Export test-run traces to Datadog, Grafana, Honeycomb, or any OTLP backend — and pivot from a failed test step to the matching production trace.

Autonomy exports every test run as an OpenTelemetry trace to the observability backend your team already uses. A run becomes an autonomy.run span with nested autonomy.step and autonomy.action.* children — failed steps carry OTLP error status, so they surface in the same error tracking, dashboards, and alerts as your production traffic.

What it does

Exports one trace per run: autonomy.run → autonomy.step (one per plan step) → autonomy.action.* (navigate, fill, tap, wait, …) with real timings.
Marks failed runs, steps, and actions with OTLP error status — trace explorers paint them red and count them in error rates.
Attaches autonomy.run_url to every span: a deep link from any span in your backend straight to the run's evidence page in Autonomy.
Delivers vendor credentials to hosted runners automatically — configure once in the dashboard, no runner environment changes.
Export is fire-and-forget: a slow or unreachable endpoint never delays or fails a test run.

What gets exported

Spans carry a fixed, allowlisted set of attributes:

Attribute	Content
`test.status`	`passed` / `failed` per step, final status on the run span
`test.platform`	`web`, `ios`, `android`, or `api`
`test.plan_name`, `test.plan_id`	The test case being run
`test.step_index`, `test.step_action`, `test.step_target`	Your authored plan step
`autonomy.diagnosis.category`	Failure classification (`app_bug`, `test_flaky`, `environment`, …)
`autonomy.recovery_count`	Retries consumed before the step's outcome — a flakiness signal you can alert on
`autonomy.run_url`	Deep link to the run in the Autonomy dashboard

Nothing else leaves Autonomy. Model identity, token usage, prompts, screenshots, page content, raw error messages, and internal tooling names are never exported — failures are reduced to category codes, and the contract is enforced by an attribute allowlist in the exporter.

Setup

Before you start

A workspace admin account — the Observability settings are admin-only.
An OTLP/HTTP intake endpoint on a public https hostname.
An ingest credential for your vendor (API key, token, or basic-auth header).

Open Settings → Observability in the Autonomy dashboard.
Pick your vendor preset — it pre-fills the endpoint shape and header format.
Enter the OTLP endpoint. Autonomy appends /v1/traces automatically; https is required.
Enter the Headers as comma-separated key=value pairs. They are stored encrypted and write-only — never shown again after saving.
Click Send Test Span. Autonomy posts a synthetic autonomy.test_connection span server-side and reports the HTTP status and latency.
Confirm the span arrived in your trace explorer under service autonomy-runner-labs, then enable Trace Export and save.

Vendor setup

Datadog

Endpoint: https://otlp.datadoghq.com — use your site's domain (otlp.datadoghq.eu, otlp.us5.datadoghq.com, …).
Headers: dd-api-key=<your API key> (an API key, not an application key).
Where traces appear: APM → Traces, service autonomy-runner-labs. Failed steps show as error spans; facet on @autonomy.diagnosis.category or @test.plan_name to build monitors.

Grafana Cloud (Tempo)

Endpoint: https://otlp-gateway-<region>.grafana.net/otlp — copy it from your stack's OpenTelemetry → OTLP details page.
Headers: Authorization=Basic <base64 of instanceID:token> — generate the token with a metrics:write/traces:write scope.
Where traces appear: Explore with the Tempo data source; query {resource.service.name="autonomy-runner-labs"} in TraceQL.

Honeycomb

Endpoint: https://api.honeycomb.io (EU: https://api.eu1.honeycomb.io).
Headers: x-honeycomb-team=<your ingest key>. On Honeycomb Classic add x-honeycomb-dataset=<dataset>.
Where traces appear: the autonomy-runner-labs dataset (or the environment the ingest key belongs to). BubbleUp on test.status = failed is a good first query.

Generic OTLP

Any collector or vendor that accepts OTLP/HTTP with JSON on /v1/traces works — an OpenTelemetry Collector, SigNoz, Axiom, and most vendors' native intakes. Point the endpoint at the base URL (public https hostname — IP literals and internal hostnames are rejected) and supply whatever auth headers the intake expects.

Self-hosted runners

Dashboard configuration takes precedence and requires no runner changes. Self-hosted runners without an org config can enable export via environment instead:

Variable	Required	Description
AUTONOMY_OBSERVABILITY_EXPORT	Yes	Set to `otel` to enable env-based export.
OTEL_EXPORTER_OTLP_ENDPOINT	Yes	Base OTLP endpoint; `/v1/traces` is appended automatically.
OTEL_EXPORTER_OTLP_HEADERS	No	Comma-separated `key=value` pairs sent with every export.
OTEL_SERVICE_NAME	No	Service name on exported spans. Defaults to `autonomy-runner-labs`.

Rotating or removing credentials

Headers are write-only: to rotate, paste the new value and save — it replaces the stored secret. To disconnect a vendor, use Clear stored headers; the endpoint config stays, but exports run without auth headers (which most vendors reject) until new ones are saved.

Troubleshooting

Send Test Span fails with HTTP 401 or 403

The intake rejected the credentials. Re-check the header key for your vendor (dd-api-key, Authorization, x-honeycomb-team) and that the credential has ingest/write scope.

Send Test Span fails with HTTP 404

There is no OTLP trace intake at that URL. Enter the base endpoint only — Autonomy appends /v1/traces itself; a doubled path is the usual cause.

"Endpoint must use https" or "must be a public hostname"

Only public https hostnames are accepted — plain http, IP literals, localhost, and .internal/.local names are rejected. For an internal collector, expose an https ingress in front of it.

The test span works but runs export nothing

Confirm Trace Export is toggled on and saved, and that your runners have picked up work after the change — the config is delivered when a runner claims a run. Delivery failures never fail runs, so a wrong credential shows up as silence, not as run errors; re-run Send Test Span to re-verify.

Observability (OTLP)