OpenTelemetry export
Export a trace for every run, with one span per step, to your OTLP backend.
Autonomy can export every run as an OpenTelemetry trace to the backend your team already uses: Datadog, Grafana, Honeycomb or any OTLP endpoint. Failed steps carry an error status, so they show up in the same trace views, dashboards and alerts as your application's traffic.
What it does
- Exports one trace per run: an
autonomy.runspan with oneautonomy.stepchild per step. - Marks failed runs and failed steps with an OTLP error status.
- Adds a link back to the run in Autonomy on every span.
- Tags every span with its project, so you can filter one project's runs.
- Sends the settings to hosted runners for you. Configure it once; nothing changes on the runner.
- Never slows a run down: if your endpoint is slow or down, the run carries on and nothing is retried.
The spans
| Span | One per | Status |
|---|---|---|
autonomy.run | Run | Error when the run's result is Failed. |
autonomy.step | Step of the test case | Error when the step failed. |
A step span covers the whole step, from the agent's first action to the Final check. The agent's individual actions are not exported as spans: they are in the run's Step Actions in Autonomy.
What gets exported
Spans carry only an allowlisted set of attributes:
| Attribute | On | Content |
|---|---|---|
test.run_id, test.run_execution_id | Run, step | The run. |
test.plan_id, test.plan_name | Run | The test case that ran. |
test.platform | Run, step | web, ios, android or api. |
test.status | Run, step | Step: passed or failed. Run: passed, failed, couldnt_start or cancelled. |
test.step_index | Step | The step's position in the test case. |
test.step_action | Step | Always check. |
test.step_target | Step | The step's outcome, as written in the test case. |
autonomy.diagnosis.category | Step | Why a step didn't pass, as a category code, when there is one. |
autonomy.project_id, autonomy.project_name | Run, step | The run's project. |
autonomy.org_id | Run | Your organization. |
autonomy.run_url | Run, step | A link to the run in Autonomy. |
Nothing else leaves Autonomy. AI model details, cost, prompts, tool names, screenshots, page content and raw error messages are never exported. The exporter drops any attribute that isn't on the list.
Set up
Before you start
- Organization owner or admin access in Autonomy.
- An OTLP/HTTP trace intake on a public HTTPS address.
- An ingest credential for your backend: an API key, token or basic-auth header.
- Open Organization → Integrations → OpenTelemetry.
- Under Trace export, pick your Vendor: Datadog, Grafana, Honeycomb or Generic. It fills in the shape of the endpoint and headers.
- Enter the OTLP endpoint. Autonomy adds
/v1/tracesitself. It must use HTTPS. - Enter the Headers as comma-separated
key=valuepairs. They are stored as a secret and never shown again. - Click Send test span. Autonomy sends an
autonomy.test_connectionspan and shows the HTTP status and time it took. - Find the span in your trace explorer under the service
autonomy-runner-labs. Then turn on Trace export and click Save settings.
Runs that start after you save are exported.
Vendor setup
Datadog
See Datadog for the endpoint of each Datadog site, the API key header, and searches for failed steps.
Grafana Cloud (Tempo)
- Endpoint:
https://otlp-gateway-<region>.grafana.net/otlp, from your stack's OpenTelemetry details page. - Headers:
Authorization=Basic <base64 of instanceID:token>, with a token that can write traces. - Where to look: Explore with the Tempo data source, and query
{resource.service.name="autonomy-runner-labs"}.
Honeycomb
- Endpoint:
https://api.honeycomb.io, orhttps://api.eu1.honeycomb.ioin the EU. - Headers:
x-honeycomb-team=<your ingest key>. On Honeycomb Classic, addx-honeycomb-dataset=<dataset>. - Where to look: the
autonomy-runner-labsdataset. Trytest.status = failedfirst.
Generic OTLP
Any collector or vendor that takes OTLP/HTTP with JSON on /v1/traces works, such as an OpenTelemetry Collector. Use its base URL on a public HTTPS hostname, and whatever auth headers it expects.
Filter by project
Every span has autonomy.project_id and autonomy.project_name. Use them to build one dashboard or alert per project, for example failed steps where autonomy.project_name is Storefront.
Self-hosted runners
The dashboard settings reach self-hosted runners too. A self-hosted runner without them can export on its own with environment variables on the runner machine:
| Variable | Required | Meaning |
|---|---|---|
AUTONOMY_OBSERVABILITY_EXPORT | Yes | Set to otel to turn export on. |
OTEL_EXPORTER_OTLP_ENDPOINT | Yes | The base OTLP endpoint. /v1/traces is added. |
OTEL_EXPORTER_OTLP_HEADERS | No | Comma-separated key=value pairs sent with every export. |
OTEL_SERVICE_NAME | No | The service name on spans. Defaults to autonomy-runner-labs. |
See Runners.
Change or remove credentials
Headers are write-only. To change them, enter the new value and save; it replaces the old one. To remove them, click Clear stored headers. The endpoint stays, but exports go without credentials, which most backends refuse, until you save new ones.
Troubleshooting
Send test span returns 401 or 403
Your backend refused the credential. Check the header name for your vendor (dd-api-key, Authorization, x-honeycomb-team) and that the credential can write traces.
Send test span returns 404
There's no trace intake at that address. Enter the base endpoint only: Autonomy adds /v1/traces, so a doubled path is the usual cause.
The endpoint is refused
Only public HTTPS hostnames are accepted. Plain HTTP, IP addresses, localhost and .internal or .local names are refused. For a collector inside your network, put an HTTPS ingress in front of it.
The test span arrives, but runs export nothing
Check that Trace export is on and saved, and that the run started after you saved. Export problems never change a run, so a wrong credential shows up as missing spans, not as run errors. Send a test span again to check.
Didn’t find it?