Send Autonomy run traces to Datadog APM, find failed steps, and compare them with your application's traces.

Autonomy sends run traces to Datadog through its OTLP/HTTP trace intake. Each run becomes an autonomy.run span with one autonomy.step span per step. Set it up in Organization → Integrations → OpenTelemetry. For what the spans contain, see OpenTelemetry export.

Before you start

Required access

Organization owner or admin access in Autonomy.
A Datadog organization with APM ingestion turned on, and access to its Trace Explorer.
A Datadog API key from Organization Settings → API Keys in that Datadog organization.
Your Datadog site, which you can tell from the address you sign in at.

The credential is a Datadog API key. Datadog application keys and Autonomy aut_ keys don't work. Put it in the Headers field, where Autonomy stores it as a secret and never shows it again.

Choose your Datadog endpoint

Enter the base URL of your Datadog site as the OTLP endpoint. Autonomy adds /v1/traces and sends JSON over HTTPS, which Datadog accepts.

Datadog site	Sign-in address	OTLP endpoint
US1	`app.datadoghq.com`	`https://otlp.datadoghq.com`
US3	`us3.datadoghq.com`	`https://otlp.us3.datadoghq.com`
US5	`us5.datadoghq.com`	`https://otlp.us5.datadoghq.com`
EU	`app.datadoghq.eu`	`https://otlp.datadoghq.eu`
AP1	`ap1.datadoghq.com`	`https://otlp.ap1.datadoghq.com`
AP2	`ap2.datadoghq.com`	`https://otlp.ap2.datadoghq.com`
UK1	`uk1.datadoghq.com`	`https://otlp.uk1.datadoghq.com`
US1-FED	`app.ddog-gov.com`	`https://otlp.ddog-gov.com`
US2-FED	`us2.ddog-gov.com`	`https://otlp.us2.ddog-gov.com`

These follow Datadog's OTLP trace intake site selector. A key from one site doesn't work at another site's intake.

Connect and verify

Open Organization → Integrations → OpenTelemetry and pick Datadog as the Vendor.
Paste your site's base URL into OTLP endpoint. Type it even if the placeholder already shows the right address.
Enter this in Headers, with your API key in place of YOUR_DATADOG_API_KEY:
```
dd-api-key=YOUR_DATADOG_API_KEY
```
Click Send test span. A successful response means Datadog accepted the request; the span may take a moment to become searchable.
In Datadog, open APM → Trace Explorer, pick a recent time range, and search service:autonomy-runner-labs. Find the autonomy.test_connection span.
Turn on Trace export and click Save settings. Start a new run.
When the run finishes, search for it. Replace RUN_ID with the ID at the end of the run's address in Autonomy:

service:autonomy-runner-labs @test.run_id:"RUN_ID"

For Datadog's APM trace metrics, add ,compute_stats=true to Headers. Datadog doesn't compute them for directly ingested traces otherwise. When you change or rotate the headers, save the whole value, including dd-api-key.

Find a failed step

Search for failed steps:

service:autonomy-runner-labs status:error @test.status:failed @test.step_index:*

Narrow it to one project with @autonomy.project_name:"Storefront".

A failed autonomy.step span has an error status. Its test.step_target is the step's outcome as written in the test case, and autonomy.diagnosis.category, when present, says why it didn't pass. The test case's name (test.plan_name) is on the run span.

Every span has autonomy.run_url. Open it to see the run in Autonomy, including the Step Actions the agent took and the Final check: those details stay in Autonomy and aren't exported. AI model details, prompts, screenshots and raw error messages are never sent to Datadog.

Datadog may show span names as its operation and resource fields. Filter on the attributes above to be precise. See Datadog's trace search syntax.

Compare with an application trace

Each run has its own trace ID. Autonomy doesn't pass that ID or a traceparent header to your app, so the run's trace and your app's request traces are separate. A run ID search won't find your app's requests.

Instrument your app and its backend to send their own traces to Datadog, with the service, environment and request resource. Request or order IDs help narrow a match.
From the failed step, note its time range and outcome. Open autonomy.run_url and look in the run's evidence for a request ID or other identifier your app shows.
In a second Trace Explorer tab, at the same time range, search your app's service and environment, then narrow by resource or request ID. Adapt this to your app's tags:
```
service:checkout-api env:production resource_name:"POST /orders"
```
Check that the identifiers and timing match before you blame that request. If the run tested a preview, search the preview's environment: a production request at a similar time isn't a match by itself.

This is manual. Without app traces or an identifier in the run's evidence, there's no way to pick out one request.

Troubleshooting

401 or 403: check the Datadog site and the API key. Use dd-api-key, not an application key or an Autonomy key.
404: copy the base URL from the table. The request path must end up as /v1/traces.
The test span arrives, but run spans don't: check that Trace export is on and saved, and that the run started afterwards.
Spans disappear from searches: check the time range and Datadog's trace retention settings. An accepted export isn't always indexed for later searches.
A run passed during a Datadog outage: export never changes a run's result. Use the result in Autonomy to gate CI, not the trace.

For self-hosted runners and removing credentials, see OpenTelemetry export.

Datadog