Runs and evidence
Follow a run until it ends, read its result and Step Actions, and get short-lived evidence links.
run.trigger queues runs and returns right away. Follow each returned runId with run.get, read what the agent did with run.step_actions, and get the evidence behind each step's result with run.evidence.
Follow a run with a cursor
The first run.get call needs only the run id. It returns right away:
1{2 "runId": "jh7examplerunid"3}Save the numeric cursor from the response. To wait for a change, send the run id, that cursor and waitMs:
1{2 "runId": "jh7examplerunid",3 "cursor": 42,4 "waitMs": 250005}The request returns as soon as the run changes (its cursor goes up) or reaches a final status. waitMs can be 0 to 25000 milliseconds; larger values are treated as 25000. If nothing changed in that time, the response has timedOut: true. That isn't an error: send the same cursor again, or stop waiting.
The final statuses are passed, failed, couldnt_start and cancelled.
What run.get returns
| Field | What it holds |
|---|---|
status | The run's result (see the table below). |
flaky | true when the run passed but a step needed a retry. |
progress | { completed, total } steps. |
steps | One entry per step and platform: index, platform, outcome (the step as written), state, result, and when known failureCategory, flaky and retryCount. |
createdResources | Accounts, inboxes, phone numbers and files the run created, each with its label, identifier (an address, number or username, never a secret), stepIndex, and cleanup status. |
evidence.available | Whether run.evidence has anything to return yet. |
next | A suggestion for what to do next. |
cursor, timedOut | For long-polling, as above. |
It also has the run summary that run.list returns: runId, projectId, testCaseId, testCaseName, platforms, environment, trigger (branch, PR, commit), planGroupRunId, startedAt and completedAt.
Run status
status uses the same run results as the dashboard:
status | Dashboard | Meaning |
|---|---|---|
queued | Queued | Waiting for a free runner. |
running | Running | In progress. |
passed | Passed | Every step reached its outcome, and each Final check agrees. Check flaky. |
failed | Failed | A step didn't reach its outcome, or the agent couldn't show that it did. Each failed step's explanation says why. |
couldnt_start | Couldn't start | The environment couldn't be tested, so no step ran. Not billed. |
cancelled | Cancelled | Someone stopped the run. |
If you use runs as a CI gate, treat only passed as passing. Runs stored before this change with a result that couldn't be proven are returned as failed, with their reason.
Step state and result
Each step has a state: not_run (not reached yet, or the run stopped or couldn't start first), running, finished or cancelled. Only a finished step has a result:
result | Dashboard | Meaning |
|---|---|---|
passed | Passed | The step reached its outcome. |
failed | Failed | The step didn't reach its outcome, or the agent couldn't show that it did. |
null | The step hasn't finished. |
When the cause is known, failureCategory is app_bug, test_flaky, environment, test_error, runner or unknown. runner means something broke on Autonomy's side, such as the AI model being unavailable or the runner's browser stopping. Autonomy runs such a run again once automatically; if it breaks again, it's failed with a Runner problem tag in the dashboard, and those attempts aren't billed. A step that ran out of time is environment, not runner.
next
next is advice, never authority. Decide from status and steps.
next | When |
|---|---|
wait | The run is queued or running. |
evidence | The run is failed, or it passed with flaky: true. |
retry | The run is couldnt_start. |
none | Anything else. |
Read the Step Actions
run.step_actions lists everything the agent did to reach each step's outcome, ordered by when each action started:
1{2 "runId": "jh7examplerunid",3 "stepIndex": 0,4 "limit": 1005}Each item has actionId, stepIndex (null for run-level work such as getting ready and cleanup), platform, kind, label, optional summary and detail, outcome (running, ok or error), startedAt and endedAt. kind is one of:
tool: an action on the app, the inbox or the phone.final_check: the fresh look at the product that the step's result must agree with.cleanup: removing what the run created.note: something the agent recorded along the way.
Values the run made or received are shown as they were; values you saved as secrets are masked and never appear.
Each response is a snapshot, and its cursor pages through that one snapshot. Actions can arrive late with an earlier start time, so the cursor can't follow a live run. Every response has runCursor, the same counter as run.get's cursor. While the run is going, and after it ends while cleanup is still reported, read again from the start whenever runCursor changes. If it hasn't changed since your last full read, there is nothing new.
Get the evidence
Call run.evidence with runId and, optionally, a numeric step:
1{2 "runId": "jh7examplerunid",3 "step": 14}The response is { run: { runId, status }, steps }, with one entry for each step that ran:
| Field | What it holds |
|---|---|
stepIndex, platform | Which step. |
outcome | The step's outcome as written. |
result | passed or failed. |
explanation | One line on what the evidence shows, or why the step failed. |
failureCategory | When known: app_bug, test_flaky, environment, test_error, runner or unknown. |
finalCheck | { agrees, detail }: whether the Final check agrees with the result. |
media | Screenshots, frames and video, each with a signed url and expiresAt. |
durationMs | How long the step took. |
Evidence never includes AI model or provider names, prompts, token or cost figures, internal errors or raw screen data.
Signed media links
Each media url looks like this:
GET https://api.autonomyqa.com/api/v1/evidence/<id>?kind=<screenshot|frame|video>&exp=<epoch-ms>&sig=<signature>A link works for 15 minutes and needs no bearer credential: the signature carries the permission. A valid link redirects (302) to the file. An invalid or expired link returns 403 with "The evidence link is invalid or has expired."; missing media returns 404. To get fresh links, call run.evidence again. Don't edit the query string.
Environment values and secrets
Environments hold named values that steps use as {{env.NAME}}, in the outcome text or in an attachment's value, address or number. Each run fills them in from the environment it runs in. Here a step uses an invite code saved in the environment:
1{2 "id": "join",3 "text": "The grower joins the cooperative with the invite code",4 "attachments": [5 {6 "id": "invite-code",7 "kind": "data",8 "label": "Invite code",9 "value": "{{env.COOP_INVITE_CODE}}",10 "exact": true11 }12 ]13}A value marked secret is write-only. environment.list and environment.get return only value names (runtimeValueNames), never values. When a runner picks up the run, Autonomy sends it the secret values the steps reference. They're filled in by reference, so the agent never sees them, and they're masked in Step Actions, evidence and logs. The API never returns a secret value.
For a fresh API target, run.trigger can carry values in targets.api.runtimeValues. See Environments for how to manage values in the dashboard.
Didn’t find it?