A run is one execution of a test case in one environment. The run page shows what the agent did for each step, what it saw, and the result, so you can decide quickly whether a failure is real and who should act on it.

Start a run

Click Run on a test case, or open New run in a project:

What to run: a test case, or a whole test plan (one run per test case).
Where: the environment. The project's default environment is picked first.
Signs in as: the accounts the steps attach, and whether each one is set up in that environment.

A run uses every platform of its test case. Runs also start from schedules, pull requests, deploys, Slack, the API and AI assistants.

Every run starts signed out, with a fresh browser, app install or HTTP client. While it waits for a runner it shows Queued; before the first step it shows Getting ready while Autonomy opens the app, creates the inbox and loads the sign-ins.

What we checked

The run page lists every step of the test case with its result. Under each step you see:

Its attachments: the account, inbox, values and files it used. Exact values carry a lock.
Its Step Actions: every action the agent took to reach the outcome, in order, as it happens: opening a page, filling a field, reading the inbox, waiting for an email. Values the run made or received are shown as they were. A value you saved as a secret shows as a lock: it's filled by reference, and the agent never sees it.
Its Final check: the agent's fresh look at the product at the end of the step. The step's result must agree with it.
Its outcome: one line on what the evidence shows.

For a test case on several platforms, switch platforms above the steps. Each platform has its own steps and results.

Results

Each step ends Passed or Failed. The run's result follows from its steps:

Result	When
Passed	Every step reached its outcome. flaky means a step needed a retry.
Failed	A step didn't reach its outcome, or the agent couldn't show that it did.
Couldn't start	The environment couldn't be tested, so no step ran.
Cancelled	Someone stopped the run.

A failed step says why, in plain words. Besides an outcome that wasn't reached, a step also fails when the agent couldn't show the outcome with fresh evidence, for example:

The step ran out of time before its outcome could be confirmed.
The agent was interrupted before it could confirm the outcome.
The evidence for this step couldn't be read.
A final check didn't agree with the result.
An account this step needed couldn't sign in.

Read the reason before you treat a failure as a bug: a step that ran out of time may do too much and need splitting, and an account that couldn't sign in shows Needs you. See Troubleshooting.

Runner problem. If a run fails because something broke on Autonomy's side, such as the AI model being unavailable, the runner crashing or its browser or device stopping, Autonomy runs it again once, automatically. If the second attempt breaks the same way, the run is Failed with a Runner problem tag. Runner-problem attempts aren't billed.

Runs from before this change that couldn't prove a step now show as Failed, with the reason they recorded.

A run that couldn't start says why, for example "The app didn't open, so the run couldn't start." or "The app blocked the test browser as a bot, so the run couldn't start." Couldn't start runs aren't billed and are left out of the pass rate.

Issues found

Problems the agent or an automatic check noticed are listed under Issues found, next to the result. An issue never changes the result: a passing run can still report a console error or a failed request. Each issue shows its kind (Bug or Risk), who found it ("Seen by the agent", "Automatic check" or "Added by a person"), the step, and what was expected. See Issues.

Click Add issue to report one yourself from the run. On a failed run, or one with issues, Copy prompt to fix copies a prompt for your coding agent.

Evidence

The evidence panel below the steps has:

Tab	What it shows
Replay	The run's video, with a marker for each step. Live while the run is going.
Screenshots	A screenshot taken at the end of every step, plus the ones steps asked to keep.
Network	Every request the page, its frames and its workers made, with headers, cookies, request and response bodies (the first 256 KB of each), status, timing and a waterfall, divided by step.
Console	Every console message at every level, uncaught errors and unhandled promise rejections, and the browser's own messages, divided by step. Logs on mobile, with the app's logs at every level.
Credentials	Accounts the run created, which you can keep as saved sign-ins.
Email previews	Emails the run's inboxes received, as HTML, text and headers.

API runs show Requests and Console instead.

Evidence shows what the run saw, as it saw it: received codes and links, generated passwords, tokens in URLs and request bodies are all shown in clear. Only values you saved as secrets (environment secrets, sign-in passwords, authenticator secrets and saved sessions) are masked everywhere: in Step Actions, screenshots, video, network details, logs and emails.

Accounts the run created

When a step signs up a new account, the Credentials tab lists it with how it signs in. Click Keep as saved sign-in to save it to the project's sign-ins: the password the agent created comes with it. The tab shows a generated password until you keep the account; from then on it's a saved secret. A magic-link account brings its address back as a project inbox.

You can keep an account for as long as your plan keeps run history, up to 90 days. Accounts that signed in with a rented phone number can't be kept, because the number goes back when the run ends.

Also created lists the inboxes, phone numbers and files the run created, and whether each was cleaned up.

Rerun, cancel and delete

Rerun starts a new run of the same test case in the same environment.
Cancel run stops a run in flight. A cancelled run pays for what it used.
Delete run removes the run and its replay, screenshots and logs. Issues that cite it keep a placeholder.

Runs from before v0.5 used the old step format. They stay readable, but they can't be rerun: start a new run of the test case instead.

Runs and results