What each Couldn't start and failure reason means, what a Runner problem is, why a sign-in needs you, and what to do about it.

Start with the run page. Every run that didn't pass says why, in a fixed sentence: on the first step for Couldn't start, and on each failed step for Failed. Find that sentence below. For what the results mean, see Runs and results.

Couldn't start

No step ran, so nothing in your app was tested. Couldn't start runs aren't billed and are left out of the pass rate.

Reason on the run	What to do
The app didn't open, so the run couldn't start.	Open the environment's URL or build yourself. Check that it's deployed, that the URL is public (runners can't reach `localhost`, a private IP or a VPN-only host), and that a mobile build installs on a simulator or emulator. Then rerun.
The app blocked the test browser as a bot, so the run couldn't start.	Your app's bot protection or firewall refused the test browser. Allow it for this environment in that tool, or run against an environment without bot protection, such as staging.
A test inbox wasn't available, so the run couldn't start.	If you receive mail on your domain, check that forwarding is active: runs never fall back to Autonomy addresses. Otherwise rerun.
A test phone number wasn't available, so the run couldn't start.	Rerun. If it keeps happening, contact us.
A test device wasn't available, so the run couldn't start.	Rerun. If it keeps happening, contact us.
The run's setup didn't finish, so the run couldn't start.	Check the environment: its target, its build and the values the test case uses. A run also can't start if a value it needs was removed while it was queued.
This run couldn't start because the runner had a problem. Run it again.	Rerun. On your own runners, run `autonomy-labs doctor`. See Runners.
No runner was available to start this run in time.	The run waited in the queue too long. See A run stays Queued.
This test case needs its steps migrated.	The test case still has its steps from before v0.5. Open it, write each step as an outcome, and save. See Authoring.

A step failed

A step fails when its outcome wasn't reached, or when the agent couldn't show with fresh evidence that it was. The step says why. When the agent couldn't show the outcome, the reason is often one of these:

Reason on the step	What to do
The step ran out of time before its outcome could be confirmed.	Open the step's Step Actions to see where the time went. A step that does a lot may be two outcomes: split it. If the app is slow at that point, add a memory such as "The receipt email can take two minutes".
The step stopped before its outcome could be confirmed.	Same as running out of time: split the step, or rerun.
The agent was interrupted before it could confirm the outcome.	Rerun.
The evidence for this step couldn't be read.	Rerun. If it keeps happening, contact us with the run ID.
A final check didn't agree with the result.	The agent's fresh look at the end of the step didn't match what it reported. Read the step's Final check and screenshot. Make the outcome more specific, so there's one clear way to check it.
An account this step needed couldn't sign in.	The attached sign-in shows Needs you. Fix it (see below), then rerun.
The outcome couldn't be confirmed with fresh evidence.	Read the Step Actions and the last screenshot, then rerun. If the outcome is vague, rewrite it as something a person could check.

Runner problem

A run with the Runner problem tag failed because something broke on Autonomy's side, not in your app: the AI model was unavailable, the runner crashed or lost its connection, or its browser or device stopped. Autonomy already ran it again once, and the second attempt broke the same way. Runner-problem attempts aren't billed. Run it again later; if it keeps happening, contact us with the run ID.

A sign-in shows Needs you after a run couldn't sign in with it. The reason says what to fix:

Reason	What to do
The saved sign-in expired. Sign in again to save a fresh one.	For a Saved browser session, sign in again in Chrome and click Renew login in the extension.
The app didn't accept the saved password.	Check the password in your app, then paste the current one on the sign-in's page.
The sign-in provider asked for an extra check (for example a phone approval). Add the authenticator secret or approve it once.	Add the account's authenticator secret, so the agent can type the code itself. See Authenticator codes.
The app says this account is locked.	Unlock the account in your app, or use another test account.
Signing in stopped working. Check the account and try again.	Sign in as the account yourself to see what changed.

The next run that signs in with it sets it back to Working. Only owners and admins can replace a saved secret. See Sign-ins.

Codes or emails don't arrive

Open the run's Email previews tab.

The email is there. The product sent it. The step's Step Actions show what the agent did with it, including the code or link it used.
The email isn't there. Check that the product sent to the address this run created, not one from an earlier run: a run's inbox closes when the run ends, and later mail is refused. The agent only reads mail that arrived after its inbox was created or its account was taken.
Your domain. Until forwarding is active, runs can't get an inbox on your domain. Finish the steps in Your domain.
Test mode gives a fixed code, not a link. Journeys that verify with a magic link need hosted inboxes or your domain.
SMS. Some senders, such as banks, Google and WhatsApp, refuse virtual numbers. For sign-ins with SMS codes, use your auth provider's test number with a static code. See Inboxes and phones.

A run stays Queued

A run waits in the queue until a runner is free and your organization has a concurrent run to spare. The run page shows "All runners are busy. Your run starts as soon as one is free."

Organization → Billing shows Concurrent runs: how many are in use and your limit. Extra runs wait.
Owners and admins on paid plans can add Concurrent runs under Billing extensions, or upgrade. See Billing.
A run that waits too long ends Couldn't start with "No runner was available to start this run in time." It isn't billed. Rerun it when fewer runs are going.

A run from before v0.5 can't be rerun

Runs from before v0.5 used the old step format. They stay readable, but Rerun is hidden and the API refuses it with "This run used the old step format, so it can't be rerun. Start a new run of the test case instead." Start a new run of the test case. If the test case itself still has old steps, write its outcomes first.

A step was judged on the wrong value

If the product refused a value the step was about, check whether the value is marked Exact. An Exact value is used as written, and a refusal is what the step is judged on. Test data isn't: if the product says it's taken, the agent uses a fresh value of the same kind and says so. See Exact or test data.

Still stuck

Email support@autonomyqa.com with the run ID (#xxxx on the run page). Use Copy prompt to fix on a failed run to hand the details to your coding agent.

Troubleshooting