Run Evidence

Every Autonomy run produces a reviewable evidence workspace. It is designed for the moment after a run finishes, when a reviewer needs to decide whether the product is safe to merge, whether a failure is real, and who should act on it.

Summary first

Start with the run summary. It shows the test case, target environment, status, failure point, and the most useful artifact links. Passing runs should be quick to scan. Failed runs should make the next action obvious.

Watching a run live

While a web run is executing, the workspace streams it. The badge above the viewport tells you which transport you are on:

The fallback is not an error state. If the stream cannot connect, or degrades mid-run, the workspace keeps showing frame evidence and the run is unaffected. Live video is available for web runs; mobile runs use the frame cadence.

Streaming only runs while someone is watching, so leaving the page open is not required for the run to complete.

Timeline and replay

The timeline shows each step Autonomy attempted and what it observed. For browser and mobile runs, screenshots and replay help confirm whether the issue is user-visible. For API, inbox, and SMS checks, the timeline records the request, response, message, or wait condition that proved the result.

Cross-channel artifacts

Autonomy can attach multiple evidence types to the same journey:

• Browser replay and screenshots.
• Mobile screenshots and device-state snapshots.
• API request and response assertions.
• Network waterfall and HAR export.
• Email inbox and OTP messages.
• SMS messages and verification codes.
• Visual comparisons against a Figma frame or uploaded reference, with the diff and the reviewer findings.
• Generated checks, warnings, and repro notes.

API assurance

When the test case has an OpenAPI document attached, the workspace also shows a contract verdict for the run: the gate status, how many documented operations the journey observed, deterministic contract failures and indeterminates, and finding counts. Inferred OpenAPI Overlay patches are listed with their rationale and can be downloaded. See API Assurance.

Failure review

For a failed run, open the failed check, compare it with the raw artifact, then use the linked replay, screenshot, request, or message to confirm the behavior. If the failure belongs in a pull request, Autonomy's PR comment should point to the same evidence rather than creating a second summary.

Suggested refinements

After a run finishes, Autonomy reads its evidence once more and looks for steps whose failure was caused by how the step was written rather than by the product: a target the screen cannot disambiguate, identity that is not visible on that screen, a credential typed as a literal, non-idempotent test data, a stale baseline, a prose assertion that has a checkable contract, or a wait condition too weak to prove completion.

When it finds one, a sparkle marker appears beside that step in the timeline. Opening it shows the current step, the proposed step, why it is suggested, and the exact trace entries and screenshot that motivated it. You accept or dismiss it; nothing changes until you do.

A suggestion can sharpen a step but never weaken it. It cannot remove or loosen an assertion, change the step action, delete a step, or introduce a secret. Accepting writes an ordinary test case revision that records the run and evidence it came from. If the step is edited in the meantime, the suggestion is marked out of date and cannot be applied. Data and environment advisories (stale fixtures, duplicate uploads) have no step edit; noting them records the follow-up without touching the case. The run's own result is never altered.

Access and retention

Evidence is scoped to the organization that owns the run. Retention, sharing, and environment access depend on your organization settings and deployment mode.