API Assurance

Attach an OpenAPI document to a test case and every run through it becomes contract evidence. Autonomy attributes the traffic the journey produced to the operations you documented, validates the responses against the schema, and derives one verdict for the run.

The point is separation of concerns. A model-authored assertion says "the response looked right". A contract check says "this response violates the schema you published" — and that verdict cannot be overridden by a model.

Contract revisions

An accepted OpenAPI 3 or Swagger 2 document — JSON or YAML — is compiled into a contract revision: a canonical, deterministic artifact with local references resolved, examples stripped, operations normalized, and the whole thing hashed.

Revisions are immutable and deduplicated by digest:

• A test case links to a revision.
• A run snapshots the revision it used.
• Re-importing a changed document creates a new revision. It never mutates the old one, and never reinterprets a historical run.

That immutability is what makes the evidence defensible six months later. The verdict on last quarter's run still refers to the contract as it existed that day.

Compilation is strict about references. Local $ref values are resolved; external, sibling-file, and dangling references are rejected outright rather than quietly weakened. Recursive or depth-limited schemas are marked as such, and validation reports them as indeterminate — an unresolved branch can never be reported as a pass.

Attach a contract

Upload the document as an attachment when you author a test case with Describe the flow — the file picker accepts OpenAPI / Swagger (.yaml, .json). It is security-scanned like any other upload, compiled, and the resulting revision is linked to the case.

Two things follow from having a contract attached:

• Autonomy can draft network assertions alongside the UI steps, because it knows what the API is supposed to do.
• Every run of that case derives an assurance report.

What a run derives

When a run reaches a terminal state, one idempotent assurance pass produces:

• Operation coverage — how many documented operations the journey actually exercised, and which it never touched.
• Drift, both directions — traffic that does not match any documented operation, and documented operations that never appear in traffic.
• Contract findings — deterministic status, content-type, required-header, and response-schema violations.
• OpenAPI Overlay documents — review-only, value-free patches describing the change the evidence implies, tied to the source digest and downloadable from the run.
• Response shape and latency baselines, with deltas against previous runs.
• A gate verdict: passing, failed, or indeterminate, with explicit reasons.

The run evidence workspace shows this as a row of facts: the gate status, n/m operations observed, contract failures and indeterminates, and finding counts broken out by isolation, cross-channel, and fuzz probes. Inferred overlay patches are listed with their rationale and can be downloaded as JSON.

indeterminate is a real answer, not a soft failure. It means the evidence could not prove the contract either way — an unresolved schema branch, or network capture the platform could not observe reliably. Treat it as "unknown", not "fine".

Evidence safety

Runtime traffic carries credentials, personal data, and tenant identifiers, so observations are structural by construction. An observation records the method, canonical path shape, the names of query parameters and headers, status, content type, bounded timing and byte counts, JSON shape fingerprints, the verified operation identity, and fixed validation issue codes.

It never records a hostname, a raw path or query value, a header value, a cookie, a body, a schema sample, a credential, or a personal datum. Raw capture stays inside the runner process and is bounded there. No customer request is ever replayed.

Credentials and multi-identity probes

Plans store credential references, never values. A profile sources a credential from a runner environment variable or from a value extracted earlier in the same run, and sensitive step values use explicit placeholders such as {{env.API_TOKEN}}. The runner resolves them inside its dispatch boundary — resolved values never enter model context, traces, observations, or backend function arguments.

This is what makes authorization probes possible. A BOLA/IDOR probe must reference an earlier mutating step that created a resource, use two distinct identity profiles, and prove before dispatch that the two principals genuinely differ. If it cannot prove that — opaque tokens, missing claims, equal principals — the probe reports indeterminate rather than claiming a pass.

In pull requests

When a test case has a contract attached, the assurance result is appended to the Autonomy pull request comment alongside the run matrix, and folds into the check run conclusion. See GitHub App.

Troubleshooting

Coverage is zero

The journey did not produce traffic Autonomy could attribute. Confirm the run used a platform with supported network capture, and that the documented server paths match the target the run actually hit.

Everything is indeterminate

Usually an unresolvable schema. Check the compiler rejected nothing at import, and look for recursive or depth-limited definitions in the operations involved.

The contract changed but runs still cite the old one

Revisions are immutable by design. Re-import the document to create a new revision and link the test case to it; historical runs keep citing the revision they ran against.