Run Autonomy from CircleCI with a web preview URL or a mobile build, then gate the pipeline on the verdict.

Use CircleCI to queue Autonomy runs after a deployment or mobile build. This page includes the API helper, complete configurations, and the verdict gate.

Workflow placement

Run Autonomy after the preview is reachable or the mobile artifact exists. Each configuration below is a complete alternative .circleci/config.yml; copy the one for your platform. In an existing workflow, retain your deployment job and make the Autonomy job depend on it with requires. Pass the final deployed URL, not a CI container's localhost.

Required secrets

In CircleCI, open Project Settings → Environment Variables and add the inputs below. CircleCI injects them into the job shell; do not write the API key into YAML. A restricted context can hold the same inputs instead: attach it using context on the job's entry under workflows, not under jobs.

AUTONOMY_API_URL: your Autonomy API base URL, without /api, using the Convex HTTP URL ending in .convex.site.
AUTONOMY_API_KEY: an aut_ organization API key from the same deployment and organization as the plan.
AUTONOMY_TEST_PLAN_ID: open an existing plan in Test Plans and copy its ID from /test-plans/<id> in the dashboard URL. The plan must contain at least one test case for the selected platform. This is a plan ID, not an individual test case ID.
Web: PREVIEW_URL, the reachable HTTPS deployment URL.
Mobile: APP_IDENTIFIER, the iOS bundle ID or Android package name matching the artifact.
Test cases that can run with these explicit targets, without runtime values or credentials supplied only by a saved Environment. A runner must be available for the selected platform.

Use a separate plan for each platform. The configurations below expect AUTONOMY_TEST_PLAN_ID and APP_IDENTIFIER to match the chosen recipe. If combining platforms, supply the appropriate values through separate contexts or platform-specific variables. Read Test Plans for how cases are grouped.

Web preview URL

Create .ci/ in your repository, save the following block as .ci/autonomy.sh, and commit it alongside .circleci/config.yml. The jobs invoke it with Bash, so an executable file bit is not required. It needs Bash, curl, and jq, provided by the Linux images below.

#!/usr/bin/env bash
set -euo pipefail
set +x
: "${AUTONOMY_API_URL:?Set the API base URL, without /api}"
: "${AUTONOMY_API_KEY:?Set an aut_ organization API key}"
: "${AUTONOMY_TEST_PLAN_ID:?Set the Test Plans ID for this platform}"
: "${AUTONOMY_REQUEST_ID:?Set a unique CI job/attempt ID}"
platform="${1:-web}"
api="${AUTONOMY_API_URL%/}"
work=$(mktemp -d)
trap 'rm -rf "$work"' EXIT

post() {
  curl --fail-with-body --silent --show-error \
    --connect-timeout 15 --max-time 90 \
    -X POST "$api$1" \
    -H "Authorization: Bearer $AUTONOMY_API_KEY" \
    -H 'Content-Type: application/json' --data-binary "$2"
}

case "$platform" in
  web)
    : "${PREVIEW_URL:?Set the reachable preview URL after deployment}"
    targets=$(jq -n --arg url "$PREVIEW_URL" '{web:{baseUrl:$url}}')
    ;;
  ios|android)
    : "${ARTIFACT_PATH:?Set the path to the app archive or APK}"
    : "${APP_IDENTIFIER:?Set the bundle ID or Android package name}"
    test -s "$ARTIFACT_PATH"
    file=$(basename "$ARTIFACT_PATH")
    bytes=$(wc -c < "$ARTIFACT_PATH" | tr -d '[:space:]')
    content_type=application/zip
    if [ "$platform" = android ]; then
      content_type=application/vnd.android.package-archive
    fi
    upload=$(post /api/artifacts/upload-url "$(jq -n \
      --arg platform "$platform" --arg fileName "$file" \
      --arg contentType "$content_type" --argjson declaredBytes "$bytes" \
      '{platform:$platform,fileName:$fileName,contentType:$contentType,declaredBytes:$declaredBytes}')")
    url=$(jq -er '.uploadUrl | strings | select(length > 0)' <<< "$upload")
    storage=$(jq -er '.storageId | strings | select(length > 0)' <<< "$upload")
    intent=$(jq -er '.intentId | strings | select(length > 0)' <<< "$upload")
    jq -e '.method == "PUT"' <<< "$upload" > /dev/null
    curl --fail-with-body --silent --show-error \
      --connect-timeout 15 --max-time 600 -X PUT "$url" \
      -H "Content-Type: $content_type" --data-binary "@$ARTIFACT_PATH"
    scan=$(post /api/artifacts/scan "$(jq -n \
      --arg storageId "$storage" --arg intentId "$intent" \
      '{storageId:$storageId,intentId:$intentId}')")
    if ! jq -e '.status == "clean"' <<< "$scan" > /dev/null; then
      echo 'Artifact scan did not pass; refusing to trigger.' >&2
      exit 1
    fi
    targets=$(jq -n --arg platform "$platform" --arg storageId "$storage" \
      --arg fileName "$file" --arg identifier "$APP_IDENTIFIER" \
      '{($platform):({storageId:$storageId,sourceMode:"upload",fileName:$fileName} +
        (if $platform == "ios" then {bundleId:$identifier} else {packageName:$identifier} end))}')
    ;;
  *) echo 'Usage: bash .ci/autonomy.sh web|ios|android' >&2; exit 2 ;;
esac

body=$(jq -n --arg testPlanId "$AUTONOMY_TEST_PLAN_ID" \
  --arg platform "$platform" --argjson targets "$targets" \
  --arg branch "${AUTONOMY_BRANCH:-manual}" \
  --arg commitSha "${AUTONOMY_COMMIT_SHA:-}" \
  --arg key "$AUTONOMY_REQUEST_ID:$platform" \
  '{testPlanId:$testPlanId,platforms:[$platform],targets:$targets,
    branch:$branch,deployment:{branch:$branch,commitSha:$commitSha},idempotencyKey:$key}')
post /api/v1/run.trigger "$body" > "$work/trigger.json"
jq -e '.runIds | type == "array" and length > 0' "$work/trigger.json" > /dev/null
jq -er '.runIds[] | strings | select(length > 0)' "$work/trigger.json" > "$work/run-ids"
printf 'Queued Autonomy runs:\n'
cat "$work/run-ids"

deadline=$((SECONDS + ${AUTONOMY_WAIT_SECONDS:-1200}))
while IFS= read -r run_id; do
  while :; do
    if (( SECONDS >= deadline )); then
      echo "Timed out waiting for $run_id; failing the CI job." >&2
      exit 1
    fi
    result=$(post /api/v1/run.get "$(jq -n --arg runId "$run_id" '{runId:$runId}')")
    status=$(jq -er '.status' <<< "$result")
    case "$status" in
      queued|running|retrying) sleep 10 ;;
      passing|failed|unverified|invalid|canceled)
        printf '%s: %s\n' "$run_id" "$status"
        if ! jq -e '.verdict == "passed" and .validity == "valid"' <<< "$result" > /dev/null; then
          echo "Autonomy gate failed for $run_id." >&2
          exit 1
        fi
        break ;;
      *) echo "Unexpected run status: $status" >&2; exit 1 ;;
    esac
  done
done < "$work/run-ids"
echo 'Every Autonomy run passed.'

Save this configuration as .circleci/config.yml. Set PREVIEW_URL to an already deployed preview for a first run, then push the files to staging. The HTTP readiness check must pass before Autonomy is triggered.

For per-commit previews, export PREVIEW_URL and invoke the helper in the same run step. A plain export does not persist to another step. To use a later run step in the same job, add this line after URL discovery in the deployment step; CircleCI loads BASH_ENV in subsequent steps:

printf 'export PREVIEW_URL=%q\n' "$PREVIEW_URL" >> "$BASH_ENV"

For separate jobs, save the discovered URL as a single line in /tmp/autonomy-preview/preview-url.txt. Use persist_to_workspace with root: /tmp/autonomy-preview and paths: [preview-url.txt] in the deployment job. In the Autonomy job, use attach_workspace with at: /tmp/autonomy-preview, then run export PREVIEW_URL="$(cat /tmp/autonomy-preview/preview-url.txt)" before the readiness check. BASH_ENV does not transfer values between jobs.

version: 2.1

jobs:
  autonomy-web:
    docker:
      - image: cimg/base:current
    steps:
      - checkout
      - run:
          name: Check preview and run Autonomy
          no_output_timeout: 30m
          command: |
            set -euo pipefail
            set +x
            : "${PREVIEW_URL:?Set the deployed preview URL}"
            curl --fail --silent --show-error --location \
              --connect-timeout 15 --max-time 30 \
              --retry 10 --retry-delay 3 --retry-all-errors \
              --output /dev/null "$PREVIEW_URL"
            export AUTONOMY_BRANCH="${CIRCLE_BRANCH:-local}"
            export AUTONOMY_COMMIT_SHA="${CIRCLE_SHA1:-$(git rev-parse HEAD)}"
            export AUTONOMY_REQUEST_ID="circleci:${CIRCLE_WORKFLOW_ID:-local}:${CIRCLE_BUILD_NUM:-$(date +%s)-$$}:web"
            bash .ci/autonomy.sh web

workflows:
  preview-qa:
    jobs:
      - autonomy-web:
          filters:
            branches:
              only: staging

For a local check on macOS or Linux, start Docker and download the official legacy CLI v0.1.47860 release. Choose the .tar.gz asset for your OS (darwin for macOS, linux for Linux) and CPU (arm64 for Apple silicon or ARM, amd64 for Intel or AMD). Extract it and place the circleci-v0 executable in a directory on your PATH. The current CLI v1 removed local execution; the commands below deliberately use the legacy binary.

Export the four inputs in your local shell, then run these commands from the committed repository. CircleCI does not import project secrets or contexts locally; each -e is required. Version 2.1 configuration must be processed before execution.

circleci-v0 config validate .circleci/config.yml
circleci-v0 config process .circleci/config.yml > /tmp/autonomy-circleci.yml
circleci-v0 local execute -c /tmp/autonomy-circleci.yml \
  -e "AUTONOMY_API_URL=$AUTONOMY_API_URL" \
  -e "AUTONOMY_API_KEY=$AUTONOMY_API_KEY" \
  -e "AUTONOMY_TEST_PLAN_ID=$AUTONOMY_TEST_PLAN_ID" \
  -e "PREVIEW_URL=$PREVIEW_URL" \
  -e "AUTONOMY_WAIT_SECONDS=${AUTONOMY_WAIT_SECONDS:-1200}" \
  autonomy-web

Local execution runs this Docker job only. It does not prove workflow scheduling, context restrictions, macOS execution, or the iOS workspace handoff.

Mobile artifact handoff

The helper requests an upload URL with the file size, uploads with PUT, and scans using both storageId and intentId. It triggers only after a clean scan. Keep .ci/autonomy.sh from above and replace .circleci/config.yml with the relevant configuration below.

iOS

Build the Simulator .app on macOS, archive it before transport to preserve executable permissions and symbolic links, then pass the ZIP to a Linux job using persist_to_workspace and attach_workspace. Uploading and waiting use Linux credits.

This example assumes a native Xcode project at ios/MyApp.xcodeproj, a shared MyApp scheme, and a MyApp.app product. Replace those names with your project's names. If your app uses an Xcode workspace, replace -project with -workspace and its path; keep your existing dependency installation steps before xcodebuild. Do not upload a device .ipa.

CircleCI currently documents 27.0.0 on m4pro.medium. macOS is available on the Free plan within its credit allowance; check the available balance before running builds. circleci-v0 local execute cannot run this macOS job.

version: 2.1

jobs:
  build-ios:
    macos:
      xcode: "27.0.0"
    resource_class: m4pro.medium
    steps:
      - checkout
      - run:
          name: Build and archive the Simulator app
          command: |
            set -euo pipefail
            xcodebuild \
              -project ios/MyApp.xcodeproj \
              -scheme MyApp \
              -configuration Release \
              -sdk iphonesimulator \
              -destination 'generic/platform=iOS Simulator' \
              -derivedDataPath ios/build \
              CODE_SIGNING_ALLOWED=NO build
            mkdir -p /tmp/autonomy-ios
            (cd ios/build/Build/Products/Release-iphonesimulator && \
              zip -qry /tmp/autonomy-ios/MyApp.app.zip MyApp.app)
      - persist_to_workspace:
          root: /tmp/autonomy-ios
          paths:
            - MyApp.app.zip

  autonomy-ios:
    docker:
      - image: cimg/base:current
    steps:
      - checkout
      - attach_workspace:
          at: /tmp/autonomy-ios
      - run:
          name: Upload and run Autonomy
          no_output_timeout: 30m
          command: |
            set -euo pipefail
            set +x
            export ARTIFACT_PATH=/tmp/autonomy-ios/MyApp.app.zip
            export AUTONOMY_BRANCH="$CIRCLE_BRANCH"
            export AUTONOMY_COMMIT_SHA="$CIRCLE_SHA1"
            export AUTONOMY_REQUEST_ID="circleci:$CIRCLE_WORKFLOW_ID:$CIRCLE_BUILD_NUM:ios"
            bash .ci/autonomy.sh ios

workflows:
  ios-qa:
    jobs:
      - build-ios:
          filters:
            branches:
              only: staging
      - autonomy-ios:
          requires:
            - build-ios
          filters:
            branches:
              only: staging

Android

The Linux Android image includes the SDK, Java, curl, and jq. This example assumes a native Gradle project under android/, its committed executable gradlew, and the app module's debug APK. Adjust the Gradle task and APK path for your module or product flavor. Retain your existing dependency preparation if the app also needs JavaScript or other build tooling.

version: 2.1

jobs:
  autonomy-android:
    docker:
      - image: cimg/android:2026.08
    resource_class: medium
    steps:
      - checkout
      - run:
          name: Build the debug APK
          command: |
            set -euo pipefail
            cd android
            ./gradlew --no-daemon :app:assembleDebug
      - run:
          name: Upload and run Autonomy
          no_output_timeout: 30m
          command: |
            set -euo pipefail
            set +x
            export ARTIFACT_PATH=android/app/build/outputs/apk/debug/app-debug.apk
            export AUTONOMY_BRANCH="$CIRCLE_BRANCH"
            export AUTONOMY_COMMIT_SHA="$CIRCLE_SHA1"
            export AUTONOMY_REQUEST_ID="circleci:$CIRCLE_WORKFLOW_ID:$CIRCLE_BUILD_NUM:android"
            bash .ci/autonomy.sh android

workflows:
  android-qa:
    jobs:
      - autonomy-android:
          filters:
            branches:
              only: staging

Gate the pipeline on the verdict

A queued run is not a passing test. The helper prints every returned run ID and polls POST /api/v1/run.get. It succeeds only when every run returns verdict: "passed" and validity: "valid"; failures, cancellation, invalid or unverified runs, API errors, and timeouts fail the job.

The wait budget is 1,200 seconds across the plan. Set AUTONOMY_WAIT_SECONDS if your plan needs longer, and increase no_output_timeout accordingly. CircleCI's default silence timeout can otherwise stop a healthy poll loop. Upload and scan time come before this wait budget. Put release jobs after the Autonomy job with requires so a failed gate prevents promotion.

Branch and event strategy

These examples run on staging. Expand filters only to trusted branches. Use a small smoke plan for reviewed preview changes and larger plans for release or scheduled pipelines. Configure push, pull request, or scheduled triggers in the CircleCI project for your connected VCS; branch filters alone do not create those triggers.

Keep Pass secrets to builds from forked pull requests disabled where that setting is available. Review untrusted code before running it in a job with the org API key, and restrict any shared context to the intended projects and trusted actors. The request ID combines the workflow and job number so a new job attempt can queue a new run while a repeated request in that attempt is idempotent.

Vendor references

Checked against CircleCI's public documentation on 20 September 2026:

Configuration reference, workspaces, and project variables.
Project environment variables and contexts.
Base image, Android image, and macOS execution.
Free plan macOS resource class, local execution, and forked pull request secrets.

CircleCI

On this page