Start Autonomy runs from a GitHub Actions workflow for a web preview or an Android APK, and fail the workflow unless every run passes.

Use GitHub Actions when you want Autonomy as a step in your own workflow: after a custom deploy script, for a mobile build, or before a release job. The workflow starts the runs, waits for their results and fails unless every run is Passed.

To run a test plan on every pull request without a workflow file, use the GitHub App instead. It runs the project's pull request plan against the preview deployment and reports a GitHub check. See Pull request testing.

Workflow placement

Put the Autonomy step after the job that deploys the preview or builds the app. The run must receive the final URL a reviewer would open, or the build file itself. The web example below takes an already deployed URL; it doesn't deploy your app.

Each workflow below is a complete file. When adding one to an existing workflow, keep your build and deploy jobs and point needs at the job that produces the URL or the build.

Required secrets

In Settings → Secrets and variables → Actions of the repository, add:

Name	Kind	Value
`AUTONOMY_API_KEY`	Secret	An organization API key starting with `aut_`. Owners and admins create keys in Organization → API keys.
`AUTONOMY_TEST_PLAN_ID`	Variable	The test plan to run. Open it in the project's Test plans: the ID is the last part of the page URL, `/p/<project>/plans/<id>`.
`PREVIEW_URL`	Variable	Web only: a deployed HTTPS URL that Autonomy can reach from the internet, not `localhost`.
`AUTONOMY_IOS_BUNDLE_ID`	Variable	iOS only: the bundle ID of the built app.
`AUTONOMY_ANDROID_PACKAGE_NAME`	Variable	Android only: the application ID of the built APK.

Optional values the helper reads:

AUTONOMY_TEST_CASE_ID: run one test case instead of a plan. Its ID is the last part of /p/<project>/tests/<id>. Set exactly one of AUTONOMY_TEST_CASE_ID and AUTONOMY_TEST_PLAN_ID.
AUTONOMY_ENVIRONMENT_SLUG: run against a saved environment of the same project, with its targets and environment values, instead of this job's URL or build. The helper then skips the upload.
AUTONOMY_WAIT_SECONDS: the total wait time for all runs, 1,200 seconds by default.

Without an environment, a run has no environment values: a step that uses {{env.NAME}} ends as Couldn't start. Set AUTONOMY_ENVIRONMENT_SLUG, or use test cases that don't need environment values. Use a separate test plan for each platform, so a result points at the right app.

Web preview URL

Create a .ci folder in the repository root and save this script as .ci/autonomy.sh. Every workflow on this page, and the GitLab, Jenkins, Bitrise and CircleCI recipes, use this same file. It needs Bash, jq and curl 7.76 or later; GitHub-hosted runners have them.

.ci/autonomy.sh

#!/usr/bin/env bash
# Usage: bash .ci/autonomy.sh web|ios|android
# Starts Autonomy runs for this CI job and exits 0 only if every run passed.
set -euo pipefail
set +x
: "${AUTONOMY_API_KEY:?Set an aut_ organization API key}"
: "${AUTONOMY_REQUEST_ID:?Set an ID that is unique for this CI job attempt}"
platform="${1:?Usage: bash .ci/autonomy.sh web|ios|android}"
api="${AUTONOMY_API_URL:-https://api.autonomyqa.com}"
api="${api%/}"

post() {
  curl --fail-with-body --silent --show-error \
    --connect-timeout 15 --max-time 90 \
    -X POST "$api$1" \
    -H "Authorization: Bearer $AUTONOMY_API_KEY" \
    -H 'Content-Type: application/json' --data-binary "$2"
}

# 1. What to run: a test plan or one test case.
if [ -n "${AUTONOMY_TEST_PLAN_ID:-}" ] && [ -z "${AUTONOMY_TEST_CASE_ID:-}" ]; then
  what=$(jq -n --arg id "$AUTONOMY_TEST_PLAN_ID" '{testPlanId:$id}')
elif [ -n "${AUTONOMY_TEST_CASE_ID:-}" ] && [ -z "${AUTONOMY_TEST_PLAN_ID:-}" ]; then
  what=$(jq -n --arg id "$AUTONOMY_TEST_CASE_ID" '{testCaseId:$id}')
else
  echo 'Set exactly one of AUTONOMY_TEST_PLAN_ID or AUTONOMY_TEST_CASE_ID.' >&2
  exit 2
fi

# 2. Where to run: a saved environment, or this job's preview URL or build.
if [ -n "${AUTONOMY_ENVIRONMENT_SLUG:-}" ]; then
  where=$(jq -n --arg slug "$AUTONOMY_ENVIRONMENT_SLUG" '{environmentSlug:$slug}')
else
  case "$platform" in
    web)
      : "${PREVIEW_URL:?Set PREVIEW_URL, or AUTONOMY_ENVIRONMENT_SLUG}"
      where=$(jq -n --arg url "$PREVIEW_URL" '{targets:{web:{baseUrl:$url}}}')
      ;;
    ios|android)
      : "${ARTIFACT_PATH:?Set the path to the zipped .app, .apk or .aab}"
      : "${APP_IDENTIFIER:?Set the iOS bundle ID or Android package name}"
      file=$(basename "$ARTIFACT_PATH")
      bytes=$(wc -c < "$ARTIFACT_PATH" | tr -d '[:space:]')
      if [ "$platform" = ios ]; then
        type=application/zip
      else
        type=application/vnd.android.package-archive
      fi
      upload=$(post /api/artifacts/upload-url "$(jq -n \
        --arg platform "$platform" --arg fileName "$file" \
        --arg contentType "$type" --argjson declaredBytes "$bytes" \
        '{platform:$platform,fileName:$fileName,contentType:$contentType,declaredBytes:$declaredBytes}')")
      storage=$(jq -er '.storageId' <<< "$upload")
      intent=$(jq -er '.intentId' <<< "$upload")
      curl --fail-with-body --silent --show-error \
        --connect-timeout 15 --max-time 600 \
        -X PUT "$(jq -er '.uploadUrl' <<< "$upload")" \
        -H "Content-Type: $type" --data-binary "@$ARTIFACT_PATH"
      scan=$(post /api/artifacts/scan "$(jq -n \
        --arg storageId "$storage" --arg intentId "$intent" \
        '{storageId:$storageId,intentId:$intentId}')")
      if [ "$(jq -r '.status' <<< "$scan")" != clean ]; then
        echo 'The build did not pass the safety check, so no run was started:' >&2
        jq -r '.findings[]?.message' <<< "$scan" >&2
        exit 1
      fi
      where=$(jq -n --arg p "$platform" --arg storageId "$storage" \
        --arg fileName "$file" --arg id "$APP_IDENTIFIER" \
        '{targets:{($p):({storageId:$storageId,sourceMode:"upload",fileName:$fileName}
          + (if $p == "ios" then {bundleId:$id} else {packageName:$id} end))}}')
      ;;
    *) echo 'Usage: bash .ci/autonomy.sh web|ios|android' >&2; exit 2 ;;
  esac
fi

# 3. Start the runs. A plan starts one run per test case.
#    The request ID's prefix (github, gitlab, ...) names the CI provider.
body=$(jq -n --argjson what "$what" --argjson where "$where" \
  --arg platform "$platform" \
  --arg branch "${AUTONOMY_BRANCH:-}" --arg pr "${AUTONOMY_PR:-}" \
  --arg commitSha "${AUTONOMY_COMMIT_SHA:-}" \
  --arg provider "${AUTONOMY_REQUEST_ID%%:*}" \
  --arg key "$AUTONOMY_REQUEST_ID:$platform" \
  '$what + $where + ({
     platforms: [$platform],
     branch: $branch,
     pr: $pr,
     deployment: ({provider: $provider, branch: $branch, commitSha: $commitSha}
       | with_entries(select(.value != ""))),
     idempotencyKey: $key
   } | with_entries(select(.value != "")))')
run_ids=$(post /api/v1/run.trigger "$body" | jq -er '.runIds[]')
printf 'Started Autonomy runs:\n%s\n' "$run_ids"

# 4. Follow each run until it ends. run.get waits up to 25 s for a change;
#    sending the last cursor back makes each call wait for something new.
deadline=$((SECONDS + ${AUTONOMY_WAIT_SECONDS:-1200}))
all_passed=true
for run_id in $run_ids; do
  cursor=null
  while :; do
    left=$((deadline - SECONDS))
    if [ "$left" -le 0 ]; then
      echo "Stopped waiting for run $run_id after AUTONOMY_WAIT_SECONDS. It keeps going in Autonomy." >&2
      exit 1
    fi
    wait_ms=$(( left < 25 ? left * 1000 : 25000 ))
    run=$(post /api/v1/run.get "$(jq -n --arg runId "$run_id" \
      --argjson cursor "$cursor" --argjson waitMs "$wait_ms" \
      '{runId:$runId,cursor:$cursor,waitMs:$waitMs}')")
    cursor=$(jq '.cursor' <<< "$run")
    status=$(jq -r '.status' <<< "$run")
    [ "$status" = queued ] || [ "$status" = running ] || break
  done

  name="$(jq -r '.testCaseName' <<< "$run") ($run_id)"
  case "$status" in
    passed)
      echo "Passed: $name" ;;
    failed)
      echo "Failed: $name. Open the run in Autonomy to see which step failed." >&2
      all_passed=false ;;
    couldnt_start)
      echo "Couldn't start: $name. The environment couldn't be tested, so this says nothing about your app. Open the run for the reason, fix it and run again." >&2
      all_passed=false ;;
    cancelled)
      echo "Cancelled: $name." >&2
      all_passed=false ;;
    *)
      echo "Unexpected status $status: $name." >&2
      all_passed=false ;;
  esac
done

if [ "$all_passed" != true ]; then
  echo 'Not every Autonomy run passed.' >&2
  exit 1
fi
echo 'Every Autonomy run passed.'

Save this as .github/workflows/autonomy-web.yml. Start it from Actions → Autonomy web → Run workflow, optionally with a preview URL.

.github/workflows/autonomy-web.yml

name: Autonomy web

on:
  push:
    branches: [main]
  workflow_dispatch:
    inputs:
      preview_url:
        description: Preview URL to test
        required: false

jobs:
  autonomy:
    runs-on: ubuntu-latest
    timeout-minutes: 30
    steps:
      - uses: actions/checkout@v5
      - name: Run Autonomy
        env:
          AUTONOMY_API_KEY: ${{ secrets.AUTONOMY_API_KEY }}
          AUTONOMY_TEST_PLAN_ID: ${{ vars.AUTONOMY_TEST_PLAN_ID }}
          PREVIEW_URL: ${{ inputs.preview_url || vars.PREVIEW_URL }}
          AUTONOMY_REQUEST_ID: github-actions:${{ github.run_id }}:${{ github.run_attempt }}
          AUTONOMY_BRANCH: ${{ github.head_ref || github.ref_name }}
          AUTONOMY_COMMIT_SHA: ${{ github.sha }}
        run: bash .ci/autonomy.sh web

If an earlier job deploys the preview, expose its URL as a job output, add that job to needs, and set PREVIEW_URL: ${{ needs.deploy.outputs.url }}.

Mobile artifact handoff

Use the same .ci/autonomy.sh. For iOS and Android it asks for an upload URL, uploads the build, waits for the safety check to come back clean and then starts the runs with the uploaded build.

iOS

iOS testing is coming soon. Keep this for when it's available. Until then, Autonomy doesn't start runs on iOS.

Build the simulator .app on macOS, zip it there so permissions and symlinks survive, and pass the zip to a Linux job as an Actions artifact. The upload and the wait then run on Linux minutes. Replace MyApp and the workspace path with your own; for a project without a workspace, use -project ios/MyApp.xcodeproj. Build for the simulator, not a device .ipa.

.github/workflows/autonomy-ios.yml

name: Autonomy iOS

on:
  push:
    branches: [main]
  workflow_dispatch:

jobs:
  build:
    runs-on: macos-latest
    timeout-minutes: 60
    steps:
      - uses: actions/checkout@v5
      - name: Build and zip the simulator app
        run: |
          set -euo pipefail
          xcodebuild \
            -workspace ios/MyApp.xcworkspace \
            -scheme MyApp \
            -configuration Release \
            -sdk iphonesimulator \
            -destination 'generic/platform=iOS Simulator' \
            -derivedDataPath ios/build \
            CODE_SIGNING_ALLOWED=NO \
            build
          mkdir -p .build
          (cd ios/build/Build/Products/Release-iphonesimulator && \
            zip -qry "$GITHUB_WORKSPACE/.build/MyApp.app.zip" MyApp.app)
      - uses: actions/upload-artifact@v5
        with:
          name: ios-simulator-app
          path: .build/MyApp.app.zip
          retention-days: 1

  autonomy:
    needs: build
    runs-on: ubuntu-latest
    timeout-minutes: 35
    steps:
      - uses: actions/checkout@v5
      - uses: actions/download-artifact@v5
        with:
          name: ios-simulator-app
          path: .build
      - name: Run Autonomy
        env:
          AUTONOMY_API_KEY: ${{ secrets.AUTONOMY_API_KEY }}
          AUTONOMY_TEST_PLAN_ID: ${{ vars.AUTONOMY_TEST_PLAN_ID }}
          APP_IDENTIFIER: ${{ vars.AUTONOMY_IOS_BUNDLE_ID }}
          ARTIFACT_PATH: .build/MyApp.app.zip
          AUTONOMY_REQUEST_ID: github-actions:${{ github.run_id }}:${{ github.run_attempt }}
          AUTONOMY_BRANCH: ${{ github.ref_name }}
          AUTONOMY_COMMIT_SHA: ${{ github.sha }}
        run: bash .ci/autonomy.sh ios

Android

Android builds run on Linux, so build and upload fit in one job. Adjust the Gradle task and the APK path together if you use another module or build variant. An .aab works too.

.github/workflows/autonomy-android.yml

name: Autonomy Android

on:
  push:
    branches: [main]
  workflow_dispatch:

jobs:
  autonomy:
    runs-on: ubuntu-latest
    timeout-minutes: 60
    steps:
      - uses: actions/checkout@v5
      - uses: actions/setup-java@v4
        with:
          distribution: temurin
          java-version: '17'
      - name: Build the debug APK
        run: |
          set -euo pipefail
          cd android
          ./gradlew --no-daemon assembleDebug
      - name: Run Autonomy
        env:
          AUTONOMY_API_KEY: ${{ secrets.AUTONOMY_API_KEY }}
          AUTONOMY_TEST_PLAN_ID: ${{ vars.AUTONOMY_TEST_PLAN_ID }}
          APP_IDENTIFIER: ${{ vars.AUTONOMY_ANDROID_PACKAGE_NAME }}
          ARTIFACT_PATH: android/app/build/outputs/apk/debug/app-debug.apk
          AUTONOMY_REQUEST_ID: github-actions:${{ github.run_id }}:${{ github.run_attempt }}
          AUTONOMY_BRANCH: ${{ github.ref_name }}
          AUTONOMY_COMMIT_SHA: ${{ github.sha }}
        run: bash .ci/autonomy.sh android

If the build is already at a short-lived HTTPS link, you can skip the upload: call run.trigger with targets.ios.artifactUrl or targets.android.artifactUrl and sourceMode: "url". See Mobile builds.

Gate the pipeline on the result

The helper starts the runs with POST /api/v1/run.trigger and prints their IDs. A test plan starts one run per test case. It then follows each run with POST /api/v1/run.get, sending back the last cursor with waitMs: 25000, so each call waits until the run changes. A response with timedOut: true only means nothing changed yet. See Runs and evidence.

Run result	The job	What the log says
Passed	Passes, if every other run passed too. A flaky pass counts as passed.	`Passed`
Failed	Fails.	Open the run to see which step failed and why, including a Runner problem.
Couldn't start	Fails.	The environment couldn't be tested, for example the build didn't install or a value was missing. This says nothing about your app; fix the cause and run again.
Cancelled	Fails.	`Cancelled`

API errors, a build that fails the safety check and running out of wait time also fail the job. AUTONOMY_WAIT_SECONDS is the total wait time for all runs, 1,200 seconds by default. Keep the CI job's time limit longer than that plus build and upload time. Stopping the CI job doesn't cancel runs that already started.

AUTONOMY_REQUEST_ID, with the platform added, becomes the idempotencyKey of run.trigger: if the same job attempt sends the request again, it gets the same runs back instead of new ones. Give every job attempt its own ID. The part before the first colon is sent as the deployment provider, and the branch and commit go with the runs. The full request and response shapes are in Operations.

GitHub Actions

Workflow placement

Required secrets

Web preview URL

Mobile artifact handoff

iOS

Android

Gate the pipeline on the result

Branch and event strategy

Troubleshooting

The web run starts before the preview is ready

The upload is rejected

Steps that use environment values couldn't start

The Android build fails to find the SDK

On this page