Start Autonomy runs from a Bitrise workflow for a web preview or an Android APK, and fail the build unless every run passes.

Use a Bitrise Script Step to upload a mobile build or test a deployed web preview, then keep the workflow open until every Autonomy run has a result. The step passes only when every run is Passed.

Workflow placement

Add the Autonomy step after your existing build and deployment steps. For web, export the final public preview URL as PREVIEW_URL from the deployment step with envman add --key PREVIEW_URL --value "$PREVIEW_URL"; later steps can read it. For mobile, keep the build and the upload in the same workflow so the build stays in the workspace.

The configurations below are complete alternatives for bitrise.yml. Copy the one for your platform, or merge its steps into your existing workflow. Set Bitrise to read bitrise.yml from the repository and commit .ci/autonomy.sh next to it. The web workflow can also run locally in a checked-out repository.

Required secrets

Before the first run

Create an organization API key starting with aut_ in Organization → API keys. Owners and admins can create keys.
Create a test plan under Test plans with at least one test case for the platform, and copy its ID.
For this recipe without an environment, use test cases that don't use environment values, or set AUTONOMY_ENVIRONMENT_SLUG.
Make the preview URL reachable from the internet.

In Workflows → Secrets, add AUTONOMY_API_KEY. Leave Expose for pull requests off and Replace variables in inputs unchecked. Secrets become shell environment variables automatically; don't put the key in bitrise.yml.

In Workflows → Env Vars, set these values for the workflow:

Variable	Value
`AUTONOMY_TEST_PLAN_ID`	The test plan to run: the last part of `/p/<project>/plans/<id>` in Test plans. Use a separate plan for each platform. To run one test case instead, set `AUTONOMY_TEST_CASE_ID` (from `/p/<project>/tests/<id>`) and leave this unset.
`PREVIEW_URL`	Web only: the final deployed URL, without a login or preview protection page. A deployment step can set it instead.
`APP_IDENTIFIER`	Mobile only: the iOS bundle ID or Android application ID of the build.
`AUTONOMY_ENVIRONMENT_SLUG`	Optional: run against a saved environment of the same project, with its targets and values, instead of this build's URL or app. The helper then skips the upload.
`AUTONOMY_WAIT_SECONDS`	Optional: the total wait time in seconds, `1200` by default.

Without an environment, a run has no environment values: a step that uses {{env.NAME}} ends as Couldn't start.

Web preview URL

Create a .ci folder in the repository and save this whole script as .ci/autonomy.sh. It needs Bash, curl 7.76 or newer and jq; the hosted stacks below have them. Install them yourself for a local CLI run. Keep this file the same for every platform.

.ci/autonomy.sh

#!/usr/bin/env bash
# Usage: bash .ci/autonomy.sh web|ios|android
# Starts Autonomy runs for this CI job and exits 0 only if every run passed.
set -euo pipefail
set +x
: "${AUTONOMY_API_KEY:?Set an aut_ organization API key}"
: "${AUTONOMY_REQUEST_ID:?Set an ID that is unique for this CI job attempt}"
platform="${1:?Usage: bash .ci/autonomy.sh web|ios|android}"
api="${AUTONOMY_API_URL:-https://api.autonomyqa.com}"
api="${api%/}"

post() {
  curl --fail-with-body --silent --show-error \
    --connect-timeout 15 --max-time 90 \
    -X POST "$api$1" \
    -H "Authorization: Bearer $AUTONOMY_API_KEY" \
    -H 'Content-Type: application/json' --data-binary "$2"
}

# 1. What to run: a test plan or one test case.
if [ -n "${AUTONOMY_TEST_PLAN_ID:-}" ] && [ -z "${AUTONOMY_TEST_CASE_ID:-}" ]; then
  what=$(jq -n --arg id "$AUTONOMY_TEST_PLAN_ID" '{testPlanId:$id}')
elif [ -n "${AUTONOMY_TEST_CASE_ID:-}" ] && [ -z "${AUTONOMY_TEST_PLAN_ID:-}" ]; then
  what=$(jq -n --arg id "$AUTONOMY_TEST_CASE_ID" '{testCaseId:$id}')
else
  echo 'Set exactly one of AUTONOMY_TEST_PLAN_ID or AUTONOMY_TEST_CASE_ID.' >&2
  exit 2
fi

# 2. Where to run: a saved environment, or this job's preview URL or build.
if [ -n "${AUTONOMY_ENVIRONMENT_SLUG:-}" ]; then
  where=$(jq -n --arg slug "$AUTONOMY_ENVIRONMENT_SLUG" '{environmentSlug:$slug}')
else
  case "$platform" in
    web)
      : "${PREVIEW_URL:?Set PREVIEW_URL, or AUTONOMY_ENVIRONMENT_SLUG}"
      where=$(jq -n --arg url "$PREVIEW_URL" '{targets:{web:{baseUrl:$url}}}')
      ;;
    ios|android)
      : "${ARTIFACT_PATH:?Set the path to the zipped .app, .apk or .aab}"
      : "${APP_IDENTIFIER:?Set the iOS bundle ID or Android package name}"
      file=$(basename "$ARTIFACT_PATH")
      bytes=$(wc -c < "$ARTIFACT_PATH" | tr -d '[:space:]')
      if [ "$platform" = ios ]; then
        type=application/zip
      else
        type=application/vnd.android.package-archive
      fi
      upload=$(post /api/artifacts/upload-url "$(jq -n \
        --arg platform "$platform" --arg fileName "$file" \
        --arg contentType "$type" --argjson declaredBytes "$bytes" \
        '{platform:$platform,fileName:$fileName,contentType:$contentType,declaredBytes:$declaredBytes}')")
      storage=$(jq -er '.storageId' <<< "$upload")
      intent=$(jq -er '.intentId' <<< "$upload")
      curl --fail-with-body --silent --show-error \
        --connect-timeout 15 --max-time 600 \
        -X PUT "$(jq -er '.uploadUrl' <<< "$upload")" \
        -H "Content-Type: $type" --data-binary "@$ARTIFACT_PATH"
      scan=$(post /api/artifacts/scan "$(jq -n \
        --arg storageId "$storage" --arg intentId "$intent" \
        '{storageId:$storageId,intentId:$intentId}')")
      if [ "$(jq -r '.status' <<< "$scan")" != clean ]; then
        echo 'The build did not pass the safety check, so no run was started:' >&2
        jq -r '.findings[]?.message' <<< "$scan" >&2
        exit 1
      fi
      where=$(jq -n --arg p "$platform" --arg storageId "$storage" \
        --arg fileName "$file" --arg id "$APP_IDENTIFIER" \
        '{targets:{($p):({storageId:$storageId,sourceMode:"upload",fileName:$fileName}
          + (if $p == "ios" then {bundleId:$id} else {packageName:$id} end))}}')
      ;;
    *) echo 'Usage: bash .ci/autonomy.sh web|ios|android' >&2; exit 2 ;;
  esac
fi

# 3. Start the runs. A plan starts one run per test case.
#    The request ID's prefix (github, gitlab, ...) names the CI provider.
body=$(jq -n --argjson what "$what" --argjson where "$where" \
  --arg platform "$platform" \
  --arg branch "${AUTONOMY_BRANCH:-}" --arg pr "${AUTONOMY_PR:-}" \
  --arg commitSha "${AUTONOMY_COMMIT_SHA:-}" \
  --arg provider "${AUTONOMY_REQUEST_ID%%:*}" \
  --arg key "$AUTONOMY_REQUEST_ID:$platform" \
  '$what + $where + ({
     platforms: [$platform],
     branch: $branch,
     pr: $pr,
     deployment: ({provider: $provider, branch: $branch, commitSha: $commitSha}
       | with_entries(select(.value != ""))),
     idempotencyKey: $key
   } | with_entries(select(.value != "")))')
run_ids=$(post /api/v1/run.trigger "$body" | jq -er '.runIds[]')
printf 'Started Autonomy runs:\n%s\n' "$run_ids"

# 4. Follow each run until it ends. run.get waits up to 25 s for a change;
#    sending the last cursor back makes each call wait for something new.
deadline=$((SECONDS + ${AUTONOMY_WAIT_SECONDS:-1200}))
all_passed=true
for run_id in $run_ids; do
  cursor=null
  while :; do
    left=$((deadline - SECONDS))
    if [ "$left" -le 0 ]; then
      echo "Stopped waiting for run $run_id after AUTONOMY_WAIT_SECONDS. It keeps going in Autonomy." >&2
      exit 1
    fi
    wait_ms=$(( left < 25 ? left * 1000 : 25000 ))
    run=$(post /api/v1/run.get "$(jq -n --arg runId "$run_id" \
      --argjson cursor "$cursor" --argjson waitMs "$wait_ms" \
      '{runId:$runId,cursor:$cursor,waitMs:$waitMs}')")
    cursor=$(jq '.cursor' <<< "$run")
    status=$(jq -r '.status' <<< "$run")
    [ "$status" = queued ] || [ "$status" = running ] || break
  done

  name="$(jq -r '.testCaseName' <<< "$run") ($run_id)"
  case "$status" in
    passed)
      echo "Passed: $name" ;;
    failed)
      echo "Failed: $name. Open the run in Autonomy to see which step failed." >&2
      all_passed=false ;;
    couldnt_start)
      echo "Couldn't start: $name. The environment couldn't be tested, so this says nothing about your app. Open the run for the reason, fix it and run again." >&2
      all_passed=false ;;
    cancelled)
      echo "Cancelled: $name." >&2
      all_passed=false ;;
    *)
      echo "Unexpected status $status: $name." >&2
      all_passed=false ;;
  esac
done

if [ "$all_passed" != true ]; then
  echo 'Not every Autonomy run passed.' >&2
  exit 1
fi
echo 'Every Autonomy run passed.'

Save this configuration as bitrise.yml. Set the workflow variables above, then choose Start build → web after deploying the preview. The checkout step runs on Bitrise; the local CLI uses the repository you already checked out.

bitrise.yml

format_version: '13'
default_step_lib_source: https://github.com/bitrise-io/bitrise-steplib.git
workflows:
  web:
    meta:
      bitrise.io:
        stack: linux-docker-android-22.04
        machine_type_id: standard
    steps:
      - activate-ssh-key@4:
          run_if: '{{getenv "SSH_RSA_PRIVATE_KEY" | ne ""}}'
      - git-clone@8:
          run_if: '{{getenv "BITRISE_IO" | eq "true"}}'
      - script@1:
          title: Autonomy
          inputs:
          - content: |-
              #!/usr/bin/env bash
              set -euo pipefail
              set +x
              : "${PREVIEW_URL:?Set the deployed preview URL}"
              curl --fail --silent --show-error --retry 12 \
                --retry-delay 5 --retry-connrefused --max-time 20 \
                "$PREVIEW_URL" > /dev/null
              attempt="$(cat /proc/sys/kernel/random/uuid 2>/dev/null || uuidgen)"
              export AUTONOMY_REQUEST_ID="bitrise:${BITRISE_BUILD_SLUG:-local}:${BITRISE_TRIGGERED_WORKFLOW_ID}:$attempt"
              export AUTONOMY_BRANCH="${BITRISE_GIT_BRANCH:-$(git branch --show-current)}"
              export AUTONOMY_COMMIT_SHA="$(git rev-parse HEAD)"
              bash .ci/autonomy.sh web

To run it locally, install the Bitrise CLI, then save this as .bitrise.secrets.yml next to bitrise.yml. Replace the three example values with your API key, test plan ID and preview URL:

.bitrise.secrets.yml

envs:
- AUTONOMY_API_KEY: 'aut_REPLACE_WITH_ORG_KEY'
  opts:
    is_expand: false
- AUTONOMY_TEST_PLAN_ID: 'REPLACE_WITH_TEST_PLAN_ID'
  opts:
    is_expand: false
- PREVIEW_URL: 'https://your-preview.example.com'
  opts:
    is_expand: false

Run these commands from the repository root. Keep the secrets file out of version control. The local CLI runs on your machine; the meta stack only applies to hosted builds.

printf '\n.bitrise*\n' >> .gitignore
chmod 600 .bitrise.secrets.yml
bitrise setup
bitrise run web

Mobile artifact handoff

Use the same .ci/autonomy.sh. For iOS and Android it asks for an upload URL, sends the build with PUT, waits for the safety check to come back clean and starts the runs with the uploaded build. Build and upload share the workspace, so there's no artifact transfer to set up.

iOS

iOS testing is coming soon. Keep this for when it's available. Until then, Autonomy doesn't start runs on iOS.

This complete alternative bitrise.yml uses a macOS Xcode stack. Set AUTONOMY_TEST_PLAN_ID to your iOS plan and APP_IDENTIFIER to its bundle ID. Replace ios/MyApp.xcodeproj, MyApp and the .app name with your project, shared scheme and product. For a workspace, replace -project ios/MyApp.xcodeproj with -workspace ios/MyApp.xcworkspace. Keep any dependency installation steps before xcodebuild.

Build a simulator .app and zip it; a device .ipa can't be used. The upload and the wait also run on macOS and use its build time.

bitrise.yml

format_version: '13'
default_step_lib_source: https://github.com/bitrise-io/bitrise-steplib.git
workflows:
  ios:
    meta:
      bitrise.io:
        stack: osx-xcode-26.4.x
        machine_type_id: g2.mac.medium
    steps:
      - activate-ssh-key@4:
          run_if: '{{getenv "SSH_RSA_PRIVATE_KEY" | ne ""}}'
      - git-clone@8:
          run_if: '{{getenv "BITRISE_IO" | eq "true"}}'
      - script@1:
          title: Autonomy
          inputs:
          - content: |-
              #!/usr/bin/env bash
              set -euo pipefail
              set +x
              xcodebuild -project ios/MyApp.xcodeproj -scheme MyApp \
                -configuration Release -sdk iphonesimulator \
                -destination 'generic/platform=iOS Simulator' \
                -derivedDataPath .build/ios CODE_SIGNING_ALLOWED=NO build
              mkdir -p .build/artifacts
              ditto -c -k --sequesterRsrc --keepParent \
                .build/ios/Build/Products/Release-iphonesimulator/MyApp.app \
                .build/artifacts/MyApp.app.zip
              export ARTIFACT_PATH="$PWD/.build/artifacts/MyApp.app.zip"
              attempt="$(cat /proc/sys/kernel/random/uuid 2>/dev/null || uuidgen)"
              export AUTONOMY_REQUEST_ID="bitrise:${BITRISE_BUILD_SLUG:-local}:${BITRISE_TRIGGERED_WORKFLOW_ID}:$attempt"
              export AUTONOMY_BRANCH="${BITRISE_GIT_BRANCH:-$(git branch --show-current)}"
              export AUTONOMY_COMMIT_SHA="$(git rev-parse HEAD)"
              bash .ci/autonomy.sh ios

Android

This complete alternative bitrise.yml uses the Linux Android stack. Set AUTONOMY_TEST_PLAN_ID to your Android plan and APP_IDENTIFIER to its application ID. It assumes an Android project in android/, a checked-in Gradle wrapper and the app module. Change the folder, task and APK path together for another module or build variant. Keep the JDK and Android SDK versions your build needs. An .aab works too.

bitrise.yml

format_version: '13'
default_step_lib_source: https://github.com/bitrise-io/bitrise-steplib.git
workflows:
  android:
    meta:
      bitrise.io:
        stack: linux-docker-android-22.04
        machine_type_id: standard
    steps:
      - activate-ssh-key@4:
          run_if: '{{getenv "SSH_RSA_PRIVATE_KEY" | ne ""}}'
      - git-clone@8:
          run_if: '{{getenv "BITRISE_IO" | eq "true"}}'
      - script@1:
          title: Autonomy
          inputs:
          - content: |-
              #!/usr/bin/env bash
              set -euo pipefail
              set +x
              (cd android && bash ./gradlew --no-daemon assembleDebug)
              export ARTIFACT_PATH="$PWD/android/app/build/outputs/apk/debug/app-debug.apk"
              attempt="$(cat /proc/sys/kernel/random/uuid 2>/dev/null || uuidgen)"
              export AUTONOMY_REQUEST_ID="bitrise:${BITRISE_BUILD_SLUG:-local}:${BITRISE_TRIGGERED_WORKFLOW_ID}:$attempt"
              export AUTONOMY_BRANCH="${BITRISE_GIT_BRANCH:-$(git branch --show-current)}"
              export AUTONOMY_COMMIT_SHA="$(git rev-parse HEAD)"
              bash .ci/autonomy.sh android

Bitrise currently lists macOS Medium and Linux Medium for its free Hobby plan, with 300 credits a month and a 90-minute build time limit. The examples select g2.mac.medium and standard. A local bitrise run ios still needs your own Mac with Xcode; choosing a macOS stack in YAML doesn't create a remote Mac. Check your remaining credits before hosted mobile builds. See Bitrise pricing.

Gate the pipeline on the result

The helper starts the runs with POST /api/v1/run.trigger and prints their IDs. A test plan starts one run per test case. It then follows each run with POST /api/v1/run.get, sending back the last cursor with waitMs: 25000, so each call waits until the run changes. A response with timedOut: true only means nothing changed yet. See Runs and evidence.

Run result	The job	What the log says
Passed	Passes, if every other run passed too. A flaky pass counts as passed.	`Passed`
Failed	Fails.	Open the run to see which step failed and why, including a Runner problem.
Couldn't start	Fails.	The environment couldn't be tested, for example the build didn't install or a value was missing. This says nothing about your app; fix the cause and run again.
Cancelled	Fails.	`Cancelled`

API errors, a build that fails the safety check and running out of wait time also fail the job. AUTONOMY_WAIT_SECONDS is the total wait time for all runs, 1,200 seconds by default. Keep the CI job's time limit longer than that plus build and upload time. Stopping the CI job doesn't cancel runs that already started.

AUTONOMY_REQUEST_ID, with the platform added, becomes the idempotencyKey of run.trigger: if the same job attempt sends the request again, it gets the same runs back instead of new ones. Give every job attempt its own ID. The part before the first colon is sent as the deployment provider, and the branch and commit go with the runs. The full request and response shapes are in Operations.

A nonzero exit fails the Script Step and stops later normal steps. Don't add is_skippable: true to it. Each workflow start makes a new UUID next to the build slug and workflow ID, so a retry or a local run starts new runs. Open a run in the project's Runs using the printed ID to see its evidence.

Branch and event strategy

The examples are manual workflows with no automatic trigger, so setting them up doesn't expose an API key to untrusted code. Start with a small smoke plan, then set a push trigger for a trusted staging or release branch in Bitrise. Add the Autonomy step after deployment in that workflow and keep its branch filters. Keep larger plans for scheduled or manual runs.

Keep secrets away from pull request builds, including forked pull requests. Run tests on reviewed code once it reaches a trusted branch, or start a build of a reviewed commit by hand. Don't turn on pull request secret exposure to make a recipe pass. Branch and commit are sent to Autonomy; this recipe doesn't post a pull request comment.

Bitrise

Workflow placement

Required secrets

Web preview URL

Mobile artifact handoff

iOS

Android

Gate the pipeline on the result

Branch and event strategy

Vendor references

On this page