Start Autonomy runs from Jenkins for a preview URL or a mobile build, and fail the build unless every run passes.

Add Autonomy to a Jenkins Pipeline after your preview deployment or mobile build. This page has the shared helper and three complete Jenkinsfiles; choose the one for your platform. The build passes only when every run is Passed.

Workflow placement

Use a Pipeline job with Pipeline script from SCM, pointing at a trusted branch and the Jenkinsfile in your repository. Commit .ci/autonomy.sh next to it, create the credentials below, and click Build Now. No webhook or Multibranch Pipeline is needed.

In an existing pipeline, put the Autonomy stage after deployment has finished and the preview responds. The web example uses an already deployed URL. For a dynamic preview, remove the autonomy-preview-url binding and set env.PREVIEW_URL from your deployment stage before this stage.

Install the Pipeline, Pipeline: Declarative, Git and Credentials Binding plugins and their dependencies. Provide a Linux agent labeled linux with Bash, Git, jq and curl 7.76 or later. Labels select your agents; Jenkins doesn't supply hosted Linux or macOS machines.

Required secrets

In Manage Jenkins → Credentials, add Secret text credentials this job can see. Keep the IDs exactly as shown. Create only the entries your platform needs.

Credential ID	Value
`autonomy-api-key`	An organization API key starting with `aut_`. Owners and admins create keys in Organization → API keys.
`autonomy-web-plan-id`	ID of your web test plan: the last part of `/p/<project>/plans/<id>` in Test plans.
`autonomy-preview-url`	The final HTTPS preview URL, for the web example.
`autonomy-ios-plan-id`	ID of your iOS test plan.
`autonomy-ios-bundle-id`	The built app's bundle ID, for example `com.example.app`.
`autonomy-android-plan-id`	ID of your Android test plan.
`autonomy-android-package-name`	The built APK's application ID, for example `com.example.app`.

Create a test plan with at least one test case for the platform. To run one test case instead, bind its ID (from /p/<project>/tests/<id>) to AUTONOMY_TEST_CASE_ID in place of AUTONOMY_TEST_PLAN_ID. Set exactly one of the two.
Make the preview reachable from the internet. A URL that only Jenkins can reach won't work.
These examples send the URL or build directly, without an environment, so runs have no environment values: a step that uses {{env.NAME}} ends as Couldn't start. To use a saved environment of the same project instead, with its targets and values, set AUTONOMY_ENVIRONMENT_SLUG to its slug; the helper then skips the upload.

withCredentials injects values only around the API call. The single-quoted Groovy shell blocks leave expansion to Bash; keep set +x and never print the key.

Web preview URL

Save this shared helper as .ci/autonomy.sh. All three Jenkinsfiles below use this same file. It starts the runs, prints their IDs and waits for every result.

.ci/autonomy.sh

#!/usr/bin/env bash
# Usage: bash .ci/autonomy.sh web|ios|android
# Starts Autonomy runs for this CI job and exits 0 only if every run passed.
set -euo pipefail
set +x
: "${AUTONOMY_API_KEY:?Set an aut_ organization API key}"
: "${AUTONOMY_REQUEST_ID:?Set an ID that is unique for this CI job attempt}"
platform="${1:?Usage: bash .ci/autonomy.sh web|ios|android}"
api="${AUTONOMY_API_URL:-https://api.autonomyqa.com}"
api="${api%/}"

post() {
  curl --fail-with-body --silent --show-error \
    --connect-timeout 15 --max-time 90 \
    -X POST "$api$1" \
    -H "Authorization: Bearer $AUTONOMY_API_KEY" \
    -H 'Content-Type: application/json' --data-binary "$2"
}

# 1. What to run: a test plan or one test case.
if [ -n "${AUTONOMY_TEST_PLAN_ID:-}" ] && [ -z "${AUTONOMY_TEST_CASE_ID:-}" ]; then
  what=$(jq -n --arg id "$AUTONOMY_TEST_PLAN_ID" '{testPlanId:$id}')
elif [ -n "${AUTONOMY_TEST_CASE_ID:-}" ] && [ -z "${AUTONOMY_TEST_PLAN_ID:-}" ]; then
  what=$(jq -n --arg id "$AUTONOMY_TEST_CASE_ID" '{testCaseId:$id}')
else
  echo 'Set exactly one of AUTONOMY_TEST_PLAN_ID or AUTONOMY_TEST_CASE_ID.' >&2
  exit 2
fi

# 2. Where to run: a saved environment, or this job's preview URL or build.
if [ -n "${AUTONOMY_ENVIRONMENT_SLUG:-}" ]; then
  where=$(jq -n --arg slug "$AUTONOMY_ENVIRONMENT_SLUG" '{environmentSlug:$slug}')
else
  case "$platform" in
    web)
      : "${PREVIEW_URL:?Set PREVIEW_URL, or AUTONOMY_ENVIRONMENT_SLUG}"
      where=$(jq -n --arg url "$PREVIEW_URL" '{targets:{web:{baseUrl:$url}}}')
      ;;
    ios|android)
      : "${ARTIFACT_PATH:?Set the path to the zipped .app, .apk or .aab}"
      : "${APP_IDENTIFIER:?Set the iOS bundle ID or Android package name}"
      file=$(basename "$ARTIFACT_PATH")
      bytes=$(wc -c < "$ARTIFACT_PATH" | tr -d '[:space:]')
      if [ "$platform" = ios ]; then
        type=application/zip
      else
        type=application/vnd.android.package-archive
      fi
      upload=$(post /api/artifacts/upload-url "$(jq -n \
        --arg platform "$platform" --arg fileName "$file" \
        --arg contentType "$type" --argjson declaredBytes "$bytes" \
        '{platform:$platform,fileName:$fileName,contentType:$contentType,declaredBytes:$declaredBytes}')")
      storage=$(jq -er '.storageId' <<< "$upload")
      intent=$(jq -er '.intentId' <<< "$upload")
      curl --fail-with-body --silent --show-error \
        --connect-timeout 15 --max-time 600 \
        -X PUT "$(jq -er '.uploadUrl' <<< "$upload")" \
        -H "Content-Type: $type" --data-binary "@$ARTIFACT_PATH"
      scan=$(post /api/artifacts/scan "$(jq -n \
        --arg storageId "$storage" --arg intentId "$intent" \
        '{storageId:$storageId,intentId:$intentId}')")
      if [ "$(jq -r '.status' <<< "$scan")" != clean ]; then
        echo 'The build did not pass the safety check, so no run was started:' >&2
        jq -r '.findings[]?.message' <<< "$scan" >&2
        exit 1
      fi
      where=$(jq -n --arg p "$platform" --arg storageId "$storage" \
        --arg fileName "$file" --arg id "$APP_IDENTIFIER" \
        '{targets:{($p):({storageId:$storageId,sourceMode:"upload",fileName:$fileName}
          + (if $p == "ios" then {bundleId:$id} else {packageName:$id} end))}}')
      ;;
    *) echo 'Usage: bash .ci/autonomy.sh web|ios|android' >&2; exit 2 ;;
  esac
fi

# 3. Start the runs. A plan starts one run per test case.
#    The request ID's prefix (github, gitlab, ...) names the CI provider.
body=$(jq -n --argjson what "$what" --argjson where "$where" \
  --arg platform "$platform" \
  --arg branch "${AUTONOMY_BRANCH:-}" --arg pr "${AUTONOMY_PR:-}" \
  --arg commitSha "${AUTONOMY_COMMIT_SHA:-}" \
  --arg provider "${AUTONOMY_REQUEST_ID%%:*}" \
  --arg key "$AUTONOMY_REQUEST_ID:$platform" \
  '$what + $where + ({
     platforms: [$platform],
     branch: $branch,
     pr: $pr,
     deployment: ({provider: $provider, branch: $branch, commitSha: $commitSha}
       | with_entries(select(.value != ""))),
     idempotencyKey: $key
   } | with_entries(select(.value != "")))')
run_ids=$(post /api/v1/run.trigger "$body" | jq -er '.runIds[]')
printf 'Started Autonomy runs:\n%s\n' "$run_ids"

# 4. Follow each run until it ends. run.get waits up to 25 s for a change;
#    sending the last cursor back makes each call wait for something new.
deadline=$((SECONDS + ${AUTONOMY_WAIT_SECONDS:-1200}))
all_passed=true
for run_id in $run_ids; do
  cursor=null
  while :; do
    left=$((deadline - SECONDS))
    if [ "$left" -le 0 ]; then
      echo "Stopped waiting for run $run_id after AUTONOMY_WAIT_SECONDS. It keeps going in Autonomy." >&2
      exit 1
    fi
    wait_ms=$(( left < 25 ? left * 1000 : 25000 ))
    run=$(post /api/v1/run.get "$(jq -n --arg runId "$run_id" \
      --argjson cursor "$cursor" --argjson waitMs "$wait_ms" \
      '{runId:$runId,cursor:$cursor,waitMs:$waitMs}')")
    cursor=$(jq '.cursor' <<< "$run")
    status=$(jq -r '.status' <<< "$run")
    [ "$status" = queued ] || [ "$status" = running ] || break
  done

  name="$(jq -r '.testCaseName' <<< "$run") ($run_id)"
  case "$status" in
    passed)
      echo "Passed: $name" ;;
    failed)
      echo "Failed: $name. Open the run in Autonomy to see which step failed." >&2
      all_passed=false ;;
    couldnt_start)
      echo "Couldn't start: $name. The environment couldn't be tested, so this says nothing about your app. Open the run for the reason, fix it and run again." >&2
      all_passed=false ;;
    cancelled)
      echo "Cancelled: $name." >&2
      all_passed=false ;;
    *)
      echo "Unexpected status $status: $name." >&2
      all_passed=false ;;
  esac
done

if [ "$all_passed" != true ]; then
  echo 'Not every Autonomy run passed.' >&2
  exit 1
fi
echo 'Every Autonomy run passed.'

Save this as Jenkinsfile for a web job. The three web credentials above make the first manual build work without job parameters.

Jenkinsfile

pipeline {
  agent { label 'linux' }
  options {
    skipDefaultCheckout(true)
    timeout(time: 30, unit: 'MINUTES')
  }
  stages {
    stage('Checkout') {
      steps { checkout scm }
    }
    stage('Autonomy web') {
      steps {
        withCredentials([
          string(credentialsId: 'autonomy-api-key', variable: 'AUTONOMY_API_KEY'),
          string(credentialsId: 'autonomy-web-plan-id', variable: 'AUTONOMY_TEST_PLAN_ID'),
          string(credentialsId: 'autonomy-preview-url', variable: 'PREVIEW_URL')
        ]) {
          sh '''#!/usr/bin/env bash
set -euo pipefail
set +x
export AUTONOMY_REQUEST_ID="jenkins:${JOB_NAME}:${BUILD_NUMBER}"
export AUTONOMY_BRANCH="${BRANCH_NAME:-${GIT_BRANCH:-manual}}"
export AUTONOMY_COMMIT_SHA="$(git rev-parse HEAD)"
bash .ci/autonomy.sh web
'''
        }
      }
    }
  }
}

Mobile artifact handoff

Use a separate test plan for each platform. Keep .ci/autonomy.sh from the web section and replace Jenkinsfile with the example you need. For iOS and Android the helper asks for an upload URL, uploads the build, waits for the safety check to come back clean and then starts the runs.

iOS

iOS testing is coming soon. Keep this for when it's available. Until then, Autonomy doesn't start runs on iOS.

Provide a macOS agent labeled macos with Xcode, its Simulator SDK, Git and zip. Jenkins has no hosted macOS machines, so use a Mac of your own. Install your project's dependencies before xcodebuild, and replace MyApp, its workspace path and the output name with your own.

Build a simulator .app, not a device .ipa. zip -y keeps symlinks, and the zip keeps executable permissions. stash passes the zip to the Linux stage, which uploads it as is. For large apps, configure a remote Jenkins artifact manager so stashing doesn't load the controller.

Jenkinsfile

pipeline {
  agent none
  options { skipDefaultCheckout(true) }
  stages {
    stage('Build iOS simulator app') {
      agent { label 'macos' }
      options { timeout(time: 60, unit: 'MINUTES') }
      steps {
        checkout scm
        sh '''#!/usr/bin/env bash
set -euo pipefail
xcodebuild \
  -workspace ios/MyApp.xcworkspace \
  -scheme MyApp \
  -configuration Release \
  -sdk iphonesimulator \
  -destination 'generic/platform=iOS Simulator' \
  -derivedDataPath ios/build \
  CODE_SIGNING_ALLOWED=NO build
mkdir -p .build
rm -f .build/MyApp.app.zip
(cd ios/build/Build/Products/Release-iphonesimulator && \
  zip -qry "$WORKSPACE/.build/MyApp.app.zip" MyApp.app)
'''
        stash name: 'ios-simulator', includes: '.build/MyApp.app.zip'
      }
    }
    stage('Autonomy iOS') {
      agent { label 'linux' }
      options { timeout(time: 35, unit: 'MINUTES') }
      steps {
        checkout scm
        unstash 'ios-simulator'
        withCredentials([
          string(credentialsId: 'autonomy-api-key', variable: 'AUTONOMY_API_KEY'),
          string(credentialsId: 'autonomy-ios-plan-id', variable: 'AUTONOMY_TEST_PLAN_ID'),
          string(credentialsId: 'autonomy-ios-bundle-id', variable: 'APP_IDENTIFIER')
        ]) {
          sh '''#!/usr/bin/env bash
set -euo pipefail
set +x
export ARTIFACT_PATH='.build/MyApp.app.zip'
export AUTONOMY_REQUEST_ID="jenkins:${JOB_NAME}:${BUILD_NUMBER}"
export AUTONOMY_BRANCH="${BRANCH_NAME:-${GIT_BRANCH:-manual}}"
export AUTONOMY_COMMIT_SHA="$(git rev-parse HEAD)"
bash .ci/autonomy.sh ios
'''
        }
      }
    }
  }
}

Android

Provide an agent labeled linux-android with Bash, Git, jq, curl 7.76 or later, a JDK that matches your Gradle version, and the Android SDK packages your project needs. Set JAVA_HOME and ANDROID_HOME on the agent and accept the SDK licenses. The example assumes an executable Gradle wrapper at android/gradlew; adjust the folder and APK path for your project. Build and upload share the workspace. An .aab works too.

Jenkinsfile

pipeline {
  agent { label 'linux-android' }
  options {
    skipDefaultCheckout(true)
    timeout(time: 60, unit: 'MINUTES')
  }
  stages {
    stage('Build Android APK') {
      steps {
        checkout scm
        sh '''#!/usr/bin/env bash
set -euo pipefail
cd android
./gradlew --no-daemon assembleDebug
'''
      }
    }
    stage('Autonomy Android') {
      steps {
        withCredentials([
          string(credentialsId: 'autonomy-api-key', variable: 'AUTONOMY_API_KEY'),
          string(credentialsId: 'autonomy-android-plan-id', variable: 'AUTONOMY_TEST_PLAN_ID'),
          string(credentialsId: 'autonomy-android-package-name', variable: 'APP_IDENTIFIER')
        ]) {
          sh '''#!/usr/bin/env bash
set -euo pipefail
set +x
export ARTIFACT_PATH='android/app/build/outputs/apk/debug/app-debug.apk'
export AUTONOMY_REQUEST_ID="jenkins:${JOB_NAME}:${BUILD_NUMBER}"
export AUTONOMY_BRANCH="${BRANCH_NAME:-${GIT_BRANCH:-manual}}"
export AUTONOMY_COMMIT_SHA="$(git rev-parse HEAD)"
bash .ci/autonomy.sh android
'''
        }
      }
    }
  }
}

Gate the pipeline on the result

The helper starts the runs with POST /api/v1/run.trigger and prints their IDs. A test plan starts one run per test case. It then follows each run with POST /api/v1/run.get, sending back the last cursor with waitMs: 25000, so each call waits until the run changes. A response with timedOut: true only means nothing changed yet. See Runs and evidence.

Run result	The job	What the log says
Passed	Passes, if every other run passed too. A flaky pass counts as passed.	`Passed`
Failed	Fails.	Open the run to see which step failed and why, including a Runner problem.
Couldn't start	Fails.	The environment couldn't be tested, for example the build didn't install or a value was missing. This says nothing about your app; fix the cause and run again.
Cancelled	Fails.	`Cancelled`

API errors, a build that fails the safety check and running out of wait time also fail the job. AUTONOMY_WAIT_SECONDS is the total wait time for all runs, 1,200 seconds by default. Keep the CI job's time limit longer than that plus build and upload time. Stopping the CI job doesn't cancel runs that already started.

AUTONOMY_REQUEST_ID, with the platform added, becomes the idempotencyKey of run.trigger: if the same job attempt sends the request again, it gets the same runs back instead of new ones. Give every job attempt its own ID. The part before the first colon is sent as the deployment provider, and the branch and commit go with the runs. The full request and response shapes are in Operations.

A nonzero exit fails the Jenkins stage. Raise the Jenkins timeout together with AUTONOMY_WAIT_SECONDS for longer plans.

Branch and event strategy

Start with manual builds from a trusted default branch. Once the deployment handoff works, add your existing SCM trigger or schedule to the job. Use short smoke plans for trusted change builds and larger plans for release or scheduled builds.

The checked-out scripts can read these credentials. Don't run an untrusted pull request's Jenkinsfile or helper with them, even if its branch name matches a filter. Use a trusted job to test an approved preview URL. Keep agents that hold these credentials apart from agents that run untrusted jobs.

Vendor references

The examples use the documented Declarative Pipeline syntax, Secret text bindings, stash and unstash steps and Jenkins agents. See Using a Jenkinsfile for Pipeline from SCM setup.

Jenkins

On this page