Autonomy Docs
Integrations

Netlify

Automatically run QA against every Netlify deploy preview and report the result via your linked GitHub repository.

The Netlify integration watches deploy previews and production deploys via webhook. When a deploy reaches the ready state, Autonomy runs the configured test case or test plan against the immutable per-deploy permalink and, if a GitHub repository is linked, reports the result as a GitHub check run on the commit.

What it does

  • Receives a webhook on every deploy state change (deploy_created, deploy_building, deploy_succeeded, deploy_failed, deploy_request_pending).
  • Filters by deploy context (deploy-preview, production, or branch-deploy) and branch name.
  • Runs the configured test case or test plan against the immutable per-deploy permalink URL.
  • Reports the result as a GitHub check run on the commit when a GitHub repository is linked.
  • Netlify has no native check surface — status reporting relies entirely on the linked GitHub repository.
  • For forked-PR deploys held by Netlify for team-owner approval (deploy_request_pending), the event appears with state "pending" in the dashboard until the deploy is approved.

Autonomy targets the immutable per-deploy permalink (https://<deploy_id>--site.netlify.app) rather than the mutable branch-scoped URL. This ensures QA results attribute to an exact commit — a newer push to the same PR cannot change what the runner sees mid-test. The branch-scoped deploy URL (deploy_ssl_url) is used only as a fallback when the permalink is absent.

Setup

Before you start

  • A Netlify account with at least one site that produces deploy previews.
  • At least one Autonomy test case that passes against a web target.
  • An Autonomy environment with a web target.
  • For status reporting: the Autonomy GitHub App installed on the repository that Netlify deploys from.
  • For self-hosted Autonomy: NETLIFY_CLIENT_ID and NETLIFY_CLIENT_SECRET configured on the Convex backend.
  1. Open Settings → Integrations in the Autonomy dashboard.
  2. Click Connect on the Netlify card. You are redirected to Netlify to authorize the OAuth application.
  3. Back in Autonomy, click Link Site. Autonomy lists the Netlify sites visible to your connected account — pick the one you want to test.
  4. Choose a Test Case or Test Plan as the default run target.
  5. Choose the Environment whose configuration should apply.
  6. Under Trigger contexts, select which deploy types start a run — typically preview. Add Branch filters to limit runs to specific branches.
  7. To report results back as a GitHub check, enable Report checks and select the Linked repository. The Autonomy GitHub App must be installed on that repository.
  8. Open a pull request and confirm that a run starts when the Netlify deploy preview is ready.

How a run is triggered

When Netlify sends a deploy_succeeded webhook:

  1. Autonomy looks up the linked site by the Netlify site ID.
  2. The Netlify deploy context (deploy-preview, production, branch-deploy) is normalized to Autonomy's vocabulary: preview, production, or branch.
  3. The context is checked against the site's trigger contexts and the branch against branch filters.
  4. If a default test target is configured, Autonomy creates a run targeting the immutable per-deploy permalink URL.

Forked-PR deploys

When a deploy comes from a forked pull request, Netlify holds it for team-owner approval and sends a deploy_request_pending webhook. Autonomy records the event with state "pending" — it appears in the dashboard as awaiting approval. Once the deploy is approved and succeeds, a deploy_succeeded webhook triggers the run as usual.

Password-protected sites

If the Netlify site has password protection enabled (site-wide or per-deploy), the runner cannot reach the deploy preview. To configure bypass:

  1. In Autonomy, open the linked site's settings.
  2. Paste the site password into Preview access secret.

Autonomy injects the password as required by Netlify's protection mechanism. The secret never leaves the backend.

Status reporting

Netlify does not expose a native check or status surface for third-party integrations. Autonomy cannot post results directly to the Netlify dashboard.

To report results, link a GitHub repository to the Netlify site in Autonomy. When enabled, Autonomy posts a GitHub check run on the commit — the same mechanism the GitHub App uses. The check appears on the pull request and can be made a required status check under branch protection.

Important: If Report checks is enabled but no GitHub repository is linked, results will only appear in the Autonomy dashboard — not on the pull request or anywhere in Netlify. The dashboard shows a warning when this is the case.

Monorepo and multiple sites

A single GitHub repository can deploy multiple Netlify sites. Link each site independently in Autonomy — each gets its own test target, environment, trigger contexts, and branch filters. All linked sites can share the same linked GitHub repository for status reporting.

When Netlify sends a webhook, Autonomy matches it to the correct linked site by the Netlify site ID.

Environment variables (self-hosted)

VariableRequiredDescription
NETLIFY_CLIENT_IDYesOAuth client ID from the Netlify OAuth application settings.
NETLIFY_CLIENT_SECRETYesOAuth client secret from the Netlify OAuth application settings.

Troubleshooting

No run starts when a deploy preview is ready

Check the following:

  • The Netlify account is connected and showing Active in Autonomy.
  • The Netlify site is linked. Autonomy only watches linked sites.
  • The site is enabled. A disabled site records the event but skips the run (project disabled).
  • The deploy context is in the site's trigger contexts. Netlify's deploy-preview maps to preview, production maps to production, and branch-deploy maps to branch. A mismatch skips with context <x> not enabled.
  • The branch matches the site's branch filters. A mismatch skips with branch <x> filtered.
  • A default test case or test plan is selected. Without one the event is skipped with no default test target.
  • The deploy reached the ready state. A deploy in building or error state does not trigger a run (deployment not ready).
  • The deploy preview URL is present. If the webhook included no preview URL, the event is skipped with no preview URL.

A deploy is stuck as "pending"

If a deploy appears as "pending" in Autonomy and never triggers a run, it is likely a forked-PR deploy held by Netlify for team-owner approval (deploy_request_pending). Approve the deploy in Netlify — a deploy_succeeded webhook will follow and trigger the run.

The run cannot reach the deploy preview

The deploy preview URL may be blocked by site-level password protection. If the run fails on a password page, add the site password as the Preview access secret.

No check appears on the pull request

Netlify has no native check surface. Status reporting requires:

  1. Report checks enabled on the linked site.
  2. A Linked repository selected.
  3. The Autonomy GitHub App installed on that repository.

If any of these are missing, the event's report status is unsupported and no check is posted. If checks are enabled but no repository is linked, results appear only in the Autonomy dashboard.

Duplicate runs for the same deploy

Autonomy deduplicates by a key that includes the Netlify deploy ID and state. A redelivered webhook produces no second run.

Vercel

Automatically run QA against every Vercel preview deployment and report the result as a deployment check.

Expo EAS

Automatically run mobile QA against every EAS build and report the result via your linked GitHub repository.

On this page

What it doesWhy the permalinkSetupHow a run is triggeredForked-PR deploysPassword-protected sitesStatus reportingMonorepo and multiple sitesEnvironment variables (self-hosted)TroubleshootingNo run starts when a deploy preview is readyA deploy is stuck as "pending"The run cannot reach the deploy previewNo check appears on the pull requestDuplicate runs for the same deploy