Vercel

The Vercel integration watches your Vercel deployments. When a deployment is ready, Autonomy runs a project's test case or test plan against its URL and reports the result as a check on the deployment. If the repository is also linked, the same result appears as a GitHub check on the commit.

What it does

• Starts runs when a Vercel deployment is ready, filtered by deployment type (preview, production, branch) and branch.
• Tests the deployment's own URL, in an environment of the project you chose.
• Posts a Vercel deployment check named Autonomy QA, and can hold promotion to production until it concludes.
• Starts a new run when someone clicks Re-run on the check in Vercel.

Set up

1. Open Organization → Integrations → Vercel and click Connect. Vercel asks you to choose a team or account and approve the requested permissions, including deployment checks.
2. Click Link project and pick the Vercel project to test. Its check definition is created for you.
3. Under Project, choose the Autonomy project whose tests these deployments run. Changing it later clears the defaults below.
4. Under Default test case or plan, choose what runs on each deployment. Only the project's test cases and test plans are listed.
5. Under Default environment, choose the project's environment. Runs use its values and sign-ins, with the deployment URL as the web target. Without an environment, deployments are skipped.
6. Under Trigger on, choose the deployment types that start a run, usually Preview. Add Branch filters such as main or release/* to limit it to some branches.
7. Turn on Post a Vercel deployment check. Turn on Block deployment promotion until check completes if a failing check should hold promotion.
8. Push a commit and check that a run starts when the preview is ready.

How a run starts

When a deployment is ready, Autonomy:

1. Finds the linked Vercel project and its Autonomy project. Nothing runs if that project is archived.
2. Checks the deployment type against Trigger on and the branch against Branch filters.
3. Starts the default test case or test plan in the default environment, against the deployment URL.

Each deployment runs once. A webhook Vercel sends twice doesn't start a second run.

The check

The check is named Autonomy QA and shows a link to each run. It concludes as:

A failed run fails the deployment check, whether its outcome wasn't reached or the agent couldn't show that it was. The summary gives the reason.

Fail the check on issues

Issues never change a run's result, so a passing run passes the check even if it reported an issue. To make bugs fail the deployment check too, turn on Fail the check on issues in the project's Settings → Integrations → Pull requests. It's off by default. When it's on, an open Bug that one of the check's runs found or reopened turns the check failed: "Failed: a bug was found". Risks, and bugs you resolved or ignored, don't count. See Pull request testing.

Blocking promotion

With Block deployment promotion until check completes on, Vercel won't promote the deployment to production until the check concludes. The check times out after 15 minutes. A run may still finish in Autonomy after that, but the Vercel check doesn't change.

Re-run

Clicking Re-run on the check in Vercel starts new runs against the same deployment, with the link's current settings.

Protected previews

If Vercel Deployment Protection is on, the runner can't open the preview without a bypass:

1. In Vercel, open the project's Settings → Deployment Protection and copy the Protection Bypass for Automation secret.
2. In Autonomy, open the linked Vercel project and paste it into Preview access secret.

Autonomy sends it with every request to that project's previews. It is stored as a secret and never shown again.

Several Vercel projects

Link each Vercel project on its own. Each link has its own Autonomy project, test plan, environment, trigger types and branch filters, so a monorepo with a web app, docs and an admin panel can run a different plan for each.

Troubleshooting

No run starts when a preview deploys

Open the linked Vercel project in Autonomy and look at the recent deployments. A skipped deployment says why:

• project disabled: the link is turned off.
• context <x> not enabled: the deployment type isn't in Trigger on.
• branch <x> filtered: the branch doesn't match Branch filters.
• no default test target: choose a Default test case or plan.
• "Choose an environment for this deploy link in Autonomy.": choose a Default environment.
• project archived, or a default that is archived or in another project: restore the project or choose new defaults.
• no preview URL: Vercel sent no URL for the deployment.

Also check that Vercel shows as Active in Autonomy.

The run can't reach the preview

Deployment Protection is blocking it. Add or refresh the Preview access secret.

The check failed but every run passed

Fail the check on issues is on, and a run found a bug. Open the run's Issues found.

The check shows neutral

A run couldn't start, for example because the preview didn't open or blocked the test browser as a bot. Open the run to see why, fix it, and click Re-run in Vercel.