Autonomy Docs
Authoring

Overview

Write a test case as outcome steps, describe what to test, or record it once.

A test case is one journey written as a short list of steps. Each step is an outcome: something a person could check when the step is done. The agent works out how to reach each outcome on the live product and shows every action it took.

Start one from your project's Test cases page with New test case. The wizard asks for the details, then how you want to create the steps, then lets you review them.

Details

  • Name: up to 100 characters, such as "Sign up with email".
  • Description: optional. It's shown on the run report and in Slack alerts.
  • Platforms: Web (Chrome), API (HTTP requests, no browser or device) and Android (emulator). iOS testing is coming soon. A platform is available when the project has an environment with a target for it.
  • Environment: where the steps are drafted and where runs go by default. A run can pick another environment; the test case doesn't change.

Choose a method

MethodTagBest for
Write the stepsPreciseYou already know the outcomes. You write each one yourself.
Describe what to testRecommendedMost journeys. You describe it in plain words and Autonomy drafts the outcome steps.
Record it onceWeb and mobileLong journeys. You do it once in a browser or on an Android emulator and Autonomy drafts the steps. See Record a journey.

Every method ends in the same steps editor, and you can edit the steps later whichever one you picked.

Write steps as outcomes

Write what should be true, not where to click:

Instead ofWrite
Click "Sign up", type an email, click "Continue"A new grower account exists for a fresh email.
Open the inbox and click the linkThe verification email arrives and its link verifies the account.
Check that the text "Maize" is visible on /profileThe profile shows Maize as the farming preference.

The agent handles the clicks, dialogs, retries and waits on its own. It won't add steps or go beyond the test case, and every action it takes is shown under the step in the run.

If the way matters, say it in the outcome: "Onboarding is completed as an experienced grower" is judged on the onboarding itself, so reaching the same end state another way, such as skipping onboarding and setting it later in settings, doesn't pass the step. If the way doesn't matter, leave it out and the agent picks one.

A test case has up to 12 steps in the dashboard, and each step up to 2,000 characters. Signing in is a step like any other: write "The grower is signed in" and attach the account.

Attachments

Click + Attach on a step to give the agent what it needs:

MenuAttachmentWhat it does
Sign in asAn accountA saved sign-in of the project.
InboxFresh inbox per runAn inbox the run creates, for codes, links and receipts. See Inboxes and phones.
PhoneTest phone numberA number that can receive SMS codes.
FileA project fileA file to upload, from the project's Files.
EvidenceKeep a screenshotKeeps a screenshot of the outcome as evidence.
Generated dataGenerated password, name and address, or A value to use…A value the agent generates each run, or one you give it.

A value to use… takes a name and a value, such as "Farm name: Green Acres". Tick Hide the value for anything secret: it's filled by reference and never shown again, not even to you. A value can also be an environment value, such as {{env.COUPON_CODE}}. See Environments.

Don't put real passwords in steps. Attach a saved sign-in instead, or let the agent generate a strong password.

Exact or test data

Attached values are test data by default. If the product refuses one only because it's already taken, such as an email that's already registered, the agent uses a fresh value of the same kind and says so in the step.

Click Exact on a value to keep it as written. The agent then uses exactly that value, and if the product refuses it, the step is judged on that refusal. Use Exact for a value the outcome is about: a coupon code that must apply, or a phone number that must be rejected. Accounts, values with a value, and fixed inboxes and phone numbers can be Exact.

Describe what to test

Describe the journey in plain words, then click Generate draft. You can attach a text file, such as notes, a .feature file or an OpenAPI .yaml or .json spec, and its text is added to the description.

  • Autonomy asks at most one question, and only when the answer changes the steps, such as "Which sign-up method should the test use?". Tick Skip questions to let it pick the usual way.
  • It knows the project's description, its product knowledge, the environment's URLs and value names, and your saved sign-ins. It never sees values.
  • Values you write in the description, such as a coupon code or "sign in as the store owner", become attachments with exactly what you wrote. Autonomy never invents a value.
  • Don't paste passwords. If the description holds one, Autonomy refuses it and asks you to pick a saved sign-in instead.

Review before saving

Before saving, Autonomy reads the steps in order, the way the agent will in a run, and lists anything it would have to guess:

  • Needs you: for example, a step signs in but no account is attached ("Which account should sign in here?"), or a step uploads a file but none is attached.
  • Suggestions: for example, a code arrives at the inbox from an earlier step, or a step reads like a click instead of an outcome.

Resolve what needs you, then click Create test case. Nothing runs until you start it, and every run starts signed out with a fresh browser or app session.

Test cases from before v0.5

Test cases written before v0.5 used one action per step. Most were rewritten as outcome steps when your organization moved to v0.5. One that still has its old steps shows a warning, and its runs end as Couldn't start until you write its outcomes. The old steps stay at the bottom of the editor, read-only.

Where to go next

Read Record a journey for recording, Visual checks for checking how a screen looks, and Test plans and schedules to run test cases together.

Didn’t find it?

On this page