Tests

Every test in OttoTester has two sides:

The test case is the spec — what the test should do, in plain English. A name, a description, an intent, a priority, and a list of steps. It’s readable; it isn’t code.
The test script is the executable — the Playwright code that actually runs.

The planner writes the test case. The generator writes the test script from it. The executor runs the script. The two are linked but kept as separate records, so you can read and reason about what a test does without opening Playwright code.

Anatomy of a test case

The test-case side carries:

Name and description — what the scenario is.
Intent — the kind of test: happy-path, negative, or edge-case.
Priority — P0 through P3.
Steps — the ordered actions and checks.
Status — planned (case exists, no script yet), scripted (script written, ready to run), or blocked (can’t proceed — usually a real app issue the healer couldn’t get past).

Pass/fail isn’t part of the test case. That’s a result, recorded on the script side after a run.

Changing a test

You can change a test in four ways:

Edit the test’s steps. Change what the test should do, in plain words, on the Test Case view. When you save changed steps, the old script no longer matches them, so the next run that writes tests writes a new one from your steps. The test shows Write this Test until then, so you can do it straight away. If you edit the script in the same save, your script edit is kept instead.
Edit the test script directly. Your edit is saved as the test’s current version — the next run uses it, and if the test later fails, the healer fixes your version rather than throwing your edit away. The fastest way to fix one test.
Regenerate the test. Re-run planning and generation to rebuild the test from scratch — useful when the app changed enough that a patch won’t do.
Add a hint. A hint doesn’t change this script directly; it teaches the generator, so future generations — of this test and others — handle the page correctly.

Edit the script to fix this test now. Add a hint to stop the same problem coming back across regenerations and other tests.

Edit and pin

To change a test for the long haul, open it from the Tests page and edit the script directly. Your edit becomes the test’s current version — the next run uses it, and if the test later fails the healer repairs your version rather than throwing your edit away.

That’s what “pin” means in practice: there’s no separate pin toggle, because every saved edit is treated as the authoritative version until the next edit or a rollback. To restore an earlier version, see version history below.

Version history

OttoTester keeps a history of each test. When a test changes — an agent updating it, or you editing it by hand — the new version is saved alongside the old ones.

For any test you can:

See its history — the changes to the test, newest first, with who made each.
Look at an earlier version — see what the test’s spec and script were at an earlier point.
Roll back — restore a test’s script to an earlier version. The rollback is saved as a new step, so nothing in the history is lost.

A test’s history starts when the test was made. If you delete a test and later make a new one with the same name, the new test’s history shows only its own changes, not the deleted test’s.

Nothing you or an agent does to a test is gone for good — there’s always a way back.

Fixtures are a kind of test

Most tests are ordinary tests. Some are fixtures — a test whose job is to sign in, so the tests after it start already authenticated. A fixture has the same two facets (a case and a script), but is scoped to a role. See Fixtures and test data.

Tests that need another test first

Some tests need data that another test makes. “Terminate an employee” needs an active employee, so “Make the employee active” runs first, and “Create an employee” runs before that. OttoTester plans these links and runs the tests in that order. The Tests page shows them:

Needs is the test that has to run first. In the list, a pill counts them, such as Needs 1. Under the test’s name, a chip names the test. Click it to open that test.
Needed by is the tests that wait for this one. The list counts them the same way, and the chip names the first. With more than one, the chip opens Details, which lists them all.
Sets up, in Details, says what the test leaves in the app for the tests that need it, such as “an active employee”.

Only tests that set up data count. Every signed-in test also needs its sign-in step, so that link isn’t shown here; the Auth line in Details names the login. If a prerequisite test fails, the tests that need it are skipped, and the run says which test they were waiting for. Once the tests that need it have run, OttoTester removes what the prerequisite test made, when the test has a cleanup step.

Most tests that need the same thing share one copy of it. A test that uses its data up, such as “Terminate an employee”, gets its own copy of the tests it needs, so it gets its own employee and the others keep theirs. In a run, the test they need runs once for each copy, and its row shows each copy’s result, such as Shared: passed and For “Terminate an employee”: passed. It still counts as one test.

Deleting a test that other tests need asks first: the Delete test? dialog names the tests that need it, and its button reads Delete anyway. Once it’s deleted, those tests don’t run until another test sets up the same thing, and their Needs line shows a deleted test. Create Tests can plan a new one.

Tests and suites

Tests are grouped into test suites — named sets you run together. A test can belong to more than one suite.

Concept → Test Authoring Templates, Test suites, Fixtures and test data, Healing and hints
Workflow → Fix a failing test