Read a run report

The run detail

Open a run from the run list to see its report. At the top: the run’s overall status, how long it took, and a summary count — passed, failed, skipped. Below that, a row per test.

Each test row shows its status:

• passed — the test ran and every check held.
• failed — the test ran and a check failed.
• skipped — the test didn’t run, often because something it depended on failed first.
• running / pending — still going, or not started yet.

Healing isn’t a per-test status. It’s a phase of the run as a whole — if the run went through a healing phase, the healer took a pass at the failures before the report was finalized.

When a test may have left data

When OttoTester writes a test that creates something, such as a new employee, it can add a cleanup step that removes it again after the test runs, whether the test passed or failed. A test that other tests rely on, such as one that makes an employee for them to use, keeps what it made until those tests have all run, and then OttoTester removes it once. A test without a cleanup step leaves what it made, and so does a test written before cleanup steps existed. When OttoTester can’t confirm a removal worked, for example because it couldn’t find the record to remove, a line in amber under the counts reads 1 test may have left data, and that test’s row says Check that Casey-1b534l was removed, naming each record it typed. A test whose cleanup didn’t finish in more than one pass, such as a heal check, names every record. A test that stopped before it reported has no row, so hold the pointer over the amber line to see its name and the records to check.

The test keeps its own result: a passed test is still passed. If the record is still in your app, delete it by hand. A test that created nothing, or whose cleanup worked, shows nothing extra.

When a run has healing turned on, OttoTester also tries to fix a cleanup that couldn’t confirm, changing only the steps that find and remove the record. It keeps the fix only once it has seen the cleanup work. It checks the fix by running only that test and the tests it needs first, and the check never changes whether the test passed in that run. If the warning keeps coming back, open the test and choose Run and Fix, which is offered for a test whose cleanup didn’t confirm even though the test passed.

Drill into a failure

To find out why a test failed:

1. Open the failed test from the report.
2. Walk its steps — each action and check, in order — down to the one that broke.
3. Open that test’s Playwright HTML report.

Read the Playwright trace

The Playwright report is the detailed record: every step, a screenshot of the page at each action, the network activity, and the console. For a failed test it includes a trace you can step through — scrub back and forth and see exactly what the page looked like when the script expected one thing and found another.

That’s almost always where the answer is: the script thought it was clicking X; the trace shows what was actually on screen.

Fix it from here

Once you know what broke, head to Fix a failing test — it covers the healer’s attempt log and the three ways to get back to green.

Related

• Concept → Test runs and reports, Healing and hints
• Workflow → Fix a failing test