Skip to content
OttoTester docs

Use OttoTester from a coding agent

A coding agent is an AI assistant that works in your editor or terminal, such as Claude Code or Cursor. OttoTester gives these agents a set of tools over MCP, the standard way a coding agent calls outside tools. With them, the agent can find your apps, start a run, wait for it, and explain the failures, without you opening OttoTester.

What you need

Section titled “What you need”
  • The address. Your OttoTester address followed by /mcp. On a self-hosted install that is usually http://localhost:8090/mcp.

That’s all. The first time the agent connects, it opens your browser at OttoTester. Sign in as usual, then approve the agent on the screen that follows. You pick which workspaces it may use: all of yours, or only some.

The agent can do exactly what you can, and nothing more, in the workspaces you picked. It can never open Global Admin, even if you are a Global Admin.

Add OttoTester to Claude Code

Section titled “Add OttoTester to Claude Code”

Run this once, with your address:

Terminal window
claude mcp add --transport http ottotester http://localhost:8090/mcp

Then start Claude Code and type /mcp. Your browser opens OttoTester to sign in. After you allow it, Claude Code shows OttoTester as connected.

Add OttoTester to Cursor

Section titled “Add OttoTester to Cursor”

Add this to ~/.cursor/mcp.json (create the file if it isn’t there), with your address:

{
  "mcpServers": {
    "ottotester": {
      "url": "http://localhost:8090/mcp"
    }
  }
}

Cursor opens your browser to sign in the first time it uses OttoTester.

Agents that can’t open a browser

Section titled “Agents that can’t open a browser”

Some agents, and anything running unattended, can’t sign in through a browser. Give them a personal access token instead. Make one in Settings → Account → Personal access tokens (see API tokens), with its own name, such as “Claude Code on the build server”, so you can revoke just that one later. Then add it as a header:

Terminal window
claude mcp add --transport http ottotester http://localhost:8090/mcp --header "Authorization: Bearer <your token>"

In Cursor, add "headers": { "Authorization": "Bearer <your token>" } next to "url". Other coding agents that support MCP over HTTP take the same two things: the address, and an Authorization: Bearer <token> header.

Ask for something

Section titled “Ask for something”

Once it is connected, ask the agent in plain words. For example:

Find the Shop app’s staging variant in OttoTester, run its saved tests, and when the run is done, tell me which tests failed and why.

The agent lists your workspaces, apps and variants to find staging, picks a run setup, and starts the run. A run carries on in the background, so the agent checks back until it has ended, then reads each test’s result and failure reason.

What the agent can do

Section titled “What the agent can do”
  • Find workspaces, apps and variants, add an app or a variant, and rescan a variant’s site. A rescan carries on in the background, and the agent checks back to see what it found.
  • List run setups, start a run, and test another address for one run only, such as a preview build.
  • Check a run, read its log, and read its results: each test’s result, failures grouped by kind, what changed since the last run, and the tests’ quality scores and cost.
  • Add references (pasted text, or a link to a web page), and see how far OttoTester has got reading them.
  • Read the app’s issue tracker settings and its issues, find out why a list of issues comes back empty, check and count a changed filter, save it, and import issues.
  • List, add and edit proposed tests, and have OttoTester propose tests again.
  • Create tests from proposed tests, or from a plain description such as “a new user can reset their password”.
  • List a variant’s tests, see a test’s recent results, and read its script.
  • Group tests into suites, and run a suite at set times: schedule it, change the schedule, pause, resume or delete it.
  • Add a run setup for CI, and set which GitHub deployments start a run (workspace admins only).
  • See which Slack channels are connected, and choose which channel hears about which variant’s runs (changing that is for workspace admins only).
  • Read what Reports shows: pass rates over time, new failures, ongoing failures, flaky and fixed tests, how two releases or two periods compare, and how much of the app’s requirements and defects is covered.
  • List the files a run left behind, and look at a failed test’s screenshot.

Connecting Jira, GitHub or Slack needs you to sign in at that service, so the agent can’t do it. It tells you which page to open instead.

For example:

Our Jira import for the Shop app finds nothing. Work out why, and suggest a filter that brings in this sprint’s stories.

AI credits. Starting a run may spend your organization’s AI credits: writing new tests always does, and running saved tests does when a failing test is repaired. So do creating tests, adding a reference, importing issues and proposing tests again, because OttoTester reads what was added and proposes tests from it, and so does rescanning a variant. Adding a variant does not scan the site, and adding a proposed test by hand reads nothing, so those spend none.

What the agent asks you first. Saving an issue tracker filter changes what the whole app imports, the automatic sync included, and proposing tests again removes the proposed tests nobody edited or used. The agent is told to check with you before doing either, and before it deletes a schedule, a CI trigger rule or a Slack routing rule.

What the agent never sees. Replies leave out saved passwords, keys and tokens. Log lines and error messages are cleaned before they reach the agent, so anything that looks like a password or a token is masked.

Screenshots, traces and videos. The agent can look at a screenshot under 1 MB. It is the app’s screen as the run saw it: password fields show dots, but anything the page shows in plain text is in the picture, just as in the report. Traces and videos can hold what was typed into the app, so the agent never gets them. It gets a link that opens each one in your browser, where you are signed in.

Stop an agent

Section titled “Stop an agent”

Open Settings → Account → Personal access tokens. An agent you connected through the browser is listed under Connected coding agents, with when it was last used: select Disconnect. An agent using a token you pasted is listed under API tokens: select Revoke. Either way, the agent’s next request is refused.

An agent you connected through the browser is also disconnected on its own after 90 days without use.

Section titled “Related”