Browser automation tools have converged on two approaches: high-level frameworks like Playwright and Cypress that abstract away the browser, and MCP-based tools like chrome-devtools MCP that expose browser control to AI agents. Both approaches have a fundamental limitation — they're tightly coupled to specific runtimes. Playwright tests run in Node.js. MCP tools only work with MCP-aware AI clients.

ToolPiper takes a third approach. It holds a persistent CDP (Chrome DevTools Protocol) connection and exposes the browser's accessibility tree as structured context that any AI model can consume — whether it's a local llama.cpp model, an OpenAI API call, an Anthropic model, or anything behind an OpenAI-compatible endpoint. The AI doesn't need MCP support. It doesn't need tool-calling capability. It just needs to read text.

The Accessibility Tree as Primary Representation

Most browser automation tools operate on the DOM. Selectors target elements by CSS class, ID, XPath, or data attributes. This works for hand-written tests but fails for AI-generated tests because DOM structure is noisy, unstable, and semantically opaque.

The accessibility tree (AX tree) is Chrome's semantic representation of the page. It describes what users see and interact with: buttons, links, text fields, headings, landmarks. It strips away presentational markup, CSS-only elements, and framework-specific wrapper divs. A React app with 2,000 DOM nodes might have 200 AX tree nodes — the ones that actually matter for interaction.

ToolPiper's browser_snapshot tool returns a pruned, formatted accessibility tree. When an AI model receives this snapshot, it can reason about the page the way a human would: "I see a login form with an email field, a password field, and a Sign In button." It doesn't need to know that the button is div.MuiButton-root.MuiButton-contained > span.MuiButton-label.

AX Selectors

Instead of CSS selectors, ToolPiper uses accessibility-role-based selectors throughout:

role:button:Sign In
label:Email
text:Welcome to the app
testid:submit-btn
role:form:Login > role:button:Submit

These selectors target the accessibility tree directly. role:button:Sign In matches a node with role "button" and accessible name "Sign In". The hierarchical syntax (>) enables scoping — role:form:Login > role:button:Submit finds the Submit button specifically within the Login form, even if other Submit buttons exist on the page.

AX selectors are inherently more stable than DOM selectors. A CSS refactor that changes class names breaks CSS selectors. A component library migration that replaces <button> with <Button> breaks tag-based selectors. But the accessibility tree rarely changes unless the actual user-visible behavior changes — which is exactly when tests should break.

Why ToolPiper Owns the CDP Connection

MCP-based browser tools (chrome-devtools MCP, Playwright MCP) are tightly coupled to MCP-aware AI clients. A local llama model running through llama.cpp has no way to call MCP tools. An OpenRouter-routed model doesn't have MCP access. Any model accessed through a standard chat completion API is excluded.

ToolPiper bridges this gap with a clean separation of concerns:

  • ToolPiper holds the CDP WebSocket connection and owns all browser interaction — snapshots, actions, assertions, recording
  • ModelPiper (the web app) injects browser context into the AI conversation as plain text or images
  • Any AI provider can generate tests because browser context is just text in the prompt, not a tool call

The AI sees an accessibility tree snapshot as part of its conversation context. It reasons about what to click, what to type, what to assert. Its response is parsed into structured test steps. No MCP required. No tool-calling required. The model just needs to understand text.

This architecture means that switching from Claude to GPT-4 to a local Qwen model doesn't require any changes to the browser automation layer. The CDP connection, AX tree formatting, action execution, and assertion checking all remain the same.

Chrome 148+ Compatibility: A Real-World CDP Story

Building on raw CDP instead of a framework like Playwright means dealing with Chrome's evolving protocol directly. Chrome 148 introduced several breaking changes that illuminate why most tools use abstraction layers — and why we chose not to.

The Debug Profile Requirement

Chrome 148 requires --user-data-dir pointing to a non-default directory when using --remote-debugging-port. Without it, Chrome silently ignores the debugging flag — no error, no port file, no CDP access. ToolPiper creates a dedicated debug profile per Chrome channel at ~/Library/Application Support/ToolPiper/chrome-debug/{channel}/.

This is actually advantageous for test automation. The debug Chrome instance is isolated from the user's normal browser — no extensions, no cached login sessions, no bookmarks interfering with test state. Clean and reproducible by default.

AX Node ID Type Coercion

The CDP specification defines AXNodeId as a string type. Chrome 148 returns node IDs and parent IDs as integers in getFullAXTree responses. Our axNodeId() helper handles both via String/Int/NSNumber coercion. This affects every part of the AX pipeline — tree rendering, diffing, path extraction, and ancestor discovery.

Frame-Scoped AX Queries

Chrome 148 requires an explicit frameId parameter on Accessibility.getFullAXTree. Without it, only the root RootWebArea node is returned. And queryAXTree requires explicit backendNodeId scoping to the document root — without it, all queries return empty results. We also discovered that Chrome builds the accessibility tree lazily, so the first AX selector resolution on a new page must prime the tree with a getFullAXTree call.

These aren't bugs. They're security and performance improvements in Chrome's implementation. But they broke every CDP-based AX tool that hadn't been updated — because most CDP tools don't use the Accessibility domain at all. They use the DOM domain with CSS selectors. Our AX-first approach meant we hit these issues early and fixed them proactively.

Self-Healing Selectors

When a test step's selector no longer matches (the button was renamed, the form was restructured), ToolPiper doesn't fail immediately. The test runner enters a healing loop:

  1. Take a fresh AX tree snapshot
  2. Search for nodes that match the original selector's role and approximate name
  3. Score candidates by similarity to the original target (role match, name edit distance, tree position)
  4. If a high-confidence match is found, execute the action on the healed selector and record the mapping

Heal history is persisted with the test session, so subsequent runs use the healed selector directly. This makes tests resilient to minor UI changes — a button renamed from "Submit" to "Save" is healed automatically, while a fundamentally different page structure correctly fails the test.

The PiperTest Format

ToolPiper stores tests in its own format — PiperTest — rather than generating Playwright or Cypress code directly. Each test is a sequence of typed steps:

[
  { "action": "navigate", "url": "https://example.com/login" },
  { "action": "fill", "selector": "label:Email", "value": "[email protected]" },
  { "action": "fill", "selector": "label:Password", "value": "secret" },
  { "action": "click", "selector": "role:button:Sign In" },
  { "action": "assert", "type": "text", "selector": "role:heading", "expected": "Dashboard" }
]

This format is framework-agnostic. The same test can be exported as Playwright code, Cypress code, or executed directly by ToolPiper's built-in test runner. The export renderer maps AX selectors to each framework's selector format — getByRole('button', { name: 'Sign In' }) for Playwright, cy.contains('button', 'Sign In') for Cypress.

Seven assertion types are supported: visible, hidden, text content, URL match, element count, attribute value, and console message. Assertions use polling with configurable timeouts — they retry until the condition is met or the timeout expires, then capture an AX snapshot on failure for debugging.

Smart Fill: Beyond Simple Text Input

The fill action isn't just input.value = text. ToolPiper auto-detects the input type and uses the appropriate strategy:

  • <select> elements — programmatically selects the matching option by value or visible text
  • Date/time inputs — uses the native value setter to bypass the browser's date picker UI
  • Range/slider inputs — sets the value and dispatches input+change events
  • Color inputs — validates hex format and sets via the native value setter
  • Standard text inputs — dispatches keyboard events for realistic interaction

Detection happens via CDP's DOM.getDocument tree walk, not the AX tree — because the AX tree doesn't expose input subtypes. The AX tree tells us "this is a textbox"; the DOM tells us it's <input type="date">.

AX Path Enrichment

Every browser action returns an axPath — the accessibility-tree path from the document root to the target element. For example:

RootWebArea > navigation:Main > list > listitem > link:Dashboard

This serves two purposes: debugging (you can see exactly where in the AX tree the action occurred) and test stability (the path provides structural context for self-healing). The path is computed from raw AX nodes in a single CDP call using the extractAXPath function — no extra round trips.

Actions also return an elementMeta object with the element's tag name, type, bounding box, and other DOM-level metadata. This enrichment happens in a single pass with the AX path extraction via the enrichForSelector method — one CDP call for both pieces of information.

14 Browser MCP Tools

For AI clients that do support MCP, ToolPiper exposes 14 browser-specific tools. These cover the full spectrum of browser automation:

  • Observation: browser_snapshot (AX tree), browser_console (logs + network errors), browser_network (request/response capture), browser_performance (Web Vitals + runtime metrics)
  • Interaction: browser_action (click, fill, select, hover, scroll, keyboard), browser_autofill (credit card + address forms), browser_eval (execute JavaScript)
  • Testing: browser_assert (7 assertion types with polling), browser_record (capture user interactions), browser_coverage (JS + CSS code coverage)
  • Infrastructure: browser_manage (connect/disconnect/page switching), browser_storage (cookies + localStorage + sessionStorage CRUD), browser_intercept (mock network responses), browser_webauthn (virtual authenticator for passkey testing)

Each tool returns semantic plain text. browser_snapshot returns a formatted AX tree with indentation and role labels. browser_action returns a structured AX diff showing what changed after the action — added nodes with +, removed with -, modified with ~.

Connection Stability

A persistent CDP WebSocket connection needs robust error handling. Chrome can crash, the user can close the debug window, DevTools can steal the session. ToolPiper's CDPClient actor implements:

  • Adaptive heartbeat — Target.getTargets at 5s intervals during recording (eager mode), 15s during idle. Detects both dead sockets and unresponsive Chrome.
  • Two-phase reconnection — (1) Rapid: 5 attempts with exponential backoff from 500ms to 8s. (2) Background: indefinite retries at 5s (eager) or 30s (idle) intervals.
  • Handshake verification — raw CDP message sent during connect() to verify Chrome responds before declaring the connection established.
  • Inspector.detached handling — when Chrome DevTools opens and steals the session, ToolPiper stops recording, clears page state, and enters reconnection.

What This Enables

The combination of AX-tree-first representation, provider-agnostic architecture, and persistent CDP ownership enables a testing workflow that no existing tool provides:

  1. Open any web app in Chrome
  2. Ask any AI model (local or cloud, any provider) to write tests for what you see
  3. The AI receives the AX tree as context and generates PiperTest steps
  4. ToolPiper executes the steps with self-healing selectors
  5. Export to Playwright or Cypress when ready for CI

No framework lock-in. No specific AI provider required. No MCP dependency. Just a browser, an accessibility tree, and an AI model that can read text.

ToolPiper's browser automation is available at modelpiper.com. The 14 browser MCP tools work with any MCP-aware client. The provider-agnostic testing workflow works with everything else.