Skip to main content

Autonomous Building and Testing with Browsr in the Loop

· 5 min read
Vivek

The most powerful AI development setups I've seen share a common trait: the agent has both code access and browser access at the same time. Not sequentially — simultaneously. It can edit a file, save it, reload the browser, take a screenshot, catch an error, fix the code, and reload again — all without stopping to ask what you see on screen. Here's what that loop looks like in practice.

The Loop​

The build→test→fix cycle is nothing new. What's new is that an AI agent can drive the entire thing. The agent:

  1. Writes or modifies code
  2. Triggers a rebuild (or relies on hot reload)
  3. Opens the relevant page via Browsr Relay
  4. Takes a screenshot and analyzes the visual result
  5. Checks for JavaScript errors and network failures
  6. If something's wrong, diagnoses and fixes it
  7. Retests

This loop runs fast. Each iteration takes seconds, not minutes. And because the agent is operating in your actual browser — not a headless clone — it catches the bugs that only appear in real environments.

A Concrete Example: Building a Dashboard​

Let's walk through what this looks like when an agent is building an analytics dashboard.

The agent scaffolds the component, starts the dev server in a shell session, and opens the page via Relay. The first screenshot comes back looking roughly right — the layout is there, the charts are rendering. But in the top-right corner, the "Last updated" timestamp is showing as Invalid Date.

Without Relay, the agent would have had no way to know this. The unit tests pass. The component renders without throwing. The bug only shows up visually, in the browser, with real data.

With Relay, the agent catches it immediately. It evaluates the expression that populates the timestamp, finds that the API is returning a Unix timestamp in seconds rather than milliseconds, and fixes the date parsing. One more screenshot confirms it. The loop continues.

How Console Errors Get Caught Automatically​

The screenshot tells you what the page looks like. The events stream tells you what's happening underneath.

When you connect an agent to a Relay session and point it at the events endpoint, it receives a live stream of CDP events: console messages, network requests, navigation lifecycle events. Console errors appear as they're thrown. Failed API calls surface as network events with non-2xx status codes.

This means the agent doesn't have to explicitly check for errors — they come to it. A console.error thrown deep in a third-party component shows up in the events stream immediately. A 401 on a data-fetching request appears as a network event before the UI even has a chance to show a loading state.

The agent's feedback loop is not just visual. It's the full picture of what's happening in the browser.

Code Access and Browser Access, Simultaneously​

The combination that makes this work is that the agent has both sides of the loop at once.

With a shell session, the agent can edit files, run builds, install packages, and restart servers. With Browsr Relay, the agent can see the result of those actions in a real browser. These aren't separate tools that the agent has to switch between — they're both live, both accessible, both responding in real time.

The shell session might be running npm run dev in watch mode. The Relay session is connected to the browser tab that's pointing at localhost:3000. When the agent edits a component and saves it, the dev server rebuilds, the browser hot-reloads, and the agent can screenshot the result within a few seconds.

This is meaningfully different from a CI pipeline that runs headless tests after a push. The feedback is immediate, the environment is real, and the agent is driving both sides.

Beyond Screenshots: Testing Real User Flows​

Screenshots are the most obvious use of Relay in a build loop, but they're not the only one. The agent can also:

  • Click through user flows using Input.dispatchMouseEvent and Input.dispatchKeyEvent CDP commands, testing that buttons work, forms submit, and navigation behaves correctly
  • Inspect the DOM using DOM.getDocument and DOM.querySelector to verify that elements are present, have the right attributes, and contain the expected text
  • Intercept network requests to verify that the right API calls are being made with the right payloads
  • Emulate viewport sizes to catch responsive layout bugs before they reach production

The same Relay URL that gives the agent a screenshot also gives it the full Chrome DevTools Protocol surface. The screenshot is just the most immediately useful starting point.

Getting Started​

If you want to set up this loop for your own project:

  1. Install the Browsr Relay Chrome extension
  2. Open your app in Chrome and start your dev server
  3. Click the Relay extension icon and share the tab — you'll get a URL
  4. Give that URL to your agent alongside your codebase access
  5. Tell the agent what you want to build, and let it run

For shell access alongside the browser session, Browsr's cloud shell sessions run in the same environment and can be connected to the same Relay URL — the agent gets both in one place.

The loop closes. The agent builds, sees, fixes. You review the result.