AI Browser MCP server

What it is. An Electron app that exposes its tabs over two equivalent interfaces — an MCP server (17 browse_* tools) and a raw WebSocket JSON-RPC API (ws://localhost:9223, methods ui.*). Your agent navigates, reads a structured semantic tree (role / label / value / states / actions), and clicks or types by data-ai-id. Reading never goes through a screenshot and there is no OCR — that's the point of the tree. browse_screenshot exists, but as a diagnostic for the cases the tree can't see (overlays, canvas, mis-rendered layout), not as the read path.

What it is not. It is not headless. The window really opens, really renders, and stays interactive — see Letting a human step in. Nor does it "co-drive" with you: the hand-off is a conversation between the agent and the user, not a browser API.

Security note. The WebSocket server binds loopback only and refuses any handshake that carries an Origin header — that's what a browser always sends, so a page running inside AI Browser (or any other browser) cannot drive the control plane, while Node/CLI/MCP clients (which send no Origin) work normally. There is still no token: a local process running as you can connect. Do not expose port 9223 beyond loopback. (AI_BROWSER_WS_ALLOW_ORIGIN=1 disables the check — debug only, it re-opens the page → browser path.)

Sandbox is off by default, deliberately. Chromium's OS sandbox cannot initialise in restricted shells (IDE terminals, some CI), where it fails with sandbox initialization failed: Operation not permitted and the app exits before any of our code runs — so there is no error to report, just a silent quit. AI_BROWSER_SANDBOX=1 turns it on; if the app then quits instantly with no output at all, that's this — unset the variable. Renderers are still isolated from the page either way (contextIsolation:true, nodeIntegration:false); this switch is about the OS layer under them.

---

Install

Install the prebuilt package unless you have a reason not to. Option A is one file you double-click: no Node.js, no dependency tree, no build step. Source install (Option C) pulls a ~276 MB node_modules plus a ~100 MB Electron engine download on top — only use it when no package fits your platform, the package misbehaves, or you specifically need to run from source.

Option A — macOS install package (recommended)

Grab AI-Browser-<version>-arm64.dmg from GitHub Releases (~110 MB to download, ~245 MB on disk — most of it is the bundled Chromium engine, which is the product). Open the dmg, drag AI Browser to /Applications, launch it. Done.

Two things to know before you download it:

• It is not notarized. macOS Gatekeeper will refuse to open it on the first try. Right-click the app → Open, confirm once, and it runs normally afterwards. (A paid Apple Developer account is what removes this step; until then, this is the honest cost.)
• Only macOS arm64 and Windows x64 are built. Linux has no package yet — see Compatibility.

Option A2 — Windows installer

Grab AI-Browser-<version>-win32-x64-Setup.exe from the same Releases page (~79 MB). Run it and click through: it installs per user to %LOCALAPPDATA%\Programs\AI Browser (no admin prompt, no UAC), creates the Start Menu entry and a Desktop shortcut by itself, and offers to launch the browser when it finishes. Uninstall is a normal entry in Settings → Apps; it removes the program and both shortcuts but leaves ~/.ai-browser alone, so your profiles and logins survive a reinstall.

Two things to know:

• It is not code-signed. SmartScreen may show "Windows protected your PC" → More info → Run anyway. (A code-signing certificate is what removes this step.)
• The installer is a plain NSIS wizard, so you can pick the install location; the default is the per-user path above.

Agents still connect through the MCP server (Option B below), and the two channels cooperate: when the packaged app is already running, the MCP server finds it on port 9223 and reuses it instead of spawning a second copy of Electron — one browser, one login session, whichever way you start it. So "package for the engine + npx for the MCP bridge" is the normal agent setup too.

Option B — npx (the agent channel)

The package is published as @glittering0717/ai-browser; no clone needed:

npx -y @glittering0717/ai-browser --help

To open the visible browser window directly:

npx -y @glittering0717/ai-browser start

Requires Node ≥ 18 and a desktop session (this is a GUI app). If a launch needs its own engine, the first run prepares it (~100 MB, downloaded once). npm ≥ 11 blocks dependency install scripts by default — the launcher detects that and runs Electron's own downloader itself, so nothing extra is needed. If the download from GitHub releases is slow or blocked (common on mainland-China networks), point it at a mirror:

ELECTRON_MIRROR=https://npmmirror.com/mirrors/electron/ npx -y @glittering0717/ai-browser start

Option C — from source (fallback)

Use this when there is no package for your platform, the package gives you trouble, or you need to modify the code. It is the heaviest of the three: npm install pulls the full dependency tree (~276 MB), then the Electron engine (~100 MB) on top.

git clone https://github.com/Glittering/ai-browser.git
cd ai-browser
npm install

If the Electron binary download is slow or fails outright, use the mirror:

ELECTRON_MIRROR=https://npmmirror.com/mirrors/electron/ npm install

Or make it sticky by creating a .npmrc in the project root:

electron_mirror=https://npmmirror.com/mirrors/electron/

Confirm the binary actually landed before blaming anything else — a missing binary still leaves you with an "install that succeeded":

ls node_modules/electron/dist

To check the install end to end:

npm start

A 1280×800 window opens and the terminal prints AI Browser WS server listening on ws://localhost:9223. (The first tab opens on the default start page; ignore it and navigate away.)

To build the package yourself:

npm install
node scripts/build_dmg.cjs              # current arch
node scripts/build_dmg.cjs --arch arm64 # or x64

Output lands in dist/. The script uses @electron/packager plus macOS's built-in hdiutil (no external binary downloads); the reasoning — including why electron-builder refuses to build this project — is documented in docs/DEPENDENCY_AND_DISTRIBUTION_AUDIT.md.

---

Connect your agent

Add this to your MCP client's config file — Claude Desktop's config, ~/.cursor/mcp.json, or a project-level .mcp.json:

{
  "mcpServers": {
    "ai-browser": {
      "command": "npx",
      "args": ["-y", "@glittering0717/ai-browser", "mcp"]
    }
  }
}

A ready-made copy sits in examples/mcp-config.json. From-source users can instead point command: "node" at /absolute/path/to/ai-browser/src/main/mcp_server.js.

You do not need to start the browser yourself. On the first browse_* call the MCP server probes port 9223; if nothing is listening it spawns Electron detached and waits up to 30 seconds for the WebSocket server (the very first call may additionally spend time preparing the engine). If you would rather keep one long-lived window shared by every client, launch the AI Browser app you installed in Option A (or run npx -y @glittering0717/ai-browser start, or npm start from source) — a second launch just focuses the existing window.

examples/AGENT_INSTRUCTIONS.md has a copy-paste prompt you can drop into a CLAUDE.md, Cursor Rule, or any system prompt; it teaches the tool loop, the token-saving params, and when to ask a human. Agents installing this themselves (Cline, etc.) can read the short machine-oriented llms-install.md instead of this README.

---

The MCP tools

Source of truth: src/main/mcp_tools.js. There are 17 — a deliberately small surface, since every tool schema is injected into your agent's context on every turn.

Tool	One line
browse_navigate	Load a URL in your tab or the given tab.
browse_get_tree	Read the page as a semantic tree + page context (modals, messages, stats).
browse_act	click / type (append at caret) / setContent (replace all) / clear / focus / hover / scroll_to / upload / get_value (read-only) / drag / press / wheel on a data-ai-id.
browse_diff	snapshot a baseline, then diff to get added / removed / changed paths with before/after values. This is how you prove an action did anything.
browse_capabilities	Probe what a page affords and which route to take on it (see below).
browse_repaint	Force the compositor to redraw the tab when the window shows a stale page; returns repaint_count.
browse_downloads	List a tab's downloads with their on-disk paths. Downloads are asynchronous — poll this after clicking an export button.
browse_canvas	Read <canvas> through captured draw calls: list / read / configure / capture. Experimental — see the canvas section below.
browse_evaluate	Run a JS expression in page context (5000-char cap, Node identifiers rejected).
browse_scroll	Scroll the page, or scroll an element into view.
browse_wait	Poll until button_enabled / modal_appeared / text_contains / url_contains.
browse_tabs	Tabs: list / new / close / switch / window. switch = default; add show:true. new never steals the view unless show:true.
browse_network	Chrome-DevTools-style network inspection: list (filter by method / URL / status / type), get (request + response headers and bodies), clear, configure.
browse_auth_state	Login-state self-check for a tab: are we logged in, or bounced to a login wall. Cheap — no tree is built.
browse_screenshot	Diagnostic viewport/element capture (png/jpeg/webp, clip, fullPage, source: auto/surface/cdp). Not the read path — use it when the tree and the screen disagree. The receipt says which tier answered.
browse_errors	Recent errors the browser recorded (crashes, failed navigations, RPC failures) with hints. Ask here when behaviour doesn't match the docs.
browse_quit	Shut the whole browser down and release the process.

What every act result already tells you: effect_state

browse_act no longer leaves you guessing whether your input reached the page. Every result carries an explicit verdict:

effect_state	Meaning	retry
committed	dispatched and acked by the page	no
none	provably nothing happened (no such target, bad params, CDP down)	safe
unknown	cannot prove nothing happened — timeout, paste dispatched but never consumed, half a key sequence	verify_first

The default is unknown, not none: when the browser cannot tell, it says so. Retrying an unknown blindly is how a form gets submitted twice — read the page first. unknown results also carry a one-sentence effect_hint saying exactly that.

When a target cannot be found, the result adds stale_ref / ref_epoch / tree_epoch / ref_diagnosis, because "no such target" hides three very different situations:

• ref_diagnosis: "stale" — the page was re-read after you took that ref (ref_epoch < tree_epoch). Get a fresh one; retrying the same ref will keep failing.
• ref_diagnosis: "never_seen" — the ref has never been in any tree read so far. Re-reading will not help: wrong value (visible text instead of a data-ai-id), a typo, or another tab's ref.
• ref_diagnosis: "present_in_tree" — in the current tree but not found in the DOM: iframe, shadow root, or a detached node.

All three report effect_state: "none" (nothing happened) with retry: "verify_first" — no side effects, but repeating the identical call is pointless.

Prove it happened: browse_diff

A tool reporting success is not evidence that anything changed — the browser will happily tell you a click landed on an element that ignored it, a drag completed when the app snapped the node somewhere else entirely, or a keystroke went into a field that then cleared itself. Every write should be wrapped in a snapshot/diff pair:

const base = await browse_diff({ operation: "snapshot", js: "app.graph.serialize()" });
await browse_act({ action: "click", target: "e:button-25" });
const d = await browse_diff({ operation: "diff", snapshot: base.snapshot });
// d.verdict === "changed"
// d.changed_paths → [{ path: "nodes[7].widgets.steps", from: 20, to: 21, reason: "value" }]

• verdict is an explicit "changed" / "unchanged", so "nothing happened" is never inferred from an empty array you might have built wrong.
• Every entry carries a path plus both values, so you can check why it changed.
• Arrays are compared element-wise by index, not as sets: a moved node reads as changed entries, never as "one removed + one added" (which would look like two events instead of one reorder).
• Float noise is tolerated (201354.48743718592 vs ...1859 is not a change), but a real 22-unit coordinate shift is.
• Omit js for a site-agnostic DOM snapshot — role / label / value / disabled / size per element plus the page's visible text. No per-site JS required, and plain <div> status text changes are caught.
• Omit js and it also fingerprints the app's own data model when the page hangs one off window. A canvas-rendered app's DOM is a still photograph — dragging a node or editing a widget repaints <canvas> and moves nothing the element walk can see, so a DOM-only diff answers unchanged to a write that demonstrably happened. ComfyUI's documented write path (node.widgets[].value) moved the DOM by exactly zero bytes. With the fingerprint, that same write shows up as model.fp (and model.count when the node count moved). It is read-only (loadGraphData is a loader and is never called), it picks whichever candidate can actually be read back rather than whichever looks most promising, and a library namespace — which scores well on names and has no model to hand back — is skipped.
• rearm: true moves the baseline forward for the next diff, so you can loop "assert nothing changed" across many steps; forget: true drops it.
• A throwing expression, a missing id, or a cross-tab comparison all return an explicit error. They never degrade into an empty snapshot, which would make every subsequent diff report "unchanged" — the one failure mode that turns all verification into false green.

Which route on this page? browse_capabilities

Before improvising on an unfamiliar site, ask. One call reports the semantic-tree size and its role/action histogram, canvas count, same-origin iframes and their internal size, objects the app hangs off window (auto-detected by shape: has .graph / ._nodes / getState / serialize), the JSON endpoints it has already called, and a recommended route with the evidence for it:

• L1_structured_data — the app exposes its own model; read and write the whole graph as JSON.
• L2_semantic_tree — enough elements to find targets by role/label and drive them by data-ai-id.
• L3_pointer_gestures — only for what has no structured path (drag a node, draw a connection, wheel pan/zoom). Always follow with browse_diff.

It also flags traps explicitly: "the body is in a same-origin iframe" or "there are 7 canvases but only 17 semantic elements — stop trying to hit canvas-drawn items with coordinates and go find the data model instead".

Notes that save round-trips:

• browse_get_tree accepts subset: "interactive" to drop layout-only branches, mode: "diff" to return only nodes that appeared/disappeared since the last read, and ax: true to read through the accessibility layer (links then carry url).
• type appends; setContent replaces. They used to be identical (both replaced the whole content), which made incremental editing impossible — insert a Weibo-style topic tag and the next type wiped it. Now type writes at the caret (params.at: end (default) / start / cursor), so "insert a topic, then keep writing the body" works. To refill a field, use setContent (or clear first).
• browse_act also takes action: "get_value" — a read-only way to fetch an element's full text, which matters because browse_get_tree truncates long values at 200 characters (see below).
• Pointer gestures are first-class: drag, press and wheel. Canvas / board / sortable UIs are built on them, and no amount of evaluate can substitute — synthetic PointerEvents do not move a framework that uses Pointer Capture (measured on a React Flow canvas: the node's position changed by exactly 0,0), so these go through CDP trusted input.
  • drag: dx/dy, to_x/to_y, or to_target (drop onto another element). from_anchor / to_anchor (left/right/top/bottom/center) grab a side — a side prefers that element's connection handle, but only if the handle is actually hit-testable (see the note below). from_x/from_y start from a bare point (panning empty canvas), in which case no target is needed.
  • press: key (Delete, Escape, ArrowRight, Enter, … or a combo like "Meta+a"), or keys for a sequence. Deleting a node, undo, deselection — all keyboard-only.
  • wheel: delta_x / delta_y plus hold: ["Control"] for canvas zoom. browse_scroll cannot do this — it calls window.scrollBy, which an infinite canvas ignores entirely.
  • The drag result reports drag_from / drag_to and which element actually sits at each point (from_at_id), plus warning + start_hit_target: false when the press landed on a different element — visual overlap otherwise looks exactly like "drag does nothing".
• actions is inferred from behaviour, not from role. An element whose computed cursor is a drag cursor, or that has draggable="true" / cursor: pointer / an onclick, now advertises drag / click in the tree even when its role is group. Without this, a canvas full of role: group nodes reported actions: [] and the agent could not tell they were interactive at all.

---

• browse_network captures requests from the moment a tab opens, including POST request bodies — no subscription needed. Request headers are redacted by default (authorization / cookie → [REDACTED]); pass include_sensitive_headers: true to see them.
• Most tools take an optional tab for multi-tab work; omit tab to target your own session tab.

---

Letting a human step in