ComputerPilot MCP server
Commit Nov 13, 2026Provides a cross-platform runtime for AI agents to perform supervised computer use, including file editing, terminal execution, and browser automation.
Also known as dibbed/computerpilot-mcp
- Stars
- 2
- Tools
- —
- Downloads, 30 days
- —
- Latest version
- —
- Last commit
- Nov 13, 2026
- License
- Apache-2.0
What is ComputerPilot MCP?
ComputerPilot provides a cross-platform runtime for AI agents to perform supervised computer use, including file editing, terminal execution, and browser automation.
Summary generated from the project's README
About ComputerPilot MCP server
Automating complex development workflows and computer tasks from an AI client is what ComputerPilot MCP enables. It provides a local runtime that allows agents to perform supervised actions across filesystems, terminals, and web browsers. Developers can use it to manage code intelligence through LSP, execute Git commands, or run multi-step workflows that survive MCP restarts using durable jobs and crash recovery.
The server runs locally on Windows, Linux, or macOS, requiring Python 3.10 or newer. It can be accessed via a local HTTP endpoint or through a managed secure tunnel. While the core functionality is cross-platform, Windows users gain access to specific tools like semantic UI Automation, native screenshots, and mouse or keyboard input. This setup is intended for users who want to delegate real OS-level work to an agent while maintaining oversight through a local control panel and explicit recovery mechanisms for interrupted mutations.
What you can do with it
- 1Resilient long-running processesUsers can run background tasks that continue working even if the MCP connection is interrupted.
Written from the project's README. Every use case is checked against a quote from it.
Configuration
MCP_EXECUTION_ROUTER_ENABLEDMCP_EXECUTION_ROUTER_EXPLAINMCP_EXECUTION_ROUTER_POLICYMCP_START_MODEMCP_TOOL_PROFILE
Automated checks
Ran Nov 14, 2026Full report →Not verified yet. A server needs a score of 80, no serious finding, and enough data for us to check. How checks work · What to fix for more points
- Source repository linked and reachabledibbed/computerpilot-mcp is public and reachablegithub.com/dibbed/computerpilot-mcp ↗Pass
- Package names this repository as its sourceNot checked
- Build provenance (Sigstore attestation)Not checked
- Open source license (OSI)Apache-2.0spdx.org/licenses/Apache-2.0.html ↗Pass
- Commit or release in the last yearLast commit or release 2026-11-13github.com/dibbed/computerpilot-mcp ↗Pass
- OpenSSF ScorecardNot checked
- Package name not close to a popular packageNot checked
- No critical advisory in the package or its dependenciesNot checked
- No hidden instructions to the model0 tool descriptions and the README read; no override phrases, credential paths or hidden charactersPass
- Destructive tools are labelledNot checked
- Remote endpoint answers the MCP handshakeNot checked
README
View on GitHubComputerPilot MCP
Supervised computer use and developer automation over the Model Context Protocol.
Current source includes the cross-platform runtime-hardening integration validated on Linux and Windows and covered by hosted Linux, macOS, and Windows CI. The latest published release remains 0.9.0 (release notes, validation); the hardening work described here is unreleased and no new version or tag is implied. Optional capabilities keep their documented defaults and require operator configuration.
ComputerPilot MCP gives MCP-capable AI agents a local runtime for files, native document intelligence, terminal and process execution, Git, testing, code intelligence, browser automation, durable jobs, crash recovery, and multi-step workflows. On Windows it also exposes native screenshots, input, and semantic UI Automation.
It is designed for developers who want an agent to do real work on a machine without collapsing everything into one unstructured shell tool. Capabilities are exposed as typed MCP tools, platform-specific features are registered only when supported, long-running work can survive MCP restarts, and ambiguous mutations are never replayed blindly.
Security: this project intentionally has broad local capabilities. Treat access to the MCP endpoint as equivalent to granting a development agent access to the OS account running it. See SECURITY.md.
Cross-Platform Validation
ComputerPilot MCP has been tested with its full automated suite and representative real MCP integrations on both Linux and Windows. macOS passes the same hosted CI matrix and the Platform Release Validation gate; native interactive macOS desktop acceptance is not claimed.
| Capability / validation | Linux amd64 | Windows amd64 | macOS |
|---|---|---|---|
| Automated CI (Python 3.10 and 3.12) | Verified | Verified | Verified |
| Platform Release Validation | Verified | Verified | Verified |
| Native terminal execution | Verified (run_shell, run_process) |
Verified (CMD, PowerShell, run_process) |
Hosted CI coverage |
| Process cancellation and recovery | Verified | Verified | Hosted CI coverage |
| SQLite schema/migration tests | Verified | Verified | Hosted CI coverage |
| Real Chromium acceptance (local fixture) | Verified | Verified | Not independently verified |
| Real Pyright language-server acceptance | Verified | Verified | Not independently verified |
| Windows UI Automation | Not applicable | Verified with isolated fixture | Not applicable |
| Sustained bounded MCP soak | Verified (181 s) | Verified (181 s) | Not independently verified |
Full automated regression suite (hosted CI at c33004e) |
2,516 passed, 24 skipped | 2,526 passed, 14 skipped | 2,516 passed, 24 skipped |
At c33004e, hosted CI reported the counts above. A local Windows grouped acceptance run recorded 2,505 passed, 35 skipped, 0 failed across 160 test files, and a final ungrouped local run after the Pyright launcher fix recorded 2,508 passed, 35 skipped, 0 failed; the difference from hosted counts is mostly symlink-privilege skips on the local host. The final integration adds three Pyright launcher regression tests, so its own hosted logs are the authority for updated totals. Counts also vary by platform and selection. The Linux and macOS suites exercised real MCP stdio/HTTP sessions, POSIX process ownership and cancellation, stale-claim reconciliation, durable jobs/workflows, schema v12 migration coverage, Chromium, Pyright and a multi-minute live MCP soak. Windows native acceptance additionally covered Job Objects, CMD/PowerShell, process-tree cleanup, UI Automation, browser uploads/downloads and an isolated 181-second resource soak.
Linux provides the portable ComputerPilot tool catalog and full authorized operating-system capabilities (files, terminal, Git, testing, documents, browser, jobs, recovery and workflows). Windows-only UI Automation tools are not registered on Linux or macOS. Hosted macOS results validate portable functionality; they do not establish that every interactive or hardware-dependent workflow was tested on a native Mac.
Evidence:
- Cross-platform runtime hardening report
- Windows native acceptance report
- Linux deep integration audit
- Linux reliability hardening report
- CI run 42442780885 and Platform Validation run 42442780796 at
c33004e - CI workflow and Platform Validation workflow
Those runs validated c33004e; later commits require their own hosted results. Detailed measurements and known gaps stay in the reports rather than being presented as universal guarantees.
Why ComputerPilot MCP
- One supervised runtime for coding and computer use. Files, code search, LSP, Git, diagnostics, processes, browser automation, jobs, workflows, recovery, and system inspection share one MCP surface.
- Durable instead of request-bound. Long-running commands can use persistent jobs with idempotency keys, bounded concurrency, cancellation, incremental output, and restart survival.
- Recovery is explicit. Interrupted mutations can become uncertain and require evidence-based reconciliation or operator acknowledgement instead of blind replay.
- Cross-platform core, capability-gated edges. Windows, Linux, and macOS share the portable core; native Windows desktop/UI Automation is only registered on supported hosts.
- Verified remote tunnel bootstrap. Tunnel mode downloads the official OpenAI Secure MCP Tunnel runtime into local state and verifies release metadata, checksums, and binary version before publication.
- Release evidence is reproducible. CI, platform validation, deterministic packaging, checksums, Doctor checks, benchmarks, and release validation live in the repository.
Security Model
ComputerPilot is a full-capability local automation runtime, not an OS sandbox. Access to the MCP endpoint is equivalent to granting the connected agent the local permissions of the account that runs the server.
- Run the server only under an account whose filesystem, process and desktop access match what you intend to delegate.
- The local HTTP endpoint and control panel bind to loopback by default; do not expose them directly to untrusted networks. Use the managed Secure Tunnel path for remote access and protect its credentials.
- Arbitrary terminal commands are conservatively coordinated as exclusive.
access="read"remains a wire-compatibility hint; it does not make arbitrary shell code safe to run concurrently. Native typed readers retain read concurrency. - Cancellation operation IDs are process-local bearer capabilities, not client authentication. They are not persisted, and durable audit records store only a keyed correlation rather than the raw ID.
- Uncertain external side effects are not replayed blindly. Durable jobs use their own cancellation contract and are not cancelled by
cancel_operation. - External extensions and language providers run with the server account's permissions. Minimal environments, hashes, process ownership and filters reduce risk but do not create an OS sandbox.
See SECURITY.md for reporting, the local authority model and operational guidance.
Quick Start
The fastest way to try the project without tunnel credentials is loopback-only local HTTP mode.
Requirements: Python 3.10 or newer, Git, and sh on Linux/macOS. Browser automation and LSP intelligence have separate optional dependencies.
Windows
git clone https://github.com/dibbed/computerpilot-mcp.git
Set-Location computerpilot-mcp
$env:MCP_START_MODE = "local-http"
.\START_MCP.bat
Linux / macOS
git clone https://github.com/dibbed/computerpilot-mcp.git
cd computerpilot-mcp
MCP_START_MODE=local-http ./start_mcp.sh
If executable mode was lost while copying files on POSIX:
chmod +x start_mcp.sh
The launchers create or reuse .venv, validate Python and dependencies, run cached/full startup checks, and start the supervisor.
- MCP endpoint:
http://127.0.0.1:8765/mcp - Local control panel:
http://127.0.0.1:8766/
For Secure Tunnel mode, browser setup, language-server configuration, release archives, and platform-specific notes, see Installation.
What You Can Do
| Area | Capabilities |
|---|---|
| Files & editing | bounded reads, search, atomic writes, exact/anchored edits, AST symbol-body edits, transactional unified patches, rollback-capable refactors |
| Code intelligence | project summaries, Python AST metadata, dependency graphs, consolidated code context, LSP definition/references/symbols/hover/call hierarchy/diagnostics |
| Git & verification | status/diff/show/blame/merge-base, guarded branch/stage/commit/restore, affected-test selection, pytest, Ruff, mypy, change-aware verification |
| Terminal & processes | native process execution, background processes, durable jobs, Windows CMD, PowerShell where available, POSIX shell on Linux/macOS; arbitrary commands take exclusive coordination, while typed native readers retain read concurrency |
| Documents | native XLSX/XLSM inspection and guarded edits, DOCX inspection/search/conservative mutation, bounded PDF metadata/text extraction, deterministic text/Markdown PDF creation |
| Browser | Playwright sessions, bounded semantic snapshots/queries, generation-scoped refs, semantic click/fill/select/waits, tabs, extraction, controlled upload/download, screenshots, isolated contexts, shared compatible browser processes, bounded shutdown and idle cleanup |
| Windows desktop | native screenshots, mouse/keyboard input, semantic UI Automation with bounded locators |
| Recovery & workflows | mutation journal, evidence-based reconciliation, durable workflow plans, optimistic versions, leases, restart recovery, bounded retries, stale-claim reconciliation |
| Adaptive execution routing | deterministic route candidates/recommendations across native, semantic, and explicitly allowed raw-desktop paths; hard safety constraints and explainable policy metadata |
| Operations | CPU/memory/disk/process/service/software inspection, health/resource budgets, local control panel, supervised restart/stop |
| Project memory | bounded versioned records with provenance and optimistic revisions |
| Secure Tunnel | verified managed runtime download/update, pinning, checksums, version validation, managed offline cache |
See Tooling and workflows for the public tool model and practical examples.
Architecture
flowchart TD
A[MCP client / AI agent] -->|Local HTTP| B[ComputerPilot MCP]
A -->|OpenAI Secure MCP Tunnel| T[Managed tunnel runtime]
T -->|local stdio| B
B --> D[Typed tool domains]
D --> F[Filesystem / project / LSP]
D --> P[Terminal / process / system]
D --> G[Git / testing]
D --> R[Jobs / recovery / workflows]
D --> O[Documents: Excel / DOCX / PDF]
D --> U[Browser / desktop]
B --> S[Supervisor + lifecycle]
S --> H[Health / control panel]
S --> L[.agent_state]
R --> L
Key runtime invariants:
- local HTTP and the control panel bind to loopback by default;
- filesystem mutations use scoped locks and guarded edit primitives;
- durable jobs keep persistent state and disk-backed output;
- uncertain side effects are not automatically replayed;
- arbitrary terminal commands are coordinated exclusively because arbitrary code cannot be proven read-only;
access="read"is a compatibility hint, not a concurrency grant; - Windows process ownership uses Job Objects, while POSIX uses sessions/process groups, with bounded cancellation and stale-claim reconciliation;
- browser sessions use isolated contexts with bounded pool/session lifetimes; the shared browser manager is drained and Playwright is stopped when the last server lifetime ends;
- language servers run under native process ownership with a minimal environment; the Node runtime directory is added only for known launcher dependencies such as the pip
pyright-langserverwrapper; - document mutations use scoped same-file locks, optional SHA-256 preconditions, staged validation, backups, atomic publication, and durable file-hash recovery checkpoints;
- generated runtime state lives under
.agent_state/and is excluded from Git.
A deeper component map is in Architecture.
The opt-in native_read_batch tool reduces multiple independent file-window requests to one call. The 2026-10-28 quiet local single-file gate and real GPT/Tunnel acceptance passed: N16 p95 fell 83.37% against direct parallel reads on the measured connection. A loaded stdio run exceeded the local target, so this is not a universal load guarantee. Keep read_file for a single file. See batch limits and measured results.
Platform Support
| Platform | Release label | Validation status | Native desktop / UIA | Notes |
|---|---|---|---|---|
| Windows amd64 | Supported | CI + Platform Validation + native interactive acceptance | Yes | Full portable core plus Windows desktop capabilities; 152-tool full profile observed |
| Linux amd64 | Preview | CI + Platform Validation + native Linux acceptance | No | Portable filesystem/documents/terminal/browser/Git/testing/jobs/recovery/workflows; 138-tool POSIX full profile observed |
| macOS arm64 | Preview | CI + Platform Validation | No | Portable core; browser is the UI automation path; native interactive acceptance not claimed |
| Linux arm64 | Preview package target | Packaging/updater mapping | No | No dedicated hosted-runner execution claimed |
| macOS amd64 | Preview package target | Packaging/updater mapping | No | No dedicated hosted-runner execution claimed |
The release workflow builds five target archives from the same tracked source. Linux and macOS remain labeled Preview because the project does not claim exhaustive coverage of every distribution, desktop environment, interactive workflow, or architecture. The portable tool catalog and full authorized OS capabilities are supported on Linux; Windows-only UI Automation is not emulated there. Platform claims and limitations are documented in Platform support.
Practical Agent Workflows
Inspect → change → verify
code_context
↓
apply_patch / rename_symbol / replace_exact
↓
affected_tests
↓
verify_changes
Run a long task without tying it to one MCP request
submit_job
↓
job_wait
↓
job_output
↓
cancel_job (when needed)
Recover an interrupted mutation
list_uncertain_operations
↓
inspect_uncertain_operation
↓
reconcile_operation
↓
acknowledge_uncertain_operation (only when evidence cannot resolve it)
Durable multi-step automation
workflow_plan
↓
workflow_start
↓
workflow_execute
↓
workflow_status / workflow_operations
↓
workflow_reconcile when a side effect is uncertain
Built-in workflows include implement_and_verify, safe_git_commit, and prepare_release. They do not push, tag, or deploy automatically.
Tool Profiles
The default full profile preserves the complete catalog. Smaller profiles reduce the active domain set for specialized agents:
minimal, coding, coding_core, read_batch, git, testing, documents, desktop, browser, operations, full.
The opt-in coding_core keeps an exact 43-tool native coding catalog for a client that lists tools once. It includes guarded file/LSP edits, verification, Git, recovery and job observation; extensions need explicit profile eligibility. Its static SDK journey is tested, while the separately budgeted live-model UX pilot remains unverified. It does not change the default. See the profile contract.
The separate opt-in read_batch profile exposes bounded reads for independent local files in one MCP call. It does not provide an atomic multi-file snapshot or guarantee a local speedup; see native read batching.
The coding evolution plan still has open acceptance gates, including the separately controlled live-model comparison. The file-only Batch feature passed its quiet local and real GPT/Tunnel acceptance; contention limits remain documented. Optional grounding, delegation and related extensions retain their separate design gates. The reviewed plan snapshot is historical; current release scope and evidence are in the 0.9.0 notes and validation.
discover_tool_domains reports the current profile and platform capabilities. recommend_tools ranks tools that are actually registered in the active profile and enriches recognized tools with shared execution-route metadata. The read-only execution_candidates and execution_recommend diagnostics use that same registered catalog; metadata alone cannot make an unavailable tool routable.
Set a profile with:
MCP_TOOL_PROFILE=coding
Set the variable before startup and restart the runtime to change profiles. coding omits browser, desktop/UIA, and native document tools. Keep full for mixed coding, desktop, and document work; documents includes filesystem, native documents, recovery, and workflows. A smaller catalog saves serialized discovery bytes by removing capabilities; it does not establish a model-token or task-quality improvement.
Profiles filter registration. They do not grant execution permission or implicitly enable extensions; typed tools and external operator policy retain their existing execution checks.
Adaptive Execution Routing
The routing layer introduced in v0.7 is a decision layer, not a universal dispatcher. It receives a structured intent, generates candidates only from tools registered in the active MCP profile, filters platform/capability and safety constraints, and applies the versioned deterministic policy deterministic-v1.
Initial route classes are:
- native: filesystem, code/LSP, Git, process, system, Excel, DOCX, and PDF;
- semantic: browser and Windows UI Automation;
raw.desktop, only when the request explicitly permits raw desktop fallback and the action is not destructive;visual.desktop, reserved for a later visual-grounding phase and unavailable in v0.9.
The public diagnostics are read-only:
execution_candidates
execution_recommend
There is intentionally no execution_run or hidden arbitrary-action dispatcher. Hard constraints run before ranking: unsupported platform/profile routes are excluded, stale semantic references never degrade to coordinates, destructive ambiguous semantic targets fail closed, destructive raw-desktop fallback is forbidden, and fidelity requirements such as Excel macro preservation cannot be overridden by ranking.
A workflow step may optionally persist a bounded execution_intent. Its selected route, router policy version, compact decision metadata, and bounded fallback history are stored before the side effect begins. If an operation becomes uncertain, the existing evidence/reconciliation path remains authoritative; the router does not blindly reroute and replay it.
Router configuration:
MCP_EXECUTION_ROUTER_ENABLED=1
MCP_EXECUTION_ROUTER_POLICY=deterministic-v1
MCP_EXECUTION_ROUTER_EXPLAIN=1
Runtime health and the loopback Control Panel expose bounded route distribution, fallback/no-route counts, native/semantic/raw usage, latency, and recent decisions without storing raw prompts, document contents, screenshots, credentials, or secrets.
Secure Tunnel Mode
Tunnel mode is the default launcher mode. It requires a control-plane API key. A profile can be supplied explicitly or resolved by the runtime configuration.
Recommended secret file:
.secrets/control_plane_api_key.txt
Or set the key in the environment:
$env:CONTROL_PLANE_API_KEY = "<your-key>"
.\START_MCP.bat
The project does not vendor tunnel-client or Cloudflared executables in Git or release archives. The managed updater downloads the matching official openai/tunnel-client runtime, validates the GitHub asset digest and upstream SHA256SUMS.txt, verifies the reported version, and publishes the selected runtime under .agent_state/tunnel-runtime/.
See Binary provenance.
Browser Automation
Browser tools are optional.
python -m pip install -r requirements-browser.txt
python -m playwright install chromium
python -m scripts.doctor --mode local-http --browser
Playwright sessions use isolated contexts. Compatible sessions can share a browser process, and idle sessions/pools are bounded by runtime budgets. When the last MCP server lifetime ends, the shared browser manager fences new work, drains sessions and pools, and stops the Playwright driver; incomplete cleanup stays fenced and is retried before reuse.
Real Chromium acceptance on Linux and Windows covered semantic snapshots, fill/select/click, extraction, context isolation, uploads, downloads and shutdown with no remaining browser process. Public-site behavior, browser crash recovery and live idle-eviction timing are not universal guarantees.
The semantic browser layer prefers accessible meaning over raw selectors or coordinates:
browser_open_page
↓
browser_snapshot / browser_query
↓
browser_click_semantic / browser_fill_semantic / browser_select
↓
browser_wait_for
Snapshot node refs are generation-scoped. Navigation, semantic mutations, or tab changes invalidate old refs, so stale targets fail closed instead of silently acting on a changed page. browser_extract returns bounded text, links, tables, form fields, or a selected subtree. Uploads require an explicit local file, and downloads are saved under controlled runtime storage with filename sanitization and a configured byte limit.
Language Intelligence (LSP)
The seven read-only LSP tools default to a trusted discovered pyright-langserver --stdio. ComputerPilot does not install a language server automatically and never accepts an arbitrary language-server command from an MCP caller.
Install Pyright with the operator's normal tooling, for example:
python -m pip install pyright
ComputerPilot MCP questions
Is ComputerPilot an OS sandbox?
No, it is a full-capability local automation runtime. Access to the MCP endpoint is equivalent to granting the connected agent the local permissions of the account running it.
What platforms are supported?
The core is cross-platform, supporting Linux, Windows, and macOS, with specific native features like UI Automation available on Windows.
How can I try ComputerPilot without tunnel credentials?
You can use the loopback-only local HTTP mode by setting the MCP_START_MODE environment variable to 'local-http'.