voltcrawl MCP server
Commit Oct 18, 2026Connects AI coding agents to headless Chromium rendering, sitemap navigation, and verified web evidence extraction.
Also known as adi-IL/voltcrawl
- Stars
- 0
- Tools
- 6
- Downloads, 30 days
- —
- Latest version
- —
- Last commit
- Oct 18, 2026
- License
- MIT
What is voltcrawl MCP?
voltcrawl connects AI coding agents to headless Chromium rendering, sitemap navigation, and verified web evidence extraction. It exposes 6 tools, such as volt_crawl_scrape, volt_crawl_search, volt_crawl_research.
Summary generated from the project's README
About voltcrawl MCP server
Coding agents that need to verify library APIs or debug code using real-time web data can use voltcrawl to access headless Chromium rendering and sitemap navigation. It bridges the gap between simple search snippets and actual web content by executing JavaScript and stripping away noise like cookie banners, paywalls, and navigation sidebars. This ensures that the information provided to the agent is clean Markdown or structured JSON rather than truncated or obsolete snippets.
Developers can deploy this server using stdio for local integration with clients like Cursor, Claude Desktop, or OpenCode, or via Streamable HTTP for remote deployments. The setup requires a connection to a Firecrawl instance, which can be a local Docker container, a self-hosted service, or a cloud endpoint. It also supports search grounding through a search proxy or Vertex AI ADC to discover candidate URLs before rendering them for evidence extraction.
What you can do with it
- 1Verify documentation with real evidenceAn AI agent can resolve natural-language questions by pulling verbatim proof lines from rendered web pages.
- 2Scrape modern single-page applicationsUse headless Chromium to render client-side JavaScript and extract content from complex web apps.
- 3Navigate entire domains via sitemapsA developer can instruct an agent to traverse a website's structure using its sitemap.xml file.
- 4Clean web data for agentsRemove distracting elements like OneTrust dialogs and ad-block warnings to get pure content.
Written from the project's README. Every use case is checked against a quote from it.
voltcrawl MCP tools 6
Detected from the READMEvolt_crawl_scrapevolt_crawl_searchvolt_crawl_researchvolt_crawl_mapvolt_crawl_crawlvolt_crawl_status
Configuration
FIRECRAWL_API_URLMASTER_API_KEYSEARCH_PROXY_URL
Automated checks
Ran Nov 14, 2026Full report →Not verified yet. A server needs a score of 80, no serious finding, and enough data for us to check. How checks work · What to fix for more points
- Source repository linked and reachableadi-il/voltcrawl is public and reachablegithub.com/adi-il/voltcrawl ↗Pass
- Package names this repository as its sourceNot checked
- Build provenance (Sigstore attestation)Not checked
- Open source license (OSI)MITspdx.org/licenses/MIT.html ↗Pass
- Commit or release in the last yearLast commit or release 2026-10-18github.com/adi-il/voltcrawl ↗Pass
- OpenSSF ScorecardNot checked
- Package name not close to a popular packageNot checked
- No critical advisory in the package or its dependenciesNot checked
- No hidden instructions to the model6 tool descriptions and the README read; no override phrases, credential paths or hidden charactersPass
- Destructive tools are labelledNo tool named like delete, drop or execPass
- Remote endpoint answers the MCP handshakeNot checked
README
View on GitHubvoltcrawl
Verified web evidence and headless browser rendering for coding agents.
voltcrawl is an open-source Model Context Protocol (MCP) server that connects AI coding agents to real headless Chromium rendering, sitemap navigation, and search candidate discovery. It strips cookie consent banners, paywalls, and navigation chrome from rendered pages, returning clean Markdown, structured JSON schemas, and verbatim quoted evidence.
flowchart TD
subgraph Clients["MCP Clients"]
OpenCode["OpenCode"]
Cursor["Cursor"]
Claude["Claude Desktop"]
end
subgraph Transports["Transports"]
Stdio["stdio (JSON-RPC)"]
StreamableHTTP["Streamable HTTP (JSON)"]
end
subgraph Core["voltcrawl Engine"]
Server["MCP Server (server.ts)"]
NoiseFilter["DOM Noise Filter (research.ts)"]
ProofExtractor["Proof Extractor (research.ts)"]
end
subgraph Upstream["Upstream Services"]
SearchProxy["Search Grounding Proxy<br/>(Vertex ADC / Custom HTTP)"]
Firecrawl["Firecrawl Engine<br/>(Headless Chromium + Crawl Queue)"]
end
OpenCode --> Stdio
Cursor --> Stdio
Claude --> Stdio
OpenCode --> StreamableHTTP
Stdio --> Server
StreamableHTTP --> Server
Server -->|"Search queries"| SearchProxy
SearchProxy -->|"Candidate URLs & snippets"| Server
Server -->|"Render & crawl requests"| Firecrawl
Firecrawl -->|"Rendered DOM & Markdown"| NoiseFilter
NoiseFilter --> ProofExtractor
ProofExtractor -->|"Verified evidence pack"| Server
Why voltcrawl
AI coding agents need current web information to debug code, verify library APIs, and inspect documentation. Standard web search tools fail agents in two ways:
- Search indexers return snippet summaries. These snippets frequently contain truncated code, obsolete syntax, or hallucinated facts.
- Basic HTTP scrapers fail on modern single-page applications. They return blank root containers, cookie consent walls, and scriptless fallbacks.
voltcrawl uses Firecrawl as an honest operational engine for headless browser rendering and sitemap parsing. It pairs that engine with search candidate discovery and deterministic DOM noise filtering. It is not an affiliate pitch or a proprietary cloud wrapper. It connects to any self-hosted Firecrawl instance, local Docker container, or cloud endpoint.
Architecture and highlights
- Headless Chromium rendering: Executes client-side JavaScript and single-page applications before extracting text.
- Deterministic noise filtering: Strips OneTrust dialogs, cookie banners, ad-block warnings, and navigation sidebars. Preserves author bylines, timestamps, code blocks, and data tables.
- Truth contract enforcement: Separates search candidates from rendered evidence. Agents cite verified DOM output rather than search index snippets.
- Decoupled backends: Connects to self-hosted Firecrawl instances on localhost or remote hosts. Pairs with Google Search Grounding via Vertex AI ADC, custom search proxies, or offline test doubles.
- Offline determinism: The 144 unit tests run completely offline with no cloud credentials or network access. The stdio integration test additionally probes a local Firecrawl instance and is the one test that expects a service on
http://localhost:3002.
Research workflow
The volt_crawl_research tool resolves natural-language questions to verified evidence in a single MCP call:
sequenceDiagram
autonumber
actor Agent as Coding Agent
participant VC as voltcrawl (MCP)
participant SG as Search Grounding (Vertex ADC)
participant FC as Firecrawl (Chromium)
participant NF as Noise Filter & Proof Extractor
Agent->>VC: volt_crawl_research(query, scrapeTop, waitFor)
VC->>SG: searchGoogle(query, numResults)
SG-->>VC: Candidate URLs & snippets
loop Render Top Candidates Sequentially
VC->>FC: Scrape candidate URL (waitFor)
FC-->>VC: Rendered page Markdown
VC->>NF: Clean page chrome & extract proof
NF-->>VC: Verified proof lines & excerpt
end
VC-->>Agent: Evidence pack (candidates + rendered proof)
Quickstart
Stdio mode (Default)
Connect to a local or remote Firecrawl instance. Run immediately without manual installation:
export FIRECRAWL_API_URL="http://localhost:3002"
# Run directly via npx
npx voltcrawl
# Or run via bunx
bunx voltcrawl
# Or install globally
npm install -g voltcrawl
voltcrawl
Streamable HTTP mode
Run as a standalone HTTP MCP service with token authentication:
export MASTER_API_KEY="your-secret-master-token"
export FIRECRAWL_API_URL="http://localhost:3002"
# Launch HTTP server on port 8787
npx voltcrawl --http --port 8787 --host 127.0.0.1
Verify service readiness:
curl http://127.0.0.1:8787/healthz
Client configuration
Add voltcrawl to your MCP client configuration file.
OpenCode
Add to ~/.config/opencode/opencode.json or your project opencode.json:
{
"mcpServers": {
"voltcrawl": {
"type": "local",
"command": ["npx", "-y", "voltcrawl"],
"environment": {
"FIRECRAWL_API_URL": "http://localhost:3002"
}
}
}
}
For remote HTTP deployments:
{
"mcpServers": {
"voltcrawl": {
"type": "remote",
"url": "https://mcp.example.com/mcp",
"headers": {
"Authorization": "Bearer YOUR_MASTER_API_KEY"
},
"timeout": 180000
}
}
}
Cursor
Add to .cursor/mcp.json or ~/.cursor/mcp.json:
{
"mcpServers": {
"voltcrawl": {
"command": "npx",
"args": ["-y", "voltcrawl"],
"env": {
"FIRECRAWL_API_URL": "http://localhost:3002"
}
}
}
}
Claude Desktop / Claude Code
Add to claude_desktop_config.json or .mcp.json:
{
"mcpServers": {
"voltcrawl": {
"command": "npx",
"args": ["-y", "voltcrawl"],
"env": {
"FIRECRAWL_API_URL": "http://localhost:3002"
}
}
}
}
Available tools
The server exposes six tools across HTTP and stdio transports:
| Tool | Parameters | Description |
|---|---|---|
volt_crawl_scrape |
url (req), formats, onlyMainContent, waitFor, jsonPrompt, jsonSchema |
Renders a target URL with headless Chromium. Extracts Markdown, HTML, or structured JSON. |
volt_crawl_search |
query (req), numResults |
Discovers search candidates using Google Search Grounding. Returns ranked URLs, titles, and snippets. |
volt_crawl_research |
query (req), numResults, scrapeTop, waitFor, mode, maxExcerptChars |
Resolves natural-language questions to verified evidence. Strips DOM noise and pulls verbatim proof lines. |
volt_crawl_map |
url (req), search, limit |
Traverses domain sitemaps via sitemap.xml. Returns an empty list when no sitemap exists. |
volt_crawl_crawl |
url (req), limit, maxDiscoveryDepth |
Starts asynchronous recursive crawling following links from rendered pages. |
volt_crawl_status |
id (opt), includeDocs, limitDocs |
Polls crawl job progress or verifies instance readiness. |
Error handling
Failed operations return { isError: true } with structured error categories:
stateDiagram-v2
[*] --> RequestReceived
RequestReceived --> ValidateParams
ValidateParams --> ExecuteTool: Valid parameters
ValidateParams --> [*]: Invalid schema (error)
state ExecuteTool {
[*] --> CallUpstream
CallUpstream --> UpstreamSuccess
CallUpstream --> UpstreamError
}
ExecuteTool --> SuccessResponse: UpstreamSuccess
ExecuteTool --> ClassifyError: UpstreamError
state ClassifyError {
[*] --> CheckAntibot
CheckAntibot --> BLOCKED: 403 / Cloudflare / Turnstile
CheckAntibot --> CheckRateLimit: Not antibot
CheckRateLimit --> RATE_LIMITED: 429 status
CheckRateLimit --> CheckTimeout: Not rate limit
CheckTimeout --> TIMEOUT: Deadline exceeded
CheckTimeout --> CheckNetwork: Not timeout
CheckNetwork --> NETWORK_ERROR: Connection refused
CheckNetwork --> UPSTREAM_ERROR: Unclassified
}
ClassifyError --> ErrorResponse: Structured ToolErrorDetail
SuccessResponse --> [*]
ErrorResponse --> [*]
Dependency mapping and packaging
voltcrawl eliminates installation friction for users and agents:
When running via npx or bunx
- Zero dependency installation. The package ships a pre-bundled JavaScript artifact per entrypoint in
dist/, with dependencies inlined. - Executing
npx voltcrawlorbunx voltcrawlruns immediately on any system with Node >= 20 or Bun installed. Both HTTP and stdio transports are verified on both runtimes in CI.
When installed via npm
Installing voltcrawl pulls in two production dependencies, which are also inlined into the published bundles:
@modelcontextprotocol/sdk(^1.6.1): The official Model Context Protocol TypeScript SDK implementing JSON-RPC 2.0 framing,StdioServerTransport, andWebStandardStreamableHTTPServerTransport.zod(^3.24.2): Type-safe schema validation engine powering all six tool parameter contracts.
Configuration
voltcrawl reads configuration from the process environment. It does not load a .env file. For a local run:
export FIRECRAWL_API_URL="http://localhost:3002"
export SEARCH_PROXY_URL="http://localhost:8088"
Environment variables
| Variable | Default | Purpose |
|---|---|---|
FIRECRAWL_API_URL |
http://localhost:3002 |
Upstream Firecrawl v2 REST endpoint |
FIRECRAWL_API_KEY |
(empty) | Optional Bearer token for cloud or authenticated Firecrawl |
SEARCH_PROXY_URL |
http://localhost:8088 |
Upstream search proxy endpoint for Google Search Grounding. VERTEX_PROXY_URL is accepted as a legacy alias. |
MCP_HTTP_PORT |
8787 |
Port for Streamable HTTP transport. PORT is accepted as a fallback. |
MCP_HTTP_HOST |
127.0.0.1 |
Host binding for HTTP transport. HOST is accepted as a fallback. |
MASTER_API_KEY |
(empty) | Enforced on HTTP transport; stdio transport is unauthenticated |
Development and testing
# Install dependencies
bun install
# Run static typecheck
bun run typecheck
# Run 10 offline unit suites (144 tests, reported by bun test)
bun test
# Run stdio integration test
bun run tests/stdio-test.ts
# Build production bundle for Node.js
bun run build
Documentation
Longer guides live in docs/:
- Truth contract — why search snippets are never cited as evidence
- Configure self-hosted Firecrawl — point voltcrawl at your own instance
- Tools reference — every tool, parameter, default, and range
- Quickstart — first verified research call
License
MIT (c) 2026 adi-IL
voltcrawl MCP questions
What tools does voltcrawl MCP provide?
It exposes 6 tools, including volt_crawl_scrape, volt_crawl_search, volt_crawl_research, volt_crawl_map, volt_crawl_crawl.
What makes voltcrawl different from standard web search tools?
Standard tools often return truncated snippets or fail on single-page applications, whereas voltcrawl uses headless Chromium to render pages and strips noise like cookie banners to provide clean Markdown and verified evidence.
Can I use my own Firecrawl instance?
Yes, the server is decoupled and can connect to self-hosted Firecrawl instances on localhost or remote hosts.
Does the server require an internet connection for testing?
No, the unit tests are designed to run completely offline without any network access.