AgentSpan MCP server
Commit Sep 1, 2026Gives AI agents cached read and search access to 52 web platforms through a REST API, SSE stream, and 91-tool MCP server.
Also known as oxbshw/Agent-Span
- Stars
- 45
- Tools
- —
- Downloads, 30 days
- —
- Latest version
- —
- Last commit
- Sep 1, 2026
- License
- MIT
What is AgentSpan MCP?
AgentSpan gives AI agents cached read and search access to 52 web platforms through a REST API, SSE stream, and 91-tool MCP server.
Summary generated from the project's README
About AgentSpan MCP server
Agent-Span gives an AI agent a single gateway for reading and searching 52 web platforms, covering everything from GitHub, YouTube, Hacker News and arXiv through to Twitter, Reddit, Bilibili, Notion, Slack and flight data. It is built for developers who want their client to fetch pages and run channel-scoped searches without wiring each platform separately. Fifty-one of those channels sit in a zero-config tier; the remaining ones need a key or cookie.
The gateway is a self-hosted async Rust binary started with `agentspan serve` on port 8080, not a hosted service, and is meant for use behind your own infrastructure. For AI clients there is a native MCP server exposing 91 tools, which `agentspan mcp install --client claude-code` (or cursor, windsurf, cline) writes into the right config file, plus `agentspan mcp tools` to list them. Access also comes through a REST API with SSE, nine language SDKs, a CLI and a React dashboard, with API keys, tenants, RBAC, rate limiting and audit. A three-tier cache, adaptive backend routing, federated search across channels, request coalescing and a namespaced key/value memory scratchpad distinguish it from simpler installers.
What you can do with it
- 1Read a page for an agentHave the client fetch any URL through the gateway so the page comes back with a summary and reading stats attached.
- 2Search one platformAsk the client to pull Hacker News results for a query through the dedicated channel endpoint.
- 3Query many channels at onceFan a topic out across several channels in a single call, with duplicates merged and cross-source hits ranked first.
- 4Keep state between requestsStore a pagination cursor or seen-set in the scratchpad so a multi-step agent run can pick up where it stopped.
- 5Connect Claude Code or CursorWrite the MCP configuration into the right file for the chosen client with one command instead of editing JSON by hand.
Written from the project's README. Every use case is checked against a quote from it.
Automated checks
Ran Nov 15, 2026Full report →Not verified yet. A server needs a score of 80, no serious finding, and enough data for us to check. How checks work · What to fix for more points
- Source repository linked and reachableoxbshw/agent-span is public and reachablegithub.com/oxbshw/agent-span ↗Pass
- Package names this repository as its sourceNot checked
- Build provenance (Sigstore attestation)Not checked
- Open source license (OSI)MITspdx.org/licenses/MIT.html ↗Pass
- Commit or release in the last yearLast commit or release 2026-09-01github.com/oxbshw/agent-span ↗Pass
- OpenSSF ScorecardNot checked
- Package name not close to a popular packageNot checked
- No critical advisory in the package or its dependenciesNot checked
- No hidden instructions to the model0 tool descriptions and the README read; no override phrases, credential paths or hidden charactersPass
- Destructive tools are labelledNot checked
- Remote endpoint answers the MCP handshakeNot checked
README
View on GitHubAgentSpan gives AI agents persistent, scalable, cached access to 52 internet platforms through one REST API, an SSE event stream, a native MCP server (91 tools), 9 language SDKs, a CLI, and a React dashboard — all on an async Rust core.
The channels also heal themselves — a background monitor auto-switches failing backends, reinstalls broken CLI tools, and alerts on sustained outages — and the gateway learns from its own traffic, suggesting cache-TTL tweaks, faster backends, and platforms worth adding next.
Why AgentSpan?
Agent Reach proved the "capability layer" model (install → diagnose → route) for agent web access. AgentSpan takes that model and turns it into a real gateway — it does the reading itself, behind one API.
| Capability | Agent Reach | AgentSpan |
|---|---|---|
| Architecture | Python installer (agent calls tools) | Async Rust gateway |
| Channels | 13 | 52 |
| Caching | ❌ | ✅ L1/L2/L3 |
| REST API | ❌ (CLI only) | ✅ + SSE + OpenAPI |
| MCP tools | 1 | 91 |
| SDKs | 0 | 9 (Py, JS/TS, Rust, Go, Ruby, Java, PHP, C#, Swift) |
| Multi-tenant / RBAC / audit | ❌ | ✅ |
| Web dashboard | ❌ | ✅ React |
| Docker | ❌ | ✅ |
| Cookie auth · transcription · installer · SKILL.md | ✅ | ✅ |
What AgentSpan is not
- Not a browser-automation agent (can't click, log in, fill forms). Use Browser-Use or Playwright for that.
- Not a generic crawler competing with Firecrawl on arbitrary-site quality. AgentSpan is optimized for the 52 platforms it knows.
- Not a hosted SaaS. It's a self-hosted binary you run on your own infra.
"Why not MCP servers / OpenRouter / Composio / SearXNG / LangChain / a browser agent?" — each gets a concrete answer in docs/why-agentspan.md. Performance claims are CI-verified on every PR: BENCHMARKS.md.
Website

Two surfaces, one design system — built with React 18 + TypeScript + Vite + GSAP (ScrollTrigger) + Lenis, pure CSS (no Tailwind/Bootstrap):
- Home (
/) — the marketing page; the whole scroll is the landing experience. - Status (
/status) — a live, read-only gateway dashboard.
Home (/)
A section-based editorial / futuristic-HUD clone reverse-engineered from a reference design and filled with real AgentSpan content:
| # | Section | What it shows |
|---|---|---|
| 0 | Preloader | gradient gateway orb with pulsing rings, 0→100 boot counter, terminal readouts (<91 MCP TOOLS>, // 2026) |
| 1 | Hero | Web Access Gateway display headline over a distortion-shaded coral gateway figure (its head an open ring routing a request through the sky), pink cloud-sky background, scattered coordinate tags, flank labels, EXPLORE / GITHUB pills |
| 2 | Channels | circular portal with the orb spinning inside + 52 CHANNELS + 4 numbered category cards |
| 3 | Network | Network grid — all 52 channels as brand-colored silhouette avatars, each wearing the platform's real logo (or a hand-drawn glyph where no logo exists), with a live status dot + category |
| 4 | Architecture | pink orbital-ring diagram + the 9 Rust crates — pinned on scroll |
| 5 | Features | the 6 killer capabilities on the pink panel |
| 6 | Showcase | the AGENTSPAN waveform wordmark, stats, and the install snippet |
| 7 | Footer | links, MIT, version |
Signature chrome: corner-frame brackets, a live mouse-coordinate HUD, a sound toggle, dot-grid section texture, animated chevron pill CTAs, a 1s per-section background cross-fade, and GSAP scroll-reveal on every section.
Status dashboard (/status)

A read-only at-a-glance view of the gateway, sharing the home page's chrome, fonts, colors and animations: gateway status header (count-up stat cards), a searchable/filterable grid of all 52 channels, performance charts (request volume + latency, recharts), a health monitor table with fallback status, quick actions, and a copy-to-clipboard install snippet.
Launch: cd web && npm install && npm run dev → / and /status.
(?static renders without the preloader/smooth-scroll, handy for screenshots.)
Architecture
flowchart TD
subgraph Clients
SDKs["9 SDKs (Py/JS/Rust/Go/Ruby/Java/PHP/C#/Swift)"]
CLI["agentspan CLI"]
Dash["React Dashboard"]
MCP["MCP clients (Claude/Cursor/...)"]
end
subgraph Gateway["AgentSpan Core (Rust, async)"]
API["REST API + SSE"]
MCPS["MCP Server (91 tools)"]
Router["Router: probe · select · retry · circuit-break"]
Cache["3-tier Cache (L1/L2/L3)"]
Auth["Auth: keys · tenants · RBAC · rate-limit · audit"]
end
Backends["Backends: HTTP APIs · CLI tools · Jina Reader"]
Clients --> API & MCPS
API --> Router --> Backends
Router --> Cache
API --> Auth
Quick start (3 commands)
cargo run --bin agentspan -- serve # 1. start the gateway on :8080
curl "localhost:8080/api/v1/read?url=https://example.com" # 2. read any page
curl "localhost:8080/api/v1/channels/hackernews/search?q=rust" # 3. search
Or with Docker:
docker compose up --build # API + UI + Redis + PostgreSQL + Prometheus + Grafana
Use with AI agents (Claude Code, Cursor, Windsurf)
# One command — writes the MCP config to the right file automatically:
agentspan mcp install --client claude-code # or: cursor, windsurf, cline
# Or print the config and paste it yourself:
agentspan mcp print-config --client cursor
# See all 91 tools:
agentspan mcp tools
See the full guides: Claude Code · Cursor · Windsurf.
Use an SDK
# Once published to PyPI:
# pip install agentspan
# Until then, install from source:
pip install -e sdk/python
from agentspan import AgentSpanClient
client = AgentSpanClient(base_url="http://localhost:8080")
print((await client.read("https://example.com")).body)
See sdk/README.md for all 9 languages and docs/api-reference.md for the full API.
Channels (52)
| Tier 0 (zero-config) | Tier 1 (needs key/cookie) |
|---|---|
| web, github, youtube, tiktok, rss, hackernews, v2ex, exa, wikipedia, arxiv, quora, pinterest, npm, crates, pypi, gitlab, dockerhub, wayback, maps, weather, coinbase, duckduckgo, gnews, statuspage, huggingface, devto, openlibrary, gutenberg, lobsters, wikidata | twitter, reddit, bilibili, xiaohongshu, instagram, linkedin, xueqiu, xiaoyuzhou, discord, telegram, spotify, twitch, scholar, podcasts, openai, anthropic, brave, bing, google, notion, slack, flight |
Every channel implements read and/or search, runs through a health-checked
backend router, and reduces tokens via format_for_llm. Run agentspan doctor
to see which backend is serving each channel right now.
CLI
agentspan serve | doctor | watch | format | benchmark | transcribe | tunnel
| plugin | install | uninstall | setup | config | skill | mcp | completions | update
Highlights: benchmark (cache-hit vs backend p50/p99), tunnel (public URL via
cloudflared), plugin (community channels), transcribe (Whisper), format
(per-channel token-reduction rules), completions <shell> (bash/zsh/fish/
powershell/elvish), and config backup/config restore.
Observability & operations
GET /metrics— Prometheus text exposition (request counts, errors, shed requests, latency, channel gauge). Public, so scrapers need no API key.- Request tracing — every request/response carries an
x-trace-id(an inboundx-trace-id/x-request-idis honoured); the id is attached to audit events for correlation. - Resource limits — a 2 MiB request-body cap plus a global in-flight
concurrency limit that sheds excess load with
503instead of unbounded queueing. - Graceful shutdown —
agentspan servedrains in-flight requests on Ctrl-C and (on Unix)SIGTERM.
Smart routing & content
- Adaptive routing — opt in with
BackendRouter::with_adaptive_routing(). The router learns each backend's EWMA latency and success rate from live traffic and prefers the better performer within a health tier. Unknown backends stay optimistic so they still get tried. - Content intelligence — fetched pages are classified (article / forum /
code / docs) with key facts (URLs, dates, code blocks), reading stats
(word/sentence count, reading time), and an extractive summary (the most
salient sentences) pulled out and attached as metadata;
smart_truncatetrims on sentence boundaries, not mid-word. All dependency-free heuristics — no model calls. - Conditional revalidation — the direct-HTTP path remembers
ETag/Last-Modifiedand revalidates withIf-None-Match/If-Modified-Since, so unchanged pages come back as a cheap304instead of a full re-download. - Federated search —
POST /api/v1/search/federatedqueries many channels at once, de-duplicates by URL (merging which channels found each result), and ranks cross-source hits first. One channel failing doesn't sink the query. Pass"rerank": trueto instead order results by a dependency-free lexical relevance score (title-weighted TF with a phrase bonus), and"collapse": trueto merge near-duplicate hits (the same story re-syndicated under different URLs) by title similarity. - Request coalescing —
with_request_coalescing()collapses a dogpile of concurrent identical reads into a single upstream fetch (single-flight), so five agents asking for the same URL in the same second hit upstream once.
Agent memory
A namespaced key/value scratchpad so agents can persist small state — cursors, seen-sets, notes — across requests without standing up their own store:
curl -X PUT localhost:8080/api/v1/memory/agent1/cursor \
-H 'content-type: application/json' -d '{"value":{"page":3},"ttl_secs":3600}'
curl localhost:8080/api/v1/memory/agent1/cursor # -> {"value":{"page":3},...}
curl localhost:8080/api/v1/memory/agent1 # -> {"keys":["cursor"]}
In-memory and process-local (survives requests, not a restart); entries take an
optional TTL and are swept lazily. GET/PUT/DELETE on
/api/v1/memory/{namespace}/{key}, GET on /api/v1/memory/{namespace}.
AI-native content primitives
Dependency-free, deterministic building blocks in agentspan-channels that make
reads cheaper and more useful for agents — no model calls, no extra services:
- Token-budget compiler (
budget::fit_to_budget) — "give me this page in ≤ 2 000 tokens." Fits any text to a hard token ceiling, degrading gracefully from extractive summary to boundary-aware truncation, and reports which strategy it used. The estimate is a safe upper bound, so the ceiling holds. - Content fingerprinting (
fingerprint) — an exact FNV-1a hash plus a 64-bit SimHash, so an agent can ask "did this page meaningfully change since I last read it, and by how much?" without diffing bodies. Complements ETag revalidation (which only says the bytes differ). - Structured extraction (
extract) — pull typed fields (title, links, dates, emails, prices, summary) into JSON and project just the keys you asked for, so a channel can answer in a fixed schema with zero LLM-extraction spend.
These ship as library primitives; REST/MCP surfacing is the next wave.
Project structure
crates/ Rust workspace (core, probe, router, cache, auth, channels, mcp, api, cli)
sdk/ 9 SDKs (python, js, rust, go, ruby, java, php, csharp, swift)
web/ React marketing site + admin dashboard (Vite + GSAP + Lenis)
integrations/ MCP client configs, VS Code/JetBrains/Nvim plugins, GitHub Action
plugins/ Community channel registry
docs/ API reference, guides, mdbook site, i18n
tests/ Integration + load test suites
See KNOWN_ISSUES.md for current rough edges and build quirks.
Community
- Issues — bugs and feature requests
- Discussions — questions and show & tell
- CONTRIBUTING.md — how to add a channel or MCP tool
Security
- Binds to
127.0.0.1by default; the Docker image sets0.0.0.0. - Single-user mode (default): admin routes return
403. Setauth.require_api_key=trueto enable scoped API keys. - API keys stored as SHA-256 hashes;
~/.agentspan/config.yamlis0600and secrets are masked in output/logs. See SECURITY.md.
License
MIT — see LICENSE.
AgentSpan MCP questions
How do I install AgentSpan's MCP server in Claude Code or Cursor?
Run 'agentspan mcp install --client claude-code' (or cursor, windsurf, cline) to write the MCP config automatically, or print the config with 'agentspan mcp print-config' and paste it yourself.
How many MCP tools and channels does AgentSpan provide?
It exposes 91 MCP tools covering 52 internet platforms, each implementing read and/or search.
Does AgentSpan require API keys for every channel?
No, 30 platforms are zero-config Tier 0 channels, while others such as twitter, reddit, and google need a key or cookie. Run 'agentspan doctor' to check which backend serves each channel.
Can agents store small state between requests with AgentSpan?
Yes, a namespaced key/value memory API lets agents persist cursors, seen-sets, and notes with optional TTL, though it is in-memory and process-local.
What caching or routing does AgentSpan use for reads?
It uses a three-tier L1/L2/L3 cache plus an adaptive router that probes, selects, retries, and circuit-breaks backends.