local-websearch MCP server
Commit Oct 19, 2026Performs DuckDuckGo web searches and retrieves webpage content or images for AI models via the MCP stdio protocol.
Also known as bennmann/local-websearch
- Stars
- 2
- Tools
- 3
- Downloads, 30 days
- —
- Latest version
- —
- Last commit
- Oct 19, 2026
- License
- Apache-2.0
What is local-websearch MCP?
local-websearch performs DuckDuckGo web searches and retrieves webpage content or images for AI models via the MCP stdio protocol. It exposes 3 tools, such as search, fetch_page_content, fetch_image.
Summary generated from the project's README
About local-websearch MCP server
Searching the web and extracting webpage content for AI models is what local-websearch enables through the MCP stdio protocol. This Python-based server allows developers to connect an AI client to DuckDuckGo to perform queries and retrieve specific information from URLs. It is designed to be a respectful crawler, implementing rate limits of 30 searches per minute and a one-second minimum interval per host during content fetching.
Users can access three primary tools: search for finding titles and snippets, fetch_page_content for retrieving cleaned text from HTML or verbatim data from JSON and XML, and an optional fetch_image tool. The image capability is only active if the operator enables the WEBSEARCH_VISION setting. The server handles content intelligently by stripping unnecessary HTML elements like scripts and navbars while refusing binary data that could corrupt the model's context. It requires a local Python environment and is configured via standard MCP client settings.
What you can do with it
- 1Perform web searchesAsk an AI to find recent news or specific information using DuckDuckGo.
- 2Extract webpage textProvide a URL to an AI so it can read and summarize the main article content.
- 3Read structured dataRetrieve raw JSON or XML data from a web endpoint for programmatic analysis.
Written from the project's README. Every use case is checked against a quote from it.
local-websearch MCP tools 3
Detected from the READMEsearchfetch_page_contentfetch_image
| search | — | |
| fetch_page_content | — | |
| fetch_image | — |
Automated checks
Ran Nov 14, 2026Full report →Not verified yet. A server needs a score of 80, no serious finding, and enough data for us to check. How checks work · What to fix for more points
- Source repository linked and reachablebennmann/local-websearch is public and reachablegithub.com/bennmann/local-websearch ↗Pass
- Package names this repository as its sourceNot checked
- Build provenance (Sigstore attestation)Not checked
- Open source license (OSI)Apache-2.0spdx.org/licenses/Apache-2.0.html ↗Pass
- Commit or release in the last yearLast commit or release 2026-10-19github.com/bennmann/local-websearch ↗Pass
- OpenSSF ScorecardNot checked
- Package name not close to a popular packageNot checked
- No critical advisory in the package or its dependenciesNot checked
- No hidden instructions to the model3 tool descriptions and the README read; no override phrases, credential paths or hidden charactersPass
- Destructive tools are labelledNo tool named like delete, drop or execPass
- Remote endpoint answers the MCP handshakeNot checked
README
View on GitHublocal-websearch
A local DuckDuckGo web search MCP server in a single Python file (414 lines). Two tools over the MCP stdio protocol, plus an opt-in third for images.
Searches are rate-limited to about 30 per minute to stay respectful to DuckDuckGo. Fetches are throttled per host, identify this tool instead of impersonating a browser, and never HTML-parse a non-HTML body. Exactly what is and is not honoured is listed under Web etiquette.
Tools
search(query, max_results)— Searches DuckDuckGo and returns titles, URLs, and snippets.fetch_page_content(url, max_length)— Retrieves the content of a URL.text/htmlandapplication/xhtml+xmlare reduced to their main text (script,style,header,footer,nav,asideremoved); every other content type — JSON, XML, YAML, plain text — is returned verbatim. HTML extraction is never applied to non-HTML: it deletes<...>tokens and injects newlines into JSON string literals. Binary bodies are refused, never returned as decoded garbage. Defaultmax_lengthis 15000 characters, clamped to 1-50000; the result is never longer than requested and the truncation marker is counted inside that budget (Wikipedia articles average 10k-20k chars; short articles 3k-5k; long articles 50k+).fetch_image(url)— Optional, hidden unless enabled. Returns a raster image (image/png,image/jpeg,image/gif,image/webp) as MCP image content for a vision model. Registered only when the operator setsWEBSEARCH_VISION=1, sotools/listadvertises what this deployment can actually serve. Images are never truncated: one overWEBSEARCH_MAX_IMAGE_BYTESis refused rather than damaged, and bytes that do not match their declared type are refused too. PDFs, audio, and video are not images and are refused;image/svg+xmlis markup and is served as text byfetch_page_content.
Failures from either tool are reported as MCP tool errors (isError, message prefixed [SEARCH FAILED] / [FETCH FAILED]) instead of being returned as text, so a failed request can never be read as page content.
Installation
pip install -e .
Then point an MCP client at server.py:
Vibe
[[mcp_servers]]
name = "local-websearch"
transport = "stdio"
command = ["python3", "/path/to/mcp-local-websearch/server.py"]
Any MCP client using stdio
{
"mcpServers": {
"local-websearch": {
"command": "python3",
"args": ["/path/to/mcp-local-websearch/server.py"]
}
}
}
Configuration
| Variable | Default | Effect |
|---|---|---|
WEBSEARCH_USER_AGENT |
local-websearch/<version> (+https://github.com/bennmann/local-websearch; automated MCP tool; not a browser; driven by an AI agent harness) |
User-Agent sent by fetch_page_content. |
WEBSEARCH_VISION |
unset | 1/true/yes/on/images registers fetch_image. An operator's claim that this client's model can consume image content; off by default because guessing wrong puts an unusable blob in the transcript. |
WEBSEARCH_MAX_IMAGE_BYTES |
4194304 (4 MiB) |
Raw byte limit for fetch_image. Base64 inflates the payload by about a third, so the wire cost is larger than this number. |
The default User-Agent names the tool, its version, and a contact URL, and says the client is automated. It deliberately does not name the agent harness or the model behind it, and it does not impersonate a browser. If you override it, keep it identifying: API hosts (GitHub included) ask for a client name, and a spoofed browser string stops operators from applying a policy to you.
This applies to fetch_page_content only. search goes through ddgs, which builds its own primp client with impersonate="random" and exposes no header hook (DDGS(proxy, timeout, verify)), so search requests reach DuckDuckGo as a random browser, not as this tool.
The remaining limits are constants in server.py: _MIN_SEARCH_INTERVAL (2s), _MIN_FETCH_INTERVAL (1s per host), _MAX_RETRY_WAIT (10s), and a 10s request timeout. _VERSION is duplicated from pyproject.toml; bump both together.
Web etiquette
- Search: 2-second minimum interval (~30 requests/min) against DuckDuckGo.
- Fetch: 1-second minimum interval per host, so a caller looping over URLs cannot hammer one target unnoticed.
- Back-off: a
Retry-Afteron 429/503 up to 10s is waited out and the request retried once; a longer hint is reported in the error rather than slept through silently. - Identification:
fetch_page_contentsends an identifyingUser-Agent, never a browser impersonation.searchcannot identify itself whileddgsowns the HTTP client — see Configuration. - Not implemented:
robots.txtand its directives are not consulted, and there is noIf-Modified-Since/ETag revalidation. Check a host's terms yourself before looping over its URLs. - Images: fetching one costs the same per-host interval as fetching text, and the byte limit bounds the damage of a loop over a gallery. A host that mislabels its content type gets an error, not a surprise payload in front of the model.
Tests
test_fetch.py (local-only, gitignored) runs offline — no network, no MCP client — and covers the content-type gate, the max_length budget, Retry-After parsing, per-host throttling, redirect accounting in the throttle, magic-byte checks, the WEBSEARCH_VISION capability gate, and the error shape. The vision cases spawn their own interpreter because the flag is read at import.
python3 test_fetch.py
test_mcp.py is a manual smoke script that drives the real server over stdio and needs network access.
Dependencies
- mcp==1.26.0
- ddgs==9.13.0
- httpx==0.28.1
- beautifulsoup4==4.14.3
Future Improvements
robots.txtcompliance forfetch_page_content: the per-host interval andRetry-Afterhandling are in place, crawler exclusion rules are not.- Now that non-HTML bodies come back verbatim, jq-style flags for parsing GitHub comments and other common JSON-on-the-web.
- A
fetch_documentfor PDFs would need a real extraction story; today they are refused rather than silently mangled, which is the honest default. - An
Acceptheader that says what we can consume (Accept: text/html, application/xhtml+xml, */*) instead of httpx's*/*.
License
Licensed under Apache 2.0.
Dependencies use permissive licenses (MIT or BSD-3-Clause) compatible with Apache 2.0:
| Dependency | License |
|---|---|
| mcp | MIT |
| ddgs | MIT |
| httpx | BSD-3-Clause |
| beautifulsoup4 | MIT |
| lxml (transitive) | BSD-3-Clause |
| primp (transitive) | MIT |
| click (transitive) | BSD-3-Clause |
AI Usage Disclosure
This project was developed with assistance from AI models: Mistral Vibe and Qwen3 27B (various Unsloth quants).
local-websearch MCP questions
What tools does local-websearch MCP provide?
It exposes 3 tools, including search, fetch_page_content, fetch_image.
How does the server handle rate limiting?
Searches are limited to approximately 30 per minute, and content fetching is throttled per host with a 1-second minimum interval.
Does the server respect robots.txt?
No, the server does not currently consult robots.txt or its directives.