Compresses tool outputs into recoverable handles and expands them to exact original bytes on demand to reduce LLM context costs.
Also known as dshakes/distil, distil-llm
- Stars
- 21
- Tools
- 3
- Downloads, 30 days
- 2,211
- Latest version
- 1.30.0
- Last commit
- Nov 15, 2026
- License
- —
What is Distil MCP?
Distil compresses tool outputs into recoverable handles and expands them to exact original bytes on demand to reduce LLM context costs. It exposes 3 tools, such as distil_compress, distil_expand, distil_savings.
Summary generated from the project's README
About Distil MCP server
Reducing LLM context costs for coding agents is what Distil accomplishes by compressing tool outputs into recoverable handles. Developers using agents like Claude, Aider, or OpenHands can use this MCP server to provide a recall tool that allows an agent to expand a compressed digest back into its exact original bytes on demand. This ensures that large amounts of data do not bloat the context window while maintaining the ability to retrieve precise information mid-task.
Users can integrate this via a local setup using the distil-llm package or by running it as a proxy for various providers including Anthropic, OpenAI, and Gemini. The system is designed to be lossless by default, meaning every compressed byte is recoverable through the distil_expand tool. It also includes auditing capabilities to measure how different compression methods affect both the total bill and the decision-making quality of the agent.
What you can do with it
- 1Recovering large tool outputsAn AI agent can use the expansion tool to retrieve the original, uncompressed bytes of a previous tool result when needed.
- 2Auditing context compression costsDevelopers can run an audit to see if specific compression strategies change the agent's actions or their total expenditure.
- 3Managing long coding sessionsThe server helps manage token usage during extended development tasks by folding away non-essential data into handles.
Written from the project's README. Every use case is checked against a quote from it.
How to install Distil MCP
Formats verified 10 Nov 2026Add to CursorAdd to GooseInstall Distil MCP in Claude Code
$ claude mcp add distil -- uvx distil-llmInstall Distil MCP in Cursor
~/.cursor/mcp.json
{
"mcpServers": {
"distil": {
"command": "uvx",
"args": [
"distil-llm"
]
}
}
}Install Distil MCP in VS Code
.vscode/mcp.json
{
"servers": {
"distil": {
"type": "stdio",
"command": "uvx",
"args": [
"distil-llm"
]
}
}
}Install Distil MCP in VS Code CLI
$ code --add-mcp '{"name":"distil","type":"stdio","command":"uvx","args":["distil-llm"]}'Install Distil MCP in Claude Desktop
claude_desktop_config.json
{
"mcpServers": {
"distil": {
"command": "uvx",
"args": [
"distil-llm"
]
}
}
}Install Distil MCP in Codex
$ codex mcp add distil -- uvx distil-llmInstall Distil MCP in Codex config.toml
~/.codex/config.toml
[mcp_servers.distil] command = "uvx" args = ["distil-llm"]
Install Distil MCP in Gemini CLI
$ gemini mcp add -s user distil uvx -- distil-llmInstall Distil MCP in Gemini settings.json
~/.gemini/settings.json
{
"mcpServers": {
"distil": {
"command": "uvx",
"args": [
"distil-llm"
]
}
}
}Started by HostedMCP on 16 Nov 2026
We ran this command for distil-llm 1.30.0 in an isolated container with no network and no keys. It answered and listed 3 tools.
Distil MCP tools 3
Read from the server with tools/listdistil_compressActionReversibly compress a text blob — typically a large tool output you want to keep in context cheaply. Returns JSON {"compressed": str, "handle": str|null, "tokens_saved": int}: a compact digest to keep in the conversation, plus an 8-hex handle that recovers the exact original bytes via distil_expand. The original is stored locally (encrypted, owner-only) and never sent anywhere. Use when a tool result is large enough that carrying it verbatim is wasteful; skip it for short text, which comes back unchanged with handle=null and tokens_saved=0. Errors return "error: ..." with isError set; nothing is stored.
distil_expandRead-onlyRecover the exact original text for an 8-hex handle returned by distil_compress. Returns the original bytes as plain text — not JSON, not a summary. Use when the digest in context lacks a detail you now need (an exact line, value, or stack frame); prefer reading the digest first, since expanding spends the tokens compression saved. Reads a local, encrypted store, so it works across sessions and processes but not across machines. An unknown, expired, or evicted handle returns "error: no original found for handle ..." with isError set — re-run the original tool rather than retrying; the answer will not change. Handles age out after DISTIL_RESTORE_TTL_DAYS (default 14).
distil_savingsRead-onlyReport cumulative savings from the local distil ledger as JSON {"runs": int, "tokens_saved": int, "dollars_saved": float}. Covers every request distil has compressed on this machine, not just this session. Use to answer 'how much has distil saved me'; it says nothing about whether any one compression was correct. Takes no arguments; an empty ledger reports zeros.
Automated checks
Ran Nov 16, 2026Full report →Passed our automated checks. That is not a security audit: read the code and give the server the narrowest access it needs. How checks work · What to fix for more points
- Source repository linked and reachabledshakes/distil is public and reachablegithub.com/dshakes/distil ↗Pass
- Package names this repository as its sourcedistil-llm names dshakes/distil as sourcepypi.org/project/distil-llm/ ↗Pass
- Build provenance (Sigstore attestation)Sigstore attestation for distil-llmpypi.org/project/distil-llm/ ↗Pass
- Open source license (OSI)No licenseFinding
- Commit or release in the last yearLast commit or release 2026-11-15github.com/dshakes/distil ↗Pass
- OpenSSF ScorecardOpenSSF Scorecard 5.6 / 10; weakest: Code-Review 0, Token-Permissions 0, CII-Best-Practices 0scorecard.dev/viewer/?uri=github.com/dshakes/distil ↗Partial 8/15
- Package name not close to a popular packageNo popular package one edit away from distil-llmPass
- No critical advisory in the package or its dependencies1 resolved dependencies checked; no critical advisoryosv.dev ↗Pass
- No hidden instructions to the model3 tool descriptions and the README read; no override phrases, credential paths or hidden charactersPass
- Destructive tools are labelledNo tool named like delete, drop or execPass
- Remote endpoint answers the MCP handshakeNot checked
README
View on GitHubuv tool install distil-llm && distil setup
Other installs (curl, Homebrew, pipx, Windows): Install your way
distil wrap -- claude # run your agent through distil (or let `distil setup` make it always-on)
distil savings # what it saved you, from your own traffic
distil doctor # if something looks off
distil audit --compressor anthropic-context-editing --cap-usd 1 # did it change what your agent does, and what it cost? (opt-in, capped)
distil audit report # helped, hurt, or can't tell yet, per compressor
Distil also compresses (lossless by default, every byte recoverable), and holds that compressor to the same measurements.
What the measurements say today (2026-10-06; every figure links to its artifact)
- On short coding tasks no compressor is shown to save money. SWE-bench Lite, 298 tasks, same model and grader: distil is at parity with no compression ($0.0393 vs $0.0387 per solved task). RTK looked 20.7% cheaper per task until we found that its arm read other arms' prompt cache; re-priced as a cold run it is −6.5% [−13.1%, +0.1%], not significant. Selective Context and Anthropic context editing are not cheaper. Headroom has not run there yet. Scoreboard →
- Long sessions: inconclusive. A 7-task long-horizon pilot (median 63–69 steps) shows no cost difference for distil or RTK; on the 6 tasks without one crashed attempt, distil cost 9.7% more than plain. Run →
- Why. An offline simulator that reproduces the provider's bill within 2.5% on runs it never saw finds that lossless policies change cost by less than 0.2%, and that lossy compression saves money only if the agent does not take extra steps to make up for what was removed. Measuring that is the next live A/B. Simulator → · Why RTK looked cheaper →
- On the maintainer's own metered Claude Code traffic (13,191 requests, September 2026) distil removed an estimated 8.3–9.2% of the bill, cache priced in, its own spend netted out (source, method). That is a same-trajectory estimate: it assumes the agent behaved the same, which is exactly what the A/B above has to check.
Why trust the measurements
- It doesn't break your prompt cache. Distil never rewrites bytes the provider still has cached; older context changes only once that cache has already expired. We shipped that bug once, measured what it cost, and made the rule an enforced invariant. The cache contract →
- Every compressed byte is recoverable. What distil folds away it keeps in a local store, and the agent gets a
distil_expandtool to pull the exact original back mid-task. How the digest works → - It's measured on a real bill, receipts included. Savings are counted per request and calibrated against the
usageyour provider actually bills; every request writes a hash-chained receipt you can verify. The same harness runs every other compressor, and when it found a flaw in its own head-to-head it published the correction. How it's measured →
And it tells you when it is hurting you. Shadow mode replays a sample of requests on the original and the compressed context and reports the difference per compression mode; when the digest's harm bound goes over budget, the proxy holds itself at lossless-only. Pointed at the providers instead, the same check found Anthropic's context clearing changed the agent's next action in 37 of 40 cases with every old tool result cleared (keep=0, synthetic corpus) (study).
Going deeper: what distil checks, and what it found when it pointed those checks at the providers' own compaction ↓
What it does
- Referee your context tooling —
distil auditreplays a capped sample of your own requests as a named compressor (distil, Anthropic context editing, OpenAI compaction, Headroom) would shape them and reports helped / hurt / can't tell yet, in billed dollars and in the agent's next action (ADR 0024). Offline, the SWE-bench outcome harness scores any arm on dollars per solved task, with each arm's prompt cache isolated; results are on the scoreboard.
- Wrap your agent — 19 presets:
distil wrap -- <agent>foraider·claude·gemini·copilot·goose·grok·kilo·kimi·vibe·openclaude·opencode·openhands·qwen·cline·codex·cn·crush·droid·omp. Zero config, no code change.distil wrap --listprints every target, its mechanism and the provider shape it speaks.
- Run a proxy — point any
base_urlclient at it. Python, TypeScript, any language, any framework. Sync proxy, async proxy, and a standalone gateway, with the same provider coverage in each: Anthropic Messages, OpenAI Chat Completions and the Responses API, Azure OpenAI, GeminigenerateContent, AWS Bedrock InvokeModel and Google Vertex AI (Anthropic and Gemini). Which routes compress → - Call it as a library —
from distil import compress_messagesin your own agent loop. - Give your agent a recall tool — MCP server: it compresses its own output and gets the exact bytes back on demand.
- Framework hooks — LangChain · LangGraph · LiteLLM · Agno · Strands · AutoGen · LlamaIndex, in-process, no network hop — plus an ASGI middleware for any Starlette/FastAPI app that hosts its own LLM endpoint, and the npm package for the Vercel AI SDK. A strict-typed TypeScript SDK (
sdk/typescript, ADR 0029) addscompress(messages, { provider })for Anthropic and OpenAI shapes; like every in-process library it is lossless-only, byte-identical to the Python engine. - Where a proxy can't reach —
distil setup --hooks: Claude Code, Cursor (MCP output), Gemini CLI and Codex CLI compress tool output through their documented post-tool hooks. Lossless-only on a subscription unless you add--digest; every digest recoverable withdistil expand <handle>. No proxy, no credentials touched.distil quotashows the rate-limit window it buys back. Hooks → - Shape shell output at the source —
distil sh -- <command>(opt-in, ADR 0026) keeps every failure, error and summary line of a test run orgit statusverbatim and folds the passing-test noise behind adistil expandhandle;distil hook install --shellmakes Claude Code use it. On SWE-bench Lite transcripts test runs are rare, so it removes 0.06% of bash output there and stays opt-in. Shell → - Same mechanisms on every provider — re-fetch verbatim, cold-point eviction, the edit-quote guard and census run on Anthropic Messages, OpenAI Chat and Responses, and Gemini; OpenAI and Gemini are priced, so savings and
distil auditstop booking $0 there; thedistil savingswhat-if replays Codex CLI and Gemini CLI sessions as well as Claude Code's. Parity matrix → - VS Code Copilot Chat — its BYOK Custom Endpoint can point at a distil proxy:
distil setup --vscode. - Keep a span verbatim —
<distil:keep>…</distil:keep>in a prompt or tool output is never compressed. - Keeps MCP tool search on —
distil wrap -- claudekeeps Claude Code's MCP tool search switched on, which Claude Code otherwise turns off behind a proxy, so unused connectors stay deferred instead of riding along on every turn. Verified live on 1.54.0 (2026-09-25):tools_deferredof 4–5 on every request, tool payload 9,733 tokens, prompt-cache reads intact. (ADR 0013) - Real code skeletons —
pip install 'distil-llm[code]'adds tree-sitter parses for Go, Rust, Java, C/C++, Ruby and TS/JS. Code skeletons → - See what it did — live status line, session dissect, per-request headers, OTel spans, Prometheus metrics.
uv tool install distil-llm && distil setup # detects your agent + billing, wires everything
Not sure which of those you want? Two questions pick your mode → — plain language, honest savings ranges, no jargon.
Will it save you money? On metered billing (an API key) it cuts the tokens you send — an estimated 8.3–9.2% of the bill on the maintainer's traffic, if the agent behaves the same. Whether that shows up as a lower bill depends on the workload: on short SWE-bench tasks the net cost effect measured cost-neutral, and the long-session pilot (7 tasks) is inconclusive. On a flat-rate Pro/Max subscription there is no per-token bill to cut, but there is a rate-limit window, and spending fewer tokens per turn leaves more of it for the next task.
distil quotashows that window live. Savings come from large, repetitive tool output: verbose JSON and duplicated log runs compress 25–99%, while prose and unique-line output compress ~0% — a short session that never reads a big file showing near 0% is the tool working correctly, not failing. Why → Shell output (grep,git, ad-hoc scripts) goes through that same generic, reversible digest (ADR 0015); search, listing and reads stay verbatim. The one exception is opt-in:distil shshapes test-runner andgit statusoutput at the source (ADR 0026).
🧩 Use it as a library
Building the agent yourself? Compress the message list where it lives — no proxy, no network hop:
from distil import compress_messages, expand_handle
result = compress_messages(messages) # OpenAI/Anthropic-style dicts
print(f"{result.saved_pct:.1f}% smaller")
response = client.messages.create(model=..., messages=result.messages)
original = expand_handle(result.handles[0]) # byte-exact, any time, any process
Tool results get the reversible digest; user and system text get lossless transforms only; the model's own turns are never rewritten. Handles resolve across processes and restarts, so a digest made by the proxy expands here and vice versa. verbatim=True disables digests entirely.
Named compress_messages/expand_handle rather than compress/expand because distil.compress and distil.expand are modules — a top-level export sharing those names would resolve to the function or the module depending on unrelated import order.
TypeScript too — compress(messages) from the npm package, byte-identical to the Python engine. Full reference: Library API → · runnable examples: python_library.py · js_library.ts.
Maintain a framework? docs/INTEGRATING.md is the ~20 lines and the four rules — we would rather the integration live in your repo than ours.
Distil MCP questions
How do I add Distil MCP to Claude Code?
Run claude mcp add distil -- uvx distil-llm in your terminal. Add --scope user to make it available in every project.
How do I add Distil MCP to Cursor?
Open ~/.cursor/mcp.json and add the server under mcpServers, using the Cursor config shown in the install section above.
What tools does Distil MCP provide?
It exposes 3 tools, including distil_compress, distil_expand, distil_savings.
Is the compression reversible?
Yes, the compression is lossless by default; every compressed byte is recoverable via a local store and a distil_expand tool.
How does Distil handle prompt caching?
Distil ensures it never rewrites bytes that the provider still has cached, maintaining the cache contract.
Can I use Distil with my existing coding agent?
Yes, it provides presets to wrap many popular agents like Claude, Aider, Gemini, and Copilot without code changes.