lap MCP server
Commit Nov 3, 2026Measures and improves the token efficiency of OpenAPI and MCP server tool definitions through scoring, linting, and automated patching.
Also known as lCrazyblindl/lap
- Stars
- 6
- Tools
- —
- Downloads, 30 days
- —
- Latest version
- —
- Last commit
- Nov 3, 2026
- License
- MIT
What is lap MCP?
lap measures and improves the token efficiency of OpenAPI and MCP server tool definitions through scoring, linting, and automated patching.
Summary generated from the project's README
About lap MCP server
Developers building APIs or MCP servers can use lap to measure and reduce the token costs associated with tool definitions. The tool analyzes OpenAPI specifications and live MCP servers to provide a 0–100 grade based on how many tokens an LLM agent must pay just to load the tool menu. It decomposes costs into menu, call, and result categories to show exactly where expenses occur.
Users interact with the toolkit via a command line interface, installing it through pip. The system uses deterministic pipelines with tiktoken or Anthropic's count_tokens to ensure reproducible results without requiring an LLM in the loop for core measurements. It identifies specific rule violations through linting and can generate an OpenAPI Overlay patch to automatically fix issues that bloat token usage.
This utility is designed for API providers wanting to optimize their documentation, agent builders auditing their own configurations with the lap stack command, and teams looking to integrate token-usage gates into their CI/CD pipelines. It provides a neutral way to verify if tool definitions are efficient or if they unnecessarily increase session costs.
What you can do with it
- 1Audit agent configuration costsA developer can run a command to see the total tokens consumed by all installed MCP servers before any messages are sent.
- 2Optimize OpenAPI specificationsYou can identify specific linting violations in your API definition and apply an automated patch to improve the grade.
- 3Display API quality badgesAPI maintainers can generate a shield.io badge to show their tool definition grade directly in their documentation.
Written from the project's README. Every use case is checked against a quote from it.
Automated checks
Ran Nov 15, 2026Full report →Not verified yet. A server needs a score of 80, no serious finding, and enough data for us to check. How checks work · What to fix for more points
- Source repository linked and reachablelcrazyblindl/lap is public and reachablegithub.com/lcrazyblindl/lap ↗Pass
- Package names this repository as its sourceNot checked
- Build provenance (Sigstore attestation)Not checked
- Open source license (OSI)MITspdx.org/licenses/MIT.html ↗Pass
- Commit or release in the last yearLast commit or release 2026-11-03github.com/lcrazyblindl/lap ↗Pass
- OpenSSF ScorecardNot checked
- Package name not close to a popular packageNot checked
- No critical advisory in the package or its dependenciesNot checked
- No hidden instructions to the model0 tool descriptions and the README read; no override phrases, credential paths or hidden charactersPass
- Destructive tools are labelledNot checked
- Remote endpoint answers the MCP handshakeNot checked
README
View on GitHublap — how many tokens does your API cost an LLM agent?
· MIT · Changelog · Live leaderboard
Every agent session starts by paying for tool definitions it mostly won't use — and pays again for every call and every response. lap measures that, for any OpenAPI spec, live MCP server, or your own agent config: the token cost decomposed (A menu / B call / C result), a 0–100 grade, the rule violations driving the cost, and an applicable patch for the fixable ones. Free, neutral, reproducible — a measuring stick, not a product.
Try it (60 seconds)
pip install "lap-score[mcp]"
lap stack # YOUR installed MCP servers: tokens paid before you type a word
lap score openapi.json # A/B/C decomposition + grade (also --mcp-url for live servers)
lap lint openapi.json # the violations driving the cost (also --mcp "cmd" for servers)
lap fix openapi.json --apply patched.json # fixable findings as an OpenAPI Overlay patch
lap badge openapi.json # shields.io grade badge for your README
$ lap stack
server kind tools menu tokens compact
time stdio 2 283 31
git stdio 12 1418 153
TOTAL 14 1701 184
Your agent pays ~1,701 tokens of tool menus at session start - before you type a word.
$ lap fix api/openapi.json --apply patched.json
[written] lap-overlay.yaml (6 action(s))
[written] patched.json (lint findings: 15 -> 3) # grade: B (72) -> A (91)
Everything is --json-able and CI-gateable (--max-menu-tokens, --diff --max-growth,
--fail-on; a composite GitHub Action and a Spectral ruleset
ship in-repo). Full CLI docs: lap/README.md.
How the measurement works
No LLM in the loop for the core numbers — a deterministic pipeline anyone can rerun:
- List, don't execute. For an MCP server, connect over stdio/HTTP and call the standard
tools/list— the exact definitions (name, description, inputSchema) it advertises. Credential-free: tool listing only, dummy env vars where a server won't boot without them. (For OpenAPI, parse the spec instead.) - Count the menu (bucket A). Serialize those definitions the way they enter a model's
context and tokenize. Offline that's tiktoken — a fixed BPE vocabulary, an algorithm not a
model, so the same input always gives the same number; set
ANTHROPIC_API_KEYfor Anthropic's owncount_tokens. This is what a session pays before the first user message. - Compact what-if. Re-render the same tools as compact signatures and count that — how much is recoverable without dropping a single tool.
- Lint the hygiene. Rules flag what burns tokens or accuracy: missing/short descriptions, undescribed params, >600-token single definitions, a whole-menu-too-heavy check.
- Fold into a grade. menu-tokens-per-tool + hygiene findings, log-scaled → 0–100 + a letter.
Static menu costs (bucket A) are exact and reproducible; whether a deferred/tool-search
server actually works and what its per-call costs are is behavioral. For that half we go
beyond the static pipeline and run live A/Bs with real LLMs — actual billed calls, measuring
accuracy and tokens — kept clearly separate: Anthropic's Tool Search verified
at ~90%, its code-execution the one case that didn't hold up, a real
server's facade that held up 8.1× at equal accuracy. Run the static check
yourself: lap lint --mcp "npx -y your-server".
What the measurements show
The leaderboard — 50 real public APIs, refreshed monthly. Their naive agent menus total ~11.2M tokens; rendering the same operations compactly recovers ~82% on average (lazy tool-search: ~86%). Nobody ships the compact form.
| API | naive menu (bucket A) | LAP compact | saved |
|---|---|---|---|
| Xero Accounting | 4,041,667 | 7,800 | +99% |
| Kubernetes | 2,864,414 | 45,015 | +98% |
| Amazon EC2 | 1,046,048 | 86,031 | +92% |
| …47 more, sortable, with history |
Real tools, not just our own variants — the same accounting pointed at the ecosystem, live, with billed calls where it matters:
- 3 real OpenAPI→MCP generators all emit menus heavier than the naive baseline, 5–28× heavier than compact (GENERATORS, MCP-SERVERS).
- 20 popular published MCP servers, scored as installed — menus from 42 to 21,411 tokens per session (Notion's official server, grade F); ~64k tokens if you connect them all, before the first user message (MCP-LEADERBOARD, refreshed monthly, incl. a grade cross-check against another grader).
- Anthropic's Tool Search: verified live — ~90% billed-token cut on a real 290-op API, server-enforced (TOOL-SEARCH). Their code-execution: disputed on our workload — heavier than naive in 5/5 repeats; its saving is behavioral, not structural (CODE-EXEC).
- A third-party optimizer's self-reported % was mismeasuring — root-caused in its own source: character counts with an asymmetric formula (MCP-COMPRESSOR).
- Compression doesn't cost accuracy — a 500-run live matrix (2 models × 10 tasks × 5 forms): every compressed form matched or beat the naive menu; the cheapest correct answer turned out to be model-dependent (validation.md).
- What the replies cost, measured — a price list of 29 real read-only calls across 13 servers (1 token to 18,439 per reply), the data nobody had published. Structurally: 13 of 29 replies are free text a caller can't filter or paginate, several heavy tools accept no limit parameter at all, and MCP itself has no caller-side field/page selection — the response-side gap every menu optimization leaves untouched (RESPONSE-COSTS).
Results that don't flatter the thesis ship as prominently as the ones that do — the verified/disputed registry of the field's headline claims is docs/FIELD.md, and the standard objections are priced out in CACHE-ECONOMICS ("isn't it cached?") and TOKENIZERS ("whose tokens?").
Who it's for
- You ship an API or MCP server →
lap score/lint/fixgive you a number, the violations behind it, and a patch; the Action gates PRs that bloat the menu;lap badgeshows the grade in your README. - You build agents →
lap stackaudits what your own config burns; the leaderboard is due-diligence before wiring up an API. - You're choosing between MCP / Tool Search / code-execution / a query DSL → this repo measured all of them on the same tasks with the same accounting (token-bench, LANDSCAPE).
- You design API conventions → the LAP profile is the rule set behind the linter; every rule cites its measurement, including the two that earned honest caveats.
Project map
lap/— the pip-installable toolkit (start here).profile/— the LAP profile: measured conventions, L1–L4 levels, the grade formula.experiments/— the benchmark (pet-zoo testbed) + every measurement script behind the docs (leaderboard, spec-#2808 simulation, cache economics, tokenizer matrix, …).docs/— the receipts: leaderboard (+ its MCP-server twin), real-tool tests, FIELD claims registry, SPEC-2808 input for the MCP spec discussion.spectral/— the lint rules for existing Spectral setups.ROADMAP.md— the full staged history and what's next.
Status & contributing
0.8.x, pre-1.0, actively maintained, MIT — no telemetry, no paid tier, no company. Issues and PRs welcome: CONTRIBUTING.md covers dev setup and the house policies (vendor neutrality, claims need receipts), and there's a "Score my API" issue template — disputes of our numbers are explicitly invited.
lap MCP questions
Does lap require an LLM to calculate token counts?
No, it uses a deterministic pipeline with tokenizers like tiktoken to ensure exact and reproducible results without an LLM in the loop.
How can I see the token cost of my currently installed MCP servers?
You can use the 'lap stack' command to audit the tokens paid for tool menus before you even start typing.