Nerf Watch MCP server
Commit Oct 21, 2026Provides read-only access to daily model performance scores and historical trends from a private, code-graded evaluation suite.
Also known as stdcmms-lang/tokenusage-codex-mcp, nerfwatch
- Stars
- 0
- Tools
- 2
- Downloads, 30 days
- —
- Latest version
- 1.0.0
- Last commit
- Oct 21, 2026
- License
- MIT
What is Nerf Watch MCP?
Nerf Watch provides read-only access to daily model performance scores and historical trends from a private, code-graded evaluation suite. It exposes 2 tools, such as get_nerfwatch_scores, get_nerfwatch_history.
Summary generated from the project's README
About Nerf Watch MCP server
Nerf Watch provides read-only access to public performance scores and historical trends from a private, code-graded evaluation suite. Developers can use this hosted MCP server to monitor model performance data without needing an API key, account, or sign-in process. It connects via a streamable HTTP endpoint, making it compatible with Claude Code, Codex, and other clients that support the mcpServers HTTP format.
The server offers two specific tools for data retrieval. The get_nerfwatch_scores tool provides the latest public scores, previous runs, changes, baselines, and trends including model ids. The get_nerfwatch_history tool allows users to pull the retained history for a specific model id, ordered from oldest to newest. Both tools share a single 300-second read cache per server instance to manage data access efficiently.
What you can do with it
- 1Monitor latest model performanceAsk the AI to retrieve the most recent public scores and baseline trends for various models.
- 2Analyze historical model trendsRequest the full historical data for a specific model to see how its performance has changed over time.
- 3Identify model performance changesUse the server to find recent changes in model scores within a specific suite version.
Written from the project's README. Every use case is checked against a quote from it.
How to install Nerf Watch MCP
Formats verified 10 Nov 2026Add to CursorAdd to GooseInstall Nerf Watch MCP in Claude Code
$ claude mcp add --transport http nerfwatch https://tokenusage.dev/mcp/nerfwatchInstall Nerf Watch MCP in Cursor
~/.cursor/mcp.json
{
"mcpServers": {
"nerfwatch": {
"url": "https://tokenusage.dev/mcp/nerfwatch"
}
}
}Install Nerf Watch MCP in VS Code
.vscode/mcp.json
{
"servers": {
"nerfwatch": {
"type": "http",
"url": "https://tokenusage.dev/mcp/nerfwatch"
}
}
}Install Nerf Watch MCP in VS Code CLI
$ code --add-mcp '{"name":"nerfwatch","type":"http","url":"https://tokenusage.dev/mcp/nerfwatch"}'Install Nerf Watch MCP in Codex
$ codex mcp add nerfwatch --url https://tokenusage.dev/mcp/nerfwatchInstall Nerf Watch MCP in Codex config.toml
~/.codex/config.toml
[mcp_servers.nerfwatch] url = "https://tokenusage.dev/mcp/nerfwatch"
Install Nerf Watch MCP in Gemini CLI
$ gemini mcp add -s user --transport http nerfwatch https://tokenusage.dev/mcp/nerfwatchInstall Nerf Watch MCP in Gemini settings.json
~/.gemini/settings.json
{
"mcpServers": {
"nerfwatch": {
"url": "https://tokenusage.dev/mcp/nerfwatch",
"type": "http"
}
}
}Nerf Watch MCP tools 2
Detected from the READMEget_nerfwatch_scoresget_nerfwatch_history
| get_nerfwatch_scores | — | |
| get_nerfwatch_history | — |
Automated checks
Ran Nov 14, 2026Full report →Not verified yet. A server needs a score of 80, no serious finding, and enough data for us to check. How checks work · What to fix for more points
- Source repository linked and reachablestdcmms-lang/tokenusage-codex-mcp is public and reachablegithub.com/stdcmms-lang/tokenusage-codex-mcp ↗Pass
- Package names this repository as its sourceNot checked
- Build provenance (Sigstore attestation)Not checked
- Open source license (OSI)MITspdx.org/licenses/MIT.html ↗Pass
- Commit or release in the last yearLast commit or release 2026-10-21github.com/stdcmms-lang/tokenusage-codex-mcp ↗Pass
- OpenSSF ScorecardNot checked
- Package name not close to a popular packageNot checked
- No critical advisory in the package or its dependenciesNot checked
- No hidden instructions to the model2 tool descriptions and the README read; no override phrases, credential paths or hidden charactersPass
- Destructive tools are labelledNo tool named like delete, drop or execPass
- Remote endpoint answers the MCP handshakeinitialize answered with protocol 2025-06-18 in 230 msPass
README
View on GitHubNerf Watch MCP server
A hosted, read-only MCP server for the public scores and history behind Nerf Watch.
A lower score is a lower score on this suite, not proof that a provider changed a model.
Runs are noisy, and a provider's defaults, the endpoint or the runner itself can change.
Data: tokenusage.dev linked to https://tokenusage.dev/nerfwatch.
- Endpoint:
https://tokenusage.dev/mcp/nerfwatch(Streamable HTTP) - No API key, account or sign-in
- Read-only public score data
- Both tools share one 300-second read cache per server instance
This directory contains client documentation and Registry metadata. The hosted implementation source is not in this repository.
Tools
| Tool | Arguments | Result |
|---|---|---|
get_nerfwatch_scores |
none | Latest public scores, previous runs, changes, baselines and trends; model ids are included. |
get_nerfwatch_history |
required model id from get_nerfwatch_scores |
That model's retained history, oldest first, with each point's suite version. |
Missing days are missing, not zero. Scores and changes are rounded to two decimals. Comparisons are within one suite version. Retention and baseline settings come from the responses. Evidence answers, grader check lines and test questions are not returned.
Install
Claude Code:
claude mcp add --transport http nerfwatch https://tokenusage.dev/mcp/nerfwatch
Codex:
codex mcp add nerfwatch --url https://tokenusage.dev/mcp/nerfwatch
For Claude Code project scope, add this to .mcp.json. Other clients that accept
this HTTP mcpServers format can use the same entry:
{
"mcpServers": {
"nerfwatch": {
"type": "http",
"url": "https://tokenusage.dev/mcp/nerfwatch"
}
}
}
Discovery
- Registry name:
dev.tokenusage/nerfwatch - Manifest: server.json
- Server card: https://tokenusage.dev/mcp/nerfwatch/server-card
- AI catalog: https://tokenusage.dev/.well-known/ai-catalog.json
- Fields, caching and access: https://tokenusage.dev/developers
The separate Codex reset server remains at https://tokenusage.dev/mcp.
Credit and terms
When showing this data to other people, show Data: tokenusage.dev linked to
https://tokenusage.dev/nerfwatch on the same screen or in the same message.
Personal use needs no visible credit. Relays retain the source URL and pass
on this credit condition.
Please poll no faster than every five minutes and retain the score qualification above.
Terms: https://tokenusage.dev/terms
Privacy: https://tokenusage.dev/privacy
Questions and corrections: https://tokenusage.dev/contact
Unofficial; not affiliated with OpenAI or Anthropic.
Nerf Watch MCP questions
How do I add Nerf Watch MCP to Claude Code?
Run claude mcp add --transport http nerfwatch https://tokenusage.dev/mcp/nerfwatch in your terminal. Add --scope user to make it available in every project.
How do I add Nerf Watch MCP to Cursor?
Open ~/.cursor/mcp.json and add the server under mcpServers, using the Cursor config shown in the install section above.
What tools does Nerf Watch MCP provide?
It exposes 2 tools, including get_nerfwatch_scores, get_nerfwatch_history.
What data does the server provide?
The server provides the latest public scores, previous runs, changes, baselines, trends, and model history.
Do I need an API key to use this server?
No, the server does not require an API key, account, or sign-in.
How often should I poll the server?
You should poll the server no more frequently than every five minutes.
Does a lower score mean a model has changed?
No, a lower score is simply a lower score on this specific suite and is not definitive proof of a model change.