toolprint-watch MCP server

toolprint-watch

A fixed set of 36 public MCP servers, checked daily against an approved baseline, to answer a question the product depends on: how often do MCP tool definitions actually change without anyone reviewing them?

Collected with toolprint. Split out of that repository so a year of daily observation commits does not bury the tool's own history.

The git history of baseline.json is the dataset. Every commit that touches it is a definition change, with a timestamp and a diff:

git log -p --follow baseline.json

observations.csv carries the denominator — how many server-days produced no drift — because a drift rate needs both halves and "we saw three rug pulls" is meaningless without "out of how many observations".

How servers were selected

Every entry was verified against the npm or PyPI registry and then probed: it is here only because it actually starts and serves tools/list with no credentials. Nothing was included on the strength of a package name appearing in a directory.

Deliberately not padded. A larger list built from npm keyword search fills up with abandoned single-author packages that will never change, which inflates the denominator and teaches nothing. 36 servers people actually run beats 60 that include 30 dead ones.

Three groups, on purpose:

Group	Why
Actively published reference servers	filesystem, memory, everything, sequential-thinking, git, time, fetch. Released frequently, so they set the baseline rate for routine churn.
Archived, frozen reference servers	puppeteer, github, postgres. Last published 2024–2025. Controls: if these drift, something is wrong.
Vendor and community servers	Where a real rug pull would most plausibly appear.

The finding that shaped this list

Of 18 well-known public remote MCP endpoints probed, two served tools/list without credentials — DeepWiki and Context7. The other sixteen returned 401 or 403: Linear, Notion, Stripe, Vercel, Sentry, GitHub, Cloudflare, Hugging Face, PayPal, Grafana, Neon, Prisma, Semgrep, Webflow, Sanity, GitMCP.

That matters more than it first looks. Remote servers are the category where a definition change is completely invisible to the people using them: no package update, no lockfile line, no local signal at all. And they are also the category nobody outside the vendor can audit, because tools/list is behind OAuth. The people best placed to detect a remote rug pull are the customers who already hold credentials — which is an argument for toolprint check running in their CI, not for a public observatory.

So the watchlist is stdio-weighted by necessity, not by preference. That is a limitation of the dataset and should be stated whenever it is cited.

Protocol versions observed

Of the servers that responded, one spoke 2026-07-28, most spoke 2025-11-25, and two still spoke 2024-11-05. The ecosystem is spread across three protocol eras, twenty months apart.

Running it by hand

toolprint baseline --connect --yes --config servers.json \
                   --baseline baseline.json
toolprint check    --connect --yes --config servers.json \
                   --baseline baseline.json

No credentials, by decision

Nothing in this repository is a secret, and nothing here needs one. That is a deliberate position, not an accident of what was easy, and it is enforced in CI: every run first points toolprint at its own watchlist and fails the job if servers.json contains a literal credential rather than a ${VAR} reference.

The reasoning is below. The short version: watching a server means running it, and running it with a real token means handing that token to a package you are watching because you suspect packages can change. A placeholder gets the same data with none of that.

Servers that ask for an API key

Most of them do not actually need one. MCP servers check credentials at tools/call, not at tools/list — the gate is whether the environment variable is set, not whether it is valid. Ten servers that refused to start without a key were tested with the literal string not-a-real-key-0000000000, and all ten served their full tool definitions.

So nine of them are on this list with ${WATCH_PLACEHOLDER}, a value the workflow sets in plain sight. There is no credential here to protect, and nothing in this repository is a secret.

Two limits on that:

• Remote HTTP endpoints are different. Linear, Notion, Stripe, Vercel, Sentry and Hugging Face all reject a placeholder bearer token with 401 or 403, because their auth happens at the transport layer rather than inside the server. Watching those needs a real token.
• A placeholder may not see everything. A server could return a different tool set to an authenticated caller — tools gated by plan or scope. What is recorded here is the unauthenticated view, which may be a subset. Treat a no-drift result as "the public surface did not change".

Adding servers as it runs

Yes — the list is meant to grow. Edit servers.json and commit; the next daily run picks it up.

# add an entry to servers.json, then either wait for the schedule or:
toolprint check   --yes --config servers.json --baseline baseline.json
toolprint approve --yes --config servers.json --baseline baseline.json --by "$USER"

A newly watched server produces no drift, which is correct — nothing moved. check reports it separately as newly watched, not yet in the baseline, and approve adopts it, stamping first_observed with that date. The scheduled workflow runs approve every day for exactly this reason, so additions are adopted without anyone intervening.

Observation windows are what keep the arithmetic honest. Servers joined at different times, so they have different windows. analyse.py --by-server divides each server's changes by its own; a flat server-day total would credit a server added last week with the whole run's quiet time, which is an error in the direction that flatters the result. Windows are counted in observation days rather than calendar days, for the same reason — the collector has already missed a day, and charging a server for a day nobody looked at inflates its window.

Those windows are read from the git history, not from the first_observed field. approve stamps that field on adoption, but approve --refresh rewrites every server from the live snapshot and does not carry it across, so it was silently dropped at the 0.1.0 → 0.2.2 refresh on 2026-09-04 and --by-server printed ? for every server until analyse.py stopped depending on it. The history cannot be wiped by a refresh.

Adoption is not drift. A server joining brings its whole tool surface with it, and counting that as change measures this repository's growth rather than the ecosystem's: adding nine servers on 2026-09-01 accounted for 89 of the 235 changes the earlier counting reported. analyse.py compares servers only across revisions in which they were already being watched, and reports what they arrived with — 501 tools across 36 servers so far — separately.

Removing a server is the same idea in reverse: drop it from servers.json and it stops being observed, but its baseline entry and its history stay. Deleting those would erase the record of what it looked like. check lists servers it expected and did not see, which also catches a server that has simply broken.

Good candidates are servers that (a) exist on npm or PyPI, and (b) actually serve tools/list with no credentials. Check the second before adding — roughly a third of the ones tried failed on a missing API key:

toolprint scan --connect --config /tmp/candidate.json

Caveats

• Runs on a GitHub-hosted runner, so npx -y pkg resolves the latest published version each day. Package churn and definition churn are therefore the same signal here; distinguishing them needs a pinned second run.
• Only tool definitions are observed. A server whose runtime behaviour changes without its definitions changing — the postmark-mcp case — is invisible to this, and to the tool generally.
• No credentials are used, so servers that gate tools/list are absent.

Layout

Path	What it is
servers.json	The watchlist, in MCP client config format
server-tiers.json	The watchlist's three tiers (reference / frozen-control / vendor-community), by server. See METHODOLOGY.md
charts/	Charts generated from the data (daily changes; rule distribution), with a reproducible generator
METHODOLOGY.md	How the monitor works and why the frozen tier is a control group
baseline.json	The approved state. Its git history is the dataset.
observations.csv	One row per date: servers watched, how many were actually reachable, tools seen, changes found. A re-run supersedes that date's row rather than adding one, and keeps the highest change count seen
observations/	Per-date drift detail: which rules fired, on which tools
findings/	Per-date full assessment — auth posture, protocol era, versions, token cost, effect classes, every finding with evidence, and fetch_status/fetch_detail for servers that failed
surfaces/	Per-date tool definitions — names, descriptions, schemas, annotations. Untrusted model-facing text; see surfaces/README.md
analyse.py	Reads the history and reports a drift rate, with adoption excluded

A drift report records that a description changed and never what it said — hashes are one way. surfaces/ is how the text is kept, so the corpus can show a rug pull rather than only prove one happened. findings/ keeps everything else the collector measures; both used to be computed daily and discarded, and a day not captured cannot be recovered afterwards. | .github/workflows/watch.yml | Daily collector |

Licence

Apache-2.0 for the code. The observations are factual records about publicly published packages and endpoints; use them freely, and cite the limitations above if you do.