mcprobe MCP server
Commit Nov 8, 2026Audits MCP servers by linting tool schemas and fuzzing them with malformed inputs to generate a conformance score.
Also known as alitiknazoglu/mcprobe
- Stars
- 6
- Tools
- 10
- Downloads, 30 days
- —
- Latest version
- —
- Last commit
- Nov 8, 2026
- License
- MIT
What is mcprobe MCP?
mcprobe audits MCP servers by linting tool schemas and fuzzing them with malformed inputs to generate a conformance score. It exposes 10 tools, such as probe_connect, probe_lint, probe_fuzz.
Summary generated from the project's README
About mcprobe MCP server
Auditing MCP servers for schema quality and behavioral reliability is what mcprobe enables for developers building Model Context Protocol integrations. It functions as a stdio MCP server that connects to any target via stdio or HTTP to perform both static linting and active behavioral fuzzing. By analyzing tool schemas against twelve specific rules and then deliberately sending malformed inputs like wrong types or missing required fields, it determines how well a server handles errors.
Users can run the engine locally via the command line or through an MCP client like Claude Desktop or Cursor. The tool generates a conformance score from 0 to 100, providing a Markdown report with a breakdown across four dimensions. For those preferring a managed experience, a hosted version provides daily audits and historical tracking. Developers can also integrate these checks into CI pipelines using a GitHub Action to gate deployments based on a minimum conformance grade.
What you can do with it
- 1Verify tool schema qualityPerform a static audit to check for missing descriptions, untyped parameters, or non-snake_case names in your tool definitions.
- 2Test error handling behaviorDetermine if your server gracefully rejects bad inputs or if it silently accepts garbage and returns incorrect values.
- 3Automate conformance gating in CIFail your build automatically if a new code change causes the MCP server conformance grade to drop below a specific threshold.
- 4Audit remote HTTP serversConnect to a live MCP server hosted at a URL to receive a full graded report without any local installation.
Written from the project's README. Every use case is checked against a quote from it.
mcprobe MCP tools 10
Detected from the READMEprobe_connectprobe_lintprobe_fuzzprobe_reportprobe_listprobe_disconnectmissing_requiredwrong_typeout_of_enumextra_garbage
Configuration
MCPROBE_ALLOW_STDIOMCPROBE_PASS_ENVMY_SERVER_API_KEY
Automated checks
Ran Nov 14, 2026Full report →Not verified yet. A server needs a score of 80, no serious finding, and enough data for us to check. How checks work · What to fix for more points
- Source repository linked and reachablealitiknazoglu/mcprobe is public and reachablegithub.com/alitiknazoglu/mcprobe ↗Pass
- Package names this repository as its sourceNot checked
- Build provenance (Sigstore attestation)Not checked
- Open source license (OSI)MITspdx.org/licenses/MIT.html ↗Pass
- Commit or release in the last yearLast commit or release 2026-11-08github.com/alitiknazoglu/mcprobe ↗Pass
- OpenSSF ScorecardNot checked
- Package name not close to a popular packageNot checked
- No critical advisory in the package or its dependenciesNot checked
- No hidden instructions to the model10 tool descriptions and the README read; no override phrases, credential paths or hidden charactersPass
- Destructive tools are labelledNo tool named like delete, drop or execPass
- Remote endpoint answers the MCP handshakeNot checked
README
View on GitHubMCProbe
A stdio MCP server that audits other MCP servers over the live protocol. It connects to any MCP target (stdio or HTTP), lints every tool's schema for agent-usability, then actually calls the tools with deliberately broken inputs to see how the server handles them, and returns a 0–100 conformance score with a per-dimension breakdown rendered as Markdown.
The behavioral pass is the part that matters. Static schema audits tell you
that a tool exists and looks reasonable. MCProbe then picks up a phone and
dials each tool with missing_required, wrong_type, out_of_enum, and
extra_garbage inputs — the same mistakes a language model will make on a
bad day — and classifies the response. A server that returns a clean
isError: true rejected the input correctly. A server that says "OK" to
garbage (silently accepted it) or crashes the JSON-RPC transport both
failed to reject it — and the Error Handling score is the fraction of
bad inputs the server rejected cleanly.
Problem statement
The Model Context Protocol is new. Servers proliferate. Most ship with
tool schemas that an agent can call, but few ship with tool schemas that
an agent can call correctly: parameters are untyped, descriptions are
missing, names are not snake_case, and a quick look at the code reveals
that the handler is doing Number(x) / Number(y) with no guard at all.
The convention in the wider ecosystem is to ship a static schema audit
that flags the obvious smells and then declare the server ready. The
smells are real, but a static audit cannot tell you whether the server
behaves: it cannot tell you that divide("x", "y") silently returns
NaN, or that an extra unknown key is just stripped and ignored.
MCProbe does both, on a single connection:
- Static lint. Twelve rules over every tool's schema: missing or thin descriptions, duplicate or unusual names, an empty or non-object schema, untyped or undocumented parameters, and a server-wide rule for "I said I had tools but I have none."
- Behavioral fuzz. For each tool, the generator produces one valid
case and at least three malformed variants, calls the target over
the live JSON-RPC transport, and classifies the outcome as
ok(the tool shrugged),toolError(graceful rejection), orprotocolCrash(worst case). A malformed case that comes back withoutisError: trueis flagged assilentlyAccepted— exactly the failure mode the linter cannot see. - Scoring. The findings and the fuzz results are combined into a 0–100 score on four dimensions, mapped to an A–F grade, and rendered as a Markdown report the host (or a human) can read.
Hosted version — mcprobe.org
Don't want to install anything? mcprobe.org is the hosted version of this engine — paste an MCP server's URL in your browser and get the same graded report, no Node or setup required.
- Free — 2 audits/day with a soft report (score, grade, dimension scores, finding counts).
- Pro ($9.90 once, lifetime) — the full report (per-dimension reasons, every finding, the fuzz table, recommended fixes), 30 audits/day, saved history, the public gallery, Markdown export, and shareable links.
- Local (stdio) servers — a Pro feature: run
mcprobe push --stdio "…" --token <key>to audit a server on your machine and send the report to your account (see the CLI).
This engine stays MIT-licensed and free — the hosted app only adds accounts, persistence, the gallery, and those conveniences. Run it yourself for nothing, or pay once for the hosted experience.
Install
npm install
npm run build # tsc -p tsconfig.json && tsc -p examples/demo-target/tsconfig.json
The build emits:
dist/index.js— the probe (run this as a stdio MCP server).examples/demo-target/dist/index.js— a deliberately flawed MCP server used by the tests and the demo.
To launch the probe as a stdio MCP server so any host can talk to it:
npm start
No port, no daemon, no config file. The probe speaks JSON-RPC on stdin/stdout and writes operator logs to stderr.
Quickstart — audit any MCP server
Two ways to point MCProbe at a target. You only ever register MCProbe; it dials the target itself, so the target needs no setup.
Option 1 — from an MCP client (Claude Desktop, Cursor, any host)
Add MCProbe to your client's MCP config (use the absolute path to the
built dist/index.js):
{
"mcpServers": {
"mcprobe": {
"command": "node",
"args": ["/absolute/path/to/mcprobe/dist/index.js"]
}
}
}
To let the host audit local stdio servers too, allow the exact command lines in the probe's own env (see Security for why this is opt-in):
"mcprobe": {
"command": "node",
"args": ["/absolute/path/to/mcprobe/dist/index.js"],
"env": {
"MCPROBE_ALLOW_STDIO": "node /absolute/path/to/my-server/dist/index.js",
"MCPROBE_PASS_ENV": "MY_SERVER_API_KEY",
"MY_SERVER_API_KEY": "..."
}
}
Then ask in plain English:
Use mcprobe to audit
https://docs.base.org/mcpover http — connect, then run a full report with fuzz and show me the score.
The host calls probe_connect then probe_report for you. MCProbe also
advertises server instructions, so the model is told the flow on
connect — no need to memorise the tool names.
Option 2 — the mcprobe CLI (no host)
Get the project and build it, then audit any server straight from the terminal:
git clone https://github.com/alitiknazoglu/mcprobe
cd mcprobe && npm install && npm run build
# audit an HTTP server
node dist/index.js audit https://docs.base.org/mcp --fuzz
# audit a LOCAL stdio server (no URL — the `npx some-server` style)
node dist/index.js audit --stdio "npx @acme/my-mcp-server" --fuzz
# audit a server that requires credentials
node dist/index.js audit https://api.acme.com/mcp --bearer sk_live_xxx --fuzz
node dist/index.js audit https://api.acme.com/mcp --header "X-API-Key: xxx"
--bearer/--header authenticate you to the server being audited, and are
sent on every request to it — so only point them at a server you trust. Set
MCPROBE_TARGET_TOKEN instead of --bearer to keep the key out of your shell
history. (Don't confuse these with push --token, which authenticates you to
MCProbe; passing that as --bearer would hand your MCProbe key to a third
party.) A stdio server takes its credentials through its own environment
instead — and it does not inherit your shell's environment (so
MCPROBE_TOKEN and CI secrets never reach it). Forward what it needs with
--pass-env NAME (repeatable) or set it with --env NAME=VALUE:
node dist/index.js audit --stdio "node ./server.js" --pass-env MY_API_KEY --fuzz
(After npm install -g . or npm link, the command is just mcprobe audit ….)
It prints the full Markdown report to stdout — add --json for a
machine-readable report (what the GitHub Action
and other tooling consume). --fuzz also calls each tool for real — once
with a plausible valid input and several times with malformed input — to score
Error Handling & Liveness. By default it only touches tools the target
annotates readOnlyHint: true or destructiveHint: false; the MCP spec
presumes every other tool destructive. --fuzz-unannotated adds tools with no
safety annotations, and --fuzz-destructive adds everything. Only use those
two against a server you own, on test data — a send_payment or delete_post
tool will run. Omit --fuzz for a read-only static audit (metadata + schema
quality only).
Save an audit to your account. push runs the same audit and uploads the
report to an ingest endpoint (default https://mcprobe.org/api/ingest) with a
bearer token:
node dist/index.js push --stdio "npx @acme/my-mcp-server" --fuzz --token mcp_xxx
The token comes from your mcprobe.org profile; --to <url> (or MCPROBE_API)
points it at a different endpoint, which must be https:// (plain http:// is
accepted only for localhost). The upload describes the target without its
secrets: values after flags like --api-key/--token, TOKEN=... pairs,
key-shaped strings and URL credentials are replaced with ***. Run
mcprobe help for all flags.
--stdio is split like a shell command line, so quote arguments that contain
spaces: --stdio "node '/path with spaces/server.js'".
Audit in CI (GitHub Action)
Gate your MCP server on every push — audit it and fail the build if its conformance grade drops. Free and self-contained (it runs the open-source engine on your own runner; no account required):
# .github/workflows/mcprobe.yml
name: MCP audit
on: [push, pull_request]
jobs:
audit:
runs-on: ubuntu-latest
steps:
- uses: alitiknazoglu/mcprobe@v1
with:
url: https://your-server.example.com/mcp
fuzz: true # behavioral testing — really calls read-only/non-destructive tools (default false)
# fuzz-unannotated: true # also call tools with no safety annotations
min-score: "75" # fail the job below this (A≥90 B≥75 C≥60 D≥40); omit to report only
The step prints the score to the job summary and exposes score / grade
outputs. It also writes a full mcprobe-report.json you can upload as an
artifact. Leave off min-score to report without ever failing the build.
Upload runs to your dashboard (Pro): add one line — a token: (your
mcprobe.org Pro key, stored as a GitHub secret). The audit stays the same; the
run is also uploaded to your history/dashboard on mcprobe.org.
- uses: alitiknazoglu/mcprobe@v1
with:
url: https://your-server.example.com/mcp
min-score: "75"
token: ${{ secrets.MCPROBE_TOKEN }} # ← only new line; uploads to your dashboard
The audit itself is always free and local; the hosted tracking (history, gallery, badge) is the Pro tier — see Hosted version.
Agent skill
This repo ships an agent skill so your
coding agent knows how to drive MCProbe on its own — just say "audit this MCP
server" and it runs the right probe_* tools or mcprobe CLI command and
explains the score. To install it, copy the folder into your agent's skills
directory:
# Claude Code (project- or user-level)
cp -r .agents/skills/mcp-audit /path/to/your/project/.claude/skills/
# Other agents that use the open skills format (Codex, Opencode, Cursor, …)
cp -r .agents/skills/mcp-audit /path/to/your/project/.agents/skills/
It's a single SKILL.md — the same file works in either location.
Security
MCProbe connects to servers you point it at and, with --fuzz, calls their
tools. Treat both directions as untrusted.
probe_connect is driven by an agent, so it is gated. Whoever drives the
MCP server — usually an LLM, possibly acting on a prompt injection in a page or
a tool result — chooses the command, its arguments, its environment and the
URL. Ungated, that is arbitrary code execution and an SSRF primitive. So:
| Env (set in the probe's MCP config) | Effect |
|---|---|
MCPROBE_ALLOW_STDIO unset |
probe_connect refuses transport: "stdio". |
MCPROBE_ALLOW_STDIO="node /abs/server.js; npx -y @acme/srv" |
Only these exact command lines (command + args, ;-separated). The agent may not set code-loading env vars (PATH, NODE_OPTIONS, LD_PRELOAD, DYLD_*, PYTHONPATH, ...). |
MCPROBE_ALLOW_STDIO="*" |
Any command, any env. Only for a probe no untrusted text can reach. |
MCPROBE_PASS_ENV="A,B" |
Forward these variables from the probe's env to spawned targets (so the agent never handles the secret). |
MCPROBE_ALLOW_PRIVATE_URLS=1 |
Allow http targets on loopback / private networks (local dev servers). Off by default. Link-local and cloud-metadata addresses (169.254.0.0/16, fe80::/10, metadata.google.internal) are always refused. Hostnames are resolved and every address checked; each request is re-checked and redirects are refused. |
Allowlisting an interpreter with an open-ended argument (npx -y <anything>)
is only as safe as that exact line — the allowlist is matched on the full
command line, so prefer an absolute path to the server's entry point.
The CLI and the mcprobe/audit library are not gated: there, the person
running the command is the operator.
Spawned targets get a minimal environment. A stdio target sees only PATH,
HOME, USER, SHELL, TERM (the MCP SDK's default set) plus what you pass
explicitly (--env, --pass-env, env/passEnv in the library,
MCPROBE_PASS_ENV for the server). Your MCPROBE_TOKEN, MCPROBE_TARGET_TOKEN
and CI secrets never reach it.
The six probe_* tools
MCProbe registers four core tools and two optional helpers. The core four cover the full lint → fuzz → score pipeline; the two helpers cover the everyday ergonomics of managing connections.
| Tool | Purpose | Returns |
|---|---|---|
probe_connect |
Open a connection to a target. | { connectionId, name, version, capabilities, counts, defaultConnectionId } |
probe_lint |
Run the 12 lint rules over the target's cached tool summaries. | { connectionId, server, findings, summary } |
probe_fuzz |
Generate valid + malformed inputs per tool, call each, classify the outcome. Only fuzzes tools annotated read-only / non-destructive by default. | { connectionId, server, results, coverage, summary } |
probe_report |
Run lint (and fuzz when requested), score, render Markdown. | { connectionId, server, overall, grade, dimensions, coverage, findings, fuzz, markdown } |
probe_list |
(optional) Enumerate the target's tools. | { connectionId, server, tools } |
probe_disconnect |
(optional) Close one connection (by id) or every connection. | { removed, remaining, defaultConnectionId } |
All tools default to the most recently opened connection when
connectionId is omitted, so a single-target audit is a three-call
sequence: probe_connect → probe_report → probe_disconnect.
Every tool also declares MCP annotations so a host can reason about
side effects before calling: probe_lint and probe_list are
readOnlyHint: true, while probe_fuzz is destructiveHint: true
(it invokes the target's tools), and the tools that reach a target
(probe_connect, probe_fuzz, probe_report) set openWorldHint: true.
MCProbe audits other servers for agent-usability, so it declares these
hints on its own tools too.
probe_connect
Two transports: stdio (spawns a child process) and http (speaks
the streamable HTTP transport, with SSE fallback). For stdio,
command is required; for http, url is required. The target's
initialize handshake is run synchronously, the server's identity
and capabilities are cached, and a stable connectionId is returned.
Both transports are gated by the operator — see Security:
stdio needs MCPROBE_ALLOW_STDIO, and loopback/private URLs need
MCPROBE_ALLOW_PRIVATE_URLS=1.
probe_lint
A pure pass over the connection's cached tool summaries — no extra
round-trip. Each finding carries a stable code, a severity
(error, warning, info), a human-readable message, a
location ({ tool, param? }), and a hint with a concrete fix.
The twelve rules are:
| Code | Severity | What it catches |
|---|---|---|
tool.missing_description |
error | A tool with no description at all. |
tool.thin_description |
warning | A description under 12 characters. |
tool.duplicate_name |
error | Two tools registered with the same name. |
tool.unusual_name |
warning | A name that is not snake_case or kebab-case. |
tool.no_input_schema |
warning | An empty or missing inputSchema. |
tool.no_annotations |
info | A tool that declares no MCP annotations (readOnlyHint, destructiveHint, etc.). |
schema.invalid |
error | A schema that fails to compile (Ajv). |
schema.root_not_object |
warning | A root type that is not object. |
schema.no_required |
info | Properties declared but no required array. |
param.untyped |
warning | A property with no type/enum/const/oneOf. |
param.missing_description |
warning | A property with no description. |
server.no_tools |
warning | The server claims tools but registers none. |
probe_fuzz
For every tool (capped at maxTools, default 10), the generator
emits one valid case and at least three malformed variants:
missing_required:<field>— drop each required field in turn.wrong_type:<field>— replace each typed field with a value of a different primitive type.out_of_enum:<field>— forenumorconstfields, send a value the schema forbids.extra_garbage— append a sentinel key to the valid args.
Each case is sent to the target over the live JSON-RPC transport. The classifier assigns one of three outcomes:
| Outcome | Meaning |
|---|---|
ok |
The target returned a result with isError: false. For a malformed case this is silentlyAccepted: true; for a valid case with no usable content it is emptySuccess: true, and a valid success that breaks the tool's declared outputSchema is outputSchemaViolation: true. |
toolError |
The target returned a result with isError: true (graceful rejection). |
protocolCrash |
The call rejected or the transport closed. |
Hallucinated success (emptySuccess). A valid call that returns success but
with an empty / contentless result — e.g. a write tool that answers 200
with an empty body and never persists anything. The agent reads "done" while
nothing happened. MCProbe flags this on the critical line, drops Liveness
credit for that call (it isn't a real success), and recommends returning a
confirmation payload. This is the "the agent said done, nothing happened" bug.
(A tool that returns valid structuredContent but empty text content is not
flagged — the structured payload is a real result.)
Output-schema conformance (outputSchemaViolation). When a tool declares an
outputSchema (MCP structured output), MCProbe checks that a valid call actually
honors it — the success must return structuredContent that validates against
the declared schema. A tool that advertises an output contract and then returns
no structured content, or a payload that doesn't match, is flagged (and reported
as an output-schema violation rather than a misleading "protocol crash"). Not
credited in Liveness — a success that breaks its own contract isn't a real
success.
Dry-run safety. Fuzzing really calls the target's tools — including a
"valid" call that does whatever the tool does. So by default only tools
annotated readOnlyHint: true or destructiveHint: false are fuzzed. Per the
MCP spec, a tool without those hints is presumed destructive: it is skipped
unless you pass fuzzUnannotated: true (and MCProbe logs a warning naming
each such tool it calls). Tools annotated destructiveHint: true additionally
need fuzzDestructive: true. probe_fuzz (and the report) return a
coverage summary listing how many tools were fuzzed and which were skipped
(as destructive, as unannotated, or over the maxTools cap).
probe_report
The convenience entry point. Calls probe_lint (always) and
probe_fuzz (when fuzz: true), scores the result on the four
dimensions described below, and returns the structured
ConformanceReport and a rendered Markdown string. The
Markdown is the canonical payload; downstream tools that need the
numbers can pull them out of the structured fields.
Scoring model — four dimensions
The overall 0–100 score is the mean of the measured dimensions.
Dimensions that were not measured (e.g. the two behavioral ones when
fuzz: false, or when every tool was skipped) are reported as "not
measured" and excluded from the average rather than penalized with a
fake value. This is what lets a static audit of a clean server still
score 100/100.
The two static dimensions are subtractive (start at 10, lose points per finding). The two behavioral dimensions are normalized rates, so a score is comparable across servers of different sizes — and the fuzz cases are partitioned by kind (malformed → Error Handling, valid → Liveness) so no outcome is ever counted twice.
Letter grades: A ≥ 90, B ≥ 75, C ≥ 60, D ≥ 40, F < 40.
| Dimension | Always measured? | What it captures |
|---|---|---|
| Metadata & Documentation | yes | Server identity (name, version), advertised capabilities, presence of instructions (+1 bonus). |
| Schema Quality | yes | Rate over tools: findings are weighted (1 per error, 0.5 per warning, 0.25 per info), summed per tool and capped at 2 each, then averaged across every tool the server advertises. A server isn't penalized for having more tools — 50 tools with one minor nit each score the same as 5. |
| Error Handling | only with fuzz: true |
Rate over malformed cases: 10 × (gracefully-rejected / total malformed). A silent accept (garbage let through) or a protocol crash both count as failed rejections. |
| Liveness & Performance | only with fuzz: true |
Rate over valid cases: 10 × (successful / total valid), minus 0.5 per 100ms that the valid-call p50 latency exceeds a 200ms target. |
The per-dimension reasons and counts are emitted in the Markdown report so the score is auditable by a human. When fuzzing runs, the report header also shows two extra lines:
- a Coverage line (how many tools were fuzzed, and which were skipped
as destructive, as unannotated, or over the
maxToolscap); and - a critical-issues callout — a flag, not a second score — hoisting
the dangerous findings to the top, e.g.
⚠ Critical: 4 tool(s) silently accept malformed input (…); 1 protocol crash(es), or✓ No critical behavioral issueswhen there are none. The normalized scores are unchanged; this just makes the scary stuff visible above the fold.
The report ends with a Recommended fixes section: a prioritized to-do list (worst severity first) that turns each finding into a concrete action — the fix hint plus the exact tools/parameters it affects — followed by behavioral fixes for tools that silently accept input or crash. So the report is a prescription, not just a diagnosis. A clean server gets "Nothing to fix — this server passes every check."
30-second demo
The probe ships with a deliberately flawed demo target at
examples/demo-target/ and a smoke script that runs the full
probe_report pipeline against it. From a clean clone:
npm install
npm run build
node scripts/smoke-report.mjs
The script spawns the probe as a stdio MCP server, opens a
connection to the demo target, calls probe_report with
fuzz: true, and prints the Markdown report to stdout. The demo
target is wired to fail loudly: greet has no description,
divide returns NaN on bad input, set_mode has a thin
description, and well_behaved is the only tool with a clean,
validated schema. The report will show a low overall score with
concrete findings, a coverage line, a critical-issues callout, and a
fuzz table that classifies the broken cases.
For an interactive tour, the official MCP inspector works as a host against the built probe:
npx @modelcontextprotocol/inspector node dist/index.js
The inspector UI lists the six probe_* tools; calling them
manually is a good way to see the request/response shape.
mcprobe MCP questions
What tools does mcprobe MCP provide?
It exposes 10 tools, including probe_connect, probe_lint, probe_fuzz, probe_report, probe_list.
What does MCProbe actually test?
It performs a static lint of tool schemas for agent-usability and a behavioral fuzz test that calls tools with malformed inputs to check error handling.
Can I audit a local MCP server?
Yes, you can audit local stdio servers using the CLI with the --stdio flag or by configuring the probe to allow specific command lines in its environment.
Is it safe to use the fuzzing feature?
Fuzzing calls tools for real, so you should only use it against servers you own on test data, as it may trigger destructive actions if not properly annotated.
How can I integrate MCProbe into my CI/CD pipeline?
You can use the official GitHub Action to gate your MCP server and fail builds if the conformance grade falls below a specified minimum score.