skillcheck MCP server
Commit Oct 28, 2026Analyze and test AI agent skills to identify request competition, name clashes, and description effectiveness within Claude Code and Codex environments.
Also known as Noem836/skillcheck
- Stars
- 1
- Tools
- —
- Downloads, 30 days
- —
- Latest version
- 0.1.2
- Last commit
- Oct 28, 2026
- License
- MIT
What is skillcheck MCP?
skillcheck analyzes and test AI agent skills to identify request competition, name clashes, and description effectiveness within Claude Code and Codex environments.
Summary generated from the project's README
About skillcheck MCP server
skillcheck helps developers identify why AI agent skills fail to trigger in real-world environments. While official evaluations test skills in isolation, this tool analyzes how they behave alongside a full suite of existing plugins in Claude Code and Codex. It detects name clashes, description effectiveness, and instances where one skill accidentally steals a request intended for another.
Users can run the tool locally via a Python script or connect to a hosted, read-only MCP server for static checks on public GitHub repositories. The local version uses your existing CLI login and subscription to run real tests through the Claude or Codex command line interfaces. It provides a preflight mode to test new skills in a throwaway folder before installation, ensuring they do not shadow your current setup. The system generates detailed reports to compare how description changes impact the fire rate of your prompts.
What you can do with it
- 1Detect request competitionIdentify which installed skill is capturing prompts intended for a different skill.
- 2Validate new skillsCheck if a new skill will clash with or shadow your existing installed skills before you install it.
- 3Optimize skill descriptionsMeasure how rewriting a skill description improves its ability to trigger correctly.
- 4Perform static repo analysisRun automated checks on a public GitHub repository to find YAML errors or duplicate names.
Written from the project's README. Every use case is checked against a quote from it.
How to install skillcheck MCP
Formats verified 10 Nov 2026Add to CursorAdd to GooseInstall skillcheck MCP in Claude Code
$ claude mcp add --transport http skillcheck https://skillcheck-alpha.vercel.app/mcpInstall skillcheck MCP in Cursor
~/.cursor/mcp.json
{
"mcpServers": {
"skillcheck": {
"url": "https://skillcheck-alpha.vercel.app/mcp"
}
}
}Install skillcheck MCP in VS Code
.vscode/mcp.json
{
"servers": {
"skillcheck": {
"type": "http",
"url": "https://skillcheck-alpha.vercel.app/mcp"
}
}
}Install skillcheck MCP in VS Code CLI
$ code --add-mcp '{"name":"skillcheck","type":"http","url":"https://skillcheck-alpha.vercel.app/mcp"}'Install skillcheck MCP in Codex
$ codex mcp add skillcheck --url https://skillcheck-alpha.vercel.app/mcpInstall skillcheck MCP in Codex config.toml
~/.codex/config.toml
[mcp_servers.skillcheck] url = "https://skillcheck-alpha.vercel.app/mcp"
Install skillcheck MCP in Gemini CLI
$ gemini mcp add -s user --transport http skillcheck https://skillcheck-alpha.vercel.app/mcpInstall skillcheck MCP in Gemini settings.json
~/.gemini/settings.json
{
"mcpServers": {
"skillcheck": {
"url": "https://skillcheck-alpha.vercel.app/mcp",
"type": "http"
}
}
}Automated checks
Ran Nov 14, 2026Full report →Not verified yet. A server needs a score of 80, no serious finding, and enough data for us to check. How checks work · What to fix for more points
- Source repository linked and reachablenoem836/skillcheck is public and reachablegithub.com/noem836/skillcheck ↗Pass
- Package names this repository as its sourceNot checked
- Build provenance (Sigstore attestation)Not checked
- Open source license (OSI)MITspdx.org/licenses/MIT.html ↗Pass
- Commit or release in the last yearLast commit or release 2026-10-28github.com/noem836/skillcheck ↗Pass
- OpenSSF ScorecardNot checked
- Package name not close to a popular packageNot checked
- No critical advisory in the package or its dependenciesNot checked
- No hidden instructions to the model0 tool descriptions and the README read; no override phrases, credential paths or hidden charactersPass
- Destructive tools are labelledNot checked
- Remote endpoint answers the MCP handshakeinitialize answered with protocol 2025-06-18 in 430 msPass
README
View on GitHubskillcheck
Find out which of your skills is stealing the request.
Your skill passed eval. Does it fire in your setup, next to everything else you installed?
Official skill evals load one skill in a clean environment. In real use it sits next to dozens of other skills and plugins, and one of them quietly takes the request. In the author's setup, prompts that fired 21/21 in an isolated claude plugin eval missed 5 times in the real harness.
skillcheck runs your prompts through your real claude and codex CLIs with everything you have installed, and reports which skill fired, which one stole the request, how flaky each prompt is, and whether your description fix helped.
New: check a skill before you install it. preflight tells you which of your installed skills a new one will take requests from, by running your existing skills' prompts with and without it in a throwaway folder. Your real skill folders are not touched. Word overlap only suggests suspects (a narrow skill can take requests with little shared wording), so name the skills you care about with --skills:
python3 skillcheck.py preflight path/to/new-skill # instant: name clashes, likely rivals, listing size
python3 skillcheck.py preflight path/to/new-skill --measure # real runs: your skills' fire rate without → with it
python3 skillcheck.py preflight path/to/new-skill --measure --skills my-a,my-b # pick which of yours to measure
Example report: two skills went from 0/4 and 1/4 to 4/4 after rewriting their descriptions.
Install
As a Claude Code plugin (the skillcheck skill walks you through scan → run → fix → re-run):
/plugin install skillcheck --marketplace Noem836/skillcheck
Or as one file (Python 3 standard library; PyYAML is used for frontmatter checks if installed). Pin a release and check its hash:
curl -fsSLO https://raw.githubusercontent.com/Noem836/skillcheck/v0.1.2/skillcheck.py
shasum -a 256 skillcheck.py # compare with the release notes
Use
python3 skillcheck.py scan # static checks, no model calls
python3 skillcheck.py gen --skills my-skill # draft should-fire / near-miss prompts → suite.json
python3 skillcheck.py run --agent claude --tag before --runs 3
python3 skillcheck.py report --tag before # fire rate, false fires, who took it
# edit the description, then
python3 skillcheck.py run --agent claude --tag after --runs 3
python3 skillcheck.py compare --before before --after after
python3 skillcheck.py html --tag before,after # one-file HTML report
python3 skillcheck.py upload --tag before,after # optional share link
For AI agents (remote MCP, no install)
A hosted, read-only MCP server runs the static checks on any public GitHub repo, which is useful before installing or publishing a skill pack:
claude mcp add --transport http skillcheck https://skillcheck-alpha.vercel.app/mcp
Tools:
preflight_skill(github_url, installed): call it before installing a skill. Pass the skills the user already has (name and description, which an agent can read from its own skill listing). It returns the name clashes that would shadow an installed skill, which installed skills the new one is most likely to take requests from, and how much the skill listing grows.check_skill_repo(github_url)returns the frontmatter/YAML errors, name mismatches, weak or slash-only descriptions, nestedSKILL.mdfiles, duplicate names, and pairs of skills whose descriptions overlap enough to compete. Real trigger measurement still needs the CLI, because it runs in your setup.
What it checks
- Discovery: Claude Code (
~/.claude/skills,.claude/skills, enabled plugins,skillOverrides) and Codex (~/.agents/skills,~/.codex/skills,.agents/skills) - Static: missing/broken frontmatter, name vs folder mismatch, too short/long descriptions, nested
SKILL.md, slash-only wording,disable-model-invocation, duplicate names - Real triggers: each prompt run N times; the first skill loaded is recorded and the run is stopped
- Competition: which other skill grabbed a should-fire prompt, and where near-miss prompts went
Safety
Prompt suites and third-party skills may contain prompt injection, so runs are locked down:
Claude runs with --tools Skill only; Codex runs with --ignore-user-config, no web search, no apps/plugins, read-only sandbox. python3 check_isolation.py verifies this with a canary file.
Runs use your own CLI login and subscription. Nothing is sent anywhere unless you run upload, which sends skill names and hit/miss rows only — never prompt texts.
Report server
server.py is the share-link server (stdlib, sqlite for local runs, Vercel Blob when configured). check_web.py [url] attacks a running instance (XSS, oversized/deep JSON, slow clients, id guessing, rate-limit spoofing). Deploy on Vercel; the Dockerfile remains for self-hosting.
FAQ
Why isn't my Claude Code skill triggering?
Usually another installed skill wins the request, or the description reads as slash-command only ("use as /name"). Run scan, then run + report and look at the taken by column. See the guide.
My skill passes claude plugin eval but doesn't fire in my normal setup. Why?
The official eval loads the skill in a clean environment. skillcheck measures it next to everything you actually have installed.
Which of my skills is stealing requests from another?
report lists, per skill, which other skill loaded on its should-fire prompts.
Does it work with Codex CLI?
Yes: run --agent codex. Skill activation is read from the SKILL.md read in codex exec --json events.
How do I test SKILL.md descriptions before publishing to a marketplace?
gen drafts should-fire and near-miss prompts, run measures, you edit, then compare shows before/after. The remote MCP gives a quick static check of the whole repo.
Does it send my prompts or code anywhere?
No. Runs use your own CLI login locally. upload is opt-in and sends skill names and hit/miss rows only.
Limits
- Claude Code and Codex only. Gemini CLI and Cursor: activation can't be read reliably yet — PRs welcome.
- A run takes about a minute per prompt; 10 skills × 6 prompts × 3 runs is a few hours of subscription usage.
MIT license.
skillcheck MCP questions
How do I add skillcheck MCP to Claude Code?
Run claude mcp add --transport http skillcheck https://skillcheck-alpha.vercel.app/mcp in your terminal. Add --scope user to make it available in every project.
How do I add skillcheck MCP to Cursor?
Open ~/.cursor/mcp.json and add the server under mcpServers, using the Cursor config shown in the install section above.
Why isn't my Claude Code skill triggering?
It is likely that another installed skill is winning the request or your description is written as a slash-command only. Use the scan, run, and report commands to identify the culprit.
Does it send my prompts or code anywhere?
No, runs use your local CLI login. The upload feature is optional and only transmits skill names and hit/miss data, never the prompt text itself.
Does it work with Codex CLI?
Yes, you can use the run command with the codex agent to measure skill activation.
How can I check a skill before installing it?
The preflight command allows you to check for name clashes and likely rivals by running existing skills' prompts with and without the new skill in a temporary folder.