agent-regression-evals plugin
Prove an agent, prompt, model or tool change did not make the agent worse. Three deterministic stdlib scripts: a trajectory checker that grades tool-call paths (strict / unordered / subset / superset match modes, argument matching, forbidden tools that block in every mode, call ceilings); a flake detector that classifies each case as stable or flaky from repeated trials and reports pass@k and pass^k with Wilson intervals; and a paired regression gate that averages trials per case, tests per-case differences with a t-based CI and an exact one-sided McNemar test, and blocks the merge on a significant drop, a critical-case regression, a dropped case, or a quarantine entry without owner and expiry. Ships cs-eval-gatekeeper agent, /cs-agent-evals and /cs-eval-gate commands, 3 references, and a CI job example.
How to install the agent-regression-evals plugin
Run both commands inside Claude Code. The first adds the marketplace once; the second installs the plugin.
/plugin marketplace add alirezarezvani/claude-skills
/plugin install agent-regression-evals@claude-code-skillsWhat's inside agent-regression-evals
4 parts read from the plugin's own files.
Commands 2
Run as /agent-regression-evals:name- /agent-regression-evals:cs-agent-evals
No description in the file.
- /agent-regression-evals:cs-eval-gate
No description in the file.
Agents 1
Call as @agent-agent-regression-evals:name- @agent-agent-regression-evals:cs-eval-gatekeeper
No description in the file.
Skills 1
Claude loads it when the task matches- agent-regression-evals
No description in the file.
Using agent-regression-evals in Claude Code
Type a command in a Claude Code session:
/agent-regression-evals:cs-agent-evals
/agent-regression-evals:cs-eval-gateAsk for an agent by name:
@agent-agent-regression-evals:cs-eval-gatekeeperSkills need no command. Claude reads them when your request matches what they describe.
Update, turn off or remove
- Update the plugin (shell)
- claude plugin update agent-regression-evals@claude-code-skills
- Refresh the marketplace
- /plugin marketplace update claude-code-skills
- Turn it off for now
- /plugin disable agent-regression-evals@claude-code-skills
- Remove it
- /plugin uninstall agent-regression-evals@claude-code-skills
agent-regression-evals plugin questions
How do I install the agent-regression-evals plugin?
In Claude Code run /plugin marketplace add alirezarezvani/claude-skills, then /plugin install agent-regression-evals@claude-code-skills.
What does the agent-regression-evals plugin include?
agent-regression-evals bundles 2 commands, 1 agent, 1 skill.
What commands does the agent-regression-evals plugin add?
It adds /agent-regression-evals:cs-agent-evals, /agent-regression-evals:cs-eval-gate. Run them in Claude Code with the plugin prefix, for example /agent-regression-evals:cs-agent-evals.
Which agents come with agent-regression-evals?
cs-eval-gatekeeper. Call one with @agent-agent-regression-evals:cs-eval-gatekeeper.
How do I update or remove the agent-regression-evals plugin?
Update it with claude plugin update agent-regression-evals@claude-code-skills in your shell, or from the Installed tab of /plugin. Remove it with /plugin uninstall agent-regression-evals@claude-code-skills.