qa-llm-evaluation plugin
LLM and prompt evaluation with Promptfoo, DeepEval and Ragas, Giskard vulnerability scans, Langfuse tracing, and golden-dataset regression suites. Audits eval suites for methodology errors. Agents (prompt-eval-reviewer, llm-red-team-planner) review an eval suite and plan adversarial probe campaigns across the OWASP LLM Top 10.
How to install the qa-llm-evaluation plugin
Run both commands inside Claude Code. The first adds the marketplace once; the second installs the plugin.
/plugin marketplace add testland/qa
/plugin install qa-llm-evaluation@testland-qaWhat's inside qa-llm-evaluation
9 parts read from the plugin's own files.
Agents 2
Call as @agent-qa-llm-evaluation:name- @agent-qa-llm-evaluation:llm-red-team-plannersonnet
Use proactively whenever an LLM app needs a red-team campaign beyond default scanner presets. Enumerates attack classes (jailbreaks, indirect prompt injection, data exfiltration, OWASP LLM Top 10), maps each to a Giskard detector or promptfoo red-team plugin and writes the scan scripts and configs.
- @agent-qa-llm-evaluation:prompt-eval-reviewersonnet
Use proactively whenever an LLM eval suite (Promptfoo, OpenAI Evals, DeepEval, Ragas, Giskard or custom) is added or changed. Flags too few cases, one provider only, missing model-graded or semantic checks, no baseline diff, no cost or latency cap, unpinned models and no adversarial cases.
Skills 7
Claude loads it when the task matches- deepeval-evaluation
No description in the file.
- giskard-llm
No description in the file.
- langfuse-tracing
No description in the file.
- llm-eval-anti-patterns
No description in the file.
- llm-regression-suite-author
No description in the file.
- promptfoo-evaluation
No description in the file.
- ragas-evaluation
No description in the file.
Using qa-llm-evaluation in Claude Code
Ask for an agent by name:
@agent-qa-llm-evaluation:llm-red-team-planner
@agent-qa-llm-evaluation:prompt-eval-reviewerSkills need no command. Claude reads them when your request matches what they describe.
Update, turn off or remove
- Update the plugin (shell)
- claude plugin update qa-llm-evaluation@testland-qa
- Refresh the marketplace
- /plugin marketplace update testland-qa
- Turn it off for now
- /plugin disable qa-llm-evaluation@testland-qa
- Remove it
- /plugin uninstall qa-llm-evaluation@testland-qa
qa-llm-evaluation plugin questions
How do I install the qa-llm-evaluation plugin?
In Claude Code run /plugin marketplace add testland/qa, then /plugin install qa-llm-evaluation@testland-qa.
What does the qa-llm-evaluation plugin include?
qa-llm-evaluation bundles 2 agents, 7 skills.
Which agents come with qa-llm-evaluation?
llm-red-team-planner, prompt-eval-reviewer. Call one with @agent-qa-llm-evaluation:llm-red-team-planner.
How do I update or remove the qa-llm-evaluation plugin?
Update it with claude plugin update qa-llm-evaluation@testland-qa in your shell, or from the Installed tab of /plugin. Remove it with /plugin uninstall qa-llm-evaluation@testland-qa.
What license is the qa-llm-evaluation plugin under?
MIT, as stated in the plugin's manifest.