決定論的なルールとオプションのLLM judgeを使用して、エージェントの出力を品質、安全性、コストの観点からスコアリングします。
別名 iris-eval/mcp-server, mcp-server, @iris-eval/mcp-server
- スター
- 10
- ツール
- 17
- ダウンロード数(30日間)
- 1,137
- 最新バージョン
- 0.21.0
- 最終コミット
- 2026年11月13日
- ライセンス
- MIT
Iris MCPとは
IrisはMCPサーバーです。 決定論的なルールとオプションのLLM judgeを使用して、エージェントの出力を品質、安全性、コストの観点からスコアリングします。. 17個のツールを提供しており、log_trace, evaluate_output, get_tracesなどがあります。
プロジェクトのREADMEから生成された概要
Iris MCP server について
Irisは、出力の品質、安全性、コストをスコアリングすることで、エージェントのパフォーマンスを監査する手段を提供します。手動でのチェックに頼る代わりに、開発者は28個の組み込み決定論的ルールを使用して、PII(個人情報)の漏洩、プロンプトインジェクション、ハルシネーション、コスト閾値の超過などの問題を検出できます。システムはすべてのエージェントの実行をマシン上のローカルなSQLiteデータベースに保存するため、OpenTelemetryエンドポイント、webhook、またはLLM judgeを明示的に有効にしない限り、データはプライベートな状態に保たれます。
ユーザーは、Claude Desktop、Cursor、VS Code、Windsurfを含むさまざまなMCPクライアントにIrisを統合できます。決定論的ルールは無料でローカルで実行されますが、セマンティックな評価タスクを処理するために、独自のAPIキーを提供してオプションでLLM judgeを有効にすることも可能です。また、このサーバーにはlocalhost:6920で提供されるダッシュボード機能があり、失敗の可視化や異なるエージェント実行結果の比較ができます。自律型エージェントを評価する際に、主観的な印象ではなく監査可能な数値が必要な開発者向けに設計されています。
できること
- 1エージェントの安全性と品質の監査エージェントにタスクをログ記録させ、その結果として出力された内容に特定のルール違反がないか評価します。
- 2コストとハルシネーションのリスク監視決定論的ルールを使用して、エージェントがコスト閾値を超えていないか、またはハルシネーションの兆候を生成していないかを確認します。
- 3HTTPによるトレース取り込みの自動化LLMを介在させることなく、POSTリクエストを使用してエージェントのトレースをサーバーに直接送信します。
プロジェクトの README から作成されています。すべてのユースケースは、その引用と照らし合わせて確認されています。
Iris MCPのインストール方法
フォーマット検証日 2026年11月10日Cursor に追加Gooseに追加Claude CodeにIris MCPをインストール
$ claude mcp add --env 'IRIS_ANTHROPIC_API_KEY=<YOUR_IRIS_ANTHROPIC_API_KEY>' --env 'IRIS_API_KEY=<YOUR_IRIS_API_KEY>' --env 'IRIS_OPENAI_API_KEY=<YOUR_IRIS_OPENAI_API_KEY>' --env 'IRIS_WEBHOOK_SECRET=<YOUR_IRIS_WEBHOOK_SECRET>' mcp-server -- npx -y @iris-eval/mcp-serverCursorにIris MCPをインストール
~/.cursor/mcp.json
{
"mcpServers": {
"mcp-server": {
"command": "npx",
"args": [
"-y",
"@iris-eval/mcp-server"
],
"env": {
"IRIS_ANTHROPIC_API_KEY": "<YOUR_IRIS_ANTHROPIC_API_KEY>",
"IRIS_API_KEY": "<YOUR_IRIS_API_KEY>",
"IRIS_OPENAI_API_KEY": "<YOUR_IRIS_OPENAI_API_KEY>",
"IRIS_WEBHOOK_SECRET": "<YOUR_IRIS_WEBHOOK_SECRET>"
}
}
}
}VS CodeにIris MCPをインストール
.vscode/mcp.json
{
"servers": {
"mcp-server": {
"type": "stdio",
"command": "npx",
"args": [
"-y",
"@iris-eval/mcp-server"
],
"env": {
"IRIS_ANTHROPIC_API_KEY": "<YOUR_IRIS_ANTHROPIC_API_KEY>",
"IRIS_API_KEY": "<YOUR_IRIS_API_KEY>",
"IRIS_OPENAI_API_KEY": "<YOUR_IRIS_OPENAI_API_KEY>",
"IRIS_WEBHOOK_SECRET": "<YOUR_IRIS_WEBHOOK_SECRET>"
}
}
}
}VS Code CLIにIris MCPをインストール
$ code --add-mcp '{"name":"mcp-server","type":"stdio","command":"npx","args":["-y","@iris-eval/mcp-server"],"env":{"IRIS_ANTHROPIC_API_KEY":"<YOUR_IRIS_ANTHROPIC_API_KEY>","IRIS_API_KEY":"<YOUR_IRIS_API_KEY>","IRIS_OPENAI_API_KEY":"<YOUR_IRIS_OPENAI_API_KEY>","IRIS_WEBHOOK_SECRET":"<YOUR_IRIS_WEBHOOK_SECRET>"}}'Claude DesktopにIris MCPをインストール
claude_desktop_config.json
{
"mcpServers": {
"mcp-server": {
"command": "npx",
"args": [
"-y",
"@iris-eval/mcp-server"
],
"env": {
"IRIS_ANTHROPIC_API_KEY": "<YOUR_IRIS_ANTHROPIC_API_KEY>",
"IRIS_API_KEY": "<YOUR_IRIS_API_KEY>",
"IRIS_OPENAI_API_KEY": "<YOUR_IRIS_OPENAI_API_KEY>",
"IRIS_WEBHOOK_SECRET": "<YOUR_IRIS_WEBHOOK_SECRET>"
}
}
}
}CodexにIris MCPをインストール
$ codex mcp add mcp-server --env 'IRIS_ANTHROPIC_API_KEY=<YOUR_IRIS_ANTHROPIC_API_KEY>' --env 'IRIS_API_KEY=<YOUR_IRIS_API_KEY>' --env 'IRIS_OPENAI_API_KEY=<YOUR_IRIS_OPENAI_API_KEY>' --env 'IRIS_WEBHOOK_SECRET=<YOUR_IRIS_WEBHOOK_SECRET>' -- npx -y @iris-eval/mcp-serverCodex config.tomlにIris MCPをインストール
~/.codex/config.toml
[mcp_servers.mcp-server] command = "npx" args = ["-y", "@iris-eval/mcp-server"] [mcp_servers.mcp-server.env] IRIS_ANTHROPIC_API_KEY = "<YOUR_IRIS_ANTHROPIC_API_KEY>" IRIS_API_KEY = "<YOUR_IRIS_API_KEY>" IRIS_OPENAI_API_KEY = "<YOUR_IRIS_OPENAI_API_KEY>" IRIS_WEBHOOK_SECRET = "<YOUR_IRIS_WEBHOOK_SECRET>"
Gemini CLIにIris MCPをインストール
$ gemini mcp add -s user -e 'IRIS_ANTHROPIC_API_KEY=<YOUR_IRIS_ANTHROPIC_API_KEY>' -e 'IRIS_API_KEY=<YOUR_IRIS_API_KEY>' -e 'IRIS_OPENAI_API_KEY=<YOUR_IRIS_OPENAI_API_KEY>' -e 'IRIS_WEBHOOK_SECRET=<YOUR_IRIS_WEBHOOK_SECRET>' mcp-server npx -- -y @iris-eval/mcp-serverGemini settings.jsonにIris MCPをインストール
~/.gemini/settings.json
{
"mcpServers": {
"mcp-server": {
"command": "npx",
"args": [
"-y",
"@iris-eval/mcp-server"
],
"env": {
"IRIS_ANTHROPIC_API_KEY": "<YOUR_IRIS_ANTHROPIC_API_KEY>",
"IRIS_API_KEY": "<YOUR_IRIS_API_KEY>",
"IRIS_OPENAI_API_KEY": "<YOUR_IRIS_OPENAI_API_KEY>",
"IRIS_WEBHOOK_SECRET": "<YOUR_IRIS_WEBHOOK_SECRET>"
}
}
}
}Iris MCPツール 17
READMEから検出log_traceevaluate_outputget_traceslist_rulesdeploy_ruledelete_ruledelete_traceevaluate_with_llm_judgeverify_citationscompare_runscompare_tracesevaluate_runs
| log_trace | — | |
| evaluate_output | — | |
| get_traces | — | |
| list_rules | — | |
| deploy_rule | — | |
| delete_rule | — | |
| delete_trace | — | |
| evaluate_with_llm_judge | — | |
| verify_citations | — | |
| compare_runs | — | |
| compare_traces | — | |
| evaluate_runs | — |
17個のツールをすべて表示ツールを折りたたむ
scorenametypeconfigweight
| score | — | |
| name | — | |
| type | — | |
| config | — | |
| weight | — |
設定
IRIS_ANTHROPIC_API_KEYシークレットEnables evaluate_with_llm_judge and verify_citations with an Anthropic model. Optional: the deterministic rules never need a key
IRIS_API_KEYシークレットAPI key for HTTP authentication. Required to bind the HTTP transport or the dashboard beyond loopback (0.0.0.0, a LAN address, a container): without it the server refuses to start
IRIS_OPENAI_API_KEYシークレットEnables evaluate_with_llm_judge and verify_citations with an OpenAI model. Optional: the deterministic rules never need a key
IRIS_WEBHOOK_SECRETシークレットThe signing key of that webhook (any string, or whsec_ + base64); the iris format refuses to run without one
IRIS_ALLOWED_ORIGINSComma-separated Origin allowlist for the HTTP transport and dashboard CORS (default http://localhost:*)
IRIS_ALLOW_UNAUTHENTICATEDSet to 1 to run a non-loopback bind with NO API key on purpose (lifts the startup refusal; the network is then your boundary)
IRIS_API_KEY_FILEPath to a file whose trimmed contents are the API key (the secret-file pattern Docker and Kubernetes mount). Set this or IRIS_API_KEY, not both; further keys and rotation are security.apiKeys in config.json
IRIS_CITATION_ALLOW_FETCHLet verify_citations fetch URLs (off by default); a tool argument can turn it off for a call, never on
IRIS_CITATION_DOMAINSComma-separated domain allowlist for verify_citations fetches (only consulted when IRIS_CITATION_ALLOW_FETCH is on); a tool argument can narrow it, never add to it
IRIS_CITATION_MAX_COST_USD_TOTALThe most one verify_citations call may spend on judge calls, in USD (default 1); max_cost_usd_total can lower it, never raise it
IRIS_DASHBOARDStart the dashboard alongside the server (same as --dashboard)
IRIS_DASHBOARD_HOSTBind host for the dashboard (default 127.0.0.1); same as --dashboard-host
IRIS_DASHBOARD_PORTDashboard port (default 6920); IRIS_PORT is the MCP HTTP transport port (default 3000)
IRIS_DB_PATHSQLite database path
IRIS_HOMEDirectory for the database, custom rules and preferences (default ~/.iris)
IRIS_HOSTBind host for the HTTP transport (default 127.0.0.1); only used with --transport http
IRIS_LLM_JUDGE_DAILY_BUDGET_USDWhat every judge call together may spend per UTC day, per tenant, in USD (default 1): the relevance judge, evaluate_with_llm_judge and verify_citations. Past it no judge call is made
IRIS_LLM_JUDGE_MAX_COST_USD_PER_EVALHard cap on the worst-case cost of one LLM judge call (default 0.25); max_cost_usd can lower it for a call, never raise it
IRIS_LOG_LEVELLog level: debug, info, warn, error
IRIS_MAX_INPUT_ELEMENTSThe most array elements and object keys one input may hold before it is validated (default 100000): a tool call's arguments, and a dashboard or OTLP JSON request body. A tool call over it gets an error result; a body, 413. OTLP protobuf has its own ceiling, one element per 5 bytes of the request size limit
IRIS_MAX_TOOL_INPUT_ELEMENTSThe name of IRIS_MAX_INPUT_ELEMENTS before 0.21.0, read when the new name is unset
IRIS_OTEL_ENDPOINTOTLP/HTTP collector URL; when set, every stored trace is also exported as OpenTelemetry spans (best-effort, asynchronous)
IRIS_OTEL_HEADERSComma-separated key=value headers sent with every OTLP export (for example an auth token for your collector)
IRIS_OTEL_SERVICE_NAMEservice.name resource attribute on exported spans (default iris-eval)
IRIS_OTEL_TIMEOUT_MSTimeout in milliseconds for each OTLP export request
IRIS_PORTMCP HTTP transport port (default 3000); only used with --transport http
IRIS_RELEVANCE_JUDGE_DAILY_BUDGET_USDThe name of IRIS_LLM_JUDGE_DAILY_BUDGET_USD before 0.21.0. Read when the new name is unset, and it then limits every judge call
IRIS_RELEVANCE_JUDGE_MAX_CALLS_PER_REQUESTRelevance judge calls one request may make (default 20)
IRIS_RELEVANCE_JUDGE_MODELA priced judge model; answers_the_ask then asks it on every evaluation with input, sending that input and output to the model's provider, and gates on its verdict (off by default)
IRIS_RELEVANCE_JUDGE_REDACTon (default) replaces the PII and credentials no_pii flags before the relevance judge sends input and output to its provider; off sends them as they are
IRIS_SEARCH_BUDGET_MSHow long one trace search may read before it answers with the matches found so far, in milliseconds (50-60000, default 1000)
IRIS_SEARCH_INDEXon (default) or off. off keeps no full-text index of the traces: writes are cheaper, and a search reads the traces within its time budget
IRIS_SQLITE_DRIVERWhich SQLite driver holds the database: native (better-sqlite3, the default) or node (Node's built-in node:sqlite, Node 22.13+). Unset: native, falling back to node when the native module cannot load
IRIS_TRANSPORTTransport: stdio (default) or http; same as --transport
IRIS_WEBHOOK_URLThe receiver of the webhook that fires on a moment (docs/webhooks.md); merged over notify.webhook in config.json
| 名前 | 説明 | 種類 |
|---|---|---|
| IRIS_ANTHROPIC_API_KEY | Enables evaluate_with_llm_judge and verify_citations with an Anthropic model. Optional: the deterministic rules never need a key | シークレット |
| IRIS_API_KEY | API key for HTTP authentication. Required to bind the HTTP transport or the dashboard beyond loopback (0.0.0.0, a LAN address, a container): without it the server refuses to start | シークレット |
| IRIS_OPENAI_API_KEY | Enables evaluate_with_llm_judge and verify_citations with an OpenAI model. Optional: the deterministic rules never need a key | シークレット |
| IRIS_WEBHOOK_SECRET | The signing key of that webhook (any string, or whsec_ + base64); the iris format refuses to run without one | シークレット |
| IRIS_ALLOWED_ORIGINS | Comma-separated Origin allowlist for the HTTP transport and dashboard CORS (default http://localhost:*) | |
| IRIS_ALLOW_UNAUTHENTICATED | Set to 1 to run a non-loopback bind with NO API key on purpose (lifts the startup refusal; the network is then your boundary) | |
| IRIS_API_KEY_FILE | Path to a file whose trimmed contents are the API key (the secret-file pattern Docker and Kubernetes mount). Set this or IRIS_API_KEY, not both; further keys and rotation are security.apiKeys in config.json | |
| IRIS_CITATION_ALLOW_FETCH | Let verify_citations fetch URLs (off by default); a tool argument can turn it off for a call, never on | |
| IRIS_CITATION_DOMAINS | Comma-separated domain allowlist for verify_citations fetches (only consulted when IRIS_CITATION_ALLOW_FETCH is on); a tool argument can narrow it, never add to it | |
| IRIS_CITATION_MAX_COST_USD_TOTAL | The most one verify_citations call may spend on judge calls, in USD (default 1); max_cost_usd_total can lower it, never raise it | |
| IRIS_DASHBOARD | Start the dashboard alongside the server (same as --dashboard) | |
| IRIS_DASHBOARD_HOST | Bind host for the dashboard (default 127.0.0.1); same as --dashboard-host | |
| IRIS_DASHBOARD_PORT | Dashboard port (default 6920); IRIS_PORT is the MCP HTTP transport port (default 3000) | |
| IRIS_DB_PATH | SQLite database path | |
| IRIS_HOME | Directory for the database, custom rules and preferences (default ~/.iris) | |
| IRIS_HOST | Bind host for the HTTP transport (default 127.0.0.1); only used with --transport http | |
| IRIS_LLM_JUDGE_DAILY_BUDGET_USD | What every judge call together may spend per UTC day, per tenant, in USD (default 1): the relevance judge, evaluate_with_llm_judge and verify_citations. Past it no judge call is made | |
| IRIS_LLM_JUDGE_MAX_COST_USD_PER_EVAL | Hard cap on the worst-case cost of one LLM judge call (default 0.25); max_cost_usd can lower it for a call, never raise it | |
| IRIS_LOG_LEVEL | Log level: debug, info, warn, error | |
| IRIS_MAX_INPUT_ELEMENTS | The most array elements and object keys one input may hold before it is validated (default 100000): a tool call's arguments, and a dashboard or OTLP JSON request body. A tool call over it gets an error result; a body, 413. OTLP protobuf has its own ceiling, one element per 5 bytes of the request size limit | |
| IRIS_MAX_TOOL_INPUT_ELEMENTS | The name of IRIS_MAX_INPUT_ELEMENTS before 0.21.0, read when the new name is unset | |
| IRIS_OTEL_ENDPOINT | OTLP/HTTP collector URL; when set, every stored trace is also exported as OpenTelemetry spans (best-effort, asynchronous) | |
| IRIS_OTEL_HEADERS | Comma-separated key=value headers sent with every OTLP export (for example an auth token for your collector) | |
| IRIS_OTEL_SERVICE_NAME | service.name resource attribute on exported spans (default iris-eval) | |
| IRIS_OTEL_TIMEOUT_MS | Timeout in milliseconds for each OTLP export request | |
| IRIS_PORT | MCP HTTP transport port (default 3000); only used with --transport http | |
| IRIS_RELEVANCE_JUDGE_DAILY_BUDGET_USD | The name of IRIS_LLM_JUDGE_DAILY_BUDGET_USD before 0.21.0. Read when the new name is unset, and it then limits every judge call | |
| IRIS_RELEVANCE_JUDGE_MAX_CALLS_PER_REQUEST | Relevance judge calls one request may make (default 20) | |
| IRIS_RELEVANCE_JUDGE_MODEL | A priced judge model; answers_the_ask then asks it on every evaluation with input, sending that input and output to the model's provider, and gates on its verdict (off by default) | |
| IRIS_RELEVANCE_JUDGE_REDACT | on (default) replaces the PII and credentials no_pii flags before the relevance judge sends input and output to its provider; off sends them as they are | |
| IRIS_SEARCH_BUDGET_MS | How long one trace search may read before it answers with the matches found so far, in milliseconds (50-60000, default 1000) | |
| IRIS_SEARCH_INDEX | on (default) or off. off keeps no full-text index of the traces: writes are cheaper, and a search reads the traces within its time budget | |
| IRIS_SQLITE_DRIVER | Which SQLite driver holds the database: native (better-sqlite3, the default) or node (Node's built-in node:sqlite, Node 22.13+). Unset: native, falling back to node when the native module cannot load | |
| IRIS_TRANSPORT | Transport: stdio (default) or http; same as --transport | |
| IRIS_WEBHOOK_URL | The receiver of the webhook that fires on a moment (docs/webhooks.md); merged over notify.webhook in config.json |
自動チェック
実行日 2026年11月12日完全なレポート →自動チェックに合格しました。これはセキュリティ監査ではありません。コードを読み、サーバーに必要最小限のアクセス権を与えてください。 チェックの仕組み
- ソースリポジトリがリンクされており、アクセス可能合格
- オープンソースライセンス (OSI)合格
- 過去1年以内にコミットまたはリリースあり合格
- パッケージ名が有名なパッケージに酷似していない合格
- リモートエンドポイントがHTTPSを使用未チェック
- 破壊的なツールがラベル付けされている未チェック
- モデルへの隠れた指示がない合格
- パッケージおよびその依存関係に重大な勧告なし合格
README
GitHubで見るIris — stop shipping agents on vibes
Send Iris an agent run and ask for a verdict on its quality, safety, and cost — on your machine, with no SDK and no account. Most agent projects check quality by running a few remembered prompts and eyeballing the output. Iris replaces that with numbers you can audit: your agent's runs land in a SQLite database on your disk, 28 built-in rules score them deterministically — PII, prompt injection, hallucination markers, cost thresholds, and the agent's own tool calls — free, with no LLM calls, and an optional LLM judge with a hard per-eval cost cap handles the semantic questions. Every rule is inspectable and editable, because a judge you can't audit is just vibes with a number on it. MIT licensed, no telemetry. Nothing leaves your machine unless you turn one of these on: an OpenTelemetry endpoint (IRIS_OTEL_ENDPOINT), which exports traces to the collector you name; the LLM judge with your own key, which sends the text it judges to that provider, and whose citation check fetches the pages an output cites; or a webhook, which posts ids, the verdict and rule names, never the text, to the address you set.
Requires Node.js 22.13 or later. Check with node --version.

The demo database, recorded by scripts/demo-media.mts; the source is demo.mp4. A still: dashboard-overview.png.
A failure on screen in 60 seconds
No agent wiring, no config — one command:
npx @iris-eval/mcp-server --demo
This seeds a demo database — five small agents, two weeks of runs, every verdict the engine's own — and serves the dashboard against it at http://localhost:6920 (your browser opens automatically on first run). The dashboard lands on Failures: what failed, worst and newest first, each card naming the rule and its evidence. Worth clicking into — a PII leak caught by the safety rules, a hidden directive in a forum post that the summarizer complied with, a number the source document never said, two runs on the same twelve questions compared with an interval (Runs), a deployed custom rule and a paused one with their audit rows, and a failed LLM-judge score with its rationale.
Demo data lives in its own database (demo.db in your Iris home directory — ~/.iris on macOS/Linux, %USERPROFILE%\.iris on Windows) and never mixes with your real traces. Remove all of it with one command:
npx @iris-eval/mcp-server --demo-clear
Hook up your own agent
First, prove the install works on this machine — it runs offline and opens nothing of yours:
npx @iris-eval/mcp-server --self-test # exit 0 = healthy
Then add Iris to your MCP client. One command writes the client's own config file, keeps every other server in it, and pins the version you ran:
npx -y @iris-eval/mcp-server install claude-code
The clients: claude-code, claude-desktop, cursor, windsurf, continue, vscode, cline, zed, codex, gemini. install --list shows the ones found on this machine, the Iris each one runs and the file it reads; install <client> --uninstall takes Iris out again. Every client shares one database, so after an upgrade move them all at once with install --upgrade (Updating). Restart the client to load it.
Claude Desktop: one click. Each release from 0.20.0 on attaches iris-eval.mcpb, an MCP Bundle: download the latest, open it, and Claude Desktop shows an install dialog. Nothing on it is required — an Anthropic or OpenAI key for the LLM judge is optional, and the dashboard is a switch that starts off. The bundle holds the npm package and its dependencies, so nothing else needs installing: Claude Desktop runs it under the Node it ships when that Node is 22.13 or newer (Claude Desktop 1.1.6679 ships 24.13), and Iris stores traces with Node's built-in SQLite, in the same ~/.iris every other install uses. The release notes show how to verify its signature and build attestation.
It runs in any MCP client, and every client it names has a row saying how it is checked. Verified on every CI run: Claude Code, Gemini CLI — the real client starts Iris from the config the installer wrote and reports it connected (claude mcp list, gemini mcp list), on Linux, macOS and Windows; Claude Code's capture plugin hooks are driven through the real scripts too. Configured from each client's own MCP documentation — the installer writes the file each client documents, at its documented path and in its own format (an mcpServers block, VS Code's servers, Zed's context_servers, Codex's TOML), keeping comments and every other server, and unit tests hold each path and every format writer to that documentation: Claude Desktop, Cursor, Devin Desktop (Windsurf), Continue, VS Code, Cline, Zed, OpenAI Codex CLI. Every row with its source and the date it was read: https://iris-eval.com/clients. By hand instead, one block, dashboard included:
{
"mcpServers": {
"iris-eval": {
"command": "npx",
"args": ["-y", "@iris-eval/mcp-server", "--dashboard"]
}
}
}
Your client lists Iris's twelve tools on connect, and the dashboard serves at http://localhost:6920. Now paste this to your agent:
Log that last task to Iris and evaluate the output.
The trace lands on the dashboard with its scores. Prefer the MCP server headless? Drop --dashboard from the args — you can open the same dashboard any time with npx @iris-eval/mcp-server --dashboard.
One thing worth knowing up front: MCP tools are called when the model decides to call them. Iris doesn't intercept your agent, so traces are logged when your agent asks it to log them — either because you told it to, or because your code calls the tools directly. Ask your agent to "log this to Iris and evaluate it" and it will. If you want capture that doesn't depend on the model choosing, POST /api/v1/traces does exactly that — your code sends the trace over plain HTTP, no model in the loop (see docs/http-ingest.md). The CLI and host hooks on the roadmap will be thin clients over the same endpoint.
Capture over HTTP (no model in the loop)
The ingest endpoint lives on the dashboard port — 6920 by default, not the MCP transport port — and it exists only while the dashboard is running. Pass --dashboard (or set IRIS_DASHBOARD=true); --transport http on its own does not start it, and a request to the transport port returns 404. With the dashboard up, anything that can send an HTTP request can log a trace — and optionally run the deterministic evals in the same request. GET /api/v1/capabilities on the same port says what this server can judge, what each rule needs, the judge state with the steps that enable it, and the limits — the same object the MCP resource iris://capabilities serves — so an HTTP caller has the frame an MCP client gets at initialize:
curl -s -X POST "http://127.0.0.1:6920/api/v1/traces" \
-H "Content-Type: application/json" \
-d '{
"agent_name": "support-bot",
"input": "What is the refund policy?",
"output": "Refunds are available within 30 days of purchase.",
"evaluate": true,
"eval_type": "safety"
}'
Returns 201 with the stored trace_id and the evaluation result (in --demo mode the endpoint refuses writes with 403, so demo data never mixes with yours). The endpoint accepts the same body as the log_trace tool and sits behind the same middleware stack as the rest of the dashboard: loopback bind and the DNS-rebinding guard by default, plus Bearer auth when you set one. Two plain facts about it: it accepts unauthenticated writes unless Iris was started with --api-key (or IRIS_API_KEY) — the loopback bind is what keeps it to your machine by default, so set a key before binding beyond loopback; and what it stores is verbatim — input and output land in iris.db exactly as sent, including any text no_pii goes on to flag. Full contract, field reference, and error semantics: docs/http-ingest.md.
Capture every Claude Code turn (optional)
/plugin marketplace add iris-eval/mcp-server
/plugin install iris-eval-capture@iris-eval
A second, separately installed plugin: its hooks record each turn's prompt, tool calls (the failed ones included, with the error the agent received) and final answer, keyed on the prompt, so an interrupted turn's calls and a background sub-agent's are sent too, and hand them to iris-eval ingest, detached, with critical spans redacted in the stored evaluation text — capture that does not depend on the model deciding to call a tool. A turn that ended in an API error is stored without being judged; when the model logged the turn too, both traces are kept and the plugin's names the model's. It never prints, never blocks, never sends anything anywhere. Installing iris-eval alone changes nothing about your turn loop. Limits and removal: claude-plugin-capture/README.md.
Python
pip install iris-eval
from iris_eval import IrisClient
iris = IrisClient() # IRIS_URL, or the running dashboard's runtime.json
iris.evaluate_output("…", input="…", agent_name="support-bot")["verdict"] # {"state": "pass", "basis": "clean", "by": []}
A thin client over the HTTP API of server 0.16.0 and later, versioned on its own — iris_eval.__version__ and the PyPI page carry its number, which is not the server's: log_trace(), evaluate_output(), get_traces(), get_trace(), health(), capabilities(), sync and async, typed answers, the server's own sentence on a refusal — and a pytest plugin: an iris fixture and assert_iris(output, expect="pass") that asserts on the verdict's state. packages/python/README.md.
Record every OpenAI and Anthropic call
from iris_eval import wrap_openai
client = wrap_openai(OpenAI(), agent_name="support-bot") # every call: a GenAI span to POST /v1/traces, scored
import { wrapOpenAI } from '@iris-eval/sdk';
const openai = wrapOpenAI(new OpenAI(), { agentName: 'support-bot' });
Wrap the provider client once and each model call becomes one OpenTelemetry GenAI span sent to the OTLP door, stored with its input, output, token usage and tool calls, and scored: capture that does not depend on the model calling a tool. wrap_openai / wrap_anthropic in the Python client; wrapOpenAI, wrapAnthropic and irisMiddleware for the Vercel AI SDK in @iris-eval/sdk. Both are not yet published (the next iris-eval release on PyPI; @iris-eval/sdk is built from source until its first npm release). Streams, the SDKs' stream helpers and tool calls are covered, the original client is not changed, and Iris being down never breaks a call — packages/sdk/README.md, packages/python/README.md.
Score every LangChain and LangGraph run
from iris_eval.langchain import IrisCallbackHandler
graph.invoke(inputs, config={"callbacks": [IrisCallbackHandler(agent_name="support-bot")]})
Each top-level run becomes one trace (the run, its model calls, its tool calls and its graph nodes as GenAI spans) with its input, output, tool calls, token usage and a verdict. Python in the client (pip install iris-eval, 0.2.0 and later), JavaScript as @iris-eval/langchain (in this repository, not yet published to npm). Both are proven in CI against a real LangGraph app with a scripted model; LangSmith's own OpenTelemetry export is proven the same way — docs/otel-recipes.md.
A CI gate, no server needed
npx -y @iris-eval/mcp-server ingest --file traces.ndjson --evaluate --fail-on any
any fails the job on every verdict that is not a pass: a failure, and a verdict that was not checked. Name one basis instead (--fail-on detector_veto) to fail only on that one.
Or the GitHub Action (0.16.0), which fails the job on the verdicts you name, writes the receipt to the job summary and posts it as one pull-request comment updated in place: uses: iris-eval/mcp-server/.github/actions/gate@v0.21.0 with traces: traces.ndjson — docs/ci-gate.md.
A fourth door (0.15.0): POST /v1/traces on the dashboard port takes the OTLP/HTTP JSON or protobuf your OpenTelemetry instrumentation already emits (the Python SDK's exporter speaks protobuf only, so this is the Python door too), and each OTLP trace becomes an Iris trace with its spans — docs/otel-integration.md; one recipe per framework (Pydantic AI, Google ADK, LangGraph via LangSmith, CrewAI, the OpenAI Agents SDK in Python and JavaScript, LlamaIndex, AutoGen, Microsoft Agent Framework, Semantic Kernel, the Vercel AI SDK and Mastra), each proved by a fixture, in docs/otel-recipes.md. ingest reads one JSON trace (or NDJSON, one per line) from stdin or a file, stores it, evaluates it under exactly the rules evaluate_output runs, prints one JSON line per trace with the verdict and its basis, and exits 1 when a verdict matches --fail-on. --dataset <id|label> restricts that gate to the case keys in a dataset (POST /api/v1/datasets promotes a run's case keys into one), so a job fails only on the cases you chose. The full recipe, the exit codes and the eight bases are in docs/ci-gate.md.
Write a rule as code
eval.plugins in config.json loads rules you wrote — an ES module whose default export is { name, kind, mechanism, version, needs, evaluate(ctx) } — pinned by the sha256 of the file, so a file that changed since you pinned it refuses startup rather than running. A loaded plugin fires like a built-in and shows on list_rules under plugins. The contract, the hash recipe and what a plugin may return: docs/plugins.md.
Use the engine in your own process
The evaluation engine is importable — no server, no database, no model:
import { EvalEngine, defaultConfig } from '@iris-eval/mcp-server/engine';
const engine = new EvalEngine(defaultConfig.eval.defaultThreshold, defaultConfig.eval.ruleThresholds, defaultConfig.eval);
const result = await engine.evaluateAll({ output: answer, input: prompt, toolCalls, costUsd });
result.verdict.state; // 'pass' | 'fail' | 'unknown', with result.verdict.basis and result.interpretations
The same engine, the same rules and the same composer the server runs; builtInRules(), createCustomRule(), compose() and the published-accuracy readers are exported beside it.
A typed client for the HTTP route
import { createClient } from '@iris-eval/mcp-server/client';
const iris = createClient({ baseUrl: 'http://127.0.0.1:6920', apiKey: process.env.IRIS_API_KEY });
const { trace_id, evaluation } = await iris.logTrace({ agent_name: 'support-bot', input, output, tool_calls, evaluate: true });
evaluation?.verdict?.state; // the same object evaluate_output returns
One body on every door: it is what log_trace and iris-eval ingest accept. A refusal throws IrisClientError with the server's own sentence and status. Both subpaths are checked from a packed tarball on every build.
Verify your install
npx @iris-eval/mcp-server --self-test # offline diagnostic; exit 0 = healthy, 1 = a check failed
npx @iris-eval/mcp-server --version # prints the bare version, e.g. 1.2.3
--self-test first creates your Iris home if it is missing and checks that it is writable (exit 1, naming the path, if it is not), reports where your database's search index is (whole, how many traces a background build has indexed so far, or no FTS5 on this SQLite), reads your database's schema (exit 1, with the fix, when this version or an MCP client pinned to an older release cannot open it), then runs its checks — storage round-trip, a planted SSN and a planted injection caught by the safety rules, dashboard boot, the DNS-rebinding guard — inside an isolated temp home. Your real database is only read, never changed. Everything Iris writes lives under one directory, your Iris home: ~/.iris by default (%USERPROFILE%\.iris on Windows), or wherever IRIS_HOME points. That is where iris.db, config.json, custom-rules.json, audit.log, preferences.json and the demo files live; point IRIS_HOME at a scratch directory to try Iris without touching your real data.
| Client | Status | What that means | Read |
|---|---|---|---|
| Claude Code | verified | a test drives the real client on every CI run | 2026-09-25 |
| Claude Desktop | claimed | configured from the client's own MCP documentation; the installer's path and format for it are unit-tested | 2026-09-25 |
| Cursor | claimed | configured from the client's own MCP documentation; the installer's path and format for it are unit-tested | 2026-09-25 |
| Devin Desktop (Windsurf) | claimed | configured from the client's own MCP documentation; the installer's path and format for it are unit-tested | 2026-09-25 |
| Continue | claimed | configured from the client's own MCP documentation; the installer's path and format for it are unit-tested | 2026-09-25 |
| VS Code | claimed | configured from the client's own MCP documentation; the installer's path and format for it are unit-tested | 2026-09-25 |
| Cline | claimed | configured from the client's own MCP documentation; the installer's path and format for it are unit-tested | 2026-09-25 |
| Zed | claimed | configured from the client's own MCP documentation; the installer's path and format for it are unit-tested | 2026-09-25 |
| OpenAI Codex CLI | claimed | configured from the client's own MCP documentation; the installer's path and format for it are unit-tested | 2026-09-25 |
| Gemini CLI | verified | a test drives the real client on every CI run | 2026-09-25 |
Every row with what was checked: iris-eval.com/clients. No client is called supported without a row.
npx -y @iris-eval/mcp-server install <client> writes each of these for you. By hand, per client:
Claude Desktop
Edit your MCP config file:
- macOS:
~/Library/Application Support/Claude/claude_desktop_config.json - Windows:
%APPDATA%\Claude\claude_desktop_config.json
Add the JSON config above, then restart Claude Desktop.
Claude Code
claude mcp add --transport stdio iris-eval -- npx -y @iris-eval/mcp-server
Then restart the session (/clear or relaunch) for tools to load.
Windows note: Do not use
cmd /cwrapper — it causes path parsing issues. Thenpxcommand works directly.
Cursor
Add the JSON config above to ~/.cursor/mcp.json (every project) or .cursor/mcp.json in a workspace, with "type": "stdio" in the iris-eval entry — Cursor's docs mark it required.
Devin Desktop (Windsurf)
Add the JSON config above to mcp_config.json: ~/.config/devin/mcp_config.json on macOS and Linux, %APPDATA%\devin\mcp_config.json on Windows.
Continue
Save the JSON config above as its own file in Continue's mcpServers folder: ~/.continue/mcpServers/iris-eval.json (every workspace) or .continue/mcpServers/iris-eval.json in one.
VS Code (native MCP)
Add to .vscode/mcp.json in your workspace (note: VS Code uses servers, not mcpServers):
{
"servers": {
"iris-eval": {
"command": "npx",
"args": ["-y", "@iris-eval/mcp-server"]
}
}
}
Cline
Open Cline's MCP Servers panel → Configure MCP Servers, and add the mcpServers JSON config above to cline_mcp_settings.json (~/.cline/data/settings/cline_mcp_settings.json, shared by Cline in VS Code, JetBrains and the CLI).
Zed
Add to Zed settings.json:
{
"context_servers": {
"iris-eval": {
"command": "npx",
"args": ["-y", "@iris-eval/mcp-server"],
"env": {}
}
}
}
OpenAI Codex CLI
Add to ~/.codex/config.toml:
[mcp_servers.iris-eval]
command = "npx"
args = ["-y", "@iris-eval/mcp-server"]
Gemini CLI
Add the mcpServers JSON config above to ~/.gemini/settings.json. Gemini CLI connects to MCP servers only in folders it trusts: if gemini mcp list shows iris-eval as Disabled, run /permissions in that folder.
Anything else that speaks MCP
Iris is a standard stdio MCP server — one npx @iris-eval/mcp-server command, no SDK, no code changes. If your client supports MCP, it supports Iris. Client config formats change; when in doubt, check your client's MCP docs and point it at that command.
Other Install Methods
# Global install (recommended for persistent data and faster startup)
npm install -g @iris-eval/mcp-server
iris-eval --dashboard
# Docker — two servers, two ports: 3000 = MCP HTTP transport,
# 6920 = dashboard (which also serves the POST /api/v1/traces ingest endpoint).
Iris MCPのよくある質問
Iris MCPをClaude Codeに追加するには
ターミナルでclaude mcp add --env 'IRIS_ANTHROPIC_API_KEY=<YOUR_IRIS_ANTHROPIC_API_KEY>' --env 'IRIS_API_KEY=<YOUR_IRIS_API_KEY>' --env 'IRIS_OPENAI_API_KEY=<YOUR_IRIS_OPENAI_API_KEY>' --env 'IRIS_WEBHOOK_SECRET=<YOUR_IRIS_WEBHOOK_SECRET>' mcp-server -- npx -y @iris-eval/mcp-serverを実行します。すべてのプロジェクトで利用するには--scope userを追加します。
Iris MCPをCursorに追加するには
~/.cursor/mcp.jsonを開き、上記のインストールセクションに示すCursor設定を使ってmcpServersの下にサーバーを追加します。
Iris MCPが提供するツールは
17個のツールを提供しており、log_trace, evaluate_output, get_traces, list_rules, deploy_ruleなどがあります。
Irisの動作にアカウントやLLMは必要ですか?
いいえ、アカウントやSDKなしでローカルマシン上で実行できます。LLMの呼び出しを必要としない28個の組み込み決定論的ルールを使用します。
データはどこに保存されますか?
エージェントの実行結果は、ディスク上のローカルなSQLiteデータベースに保存されます。
モデルがツール呼び出しを選択しなかった場合でも、エージェントのトレースをキャプチャできますか?
HTTP ingestエンドポイントを使用してプレーンなHTTP経由でトレースを送信するか、専用のClaude Codeキャプチャプラグインを使用できます。
データはクラウドに送信されますか?
OpenTelemetryエンドポイント、LLM judge、またはwebhookを明示的に有効にしない限り、デフォルトではデータがマシンから外部へ送信されることはありません。