agent-observability MCP server
✓ VerifiedTrace AI agent execution by monitoring tool calls, errors, token usage, and costs through a local-first dashboard.
Also known as KryptosAI/agent-observability, agent-obs
- Stars
- 2
- Tools
- 6
- Downloads, 30 days
- —
- Latest version
- 1.0.7
- Last commit
- Jul 27, 2026
- License
- MIT
What is agent-observability MCP?
agent-observability is an MCP server. Trace AI agent execution by monitoring tool calls, errors, token usage, and costs through a local-first dashboard. It exposes 6 tools, such as obs_record_tool_call, obs_record_token_usage, obs_record_decision.
Summary generated from the project's README
About agent-observability MCP server
Monitoring AI agent execution through a local dashboard is what agent-observability enables for developers using tools like Claude Code, Cursor, or opencode. It provides a way to trace tool calls, monitor token usage, and track costs without sending any data to the cloud. All information is stored in a local SQLite database on your machine, ensuring privacy while offering a clear audit trail of every action taken by your agent.
Users can set up the system with a single command that auto-detects their agent and configures the necessary MCP settings. The dashboard, accessible via localhost, displays session grades from A to F based on efficiency, along with detailed breakdowns of tool durations, statuses, and specific reasoning behind agent decisions. For agents that do not support self-reporting, a proxy mode is available to intercept and log MCP tool calls automatically.
This tool helps developers understand the exact cost of a session in both tokens and dollars. By using tools like obs_record_tool_call and obs_record_token_usage, the server captures a complete history of what was read, edited, or executed during a task.
What you can do with it
- 1Analyze agent efficiency and costsA developer can review a session to see exactly how many tokens were consumed and the estimated dollar cost for a specific task.
- 2Debug failed agent tasksYou can investigate why an agent failed to complete a task by examining the full audit trail of tool calls and error statuses.
- 3Understand agent reasoningAn AI client can record the logic behind choosing one specific action over several other possible options.
- 4Monitor tool call performanceDevelopers can track the duration and success rate of specific tool invocations to identify bottlenecks.
Written from the project's README. Every use case is checked against a quote from it.
How to install agent-observability MCP
Formats verified 10 Nov 2026Add to CursorAdd to GooseInstall agent-observability MCP in Claude Code
$ claude mcp add agent-observability -- npx -y agent-obsInstall agent-observability MCP in Cursor
~/.cursor/mcp.json
{
"mcpServers": {
"agent-observability": {
"command": "npx",
"args": [
"-y",
"agent-obs"
]
}
}
}Install agent-observability MCP in VS Code
.vscode/mcp.json
{
"servers": {
"agent-observability": {
"type": "stdio",
"command": "npx",
"args": [
"-y",
"agent-obs"
]
}
}
}Install agent-observability MCP in VS Code CLI
$ code --add-mcp '{"name":"agent-observability","type":"stdio","command":"npx","args":["-y","agent-obs"]}'Install agent-observability MCP in Claude Desktop
claude_desktop_config.json
{
"mcpServers": {
"agent-observability": {
"command": "npx",
"args": [
"-y",
"agent-obs"
]
}
}
}Install agent-observability MCP in Codex
$ codex mcp add agent-observability -- npx -y agent-obsInstall agent-observability MCP in Codex config.toml
~/.codex/config.toml
[mcp_servers.agent-observability] command = "npx" args = ["-y", "agent-obs"]
Install agent-observability MCP in Gemini CLI
$ gemini mcp add -s user agent-observability npx -- -y agent-obsInstall agent-observability MCP in Gemini settings.json
~/.gemini/settings.json
{
"mcpServers": {
"agent-observability": {
"command": "npx",
"args": [
"-y",
"agent-obs"
]
}
}
}agent-observability MCP tools 6
Detected from the READMEobs_record_tool_callobs_record_token_usageobs_record_decisionobs_record_gradeobs_get_session_reportobs_list_sessions
Automated checks
Ran Nov 15, 2026Full report →Passed our automated checks. That is not a security audit: read the code and give the server the narrowest access it needs. How checks work
- Source repository linked and reachablekryptosai/agent-observability is public and reachablegithub.com/kryptosai/agent-observability ↗Pass
- Package names this repository as its sourceagent-obs names kryptosai/agent-observability as sourcenpmjs.com/package/agent-obs ↗Pass
- Build provenance (Sigstore attestation)No build attestation published for this releaseNot checked
- Open source license (OSI)MITspdx.org/licenses/MIT.html ↗Pass
- Commit or release in the last yearLast commit or release 2026-07-27github.com/kryptosai/agent-observability ↗Pass
- OpenSSF ScorecardNot checked
- Package name not close to a popular packageNo popular package one edit away from agent-obsPass
- No critical advisory in the package or its dependencies109 resolved dependencies checked; no critical advisoryosv.dev ↗Pass
- No hidden instructions to the model6 tool descriptions and the README read; no override phrases, credential paths or hidden charactersPass
- Destructive tools are labelledNo tool named like delete, drop or execPass
- Remote endpoint answers the MCP handshakeNot checked
README
View on GitHubagent-observability
Open source agent observability — see what your agents did, why they failed, and what it cost. Runs locally. No cloud required.
Quick Start
One command. No manual config. No copy-paste.
npx agent-obs@latest setup
This auto-detects your AI agent (opencode, Claude Code, Cursor) and configures everything:
- Adds agent-obs to your MCP config
- Adds the self-reporting instruction
- Tells you what to do next
Then restart your agent. Every action will now self-report.
agent-obs dashboard # http://localhost:9400 — see your sessions appear
What you'll see
After running a task in your agent, the dashboard shows:
- Every tool call with duration and status
- A-F session grade (how efficient was your agent?)
- Cost per session in tokens and dollars
- Full audit trail of what was read, edited, and executed
Manual setup (if setup can't detect your agent)
opencode: Add to opencode.json:
{
"mcp": {
"agent-obs": {
"command": "npx",
"args": ["-y", "agent-obs@latest", "server"],
"enabled": true
}
}
}
Claude Code: Create .mcp.json:
{
"mcpServers": {
"agent-obs": {
"command": "npx",
"args": ["-y", "agent-obs@latest", "server"]
}
}
}
Cursor: Add to Cursor Settings > MCP:
{
"mcpServers": {
"agent-obs": {
"command": "npx",
"args": ["-y", "agent-obs@latest", "server"]
}
}
}
See Agent-Specific Setup below for detailed per-platform instructions including self-reporting directives.
Note: All data is written to a local SQLite database in
~/.agent-observability/. No data leaves your machine.
Proxy mode (fallback)
For agents that can't self-report, wrap any MCP server and every tool invocation gets traced automatically:
agent-obs proxy --desc "fix login bug" -- npx @modelcontextprotocol/server-filesystem /tmp
Proxy mode captures only MCP tool calls (~30% of typical agent actions). Prefer the setup / MCP server approach above.
Agent-Specific Setup
Claude Code
- MCP Config — Create
.mcp.jsonin your project root:
{
"mcpServers": {
"agent-obs": {
"command": "npx",
"args": ["-y", "agent-obs@latest", "server"]
}
}
}
- Self-reporting instruction — Create
.claude/instructions.md(or reference the existing SKILL.md under.claude/skills/agent-obs/SKILL.md):
After every tool call, report to agent-obs's log_tool_call.
Start each task with start_session. End with end_session.
- Verify — Run
agent-obs dashboard, open http://localhost:9400, and look for your session after the agent completes a task.
Cursor
- MCP Config — Add to Cursor's MCP settings (
~/.cursor/mcp.json, or Settings → MCP → Add new global MCP server):
{
"mcpServers": {
"agent-obs": {
"command": "npx",
"args": ["-y", "agent-obs@latest", "server"]
}
}
}
- Self-reporting instruction — Create
.cursorrules(or.cursor/rules/agent-obs.md):
After every tool call, you MUST call the agent-obs MCP server's log_tool_call tool.
Include toolName, status (success/error), outputSummary, and durationMs.
Start each session with start_session and end with end_session.
- Verify — Run
agent-obs dashboard, open http://localhost:9400, and look for your session after the agent completes a task.
Architecture
┌──────────────────────────┐
│ agent-obs server │
│ (self-reporting) │
│ │
│ Agent calls obs_record_* │
│ tools directly via MCP │
│ • tool calls │
│ • token usage │
│ • decisions │
│ • grades │
└──────────┬───────────────┘
│
▼
┌──────────────────────────┐
│ agent-obs proxy │
│ (fallback — MCP-only) │
│ │
MCP Client ───────▶│ intercepts tool calls ──▶│ MCP Server
(Claude/Cursor) │ logs to SQLite │ (filesystem,
└──────────┬───────────────┘ github, etc.)
│
▼
┌──────────────────────────┐
│ ~/.agent-observability │
│ SQLite DB │
│ │
│ • sessions │
│ • tool_calls │
│ • token_usage │
│ • decisions │
│ • grades │
└──────────┬───────────────┘
│
▼
┌──────────────────────────┐
│ agent-obs dashboard │
│ (port 9400) │
│ │
│ GET /api/sessions │
│ GET /api/sessions/:id │
│ GET /api/search?q= │
│ GET /api/stats │
│ POST /api/export │
└──────────────────────────┘
API Reference
REST Endpoints (Dashboard)
| Method | Path | Description |
|---|---|---|
GET |
/api/sessions |
List all sessions. Query params: ?limit=20&offset=0&grade=B |
GET |
/api/sessions/:id |
Get full session detail with all tool calls, tokens, decisions, and grades |
GET |
/api/sessions/:id/tool-calls |
List tool calls for a session. Query params: ?status=error&server=filesystem |
GET |
/api/sessions/:id/tokens |
Get token usage history for a session |
GET |
/api/sessions/:id/decisions |
Get decision points for a session |
GET |
/api/search |
Full-text search across tool call inputs/outputs. Query param: ?q=read_file |
GET |
/api/stats |
Aggregate statistics: total sessions, avg grade, total tokens, total cost |
POST |
/api/export |
Export session data as JSON. Body: { "sessionIds": ["abc123"], "format": "json" } |
GET |
/api/health |
Health check. Returns { "status": "ok", "dbSize": "2.4MB", "sessionCount": 47 } |
MCP Tools (Server Mode)
| Tool | Parameters | Returns |
|---|---|---|
obs_record_tool_call |
sessionId, toolName, serverName, duration (ms), status (success/error), input, output |
{ "id": "call-uuid", "recorded": true } |
obs_record_token_usage |
sessionId, inputTokens, outputTokens, model |
{ "totalTokens": 1850, "estimatedCost": "$0.023" } |
obs_record_decision |
sessionId, context, options, chosen, reasoning |
{ "id": "decision-uuid", "recorded": true } |
obs_record_grade |
sessionId, grade (A/B/C/D/F), reasoning |
{ "grade": "B", "recorded": true } |
obs_get_session_report |
sessionId |
Full session JSON with all calls, tokens, decisions, grade |
obs_list_sessions |
limit, offset |
Array of { id, description, grade, createdAt, tokenTotal, estimatedCost } |
What Gets Tracked
Every tool call
- Tool name — e.g.
read_file,execute_command,search_code - Server — which MCP server handled it
- Duration — wall-clock milliseconds
- Status —
successorerror - Input — full arguments passed to the tool
- Output — full result returned (truncated at 64KB for storage)
Token consumption
- Input tokens — prompt and context
- Output tokens — generated response
- Total tokens — sum per session
- Per-call breakdown — token delta for each LLM round-trip
Cost estimation
Costs are estimated based on published API pricing:
| Model | Input (per 1M tokens) | Output (per 1M tokens) |
|---|---|---|
| Claude 3.5 Sonnet | $3.00 | $15.00 |
| Claude 3 Opus | $15.00 | $75.00 |
| GPT-4o | $2.50 | $10.00 |
| GPT-4 Turbo | $10.00 | $30.00 |
Costs are tracked per session and displayed in the dashboard. You can add custom model pricing via ~/.agent-observability/models.json.
Session grades
Every session receives a letter grade based on efficiency and correctness:
| Grade | Label | Criteria |
|---|---|---|
| A | Clean | Zero errors, minimal token waste, no unnecessary tool calls |
| B | Minor issues | Some inefficiencies, but no failures |
| C | Inefficient | Excessive token usage, redundant tool calls, recoverable errors |
| D | Risky | Significant problems — failed tool calls, high cost, wrong tools chosen |
| F | Failed | Errors prevented task completion, or agent abandoned the session |
Decision points
When the agent has multiple possible actions, it can record why it chose one over another. Example:
{
"context": "need to read a file — it could be at src/config.ts or lib/config.ts",
"options": ["read_file src/config.ts", "glob **/config.ts", "grep 'CONFIG' *.ts"],
"chosen": "glob **/config.ts",
"reasoning": "File might not exist at the expected path; glob guarantees finding it regardless of location"
}
Decision points create an audit trail of agent reasoning, making it possible to understand not just what happened, but why.
Full audit trail
Every action is timestamped and linked to a session. The audit trail answers:
- Who (which agent/user) accessed what data?
- When did each tool call happen?
- What data was read or modified?
- Was the action authorized?
Grade System
Grades are assigned manually by the agent via obs_record_grade or automatically by the dashboard based on session statistics:
- A = Clean — No issues detected. Every tool call succeeded, token usage was efficient relative to task complexity, and the session completed its stated goal.
- B = Minor issues — Some inefficiencies (e.g., reading the same file twice, calling a tool that returned empty results and then trying it again). No failures.
- C = Inefficient — Needs attention. Redundant tool calls, excessive token consumption, or recoverable errors that were eventually resolved.
- D = Risky — Significant problems. Multiple failed tool calls, unusually high cost for the task, or the agent chose demonstrably wrong tools.
- F = Failed — Errors occurred that prevented task completion, or the agent abandoned the session.
Grades accumulate over time. The /api/stats endpoint shows your average grade and grade distribution.
Why This Exists
AI agents are a black box. You tell Claude Code or Cursor to fix a bug, refactor a module, or add a feature — and minutes later you have a diff. But what actually happened in between? How many tool calls did it make? How many tokens did it burn? Which files did it read? Did it try three approaches before landing on one? You have no idea.
This isn't just academic curiosity. Without observability:
- You can't estimate costs. A session that feels quick might have burned $5 in API calls. A session that felt slow might have cost $0.50. You're operating blind.
- You can't debug failures. The agent produced broken code — was it because it read the wrong file? Used a deprecated API? Misunderstood the task? Without traces, you're guessing.
- You can't improve your prompts. Is the agent reading too many files? Calling too many tools? Wasting tokens on irrelevant context? You need data to tune your system instructions.
- You can't audit access. If an agent reads a file containing secrets or API keys, you should know about it. If it makes an unexpected network call, you need to see it.
Agent Observability gives you a flight recorder for every agent session. It answers: what happened, why it happened, and what it cost.
Provenance
This project combines ideas from three prior projects:
AgentShelf (Shopify AI readiness scanner)
AgentShelf scans a Shopify store's theme, apps, admin settings, and custom code to produce an AI readiness score. As it scans, it builds a full audit trail — every file read, every API call made, every finding logged with a timestamp and data source. Pattern adopted here: structured audit trail with per-action timestamps, data source attribution, and session-level summarization.
Veros (FHIR clinical ground truth)
Veros generates synthetic longitudinal patient records (FHIR R4) and uses them to validate clinical decision support agents. Its core principle: "no trace, no answer." Every snippet of clinical reasoning produced by an AI must be traceable back to the specific data element (lab result, medication, condition) that supports it. Pattern adopted here: decision point tracking — every agent choice must cite its basis, making reasoning auditable and contestable.
MCP Observatory
MCP Observatory secures MCP servers — testing them for vulnerabilities, schema drift, and attack surfaces before agents depend on them. It's used by 850+ developers weekly and powers CI pipelines for MCP server security. Use Observatory to secure your MCP servers. Use agent-obs to trace the agents that depend on them. Observatory validates; agent-obs observes.
Pattern adopted here: tool health scoring, session grading (A-F), and automatic degradation flags.
Open Source vs Cloud
This open source package (agent-observability on npm) is the core engine. It's MIT licensed and will always remain free and local-first:
- CLI tools (proxy, server, dashboard)
- Local SQLite storage
- REST API
- MCP server tools
- Full audit trail, cost estimation, and session grading
A cloud version (app.agentobservability.dev) is in development with additional features:
- Team dashboards with shared session history
- Alerting (Slack/email when an agent's grade drops below C)
- Historical analytics (cost trends, efficiency trends over months)
- Role-based access control for audit trails
- SOC 2 compliant infrastructure
The open source package will always be able to run independently. The cloud version is additive, not a replacement.
License
MIT
agent-observability MCP questions
How do I add agent-observability MCP to Claude Code?
Run claude mcp add agent-observability -- npx -y agent-obs in your terminal. Add --scope user to make it available in every project.
How do I add agent-observability MCP to Cursor?
Open ~/.cursor/mcp.json and add the server under mcpServers, using the Cursor config shown in the install section above.
What tools does agent-observability MCP provide?
It exposes 6 tools, including obs_record_tool_call, obs_record_token_usage, obs_record_decision, obs_record_grade, obs_get_session_report.
Does my data leave my machine?
No, all data is stored in a local SQLite database and no data leaves your machine.
How can I see my agent's sessions?
You can view your sessions by running the dashboard command, which hosts a local web interface.
What happens if my agent cannot self-report?
You can use Proxy mode to wrap any MCP server, which automatically traces every tool invocation.