agentflight MCP server

AgentFlight (agentflight)

AI Agent Black-Box Flight Recorder, Deterministic Trajectory Replay (<0.02 ms, $0.00 CI Spend), Differential Regression Detector, OpenTelemetry / W3C TraceContext Exporter (for stackpulse), Real-Time Loop/Cost Circuit Breaker & 10-Rule MCP Tool-Smell Linter.

🚀 Live Interactive Developer Portal & Playground: Visit https://agentflight-tau.vercel.app in your web browser to test tool linting live, inspect schemas, and copy 1-click configurations for Claude Code, Cursor, Windsurf, and Claude Desktop.

⭐ Like what you see? Give this repository a Star and follow creator Akandwanaho Ivan Alvin!

---

Why AgentFlight Exists (The 2026 Agent Reliability Pain Point)

Production AI agent engineering in 2026 faces two compounding bottlenecks:

1. Non-Deterministic, Expensive Multi-Step Agent Debugging & CI Testing: When a 15-step tool-calling or MCP agent fails on step 12, re-running live LLM turns and external API calls in CI is slow (10–45s), flaky, and expensive. Engineers need a "VCR / Black-Box Flight Recorder for AI Agents" that captures every LLM turn, MCP RPC, and tool invocation into a tamper-evident SHA-256 hash chain cassette and replays the entire trajectory deterministically in <0.02 ms with $0.00 token spend—raising a structural diff (TrajectoryDivergenceError) the instant an agent's behavior drifts from the golden baseline.
2. "MCP Tool Smell" (Schema Ergonomics Bugs): Empirical evaluations across Model Context Protocol (MCP) servers and function-calling suites show that most tool-selection failures are caused not by model reasoning limits, but by poorly authored tool schemas: missing parameter descriptions, ambiguous boolean flags (flag, mode), deeply nested input objects (depth > 3), unconstrained finite-set strings (status, format without enum), overlapping tool names (search_records vs. find_records), and destructive tools (delete_*, drop_*) without confirmation guards.

agentflight solves both problems with a pure-Python 3.10+ stdlib core engine (src/agentflight/ uses zero third-party runtime dependencies), paired with a FastAPI/ASGI portal gateway.

---

Architecture Blueprint

+-----------------------------------------------------------------------------------+
|                                   agentflight                                     |
+-----------------------------------------------------------------------------------+
|                                                                                   |
|  1. Pre-Deploy Static Gate                2. Runtime Execution & Guard            |
|  +----------------------------------+     +------------------------------------+  |
|  | ToolSmellLinter (linter.py)      |     | TrajectoryCircuitBreaker (guard.py)|  |
|  | - 10 Deterministic Rules         |     | - Exact Call Loop (N repeats)      |  |
|  |   (TS001 .. TS010)               |     | - Ping-Pong Cycle (A->B->A->B,     |  |
|  | - Agent Ergonomics Score (0-100) |     |   A->B->C->A->B->C)                |  |
|  | - Token Footprint Estimator      |     | - Consecutive Tool Error Storm     |  |
|  | - Actionable Remediations        |     | - Hard Token & USD Budget Caps     |  |
|  +-----------------+----------------+     +-----------------+------------------+  |
|                    |                                        |                     |
|                    v                                        v                     |
|  3. Tamper-Evident Flight Recorder        4. CI Replay & Golden Diff Engine       |
|  +----------------------------------+     +------------------------------------+  |
|  | FlightRecorder (recorder.py)     |     | CassetteReplayer & Evaluator       |  |
|  | - Append-only TrajectorySpan     | --> | - Sub-ms $0.00 Deterministic Stub  |  |
|  | - Canonical JSON SHA-256 Chain   |     | - Structural Diff on Divergence    |  |
|  |   (prev_hash -> step_hash)       |     | - Tool Precision / Recall / F1     |  |
|  | - .flight.json & .flight.jsonl   |     | - Step Efficiency & Cost Delta     |  |
|  | - verify_cassette() Tamper Check |     | - Goal Completion Score (0.0-1.0)  |  |
|  +----------------------------------+     +------------------------------------+  |
+-----------------------------------------------------------------------------------+

---

Installation

# Install from PyPI (module import & CLI command remain `agentflight`):
pip install agentflight-mcp

# Or install v1.4.0 directly from GitHub:
pip install "git+https://github.com/codesphere-u/agentflight.git@v1.4.0"

# Contributors:
pip install -e ".[dev]"

---

Quickstart

0. 1-Line Drop-In SDK & Pytest Integration (v1.3.0)

Wrap OpenAI, AsyncOpenAI, OpenRouter, vLLM, OpenAny, or Anthropic clients with zero boilerplate:

from openai import OpenAI
from agentflight import wrap_openai, wrap_anthropic, TrajectoryCircuitBreaker

client = wrap_openai(OpenAI(), guard=TrajectoryCircuitBreaker(max_cost_usd=0.25))
resp = client.chat.completions.create(model="gpt-4o-mini", messages=[{"role": "user", "content": "Health?"}])
client.agentflight_recorder.save("tests/cassettes/health.flight.json")

Pytest fixtures (flight_recorder, flight_guard, flight_session) and @pytest.mark.flight_cassette register automatically on pip install agentflight-mcp:

import pytest

@pytest.mark.flight_cassette("tests/cassettes/checkout.flight.json")
def test_checkout_deterministic(flight_session, flight_guard):
    search = flight_session.stub("search_catalog", fallback_fn=lambda q: {"hits": [q]})
    assert search(q="router")["hits"] == ["router"]

1. Record & Cryptographically Verify an Agent Trajectory (FlightRecorder)

from agentflight import FlightRecorder, CassetteReplayer, verify_cassette

recorder = FlightRecorder(agent_id="sre-incident-agent", metadata={"env": "prod"})

recorder.record(
    kind="llm_turn",
    actor="planner_model",
    input_payload={"goal": "Investigate payment-api p99 latency spike"},
    output_payload={"next_tool": "prometheus.query_metrics"},
    latency_ms=410.0,
    prompt_tokens=580,
    completion_tokens=90,
    cost_usd=0.0022,
)
recorder.record(
    kind="mcp_rpc",
    actor="prometheus.query_metrics",
    input_payload={"service": "payment-api", "window": "15m"},
    output_payload={"p99_latency_ms": 1840.0, "anomaly": True},
    latency_ms=85.0,
)

# Persist as tamper-evident .flight.json or .flight.jsonl
recorder.save("cassettes/incident_8841.flight.json")
assert verify_cassette(recorder) is True

2. Replay Deterministically in <0.02 ms with $0.00 Spend (CassetteReplayer)

replayer = CassetteReplayer("cassettes/incident_8841.flight.json", strict_order=True)

# Stub any MCP or tool callable directly in pytest:
query_metrics_stub = replayer.stub_tool("prometheus.query_metrics", kind="mcp_rpc")

# Replaying matching steps returns recorded payloads in ~2.8 microseconds:
replayer.replay_step("llm_turn", "planner_model", {"goal": "Investigate payment-api p99 latency spike"})
result = query_metrics_stub(window="15m", service="payment-api")  # Key-order invariant!
assert result["anomaly"] is True

If a code change causes the agent to call the wrong tool or pass altered parameters, CassetteReplayer immediately raises TrajectoryDivergenceError with missing_keys, extra_keys, and changed_values.

3. Lint MCP Tool Schemas Across 10 Ergonomics Rules (ToolSmellLinter)

from agentflight import ToolSmellLinter, SHOWCASE_FLAWED_MCP_TOOLS

linter = ToolSmellLinter()
report = linter.lint(SHOWCASE_FLAWED_MCP_TOOLS)

print(f"Ergonomics Score: {report.ergonomics_score}/100 (Grade {report.grade})")
for finding in report.findings:
    print(f"[{finding.rule_id}] {finding.tool_name}: {finding.message}")
    print(f"  Fix -> {finding.remediation}")

4. Guard Live Agents Against Loops, Ping-Pong Cycles & Runaway Spend (TrajectoryCircuitBreaker)

from agentflight import TrajectoryCircuitBreaker, CircuitBreakerTripped

breaker = TrajectoryCircuitBreaker(
    max_identical_repeats=3,
    max_ping_pong_cycles=2,
    max_consecutive_errors=3,
    max_total_tokens=25_000,
    max_cost_usd=0.50,
)

try:
    breaker.observe("search_logs", {"query": "timeout"})
    breaker.observe("refine_query", {"query": "timeout"})
    breaker.observe("search_logs", {"query": "timeout"})
    breaker.observe("refine_query", {"query": "timeout"})  # Trips on A -> B -> A -> B!
except CircuitBreakerTripped as exc:
    print(exc.reason, exc.step_index, exc.evidence)
    # -> ping_pong_oscillation 3 {'period': 2, 'cycles': 2, 'pattern_actors': ['search_logs', 'refine_query']}

5. Compare Candidate vs. Golden Trajectories (compare_cassettes & TrajectoryEvaluator)

from agentflight import compare_cassettes, diff_trajectories

# v1.1.0 Trajectory Differential Analyzer (Levenshtein/LCS sequence diff + loop/error/cost regression check)
report = compare_cassettes("cassettes/baseline.flight.jsonl", "cassettes/candidate.flight.jsonl")
print(report.regression_detected, report.similarity_score, report.added_tools, report.cost_delta_pct)
print(report.summary_markdown)

# Golden Evaluation Metrics (Precision, Recall, F1, Step Efficiency)
diff = diff_trajectories(golden_recorder, candidate_recorder)
print(diff.tool_precision, diff.tool_recall, diff.step_efficiency_ratio, diff.goal_completion_score)

6. Export W3C TraceContext / OpenTelemetry Spans for stackpulse (export_cassette_to_otel)

from agentflight import export_cassette_to_otel

spans = export_cassette_to_otel(
    "cassettes/incident_8841.flight.json",
    service_name="sre-incident-agent",
    output_path="spans.json",
)
# Directly ingestible by `codesphere-u/stackpulse` for critical-path DAG analysis and SLO burn-rate monitoring!

---

MCP Tool-Smell Rule Catalog (TS001–TS010)

Rule ID	Severity	Penalty	Rule Title & Ergonomic Failure Mode
TS001	ERROR	-12	Missing or Ultra-Short Tool Description (<15 chars): Leaves the LLM guessing when to invoke the tool.
TS002	WARNING	-5	Property Missing description: A parameter inside inputSchema.properties has no semantic description or format hint.
TS003	WARNING	-8	Deeply Nested Input Schema (depth > 3): Deep object nesting degrades JSON argument serialization accuracy on smaller/faster models.
TS004	WARNING	-7	Excessive Top-Level Parameter Count (> 8): Increases parameter hallucination and omission rates.
TS005	WARNING	-5	Ambiguous Boolean Flag Name: Boolean named flag, mode, option, data, value, switch, or toggle without clear polarity.
TS006	WARNING	-6	Unconstrained Finite-Set String Parameter: Parameter named status, sort, format, env, role, type, or priority lacks enum or pattern.
TS007	WARNING	-6	Missing required Array on Non-Empty Schema: Causes models to omit mandatory arguments.
TS008	ERROR	-10	High Lexical Overlap Between Tools (cosine >= 0.72): Overlapping tool names/descriptions in the same suite cause nondeterministic routing confusion.
TS009	WARNING	-7	Bloated Schema Token Footprint (> 450 tokens): Excessive schema verbosity inflates prompt cost on every agent turn.
TS010	ERROR	-10	Destructive Tool Missing Safety Guard: Tool prefixed with delete_, drop_, remove_, transfer_, or exec_ lacks a confirm/dry_run/idempotency_key parameter or irreversibility warning.

---

Measured Benchmark Results

Reproducible via python -m agentflight.cli benchmark (measured on Python 3.12):

Subsystem Operation	Workload Size	Total Latency (ms)	Per-Unit Latency	Throughput
FlightRecorder.record (SHA-256 chain)	1,000 spans	7.42 ms	7.42 us/span	134,770 spans/s
verify_cassette (Tamper Verification)	1,000 spans	3.40 ms	3.40 us/span	294,117 spans/s
CassetteReplayer.replay_step ($0 Stub)	1,000 spans	2.83 ms	2.83 us/span	353,356 spans/s
TrajectoryCircuitBreaker.observe_span	1,000 spans	3.01 ms	3.01 us/span	332,225 spans/s
ToolSmellLinter.lint (10 Rules + TS008)	200 MCP tools	119.27 ms	596.35 us/tool	1,676 tools/s
TrajectoryEvaluator.diff_trajectories	1,000 vs 1,000 spans	4.06 ms	4.06 us/span	246,305 spans/s

---

CLI Reference

# 1. Lint showcase flawed vs. hardened MCP tool suites (or pass --schema-file path/to/tools.json)
agentflight lint

# 2. Automated Schema Repair: harden smelly tools to Glama TDQS Gold Standard
agentflight fix tools.json --output hardened_tools.json

# 3. CI/CD Pre-Flight Gate: lint tool smells, verify cassette integrity, enforce budget SLAs
agentflight ci --schema-file tools.json --cassette runs/test.cassette.json --fail-on-smell

# 4. Forensic Post-Mortem Investigator: generate AAIB/NTSB incident reports from flight cassettes
agentflight post-mortem crash_incident.cassette.json --output report.md

# 5. Run the end-to-end recording, <0.02ms replay, circuit breaker & diff demo
agentflight demo

# 6. Run the reproducible subsystem benchmark suite
agentflight benchmark

# 7. Compare baseline vs. candidate cassettes for behavioral regressions
agentflight diff baseline.flight.json candidate.flight.json --json

# 8. Export cassette to W3C TraceContext / OpenTelemetry JSON spans for StackPulse
agentflight export-otel incident_8841.flight.json --output spans.json

# 9. Start the JSON-RPC 2.0 MCP Server over stdio
agentflight serve-mcp

# 10. Run offline self-diagnostics (stdlib core, pytest11, redaction, SHA-256 chain, guard)
agentflight doctor --json

---

Deterministic Testing with @flight_test (VCR for AI Agents)

Eliminate non-determinism, API cost, and network latency from your CI test suite:

import pytest
from agentflight.testing import flight_test

@flight_test("tests/fixtures/checkout.cassette.json")
def test_checkout_agent(flight):
    # During first run: executes live and records cryptographic cassette.
    # During subsequent runs (e.g. CI): replays in <3 μs with $0.00 network cost!
    agent = OrderAgent()
    result = agent.execute("Order item sku_881 with coupon SAVE10")
    
    assert result.status == "confirmed"
    assert flight.step_count == 3
    assert flight.total_cost_usd <= 0.05

---

MCP Tools Reference & Usage Guide

AgentFlight provides 4 specialized MCP tools for AI coding agents and LLM tool designers:

Tool Name	What is its use? (Purpose)	Key Arguments	Hosted on Vercel?	Typical Output
agentflight_lint_tools	Statically analyzes MCP tool definitions against 10 ergonomics rules (TS001–TS010), flags schema anti-patterns, and computes a 0–100 Agent Ergonomics Score.	tools (list of tool schema dicts)	Yes (Public)	{"overall_score": 88.5, "passed": true, "findings": [...]}
agentflight_diff_cassettes	Compares a baseline .flight.json cassette against a candidate cassette across model or prompt upgrades to detect tool sequence divergence, new errors, and cost/token drift.	baseline_path (str), candidate_path (str), similarity_threshold (float)	Local stdio	{"verdict": "PASSED", "similarity_score": 0.95, "cost_delta_usd": 0.00}
agentflight_export_otel	Converts recorded agent trajectories into W3C TraceContext / OpenTelemetry JSON spans for critical-path DAG analysis and SLO monitoring.	cassette_path (str), output_path (str), service_name (str)	Local stdio	{"spans_count": 14, "trace_id": "...", "exported": true}
agentflight_verify_cassette	Verifies the cryptographic SHA-256 hash chain of a .flight.json cassette to guarantee zero tampering or silent step mutations.	cassette_path (str)	Local stdio	{"verified": true, "step_count": 8, "merkle_root": "..."}

Tool Usage Examples

1. agentflight_lint_tools — Auditing MCP Tool Ergonomics Before Deployment

AI agent frameworks often suffer from confusing tool descriptions, ambiguous parameters, or overlapping tools. Call agentflight_lint_tools to validate your schemas before registering them:

{
  "jsonrpc": "2.0",
  "id": 1,
  "method": "tools/call",
  "params": {
    "name": "agentflight_lint_tools",
    "arguments": {
      "tools": [
        {
          "name": "search_database",
          "description": "Searches customer database by email address or user ID. Returns matched records.",
          "inputSchema": {
            "type": "object",
            "properties": {
              "query": { "type": "string", "description": "Email or user ID string." }
            },
            "required": ["query"]
          }
        }
      ]
    }
  }
}

2. agentflight_diff_cassettes — Detecting Agent Behavioral Regressions in CI

When updating prompts or upgrading from GPT-4o to Claude 3.5 Sonnet, verify that the new agent's tool execution trajectory does not deviate or loop:

{
  "jsonrpc": "2.0",
  "id": 2,
  "method": "tools/call",
  "params": {
    "name": "agentflight_diff_cassettes",
    "arguments": {
      "baseline_path": "fixtures/golden_deploy_workflow.flight.json",
      "candidate_path": "test_runs/candidate_deploy.flight.json",
      "similarity_threshold": 0.85,
      "max_cost_increase_pct": 20.0
    }
  }
}

Exposed MCP Resources (resources/list, resources/read)

Resource URI	MIME Type	Purpose / Content Description
agentflight://rules/catalog	application/json	The complete 10-rule ergonomics catalog (TS001–TS010) detailing antipattern rules, penalty weights, and severities.
agentflight://metrics/summary	application/json	Verified execution benchmark snapshot (sub-0.02ms replay latency, 1,676 tools/sec throughput, circuit-breaker overhead).

Exposed MCP Prompts (prompts/list, prompts/get)

Prompt Name	Purpose / Use Case	Required Arguments	Workflow Output
lint-tool-definitions	Guides an LLM agent to audit, critique, and refactor flawed MCP tool definitions into high-ergonomics schemas.	tools_json (string)	Pre-formatted audit prompt instructing the model to check for ambiguous flags, missing descriptions, and collision risks.
review-agent-trajectory	Guides an LLM agent through a structured post-mortem of an agent trajectory log to diagnose loops, errors, and cost spikes.	trajectory_json (string)	Pre-formatted diagnostic prompt breaking down execution phases, tool calls, and failure boundaries.

---

Architectural Decision Records (ADRs)

ADR-001: Canonical JSON SHA-256 Chaining for Trajectory Cassettes

• Context: Golden test fixtures for multi-step agents frequently suffer from dictionary key-order drift across Python/JSON runtimes and silent fixture edits.
• Decision: Every TrajectorySpan computes input_hash over json.dumps(..., sort_keys=True, separators=(",", ":")) and chains step_hash = SHA256(step_index || kind || actor || input_payload || output_payload || status || prev_hash).
• Consequence: Reordered keyword arguments match deterministically during replay while any mutation to recorded inputs, outputs, or step ordering triggers CassetteTamperError.

ADR-002: Zero-Dependency Static Ergonomics Linting (TS001–TS010)

• Context: Calling an LLM judge to critique MCP tool definitions is slow, non-deterministic in CI, and requires API credentials.
• Decision: Implement a fast, deterministic 10-rule static analyzer combining JSON Schema AST traversal with tokenized bag-of-words cosine similarity (TS008).
• Consequence: 200 MCP tool definitions are linted in ~119 ms (1,676 tools/sec) with zero external dependencies.

---

Hosted endpoint (Vercel)

A free, public, stateless deployment of AgentFlight runs over MCP Streamable HTTP:

MCP endpoint	https://agentflight-tau.vercel.app/mcp (JSON-RPC 2.0 over POST)
Health probe	https://agentflight-tau.vercel.app/health

curl -s -X POST https://agentflight-tau.vercel.app/mcp -H 'Content-Type: application/json' \
  -d '{"jsonrpc":"2.0","id":1,"method":"tools/list"}'

Client Configuration

1. Claude Code CLI (Recommended)

# Add to current project:
claude mcp add --transport http agentflight https://agentflight-tau.vercel.app/mcp

# Or add globally across all projects:
claude mcp add --scope user --transport http agentflight https://agentflight-tau.vercel.app/mcp

2. Claude Desktop / Cursor / Windsurf (mcp.json)

{
  "mcpServers": {
    "agentflight": {
      "type": "http",
      "url": "https://agentflight-tau.vercel.app/mcp"
    }
  }
}

3. Smithery Registry (1-Click Install)

npx -y @smithery/cli install @codesphere-u/agentflight --client claude