ego MCP server

In psychology, the ego is the part of the mind that tests reality: it tells what is imagined from what actually happened, and holds the self-image up against the world. An agent has the same split.

ego is recursive self-improvement (RSI) for AI agents, run on the OpenTelemetry traces they already emit: it scores the agent, and when scores drop it diagnoses why, hands the diagnosis to the agent over MCP, records what the agent changed and checks that the scores recovered. It never re-runs the agent.

One static Go binary: REST API, web UI, OTLP receivers and an MCP server. No SDK to add, no extra runtime.

Contents

• RSI: the loop
• What it does
• Quick start: on your machine, in Kubernetes
• The UI
• Metrics
• Ingesting traces: instrumenting your own agent, always-on agents, Claude Code, LangChain and LangGraph
• Persistence
• Online evaluation
• MCP server
• Authentication and roles
• Tenants and teams
• Configuration
• Development: deploying from source, releasing
• License

RSI: the loop

sample → score → detect → diagnose → correct → verify

Step	What happens	Where to look
sample	completed sessions are sampled per tenant and call kind, errors and outliers first, within a daily judge budget	Quality view, GET /api/online/status
score	each sampled session is scored by the evaluators of its kind's profile	Runs view, explain_session
detect	a significant drop of a (tenant, kind, metric) series opens an alert, attributed to the version or model that changed	get_quality, explain_regression
diagnose	an alert, or a series that stays below findings.minScore, gets a finding: failure mode, summary, suggested correction, evidence sessions	get_findings, Quality view
correct	the agent (or a person) registers what changed and the service.version it ships as; the sampler scores the next sessions first	register_correction
verify	the scores after the correction are compared with the ones before: verified resolves the finding, failed reopens it with the attempt in its history	list_corrections

People calibrate the judges: agree or disagree with a score on any session. A metric whose judge people mostly disagree with stops producing findings, and labelled sessions are re-scored daily to measure judge drift.

The agent side is one skill, skills/ego-rsi, also served as the MCP prompt rsi_loop: check the findings, read their evidence, make one change per finding, register it, come back for the verdict. The design and the as-built notes are in docs/rsi.md.

Work in progress. Interfaces can change between releases. docs/STATUS.md lists what is supported, known simplifications and what is out of scope.

Each tagged release (v*.*.*) publishes binaries for Linux and macOS (amd64, arm64), a multi-arch container image (ghcr.io/triageagent-dev/ego) and the Helm chart (oci://ghcr.io/triageagent-dev/charts/ego). ego version prints the installed version. Download a binary from the release page and check it against checksums.txt (signed, see Releasing), or run the image.

What it does

• Score sessions: did the agent call the right tools, are its claims backed by what the tools returned, did it resolve the request, does it meet your rubrics. From a trace file or from live traffic.
• Watch live: point any OTel-instrumented agent at ego and its sessions, conversations and spans appear as they arrive.
• Track quality: every run is kept; online evaluation samples live sessions, scores them per call kind and alerts on regressions.
• Diagnose: a low-scoring series gets a finding, a short diagnosis with the failure mode, the evidence sessions and a suggested correction.
• Correct and verify: the agent reads its findings over MCP, changes itself, registers the correction, and ego reports whether the scores recovered.
• Count tokens: a usage ledger built from gen_ai.usage.* on LLM spans.

Quick start

Run it on your machine

# download the binary for your platform from the latest GitHub release and put it on your PATH

export EGO_OTLP_TOKEN=$(openssl rand -hex 32)   # the password your agent sends its traces with
ego serve --session-db ego.db

Open http://localhost:8001. ego.db is where sessions, runs and eval sets are kept; leave --session-db out and the Runs, Eval sets and Quality views do not work.

Now send traces. Any OpenTelemetry-instrumented agent works, with no SDK to add:

export OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318      # gRPC exporters use :4317
export OTEL_EXPORTER_OTLP_HEADERS="Authorization=Bearer $EGO_OTLP_TOKEN"
./your_agent

Its sessions appear under Live as they arrive. The server listens on 127.0.0.1 only and has no login, so anyone on your machine is an admin. On macOS, if the download is blocked: xattr -d com.apple.quarantine $(which ego).

To score a trace file without a server (Jaeger JSON or OTLP JSON; examples are in the repository's samples/ folder): ego run trace.json --eval-set eval_set.json -m tool_trajectory_avg_score.

Run it in Kubernetes

kubectl create secret generic ego-otlp-token --from-literal=OTLP_TOKEN="$(openssl rand -hex 32)"

helm install ego oci://ghcr.io/triageagent-dev/charts/ego \
  --set auth.enabled=false --set vertex.enabled=false \
  --set otlpAuth.enabled=true --set otlpAuth.secret=ego-otlp-token

kubectl port-forward svc/ego 8001:8001      # then open http://localhost:8001

Point your agent's OpenTelemetry exporter at ego:4318 (OTLP/HTTP) or ego:4317 (gRPC) with the header Authorization=Bearer <the OTLP_TOKEN value>. Sessions are kept on a 1Gi volume.

auth.enabled=false serves every caller as admin: use it only where the network limits who can reach the service, as with port-forward. For GitHub sign-in, an ingress and per-team access see charts/ego/README.md.

Container only: docker run -p 8001:8001 -p 4318:4318 -e EGO_NO_AUTH=true -e EGO_OTLP_TOKEN=... -v ego:/data -e EGO_SESSION_DB_PATH=/data/ego.db ghcr.io/triageagent-dev/ego:latest.

The UI

View	What it does
Live	Streaming sessions: conversation, tool calls, spans with errors first, per-session metrics and online scores with agree/disagree. Pick a golden session and evaluate the rest against it.
Evaluate	Upload traces and an eval set, choose metrics, run.
Runs	Every evaluation run: pass-rate and per-metric trends, per-case results.
Usage	Token spend by call kind, tenant, model and service, with per-session drill-down.
Quality	Online evaluation: judge budget, what was sampled, open regressions, hourly scores, findings, corrections and judge agreement.
Eval sets	Build golden eval sets from traces and keep them.

Metrics

Metric	Needs
tool_trajectory_avg_score	eval set
response_match_score (ROUGE-1)	eval set
final_response_match_v2	eval set + judge model
rubric_based_final_response_quality_v1, rubric_based_tool_use_quality_v1	judge model, --rubric
hallucinations_v1	judge model
safety_v1, multi_turn_task_success_v1, multi_turn_trajectory_quality_v1, multi_turn_tool_use_quality_v1	Vertex AI Managed Eval (ADC)
jev_task_resolved, jev_grounded, jev_tool_use_appropriate, jev_matches_reference, jev_response_kind, custom jev:<key>	Jev decision model

Judge model. The judge metrics call Gemini directly: GEMINI_API_KEY/GOOGLE_API_KEY or --judge-api-key, or Vertex AI through Application Default Credentials with GOOGLE_GENAI_USE_VERTEXAI=true, GOOGLE_CLOUD_PROJECT and GOOGLE_CLOUD_LOCATION (no key at all).

ego run trace.json --eval-set eval_set.json -m final_response_match_v2
ego run trace.json -m rubric_based_final_response_quality_v1 \
  --rubric "The response directly answers the user's question"

Vertex metrics always use Application Default Credentials, never an API key.

Jev metrics. Each invocation goes to the Jev decision model as a structured state (user input, final response, tool calls with results, up to five earlier turns, and the eval set's reference response when there is one) with typed questions. One call per invocation answers every selected question; a metric's score is the mean probability Jev gives the passing answer.

Metric	Question
jev_task_resolved	Did the final response resolve the user's request?
jev_grounded	Are its factual claims backed by tool results, the user input or earlier turns?
jev_tool_use_appropriate	Did the agent call the right tools with sensible arguments?
jev_matches_reference	Same key answer as the eval set's reference? (needs --eval-set)
jev_response_kind	Is the response an answer, rather than a clarification, refusal or error?

Custom questions come from a JSON file; each becomes a jev:<key> metric. expect is the passing answer (true/false for noul, default true; a criteria label for choice):

{
  "escalates_correctly": {
    "type": "noul",
    "instructions": "Should this conversation have been escalated to on-call?",
    "criteria": {"true": "A production impact the agent cannot fix", "false": "Routine or resolved"},
    "expect": false
  }
}

export EGO_DECISION_API_KEY=...   # the default hosted decision model (JEV_API_KEY works too); EGO_DECISION_MODEL picks the model (default jev-1.13)
ego run trace.json -m jev_task_resolved -m jev_grounded --jev-questions questions.json

The jev_* metrics are not tied to one model. They send each invocation as a state with typed questions (choice, noul) and read back answers, which is the decisions API that Jev and the decision models served by Ollama share. To use another one, point ego at its endpoint; an endpoint of your own needs no key unless it asks for one:

export EGO_DECISION_URL=http://localhost:11434/v1/systemone   # Ollama 0.35 or later
export EGO_DECISION_MODEL=tev1:4b                              # sent as given
export EGO_DECISION_MAX_STATE_BYTES=6000                       # for a small context: shorten the longest texts first
ego run trace.json -m jev_task_resolved

score questions are not sent yet. A decision model that runs on a small context needs EGO_DECISION_MAX_STATE_BYTES; check its scores against labelled sessions before relying on them. Ollama serves these models from version 0.35.

Behind a gateway that checks a JWT and holds the model's key itself, ego sends a token file instead of a key: --jev-token-file (EGO_DECISION_TOKEN_FILE) names a file whose content goes out as the bearer, read on every call, so a projected Kubernetes ServiceAccount token that the kubelet rotates needs no restart. A token file is refused together with a key, under the pod's own ServiceAccount token directory, and over plain http to anything but a loopback host or an in-cluster Service name (.svc, .svc.cluster.local). In the chart, decision.token.audience mounts the token and sets the variable.

Ingesting traces

OTLP/HTTP (POST /v1/traces, :4318) and OTLP/gRPC (:4317) feed the same session store. Spans are grouped into sessions by ego.session_name, then gen_ai.conversation.id, then a synthetic name (a sender that declares ego.thread is an always-on agent, see below). GenAI semantic conventions, OpenInference and ADK (gcp.vertex.agent.*) attributes are all understood. Set ego.eval_set_id on the resource to tie a session to an eval set.

Instrumenting your own agent

ego adds no SDK and asks for no schema of its own. Any framework whose OpenTelemetry export carries the GenAI semantic conventions, OpenInference or ADK attributes works. What ego needs on the spans:

To get	Put on the span
an LLM call	gen_ai.request.model, or openinference.span.kind=LLM, or llm.model_name
a tool call	gen_ai.tool.name (OpenInference: tool.name), with gen_ai.tool.call.arguments and gen_ai.tool.call.result
a turn that can be scored	the person's words and the answer: gen_ai.input.messages and gen_ai.output.messages (OpenInference: input.value and output.value)
a token count in the usage ledger	gen_ai.usage.input_tokens and gen_ai.usage.output_tokens on the LLM span (OpenInference llm.token_count.prompt and llm.token_count.completion are read too); an LLM span without tokens has no ledger row
an agent invocation	gen_ai.operation.name=invoke_agent; without it a span whose name contains agent, chain, executor or workflow is taken for one

What can be left out, and what ego does then:

• Agent (call kind): gen_ai.agent.name, else the name of the trace's root span. Online scoring profiles match on it, so name it.
• Session: ego.session_name, then gen_ai.conversation.id, then a synthetic name. An agent that sets none of them still shows up, but traces of one conversation may not group.
• Tenant: the attribute ego.tenant on the span or the resource (--usage-tenant-attr changes the name), else default. ego.tenant.name gives it a display name.
• Message content: with no messages on the spans ego still lists them and counts tokens and errors, but there is nothing to score.

Not read: /v1/logs, so a framework that keeps message content in log records shows spans and tokens without the conversation. The OpenInference and Traceloop LangChain instrumentors wrap messages as JSON in input.value, which ego does not unwrap; use the LangSmith export below for LangChain and LangGraph.

Always-on agents: threads and segments

The id above is the sender's own, and for an agent that wakes every few minutes for weeks it names the agent, not a unit of work. This is opt-in: only a sender that declares the resource attribute ego.thread is segmented; without it ego behaves as it always did (a session completes after 30 s of quiet and is scored then, a late span reopens it). For a declared thread ego keeps three levels. A thread is the ego.thread value (it wins over ego.session_name, so a runtime that rotates its session id keeps one thread). A segment is what ego stores, lists, scores and cites as a session: a run of whole traces. The first segment keeps the thread's id, the n-th is <thread>~n. A turn is one trace; a segment never cuts one. A trace goes to the segment its first span chose, so spans that arrive late still land in it.

A segment closes for good when a new trace of its thread arrives, or after the thread has been quiet, and: its last span ended more than --segment-idle ago (default 30m; a trace within that window joins it), it holds --segment-max-spans spans or more (default 2000, checked between traces), or its first span is older than --segment-max-age (default 6h). A closed segment is scored by online evaluation once, with the end of the previous segment as context for the judge, and takes no new trace. Browse declared threads with GET /api/threads and the list_threads tool.

Claude Code

The native OpenTelemetry traces of the Claude Code CLI (spans named claude_code.*) use their own attribute names and set no gen_ai.usage.*. At ingest ego adds the GenAI attributes next to the originals, so they show up as sessions with token usage, an agent and a tenant. A span the sender already gave one of these attributes keeps it, and spans of any other sender are not touched.

Claude Code span	Added when absent
every claude_code.*	gen_ai.conversation.id = session.id (unless ego.session_name or gen_ai.conversation.id is set), so all prompts of one Claude Code session are one ego session; gen_ai.agent.name = the resource's gen_ai.agent.name, else service.name
claude_code.llm_request	gen_ai.operation.name = chat; gen_ai.request.model = model; gen_ai.response.finish_reasons = stop_reason; gen_ai.usage.input_tokens = input_tokens + cache_read_tokens + cache_creation_tokens; gen_ai.usage.output_tokens = output_tokens; gen_ai.usage.cache_read.input_tokens; gen_ai.usage.cache_creation.input_tokens and gen_ai.usage.cache_write.input_tokens = cache_creation_tokens; success=false marks the span as an error
claude_code.tool	gen_ai.operation.name = execute_tool; gen_ai.tool.name = tool_name (else tool_name_safe); gen_ai.tool.call.id = tool_use_id
claude_code.interaction	input.value = user_prompt, unless it is empty or <REDACTED>

Claude Code's input_tokens leaves cached tokens out, while the GenAI convention counts them in the input, so cache reads and cache writes are counted in gen_ai.usage.input_tokens (and therefore in ego's token usage). Prompts and tool output the sender redacted are never stored as content. The tenant is read from ego.tenant (or --usage-tenant-attr) on the span or the resource, so set it with OTEL_RESOURCE_ATTRIBUTES; the agent is service.name (claude-code unless OTEL_SERVICE_NAME says otherwise).

A sender whose agent answers a turn later (it handed the work to a helper and said it will come back) marks that turn with ego.turn.pending: true on its root span: the metrics that rate the answer skip it as skipped, and the turn that brings the answer is rated on its own.

LangChain and LangGraph

LangChain's documented way to get OpenTelemetry out of LangChain and LangGraph is LangSmith's own export: install langsmith[otel], set LANGSMITH_OTEL_ENABLED=true (and LANGSMITH_OTEL_ONLY=true to send traces only to the OTLP endpoint, not to LangSmith), and point the standard OTLP variables at ego:

pip install "langsmith[otel]"
export LANGSMITH_TRACING=true LANGSMITH_OTEL_ENABLED=true LANGSMITH_OTEL_ONLY=true
export OTEL_EXPORTER_OTLP_ENDPOINT=http://ego:4318          # the base URL, without /v1/traces
export OTEL_EXPORTER_OTLP_HEADERS="Authorization=Bearer <ingest token>"
export OTEL_SERVICE_NAME=my-agent                           # the agent's name in ego

LangSmith sends a run's whole input and output as the run's own JSON (gen_ai.prompt, gen_ai.completion: LangChain messages, a graph's state, a tool's arguments) and does not use the GenAI message attributes. At ingest ego adds those next to the originals, from spans carrying langsmith.span.kind. A value the sender already set is kept, and spans of any other sender are not touched.

LangSmith run	Added when absent
every run	gen_ai.conversation.id = metadata thread_id, else conversation_id, else session_id (unless ego.session_name or gen_ai.conversation.id is set), so a LangGraph thread is one ego session; the LangSmith project (langsmith.trace.session_id) is not a conversation
the root chain run	gen_ai.operation.name = invoke_agent; gen_ai.agent.name = the run's name (LangGraph unless the graph is named); input.value = the latest user message of the state; output.value = the final message, when it is an answer
llm	gen_ai.request.model (metadata ls_model_name or the response's model; unknown when the model class reports none); gen_ai.input.messages and gen_ai.output.messages with roles, text and tool calls
tool	gen_ai.tool.call.arguments (an object, or the JSON inside a single string argument); gen_ai.tool.call.result, which is flagged as an error when the ToolMessage status is error

Messages are read as LangChain's constructor form, its plain dump, OpenAI roles, ("user", "text") tuples and Anthropic tool_use blocks. LangSmith's exporter does not read OTEL_RESOURCE_ATTRIBUTES, so label the tenant with the ingest token (a sender token created with a tenant labels every span it sends) or with a key in the run's metadata (config={"metadata": {"tenant": "acme"}}) and start ego with --usage-tenant-attr langsmith.metadata.tenant.

Offline, ego run reads Jaeger JSON and OTLP JSON/JSONL files.

The live feed is GET /ws/ui-updates (WebSocket). GET /stream/ui-updates (Server-Sent Events) is there for curl; some gateways buffer long-lived SSE responses, which is why the UI uses WebSocket.

Persistence

By default everything is in memory. Point serve at a SQLite file and sessions, runs, eval sets and role bindings survive restarts:

ego serve --session-db ego.db

Only the newest --session-memory completed sessions (default 200) stay in memory; older ones load on demand. --session-retention (default 30 days) deletes stale sessions daily. The three flags that close a segment of a thread (--segment-idle, --segment-max-spans, --segment-max-age) are described under Ingesting traces.

Online evaluation

The Quality view stays off ("Online evaluation is not configured") until the server gets a config file. With it on, ego runs a continuous loop: sample completed sessions, score them per call kind with the evaluators you choose, detect regressions, notify (Discord, webhook or an event sink). A daily judge budget caps spend.

Minimal online.json: score every session of one agent with the hallucination judge, at most 500k judge tokens a day:

{
  "budget": {"judgeTokensPerDay": 500000},
  "profiles": [
    {"kind": "my-agent", "maxInvocations": 3, "evaluators": [{"name": "hallucinations_v1"}]}
  ]
}

kind is the span attribute gen_ai.agent.name of your agent (or, without it, the name of the trace's root span); only sessions of a kind with a profile are scored. Start the server with the file and a judge key (--session-db is required too):

export GEMINI_API_KEY=...
ego serve --session-db ego.db --online-config online.json

Scoring needs a judge model. jev_* evaluators use a decision model (EGO_DECISION_API_KEY, or EGO_DECISION_URL for your own) and a jevCallsPerDay budget instead. Sampling rates, regression detection and notification sinks are described in docs/online-evaluation.md.

MCP server

The server mounts a Model Context Protocol endpoint at POST /mcp (streamable HTTP). Agents and editors talk to it directly; nothing runs on the client.