Bietet eine kontrollierte Memory-Layer für Multi-Agent-Systeme mit rollenbasierter Zugriffskontrolle, Provenance und temporalem Replay.
Auch bekannt als bolnet/agent-memory, agent-memory
- Stars
- 14
- Tools
- —
- Downloads, 30 Tage
- 124
- Neueste Version
- 0.1.3
- Letzter Commit
- 18. Sept. 2026
- Lizenz
- MIT
Was ist Attestor MCP?
Attestor ist ein MCP server. Bietet eine kontrollierte Memory-Layer für Multi-Agent-Systeme mit rollenbasierter Zugriffskontrolle, Provenance und temporalem Replay.
Zusammenfassung generiert aus dem README des Projekts
Über Attestor MCP server
Attestor bietet eine kontrollierte Memory-Layer, die speziell für Multi-Agent-Production-Meshes entwickelt wurde, bei denen Sicherheit und Auditierbarkeit entscheidend sind. Im Gegensatz zu Standard-Memory-Tools für Coding-Assistenten verwaltet dieser Service komplexe Interaktionen, bei denen mehrere Agenten einen gemeinsamen Memory-Space nutzen. Er erzwingt eine strikte rollenbasierte Zugriffskontrolle über eine AgentContext-Layer, die es Entwicklern ermöglicht, spezifische Berechtigungen für Rollen wie Orchestrator, Planner, Executor, Researcher, Reviewer und Monitor zu definieren.
Nutzer können dies als Python-Library, als REST Sidecar oder als MCP server bereitstellen. Das System integriert sich mit Postgres als Source of Truth, Pinecone für Vektoren und Neo4j für Graphdaten, um eine strikte Tenant-Isolation über alle Storage-Rollen hinweg zu gewährleisten. Es bietet Temporal-Replay-Funktionen über den as_of Parameter, mit denen Nutzer abfragen können, was das Memory-Mesh zu einem beliebigen vergangenen Zeitpunkt wusste. Governance-Aufgaben wie Datenaufbewahrung und Nutzerlöschung werden als durable Temporal Workflows gehandhabt, um Zuverlässigkeit und Auditierbarkeit sicherzustellen.
Was du damit machen kannst
- 1Historische Zustände rekonstruierenFrage den Memory-Zustand so ab, wie er zu einem bestimmten Zeitpunkt existierte, um vergangene Agenten-Entscheidungen zu debuggen oder zu auditieren.
- 2Agenten-Ressourcenverbrauch verwaltenSetze Limits dafür, wie viele Daten ein Agent abrufen darf oder wie viele Schreibvorgänge er pro Tag durchführen kann.
- 3Daten-Provenance sicherstellenVerfolge genau, welcher Agent ein bestimmtes Memory-Element erstellt hat, und verifiziere dessen Authentizität mittels digitaler Signaturen.
Geschrieben aus dem README des Projekts. Jeder Use Case wurde gegen ein Zitat daraus geprüft.
So installierst du Attestor MCP
Formate geprüft am 10. Nov. 2026Attestor MCP in Claude Code installieren
$ claude mcp add --env NEO4J_PASSWORD=<YOUR_NEO4J_PASSWORD> --env OPENAI_API_KEY=<YOUR_OPENAI_API_KEY> --env OPENROUTER_API_KEY=<YOUR_OPENROUTER_API_KEY> agent-memory -- uvx memwrightAttestor MCP in Cursor installieren
~/.cursor/mcp.json
{
"mcpServers": {
"agent-memory": {
"command": "uvx",
"args": [
"memwright"
],
"env": {
"NEO4J_PASSWORD": "<YOUR_NEO4J_PASSWORD>",
"OPENAI_API_KEY": "<YOUR_OPENAI_API_KEY>",
"OPENROUTER_API_KEY": "<YOUR_OPENROUTER_API_KEY>"
}
}
}
}Attestor MCP in VS Code installieren
.vscode/mcp.json
{
"servers": {
"agent-memory": {
"type": "stdio",
"command": "uvx",
"args": [
"memwright"
],
"env": {
"NEO4J_PASSWORD": "<YOUR_NEO4J_PASSWORD>",
"OPENAI_API_KEY": "<YOUR_OPENAI_API_KEY>",
"OPENROUTER_API_KEY": "<YOUR_OPENROUTER_API_KEY>"
}
}
}
}Attestor MCP in Claude Desktop installieren
claude_desktop_config.json
{
"mcpServers": {
"agent-memory": {
"command": "uvx",
"args": [
"memwright"
],
"env": {
"NEO4J_PASSWORD": "<YOUR_NEO4J_PASSWORD>",
"OPENAI_API_KEY": "<YOUR_OPENAI_API_KEY>",
"OPENROUTER_API_KEY": "<YOUR_OPENROUTER_API_KEY>"
}
}
}
}Attestor MCP in Codex installieren
~/.codex/config.toml
[mcp_servers.agent-memory] command = "uvx" args = ["memwright"] [mcp_servers.agent-memory.env] NEO4J_PASSWORD = "<YOUR_NEO4J_PASSWORD>" OPENAI_API_KEY = "<YOUR_OPENAI_API_KEY>" OPENROUTER_API_KEY = "<YOUR_OPENROUTER_API_KEY>"
Attestor MCP in Gemini CLI installieren
~/.gemini/settings.json
{
"mcpServers": {
"agent-memory": {
"command": "uvx",
"args": [
"memwright"
],
"env": {
"NEO4J_PASSWORD": "<YOUR_NEO4J_PASSWORD>",
"OPENAI_API_KEY": "<YOUR_OPENAI_API_KEY>",
"OPENROUTER_API_KEY": "<YOUR_OPENROUTER_API_KEY>"
}
}
}
}Konfiguration
NEO4J_PASSWORDSecretNeo4j database password (default: memwright)
OPENAI_API_KEYSecretOpenAI API key for embeddings (alternative to OpenRouter)
OPENROUTER_API_KEYSecretOpenRouter API key for embeddings (required for semantic search)
ATTESTOR_DISABLE_LOCAL_EMBEDATTESTOR_EMBEDDING_MODELATTESTOR_EMBEDDING_PROVIDERATTESTOR_LIVE_PGPG_CONNECTION_STRINGPostgreSQL connection string for pgvector (default: postgresql://memwright:memwright@localhost:5432/memwright)
| Name | Beschreibung | Typ |
|---|---|---|
| NEO4J_PASSWORD | Neo4j database password (default: memwright) | Secret |
| OPENAI_API_KEY | OpenAI API key for embeddings (alternative to OpenRouter) | Secret |
| OPENROUTER_API_KEY | OpenRouter API key for embeddings (required for semantic search) | Secret |
| ATTESTOR_DISABLE_LOCAL_EMBED | — | |
| ATTESTOR_EMBEDDING_MODEL | — | |
| ATTESTOR_EMBEDDING_PROVIDER | — | |
| ATTESTOR_LIVE_PG | — | |
| PG_CONNECTION_STRING | PostgreSQL connection string for pgvector (default: postgresql://memwright:memwright@localhost:5432/memwright) |
Automatisierte Checks
Geluften am 14. Nov. 2026Vollständiger Bericht →Unsere automatisierten Checks bestanden. Das ist kein Security-Audit: Lies den Code und gib dem Server nur den minimal notwendigen Zugriff. Wie die Checks funktionieren
- Source Repository verknüpft und erreichbarbolnet/agent-memory is public and reachablegithub.com/bolnet/agent-memory ↗Bestanden
- Paket nennt dieses Repository als Quellememwright names bolnet/agent-memory as sourcepypi.org/project/memwright/ ↗Bestanden
- Build-Herkunft (Sigstore-Attestierung)Sigstore attestation for memwrightpypi.org/project/memwright/ ↗Bestanden
- Open Source Lizenz (OSI)MITspdx.org/licenses/MIT.html ↗Bestanden
- Commit oder Release im letzten JahrLast commit or release 2026-09-18github.com/bolnet/agent-memory ↗Bestanden
- OpenSSF ScorecardNicht geprüft
- Package-Name ähnelt keinem populären PackageNo popular package one edit away from memwrightBestanden
- Kein kritischer Hinweis im Paket oder seinen Abhängigkeiten1 resolved dependencies checked; no critical advisoryosv.dev ↗Bestanden
- Keine versteckten Anweisungen an das Modell0 tool descriptions and the README read; no override phrases, credential paths or hidden charactersBestanden
- Destruktive Tools sind markiertNicht geprüft
- Remote-Endpoint beantwortet den MCP-HandshakeNicht geprüft
README
Auf GitHub ansehenAttestor
Governed memory for multi-agent production meshes.
Attestor is no longer "memory for your coding assistant." Three things happened: the editor vendors shipped memory for free, the benchmark race became a race about which LLM answers the question, and every serious conversation about a memory layer was about something else entirely: when many agents share one memory, who is allowed to write, who is allowed to forget, and can you prove it afterwards. So that is what Attestor is now.
When many agents share one memory, recall is the easy part. The hard part is who may write, who may forget, what an answer was built from, and whether you can prove all of it later. Attestor is a self-hosted memory service that makes those properties enforceable in code rather than in policy documents: role-based access at the AgentContext layer, provenance on every memory (optionally Ed25519-signed), per-agent token budgets and write quotas, a deterministic ranking path with no LLM in it, temporal supersession you can query as of any past moment, hard tenant isolation across all three storage roles, an auditable, audit-first forget-user path with declarative retention, and governance jobs that run as durable Temporal workflows: retried by policy, resumed after a crash, visible in an operator UI, and never on the read path. It runs as a Python library, a REST sidecar, or an MCP server with the same API. Role enforcement is an AgentContext guarantee and runs in process. The REST and MCP surfaces call the store directly and rely on their own auth.
ctx = AgentContext(agent_id="reviewer-1", role=AgentRole.REVIEWER, namespace="acme-prod")
ctx.add_memory("...") # PermissionError: REVIEWER lacks 'write'
mem.recall("...", context=ctx, as_of=last_friday) # what the mesh believed then — nothing is deleted
pip install attestor
| Version | 4.1.11 (stable; greenfield rebuild — no v3 migration path) |
| PyPI | attestor |
| Import | attestor |
| Live site | https://attestor.dev/ |
| Repo | https://github.com/bolnet/attestor |
| License | MIT |
Designed and built by Surendra Singh — building auditable infrastructure for multi-agent AI, with fifteen years of production-systems discipline brought to the memory layer. Companion projects:
claude-finance(Claude-powered financial analytics) ·private-equity(PE × AI workshop). Reach out if you're hiring senior IC for AI infrastructure.
What it is
Attestor is the memory layer for an agent mesh that has to answer to someone — a security review, a regulator, a customer, or the engineer paged at 3 a.m. Every memory is stored once in Postgres (the source of truth) and derived into Pinecone (vectors) and Neo4j (graph). Every read and write goes through an AgentContext that carries identity, role, namespace, and budget, and the store keeps enough history that any past state can be reconstructed.
What it enforces
Each guarantee below is a property of the code, not a roadmap item. File references point at the enforcing implementation.
| Guarantee | What the code does | Where |
|---|---|---|
| Role-based access, six roles | ORCHESTRATOR = read + write + forget; PLANNER / EXECUTOR / RESEARCHER = read + write; REVIEWER / MONITOR = read only. read_only=True is an independent kill switch that strips write + forget regardless of role. Violations raise PermissionError before any backend is touched. |
attestor/context.py — ROLE_PERMISSIONS, _require_permission |
| Provenance on every memory | Every write carries agent_id, session_id, namespace, and role; child contexts accumulate an agent_trail, so "which agent said this, on whose behalf" is answerable. source_episode_id is set by the episode consolidation path, not by every write. Supersession never deletes. Rows leave only through an explicit forget_user() (audited first), a retention delete policy (audited after), or compact() (not audited). Optional Ed25519 signing of id ‖ agent_id ‖ t_created ‖ content_hash, verified with verify_memory(). |
attestor/context.py, attestor/identity/signing.py |
| Per-agent budgets and write quotas | token_budget bounds what recall() packs for one AgentContext; it never limits writes. max_writes_per_agent is an in-memory counter on the context and resets with a new context. set_quota() sets a per-user daily write limit enforced on add(); it is per user, not per agent. |
attestor/context.py, attestor/core/quota_service.py, attestor/core/agent_memory.py |
| Deterministic read path | Ranking is vector → BM25 → RRF → graph → MMR → budget fit, with no LLM in scoring by default. Same query, same order, unit-testable and replayable. Query-expansion lanes (HyDE, multi-query) are YAML flags that add candidates; they never touch scoring. An optional reranker (bge, cohere, or llm provider) can re-score candidates; it is off by default. |
attestor/retrieval/orchestrator/core.py, attestor/retrieval/scorer.py, attestor/retrieval/reranker.py |
Temporal supersession, as_of replay |
Contradictions supersede older facts; supersession never deletes. Every fact carries valid time (valid_from, valid_until) and transaction time (t_created, t_expired). recall(as_of=...) filters candidates on transaction time at document hydration and in the BM25 lane; the Pinecone lane ignores as_of and logs a warning. |
attestor/temporal/manager.py, attestor/store/schema.sql, attestor/store/pinecone_backend.py |
| Hard tenant isolation, all three roles | Tenant is the user. Postgres: row-level security keyed to attestor.current_user_id, with namespace stamped into metadata. Pinecone: one index, partitioned by namespace= on every call. Neo4j: namespace property with a composite (key, namespace) constraint and namespace-scoped traversal; the PageRank projection is global and is a ranking signal only. |
attestor/store/schema.sql, attestor/store/pinecone_backend.py, attestor/store/neo4j_backend.py |
| Auditable forget-user and retention | forget_user() writes an RLS-exempt, append-only by convention forget_audit row before deleting, then walks document → state → vector → graph and reports per-backend counts so partial success is visible, never silent. Declarative retention policies (archive / delete, filtered by namespace, category, layer, tags, age) apply first and write a forget_audit row after; only forget_user() is audit-first. Full export for subject-access requests. |
attestor/compliance/retention.py, attestor/core/agent_memory.py — export_user, purge_user, deletion_audit_log |
| Durable governance jobs | Consolidation, derived-state repair (attestor durable rebuild), the audit-first forget-user saga, and retention / session sweeps run as Temporal workflows with explicit retry policies and an operator UI. The forget saga writes the audit row, deletes document rows, then runs vector, graph, and state concurrently; it retries each backend independently and reports partial state as a failure. Opt-in extra (attestor[durable]); the read path and the Claude Code hooks never touch it. |
attestor/durable/, docs/INSTALL.md (Ch. 04) |
How to evaluate it
The nearest neighbours are the temporal-knowledge-graph memory layers (Zep, Graphiti). For a production mesh the useful comparison is not benchmark accuracy — it is whether the memory layer itself enforces roles, budgets, tenancy, provenance, and auditable deletion, or leaves each of those to the calling application. The table above is Attestor's answer; hold any alternative to the same checklist, with file references.
Designed for
- Multi-agent products where many LLM-driven agents with different trust levels write to the same memory
- Regulated systems that need point-in-time reconstruction and a defensible deletion story
- Self-hosted deployments — your VPC, your Postgres, your Pinecone, your Neo4j
Not designed for
- A personal memory for a single coding assistant — the Claude Code plugin below is an on-ramp demo of the same engine, not the product
- A general-purpose vector database or a RAG framework with chunking / reranking / orchestration
- An LLM agent runtime — Attestor is the memory backend; the agent loop is yours
Try it locally in one command.
pipx install attestor && attestor quickstartbrings up Postgres + Pinecone Local + Neo4j in Docker and wires the Claude Code MCP server + hooks so you can watch the governance layer work on a real project. Details in Quick start and Install for Claude Code below.
Quick start
1. Install
pip install attestor # or: pipx install attestor
Or pull the container (introspection-grade image, single layer over python:3.12-slim, currently linux/amd64):
docker pull ghcr.io/bolnet/attestor:latest # recommended — anonymous pull, mirrored to all registries below
Same image is mirrored to:
| Registry | Pull address |
|---|---|
| GHCR | ghcr.io/bolnet/attestor:latest |
| Docker Hub | bolnet2025/attestor:latest |
| Quay | quay.io/bolnet/attestor:latest |
| AWS ECR Public | public.ecr.aws/m6h5j7o3/attestor:latest |
| GCP AR | us-central1-docker.pkg.dev/coral-marker-452616-n4/attestor/attestor:latest |
(An internal Azure ACR mirror exists at memwright.azurecr.io/attestor but is private — Azure customers should use az acr import from one of the public registries above.)
The image's default entrypoint is attestor mcp (MCP server over stdio). For full production use, point the container at an external Postgres + Neo4j via env vars (or compose them with attestor/infra/local/docker-compose.yml); override the entrypoint to run attestor doctor, attestor api, etc.
2. Stand up the local stack — one command, zero questions
attestor quickstart
attestor quickstart does the whole local install non-interactively and prints every step: it writes ~/.attestor/{config.toml,attestor.yaml,.env}, brings up the three-role local stack in Docker, uses a local Ollama bge-m3 embedder (no cloud key), wires the Claude Code MCP server (./.mcp.json) + lifecycle hooks, and runs attestor doctor.
Prerequisites: Docker running, and Ollama serving bge-m3 (ollama pull bge-m3). quickstart runs a preflight that scans the ports/tools and tells you if anything is missing — it never prompts.
| Container | Role | Port | Purpose |
|---|---|---|---|
| Postgres 16 | Document | 5432 |
Source of truth — content, tags, entity, ts, provenance, RLS-isolated by user_id |
| Pinecone Local | Vector | 5080-5089 |
Dense embeddings, per-namespace isolation, plain gRPC (no HTTPS) |
| Neo4j 5 + GDS | Graph | 7687 |
Entity nodes + typed edges, PageRank / BFS / Leiden |
To reverse it later: attestor teardown (zero-question; keeps your data volumes by default — --purge also wipes them, --dry-run previews).
In Claude Code, drive the same install conversationally: /plugin marketplace add bolnet/attestor → /plugin install attestor (then enable it), and run /attestor:install-attestor — it runs attestor quickstart for you. Cloud/managed backends (Neon / RDS / Cloud SQL, Pinecone Cloud, Neo4j AuraDB) and alternative embedders (Pinecone Inference llama-text-embed-v2, Voyage voyage-4, OpenAI text-embedding-3) are configured in ~/.attestor/attestor.yaml (the single source of truth) — see docs/INSTALL.md.
attestor doctor (run automatically at the end, or any time) checks all four subsystems: Document Store (Postgres), Vector Store (Pinecone), Graph Store (Neo4j), Retrieval Pipeline. The only hard dependency that cannot be down is the document store (Postgres); transient vector-probe failures are surfaced in the response trace rather than swallowed (retrieval/orchestrator/debug.py — vector_error field).
3. Use it
from attestor import AgentMemory, AgentContext, AgentRole
mem = AgentMemory() # picks up env / ~/.attestor.toml automatically
ctx = AgentContext(
agent_id="researcher-1",
role=AgentRole.RESEARCHER,
namespace="acme-prod",
)
mem.add(
content="Alice is the engineering manager",
entity="alice",
category="role",
context=ctx,
)
results = mem.recall(query="who runs engineering?", context=ctx)
for r in results:
print(r.score, r.memory.content)
SOLO mode (zero-config). In v4,
AgentMemory().add('foo')auto-provisions a singletonlocaluser, an Inbox project (metadata.is_inbox=true), and a daily session — so the snippet above works on a fresh database without configuring identity (core.py:179-209). For multi-tenant production use, pass an explicitAgentContextwith a realnamespace.
4. Run a smoke benchmark (optional)
Verify your install end-to-end against a tiny LongMemEval slice. Defaults come from configs/attestor.yaml: Pinecone Inference llama-text-embed-v2 (1024-D) embedder + Pinecone vector store, openai/gpt-5.5 answerer, dual judges (openai/gpt-5.5 + anthropic/claude-sonnet-4-6), parallel=2.
set -a && source .env && set +a # OPENROUTER_API_KEY, PINECONE_API_KEY, NEO4J_PASSWORD
.venv/bin/python scripts/lme_smoke_local.py --n 2 --yes
Every model and parameter comes from YAML — see § Benchmarking below for the full bench harness.
Benchmarking
Every benchmark — smoke, single slice, full sweep, synthetic supersession — reads its knobs from two YAMLs:
| File | What lives there |
|---|---|
configs/attestor.yaml |
Stack — embedder, models, retrieval features, DBs, registries, clouds |
configs/bench.yaml |
Bench-only — variants, category iteration order, target scores, output paths |
The two files must have disjoint keys. The CI test tests/test_config_no_duplicate_keys.py enforces this; the bench loader (attestor.bench_config.get_bench) crashes on overlap. If you want a one-off override (different model for one bench run), use an env var or CLI flag — never duplicate the key in bench.yaml.
What LongMemEval is
LongMemEval (Wu et al., 2024 — published at ICLR '25) is the canonical benchmark for memory-augmented chat assistants. It measures whether an AI system can correctly answer questions that require recalling facts from long, multi-session conversation histories — the exact scenario Attestor is built for.
500 questions, 6 reasoning categories, 3 haystack sizes. Same questions across all three sizes; only the noise around the answer-bearing session changes:
| Variant | Tokens / Q | Sessions | What it measures |
|---|---|---|---|
oracle |
~3-15k | 1-3 gold | Reasoning ceiling — what the answerer can do with perfect retrieval. If you score low here, your prompt or LLM is broken (retrieval can't help). |
s (Standard / Small) |
~115k | ~50 | Public leaderboard — the canonical comparison. Fits in a single Claude/GPT context window, so Attestor's retrieval is benchmarked against the "just stuff everything into long context" baseline. |
m (Plus / Medium) |
~1M+ | ~500 | Pure retrieval — too big for any context window. Memory layer is forced; no long-context shortcut available. |
LME-S is the headline number to beat. A memory layer that scores within 5% of a long-context baseline at 30× lower token cost is the marketing pitch.
The 6 reasoning categories (cleaned LME-S, 500 questions total — note: no abstention slice in the cleaned split, which the synthetic supersession suite covers):
| Category | N | What it tests |
|---|---|---|
multi-session |
133 | Fact spans across multiple sessions — must track an entity over time |
temporal-reasoning |
133 | Date arithmetic ("two weeks ago", "before X") — Attestor's bi-temporal layer is built for this slice |
knowledge-update |
78 | Supersession — newer fact must beat older fact when both exist |
single-session-user |
70 | One session, fact stated by the user |
single-session-assistant |
56 | One session, fact stated by the assistant |
single-session-preference |
30 | One session, user preference |
Why this benchmark for Attestor: the temporal-reasoning and knowledge-update slices directly exercise features that distinguish Attestor from a vanilla RAG: bi-temporal recall, supersession-on-contradiction, event-time vs transaction-time disambiguation. A high score on those slices is the regulated-AI / audit / compliance pitch.
For the published Attestor numbers, see docs/bench/ — bench artifacts persist as lme-{variant}-{category}-{date}.{report,summary}.json. The Reporting section below shows how to render them as a table.
Download the LongMemEval dataset (one-time, before any bench run)
All lme_*.sh scripts use the cleaned LongMemEval split published on HuggingFace by xiaowu0162/longmemeval-cleaned. It auto-downloads on first use, but you'll want to know what's happening.
Cache location (created on first call):
~/.cache/attestor/longmemeval/
(Or $XDG_CACHE_HOME/attestor/longmemeval/ if you set XDG_CACHE_HOME.)
Variants and on-disk sizes:
| Variant | Filename | Size | Tokens / Q | Use |
|---|---|---|---|---|
oracle |
longmemeval_oracle.json |
~5 MB | ~3-15k | Reasoning ceiling — cheapest smoke |
s |
longmemeval_s_cleaned.json |
~250 MB | ~115k | Public leaderboard (canonical) |
m |
longmemeval_m_cleaned.json |
~2 GB | ~1M+ | Forces retrieval (no long-context shortcut) |
Option A — auto-download (recommended)
Just run any bench command. The first call downloads and caches; every subsequent call reads from disk:
# Will download longmemeval_oracle.json (~5 MB) the first time
.venv/bin/python scripts/lme_smoke_local.py --n 2 --yes --variant oracle
# Will download longmemeval_s_cleaned.json (~250 MB) the first time
scripts/bench/lme_run.sh knowledge-update
You only pay the download cost once per variant. Internet flake during the first run? Delete the partial file in the cache dir and rerun.
Option B — pre-warm the cache (offline / CI)
Pre-fetch every variant you plan to use before the bench day:
.venv/bin/python -c "
from attestor.longmemeval import load_or_download
for v in ('oracle', 's', 'm'):
samples = load_or_download(variant=v)
print(f'{v}: {len(samples)} samples')
"
Expected output:
oracle: 500 samples
s: 500 samples
m: 500 samples
Option C — manual download (firewalled environments)
If your runner can't reach huggingface.co, fetch the files on a connected machine and drop them into the cache dir manually:
mkdir -p ~/.cache/attestor/longmemeval
cd ~/.cache/attestor/longmemeval
# pick the variants you need
curl -L -o longmemeval_oracle.json \
https://huggingface.co/datasets/xiaowu0162/longmemeval-cleaned/resolve/main/longmemeval_oracle.json
curl -L -o longmemeval_s_cleaned.json \
https://huggingface.co/datasets/xiaowu0162/longmemeval-cleaned/resolve/main/longmemeval_s_cleaned.json
curl -L -o longmemeval_m_cleaned.json \
https://huggingface.co/datasets/xiaowu0162/longmemeval-cleaned/resolve/main/longmemeval_m_cleaned.json
The bench harness checks for these filenames exactly — don't rename them.
Verify the dataset is loadable
After download (auto or manual), confirm the loader picks it up cleanly:
.venv/bin/python -c "
from attestor.longmemeval import load_or_download
from collections import Counter
samples = load_or_download(variant='s')
cnt = Counter(s.question_type for s in samples)
print(f'Loaded {len(samples)} samples')
for cat, n in sorted(cnt.items(), key=lambda x: -x[1]):
print(f' {cat}: {n}')
"
Expected for the cleaned s variant (500 questions, 6 categories — note: no abstention slice in the cleaned split):
Loaded 500 samples
multi-session: 133
temporal-reasoning: 133
knowledge-update: 78
single-session-user: 70
single-session-assistant: 56
single-session-preference: 30
If counts don't match, the file is truncated — re-download.
Quick smoke (≤ 1 minute, ≤ $0.10)
Confirm the pipeline runs end-to-end before committing or running anything bigger:
.venv/bin/python scripts/lme_smoke_local.py --n 2 --yes --variant oracle
oracle is the cheapest variant (gold sessions only, no distractor haystack). Schema is reapplied automatically; pass --skip-schema if you want to keep a populated DB between runs.
Single category — scripts/bench/lme_run.sh
# all 6 categories, current variant from bench.yaml (default: s)
scripts/bench/lme_run.sh
# one slice — full
scripts/bench/lme_run.sh knowledge-update
# one slice — capped at N samples (smoke)
scripts/bench/lme_run.sh knowledge-update 10
# one slice on a different variant (oracle = cheapest, m = ~1M tokens)
scripts/bench/lme_run.sh knowledge-update "" oracle
Valid --category values: single-session-user, single-session-assistant, single-session-preference, multi-session, temporal-reasoning, knowledge-update. See What LongMemEval is above for sample counts and what each category tests.
Each run persists two files:
docs/bench/lme-{variant}-{category}-{YYYYMMDD}.report.json # full LMERunReport
docs/bench/lme-{variant}-{category}-{YYYYMMDD}.summary.json # BenchmarkSummary
Full sweep — scripts/bench/lme_all.sh
Iterates bench.yaml's lme.categories list in order. Adding/removing slices is a YAML edit, not a script edit:
# All 6 slices, current variant
scripts/bench/lme_all.sh
# All 6 slices, capped at 10 samples each (smoke)
scripts/bench/lme_all.sh 10
# All 6 slices on Oracle variant
scripts/bench/lme_all.sh "" oracle
If one slice fails, the script logs it and moves on to the next.
Attestor MCP Fragen
Wie füge ich Attestor MCP zu Claude Code hinzu?
Führe claude mcp add --env NEO4J_PASSWORD=<YOUR_NEO4J_PASSWORD> --env OPENAI_API_KEY=<YOUR_OPENAI_API_KEY> --env OPENROUTER_API_KEY=<YOUR_OPENROUTER_API_KEY> agent-memory -- uvx memwright in deinem Terminal aus. Füge --scope user hinzu, damit der Server in jedem Projekt verfügbar ist.
Wie füge ich Attestor MCP zu Cursor hinzu?
Öffne ~/.cursor/mcp.json und füge den Server unter mcpServers hinzu, mit der Cursor-Config aus dem Installationsabschnitt oben.
Was ist Attestor?
Attestor ist ein selbst gehosteter Memory-Service für Multi-Agent-Production-Meshes, der rollenbasierte Zugriffskontrolle, Provenance und eine prüfbare Löschung erzwingt.