Menyediakan layer memori yang terkelola untuk sistem multi-agent dengan fitur role-based access, provenance, dan temporal replay.
Juga dikenal sebagai bolnet/agent-memory, agent-memory
- Stars
- 14
- Tools
- —
- Unduhan, 30 hari
- 124
- Versi terbaru
- 0.1.3
- Commit terakhir
- 18 Sep 2026
- Lisensi
- MIT
Apa itu Attestor MCP?
Menyediakan layer memori yang terkelola untuk sistem multi-agent dengan fitur role-based access, provenance, dan temporal replay.
Ringkasan dibuat dari README proyek
Tentang Attestor MCP server
Attestor menyediakan layer memori terkelola yang dirancang khusus untuk multi-agent production meshes di mana keamanan dan auditabilitas sangat krusial. Berbeda dengan tool memori standar untuk coding assistant, layanan ini mengelola interaksi kompleks di mana beberapa agent berbagi satu ruang memori yang sama. Attestor menerapkan role-based access control yang ketat melalui layer AgentContext, memungkinkan developer untuk menentukan izin spesifik untuk role seperti Orchestrator, Planner, Executor, Researcher, Reviewer, dan Monitor.
Anda dapat men-deploy ini sebagai library Python, REST sidecar, atau MCP server. Sistem ini terintegrasi dengan Postgres untuk data truth, Pinecone untuk vector, dan Neo4j untuk data graph, guna memastikan isolasi tenant yang kuat di semua role penyimpanan. Attestor memiliki kemampuan temporal replay melalui parameter as_of, yang memungkinkan Anda melakukan query terhadap apa yang diketahui oleh memory mesh pada momen spesifik di masa lalu. Tugas governance, seperti retensi data dan penghapusan user, ditangani sebagai workflow Temporal yang durable untuk memastikan reliabilitas dan auditabilitas.
Apa yang bisa Anda lakukan dengannya
- 1Rekonstruksi state historisLakukan query terhadap state memori sebagaimana adanya pada titik waktu tertentu untuk melakukan debug atau audit terhadap keputusan agent di masa lalu.
- 2Kelola konsumsi resource agentTetapkan limit pada seberapa banyak data yang dapat diambil oleh agent atau seberapa banyak operasi write yang dapat dilakukan setiap harinya.
- 3Pastikan data provenanceLacak secara akurat agent mana yang membuat potongan memori tertentu dan verifikasi autentisitasnya menggunakan digital signatures.
Ditulis berdasarkan README proyek. Setiap use case diperiksa terhadap kutipan dari README tersebut.
Cara menginstal Attestor MCP
Format diverifikasi 10 Nov 2026Tambahkan ke CursorTambahkan ke GooseInstal Attestor MCP di Claude Code
$ claude mcp add --env 'NEO4J_PASSWORD=<YOUR_NEO4J_PASSWORD>' --env 'OPENAI_API_KEY=<YOUR_OPENAI_API_KEY>' --env 'OPENROUTER_API_KEY=<YOUR_OPENROUTER_API_KEY>' agent-memory -- uvx memwrightInstal Attestor MCP di Cursor
~/.cursor/mcp.json
{
"mcpServers": {
"agent-memory": {
"command": "uvx",
"args": [
"memwright"
],
"env": {
"NEO4J_PASSWORD": "<YOUR_NEO4J_PASSWORD>",
"OPENAI_API_KEY": "<YOUR_OPENAI_API_KEY>",
"OPENROUTER_API_KEY": "<YOUR_OPENROUTER_API_KEY>"
}
}
}
}Instal Attestor MCP di VS Code
.vscode/mcp.json
{
"servers": {
"agent-memory": {
"type": "stdio",
"command": "uvx",
"args": [
"memwright"
],
"env": {
"NEO4J_PASSWORD": "<YOUR_NEO4J_PASSWORD>",
"OPENAI_API_KEY": "<YOUR_OPENAI_API_KEY>",
"OPENROUTER_API_KEY": "<YOUR_OPENROUTER_API_KEY>"
}
}
}
}Instal Attestor MCP di VS Code CLI
$ code --add-mcp '{"name":"agent-memory","type":"stdio","command":"uvx","args":["memwright"],"env":{"NEO4J_PASSWORD":"<YOUR_NEO4J_PASSWORD>","OPENAI_API_KEY":"<YOUR_OPENAI_API_KEY>","OPENROUTER_API_KEY":"<YOUR_OPENROUTER_API_KEY>"}}'Instal Attestor MCP di Claude Desktop
claude_desktop_config.json
{
"mcpServers": {
"agent-memory": {
"command": "uvx",
"args": [
"memwright"
],
"env": {
"NEO4J_PASSWORD": "<YOUR_NEO4J_PASSWORD>",
"OPENAI_API_KEY": "<YOUR_OPENAI_API_KEY>",
"OPENROUTER_API_KEY": "<YOUR_OPENROUTER_API_KEY>"
}
}
}
}Instal Attestor MCP di Codex
$ codex mcp add agent-memory --env 'NEO4J_PASSWORD=<YOUR_NEO4J_PASSWORD>' --env 'OPENAI_API_KEY=<YOUR_OPENAI_API_KEY>' --env 'OPENROUTER_API_KEY=<YOUR_OPENROUTER_API_KEY>' -- uvx memwrightInstal Attestor MCP di Codex config.toml
~/.codex/config.toml
[mcp_servers.agent-memory] command = "uvx" args = ["memwright"] [mcp_servers.agent-memory.env] NEO4J_PASSWORD = "<YOUR_NEO4J_PASSWORD>" OPENAI_API_KEY = "<YOUR_OPENAI_API_KEY>" OPENROUTER_API_KEY = "<YOUR_OPENROUTER_API_KEY>"
Instal Attestor MCP di Gemini CLI
$ gemini mcp add -s user -e 'NEO4J_PASSWORD=<YOUR_NEO4J_PASSWORD>' -e 'OPENAI_API_KEY=<YOUR_OPENAI_API_KEY>' -e 'OPENROUTER_API_KEY=<YOUR_OPENROUTER_API_KEY>' agent-memory uvx -- memwrightInstal Attestor MCP di Gemini settings.json
~/.gemini/settings.json
{
"mcpServers": {
"agent-memory": {
"command": "uvx",
"args": [
"memwright"
],
"env": {
"NEO4J_PASSWORD": "<YOUR_NEO4J_PASSWORD>",
"OPENAI_API_KEY": "<YOUR_OPENAI_API_KEY>",
"OPENROUTER_API_KEY": "<YOUR_OPENROUTER_API_KEY>"
}
}
}
}Konfigurasi
NEO4J_PASSWORDRahasiaNeo4j database password (default: memwright)
OPENAI_API_KEYRahasiaOpenAI API key for embeddings (alternative to OpenRouter)
OPENROUTER_API_KEYRahasiaOpenRouter API key for embeddings (required for semantic search)
ATTESTOR_DISABLE_LOCAL_EMBEDATTESTOR_EMBEDDING_MODELATTESTOR_EMBEDDING_PROVIDERATTESTOR_LIVE_PGPG_CONNECTION_STRINGPostgreSQL connection string for pgvector (default: postgresql://memwright:memwright@localhost:5432/memwright)
| Nama | Deskripsi | Tipe |
|---|---|---|
| NEO4J_PASSWORD | Neo4j database password (default: memwright) | Rahasia |
| OPENAI_API_KEY | OpenAI API key for embeddings (alternative to OpenRouter) | Rahasia |
| OPENROUTER_API_KEY | OpenRouter API key for embeddings (required for semantic search) | Rahasia |
| ATTESTOR_DISABLE_LOCAL_EMBED | — | |
| ATTESTOR_EMBEDDING_MODEL | — | |
| ATTESTOR_EMBEDDING_PROVIDER | — | |
| ATTESTOR_LIVE_PG | — | |
| PG_CONNECTION_STRING | PostgreSQL connection string for pgvector (default: postgresql://memwright:memwright@localhost:5432/memwright) |
Pengecekan otomatis
Berjalan 14 Nov 2026Laporan lengkap →Lolos checks otomatis kami. Ini bukan audit keamanan: baca kodenya dan berikan server akses paling terbatas yang dibutuhkannya. Cara kerja checks
- Source repository terhubung dan dapat dijangkaubolnet/agent-memory is public and reachablegithub.com/bolnet/agent-memory ↗Lolos
- Nama paket mencantumkan repositori ini sebagai sumbernyamemwright names bolnet/agent-memory as sourcepypi.org/project/memwright/ ↗Lolos
- Provenance build (atestasi Sigstore)Sigstore attestation for memwrightpypi.org/project/memwright/ ↗Lolos
- Lisensi open source (OSI)MITspdx.org/licenses/MIT.html ↗Lolos
- Ada commit atau release dalam satu tahun terakhirLast commit or release 2026-09-18github.com/bolnet/agent-memory ↗Lolos
- OpenSSF ScorecardTidak diperiksa
- Nama package tidak mirip dengan package populerNo popular package one edit away from memwrightLolos
- Tidak ada advisory kritis pada paket atau dependensinya1 resolved dependencies checked; no critical advisoryosv.dev ↗Lolos
- Tidak ada instruksi tersembunyi untuk model0 tool descriptions and the README read; no override phrases, credential paths or hidden charactersLolos
- Tools yang bersifat destruktif telah diberi labelTidak diperiksa
- Endpoint remote merespons handshake MCPTidak diperiksa
README
Lihat di GitHubAttestor
Governed memory for multi-agent production meshes.
Attestor is no longer "memory for your coding assistant." Three things happened: the editor vendors shipped memory for free, the benchmark race became a race about which LLM answers the question, and every serious conversation about a memory layer was about something else entirely: when many agents share one memory, who is allowed to write, who is allowed to forget, and can you prove it afterwards. So that is what Attestor is now.
When many agents share one memory, recall is the easy part. The hard part is who may write, who may forget, what an answer was built from, and whether you can prove all of it later. Attestor is a self-hosted memory service that makes those properties enforceable in code rather than in policy documents: role-based access at the AgentContext layer, provenance on every memory (optionally Ed25519-signed), per-agent token budgets and write quotas, a deterministic ranking path with no LLM in it, temporal supersession you can query as of any past moment, hard tenant isolation across all three storage roles, an auditable, audit-first forget-user path with declarative retention, and governance jobs that run as durable Temporal workflows: retried by policy, resumed after a crash, visible in an operator UI, and never on the read path. It runs as a Python library, a REST sidecar, or an MCP server with the same API. Role enforcement is an AgentContext guarantee and runs in process. The REST and MCP surfaces call the store directly and rely on their own auth.
ctx = AgentContext(agent_id="reviewer-1", role=AgentRole.REVIEWER, namespace="acme-prod")
ctx.add_memory("...") # PermissionError: REVIEWER lacks 'write'
mem.recall("...", context=ctx, as_of=last_friday) # what the mesh believed then — nothing is deleted
pip install attestor
| Version | 4.1.11 (stable; greenfield rebuild — no v3 migration path) |
| PyPI | attestor |
| Import | attestor |
| Live site | https://attestor.dev/ |
| Repo | https://github.com/bolnet/attestor |
| License | MIT |
Designed and built by Surendra Singh — building auditable infrastructure for multi-agent AI, with fifteen years of production-systems discipline brought to the memory layer. Companion projects:
claude-finance(Claude-powered financial analytics) ·private-equity(PE × AI workshop). Reach out if you're hiring senior IC for AI infrastructure.
What it is
Attestor is the memory layer for an agent mesh that has to answer to someone — a security review, a regulator, a customer, or the engineer paged at 3 a.m. Every memory is stored once in Postgres (the source of truth) and derived into Pinecone (vectors) and Neo4j (graph). Every read and write goes through an AgentContext that carries identity, role, namespace, and budget, and the store keeps enough history that any past state can be reconstructed.
What it enforces
Each guarantee below is a property of the code, not a roadmap item. File references point at the enforcing implementation.
| Guarantee | What the code does | Where |
|---|---|---|
| Role-based access, six roles | ORCHESTRATOR = read + write + forget; PLANNER / EXECUTOR / RESEARCHER = read + write; REVIEWER / MONITOR = read only. read_only=True is an independent kill switch that strips write + forget regardless of role. Violations raise PermissionError before any backend is touched. |
attestor/context.py — ROLE_PERMISSIONS, _require_permission |
| Provenance on every memory | Every write carries agent_id, session_id, namespace, and role; child contexts accumulate an agent_trail, so "which agent said this, on whose behalf" is answerable. source_episode_id is set by the episode consolidation path, not by every write. Supersession never deletes. Rows leave only through an explicit forget_user() (audited first), a retention delete policy (audited after), or compact() (not audited). Optional Ed25519 signing of id ‖ agent_id ‖ t_created ‖ content_hash, verified with verify_memory(). |
attestor/context.py, attestor/identity/signing.py |
| Per-agent budgets and write quotas | token_budget bounds what recall() packs for one AgentContext; it never limits writes. max_writes_per_agent is an in-memory counter on the context and resets with a new context. set_quota() sets a per-user daily write limit enforced on add(); it is per user, not per agent. |
attestor/context.py, attestor/core/quota_service.py, attestor/core/agent_memory.py |
| Deterministic read path | Ranking is vector → BM25 → RRF → graph → MMR → budget fit, with no LLM in scoring by default. Same query, same order, unit-testable and replayable. Query-expansion lanes (HyDE, multi-query) are YAML flags that add candidates; they never touch scoring. An optional reranker (bge, cohere, or llm provider) can re-score candidates; it is off by default. |
attestor/retrieval/orchestrator/core.py, attestor/retrieval/scorer.py, attestor/retrieval/reranker.py |
Temporal supersession, as_of replay |
Contradictions supersede older facts; supersession never deletes. Every fact carries valid time (valid_from, valid_until) and transaction time (t_created, t_expired). recall(as_of=...) filters candidates on transaction time at document hydration and in the BM25 lane; the Pinecone lane ignores as_of and logs a warning. |
attestor/temporal/manager.py, attestor/store/schema.sql, attestor/store/pinecone_backend.py |
| Hard tenant isolation, all three roles | Tenant is the user. Postgres: row-level security keyed to attestor.current_user_id, with namespace stamped into metadata. Pinecone: one index, partitioned by namespace= on every call. Neo4j: namespace property with a composite (key, namespace) constraint and namespace-scoped traversal; the PageRank projection is global and is a ranking signal only. |
attestor/store/schema.sql, attestor/store/pinecone_backend.py, attestor/store/neo4j_backend.py |
| Auditable forget-user and retention | forget_user() writes an RLS-exempt, append-only by convention forget_audit row before deleting, then walks document → state → vector → graph and reports per-backend counts so partial success is visible, never silent. Declarative retention policies (archive / delete, filtered by namespace, category, layer, tags, age) apply first and write a forget_audit row after; only forget_user() is audit-first. Full export for subject-access requests. |
attestor/compliance/retention.py, attestor/core/agent_memory.py — export_user, purge_user, deletion_audit_log |
| Durable governance jobs | Consolidation, derived-state repair (attestor durable rebuild), the audit-first forget-user saga, and retention / session sweeps run as Temporal workflows with explicit retry policies and an operator UI. The forget saga writes the audit row, deletes document rows, then runs vector, graph, and state concurrently; it retries each backend independently and reports partial state as a failure. Opt-in extra (attestor[durable]); the read path and the Claude Code hooks never touch it. |
attestor/durable/, docs/INSTALL.md (Ch. 04) |
How to evaluate it
The nearest neighbours are the temporal-knowledge-graph memory layers (Zep, Graphiti). For a production mesh the useful comparison is not benchmark accuracy — it is whether the memory layer itself enforces roles, budgets, tenancy, provenance, and auditable deletion, or leaves each of those to the calling application. The table above is Attestor's answer; hold any alternative to the same checklist, with file references.
Designed for
- Multi-agent products where many LLM-driven agents with different trust levels write to the same memory
- Regulated systems that need point-in-time reconstruction and a defensible deletion story
- Self-hosted deployments — your VPC, your Postgres, your Pinecone, your Neo4j
Not designed for
- A personal memory for a single coding assistant — the Claude Code plugin below is an on-ramp demo of the same engine, not the product
- A general-purpose vector database or a RAG framework with chunking / reranking / orchestration
- An LLM agent runtime — Attestor is the memory backend; the agent loop is yours
Try it locally in one command.
pipx install attestor && attestor quickstartbrings up Postgres + Pinecone Local + Neo4j in Docker and wires the Claude Code MCP server + hooks so you can watch the governance layer work on a real project. Details in Quick start and Install for Claude Code below.
Quick start
1. Install
pip install attestor # or: pipx install attestor
Or pull the container (introspection-grade image, single layer over python:3.12-slim, currently linux/amd64):
docker pull ghcr.io/bolnet/attestor:latest # recommended — anonymous pull, mirrored to all registries below
Same image is mirrored to:
| Registry | Pull address |
|---|---|
| GHCR | ghcr.io/bolnet/attestor:latest |
| Docker Hub | bolnet2025/attestor:latest |
| Quay | quay.io/bolnet/attestor:latest |
| AWS ECR Public | public.ecr.aws/m6h5j7o3/attestor:latest |
| GCP AR | us-central1-docker.pkg.dev/coral-marker-452616-n4/attestor/attestor:latest |
(An internal Azure ACR mirror exists at memwright.azurecr.io/attestor but is private — Azure customers should use az acr import from one of the public registries above.)
The image's default entrypoint is attestor mcp (MCP server over stdio). For full production use, point the container at an external Postgres + Neo4j via env vars (or compose them with attestor/infra/local/docker-compose.yml); override the entrypoint to run attestor doctor, attestor api, etc.
2. Stand up the local stack — one command, zero questions
attestor quickstart
attestor quickstart does the whole local install non-interactively and prints every step: it writes ~/.attestor/{config.toml,attestor.yaml,.env}, brings up the three-role local stack in Docker, uses a local Ollama bge-m3 embedder (no cloud key), wires the Claude Code MCP server (./.mcp.json) + lifecycle hooks, and runs attestor doctor.
Prerequisites: Docker running, and Ollama serving bge-m3 (ollama pull bge-m3). quickstart runs a preflight that scans the ports/tools and tells you if anything is missing — it never prompts.
| Container | Role | Port | Purpose |
|---|---|---|---|
| Postgres 16 | Document | 5432 |
Source of truth — content, tags, entity, ts, provenance, RLS-isolated by user_id |
| Pinecone Local | Vector | 5080-5089 |
Dense embeddings, per-namespace isolation, plain gRPC (no HTTPS) |
| Neo4j 5 + GDS | Graph | 7687 |
Entity nodes + typed edges, PageRank / BFS / Leiden |
To reverse it later: attestor teardown (zero-question; keeps your data volumes by default — --purge also wipes them, --dry-run previews).
In Claude Code, drive the same install conversationally: /plugin marketplace add bolnet/attestor → /plugin install attestor (then enable it), and run /attestor:install-attestor — it runs attestor quickstart for you. Cloud/managed backends (Neon / RDS / Cloud SQL, Pinecone Cloud, Neo4j AuraDB) and alternative embedders (Pinecone Inference llama-text-embed-v2, Voyage voyage-4, OpenAI text-embedding-3) are configured in ~/.attestor/attestor.yaml (the single source of truth) — see docs/INSTALL.md.
attestor doctor (run automatically at the end, or any time) checks all four subsystems: Document Store (Postgres), Vector Store (Pinecone), Graph Store (Neo4j), Retrieval Pipeline. The only hard dependency that cannot be down is the document store (Postgres); transient vector-probe failures are surfaced in the response trace rather than swallowed (retrieval/orchestrator/debug.py — vector_error field).
3. Use it
from attestor import AgentMemory, AgentContext, AgentRole
mem = AgentMemory() # picks up env / ~/.attestor.toml automatically
ctx = AgentContext(
agent_id="researcher-1",
role=AgentRole.RESEARCHER,
namespace="acme-prod",
)
mem.add(
content="Alice is the engineering manager",
entity="alice",
category="role",
context=ctx,
)
results = mem.recall(query="who runs engineering?", context=ctx)
for r in results:
print(r.score, r.memory.content)
SOLO mode (zero-config). In v4,
AgentMemory().add('foo')auto-provisions a singletonlocaluser, an Inbox project (metadata.is_inbox=true), and a daily session — so the snippet above works on a fresh database without configuring identity (core.py:179-209). For multi-tenant production use, pass an explicitAgentContextwith a realnamespace.
4. Run a smoke benchmark (optional)
Verify your install end-to-end against a tiny LongMemEval slice. Defaults come from configs/attestor.yaml: Pinecone Inference llama-text-embed-v2 (1024-D) embedder + Pinecone vector store, openai/gpt-5.5 answerer, dual judges (openai/gpt-5.5 + anthropic/claude-sonnet-4-6), parallel=2.
set -a && source .env && set +a # OPENROUTER_API_KEY, PINECONE_API_KEY, NEO4J_PASSWORD
.venv/bin/python scripts/lme_smoke_local.py --n 2 --yes
Every model and parameter comes from YAML — see § Benchmarking below for the full bench harness.
Benchmarking
Every benchmark — smoke, single slice, full sweep, synthetic supersession — reads its knobs from two YAMLs:
| File | What lives there |
|---|---|
configs/attestor.yaml |
Stack — embedder, models, retrieval features, DBs, registries, clouds |
configs/bench.yaml |
Bench-only — variants, category iteration order, target scores, output paths |
The two files must have disjoint keys. The CI test tests/test_config_no_duplicate_keys.py enforces this; the bench loader (attestor.bench_config.get_bench) crashes on overlap. If you want a one-off override (different model for one bench run), use an env var or CLI flag — never duplicate the key in bench.yaml.
What LongMemEval is
LongMemEval (Wu et al., 2024 — published at ICLR '25) is the canonical benchmark for memory-augmented chat assistants. It measures whether an AI system can correctly answer questions that require recalling facts from long, multi-session conversation histories — the exact scenario Attestor is built for.
500 questions, 6 reasoning categories, 3 haystack sizes. Same questions across all three sizes; only the noise around the answer-bearing session changes:
| Variant | Tokens / Q | Sessions | What it measures |
|---|---|---|---|
oracle |
~3-15k | 1-3 gold | Reasoning ceiling — what the answerer can do with perfect retrieval. If you score low here, your prompt or LLM is broken (retrieval can't help). |
s (Standard / Small) |
~115k | ~50 | Public leaderboard — the canonical comparison. Fits in a single Claude/GPT context window, so Attestor's retrieval is benchmarked against the "just stuff everything into long context" baseline. |
m (Plus / Medium) |
~1M+ | ~500 | Pure retrieval — too big for any context window. Memory layer is forced; no long-context shortcut available. |
LME-S is the headline number to beat. A memory layer that scores within 5% of a long-context baseline at 30× lower token cost is the marketing pitch.
The 6 reasoning categories (cleaned LME-S, 500 questions total — note: no abstention slice in the cleaned split, which the synthetic supersession suite covers):
| Category | N | What it tests |
|---|---|---|
multi-session |
133 | Fact spans across multiple sessions — must track an entity over time |
temporal-reasoning |
133 | Date arithmetic ("two weeks ago", "before X") — Attestor's bi-temporal layer is built for this slice |
knowledge-update |
78 | Supersession — newer fact must beat older fact when both exist |
single-session-user |
70 | One session, fact stated by the user |
single-session-assistant |
56 | One session, fact stated by the assistant |
single-session-preference |
30 | One session, user preference |
Why this benchmark for Attestor: the temporal-reasoning and knowledge-update slices directly exercise features that distinguish Attestor from a vanilla RAG: bi-temporal recall, supersession-on-contradiction, event-time vs transaction-time disambiguation. A high score on those slices is the regulated-AI / audit / compliance pitch.
For the published Attestor numbers, see docs/bench/ — bench artifacts persist as lme-{variant}-{category}-{date}.{report,summary}.json. The Reporting section below shows how to render them as a table.
Download the LongMemEval dataset (one-time, before any bench run)
All lme_*.sh scripts use the cleaned LongMemEval split published on HuggingFace by xiaowu0162/longmemeval-cleaned. It auto-downloads on first use, but you'll want to know what's happening.
Cache location (created on first call):
~/.cache/attestor/longmemeval/
(Or $XDG_CACHE_HOME/attestor/longmemeval/ if you set XDG_CACHE_HOME.)
Variants and on-disk sizes:
| Variant | Filename | Size | Tokens / Q | Use |
|---|---|---|---|---|
oracle |
longmemeval_oracle.json |
~5 MB | ~3-15k | Reasoning ceiling — cheapest smoke |
s |
longmemeval_s_cleaned.json |
~250 MB | ~115k | Public leaderboard (canonical) |
m |
longmemeval_m_cleaned.json |
~2 GB | ~1M+ | Forces retrieval (no long-context shortcut) |
Option A — auto-download (recommended)
Just run any bench command. The first call downloads and caches; every subsequent call reads from disk:
# Will download longmemeval_oracle.json (~5 MB) the first time
.venv/bin/python scripts/lme_smoke_local.py --n 2 --yes --variant oracle
# Will download longmemeval_s_cleaned.json (~250 MB) the first time
scripts/bench/lme_run.sh knowledge-update
You only pay the download cost once per variant. Internet flake during the first run? Delete the partial file in the cache dir and rerun.
Option B — pre-warm the cache (offline / CI)
Pre-fetch every variant you plan to use before the bench day:
.venv/bin/python -c "
from attestor.longmemeval import load_or_download
for v in ('oracle', 's', 'm'):
samples = load_or_download(variant=v)
print(f'{v}: {len(samples)} samples')
"
Expected output:
oracle: 500 samples
s: 500 samples
m: 500 samples
Option C — manual download (firewalled environments)
If your runner can't reach huggingface.co, fetch the files on a connected machine and drop them into the cache dir manually:
mkdir -p ~/.cache/attestor/longmemeval
cd ~/.cache/attestor/longmemeval
# pick the variants you need
curl -L -o longmemeval_oracle.json \
https://huggingface.co/datasets/xiaowu0162/longmemeval-cleaned/resolve/main/longmemeval_oracle.json
curl -L -o longmemeval_s_cleaned.json \
https://huggingface.co/datasets/xiaowu0162/longmemeval-cleaned/resolve/main/longmemeval_s_cleaned.json
curl -L -o longmemeval_m_cleaned.json \
https://huggingface.co/datasets/xiaowu0162/longmemeval-cleaned/resolve/main/longmemeval_m_cleaned.json
The bench harness checks for these filenames exactly — don't rename them.
Verify the dataset is loadable
After download (auto or manual), confirm the loader picks it up cleanly:
.venv/bin/python -c "
from attestor.longmemeval import load_or_download
from collections import Counter
samples = load_or_download(variant='s')
cnt = Counter(s.question_type for s in samples)
print(f'Loaded {len(samples)} samples')
for cat, n in sorted(cnt.items(), key=lambda x: -x[1]):
print(f' {cat}: {n}')
"
Expected for the cleaned s variant (500 questions, 6 categories — note: no abstention slice in the cleaned split):
Loaded 500 samples
multi-session: 133
temporal-reasoning: 133
knowledge-update: 78
single-session-user: 70
single-session-assistant: 56
single-session-preference: 30
If counts don't match, the file is truncated — re-download.
Quick smoke (≤ 1 minute, ≤ $0.10)
Confirm the pipeline runs end-to-end before committing or running anything bigger:
.venv/bin/python scripts/lme_smoke_local.py --n 2 --yes --variant oracle
oracle is the cheapest variant (gold sessions only, no distractor haystack). Schema is reapplied automatically; pass --skip-schema if you want to keep a populated DB between runs.
Single category — scripts/bench/lme_run.sh
# all 6 categories, current variant from bench.yaml (default: s)
scripts/bench/lme_run.sh
# one slice — full
scripts/bench/lme_run.sh knowledge-update
# one slice — capped at N samples (smoke)
scripts/bench/lme_run.sh knowledge-update 10
# one slice on a different variant (oracle = cheapest, m = ~1M tokens)
scripts/bench/lme_run.sh knowledge-update "" oracle
Valid --category values: single-session-user, single-session-assistant, single-session-preference, multi-session, temporal-reasoning, knowledge-update. See What LongMemEval is above for sample counts and what each category tests.
Each run persists two files:
docs/bench/lme-{variant}-{category}-{YYYYMMDD}.report.json # full LMERunReport
docs/bench/lme-{variant}-{category}-{YYYYMMDD}.summary.json # BenchmarkSummary
Full sweep — scripts/bench/lme_all.sh
Iterates bench.yaml's lme.categories list in order. Adding/removing slices is a YAML edit, not a script edit:
# All 6 slices, current variant
scripts/bench/lme_all.sh
# All 6 slices, capped at 10 samples each (smoke)
scripts/bench/lme_all.sh 10
# All 6 slices on Oracle variant
scripts/bench/lme_all.sh "" oracle
If one slice fails, the script logs it and moves on to the next.
Pertanyaan tentang Attestor MCP
Bagaimana cara menambahkan Attestor MCP ke Claude Code?
Jalankan claude mcp add --env 'NEO4J_PASSWORD=<YOUR_NEO4J_PASSWORD>' --env 'OPENAI_API_KEY=<YOUR_OPENAI_API_KEY>' --env 'OPENROUTER_API_KEY=<YOUR_OPENROUTER_API_KEY>' agent-memory -- uvx memwright di terminal Anda. Tambahkan --scope user agar tersedia di semua proyek.
Bagaimana cara menambahkan Attestor MCP ke Cursor?
Buka ~/.cursor/mcp.json dan tambahkan server di bawah mcpServers, menggunakan config Cursor yang ditunjukkan di bagian instalasi di atas.
Apa itu Attestor?
Attestor adalah layanan memori self-hosted yang dirancang untuk multi-agent production meshes yang menerapkan role-based access, provenance, dan penghapusan yang dapat diaudit.