laya-rag MCP server
Commit Nov 3, 2026A local-first RAG engine that provides hybrid retrieval, relevance re-ranking, and answerability gating using calibrated probabilities.
Also known as thread3d/laya-rag
- Stars
- 0
- Tools
- —
- Downloads, 30 days
- —
- Latest version
- —
- Last commit
- Nov 3, 2026
- License
- Apache-2.0
What is laya-rag MCP?
laya-rag is a local-first RAG engine that provides hybrid retrieval, relevance re-ranking, and answerability gating using calibrated probabilities.
Summary generated from the project's README
About laya-rag MCP server
Developers needing high-precision retrieval for local applications can use this engine to provide evidence-based answers without the risk of hallucinations. It functions as a non-autoregressive decision engine that focuses on finding and validating relevant text passages rather than generating new prose. By using a hybrid recall method that combines dense embeddings with BM25, the system ensures that specific queries are matched to the correct documents even when standard cosine similarity fails to distinguish them.
The server runs locally on macOS or Linux via a Python environment and can utilize hardware accelerators like CUDA, ROCm, or Metal. Users interact with the engine through a command line interface, a web interface, or a local HTTP service on port 8077. It requires a Laya torch checkpoint to function and supports multi-tenant setups through API keys. Unlike generative models, it returns an evidence bundle with calibrated probabilities, allowing the caller to decide whether to compose a final answer based on verbatim spans.
What you can do with it
- 1Verify claim accuracyCheck if a generated answer is supported by specific source documents using deterministic citation checks.
- 2Filter irrelevant search resultsUse calibrated probabilities to set thresholds that separate highly relevant passages from noise.
- 3Prevent answering unsupported questionsGate the ability to answer queries by checking if the retrieved evidence actually contains the necessary information.
- 4Improve retrieval for technical queriesCombine dense and BM25 search methods to find specific documentation that standard vector search might miss.
Written from the project's README. Every use case is checked against a quote from it.
Configuration
LAYA_RAG_COMPOSE_BASE_URLLAYA_RAG_COMPOSE_MODEL
| Name | Description | |
|---|---|---|
| LAYA_RAG_COMPOSE_BASE_URL | — | |
| LAYA_RAG_COMPOSE_MODEL | — |
Automated checks
Ran Nov 15, 2026Full report →Not verified yet. A server needs a score of 80, no serious finding, and enough data for us to check. How checks work · What to fix for more points
- Source repository linked and reachablethread3d/laya-rag is public and reachablegithub.com/thread3d/laya-rag ↗Pass
- Package names this repository as its sourceNot checked
- Build provenance (Sigstore attestation)Not checked
- Open source license (OSI)Apache-2.0spdx.org/licenses/Apache-2.0.html ↗Pass
- Commit or release in the last yearLast commit or release 2026-11-03github.com/thread3d/laya-rag ↗Pass
- OpenSSF ScorecardNot checked
- Package name not close to a popular packageNot checked
- No critical advisory in the package or its dependenciesNot checked
- No hidden instructions to the model0 tool descriptions and the README read; no override phrases, credential paths or hidden charactersPass
- Destructive tools are labelledNot checked
- Remote endpoint answers the MCP handshakeNot checked
README
View on GitHublaya_rag
A retrieval-augmented generation system that uses Laya as its retrieval and decision engine, with a command line, an HTTP API and a web interface.
Version 3.0.10 — see CHANGELOG.md for what it contains, and what it
deliberately does not claim.
Clients meet the engine over a local HTTP service:

Why the split is shaped this way
Laya is not a generative model. It is a non-autoregressive decision engine: it evaluates typed
questions (choice, score, noul) over a body of text in a single forward pass and returns
calibrated probabilities. Its README is explicit — "No text generation, so nothing to parse and
nothing to hallucinate." It also ships no document loader, no chunker and no vector store.
That determines the architecture:
| Concern | Owner | Why |
|---|---|---|
| Chunking, persistence, retrieval | this project | Laya ships none of it |
| Embeddings | Laya | embed_fn_from_agent mean-pools the checkpoint encoder |
| Relevance re-ranking | Laya | calibrated noul decision, thresholdable |
| Answerability gating | Laya | calibrated noul decision over the evidence set |
| Writing the answer | the caller, or the optional compose step | Laya cannot generate text |
The engine returns evidence, not prose. It never invents an answer; it reports which passages support one, with a calibrated probability that they do, and abstains when they do not. Whoever consumes it — a person reading the interface, or the optional compose step — writes the final wording from exactly those passages.
What is actually measured here
Laya's decision head is dramatically sharper than the encoder's cosine similarity, which is why the engine spends compute on re-ranking rather than trusting dense retrieval:
| Passage (query: "How long does a refund take?") | Dense cosine | Laya relevance |
|---|---|---|
| Refunds are processed within five business days… | 0.357 | 0.790 |
| Shipping is free on orders over fifty dollars… | 0.284 | 0.001 |
| To reset your password, open Settings… | 0.283 | 0.001 |
| Our API rate limit is 600 requests per minute… | 0.138 | 0.011 |
Cosine barely separates the correct passage from unrelated ones; a 0.5 threshold on it would be meaningless. Laya's calibrated probability separates them by ~0.78, so thresholds are usable. Answerability gates the same way: 0.735 for evidence that answers the query against 0.0007 for a question the evidence cannot answer.
Cosine compression is also why the engine runs hybrid recall. Laya's encoder is mean-pooled and not retrieval-trained — its own README says a dedicated bi-encoder "will usually shortlist better" — and measured on this repository's own documentation it put the expected file outside the top 20 for 2 of 5 unambiguous queries. Adding BM25 and fusing with Reciprocal Rank Fusion fixed that:
| Query | Dense only | Hybrid (BM25 + dense) |
|---|---|---|
| "How do I use Laya with LangChain?" | miss | rank 1 (0.94) |
| "How do I run Laya in Docker?" | miss | rank 1 (0.91) |
| "How does structured output with Pydantic work?" | rank 2 | rank 1 (0.83) |
Requirements
- macOS or Linux, Python 3.10+
- ~3 GB disk for the Laya checkpoints and ~1 GB for the virtualenv
- No GPU required. Laya runs fp32 on CPU, and picks up an accelerator automatically when
available: CUDA, ROCm (which torch reaches through its CUDA API, so a ROCm host reports
cuda) or Metal (MPS)
This machine is an Intel Mac with no CUDA; the engine auto-selects MPS and answers a query in 2–5 s end to end. On CPU-only hosts it works but is slower.
DEVICE=auto — the default for every model this package loads itself, not only for Laya — means
"the fastest device the installed torch can reach". That is a property of the wheel as much as
of the hardware, so doctor prints what it resolved to alongside the request:
checkpoint : multilingual on auto -> cuda:0 - AMD Radeon Graphics (ROCm 6.1.40091-a8dbc0c19)
An explicit DEVICE=cpu, cuda, cuda:1 or mps is honoured verbatim, including one this host
cannot serve: a caller who asked for cuda gets the load error rather than a silent CPU fallback.
Setup
The Laya checkout lives in the vendor/laya submodule and ships a setup script pinned for macOS
Intel (PyTorch stopped publishing macOS x86_64 wheels after 2.2.2):
git clone --recurse-submodules <repository-url> laya-rag
cd laya-rag
# 1. Laya itself: venv, pinned deps, checkpoints, integrity check, smoke test
PYTHON=python3 vendor/laya/setup_laya.sh
# 2. This project: its own venv plus the service dependencies
make setup
make setup creates .venv, installs the same pins, installs Laya and this package editable, and
reuses the pip cache from step 1 so it is quick.
On an AMD host, install the ROCm wheel instead of the portable default:
make setup-rocm # torch==2.6.0+rocm6.1, from the ROCm wheel index
make setup-rocm ROCM_TORCH_SPEC=torch==2.6.0+rocm6.2 ROCM_TORCH_INDEX=https://download.pytorch.org/whl/rocm6.2
The accelerator is a property of the wheel, so this is the step that decides whether the GPU is
reachable at all: a CPU wheel on a ROCm host reports torch.cuda.is_available() == False and every
model resolves to cpu, producing CPU numbers with nothing in them saying so. make setup takes
the same choice through TORCH_SPEC and TORCH_INDEX for a CUDA box, and derives its numpy pin
from the torch spec because only the 2.2.2 wheel predates numpy 2.
Verify everything is wired up:
.venv/bin/laya-rag doctor
Using it
Command line
# Index a corpus (Markdown, source code, config, plain text)
.venv/bin/laya-rag index vendor/laya/docs --rebuild
# Ask for evidence
.venv/bin/laya-rag query "How do I run Laya in Docker?"
# Show configuration, model and corpus state
.venv/bin/laya-rag stats --json
# Deterministic citation/quote check of a composed answer against evidence files
.venv/bin/laya-rag verify "Refunds take five business days [1]." --evidence passage.txt
# Ask Laya whether the whole corpus supports each claim in an answer
.venv/bin/laya-rag adjudicate --answer-file answer.txt --tenant acme
# Report index drift: deleted documents, duplicate and near-duplicate chunks
.venv/bin/laya-rag integrity --near-duplicates
# Remove specific documents by id or path, without a rebuild
.venv/bin/laya-rag delete --doc-id doc-17 path/to/gone.md
# On the append-only on-disk backend, reclaim tombstoned space without re-embedding
.venv/bin/laya-rag compact
query prints Laya's calibrated verdict, the passages it selected with their sources, which
recall channel found each one, and the shape of the retrieval pool it built (requested vs
searched, each channel's contribution). verify is model-free and checks citations and quoted
spans only; adjudicate re-retrieves across the corpus and reuses the measured answerability
question. delete removes documents by id and/or path; on the append-only on-disk backend it
tombstones them and compact rewrites the store without the tombstones. integrity reads the index
and reports; it never repairs, because the one safe automatic rule (prune the paths of a directory
being re-indexed) already lives in index. Use --json for machine-readable output on any of them.
HTTP service
.venv/bin/laya-rag-serve # 127.0.0.1:8077
| Route | Purpose |
|---|---|
GET / |
the web GUI (see below) |
GET /healthz |
liveness: the process serves HTTP (cheap, never loads the model) |
GET /readyz |
readiness: model loadable + index present → 200, else 503 + Retry-After |
GET /health |
the legacy combined health report (deprecated in favour of the two probes) |
GET /v1/stats |
config, model and corpus statistics |
GET /v1/documents |
every indexed document with its chunk and character counts (?limit=&cursor=) |
GET /v1/chunks |
indexed chunks in document order, optionally ?path= one file (?limit=&cursor=) |
GET /v1/integrity |
index drift: deleted documents, duplicate and near-duplicate chunks (?near_duplicates=true&threshold=0.85) |
POST /v1/index |
{paths: [...], rebuild?: bool} — idempotent with Idempotency-Key |
DELETE /v1/index |
delete the index (standard verb; POST /v1/reset is the deprecated alias) |
DELETE /v1/documents |
{ids: [...]} or {paths: [...]} — remove specific documents without a rebuild; on the append-only on-disk backend this tombstones and compacts |
POST /v1/index/stream |
the same body, but streams running counters (progress events, then one done); NDJSON, or SSE with Accept: text/event-stream |
POST /v1/query |
{query, top_k?, retrieve_k?, relevance_threshold?, answerability_threshold?, max_passage_chars?} |
POST /v1/extract |
{query, top_k?, max_sentences?, min_score?, …} — answers by quoting the evidence; every sentence is a verbatim span with its [n], path and chunk id, and the passages travel with it, so the row is the engine's alone |
POST /v1/compose |
{query, passages: [...]} — streams composed prose (NDJSON or SSE) |
POST /v1/verify |
{answer, passages: [...]} — deterministic citation and quote checks |
POST /v1/adjudicate |
{answer, …} — Laya decides whether the corpus supports each claim |
POST /v1/reset |
delete the index (deprecated) |
The contract
Other software is meant to depend on this service, so the surface is documented and tested rather than incidental:
- Errors are RFC 9457
application/problem+jsonwith a stablecode(e.g.index_not_built), arequest_id, and field-levelerrorswith JSON Pointers.detailis always a string. TheWWW-Authenticatechallenge is sent on 401/403. - Correlation: every response carries
X-Request-ID(an inbound one is echoed). A valid W3Ctraceparentis reused and its trace id returned inX-Trace-IDand in problem bodies. - OpenAPI: the typed document is at
/openapi.json(and/docs). Every success and error response is a named schema, including the two stream media types. SetLAYA_RAG_SCHEMA_PUBLIC=falseto disable the schema routes on an authenticated service. - Pagination on the two listings is keyset via an opaque
cursor; withoutlimitthe full listing is returned, andnext_cursoris null on the last page. - Idempotency:
Idempotency-Keyon the mutating routes replays the first response (withIdempotency-Replayed: true); a key reused with a different payload is 422, and a duplicate while the first is running is 409. - Versioning: routes live under
/v1. A replaced route is marked with the RFC 9745Deprecationheader (@-Unix-seconds) and aLink: rel="successor-version".
Authorization and tenancy
LAYA_RAG_API_KEY is a single admin token, kept for existing deployments. For per-token
tenants and scopes, set LAYA_RAG_API_KEYS to a JSON map (or a path to one):
{
"acme-read": {"tenant": "acme", "scopes": ["read"]},
"acme-admin": {"tenant": "acme", "scopes": ["read", "write"]},
"ops": {"scopes": ["read", "write"]}
}
A token's tenant is the only tenant it may query; a body tenant that disagrees is
403, so the trust manifest's per-tenant ACL can no longer be spoofed. A principal with no
tenant (or the legacy admin key) may select one. read covers query/verify/adjudicate and
the listings; write covers index/delete. Indexing is confined to LAYA_RAG_INDEX_ROOTS;
with no allowlist it is permitted only on a loopback bind, so a service reachable from
elsewhere cannot be asked to read arbitrary host paths.
MCP server
The engine also speaks the Model Context Protocol, so an agent host can call it directly. The SDK is an optional extra:
.venv/bin/pip install -e '.[mcp]'
make mcp # stdio transport (what Claude Desktop / Cursor launch)
make mcp-http # streamable HTTP on MCP_PORT (default 8078)
Claude Desktop / Cursor config (use absolute paths):
{
"mcpServers": {
"laya-rag": {
"command": "/absolute/path/to/laya-rag/.venv/bin/laya-rag-mcp"
}
}
}
Tools: search_evidence (the calibrated evidence call), list_documents,
read_document, corpus_stats, check_index_integrity, extract_answer (answers by
quoting the retrieved passages verbatim, with no model in the path, and abstains when
nothing qualifies), verify_answer, adjudicate_answer, compose_answer, and — only with
LAYA_RAG_MCP_ALLOW_WRITES=1 — index_documents and reset_index (the latter requires
confirm="reset"). The search_evidence description states the honesty rule plainly: when
answerable is false, the passages do not support an answer and the agent should abstain. Resources
(laya-rag://stats, laya-rag://documents) and two prompts (answer_from_corpus,
audit_answer) are also exposed. By default the server runs the engine in-process, like
the CLI; LAYA_RAG_MCP_BACKEND=http proxies to a running service instead. The HTTP
transport refuses to start beyond loopback without a bearer token.
Web GUI
The engine serves its own interface at the root, so one command is the whole setup:
.venv/bin/laya-rag-serve # then open http://127.0.0.1:8077/
It is plain HTML, CSS and JavaScript shipped inside the Python package — no bundler, no npm, no CDN and no network access, which keeps the "works offline" property the rest of the project has. Serving the GUI from the same process as the JSON API is also what makes it need no configuration: the page calls the API on its own origin.
What it does:
- Evidence requests. Ask a question and read Laya's verdict as the primary result — a calibrated
P(answerable)with the decision threshold drawn on the meter, the candidate funnel (considered → kept → below relevance), and per-stage timings. Retrieval controls are seeded from the engine's own configuration rather than hard-coded, and each passage shows its Laya relevance beside its recall score and which recall channel found it (dense,lexical, or both). - One entry per document. Retrieval works in chunks, so a single file can arrive several times.
The evidence is grouped by source document — at its best-ranked passage's position, with that
document's chunks joined in rank order and the chunker's overlap trimmed — so a document is
numbered once instead of spending several numbers on the same file. The composed answer's context,
the numbered passages and the citation check all use that grouping, so
[n]names the same evidence in all three. - Honest abstention. When nothing clears the relevance bar the engine short-circuits and reports
answerability = 0.0without ever asking Laya the answerability question. The GUI detects this and renders "gate not reached" instead of drawing that default as a measured probability. - Corpus browser. Every indexed file with chunk counts, and a viewer for the exact chunks Laya sees, so a surprising ranking can be traced to the text that produced it.
- Indexing and reset. Index paths or directories (optionally rebuilding), with the engine's report — including which paths it refused and why. A build reports running counts as it goes (files read, documents chunked, chunks embedded) rather than leaving a bare spinner.
- Evidence export. Copy the grouped evidence as markdown, numbered to match the citations, or the raw query response as JSON — chunks in retrieval order, exactly as the engine sent it.
Composing an answer (opt-in, and not Laya's). The switch sits at the end of the evidence, below the passages. Off — the default — the engine behaves exactly as described above and returns evidence only. On, a generative model writes a prose answer from the grouped evidence on screen, streamed into the page as it is written, and the footer banner turns into a warning that the answer may contain hallucinations. It appears after the passages it was written from — Laya's calibrated verdict and evidence come first — badged as unverified, and it is never included in the "Copy markdown" evidence export, which stays a record of Laya's evidence, never the answer.
This is the one part of the pipeline Laya is not responsible for, so it is worth being precise about what is and is not calibrated:
| Laya's verdict and passages | The composed answer | |
|---|---|---|
| Source | calibrated noul decisions |
a generative model |
| Thresholdable | yes | no |
| Can invent facts | no — it cannot generate text | yes |
The model is configured here: LAYA_RAG_COMPOSE_MODEL names it and LAYA_RAG_COMPOSE_BASE_URL the
endpoint, both defaulting to a local Ollama and the model ornith-1.5:35b and both overridable from
this project's own environment.
Composition is unavailable — with the reason shown in the interface — when it is turned off with
LAYA_RAG_COMPOSE_MODEL=none, and the switch is disabled rather than offering something that cannot
work. If the model is unreachable or times out mid-answer, the prose is dropped or marked incomplete,
the evidence below is left untouched, and the warning stays up.
Three implementation details behind that. The request disables the model's thinking mode, because
otherwise the reasoning tokens can consume the entire budget and return empty content — measured at
41 s for no answer versus sub-second with it off. A truncated answer (the model hitting its token
limit) is reported as truncated, since a cut-off answer otherwise looks identical to a complete one.
And the question is stated twice, before and after the evidence, which is not redundancy: on a
short topical query (Laya in docker) the model asked for the question in 3 of 5 runs when it
appeared only after the evidence, in 3 of 3 when it appeared only before, and in 0 of 8 when
it appeared at both ends — with answers several times longer besides. A large evidence block buries a
short trailing question, so do not tidy that repetition away without re-measuring.
The first two depend on think: false actually being honoured, and it is not honoured on every
route. Ollama's OpenAI-compatible /v1/chat/completions silently ignores the flag: measured over
three calls it still produced 561–614 characters of reasoning each time, took 19–21 s, and once spent
an entire 1200-token budget thinking and wrote no answer at all. The native /api/chat route honours
it — thinking: 0, ~7 s for the same prompt. That is why COMPOSE_API defaults to ollama and
reaches for the native route, deriving <root>/api/chat by dropping the base URL's /v1 suffix. Set
COMPOSE_API=openai only for an endpoint that is not Ollama.
The citation check
Every composed answer is followed by a citation check, which is deterministic: no model, no threshold, no probability to misread. It reports whether each claim cites in-range evidence, whether any citation points outside the passages, and whether each quoted span actually appears in the evidence. Quotes are matched after folding case, whitespace and quote style — a fold, not a fuzzy match, so "found" means the text really is there.
It is deliberately not a green "verified" badge. The wording bounds the claim, because the check answers a narrower question than it looks like it does:
| The citation check does | The citation check does not |
|---|---|
| confirm every claim carries a citation | confirm the citation supports the claim |
| catch citations to passages that do not exist | catch a misreading of a real passage |
| catch quotes that are not in the evidence | catch a claim that is true but uncited |
| count the claims it could not check | establish that the answer is true |
The two directions matter differently. "Citation out of range" and "quote not found" are decided mechanically, so they cannot themselves be wrong. "Uncited" is a prompt-compliance signal, not a truth signal — which is why the composition prompt states the citation rule as a hard requirement: a first real run produced twelve uncited claims out of twelve while every quoted span was verbatim, and a check that always reports "uncited" tells you nothing. With the rule stated explicitly, the same query produced 8 of 9 claims cited with all 8 quoted spans verbatim.
What is still missing is the harder half — whether a correctly-cited claim is a fair reading of its source. That needs Laya's adjudication, not string matching.
Corpus support
Below the citation check, the second half of verification asks Laya — not the model, and not a string comparison — whether the corpus actually supports each claim. For every claim it re-retrieves candidates across the whole index (dense + BM25 fused by RRF) and adjudicates them with Laya's calibrated decisions, so each claim ends up with a probability and the passages that produced it.
Three choices shape it:
- The corpus, not just the shown passages. A claim can be true of the corpus and simply not have been among the five passages the model was handed. Re-retrieval is what tells those apart.
- The already-measured question. It reuses
ANSWERABILITY_QUESTION— "Canquerybe answered using onlyevidence?" — with the claim standing in as the query. A newly-worded claim-support question would carry a calibration nobody has measured, and Laya's own docs warn thatnoulis sensitive to wording and that the question type in use must be tested on the data you intend to serve. The report names the question it used so it cannot be mistaken for a benchmarked claim-support decision. - "Not measured" is not zero. When no candidate clears the relevance bar, the engine
short-circuits and Laya is never asked. Those claims are reported as not measured rather than
drawn as a confident
0.000— the same distinction the query verdict makes.
Cost is linear in claims × candidates, which is why the candidate pool here is narrower than the
query path's: measured warm over five claims, CLAIM_RETRIEVE_K=20 took 21.6 s against 8.7 s at
10, for the same verdicts on the sample. The interface shows elapsed progress rather than a bare
spinner, because this stage takes seconds.
Together the two layers catch different failures, which is why both are worth having: a fabricated claim that does carry a valid citation — "Laya was first released in 2019 by a team in Iceland [1]" — passes the citation check cleanly and is caught only by adjudication, while a misquoted passage is caught by the citation check alone.
Two small things worth knowing. The page itself loads without a token so it can ask for one: if
LAYA_RAG_API_KEY is set, paste it under Connection and it is kept in this browser's local
storage only, then sent as a bearer token. And the GUI needs no restart to reflect a change on disk —
it re-reads engine and corpus state, and has a Refresh button besides.