MCP server PayPilot

PayPilot

A worked example of an agent that handles money and can be told no: failed subscription payments, decided by a fixed rules table, recorded with the rule that fired, and sent only after a person approves.

Live demo -> paypilot.fly.dev - try it in the browser, no setup or API key required.

The hard part of putting a model near money is not the model. It is the part where the agent can act and somebody has to answer for it. PayPilot is built around that part. A failed payment arrives, a fixed rules table (app/decision.py) decides what happens next, the model writes only the wording, every decision stores the rule that fired, and nothing leaves without a person's approval. The recovery email is the worked example; the refusals are the point.

When a recurring charge fails, PayPilot turns the invoice.payment_failed event into a diagnosis, a retry strategy and a drafted dunning email in a single API call.

It's built as a small, readable LangGraph agent with retrieval-augmented generation (RAG) over a dunning playbook, exposed through a FastAPI endpoint. The whole thing runs its test suite with no API key and no network.

---

How it works

A failed-payment event flows through a seven-node LangGraph StateGraph. Each node enriches a shared, typed RecoveryState and hands it to the next:

flowchart LR
    A[retrieve_context] --> R[assess_risk]
    R --> B[diagnose_reason]
    B --> C[choose_strategy]
    C --> S[schedule_retry]
    S --> D[draft_message]
    D --> E[finalize]
    E --> F([END])

Node	What it does
retrieve_context	Loads the customer record and pulls relevant snippets from the dunning playbook via the RAG retriever.
assess_risk	Deterministic (no LLM): scores churn risk (low/medium/high) from the dunning attempt number and the customer's recent failure streak.
diagnose_reason	Committed template by default, LLM only with PAYPILOT_LLM_DRAFT=1: a 1-2 sentence, playbook-grounded diagnosis of why the payment failed, reflecting the churn risk.
choose_strategy	Deterministic (no LLM): maps the failure code to a fixed action + retry cadence, then tightens it when churn risk is high. Stable and unit-testable.
schedule_retry	Deterministic (no LLM): turns the cadence into a concrete next_retry_at UTC time, ready to hand to a scheduler.
draft_message	Committed template by default, LLM only with PAYPILOT_LLM_DRAFT=1: a short, warm dunning email with one clear call to action.
finalize	Assembles the {diagnosis, risk, strategy, schedule, message, impact} response payload.

Why RAG?

The recovery quality depends on dunning best-practice - retry timing, tone, when to offer a grace period. Rather than bake that into prompts, PayPilot keeps it in an editable knowledge source (data/playbook.md) that the retriever feeds into the diagnosis and drafting nodes.

Which retriever depends on configuration, and it is worth being precise about this. With OPENAI_API_KEY set, app/ingest.py builds a FAISS index over the playbook using OpenAI embeddings (k=3). With no key - which is how the public demo runs - it falls back to a lexical keyword retriever, so no FAISS index and no embedding call is involved in anything a visitor sees.

And the playbook only changes the output on the LLM path. On the default zero-inference path the committed templates are rendered as-is and retrieved context is not consulted, so editing playbook.md changes nothing until PAYPILOT_LLM_DRAFT=1 is set. Playbook edits are input to the build-time generation step, not to every request.

What it refuses, and what is not built

• It does not send unless sending is switched on, a person approved the message and the recipient is on the allowlist. An empty allowlist means nobody (app/mailer.py).
• A draft carrying a link it did not sanction, or anything shaped like a secret, is swapped for fixed copy; a send that fails the same check is refused (app/safety.py).
• It does not return a decision it could not record: no audit row, a 503 (app/decision.py).
• A code the table does not list fires a row named default, and the decision's gates_checked receipt says fell_through rather than matched.

A refusal view, in two parts. A body POST /decide refuses leaves a row in decision_refusals: the gate that refused and a machine reason code, never the rejected input, and the 422 carries the row's refusal_id. Every mailer and approval refusal was already recorded as a suppressed message with a reason code, so those are not copied. GET /refusals (admin-gated, read-only) lists both, newest first, with the gate each reason belongs to (app/refusals.py). A test walks the mailer and approval source and fails when a reason code the code can write has no gate, and a reason the map does not know is listed as unclassified rather than dropped. It also lists decisions that fell through to the default row, and drafts the output guard replaced with fixed copy. That last source is the optional event log (PAYPILOT_AUDIT_DB_PATH), which is absent on the public deployment on purpose, so the response carries a sources map saying which sources were read: an empty swap list with audit_event_log: not_configured is not the same as no swaps. In multi mode that log is platform-level and is not listed (not_tenant_scoped). In single mode the view reads decision refusals only for the client id equal to the tenant id; a different token map client id is not listed. The decision audit has no update or delete on the record path; erasing a client for GDPR is a separate, named function. See docs/claims.md.

Why a deterministic strategy node?

choose_strategy is intentionally not an LLM call. Retry cadence and the chosen action come from a fixed rules table (app/decision.py) keyed on the failure code, which comes from Stripe, Recharge, Chargebee or Loop:

Failure code	Retry in	Action	Tone
card_expired	~1 day	request card update	Friendly, routine
insufficient_funds	~3 days	wait and retry	Soft, no pressure
generic_decline	~2 days	retry and verify	Calm, helpful
issuer_do_not_retry	never	request new payment method	Plain, no blame
card_details_invalid	never	request card update	Plain, no blame
direct_debit_not_retried	never	request manual payment	Plain, no blame
retries_exhausted	never	request new payment method	Plain, no blame

The LLM writes the message; the policy stays predictable.

irreversible-reach maps which model-produced values in an agent can reach an action that cannot be undone. Its PayPilot manifest marks sending an email as PayPilot's only irreversible action, with deterministic guards in front of it; remove those guards and it fails. A manifest describes code, it does not test it.

Risk-aware escalation

assess_risk reads the dunning attempt number and the customer's recent payment history (from data/customers.json) and buckets churn risk. When it's high - a third attempt, or a run of recent failures - choose_strategy tightens the retry cadence and marks the strategy escalated, the diagnosis calls out the urgency, and impact discounts the recovery odds for a customer who keeps bouncing. So the agent reasons about history, not just the single event in front of it.

---

Quickstart

# 1. Install
python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt

# 2. Configure (only needed to call the live LLM; tests don't need it)
cp .env.example .env   # then add your OPENAI_API_KEY

# 3. Run the API. PAYPILOT_TENANT=demo serves the bundled demo fixtures (data/customers.json).
PAYPILOT_TENANT=demo uvicorn app.api:app --reload

Call it

curl -s http://localhost:8000/payment-failed \
  -H 'Content-Type: application/json' \
  -d '{
        "customer_id": "cust_001",
        "amount": 1499.0,
        "currency": "usd",
        "failure_code": "card_expired",
        "attempt": 1
      }' | jq

{
  "diagnosis": "The card on file for Acme Robotics has expired, so the Scale renewal couldn't be charged; ...",
  "risk": { "attempt": 1, "prior_failures": 0, "churn_risk": "low", "escalate": false },
  "strategy": { "action": "request_card_update", "retry_in_days": 1, "offer": "...", "escalated": false },
  "schedule": { "retry_in_days": 1, "next_retry_at": "2026-07-02T09:00:00+00:00", "retry_on": "2026-07-02", "timezone": "UTC" },
  "message": "Hello Acme Robotics, we tried to renew your Scale plan but the card we have on file has expired ...",
  "impact": { "amount_at_risk": 1499.0, "currency": "USD", "recovery_likelihood": 0.7, "expected_recovered": 1049.3, "annual_value_at_risk": 17988.0, "churn_risk": "low" }
}

GET /health returns {"status": "ok"} for liveness checks. The full response schema (typed with pydantic) is browsable at /docs.

The endpoint is rate limited per client IP, and the response payload is a typed RecoveryResponse (diagnosis, risk, strategy, schedule, message, impact), so the contract shows up precisely in the OpenAPI docs.

POST /payment-failed/batch runs a whole billing run (up to 50 events) in one call and adds a portfolio aggregate - total at risk, total expected recovered, and how many accounts are high churn risk. GET /portfolio-impact rolls that up across the demo customers and powers the recoverable-revenue headline on the landing page.

Speaks Stripe

Status: no client traffic reaches this deployment, so no merchant's live Stripe delivery has either. The handler is exercised offline by tests/test_stripe.py.

POST /webhooks/stripe accepts a real Stripe invoice.payment_failed event. It verifies the Stripe-Signature header (HMAC-SHA256) when STRIPE_WEBHOOK_SECRET is set, acknowledges other event types with a 200 so Stripe won't retry, maps Stripe decline codes (expired_card, insufficient_funds, ...) to PayPilot's failure codes, and runs the recovery graph. Point a webhook (or stripe trigger invoice.payment_failed) at it; add metadata.paypilot_customer_id to resolve a demo customer.

Speaks Recharge

Status: this deployment carries no client traffic, so no merchant's live Recharge delivery has reached it. The handler is exercised offline by tests/test_recharge.py.

POST /webhooks/recharge accepts Recharge charge/failed and charge/max_retries_reached deliveries for Shopify subscription stores. It checks X-Recharge-Hmac-Sha256 against RECHARGE_CLIENT_SECRET the way Recharge documents it, and maps every one of Recharge's 109 published error types to a PayPilot failure code (the full table). Recharge runs its own retries, so when it will retry, its retry_date is reported as the schedule with source="recharge" and no customer email is drafted (message_suppressed: "recharge_retry_scheduled"). Errors that are not a payment problem the customer can fix (inventory, shipping, tax, test mode) and error types PayPilot does not recognise are acknowledged and not run, so no customer gets an email about them.

A merchant comes on in three steps. They create an API token and copy the API Client Secret from their Recharge admin; the secret goes on the deployment as RECHARGE_CLIENT_SECRET; then RECHARGE_API_TOKEN=... python scripts/recharge_register_webhooks.py --apply registers both webhooks against this deployment (dry run without --apply, and safe to re-run).

The webhook is idempotent on the Stripe event id, so a retried delivery replays the stored result instead of re-running the graph. POST /payment-failed and /batch accept an optional Idempotency-Key header for the same guarantee. Every response carries X-Process-Time and X-Request-ID headers, emits a structured JSON access log, and 429s include Retry-After. GET /metrics (admin token required) returns a JSON snapshot (request counts by status, average latency, recoveries run, total expected recovered). Batches roll up per currency, so a mixed USD/EUR/GBP billing run stays correct (aggregate.by_currency). GET /usage (admin token, hidden from the schema) answers "has anyone outside called the API": a tally per route and client network prefix that survives restarts, stored beside the database on the volume. It keeps the route template, status, a coarse /24 prefix and a count; no body, no headers, no query string, no full address, and rows are deleted after 90 days. Pass ?exclude=<your prefixes> to remove your own traffic.

For discovery, the app also serves /robots.txt, /sitemap.xml, and an /llms.txt summary for AI answer engines, and the landing page ships SoftwareApplication + FAQPage JSON-LD.

---

The closed recovery loop

PayPilot does not stop at drafting. It records what failed, what it sent, and what happened next, so "we recovered X" is a figure you can audit rather than a claim.

invoice.payment_failed  ->  record  ->  strategy  ->  portal link  ->  email
                                                                        |
        invoice.paid / payment_succeeded  ->  recovered  <---------------+
        customer.subscription.deleted     ->  churned

Per-invoice state machine: failed -> messaged -> clicked -> recovered | churned | exhausted. Illegal moves raise rather than silently overwrite, so the dashboard can never contradict revenue it already reported. Amounts are stored in integer minor units with a currency code. Floats appear only at the presentation edge, never in storage or arithmetic.

State only advances to messaged on a real send. A dry run, a suppressed recipient or a provider failure leaves the invoice at failed, because claiming we contacted someone we did not is exactly what this ledger exists to prevent.

An invoice we never saw fail is never counted as a recovery. Most invoices in a Stripe account are paid without ever failing, and counting those would inflate the one number the product is judged on.

Honest attribution

Some failed invoices recover on their own: Stripe retries them, and customers fix their cards unprompted. So /report splits invoices into three arms - treated (actually messaged), holdout (deliberately withheld) and untouched (a dry run or suppressed send) - and reports each separately. A lift figure is withheld until both arms reach 30 invoices, and says so rather than printing a confident percentage from four data points.

Holdout assignment is a deterministic hash of the invoice id, stable across restarts and reproducible from invoice ids alone. It defaults to 0 percent: withholding dunning from paying customers is a decision, not a default.

A public sample of the dashboard, on a fixed cohort and labelled as sample data, is at /report/sample. The real one is admin-gated: it holds revenue data and reveals which invoices were withheld from dunning.

Watch every refusal gate fire

make refusal-walk sends seven invented requests through the real code paths (a refused body, a fall-through, an allowlist refusal, an output-guard refusal, a swapped draft) and two controls that must NOT appear (a matched decision and a provider failure), prints what each did and the view GET /refusals serves, and exits non-zero if an expected row is missing or an unexpected one is present. No key, no network, nothing leaves the machine. tests/test_refusal_walk.py runs it in the suite. The same run is published at /refusals-sample, rendered by scripts/render_refusals_page.py from what the walk returned, so a gate that stops refusing makes the page stale and the build red. Every row on it is invented.

Prove it end to end

PAYPILOT_DEMO_EMAIL=you@example.com make demo-loop

Drives Stripe test mode through the whole cycle with a test clock: a subscription is created and paid, its card goes bad, a month passes, the renewal genuinely fails, the recovery runs, the card is fixed, the invoice is paid, and the dashboard is printed before and after. It refuses to run against a live key.

---

Zero-token architecture

The default path performs no inference at all. A dunning email for a given failure code is the same class of output every time, so the copy is generated once, reviewed by a human, committed as data/templates/dunning.json, and filled deterministically at runtime. That makes what a customer reads reviewable the way code is reviewable: it diffs, and changing it is a pull request.

An OPENAI_API_KEY alone does not enable chat inference; live drafting also requires PAYPILOT_LLM_DRAFT=1.

One honest caveat: a key does still enable embeddings. With OPENAI_API_KEY set, the retriever builds a FAISS index over the playbook once per process (a lazy singleton in app/ingest.py) and then embeds the query on each request. So the index cost is paid once, but per-request query embedding is a real, small token cost on a path otherwise described as zero-inference.

Three CI gates keep the rest honest: a full recovery must construct no chat model, a ChatOpenAI( call site without a written BUILD-TIME / CACHEABLE / TRUE-RUNTIME classification fails the build, and the committed copy must cover every failure code and contain no URL. The call-site gate matches on that literal string, so it would not catch a different SDK or a call outside app/.

Multilingual copy, on the same seam

A locale is a dimension on the committed catalogue rather than a second system: the generator writes it, a human reviews it, the runtime fills it, and no model is called on the path. Two decisions are worth naming.

A locale is all or nothing. A half-translated catalogue is not merged key by key with English, because that mails one customer a Spanish subject over an English body, which reads as broken software rather than as a missing translation. A catalogue that does not cover every kind and every failure code is refused as a locale and English is served whole, with the reason readable per locale (incomplete_locales()). Completeness is measured against the English catalogue rather than a fixed list, so adding a failure code to English makes every translation incomplete until it is translated, which is the direction that fails safe.

The fallback is reported, not silent. render_with_locale() returns the locale actually used, because a silent fallback is indistinguishable from a translation that is working, and a climbing fallback count is the measurement that says which language to translate next. A missing catalogue falls back rather than failing closed: an English dunning email recovers money where no email recovers none. Missing copy altogether is still fatal.

A language is never derived from the currency, and a test greps the module's own source to keep it that way. EUR spans twenty countries, USD is legal tender in Ecuador and Panama, and GBP is not a language.

State today, measured rather than implied: available_locales() returns ['en']. The machinery is complete and runnable and there is one catalogue in it. scripts/generate_templates.py --locale translates the live English rather than regenerating per language, so the languages cannot drift apart in meaning, and it refuses a draft that renamed a {name}, {plan} or {business} placeholder, since str.format would then leave a customer reading a literal brace. Producing a reviewed Spanish catalogue is one command and a speaker's review, not a build.

---

An MCP server over the decision table

An agent asks, the table decides, and the answer carries which rule fired. The model scores, suggests and drafts; governed logic decides. app/mcp_server.py is that sentence made callable: an MCP server whose three tools read a fixed decision table, so the answer cannot be argued with by anything the agent read on the way in.

Seventeen ACH return codes, five decisions. There are five answers to the one question that decides what happens next: what has to change before this debit can exist again. Nothing, the account details, their instruction, our own entry, or nothing ever will. A table with one row per code is a reference document and the registrar publishes that for free. A table with one row per decision is a rule set, and that is the part a team cannot download.

Row	Goes again	What has to change first	A person acts first	Codes
1	yes	nothing	no	R01, R09
2	not on this entry	account details	no	R02, R03, R04, R13, R20, R28
3	not until they ask	their instruction	no	R08
4	corrected only	our own entry	no	R11
5	no	nothing will	yes	R05, R07, R10, R14, R15, R16, R29

It is published at paypilot.fly.dev/ach-returns and readable in the repository at docs/ach-return-decisions.md. Both are generated from app/ach_return_map.py by scripts/render_ach_page.py, so neither holds the table and neither can disagree with it.

R11 is the row worth the table. It arrives in the unauthorized family and is still correctable, so it is the one code where the family and the decision disagree, and the one thing here that is hard to know.

The default is visible as a default. Every code from R00 to R99 that the table does not list lands on row 5, where nothing automatic happens, and the answer comes back with matched set to false and a reason saying it fell through rather than matched. It is never null and it is never row 1. A caller that cannot tell a match from a fallback has been handed a guess dressed as an answer.

The three tools

Tool	Answers
lookup_return_code	one code: its row, whether the entry goes again, what has to change first, the wording for the customer, and whether a named person acts first. R01 and R09 also carry the reinitiation limit (at most two times, within 180 days of the original settlement date) with its source; no other code states one
list_decisions	the five rows and the codes on each, so the table is legible to an agent rather than only queryable
explain	which row fired and why, plus what the table does and does not state about the code's family

Nothing writes, nothing calls a model, and nothing reaches the network. That is declared per tool as read_only_hint and open_world_hint: false rather than promised in prose, so a client can read it off the tool list and tests/test_mcp_server.py asserts it. The same suite runs every tool again with socket.socket replaced by a function that raises, which is the claim measured rather than intended.