Convierte descripciones de empleo desordenadas en JSON con tipos y validación de esquema mediante agentes LLM y reintentos con retroalimentación de errores.
También conocido como nilayp-codes/typecast, typecast-mcp
- Stars
- 1
- Herramientas
- —
- Descargas, 30 días
- 180
- Última versión
- 0.1.0
- Último commit
- 6 nov 2026
- Licencia
- MIT
¿Qué es Typecast MCP?
Convierte descripciones de empleo desordenadas en JSON con tipos y validación de esquema mediante agentes LLM y reintentos con retroalimentación de errores.
Resumen generado a partir del README del proyecto
Acerca del MCP server de Typecast
Los desarrolladores que necesiten transformar descripciones de empleo no estructuradas en datos estructurados y tipados pueden usar Typecast para automatizar el proceso de extracción. Esta herramienta utiliza agentes LLM para convertir texto desordenado en JSON validado mediante esquemas, enfocándose específicamente en la creación de modelos de ofertas de empleo fiables. Está diseñada para funcionar con modelos de pesos abiertos como Qwen a través de OpenRouter, proporcionando una forma de medir la precisión y la rentabilidad de diferentes estrategias de prompting.
El sistema funciona generando prompts a partir de modelos Pydantic para asegurar que el esquema nunca se escriba a mano. Admite varios modos operativos, incluyendo prompting simple, modo schema específico del proveedor y bucles de reintento con retroalimentación de errores que permiten hasta tres intentos para corregir errores de validación. Los usuarios pueden elegir entre configuraciones que priorizan el bajo costo o una mayor precisión de extremo a extremo, mientras el servidor se encarga del trabajo pesado de validar strings sin procesar contra el esquema JSON esperado y registrar cualquier fallo de extracción.
Qué puedes hacer con él
- 1Convertir texto de empleo a JSONTransforma una descripción de puesto desordenada en un objeto JSON estructurado y validado mediante esquema.
- 2Automatizar la corrección de erroresUtiliza reintentos con retroalimentación de errores para corregir automáticamente los fallos de validación en las salidas del LLM.
- 3Evaluar la precisión de la extracciónCompara el costo y la precisión de los campos entre diferentes configuraciones de modelos y métodos de prompting.
- 4Imponer restricciones de datos estrictasAsegura que las salidas del modelo cumplan con restricciones específicas, como límites de elementos dentro de un esquema.
Escrito a partir del README del proyecto. Cada caso de uso se verifica con una cita del mismo.
Cómo instalar Typecast MCP
Formatos verificados el 10 nov 2026Añadir a CursorAñadir a GooseInstalar Typecast MCP en Claude Code
$ claude mcp add typecast -- uvx typecast-mcpInstalar Typecast MCP en Cursor
~/.cursor/mcp.json
{
"mcpServers": {
"typecast": {
"command": "uvx",
"args": [
"typecast-mcp"
]
}
}
}Instalar Typecast MCP en VS Code
.vscode/mcp.json
{
"servers": {
"typecast": {
"type": "stdio",
"command": "uvx",
"args": [
"typecast-mcp"
]
}
}
}Instalar Typecast MCP en VS Code CLI
$ code --add-mcp '{"name":"typecast","type":"stdio","command":"uvx","args":["typecast-mcp"]}'Instalar Typecast MCP en Claude Desktop
claude_desktop_config.json
{
"mcpServers": {
"typecast": {
"command": "uvx",
"args": [
"typecast-mcp"
]
}
}
}Instalar Typecast MCP en Codex
$ codex mcp add typecast -- uvx typecast-mcpInstalar Typecast MCP en Codex config.toml
~/.codex/config.toml
[mcp_servers.typecast] command = "uvx" args = ["typecast-mcp"]
Instalar Typecast MCP en Gemini CLI
$ gemini mcp add -s user typecast uvx -- typecast-mcpInstalar Typecast MCP en Gemini settings.json
~/.gemini/settings.json
{
"mcpServers": {
"typecast": {
"command": "uvx",
"args": [
"typecast-mcp"
]
}
}
}Verificaciones automáticas
Ejecutado 14 nov 2026Informe completo →Pasó nuestras verificaciones automáticas. Esto no es una auditoría de seguridad: lee el código y dale al servidor el acceso más limitado que necesite. Cómo funcionan las verificaciones
- Repositorio de origen vinculado y accesiblenilayp-codes/typecast is public and reachablegithub.com/nilayp-codes/typecast ↗Aprobado
- El paquete indica este repositorio como su origentypecast-mcp names nilayp-codes/typecast as sourcepypi.org/project/typecast-mcp/ ↗Aprobado
- Procedencia de la compilación (atestación de Sigstore)No build attestation published for this releaseNo verificado
- Licencia de código abierto (OSI)MITspdx.org/licenses/MIT.html ↗Aprobado
- Commit o lanzamiento en el último añoLast commit or release 2026-11-06github.com/nilayp-codes/typecast ↗Aprobado
- OpenSSF ScorecardNo verificado
- El nombre del paquete no es similar a un paquete popularNo popular package one edit away from typecast-mcpAprobado
- No hay avisos críticos en el paquete ni en sus dependencias29 resolved dependencies checked; no critical advisoryosv.dev ↗Aprobado
- Sin instrucciones ocultas para el modelo0 tool descriptions and the README read; no override phrases, credential paths or hidden charactersAprobado
- Las herramientas destructivas están etiquetadasNo verificado
- El endpoint remoto responde al handshake de MCPNo verificado
README
Ver en GitHubTypecast
LLM agent that turns messy job descriptions into schema-validated, typed JSON — and measures how often that fails.
Key findings (two Qwen models via OpenRouter, 30 real job descriptions):
- Schema mode makes output valid, not correct. In setup C both models were valid 98.9% of the time, at lower cost than retry, yet end-to-end accuracy was 67.1% (7B) and 84.7% (235B).
- Error-feedback retry fixed the small model, not the large one. Setup B was 98.9% valid for the 7B but 88.9% for the 235B, which failed on jd_002 in all four setup-B runs. A count-aware retry message did not change that.
- No run reached my 90% field-accuracy goal (best: 87.3%), and runs vary. The same setup moved by about 8 points end to end between runs, so a gap between two single runs is not conclusive.
60-second walkthrough:
https://github.com/user-attachments/assets/c4a2572c-88a2-49a3-8e20-176010877154
CLI and package name: soa · MCP server on PyPI: typecast-mcp
Results
Two open-weight models through OpenRouter, 30 real job descriptions, four setups. Field accuracy is scored against 20 hand-verified gold labels (see Limitations for how they were made). With 3 runs, the figure is the mean and the brackets are min–max across the runs; with 1 run it is that single run. A = plain prompt, no retry; B = plain prompt + error-feedback retry; C = provider schema mode, no retry; D = schema mode + retry.
| model | setup | runs | final valid | end-to-end accuracy | cost per 30-doc run |
|---|---|---|---|---|---|
| qwen-2.5-7b-instruct | A · plain | 1 | 83.3% | 50.4% | $0.0081 |
| qwen-2.5-7b-instruct | B · plain + retry | 3 | 98.9% (96.7%-100.0%) | 67.1% (62.9%-70.4%) | $0.0103 ($0.0101-$0.0105) |
| qwen-2.5-7b-instruct | C · schema | 3 | 98.9% (96.7%-100.0%) | 67.1% (65.0%-69.8%) | $0.0076 ($0.0076-$0.0076) |
| qwen-2.5-7b-instruct | D · schema + retry | 1 | 100.0% | 70.0% | $0.0080 |
| qwen3-235b-a22b-2507 | A · plain | 1 | 70.0% | 60.4% | $0.0120 |
| qwen3-235b-a22b-2507 | B · plain + retry | 3 | 88.9% (86.7%-93.3%) | 75.6% (72.8%-80.9%) | $0.0211 ($0.0199-$0.0234) |
| qwen3-235b-a22b-2507 | C · schema | 3 | 98.9% (96.7%-100.0%) | 84.7% (82.8%-85.8%) | $0.0136 ($0.0134-$0.0140) |
| qwen3-235b-a22b-2507 | D · schema + retry | 1 | 100.0% | 85.4% | $0.0143 |
| model | setup | runs | first-try valid | final valid | avg calls | field accuracy (valid outputs) | end-to-end accuracy | cost per 30-doc run |
|---|---|---|---|---|---|---|---|---|
| qwen-2.5-7b-instruct | A: plain prompt, no retry | single run | 83.3% | 83.3% | 1.00 | 62.9% | 50.4% | $0.0081 |
| qwen-2.5-7b-instruct | B: plain prompt + error-feedback retry | mean of 3 runs (min–max) | 82.2% (80.0%-83.3%) | 98.9% (96.7%-100.0%) | 1.20 (1.17-1.23) | 68.2% (66.0%-70.4%) | 67.1% (62.9%-70.4%) | $0.0103 ($0.0101-$0.0105) |
| qwen-2.5-7b-instruct | C: schema mode, no retry | mean of 3 runs (min–max) | 98.9% (96.7%-100.0%) | 98.9% (96.7%-100.0%) | 1.00 (1.00-1.00) | 68.2% (66.5%-69.8%) | 67.1% (65.0%-69.8%) | $0.0076 ($0.0076-$0.0076) |
| qwen-2.5-7b-instruct | D: schema mode + retry | single run | 96.7% | 100.0% | 1.03 | 70.0% | 70.0% | $0.0080 |
| qwen3-235b-a22b-2507 | A: plain prompt, no retry | single run | 70.0% | 70.0% | 1.00 | 86.6% | 60.4% | $0.0120 |
| qwen3-235b-a22b-2507 | B: plain prompt + error-feedback retry | mean of 3 runs (min–max) | 76.7% (73.3%-80.0%) | 88.9% (86.7%-93.3%) | 1.34 (1.33-1.37) | 85.6% (85.1%-86.0%) | 75.6% (72.8%-80.9%) | $0.0211 ($0.0199-$0.0234) |
| qwen3-235b-a22b-2507 | C: schema mode, no retry | mean of 3 runs (min–max) | 98.9% (96.7%-100.0%) | 98.9% (96.7%-100.0%) | 1.00 (1.00-1.00) | 86.1% (85.6%-87.0%) | 84.7% (82.8%-85.8%) | $0.0136 ($0.0134-$0.0140) |
| qwen3-235b-a22b-2507 | D: schema mode + retry | single run | 100.0% | 100.0% | 1.00 | 85.4% | 85.4% | $0.0143 |
How to read it:
- "Mean of 3 runs (min–max)" (B and C) is the mean of three runs on the same 30 documents, with the lowest and highest run in brackets. It is a spread across runs, not an interval estimate. The B runs are the three runs made after the retry-feedback fix described below. The C runs are the first C run plus two repeats. "Single run" (A and D) is one draw each: the repeats show the same setup moving by about 8 points end to end between runs (235B setup B: 72.8% to 80.9%). Each run's own 95% interval (Wilson for the validity columns, bootstrap for the accuracy columns) is in the per-run table at the end of this section; the single runs of A and D have none, because they were made before the interval code existed.
- First-try / final valid: share of the 30 documents whose first reply, or any reply within 3 attempts, passed validation. Avg calls: model calls per document.
- Field accuracy (valid outputs) is measured only on documents that produced a valid posting, so it flatters a setup that fails often (235B setup A: 86.6% field accuracy, but only 70.0% of the labelled documents had a valid output to score). End-to-end accuracy counts a failed document as wrong on every field and is the number to trust when setups differ in validity.
- Cost is the provider-reported cost of one run over the 30 documents. Temperature was not set (the provider default applied, value unknown).
My targets, and whether they were met
I set three targets for setup B before running anything: final validity of 30/30, at most 1.3 calls per document on average, and at least 90% field accuracy. Result, per run (the three runs after the retry fix):
| target (setup B) | qwen-2.5-7b-instruct | qwen3-235b-a22b-2507 |
|---|---|---|
| final validity 30/30 | met in 2 of 3 runs (100.0%, 100.0%, 96.7%) | missed in all 3 (86.7%, 93.3%, 86.7%) |
| avg calls <= 1.3 | met (1.20, 1.17, 1.23) | missed (1.37, 1.33, 1.33) |
| field accuracy >= 90% | missed (70.4%, 68.1%, 66.0%) | missed (85.7%, 85.1%, 86.0%) |
The 90% field-accuracy target was missed by every run of both models, in every setup. The best number anywhere is 87.3% (235B, setup B, first-round run).
Rows 1-8: the first round, one run per model and setup (A, B, C, D for the 7B, then for the 235B). These were written
before the interval code existed, so they have none. Rows 9-18: the repeat round, in order 7B B x3, 7B C x2, 235B B x3,
235B C x2. Intervals are in brackets. Columns: field acc is over valid outputs, coverage is the share of the 20
labelled documents with a valid output, e2e acc counts failures as wrong.
| model | setup | n | first-try | final | avg calls | field acc (norm) | coverage | e2e acc (norm) | cost |
|---|---|---|---|---|---|---|---|---|---|
| openrouter/qwen/qwen-2.5-7b-instruct | A | 30 | 83.3% | 83.3% | 1.00 | 62.9% | 80.0% | 50.4% | $0.0081 |
| openrouter/qwen/qwen-2.5-7b-instruct | B | 30 | 86.7% | 96.7% | 1.17 | 68.3% | 95.0% | 64.8% | $0.0102 |
| openrouter/qwen/qwen-2.5-7b-instruct | C | 30 | 100.0% | 100.0% | 1.00 | 69.8% | 100.0% | 69.8% | $0.0076 |
| openrouter/qwen/qwen-2.5-7b-instruct | D | 30 | 96.7% | 100.0% | 1.03 | 70.0% | 100.0% | 70.0% | $0.0080 |
| openrouter/qwen/qwen3-235b-a22b-2507 | A | 30 | 70.0% | 70.0% | 1.00 | 86.6% | 70.0% | 60.4% | $0.0120 |
| openrouter/qwen/qwen3-235b-a22b-2507 | B | 30 | 83.3% | 86.7% | 1.33 | 87.3% | 85.0% | 74.3% | $0.0186 |
| openrouter/qwen/qwen3-235b-a22b-2507 | C | 30 | 100.0% | 100.0% | 1.00 | 85.8% | 100.0% | 85.8% | $0.0140 |
| openrouter/qwen/qwen3-235b-a22b-2507 | D | 30 | 100.0% | 100.0% | 1.00 | 85.4% | 100.0% | 85.4% | $0.0143 |
| openrouter/qwen/qwen-2.5-7b-instruct | B | 30 | 80.0% [62.7-90.5] | 100.0% [88.6-100.0] | 1.20 | 70.4% [64.4-76.9] | 100.0% | 70.4% [64.4-76.9] | $0.0104 |
| openrouter/qwen/qwen-2.5-7b-instruct | B | 30 | 83.3% [66.4-92.7] | 100.0% [88.6-100.0] | 1.17 | 68.1% [61.4-74.7] | 100.0% | 68.1% [61.4-74.7] | $0.0101 |
| openrouter/qwen/qwen-2.5-7b-instruct | B | 30 | 83.3% [66.4-92.7] | 96.7% [83.3-99.4] | 1.23 | 66.0% [59.6-72.2] | 95.0% | 62.9% [54.5-70.5] | $0.0105 |
| openrouter/qwen/qwen-2.5-7b-instruct | C | 30 | 100.0% [88.6-100.0] | 100.0% [88.6-100.0] | 1.00 | 66.5% [59.6-73.8] | 100.0% | 66.5% [59.6-73.8] | $0.0076 |
| openrouter/qwen/qwen-2.5-7b-instruct | C | 30 | 96.7% [83.3-99.4] | 96.7% [83.3-99.4] | 1.00 | 68.4% [61.3-75.0] | 95.0% | 65.0% [55.2-73.3] | $0.0076 |
| openrouter/qwen/qwen3-235b-a22b-2507 | B | 30 | 76.7% [59.1-88.2] | 86.7% [70.3-94.7] | 1.37 | 85.7% [81.3-89.6] | 85.0% | 72.8% [58.4-84.8] | $0.0234 |
| openrouter/qwen/qwen3-235b-a22b-2507 | B | 30 | 73.3% [55.6-85.8] | 93.3% [78.7-98.2] | 1.33 | 85.1% [81.1-88.9] | 95.0% | 80.9% [71.1-87.9] | $0.0199 |
| openrouter/qwen/qwen3-235b-a22b-2507 | B | 30 | 80.0% [62.7-90.5] | 86.7% [70.3-94.7] | 1.33 | 86.0% [82.0-89.6] | 85.0% | 73.1% [58.7-84.9] | $0.0202 |
| openrouter/qwen/qwen3-235b-a22b-2507 | C | 30 | 96.7% [83.3-99.4] | 96.7% [83.3-99.4] | 1.00 | 87.0% [83.7-89.9] | 95.0% | 82.8% [72.3-89.0] | $0.0136 |
| openrouter/qwen/qwen3-235b-a22b-2507 | C | 30 | 100.0% [88.6-100.0] | 100.0% [88.6-100.0] | 1.00 | 85.6% [82.0-88.8] | 100.0% | 85.6% [82.0-88.8] | $0.0134 |
The A and D figures in the main table are rows 1-8 (A and D) above. The first-round B rows (7B: 86.7% / 96.7%,
235B: 83.3% / 86.7% first-try / final) are the runs from before the retry fix; the first-round C rows are the first
of the three C runs in the main table. The aggregate over the repeats is in results/aggregate.md; the raw per-run
reports are results/2026-11-04_*.json.
How it works
text ──► build prompt (schema + instructions)
│
▼
LLM call ──► raw string
│
▼
JobPosting.model_validate_json(raw)
│ ok │ ValidationError
▼ ▼
log success log failure (error + raw)
return model attempt < MAX? ──► append error to messages, retry
│ no
▼
raise ExtractionFailed (logged)
The schema shown to the model is generated from the Pydantic model (JobPosting.model_json_schema()), never written
by hand, and the reply is checked with the same model. A failed reply goes back to the model with the validator's
errors, one line per error, for up to 3 attempts. After a valid reply, two deterministic tools tidy the result
(canonical skill names, a location check against the document header). They are plain code, not model calls, so they
add no LLM calls and the setups below stay comparable. Details: docs/ARCHITECTURE.md.
The four setups: A plain prompt, no retry. B plain prompt, errors fed back (up to 3 attempts). C the provider's strict JSON-schema mode, no retry. D schema mode plus retry.
Why not just use native structured outputs?
Schema mode helped a lot, and it is cheaper. It also does not do the whole job.
What it fixed. Setup C was valid on the first try 98.9% (96.7%-100.0%) of the time for both models, against a
single plain-prompt run (A) of 83.3% for the 7B and 70.0% for the 235B. The 30-item cap on skills was the constraint
that models actually broke in plain mode; under schema mode the report counts replies that broke a constraint the
schema carries, and it was 0 in every C and D run (6 C runs, 2 D runs). C also cost less than B: $0.0076 against
$0.0103 for the 7B, $0.0136 against $0.0211 for the 235B (means per run).
What it does not do:
- It guarantees shape, not meaning. The remaining failures in C were rules no JSON schema can express. Two of the six C runs had one document rejected for duplicate skill names ("LangChain" and "Langchain"; "Python" twice; both rejections are in results/failure_examples.jsonl), which is why C's final validity is 98.9% (96.7%-100.0%) and not 100%. The intern-implies-no-experience rule has the same property (it cannot be stated in a schema); it did not fire in any of the 637 logged attempt lines of this project (the last line of that file records the count), so I have no evidence either way on it.
- Valid is not correct. Schema mode made the 235B valid 98.9% (96.7%-100.0%) of the time, and its field accuracy is 86.1% (85.6%-87.0%), with end-to-end accuracy 84.7% (82.8%-85.8%): below my 90% goal. The 7B in C is just as valid and gets 68.2% (66.5%-69.8%) field accuracy; its seniority accuracy was 55.0%, 55.0% and 52.6% over the three C runs. Validity says nothing about whether the seniority is right.
- It may trade recall for validity. The cap is enforced by the model emitting shorter skill lists, and skills are where accuracy is weakest. For the 235B, skills F1 was 66.9%, 72.1% and 65.7% in the three C runs, against 71.4%, 71.6% and 68.3% in the three B runs (means 68.2% vs 70.4%, with overlapping ranges). In the first-round single runs it was 72.1% (A) and 73.8% (B) against 66.9% (C) and 63.4% (D). The repeats show this is a tendency, not a clean effect, and with n=20 labelled documents I would not claim more.
- It depends on the provider. Both models accepted strict
json_schemahere, but only because requests were sent with OpenRouter'srequire_parametersso they could only route to providers that support it. The 7B was always served by one provider; the 235B was split across three.
Most common failure types, and what fixed them
From the plain-prompt runs and the retry runs (counts are failing attempts whose first error was at that field,
failures.by_first_loc in the reports):
| failure | setup A (one run) | setup B, 3 runs | what fixed it |
|---|---|---|---|
skills over the 30-item cap |
7B: 3 of 5 failing docs; 235B: 8 of 9 failing docs | 7B: 5, 5, 3 failing attempts; 235B: 14, 11, 14 | Retry feedback fixes most of it. Schema mode removes it. |
skill name over 40 characters (skills.*.name) |
7B 2, 235B 1 | 7B: 0, 0, 3; 235B: 1, 1, 0 | Retry |
null instead of true/false (skills.*.required, salary_mentioned) |
none | 7B: 2 attempts at skills.*.required in one run, 1 at salary_mentioned in another |
Retry |
duplicate skill names (also counted under skills, since that is where the error is reported) |
not counted separately | not counted separately | Retry fixes it: all 5 duplicate-name rejections in setup B were valid on the next attempt (each rejected attempt and its valid successor are in results/failure_examples.jsonl). Schema mode cannot: it was the only failure left in C (two documents, same file). |
| malformed JSON, markdown fences, prose, extra fields, invalid enum values | none: no run has a failure at the root, at seniority, mode or any other enum or extra field |
none | Nothing needed |
Almost everything that goes wrong is a count cap, not a formatting problem. In the first 11-document baseline every
failure of both models was a too-long skills list (results/2026-11-03_baseline_*.md). The first thing I tried was
to make the feedback more specific. The default feedback line is the validator's own wording (for example skills: List should have at most 30 items after validation, not 42). I replaced it for over-long lists with a count and a
fix, for example skills: has 39 items, maximum is 30 - remove 9, starting with required=false ones. The result is
mixed:
- It did not lift the 235B's final validity in setup B: 86.7% before, then 86.7%, 93.3% and 86.7% across three runs. Each run's interval contains the old figure, so at n=30 no effect is shown.
- The four 235B documents that failed in the first B run (jd_002, jd_003, jd_005, jd_023) passed in 6 of 12 document-runs afterwards (jd_002 0 of 3, jd_003 1 of 3, jd_005 2 of 3, jd_023 3 of 3), but other documents (jd_029, jd_030, jd_004) failed instead. I cannot separate the fix from run-to-run variation. The first attempt is identical before and after the fix, yet first-try validity moved from 83.3% to 76.7%, 73.3% and 80.0%; that spread is run-to-run variation (sampling plus a provider mix I did not log in the first round).
- jd_002 failed for the 235B in all four setup-B runs. Every attempt is in
results/failure_examples.jsonl: 42, 31, 31 skills in the first run (before the
fix), then 70, 31, 31 and 47, 31, 31 and 44, 31, 31 in the three runs after it. The model goes from 42-70 skills down
to 31 on the first retry and then stays at 31. For a 31-item reply the new feedback line is
skills: has 31 items, maximum is 30 - remove 1, starting with required=false ones(built by the code; the prompt text itself is not logged). This is the clearest open failure: the model does not make the last cut.
Design decisions
strict=Trueon every model. Pydantic's default lax mode quietly turns"3"into3and"yes"intoTrue, so a sloppy reply passed and the retry loop had nothing to do. Strict mode rejects both.- Required-but-nullable fields (
company,city,country,min_years_experience). No default: the model has to writenullexplicitly, so "not stated" is a decision the model made, and a forgotten key is an error rather than a silentnull. - One line per error in the retry prompt,
loc: msg (got: <input, clipped to 60 chars>), rendered fromValidationError.errors()instead ofstr(error). The raw string carries a version-specific prefix, a truncatedinput_value, and a documentation URL, which are noise for a model. The one exception is an over-long list, which gets the count-aware line shown above. I did not run a blind-retry control (retry with no error text), so I have no measurement of feedback versus plain resampling; setup A against B is retry-with-feedback against nothing. - Tools are deterministic post-processing, not model-called. Canonical skill names and a location check run
after validation as plain code. That keeps every setup at one prompt and the retry/attempt/cost counts clean, and
the model's own output is kept (
raw_posting) so validity is measured on what the model actually said. Letting the model call the tools is a separate experiment I have not run. - API errors are separate from validation errors. A 429 or 5xx is retried with backoff (2, 4, 8 s) and never uses up one of the 3 validation attempts.
- The 30-skill cap stays. The first plain-prompt baseline showed every failure was this one constraint, so I kept it as the test of what retries and schema modes can do rather than loosening it to make numbers look better.
Limitations
- 30 job descriptions, 20 gold labels. One document moves a validity rate by 3.3 points and a field accuracy by 5 points; the intervals above are wide for that reason (for example 235B setup B final validity 93.3% [78.7-98.2] in one run). Five of the 30 are non-technical or semi-technical roles, kept on purpose as an out-of-domain slice; two of them (jd_012, jd_019) are among the 20 labelled.
- The gold labels were drafted by a model (Claude) and then checked by me. I verified every field and made 17 corrections (docs/label_corrections.log). One person, one pass, so any systematic mistake of mine is in the labels. The labelling rules were written into the schema's field descriptions after the first baseline, so the models and the labels follow the same rules; runs on the earlier prompt are not comparable.
- Noise floor on accuracy, from a blind check. On 5 other documents (jd_004, 005, 008, 014, 015, none of them in the gold set) I labelled blind and compared against Claude's sealed labels written beforehand, with the same per-field metrics as the eval (my labels as the reference): title 80.0%, company 100.0%, seniority 80.0%, mode 80.0%, salary_mentioned 100.0%, min_years_experience 100.0%, city 80.0%, country 100.0%, skills F1 61.0% (precision 52.9%, recall 72.0%), overall 86.8% (results/label_agreement.md). Five documents is a sanity check, not a rate: one disagreement moves a field by 20 points. It does suggest that on skills two careful labelers disagree about as much as the models disagree with the gold labels, and that the 235B's roughly 86% field accuracy is close to the level two labelers reach with each other, so the remaining gap to my 90% target may partly be label noise rather than model error. I have not shown that.
- Runs are from one day, two models of one family, one provider layer. Temperature was not set. The 7B was always routed to one provider; the 235B to three, and the provider was not logged in the first round, so provider mix and sampling noise are mixed into the run-to-run spread.
- The post-processing tools were not tuned against the final labelling rules, so the location check and the labels can disagree on some documents.
- Setup D is a single run per model; only B and C were repeated.
- Data. The 30 job descriptions in
data/jds/were copied from public postings (LinkedIn 13, Work at a Startup 11, Wellfound 6, counted fromdata/jds/manifest.csv) for evaluation only; I will remove any of them on request. Five rows have no source URL recorded inmanifest.csv: jd_001, jd_002, jd_003, jd_004 and jd_010 (all LinkedIn; their notes say "URL not recorded").
Run it yourself
Needs Python 3.11+ and uv.
git clone https://github.com/nilayp-codes/typecast.git
cd typecast
uv sync
cp .env.example .env # then set OPENROUTER_API_KEY in .env (never commit it)
uv run soa extract data/jds/jd_001.txt # the JSON on stdout, a status line on stderr
uv run soa extract data/jds/jd_001.txt --json-mode # provider schema mode
uv run soa extract data/jds/jd_001.txt > out.json
uv run soa eval --model openrouter/qwen/qwen-2.5-7b-instruct --setup B --limit 5
uv run soa eval --model openrouter/qwen/qwen3-235b-a22b-2507 --setup all
Known issue: on Windows, clone into a short path, because LiteLLM writes a cache file under .venv and a very deep path exceeds Windows' path-length limit (soa extract then fails with APIConnectionError ... [Errno 2]).
soa eval makes real model calls: it prints the plan and the estimated number of calls and asks before sending
anything (--yes skips the question). In my runs one setup over the 30 documents cost between $0.0076 and $0.0234.
It saves results/<date>_<model>_<setup>.json and appends a row to results/summary.md; --resume <eval id>
continues an interrupted run. To combine repeated runs: uv run python scripts/aggregate_runs.py results/<files>.json
(writes results/aggregate.md). The 30 job descriptions are in data/jds/ and the gold labels in data/labels/.
Tests and type checks (no real model is ever called in tests):
uv run pytest
uv run mypy --strict src/ packages/typecast-mcp/src/
Preguntas sobre Typecast MCP
¿Cómo agrego Typecast MCP a Claude Code?
Ejecuta claude mcp add typecast -- uvx typecast-mcp en tu terminal. Añade --scope user para que esté disponible en todos los proyectos.
¿Cómo agrego Typecast MCP a Cursor?
Abre ~/.cursor/mcp.json y añade el servidor en mcpServers, usando la configuración de Cursor que se muestra en la sección de instalación de arriba.
¿Cómo funciona el proceso de validación?
El servidor utiliza un modelo Pydantic para generar un esquema JSON, valida la salida de texto sin procesar del LLM y realiza hasta 3 reintentos con retroalimentación de errores si la validación falla.
¿El uso del modo schema garantiza que los datos sean correctos?
No, el modo schema garantiza la forma y estructura del JSON, pero no garantiza la corrección semántica del contenido.
¿Qué configuraciones de ejecución están disponibles?
Hay cuatro configuraciones: A (prompt simple, sin reintentos), B (prompt simple con reintentos de retroalimentación de errores), C (modo schema del proveedor, sin reintentos) y D (modo schema con reintentos).