PDF, Office belgeleri, görseller, ses dosyaları ve web sitelerini, düzen farkındalıklı semantik chunking yöntemiyle temiz ve LLM uyumlu Markdown formatına dönüştürür.
Ayrıca bilinen adı funkyfunc/filegoblin
- Yıldızlar
- 1
- Araçlar
- —
- İndirmeler, 30 gün
- —
- En son sürüm
- —
- Son commit
- 7 Kas 2026
- Lisans
- MIT
filegoblin MCP nedir?
filegoblin bir MCP sunucusudur. PDF, Office belgeleri, görseller, ses dosyaları ve web sitelerini, düzen farkındalıklı semantik chunking yöntemiyle temiz ve LLM uyumlu Markdown formatına dönüştürür.
Özet, projenin README dosyasından oluşturuldu
filegoblin MCP sunucusu hakkında
filegoblin, karmaşık belgeleri yapay zeka için yapılandırılmış bağlama dönüştürmenizi sağlar. Bu tek bir binary'den oluşan CLI aracı; PDF, Office belgeleri, görseller, ses dosyaları ve web sitelerini işleyerek bunları temiz Markdown formatına dönüştürür. Bulut API'lerine bağımlı kalmadan RAG veya LLM istemleri için yüksek kaliteli veriye ihtiyaç duyan geliştiriciler ve agent'lar için tasarlanmıştır. Tüm ayrıştırma ve PII temizleme işlemleri yerel makinenizde gerçekleşerek gizliliği sağlar.
Kullanıcılar, hızlı geometrik dönüşüm için hafif (lite) bir sürüm veya karmaşık düzenler için vision modelleri içeren tam sürüm arasında seçim yapabilirler. Vision hattı; okuma sırasını, LaTeX formül çıkarımını ve tablo yeniden yapılandırmayı yönetir. Ayrıca macOS üzerinde Apple Vision aracılığıyla OCR desteği sunarken, Linux ve Windows için ek OCR özellikleri de mevcuttur. Araç, tabloların ve kod bloklarının işleme sırasında yanlış şekilde bölünmesini önlemek için düzen farkındalıklı semantik chunking sağlar.
Neler yapabilirsiniz
- 1RAG için karmaşık PDF'leri hazırlayınÇok sütunlu düzenlere ve tablolara sahip bir PDF'i, okuma sırasını koruyan yapılandırılmış Markdown formatına dönüştürün.
- 2Matematiksel formülleri LaTeX olarak çıkarınMatematik sembolleri içeren bilimsel belgeleri, bir LLM'in LaTeX kullanarak anlayabileceği bir formata dönüştürün.
- 3Hassas verileri güvenli bir şekilde işleyinVerileri sisteme dahil etmeden önce gizliliği korumak amacıyla, kişisel verileri (PII) dosyalar üzerinden yerel olarak temizleyin.
Projenin README dosyasından alınmıştır. Her kullanım durumu, oradaki bir alıntı ile kontrol edilir.
filegoblin MCP nasıl yüklenir
Biçimler 10 Kas 2026 tarihinde doğrulandıCursor'a ekleGoose'a eklefilegoblin MCP'yi Claude Code içine yükleyin
$ claude mcp add filegoblin -- npx -y filegoblinfilegoblin MCP'yi Cursor içine yükleyin
~/.cursor/mcp.json
{
"mcpServers": {
"filegoblin": {
"command": "npx",
"args": [
"-y",
"filegoblin"
]
}
}
}filegoblin MCP'yi VS Code içine yükleyin
.vscode/mcp.json
{
"servers": {
"filegoblin": {
"type": "stdio",
"command": "npx",
"args": [
"-y",
"filegoblin"
]
}
}
}filegoblin MCP'yi VS Code CLI içine yükleyin
$ code --add-mcp '{"name":"filegoblin","type":"stdio","command":"npx","args":["-y","filegoblin"]}'filegoblin MCP'yi Claude Desktop içine yükleyin
claude_desktop_config.json
{
"mcpServers": {
"filegoblin": {
"command": "npx",
"args": [
"-y",
"filegoblin"
]
}
}
}filegoblin MCP'yi Codex içine yükleyin
$ codex mcp add filegoblin -- npx -y filegoblinfilegoblin MCP'yi Codex config.toml içine yükleyin
~/.codex/config.toml
[mcp_servers.filegoblin] command = "npx" args = ["-y", "filegoblin"]
filegoblin MCP'yi Gemini CLI içine yükleyin
$ gemini mcp add -s user filegoblin npx -- -y filegoblinfilegoblin MCP'yi Gemini settings.json içine yükleyin
~/.gemini/settings.json
{
"mcpServers": {
"filegoblin": {
"command": "npx",
"args": [
"-y",
"filegoblin"
]
}
}
}Yapılandırma
FILEGOBLIN_ALLOW_LOCAL_URLS
| Ad | Açıklama | Sunucu |
|---|---|---|
| FILEGOBLIN_ALLOW_LOCAL_URLS | — |
Otomatik kontroller
Çalıştırıldı 14 Kas 2026Tam rapor →Otomatik kontrollerimizden geçti. Bu bir güvenlik denetimi değildir: kodu okuyun ve sunucuya ihtiyaç duyduğu en dar erişimi verin. Kontroller nasıl çalışır
- Kaynak deposu bağlantılı ve erişilebilirfunkyfunc/filegoblin is public and reachablegithub.com/funkyfunc/filegoblin ↗Geçti
- Paket, bu depoyu kaynağı olarak gösteriyorfilegoblin names funkyfunc/filegoblin as sourcenpmjs.com/package/filegoblin ↗Geçti
- Derleme kökeni (Sigstore onayı)Sigstore attestation for filegoblinnpmjs.com/package/filegoblin/v/1.17.0#provenance ↗Geçti
- Açık kaynak lisansı (OSI)MITspdx.org/licenses/MIT.html ↗Geçti
- Son bir yıl içinde commit veya sürüm mevcutLast commit or release 2026-11-07github.com/funkyfunc/filegoblin ↗Geçti
- OpenSSF ScorecardKontrol edilmedi
- Paket adı popüler bir pakete yakın değilNo popular package one edit away from filegoblinGeçti
- Pakette veya bağımlılıklarında kritik uyarı yokKontrol edilmedi
- Modele yönelik gizli talimat yok0 tool descriptions and the README read; no override phrases, credential paths or hidden charactersGeçti
- Yıkıcı araçlar etiketlenmişKontrol edilmedi
- Uzak uç nokta MCP el sıkışmasına yanıt veriyorKontrol edilmedi
(o_o) filegoblin (filegoblin)
"Ingesting the messy world, spitting out clean context."
filegoblin is a single-binary CLI that "gobbles" almost any file or website — PDF, Word, Excel, PowerPoint, email, EPUB, HTML, whole repos — and converts it into clean, LLM-ready markdown.
Philosophy
- It just works.
filegoblin document.pdf > context.mdneeds no manual. Power flags exist, but the default path is the product. - Format invisibility. You shouldn't have to think about extensions; detection is extension-first with magic-byte fallback, so renamed and extension-less files route correctly.
- Fail loudly, never silently. An empty result exits non-zero, parse failures are reported in-band, and invalid flags are hard errors. An agent can trust every exit code.
- Local and private. Parsing and PII scrubbing happen entirely on your machine. No cloud APIs, no runtimes, no
pip install. - Not just a concatenator.
cat * > out.txtexists; filegoblin adds structure (real markdown from Word, one consistent table dialect, readability-grade web extraction), token budgeting, and redaction.
📦 Installation
Every release comes in two flavours:
- with the PDF vision models for layout and reading order, formulas as LaTeX, tables, OCR on scanned pages, and a page reader for handwritten and math scans (~1.3 GB; ~510 MB without the page reader);
- lite, without them: the fast geometric converter, plus Apple Vision OCR on macOS.
Everything else is the same.
| Channel | Lite | With the models |
|---|---|---|
| GitHub release (latest) | filegoblin-<target> archive |
filegoblin-full-<target> archive: one binary with the models compiled in |
| PyPI | pip install filegoblin |
pip install filegoblin, then filegoblin --install-models (the filegoblin[models] packages aren't on PyPI yet) |
| npm | npm i -g filegoblin |
npm i -g filegoblin @filegoblin/models |
| Cargo | cargo install --git https://github.com/funkyfunc/filegoblin filegoblin |
then filegoblin --install-models |
Builds are published for macOS (Apple Silicon and Intel), Linux (x86_64 and aarch64), and Windows (x86_64).
Getting the models later.
filegoblin --install-modelsdownloads this version's model archive from the GitHub release, checks its SHA-256 and unpacks it into~/.filegoblin/models. You can pass a directory instead, and setFILEGOBLIN_MODELS_URLto point at an internal mirror. filegoblin only downloads when you ask it to.filegoblin --models-statusshows where each model was found.
Behind a corporate network. The npm model packages (and the PyPI ones, once they're published) carry the weights themselves, one package per model, with the two largest models in halves to stay under the registries' 100 MB per-file limit. They install from a company mirror (Artifactory, Nexus, CodeArtifact) like any other package, with no access to github.com.
Air-gapped machines. Copy the full download. Alternatively, unpack the filegoblin-vision-models.tar.gz archive on a shared read-only path and name that directory in FILEGOBLIN_MODELS (the archive itself is not read). That variable takes a PATH-style list. Licenses are in LICENSES.md inside the archive.
From source — needs a Rust toolchain (1.87+):
cargo install --git https://github.com/funkyfunc/filegoblin filegoblin
That gives you everything in the "always on" list below, including web fetching and crawling. The optional engines are opt-in because each one pulls in a heavy dependency:
| Want | Add | Build needs |
|---|---|---|
| OCR on Linux/Windows | --features ocr |
just get-brains for the weights (macOS uses Apple Vision, no flag) |
| PDF vision models (layout, reading order, formulas → LaTeX, tables, scan OCR) | --features vision (on in release builds) |
The weights: filegoblin --install-models (the release's filegoblin-vision-models.tar.gz, SHA-256 checked) or just build-vision-models. filegoblin finds them in models/ next to the binary, in ~/.filegoblin/models, or in $FILEGOBLIN_MODELS. Add bundled-models with FILEGOBLIN_MODELS_DIR set to compile them into one binary |
| Audio/video transcription | --features transcribe |
cmake + a C/C++ compiler; downloads a ~574 MB model on first use |
| Metal-accelerated transcription (macOS only) | --features transcribe-metal |
Xcode Command Line Tools. Does not build on Linux/Windows |
| WASM plugins | --features wasm |
— |
BM25 search (--search) |
--features rag |
— |
# Example: a macOS build with GPU-accelerated transcription
cargo install --git https://github.com/funkyfunc/filegoblin filegoblin --features transcribe-metal
Without the models, --vision smart (the default) behaves exactly like the fast converter, with one note per run on stderr; --vision off silences it. With the models, it uses them only on the pages that need them. Anyone building from source can make a single self-contained binary with --features vision,bundled-models and FILEGOBLIN_MODELS_DIR set.
Measuring quality. just gauntlet runs the 29-case adversarial gauntlet
(just gauntlet off for the fast path). Each case gets pass/fail canaries and
a 0–100 quality score graded against hand-checked gold excerpts in
test_assets/gauntlet/gold/, plus checks that --format plain and
--chunk-size lose nothing. just gauntlet-compare A B shows what a change
did case by case. Results are in GAUNTLET_SCORECARD.md.
🏗 Current Project Structure
Note to Agent: Maintain this section as you add modules.
.
├── Cargo.toml (Virtual Workspace Root)
├── crates/
│ ├── core/ (filegoblin-core: The Engine & Parsers)
│ │ ├── Cargo.toml
│ │ ├── build.rs (Man page & shell completions)
│ │ └── src/
│ │ ├── lib.rs (Core Pipeline Logic)
│ │ ├── cli.rs (Clap Flag Definitions)
│ │ ├── formatters.rs (Output Assembly & Splitting)
│ │ ├── formatters/ (Plain text & specialized formatters)
│ │ ├── chunking/ (Layout-aware semantic chunking engine)
│ │ ├── curation/ (BM25 Search & Auto-Pruning)
│ │ ├── privacy_shield.rs (PII Redaction Engine)
│ │ ├── compressor/ (Token Compression Pipeline)
│ │ ├── parsers/ (File parsing engines; pdf_vision.rs = layout-guided PDF assembly,
│ │ │ pdf_furniture.rs + pdf_edges.rs = page furniture, pdf_words.rs = glyph-level
│ │ │ words, pdf_math.rs = inline-math clean-up, pdf_worker.rs = PDFs converted
│ │ │ in child processes)
│ │ ├── vision/ (Bundled ONNX models on rten: layout, formula, table, OCR, page reader; hayro rendering)
│ │ └── pipeline/ (Pipeline execution stages)
│ └── cli/ (filegoblin: The Headless Binary)
│ ├── Cargo.toml
│ └── src/main.rs (CLI Entry & Mascot)
│ └── walk.rs (--walk interactive picker)
├── scripts/ (Dev tooling: run_gauntlet.py = graded gauntlet, olmocr_bench.py = olmOCR-bench
│ on a fixed dev/test split, engines.py = other tools graded the same way
│ (--engine; docs/COMPARISON.md), model packaging and release helpers)
├── npm/ (npx filegoblin Zero-Install Launcher)
│ ├── package.json
│ └── bin/filegoblin.js
└── .cargo/
└── config.toml (Aliases)
⚡ Current Capabilities (v1.17.0)
Some capabilities are gated behind opt-in Cargo features (built off by default to keep the binary lean). Feature-gated items are marked below.
- Multi-Format Output Engine (
--format markdown|plain|json/-f): GFM markdown (default), syntax-stripped plain text with space-aligned tables for vector embeddings (e.g. OpenAItext-embedding-3, BGE) and BM25 search, and structured JSON documents - Layout-Aware Semantic Chunking (
--chunk-size,--chunk-overlap,--chunk-strategy layout|tokens): splits along structural headings (H1 > H2 > H3), protects tables and code blocks from mid-block slicing, automatically replicates table headers across chunk splits, and embeds hierarchical outline breadcrumbs - Core Parsers (PDF, Word, Excel/CSV/TSV, PowerPoint, OpenDocument, Jupyter, Code, Email, RTF, HTML, EPUB) — always on, including Office's macro-enabled and template variants (
.docm/.dotx/.dotm,.xlsm/.xltx/.xltm/.xlsb/.xlam,.pptm/.potx/.ppsx/.ppsm) and the pre-2007 binaries (.doc/.dot,.xls,.ppt/.pps/.pot) - Legacy text encodings: plain text, markdown, source code, CSV/TSV, subtitles and stdin are decoded from a byte-order mark (UTF-8/UTF-16), as UTF-8, or else as Windows-1252/Latin-1 — binary files are still refused. CSV delimiters (
,;tab|) are sniffed, so European;exports keep their columns - PDF → markdown via
pdf_oxide: headings, pipe tables, reading order, hyphenation — with per-page scanned-vs-text classification. Word spacing is re-read from glyph positions, so kerned, letter-spaced and ligature-split words ("infl uenced") and accents drawn apart come out whole; text outside the crop box (printer's slugs) is dropped; a password-protected PDF gets a clear error - Scanned PDFs and images OCR automatically on macOS (Apple Vision, zero setup; parallelized page conversion, 30+ pages/s; every page of a multi-page TIFF, under
## Page N); Linux/Windows via--features ocr - PDF vision pipeline (
--features vision+ model weights;--vision smart|all|off, defaultsmart): pages are rendered in pure Rust (hayro) and read by bundled, permissively licensed models onrten— no Python, system libraries or general-purpose LLM at runtime. PP-DocLayoutV3 places every text-layer word in its region (title, heading, paragraph, caption, footnote, table, figure, running header/footer) in a learned reading order; display and inline formulas are re-read from the image as LaTeX (pix2text MFR); tables get rows/columns/spanning cells from Table Transformer and their cell values from the text layer; scanned pages, text regions the text layer leaves empty, and scans whose own OCR layer is too damaged to keep are read with PP-OCRv6 (medium). A scanned page PP-OCR is unsure of (handwriting) or that is full of formulas is read whole by the optional page reader, LightOnOCR-2-1B (Apache-2.0, 4-bit onrten, about half a minute to two minutes a page); every other page keeps its speed.--vision smartdecides per page from cheap text-layer facts — scans, math fonts/symbols, column-aligned tables and forms, multi-column text, pictures, one- or two-page documents go through the models; plain single-column prose stays on the fast path. On the full olmOCR-bench (1,403 PDFs, official scorer; the 180 it was tuned on included) the models take filegoblin from 41.1% to 74.8% (arXiv math 0% → 87%, tables 35% → 80%, multi-column 50% → 73%, headers/footers 74% → 92%). On the 1,223 PDFs it was not tuned on, it scores 78.3% on the CPU alone (74.3% before the page reader); MinerU 79.0%, marker (fast mode) 65.6%, docling 54.6% and pymupdf4llm 42.3%, each in its default configuration (docs/COMPARISON.md). Ruled tables take their grid from the page's own rulings, and rotated (/Rotate) pages are read as displayed - Geometric PDF page furniture removal: running headers/footers, page numbers, and line-number gutters (pleading paper, patents) are identified from position, cross-page recurrence, font size and alignment — before conversion — so prose, numbers and headings are never rewritten by text heuristics. One-off furniture that cannot recur (letterheads, print stamps, document IDs, URLs, copyright and "downloaded from" lines) is found on each page from its place, size and wording. Line-break hyphenation and words run together across visible gaps are repaired from page geometry plus the document's own vocabulary
- Semantic Chapter Boundaries: Automated detection and injection of thematic dividers and XML section markers (
<!-- filegoblin:section ... -->) - Word → real markdown: heading levels, nested lists, bold/italic, hyperlinks with URLs, and equations
- Word Resilient Fallback: Streaming OpenXML reader handling
<w:gridSpan>merged tables and<mc:AlternateContent>floating text boxes - Word Office Math (OMML):
<m:oMath>and<m:oMathPara>converted directly to standard LaTeX ($ ... $and$$ ... $$) with radicals, delimiters, matrices, and functions - Embedded Office Charts: PowerPoint (
.pptx) and Word (.docx) OpenXML DrawingML charts (c:chartSpace) deconstructed into clean Markdown pipe tables, Mermaidxychart-beta, and native Mermaidpie showDatadiagrams - DrawingML Connectors & SmartArt: PowerPoint and Word shape connections, flowcharts, and SmartArt process hierarchies compiled into Mermaid
flowchartdiagrams - Zero-install npm distribution: Execute via
npx filegoblinin environments without Rust toolchains - DOM-based HTML → markdown (readability-style content selection, link-farm pruning, preserved link URLs, title/source front-matter, ATX headings)
- One tabular dialect everywhere: standard markdown pipe tables across PDF, docx, pptx, OpenDocument, and CSV/XLSX
- Pure Rust Ruling-Line Table Extraction: Native port of
pdfplumber.tablevector ruling-line and intersection table algorithm, supporting collinear edge snapping, bounding box cycle reconstruction, multi-column merged headers, and fallback gutter detection - Trafilatura DOM Content Extraction: Native port of
trafilatura.coretext/link-density heuristics, boilerplate token pruning, syntax-highlighted code block preservation, and citation-link dampening - PowerPoint: manifest-true slide order + speaker notes + embedded chart tables
- Legacy Office binaries (Word, PowerPoint and Excel 97–2003) via
office_oxide(pure Rust):.docheading levels from the document's styles, tables, nested lists, links and footnotes;.pptslides in the same shape as.pptx, with speaker notes;.xlssheets in the same layout as.xlsx, cells as Excel displays them. The file's own streams decide whether it is Word, PowerPoint or Excel, so renamed and extension-less files convert; RTF or HTML saved as.docgoes to its own parser; password-protected files and Outlook.msgget a clear error - Magic-byte format detection — extension-first, falling back to content signatures, so extension-less files and files with an unknown extension still route correctly (zip, tar and gzip archives; Office documents are recognised by the parts inside their zip, legacy ones by the streams inside their OLE2 container). A known but wrong extension (a
.docxrenamed to.pdf) still goes to that extension's parser - URL fetching routes by Content-Type: HTML → extractor, raw markdown/JSON/text → passthrough, binary documents (a PDF URL!) → the normal parser pipeline
-
--browser: render JavaScript-built pages (SPAs) with your installed Chrome/Edge/Brave — headless, throwaway profile, no bundled browser - Recursive crawling (
--horde) + GitHub repo cloning — on by default in the CLI binary (webfeature; disable with--no-default-features) - Fail-loud contract for agents: exit
4whenever nothing could be extracted, hard errors on invalid flags, quiet exit on broken pipes, explicit paths take precedence over piped stdin - MCP server (
--mcp): stdio Model Context Protocol server exposing aconverttool — plug filegoblin into Claude Desktop/Code with one config line - Subtitles (
.srt/.vtt): cue files → the same timestamped transcript markdown whisper produces, with speaker labels and rolling-caption dedup - RSS/Atom feeds: a feed URL or saved feed file → a clean markdown list of entries
- Conversion cache: repeat URL fetches and OCR/transcription results come back instantly (
~/.filegoblin/cache,--no-cacheto bypass) - Audio/video transcription (
--features transcribe; addtranscribe-metalon macOS for GPU): mp3/m4a/wav/flac/ogg/opus/mp4/mov/webm/mkv (audio track) → timestamped transcript markdown via whisper.cpp, on-device, model auto-downloads once - Golden-file test suite: end-to-end snapshots of real fixture output per format (
cargo insta reviewto inspect changes) - WASM Plugin Extensibility (
--plugin) — requires--features wasm - Semantic BM25 Search with Relevance Scores (
--search) — requires--features rag - Conversion cache management:
--no-cacheto bypass,--cache-clearto empty it - Glob Filtering (
--include,--exclude) & Depth Control (--depth) - Token Estimation (
--tokens), Token-Only Mode (--tokens-only) - Token Compression (
--compress safe|contextual) - Auto-Pruning to Token Budgets (
--max-tokens) - Manifest Table of Contents (
--manifest) - Git-Aware Diffing (
--git-diff) with Unified Diff Output (--diff-format) - Rust Symbol Extraction (
--extract symbols) via tree-sitter - Local PII / Secret Redaction (
--scrub) — deterministic + entropy heuristics - Output Splitting (
--split), Chunking (--chunk), JSON (--json) - Clipboard Ingestion (
--clipboard), Clipboard Output (--copy), OS Open (--open), File Write (--write) - Heuristic Project Summaries (
--summary) - Out-of-core streaming for the default dump path (flat RAM footprint)
- Pipeline-Safe Streams (
-qfor clean stdout)
✨ Key Features
- Parsers:
pdf_oxide,docx-rs,tree-sitter,quick-xml,calamine,scraper(Statically Linked). - OCR built in on macOS: Apple Vision (
objc2) with no weights or flags — scanned PDFs and images just work. Linux gets pure-Rustocrs/rteninference behind--features ocr. - WASM Extensibility:
wasmtimefor pure-rust statically-linked component execution. - Privacy First: 100% local PII/secret scrubbing — deterministic patterns (SSN, email, cloud/token prefixes) plus a Shannon-entropy tier that redacts high-entropy secrets. No model weights, no data egress.
- Structural Minification: Tree-sitter powered
--extract symbolsmode for Rust source.
| Source | Path Example | Parser Used | Output Format |
|---|---|---|---|
~/receipt.pdf |
PdfGobbler |
Markdown (headings, pipe tables, OCR for scans) | |
| Word | ~/notes.docx |
OfficeGobbler |
Markdown (headings, lists, emphasis, links, equations, charts) |
| Excel/CSV | ~/data.xlsx |
SheetGobbler |
Markdown pipe tables (header row named) |
| PowerPoint | ~/deck.pptx |
PptxGobbler |
Per-slide markdown + speaker notes + embedded charts |
| OpenDocument | ~/doc.odt, ~/deck.odp |
OpenDocGobbler |
Markdown (headings, lists, tables) |
| Source Code | ~/project/src |
CodeGobbler |
Full content (or --extract symbols) |
~/thread.eml, ~/inbox.mbox |
EmailGobbler |
Header block + markdown body + attachment list | |
| RTF | ~/notes.rtf |
RtfGobbler |
Plain text |
| Images | ~/screenshot.png, ~/IMG_0042.HEIC |
OcrGobbler (built-in on macOS) |
OCR Text |
| Subtitles | ~/movie.srt, ~/talk.vtt |
SubtitleGobbler |
Timestamped transcript |
| RSS/Atom | https://hnrss.org/frontpage |
FeedGobbler |
Entry list (title, link, date, body) |
| Webpages | https://news.ycombinator.com |
HtmlGobbler (default build) |
DOM extraction + front-matter |
- Semantic Search: BM25 relevance scoring via
tantivywith zero model weights. - Manifest TOC: Auto-generated table of contents with per-file token counts.
⚙️ Development & Asset Pipeline
filegoblin maintains a strict zero-dependency profile for the end user. However, building the project requires assembling several logic assets.
build.rs generates the manpage and shell completions into target/generated/. The .rten tensor weights for pure-Rust OCR fallback on Linux/Windows are not downloaded automatically — fetch them with the provided justfile when building with --features ocr:
just get-brains
(On macOS, OCR uses Apple's Vision framework natively; no weights needed.)
🚀 Quick Start
Basic Gobble (Files):
# Output a single file to stdout
filegoblin my_notes.pdf > context.md
# Merge multiple files natively
filegoblin src/main.rs src/lib.rs README.md --write context.md
URL Ingestion (Web):
filegoblin https://example.com/api-docs > context.md
Web Horde (Split into mapping directory):
filegoblin https://bettercli.org/ --horde --split
# Translates to -> ./bettercli.org_gobbled/
Privacy Shield (Redact PII & Secrets locally):
# Typed redaction (default): values become [EMAIL], [AWS_KEY], [CREDIT_CARD], …
filegoblin ./src/api_keys.ts --scrub > safe_context.md
# Pipe a log through it as a pre-flight gate before any cloud LLM
cat prod.log | filegoblin - --scrub --scrub-style plain
# CI / pre-commit guard: non-zero exit if anything sensitive is found
git ls-files -z | xargs -0 cat | filegoblin - --scrub --scrub-fail -q || echo "secrets detected!"
# Exempt a known-public value, or add your own pattern via ./.filegoblin-scrub.json
filegoblin ./config --horde --scrub --scrub-allow "support@example.com"
# Consistent pseudonyms: the same value maps to [EMAIL_1] everywhere, so an LLM can
# still tell that an entity recurs — without ever seeing it.
cat ticket.txt | filegoblin - --scrub --scrub-style pseudonym
Detection is 100% local with zero model weights: validated rules (Luhn for cards, range
checks for SSN, mod-97 for IBAN) plus cloud/token prefixes (AWS, GitHub, Stripe, Slack, Google,
OpenAI, JWT, private-key blocks, connection strings) and a Shannon-entropy catch-all for unknown
secrets. A value-free audit tally is printed to stderr (🛡️ Privacy Shield redacted 4 item(s): …).
An optional .filegoblin-scrub.json supplies an allowlist and custom patterns.
What filegoblin does NOT do: maintain a secrets blocklist. It converts exactly what you point it at — including a
.envfile if you name one. Deciding what's sensitive is yours: the directory walker respects.gitignore(where secrets usually already live),--excludefilters by glob, and--scrubredacts what does get through.
Machine Contract (--json) — extract any file to clean text for agents/scripts:
# One weird binary format → clean text + metadata, in a single call
filegoblin report.pdf --json | jq '.[0] | {format, tokens}'
# → { "format": "pdf", "tokens": 1234 }
filegoblin ./src/ --horde -q --json | jq '.[].path'
--json emits a deterministic, alphabetically-sorted array. A successful record is
{ "path", "format", "tokens", "content" } (format is a coarse type like pdf,
spreadsheet, code, image, text; tokens is an LLM token estimate). A file that
fails to parse appears in-band as { "path", "format", "error" } rather than silently
vanishing — so an agent can trust that every input is accounted for. Exit codes: 0 success,
4 when nothing could be extracted (empty result vs. total failure), 3 when
--scrub --scrub-fail redacted something, 5 when the run went over its memory ceiling (--memory).
Multi-Format Output (--format markdown|plain|json / -f):
# Default GFM markdown with tables, math, and code fences
filegoblin document.pdf -f markdown > doc.md
# Strip markdown punctuation for dense vector embeddings & BM25 search
# (Pipe tables become space-aligned text columns; headings & math stripped of tokens)
filegoblin document.pdf -f plain > embedding_input.txt
filegoblin MCP soruları
filegoblin MCP'yi Claude Code'a nasıl eklerim?
Terminalinizde claude mcp add filegoblin -- npx -y filegoblin komutunu çalıştırın. Her projede kullanılabilir olması için --scope user ekleyin.
filegoblin MCP'yi Cursor'a nasıl eklerim?
~/.cursor/mcp.json dosyasını açın ve yukarıdaki yükleme bölümünde gösterilen Cursor yapılandırmasını kullanarak sunucuyu mcpServers altına ekleyin.
filegoblin dosyaları yerel olarak mı işler?
Evet, tüm ayrıştırma ve PII temizleme işlemleri bulut API'leri kullanılmadan tamamen kendi makinenizde gerçekleşir.
Hangi dosya formatları destekleniyor?
PDF, Word, Excel, PowerPoint, e-posta, EPUB, HTML, Jupyter, Kod ve daha fazlasını destekler.
Taranmış belgeleri işleyebilir mi?
Evet, düzen ve okuma sırası için özelleşmiş vision modelleri ile taranmış PDF'ler ve görseller için OCR yeteneklerine sahiptir.
Büyük belgeleri LLM'ler için nasıl işler?
Tabloları ve kod bloklarını koruyan ve tablo başlıklarını parçalar arasında yeniden oluşturan, düzen farkındalıklı semantik chunking özelliğine sahiptir.