llm.energy MCP server
Commit Nov 15, 2026Extracts llms.txt and install.md documentation from websites into structured, agent-ready markdown for AI clients.
Also known as nirholas/extract-llms-docs, extract-llms-docs
- Stars
- 35
- Tools
- 9
- Downloads, 30 days
- โ
- Latest version
- 1.0.0
- Last commit
- Nov 15, 2026
- License
- โ
What is llm.energy MCP?
llm.energy extracts llms.txt and install.md documentation from websites into structured, agent-ready markdown for AI clients. It exposes 9 tools, such as extract_docs, validate_url, verify_llms_txt.
Summary generated from the project's README
About llm.energy MCP server
Converting website documentation into structured formats for AI agents is what Documentation Extractor enables. It specifically targets websites that implement the llms.txt and install.md standards, transforming raw data into agent-ready markdown, JSON, YAML, or ZIP archives. Developers can use this to feed up-to-date context into coding assistants like Cursor or Windsurf.
Users connect the server to clients such as Claude Desktop or Cursor via npx. The server provides specialized tools including extract_docs, validate_url, and discover_documentation_urls to help agents navigate and ingest technical information. It can also process multiple URLs at once through batch processing capabilities.
Beyond simple extraction, the system can generate new documentation. It features an AI-powered generator that creates install.md files by analyzing public GitHub repositories, existing documentation URLs, or manual input. This makes it a versatile tool for both consuming external knowledge and preparing your own projects for AI-driven workflows.
What you can do with it
- 1Feed documentation to coding assistantsProvide up-to-date technical context to AI tools like Cursor or Windsurf.
- 2Generate installation instructions from GitHubCreate executable installation files by analyzing a public repository's structure.
- 3Automate documentation pipelines for RAGBuild automated workflows that aggregate and structure documentation for retrieval-augmented generation systems.
- 4Verify documentation standardsCheck if a specific website supports the llms.txt standard before attempting extraction.
Written from the project's README. Every use case is checked against a quote from it.
llm.energy MCP tools 9
Detected from the READMEextract_docsvalidate_urlverify_llms_txtdiscover_documentation_urlslist_sitessearch_sitesfetch_llms_txtfetch_llms_full_txtextract_documentation
| extract_docs | โ | |
| validate_url | โ | |
| verify_llms_txt | โ | |
| discover_documentation_urls | โ | |
| list_sites | โ | |
| search_sites | โ | |
| fetch_llms_txt | โ | |
| fetch_llms_full_txt | โ | |
| extract_documentation | โ |
Automated checks
Ran Nov 14, 2026Full report โNot verified yet. A server needs a score of 80, no serious finding, and enough data for us to check. How checks work ยท What to fix for more points
- Source repository linked and reachablenirholas/extract-llms-docs is public and reachablegithub.com/nirholas/extract-llms-docs โPass
- Package names this repository as its sourceNot checked
- Build provenance (Sigstore attestation)Not checked
- Open source license (OSI)No licenseFinding
- Commit or release in the last yearLast commit or release 2026-11-13github.com/nirholas/extract-llms-docs โPass
- OpenSSF ScorecardNot checked
- Package name not close to a popular packageNot checked
- No critical advisory in the package or its dependenciesNot checked
- No hidden instructions to the model9 tool descriptions and the README read; no override phrases, credential paths or hidden charactersPass
- Destructive tools are labelledNo tool named like delete, drop or execPass
- Remote endpoint answers the MCP handshakeNot checked
README
View on GitHubllm.energy
๐ Overview
llm.energy is a web application and MCP server that fetches, parses, and organizes documentation from websites implementing the llms.txt and install.md standards. It transforms raw documentation into structured, agent-ready formats optimized for large language models, AI assistants, and developer tooling.
| Standard | Description | Learn More |
|---|---|---|
| llms.txt | Machine-readable documentation for AI systems | llmstxt.org |
| install.md | LLM-executable installation instructions | installmd.org |
โจ Key Features
| Feature | Description |
|---|---|
| ๐ Smart Detection | Automatically finds llms.txt, llms-full.txt, and install.md |
| ๐ Organized Output | Splits content into individual markdown files by section |
| ๐ค Agent-Ready | Includes AGENT-GUIDE.md optimized for AI assistants |
| ๐ฆ Multiple Formats | Export as Markdown, JSON, YAML, or ZIP archive |
| ๐ MCP Server | Integrate with Claude Desktop, Cursor, and more |
| โก Batch Processing | Process multiple URLs simultaneously |
| ๐ Site Directory | Browse 19+ curated llms.txt-enabled websites |
| โ๏ธ llms.txt Generator | Create your own llms.txt files with a guided wizard |
| ๐ install.md Generator | AI-powered: generate from GitHub repos, docs URLs, or manually |
๐ฏ Use Cases
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
๐ Feed docs to AI coding assistants (Cursor, Windsurf)
๐ค Build context-aware AI agents with up-to-date docs
๐ Create documentation pipelines for RAG systems
๐ฆ Aggregate docs from multiple sources automatically
โ๏ธ Generate llms.txt/install.md for your own projects
๐ Auto-generate install.md from any GitHub repo
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
๐ install.md Generator
Generate LLM-executable installation instructions from any source:
| Mode | Description | Use Case |
|---|---|---|
| From GitHub | Analyze any public GitHub repo and generate install.md | Perfect for creating install.md for existing projects |
| From URL | Extract from any documentation page | Convert existing docs to install.md format |
| Manual | Build from scratch with guided wizard | Full control over every detail |
How It Works
- GitHub Mode: Analyzes README, package.json/pyproject.toml/Cargo.toml, GitHub Actions, and releases
- URL Mode: Scrapes documentation pages, detects platforms (Mintlify, Docusaurus, GitBook, etc.)
- AI Synthesis: Uses Claude to generate a properly formatted install.md
Quick Example
# Generate install.md for any GitHub project
curl -X POST https://llm.energy/api/generate-install \
-H "Content-Type: application/json" \
-d '{"url": "https://github.com/anthropics/anthropic-sdk-python", "type": "github"}'
Visit llm.energy/install-generator to use the web interface.
๐ Documentation
Full documentation is available at llm.energy/docs
๐ Installation
Web Application
Visit llm.energy to use the hosted version.
Local Development
# Clone the repository
git clone https://github.com/nirholas/extract-llms-docs.git
cd extract-llms-docs
# Install dependencies
pnpm install
# Start development server
pnpm dev
๐ The application runs on http://localhost:3001
MCP Server
Add to your MCP client configuration (Claude Desktop, Cursor, etc.):
{
"mcpServers": {
"llm-energy": {
"command": "npx",
"args": ["-y", "@llm-energy/mcp-server"]
}
}
}
See MCP Server Documentation for detailed setup.
๐ API Reference
curl -X POST https://llm.energy/api/extract \
-H "Content-Type: application/json" \
-d '{"url": "docs.anthropic.com"}'
Response includes parsed sections, metadata, and download URLs.
curl -X POST https://llm.energy/api/validate \
-H "Content-Type: application/json" \
-d '{"url": "docs.example.com"}'
curl -X POST https://llm.energy/api/batch \
-H "Content-Type: application/json" \
-d '{"urls": ["docs.anthropic.com", "docs.stripe.com"]}'
# Formats: markdown, json, yaml, zip
curl "https://llm.energy/api/download?url=docs.anthropic.com&format=zip"
# Generate from a GitHub repository
curl -X POST https://llm.energy/api/generate-install \
-H "Content-Type: application/json" \
-d '{"url": "https://github.com/anthropics/anthropic-sdk-python", "type": "github"}'
# Generate from a documentation URL
curl -X POST https://llm.energy/api/generate-install \
-H "Content-Type: application/json" \
-d '{"url": "https://docs.anthropic.com/en/docs/quickstart", "type": "docs"}'
# Analyze only (preview mode)
curl "https://llm.energy/api/generate-install?url=https://github.com/anthropics/anthropic-sdk-python&type=github"
๐ค MCP Server Tools
The MCP server exposes the following tools for AI agents:
| Tool | Description |
|---|---|
extract_docs |
Extract documentation from a URL with llms.txt support |
validate_url |
Check if a URL has llms.txt available |
verify_llms_txt |
Verify llms.txt exists and get file info |
discover_documentation_urls |
Find documentation URLs for a domain |
list_sites |
Get directory of known llms.txt-enabled sites |
search_sites |
Search the site directory by category or keyword |
๐๏ธ Architecture
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ llm.energy โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโค
โ โ
โ โโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโ โ
โ โ Web App โ โ REST API โ โ MCP Server โ โ
โ โ (Next.js) โ โ /api/* โ โ (stdio) โ โ
โ โโโโโโโโฌโโโโโโโ โโโโโโโโฌโโโโโโโ โโโโโโโโฌโโโโโโโ โ
โ โ โ โ โ
โ โโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโ โ
โ โ โ
โ โโโโโโโโผโโโโโโโ โ
โ โ Core โ โ
โ โ - Parser โ โ
โ โ - Extractorโ โ
โ โ - Cache โ โ
โ โโโโโโโโฌโโโโโโโ โ
โ โ โ
โ โโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโ โ
โ โผ โผ โผ โ
โ โโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโ โ
โ โ llms.txt โ โ install.md โ โ Sitemap โ โ
โ โโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโ โ
โ โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
extract-llms-docs/
โโโ src/
โ โโโ app/ # Next.js pages and API routes
โ โ โโโ api/ # REST API endpoints
โ โ โ โโโ extract/ # Documentation extraction API
โ โ โ โโโ generate-install/ # AI-powered install.md generation
โ โ โ โโโ ... # Other API endpoints
โ โ โโโ extract/ # Extraction interface
โ โ โโโ batch/ # Batch processing page
โ โ โโโ directory/ # Site directory browser
โ โ โโโ generator/ # llms.txt generator wizard
โ โ โโโ install-generator/ # install.md generator with tabs
โ โโโ components/ # React UI components
โ โ โโโ install-generator/ # GitHubTab, UrlTab, Preview
โ โโโ lib/ # Core utilities
โ โ โโโ github-analyzer.ts # GitHub repo analysis
โ โ โโโ docs-analyzer.ts # Documentation URL scraping
โ โ โโโ ... # Parser, extractor, cache
โ โโโ hooks/ # React hooks
โ โโโ types/ # TypeScript definitions
โโโ packages/core/ # Shared parser and types
โโโ mcp-server/ # MCP server package
โโโ docs-site/ # MkDocs documentation source
๐ ๏ธ Technology Stack
โ๏ธ Configuration
All environment variables are optional โ the app runs without them. Copy .env.example to
.env.local and fill in only what you need:
cp .env.example .env.local
| Variable | Used by | Default | Purpose |
|---|---|---|---|
ADMIN_KEY |
/api/cache |
unset | Value the x-admin-key header must match for cache management. Both GET and POST return 500 when it is unset. Production reads it from Secret Manager (llm-energy-admin-key). |
ANTHROPIC_API_KEY |
/api/generate-install |
unset | Enables Claude-backed generation of install.md. The endpoint returns a configuration error without it. |
VERIFY_TIMEOUT_MS |
/api/sites/verify |
10000 |
Per-site fetch timeout, in milliseconds. |
VERIFY_CACHE_TTL_MS |
/api/sites/verify |
300000 |
How long verification results are cached, in milliseconds. |
HEALTH_TIMEOUT_MS |
/api/sites/health |
8000 |
Timeout for site health probes, in milliseconds. |
๐ Deployment
| Hostname | Serves | Hosted on |
|---|---|---|
llm.energy, www.llm.energy |
The Next.js app and its /api/* routes |
Cloud Run service llm-energy, fronted by the Cloudflare Worker llm-energy-proxy |
| docs.llm.energy | MkDocs site built from docs-site/ |
Cloudflare Pages project llm-energy |
Redeploy commands, DNS layout and rollback: docs/deploy-cloudflare.md.
๐ป Development
pnpm dev # ๐ Start development server
pnpm build # ๐ฆ Production build
pnpm start # โถ๏ธ Start production server
pnpm test # ๐งช Run tests (163 tests)
pnpm test:coverage # ๐ Tests with coverage report
pnpm typecheck # โ
TypeScript validation
pnpm lint # ๐ ESLint check
๐ Related Projects
| Project | Description |
|---|---|
| llms.txt Standard | Machine-readable documentation format |
| install.md Standard | LLM-executable installation format |
| Model Context Protocol | Protocol for AI tool integration |
| MCP Servers Directory | Community MCP servers |
๐ Sites with llms.txt Support
Browse 19+ curated websites with verified llms.txt support at llm.energy/directory
Featured sites include:
- ๐ค Anthropic Documentation
- โก Vercel Documentation
- ๐ณ Stripe API Reference
- ๐๏ธ Supabase Docs
- ๐ Mintlify Documentation
- ...and more!
๐ค Contributing
Contributions are welcome! Please follow these steps:
- ๐ด Fork the repository
- ๐ฟ Create a feature branch (
git checkout -b feature/amazing-feature) - ๐พ Commit your changes (
git commit -m 'Add amazing feature') - ๐ค Push to the branch (
git push origin feature/amazing-feature) - ๐ Open a Pull Request
Adding a site to the directory? Edit src/data/sites.ts and submit a PR.
๐ License
Proprietary โ Copyright 2026 nirholas, all rights reserved. See LICENSE for details.
๐ Links
๐ค Author
Built by nich
Author
Built by nich - https://x.com/nichxbt
Comprehensive keyword list for ERC-8004 Trustless Agents ecosystem
Core Protocol Keywords
ERC-8004, ERC8004, EIP-8004, EIP8004, Trustless Agents, trustless agent, trustless AI, trustless AI agents, agent protocol, agent standard, Ethereum agent standard, blockchain agent protocol, on-chain agents, onchain agents, on-chain AI, onchain AI, decentralized agents, decentralized AI agents, autonomous agents, autonomous AI agents, AI agent protocol, AI agent standard, agent discovery, agent trust, agent reputation, agent validation, agent identity, agent registry, identity registry, reputation registry, validation registry, agent NFT, ERC-721 agent, agent tokenId, agentId, agentURI, agentWallet, agent registration, agent registration file, agent-registration.json, agent card, agent metadata, agent endpoints, agent discovery protocol, agent trust protocol, open agent protocol, open agent standard, permissionless agents, permissionless AI, censorship-resistant agents, portable agent identity, portable AI identity, verifiable agents, verifiable AI agents, accountable agents, accountable AI, agent accountability
Blockchain & Web3 Keywords
Ethereum, Ethereum mainnet, ETH, EVM, Ethereum Virtual Machine, smart contracts, Solidity, blockchain, decentralized, permissionless, trustless, on-chain, onchain, L2, Layer 2, Base, Optimism, Polygon, Linea, Arbitrum, Scroll, Monad, Gnosis, Celo, Sepolia, testnet, mainnet, singleton contracts, singleton deployment, ERC-721, NFT, non-fungible token, tokenURI, URIStorage, EIP-712, ERC-1271, wallet signature, EOA, smart contract wallet, gas fees, gas sponsorship, EIP-7702, subgraph, The Graph, indexer, blockchain indexing, IPFS, decentralized storage, content-addressed, immutable data, public registry, public good, credibly neutral, credibly neutral infrastructure, open protocol, open standard, Web3, crypto, cryptocurrency, DeFi, decentralized finance
AI & Agent Technology Keywords
AI agents, artificial intelligence agents, autonomous AI, AI autonomy, LLM agents, large language model agents, machine learning agents, ML agents, AI assistant, AI chatbot, intelligent agents, software agents, digital agents, virtual agents, AI automation, automated agents, agent-to-agent, A2A, A2A protocol, Google A2A, Agent2Agent, MCP, Model Context Protocol, agent communication, agent interoperability, agent orchestration, agent collaboration, multi-agent, multi-agent systems, agent capabilities, agent skills, agent tools, agent prompts, agent resources, agent completions, AgentCard, agent card, agent endpoint, agent service, AI service, AI API, agent API, AI infrastructure, agent infrastructure, agentic, agentic web, agentic economy, agentic commerce, agent economy, agent marketplace, AI marketplace, agent platform, AI platform
Trust & Reputation Keywords
trust, trustless, reputation, reputation system, reputation protocol, reputation registry, feedback, client feedback, user feedback, on-chain feedback, on-chain reputation, verifiable reputation, portable reputation, reputation aggregation, reputation scoring, reputation algorithm, trust signals, trust model, trust verification, trust layer, recursive reputation, reviewer reputation, spam prevention, Sybil attack, Sybil resistance, anti-spam, feedback filtering, trusted reviewers, rating, rating system, quality rating, starred rating, uptime rating, success rate, response time, performance history, track record, audit trail, immutable feedback, permanent feedback, feedback response, appendResponse, giveFeedback, revokeFeedback, feedback tags, feedback value, valueDecimals, feedbackURI, feedbackHash, clientAddress, reviewer address
Validation & Verification Keywords
validation, validation registry, validator, validator contract, cryptographic validation, cryptographic proof, cryptographic attestation, zero-knowledge, ZK, zkML, zero-knowledge machine learning, ZK proofs, trusted execution environment, TEE, TEE attestation, TEE oracle, stake-secured, staking validators, crypto-economic security, inference re-execution, output validation, work verification, third-party validation, independent validation, validation request, validation response, validationRequest, validationResponse, requestHash, responseHash, verifiable computation, verified agents, verified behavior, behavioral validation, agent verification
Payment & Commerce Keywords
x402, x402 protocol, x402 payments, programmable payments, micropayments, HTTP payments, pay-per-request, pay-per-task, agent payments, AI payments, agent monetization, AI monetization, agent commerce, AI commerce, agentic commerce, agent economy, AI economy, agent marketplace, service marketplace, agent-to-agent payments, A2A payments, stablecoin payments, USDC, crypto payments, on-chain payments, payment settlement, programmable settlement, proof of payment, proofOfPayment, payment receipt, payment verification, Coinbase, Coinbase x402, agent pricing, API pricing, service pricing, subscription, API keys, revenue, trading yield, cumulative revenues, agent wallet, payment address, toAddress, fromAddress, txHash
Discovery & Registry Keywords
agent discovery, service discovery, agent registry, identity registry, agent registration, register agent, mint agent, agent NFT, agent tokenId, agent browsing, agent explorer, agent scanner, 8004scan, 8004scan.io, agentscan, agentscan.info, 8004agents, 8004agents.ai, agent leaderboard, agent ranking, top agents, agent listing, agent directory, agent catalog, agent index, browse agents, search agents, find agents, discover agents, agent visibility, agent discoverability, no-code registration, agent creation, create agent, my agents, agent owner, agent operator, agent transfer, transferable agent, portable agent
Endpoints & Integration Keywords
endpoint, agent endpoint, service endpoint, API endpoint, MCP endpoint, A2A endpoint, web endpoint, HTTPS endpoint, HTTP endpoint, DID, decentralized identifier, ENS, Ethereum Name Service, ENS name, agent.eth, vitalik.eth, email endpoint, OASF, Open Agent Specification Format, endpoint verification, domain verification, endpoint ownership, .well-known, well-known, agent-registration.json, endpoint domain, endpoint URL, endpoint URI, base64 data URI, on-chain metadata, off-chain metadata, metadata storage, JSON metadata, agent JSON, registration JSON
SDK & Developer Tools Keywords
SDK, Agent0 SDK, Agent0, ChaosChain SDK, ChaosChain, Lucid Agents, Daydreams AI, create-8004-agent, npm, TypeScript SDK, Python SDK, JavaScript, Solidity, smart contract, ABI, contract ABI, deployed contracts, contract addresses, Hardhat, development tools, developer tools, dev tools, API, REST API, GraphQL, subgraph, The Graph, indexer, blockchain explorer, Etherscan, contract verification, open source, MIT license, CC0, public domain, GitHub, repository, code repository, documentation, docs, best practices, reference implementation
Ecosystem & Community Keywords
ecosystem, community, builder, builders, developer, developers, contributor, contributors, partner, partners, collaborator, collaborators, co-author, co-authors, MetaMask, Ethereum Foundation, Google, Coinbase, Consensys, AltLayer, Virtuals Protocol, Olas, EigenLayer, Phala, ElizaOS, Flashbots, Polygon, Base, Optimism, Arbitrum, Scroll, Linea, Monad, Gnosis, Celo, Near Protocol, Filecoin, Worldcoin, ThirdWeb, ENS, Collab.land, DappRadar, Giza Tech, Theoriq, OpenServ, Questflow, Semantic, Semiotic, Cambrian, Nevermined, Oasis, Towns Protocol, Warden Protocol, Terminal3, Pinata Cloud, Silence Labs, Rena Labs, Index Network, Trusta Network, Turf Network
Key People & Organizations Keywords
Marco De Rossi, MetaMask AI Lead, Davide Crapis, Ethereum Foundation AI, Head of AI, Jordan Ellis, Google engineer, Erik Reppel, Coinbase engineering, Head of Engineering, Sumeet Chougule, ChaosChain founder, YQ, AltLayer co-founder, Wee Kee, Virtuals contributor, Cyfrin audit, Nethermind audit, Ethereum Foundation Security Team, security audit, audited contracts
llm.energy MCP questions
What tools does llm.energy MCP provide?
It exposes 9 tools, including extract_docs, validate_url, verify_llms_txt, discover_documentation_urls, list_sites.
What does llm.energy extract from a website?
It fetches llms.txt, llms-full.txt, and install.md files from sites that implement those standards and organizes them into structured markdown.
Which MCP tools does the llm.energy server provide?
The server exposes tools such as extract_docs, validate_url, verify_llms_txt, discover_documentation_urls, list_sites, and search_sites.
How do I add the llm.energy MCP server to my client?
Add an mcpServers entry named llm-energy that runs npx -y @llm-energy/mcp-server in your client configuration.
Can llm.energy process multiple URLs at once?
Yes, the batch API accepts a list of URLs and processes them together, and export formats include markdown, JSON, YAML, and ZIP.
Can it generate an install.md for my project?
Yes, the install.md generator can analyze a public GitHub repo, scrape a docs URL, or build one manually.