# WebCrawlerAPI MCP server

> Converts any web page into clean Markdown for AI assistants, handling JavaScript-rendered content and stripping boilerplate.

- Repository: https://github.com/WebCrawlerAPI/mcp-server
- Page: https://hostedmcp.org/servers/webcrawlerapi/mcp-server
- Transport: remote
- Stars: 0
- License: MIT
- Latest version: 0.1.3

## Install

### Claude Code

```sh
claude mcp add --transport http mcp-server https://mcp.webcrawlerapi.com/mcp
```

### Cursor (`~/.cursor/mcp.json`)

```json
{
  "mcpServers": {
    "mcp-server": {
      "url": "https://mcp.webcrawlerapi.com/mcp"
    }
  }
}
```

### VS Code (`.vscode/mcp.json`)

```json
{
  "servers": {
    "mcp-server": {
      "type": "http",
      "url": "https://mcp.webcrawlerapi.com/mcp"
    }
  }
}
```

### VS Code CLI

```sh
code --add-mcp '{"name":"mcp-server","type":"http","url":"https://mcp.webcrawlerapi.com/mcp"}'
```

### Codex

```sh
codex mcp add mcp-server --url https://mcp.webcrawlerapi.com/mcp
```

### Codex config.toml (`~/.codex/config.toml`)

```toml
[mcp_servers.mcp-server]
url = "https://mcp.webcrawlerapi.com/mcp"
```

### Gemini CLI

```sh
gemini mcp add -s user --transport http mcp-server https://mcp.webcrawlerapi.com/mcp
```

### Gemini settings.json (`~/.gemini/settings.json`)

```json
{
  "mcpServers": {
    "mcp-server": {
      "url": "https://mcp.webcrawlerapi.com/mcp",
      "type": "http"
    }
  }
}
```

## Tools (1)

- `url`

## FAQ

### How do I authenticate with WebCrawlerAPI?

The server uses OAuth 2.1. When you make your first request, a browser window will open for you to sign in and approve access for your organization.

### Can the server read pages that use JavaScript?

Yes, the server runs on WebCrawlerAPI, which ensures that JavaScript-rendered pages are processed correctly.

### What happens if a page cannot be fetched?

The tool will return an error message explaining the failure, such as an unreachable site, a robots.txt block, a timeout, or an empty balance.

### Does the server log my data?

The server receives requested URLs and returns content, but it does not log tokens or full URLs containing query strings.
