# CortexScout MCP server

> Enables AI agents to perform deep web research, structured data extraction, and stateful browser automation with anti-bot bypass capabilities.

- Repository: https://github.com/DevsHero/ShadowCrawl
- Page: https://hostedmcp.org/servers/devshero/shadow-crawl
- Transport: remote
- Stars: 71
- License: MIT
- Latest version: 2.0.0-rc

## Install

### Claude Code

```sh
claude mcp add --transport http shadow-crawl https://shadowcrawl--devshero.run.tools/
```

### Cursor (`~/.cursor/mcp.json`)

```json
{
  "mcpServers": {
    "shadow-crawl": {
      "url": "https://shadowcrawl--devshero.run.tools/"
    }
  }
}
```

### VS Code (`.vscode/mcp.json`)

```json
{
  "servers": {
    "shadow-crawl": {
      "type": "http",
      "url": "https://shadowcrawl--devshero.run.tools/"
    }
  }
}
```

### VS Code CLI

```sh
code --add-mcp '{"name":"shadow-crawl","type":"http","url":"https://shadowcrawl--devshero.run.tools/"}'
```

### Codex

```sh
codex mcp add shadow-crawl --url https://shadowcrawl--devshero.run.tools/
```

### Codex config.toml (`~/.codex/config.toml`)

```toml
[mcp_servers.shadow-crawl]
url = "https://shadowcrawl--devshero.run.tools/"
```

### Gemini CLI

```sh
gemini mcp add -s user --transport http shadow-crawl https://shadowcrawl--devshero.run.tools/
```

### Gemini settings.json (`~/.gemini/settings.json`)

```json
{
  "mcpServers": {
    "shadow-crawl": {
      "url": "https://shadowcrawl--devshero.run.tools/",
      "type": "http"
    }
  }
}
```

## FAQ

### How does CortexScout handle anti-bot protections like Cloudflare or DataDome?

It uses progressive fallbacks including native retrieval, Chromium CDP rendering, and Human-in-the-Loop (HITL) workflows to bypass challenges.

### Can I use this for automated testing like Playwright?

Yes, it includes a built-in stateful CDP automation engine designed to replace frameworks like Playwright for E2E testing workflows.

### What happens if the agent encounters a CAPTCHA?

The agent can use the Agent Auth Portal to launch the browser profile in a visible window, allowing a human to solve the CAPTCHA and save the session cookies.
