# ModelWatch MCP server

> Monitors LLM applications for behavioral drift by creating and running evaluation specs directly within an MCP-compatible client.

- Repository: https://github.com/bch1212/modelwatch
- Page: https://hostedmcp.org/servers/bch1212/modelwatch
- Transport: stdio
- Stars: 0
- Latest version: 0.1.3

## Install

### Claude Code

```sh
claude mcp add --env 'MODELWATCH_API_KEY=<YOUR_MODELWATCH_API_KEY>' modelwatch -- npx -y modelwatch-mcp
```

### Cursor (`~/.cursor/mcp.json`)

```json
{
  "mcpServers": {
    "modelwatch": {
      "command": "npx",
      "args": [
        "-y",
        "modelwatch-mcp"
      ],
      "env": {
        "MODELWATCH_API_KEY": "<YOUR_MODELWATCH_API_KEY>"
      }
    }
  }
}
```

### VS Code (`.vscode/mcp.json`)

```json
{
  "servers": {
    "modelwatch": {
      "type": "stdio",
      "command": "npx",
      "args": [
        "-y",
        "modelwatch-mcp"
      ],
      "env": {
        "MODELWATCH_API_KEY": "<YOUR_MODELWATCH_API_KEY>"
      }
    }
  }
}
```

### VS Code CLI

```sh
code --add-mcp '{"name":"modelwatch","type":"stdio","command":"npx","args":["-y","modelwatch-mcp"],"env":{"MODELWATCH_API_KEY":"<YOUR_MODELWATCH_API_KEY>"}}'
```

### Claude Desktop (`claude_desktop_config.json`)

```json
{
  "mcpServers": {
    "modelwatch": {
      "command": "npx",
      "args": [
        "-y",
        "modelwatch-mcp"
      ],
      "env": {
        "MODELWATCH_API_KEY": "<YOUR_MODELWATCH_API_KEY>"
      }
    }
  }
}
```

### Codex

```sh
codex mcp add modelwatch --env 'MODELWATCH_API_KEY=<YOUR_MODELWATCH_API_KEY>' -- npx -y modelwatch-mcp
```

### Codex config.toml (`~/.codex/config.toml`)

```toml
[mcp_servers.modelwatch]
command = "npx"
args = ["-y", "modelwatch-mcp"]

[mcp_servers.modelwatch.env]
MODELWATCH_API_KEY = "<YOUR_MODELWATCH_API_KEY>"
```

### Gemini CLI

```sh
gemini mcp add -s user -e 'MODELWATCH_API_KEY=<YOUR_MODELWATCH_API_KEY>' modelwatch npx -- -y modelwatch-mcp
```

### Gemini settings.json (`~/.gemini/settings.json`)

```json
{
  "mcpServers": {
    "modelwatch": {
      "command": "npx",
      "args": [
        "-y",
        "modelwatch-mcp"
      ],
      "env": {
        "MODELWATCH_API_KEY": "<YOUR_MODELWATCH_API_KEY>"
      }
    }
  }
}
```

## Environment variables

| Name | Required | Secret | Description |
|---|---|---|---|
| `MODELWATCH_API_KEY` | yes | yes | API key from https://modelwatch.app (free tier: 5 specs, 500 runs/mo). |
| `MODELWATCH_API_BASE` | no | no | Override the API base URL for self-hosted deployments. |

## Tools (9)

- `list_endpoints`
- `create_endpoint`
- `list_specs`
- `create_spec`
- `run_spec`
- `reset_baseline`
- `get_drift_events`
- `get_spec_history`
- `get_health`

## FAQ

### How do I install ModelWatch MCP?

You can install it globally using npm by running 'npm install -g modelwatch-mcp'.

### Is there a free tier available?

Yes, you can get a free key that includes 5 specs and 500 runs per month without a credit card.
