MCP server de diff

MCP Server Diff

A GitHub Action for diffing Model Context Protocol (MCP) server public interfaces between versions. Compares the current branch against a baseline to surface any changes to your server's exposed tools, resources, prompts, and capabilities.

Also available as a standalone CLI — see CLI Documentation or install with npx mcp-server-diff

Overview

MCP servers expose a public interface to AI assistants: tools (with their input schemas), resources, prompts, and server capabilities. As your server evolves, changes to this interface are worth tracking. This action automates public interface comparison by:

1. Building your MCP server from both the current branch and a baseline (merge-base, tag, or specified ref)
2. Querying both versions for their complete public interface (tools, resources, prompts, capabilities)
3. Generating a diff report showing exactly what changed
4. Surfacing results directly in GitHub's Job Summary

This is not about testing internal logic or correctness—it's about visibility into what your server advertises to clients.

Quick Start

Create .github/workflows/mcp-diff.yml in your repository:

name: MCP Server Diff

on:
  pull_request:
    branches: [main]
  push:
    branches: [main]
    tags: ['v*']

permissions:
  contents: read

jobs:
  mcp-diff:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.0
        with:
          fetch-depth: 0

      - uses: SamMorrowDrums/mcp-server-diff@v3
        with:
          setup_node: true
          install_command: npm ci
          build_command: npm run build
          start_command: node dist/stdio.js

Language Examples

Node.js / TypeScript

- uses: SamMorrowDrums/mcp-server-diff@v3
  with:
    setup_node: true
    node_version: '22'
    install_command: npm ci
    build_command: npm run build
    start_command: node dist/stdio.js

Python

- uses: SamMorrowDrums/mcp-server-diff@v3
  with:
    setup_python: true
    python_version: '3.12'
    install_command: pip install -e .
    start_command: python -m my_mcp_server

Go

- uses: SamMorrowDrums/mcp-server-diff@v3
  with:
    setup_go: true
    install_command: go mod download
    build_command: go build -o bin/server ./cmd/stdio
    start_command: ./bin/server

Rust

- uses: SamMorrowDrums/mcp-server-diff@v3
  with:
    setup_rust: true
    install_command: cargo fetch
    build_command: cargo build --release
    start_command: ./target/release/my-mcp-server

C# / .NET

- uses: SamMorrowDrums/mcp-server-diff@v3
  with:
    setup_dotnet: true
    dotnet_version: '9.0.x'
    install_command: dotnet restore
    build_command: dotnet build -c Release
    start_command: dotnet run --no-build -c Release

Custom Setup

If you need more control over environment setup (caching, specific registries, etc.), do your own setup before calling the action:

steps:
  - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.0
    with:
      fetch-depth: 0

  - uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
    with:
      node-version: '22'
      cache: 'npm'
      registry-url: 'https://npm.pkg.github.com'

  - uses: SamMorrowDrums/mcp-server-diff@v3
    with:
      install_command: npm ci
      build_command: npm run build
      start_command: node dist/stdio.js

Testing Multiple Transports

Test both stdio and HTTP transports in a single run using the configurations input:

- uses: SamMorrowDrums/mcp-server-diff@v3
  with:
    setup_node: true
    install_command: npm ci
    build_command: npm run build
    configurations: |
      [
        {
          "name": "stdio",
          "transport": "stdio",
          "start_command": "node dist/stdio.js"
        },
        {
          "name": "streamable-http",
          "transport": "streamable-http",
          "start_command": "node dist/http.js",
          "server_url": "http://localhost:3000/mcp"
        }
      ]

Inputs Reference

Language Setup (Optional)

Input	Description	Default
setup_node	Set up Node.js environment	false
node_version	Node.js version	22
setup_python	Set up Python environment	false
python_version	Python version	3.11
setup_go	Set up Go environment	false
go_version	Go version (reads from go.mod if empty)	""
setup_rust	Set up Rust environment	false
rust_toolchain	Rust toolchain	stable
setup_dotnet	Set up .NET environment	false
dotnet_version	.NET version	8.0.x

Required Inputs

Input	Description
install_command	Command to install dependencies (e.g., npm ci, pip install -e ., go mod download)

Server Configuration

Input	Description	Default
build_command	Command to build the server. Optional for interpreted languages.	""
start_command	Command to start the server for stdio transport	""
transport	Transport type: stdio or streamable-http	stdio
server_url	Server URL for HTTP transport (e.g., http://localhost:3000/mcp)	""
configurations	JSON array of test configurations for testing multiple transports	""
server_timeout	Timeout in seconds to wait for server response	10
env_vars	Environment variables as newline-separated KEY=VALUE pairs	""
client_capabilities	JSON object advertised as the MCP client's capabilities. See Client Capability Profile.	Standard core profile

Either start_command (for stdio) or server_url (for HTTP) must be provided, unless using configurations.

Comparison Configuration

Input	Description	Default
compare_ref	Git ref to compare against. Auto-detects merge-base on PRs or previous tag on tag pushes if not specified.	""
fail_on_diff	Fail the action if API changes are detected. Useful for release validation workflows.	false
fail_on_error	Fail the action if a genuine probe error occurs (e.g. the server fails to start on both sides). A configuration that starts on only one side is reported as "missing on one side" and is not treated as a probe error — see One-Sided Startup Failures.	true

Configuration Object Schema

When using configurations, each object supports:

Field	Description	Required
name	Identifier for this configuration (appears in report)	Yes
transport	stdio or streamable-http	No (default: stdio)
start_command	Server start command (stdio: spawns process, HTTP: starts server in background)	Yes for stdio, optional for HTTP
server_url	URL for HTTP transport	Required for streamable-http
startup_wait_ms	Milliseconds to wait for HTTP server to start (when using start_command)	No (default: 2000)
pre_test_command	Command to run before probing (alternative to start_command for HTTP)	No
pre_test_wait_ms	Milliseconds to wait after pre_test_command	No
post_test_command	Command to run after probing (cleanup, used with pre_test_command)	No
headers	HTTP headers for this configuration	No
env_vars	Additional environment variables	No
custom_messages	Config-specific custom messages	No
client_capabilities	Config-specific capability object; replaces the global/default profile	No
base_start_command	Command for baseline comparison (skips git checkout for this config)	No
base_server_url	URL for baseline HTTP server (used with base_start_command)	No

Client Capability Profile

Servers can conditionally expose public-interface elements based on what an MCP client advertises. The probe sends the same client capability object through both protocol paths:

• On MCP 2026-07-28+, every stateless request includes it at _meta.io.modelcontextprotocol/clientCapabilities, including server/discover, list requests, and custom requests.
• On earlier protocols, the SDK client includes it in initialize.params.capabilities.

When client_capabilities is empty or omitted, the probe advertises every standardized core client capability from MCP 2026-07-28:

{
  "roots": {},
  "sampling": {
    "context": {},
    "tools": {}
  },
  "elicitation": {
    "form": {},
    "url": {}
  }
}

This default makes capability-gated interface surfaces visible, including tools that require form elicitation. It intentionally does not opt into extensions such as Tasks.

Provide a JSON object to replace the profile completely. Use {} to advertise no optional capabilities. Capability names are open-ended and each top-level value must be a JSON object, so future and vendor-defined capabilities round-trip without a release of this action:

- uses: SamMorrowDrums/mcp-server-diff@v3
  with:
    start_command: ./server
    client_capabilities: |
      {
        "elicitation": {
          "form": {}
        },
        "extensions": {
          "io.modelcontextprotocol/tasks": {},
          "com.example/future-client": {
            "version": 2
          }
        },
        "com.example/top-level": {
          "enabled": true
        }
      }

client_capabilities inside a configurations entry overrides the global input for that configuration, which is useful when intentionally comparing capability-dependent surfaces.

[!IMPORTANT] Capabilities are an interface-discovery declaration, not an interactive client implementation. The probe sends discovery and list requests; it does not invoke tools, provide model sampling, collect elicited user input, follow elicitation URLs, or execute extension workflows. Advertising additional capabilities is safe for inspecting conditional interface visibility, but a server that actively exercises a client callback while handling discovery/list requests can cause that probe to fail. Only declare extensions whose conditional surface you intend to validate.

Comparing Against External Servers

When comparing against external servers (e.g., Docker images, remote services), use base_start_command to specify a different command for the baseline. This skips git checkout for that configuration and probes the specified server directly:

configurations: |
  [
    {
      "name": "compare-versions",
      "transport": "stdio",
      "start_command": "docker run -i ghcr.io/example/mcp-server:v2.0.0",
      "base_start_command": "docker run -i ghcr.io/example/mcp-server:v1.0.0"
    }
  ]

This is useful for:

• Version comparison: Compare a new release against the previous version
• Golden reference testing: Compare your local code against a known-good reference
• Cross-implementation testing: Compare different implementations of the same server
• Self-testing CI: Verify the action detects diffs by comparing two known-different servers

For HTTP transport, use base_server_url alongside base_start_command:

configurations: |
  [
    {
      "name": "http-comparison",
      "transport": "streamable-http",
      "start_command": "docker run -p 3000:3000 myserver:latest",
      "server_url": "http://localhost:3000/mcp",
      "base_start_command": "docker run -p 3001:3000 myserver:v1.0.0",
      "base_server_url": "http://localhost:3001/mcp"
    }
  ]

Security

The action executes configured install, build, and server commands, and its server processes inherit the job environment. Use trusted configuration and avoid exposing secrets or write-capable tokens when running it against untrusted pull request code. Grant workflows only the permissions they need, and pass servers only the credentials required for the probe.

How It Works

Execution Flow

1. Baseline Detection: Determines the comparison ref:
   • For pull requests: merge-base with target branch
   • For tag pushes: previous tag (e.g., v1.1.0 compares against v1.0.0)
   • Explicit: uses compare_ref if provided
2. Build Baseline: Creates a git worktree at the baseline ref and builds the server
3. Build Current: Builds the server from the current branch
4. Conformance Testing: Sends MCP protocol requests to both servers:
   • initialize - Server capabilities and metadata
   • tools/list - Available tools and their schemas
   • resources/list - Available resources
   • prompts/list - Available prompts
5. Report Generation: Produces a Markdown report with diffs, uploaded as an artifact and displayed in Job Summary

What Gets Compared

The action queries the public interface of both server versions and compares the responses:

Method	What It Reveals
initialize	Server name, version, capabilities
tools/list	Available tools and their JSON schemas
resources/list	Exposed resources
prompts/list	Available prompts

Differences appear as unified diffs in the report. Common changes include:

• New tools, resources, or prompts added
• Schema changes (new parameters, updated descriptions)
• Capability changes (new features enabled)
• Version string updates

One-Sided Startup Failures

Sometimes a configuration starts on one side of the comparison but fails to start on the other. The most common cause is a brand-new configuration that depends on a CLI flag or option that doesn't exist on the compare ref yet — for example a PR that introduces a new scoped server mode. On the compare ref the unknown flag is rejected and the server exits with something like MCP error -32000: Connection closed.

When exactly one side fails to start (and the other probes successfully), the action:

1. Reports a fail on the side that could not start, naming the version (current branch or compare ref) in a dedicated "🚫 missing on one side" section and a per-config callout, instead of an opaque probe error.
2. Diffs the working side against an empty baseline, so the entire surface of the new (or removed) configuration renders as added/removed — giving you a complete picture of what it exposes.
3. Does not trip fail_on_error. A one-sided startup failure is treated as a surface difference (config-missing), not a genuine probe error, so an introducing PR isn't failed for the expected "this config doesn't exist on the base ref yet" case. It still counts as a difference, so fail_on_diff will flag it if enabled.

When both sides fail to start, that is a genuine probe failure: it is reported as an error and trips fail_on_error (when enabled).

Transport Support

stdio Transport

The default transport communicates with your server via stdin/stdout using JSON-RPC. For stdio, each configuration spawns a fresh server process:

- uses: SamMorrowDrums/mcp-server-diff@v3
  with:
    setup_node: true
    install_command: npm ci
    build_command: npm run build
    start_command: node dist/stdio.js

Streamable HTTP Transport

For HTTP servers, you typically want to start the server once and test multiple configurations against it. Use start_command at the configuration level—the action spawns the server, waits for startup, probes it, then terminates it after that configuration completes:

configurations: |
  [{
    "name": "http-server",
    "transport": "streamable-http",
    "start_command": "node dist/http.js",
    "server_url": "http://localhost:3000/mcp",
    "startup_wait_ms": 2000
  }]

Per-configuration server lifecycle: If your use case requires a fresh server instance per configuration (e.g., testing different flags or environment variables), include start_command in each configuration—each will get its own server process started and stopped.

Shared server for multiple configurations: If you want one HTTP server to handle multiple test configurations, use pre_test_command/post_test_command on the first/last configuration, or start the server in a prior workflow step:

configurations: |
  [
    {
      "name": "config-a",
      "transport": "streamable-http",
      "server_url": "http://localhost:3000/mcp",
      "pre_test_command": "node dist/http.js &",
      "pre_test_wait_ms": 2000
    },
    {
      "name": "config-b",
      "transport": "streamable-http",
      "server_url": "http://localhost:3000/mcp"
    },
    {
      "name": "config-c",
      "transport": "streamable-http",
      "server_url": "http://localhost:3000/mcp",
      "post_test_command": "pkill -f 'node dist/http.js' || true"
    }
  ]

Pre-deployed servers: For already-running servers (staging, production), omit lifecycle commands entirely:

- uses: SamMorrowDrums/mcp-server-diff@v3
  with:
    install_command: 'true'
    transport: streamable-http
    server_url: https://mcp.example.com/api

Version Comparison Strategies

Pull Requests

On pull requests, the action automatically compares against the merge-base with the target branch. This shows exactly what changes the PR introduces.

Tag Releases

When triggered by a tag push matching v*, the action finds the previous tag and compares against it:

on:
  push:
    tags: ['v*']

# v1.2.0 will automatically compare against v1.1.0

Explicit Baseline

Specify any git ref to compare against:

- uses: SamMorrowDrums/mcp-server-diff@v3
  with:
    setup_node: true
    install_command: npm ci
    build_command: npm run build
    start_command: node dist/stdio.js
    compare_ref: v1.0.0

Failing on Changes (Release Validation)

For release workflows where you want to ensure no API changes, use fail_on_diff:

- uses: SamMorrowDrums/mcp-server-diff@v3
  with:
    setup_node: true
    install_command: npm ci
    build_command: npm run build
    start_command: node dist/stdio.js
    compare_ref: v1.0.0
    fail_on_diff: true  # Action fails if any API changes are detected

Artifacts and Reports

The action produces:

1. Job Summary: Inline Markdown report in the GitHub Actions UI showing test results and diffs
2. Artifact: mcp-diff-report artifact containing MCP_DIFF_REPORT.md for download or further processing

Example Output

No Changes Detected

When the MCP server's public interface hasn't changed between branches:

📊 Comparison:
  Current: HEAD
  Compare: abc1234 (v1.0.0)

🧪 Running diff...

📊 Phase 3: Comparing results...
📋 Configuration stdio: ✅ No changes

✅ No API Changes - All configurations match the baseline.

Changes Detected

When changes are detected, the action shows a semantic diff with clear paths to each change:

📋 Configuration stdio: 3 change(s) found

The generated report shows exactly what changed using path notation:

--- base/tools.json
+++ branch/tools.json

+ tools[new_tool]: {"name": "new_tool", "description": "A newly added tool", ...}
- tools[old_tool].inputSchema.properties.name.description: "Old description"
+ tools[old_tool].inputSchema.properties.name.description: "Updated description"
- tools[calculator].inputSchema.properties.precision.type: "string"  
+ tools[calculator].inputSchema.properties.precision.type: "number"

--- base/resources.json
+++ branch/resources.json

+ resources[config://settings]: {"uri": "config://settings", "name": "Settings", ...}

Each line shows:

• + for additions (new tools, resources, or changed values)
• - for removals (deleted items or previous values)
• Full path to the change: tools[tool_name].inputSchema.properties.param.type

This makes it easy to see exactly what changed without wading through entire JSON dumps

Recommended Workflow

name: MCP Server Diff

on:
  workflow_dispatch:
  pull_request:
    branches: [main]
  push:
    branches: [main]
    tags: ['v*']

permissions:
  contents: read

jobs:
  mcp-diff:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.0
        with:
          fetch-depth: 0

      - uses: SamMorrowDrums/mcp-server-diff@v3
        with:
          setup_node: true
          install_command: npm ci
          build_command: npm run build
          configurations: |
            [
              {
                "name": "stdio",
                "transport": "stdio",
                "start_command": "node dist/stdio.js"
              },
              {
                "name": "streamable-http",
                "transport": "streamable-http",
                "start_command": "node dist/http.js",
                "server_url": "http://localhost:3000/mcp"
              }
            ]

Troubleshooting

Server fails to start

• Check that start_command works locally
• Increase server_timeout for slow-starting servers
• Verify all dependencies are installed by install_command

Missing baseline

• Ensure fetch-depth: 0 in your checkout step
• For new repositories, the first run may fail (no baseline exists)

HTTP transport connection refused

• Verify server_url matches your server's listen address
• Ensure the server binds to 0.0.0.0 or 127.0.0.1, not just localhost on some systems
• Check firewall or container networking if running in Docker

---

CLI Tool

The CLI lets you diff any two MCP servers directly from your terminal—useful for local development, CI pipelines, or comparing servers across different implementations.

Installation

# Run directly with npx (no install required)
npx mcp-server-diff --help

# Or install globally
npm install -g mcp-server-diff

Basic Usage

# Compare two local stdio servers
npx mcp-server-diff -b "python -m mcp_server" -t "node dist/stdio.js"

# Compare local server vs remote HTTP endpoint
npx mcp-server-diff -b "go run ./cmd/server stdio" -t "https://mcp.example.com/api"

# Replace the default capability profile for both servers
npx mcp-server-diff -b "./server-v1" -t "./server-v2" \
  --client-capabilities '{"elicitation":{"form":{}}}'

# Output formats
npx mcp-server-diff -b "..." -t "..." -o diff      # Raw diff hunks only
npx mcp-server-diff -b "..." -t "..." -o json      # Full JSON with details
npx mcp-server-diff -b "..." -t "..." -o markdown  # Formatted report
npx mcp-server-diff -b "..." -t "..." -o summary   # One-line summary (default)

HTTP Headers & Authentication

For authenticated HTTP endpoints, pass headers with -H (target) or --base-header:

# Direct header value for target
npx mcp-server-diff -b "./server" -t "https://api.example.com/mcp" \
  -H "Authorization: Bearer your-token-here"

# Read from environment variable (keeps secrets out of shell history)
export MCP_TOKEN="your-secret-token"
npx mcp-server-diff -b "./server" -t "https://api.example.com/mcp" \
  -H "Authorization: Bearer env:MCP_TOKEN"

# Prompt for secret interactively (hidden input, named "token")
npx mcp-server-diff -b "./server" -t "https://api.example.com/mcp" \
  -H "Authorization: Bearer secret:token"