AI agent가 데스크톱을 UI 맵으로 읽고, 요소 이름을 기반으로 클릭, 타이핑 및 동작을 수행하며 각 작업을 검증합니다.
다른 이름 AmrDab/clawdcursor
- 스타
- 407
- Tools
- —
- 다운로드, 30일
- 3,358
- 최신 버전
- 2.0.0
- 마지막 커밋
- 2026년 11월 14일
- 라이선스
- MIT
Clawd Cursor MCP란?
Clawd Cursor은(는) MCP server입니다. AI agent가 데스크톱을 UI 맵으로 읽고, 요소 이름을 기반으로 클릭, 타이핑 및 동작을 수행하며 각 작업을 검증합니다.
프로젝트 README에서 생성된 요약
Clawd Cursor MCP server 정보
AI 에이전트가 데스크톱 애플리케이션 및 웹 브라우저와 상호작용해야 하는 개발자는 Clawd Cursor를 사용하여 컴퓨터를 탐색 가능한 UI 맵으로 변환할 수 있습니다. 이 서버는 단순히 시각적 좌표에 의존하는 대신, AI client가 버튼이나 텍스트 필드와 같은 인터페이스 요소의 이름을 식별하고 동작할 수 있도록 지원합니다. 브라우저 페이지나 스크린 리더가 사용하는 접근성 트리를 우선적으로 활용하고, 텍스트 인식 및 스크린샷을 보조 수단으로 사용하는 등 사용 가능한 가장 효율적인 방식을 통해 Windows, macOS, Linux에서 작동합니다.
사용자는 Claude 또는 Cursor와 같이 선호하는 MCP client를 서버에 연결하여 accessibility, system, window, browser, computer, batch와 같은 도구에 접근할 수 있습니다. AI client가 모든 결정을 내리므로 서버 자체적으로 모델을 제공하거나 API 키를 요구하지 않습니다. 모든 작업 후 서버는 변경 사항이 확인되었는지, 미확인되었는지, 또는 실패했는지를 보고하여 에이전트가 진행 상황을 검증할 수 있도록 합니다. 이 설정을 통해 에이전트는 그래픽 사용자 인터페이스 내에서만 수행 가능한 작업들을 처리할 수 있습니다.
활용 사례
- 1데스크톱 애플리케이션 워크플로 자동화요소 이름을 사용하여 네이티브 데스크톱 소프트웨어의 특정 버튼을 클릭하거나 양식을 채우도록 AI 에이전트를 지시합니다.
- 2Playwright를 통한 웹 브라우저 제어전용 브라우저 창을 통해 AI가 웹사이트를 탐색하고 페이지 요소와 상호작용하도록 합니다.
- 3비표준 UI 요소와 상호작용애플리케이션이 접근성 트리를 제공하지 않는 경우 텍스트 인식을 사용하여 레이블을 찾아 클릭합니다.
프로젝트의 README를 바탕으로 작성되었습니다. 모든 활용 사례는 해당 내용의 인용문과 대조하여 확인합니다.
Clawd Cursor MCP 설치 방법
형식 확인: 2026년 11월 10일Cursor에 추가Goose에 추가Claude Code에 Clawd Cursor MCP 설치
$ claude mcp add clawdcursor -- npx -y clawdcursorCursor에 Clawd Cursor MCP 설치
~/.cursor/mcp.json
{
"mcpServers": {
"clawdcursor": {
"command": "npx",
"args": [
"-y",
"clawdcursor"
]
}
}
}VS Code에 Clawd Cursor MCP 설치
.vscode/mcp.json
{
"servers": {
"clawdcursor": {
"type": "stdio",
"command": "npx",
"args": [
"-y",
"clawdcursor"
]
}
}
}VS Code CLI에 Clawd Cursor MCP 설치
$ code --add-mcp '{"name":"clawdcursor","type":"stdio","command":"npx","args":["-y","clawdcursor"]}'Claude Desktop에 Clawd Cursor MCP 설치
claude_desktop_config.json
{
"mcpServers": {
"clawdcursor": {
"command": "npx",
"args": [
"-y",
"clawdcursor"
]
}
}
}Codex에 Clawd Cursor MCP 설치
$ codex mcp add clawdcursor -- npx -y clawdcursorCodex config.toml에 Clawd Cursor MCP 설치
~/.codex/config.toml
[mcp_servers.clawdcursor] command = "npx" args = ["-y", "clawdcursor"]
Gemini CLI에 Clawd Cursor MCP 설치
$ gemini mcp add -s user clawdcursor npx -- -y clawdcursorGemini settings.json에 Clawd Cursor MCP 설치
~/.gemini/settings.json
{
"mcpServers": {
"clawdcursor": {
"command": "npx",
"args": [
"-y",
"clawdcursor"
]
}
}
}자동 검사
2026년 11월 12일에 실행됨전체 보고서 →자동 검사를 통과했습니다. 이는 보안 감사가 아닙니다. 코드를 직접 읽고 서버에 필요한 최소한의 권한만 부여하십시오. 검사 방식
- 소스 저장소 연결 및 접근 가능통과
- 오픈 소스 라이선스 (OSI)통과
- 최근 1년 내 커밋 또는 릴리스통과
- 패키지 이름이 인기 패키지와 유사하지 않음통과
- 원격 엔드포인트가 HTTPS 사용검사되지 않음
- 파괴적인 도구에 라벨 표시됨검사되지 않음
- 모델에 대한 숨겨진 지침 없음통과
- 패키지 및 해당 종속성에 중요한 보안 권고 없음통과
README
GitHub에서 보기What's new
2.0.0 — A tool, driven by your agent. (First published as 1.6.0; renumbered because it is a breaking change.) (full changelog)
- Breaking: the built-in agent loop and all model settings are gone. Claude, Codex, Cursor, a local Llama or any MCP client drives the six tools (
accessibility,system,window,browser,computer,batch); thetasktool and the--provider/--model/--api-keyflags are removed. - Breaking: the policy gate is gone, so
open_file, closing a window, clipboard writes and clicking "Send" by name work. What still protects you: Stop, the pause card, no stray input into AI assistant windows, a short per-OS list of session-ending keys, and your MCP host's own approval prompts. If your host auto-approves tool calls, nothing asks before an irreversible action: see SECURITY.md (an opt-in confirm mode is coming in 2.1). - A thin outline around the window the agent is working in replaces the full-screen glow; the pill sits on that window's monitor.
- Results you can trust: a value is "set" only when the field shows it;
copy_all_textnever presses keys outside a browser;open_appnever names another app's window; actions by name never press a hidden element. - Keys follow the OS: the agent is told its modifier (
cmdorctrl) up front, and shortcuts are offered only for this OS. - Second monitors above or left of the main one, Retina zoom areas, and slow-starting apps now work as expected.
1.5.27 — Results you can trust, and logs that say what happened.
- Ticks, dropdowns and expanders read and change the same way on Windows, macOS and Linux;
togglereports the state it actually reached, and actions by name aim at the control, not its label. - A stop always answers "stopped by the user" first, and also blocks hover, clipboard writes, page script and tab switches.
- Typing goes only to the window the agent was working in, dialogs like Save As included.
- Linux reads deep web pages, and a search that runs out of time or elements says so instead of "not found".
- Logs in
~/.clawdcursor/logs/show every call's timing and outcome, slow internal steps, why the server exited, and who stopped it, with no typed text, error text or window titles.
1.5.26 — The agent asks you on screen, and the pill says what it is doing.
- Security (Windows, high): text the agent typed or copied could run as PowerShell code (GHSA-9ppf-38v8-h97v). Fixed; upgrade.
- The pause prompt is a card: the agent can offer 2-6 choices to click, or a text box, and gets your exact answer back. A choice is never a safety approval.
- A redesigned pill: what the agent is doing right now ("Typing in Edge"), how long it has been running, and a Stop button.
- A field below the fold is scrolled into view instead of refused (verified on Windows; the macOS and Linux calls are in but not yet run on a real desktop).
- macOS: when a password field holds secure input,
window focusnames the app holding it instead of failing vaguely.
1.5.25 — A stop really stops, and keys only go to the window the agent was working in.
- Double-click the pill and every clawdcursor process stops: a running wait ends at once, the rest of a batch never runs, and it stays stopped (even across restarts) until you click Resume.
- Every key, paste and click first checks that the agent's window is still in front. If you switched apps, nothing is sent and the agent is told so.
- Safety gate: key combos are blocked however their modifiers are spelled,
shortcuts_executeand compound actions are gated like the tools they run, and every element an invoke may act on is checked. --no-visionnow really keeps screenshots from every model; the daemon is loopback-only (use an SSH tunnel for remote), andserver.portin config works.- macOS: the Screen Recording check works without the helper built, and
doctorasks the same question screenshots do.
1.5.24 — The AI never touches your pill, and failures stop looking like results.
- While a pause waits for you, nothing reads, clicks or types. The pill and pause prompt are never a target, even from another clawdcursor process.
- On macOS, a screenshot without Screen Recording permission is an error, not a wallpaper image, and open popovers are readable.
- The browser clawdcursor drives comes to the front when it navigates.
- A failed OCR read reports its error instead of "No text detected", and OCR
confidenceis only shown when the engine really has one.
1.5.23 — Windows: finding an element by name no longer matches stray letters. A lone "C" on another page could match "User code 0", and "Don't Save" could match Save. A partial match now has to be a real part of the name you asked for.
1.5.22 — Your turn is as short as possible. When an agent needs you (a sign-in, a 2FA code), it asks for that one step and does the rest itself. With done_when (what shows on screen once you're done), the prompt turns from blue back to amber the moment you finish, with no Resume click.
1.5.21
set_valuenever guesses a field. With no field name or role it used to type into the window's largest text box; it now asks you to name the field.- macOS: no screen-recording permission no longer breaks everything. Only screenshots and OCR fail, and they say how to fix it; windows, keys and accessibility keep working.
- Cleaner page text on Windows (no placeholder characters), and a security update to a dependency.
1.5.20
- The on-screen pill no longer blocks the agent's own clicks. It sits where browsers keep their tabs and address bar; it now steps aside for a moment before every click, drag or scroll, then comes back. Windows, macOS and Linux.
- Reading a page no longer highlights it on Linux and macOS either.
copy_all_textreads the document through accessibility, as on Windows: exact text, nothing selected, your clipboard untouched. - Your clipboard is put back after typing (
smart_typetoo), and only once the paste has landed, so a slow app can't paste what you had copied. - macOS:
window focusfocuses the process you asked for, a locked screen says so, sidebar clicks are verified, and a bare modifier key (shift) works.
1.5.19 — Typing never pastes your own clipboard. clawdcursor types by pasting; on Windows a clipboard write could fail silently and the paste then sent whatever you had copied. Every paste now checks the clipboard first and types key by key when it doesn't hold the text.
1.5.18
- "Something else" on the pause prompt. When the agent pauses for you, the prompt has a third button between Resume and End task: type an answer instead of doing the step — "use the test account instead", a code, a change of plan — and press Enter. The agent continues from what you wrote. Windows, macOS and Linux; over MCP
pause_for_userreturns{status:"replied", reply}. - Reading a page no longer highlights it (Windows).
copy_all_textreads the document the way a screen reader does: exact text, no Ctrl+A, nothing selected, your clipboard untouched. - MCP tool annotations on every tool, and
window/systemnow describe every action, its side effects and what it returns.
1.5.17
- macOS: text outside ASCII is typed and pasted as itself — "café 日本語" no longer comes out as "cafa aaa" or "caf√©".
copy_all_textnever selects an editor or a list (Ctrl+A there selected a whole document or every email).
1.5.16
- It pauses for you at a roadblock, then carries on. A sign-in, a 2FA code, a CAPTCHA, an upgrade page: instead of giving up, the agent pauses. The glow around the screen turns blue and an on-screen prompt says what it needs ("Sign in to Outlook, then click Resume"). Click Resume and it continues where it left off, or End task to stop. Your own agent can do the same over MCP with
system({ action: "pause_for_user", message }); from a terminal,clawdcursor resume(or--end). - The glow and banner on every OS. macOS and Linux now get the same screen-edge glow and banner as Windows: amber while the agent works, blue while it waits for you. Double-click the banner to stop everything.
- Blind mode on macOS caught up with Windows. A native accessibility helper lists windows in 0.3 s (was 11 s) and reads a window's tree in 0.06–0.2 s. Typing is refused while a system dialog owns the keyboard, and the agent is told when another app's dialog took its keys.
- Sharper blind mode on Windows and Linux. Classic controls no longer read as blank panes, the open dialog is what gets read, no 10 s stall on buttons that open a modal, the right field is found by name, and focusing a window no longer un-maximizes it.
- Upgrade walls are a roadblock. The agent is told to stop at
Upgrade,SubscribeorStart trialand ask you on the pause prompt. - Know what you're running.
clawdcursor versionslists every installed copy (global npm, Claude Desktop extension, Claude Code plugin), and the installers say what they updated.
Everything else is in the changelog.
What it is
Clawd Cursor is a local MCP server that gives any tool-calling agent — Claude Code, Cursor, Windsurf, OpenClaw, the Claude Agent SDK, or your own loop — safe control of the real desktop. It clicks, types, reads the screen, opens apps, and drives any GUI the way a human would: native apps, the browser, even a canvas.
Most "let an agent use the computer" tools take a screenshot and feed it to a vision model — slow, expensive, and brittle. Clawd Cursor compiles the screen into one UI map: it fuses the accessibility tree and OCR into a confidence-scored set of elements, each tagged with a stable el_NN id, and acts on elements by id — not pixel coordinates. Coordinates appear only in the last-resort screenshot/vision tier (live pixels off the current frame), for canvas-only apps or tasks that genuinely need spatial reasoning. The result is cheaper, faster, private, and — uniquely — it checks that each action actually did what it claimed.
If a human can do it on a screen, your agent can too. No API, no integration, no problem — only the right sequence of reads, clicks, keys, and waits. Use it as the last-mile fallback: native API exists? Use it. CLI? Use it. Clawd Cursor is for the click, the legacy app, the GUI with no public surface.
Why it's different
The desktop-agent space is crowded. The closest install-and-go peers are Windows-MCP and Terminator (desktop MCP servers); browser-only tools (browser-use, Playwright MCP) are adjacent; and OmniParser / UI-TARS are vision-centric parsing approaches you'd build an agent around, not products you install. Here's the honest comparison across those approaches — what Clawd Cursor does that the popular options don't:
| Clawd Cursor | browser-use | Playwright MCP | OmniParser / UI-TARS | computer-use | |
|---|---|---|---|---|---|
| Any desktop app, not just the web | ✅ | web only | web only | ✅ | ✅ |
| Cross-OS (Windows + macOS + Linux) | ✅ | — | — | varies | sandbox |
| Perception without a vision model | ✅ compiled a11y + OCR map | DOM | a11y tree | ❌ vision-centric | ❌ vision |
| Verifies its own actions (deviation) | ✅ | — | — | — | — |
| A Stop that holds (every process, across restarts) | ✅ | — | — | — | — |
| Any model / vendor | ✅ | ✅ | not an agent | model-specific | Claude only |
| MCP-native (one config, any host) | ✅ | library | test framework | — | tool-use API |
| Local-only, no cloud required | ✅ | ✅ | ✅ | needs a model | screens → cloud |
Three things here are genuinely rare:
- Cheapest-tier-first perception, fully local. Accessibility tree (free) → OCR (cheap) → screenshot (expensive — the only tier that puts pixels in the model's context; "screenshot" and "vision" are the same step). The agent climbs only when it must, so token cost tracks task difficulty — and with a local model, nothing leaves the machine. Vision-centric agents (OmniParser, UI-TARS) need a screenshot in the model for every observation.
- It verifies. Pass
expecton a consequential action and Clawd Cursor re-checks the live screen (with a short settle window for async UIs) and reports a DEVIATION instead of a hollow "success." A completed task can't be marked done on evidence that was already true before it acted. - You stay in control, and the roles are clear. clawdcursor is a tool layer that runs with your own permissions: what to do is decided by your agent and by your MCP host's permission prompts, which clawdcursor feeds with accurate read-only / destructive tool annotations. What clawdcursor guarantees itself is small and holds for every call: a Stop that every process honours, a pause only you can answer, no stray input into the AI assistant's own window, and a short list of key combos (lock, log out) it never sends. See SECURITY.md.
Plus: whenever an agent is driving, an on-screen banner that says what it is doing ("Typing in Edge") and a thin amber outline around the window it is working in (nothing over the window, nothing on the rest of the screen), on Windows, macOS and Linux — press Stop on the banner to stop. A stop holds until you give control back: the agent's next request comes up as a prompt, and only your Resume lets it act again (or run clawdcursor resume). Keys and clicks also go only to the window the agent was working in: if you switch apps while it waits, its next input is refused instead of landing in your window. When the agent needs you (a sign-in, a 2FA code), the outline turns blue and the banner says what to do, with Resume and End task. A human at the machine always knows, and always has a kill switch.
Install
clawdcursor is an MCP server published to npm — install it into any MCP-capable agent (Claude Code, Claude Desktop, Cursor, Windsurf, Zed, OpenAI Codex, or your own loop) the same way you install any other MCP server.
Claude Desktop — one-click extension (no terminal)
Download the extension for your OS and open it — Claude Desktop installs it:
| Windows | macOS | Linux |
|---|---|---|
| clawdcursor-win32.mcpb | clawdcursor-darwin.mcpb | clawdcursor-linux.mcpb |
- Double-click the file (or Settings → Extensions → Advanced settings → Install Extension… and pick it).
- Turn on Allow clawdcursor to control this computer — that is the one-time desktop-control consent; until it is on, every tool returns a consent prompt.
- macOS: the first time macOS asks, allow Claude under Privacy & Security → Accessibility and Screen Recording.
No Node.js, npm or JSON editing needed — Claude Desktop runs the bundled server itself. Everything below is for other agents, or for Claude Desktop without the extension.
1 — Install the engine + grant consent (once)
npm i -g clawdcursor
clawdcursor consent --accept # one-time desktop-control consent (required)
clawdcursor grant # macOS only — approve Accessibility + Screen Recording
Hosts that support MCP elicitation (e.g. Claude Code, VS Code) also let you skip
consent --accept: the first tool call asks "Allow clawdcursor to control this computer" right in the app.
Zero-install also works — swap
clawdcursorfornpx -y clawdcursorin any snippet below and npx fetches it on demand. A global install is recommended anyway: it's pinnable and inspectable on disk (safer for a tool with full desktop control than auto-fetchinglatestevery run), and it's the path on which the macOS native helper builds at install time. Requires Node.js 20+.
Per-OS prerequisites. Windows installs clean —
sharpand@nut-tree-fork/nut-jsship prebuilt binaries, so no C++/Python build tools are needed. macOS needs Xcode Command Line Tools (xcode-select --install) for screenshots / vision; core accessibility-driven control still works without them. Linux needs a few system packages npm can't install:tesseract-ocr(OCR),python3-gi+gir1.2-atspi-2.0(accessibility tree), and — on Wayland —ydotool(synthetic input).
2 — Add it to your agent (pick your host)
One click (Cursor, VS Code):
Every host runs the same command — clawdcursor mcp --compact, or
npx -y clawdcursor mcp --compact with no global install. Only the config shape
differs between hosts:
Claude Code
claude mcp add clawdcursor -s user -- clawdcursor mcp --compact
Cursor · Windsurf · Claude Desktop · Cline · Gemini CLI — the common mcpServers shape:
{
"mcpServers": {
"clawdcursor": { "command": "clawdcursor", "args": ["mcp", "--compact"] }
}
}
Claude Desktop reads %APPDATA%\Claude\claude_desktop_config.json (Windows) or
~/Library/Application Support/Claude/claude_desktop_config.json (macOS).
Gemini CLI reads ~/.gemini/settings.json.
VS Code (Copilot) — .vscode/mcp.json uses servers, not mcpServers:
{
"servers": {
"clawdcursor": { "command": "clawdcursor", "args": ["mcp", "--compact"] }
}
}
Zed — context_servers in settings.json:
{
"context_servers": {
"clawdcursor": { "command": "clawdcursor", "args": ["mcp", "--compact"] }
}
}
OpenAI Codex — ~/.codex/config.toml. The first launch can take a few seconds,
so raise the 10 s default startup timeout:
[mcp_servers.clawdcursor]
command = "clawdcursor"
args = ["mcp", "--compact"]
startup_timeout_sec = 30
opencode — opencode.json uses mcp, and command is an array:
{
"mcp": {
"clawdcursor": { "type": "local", "command": ["clawdcursor", "mcp", "--compact"] }
}
}
Any other MCP host (Goose, Continue, …) — register a stdio server with command
clawdcursor and args mcp --compact.
Windows: some hosts start the server without a shell, and then can't find
clawdcursorornpx(both are.cmdshims). If the server won't start, wrap it incmd /c— e.g."command": "cmd", "args": ["/c", "clawdcursor", "mcp", "--compact"], orclaude mcp add clawdcursor -s user -- cmd /c clawdcursor mcp --compact.
That's the whole setup. Ask your agent: "open Outlook and reply to the latest email from Sarah."
Or: one-command plugin (Claude Code)
Skip the manual config — this repo ships a plugin that registers the tools and
bundles the usage skill in one step. It resolves the package's bin (never a
hard-coded dist/ path), so an upgrade can't break it:
claude plugin marketplace add AmrDab/clawdcursor
claude plugin install clawdcursor@clawdcursor
# Windows (PowerShell)
powershell -c "irm https://clawdcursor.com/install.ps1 | iex"
Clawd Cursor MCP 질문
Clawd Cursor MCP를 Claude Code에 추가하려면 어떻게 하나요?
터미널에서 claude mcp add clawdcursor -- npx -y clawdcursor를 실행합니다. 모든 프로젝트에서 사용할 수 있게 하려면 --scope user를 추가합니다.
Clawd Cursor MCP를 Cursor에 추가하려면 어떻게 하나요?
~/.cursor/mcp.json를 열고 mcpServers 아래에 서버를 추가합니다. 위 설치 섹션에 표시된 Cursor 구성을 사용합니다.
Clawd Cursor는 스크린샷으로 화면을 읽나요, 아니면 접근성 기능을 사용하나요?
접근성 구조와 OCR을 우선적으로 사용하며, 스크린샷과 비전 기능은 최후의 수단으로 사용합니다.
에이전트의 데스크톱 제어를 어떻게 중단하나요?
화면에 표시된 pill을 더블 클릭하면 모든 clawdcursor 프로세스가 중단됩니다. Resume를 클릭하기 전까지 중단 상태가 유지됩니다.
Clawd Cursor에 API 키나 모델 설정이 필요한가요?
아니요. 내장된 에이전트 루프와 모델 설정이 제거되었으므로, Claude 또는 Cursor와 같은 사용자의 MCP client가 도구를 구동합니다.
에이전트가 로그인이나 2FA 프롬프트를 만나면 어떻게 되나요?
에이전트가 일시 중지되고 필요한 사항을 설명하는 온스크린 프롬프트가 표시됩니다. Resume를 클릭하면 중단된 지점부터 다시 시작합니다.