# gui-agent MCP server

> Enables AI agents to perform desktop automation on Linux and Windows using accessibility trees, OCR, and direct hardware input simulation.

- Repository: https://github.com/leandre755/gui_agent
- Page: https://hostedmcp.org/servers/leandre755/gui-agent
- Transport: —
- Stars: 5
- License: MIT

## Environment variables

| Name | Required | Secret | Description |
|---|---|---|---|
| `ALLOW_CONFIG_EDIT` | no | no |  |

## Tools (47)

- `language`
- `code`
- `timeout`
- `command`
- `background`
- `sudo_password`
- `window_id`
- `include_screenshot`
- `element_id`
- `action`
- `value`
- `text`
- `confidence`
- `show_grid`
- `grid_step`
- `output_path`
- `button`
- `double`
- `from_x`
- `from_y`
- `to_x`
- `to_y`
- `duration`
- `direction`
- `amount`
- `key`
- `modifiers`
- `monitor_index`
- `crop_box`
- `apply_grid`
- `grid_interval`
- `format`
- `quality`
- `include_base64`
- `normalized`
- `clicks`
- `delay`
- `width`
- `height`
- `template_path`
- `threshold`
- `url`
- `selector`
- `viewport_width`
- `viewport_height`
- `timeout_ms`
- `fps`

## FAQ

### What operating systems does gui-agent support?

The server is designed for desktop computer use on both Linux and Windows.

### How does the server handle UI interaction without relying solely on screenshots?

It uses a layered approach called Progressive Escalation, accessing the OS accessibility tree (AT-SPI2 / D-Bus) directly in RAM for sub-50ms actuation.

### Can the server record the screen for auditing purposes?

Yes, it includes tools to start and stop background low-overhead screen video recording using FFmpeg.