gpuq MCP server
Commit Nov 7, 2026Lets an AI client manage and coordinate cooperative GPU job queues across multiple machines for shared inference and training tasks.
Also known as unconventio/gpuq
- Stars
- 0
- Tools
- —
- Downloads, 30 days
- —
- Latest version
- —
- Last commit
- Nov 7, 2026
- License
- MIT
What is gpuq MCP?
gpuq lets an AI client manage and coordinate cooperative GPU job queues across multiple machines for shared inference and training tasks.
Summary generated from the project's README
About gpuq MCP server
Managing shared GPU resources across multiple machines is what gpuq enables for developers running various AI agent sessions and inference servers. It coordinates cooperative job queues where running tasks can yield their hardware to higher priority jobs, pausing their work and resuming later when they reach the front of the line. This system works across a fleet of Linux hosts using SSH for communication, with each host maintaining its own queue via JSON files.
Users can interact with the system through a CLI, a terminal UI, or an MCP server. The server supports NVIDIA and AMD hardware, including integrated GPUs, and allows for a declared inventory where you can reserve memory or tag specific cards. It handles resident services like Ollama, vLLM, and llama.cpp by providing tools to unload models to make room for larger tasks. Because it is advisory, it relies on jobs to heartbeat and follow the protocol rather than forcibly killing processes.
What you can do with it
- 1Find available GPU capacityAn AI agent can determine which machine has enough VRAM and the shortest wait time to start a new task.
- 2Coordinate cooperative task executionA developer can submit a job that automatically yields its GPU when a higher priority task arrives.
- 3Manage resident inference serversYou can clear memory on a specific card by unloading existing models to accommodate a large training job.
Written from the project's README. Every use case is checked against a quote from it.
Configuration
CUDA_VISIBLE_DEVICES
| Name | Description | |
|---|---|---|
| CUDA_VISIBLE_DEVICES | — |
Automated checks
Ran Nov 15, 2026Full report →Not verified yet. A server needs a score of 80, no serious finding, and enough data for us to check. How checks work · What to fix for more points
- Source repository linked and reachableunconventio/gpuq is public and reachablegithub.com/unconventio/gpuq ↗Pass
- Package names this repository as its sourceNot checked
- Build provenance (Sigstore attestation)Not checked
- Open source license (OSI)MITspdx.org/licenses/MIT.html ↗Pass
- Commit or release in the last yearLast commit or release 2026-11-07github.com/unconventio/gpuq ↗Pass
- OpenSSF ScorecardNot checked
- Package name not close to a popular packageNot checked
- No critical advisory in the package or its dependenciesNot checked
- No hidden instructions to the model0 tool descriptions and the README read; no override phrases, credential paths or hidden charactersPass
- Destructive tools are labelledNot checked
- Remote endpoint answers the MCP handshakeNot checked
README
View on GitHubgpuq
A GPU queue for a few personal machines shared by one person, many AI agent sessions and a couple of always-on
inference servers. Every card on every machine has its own numbered line. Jobs take turns and give the card up
cooperatively: a running job is asked to yield through its heartbeat, stops itself, and resumes when it is first
in line again. gpuq where tells you which card on which machine can take a job, and how soon.
- Fleet-wide placement, no broker: each host keeps its own queue in JSON files; other hosts are reached over SSH.
- NVIDIA and AMD, discrete and integrated:
nvidia-smifor NVIDIA, sysfs for AMD (no amd-smi), an iGPU sized by its shared memory. - A declared inventory: cap, reserve and tag each card's memory, and declare the resident servers that live on it
(presets for Ollama, vLLM, llama.cpp
llama-server, llama-swap).gpuq freeunloads their models for a big job. - Agents and humans share one line: a CLI with exit codes and
--json, an MCP server, and a terminal UI where you reorder the line by hand. - Short jobs first, from real numbers: without
--est, a job's estimate is the median of its past runs.
Stdlib-only Python (3.9+), Linux. gpuq is advisory: it does not kill your processes or meter memory use. It keeps an honest, visible line and flags GPU use that did not go through it.
Install
pip install . # or: pipx install . (provides `gpuq` and `gpuq-mcp`)
# no install: PYTHONPATH=. python3 -m gpuq
gpuq init # write a starter config from the cards (and servers) found here
State lives in ~/.local/state/gpuq (override: $GPUQ_STATE). Run gpuq sweep every minute from a timer
(systemd user timer or cron); it purges dead holders.
An app-menu launcher
examples/gpuq.desktop puts "GPU Queue" in your app menu, like htop's entry, so the live view opens in its own
window and closes without a "running program" prompt. Copy it to ~/.local/share/applications/. It uses your
default terminal at its default size.
A bigger window (130x40 fits the full layout with bars and the model column) takes one more step. With
Terminal=true the desktop picks the terminal, and a desktop file has no way to ask it for a size, so there is no
setting that works everywhere. Instead set Terminal=false and start a terminal yourself on the Exec= line, with
that terminal's own size option:
| terminal | Exec= |
|---|---|
| GNOME Terminal | gnome-terminal --geometry=130x40 -- gpuq |
| Konsole | konsole -p TerminalColumns=130 -p TerminalRows=40 -e gpuq |
| kitty | kitty -o initial_window_width=130c -o initial_window_height=40c gpuq |
| xterm | xterm -geometry 130x40 -e gpuq |
Some terminals (Ptyxis, for one) can't set a window size per launch, only a default size for every window;
use another terminal for the launcher, or change that default. Use the full path to gpuq if the menu's PATH
lacks ~/.local/bin.
Tab completion for bash (commands, flags, card names, hosts, queue ids):
mkdir -p ~/.local/share/bash-completion/completions
gpuq completion bash > ~/.local/share/bash-completion/completions/gpuq
Quickstart
gpuq where --vram 16 # which card, on which machine, how soon
ID=$(gpuq join --owner nightly --job "eval sweep" --vram 16 --est 90 --max 180)
gpuq run $ID -- python eval.py # waits its turn, runs, heartbeats, releases
gpuq # live view (like htop): k/j move jobs, Tab switches GPU, q quits
gpuq show # print the line once (what `gpuq` prints when piped)
gpuq inventory # what every card offers
The agent protocol
| call | meaning |
|---|---|
gpuq join ... |
get in line, prints an id. --vram GB / --tag T pick a card that fits (shortest wait, then the smallest card); --gpu names one. --est MIN (default: the median of this job's past runs); --max MIN (alias --limit) is a hard limit, after which the sweep purges it. --priority chill|normal|important|ablaze|preempt|N (below) --spec FILE (or - for stdin) reads all of it as one JSON object |
gpuq start ID |
exit 0 = you hold the card. Exit 3 = not your turn yet (or a yield was acknowledged): retry in a minute. Prints CUDA_VISIBLE_DEVICES=GPU-<uuid> (or ROCR_VISIBLE_DEVICES=…) on multi-card hosts |
gpuq beat ID [--progress 41/80] |
heartbeat, about every minute (15 min silent = purged). Exit 4 = you were purged: stop. Exit 5 = yield: stop your GPU work, go back to polling start |
gpuq done ID [--exit CODE] |
finished, leave the line. Give the exit code: a non-zero one tells the job's contact (below). --no-notice = a deliberate stop, no notice |
gpuq run ID [--free] [--on-stop SH] [--progress-file F] [--log PATH|-] [--every SEC] [--grace SEC] -- CMD... |
does all of the above, and saves CMD's output to ~/.local/state/gpuq/logs/ID.log while still showing it (CMD runs on a pseudo-terminal, so progress bars keep working; --log - saves nothing). It passes the exit code to done and samples the card's memory and load with each beat. On a yield it sends CMD's process group SIGTERM (SIGKILL after --grace, default 30 s; it beats every --every, default 60 s) and reruns CMD from the start when you are #1 again, so CMD must be safe to rerun (checkpoint yourself). --free frees the card's residents once granted and restarts what it stopped at the end |
Add --json to join, start, beat, done, env, edit, mv, yield, rm, free, evac or restore for
one JSON object instead: {"ok", "code", "state", "message", …} where state is go / wait / gone for start
and ok / yield / purged for beat. start --json carries the pin as "env": {"CUDA_VISIBLE_DEVICES": "GPU-…"};
the resident commands put what they printed in lines. On show, inventory, where and inbox, --json prints
the data itself, no envelope (show --json is one host's: add --host H for another).
Exit codes everywhere: 0 ok · 1 failed · 2 bad input · 3 not your turn (or paused) · 4 no such entry (purged,
removed, done) · 5 yield. gpuq run exits with CMD's code.
Flags: --host H works before or after the command. Flags are never abbreviated (--hos is an error, not
--hosts). List flags (--models, --tag, --contact, --hosts) take commas and may repeat. --card = --gpu.
edit takes an id, never a position.
Hand-rolled holder loop:
while :; do
until gpuq start $ID; do [ $? = 3 ] || exit; sleep 60; done # 4 = purged
launch the GPU work
while work is alive; do
gpuq beat $ID --progress "$P"; rc=$?
[ $rc = 4 ] && exit
[ $rc = 5 ] && { stop the work; break; } # yield
sleep 60
done
work finished on its own -> gpuq done $ID; exit
done
When a job dies: notices
A job names a contact with gpuq join --contact NAME[,NAME] (default: its owner); gpuq edit ID --contact … --desktop --log … adds or changes them later, even while it runs. The contact gets a notice when the job
- fails:
done --exitnon-zero, or killed by a signal (gpuq runreports both); - dies early: fails, or leaves with no exit code, within 10 minutes of getting the card (unless its own estimate was that short);
- is purged: no heartbeat for 15 minutes, or past its
--max; - stalls: its
--progresshasn't changed for 30 minutes, or its card sat at 0% load that long. Told once, until it moves again.
A notice holds the job, card, host, exit code, run time and log path. It is a line in
~/.local/state/gpuq/inbox/<contact>.jsonl on the host where the job ran; readers pull it:
gpuq inbox --contact NAME [--ack] # unread notices (--ack marks them read); --hosts a,b for other machines
gpuq inbox --contact NAME --follow # keep printing new ones, for a watcher process
The MCP server returns them on the session's next tool call, and the TUI shows them as red ⚠ lines (a dismisses
them from the screen only; each reader keeps its own "read" mark). join --desktop (alias --notify) also pops a desktop notice
(notify-send) for that job, and an ntfy URL in the config sends those to your phone too. Whether a person is told
is the job owner's choice; "notify": {"desktop": "always"} makes it yours.
History keeps how each job ended: exit, run time, the card's peak memory and average load while it ran (whole
card, residents included). Failed runs don't count toward estimates.
Priority: any integer, higher goes first; the names are defaults: chill 5 (☾), normal 10, important 20 (★),
ablaze 30 (⚑), and preempt = one above the running job's. A joining job goes just behind the last job of at least
its priority, and outranking the running job asks it to yield (it resumes after). --chill, --important, --ablaze
and --preempt are shortcuts; gpuq edit ID --priority … re-places a job.
Rules of the line: a yield is asked by moving the running job down (gpuq mv 1 3, TUI j, or gpuq yield ID).
The yielder keeps the card until its next start acknowledges. A waiting #1 that does not start within 15 min is
skipped to the back (twice: removed). A job of at most 6 h goes ahead of jobs with 10x more time left (decided at
join/edit only, never otherwise) unless they are important (priority 20+) or outrank it. A pause (gpuq pause) lets nobody start.
MCP
gpuq mcp (or gpuq-mcp) is an MCP server over stdio, stdlib only. Point your agent's MCP config at it:
{"mcpServers": {"gpuq": {"command": "gpuq-mcp"}}}
Tools: gpuq_inventory, gpuq_where, gpuq_show, gpuq_join, gpuq_start, gpuq_beat, gpuq_done,
gpuq_edit, gpuq_yield, gpuq_rm, gpuq_env, gpuq_inbox; each takes an optional host. gpuq_join and
gpuq_edit take contact, desktop and log; priority is a number or a name; gpuq_done takes exit_code
and no_notice. gpuq_rm, gpuq_yield and gpuq_env take ids only, never positions. They return the same states as --json, plus a next hint for the agent.
Agents forget to heartbeat, so any tool call also beats every job that MCP session has started, and a pending yield
for one of them comes back on any result as a notices entry, as do notices for the contacts of jobs that
session joined. A session that makes no calls at all still goes
stale after 15 minutes. For long work, gpuq run ID -- CMD in a shell beats by itself: use MCP to ask and hold,
gpuq run to run.
Inventory and placement
Cards are detected; the config says what they offer. Usable memory = the card's memory (capped by vram_gb) minus
reserve_gb minus what its residents hold. gpuq inventory shows it per card, with tags, residents, the line and
the wait. gpuq where --vram GB [--tag T] [--est MIN] ranks every card on every host: cards that fit first, then the
shortest wait (with the short-job jump applied when --est is given), then this host, then the smallest card. It
exits 1 when nothing fits.
Tags: cuda or rocm by vendor, igpu for integrated GPUs, plus any you add per card.
Residents are declared per card, either as a process name (the old form) or as an object:
"residents": {
"3090": [{"type": "ollama", "vram_gb": 8}],
"3060": ["whisper-server", {"type": "vllm", "unit": "vllm-small.service"}]
}
| field | meaning |
|---|---|
type |
a preset: ollama, vllm, llama-server, llama-swap (fills in the fields below) |
name |
label (default: the type or process name) |
match |
process name as gpuq sees it, to recognize it on a card |
unit |
its systemd user unit (evac restarts it pinned elsewhere; free may stop it) |
vram_gb |
memory it normally holds; vram_frac = a share of the card (the vLLM preset assumes 0.9) |
free |
how gpuq free releases it: ollama / llama-swap (unload over HTTP at url), stop (stop the unit; restore starts it), or a shell command |
gpuq free [CARD] unloads or stops the card's residents (--dry-run previews). Ollama and llama-swap load models
again on the next request, so nothing needs restoring; stopped units come back with gpuq restore. A resident with no
type or free is reported and left alone. gpuq evac [CARD] instead moves the residents to another card with a
systemd drop-in and a restart; gpuq restore undoes either.
gpuq only plans with what you declare. It does not read the servers' own configs, guess which model they will load, or decide when a resident may be evicted.
Config
Optional JSON at ~/.config/gpuq/config.json ($GPUQ_CONFIG); gpuq init writes a starter one, and
examples/config.json shows every key.
| key | effect |
|---|---|
default |
card (by model number, e.g. 3090) used when a job names none; else the card with most memory |
owners |
{owner: card} default card per owner |
cards |
{card: {"reserve_gb", "vram_gb", "tags"}} per card |
residents |
{card: [process name or resident object]}, above |
units |
{process name: systemd user unit} for old-form residents (default <name>.service) |
hosts |
comma list for the multi-host views and where (default: this host) |
remote_command |
gpuq on other hosts over non-interactive ssh (default ~/.local/bin/gpuq) |
rules |
{"jump_ratio": 10, "jump_max_hours": 6, "priorities": {"chill": 5, "normal": 10, "important": 20, "ablaze": 30}, "early_death_min": 10, "stall_min": 30} |
notify |
{"desktop": "never"|"always", "ntfy": URL, "command": SHELL}: how notices leave the inbox. command gets {id} {kind} {owner} {job} {exit} {log} {contact} {text}, shell-quoted |
compat_lock |
a lock file for older tools; written while a job runs, removed after |
Cards and hosts
Each card has its own line, named by model number. NVIDIA jobs get their card by UUID in CUDA_VISIBLE_DEVICES
(nvidia-smi and CUDA number cards differently). Every host keeps its own queue; gpuq --host H cmd, show, watch,
inventory, where and tui --hosts a,b reach others over SSH (key auth, gpuq installed there). where suggests
a host; nothing places jobs across hosts for you.
GPU use outside the queue is shown as a yellow line naming each process: its command, pid, user, age, how it was
started (a service, a container, tmux or screen, a terminal, VS Code, ssh, a Claude Code session) and its folder,
read from /proc. Desktop programs (OBS, remote desktop, browsers, the compositor; config desktop_apps adds more) are
listed as desktop: and never count. A job that ends early without an exit code is a yellow "check", not a red alarm.
AMD and integrated GPUs
AMD cards are found in /sys/class/drm (driver amdgpu) and named from the PCI id database (Radeon 880M -> 880M);
no vendor tool is needed. An integrated GPU has a tiny dedicated carve-out and uses system RAM, so its size is shown as
shared memory (880M (24 GB shared)), and a discrete card is preferred as the host default. Cap what an iGPU may lend
with cards.<name>.vram_gb (gpuq init proposes half). Jobs on an AMD card get ROCR_VISIBLE_DEVICES (the ROCm
device index, or GPU-<unique id> when the card has one); gpuq run sets it, and gpuq start / gpuq env ID print
it. Only ROCR_VISIBLE_DEVICES is set, not HIP_VISIBLE_DEVICES, because HIP applies its own filter on top of ROCm's
and the two would compound. "processes on this card" lists processes holding at least 1 GB, read from the kernel's DRM fdinfo
(needs kernel 5.19+ for the memory keys; you only see processes you are allowed to inspect), plus gpu_busy_percent.
Not covered: Intel GPUs, Vulkan-only devices without amdgpu, and a job that spans several cards. A host that has
neither tool still works as a plain numbered line without pinning or usage warnings. Mixed NVIDIA + AMD hosts get one
line per card of either kind.
Limits
Memory accounting is declarative: --vram is checked against what the card offers after reserves and residents,
not against live usage (live use is shown, and sampled per job as whole-card figures). Jobs on a card run one at a time. No checkpointing, nothing enforces the line against processes that ignore it (they show up as "unqueued GPU use").
Linux only. Purging a holder removes its place; it does not kill its process. There is no daemon: purges and stalls
are found by gpuq sweep, so install its timer (examples/) on every host. Logs and notices are never deleted by
gpuq.
Tests
python3 -m unittest discover -s tests
They run against temporary state directories and fake cards; nothing touches a real queue, GPU, systemd unit or inference server.
License
MIT, see LICENSE.
gpuq MCP questions
Does gpuq support both NVIDIA and AMD GPUs?
Yes, it supports NVIDIA via nvidia-smi and AMD via sysfs, including integrated GPUs.
How can an AI agent use this server?
Agents can connect via an MCP server over stdio to join queues, check inventory, and manage job status.
What happens if a job stops sending heartbeats?
If no heartbeat is received for 15 minutes, the job is purged from the queue.
Can I see which GPU is available and how long the wait is?
Yes, the 'where' command tells you which card on which machine can take a job and how soon.