Навык post-patch-validation
✓ ОфициальныеscriptsПроверяет патчи безопасности, сравнивая базовый и исправленный код с помощью воспроизводимых эксплойтов, вариантов первопричин и регрессионных тестов.
- Звёзды на GitHub
- 8 136
- Файлы
- 5
- Последний коммит
- 11 нояб. 2026 г.
- Лицензия
- CC-BY-SA-4.0
- Разрешённые инструменты
- 7
Что делает навык post-patch-validation?
Проверяет патчи безопасности, сравнивая базовый и исправленный код с помощью воспроизводимых эксплойтов, вариантов первопричин и регрессионных тестов.
Сводка сгенерирована из SKILL.md навыка.
О skill post-patch-validation
Инженеры по безопасности и разработчики используют этот SKILL, чтобы убедиться, что патч безопасности действительно устраняет уязвимость, не внося новых багов и не нарушая существующую функциональность. Инструмент работает путем сравнения уязвимой базовой версии с исправленной версией кода с использованием структурированного плана валидации. Процесс включает запуск исходных эксплойтов, тестирование вариантов первопричин, проверку на регрессии и контроль того, что корректное поведение приложения остается неизменным.
Агенту следует загрузить этот SKILL после появления патча или коммита с исправлением, но до того, как они будут объединены с основной веткой или помечены как исправленные. Работа инструмента опирается на оркестрационный скрипт на Python для создания каркаса, валидации и выполнения планов доказательств. Инструмент формирует подробные результаты, включая JSON-сводку, отчет в формате markdown и архивированные временные директории для каждой проверки. Он предназначен для локального выполнения в репозиториях и требует явного разрешения на запуск кода или тестовых наборов. В отличие от простого анализа diff, этот SKILL предоставляет конкретные доказательства через runtime assertions и слепое выполнение (side-blind execution) для предотвращения ложноположительных результатов.
Что агент может делать с ним
- 1Валидация патчей безопасности, созданных ИИПроверьте, что патч, созданный ИИ-агентом, правильно устраняет уязвимость, прежде чем человек приступит к ревью кода.
- 2Обнаружение вариантов первопричинУбедитесь, что исправление охватывает все возможные способы триггера конкретной уязвимости, а не только один путь эксплойта.
- 3Предотвращение регрессий безопасностиПроверьте, не вносит ли исправление безопасности непреднамеренную новую уязвимость в соседние пути выполнения кода.
- 4Подтверждение сохранения поведенияУбедитесь, что исправление безопасности не изменяет ожидаемый результат работы функций приложения, которые не затрагивались правками.
Написано на основе skill SKILL.md. Каждый сценарий использования проверен по цитате из него.
Как установить навык post-patch-validation
Каждая команда копирует папку навыка в каталог навыков клиента. Перезапустите клиент или начните новую сессию, чтобы загрузить его.Установка навыка post-patch-validation в Плагин
/plugin marketplace add trailofbits/skills /plugin install post-patch-validation@trailofbits
Установка навыка post-patch-validation в Claude Code
~/.claude/skills/post-patch-validation/SKILL.md
$ npx degit trailofbits/skills/plugins/post-patch-validation/skills/post-patch-validation ~/.claude/skills/post-patch-validationУстановка навыка post-patch-validation в Claude Code (project)
.claude/skills/post-patch-validation/SKILL.md
$ npx degit trailofbits/skills/plugins/post-patch-validation/skills/post-patch-validation .claude/skills/post-patch-validationУстановка навыка post-patch-validation в Codex
~/.agents/skills/post-patch-validation/SKILL.md
$ npx degit trailofbits/skills/plugins/post-patch-validation/skills/post-patch-validation ~/.agents/skills/post-patch-validationУстановка навыка post-patch-validation в Gemini CLI
~/.gemini/skills/post-patch-validation/SKILL.md
$ npx degit trailofbits/skills/plugins/post-patch-validation/skills/post-patch-validation ~/.gemini/skills/post-patch-validationУстановка навыка post-patch-validation в Claude.ai
npx degit trailofbits/skills/plugins/post-patch-validation/skills/post-patch-validation post-patch-validation && zip -r post-patch-validation.zip post-patch-validation # claude.ai → Customize → Skills → + → Create skill → Upload a skill → post-patch-validation.zip
Автоматические проверки
Запущено 15 нояб. 2026 г.Полный отчёт →Ещё не проверен. Навыку нужно 80 баллов, ни одной серьёзной находки и достаточно данных для проверки. Как работают проверки · Что исправить, чтобы получить больше баллов
- Связанный и доступный исходный репозиторийtrailofbits/skills/plugins/post-patch-validation/skills/post-patch-validation is public and reachablegithub.com/trailofbits/skills/tree/HEAD/plugins/post-patch-v ↗Пройдено
- Open source лицензия (OSI)CC-BY-SA-4.0 is not OSI-approvedspdx.org/licenses/CC-BY-SA-4.0.html ↗Проблема
- Коммит или релиз за последний годLast commit 2026-11-11github.com/trailofbits/skills/commits ↗Пройдено
- В скриптах нет загрузки с запуском и кражи учётных данных1 script read; no download-and-run, remote eval, base64 execution or credential uploadПройдено
- В SKILL.md нет скрытых инструкцийNo override phrases, safeguard switches, credential requests or hidden charactersgithub.com/trailofbits/skills/blob/HEAD/plugins/post-patch-v ↗Пройдено
- Заранее разрешённые инструменты ограниченыBash lets the agent run any shell command without askinggithub.com/trailofbits/skills/blob/HEAD/plugins/post-patch-v ↗Проблема
- SKILL.md соответствует формату Agent SkillsValid name and description; says when to use it; 262 linesgithub.com/trailofbits/skills/blob/HEAD/plugins/post-patch-v ↗Пройдено
SKILL.md
Открыть на GitHubPost-Patch Validation
Validate a security patch against the reported bug and the surrounding code it affects. Give the patch author reproducible failures to fix and identify what still needs testing. Apply the same checks to human and agent patches. The diff, author, upstream implementation, and original proof of concept alone cannot establish correctness.
When to Use
- A security fix, remediation commit, patch file, or pull request already exists.
- An AI-generated patch needs validation before human review or merge.
- A fix may cover one exploit path while missing variants of the same root cause.
- A security fix may alter legitimate behavior or introduce a new vulnerability.
- A patch author needs concrete failures and coverage gaps before another revision.
When NOT to Use
- No patch exists yet; use vulnerability discovery or fix implementation first.
- The task is to review an audit finding against a report without executing patch evidence.
- The task is only to convert a finding into a permanent project test.
- The target is remote or production. This skill executes local code and tests only.
- The user has not authorized execution of the repository's code or test suite.
Quick Start
-
Pin the vulnerable base and patched input. Prefer immutable commits. For uncommitted work, create a binary patch file first; do not validate in the user's working tree.
-
Scaffold a pinned plan:
uv run {baseDir}/scripts/post_patch_validation.py scaffold \ --repo . \ --base-ref <vulnerable-ref> \ --patched-ref <patched-ref> \ --finding-id <stable-id> \ --finding-summary "<root cause and impact>" \ --evidence-level runtime \ --output post-patch-validation/plan.jsonUse
--patch-file <path>instead of--patched-reffor a patch artifact. Choose the highest honest evidence level:sourcefor source/patch invariants only,buildwhen target code is compiled or analyzed but the reported behavior is not executed, orruntimewhen the checks execute the reported behavior and its safety assertions. -
Inspect the finding, diff, callers, sibling paths, cleanup/error paths, and existing tests. Populate
checksin the generated plan. Runprint-schemafor the structural schema:uv run {baseDir}/scripts/post_patch_validation.py print-schema -
Run
validate-planfor the complete validation, including coverage, command restrictions, and pinned inputs, before executing code:uv run {baseDir}/scripts/post_patch_validation.py validate-plan \ --plan post-patch-validation/plan.json -
Execute the evidence plan:
uv run {baseDir}/scripts/post_patch_validation.py run \ --plan post-patch-validation/plan.json \ --output post-patch-validation/results -
Report
result.json,report.md, the evidence level, and the completeassessment. Return each finding to the patch author with its check ID, assertion, and saved logs. Identify each validation gap separately, including gaps that coexist with supported findings. After the author revises the patch, pin the new inputs and save a fresh validation run. Preserve the prior evidence. Passing supplied checks still requires human review before acceptance.
Evidence Contract
The runner rejects incomplete plans. Supply at least one check of every kind:
| Kind | Required observation |
|---|---|
control |
Benign harness succeeds on both base and patch |
exploit |
Original safety assertion fails on base and succeeds on patch |
variant |
A distinct root-cause variant fails on base and succeeds on patch |
behavior |
Unaffected behavior succeeds with byte-identical selected output |
regression |
Targeted non-security regression check succeeds on both revisions |
security |
Adjacent/new-vulnerability check succeeds on base and patch |
suite |
Existing project suite, sanitizer, or deterministic fuzz campaign succeeds on patch |
Commands are argv arrays, never shell strings. Put complex setup in a checked-in or plan artifact
script and invoke it with {plan_dir}. The runner fixes locale/timezone/hash-seed inputs, executes
checks in lexical ID order, records raw stdout/stderr, and never edits the original worktree.
Each check's timeout_seconds defaults to 300 and accepts integers from 1 through 3600.
Exceeding the timeout leaves a validation gap. A timeout alone does not establish a regression.
Every plan also contains a sorted submodules array ([] when none). Scaffolding infers affected
Gitlinks from the changed-file inventory. The runner initializes those pinned commits from the
source repository's existing Git module objects, never from .gitmodules network URLs; initialize
or fetch them in the source repository before validation.
Exploit and variant checks must prove they ran
A nonzero exit does not mean the vulnerability reproduced. An import error, a failed build, a
missing dependency, and a failed safety assertion all exit nonzero and are indistinguishable to the
runner. Every exploit and variant check must print and flush PPV_REACHED immediately
before it evaluates its assertion, on both revisions:
"argv": ["python3", "-c", "import app; value = app.render('<'); print('PPV_REACHED', flush=True); assert value == '<'"]
The token is also in the environment as PPV_REACHED_MARKER. It must land on stdout, as a line
of its own. Stderr is not scanned, because a Python SyntaxError traceback echoes the offending
source and would otherwise satisfy the check for a harness that executed nothing. A run without it
leaves a marker_missing gap. Independent findings from other checks remain in the result.
Flush explicitly: a harness whose payload segfaults or calls _exit loses buffered output and forfeits its own evidence.
These checks also run side-blind. {side} is not expanded for them, PPV_SIDE is absent from
their environment, the checkout directory is randomly named, and the plan validator rejects any
exploit or variant check whose argv or env mentions either. An assertion that can see which
revision it is on can assert on that instead of on the code, which is the cheapest possible way
to fake a reproduction followed by a fix.
Environment
Checks run under a fixed minimal environment: PATH, HOME, and a handful of temp/user keys,
plus LANG/LC_ALL=C, TZ=UTC, PYTHONHASHSEED=0, NO_COLOR, TERM=dumb. Everything else in
the caller's environment is dropped. Toolchains that need more get it explicitly:
uv run {baseDir}/scripts/post_patch_validation.py run \
--plan post-patch-validation/plan.json \
--output post-patch-validation/results \
--allow-env JAVA_HOME --allow-env CARGO_HOME
Forwarded names and values are recorded in result.json. A requested variable that is unset is an
error, not an empty string. Two classes are refused outright: names that read as credentials
(*SECRET*, *TOKEN*, *API_KEY*, …), because the value would be written into the result; and
names that change what executes (LD_PRELOAD, BASH_ENV, NODE_OPTIONS, GIT_SSH_COMMAND, …),
because those variables can change which code executes.
The runner's fixed variables and every PPV_* name are also reserved and cannot be forwarded.
Placeholders expanded in argv and per-check env values: {checkout} (the revision under test),
{plan_dir} (an isolated copy of the plan artifacts for that one invocation), {scratch} (a
fresh opaque directory for that one check invocation), and
{side} (base or patched, and not available to exploit/variant checks). The same values
arrive as PPV_CHECKOUT, PPV_PLAN_DIR, PPV_SCRATCH, PPV_SIDE, and PPV_CASE_ID. Write only
under {scratch}; the evidence directory path is not passed to checks. Base and patched invocations
do not share runner-managed scratch, plan, or worktree roots. After each invocation exits, its
scratch tree is archived under the deterministic results/scratch/<check-id-and-side> path, its
private plan copy is discarded, and every readable argv element that resolves to a file is hashed
in argv_files. Files inside the isolated plan or checkout roots are additionally retained under
results/helpers/<sha256> up to 16 MiB; the record explains why any other file was not archived.
Use a dedicated directory for plan.json: its sibling files and directories are copied into each
invocation's {plan_dir}. Keep helper code under that directory's checks/ directory or checked
into the target repository so its bytes are reviewable. The machine plan containing commit pins,
the current output directory, and detected prior result trees are excluded; symlinks are rejected.
The clean snapshot remains only in runner memory, and
exploit/variant sides execute in random order while evidence filenames remain deterministic.
Stdout/stderr use anonymous or randomly named capture descriptors and are copied to the named
evidence files only after the child exits, so fd inspection cannot disclose the side label.
This isolation is not a host sandbox: checks run with the caller's privileges and a malicious helper could use arbitrary external state or deliberately infer the revision from source or Git metadata. Inspect the content-addressed helper artifacts, and use an OS/container sandbox when the check code itself is untrusted.
Active validation worktrees are Git-locked with random owner tokens backed by kernel file locks, so
another concurrent validator cannot prune them and PID reuse cannot impersonate an owner. If the
runner is forcibly killed, the next run unlocks stale validator-owned registrations. For manual
recovery, inspect git worktree list, then use git worktree unlock <path> and
git worktree remove --force <path> (or git worktree prune after the path is gone).
Read evidence-model.md when designing coverage, selecting variants, or interpreting findings and validation gaps. Do not read it for routine CLI execution.
Coverage Rules
- Derive variants from the root cause, not cosmetic mutations of the original payload.
- Enumerate sibling call sites, alternate callbacks/outputs, error paths, teardown, ownership, serialization, and boundary values touched by the fix.
- Make each exploit or variant test assert the safe behavior. It must fail on the vulnerable base; a test that passes on both revisions proves nothing about remediation. Read the base-side stderr and confirm the failure is the assertion you wrote, not a harness that never got there.
- Keep exploit and variant assertions limited to the security invariant. Test liveness, exact
error types/messages, timing, and compatibility separately as
behaviororregressionchecks; otherwise an unrelated contract change can masquerade as proof that the vulnerability remains. - Keep the
controlharness benign and make it exercise the changed component. It establishes that the harness works on both revisions. Failed controls leave gaps and prevent attributing other failures to the patch. The raw observations remain available for review. - Use
behavioronly for behavior that should remain unchanged. Exact output comparison is deliberate; move unstable values behind a deterministic test harness instead of normalizing them away in prose. - Make
securitychecks pass on the vulnerable base before treating a patched failure as newly introduced. A failed baseline leaves attribution unresolved. - Do not edit the patch during validation. Return failures to the patch author and start a new, freshly pinned run.
Reading the result
result.json schema 2.0 contains an assessment with status, findings, gaps, and
human_review_required. Status is complete when all required checks produced usable evidence,
even if some checks found failures. Status is incomplete when any gap remains. Findings name the
check ID, check kind, and failed expectation. Gaps name the check ID and missing evidence, with a
null check ID for run-level problems such as cleanup failures.
Suite checks run on the patched revision first. A completed failure triggers the same check on baseline. If baseline passes, report the failure after the patch. If both fail, preserve both logs and report that attribution is unresolved. Do not assume matching exit codes mean the same failure.
The runner exits 0 for complete checks with no findings, 1 for complete checks with findings, 10 for incomplete validation, and 64 for invalid inputs. Read the artifact even after exit 10: it can contain supported findings alongside gaps. Source or build evidence cannot establish runtime behavior. State the evidence level alongside any passing result.
Claude Dynamic Workflow
Claude Code exposes the bundled workflow as /post-patch-validation:validate-patch.
To pass structured inputs through the Workflow tool, use the name without a leading slash:
Workflow({
name: 'post-patch-validation:validate-patch',
args: {
finding: '<finding text or local path>',
baseRef: '<vulnerable-ref>',
patchRef: '<patched-ref>',
workdir: 'post-patch-validation',
},
})
Use patchFile instead of patchRef when appropriate. The workflow uses fixed coverage
lenses to propose checks and a fixed executor to run this skill. Agents may author test
artifacts. The Python runner records findings and gaps, and reviewers report coverage or evidence
concerns separately. NEEDS_REPAIR returns supported failures even when other checks left gaps.
BLOCKED means evidence is incomplete without a supported failure. Passing checks advance to
READY_FOR_HUMAN_REVIEW only after both evidence reviews approve, otherwise REVIEW_REQUIRED.
The workflow cannot ask questions after launch, so pass every input up front.
Rationalizations to Reject
| Rationalization | Required response |
|---|---|
| "The original PoC no longer works" | Test at least one independent root-cause variant |
| "The exploit failed on base, so it reproduced" | Confirm the marker and that the failure is the assertion, not a broken harness |
| "The full suite passes" | Prove baseline reproduction and targeted behavior explicitly |
| "This matches the upstream/canonical patch" | Treat provenance as context, not evidence |
| "The diff is tiny" | Exercise callers, failure paths, and teardown affected by the change |
| "All supplied checks passed" | Preserve artifacts and require human review |
| "A flaky rerun passed" | Keep the first pinned result; fix nondeterminism before retrying |
| "There is no obvious variant" | Inspect sibling sites and boundaries; otherwise report the missing coverage |
Файлы в этом навыке
5Вопросы о навыке post-patch-validation
Как установить навык post-patch-validation в Claude Code?
Выполните npx degit trailofbits/skills/plugins/post-patch-validation/skills/post-patch-validation ~/.claude/skills/post-patch-validation. Claude Code загружает навыки из ~/.claude/skills (все проекты) или .claude/skills (один проект) и использует навык, когда его описание соответствует задаче.
Какие клиенты могут использовать навык post-patch-validation?
Любой клиент, который читает формат Agent Skills: Claude Code, Codex, Gemini CLI и claude.ai (загрузить как ZIP). Скрипты в навыке выполняются только там, где клиент разрешает выполнение кода.
Когда следует использовать этот SKILL?
Используйте его, когда существует исправление безопасности, коммит с устранением уязвимости или патч, сгенерированный ИИ, который необходимо проверить на соответствие исходному багу, потенциальным вариантам или новым регрессиям.
Какие уровни доказательств доступны?
Этот SKILL поддерживает три уровня: 'source' для инвариантов, 'build' для скомпилированного/проанализированного кода без выполнения и 'runtime' для выполнения описанного поведения и проверок безопасности (assertions).
Как убедиться, что проверка эксплойта действительно достигла уязвимости?
Каждая проверка эксплойта и варианта должна выводить и сбрасывать токен 'PPV_REACHED' в stdout непосредственно перед оценкой утверждения, чтобы доказать, что путь выполнения кода был пройден.
Безопасно ли запускать этот SKILL в продакшн-средах?
Нет, этот SKILL предназначен только для выполнения локального кода и тестов; его не следует использовать на удаленных или продуктовых целях.