Skip to content

hunchom/claude-code-gauntlet

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

68 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

claude-code-gauntlet

ci license python 3.11+ claude code plugin version v0.3.2

Install · Run · Pipeline · Limitations · Compared to · Architecture


A Claude Code plugin that turns vulnerability research into an auditable pipeline.

Seventeen subagents coordinate through a six-gate ladder — A → B → B.5 → C → C.5 → D — where every gate demands concrete evidence on disk:

  • ASan stack hashes that reproduce 3/3 on the fuzz host
  • Symex witnesses replayable from a deterministic seed
  • Fresh-context validator verdicts from two quarantined agents
  • A Merkle root over every factual atom in the finding

If any gate rejects, the finding dies before a human ever sees it.

From a Claude Code session

/vuln-audit <target>             # interactive
/vuln-audit-autonomous <target>  # hands-off loop
/vuln-audit-status               # state snapshot

From the shell

./bin/vr status      # one-line state
./bin/vr dashboard   # full dashboard
./bin/vr review      # Gate-D review TUI
./bin/vr watch       # tail .vr-state/events/

Pipeline

flowchart LR
    A([scope])            --> B([reachable + ASan 3/3])
    B                     --> B5([symex witness])
    B5                    --> C([dual fresh validators])
    C                     --> C5([counter-symex on release tag])
    C5                    --> D([human send])

    classDef gate fill:#0E1116,stroke:#5A6472,stroke-width:1px,color:#E6E9EF
    classDef send fill:#0E1116,stroke:#FF5C4D,stroke-width:2px,color:#FF5C4D
    class A,B,B5,C,C5 gate
    class D send
Loading

Each arrow is a hook in hooks/gate-*.sh. A hook is a shell script, not a prompt — it reads .vr-state/ and either writes a gate-pass entry or refuses the transition. No LLM self-reporting; no "I've verified it" in natural language.

Important

Either the stack hash matches 3/3 or the gate fails. Prose doesn't move candidates forward — exit codes do.

Install

1. Clone and bootstrap. Installs host tools, Python venv, CodeQL, GitNexus, and the vr package. all auto-prompts for a backend at the end.

git clone https://github.com/hunchom/claude-code-gauntlet.git
cd claude-code-gauntlet
./setup.sh all        # tools + venv + codeql + gitnexus + interactive backend prompt
./setup.sh doctor     # diagnostic — re-run anytime to verify integrity

The all run is idempotent. It will fetch ~1GB on first install (mostly CodeQL) and is safe to re-run to catch missing deps.

2. Register the plugin with Claude Code. Two options:

# For active development against your local clone:
/plugin dev-install ~/claude-code-guantlet

# For the pinned GitHub release:
/plugin marketplace add hunchom/claude-code-gauntlet
/plugin install claude-code-gauntlet@hunchom

Both paths register the 17 subagents, gate hooks, and /vuln-audit slash commands into your session.

3. Run an audit from any Claude Code session:

/vuln-audit <git-url>             # interactive
/vuln-audit-autonomous <git-url>  # hands-off loop
/vuln-audit-status                # state snapshot

The 17 skills are also importable as a real Python package outside Claude:

python -m vr.skills.provenance_merkle build-atom \
    --claim "heap-buffer-overflow at src/parser.c:412" \
    --source-type file_content \
    --source-ref "src/parser.c#L412@a1b2c3d" \
    --repo-root /opt/src/target

Note

The lib/python/vr/ package is the authoritative implementation. Every skills/vr-*/SKILL.md points subagents at python -m vr.skills.<name> <verb> first; the pseudocode block below it is kept only for human review and may drift from the installed code.

What you run

Command Purpose
/vuln-audit <target> Interactive — scope prompt, then gate ladder
/vuln-audit-autonomous <target> Hands-off loop, resumes across compacts
/vuln-audit-status Read-only campaign state
/vuln-audit-dashboard Candidate table + recent gate events
/vuln-audit-kill Sets halt flag; next tick exits
/vuln-audit-intake <url> Import lead (OSS-Fuzz / Syzbot / revert)
/vuln-audit-postmortem After-action report for a campaign

<target> can be a git URL (https://, git@, ssh://, file://) or an absolute/~/-prefixed path to an already-cloned local repo. Remote URLs are cloned into targets/<project>; local paths are symlinked in-place (read-only) and their current HEAD is pinned as the commit. Either way, the plugin indexes via GitNexus, runs the ladder, and stops at Gate D where a human approves. Nothing else reaches external inboxes.

Execution backends

Pick one via ./setup.sh backend:

Backend Fuzz + symex host Requires Host OS
remote_vm Linux, over SSH user@host + ssh-agent key any
local_container Docker or Podman Docker / Podman installed any
local_host Current machine Linux only Linux

Repository layout

claude-code-gauntlet/
|-- .claude-plugin/plugin.json     plugin manifest
|-- ARCHITECTURE.md                82KB design document (gates, V1..V9 spine)
|
|-- agents/                        17 subagents (Opus 4.6, pinned)
|-- commands/                      15 slash commands
|-- skills/vr-*/SKILL.md           17 skill definitions (authoritative CLI + reference pseudocode)
|-- lib/python/vr/                 installable Python package
|   |-- _common/                   atomic_state, errors, hashing, ssh
|   |-- skills/*.py                17 authoritative skill modules
|   `-- __main__.py                python -m vr <skill> <verb> dispatcher
|
|-- hooks/                         17 hook scripts (gate-a..gate-d + dispatch guards)
|-- mcp/                           9 MCP servers (gitnexus, nvd, klee-runner, ...)
|-- bin/                           orchestrator, dispatcher, TUI, status line
|-- schemas/intent.schema.json     JSON Schema for orchestrator intents
`-- tests/                         pytest suite (44 tests, runs <0.5s)

How a gate chain looks

Canonical Gate-B chain (reachability + reproduction):

 1  asan_poc cycle             build with ASan+UBSan, run PoC 3x, capture stack hashes
 2  fingerprint_match stab.    require 3/3 identical signatures
 3  provenance_merkle verify   live-rehash every file / CVE / commit-SHA atom
 4  write_gate write G_B pass  append to gates.json, back-patch candidates.json

Every skill speaks a three-state exit code:

Exit Meaning Gate behavior
0 success gate-pass entry written
1 infrastructure error (disk, SSH, network) retry / surface to operator
2 verification mismatch — a real negative gate fails, candidate buried

The orchestrator reads exit codes, not prose.

All five chains
python -m vr.skills.chain list      # gate_b_pass, gate_b5_pass, gate_c_pass, gate_c5_pass, gate_d_send
python -m vr.skills.chain show --name gate_d_send

Limitations

Important

This section is longer than most because the claim-to-reality gap matters. Read it before deciding whether this is the right tool for your threat model.

  • The TP-rate target is aspirational, not measured. No calibration ledger has accumulated findings yet (ARCHITECTURE.md §13 technique 8 is scaffolded but empty). Until it does, any number in the docs is an architectural goal, not a benchmark.
  • End-to-end has not been run on a real target. Every component tests in isolation (44 pytest green); no full A -> D campaign has produced a disclosed finding. Do not expect a drop-in CVE generator.
  • Symex witness replay is untested at scale. The KLEE + angr runners dispatch correctly and the replay skill compares signatures, but no Gate B.5 SAT_CONCRETE_VERIFIED has been observed end-to-end. Harness synthesis is still LLM-drafted and fragile.
  • Fresh Validator independence is partial. Two Opus agents spawned with different system_prompt_flavor share underlying weights — a shared-prior hallucination can still pass V5. V1–V7 together partially mitigate this; they do not eliminate it.
  • macOS is dev-only. Fuzzing, symex, and ASan reproduction run on Linux. On macOS pick remote_vm or local_container.
  • Not a scanner. Consumes pre-targeted repos and works the ladder. It does not scan a whole code host looking for things to find.

Compared to

Tool Approach Reproduction verified Best for
claude-code-gauntlet LLM auditors + 6 mechanical gates ASan 3/3 + symex replay targeted research
OSS-Fuzz Continuous black-box fuzzing ASan crash reports upstream projects
Semgrep + CodeQL Static pattern matching no — triage each hit CI-time guardrails
Manual audit + fuzzer Expert time per-engineer discipline deep one-off dives

Configuration

Prompted at first /plugin install, or set via env for scripted runs:

Variable Default Meaning
FUZZ_HOST (unset) SSH user@host for remote_vm
NVD_API_KEY (unset) Raises NVD rate limit; cache works without it
GITHUB_TOKEN inherits gh auth GHSA advisory lookups
DO_API_TOKEN (unset) Needed only for DigitalOcean VM provisioning
AUTONOMOUS_MODE false true auto-approves gate-advancing calls

Anti-hallucination spine (V1..V9)

Click for the short version
Check What it enforces
V1 Physical attestation — ASan stack hash reproduces 3/3
V2 Provenance Merkle — every atom rehashable against live source
V3 Oracle triangulation — Semgrep / CodeQL find the same sink
V4 Patch inversion — the draft patch makes the PoC not crash
V5 Dual Fresh Validators — two quarantined agents reach "bug"
V6 Devil's Advocate — structured disproof attempts
V7 Gate schema — JSON Schema on every gates.json entry
V8 Calibration ledger — rolling Brier score gates the campaign
V9 Batch-review cadence — humans tie-break at Gate C and Gate D

See ARCHITECTURE.md §13 for the full rationale.

Contributing

Highest-leverage contributions, ranked:

  1. Reproducible bug reports. Include the exact command, the input, and the gate that failed.
  2. Stress the gate ladder. Find a path that moves a candidate past a gate without producing the required evidence — file a security advisory.
  3. Tighten the V1..V9 spine. Anything that eliminates a whole class of false positives is worth a PR.
  4. Better symex harness templates. Gate B.5 is the most fragile link; KLEE/angr harness improvements move the entire pipeline forward.

Full guidelines in CONTRIBUTING.md.

License

AGPL-3.0-or-later. See LICENSE.

About

Claude Code plugin for vulnerability research. Six-gate ladder (A -> B -> B.5 -> C -> C.5 -> D) enforced by hooks; 17 deterministic Python skill modules; hands-off intent-consumer loop. AGPL-3.0.

Topics

Resources

License

Contributing

Security policy

Stars

Watchers

Forks

Packages

 
 
 

Contributors