-
Notifications
You must be signed in to change notification settings - Fork 0
feat: add skill-guards plugin to enforce fresh execution on every skill invocation #196
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
JacobPEvans
wants to merge
2
commits into
main
Choose a base branch
from
feat/skill-execution-guards
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -37,6 +37,7 @@ | |
| "ENDJSON", | ||
| "RLENGTH", | ||
| "unrecognised", | ||
| "tokenisation" | ||
| "tokenisation", | ||
| "reinvocation" | ||
| ] | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,9 @@ | ||
| { | ||
| "name": "skill-guards", | ||
| "version": "2.3.2", | ||
| "description": "Ensures fresh execution of skills on every invocation via UserPromptSubmit hook", | ||
| "author": {"name": "JacobPEvans"}, | ||
| "homepage": "https://github.com/JacobPEvans/claude-code-plugins", | ||
| "keywords": ["skills", "execution", "guards", "hooks"], | ||
| "license": "Apache-2.0" | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,38 @@ | ||
| # skill-guards | ||
|
|
||
| Ensures fresh execution of skills on every invocation via a UserPromptSubmit hook. | ||
|
|
||
| ## Problem | ||
|
|
||
| When a skill like `/ship` is called a second time in a session, Claude may shortcut | ||
| by assuming previous results are still valid instead of re-running all commands against | ||
| live state. | ||
|
|
||
| ## Solution | ||
|
|
||
| A UserPromptSubmit hook detects `/skill-name` patterns in user prompts and injects a | ||
| systemMessage reminding Claude to execute every step from scratch using current live state. | ||
|
|
||
| ## Installation | ||
|
|
||
| ```bash | ||
| claude plugins add jacobpevans-cc-plugins/skill-guards | ||
| ``` | ||
|
|
||
| ## Usage | ||
|
|
||
| No manual invocation required. The hook activates automatically on every user prompt. | ||
|
|
||
| ## Hook Behavior | ||
|
|
||
| - **Fires on**: Every user prompt submission | ||
| - **Detects**: `/lowercase-with-hyphens` skill invocation patterns | ||
| - **Excludes**: Common filesystem paths (`/usr`, `/tmp`, `/etc`, `/nix`, etc.) | ||
| - **Output**: `{"systemMessage": "FRESH EXECUTION: /skill — ..."}` or `{}` | ||
| - **Exit code**: Always 0 (never blocks prompts) | ||
|
|
||
| ## Part of a Three-Layer System | ||
|
|
||
| 1. **Global rule** (`skill-execution-integrity`) — establishes the mental model | ||
| 2. **This hook** — tactical trigger at moment of invocation | ||
| 3. **Skill preambles** — skill-specific state warnings in SKILL.md files |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,16 @@ | ||
| { | ||
| "hooks": { | ||
| "UserPromptSubmit": [ | ||
| { | ||
| "matcher": "*", | ||
| "hooks": [ | ||
| { | ||
| "type": "command", | ||
| "command": "${CLAUDE_PLUGIN_ROOT}/scripts/skill-reinvocation-guard.sh", | ||
| "timeout": 3 | ||
| } | ||
| ] | ||
| } | ||
| ] | ||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,21 @@ | ||
| #!/bin/bash | ||
| # UserPromptSubmit hook — inject fresh-execution reminder on skill invocation. | ||
|
|
||
| input=$(cat) | ||
| prompt=$(jq -r '.tool_input.prompt // .tool_input.content // .prompt // .content // empty' <<< "$input" 2>/dev/null) || { printf '{}'; exit 0; } | ||
|
|
||
| [[ -z "$prompt" ]] && { printf '{}'; exit 0; } | ||
|
|
||
| if [[ "$prompt" =~ (^|[[:space:]])/([a-z][a-z0-9-]+)([^a-z0-9-]|$) ]]; then | ||
| skill_name="${BASH_REMATCH[2]}" | ||
| case "$skill_name" in | ||
| usr|tmp|etc|var|bin|dev|opt|home|nix|proc|sys|run|lib|mnt|srv|boot|root|sbin) printf '{}'; exit 0 ;; | ||
| esac | ||
| jq -n --arg skill "$skill_name" '{ | ||
| systemMessage: ("FRESH EXECUTION: /" + $skill + " — Step 1 now. New invocation, new live state. All prior outputs in this session are stale. Re-run every git, gh, and API command.") | ||
| }' || printf '{"systemMessage":"FRESH EXECUTION: /%s — Step 1 now. New invocation, new live state. All prior outputs in this session are stale. Re-run every git, gh, and API command."}' "$skill_name" | ||
| else | ||
| printf '{}' | ||
| fi | ||
|
|
||
| exit 0 |
133 changes: 133 additions & 0 deletions
133
tests/skill-guards/skill-reinvocation-guard/skill-reinvocation-guard.bats
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,133 @@ | ||
| #!/usr/bin/env bats | ||
| # Test suite for skill-guards/scripts/skill-reinvocation-guard.sh | ||
| # | ||
| # Tests the UserPromptSubmit hook's skill detection and systemMessage injection: | ||
| # - No skill in prompt → empty JSON, exit 0 | ||
| # - /ship → systemMessage with "FRESH EXECUTION" and skill name | ||
| # - /finalize-pr 42 → systemMessage with skill name | ||
| # - Regular text → empty JSON, exit 0 | ||
| # - Filesystem path /usr/bin → empty JSON (false positive exclusion) | ||
| # - Empty prompt → empty JSON, exit 0 | ||
| # | ||
| # Run with: bats tests/skill-guards/skill-reinvocation-guard/skill-reinvocation-guard.bats | ||
|
|
||
| setup() { | ||
| REPO_ROOT="$(cd "$(dirname "$BATS_TEST_FILENAME")/../../.." && pwd)" | ||
| SCRIPT="$REPO_ROOT/skill-guards/scripts/skill-reinvocation-guard.sh" | ||
|
|
||
| if [[ ! -f "$SCRIPT" ]]; then | ||
| echo "ERROR: Script not found at $SCRIPT" >&2 | ||
| return 1 | ||
| fi | ||
| } | ||
|
|
||
| # Run the hook with a given prompt string (uses jq for safe JSON construction) | ||
| run_hook_with_prompt() { | ||
| local prompt="$1" | ||
| run bash -c 'jq -n --arg p "$1" "{tool_input:{prompt:\$p}}" | /bin/bash "$2"' -- "$prompt" "$SCRIPT" | ||
| } | ||
|
|
||
| # --------------------------------------------------------------------------- | ||
| # TC1: No skill invocation → empty JSON, exit 0 | ||
| # --------------------------------------------------------------------------- | ||
|
|
||
| @test "TC1: prompt without skill outputs empty JSON" { | ||
| run_hook_with_prompt "help me fix this bug" | ||
| [ "$status" -eq 0 ] | ||
| [[ "$output" == "{}" ]] | ||
| } | ||
|
|
||
| # --------------------------------------------------------------------------- | ||
| # TC2: /ship → systemMessage with FRESH EXECUTION and skill name | ||
| # --------------------------------------------------------------------------- | ||
|
|
||
| @test "TC2: /ship triggers fresh execution message" { | ||
| run_hook_with_prompt "/ship" | ||
| [ "$status" -eq 0 ] | ||
| [[ "$output" =~ "systemMessage" ]] | ||
| [[ "$output" =~ "FRESH EXECUTION" ]] | ||
| [[ "$output" =~ "ship" ]] | ||
| } | ||
|
|
||
| # --------------------------------------------------------------------------- | ||
| # TC3: /finalize-pr 42 → systemMessage with skill name | ||
| # --------------------------------------------------------------------------- | ||
|
|
||
| @test "TC3: /finalize-pr with argument triggers fresh execution message" { | ||
| run_hook_with_prompt "/finalize-pr 42" | ||
| [ "$status" -eq 0 ] | ||
| [[ "$output" =~ "systemMessage" ]] | ||
| [[ "$output" =~ "FRESH EXECUTION" ]] | ||
| [[ "$output" =~ "finalize-pr" ]] | ||
| } | ||
|
|
||
| # --------------------------------------------------------------------------- | ||
| # TC4: /resolve-pr-threads all → systemMessage with skill name | ||
| # --------------------------------------------------------------------------- | ||
|
|
||
| @test "TC4: /resolve-pr-threads triggers fresh execution message" { | ||
| run_hook_with_prompt "/resolve-pr-threads all" | ||
| [ "$status" -eq 0 ] | ||
| [[ "$output" =~ "systemMessage" ]] | ||
| [[ "$output" =~ "resolve-pr-threads" ]] | ||
| } | ||
|
|
||
| # --------------------------------------------------------------------------- | ||
| # TC5: Regular text without slash → empty JSON | ||
| # --------------------------------------------------------------------------- | ||
|
|
||
| @test "TC5: text mentioning 'ship' without slash outputs empty JSON" { | ||
| run_hook_with_prompt "help me ship this feature" | ||
| [ "$status" -eq 0 ] | ||
| [[ "$output" == "{}" ]] | ||
| } | ||
|
|
||
| # --------------------------------------------------------------------------- | ||
| # TC6: Filesystem path /usr/bin → empty JSON (false positive exclusion) | ||
| # --------------------------------------------------------------------------- | ||
|
|
||
| @test "TC6: filesystem path /usr excluded as false positive" { | ||
| run_hook_with_prompt "check /usr/bin/python" | ||
| [ "$status" -eq 0 ] | ||
| [[ "$output" == "{}" ]] | ||
| } | ||
|
|
||
| # --------------------------------------------------------------------------- | ||
| # TC7: Filesystem path /tmp → empty JSON (false positive exclusion) | ||
| # --------------------------------------------------------------------------- | ||
|
|
||
| @test "TC7: filesystem path /tmp excluded as false positive" { | ||
| run_hook_with_prompt "read /tmp/output.log" | ||
| [ "$status" -eq 0 ] | ||
| [[ "$output" == "{}" ]] | ||
| } | ||
|
|
||
| # --------------------------------------------------------------------------- | ||
| # TC8: Empty/missing prompt → empty JSON, exit 0 | ||
| # --------------------------------------------------------------------------- | ||
|
|
||
| @test "TC8: empty input outputs empty JSON" { | ||
| run bash -c "echo '{}' | /bin/bash '$SCRIPT'" | ||
| [ "$status" -eq 0 ] | ||
| [[ "$output" == "{}" ]] | ||
| } | ||
|
|
||
| # --------------------------------------------------------------------------- | ||
| # TC9: Malformed JSON input → empty JSON, exit 0 (fail-open) | ||
| # --------------------------------------------------------------------------- | ||
|
|
||
| @test "TC9: malformed JSON input outputs empty JSON" { | ||
| run bash -c "echo 'not json at all' | /bin/bash '$SCRIPT'" | ||
| [ "$status" -eq 0 ] | ||
| [[ "$output" == "{}" ]] | ||
| } | ||
|
|
||
| # --------------------------------------------------------------------------- | ||
| # TC10: /nix path excluded as false positive | ||
| # --------------------------------------------------------------------------- | ||
|
|
||
| @test "TC10: filesystem path /nix excluded as false positive" { | ||
| run_hook_with_prompt "look at /nix/store/abc123" | ||
| [ "$status" -eq 0 ] | ||
| [[ "$output" == "{}" ]] | ||
| } | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.