Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
d4b3c4d
feat(supervision): GitHub events watcher (comments/CI/reviews) with C…
e-jung Jun 22, 2026
0e7f945
fix(ghwatch): per-PR emission, atomic seen writes, usage + test coverage
e-jung Jun 22, 2026
7ec2db4
fix(ghwatch): respect merge filter, carry-forward CI sig, doc/test fixes
e-jung Jun 22, 2026
8023854
fix(ghwatch): empty-filters, contributor derivation, reviews filter, …
e-jung Jun 22, 2026
8c3862e
fix(ghwatch): per_page=100 on list endpoints; accurate comment label
e-jung Jun 22, 2026
367796a
fix(ghwatch): empty-contributor flood guard; isolate atomic_write temps
e-jung Jun 22, 2026
9becefc
fix(ghwatch): --limit 1000 on PR search; document 100-item count cap
e-jung Jun 22, 2026
51b9cc4
fix(ghwatch): treat CLOSED as non-terminal; exclude .tmp from status …
e-jung Jun 22, 2026
595de59
fix(ghwatch): carry forward state across transient pr_state failures
e-jung Jun 22, 2026
8750066
fix(ghwatch): bound CLOSED re-probes with a close->settle window
e-jung Jun 22, 2026
620a627
no-mistakes(document): docs(ghwatch): document FM_GH_CLOSE_REPROBE_SE…
e-jung Jun 22, 2026
8b53a0a
style(ghwatch): refactor A && B || C to if-statements for shellcheck …
e-jung Jun 22, 2026
b06c9d8
perf(ghwatch): poll PRs concurrently to fit the 30s check-script budget
e-jung Jun 23, 2026
8fd01ba
fix(ghwatch): debounce CI filter to one event per overall-state trans…
e-jung Jun 24, 2026
36de048
fix(ghwatch): no-deploy flood + correct fork-PR CI roll-up for daemon…
e-jung Jun 24, 2026
42fd8fa
fix(ghwatch): skip a PR on transient GitHub API errors instead of par…
e-jung Jun 24, 2026
6bb103f
no-mistakes(document): docs(ghwatch): document FM_GH_CONCURRENCY and …
e-jung Jun 29, 2026
8403b61
feat(supervision): done-crewmate check plugin + fm-plugin.sh lifecycle
e-jung Jun 30, 2026
b40a7b1
no-mistakes(review): Fix done-crewmate check portable root resolution
e-jung Jun 30, 2026
00cfc87
no-mistakes(test): fall back to legacy merge-tree for content-landed …
e-jung Jun 30, 2026
7524ca9
no-mistakes(document): docs: cover done-crewmate plugin, fm-plugin.sh…
e-jung Jun 30, 2026
1d5afc5
no-mistakes(review): Exclude kind=secondmate from done-crewmate offen…
e-jung Jun 30, 2026
7067b8e
no-mistakes(document): docs already cover done-crewmate plugin and gi…
e-jung Jun 30, 2026
b9193af
no-mistakes(lint): Lint done-crewmate plugin and fm-plugin lifecycle …
e-jung Jun 30, 2026
4eb4d7c
fix(ci): silence SC2015 in bootstrap plugin-sync guard; mark fm-plugi…
e-jung Jul 1, 2026
6a440c0
no-mistakes(review): Stamp closed_at in build_seen to bound closed-PR…
e-jung Jul 1, 2026
b7f08dc
no-mistakes(document): Document fm-plugin test and fix FM_GH_CONTRIBU…
e-jung Jul 1, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 11 additions & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,7 @@ README.md public overview and development notes
.agents/skills/ shared skills, committed
.claude/skills symlink to .agents/skills for claude compatibility
bin/ helper scripts, committed; read each script's header before first use
check-plugins/ durable watcher check plugins, committed; bin/fm-plugin.sh symlinks each into state/*.check.sh so the watcher picks them up
.env optional X-mode pairing token; LOCAL, gitignored; presence-gates section 14
config/crew-harness crewmate harness override; LOCAL, gitignored; absent or "default" = same as firstmate. Inherited: the primary pushes this into every secondmate home's config/ (section 4), so a secondmate's own crewmates use the primary's value
config/crew-dispatch.json optional crewmate dispatch profiles; LOCAL, gitignored; firstmate-maintained but human-editable natural-language rules that choose a per-task harness/model/effort profile (section 4). Inherited by secondmate homes
Expand All @@ -90,7 +91,7 @@ state/ volatile runtime signals; gitignored
<id>.turn-ended touched by turn-end hooks
<id>.grok-turnend-token firstmate-owned grok hook registry token for the task; removed by teardown
<id>.meta written by fm-spawn: window=, worktree=, project=, harness=, model=, effort=, kind=, mode=, yolo=, tasktmp=; kind=secondmate also records home= and projects= (fm-pr-check, including through fm-pr-merge, appends pr= and GitHub's pr_head= when available; fm-x-link appends x_request= and x_request_ts= for an X-mention-originated task, section 14)
<id>.check.sh optional slow poll you write per task (e.g. merged-PR check)
<id>.check.sh optional slow poll you write per task (e.g. merged-PR check); fleet-wide plugin checks also appear here as symlinks into bin/check-plugins/ (bin/fm-plugin.sh manages them)
x-watch.check.sh generated X-mode relay poll shim; present only when opted in (section 14)
x-inbox/ generated X-mode pending mention payloads; fmx-respond drains it (section 14)
x-outbox/ generated X-mode dry-run reply and dismiss previews; inspect it when FMX_DRY_RUN is set (section 14)
Expand All @@ -102,6 +103,8 @@ state/ volatile runtime signals; gitignored
.watch-triage.log watcher's absorbed-wake debug log (size-capped); never relied on, safe to delete
.last-watcher-beat watcher liveness beacon, touched every poll (including while absorbing benign wakes); fm-guard.sh reads it
.subsuper-* .supervise-daemon.* sub-supervisor internals; never touch
.github-watch-config fm-github-watch.sh filter/contributor config (key=value); never touch unless driving that tool
.github-watch-seen/ fm-github-watch.sh per-PR seen state (high-water marks); owned by that script
.no-mistakes/ local validation state and evidence; gitignored
```

Expand Down Expand Up @@ -632,6 +635,13 @@ A secondmate may be sitting on its own watcher with no visible pane changes, so
`fm-watch.sh` therefore skips stale-pane wakes for windows whose meta records `kind=secondmate`.
This exception is narrow: ordinary crewmates still trip stale detection when their pane stops changing without a busy signature.

**Terminal-status crewmates must be progressed immediately.**
A crewmate that reports `done`, `failed`, or `blocked` and is then left idle in its tmux window is unfinished supervision work, not a quiet fleet.
The signal layer fires exactly once, on the status write; if you drop the thread after that, nothing re-nudges you - the stale-pane detector flags the idle pane, but that alarm is indistinguishable from a stuck crewmate until you re-read the status, so a busy supervisor dismisses it as noise.
The `done-crewmate` check plugin is the deterministic, recurring backstop: it scans every `state/*.meta` for a crewmate whose current status is terminal and whose window is still alive in tmux, and prints one wake line per check interval listing every offender until each is progressed (validated, merged, or torn down).
It is installed by `bin/fm-plugin.sh add done-crewmate state/done-crewmate.check.sh` and lives durably under `bin/check-plugins/` (symlinked into `state/` so the watcher's existing `*.check.sh` glob picks it up, no watcher changes required; `fm-plugin.sh sync`, called by bootstrap, restores the symlinks after a fresh clone).
Treat its wake with the same priority as a `signal:`: read the named task's status, then advance or tear it down.

**Watcher liveness is guarded, not just disciplined.**
Arming the watcher is the last action of every wake-handling turn - but the protocol no longer relies on remembering that.
While running, `fm-watch.sh` touches `state/.last-watcher-beat` every poll cycle.
Expand Down
2 changes: 2 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -85,6 +85,8 @@ tests/fm-secondmate-safety.test.sh # secondmate home safety, idle charter
tests/fm-teardown.test.sh # fm-teardown.sh landed-work safety and reminder checks: fork-remote allow, squash/content landings, dirty and unlanded refusals, PR-head metadata, no-pr= branch discovery, tasks-axi/manual backlog reminder, --force override
tests/fm-pr-merge.test.sh # fm-pr-merge.sh records pr= and available pr_head= before merging, propagates real merge failures, and forwards extra gh-axi pr merge flags
tests/fm-crew-state.test.sh # fm-crew-state.sh current-state reconciliation: run-step authority including closed panes, stale needs-decision/blocked superseded by a resumed run, genuine-parked, cross-branch attribution, pane/status-log fallback, scout skip, torn-down/missing-meta graceful
tests/fm-github-watch.test.sh # fm-github-watch.sh events, filters, rolled-up CI flips, merge/close transitions, contributor resolution, seen-state losslessness, and concurrency via a fake gh fixture
tests/fm-plugin.test.sh # fm-plugin.sh add/remove/list/sync lifecycle, invalid-name and not-found guards, and the done-crewmate.check.sh terminal-status offender scan via a fake tmux fixture
[ "$(readlink CLAUDE.md)" = "AGENTS.md" ]
[ "$(readlink .claude/skills)" = "../.agents/skills" ]
tmp=$(mktemp -d) && printf 'done: smoke\n' > "$tmp/smoke.status" && FM_STATE_OVERRIDE="$tmp" FM_SIGNAL_GRACE=1 FM_POLL=1 FM_HEARTBEAT=999999 bin/fm-watch-arm.sh # watcher re-arm smoke test (prints arm status, then an actionable signal)
Expand Down
98 changes: 98 additions & 0 deletions bin/check-plugins/done-crewmate.check.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,98 @@
#!/usr/bin/env bash
# Watcher check plugin: detect crewmates that reported a terminal status
# (done/failed/blocked) but whose tmux window is still alive - i.e. finished
# work firstmate has not yet progressed (validated / PR'd / merged) or torn down.
#
# Why this exists: a status write fires exactly once, on change. If firstmate
# gets the `done` signal, starts acting, then drops the thread, nothing re-nudges
# it - the stale-pane detector fires on the idle pane, but that alarm is
# indistinguishable from a stuck crewmate until firstmate re-reads the status, so
# a busy firstmate dismisses it as noise. This check is the deterministic,
# recurring backstop: every FM_CHECK_INTERVAL it re-asserts "done work is still
# sitting there" until the crewmate is torn down.
#
# Watcher check contract (same as bin/fm-pr-check.sh's per-task checks):
# print exactly one line -> wake firstmate (reason wrapped as
# `check: <this-script>: <line>`)
# print nothing -> fleet healthy; keep sleeping
# Runs via the watcher's state/*.check.sh glob (state/done-crewmate.check.sh is
# a symlink to this canonical copy under bin/check-plugins/; see bin/fm-plugin.sh).
# Fast by design: only tmux list-windows + small file reads, no network.
set -u

# Resolve FM_ROOT independent of cwd and of symlink indirection
# (state/<name>.check.sh -> bin/check-plugins/<name>.check.sh). Prefer an explicit
# override, then cwd (the watcher runs from FM_ROOT, so state/ and bin/ are
# siblings of $PWD), then walk up from this script's resolved real path.
fm_root() {
[ -n "${FM_ROOT_OVERRIDE:-}" ] && { printf '%s\n' "$FM_ROOT_OVERRIDE"; return; }
if [ -d state ] && [ -d bin ]; then printf '%s\n' "$PWD"; return; fi
local src="${BASH_SOURCE[0]}" real d root
# readlink -f is GNU-only; plain readlink (one symlink level) is portable on
# BSD/GNU. fm-plugin.sh points state/<name>.check.sh at an absolute
# bin/check-plugins/<name>.check.sh; resolve a relative target against the link.
if real="$(readlink "$src" 2>/dev/null)" && [ -n "$real" ]; then
case "$real" in
/*) src="$real" ;;
*) src="$(cd -P "$(dirname "$src")" && pwd)/$real" ;;
esac
fi
d="$(cd -P "$(dirname "$src")" 2>/dev/null && pwd)" || { printf '%s\n' "$PWD"; return; }
for root in "$d/../.." "$d/.."; do
[ -d "$root/bin" ] && [ -d "$root/state" ] && { (cd -P "$root" && pwd); return; }
done
printf '%s\n' "$PWD"
}
FM_ROOT="$(fm_root)"
STATE="$FM_ROOT/state"

[ -d "$STATE" ] || exit 0

# A terminal status means the crewmate's work is complete (or halted pending
# firstmate) and it should not still be occupying a tmux window. needs-decision
# is intentionally excluded: it escalates immediately through the signal layer on
# write, so it never needs this recurring backstop.
is_terminal() {
case "$1" in
done:*|failed:*|blocked:*) return 0 ;;
*) return 1 ;;
esac
}

# Live crewmate windows, one '<session>:<window>' per line (matches the watcher's
# own enumeration in bin/fm-watch.sh). Empty if tmux is absent or no fm windows
# exist - which means nothing can be idle-done, so we stay silent.
WINDOWS="$(tmux list-windows -a -F '#{session_name}:#{window_name}' 2>/dev/null | grep ':fm-' || true)"
[ -n "$WINDOWS" ] || exit 0

offenders=""
for meta in "$STATE"/*.meta; do
[ -e "$meta" ] || continue
kind="$(grep -m1 '^kind=' "$meta" 2>/dev/null | cut -d= -f2-)"
[ -n "$kind" ] || kind=ship
[ "$kind" = secondmate ] && continue
id="$(basename "$meta" .meta)"
status_file="$STATE/$id.status"
[ -f "$status_file" ] || continue # no status reported yet -> still working

# Current state = the last non-empty status line (crewmates append; a later
# `working:` means it resumed, which is not idle-done). Tolerate a missing
# trailing newline via the `|| [ -n "$line" ]` guard.
last=""
while IFS= read -r line || [ -n "$line" ]; do
[ -n "$line" ] && last="$line"
done < "$status_file"
is_terminal "$last" || continue

# Cross-reference tmux: is this crewmate's window still alive? The meta's
# window= target is authoritative (recorded by fm-spawn as <session>:<window>).
win="$(grep -m1 '^window=' "$meta" 2>/dev/null | cut -d= -f2-)"
[ -n "$win" ] || continue
case "$WINDOWS" in
*"$win"*) offenders="${offenders:+$offenders }$id" ;;
esac
done

[ -n "$offenders" ] || exit 0
# One line listing every offender so a single wake carries the whole picture.
printf 'done crewmate %s still alive in tmux - progress or tear down\n' "$offenders"
5 changes: 5 additions & 0 deletions bin/fm-bootstrap.sh
Original file line number Diff line number Diff line change
Expand Up @@ -412,4 +412,9 @@ fi
secondmate_sync
x_mode_setup
fleet_sync
# Re-arm durable watcher check plugins (state/*.check.sh symlinks into the
# tracked canonical copies under bin/check-plugins/). state/ is gitignored, so a
# fresh clone has no symlinks until this runs. Best-effort and silent on success.
# shellcheck disable=SC2015 # best-effort: a missing exe or sync failure must never abort bootstrap
[ -x "$FM_ROOT/bin/fm-plugin.sh" ] && "$FM_ROOT/bin/fm-plugin.sh" sync || true
exit 0
Loading