fix(docs): tell coordinator to roster Fact Checker on first-time cast (#1299)#1300
Merged
tamirdresher merged 2 commits intoJun 13, 2026
Merged
Conversation
…bradygaster#1299) squad init correctly creates .squad/agents/fact-checker/ on disk (per merged PR bradygaster#1223). But when the user opens copilot --agent squad and the coordinator runs first-time casting, it OMITS Fact Checker from the team.md ## Members table while including Scribe, Ralph, and Rai. Root cause: .squad-templates/squad.agent.md had two gaps: 1. Line 56 said "team size (typically 4-5 + Scribe)" — naming only Scribe 2. Rai had a dedicated ## Rai section with explicit "Rai always appears in team.md" instruction — Fact Checker had no equivalent section So the model added Rai (because instructed to) but had no instruction to add Fact Checker, even though the agent dir was scaffolded on disk. Fix: * Update team-size line to name all 4 always-on built-ins: Scribe + Ralph + Rai + Fact Checker * Add full ## Fact Checker — Verification & Devil's Advocate section mirroring the Rai pattern: roster-entry instruction, dual operating mode (per bradygaster#789 + bradygaster#1254), trigger phrase table, confidence ratings, DA brief structure, boundaries, state location Sync via sync-templates.mjs --sync propagates squad.agent.md changes to all 4 mirror targets: .squad-templates/, templates/, packages/squad-cli/ templates/, packages/squad-sdk/templates/, .github/agents/. Tests: new test/squad-agent-roster.test.ts runs against all 4 template targets and asserts: * The "Determine team size" line names all 4 built-ins * A ## Fact Checker section exists with "always appears in team.md" * The section declares dual operating mode (anchors bradygaster#789 + bradygaster#1254 design so a future PR can't accidentally split Fact Checker and Devil's Advocate again — cf. closed PR bradygaster#1294) * Existing Ralph + Rai sections still present 16/16 pass. Closes bradygaster#1299 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
🟠 Impact Analysis — PR #1300Risk tier: 🟠 HIGH 📊 Summary
🎯 Risk Factors
📦 Modules Affectedci-workflows (1 file)
root (2 files)
squad-cli (1 file)
squad-sdk (1 file)
templates (1 file)
tests (1 file)
This report is generated automatically for every PR. See #733 for details. |
Contributor
🛫 PR Readiness Check
PR Scope: 🔧 Infrastructure
|
| Status | Check | Details |
|---|---|---|
| ❌ | Single commit | 2 commits — consider squashing before review |
| ✅ | Not in draft | Ready for review |
| ✅ | Branch up to date | Up to date with dev |
| ❌ | Copilot review | No Copilot review yet — it may still be processing |
| ✅ | Changeset present | Changeset file found |
| ✅ | Scope clean | No .squad/ or docs/proposals/ files |
| ✅ | No merge conflicts | No merge conflicts |
| ✅ | Copilot threads resolved | 0 active Copilot thread(s) resolved (3 outdated skipped) |
| ❌ | CI passing | 7 check(s) still running |
Files Changed (7 files, +499 −5)
| File | +/− |
|---|---|
.changeset/fix-1299-fact-checker-roster-instructions.md |
+54 −0 |
.github/agents/squad.agent.md |
+75 −1 |
.squad-templates/squad.agent.md |
+75 −1 |
packages/squad-cli/templates/squad.agent.md.template |
+75 −1 |
packages/squad-sdk/templates/squad.agent.md.template |
+75 −1 |
templates/squad.agent.md.template |
+75 −1 |
test/squad-agent-roster.test.ts |
+70 −0 |
Total: +499 −5
This check runs automatically on every push. Fix any ❌ items and push again.
See CONTRIBUTING.md and PR Requirements for details.
tamirdresher
pushed a commit
to tamirdresher/squad
that referenced
this pull request
Jun 13, 2026
Contributor
There was a problem hiding this comment.
Pull request overview
This PR fixes a first-time casting regression where the coordinator prompt failed to explicitly instruct adding Fact Checker to the .squad/team.md ## Members roster, despite the directory being scaffolded. It updates the canonical squad.agent.md prompt (and synced mirrors) to name all always-on built-ins and adds a dedicated Fact Checker section, plus a regression test to prevent future drift.
Changes:
- Update the “Determine team size” guidance to explicitly name Scribe, Ralph, Rai, and Fact Checker as always-on built-ins.
- Add a new ## Fact Checker — Verification & Devil’s Advocate section with an explicit roster-entry instruction and operating-mode guidance.
- Add a vitest regression suite to assert these instructions exist across all template targets.
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
.squad-templates/squad.agent.md |
Canonical coordinator prompt update: team-size guidance + new Fact Checker section. |
templates/squad.agent.md.template |
Synced mirror of the updated coordinator prompt. |
packages/squad-cli/templates/squad.agent.md.template |
Synced mirror of the updated coordinator prompt for CLI distribution. |
packages/squad-sdk/templates/squad.agent.md.template |
Synced mirror of the updated coordinator prompt for SDK distribution. |
.github/agents/squad.agent.md |
Repo’s own coordinator file updated to match the canonical template. |
test/squad-agent-roster.test.ts |
New regression tests asserting the always-on roster instructions exist across template targets. |
.changeset/fix-1299-fact-checker-roster-instructions.md |
Changeset documenting the fix and triggering patch releases. |
| 2. Ask: *"What are you building? (language, stack, what it does)"* | ||
| 3. **Cast the team.** Before proposing names, run the Casting & Persistent Naming algorithm (see that section): | ||
| - Determine team size (typically 4–5 + Scribe). | ||
| - Determine team size (typically 4–5 + Scribe + Ralph + Rai + Fact Checker — the 4 always-on built-ins, see their dedicated sections below). |
|
|
||
| ### Roster Entry | ||
|
|
||
| Fact Checker always appears in `team.md`: `| Fact Checker | Fact Checker | .squad/agents/fact-checker/charter.md | 🔍 Verifier |` |
|
|
||
| **Philosophy: "Trust, but verify. Then steelman the opposition."** Fact Checker is rigorous but constructive — never gotcha-driven. Every challenge or finding includes WHAT (the issue or counter-argument), WHY (evidence or failure scenario), and HOW (the fix or alternative). | ||
|
|
||
| **On-demand reference:** Read `.squad/templates/fact-checker-charter.md` for the full charter, verification methodology, confidence rating taxonomy, and pre-ship ceremony format. |
This was referenced Jun 13, 2026
…rrect on-demand reference path Reviewer follow-ups on bradygaster#1300: 1. Team-size phrasing — the line read 'typically 4-5 + Scribe + Ralph + Rai + Fact Checker' which a model could parse as arithmetic (4-5 + 4 = 8-9, but it could also collapse). Rewrote it to make the composition explicit: '4-5 cast (user-domain) agents + 4 always-on built-ins = 8-9 total roster entries'. 2. Cast-exemption parity — Scribe, Ralph, and Rai each have an explicit 'exempt from casting' bullet but Fact Checker did not. Added the matching bullet right after Rai's. 3. Bad on-demand reference path — the FC section pointed at '.squad/templates/fact-checker-charter.md'. That file IS shipped (TEMPLATE_MANIFEST destination 'templates/fact-checker-charter.md') but only AFTER 'squad init' or 'squad upgrade' has populated .squad/templates/. A reader of squad.agent.md on an un-initialized repo (or in .github/agents/ on the cloud agent surface) would follow a dead link. Repointed to the '.squad/agents/fact-checker/charter.md' instance that ensureBuiltinAgents creates as part of the same init/upgrade path — that's where the rich charter actually lives at runtime per bradygaster#1299 + bradygaster#1301. All 4 mirrored copies re-synced via scripts/sync-templates.mjs. fact-checker-role.test.ts: 8/8 pass. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Collaborator
Author
|
All 3 reviewer threads addressed in commit 9a897ea:
All 4 mirrors re-synced via |
tamirdresher
added a commit
to tamirdresher/squad
that referenced
this pull request
Jun 13, 2026
tamirdresher
added a commit
to tamirdresher/squad
that referenced
this pull request
Jun 13, 2026
bradygaster
approved these changes
Jun 13, 2026
bradygaster
left a comment
bradygaster
left a comment
Owner
There was a problem hiding this comment.
✅ Flight approves. Template sync and first-time coordinator casting for Fact Checker are covered cleanly with tests.
tamirdresher
added a commit
that referenced
this pull request
Jun 13, 2026
…/fact-checker/ state dir (#1299 deep) (#1301) * fix(sdk): plumb Fact Checker like Rai — rich charter at init + .squad/fact-checker/ state dir (#1299 deep) PR #1300 fixed the documentation gap so the coordinator knows to roster Fact Checker. This PR fixes the structural gap behind it. Per user testing 2026-06-13: even after #1300 the actual agent on disk was still "a name on disk with a 21-line placeholder". Three structural problems: 1. squad init never used the rich {role}-charter.md templates. Both Rai and fact-checker got 478-byte generic stubs from generateCharter(). Rich templates only ran via squad upgrade's ensureBuiltinAgents path. 2. fact-checker had no state dir. Rai gets .squad/rai/{policy.md, audit-trail.md} via init.ts lines 879-941. fact-checker had nothing equivalent. 3. fact-checker-charter.md was only in packages/squad-cli/templates/ — missing from .squad-templates/ (canonical source) AND packages/ squad-sdk/templates/. SDK init's getSDKTemplatesDir() resolves to the SDK templates dir, so even if init tried to read the rich charter, the file wasn't there. Fix (4 parts): Part 1 - Rich charter at init (benefits BOTH Rai and fact-checker): * SDK init.ts agent loop now looks up {templatesDir}/{role}-charter.md for each agent and uses that as charter.md content if it exists. Falls back to generateCharter() for user-defined agents. * Result: fresh squad init produces .squad/agents/Rai/charter.md at 4525 bytes (full Rai charter) and fact-checker/charter.md at 3024 bytes (full FC charter). Previously both were 478-byte stubs. Part 2 - .squad/fact-checker/ state dir mirroring .squad/rai/: * New block in init.ts (right after the Rai seeding) creates .squad/fact-checker/policy.md (from templates/fact-checker-policy.md or inline fallback) and audit-trail.md. * New .squad-templates/fact-checker-policy.md (~6KB) is the canonical authority for dual-mode operating rules per #789 + #1254: - Mode 1 Verification: ✅/⚠️ /❌/🔍 confidence rating taxonomy - Mode 2 Devil's Advocate: required brief structure - Hard anti-fabrication rules - Advisory by default with narrow blocking exceptions - Audit trail rules (succinct, never raw source) Part 3 - Fix .squad-templates/ distribution gap: * Copied fact-checker-charter.md into .squad-templates/ so sync-templates.mjs propagates it to all 4 mirror targets including packages/squad-sdk/templates/. This unblocks Part 1. Part 4 - Plumbing: * .gitattributes: .squad/fact-checker/audit-trail.md merge=union * TEMPLATE_MANIFEST: fact-checker-policy.md * squad.agent.md Files Catalog: 2 new rows for FC state files Tests: 3 new regression tests in test/init.test.ts (28/28 pass total). npm run lint clean. Composability: This PR builds on #1300 (which adds the ## Fact Checker section to squad.agent.md and the team-size line fix). Both PRs modify squad.agent.md in disjoint regions and merge in either order. Full plumbing requires BOTH to land. Closes #1299 (deep fix; #1300 was the surface fix) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(sdk): lowercase fallback for rich-charter template lookup + sync .github/agents Two reviewer follow-ups on #1301 (#1299 deep): 1. Case-sensitive FS bug in rich-charter lookup The lookup tried '\-charter.md' and '\-charter.md' only. For Rai (role='Rai', name='Rai') this becomes 'Rai-charter.md', but the actual file shipped lowercase ('rai-charter.md'). On Windows the lookup succeeded because the filesystem is case-insensitive; on Linux CI it silently missed and fell back to the 478-byte generic stub — exactly the regression #1299 was trying to fix. Reproduced by 'should use the rich Rai-charter.md template at init' failing with 'expected 476 to be greater than 1000' on GitHub Actions. Add toLowerCase() candidates after the exact-case ones. De-dupe via a Set so we don't double-stat when role and name are already lowercase (fact-checker case). Guard each candidate against blank keys. 2. Template-sync parity The canonical .squad-templates/squad.agent.md gained two Fact Checker rows in the Files Catalog but the mirrored .github/agents/squad.agent.md copy was never re-synced, so the template-sync.test.ts byte-for-byte parity check would have fired. Run 'node scripts/sync-templates.mjs --sync' to regenerate. Verified: vitest 'rich Rai-charter' passes locally after the fix (previously failing on Linux CI run 27464079078). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <Copilot@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: Tamir Dresher <tamirdresher@users.noreply.github.com>
tamirdresher
added a commit
that referenced
this pull request
Jun 13, 2026
…ough the cross-squad skill (#1307) * fix(coordinator): route "spawn a squad" / "another squad" prompts through the cross-squad skill A coordinator initialised by `squad init` saw prompts like "spawn two squads of designers and devs" and fanned out raw `task` agents inside its own context, treating "squad" as generic English for "team / group". It never invoked the bundled `cross-squad` or `cross-squad-communication` skills, so the peer-squad delegation protocol (registry / manifest / sync CLI / git-async / GH-issue patterns) was bypassed entirely. Two structural holes in squad.agent.md allowed this: 1. The Routing table had no row mapping "spawn a squad" phrasing to the Squad-PRODUCT concept (only "upgrade squad" / "squad commands" rows covered Squad-as-a-product vocabulary). 2. The Skill-aware-routing block was process discipline ("check skill directories by domain relevance") with no hard "if the user's word matches a skill name, MUST load the skill" trigger. This fix: - Adds a new routing-table row for the squad-spawning vocabulary ("spawn a squad", "another squad", "two squads", "second squad", "fan out to squads", "delegate to a squad"). Action: invoke the skill tool on cross-squad AND cross-squad-communication BEFORE any task spawn, then delegate via Pattern 0/1/2/3. - Adds a "Hard trigger — keyword-to-skill match" paragraph at the top of the Skill-aware-routing block. If any word in the user's request matches an installed skill name (squad → cross-squad, reflect → reflect, ceremony → matching ceremony skill, fact-check → fact-checking, release → release-process), the coordinator MUST invoke the skill tool to fully load that skill before designing its approach. Includes a "failure mode this rule closes" pointer so the guard survives future paraphrasing. - Strengthens cross-squad/SKILL.md with a Read-this-FIRST callout above the existing Context paragraph, so even a coordinator that skips the routing-table row still hits the trigger when it does eventually load the skill. - Adds a regression test (template-sync.test.ts) that asserts the row + the hard-trigger paragraph + the worked example are present in every mirrored copy of squad.agent.md (5 locations). All 4 mirrors re-synced via `scripts/sync-templates.mjs --sync`. Verified: 223/223 template-sync tests pass. Composability: disjoint from #1292/#1293/#1295/#1298/#1300/#1301/#1302/ #1303/#1304/#1306 — only touches squad.agent.md mirrors + cross-squad/ SKILL.md + the template-sync test. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(coordinator): strengthen squad-spawn disambiguation — ask_user on ambiguity + anti-patterns Real-world failure (2026-06-13): even AFTER the routing-table row and hard-trigger paragraph from b5d05fb landed, a peer-squad coordinator *still* did ad-hoc `task` fan-out for "spawn two squads of engineers and QAs". Self-diagnosis surfaced four contributing failure modes: 1. Prior-session anchoring (saw earlier `reviews/squad-alpha/` folders and matched the pattern without re-evaluating user intent). 2. Ambiguous wording, lazy interpretation (silently picked the cheaper option instead of asking). 3. Coordinator doctrine biases toward `task` fan-out (the existing Eager Execution / Parallel Fan-Out section pulled the coordinator back even after it had loaded `cross-squad`). 4. Cost/overhead instinct ("two real squads for a 30-line app feels disproportionate" — judged silently instead of surfacing the trade-off). The original PR #1307 fix closed modes 1 and 3 mechanically (forces the skill to load) but left modes 2 and 4 open (didn't dictate what to DO with that knowledge). This commit closes them: A. squad.agent.md routing row — added two explicit clauses: - "**Default = literal Squad install.** Calling `task` sub-agents 'squad-alpha' / 'squad-beta' does NOT make them squads — that is the explicit anti-pattern." - "**If the request is ambiguous** ... you MUST `ask_user` with a 2-choice prompt — and never silently pick the cheaper option." No escape hatch. The coordinator can no longer rationalise the downgrade as a judgment call. B. cross-squad/SKILL.md — added a full `## Disambiguation: 'squad' vs ad-hoc agents` section with: - Default-behaviour table mapping common phrasings to expected coordinator actions (real squads vs ad-hoc agents vs ambiguous). - ask_user 2-choice protocol verbatim (heavier/persistent vs lighter/ephemeral) so the coordinator has the exact prompt shape. - Four named anti-patterns drawn directly from the observed failure: * Naming task agents "squad-alpha" doesn't make them squads * Prior-session anchoring (pattern is a hint, not a contract) * Silent cheaper-option pick (judgment call belongs to the user) * Loading the skill but doing task fan-out anyway (disambiguation rule OVERRIDES generic fan-out doctrine when "squad" was the trigger) - Sharpened `description:` so the squad skill-aware-router has better natural-language hooks. - `triggers:` frontmatter array (Copilot CLI ignores `triggers:` per sdk/index.js decompile, but the squad coordinator's skill-aware routing system uses natural-language matching against frontmatter + content, so documenting the phrases here helps that matcher fire). C. Regression tests in test/template-sync.test.ts: - Routing row must mention `ask_user` + "anti-pattern" (new × 5 mirrors = 5 assertions). - cross-squad/SKILL.md must have `## Disambiguation` section, default-behaviour rule, ask_user requirement, the squad-alpha anti-pattern, and triggers: frontmatter (5 new × 3 mirrors = 15 assertions). - 20 new assertions total; 243/243 template-sync tests pass. cross-squad/SKILL.md mirrored to packages/squad-cli/templates/skills/ and packages/squad-sdk/templates/skills/ (byte-identical). squad.agent.md re-synced to all 4 mirrors via scripts/sync-templates.mjs. Composability: still disjoint from all other open PRs. Pure additions to two files (squad.agent.md row, cross-squad/SKILL.md content) plus mirrors + tests. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Tamir Dresher <tamirdresher@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This was referenced Jun 14, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #1299.
Symptom
When a user runs
squad initfollowed bycopilot --agent squad, the coordinator's first-time casting flow correctly creates.squad/agents/fact-checker/on disk (per merged PR #1223) but omits Fact Checker from the## Memberstable in.squad/team.md. Rai is included, Ralph is included, Scribe is included — Fact Checker is silently dropped.Verified live on 2026-06-13 in
C:\Temp\test-weekend\— fresh init produced 8 agent dirs (including fact-checker) but the cast wrote a 7-member team.md (no fact-checker).Root cause
.squad-templates/squad.agent.md(the canonical coordinator instructions, mirrored to all package templates) had two gaps:| Rai | RAI Reviewer | ... | 🛡️ RAI |" — but no equivalent section existed for Fact Checker.The model added Rai correctly (saw the roster-entry instruction) but had no instruction to add Fact Checker, even though the directory was already scaffolded.
Fix
## Fact Checker — Verification & Devil's Advocatesection that mirrors the Rai pattern:Fact Checker always appears in team.md: \| Fact Checker | Fact Checker | .squad/agents/fact-checker/charter.md | 🔍 Verifier |\Sync targets
sync-templates.mjs --syncpropagatessquad.agent.mdto all 4 mirror targets:templates/squad.agent.md.templatepackages/squad-cli/templates/squad.agent.md.templatepackages/squad-sdk/templates/squad.agent.md.template.github/agents/squad.agent.md(this repo's own coordinator file)Test coverage
New
test/squad-agent-roster.test.tsruns against all 4 template targets and asserts (16 tests = 4 assertions × 4 files):## Fact Checkersection exists with an explicit "always appears in team.md" roster-entry lineAll 40 existing init/cli/init tests still pass;
npm run lintclean.Verification path
After this PR ships, a fresh
squad init+copilot --agent squadproduces ateam.md ## Memberstable that includes Fact Checker alongside Scribe, Ralph, and Rai.