Promote Fact Checker to active roster; declare dual operating mode by tamirdresher · Pull Request #1254 · bradygaster/squad

tamirdresher · 2026-06-10T11:17:25Z

What

Promotes the existing fact-checker agent (stub charter, never on roster) to a full team member with FAO-inspired charter, claim-verification methodology, and hard rules against fabrication.

Adds the Pre-Ship Fact Check ceremony so any user-facing artifact gets a claim-by-claim verification pass before it ships.

Updates identity/now.md (12 weeks stale from v0.9.1) to reflect:

Current v0.10.0 release status
Dual operating mode: Brady's upstream maintenance + Tamir's framework R&D (shared roster, shared decisions ledger)
The 7 governance/MCP findings from the 2026-06-09 investigation session (docs: state-backends upgrade flow misses pre-commit hook, file reappearance, and recovery #1226, feat: conditional .gitignore for two-layer/orphan state files (defense-in-depth) #1228, docs(skills): docs say .squad/skills/ but code writes to .copilot/skills/ — pick one and clarify #1241–Workspace-only squad_state MCP not loaded in copilot -p mode → agents forge audit-log entries #1247)
Two new state-tool gaps discovered while writing the new now.md itself (will be filed separately)

Why

The 2026-06-09 forged-audit-entry incident showed concretely why an explicit Verification + Devil's Advocate role matters. The team was already substantially staffed but lacked someone whose job is to challenge claims with evidence.

The existing fact-checker/charter.md already named the role correctly: "Devil's advocate and verification agent — validates claims, detects hallucinations, and runs counter-hypotheses." This PR makes that real: full charter, on the roster, on routing, owning a ceremony.

Charter highlights

5-step verification methodology (Restate → Source → Counter-hypothesis → Test → Verdict)
Hard rules against fabrication; every verdict needs a citation a human can re-check
Tier policy: Lightweight (manual invocation) / Standard (Pre-Ship) / Full (multi-claim batch)
Apollo FAO inspiration — "What if?" is the question that saved Apollo 13

Known gap NOT addressed here

Rai 🛡️ (RAI Reviewer) is missing from this team's roster despite being required by squad.agent.md governance. The .squad/agents/Rai/ directory exists. This is migration drift from the team predating Rai's introduction.

Deliberately deferred for a separate PR with Brady's review since it's a different concern.

Files

File	Change
`.squad/agents/fact-checker/charter.md`	Stub (522B) → full charter (4,685B)
`.squad/team.md`	+ Fact Checker row (🔍 Reviewer)
`.squad/routing.md`	+ work-type row "Claim verification & devil's advocate"
`.squad/ceremonies.md`	+ Pre-Ship Fact Check ceremony
`.squad/identity/now.md`	Rewritten (was 12 weeks stale)

No code changes. Build passes locally (npm run build clean in 16.4s).

Promotes the existing fact-checker agent (stub charter, never on roster) to a full team member with FAO-inspired charter, claim-verification methodology, and hard rules against fabrication or unverified claims. Adds the Pre-Ship Fact Check ceremony so any user-facing artifact gets a claim-by-claim verification pass before it ships. Updates identity/now.md (12 weeks stale from v0.9.1) to reflect: - Current v0.10.0 release status - Dual operating mode: Brady's upstream maintenance + Tamir's framework R&D (both sharing the same roster and decisions ledger) - The 7 governance/MCP findings from the 2026-06-09 investigation session (#1226, #1228, #1241-#1247) - Two new state-tool gaps discovered while writing the new now.md itself Rationale: the 2026-06-09 forged-audit-entry incident showed concretely why an explicit Verification + Devil's Advocate role matters. The team was already substantially staffed but lacked someone whose job is to challenge claims with evidence. Fact Checker is that role. Known gap NOT addressed here: Rai (RAI Reviewer) is required by squad.agent.md but missing from this team's roster. The .squad/agents/Rai directory exists. This is migration drift from the team predating Rai; should be addressed in a separate PR for Brady's review. No code changes. Only static team config + one new charter. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

github-actions · 2026-06-10T11:17:36Z

🟢 Impact Analysis — PR #1254

Risk tier: 🟢 LOW

📊 Summary

Metric	Count
Files changed	5
Files added	1
Files modified	4
Files deleted	0
Modules touched	1

🎯 Risk Factors

5 files changed (≤5 → LOW)
1 module(s) touched (≤1 → LOW)

📦 Modules Affected

squad-state (5 files)

.squad/agents/fact-checker/charter.md
.squad/ceremonies.md
.squad/identity/now.md
.squad/routing.md
.squad/team.md

This report is generated automatically for every PR. See #733 for details.

github-actions · 2026-06-10T11:17:44Z

🛫 PR Readiness Check

ℹ️ This comment updates on each push. Last checked: commit 04f9f99

PR Scope: 🔧 Infrastructure

⚠️ 2 item(s) to address before review

Status	Check	Details
✅	Single commit	1 commit — clean history
✅	Not in draft	Ready for review
✅	Branch up to date	Up to date with dev
❌	Copilot review	No Copilot review yet — it may still be processing
✅	Changeset present	No source files changed — changeset not required
✅	Scope clean	⚠️ PR includes 5 .squad/ file(s) — ensure these are intentional
✅	No merge conflicts	No merge conflicts
✅	Copilot threads resolved	No Copilot review threads
❌	CI passing	4 check(s) still running

Files Changed (5 files, +180 −111)

File	+/−
`.squad/agents/fact-checker/charter.md`	+93 −0
`.squad/ceremonies.md`	+20 −0
`.squad/identity/now.md`	+65 −111
`.squad/routing.md`	+1 −0
`.squad/team.md`	+1 −0

Total: +180 −111

This check runs automatically on every push. Fix any ❌ items and push again.
See CONTRIBUTING.md and PR Requirements for details.

github-actions · 2026-06-10T11:17:44Z

⚠️ Squad File Leakage Detected

The following .squad/ files were modified in this PR:

.squad/agents/fact-checker/charter.md
.squad/ceremonies.md
.squad/identity/now.md
.squad/routing.md
.squad/team.md

These files affect team routing, agent charters, and decisions.
If intentional, ensure approval from the team lead.

Copilot

Pull request overview

Promotes the existing fact-checker agent from a stub to an active, rostered reviewer role, and updates team process/identity docs to formalize claim verification as part of the workflow.

Changes:

Add Fact Checker to the team roster and routing rules as the verification/devil’s-advocate specialist.
Introduce a Pre-Ship Fact Check ceremony to ensure user-facing artifacts get claim-by-claim verification.
Rewrite .squad/identity/now.md to reflect current operating context, active workstreams, and recent governance/MCP findings.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
`.squad/agents/fact-checker/charter.md`	Adds a full charter with methodology, responsibilities, and hard rules for evidence-based verification.
`.squad/team.md`	Adds Fact Checker to the roster as a reviewer role.
`.squad/routing.md`	Adds routing entry for claim verification & devil’s-advocate work to Fact Checker.
`.squad/ceremonies.md`	Adds the Pre-Ship Fact Check ceremony definition.
`.squad/identity/now.md`	Updates the “current focus” document to v0.10.0+ context and dual operating mode.

+## Project Context
+
+- **Project:** squad-sdk — programmable multi-agent runtime for GitHub Copilot
+- **Stack:** TypeScript (strict, ESM-only), Node.js ≥20, `@github/copilot-sdk`, Vitest, esbuild


…#1299) (#1300) * fix(docs): tell coordinator to roster Fact Checker on first-time cast (#1299) squad init correctly creates .squad/agents/fact-checker/ on disk (per merged PR #1223). But when the user opens copilot --agent squad and the coordinator runs first-time casting, it OMITS Fact Checker from the team.md ## Members table while including Scribe, Ralph, and Rai. Root cause: .squad-templates/squad.agent.md had two gaps: 1. Line 56 said "team size (typically 4-5 + Scribe)" — naming only Scribe 2. Rai had a dedicated ## Rai section with explicit "Rai always appears in team.md" instruction — Fact Checker had no equivalent section So the model added Rai (because instructed to) but had no instruction to add Fact Checker, even though the agent dir was scaffolded on disk. Fix: * Update team-size line to name all 4 always-on built-ins: Scribe + Ralph + Rai + Fact Checker * Add full ## Fact Checker — Verification & Devil's Advocate section mirroring the Rai pattern: roster-entry instruction, dual operating mode (per #789 + #1254), trigger phrase table, confidence ratings, DA brief structure, boundaries, state location Sync via sync-templates.mjs --sync propagates squad.agent.md changes to all 4 mirror targets: .squad-templates/, templates/, packages/squad-cli/ templates/, packages/squad-sdk/templates/, .github/agents/. Tests: new test/squad-agent-roster.test.ts runs against all 4 template targets and asserts: * The "Determine team size" line names all 4 built-ins * A ## Fact Checker section exists with "always appears in team.md" * The section declares dual operating mode (anchors #789 + #1254 design so a future PR can't accidentally split Fact Checker and Devil's Advocate again — cf. closed PR #1294) * Existing Ralph + Rai sections still present 16/16 pass. Closes #1299 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * doc(squad.agent.md): clarify Fact Checker is exempt from casting + correct on-demand reference path Reviewer follow-ups on #1300: 1. Team-size phrasing — the line read 'typically 4-5 + Scribe + Ralph + Rai + Fact Checker' which a model could parse as arithmetic (4-5 + 4 = 8-9, but it could also collapse). Rewrote it to make the composition explicit: '4-5 cast (user-domain) agents + 4 always-on built-ins = 8-9 total roster entries'. 2. Cast-exemption parity — Scribe, Ralph, and Rai each have an explicit 'exempt from casting' bullet but Fact Checker did not. Added the matching bullet right after Rai's. 3. Bad on-demand reference path — the FC section pointed at '.squad/templates/fact-checker-charter.md'. That file IS shipped (TEMPLATE_MANIFEST destination 'templates/fact-checker-charter.md') but only AFTER 'squad init' or 'squad upgrade' has populated .squad/templates/. A reader of squad.agent.md on an un-initialized repo (or in .github/agents/ on the cloud agent surface) would follow a dead link. Repointed to the '.squad/agents/fact-checker/charter.md' instance that ensureBuiltinAgents creates as part of the same init/upgrade path — that's where the rich charter actually lives at runtime per #1299 + #1301. All 4 mirrored copies re-synced via scripts/sync-templates.mjs. fact-checker-role.test.ts: 8/8 pass. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <Copilot@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: Tamir Dresher <tamirdresher@users.noreply.github.com>

…/fact-checker/ state dir (#1299 deep) (#1301) * fix(sdk): plumb Fact Checker like Rai — rich charter at init + .squad/fact-checker/ state dir (#1299 deep) PR #1300 fixed the documentation gap so the coordinator knows to roster Fact Checker. This PR fixes the structural gap behind it. Per user testing 2026-06-13: even after #1300 the actual agent on disk was still "a name on disk with a 21-line placeholder". Three structural problems: 1. squad init never used the rich {role}-charter.md templates. Both Rai and fact-checker got 478-byte generic stubs from generateCharter(). Rich templates only ran via squad upgrade's ensureBuiltinAgents path. 2. fact-checker had no state dir. Rai gets .squad/rai/{policy.md, audit-trail.md} via init.ts lines 879-941. fact-checker had nothing equivalent. 3. fact-checker-charter.md was only in packages/squad-cli/templates/ — missing from .squad-templates/ (canonical source) AND packages/ squad-sdk/templates/. SDK init's getSDKTemplatesDir() resolves to the SDK templates dir, so even if init tried to read the rich charter, the file wasn't there. Fix (4 parts): Part 1 - Rich charter at init (benefits BOTH Rai and fact-checker): * SDK init.ts agent loop now looks up {templatesDir}/{role}-charter.md for each agent and uses that as charter.md content if it exists. Falls back to generateCharter() for user-defined agents. * Result: fresh squad init produces .squad/agents/Rai/charter.md at 4525 bytes (full Rai charter) and fact-checker/charter.md at 3024 bytes (full FC charter). Previously both were 478-byte stubs. Part 2 - .squad/fact-checker/ state dir mirroring .squad/rai/: * New block in init.ts (right after the Rai seeding) creates .squad/fact-checker/policy.md (from templates/fact-checker-policy.md or inline fallback) and audit-trail.md. * New .squad-templates/fact-checker-policy.md (~6KB) is the canonical authority for dual-mode operating rules per #789 + #1254: - Mode 1 Verification: ✅/⚠️/❌/🔍 confidence rating taxonomy - Mode 2 Devil's Advocate: required brief structure - Hard anti-fabrication rules - Advisory by default with narrow blocking exceptions - Audit trail rules (succinct, never raw source) Part 3 - Fix .squad-templates/ distribution gap: * Copied fact-checker-charter.md into .squad-templates/ so sync-templates.mjs propagates it to all 4 mirror targets including packages/squad-sdk/templates/. This unblocks Part 1. Part 4 - Plumbing: * .gitattributes: .squad/fact-checker/audit-trail.md merge=union * TEMPLATE_MANIFEST: fact-checker-policy.md * squad.agent.md Files Catalog: 2 new rows for FC state files Tests: 3 new regression tests in test/init.test.ts (28/28 pass total). npm run lint clean. Composability: This PR builds on #1300 (which adds the ## Fact Checker section to squad.agent.md and the team-size line fix). Both PRs modify squad.agent.md in disjoint regions and merge in either order. Full plumbing requires BOTH to land. Closes #1299 (deep fix; #1300 was the surface fix) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(sdk): lowercase fallback for rich-charter template lookup + sync .github/agents Two reviewer follow-ups on #1301 (#1299 deep): 1. Case-sensitive FS bug in rich-charter lookup The lookup tried '\-charter.md' and '\-charter.md' only. For Rai (role='Rai', name='Rai') this becomes 'Rai-charter.md', but the actual file shipped lowercase ('rai-charter.md'). On Windows the lookup succeeded because the filesystem is case-insensitive; on Linux CI it silently missed and fell back to the 478-byte generic stub — exactly the regression #1299 was trying to fix. Reproduced by 'should use the rich Rai-charter.md template at init' failing with 'expected 476 to be greater than 1000' on GitHub Actions. Add toLowerCase() candidates after the exact-case ones. De-dupe via a Set so we don't double-stat when role and name are already lowercase (fact-checker case). Guard each candidate against blank keys. 2. Template-sync parity The canonical .squad-templates/squad.agent.md gained two Fact Checker rows in the Files Catalog but the mirrored .github/agents/squad.agent.md copy was never re-synced, so the template-sync.test.ts byte-for-byte parity check would have fired. Run 'node scripts/sync-templates.mjs --sync' to regenerate. Verified: vitest 'rich Rai-charter' passes locally after the fix (previously failing on Linux CI run 27464079078). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <Copilot@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: Tamir Dresher <tamirdresher@users.noreply.github.com>

tamirdresher · 2026-06-14T04:02:01Z

Closing — superseded by #1300 (coordinator rosters Fact Checker) + #1301 (full Fact Checker plumbing, rich charter, dual-mode declaration). Both merged into dev as part of the v0.10 stabilisation effort and shipped in @bradygaster/squad-cli@0.10.0-insider.1.

Copilot AI review requested due to automatic review settings June 10, 2026 11:17

Copilot started reviewing on behalf of tamirdresher June 10, 2026 11:17 View session

This was referenced Jun 10, 2026

MCP squad_state_write refuses .squad/identity/* but squad.agent.md requires updating now.md #1255

Open

MCP squad_decide author validator too narrow — cannot express on-behalf-of authorship #1256

Open

Copilot AI reviewed Jun 10, 2026

View reviewed changes

Comment thread .squad/agents/fact-checker/charter.md

## Project Context

- **Project:** squad-sdk — programmable multi-agent runtime for GitHub Copilot

- **Stack:** TypeScript (strict, ESM-only), Node.js ≥20, `@github/copilot-sdk`, Vitest, esbuild

tamirdresher closed this Jun 14, 2026

tamirdresher deleted the squad/team-fact-checker-promotion branch June 14, 2026 04:02

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Promote Fact Checker to active roster; declare dual operating mode#1254

Promote Fact Checker to active roster; declare dual operating mode#1254
tamirdresher wants to merge 1 commit into
devfrom
squad/team-fact-checker-promotion

tamirdresher commented Jun 10, 2026

Uh oh!

github-actions Bot commented Jun 10, 2026

Uh oh!

github-actions Bot commented Jun 10, 2026

Uh oh!

github-actions Bot commented Jun 10, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

tamirdresher commented Jun 14, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

tamirdresher commented Jun 10, 2026

What

Why

Charter highlights

Known gap NOT addressed here

Files

Related

Uh oh!

github-actions Bot commented Jun 10, 2026

🟢 Impact Analysis — PR #1254

📊 Summary

🎯 Risk Factors

📦 Modules Affected

Uh oh!

github-actions Bot commented Jun 10, 2026

🛫 PR Readiness Check

⚠️ 2 item(s) to address before review

Files Changed (5 files, +180 −111)

Uh oh!

github-actions Bot commented Jun 10, 2026

⚠️ Squad File Leakage Detected

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

tamirdresher commented Jun 14, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants