feat(ORCH): orchestrator-first plugin spec — idea + research (closes #501)

docs/adr/0046-expand-orchestrator-tool-list-to-dispatch-authority.md

@@ -0,0 +1,117 @@
+---
+id: ADR-0046
+title: Expand orchestrator tool list from Read/Grep to full dispatch authority
+status: accepted
+date: 2026-05-13
+deciders:
+  - Luis Mendez
+consulted:
+  - Claude Sonnet 4.6
+informed:
+  - Specorator contributors
+supersedes: []
+superseded-by: []
+tags: [orchestrator, agents, security, architecture, goal-loop]
+---
+
+# ADR-0046 — Expand orchestrator tool list from Read/Grep to full dispatch authority
+
+## Status
+
+Accepted
+
+## Context
+
+The current orchestrator agent definition (`.claude/agents/orchestrator.md`) declares `tools: [Read, Grep]`. With only these two tools, the orchestrator is advisory-only: it can inspect state and recommend what should happen next, but it cannot dispatch subagents, update workflow state, or gate on user decisions. Every action requires a manual slash command from the user.
+
+The goal-loop feature (issue #501, PRD-ORCH-001) requires the orchestrator to:
+
+1. Spawn specialist subagents (researcher, architect, planner, dev, qa, reviewer) via the Agent tool (REQ-ORCH-001).
+2. Write and update `workflow-state.md` to track phase transitions and persist state before every HITL gate (REQ-ORCH-002, REQ-ORCH-022).
+3. Write new artifacts to `specs/<slug>/` as the goal-loop progresses — specifically `scope.md` and `session-summary.md`.
+4. Present synchronous HITL gates to the user via AskUserQuestion (REQ-ORCH-008, REQ-ORCH-011, REQ-ORCH-015).
+
+Granting the orchestrator `Agent` tool access is a trust-boundary expansion. The orchestrator, as the root session agent (via plugin `settings.json agent: orchestrator`), already operates with the session's full permission mode. Adding `Agent`, `Write`, and `Edit` to its tool list makes that scope explicit and exercisable rather than implicit.
+
+The platform constraint that subagents cannot spawn subagents (Claude Code hard limit) means the orchestrator must be the root session agent — it cannot itself be invoked as a subagent. This architectural constraint actually reduces risk: the orchestrator's expanded tool list does not propagate to any subagent context.
+
+## Decision
+
+We expand the orchestrator agent's tool list from `[Read, Grep]` to `[Agent, Read, Write, Edit, AskUserQuestion]`.
+
+The rationale for each addition:
+
+- **Agent** — required to dispatch specialist subagents per REQ-ORCH-001. Without this, the goal-loop cannot spawn any specialist.
+- **Write** — required to create `workflow-state.md`, `scope.md`, and `session-summary.md`. The orchestrator is the sole owner of `workflow-state.md` transitions (REQ-ORCH-002).
+- **Edit** — required to update `workflow-state.md` in-place between phases without rewriting the full file on each transition.
+- **AskUserQuestion** — required to implement the three HITL gates (post-scope, post-design, post-review) and the stall gate (REQ-ORCH-008, REQ-ORCH-011, REQ-ORCH-014, REQ-ORCH-015).
+- **Read** — retained for pre-flight precondition checks (REQ-ORCH-003) and for reading `workflow-state.md` on session resume.
+- **Grep** — removed. The orchestrator does not perform search operations in the goal-loop; search is delegated to specialist subagents. Removing Grep narrows the tool surface.
+
+The orchestrator's write boundary is restricted to `specs/<slug>/` directories and their content files. It does not gain Bash, WebSearch, or any other tool.
+
+## Considered options
+
+### Option A — Keep Read/Grep; use a wrapper subagent as the dispatch authority
+
+Have the orchestrator remain advisory-only; introduce a separate "goal-loop runner" subagent with full dispatch authority that the user invokes via a slash command.
+
+- Pros: No change to existing orchestrator trust surface.
+- Cons: The goal-loop runner would itself need Agent tool access — we have not reduced the trust surface, only moved it. Additionally, the platform constraint (subagents cannot spawn subagents) means a subagent cannot be the dispatch authority. This option is architecturally impossible under the platform constraints.
+
+### Option B — Expand orchestrator tool list (chosen)
+
+Promote the existing orchestrator to full dispatch authority by adding Agent, Write, Edit, AskUserQuestion.
+
+- Pros: Consistent with the platform model (root session agent has dispatch authority); no new agent definition; the trust expansion is explicit in the agent frontmatter; the platform constraint enforces that this authority does not cascade to subagents.
+- Cons: The orchestrator now has Write access to `specs/` directories; this is a wider trust surface than the current Read-only posture.
+
+### Option C — Use Agent Teams (experimental)
+
+Use `CLAUDE_CODE_EXPERIMENTAL_AGENT_TEAMS=1` to delegate dispatch authority to a team lead without expanding the orchestrator's own tool list.
+
+- Pros: Experimental feature; no tool list change on the orchestrator.
+- Cons: Rejected in research.md (Alternative C) as an experimental feature with known limitations: skills and MCP not applied to teammates, no session resumption, one team per lead. Reserved for v2.
+
+## Consequences
+
+### Positive
+
+- The orchestrator can now autonomously drive the full goal-loop without any user slash-command chaining (REQ-ORCH-001, REQ-ORCH-002, REQ-ORCH-006).
+- HITL gates are enforced by the orchestrator itself via AskUserQuestion, not by convention.
+- `workflow-state.md` transitions are owned and written by a single agent, preventing multi-agent write races.
+- Subagents continue to have no Agent tool access — they cannot spawn further subagents, preserving the star topology.
+
+### Negative
+
+- The orchestrator now has Write access to `specs/<slug>/` paths. A defective orchestrator implementation could overwrite spec artifacts. Mitigation: the orchestrator's write boundary is documented and lint-checked; subagents do not inherit this access.
+- Removing Grep is a mild capability reduction for the existing advisory-only use case. Any user who invokes the orchestrator outside a goal-loop and expects search behaviour will notice a gap. Mitigation: the orchestrator's advisory use case is superseded by the goal-loop; the description in the agent definition is updated to reflect the new primary role.
+
+### Neutral
+
+- The plugin's `settings.json` continues to declare `agent: orchestrator`, making the orchestrator the default session agent when the plugin is enabled. This does not change.
+- The `build-claude-plugin.ts` pipeline copies `.claude/agents/orchestrator.md` into the plugin bundle unchanged; the tool list expansion is automatically included in the next bundle build.
+
+## Compliance
+
+- The orchestrator's YAML frontmatter `tools:` field in `.claude/agents/orchestrator.md` must list `[Agent, Read, Write, Edit, AskUserQuestion]` exactly.
+- `check-agents.ts` must validate that the orchestrator's frontmatter does not declare `hooks`, `mcpServers`, or `permissionMode` (REQ-ORCH-020).
+- Design spec (Part C — Components table) must document the orchestrator's write boundary as `specs/<slug>/` only.
+- The release-criteria checklist in `requirements.md` includes verification that all 85 existing slash commands continue to function after the orchestrator tool expansion (REQ-ORCH-005, REQ-ORCH-021).
+
+## References
+
+- PRD-ORCH-001 — Goal-oriented orchestrator plugin requirements
+- REQ-ORCH-001 — Orchestrator dispatch via Agent tool
+- REQ-ORCH-002 — Orchestrator ownership of workflow-state.md transitions
+- REQ-ORCH-020 — Plugin agent frontmatter validation
+- REQ-ORCH-021 — Backward compatibility for non-plugin users
+- `specs/goal-oriented-orchestrator-plugin/design.md` Part C — Architecture
+- `research.md` — Alternative B (Anthropic native Orchestrator-Subagent pattern, recommended)
+- ADR-0043 — Plugin bundle distribution model
+- [Claude Code — Create custom subagents](https://code.claude.com/docs/en/sub-agents)
+- [GitHub Issue #19077 — Subagents cannot spawn subagents](https://github.com/anthropics/claude-code/issues/19077)
+
+---
+
+> **ADR bodies are immutable.** To change a decision, supersede it with a new ADR; only the predecessor's `status` and `superseded-by` pointer fields may be updated.

docs/adr/0047-adopt-goal-loop-workflow-state-schema-extensions.md

@@ -0,0 +1,123 @@
+---
+id: ADR-0047
+title: Extend workflow-state.md schema with goal-loop fields
+status: accepted
+date: 2026-05-13
+deciders:
+  - Luis Mendez
+consulted:
+  - Claude Sonnet 4.6 (architect agent)
+informed:
+  - Specorator contributors
+supersedes: []
+superseded-by: []
+tags: [orchestrator, goal-loop, workflow-state, schema, zod, artifact]
+---
+
+# ADR-0047 — Extend workflow-state.md schema with goal-loop fields
+
+## Status
+
+Accepted
+
+## Context
+
+`workflow-state.md` is the durable checkpoint for all Specorator session state. ADR-0042 established a typed-artifact reader seam (Zod schema) for frontmatter parsing. The existing schema captures lifecycle stage and status fields sufficient for the 11-stage manual workflow.
+
+The goal-loop (PRD-ORCH-001) introduces an orchestrator that drives multi-phase sessions autonomously. Session resume (REQ-ORCH-022), stall detection (REQ-ORCH-014), and pre-flight precondition checks (REQ-ORCH-003) all require the orchestrator to read structured state from `workflow-state.md` that is not present in the current schema. Specifically:
+
+1. **Current phase within the goal-loop** — the orchestrator must know which of the six phases (scope, research, design, plan, implement, review) is active when resuming a session.
+2. **HITL state** — which gate is pending; what the gate content was (so the gate can be replayed without re-running the phase).
+3. **Researcher count** — how many analyst subagents were dispatched in the research wave (used in status messages and for wave-cost auditing in `session-summary.md`).
+4. **Wave schedule** — the topological wave plan derived from `tasks.md` (wave index → list of task IDs). Persisted so that the orchestrator can resume mid-wave after a session interrupt without re-parsing `tasks.md`.
+5. **Stall counters** — per-task retry count, keyed by task ID. Allows the orchestrator to detect stalls across session restarts, not just within a single session.
+
+The release criteria in `requirements.md` explicitly state: "`workflow-state.md` Zod schema (ADR-0042 prerequisite) is in place before implementation of REQ-ORCH-002 and REQ-ORCH-022."
+
+## Decision
+
+We extend the `workflow-state.md` Zod schema (introduced by ADR-0042) with the following optional goal-loop fields in the YAML frontmatter:
+
+```yaml
+goal_loop:
+  current_phase: scope | research | design | plan | implement | review | complete | aborted
+  hitl_state:
+    gate: 1 | 2 | 3 | stall
+    pending: true | false
+    gate_content_ref: "specs/<slug>/workflow-state.md#gate-content"  # embedded in body
+  researcher_count: <integer 1–5>
+  wave_schedule:
+    - wave: 1
+      task_ids: [T-ORCH-001, T-ORCH-002]
+    - wave: 2
+      task_ids: [T-ORCH-003]
+  stall_counters:
+    T-ORCH-003: 2
+  artifacts_produced:
+    - specs/<slug>/scope.md
+    - specs/<slug>/research.md
+```
+
+All `goal_loop` fields are optional — their absence indicates the session is using the manual 11-stage command workflow, not the goal-loop. Existing `workflow-state.md` files without a `goal_loop` key are valid under the extended schema.
+
+The `hitl_state.gate_content_ref` points to a section in the `workflow-state.md` body (not a separate file). Gate content is embedded in the body as a Markdown block so the entire checkpoint is a single file.
+
+The Zod schema extension follows the additive-only convention established by ADR-0042: no existing required field is changed; only new optional fields are added.
+
+## Considered options
+
+### Option A — Extend workflow-state.md schema (chosen)
+
+Add optional `goal_loop` fields to the existing Zod schema. Single file; single source of truth for session state.
+
+- Pros: consistent with the existing state model; no new infrastructure; session resume reads the same file as phase tracking; the `build-claude-plugin.ts` pipeline requires no changes.
+- Cons: `workflow-state.md` grows in size during a goal-loop session. For a 5-wave session with 20 tasks, stall_counters and wave_schedule add ~30 lines of YAML.
+
+### Option B — Introduce a separate goal-loop-state.md file
+
+Store goal-loop-specific state in `specs/<slug>/goal-loop-state.md` distinct from `workflow-state.md`.
+
+- Pros: keeps the existing `workflow-state.md` schema unchanged; separates concerns.
+- Cons: creates a second "source of truth" for session state; the orchestrator must write and read two files atomically to maintain consistency; session resume becomes a two-file operation with a risk of partial-write inconsistency.
+
+### Option C — Store goal-loop state in memory only (no disk persistence)
+
+Keep goal-loop state in the orchestrator's context window, not on disk.
+
+- Pros: no schema changes; simple.
+- Cons: session resume is impossible — REQ-ORCH-022 requires state to survive session interruption. Directly contradicts NFR-ORCH-008.
+
+## Consequences
+
+### Positive
+
+- Session resume (REQ-ORCH-022) is reliable: all state needed to replay a HITL gate lives in `workflow-state.md`.
+- Stall detection (REQ-ORCH-014) is persistent across session restarts: `stall_counters` survive a process restart.
+- Pre-flight checks (REQ-ORCH-003) can use `artifacts_produced` to verify preconditions without filesystem stat calls on every field.
+- The schema extension is additive — no existing Specorator workflow or test is affected.
+
+### Negative
+
+- The Zod schema for `workflow-state.md` (ADR-0042) must be updated before implementation of REQ-ORCH-002 and REQ-ORCH-022 can begin. This is a blocking prerequisite.
+- Gate content embedded in the `workflow-state.md` body makes the file longer during a session. Mitigated by the fact that only one gate is pending at a time and old gate content can be cleared on gate resolution.
+
+### Neutral
+
+- The `workflow-state.md` Zod schema lives in the scripts layer (established by ADR-0042). The extension PR touches only the schema module and adds no new scripts.
+
+## Compliance
+
+- The Zod schema module (path established by ADR-0042) must be updated to include the `goal_loop` optional field group before the implementation phase of the goal-loop feature begins.
+- The orchestrator's system prompt must document which `goal_loop` sub-fields it writes at each phase transition.
+- `npm run verify` must include schema validation of any `workflow-state.md` file produced by the test suite.
+
+## References
+
+- PRD-ORCH-001 — REQ-ORCH-002, REQ-ORCH-003, REQ-ORCH-014, REQ-ORCH-022; NFR-ORCH-008
+- ADR-0042 — Typed artifact reader seam for frontmatter parsing (prerequisite schema)
+- DESIGN-ORCH-001 Part C — Data model section (workflow-state.md extended fields)
+- `specs/goal-oriented-orchestrator-plugin/research.md` §State management model
+
+---
+
+> **ADR bodies are immutable.** To change a decision, supersede it with a new ADR; only the predecessor's `status` and `superseded-by` pointer fields may be updated.