fix(planner): reconcile system prompt with pre-fetched GitHub context#197
Merged
Merged
Conversation
… (refs #193) The Planner's system prompt unconditionally declared "ZERO tool access", causing the LLM to tell users it could not look up GitHub issues even when PR #195's deterministic pre-fetch had already injected the issue body as a secondary system message. Users reported the bug with "Review and implement the fix for GitHub Issue #113" — the Planner replied "I don't have access to GitHub" despite the issue content being in its context window. Changes: - _PLANNER_SYSTEM_PROMPT now explains the pre-fetch flow and explicitly forbids denying GitHub access when the PRE-FETCHED GITHUB CONTEXT block is present. - _format_github_context_block wraps the payload in clear start/end markers that the system prompt references by name. - _prefetch_github_refs_for_message logs a warning (not silent skip) when refs were detected but GITHUB_TOKEN is missing or the fetch returned zero items, so operators can diagnose environment issues. - Four new tests in TestPlannerSystemPromptReconciliation cover: the system prompt no longer contains "ZERO tool access", the context block carries start/end markers, empty inputs produce no block, and the new warning log fires when refs are detected without a token. Closes the AC #3 gap in issue #193 (Planner "fetches" GitHub issues when referenced). All other ACs were already satisfied by PRs #194-#196. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
15 tasks
Abernaughty
added a commit
that referenced
this pull request
Apr 17, 2026
…mote (refs #193) After PR #197 landed the system-prompt reconciliation, manual smoke revealed the pre-fetch still didn't run because the default owner/repo was never actually resolvable: `extract_github_refs` drops same-repo refs like "Issue #113" when default_owner/default_repo are empty, and the Planner only read them from `GITHUB_OWNER`/`GITHUB_REPO` env vars (undocumented). The dashboard's REMOTE repo picker never reached the Planner either — `PlannerStartRequest` had no `github_repo` field. End-to-end plumbing: - `PlannerStartRequest` now accepts `github_repo` + `workspace_type`. - Dashboard `startSession` sends both on REMOTE and LOCAL starts. - `create_planner_session` stores `github_repo` on the session. - When `github_repo` is not passed, the server auto-detects it by parsing `remote.origin.url` from the workspace's `.git/config` (SSH and HTTPS, with/without `.git` suffix, origin-only). - `_prefetch_github_refs_for_message` prefers session.github_repo, falling back to env vars for CLI/headless use. Anti-hallucination + operator ergonomics: - System prompt now explicitly forbids inventing issue/PR contents when the "=== PRE-FETCHED GITHUB CONTEXT ===" block is absent — the LLM must tell the user the context is missing instead of fabricating plausible-sounding bodies. - New loose `#\d+` heuristic fires a warning log when the message contains `#N` references but no default repo is resolvable, so operators get a breadcrumb instead of a silent drop. Tests: 12 new tests covering explicit-repo plumbing, git-remote auto-detect (SSH/HTTPS/non-origin/non-github), session-repo used for pre-fetch, env-var fallback, silent-drop warning, and the anti-hallucination system prompt. 73/73 planner tests pass; 205/205 related tests pass. Closes amended ACs #3a-#3e in #193. AC #3f (manual smoke on LOCAL and REMOTE) follows post-merge. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
6 tasks
Abernaughty
added a commit
that referenced
this pull request
Apr 17, 2026
…mote (refs #193) (#198) After PR #197 landed the system-prompt reconciliation, manual smoke revealed the pre-fetch still didn't run because the default owner/repo was never actually resolvable: `extract_github_refs` drops same-repo refs like "Issue #113" when default_owner/default_repo are empty, and the Planner only read them from `GITHUB_OWNER`/`GITHUB_REPO` env vars (undocumented). The dashboard's REMOTE repo picker never reached the Planner either — `PlannerStartRequest` had no `github_repo` field. End-to-end plumbing: - `PlannerStartRequest` now accepts `github_repo` + `workspace_type`. - Dashboard `startSession` sends both on REMOTE and LOCAL starts. - `create_planner_session` stores `github_repo` on the session. - When `github_repo` is not passed, the server auto-detects it by parsing `remote.origin.url` from the workspace's `.git/config` (SSH and HTTPS, with/without `.git` suffix, origin-only). - `_prefetch_github_refs_for_message` prefers session.github_repo, falling back to env vars for CLI/headless use. Anti-hallucination + operator ergonomics: - System prompt now explicitly forbids inventing issue/PR contents when the "=== PRE-FETCHED GITHUB CONTEXT ===" block is absent — the LLM must tell the user the context is missing instead of fabricating plausible-sounding bodies. - New loose `#\d+` heuristic fires a warning log when the message contains `#N` references but no default repo is resolvable, so operators get a breadcrumb instead of a silent drop. Tests: 12 new tests covering explicit-repo plumbing, git-remote auto-detect (SSH/HTTPS/non-origin/non-github), session-repo used for pre-fetch, env-var fallback, silent-drop warning, and the anti-hallucination system prompt. 73/73 planner tests pass; 205/205 related tests pass. Closes amended ACs #3a-#3e in #193. AC #3f (manual smoke on LOCAL and REMOTE) follows post-merge. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
_PLANNER_SYSTEM_PROMPTno longer declares "ZERO tool access" — it now explains the deterministic pre-fetch flow from PR feat(planner): GitHub pre-fetch + pass-through to orchestrator (#193 PR 2) #195 and tells the LLM never to deny GitHub access when the pre-fetched block is present._format_github_context_blockwraps the payload in explicit=== PRE-FETCHED GITHUB CONTEXT ===/=== END PRE-FETCHED GITHUB CONTEXT ===markers that the system prompt references by name._prefetch_github_refs_for_messagenow logs awarning(not a silent skip) when refs were detected butGITHUB_TOKENis missing or the fetch returned zero items.Closes the AC #3 gap in #193: "Planner fetches GitHub issues/PRs when referenced (no more 'what language is this?')". ACs #1, #2, #4–#9 were already satisfied by #194, #195, and #196.
Root cause
With "Review and implement the fix for GitHub Issue #113", the regex correctly matched,
fetch_refs_as_context_itemspopulatedsession.task_spec.github_context, and_format_github_context_blockinjected the body as a secondary system message. But the primary system prompt was static and kept telling the LLM it had zero tool access — the LLM followed the canonical instruction and apologized to the user instead of reading the injected block. The existing integration test attests/test_architect_two_phase.py::TestFixIssue113Integrationmocked the Planner LLM response, so it never exercised this contradiction.Test plan
uv run pytest tests/test_planner.py -v— 61/61 pass (4 new inTestPlannerSystemPromptReconciliation)uv run pytest tests/test_github_fetch.py tests/test_mcp_tools.py tests/test_architect_two_phase.py— 166/166 passuv run pytest tests/ -m "not integration"— only 2 pre-existing unrelated failures (test_workspace.py::test_from_env_defaults,test_gather_context.py::test_keyword_matching), confirmed onmain"Review and implement the fix for GitHub Issue #113."in the Planner and confirm it references the issue body instead of apologizing.Refs #193
🤖 Generated with Claude Code