From d1c37b90861b8a04bdfbcea41a6660d2a42479a3 Mon Sep 17 00:00:00 2001 From: Dan Powers Date: Tue, 2 Jun 2026 13:57:25 -0500 Subject: [PATCH 01/10] docs(spec): #94 PR 2 of 2. spec/28 web search read_only + spec/35 amendments (MUST 4/5/7/11 + new MUST 15) spec/28 amends action class glosses: web search HTTP GETs are explicitly classified read_only because they do not change external state. Aligns with the framework's external_side_effect definition ('sending email, posting messages, anything the world sees'). Locked in PR 2 via AskUserQuestion gate (P1). Researcher template uses the same Cautious preset as advisor without paying judge_required overhead on every search query. spec/35 amendments: - MUST 4 expands to require path validation via safe_resolve_under AND fresh-write cleanup on failure AND staging-dir rule for Add-to-it - MUST 5 expands to the full Add-to-it staging-dir commit contract: render to .new., backup existing to .bak., rename .new to agent_dir, on success rmtree .bak / failure restore .bak. KI handler detects half-committed state and completes restoration. - MUST 7 carved out for --from-template and --list-templates per P3 AskUserQuestion: template scaffold writes file content only, no LLM call. CI users do not need ANTHROPIC_API_KEY at scaffold time. - MUST 11 amends entry guards: --list-templates MUST enumerate all --from-template choices and stay in sync across PRs. - New MUST 15: section-detection contract for Add-to-it. ATX h2 regex, skips fences + comments + frontmatter, fail-closed when schema does not match, backfill missing files, preserve operator orphan sections and h3+ subsections. spec/35 MUST count: 14 to 15 (sequential 1-15). Co-Authored-By: Claude Opus 4.7 --- docs/spec/28-judge-layer.md | 9 +++++- docs/spec/35-init-wizard.md | 57 ++++++++++++++++++++++++++++++------- 2 files changed, 55 insertions(+), 11 deletions(-) diff --git a/docs/spec/28-judge-layer.md b/docs/spec/28-judge-layer.md index a9ea533..cf4480a 100644 --- a/docs/spec/28-judge-layer.md +++ b/docs/spec/28-judge-layer.md @@ -705,11 +705,18 @@ Four classes, declared per-tool in `tools.md` (custom tools) or `mcp.md` (MCP to | Class | Examples | Default judge policy | |---|---|---| -| `read_only` | `read_file`, `search_notes`, `list_directory` | Bypass judge; no proposal required | +| `read_only` | `read_file`, `search_notes`, `list_directory`, web search (HTTP GET) | Bypass judge; no proposal required | | `reversible_write` | `write_note(staged)`, `create_draft` | Judge optional; default-allow with audit | | `external_side_effect` | `send_email`, `post_message`, `create_pr` | Judge required; default policy is judge-decides | | `high_risk` | `delete_files`, `force_push`, `production_deploy` | Judge required; default policy is escalate | +**Action class definitions:** + +- `read_only` -- Reading files, searching notes, listing directories, querying external read-only APIs (HTTP GET that does not change external state, like web search). Action is auditable but does not change persistent state. +- `reversible_write` -- Writing notes, drafting documents, staging work; the change can be undone via a restore path. +- `external_side_effect` -- Sending email, posting messages, creating pull requests; visible to the world outside the agent's vault. +- `high_risk` -- Deleting files, force-pushing code, deploying to production; irreversible or operationally consequential. + Class is the strongest input to outcome selection. Tools without an explicit class default to `external_side_effect` (safest classification for "we don't know"). Operators promote unknowns to `read_only` after observing them, via `tools.md` or `mcp.md`. ## Specialist judges diff --git a/docs/spec/35-init-wizard.md b/docs/spec/35-init-wizard.md index 68251ea..9618ddf 100644 --- a/docs/spec/35-init-wizard.md +++ b/docs/spec/35-init-wizard.md @@ -287,11 +287,26 @@ at the top. Tiebreaker for ambiguous order: alphabetical by issue number. `atomic_agents._io.safe_resolve_under(child, agent_dir)` before passing it to `atomic_write`. On a fresh-write failure (no pre-existing scaffold to restore), the wizard MUST clean up the partial `agent_dir` it created so - the operator sees either a complete scaffold or none of one. - -5. The collision Overwrite branch MUST use the backup+restore pattern: atomic - rename to `.bak.`, write all files, success rmtree the - `.bak`, failure rename back. + the operator sees either a complete scaffold or none of one. The Add-to-it + path additionally requires the wizard to render new scaffold content into a + sibling staging directory `.new.` before any + rename of the existing agent_dir. Staging-dir creation MUST use + `mkdir(parents=True, exist_ok=False)` so a stale staging dir from a prior + crashed run fails fast and triggers operator recovery. + +5. Recovery atomicity: The collision Overwrite branch MUST use the + backup+restore pattern: atomic rename to `.bak.`, + write all files, success rmtree the `.bak`, failure rename `.bak` back. The + collision Add-to-it branch MUST use the staging-dir commit pattern: render + the new scaffold under `.new.` while leaving + the existing `agent_dir` untouched; display a unified diff preview between + existing and staged content; on operator confirmation, atomically rename + `agent_dir` to `.bak.` then rename + `.new` to `agent_dir`, on success rmtree the `.bak`. On operator + decline or any failure between staging-dir creation and commit-rename, rmtree + the staging-dir and leave `agent_dir` untouched. The KeyboardInterrupt handler + MUST detect a half-committed state (`agent_dir` absent + `agent_dir.bak.*` + present + `agent_dir.new.*` present) and complete the restoration before exit. 6. The wizard MUST warn before any mkdir or file write when `ATOMIC_AGENTS_PERSONA_BACKEND_URL` is set non-empty. Decline MUST exit 0 @@ -300,10 +315,12 @@ at the top. Tiebreaker for ambiguous order: alphabetical by issue number. 7. The wizard MUST resolve the Anthropic API key via `atomic_agents._llm._get_key(env_vars=constants.ANTHROPIC_ENV_VARS, keychain_name=constants.ANTHROPIC_KEYCHAIN_NAME, - config_key=constants.ANTHROPIC_CONFIG_KEY)` at pre-flight on the paths that - may invoke the LLM (interactive Q&A and `--from-template`). The - `--list-templates` path MUST NOT require an API key (it writes no files and - makes no LLM calls). + config_key=constants.ANTHROPIC_CONFIG_KEY)` at pre-flight on the interactive + Q&A path. The `--from-template ` and `--list-templates` paths MUST NOT + require an API key at scaffold time because templates write file content only + with no LLM call. The opt-in test call at end of `--from-template` still + requires the key; when absent, the test-call prompt is skipped with a + one-line notice. 8. The wizard MUST call `atomic_agents.doctor.run_doctor()` on the new agent and MUST block the test-call prompt when @@ -331,7 +348,10 @@ at the top. Tiebreaker for ambiguous order: alphabetical by issue number. permitted. `agent_name` MUST be supplied; the wizard MUST refuse with a clear error if `--from-template` is given without `agent_name`. - `--list-templates`: no entry guards (read-only enumeration; no files - written, no LLM calls, no name required). + written, no LLM calls, no name required). `--list-templates` MUST + enumerate exactly the templates named in the `--from-template` argparse + choices list. The enumeration MUST stay in sync with `--from-template` + choices across all PRs that add or remove templates. 12. CHANGELOG `[Unreleased]` MUST interleave newest-arc-at-top with alphabetical-by-issue-number tiebreaker on conflict. @@ -350,6 +370,23 @@ at the top. Tiebreaker for ambiguous order: alphabetical by issue number. argparse `add_argument` calls with operator-facing help text on every argument, plus the subparser declaration and dispatch wiring). +15. Section-detection contract for Add-to-it: The wizard MUST detect existing + template-owned sections via ATX-style h2 header match (`^##\s+(.+)$`) + against `constants.TEMPLATE_SECTION_SCHEMA[template_name][file_relpath]`. + The section-detection parser MUST skip header-shaped lines inside code + fences (delimited by ` ``` ` or `~~~`), HTML comments (delimited by ``), and YAML frontmatter (delimited by `---` at file top). + Setext-style h2 headers (text followed by `------` underline) are NOT + supported; operators with setext-converted files MUST convert to ATX before + Add-to-it. When section-detection fails (file structure does not match + schema), the wizard MUST fail closed by offering Overwrite or Cancel only. + When a template-owned file is missing entirely, the wizard MUST backfill it + from the template; the diff-preview MUST label backfilled files as + `[new file]` and show full new content. Operator-authored h2 sections not + in the schema (orphan sections) and operator-authored h3+ subsections under + known h2 sections MUST be preserved verbatim in the rendered output, in + their original relative position. + --- ## Future work From 9ccbdc773d25f10d86288110be56bc2c453ee869 Mon Sep 17 00:00:00 2001 From: Dan Powers Date: Tue, 2 Jun 2026 13:57:25 -0500 Subject: [PATCH 02/10] feat(init): #94 PR 2 of 2. constants.py additions for templates + Add-to-it TEMPLATE_PRESET_DEFAULTS maps each template name to its locked autonomy preset (all three default to Cautious per the design decisions; comments explain the rationale). TEMPLATE_SECTION_SCHEMA is the per-template per-file h2 header schema that Add-to-it section detection validates against. Advisor schema includes 'Operating mode' (added to IDENTITY.md template in this PR per spec/01 conformance). Researcher and writer schemas populated with the actual h2 headers from the new template files. MAX_TEMPLATE_DEPTH defensive cap (16) for the iterative template walk added in this PR. redact_url_credentials helper drops user:pass@ from URL netloc while keeping scheme + host + path visible. The persona-backend warning at end of PR 2 uses this to show operators which backend is configured without leaking credentials. The existing _redact_for_error_message pattern in persona/mandate/corpus backends strips everything after :// which hid the host entirely; that defeated the operator-decision goal. MSG_NO_TTY rewrite drops the advisor-specific example; now points operators at --list-templates so they can pick from all three. Three new MSG_ constants for the new recovery paths: - MSG_SECTION_DETECTION_FAILED (Add-to-it fail-closed) - MSG_STAGING_DIR_EXISTS (stale staging from prior crashed run) - MSG_MISSING_FILE_BACKFILL (template-owned file absent on disk) Co-Authored-By: Claude Opus 4.7 --- atomic_agents/init/constants.py | 240 +++++++++++++++++++++++++++++++- 1 file changed, 238 insertions(+), 2 deletions(-) diff --git a/atomic_agents/init/constants.py b/atomic_agents/init/constants.py index ba0a220..66eb2e7 100644 --- a/atomic_agents/init/constants.py +++ b/atomic_agents/init/constants.py @@ -81,6 +81,201 @@ }, } +# Per-template autonomy preset defaults. Used by _default_template_vars in wizard.py +# when --from-template is invoked without going through the interactive Q&A. +# All three templates default to Cautious per the design decisions: +# - advisor: Cautious is the home-user-safe default per PR 1 +# - researcher: Cautious because web search APIs are classified read_only +# (spec/28 amended in PR 2), so the rare outbound action (email a summary) +# correctly escalates rather than going through judge_required overhead +# - writer: Cautious because publishing is a high-stakes external action that +# the operator must approve per draft +TEMPLATE_PRESET_DEFAULTS: Final[dict[str, str]] = { + "advisor": PRESET_CAUTIOUS, + "researcher": PRESET_CAUTIOUS, + "writer": PRESET_CAUTIOUS, +} + +# Section schema for Add-to-it recovery merge per spec/35 MUST 15. +# Maps template name -> file relpath -> ordered list of exact h2 header strings. +# The wizard's section-detection state machine compares an existing file's +# extracted h2 headers against this schema. When they match, the wizard +# offers Add-to-it. When they don't match, the wizard fails closed and +# offers Overwrite or Cancel only. +TEMPLATE_SECTION_SCHEMA: Final[dict[str, dict[str, list[str]]]] = { + "advisor": { + "persona/IDENTITY.md": [ + "Who I am", + "Mission", + "Scope", + "Operating doctrine", + "Operating mode", + "Autonomy ladder", + "What I'm NOT (the bright lines)", + ], + "persona/SOUL.md": [ + "Voice", + "Posture", + "Evolution discipline", + "Things I have learned about this operator", + ], + "persona/USER.md": [ + "Role and context", + "Communication preferences", + "Things to avoid", + "Supporting professionals (when to recommend outside help)", + ], + "tools.md": [ + "Read paths", + "Write paths (own folder ONLY)", + "External APIs", + "Hard NOs (absolute, no exceptions)", + "Soft NOs (require explicit operator override)", + "Read budget", + "Tool failure behavior", + ], + "model.md": [ + "Default model", + "Fallback", + "Token budget", + "Prompt caching strategy", + "Cost guardrail", + "Research integrity", + ], + "memory/INDEX.md": [ + "Critical Feedback", + "Locked Decisions", + "User Profile", + "Active Projects", + "Reference", + "Recently Promoted to Persona", + "Archive (superseded)", + ], + "wiki/INDEX.md": [ + "Background and context", + "Reference material", + "How wiki pages cite sources", + ], + }, + "researcher": { + "persona/IDENTITY.md": [ + "Who I am", + "Mission", + "Scope", + "Operating doctrine", + "Operating mode", + "Research integrity", + "Autonomy ladder", + "What I'm NOT (the bright lines)", + ], + "persona/SOUL.md": [ + "Voice", + "Posture", + "Evolution discipline", + "Things I have learned about this operator", + ], + "persona/USER.md": [ + "Role and context", + "Communication preferences", + "Things to avoid", + "Supporting professionals (when to recommend outside help)", + ], + "tools.md": [ + "Read paths", + "Write paths (own folder ONLY)", + "External APIs", + "Hard NOs (absolute, no exceptions)", + "Soft NOs (require explicit operator override)", + "Read budget", + "Tool failure behavior", + ], + "model.md": [ + "Default model", + "Fallback", + "Token budget", + "Prompt caching strategy", + "Cost guardrail", + "Research integrity", + ], + "memory/INDEX.md": [ + "Critical Feedback", + "Research Conclusions", + "User Profile", + "Active Investigations", + "Reference", + "Recently Promoted to Persona", + "Archive (superseded)", + ], + "wiki/INDEX.md": [ + "Source citations", + "Pending distillation", + "Background and context", + "Reference material", + "How wiki pages cite sources", + ], + }, + "writer": { + "persona/IDENTITY.md": [ + "Who I am", + "Mission", + "Scope", + "Operating doctrine", + "Operating mode", + "Autonomy ladder", + "What I'm NOT (the bright lines)", + ], + "persona/SOUL.md": [ + "Voice", + "Posture", + "Evolution discipline", + "Things I have learned about this operator", + ], + "persona/USER.md": [ + "Role and context", + "Communication preferences", + "Things to avoid", + "Revision and consistency preferences", + "Supporting professionals (when to recommend outside help)", + ], + "tools.md": [ + "Read paths", + "Write paths (own folder ONLY)", + "External APIs", + "Hard NOs (absolute, no exceptions)", + "Soft NOs (require explicit operator override)", + "Read budget", + "Tool failure behavior", + ], + "model.md": [ + "Default model", + "Fallback", + "Token budget", + "Prompt caching strategy", + "Cost guardrail", + "Research integrity", + ], + "memory/INDEX.md": [ + "Critical Feedback", + "Locked Decisions", + "User Profile", + "Active Projects", + "Reference", + "Recently Promoted to Persona", + "Archive (superseded)", + ], + "wiki/INDEX.md": [ + "Background and context", + "Reference material", + "How wiki pages cite sources", + ], + }, +} + +# Maximum directory depth the template walk will traverse. Defensive cap +# against future template trees that ship deeply nested structures. +# Current advisor template is 3 levels deep (templates//persona/IDENTITY.md). +MAX_TEMPLATE_DEPTH: Final[int] = 16 + # --------------------------------------------------------------------------- # agent_name validation # --------------------------------------------------------------------------- @@ -143,8 +338,9 @@ MSG_NO_TTY: Final = ( "This command needs an interactive terminal. For non-interactive use, run " - "`atomic-agents init --from-template advisor` to scaffold a Caldwell-shaped " - "agent. See `atomic-agents init --list-templates` for other options." + "`atomic-agents init --from-template