Skip to content

Commit c0bfd0c

Browse files
committed
docs: sync docs/, README, and update-docs skill with v0.0.5 features
- docs/cli-reference.md: add --report junit, -step-setup, -step-teardown, --isolation flags and examples - docs/writing-runbooks.md: add sub-command separator section, fix negation description to word boundary matching - docs/advanced.md: add per-step hooks, per-runbook isolation, report formats (JSON array, JUnit), config fields (step_setup, step_teardown, isolation) - README.md: expand Features table and Key Concepts with all major features - update-docs skill: extend scope to cover docs/, README.md alongside skills/ and CHANGELOG.md, add documentation tiers and cross-tier consistency checks
1 parent 11138b5 commit c0bfd0c

5 files changed

Lines changed: 239 additions & 46 deletions

File tree

.skillshare/skills/update-docs/SKILL.md

Lines changed: 121 additions & 37 deletions
Original file line numberDiff line numberDiff line change
@@ -1,22 +1,32 @@
11
---
22
name: mdproof-update-docs
33
description: >-
4-
Update skills/ documentation and CHANGELOG to match recent code changes,
5-
cross-validating every flag and config field against Go source. Use this skill
6-
whenever the user asks to: update docs, sync docs with code, document a new
7-
flag or config option, fix stale docs, or refresh the skills reference after
8-
implementing a feature. This skill covers skills/SKILL.md (main doc),
9-
skills/references/ (assertions, advanced features), and CHANGELOG.md. If you
10-
just implemented a feature and need to update documentation, this is the skill
11-
to use. Never manually edit skills/ docs without cross-validating against Go
12-
source first.
4+
Update all documentation (skills/, docs/, README.md, CHANGELOG.md) to match
5+
recent code changes, cross-validating every flag and config field against Go
6+
source. Use this skill whenever the user asks to: update docs, sync docs with
7+
code, document a new flag or config option, fix stale docs, or refresh
8+
documentation after implementing a feature. This skill covers: docs/ (user-facing
9+
guides: cli-reference.md, writing-runbooks.md, advanced.md), skills/SKILL.md
10+
(AI agent skill), skills/references/ (assertions, advanced features),
11+
README.md (project overview), and CHANGELOG.md. If you just implemented a
12+
feature and need to update documentation, this is the skill to use. Never
13+
manually edit docs without cross-validating against Go source first.
1314
argument-hint: "[feature-name | commit-range]"
1415
targets: [claude, codex]
1516
---
1617

17-
Sync skills/ documentation with recent code changes. $ARGUMENTS specifies scope: a feature name (e.g., `step-setup`), commit range, or omit to auto-detect from `git diff HEAD~1`.
18+
Sync all documentation with recent code changes. $ARGUMENTS specifies scope: a feature name (e.g., `step-setup`), commit range, or omit to auto-detect from `git diff HEAD~1`.
1819

19-
**Scope**: This skill updates `skills/`, `CHANGELOG.md`. It does NOT write Go code (use `implement-feature`).
20+
**Scope**: This skill updates `docs/`, `skills/`, `README.md`, `CHANGELOG.md`. It does NOT write Go code (use `implement-feature`).
21+
22+
**Documentation tiers** (update all that are affected):
23+
24+
| Tier | Path | Audience | What lives here |
25+
|------|------|----------|-----------------|
26+
| User guides | `docs/` | End users, GitHub visitors | CLI reference, runbook authoring, advanced features |
27+
| AI skill | `skills/` | AI agents (Claude, Codex) | Compact reference for agent consumption |
28+
| Project overview | `README.md` | Everyone | Feature highlights, install, quick start |
29+
| Release notes | `CHANGELOG.md` | Users tracking versions | Use the `changelog` skill instead |
2030

2131
## Workflow
2232

@@ -35,7 +45,22 @@ git diff HEAD~1 -- internal/core/types.go
3545

3646
Map changed files to affected documentation:
3747

38-
**Main skill doc** (`skills/SKILL.md`):
48+
**User guide — CLI reference** (`docs/cli-reference.md`):
49+
- `cmd/mdproof/main.go` → Flags table, Examples section, Subcommands
50+
- New flags or subcommands must appear here
51+
52+
**User guide — Writing runbooks** (`docs/writing-runbooks.md`):
53+
- `internal/parser/` → step heading format, code block handling
54+
- `internal/executor/session.go` → persistent session, sub-commands
55+
- `internal/assertion/` → assertion types and syntax
56+
57+
**User guide — Advanced features** (`docs/advanced.md`):
58+
- `internal/executor/session.go` → hooks, sub-commands, execution model
59+
- `internal/config/config.go` → Configuration section, all config fields
60+
- `internal/report/` → report format changes (JSON, JUnit, plain)
61+
- `cmd/mdproof/main.go` → new flags → new sections (isolation, coverage, etc.)
62+
63+
**AI skill doc** (`skills/SKILL.md`):
3964
- `cmd/mdproof/main.go` → Quick Reference, CLI Flags table
4065
- `internal/core/types.go` → Writing Runbooks section (new step fields, report structure)
4166
- `internal/config/config.go` → mention config support in CLI Flags table
@@ -51,6 +76,9 @@ Map changed files to affected documentation:
5176
- `cmd/mdproof/main.go` → new flags → new sections
5277
- `internal/report/` → report format changes (plain, JSON, JUnit)
5378

79+
**Project overview** (`README.md`):
80+
- Any new user-visible feature → Features table, Key Concepts section
81+
5482
**CHANGELOG** (`CHANGELOG.md`):
5583
- Any user-visible change → use the `changelog` skill instead
5684

@@ -73,16 +101,58 @@ For each affected area, verify docs match source:
73101
grep -n 'json:"' internal/core/types.go
74102
```
75103

76-
4. Compare each against what's documented in `skills/SKILL.md` and `skills/references/`:
77-
- **New flag in code** → add to CLI Flags table + Quick Reference
78-
- **New config field** → add to Configuration File section in advanced-features.md
79-
- **Removed flag/field** → remove from docs
80-
- **Changed behavior** → update description
104+
4. Compare each against what's documented in `docs/`, `skills/`, and `README.md`:
105+
- **New flag in code** → add to `docs/cli-reference.md` Flags table + `skills/SKILL.md` CLI Flags table + Quick Reference
106+
- **New config field** → add to `docs/advanced.md` Configuration section + `skills/references/advanced-features.md` Configuration File section
107+
- **New execution behavior** → add to `docs/writing-runbooks.md` + `skills/SKILL.md` Writing Runbooks
108+
- **New major feature** → add to `README.md` Features table + Key Concepts
109+
- **Removed flag/field** → remove from all docs
110+
- **Changed behavior** → update description in all affected docs
81111
- **Every `--flag` / `-flag` in docs** must have a matching hit in source
82112

83113
### Step 3: Update Documentation
84114

85-
Apply changes following the established structure:
115+
Apply changes following the established structure. Update **all tiers** that are affected — a new flag needs to appear in `docs/cli-reference.md`, `skills/SKILL.md`, and possibly `README.md`.
116+
117+
#### docs/cli-reference.md structure:
118+
119+
| Section | What to update |
120+
|---------|---------------|
121+
| Flags table | Must match `cmd/mdproof/main.go` flag definitions exactly |
122+
| Subcommands | New subcommands (sandbox, upgrade) |
123+
| Examples | Add usage examples for new flags |
124+
125+
#### docs/writing-runbooks.md structure:
126+
127+
| Section | What to update |
128+
|---------|---------------|
129+
| Code Blocks | New execution behaviors (sub-commands, separators) |
130+
| Persistent Shell Session | Changes to env persistence model |
131+
| Assertions | New assertion types or changed behavior |
132+
| Directives | New HTML comment directives |
133+
| Inline Testing | Changes to `--inline` mode |
134+
135+
#### docs/advanced.md structure:
136+
137+
| Section | What to update |
138+
|---------|---------------|
139+
| Hooks | Per-runbook hooks + per-step hooks |
140+
| Configuration | `mdproof.json` fields — must match Config struct |
141+
| Per-Runbook Isolation | Isolation mode changes |
142+
| Report Formats | JSON, JUnit, plain text changes |
143+
| Coverage | `--coverage` mode |
144+
| Watch Mode | `--watch` mode |
145+
| Container Safety | Strict mode, sandbox changes |
146+
| CI Integration | CI usage patterns |
147+
148+
#### README.md structure:
149+
150+
| Section | What to update |
151+
|---------|---------------|
152+
| Features table | New major features (For AI Agents / For Humans) |
153+
| Key Concepts | New concepts users should know about |
154+
| Assertions table | New assertion types |
155+
| Quick Start | Only if the basic workflow changes |
86156

87157
#### skills/SKILL.md structure:
88158

@@ -123,13 +193,15 @@ Apply changes following the established structure:
123193

124194
### Step 4: Consistency Checks
125195

126-
After updating, verify cross-references are consistent:
196+
After updating, verify cross-references are consistent across all doc tiers:
127197

128-
1. **Assertion count**`skills/SKILL.md` says "Six types" → count actual types in assertions-guide.md
129-
2. **Config example** — JSON example in advanced-features.md must include all Config struct fields
130-
3. **Flag table** — CLI Flags in SKILL.md must be a complete subset of flags in main.go
131-
4. **Quick Reference** — examples must use correct flag ordering (flags before file path — Go's `flag` package requirement)
132-
5. **Report fields** — any jq assertion examples must reference fields that actually exist in the JSON report
198+
1. **Flag completeness** — every flag in `cmd/mdproof/main.go` must appear in both `docs/cli-reference.md` and `skills/SKILL.md`
199+
2. **Config completeness** — every `json:"..."` field in Config struct must appear in both `docs/advanced.md` and `skills/references/advanced-features.md` config examples
200+
3. **Assertion count**`skills/SKILL.md` says "Six types" → count actual types in assertions-guide.md and `docs/writing-runbooks.md`
201+
4. **Cross-tier consistency**`docs/` and `skills/` must not contradict each other (e.g., different flag descriptions, different default values)
202+
5. **README coverage** — every major feature in Key Concepts should have a corresponding section in `docs/advanced.md`
203+
6. **Flag ordering** — all CLI examples across all docs must put flags before file paths (Go's `flag` package requirement)
204+
7. **Report fields** — any jq assertion examples must reference fields that actually exist in the JSON report
133205

134206
### Step 5: Verify
135207

@@ -149,36 +221,48 @@ List all changes made with rationale:
149221
== Documentation Updates ==
150222
151223
Modified:
224+
docs/cli-reference.md
225+
- Added -step-setup/-step-teardown to Flags table
226+
- Added per-step hooks example
227+
228+
docs/advanced.md
229+
- Added Per-Step Setup/Teardown section
230+
- Updated Configuration table with step_setup/step_teardown
231+
152232
skills/SKILL.md
153233
- Added -step-setup/-step-teardown to CLI Flags table
154-
- Added sub-command separator section to Writing Runbooks
155234
156235
skills/references/advanced-features.md
157-
- Added Per-Step Setup/Teardown section
158236
- Updated Configuration File example with step_setup/step_teardown
159237
238+
README.md
239+
- Added per-step hooks to Features table and Key Concepts
240+
160241
No code changes.
161242
```
162243

163244
## Source-to-Doc Mapping Quick Reference
164245

165-
| Source file | Doc file | What to check |
166-
|------------|----------|---------------|
167-
| `cmd/mdproof/main.go` | `skills/SKILL.md` | CLI Flags table, Quick Reference |
168-
| `internal/config/config.go` | `skills/references/advanced-features.md` | Configuration File section |
169-
| `internal/core/types.go` | `skills/references/assertions-guide.md` | Report field references in jq examples |
170-
| `internal/assertion/` | `skills/references/assertions-guide.md` | Assertion types and behavior |
171-
| `internal/executor/session.go` | `skills/references/advanced-features.md` | Execution model, hooks, sub-commands |
172-
| `internal/report/plain.go` | `skills/references/advanced-features.md` | Plain text report behavior |
173-
| `internal/report/junit.go` | `skills/references/advanced-features.md` | JUnit report behavior |
246+
| Source file | Doc files to update | What to check |
247+
|------------|---------------------|---------------|
248+
| `cmd/mdproof/main.go` | `docs/cli-reference.md`, `skills/SKILL.md` | Flags table, Quick Reference, Examples |
249+
| `internal/config/config.go` | `docs/advanced.md`, `skills/references/advanced-features.md` | Configuration section, config JSON example |
250+
| `internal/core/types.go` | `docs/writing-runbooks.md`, `skills/references/assertions-guide.md` | Report field references in jq examples |
251+
| `internal/assertion/` | `docs/writing-runbooks.md`, `skills/references/assertions-guide.md` | Assertion types, syntax, behavior |
252+
| `internal/executor/session.go` | `docs/writing-runbooks.md`, `docs/advanced.md`, `skills/references/advanced-features.md` | Execution model, hooks, sub-commands, persistent session |
253+
| `internal/report/` | `docs/advanced.md`, `skills/references/advanced-features.md` | Report format behavior (JSON, JUnit, plain) |
254+
| `internal/parser/` | `docs/writing-runbooks.md`, `skills/SKILL.md` | Step headings, code block handling, inline parsing |
255+
| Any new feature | `README.md` | Features table, Key Concepts |
174256
| `CHANGELOG.md` | (use `changelog` skill) | User-facing release notes |
175257

176258
## Rules
177259

178260
- **Source of truth is Go code** — docs must match what the code actually does
179261
- **Every flag/config claim must be verified** — grep source before writing docs
262+
- **Update all tiers** — a new flag must appear in `docs/`, `skills/`, and possibly `README.md`
180263
- **No speculative docs** — never document planned but unimplemented features
181-
- **No code changes** — this skill only touches `skills/` and `CHANGELOG.md`
182-
- **Preserve style** — match existing doc structure and tone
264+
- **No code changes** — this skill only touches documentation files
265+
- **Preserve style** — match existing doc structure and tone per tier
183266
- **Flag ordering** — all CLI examples must put flags before file paths
184267
- **Config field names** — use the exact `json:"..."` tag from Config struct (snake_case)
268+
- **`docs/` vs `skills/`**`docs/` is for human end users (more prose, examples, mermaid diagrams); `skills/` is for AI agents (compact, structured, precise)

README.md

Lines changed: 16 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -138,9 +138,13 @@ mdproof sandbox api-proof.md # auto-provisions a container
138138
**For Humans**
139139
- Documentation IS the test — no context switching
140140
- Readable — anyone can understand what's being tested
141-
- Lifecycle hooks — build, setup, teardown
141+
- Lifecycle hooks — `--build`, `--setup`/`--teardown`, `-step-setup`/`-step-teardown`
142142
- Container-first — safe by default, sandbox mode
143-
- Persistent sessions — env vars flow across steps
143+
- Persistent sessions — env vars flow across steps and `---` sub-commands
144+
- Per-runbook isolation — `--isolation per-runbook` for clean `$HOME`/`$TMPDIR`
145+
- Step filtering — `--steps 1,3`, `--from N`, `--fail-fast`
146+
- Coverage & watch — `--coverage` for CI gating, `--watch` for live re-runs
147+
- Inline testing — `--inline` validates code examples in any `.md`
144148
- Zero dependencies — pure Go stdlib, single binary
145149

146150
</td>
@@ -223,10 +227,17 @@ No `Expected:` section → exit code decides (0 = pass).
223227

224228
### Key Concepts
225229

226-
- **Persistent session** — all steps share one bash process; `export` vars persist across steps
230+
- **Persistent session** — all steps share one bash process; `export` vars persist across steps and `---` sub-commands
227231
- **Container-first** — strict mode (default) refuses to run outside containers; use `mdproof sandbox` or `--strict=false`
228-
- **Hooks**`--build` (once), `--setup` / `--teardown` (per runbook) for lifecycle management
229-
- **Directives**`<!-- runbook: timeout=30s retry=3 -->` for per-step control
232+
- **Hooks**`--build` (once), `--setup` / `--teardown` (per runbook), `-step-setup` / `-step-teardown` (per step)
233+
- **Directives**`<!-- runbook: timeout=30s retry=3 depends=2 -->` for per-step control
234+
- **Sub-commands**`---` separator splits a code block into independent subshells with shared env
235+
- **Isolation**`--isolation per-runbook` gives each runbook a fresh `$HOME` and `$TMPDIR`
236+
- **Step filtering**`--steps 1,3,5` or `--from N` to run a subset; `--fail-fast` to stop early
237+
- **Coverage**`--coverage` reports assertion coverage; `--coverage-min N` for CI gating
238+
- **Watch mode**`--watch` re-runs tests on file changes
239+
- **Inline testing**`--inline` extracts `<!-- mdproof:start/end -->` blocks from any `.md`
240+
- **Snapshots**`snapshot:` assertions with `-u` to update
230241

231242
## Documentation
232243

docs/advanced.md

Lines changed: 64 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -78,9 +78,26 @@ flowchart LR
7878
| `build` | Once, before all runbooks | Abort — nothing runs |
7979
| `setup` | Per runbook, before steps | All steps skipped |
8080
| `teardown` | Per runbook, after steps | Informational only |
81+
| `step-setup` | Per step, before step body | Step marked failed, body skipped |
82+
| `step-teardown` | Per step, after step body | Informational only |
8183

8284
Setup and teardown run inside the same bash session as the steps, so they share environment variables. Build runs as a separate process.
8385

86+
### Per-Step Setup/Teardown
87+
88+
Distinct from per-runbook hooks, these run before/after **each step**:
89+
90+
```bash
91+
mdproof -step-setup 'rm -rf /tmp/test-state && mkdir -p /tmp/test-state' test.md
92+
mdproof -step-teardown 'echo step done' test.md
93+
mdproof -step-setup 'reset-db' -step-teardown 'dump-logs' test.md
94+
```
95+
96+
- Step-setup stdout is **not** mixed into step stdout
97+
- JSON report includes `step_setup` and `step_teardown` objects with `exit_code`, `stdout`, `stderr`
98+
- When neither flag is provided, no `step_setup`/`step_teardown` fields appear in the report
99+
- With retry (`<!-- runbook: retry=N -->`), each attempt runs the full cycle: setup → body → teardown
100+
84101
## Configuration
85102

86103
Create `mdproof.json` in the runbook directory:
@@ -90,8 +107,11 @@ Create `mdproof.json` in the runbook directory:
90107
"build": "make build",
91108
"setup": "docker-compose up -d",
92109
"teardown": "docker-compose down",
110+
"step_setup": "rm -rf /tmp/test-state",
111+
"step_teardown": "echo step done",
93112
"timeout": "5m",
94113
"strict": false,
114+
"isolation": "per-runbook",
95115
"env": {
96116
"LOG_LEVEL": "debug",
97117
"API_URL": "http://localhost:8080"
@@ -104,8 +124,11 @@ Create `mdproof.json` in the runbook directory:
104124
| `build` | string | Command to run once before all runbooks |
105125
| `setup` | string | Command to run before each runbook |
106126
| `teardown` | string | Command to run after each runbook |
127+
| `step_setup` | string | Command to run before each step |
128+
| `step_teardown` | string | Command to run after each step |
107129
| `timeout` | string | Default per-step timeout (e.g. `"2m"`, `"30s"`) |
108130
| `strict` | boolean | Container-only execution (default: `true`) |
131+
| `isolation` | string | `"shared"` (default) or `"per-runbook"` |
109132
| `env` | object | Environment variables seeded into all steps |
110133

111134
Sandbox settings can also be configured:
@@ -122,6 +145,46 @@ Sandbox settings can also be configured:
122145

123146
CLI flags override config file values.
124147

148+
## Per-Runbook Isolation
149+
150+
By default, all runbooks share the host's `$HOME` and `$TMPDIR`. With `--isolation per-runbook`, each runbook gets a fresh temp directory as `$HOME` with `$TMPDIR` under `$HOME/tmp`, cleaned up after each runbook:
151+
152+
```bash
153+
mdproof --isolation per-runbook ./runbooks/
154+
```
155+
156+
Or in `mdproof.json`:
157+
158+
```json
159+
{ "isolation": "per-runbook" }
160+
```
161+
162+
- Build hook (`--build`) runs in the original environment — not affected by isolation
163+
- Setup/teardown hooks inherit the isolated `$HOME`/`$TMPDIR`
164+
- Invalid values produce an error at config load time
165+
- CLI `--isolation` overrides the config file value
166+
167+
## Report Formats
168+
169+
### JSON
170+
171+
```bash
172+
mdproof --report json test.md # single object to stdout
173+
mdproof --report json ./runbooks/ # JSON array to stdout
174+
mdproof -o results.json ./runbooks/ # always writes JSON array to file
175+
```
176+
177+
Single-file mode outputs one JSON object; directory mode outputs a JSON array.
178+
179+
### JUnit XML
180+
181+
```bash
182+
mdproof --report junit ./runbooks/ # stdout
183+
mdproof --report junit -o results.xml ./runbooks/ # file
184+
```
185+
186+
Produces JUnit XML for native CI test result display (GitHub Actions, GitLab CI, Jenkins). Sub-command exit codes and stderr are included in failure bodies.
187+
125188
## Coverage
126189

127190
Analyze assertion coverage of your runbooks without executing them:
@@ -251,7 +314,7 @@ Once installed, your AI agent can autonomously:
251314
252315
- Write runbook files with correct naming (`*-proof.md`, `*_runbook.md`)
253316
- Use all 6 assertion types (substring, exit_code, regex, jq, snapshot, negated)
254-
- Configure hooks (build, setup, teardown) and `mdproof.json`
317+
- Configure hooks (build, setup, teardown, step-setup, step-teardown) and `mdproof.json`
255318
- Apply directives (timeout, retry, depends)
256319
- Handle container safety (`MDPROOF_ALLOW_EXECUTE=1`)
257320
- Run with the right flags and interpret results

0 commit comments

Comments
 (0)