fix(storage): skip corrupt pages in list_pages to prevent KB-wide crashes

[RealDiligent]

Summary

Root cause: list_pages() was the only bulk list_* path that did not use the _load_or_skip resilience pattern. A single corrupt or hand-edited pages/*.md file (missing frontmatter, bad YAML, mojibake) raised an unhandled exception and crashed vouch status, vouch lint, kb.search (substring/hybrid fallback), MCP kb.list_pages, and other core paths — while list_claims(), list_proposals(), etc. already skipped bad files with a warning.

Fix: Add _load_page_or_skip() mirroring _load_or_skip and use it in list_pages(). One regression test added alongside the existing claim test.

Impact: One bad page no longer takes down the entire KB listing surface; operators get a logged warning and the rest of the vault keeps working.

Reproduction (proof of crash on main)

From repo root on main (without this fix), after placing one corrupt page file:

$ python scripts/repro_list_pages_crash.py

=== 2. store.list_pages() ===
CRASH — ValueError: page file missing YAML frontmatter

=== 3. health.lint(store) ===
CRASH — ValueError: page file missing YAML frontmatter

=== 4. health.status(store) ===
CRASH — ValueError: page file missing YAML frontmatter

On this branch, the same script returns good-page and lint/status complete (bad file skipped with a warning).

Minimal manual repro:

(store.kb_dir / "pages" / "bad.md").write_text("not valid frontmatter")
store.list_pages()  # ValueError on main; skips on this branch

Realistic corruption / failure modes (today)

Mode	Example
Hand edit / Obsidian vault sync	User edits .vouch/pages/*.md and breaks frontmatter
Mojibake / encoding	Non-UTF-8 bytes in YAML (pyyaml ReaderError) — same class of bug _load_or_skip already guards for claims/sources
Legacy / invalid YAML	Partial file, bad indentation after manual merge
Pre-validator artifacts	Already surfaced for claims via invalid_claim in lint; pages lacked the equivalent skip-on-list

This PR does not introduce a new architectural layer — it brings list_pages() to parity with list_claims() / list_entities() / list_sources(). Agree v2.0 SQLite may change the storage model; until then files are the source of truth and one bad artifact should not 500 the review gate.

Risk / tradeoffs

• Corrupt pages are silently omitted from listings (same behavior as corrupt claims today). Direct get_page(id) still raises for a targeted read — intentional, matches get_claim semantics.

Test plan

• test_list_pages_skips_unreadable_file (new)
• Mirrors existing test_list_claims_skips_unreadable_file
• python scripts/repro_list_pages_crash.py

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: e3836f78-9704-42d2-871b-57baff2a9a9e

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

📝 Walkthrough

Walkthrough

Adds a helper function _load_page_or_skip in storage.py that catches parsing/read errors when loading page files and returns None instead of raising. KBStore.list_pages() is updated to use this helper, skipping malformed page files. A corresponding test validates this behavior.

Changes

Resilient Page Listing

Layer / File(s)	Summary
Page loader helper and list_pages integration src/vouch/storage.py	Adds _load_page_or_skip to catch YAML/validation/read errors and ValueError from _deserialize_page, returning None and logging a warning; list_pages() now filters pages through this helper instead of calling _deserialize_page directly.
Test coverage for skipped pages tests/test_storage.py	Adds test_list_pages_skips_unreadable_file, verifying list_pages() returns only valid pages when a corrupt page file is present.

Estimated code review effort: 1 (Trivial) | ~5 minutes

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly summarizes the main change: skipping corrupt pages in list_pages to avoid KB-wide crashes.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[plind-junior]

Is it really crashes? Attach the proof plz

[plind-junior]

With v2.0.0, we'll be moving to a database-backed approach (e.g., SQLite) instead of Markdown files. That said, I'm still curious—under what circumstances could claims or pages become corrupted in the current or future setup? Are there specific failure modes we should be proactively guarding against (e.g., concurrent writes, partial saves, file system errors, or migration mishaps)?

[RealDiligent]

Thanks for the review — answers below.

Re: "Is it really crashes? Attach the proof plz"

Yes. On main, one corrupt pages/*.md aborts every caller of list_pages() with an unhandled exception:

ValueError: page file missing YAML frontmatter

Affected paths (all call store.list_pages() internally):

• health.status() → vouch status
• health.lint() → vouch lint (orphan-page sweep iterates pages)
• MCP / JSONL kb.list_pages
• kb.search substring/hybrid fallback paths that enumerate pages

Why claims don't crash but pages do: list_claims() already uses _load_or_skip(); list_pages() was the outlier — it called _deserialize_page() directly with no try/except:

# main — list_pages()
return [_deserialize_page(p.read_text(...)) for p in sorted(pdir.glob("*.md"))]

Proof you can run:

git checkout main
python scripts/repro_list_pages_crash.py   # added in latest commit on this PR branch

Or minimal inline repro:

store.put_page(Page(id="good", title="ok", body="x"))
(store.kb_dir / "pages" / "bad.md").write_text("not valid frontmatter")
store.list_pages()  # ValueError on main

On this branch the same call logs skipping unreadable page bad.md: ... and returns [good].

Regression test: test_list_pages_skips_unreadable_file (mirrors existing test_list_claims_skips_unreadable_file).

---

Re: corruption failure modes / v2.0 SQLite

Agree v2.0 may move durable storage to SQLite. Until then, files are the source of truth and these are the realistic ways a page/claim file becomes unreadable today:

Failure mode	How it happens
Hand edit / vault sync	Obsidian or manual edit of .vouch/pages/*.md breaks frontmatter
Encoding / mojibake	Non-UTF-8 bytes in YAML → pyyaml ReaderError (same class already handled for claims via _load_or_skip)
Invalid / partial YAML	Bad merge, truncated save, wrong indentation
Legacy pre-validator artifacts	Claims: already surfaced as invalid_claim in lint; pages had no equivalent skip-on-list

Not primary targets for this PR: concurrent writes (create path uses open("x"); updates are last-write-wins), migration mishaps (migration runner has its own error surface).

This change is parity, not new architecture — the same skip-and-warn pattern already shipped for claims, entities, relations, evidence, sessions, sources, and proposals. One bad file shouldn't take the review gate offline while the operator fixes it.

Happy to rebase onto test if helpful.

[RealDiligent]

Rebased onto latest \ est\ (\�0674f1). Both commits replayed cleanly; \ est_list_pages_skips_unreadable_file\ and \scripts/repro_list_pages_crash.py\ verified locally.