fix(sync+embed+extract): code-symbol ingest + graph extraction (#767 + #769 + extract follow-ups)

[rayers]

Summary

Three related fixes for code-symbol ingest and graph extraction:

• sync --strategy code dropped on first sync via performFullSync #767 — gbrain sync --strategy code silently ran a markdown-only import on first sync.
• Code chunks land in DB with NULL language / symbol_name / symbol_type across all languages #769 — once code chunks land, every re-embed pass clobbered their metadata to NULL.
• Extract follow-ups (no separate issue filed yet) — gbrain doctor recommended non-existent commands; body wikilinks like [[bare-name]] were silently dropped because they fell outside DIR_PATTERN and the resolver wasn't routed through markdown extraction. Surfaced while diagnosing why post-Code chunks land in DB with NULL language / symbol_name / symbol_type across all languages #769 brains showed graph_coverage 0% even after running the (correct) extract command.

All three ship together: each is a layer of the same pipeline. Without #767 the chunks don't exist; without #769 their metadata is dead; without the extract fixes the graph stays empty for any brain not formatted in YC/CRM-style [Name](people/slug) markup.

#767 changes — first-sync strategy

src/commands/import.ts

• runImport now parses --strategy markdown|code|auto (defaults to markdown for backward compat).
• Both space-separated and equals-separated forms accepted.
• auto strategy is computed as a UNION of collectMarkdownFiles ∪ collectFilesByStrategy('code') — strict superset of legacy markdown coverage.
• New collectFilesByStrategy(dir, strategy) helper. Mirrors collectMarkdownFiles symlink + hidden-dir + node_modules safety.

src/commands/sync.ts

• performFullSync threads opts.strategy through to runImport's importArgs. Same fix in the dryRun branch.

#769 changes — metadata preservation across re-embed

src/commands/embed.ts

Three re-upsert call sites (embedOnePage, embedAll non-stale, embedAllStale autopilot path) were stripping the 8 metadata fields. New preserveCodeMetadata helper threads them through consistently.

src/core/postgres-engine.ts + src/core/pglite-engine.ts

upsertChunks ON CONFLICT clause was overwriting metadata columns from EXCLUDED unconditionally. Applied the chunk_text-gated CASE pattern to all 8 metadata columns (re-chunk → trust EXCLUDED, re-embed → COALESCE preserve), symmetric with the existing embedding/embedded_at/model pattern.

test/embed.serial.test.ts

Three regression tests cover all three runEmbed paths. Each fails on pre-fix code (8 fields come back undefined), passes on the patched code.

Extract follow-ups — doctor hint + body wikilinks

src/commands/doctor.ts

graph_coverage warning pointed users at gbrain link-extract && gbrain timeline-extract. Those commands don't exist. Updated to gbrain extract all.

src/core/link-extraction.ts

• New WIKILINK_GENERIC_RE matches [[anything]] without DIR_PATTERN whitelist gating.
• extractEntityRefs runs it as pass 2c with masked-out ranges from 2a/2b. Refs tagged needsResolution: true.
• extractPageLinks step 1 routes needsResolution refs through the resolver. Unresolvable refs silently drop instead of writing dangling rows. Tagged linkSource: 'wikilink-resolved'.
• makeResolver gains step 2.5: slug-tail match. Wikilink text 2026-05-07-cost-plan matches the tail of slug topics/dragon-pilot/raw/notes/2026-05-07-cost-plan even when the title diverges. One getAllSlugs per resolver instance, in-memory map lookups thereafter. Defensive — empty index if getAllSlugs throws or is missing (legacy/test mocks).
• extractPageLinks gains opts.skipFrontmatter so callers can control frontmatter resolution independently of body resolution. (Pre-fix nullResolver blocked both together.)

src/commands/extract.ts

DB-source path always passes the active batch resolver. --include-frontmatter flag now controls only the frontmatter pass via the new opt, not body resolution.

test/link-extraction.test.ts

Four new tests cover the new behavior:

• generic wikilinks emerge with needsResolution: true
• DIR_PATTERN refs stay untagged (no double-emit)
• qualified-syntax tokens ([[wiki:topics/x]]) skipped by the generic pass
• generic wikilinks resolve via resolver, unresolvable ones silently drop, resolved ones tag as wikilink-resolved
• skipFrontmatter opt suppresses frontmatter pass

Out-of-scope follow-up

FS-source extraction (extractLinksFromDir + extractLinksFromFile) uses a separate code path that doesn't go through extractEntityRefs or extractPageLinks. Body wikilinks in --source fs mode still won't resolve. Most users are on --source db (autopilot uses it); FS is for offline Obsidian-vault mode. Worth a follow-up PR but skipped here to keep scope contained.

Verification

#767 end-to-end against a 1.6GB Dividia NVR repo (~7000 files)

$ gbrain sources add gstack-code-nvr --path /repo --federated
$ gbrain sync --source gstack-code-nvr --strategy code --no-embed
Found 1882 code files
[import.files] 1882/1882 (100%) imported=1882 skipped=0 errors=0
  31933 chunks created

Was 0 before this patch.

#769 unit test reproduction

Pre-fix: 3 fails (8 metadata fields come back undefined for --stale, --all, --slugs paths).
Post-fix: 14 pass.

#769 live verification

On a 7180-page brain, ran gbrain reindex-code --force --no-embed --yes to backfill the 4875 existing code pages whose metadata had been clobbered. Result:

• 47,908 chunks now have language + symbol_type metadata (was ~42)
• Java symbol_name coverage 99.9%, Python 98.6%, Swift 87.3%
• 0 stale embeddings (incremental-chunking dedup hit 100% — chunk_text byte-identical)
• gbrain code-def DstreamView now returns 20 hits across Java + Swift sources (was 0)

Extract follow-up live verification

On the same 7180-page brain:

• gbrain doctor warning now reads Run: gbrain extract all (was the broken link-extract && timeline-extract recommendation).
• gbrain extract links --source db --dry-run produced 20+ real edges in the wiki/topic content. Pre-fix the same content produced 0.
• After running for real, 221 real edges landed in the links table (193 mentions, 14 documents, 14 documented_by).

Tests

• 13 new tests in test/collect-files-by-strategy.test.ts (sync --strategy code dropped on first sync via performFullSync #767)
• 3 new tests in test/embed.serial.test.ts (Code chunks land in DB with NULL language / symbol_name / symbol_type across all languages #769)
• 4 new tests in test/link-extraction.test.ts (extract follow-up)
• Pre-existing surface (import-walker, import-file, import-resume, sync-classifier-widening, sources-resync-recovery, skillpack-sync-guard, embed.serial, incremental-chunking, chunk-grain-fts, chunker-version-gate, parent-scope, pglite-engine, schema-verify, search-image-column, search-lang-symbol-kind, link-extraction, extract-db, link-extraction-code-refs, cycle-patterns, post-write-lint): 178 → 178, all pass.

Backfill required after #769 deploy

Existing brains where embed --stale already clobbered metadata need a chunker re-walk:

gbrain reindex-code --force --no-embed --yes

Cost: only chunks whose chunk_text actually changed get re-embedded. The chunker is deterministic and chunker_version hasn't changed, so most chunks pass the dedup check and reuse existing embeddings. On the verification brain above, 0 chunks needed re-embedding.

Adversarial review

Two rounds for #767 (Claude subagent in hostile-reviewer mode caught three HIGH issues — auto-strategy not strict superset, --strategy= form silently dropped, lone require() — all addressed in commit d8e79f7).

For #769 the fix mirrors the embedding column's existing chunk_text-gated CASE pattern verbatim — pattern-symmetry is the strongest correctness argument.

For the extract follow-up: the body wikilink path was carefully gated with masked-range tracking so existing DIR_PATTERN matches stay untagged (preserving back-compat for the 0.13 frontmatter behavior). The slug-tail match is defensive (graceful when getAllSlugs is missing) and dir-hint scoped to avoid cross-linking unrelated dirs.

Codex CLI review attempted on #767 round-1 (codex review --base master) but stalled on this repo size; Claude subagent review was more productive.

Out of scope (separate from this PR)

• gbrain sync --source <id> without --strategy doesn't consult cfg.strategy from the source's stored config. The --all path does. Pre-existing inconsistency.
• Java tree-sitter chunker still emits headers like [Java] path:line for some node types where extractSymbolName(node) returns null. With Code chunks land in DB with NULL language / symbol_name / symbol_type across all languages #769 fixed, the symbols that DO populate (Python class/method, TS function/class, Go func/type, Java class/method) survive across re-embed. Coverage tuning is a separate chunker concern.
• FS-source extract path doesn't use the new generic wikilink + resolver flow (see "Out-of-scope follow-up" above).
• Doctor's brain_score graph component is still a page-coverage metric. For brains that primarily ingest code + transcripts + structured learnings (no Obsidian-style cross-references), the score stays low even after this fix because the content style doesn't produce wikilinks. That's a metric/expectation mismatch, not an extraction bug. Worth its own discussion.

Environment

gbrain v0.30.1 (dffb607), engine = Postgres 16.13 (pgvector + pg_trgm), bun 1.3.10, macOS 26.3.1.

🤖 Generated with Claude Code

---

^{Need help on this PR? Tag @codesmith with what you need.}

• Let Codesmith autofix CI failures and bot reviews