Skip to content

feat(search): Full-text search with SQLite FTS5 and Bleve backends#50

Merged
rgonek merged 6 commits intomainfrom
feature/full-text-search
Mar 3, 2026
Merged

feat(search): Full-text search with SQLite FTS5 and Bleve backends#50
rgonek merged 6 commits intomainfrom
feature/full-text-search

Conversation

@rgonek
Copy link
Owner

@rgonek rgonek commented Mar 3, 2026

Summary

  • Adds conf search QUERY [flags] command for heading-anchored, faceted full-text search over local Confluence Markdown workspaces
  • Two pluggable backends behind a shared Store interface: --engine sqlite (default, SQLite FTS5 via modernc.org/sqlite) and --engine bleve (Bleve scorch)
  • Index lives in .confluence-search-index/ (gitignored, local-only); updated automatically after each conf pull (non-fatal)

What's included

New packages

  • internal/search/document.go — shared Document, SearchOptions, SearchResult types
  • internal/search/store.goStore interface
  • internal/search/parser.go — Goldmark AST walker producing heading-anchored sections and code blocks
  • internal/search/indexer.go — backend-agnostic Indexer (full reindex, per-space, incremental mtime-based)
  • internal/search/sqlitestore/ — SQLite FTS5 backend (porter + unicode61 tokenizer, JSON label filtering)
  • internal/search/blevestore/ — Bleve scorch backend (disjunction query with field boosts)

CLI

  • cmd/search.gonewSearchCmd(), runSearch(), text/JSON formatters, updateSearchIndexForSpace() helper
  • cmd/search_test.go — command-level tests

Integration hooks

  • cmd/pull.go — non-fatal post-pull index update via updateSearchIndexForSpace()
  • cmd/clean.go — removes .confluence-search-index/ as part of clean
  • cmd/init.go — adds .confluence-search-index/ to gitignore template and ensureGitignore() entries
  • .gitignore — adds .confluence-search-index/ entry

Documentation

  • AGENTS.mdconf search command reference and recommended agent workflow
  • docs/usage.md — full conf search QUERY section with flag table and examples

Key flags

Flag Default Description
--space KEY Filter to a specific Confluence space
--label LABEL Filter by label (repeatable)
--heading TEXT Restrict to sections under matching headings
--engine sqlite Backend: sqlite or bleve
--format auto text, json, or auto (TTY→text, pipe→json)
--reindex false Force full index rebuild
--list-labels / --list-spaces Facet discovery

Verification

All tests pass (go test ./...), build is clean, go fmt applied.

Robert Gonek added 6 commits March 3, 2026 09:45
Add Document, SearchOptions, SearchResult types and the Store interface.
Add ParseMarkdownStructure() for Goldmark AST-based heading/code extraction.
Add sqlitestore.Store implementing search.Store with SQLite + FTS5.
Add schema DDL with triggers for FTS sync, and comprehensive tests.
Add Indexer type that orchestrates markdown file walking and calls Store.
Supports full reindex, per-space indexing, and mtime-based incremental updates.
…tion hooks, docs

- Add Bleve scorch backend (internal/search/blevestore)
- Add conf search command with sqlite/bleve backends, --list-labels/spaces, --format auto
- Register newSearchCmd() in root.go
- Phase 5: post-pull search index update in cmd/pull.go (non-fatal)
- Phase 5: remove .confluence-search-index/ in conf clean
- Phase 5: add .confluence-search-index/ to gitignore template (cmd/init.go) and .gitignore
- Phase 6: update AGENTS.md and docs/usage.md with conf search reference
@rgonek rgonek merged commit 80fed5e into main Mar 3, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant