Skip to content

Add Elasticsearch as optional search backend for library catalog #229

@jakebromberg

Description

@jakebromberg

Context

The library catalog search currently uses PostgreSQL pg_trgm (trigram similarity) via the library_artist_view database view. While functional, pg_trgm has limitations: it doesn't support stemming, synonym expansion, or sophisticated relevance ranking. Elasticsearch provides Lucene-powered full-text search with fuzzy matching, relevance tuning, and better handling of partial matches.

Plan

Add Elasticsearch as an optional search backend with graceful degradation to pg_trgm when ES is unavailable. PostgreSQL remains the source of truth.

PR 1 (this issue): ES client, index management, search service, facade with fallback, unit tests, Docker Compose, healthcheck

New files (apps/backend/services/search/):

  • elasticsearch.client.ts -- Singleton ES client. Returns null when ELASTICSEARCH_URL is unset.
  • elasticsearch.indices.ts -- Index mapping and lifecycle (ensureLibraryIndex() called at startup).
  • elasticsearch.search.ts -- ES query implementations matching existing function signatures.
  • elasticsearch.sync.ts -- Stub interfaces for dual-write sync (PR 2).
  • index.ts -- Facade: tries ES first, falls back to pg_trgm on error.

Modified files:

  • library.service.ts -- Renames pg_trgm functions with pgTrgm prefix, re-exports facade versions under original names.
  • app.ts -- Calls ensureLibraryIndex() at startup, expands /healthcheck with ES status.
  • apps/backend/package.json -- Adds @elastic/elasticsearch ^8.17.0.
  • dev_env/docker-compose.yml -- Adds ES services for dev and ci profiles.
  • scripts/ci-env.sh -- Optionally starts ci-elasticsearch.
  • .env.example and CLAUDE.md -- Documents new env vars and architecture.

Unit tests (tests/unit/services/search/):

  • elasticsearch.client.test.ts -- Client enable/disable, singleton, auth config.
  • elasticsearch.search.test.ts -- Query structure, result mapping, empty results.
  • search.facade.test.ts -- ES routing, fallback on error, direct pg_trgm when disabled.

PR 2 (follow-up): Dual-write sync from library write paths, bulk reindex job, integration tests with real ES container

Feature Flag

ELASTICSEARCH_URL env var:

  • Unset (default): pg_trgm only
  • Set (e.g., http://localhost:9200): ES with automatic fallback to pg_trgm
  • Instant rollback: unset the env var and restart

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions