Skip to content

Elasticsearch dual-write sync, bulk reindex, and reindex endpoint #231

@jakebromberg

Description

@jakebromberg

Context

PR #230 added Elasticsearch as an optional search backend with a facade that routes to ES or pg_trgm. It left stub functions in elasticsearch.sync.ts for dual-write sync and bulk reindex. This issue tracks implementing those stubs, wiring dual-write into the library service layer, adding a management reindex endpoint, and providing unit tests.

Changes

elasticsearch.sync.ts — Replace stubs with implementations

  • indexLibraryDocumentById(albumId) — Queries library_artist_view by ID, indexes into ES. Wrapped in try/catch — logs errors, never throws.
  • removeLibraryDocument(id) — Deletes from ES by string ID. Ignores 404.
  • bulkIndexLibrary() — Calls ensureLibraryIndex(), reads all rows from library_artist_view, chunks into batches of 500, uses ES _bulk API. Returns { indexed, errors } counts.

library.service.ts — Dual-write calls

Fire-and-forget sync on:

  • insertAlbum — after insert, sync the new album
  • addToRotation — after insert, sync the album (rotation_bin changes)
  • killRotationInDB — after update, sync the album (kill_date removes rotation_bin)

Errors are logged but never propagated — library operations succeed even if ES is temporarily unavailable.

library.controller.ts + library.route.ts — Reindex endpoint

POST /library/reindex — requires catalog: ['write'] permission (musicDirector or stationManager). Calls bulkIndexLibrary() and returns { message, indexed, errors }.

Unit tests

  • 18 tests for sync functions (client null handling, 404 ignore, batch chunking, error counting, rotation_bin mapping)
  • 6 tests for dual-write behavior in library service (correct album ID passed, errors don't propagate)

What's NOT included

  • Integration tests with a real ES container
  • ETL job modification — operators should call POST /library/reindex after ETL
  • CI docker-compose changes — ES remains opt-in

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions