Skip to content

Fix artist_alias FK violation and add import dedup#45

Merged
jakebromberg merged 2 commits intomainfrom
worktree-import-dedup-filter
Mar 11, 2026
Merged

Fix artist_alias FK violation and add import dedup#45
jakebromberg merged 2 commits intomainfrom
worktree-import-dedup-filter

Conversation

@jakebromberg
Copy link
Member

Summary

  • Filter artist_alias and artist_member imports to only known artist IDs, preventing FK violations when converter CSVs contain all Discogs artists
  • Add unique_key dedup configs for artist_alias, artist_member, and release_track to prevent duplicate row errors during COPY
  • Add generic id_filter/id_filter_column mechanism to import_csv() for column-based row filtering

Closes #44

Test plan

  • Unit tests verify unique_key configs on artist_alias, artist_member, release_track
  • Unit test verifies import_artist_details passes artist_id_filter
  • All 467 unit tests pass
  • All 156 PostgreSQL integration tests pass
  • Lint passes (ruff format + ruff check)

Jake Bromberg added 2 commits March 11, 2026 13:02
…event FK violations

Artist alias and member CSVs from discogs-xml-converter contain all Discogs artists, but the artist table only has stub rows for filtered releases. This caused FK violations on artist_alias.artist_id during import. Now import_artist_details queries known artist IDs and filters artist_alias/artist_member rows during COPY. Also adds unique_key dedup configs for artist_alias, artist_member, and release_track to prevent duplicate row errors.
@jakebromberg jakebromberg merged commit 03680e7 into main Mar 11, 2026
3 checks passed
@jakebromberg jakebromberg deleted the worktree-import-dedup-filter branch March 11, 2026 20:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Fix artist_alias FK violation and add dedup for all importable tables

1 participant