Skip to content

feat(media_deletion): New approach for handling async deletion of attachments#1011

Merged
cloedu87 merged 10 commits intodevelopfrom
fix/media_cascade_deletion_improvements
Jan 5, 2026
Merged

feat(media_deletion): New approach for handling async deletion of attachments#1011
cloedu87 merged 10 commits intodevelopfrom
fix/media_cascade_deletion_improvements

Conversation

@cloedu87
Copy link
Copy Markdown
Collaborator

@cloedu87 cloedu87 commented Dec 17, 2025

Pull Request

Description

The cascade destruction and soft deletion of Files.Attachment resources is currently way to slow. The main reason is, that the ash cascade_destroy, which gets triggered when Records.Collection resources get deleted, are running in the application instead of the database and are therefore very slow, are prone to memory problems and are far away acceptable when we have to deal with huge datasets.

The approach here is, to add a Records.Collection reference to the Files.Attachment resource, so we can collect all referenced attachments with one query and soft delete them within the transaction of deleting the collection.
This brings:

  • less fuzz and application managed delete cascades
  • workload delegation for constraint based record deletion to the database, where it belongs
  • remaining soft-deletion for later async, oban-based hard-deletion

Related Issue

#979
#812
#970

Type of Change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation update
  • Refactoring (no functional changes)
  • Performance improvement
  • Test update
  • Build/CI update
  • Other (please describe):

@cloedu87 cloedu87 requested a review from sidler92 December 17, 2025 07:22
@cloedu87 cloedu87 self-assigned this Dec 17, 2025
@cloedu87 cloedu87 added enhancement New feature or request elixir Pull requests that update Elixir code db Changes affecting database layer labels Dec 17, 2025
Comment thread lib/data_aggregator/files/attachment/attachment.ex
Comment thread lib/data_aggregator/records/validation_response/helpers.ex
@cloedu87 cloedu87 marked this pull request as ready for review December 17, 2025 10:06
@cloedu87 cloedu87 changed the title New approach for handling async deletion of attachments feat(media_deletion): New approach for handling async deletion of attachments Jan 5, 2026
sidler92
sidler92 previously approved these changes Jan 5, 2026
Copy link
Copy Markdown
Collaborator

@sidler92 sidler92 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks nice 👍
Only one question about the attachment migrations.

cloedu87 and others added 2 commits January 5, 2026 14:18
* chore: dependency updates and changes to make authentication work w/ new versions
* chore: cleaning up test messages
* chore(deps): bump actions/cache from 4 to 5
@cloedu87 cloedu87 requested a review from sidler92 January 5, 2026 14:18
sidler92
sidler92 previously approved these changes Jan 5, 2026
Copy link
Copy Markdown
Collaborator

@sidler92 sidler92 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One last smalls change needed.
But looks very nice! 🥳

Comment thread lib/data_aggregator/records/import/changes/import_records.ex Outdated
@cloedu87 cloedu87 requested a review from sidler92 January 5, 2026 14:47
Copy link
Copy Markdown
Collaborator

@sidler92 sidler92 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@cloedu87 cloedu87 merged commit 9e8cb90 into develop Jan 5, 2026
8 checks passed
@cloedu87 cloedu87 deleted the fix/media_cascade_deletion_improvements branch January 5, 2026 16:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

db Changes affecting database layer elixir Pull requests that update Elixir code enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants