Merged
Conversation
skipi
force-pushed
the
mk/plumber/retention-policies
branch
4 times, most recently
from
7e03b42 to
3b83c30
Compare
skipi
force-pushed
the
mk/plumber/retention-policies
branch
from
dd35670 to
18191c5
Compare
Collaborator
Author
|
@codex review |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Collaborator
Author
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
skipi
force-pushed
the
mk/plumber/retention-policies
branch
from
aaac102 to
af9fec5
Compare
loadez
previously approved these changes
Dec 4, 2025
loadez
reviewed
Dec 4, 2025
skipi
force-pushed
the
mk/plumber/retention-policies
branch
from
af9fec5 to
0d49ae0
Compare
skipi
force-pushed
the
mk/plumber/retention-policies
branch
2 times, most recently
from
45b6883 to
fd8d40d
Compare
Collaborator
Author
|
@codex review |
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
skipi
force-pushed
the
mk/plumber/retention-policies
branch
from
5e4a69a to
1b87569
Compare
radwo
reviewed
Dec 11, 2025
skipi
force-pushed
the
mk/plumber/retention-policies
branch
from
1e97afc to
8f27f9c
Compare
skipi
force-pushed
the
mk/plumber/retention-policies
branch
from
04aaf0c to
21a9590
Compare
radwo
previously approved these changes
Dec 15, 2025
DamjanBecirovic
previously approved these changes
Dec 16, 2025
[db] Add expires_at column to pipeline_requests table [db] Add idx_pipeline_requests_retention_marking index (org_id, inserted_at WHERE expires_at IS NULL) [db] Add idx_pipeline_requests_retention_deletion index (expires_at WHERE expires_at IS NOT NULL) [db] Split migrations into 3 files for safer deployment [worker] Add PolicyApplier module with batched mark/unmark operations (default 10k batch size) [worker] Add PolicyConsumer to handle RabbitMQ retention policy events [config] Add RETENTION_GRACE_PERIOD_DAYS env var (default 15, min 7) [config] Add RETENTION_APPLIER_BATCH_SIZE env var (default 10000) [docs] Add commit format guidelines to AGENTS.md
[worker] Move retention worker startup to Application [config] Add RETENTION_CONSUMER_ENABLED env var (default false)
[worker] Combine SELECT and UPDATE into single query
[config] Move retention worker config to runtime.exs [config] Add default exchange (usage_internal_api) [config] Add default routing_key (usage.apply_organization_policy)
[worker] Fix record_deleter to use id instead of ppl_artefact_id [test] Add integration test for retention deletion
skipi
force-pushed
the
mk/plumber/retention-policies
branch
from
21a9590 to
d3ab56f
Compare
skipi
force-pushed
the
mk/plumber/retention-policies
branch
from
d3ab56f to
1e6a60a
Compare
DamjanBecirovic
approved these changes
Dec 16, 2025
marco2216
pushed a commit
to marco2216/semaphore
that referenced
this pull request
Dec 30, 2025
## 📝 Description This PR implements retention policy support in plumber. When the usage service applies a retention policy, plumber marks pipeline records for expiration with a configurable grace period, allowing corrections before data is deleted. - Added `expires_at` column to `pipeline_requests` table to track when records should be deleted - Created `PolicyConsumer` to receive retention policy events from usage service via RabbitMQ - Created `PolicyApplier` to mark/unmark pipeline records based on the cutoff date - Records older than cutoff are marked with `expires_at = now + grace_period` (default 15 days) - Records newer than the cutoff have `expires_at` cleared, allowing policy corrections to "save" mistakenly marked records - Grace period is configurable via `RETENTION_GRACE_PERIOD_DAYS` env var (min: 7 days) - Record Deleter Worker - Added a GenServer that periodically deletes expired records in batches ## ✅ Checklist - [x] I have tested this change - [ ] This change requires documentation update
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
📝 Description
This PR implements retention policy support in plumber. When the usage service applies a retention policy, plumber marks pipeline records for expiration with a configurable grace period, allowing corrections before data is deleted.
expires_atcolumn topipeline_requeststable to track when records should be deletedPolicyConsumerto receive retention policy events from usage service via RabbitMQPolicyApplierto mark/unmark pipeline records based on the cutoff dateexpires_at = now + grace_period(default 15 days)expires_atcleared, allowing policy corrections to "save" mistakenly marked recordsRETENTION_GRACE_PERIOD_DAYSenv var (min: 7 days)✅ Checklist