feat: add account_id filter for sources in search_beliefs#2065
Draft
feat: add account_id filter for sources in search_beliefs#2065
Conversation
- Add account_id column (FK to account.id, nullable) to DataSource - Add account relationship (backref 'data_sources' on Account) - Update __table_args__ UniqueConstraint to include account_id - Set self.account from user.account in __init__ when user is provided
- Add account_id column (nullable, FK to account.id) - Data migration: populate account_id from user's account where user_id IS NOT NULL - Update UniqueConstraint to include account_id column - down_revision: 8b62f8129f34
…ng sensor data Context: - DataSource.__init__ now sets self.account = user.account when a user is provided - The POST sensor data endpoint calls get_or_create_source(current_user) - Need to verify account_id is populated correctly on the resulting data source Change: - Added test_post_sensor_data_sets_account_id_on_data_source - Posts sensor data as test_supplier_user_4@seita.nl - Asserts data_source.account_id == user.account_id
… changelog entry Co-authored-by: Flix6x <30658763+Flix6x@users.noreply.github.com> Agent-Logs-Url: https://github.com/FlexMeasures/flexmeasures/sessions/7969297e-88c2-4ace-99c9-ec74109f03c8
… unflushed users Co-authored-by: Flix6x <30658763+Flix6x@users.noreply.github.com> Agent-Logs-Url: https://github.com/FlexMeasures/flexmeasures/sessions/7969297e-88c2-4ace-99c9-ec74109f03c8
…d invariants Context: - PR #2058 added account_id FK to DataSource table - No migration checklist existed for reviewing/writing Alembic migrations - New domain invariant (user-type DataSources now have account_id) was undocumented - Agent failed to self-update during the PR session Change: - Added Alembic migration checklist with safe backfill pattern - Added DataSource entity to Domain Knowledge with account_id invariant - Documented invariants #5 and #6 for DataSource UniqueConstraint and account_id - Added Lessons Learned section with PR #2058 case study
…essons Context: - PR #2058 added a test for account_id on DataSource after API POST - No guidance existed for testing data source properties after API calls - Agent failed to self-update during the PR session Change: - Added 'Testing DataSource Properties After API Calls' section with pattern - Added Lessons Learned section documenting PR #2058 self-improvement failure
…Coordinator failure Context: - PR #2058 session: Coordinator was not invoked (3rd session in a row) - API Specialist was not engaged despite endpoint behavior change - No agents updated their instructions (same failure as 2026-02-06) Change: - Added 'Agent Selection Checklist' mapping code change types to required agents - Documented PR #2058 as lessons learned (3 distinct failures) - Reinforced that Coordinator must ALWAYS be last agent in every session - Clarified that endpoint behavior changes → API Specialist must be engaged
…lure pattern Context: - PR #2058 repeated the same 3 failures from sessions 2026-02-06 and 2026-02-08 - Coordinator not invoked, no agent self-updates, API Specialist not engaged - Same failures now documented 3 times in Review Lead instructions with no change in behavior Change: - Added 'Additional Pattern Discovered (2026-03-24)' with root cause analysis - Documented missed API Specialist engagement for endpoint behavior change - Flagged governance escalation if Coordinator invocation fails a 4th time - Noted code observation: or-pattern for account_id and empty Initial plan commit
Co-authored-by: Flix6x <30658763+Flix6x@users.noreply.github.com> Agent-Logs-Url: https://github.com/FlexMeasures/flexmeasures/sessions/7969297e-88c2-4ace-99c9-ec74109f03c8
…a data source before POSTing sensor data Signed-off-by: F.N. Claessen <claessen@seita.nl>
…l time (UTC+1) Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
…lineage preservation Co-authored-by: Flix6x <30658763+Flix6x@users.noreply.github.com> Agent-Logs-Url: https://github.com/FlexMeasures/flexmeasures/sessions/004f0bb9-133d-45c1-9e06-2fedbf25b258
…n delete tests Context: - PR #2058 dropped FK constraints on DataSource.user_id and account_id to preserve lineage when users/accounts are deleted - The new tests only checked one of the two lineage fields per test Change: - test_delete_user: also assert data_source.account_id is preserved after user deletion (user deletion should not nullify account_id) - test_delete_account: also save and assert data_source.user_id per data source; account deletion internally calls delete_user so both user_id and account_id lineage must survive the full deletion chain
…eservation Context: - The existing entry only mentioned adding account_id to data_source - PR #2058 also drops FK constraints on user_id and account_id to preserve historical references when users/accounts are deleted (data lineage) Change: - Changelog now states both the column addition AND the behavior change: 'historical user/account IDs are no longer nullified when users or accounts are deleted'
…s lesson Context: - PR #2058 dropped FK constraints on DataSource.user_id and account_id for data lineage preservation (a deliberate design decision) - The original changelog entry omitted the FK drop and behavior change - No existing agent pattern covered 'intentional FK removal for lineage' Change: - Added checklist for no-FK lineage pattern (passive_deletes, tests, changelog) - Added lesson: changelog must describe behavior changes, not just schema changes - Documented FK drop design decision for future reviewers
…plementation The note previously described the 'or' anti-pattern as a concern, but the final code already uses 'if account_id is not None' correctly. Clarified to point to the implemented solution.
…ource' into copilot/add-account-id-filter-sources
Documentation build overview
Show files changed (9 files in total): 📝 9 modified | ➕ 0 added | ➖ 0 deleted
|
Add an account_id parameter (int or list[int]) to: - get_source_criteria() / source_account_criterion() in queries/utils.py - Sensor.search_beliefs() and TimedBelief.search() in models/time_series.py - GenericAsset.search_beliefs() in models/generic_assets.py - BeliefsSearchConfigSchema in schemas/reporting/__init__.py Allows reporters/schedulers/forecasters to filter beliefs by the account_id of their data source, implementing issue #2046.
Context: - Issue #2046 added account_id filtering to Sensor.search_beliefs and TimedBelief.search - No tests existed for the new account_id parameter Change: - Added setup_sources_with_accounts fixture in conftest.py Creates a sensor with beliefs from three sources: one per Prosumer/Supplier account and one without an account, enabling account_id filter testing
Signed-off-by: F.N. Claessen <claessen@seita.nl>
…either is deleted Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
…-account-id-filter-sources # Conflicts: # documentation/changelog.rst
…ource' into copilot/add-account-id-filter-sources
- Resolve conflicts in agent documentation files - Keep both DataSource and Annotation documentation sections - Accept deletion of lead.md from origin/main (governance consolidated) - Integrate annotation API implementation from main branch - All conflict markers resolved
Update all agent documentation to use 'Lead' instead of 'Review Lead', reflecting the consolidated governance model where lead.md has been removed and its responsibilities are now documented in AGENTS.md. Changes: - AGENTS.md: All 31 'Review Lead' references updated to 'Lead' - coordinator.md: Updated monitoring patterns to reference Lead - test-specialist.md: Updated coordination examples to reference Lead - tooling-ci-specialist.md: Updated coordination examples to reference Lead This ensures consistent terminology across all agent instructions.
Update b2e07f0dafa1_merge.py migration to specify both parent revisions that are being merged: - 3f4a6f9d2b11 (increase_audit_event_length_to_500) - b2c3d4e5f6a7 (Drop FK constraint from audit_log.active_user_id) This resolves the multiple heads issue and creates a proper mergepoint in the migration history. The migration now correctly shows as a (mergepoint) in the history output.
Clean up leftover git merge conflict markers (<<<<<<< HEAD, =======, >>>>>>> origin/main) that were not properly resolved. Both pattern sections (2026-03-24 and 2026-02-10) have been preserved as they document complementary learning from different sessions.
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Co-authored-by: Nicolas Höning <nicolas@seita.nl> Signed-off-by: Felix Claessen <30658763+Flix6x@users.noreply.github.com>
…7450113f6) Remove the intermediate a1b2c3d4e5f6 migration that was dropping the account_id FK (only created within this PR) and the user_id FK. Instead: - Don't create the account_id FK in 9877450113f6 in the first place - Absorb the user_id FK drop into 9877450113f6 - Update b2c3d4e5f6a7 to chain off 9877450113f6 directly - Delete a1b2c3d4e5f6 Agent-Logs-Url: https://github.com/FlexMeasures/flexmeasures/sessions/4d9d8a92-d4fd-4635-bebf-1965c35667b3 Co-authored-by: nhoening <1042336+nhoening@users.noreply.github.com>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
…main) Change 9877450113f6.down_revision from 8b62f8129f34 to e26d02ed1621 so the PR's two real migrations (9877450113f6, b2c3d4e5f6a7) chain linearly after the tip of the main branch, making the two merge-only files (b2e07f0dafa1 and 2c6a4c8127c1) completely obsolete. Delete b2e07f0dafa1_merge.py and 2c6a4c8127c1_merge.py. Agent-Logs-Url: https://github.com/FlexMeasures/flexmeasures/sessions/94c3070e-a701-4a97-a895-9ea1064e3c20 Co-authored-by: Flix6x <30658763+Flix6x@users.noreply.github.com>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
…ource' into copilot/add-account-id-filter-sources # Conflicts: # .github/agents/architecture-domain-specialist.md # .github/agents/coordinator.md # flexmeasures/data/schemas/account.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Sensor.search_beliefsand related methods could filter sources by id or type, but not by the account that owns the source. This addsaccount_idas a first-class filter parameter throughout the belief query stack.Core changes
data/queries/utils.py: Newsource_account_criterion()function;account_id: int | list[int] | Noneadded toget_source_criteria(). Accepts both integer IDs andAccountmodel instances. Empty list[]at thesearch_beliefslevel matches nothing (consistent with SQLIN ()semantics).data/models/time_series.py:account_idparameter threaded throughSensor.search_beliefs()andTimedBelief.search()data/models/generic_assets.py:account_idparameter added toGenericAsset.search_beliefs()data/schemas/account.py: NewAccountIdOrListFieldclass that natively acceptsint | list[int], validates each ID against the DB, rejects empty lists, and always deserializes tolist[Account].AccountIdField._deserializeupdated to use@with_appcontext_if_needed()instead of@with_appcontext, allowing the field to be used outside CLI contexts. The normalization is encapsulated in the field definition (rather than a schema-level@pre_loadhook) so future OpenAPI generation can emit aoneOfschema for it.data/schemas/reporting/__init__.py:account_id = AccountIdOrListField()added toBeliefsSearchConfigSchema— enables account filtering in reporter/scheduler/forecaster configurations. Both42and[42]are accepted; empty lists are rejected at the field level.data/schemas/io.py: Same field added toInputschema, which is whatPandasReporterandAggregatorReporteractually deserialize for theirinputlist entries.Usage
Original prompt
📱 Kick off Copilot coding agent tasks wherever you are with GitHub Mobile, available on iOS and Android.