refactor: remove data_sources_expanded view (#845)#956
Open
Conversation
Drops the data_sources_expanded SQL view and its DataSourceExpanded ORM shadow, replacing them with a column_property on DataSource that issues an equivalent correlated subquery for record_type_name. The agencies and data_requests relationships move onto DataSource directly. Removing the view eliminates the migration friction it caused (every data_sources schema change required dropping/recreating the view). Closes #845 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes #845.
Drops the
data_sources_expandedSQL view (and itsDataSourceExpandedSQLAlchemy shadow) so futuredata_sourcesschema changes no longer require dropping/recreating a view in every migration.The view's only added value over the
data_sourcestable was the joinedrecord_types.namecolumn. That is now exposed via a SQLAlchemycolumn_propertyonDataSourcethat issues a correlated subquery againstrecord_types, producing the samerecord_type_namevalue with the sameLEFT JOINsemantics (NULL whenrecord_type_idis unset).Replacements by call site
db/models/implementations/core/data_source/core.py— addedagenciesanddata_requestsrelationships (moved off the deletedDataSourceExpanded) and therecord_type_namecolumn_property.db/client/core.py,db/dynamic_query_constructor.py,endpoints/instantiations/data_sources_/get/{by_id,by_id/agencies,many}/query.py,endpoints/instantiations/data_sources_/get/convert.py— allselect(DataSourceExpanded)/selectinload(DataSourceExpanded.*)switched to theDataSourcemodel. No query semantics change; thecolumn_propertyinjects the sameLEFT JOIN record_typesprojection.db/subquery_logic.py,middleware/primary_resource_logic/{data_sources.py,data_requests_/related/related_source/get.py}—Relations.DATA_SOURCES_EXPANDEDreferences repointed toRelations.DATA_SOURCES.db/models/implementations/core/{agency/core.py,data_request/expanded.py}—data_sourcesrelationships now point atDataSourceinstead ofDataSourceExpanded.db/models/table_reference.py,middleware/enums.py,middleware/column_permission/mapping.py— removeddata_sources_expandedregistrations; mergedrecord_type_nameinto thedata_sourcespermission entry (same role/permission settings as the deleteddata_sources_expandedentry).tests/utilities/test_result_formatter.py— instantiatesDataSourcedirectly.alembic/versions/2026_05_02_0839-a3fcd7a36a70_drop_data_sources_expanded_view.pydrops the view inupgrade()and recreates it indowngrade()for reversibility.Test plan
uv run ruff check .— cleanuv run basedpyright --level error— 0 errorsuv run alembic upgrade headthendowngrade -1thenupgrade headagainst a local PG15 — view drops and re-creates as expecteduv run pytest tests/(all sub-suites) — 257 passed locally; sametests_comprehensivefailures (11 fail / 7 error) exist ondevbaseline and are unrelatedrun_pytest.ymlagainst full DBRisk areas for reviewer
column_propertyruns a correlated scalar subquery forrecord_type_namewhenever it is loaded. For the existing call sites this matches theLEFT JOINthe view used. If any caller diddata_sources_expanded.record_type_namefiltering in aWHEREclause it would still work (SQLAlchemy will inline the subquery), but no such filter exists in the current codebase — worth a quick second look.DataSourceExpandedDTO/DataSourceExpandedSchemanames were left in place since they describe the API output schema, not the database view. Renaming is out of scope.🤖 Generated with Claude Code