Fix user profile address view and improve historical order logic#1
Open
jacobmpeters wants to merge 5 commits into
Open
Fix user profile address view and improve historical order logic#1jacobmpeters wants to merge 5 commits into
jacobmpeters wants to merge 5 commits into
Conversation
added 3 commits
May 5, 2026 12:35
…y-forward for NULL fields - Replace (element_position + 1) with ROW_NUMBER() OVER (PARTITION BY Connect_ID ORDER BY element_position DESC) in Queries 3, 4, and 6 so that historical_order = 1 is the most recent history entry, consistent with the reverse_chron_order convention in the demo - Wrap all six UNION ALL queries in an outer SELECT with a named WINDOW that applies LAST_VALUE(... IGNORE NULLS) to fill true NULLs in address_line_1, address_line_2, city, state, and zip_code using the most recent non-null value per (Connect_ID, address_nickname) - Empty strings are intentionally preserved as-is (Step 5 behavior); NULLIF/TRIM normalization continues to happen downstream in the standardized_addresses CTE in address_processing.py
extra copy of file
…ard logic to work
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request introduces important updates to the user profile address processing pipeline, focusing on how address history is handled and delivered prior to our second delivery to NORC. The changes correct the ordering of historical address entries, implement a carry-forward mechanism for NULL address fields, and provide clear documentation on the downstream impact. Additionally, a new
dry_run.pyscript is added to validate the generated SQL before execution in BigQuery.User Profile Address History Logic:
historical_orderfield inuser_profile_address_view.sqlto number history entries newest-first (ROW_NUMBER() ... ORDER BY element_position DESC), ensuringhistorical_order = 1is always the most recent history snapshot. Previously, entries were numbered oldest-first. [1] [2] [3]LAST_VALUE(... IGNORE NULLS) OVER wfor address fields (address_line_1,address_line_2,city,state,zip_code) to fill true NULLs from the nearest newer non-null entry per participant and address type. Empty strings are preserved and not filled. [1] [2]Documentation and Impact Analysis:
delivery_impact_of_change_to_up_hx.md) explaining the behavioral changes, their effect onaddress_hashcomputation, delivery state, and guidelines for merging returned geocoded data.Tooling and Validation:
core/dry_run.py, a script that renders the combined address view SQL, saves it for inspection, and performs a BigQuery dry run to validate the SQL without executing it. This helps catch errors before deployment.