Fix UC calibration: combine OBR targets, add DWP payment bands#31
Open
nikhilwoodruff wants to merge 3 commits intomainfrom
Open
Fix UC calibration: combine OBR targets, add DWP payment bands#31nikhilwoodruff wants to merge 3 commits intomainfrom
nikhilwoodruff wants to merge 3 commits intomainfrom
Conversation
…ayment bands The calibration was producing £15.5bn UC spend instead of ~£92bn for 2029/30. Three root causes: 1. OBR UC in-cap (£79bn) and outside-cap (£13bn) mapped to the same calibration matrix column with conflicting target values, causing the optimizer to compromise at ~£26bn. Now combined into a single total target. 2. Added UC payment band distribution targets from DWP stat-xplore (7 bands from £0-300/month to £2000+/month), with benunit-level min/max filter support in the Rust calibration engine. 3. Added max weight ratio (100x) to prevent degenerate weights during optimisation. Also adds pool_frs.py for combining multiple FRS survey years with ID reindexing, weight division, and column harmonisation across FRS format changes (2021/2022 vs 2023). Result: UC spend now £89.6bn vs £92.2bn target (2.8% error, down from 83%).
All target builders now produce targets from 2023/24, not just 2024/25: - DWP: added col 79 (2023/24) to forecast column map - HMRC SPI: extended CALIBRATION_YEARS to include 2023 - ONS: extended population targets to 2023 - OBR: back-extrapolated 2023 receipts/welfare from 2024 outturn using OBR earnings/CPI/council tax growth rates Every year 2023-2029 now has 206 calibration targets.
New calibration targets: - ONS household tenure distribution (owned outright, owned w/ mortgage, social rent, private rent) — 4 training targets per year - ONS households by region (12 holdout targets per year) - DWP UC claimants by age band (16-24, 25-34, 35-49, 50-64, 65+) from stat-xplore — 5 targets per year (4 training, 1 holdout) Rust changes: - Added tenure_type and region to household variable resolver - Added filter support for household-entity targets in build_matrix Total targets per year: 227 (up from 206).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
UC spending was calibrating to £15.5bn instead of ~£92bn for 2029/30 due to three issues:
OBR dual-target bug: UC appears twice in OBR sheet 4.9 (inside and outside the welfare cap). Both rows mapped to the same calibration matrix column (
universal_creditsum) but with conflicting target values (£79bn vs £13bn), so the optimizer compromised at ~£26bn. Now combined into a single £92.2bn total.Insufficient distribution constraints: Added 7 UC payment band targets from DWP stat-xplore (monthly bands from £0-300 to £2000+), with new benunit-level min/max filter support in the Rust calibration engine. This constrains the UC amount distribution, not just the total.
Weight bounds: Added max weight ratio (100x) to prevent degenerate weights where a handful of households get extreme weights during optimisation.
Also adds
pool_frs.pyfor combining multiple FRS survey years (2021+2022+2023) with ID reindexing, weight division, and column harmonisation across FRS format changes.Result: UC spend £89.6bn vs £92.2bn target (2.8% error, down from 83%).