Add enhanced FRS (wealth + consumption imputation) and VAT#8
Merged
nikhilwoodruff merged 4 commits intomainfrom Apr 5, 2026
Merged
Add enhanced FRS (wealth + consumption imputation) and VAT#8nikhilwoodruff merged 4 commits intomainfrom
nikhilwoodruff merged 4 commits intomainfrom
Conversation
nikhilwoodruff
force-pushed
the
add-efrs-and-vat
branch
from
3328162 to
8f41768
Compare
…e pension scaling Ports the EFRS pipeline from policyengine-uk-data to Rust: trains random forest models on WAS (wealth) and LCFS (consumption) survey data, predicts onto FRS households, and calibrates energy consumption to NEED 2023 targets. Adds a household-level VAT calculation (standard/reduced/zero rates by COICOP category) wired into net income, enabling distributional analysis of consumption tax reforms. Fixes state pension reform responsiveness: reported SP amounts are now scaled by reform_rate/baseline_rate so that changing the SP parameter actually affects household net income. Previously reported amounts were passed through verbatim, making SP reforms have near-zero distributional impact. New CLI flags: --extract-efrs, --was-dir, --lcfs-dir. Co-Authored-By: Claude <noreply@anthropic.com>
HICBC is now correctly modelled as an income tax charge on the highest earner (Phase 2b of simulation) rather than a child benefit reduction. Child benefit is paid in full; the charge is tapered between the HICBC threshold and taper end based on adjusted net income. Also adds auto-detection of data/efrs_clean (preferred) or data/frs_clean when no explicit data source is specified on the CLI. Co-Authored-By: Claude <noreply@anthropic.com>
…dd efrs to Python DATASETS State pension now matches policyengine-uk approach: new SP recipients (reached SP age after April 2016) get the full parameter rate directly; basic SP recipients get reported amounts scaled by reform ratio. Also adds fiscal_year parameter to Simulation constructor and adds "efrs" to the Python interface's DATASETS tuple for GCS download support. Co-Authored-By: Claude <noreply@anthropic.com>
nikhilwoodruff
force-pushed
the
add-efrs-and-vat
branch
from
cf93b4f to
602d2de
Compare
Calculate per-person state pension in Phase 1a before income tax in Phase 1b, so SP reforms correctly affect taxable income. Previously income tax used reported SP amounts regardless of reform parameters, causing £0 revenue change for SP reforms. Also fixes HMAC download truncation in Python data module (adds content-length verification and timeout). Co-Authored-By: Claude <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Ports the EFRS pipeline from policyengine-uk-data to Rust, adds a VAT model, and fixes state pension reform responsiveness.
Enhanced FRS (
--extract-efrs): trains random forest models (smartcore) on WAS Round 7 (10 wealth targets) and LCFS 2021/22 (17 consumption targets), predicts onto FRS households, and calibrates energy to NEED 2023 targets via iterative raking. New entity fields:TenureType,AccommodationType, 10 wealth fields, 17 consumption fields. Clean CSV format extended with backwards compatibility.VAT (
vatparameter block): applies standard (20%), reduced (5%), and zero rates by COICOP category when EFRS consumption data is available, or estimates from disposable income using ONS propensity-to-consume curves otherwise. Historical rates back to 1994 (17.5%/15%/20%). VAT is deducted from household net income and included in total tax revenue.State pension fix: previously, reported SP amounts were passed through verbatim — changing the SP parameter had near-zero distributional impact because
calculate_state_pensionreturnedp.state_pension(the FRS-reported value) without scaling. Now, reported amounts are scaled byreform_rate / baseline_rate, and the gross income calculation adjusts accordingly. A 5% SP increase now correctly shows +£5.8bn spending, 30% winners (pensioner households), and a progressive distributional pattern (bottom decile +1.6%, top decile +0.2%).Test plan
cargo test— 108 tests passcargo build --release— clean, no warnings--extract-efrswith WAS + LCFS data to verify RF training pipeline end-to-end