feat: Improve numerical conditioning#23
Merged
koen-vg merged 3 commits intoJul 3, 2026
Merged
Conversation
… clipping) Remove Gurobi's "large matrix coefficient range" warning on the food-system model by tightening two sources of extreme coefficients: - Denominate the CH4 and N2O emission buses in kilotonnes instead of tonnes, so their flow coefficients (rice/enteric methane, fertilizer N2O) sit within a few orders of the MtCO2 bus. Updates the source-link writers, the GWP aggregation links, the analysis converters, and the carrier unit labels. - Add a `numerics` config block and a build-time clip pass (clip_negligible_coefficients) that zeroes physically-negligible coefficients: sub-hectare land areas (the source of ~1e-21 bounds/RHS), trace irrigation water, near-zero spared-land carbon fluxes, and rounding-level cost corrections. The former `land.filtering` thresholds are folded into `numerics` so all build-time conditioning knobs live together. Central A/B (barrier + crossover): matrix range [6e-08, 1e+05] -> [5e-06, 3e+04], warning gone, solver work units 24.9 -> 22.3 (~10%); objective unchanged to 1.5e-4. Emission totals are identical. Calibration provenance stamps are refreshed (artefacts are inert to the change: calibration solves run GHG-off and the pruning thresholds keep their values).
The L1 stability penalties (crop, grassland, animal feed, diet) encoded |deviation| with an auxiliary abs_dev variable and two inequality rows per link (abs_dev >= +dev, abs_dev >= -dev). Gurobi's presolve cannot fold that pair, so on a full-resolution health solve the stability machinery accounted for 150k of 368k presolved rows. Encode instead dev_pos - dev_neg == deviation with dev_pos, dev_neg >= 0 and price their sum: one equality row per link, same optimum for any positive L1 cost. The land-conversion penalty (zero baseline, non-negative flows, |p - 0| = p) is priced directly on the link flows with no auxiliary variables at all. On the central health-on config this cuts presolved rows 368k -> 212k and total solver work 41 -> 28 work units (-33%); objectives agree to within the 0.1% MIP gap and calibrated L1 costs remain valid. The objective-breakdown bookkeeping in core.py reads dev_pos + dev_neg (and link flows for land conversion) in place of abs_dev.
Picks up the balance-arg grouping fix in _iter_balance_args: grouping on a two-column delay/cyclic frame instead of a deep copy of the full wide static frame per output port. create_model on the full-resolution model drops from ~13.4s to ~8.7s; the constructed model is verified element-for-element identical.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Improve numerical conditioning, reformulate L1 deviation costs for better presolve, and update PyPSA to include model building performance fix.