Skip to content

Commit 9faf98b

Browse files
FBumannFBumann
committed
Invalidate and lock FLow system based on solution existance (#518)
* fix: Link status parameters to flow_system * ⏺ Summary I've implemented the FlowSystem locking behavior as discussed. Here's what was done: Core Concept A FlowSystem is locked (read-only) when solution is not None. New Features 1. is_locked property (flow_system.py:1127-1131) - Returns True if the FlowSystem has a solution 2. reset() method (flow_system.py:1145-1168) - Clears: solution, model, element submodels, variable/constraint names - Returns self for method chaining - Allows the FlowSystem to be modified and re-optimized 3. _invalidate_model() helper (flow_system.py:1133-1143) - Called when adding elements/carriers to a FlowSystem with a model (but no solution) - Clears model and element state 4. copy() method (flow_system.py:733-767) - Creates a fresh FlowSystem copy without solution/model - Supports copy.copy() and copy.deepcopy() Behavior Changes | Operation | Before Optimization | After build_model() | After optimize() | |----------------|-------------------------------|------------------------------------|-------------------------------| | add_elements() | ✓ Works | ⚠ Works, warns & invalidates model | ✗ RuntimeError | | add_carriers() | ✓ Works | ⚠ Works, warns & invalidates model | ✗ RuntimeError | | copy() | Returns copy without solution | Returns copy without solution | Returns copy without solution | | reset() | No-op | Clears model | Clears solution & model | Bug Fix Fixed an issue where StatusParameters created during modeling (for prevent_simultaneous_flows and component-level status) weren't linked to the FlowSystem (elements.py:957-964, components.py:731-732). Tests Added comprehensive tests in tests/test_flow_system_locking.py (28 tests) covering: - is_locked property behavior - add_elements() locking and invalidation - add_carriers() locking and invalidation - reset() method functionality - copy() method functionality - Loaded FlowSystem behavior * Add invalidation tests * Add link_to_flow_system method * Add link_to_flow_system method * New invalidate() method (flow_system.py:1232-1275) A public method for manual model invalidation when modifying element attributes after connect_and_transform(): def invalidate(self) -> FlowSystem: """Invalidate the model to allow re-transformation after modifying elements.""" - Raises RuntimeError if FlowSystem has a solution (must call reset() first) - Returns self for method chaining - Useful when you need to modify after connect_and_transform() but before optimize() 2. Updated docstrings connect_and_transform() - Added comprehensive docstring explaining: - What steps it performs - Warning that attributes become xarray DataArrays after transformation - Note about idempotency and how to reset with invalidate() _invalidate_model() - Clarified its role and relationship to public methods 3. New tests (test_flow_system_locking.py:285-409) Added TestInvalidate class with 8 tests: - test_invalidate_resets_connected_and_transformed - test_invalidate_clears_model - test_invalidate_raises_when_locked - test_invalidate_returns_self - test_invalidate_allows_retransformation - test_modify_element_and_invalidate - full workflow with reset - test_invalidate_needed_after_transform_before_optimize - pre-optimization modification - test_reset_already_invalidates - confirms reset already handles invalidation Key insight from testing reset() already calls _invalidate_model(), so modifications after reset() automatically take effect on the next optimize(). The new invalidate() method is primarily for the case where: 1. You've called connect_and_transform() manually 2. Haven't optimized yet (no solution) 3. Need to modify element attributes * Typo
1 parent 1601cac commit 9faf98b

7 files changed

Lines changed: 793 additions & 152 deletions

File tree

‎flixopt/components.py‎

Lines changed: 32 additions & 30 deletions
Original file line numberDiff line numberDiff line change
@@ -180,11 +180,11 @@ def create_model(self, model: FlowSystemModel) -> LinearConverterModel:
180180
self.submodel = LinearConverterModel(model, self)
181181
return self.submodel
182182

183-
def _set_flow_system(self, flow_system) -> None:
183+
def link_to_flow_system(self, flow_system, prefix: str = '') -> None:
184184
"""Propagate flow_system reference to parent Component and piecewise_conversion."""
185-
super()._set_flow_system(flow_system)
185+
super().link_to_flow_system(flow_system, prefix)
186186
if self.piecewise_conversion is not None:
187-
self.piecewise_conversion._set_flow_system(flow_system)
187+
self.piecewise_conversion.link_to_flow_system(flow_system, self._sub_prefix('PiecewiseConversion'))
188188

189189
def _plausibility_checks(self) -> None:
190190
super()._plausibility_checks()
@@ -216,14 +216,13 @@ def _plausibility_checks(self) -> None:
216216
f'({flow.label_full}).'
217217
)
218218

219-
def transform_data(self, name_prefix: str = '') -> None:
220-
prefix = '|'.join(filter(None, [name_prefix, self.label_full]))
221-
super().transform_data(prefix)
219+
def transform_data(self) -> None:
220+
super().transform_data()
222221
if self.conversion_factors:
223222
self.conversion_factors = self._transform_conversion_factors()
224223
if self.piecewise_conversion:
225224
self.piecewise_conversion.has_time_dim = True
226-
self.piecewise_conversion.transform_data(f'{prefix}|PiecewiseConversion')
225+
self.piecewise_conversion.transform_data()
227226

228227
def _transform_conversion_factors(self) -> list[dict[str, xr.DataArray]]:
229228
"""Converts all conversion factors to internal datatypes"""
@@ -427,49 +426,50 @@ def create_model(self, model: FlowSystemModel) -> StorageModel:
427426
self.submodel = StorageModel(model, self)
428427
return self.submodel
429428

430-
def _set_flow_system(self, flow_system) -> None:
429+
def link_to_flow_system(self, flow_system, prefix: str = '') -> None:
431430
"""Propagate flow_system reference to parent Component and capacity_in_flow_hours if it's InvestParameters."""
432-
super()._set_flow_system(flow_system)
431+
super().link_to_flow_system(flow_system, prefix)
433432
if isinstance(self.capacity_in_flow_hours, InvestParameters):
434-
self.capacity_in_flow_hours._set_flow_system(flow_system)
433+
self.capacity_in_flow_hours.link_to_flow_system(flow_system, self._sub_prefix('InvestParameters'))
435434

436-
def transform_data(self, name_prefix: str = '') -> None:
437-
prefix = '|'.join(filter(None, [name_prefix, self.label_full]))
438-
super().transform_data(prefix)
435+
def transform_data(self) -> None:
436+
super().transform_data()
439437
self.relative_minimum_charge_state = self._fit_coords(
440-
f'{prefix}|relative_minimum_charge_state', self.relative_minimum_charge_state
438+
f'{self.prefix}|relative_minimum_charge_state', self.relative_minimum_charge_state
441439
)
442440
self.relative_maximum_charge_state = self._fit_coords(
443-
f'{prefix}|relative_maximum_charge_state', self.relative_maximum_charge_state
441+
f'{self.prefix}|relative_maximum_charge_state', self.relative_maximum_charge_state
442+
)
443+
self.eta_charge = self._fit_coords(f'{self.prefix}|eta_charge', self.eta_charge)
444+
self.eta_discharge = self._fit_coords(f'{self.prefix}|eta_discharge', self.eta_discharge)
445+
self.relative_loss_per_hour = self._fit_coords(
446+
f'{self.prefix}|relative_loss_per_hour', self.relative_loss_per_hour
444447
)
445-
self.eta_charge = self._fit_coords(f'{prefix}|eta_charge', self.eta_charge)
446-
self.eta_discharge = self._fit_coords(f'{prefix}|eta_discharge', self.eta_discharge)
447-
self.relative_loss_per_hour = self._fit_coords(f'{prefix}|relative_loss_per_hour', self.relative_loss_per_hour)
448448
if not isinstance(self.initial_charge_state, str):
449449
self.initial_charge_state = self._fit_coords(
450-
f'{prefix}|initial_charge_state', self.initial_charge_state, dims=['period', 'scenario']
450+
f'{self.prefix}|initial_charge_state', self.initial_charge_state, dims=['period', 'scenario']
451451
)
452452
self.minimal_final_charge_state = self._fit_coords(
453-
f'{prefix}|minimal_final_charge_state', self.minimal_final_charge_state, dims=['period', 'scenario']
453+
f'{self.prefix}|minimal_final_charge_state', self.minimal_final_charge_state, dims=['period', 'scenario']
454454
)
455455
self.maximal_final_charge_state = self._fit_coords(
456-
f'{prefix}|maximal_final_charge_state', self.maximal_final_charge_state, dims=['period', 'scenario']
456+
f'{self.prefix}|maximal_final_charge_state', self.maximal_final_charge_state, dims=['period', 'scenario']
457457
)
458458
self.relative_minimum_final_charge_state = self._fit_coords(
459-
f'{prefix}|relative_minimum_final_charge_state',
459+
f'{self.prefix}|relative_minimum_final_charge_state',
460460
self.relative_minimum_final_charge_state,
461461
dims=['period', 'scenario'],
462462
)
463463
self.relative_maximum_final_charge_state = self._fit_coords(
464-
f'{prefix}|relative_maximum_final_charge_state',
464+
f'{self.prefix}|relative_maximum_final_charge_state',
465465
self.relative_maximum_final_charge_state,
466466
dims=['period', 'scenario'],
467467
)
468468
if isinstance(self.capacity_in_flow_hours, InvestParameters):
469-
self.capacity_in_flow_hours.transform_data(f'{prefix}|InvestParameters')
469+
self.capacity_in_flow_hours.transform_data()
470470
else:
471471
self.capacity_in_flow_hours = self._fit_coords(
472-
f'{prefix}|capacity_in_flow_hours', self.capacity_in_flow_hours, dims=['period', 'scenario']
472+
f'{self.prefix}|capacity_in_flow_hours', self.capacity_in_flow_hours, dims=['period', 'scenario']
473473
)
474474

475475
def _plausibility_checks(self) -> None:
@@ -714,11 +714,10 @@ def create_model(self, model) -> TransmissionModel:
714714
self.submodel = TransmissionModel(model, self)
715715
return self.submodel
716716

717-
def transform_data(self, name_prefix: str = '') -> None:
718-
prefix = '|'.join(filter(None, [name_prefix, self.label_full]))
719-
super().transform_data(prefix)
720-
self.relative_losses = self._fit_coords(f'{prefix}|relative_losses', self.relative_losses)
721-
self.absolute_losses = self._fit_coords(f'{prefix}|absolute_losses', self.absolute_losses)
717+
def transform_data(self) -> None:
718+
super().transform_data()
719+
self.relative_losses = self._fit_coords(f'{self.prefix}|relative_losses', self.relative_losses)
720+
self.absolute_losses = self._fit_coords(f'{self.prefix}|absolute_losses', self.absolute_losses)
722721

723722

724723
class TransmissionModel(ComponentModel):
@@ -729,6 +728,9 @@ def __init__(self, model: FlowSystemModel, element: Transmission):
729728
for flow in element.inputs + element.outputs:
730729
if flow.status_parameters is None:
731730
flow.status_parameters = StatusParameters()
731+
flow.status_parameters.link_to_flow_system(
732+
model.flow_system, f'{flow.label_full}|status_parameters'
733+
)
732734

733735
super().__init__(model, element)
734736

‎flixopt/effects.py‎

Lines changed: 22 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -237,50 +237,56 @@ def __init__(
237237
self.minimum_over_periods = minimum_over_periods
238238
self.maximum_over_periods = maximum_over_periods
239239

240-
def transform_data(self, name_prefix: str = '') -> None:
241-
prefix = '|'.join(filter(None, [name_prefix, self.label_full]))
242-
self.minimum_per_hour = self._fit_coords(f'{prefix}|minimum_per_hour', self.minimum_per_hour)
243-
self.maximum_per_hour = self._fit_coords(f'{prefix}|maximum_per_hour', self.maximum_per_hour)
240+
def link_to_flow_system(self, flow_system, prefix: str = '') -> None:
241+
"""Link this effect to a FlowSystem.
242+
243+
Elements use their label_full as prefix by default, ignoring the passed prefix.
244+
"""
245+
super().link_to_flow_system(flow_system, self.label_full)
246+
247+
def transform_data(self) -> None:
248+
self.minimum_per_hour = self._fit_coords(f'{self.prefix}|minimum_per_hour', self.minimum_per_hour)
249+
self.maximum_per_hour = self._fit_coords(f'{self.prefix}|maximum_per_hour', self.maximum_per_hour)
244250

245251
self.share_from_temporal = self._fit_effect_coords(
246252
prefix=None,
247253
effect_values=self.share_from_temporal,
248-
suffix=f'(temporal)->{prefix}(temporal)',
254+
suffix=f'(temporal)->{self.prefix}(temporal)',
249255
dims=['time', 'period', 'scenario'],
250256
)
251257
self.share_from_periodic = self._fit_effect_coords(
252258
prefix=None,
253259
effect_values=self.share_from_periodic,
254-
suffix=f'(periodic)->{prefix}(periodic)',
260+
suffix=f'(periodic)->{self.prefix}(periodic)',
255261
dims=['period', 'scenario'],
256262
)
257263

258264
self.minimum_temporal = self._fit_coords(
259-
f'{prefix}|minimum_temporal', self.minimum_temporal, dims=['period', 'scenario']
265+
f'{self.prefix}|minimum_temporal', self.minimum_temporal, dims=['period', 'scenario']
260266
)
261267
self.maximum_temporal = self._fit_coords(
262-
f'{prefix}|maximum_temporal', self.maximum_temporal, dims=['period', 'scenario']
268+
f'{self.prefix}|maximum_temporal', self.maximum_temporal, dims=['period', 'scenario']
263269
)
264270
self.minimum_periodic = self._fit_coords(
265-
f'{prefix}|minimum_periodic', self.minimum_periodic, dims=['period', 'scenario']
271+
f'{self.prefix}|minimum_periodic', self.minimum_periodic, dims=['period', 'scenario']
266272
)
267273
self.maximum_periodic = self._fit_coords(
268-
f'{prefix}|maximum_periodic', self.maximum_periodic, dims=['period', 'scenario']
274+
f'{self.prefix}|maximum_periodic', self.maximum_periodic, dims=['period', 'scenario']
269275
)
270276
self.minimum_total = self._fit_coords(
271-
f'{prefix}|minimum_total', self.minimum_total, dims=['period', 'scenario']
277+
f'{self.prefix}|minimum_total', self.minimum_total, dims=['period', 'scenario']
272278
)
273279
self.maximum_total = self._fit_coords(
274-
f'{prefix}|maximum_total', self.maximum_total, dims=['period', 'scenario']
280+
f'{self.prefix}|maximum_total', self.maximum_total, dims=['period', 'scenario']
275281
)
276282
self.minimum_over_periods = self._fit_coords(
277-
f'{prefix}|minimum_over_periods', self.minimum_over_periods, dims=['scenario']
283+
f'{self.prefix}|minimum_over_periods', self.minimum_over_periods, dims=['scenario']
278284
)
279285
self.maximum_over_periods = self._fit_coords(
280-
f'{prefix}|maximum_over_periods', self.maximum_over_periods, dims=['scenario']
286+
f'{self.prefix}|maximum_over_periods', self.maximum_over_periods, dims=['scenario']
281287
)
282288
self.period_weights = self._fit_coords(
283-
f'{prefix}|period_weights', self.period_weights, dims=['period', 'scenario']
289+
f'{self.prefix}|period_weights', self.period_weights, dims=['period', 'scenario']
284290
)
285291

286292
def create_model(self, model: FlowSystemModel) -> EffectModel:
@@ -670,7 +676,7 @@ def _do_modeling(self):
670676
penalty_effect = self.effects._create_penalty_effect()
671677
# Link to FlowSystem (should already be linked, but ensure it)
672678
if penalty_effect._flow_system is None:
673-
penalty_effect._set_flow_system(self._model.flow_system)
679+
penalty_effect.link_to_flow_system(self._model.flow_system)
674680

675681
# Create EffectModel for each effect
676682
for effect in self.effects.values():

‎flixopt/elements.py‎

Lines changed: 52 additions & 39 deletions
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,6 @@
2020
Element,
2121
ElementModel,
2222
FlowSystemModel,
23-
Interface,
2423
register_class_for_io,
2524
)
2625

@@ -111,21 +110,23 @@ def create_model(self, model: FlowSystemModel) -> ComponentModel:
111110
self.submodel = ComponentModel(model, self)
112111
return self.submodel
113112

114-
def _set_flow_system(self, flow_system) -> None:
115-
"""Propagate flow_system reference to nested Interface objects and flows."""
116-
super()._set_flow_system(flow_system)
113+
def link_to_flow_system(self, flow_system, prefix: str = '') -> None:
114+
"""Propagate flow_system reference to nested Interface objects and flows.
115+
116+
Elements use their label_full as prefix by default, ignoring the passed prefix.
117+
"""
118+
super().link_to_flow_system(flow_system, self.label_full)
117119
if self.status_parameters is not None:
118-
self.status_parameters._set_flow_system(flow_system)
120+
self.status_parameters.link_to_flow_system(flow_system, self._sub_prefix('status_parameters'))
119121
for flow in self.inputs + self.outputs:
120-
flow._set_flow_system(flow_system)
122+
flow.link_to_flow_system(flow_system)
121123

122-
def transform_data(self, name_prefix: str = '') -> None:
123-
prefix = '|'.join(filter(None, [name_prefix, self.label_full]))
124+
def transform_data(self) -> None:
124125
if self.status_parameters is not None:
125-
self.status_parameters.transform_data(prefix)
126+
self.status_parameters.transform_data()
126127

127128
for flow in self.inputs + self.outputs:
128-
flow.transform_data() # Flow doesnt need the name_prefix
129+
flow.transform_data()
129130

130131
def _check_unique_flow_labels(self):
131132
all_flow_labels = [flow.label for flow in self.inputs + self.outputs]
@@ -269,16 +270,18 @@ def create_model(self, model: FlowSystemModel) -> BusModel:
269270
self.submodel = BusModel(model, self)
270271
return self.submodel
271272

272-
def _set_flow_system(self, flow_system) -> None:
273-
"""Propagate flow_system reference to nested flows."""
274-
super()._set_flow_system(flow_system)
273+
def link_to_flow_system(self, flow_system, prefix: str = '') -> None:
274+
"""Propagate flow_system reference to nested flows.
275+
276+
Elements use their label_full as prefix by default, ignoring the passed prefix.
277+
"""
278+
super().link_to_flow_system(flow_system, self.label_full)
275279
for flow in self.inputs + self.outputs:
276-
flow._set_flow_system(flow_system)
280+
flow.link_to_flow_system(flow_system)
277281

278-
def transform_data(self, name_prefix: str = '') -> None:
279-
prefix = '|'.join(filter(None, [name_prefix, self.label_full]))
282+
def transform_data(self) -> None:
280283
self.imbalance_penalty_per_flow_hour = self._fit_coords(
281-
f'{prefix}|imbalance_penalty_per_flow_hour', self.imbalance_penalty_per_flow_hour
284+
f'{self.prefix}|imbalance_penalty_per_flow_hour', self.imbalance_penalty_per_flow_hour
282285
)
283286

284287
def _plausibility_checks(self) -> None:
@@ -505,45 +508,49 @@ def create_model(self, model: FlowSystemModel) -> FlowModel:
505508
self.submodel = FlowModel(model, self)
506509
return self.submodel
507510

508-
def _set_flow_system(self, flow_system) -> None:
509-
"""Propagate flow_system reference to nested Interface objects."""
510-
super()._set_flow_system(flow_system)
511+
def link_to_flow_system(self, flow_system, prefix: str = '') -> None:
512+
"""Propagate flow_system reference to nested Interface objects.
513+
514+
Elements use their label_full as prefix by default, ignoring the passed prefix.
515+
"""
516+
super().link_to_flow_system(flow_system, self.label_full)
511517
if self.status_parameters is not None:
512-
self.status_parameters._set_flow_system(flow_system)
513-
if isinstance(self.size, Interface):
514-
self.size._set_flow_system(flow_system)
515-
516-
def transform_data(self, name_prefix: str = '') -> None:
517-
prefix = '|'.join(filter(None, [name_prefix, self.label_full]))
518-
self.relative_minimum = self._fit_coords(f'{prefix}|relative_minimum', self.relative_minimum)
519-
self.relative_maximum = self._fit_coords(f'{prefix}|relative_maximum', self.relative_maximum)
520-
self.fixed_relative_profile = self._fit_coords(f'{prefix}|fixed_relative_profile', self.fixed_relative_profile)
521-
self.effects_per_flow_hour = self._fit_effect_coords(prefix, self.effects_per_flow_hour, 'per_flow_hour')
518+
self.status_parameters.link_to_flow_system(flow_system, self._sub_prefix('status_parameters'))
519+
if isinstance(self.size, InvestParameters):
520+
self.size.link_to_flow_system(flow_system, self._sub_prefix('InvestParameters'))
521+
522+
def transform_data(self) -> None:
523+
self.relative_minimum = self._fit_coords(f'{self.prefix}|relative_minimum', self.relative_minimum)
524+
self.relative_maximum = self._fit_coords(f'{self.prefix}|relative_maximum', self.relative_maximum)
525+
self.fixed_relative_profile = self._fit_coords(
526+
f'{self.prefix}|fixed_relative_profile', self.fixed_relative_profile
527+
)
528+
self.effects_per_flow_hour = self._fit_effect_coords(self.prefix, self.effects_per_flow_hour, 'per_flow_hour')
522529
self.flow_hours_max = self._fit_coords(
523-
f'{prefix}|flow_hours_max', self.flow_hours_max, dims=['period', 'scenario']
530+
f'{self.prefix}|flow_hours_max', self.flow_hours_max, dims=['period', 'scenario']
524531
)
525532
self.flow_hours_min = self._fit_coords(
526-
f'{prefix}|flow_hours_min', self.flow_hours_min, dims=['period', 'scenario']
533+
f'{self.prefix}|flow_hours_min', self.flow_hours_min, dims=['period', 'scenario']
527534
)
528535
self.flow_hours_max_over_periods = self._fit_coords(
529-
f'{prefix}|flow_hours_max_over_periods', self.flow_hours_max_over_periods, dims=['scenario']
536+
f'{self.prefix}|flow_hours_max_over_periods', self.flow_hours_max_over_periods, dims=['scenario']
530537
)
531538
self.flow_hours_min_over_periods = self._fit_coords(
532-
f'{prefix}|flow_hours_min_over_periods', self.flow_hours_min_over_periods, dims=['scenario']
539+
f'{self.prefix}|flow_hours_min_over_periods', self.flow_hours_min_over_periods, dims=['scenario']
533540
)
534541
self.load_factor_max = self._fit_coords(
535-
f'{prefix}|load_factor_max', self.load_factor_max, dims=['period', 'scenario']
542+
f'{self.prefix}|load_factor_max', self.load_factor_max, dims=['period', 'scenario']
536543
)
537544
self.load_factor_min = self._fit_coords(
538-
f'{prefix}|load_factor_min', self.load_factor_min, dims=['period', 'scenario']
545+
f'{self.prefix}|load_factor_min', self.load_factor_min, dims=['period', 'scenario']
539546
)
540547

541548
if self.status_parameters is not None:
542-
self.status_parameters.transform_data(prefix)
549+
self.status_parameters.transform_data()
543550
if isinstance(self.size, InvestParameters):
544-
self.size.transform_data(prefix)
551+
self.size.transform_data()
545552
else:
546-
self.size = self._fit_coords(f'{prefix}|size', self.size, dims=['period', 'scenario'])
553+
self.size = self._fit_coords(f'{self.prefix}|size', self.size, dims=['period', 'scenario'])
547554

548555
def _plausibility_checks(self) -> None:
549556
# TODO: Incorporate into Variable? (Lower_bound can not be greater than upper bound
@@ -955,11 +962,17 @@ def _do_modeling(self):
955962
for flow in all_flows:
956963
if flow.status_parameters is None:
957964
flow.status_parameters = StatusParameters()
965+
flow.status_parameters.link_to_flow_system(
966+
self._model.flow_system, f'{flow.label_full}|status_parameters'
967+
)
958968

959969
if self.element.prevent_simultaneous_flows:
960970
for flow in self.element.prevent_simultaneous_flows:
961971
if flow.status_parameters is None:
962972
flow.status_parameters = StatusParameters()
973+
flow.status_parameters.link_to_flow_system(
974+
self._model.flow_system, f'{flow.label_full}|status_parameters'
975+
)
963976

964977
# Create FlowModels (which creates their variables and constraints)
965978
for flow in all_flows:

0 commit comments

Comments
 (0)