Clustering Parameter Refinements & Unified Slot Assignment (#552)

* Improve documentation and improve CHANGELOG.md

* FIx CHangelog and change to v6.0.0

* FIx CHangelog and change to v6.0.0

* FIx CHangelog and change to v6.0.0

* Enhanced Clustering Control

  New Parameters Added to cluster() Method

  | Parameter               | Type                          | Default              | Purpose                                                                                                            |
  |-------------------------|-------------------------------|----------------------|--------------------------------------------------------------------------------------------------------------------|
  | cluster_method          | Literal[...]                  | 'k_means'            | Clustering algorithm ('k_means', 'hierarchical', 'k_medoids', 'k_maxoids', 'averaging')                            |
  | representation_method   | Literal[...]                  | 'meanRepresentation' | How clusters are represented ('meanRepresentation', 'medoidRepresentation', 'distributionAndMinMaxRepresentation') |
  | extreme_period_method   | Literal[...]                  | 'new_cluster_center' | How peaks are integrated ('None', 'append', 'new_cluster_center', 'replace_cluster_center')                        |
  | rescale_cluster_periods | bool                          | True                 | Rescale clusters to match original means                                                                           |
  | random_state            | int | None                    | None                 | Random seed for reproducibility                                                                                    |
  | predef_cluster_order    | np.ndarray | list[int] | None | None                 | Manual clustering assignments                                                                                      |
  | **tsam_kwargs           | Any                           | -                    | Pass-through for any tsam parameter                                                                                |

  Clustering Quality Metrics

  Access via fs.clustering.metrics after clustering - returns a DataFrame with RMSE, MAE, and other accuracy indicators per time series.

  Files Modified

  1. flixopt/transform_accessor.py - Updated cluster() signature and tsam call
  2. flixopt/clustering/base.py - Added metrics field to Clustering class
  3. tests/test_clustering/test_integration.py - Added tests for new parameters
  4. docs/user-guide/optimization/clustering.md - Updated documentation

* Dimension renamed: original_period → original_cluster
  Property renamed: n_original_periods → n_original_clusters

* Problem: Expanded FlowSystem from clustering didn't have the extra timestep that regular FlowSystems have.

  Root Cause: In expand_solution(), the solution was only indexed by original_timesteps (n elements) instead of original_timesteps_extra (n+1 elements).

  Fix in flixopt/transform_accessor.py:

  1. Reindex solution to timesteps_extra (line 1296-1298):
    - Added expanded_fs._solution.reindex(time=original_timesteps_extra) for consistency with non-expanded FlowSystems
  2. Fill extra timestep for charge_state (lines 1300-1333):
    - Added special handling to properly fill the extra timestep for storage charge_state variables using the last cluster's extra timestep value
  3. Updated intercluster storage handling (lines 1340-1388):
    - Modified to work with original_timesteps_extra instead of just original_timesteps
    - The extra timestep now correctly gets the final SOC boundary value with proper decay applied

  Tests updated in tests/test_cluster_reduce_expand.py:
  - Updated 4 assertions that check solution time coordinates to expect 193 (192 + 1 extra) instead of 192

* - 'variable' is treated as a special valid facet value (since it exists in the melted DataFrame from data_var names, not as a dimension)
  - When facet_row='variable' or facet_col='variable' is passed, it's passed through directly
  - In line(), when faceting by variable, it's not also used for color (avoids double encoding)

* Add variable and color to auto resolving in fxplot

* Added 'variable' to both priority lists and updated the logic to treat it consistently:

  flixopt/config.py:
  'extra_dim_priority': ('variable', 'cluster', 'period', 'scenario'),
  'x_dim_priority': ('time', 'duration', 'duration_pct', 'variable', 'period', 'scenario', 'cluster'),

  flixopt/dataset_plot_accessor.py:
  - _get_x_dim: Now takes n_data_vars parameter; 'variable' is available when > 1
  - _resolve_auto_facets: 'variable' is available when len(data_vars) > 1 and respects exclude_dims

  Behavior:
  - 'variable' is treated like any other dimension in the priority system
  - Only available when there are multiple data_vars
  - Properly excluded when already used (e.g., for x-axis)

* Improve plotting, especially for clustering

* Drop cluster index when expanding

* Fix storage expansion

* Improve clustering

* fix scatter plot faceting

* ⏺ Fixed the documentation in the notebook:

  1. Cell 32 (API Reference table): Updated defaults to 'hierarchical', 'medoidRepresentation', and None
  2. Cell 16: Swapped the example to show k_means as the alternative (since hierarchical is now default)
  3. Cell 17: Updated variable names to match
  4. Cell 33 (Key Takeaways): Clarified that random_state is only needed for non-deterministic methods like 'k_means'

  The code review

* 1. Error handling for accuracyIndicators() - Added try/except with warning log and empty DataFrame fallback, plus handling empty DataFrames when building the metrics Dataset
  2. Random state to tsam - Replaced global np.random.seed() with passing seed parameter directly to tsam's TimeSeriesAggregation
  3. tsam_kwargs conflict validation - Added validation that raises ValueError if user tries to override explicit parameters via **tsam_kwargs (including seed)
  4. predef_cluster_order validation - Added dimension validation for DataArray inputs, checking they match the FlowSystem's period/scenario structure
  5. Out-of-bounds fix - Clamped last_original_cluster_idx to n_original_clusters - 1 to handle partial clusters at the end

* 1. DataFrame truth ambiguity - Changed non_empty_metrics.get(first_key) or next(...) to explicit if metrics_df is None: check
  2. removed random state

* Fix pie plot animation frame and add warnings for unassigned dims

* Change logger warning to regular warning

* ⏺ The centralized slot assignment system is now complete. Here's a summary of the changes made:

  Changes Made

  1. flixopt/config.py

  - Replaced three separate config attributes (extra_dim_priority, dim_slot_priority, x_dim_priority) with a single unified dim_priority tuple
  - Updated CONFIG.Plotting class docstring and attribute definitions
  - Updated to_dict() method to use the new attribute
  - The new priority order: ('time', 'duration', 'duration_pct', 'variable', 'cluster', 'period', 'scenario')

  2. flixopt/dataset_plot_accessor.py

  - Created new assign_slots() function that centralizes all dimension-to-slot assignment logic
  - Fixed slot fill order: x → color → facet_col → facet_row → animation_frame
  - Updated all plot methods (bar, stacked_bar, line, area, heatmap, scatter, pie) to use assign_slots()
  - Removed old _get_x_dim() and _resolve_auto_facets() functions
  - Updated docstrings to reference dim_priority instead of x_dim_priority

  3. flixopt/statistics_accessor.py

  - Updated _resolve_auto_facets() to use the new assign_slots() function internally
  - Added import for assign_slots from dataset_plot_accessor

  Key Design Decisions

  - Single priority list controls all auto-assignment
  - Slots are filled in fixed order based on availability
  - None means a slot is not available for that plot type
  - 'auto' triggers auto-assignment from priority list
  - Explicit string values override auto-assignment

* Add slot_order to config

* Add new assign_slots() method

* Add new assign_slots() method

* Fix heatmap and convert all to use fxplot

* Fix heatmap

* Fix heatmap

* Fix heatmap

* Fix heatmap

* Squeeze signleton dims in heatmap()

Author: FBumann

CHANGELOG.md

@@ -51,9 +51,12 @@ If upgrading from v2.x, see the [v3.0.0 release notes](https://github.com/flixOp
 
 Until here -->
 
-## [5.1.0] - Upcoming
+## [6.0.0] - Upcoming
 
-**Summary**: Time-series clustering for faster optimization with configurable storage behavior across typical periods. Improved weights API with always-normalized scenario weights.
+**Summary**: Major release introducing time-series clustering with storage inter-cluster linking, the new `fxplot` accessor for universal xarray plotting, and removal of deprecated v5.0 classes. Includes configurable storage behavior across typical periods and improved weights API.
+
+!!! warning "Breaking Changes"
+    This release removes `ClusteredOptimization` and `ClusteringParameters` which were deprecated in v5.0.0. Use `flow_system.transform.cluster()` instead. See [Migration](#migration-from-clusteredoptimization) below.
 
 ### ✨ Added
 
@@ -121,6 +124,44 @@ charge_state = fs_expanded.solution['SeasonalPit|charge_state']
     Use `'cyclic'` for short-term storage like batteries or hot water tanks where only daily patterns matter.
     Use `'independent'` for quick estimates when storage behavior isn't critical.
 
+**FXPlot Accessor**: New global xarray accessors for universal plotting with automatic faceting and smart dimension handling. Works on any xarray Dataset, not just flixopt results.
+
+```python
+import flixopt as fx  # Registers accessors automatically
+
+# Plot any xarray Dataset with automatic faceting
+dataset.fxplot.bar(x='component')
+dataset.fxplot.area(x='time')
+dataset.fxplot.heatmap(x='time', y='component')
+dataset.fxplot.line(x='time', facet_col='scenario')
+
+# DataArray support
+data_array.fxplot.line()
+
+# Statistics transformations
+dataset.fxstats.to_duration_curve()
+```
+
+**Available Plot Methods**:
+
+| Method | Description |
+|--------|-------------|
+| `.fxplot.bar()` | Grouped bar charts |
+| `.fxplot.stacked_bar()` | Stacked bar charts |
+| `.fxplot.line()` | Line charts with faceting |
+| `.fxplot.area()` | Stacked area charts |
+| `.fxplot.heatmap()` | Heatmap visualizations |
+| `.fxplot.scatter()` | Scatter plots |
+| `.fxplot.pie()` | Pie charts with faceting |
+| `.fxstats.to_duration_curve()` | Transform to duration curve format |
+
+**Key Features**:
+
+- **Auto-faceting**: Automatically assigns extra dimensions (period, scenario, cluster) to `facet_col`, `facet_row`, or `animation_frame`
+- **Smart x-axis**: Intelligently selects x dimension based on priority (time > duration > period > scenario)
+- **Universal**: Works on any xarray Dataset/DataArray, not limited to flixopt
+- **Configurable**: Customize via `CONFIG.Plotting` (colorscales, facet columns, line shapes)
+
 ### 💥 Breaking Changes
 
 - `FlowSystem.scenario_weights` are now always normalized to sum to 1 when set (including after `.sel()` subsetting)
@@ -132,12 +173,94 @@ charge_state = fs_expanded.solution['SeasonalPit|charge_state']
 
 ### 🗑️ Deprecated
 
+The following items are deprecated and will be removed in **v7.0.0**:
+
+**Classes** (use FlowSystem methods instead):
+
+- `Optimization` class → Use `flow_system.optimize(solver)`
+- `SegmentedOptimization` class → Use `flow_system.optimize.rolling_horizon()`
+- `Results` class → Use `flow_system.solution` and `flow_system.statistics`
+- `SegmentedResults` class → Use segment FlowSystems directly
+
+**FlowSystem methods** (use `transform` or `topology` accessor instead):
+
+- `flow_system.sel()` → Use `flow_system.transform.sel()`
+- `flow_system.isel()` → Use `flow_system.transform.isel()`
+- `flow_system.resample()` → Use `flow_system.transform.resample()`
+- `flow_system.plot_network()` → Use `flow_system.topology.plot()`
+- `flow_system.start_network_app()` → Use `flow_system.topology.start_app()`
+- `flow_system.stop_network_app()` → Use `flow_system.topology.stop_app()`
+- `flow_system.network_infos()` → Use `flow_system.topology.infos()`
+
+**Parameters:**
+
 - `normalize_weights` parameter in `create_model()`, `build_model()`, `optimize()`
 
+**Topology method name simplifications** (old names still work with deprecation warnings, removal in v7.0.0):
+
+| Old (v5.x) | New (v6.0.0) |
+|------------|--------------|
+| `topology.plot_network()` | `topology.plot()` |
+| `topology.start_network_app()` | `topology.start_app()` |
+| `topology.stop_network_app()` | `topology.stop_app()` |
+| `topology.network_infos()` | `topology.infos()` |
+
+Note: `topology.plot()` now renders a Sankey diagram. The old PyVis visualization is available via `topology.plot_legacy()`.
+
+### 🔥 Removed
+
+**Clustering classes removed** (deprecated in v5.0.0):
+
+- `ClusteredOptimization` class - Use `flow_system.transform.cluster()` then `optimize()`
+- `ClusteringParameters` class - Parameters are now passed directly to `transform.cluster()`
+- `flixopt/clustering.py` module - Restructured to `flixopt/clustering/` package with new classes
+
+#### Migration from ClusteredOptimization
+
+=== "v5.x (Old - No longer works)"
+    ```python
+    from flixopt import ClusteredOptimization, ClusteringParameters
+
+    params = ClusteringParameters(hours_per_period=24, nr_of_periods=8)
+    calc = ClusteredOptimization('model', flow_system, params)
+    calc.do_modeling_and_solve(solver)
+    results = calc.results
+    ```
+
+=== "v6.0.0 (New)"
+    ```python
+    # Cluster using transform accessor
+    fs_clustered = flow_system.transform.cluster(
+        n_clusters=8,           # was: nr_of_periods
+        cluster_duration='1D',  # was: hours_per_period=24
+    )
+    fs_clustered.optimize(solver)
+
+    # Results on the clustered FlowSystem
+    costs = fs_clustered.solution['costs'].item()
+
+    # Expand back to full resolution if needed
+    fs_expanded = fs_clustered.transform.expand_solution()
+    ```
+
 ### 🐛 Fixed
 
 - `temporal_weight` and `sum_temporal()` now use consistent implementation
 
+### 📝 Docs
+
+**New Documentation Pages:**
+
+- [Time-Series Clustering Guide](https://flixopt.github.io/flixopt/latest/user-guide/optimization/clustering/) - Comprehensive guide to clustering workflows
+
+**New Jupyter Notebooks:**
+
+- **08c-clustering.ipynb** - Introduction to time-series clustering
+- **08c2-clustering-storage-modes.ipynb** - Comparison of all 4 storage cluster modes
+- **08d-clustering-multiperiod.ipynb** - Clustering with periods and scenarios
+- **08e-clustering-internals.ipynb** - Understanding clustering internals
+- **fxplot_accessor_demo.ipynb** - Demo of the new fxplot accessor
+
 ### 👷 Development
 
 **New Test Suites for Clustering**:
@@ -147,6 +270,11 @@ charge_state = fs_expanded.solution['SeasonalPit|charge_state']
 - `TestMultiPeriodClustering`: Tests for clustering with periods and scenarios dimensions
 - `TestPeakSelection`: Tests for `time_series_for_high_peaks` and `time_series_for_low_peaks` parameters
 
+**New Test Suites for Other Features**:
+
+- `test_clustering_io.py` - Tests for clustering serialization roundtrip
+- `test_sel_isel_single_selection.py` - Tests for transform selection methods
+
 ---
 
 ## [5.0.4] - 2026-01-05

docs/notebooks/01-quickstart.ipynb

@@ -34,6 +34,7 @@
    "outputs": [],
    "source": [
     "import pandas as pd\n",
+    "import plotly.express as px\n",
     "import xarray as xr\n",
     "\n",
     "import flixopt as fx\n",
@@ -58,7 +59,8 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "timesteps = pd.date_range('2024-01-15 08:00', periods=4, freq='h')"
+    "timesteps = pd.date_range('2024-01-15 08:00', periods=4, freq='h')\n",
+    "print(f'Optimizing from {timesteps[0]} to {timesteps[-1]}')"
    ]
   },
   {
@@ -86,8 +88,9 @@
     "    name='Heat Demand [kW]',\n",
     ")\n",
     "\n",
-    "# Visualize the demand with fxplot accessor\n",
-    "heat_demand.to_dataset().fxplot.bar(title='Heat Demand')"
+    "# Visualize the demand with plotly\n",
+    "fig = px.bar(x=heat_demand.time.values, y=heat_demand.values, labels={'x': 'Time', 'y': 'Heat Demand [kW]'})\n",
+    "fig"
    ]
   },
   {
@@ -200,18 +203,14 @@
    "metadata": {},
    "outputs": [],
    "source": [
+    "total_costs = flow_system.solution['costs'].item()\n",
     "total_heat = float(heat_demand.sum())\n",
     "gas_consumed = total_heat / 0.9  # Account for boiler efficiency\n",
     "\n",
-    "pd.DataFrame(\n",
-    "    {\n",
-    "        'Total heat demand [kWh]': total_heat,\n",
-    "        'Gas consumed [kWh]': gas_consumed,\n",
-    "        'Total costs [EUR]': flow_system.solution['costs'].item(),\n",
-    "        'Average cost [EUR/kWh_heat]': flow_system.solution['costs'].item() / total_heat,\n",
-    "    },\n",
-    "    index=['Value'],\n",
-    ").T"
+    "print(f'Total heat demand: {total_heat:.1f} kWh')\n",
+    "print(f'Gas consumed: {gas_consumed:.1f} kWh')\n",
+    "print(f'Total costs: {total_costs:.2f} €')\n",
+    "print(f'Average cost: {total_costs / total_heat:.3f} €/kWh_heat')"
    ]
   },
   {

docs/notebooks/02-heat-system.ipynb

@@ -32,7 +32,9 @@
    "metadata": {},
    "outputs": [],
    "source": [
+    "import numpy as np\n",
     "import pandas as pd\n",
+    "import plotly.express as px\n",
     "import xarray as xr\n",
     "\n",
     "import flixopt as fx\n",
@@ -57,12 +59,33 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "from data.tutorial_data import get_heat_system_data\n",
+    "# One week, hourly resolution\n",
+    "timesteps = pd.date_range('2024-01-15', periods=168, freq='h')\n",
     "\n",
-    "data = get_heat_system_data()\n",
-    "timesteps = data['timesteps']\n",
-    "heat_demand = data['heat_demand']\n",
-    "gas_price = data['gas_price']"
+    "# Create realistic office heat demand pattern\n",
+    "hours = np.arange(168)\n",
+    "hour_of_day = hours % 24\n",
+    "day_of_week = (hours // 24) % 7\n",
+    "\n",
+    "# Base demand pattern (kW)\n",
+    "base_demand = np.where(\n",
+    "    (hour_of_day >= 7) & (hour_of_day <= 18),  # Office hours\n",
+    "    80,  # Daytime\n",
+    "    30,  # Night setback\n",
+    ")\n",
+    "\n",
+    "# Reduce on weekends (days 5, 6)\n",
+    "weekend_factor = np.where(day_of_week >= 5, 0.5, 1.0)\n",
+    "heat_demand = base_demand * weekend_factor\n",
+    "\n",
+    "# Add some random variation\n",
+    "np.random.seed(42)\n",
+    "heat_demand = heat_demand + np.random.normal(0, 5, len(heat_demand))\n",
+    "heat_demand = np.clip(heat_demand, 20, 100)\n",
+    "\n",
+    "print(f'Time range: {timesteps[0]} to {timesteps[-1]}')\n",
+    "print(f'Peak demand: {heat_demand.max():.1f} kW')\n",
+    "print(f'Total demand: {heat_demand.sum():.0f} kWh')"
    ]
   },
   {
@@ -72,13 +95,15 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "# Visualize the demand pattern with fxplot\n",
-    "demand_ds = xr.Dataset(\n",
-    "    {\n",
-    "        'Heat Demand [kW]': xr.DataArray(heat_demand, dims=['time'], coords={'time': timesteps}),\n",
-    "    }\n",
+    "# Visualize the demand pattern with plotly\n",
+    "demand_series = xr.DataArray(heat_demand, dims=['time'], coords={'time': timesteps}, name='Heat Demand [kW]')\n",
+    "fig = px.line(\n",
+    "    x=demand_series.time.values,\n",
+    "    y=demand_series.values,\n",
+    "    title='Office Heat Demand Profile',\n",
+    "    labels={'x': 'Time', 'y': 'kW'},\n",
     ")\n",
-    "demand_ds.fxplot.line(title='Office Heat Demand Profile')"
+    "fig"
    ]
   },
   {
@@ -98,13 +123,15 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "# Visualize gas price with fxplot\n",
-    "price_ds = xr.Dataset(\n",
-    "    {\n",
-    "        'Gas Price [EUR/kWh]': xr.DataArray(gas_price, dims=['time'], coords={'time': timesteps}),\n",
-    "    }\n",
+    "# Time-of-use gas prices (€/kWh)\n",
+    "gas_price = np.where(\n",
+    "    (hour_of_day >= 6) & (hour_of_day <= 22),\n",
+    "    0.08,  # Peak: 6am-10pm\n",
+    "    0.05,  # Off-peak: 10pm-6am\n",
     ")\n",
-    "price_ds.fxplot.line(title='Gas Price')"
+    "\n",
+    "fig = px.line(x=timesteps, y=gas_price, title='Gas Price [€/kWh]', labels={'x': 'Time', 'y': '€/kWh'})\n",
+    "fig"
    ]
   },
   {
@@ -282,16 +309,12 @@
    "metadata": {},
    "outputs": [],
    "source": [
+    "total_costs = flow_system.solution['costs'].item()\n",
     "total_heat = heat_demand.sum()\n",
     "\n",
-    "pd.DataFrame(\n",
-    "    {\n",
-    "        'Total operating costs [EUR]': flow_system.solution['costs'].item(),\n",
-    "        'Total heat delivered [kWh]': total_heat,\n",
-    "        'Average cost [ct/kWh]': flow_system.solution['costs'].item() / total_heat * 100,\n",
-    "    },\n",
-    "    index=['Value'],\n",
-    ").T"
+    "print(f'Total operating costs: {total_costs:.2f} €')\n",
+    "print(f'Total heat delivered: {total_heat:.0f} kWh')\n",
+    "print(f'Average cost: {total_costs / total_heat * 100:.2f} ct/kWh')"
    ]
   },
   {