REF cleaner BaseTSFMSolver with richer default adaptation strategies by rtavenar · Pull Request #42 · benchopt/benchmark_tsfm

rtavenar · 2026-06-04T11:49:23Z

Summary (cf #39 )

Abstract model_id property + base-class model caching: the loaded model is now reused across set_objective calls and only reloaded when model_id changes (previously each call reloaded).
build_adapter is no longer abstract: a non-abstract default covers all four tasks using the first available capability (see table below). Solvers only need to override it for genuinely custom adapter logic.
embed() / time_embed() public methods: wrap embed_batch / time_embed_batch with numpy conversion, making them directly usable in adapters and subclasses.
event_detection added to the _Task type alias.
forecast() type fix: numpy inputs are now converted to tensors before forecast_batch, and tensor outputs converted back to numpy before assembly (was a silent type mismatch).
Docstring fixes: swapped docstrings on embed_batch / time_embed_batch corrected; build_adapter docstring now lists numbered fallback options instead of ambiguous →.

Default adaptation strategies in `build_adapter` (discussed in #31 )

Task	Option 1	Option 2 (fallback)
`forecasting`	`forecast_batch` (zero-shot)	`embed_batch` → windowed ridge regression
`classification`	`embed_batch` → LinearProbe	`time_embed_batch` → mean-pooled + LinearProbe
`anomaly_detection`	`embed_batch` → distance-from-mean	`forecast_batch` → forecast-error residuals
`event_detection`	`time_embed_batch` → per-position LogReg	`embed_batch` → causal-windowed LogReg

Test plan

Existing solvers (Chronos, Moment, …) still extend BaseSolver directly and are unaffected by this PR
A new solver implementing only forecast_batch + supported_tasks + model_id + load_model should work end-to-end for forecasting and anomaly detection without any further code
Model is not reloaded across tasks when model_id is unchanged

🤖 Generated with Claude Code

- Add abstract model_id property; base class now caches the loaded model across set_objective calls (no reload unless model_id changes). - Replace abstract build_adapter with a non-abstract default that picks the first available capability per task: forecasting forecast_batch → windowed-embed ridge regression classification embed_batch → pooled time_embed + LinearProbe anomaly_detection embed_batch → forecast residuals event_detection time_embed → causal-windowed embed + LogReg - Add event_detection to the _Task type alias. - Add embed() and time_embed() public methods (call embed_batch / time_embed_batch and convert to numpy). - Fix forecast() to convert numpy inputs to tensors before forecast_batch and tensor outputs back to numpy before assembly. - Fix swapped docstrings on embed_batch / time_embed_batch. - Drop unused Callable import. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Reconciles the shape unification from main (benchopt#28) with the BaseTSFMSolver refactor on this branch: - Adopt TaskType alias (renamed from _Task) and ForecastOutput shape (n_cutoffs, H, C, Q) from main; update _WindowedForecastAdapter accordingly. - Add covariates parameter to forecast_batch / forecast() from main. - Import Covariates from benchmark_utils.covariates. - Carry forward all branch additions: model_id caching, non-abstract build_adapter with default strategies, embed/time_embed methods, and event_detection task type. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

felixdivo

Overall, this is a super nice set of changes!

I LOVE the simplicity of the concrete solvers 👌

felixdivo · 2026-06-13T11:18:08Z

+        forecasting
+            1. forecast_batch  (zero-shot, via _SolverForecastAdapter)
+            2. embed_batch     (windowed ridge regression, via _WindowedForecastAdapter)
+        classification
+            1. embed_batch     (flat embedding + LinearProbeAdapter)
+            2. time_embed_batch (mean-pooled temporal embedding + LinearProbeAdapter)
+        anomaly_detection
+            1. embed_batch     (distance-from-mean score, via LinearProbeAdapter)
+            2. forecast_batch  (forecast-error score, via ForecastResidualAdapter)
+        event_detection
+            1. time_embed_batch (per-position LogReg, via _TimeEmbedEventAdapter)
+            2. embed_batch      (causal-windowed LogReg, via _WindowedEventAdapter)


The strategy makes sense! However, this is very important behavioral documentation that is somewhat hidden. We should at least add a TODO to move this to a "how to add a model" docs at some point.

Maybe we should add such a documentation page anyway soon? Also, AGENTS.md could point to that docs page to make it easier for coding agents to add models.

100% agree

It's a TODO at the moment, but it would definitely make sense to cover it in this PR.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…vels, docstrings Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…Toto2 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…ertsfm # Conflicts: # solvers/mantis.py

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

felixdivo · 2026-06-15T08:48:44Z

+        # it overrides the `forecast_batch` method
+        return type(self).forecast_batch is not BaseTSFMSolver.forecast_batch
+
+    def get_quantile_levels(self) -> tuple[float, ...]:


Would it make sense to have this be a property? I think so, and in case it is a bit of work to look this up in the model config, it could then be easily cached with @functools.cached_property.

…ertsfm

…tion) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

felixdivo

Else, looks good to merge. Wonderful improvement!

Chronos2's lookup depends on self.model.quantiles (only available post-load_model), so it uses functools.cached_property; the other solvers expose static tuples via a plain property.

tomMoral

Looks great! thanks @rtavenar

rtavenar and others added 5 commits June 4, 2026 13:46

Chronos2 inherits from BaseTSFMSolver

e6291aa

bugfix

08653bb

FIX handle list and 1-D inputs in embed/time_embed encoders

372043b

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

rtavenar requested a review from felixdivo June 12, 2026 17:17

felixdivo requested changes Jun 13, 2026

View reviewed changes

This was referenced Jun 13, 2026

ADD T0 (t0-alpha) zero-shot forecasting solver #46

Open

[WIP] Adding adapter for event detection on Chronos v1 #35

Open

REF migrate Toto2, Moment, Mantis solvers to BaseTSFMSolver

27d2f19

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

felixdivo mentioned this pull request Jun 13, 2026

Add TabICL-v2 solver and UEA multivariate dataset loader #19

Open

rtavenar and others added 5 commits June 15, 2026 10:26

REF address PR review: types, match/case, sklearn import, quantile_le…

8a0106c

…vels, docstrings Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

REF migrate Chronos v1 to BaseTSFMSolver; add get_quantile_levels to …

6848afc

…Toto2 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Merge remote-tracking branch 'origin/main' into feat-cleaner-basesolv…

62730a6

…ertsfm # Conflicts: # solvers/mantis.py

STY fix E501 line-too-long errors

adbb821

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

REF remove sampling_strategy from BaseTSFMSolver (kept in objective.py)

7b02ec1

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

felixdivo reviewed Jun 15, 2026

View reviewed changes

rtavenar and others added 2 commits June 15, 2026 16:28

Merge remote-tracking branch 'origin/main' into feat-cleaner-basesolv…

9ad8636

…ertsfm

TST remove ChronosEncoder test (class removed in BaseTSFMSolver migra…

fc19d8b

…tion) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

felixdivo approved these changes Jun 16, 2026

View reviewed changes

REF convert get_quantile_levels() into a quantile_levels property

8fcbf58

Chronos2's lookup depends on self.model.quantiles (only available post-load_model), so it uses functools.cached_property; the other solvers expose static tuples via a plain property.

rtavenar merged commit fa46139 into benchopt:main Jun 16, 2026
5 checks passed

This was referenced Jun 16, 2026

Implement embed and time_embed for the BaseTSFMSolver #39

Open

Elements to be documented #52

Open

Design choices on the solver side #31

Closed

tomMoral reviewed Jun 16, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

REF cleaner BaseTSFMSolver with richer default adaptation strategies#42

REF cleaner BaseTSFMSolver with richer default adaptation strategies#42
rtavenar merged 14 commits into
benchopt:mainfrom
rtavenar:feat-cleaner-basesolvertsfm

rtavenar commented Jun 4, 2026 •

edited

Loading

Uh oh!

felixdivo left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

felixdivo Jun 13, 2026

Uh oh!

rtavenar Jun 15, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

felixdivo Jun 15, 2026 •

edited

Loading

Uh oh!

felixdivo left a comment

Uh oh!

Uh oh!

tomMoral left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

rtavenar commented Jun 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary (cf #39 )

Default adaptation strategies in build_adapter (discussed in #31 )

Test plan

Uh oh!

felixdivo left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

felixdivo Jun 13, 2026

Choose a reason for hiding this comment

Uh oh!

rtavenar Jun 15, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

felixdivo Jun 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

felixdivo left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

tomMoral left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

rtavenar commented Jun 4, 2026 •

edited

Loading

Default adaptation strategies in `build_adapter` (discussed in #31 )

felixdivo Jun 15, 2026 •

edited

Loading