Feat/clean up old rolling forecasting code#2082
Open
BelhsanHmida wants to merge 16 commits intomainfrom
Open
Conversation
Context: - Fixed-viewpoint forecasting is the supported execution path - Rolling job helpers and horizon tables were left behind in shared utilities Change: - Remove legacy rolling forecasting service entry points - Keep worker exception handling but stop re-enqueueing unsupported fallback jobs - Delete unused rolling horizon utility helpers
Context: - Some tests still created forecasting jobs through the removed rolling job helper - These tests only need queue metadata, not rolling-forecast behavior Change: - Build forecasting jobs with TrainPredictPipeline in cache/schema tests - Update assertions to match the supported forecasting queue shape
Context: - The old forecasting job test modules were centered on removed rolling behavior and fallback chaining - We still need coverage for queue metadata, failure handling, and successful fixed-viewpoint execution Change: - Rewrite forecasting job tests around TrainPredictPipeline jobs - Verify failed jobs stay failed without spawning fallback jobs - Keep fresh-db coverage for queued fixed-viewpoint forecasting
Context: - The developer forecasting helper still queued work through the removed rolling entry point - Pre-commit also refreshed the generated OpenAPI spec in this branch Change: - Switch the manual testing helper to TrainPredictPipeline jobs - Keep the generated OpenAPI spec in sync
Documentation build overview
Show files changed (13 files in total): 📝 9 modified | ➕ 0 added | ➖ 4 deleted
|
Signed-off-by: Mohamed Belhsan Hmida <mohamedbelhsanhmida@gmail.com>
Signed-off-by: Mohamed Belhsan Hmida <mohamedbelhsanhmida@gmail.com>
Signed-off-by: Mohamed Belhsan Hmida <mohamedbelhsanhmida@gmail.com>
Signed-off-by: Mohamed Belhsan Hmida <mohamedbelhsanhmida@gmail.com>
Signed-off-by: Mohamed Belhsan Hmida <mohamedbelhsanhmida@gmail.com>
Signed-off-by: Mohamed Belhsan Hmida <mohamedbelhsanhmida@gmail.com>
Signed-off-by: Mohamed Belhsan Hmida <mohamedbelhsanhmida@gmail.com>
Contributor
There was a problem hiding this comment.
Pull request overview
This PR completes the fixed-point forecasting migration by removing legacy rolling-forecasting infrastructure (including the timetomodel integration) and updating tests/tooling to queue forecasting work via TrainPredictPipeline, while also refreshing the generated OpenAPI specs.
Changes:
- Remove the old rolling forecasting job creation + helper utilities and the
timetomodeldependency. - Update forecasting-related tests and developer CLI tooling to use
TrainPredictPipeline. - Adjust forecasting exception handling to stop enqueueing legacy fallback jobs; refresh OpenAPI specs.
Reviewed changes
Copilot reviewed 16 out of 18 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
uv.lock |
Drops timetomodel from the resolved dependency set. |
pyproject.toml |
Removes the direct timetomodel dependency. |
flexmeasures/utils/time_utils.py |
Removes unused rolling-horizon helper functions. |
flexmeasures/ui/static/openapi-specs.json |
Updates generated OpenAPI version metadata. |
flexmeasures/data/tests/test_job_cache.py |
Switches forecasting job cache test to TrainPredictPipeline-queued jobs. |
flexmeasures/data/tests/test_forecasting_jobs.py |
Replaces rolling-forecasting job tests with pipeline-based job queuing/failure assertions. |
flexmeasures/data/tests/test_forecasting_jobs_fresh_db.py |
Updates fresh-DB job execution test to use pipeline jobs. |
flexmeasures/data/tests/conftest.py |
Removes legacy model-spec fixtures tied to the old forecasting stack. |
flexmeasures/data/services/forecasting.py |
Removes legacy forecasting job creation/execution code; keeps exception handler and disables fallback enqueueing. |
flexmeasures/data/scripts/data_gen.py |
Removes legacy forecast population code built on the old forecasting stack. |
flexmeasures/data/models/forecasting/utils.py |
Removes old rolling-forecasting utility functions, keeping only shared helpers still in use. |
flexmeasures/data/models/forecasting/model_specs/naive.py |
Deletes legacy naive model spec implementation. |
flexmeasures/data/models/forecasting/model_specs/linear_regression.py |
Deletes legacy linear regression model spec implementation. |
flexmeasures/data/models/forecasting/model_spec_factory.py |
Deletes the timetomodel-based model spec factory. |
flexmeasures/data/models/forecasting/__init__.py |
Removes legacy model-map lookup API tied to deleted model specs/factory. |
flexmeasures/cli/testing.py |
Updates developer manual forecasting job test to enqueue pipeline jobs. |
flexmeasures/api/common/schemas/tests/test_sensor_data_schema.py |
Updates jobs-metadata test setup to enqueue forecasting jobs via the pipeline. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Context: - Copilot review pointed out that raw exception instances in rq job meta are not JSON-serializable. - rq-dashboard and job meta consumers rely on this metadata being safe to serialize. Change: - Store forecasting exception metadata as a JSON-safe dict with type and message. - Keep start and end timestamps normalized before saving job meta. Signed-off-by: Mohamed Belhsan Hmida <mohamedbelhsanhmida@gmail.com>
Signed-off-by: Mohamed Belhsan Hmida <mohamedbelhsanhmida@gmail.com>
Signed-off-by: Mohamed Belhsan Hmida <mohamedbelhsanhmida@gmail.com>
Signed-off-by: Mohamed Belhsan Hmida <149331360+BelhsanHmida@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
TrainPredictPipelineHow to test
pytest -q flexmeasures/data/tests/test_forecasting_jobs.py flexmeasures/data/tests/test_forecasting_jobs_fresh_db.py flexmeasures/data/tests/test_job_cache.py flexmeasures/api/common/schemas/tests/test_sensor_data_schema.pypytest -q flexmeasures/data/tests/test_forecasting_pipeline.py flexmeasures/api/v3_0/tests/test_forecasting_api.pypre-commit run --all-filesRelated Items
Sign-off