Phase 1 (OS refactoring) and provides the initial backend architecture for Phase 2 (OG-Core runner)

[suyash469]

Summary
What changed:

Refactored hardcoded Windows OS paths in the OsemosysClass solver logic to allow dynamic cross-platform execution supporting Windows, Linux, and macOS (Darwin).
Introduced the
OGCoreClass
runner and OGCoreRoute endpoints into the Flask API. This creates the foundational backend structure necessary for Phase 2 of the OG-CLEWS integration roadmap by enabling the execution of
ogcore
Time Path models via asynchronous multiprocessing.
Why: This PR directly addresses Phase 1 (Cross-Platform Refactoring) and Phase 2 (OG-Core Module) of the UN OICT roadmap. The OS-abstraction allows the community to utilize native solvers on any machine, while the new
OGCoreClass
orchestrator hooks the Python mathematical models correctly into the MUIO Flask app endpoints for future UI coupling.

Related issues
Issue exists and is linked
Closes # 171
Related # 122
Validation
Tests added/updated (or not applicable)
Validation steps documented
Evidence attached (logs/screenshots/output as relevant)
Validation Steps & Evidence:

Ran
app.py
locally and verified endpoints /ogcore/run initialize the Dask multiprocessing client.
Verified if platform.system() == 'Windows' logic gracefully falls back to Linux/Darwin paths for solver lookups.
Documentation
Docs updated in this PR (or not applicable)
Any setup/workflow changes reflected in repo docs
Scope check
No unrelated refactors
Implemented from a feature branch
Change is deliverable without upstream OSeMOSYS/MUIO dependency
Base repo/branch is EAPD-DRB/MUIOGO:main (not upstream)

[NamanmeetSingh]

Hi, great work getting the foundation laid out for the OG-Core runner! Hooking up the Dask multiprocessing client is a huge step forward for Phase 2.

I was reviewing this to see how it integrates with the converging pipeline I am building in PR #24, and I noticed a few items and architectural bottlenecks we should address before merging so we don't block the next phase of development:

Integration & Architecture Feedback:

Synchronous Execution Timeout: Running og_runner.execute_run() blockingly inside the /ogcore/run route will likely cause HTTP gateway timeouts. Since OG-Core time paths can take minutes to converge, the browser will drop the connection before the model finishes. We should either offload this to a background worker (like Celery/RQ) and return a task_id for the frontend to poll, or run it in a detached thread for now.

Hardcoded Time Horizons: In _generate_outputs(), num_years=10 is currently hardcoded for the CSV generation. Since CLEWS models typically run projections out to 2050/2070, we will need OG-Core to output arrays matching that length. Could we expose num_years as an optional parameter in og_spec so the converging pipeline can synchronize the time horizons between both models?

Dask Client Initialization: Spawning a new Dask Client() directly inside the request handler might lead to port collisions ("Address already in use") if multiple runs are triggered concurrently. We might want to pass processes=False or specify a dedicated cluster address to make it thread-safe for web API usage.

Let me know if you want to collaborate on the async/threading implementation! Making sure these I/O boundaries are flexible will make coupling this with the OSeMOSYS outputs much smoother.

[SeaCelo]

Please update the PR body to include a real linked issue if you want this to continue in review. The current text still has placeholder issue fields and no actual linked issue. If there is not already a dedicated issue for this scope, please open one first and then update this PR with a real Closes #... or Related #... reference.

[suyash469]

Please update the PR body to include a real linked issue if you want this to continue in review. The current text still has placeholder issue fields and no actual linked issue. If there is not already a dedicated issue for this scope, please open one first and then update this PR with a real Closes #... or Related #... reference.

Thanks for the review! I'll open a dedicated issue for this scope now and update
the PR body with the proper reference.

[suyash469]

Please update the PR body to include a real linked issue if you want this to continue in review. The current text still has placeholder issue fields and no actual linked issue. If there is not already a dedicated issue for this scope, please open one first and then update this PR with a real Closes #... or Related #... reference.

Done — opened #171 for this scope and updated the PR body to reference it.

[suyash469]

Hi, great work getting the foundation laid out for the OG-Core runner! Hooking up the Dask multiprocessing client is a huge step forward for Phase 2.

I was reviewing this to see how it integrates with the converging pipeline I am building in PR #24, and I noticed a few items and architectural bottlenecks we should address before merging so we don't block the next phase of development:

Integration & Architecture Feedback:

Synchronous Execution Timeout: Running og_runner.execute_run() blockingly inside the /ogcore/run route will likely cause HTTP gateway timeouts. Since OG-Core time paths can take minutes to converge, the browser will drop the connection before the model finishes. We should either offload this to a background worker (like Celery/RQ) and return a task_id for the frontend to poll, or run it in a detached thread for now.

Hardcoded Time Horizons: In _generate_outputs(), num_years=10 is currently hardcoded for the CSV generation. Since CLEWS models typically run projections out to 2050/2070, we will need OG-Core to output arrays matching that length. Could we expose num_years as an optional parameter in og_spec so the converging pipeline can synchronize the time horizons between both models?

Dask Client Initialization: Spawning a new Dask Client() directly inside the request handler might lead to port collisions ("Address already in use") if multiple runs are triggered concurrently. We might want to pass processes=False or specify a dedicated cluster address to make it thread-safe for web API usage.

Let me know if you want to collaborate on the async/threading implementation! Making sure these I/O boundaries are flexible will make coupling this with the OSeMOSYS outputs much smoother.

Thanks for the detailed review! All three points are valid:

1. Async execution — agreed, blocking inside the Flask route is not
   production-ready. Will offload to a background thread in a follow-up PR.

2. num_years — will expose this as an optional parameter in og_spec so
   _generate_outputs() syncs time horizons with CLEWS.

3. Dask Client — will add processes=False to Client() initialization to
   prevent port collisions on concurrent requests.

Happy to collaborate on the async implementation!

[NamanmeetSingh]

@suyash469 Awesome, that will make the integration incredibly smooth.

Regarding the async execution, you actually don't need to build a new background worker from scratch. Over the weekend, the team extracted the async pipeline into a generic TaskManager singleton (currently sitting in PR #146).

Once PR #146 merges, you can just submit your og_runner.execute_run directly to that queue. It will automatically handle the 202 Accepted response and expose the /status/<task_id> polling endpoint for the frontend, saving you the hassle of rewriting the threading logic!

Exposing num_years and securing the Dask Client will perfectly unblock the Track 1 convergence math on my end. Once you update the og_spec to accept those dynamic time horizons, my ConvergingOrchestrator can seamlessly map the Pandas arrays right into your runner.