Add JobABC for submitting batch jobs#294
Open
TimothyWillard wants to merge 1 commit into
Open
Conversation
There was a problem hiding this comment.
Pull request overview
This PR introduces a new “job” module namespace to allow submitting CliCommands for execution via pluggable backends, adds a top-level jobs section to the configuration model, and includes a built-in ShellJob implementation plus tests.
Changes:
- Added
JobABC+JobHandleabstractions and module builder for the newjobmodule namespace. - Added
ShellJobbackend to execute CLI commands viasubprocess(detached or synchronous). - Extended
ConfigurationModelwith a new top-leveljobssection and updated module loader namespace typing; added changelog + tests.
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
src/flepimop2/job/abc/__init__.py |
Introduces JobABC and JobHandle plus a build() helper for job backends. |
src/flepimop2/job/shell/__init__.py |
Adds ShellJob backend that runs flepimop2 commands via subprocess. |
src/flepimop2/configuration/_configuration.py |
Adds jobs as a top-level configuration section and integrates it into patching/ordering. |
src/flepimop2/_utils/_module.py |
Extends the module namespace literal type to include "job". |
tests/job/abc/test_job_handle.py |
Adds unit tests for JobHandle string formatting and metadata defaults. |
tests/job/shell/test_shell_job.py |
Adds unit tests for ShellJob validation and submission behavior in detached/sync modes. |
CHANGELOG.md |
Documents the addition of jobs infrastructure and ShellJob. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+198
to
+200
| def test_submit_synchronous_returns_single_handle() -> None: | ||
| """Submit in synchronous mode should also return a one-element list.""" | ||
| job = ShellJob.model_validate({"module": "flepimop2.job.shell", "detach": False}) |
Comment on lines
34
to
40
| "engines", | ||
| "systems", | ||
| "backends", | ||
| "jobs", | ||
| "process", | ||
| "parameters", | ||
| "scenarios", |
Added the `JobABC` module type that consumes `CliCommand`s and then runs them in a seperate environment, with future use cases including HPC environments like on slurm. Also added `jobs` top level configuration key to correspond to this new module type and the `ShellJob` class that just launches the command in a subprocess as an example/simple way to introduce users to jobs. Closes #276. Closes #277. Closes #278.
Collaborator
Author
|
Addressed additional copilot review comments in https://github.com/ACCIDDA/flepimop2/compare/43ce01fb894ac1552b869a7bbeb9d581e22a936f..6cad98c689ecec55fdf2daaa4e0a9a85d5acd74b |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Added the
JobABCmodule type that consumesCliCommands and then runs them in a seperate environment, with future use cases including HPC environments like on slurm. Also addedjobstop level configuration key to correspond to this new module type and theShellJobclass that just launches the command in a subprocess as an example/simple way to introduce users to jobs.Related issues
Closes #276. Closes #277. Closes #278.
Checklist
just ci.CHANGELOG.mdor noted "no major changes" in my commit if the PR is small.