v1.0.0

First stable release of jobflow-remote.

Most notable changes since last release:

• Renamed TERMINATED state to RUN_FINISHED
• refactoring of the internal representation for batch jobs
• dropped support for python 3.9
• minor bugfixes
• improvements in the CLI

Requires the execution of jf admin upgrade after the installation with runner not active.
It is advisable that no batch jobs are running at the moment of the upgrade. In that case the output of those jobs can be lost.

Changelog

v1.0.0 (2026-01-14)

Full Changelog

Closed issues:

• Feature Request: Running Flows on Heterogeneous Resources #419
• How to run specific slurm script on server? #417
• Mixmatch between Runner log and submission file leading to error #397
• More than the max_jobs number of parallel batch jobs are submitted, after some time #391
• Runs fail the first time #379
• setting number of nodes and process in PBS jobs #378
• jf runner stop does not kill all jf processes when starting a new SSH session in a very specific way #373
• jf project check --errors does not check if the Slurm resources keys are valid but perhaps should #371
• Incorrect Environment Variable Name in Documentation (JFREMOTE_PROJECT_FOLDER vs JFREMOTE_PROJECTS_FOLDER) #362
• jobflow_remote.config.base.ConfigError: ExecutionConfig with id example_config is not defined #358
• Don't rely on there being an additional_stores field (e.g. with Atomate2) #331
• Unifying jobstore in Jobflow-Remote and JOB_STORE in Jobflow #328
• Python package checks during jf project check --errors are being done in the base environment #318
• Feature request: Pause flow to wait for user interaction #289
• Better error handling for incorrectly formatted project YAML #286
• name of "TERMINATED" state #101
• Suggestion: PSI/J for Interfacing with the Job Scheduler #10

Merged pull requests:

• remove contributing.rst #441 (gpetretto)
• Update release action #440 (gpetretto)
• qtoolkit 0.1.7 #437 (gpetretto)
• fix test and timezones #434 (gpetretto)
• Updates for batch jobs management #425 (gpetretto)
• Do not support python 3.14.1, as it breaks networkx #424 (gpetretto)
• Fix dev docs #422 (gpetretto)
• Runner updates #416 (gpetretto)
• Allow jf job set-state for many Jobs at once #413 (gpetretto)
• Fix job names in tests to prevent some random test failures #412 (davidwaroquiers)
• Add warn option #407 (FabiPi3)
• Remove support for python 3.9 #406 (gpetretto)
• fix upgrade to 1.0 #404 (gpetretto)
• Setting up uv to do lowest-direct and highest deps resolutions tests. #402 (davidwaroquiers)
• Replace --force with --yes #400 (gpetretto)
• Report option for jf flow info #399 (gpetretto)
• Update set_run_config to allow multiple changes #396 (gpetretto)
• Fix for batch submission #392 (davidwaroquiers)
• Add CLI option to edit project files #390 (gpetretto)
• CLI commands to remove job folders #388 (gpetretto)
• Changed TERMINATED state to RUN_FINISHED. #384 (davidwaroquiers)
• Added the possibility to have a custom scheduler io #383 (davidwaroquiers)
• Bugfix and small updates #376 (gpetretto)
• Clearer error message for undefined active project #374 (Andrew-S-Rosen)
• Improve warning message for flow reset validation #372 (Andrew-S-Rosen)
• Initial implementation of metadata in flow info. #366 (davidwaroquiers)
• Added check of environment variables in jf project check. #364 (davidwaroquiers)
• Removed auxiliary based management of batch jobs #363 (davidwaroquiers)
• Updates to batch process #354 (davidwaroquiers)

* This Changelog was automatically generated by github_changelog_generator

v0.1.8

Several updates and fixes.
Most important changes:

• It is now possible to resume STOPPED jobs.
• New option to extend the jf CLI from other packages.

Changelog

v0.1.8 (2025-09-23)

Full Changelog

Closed issues:

• UnboundLocalError when running jf job files get #350
• Allow for the omission of a jobstore altogether #332
• Parallel batch job shows fewer BATCH_RUNNING jobs than expected after jf admin reset #326
• The built docs render -- as – #322
• Feature Request: Flow_DB_ID option for jf job list command #320
• Unexpected keyword argument 'sort' #298
• Freshly created replaced jobs should be stopped as well #291
• Using atomate2 with jobflow-remote example failed to run: NBANDS seems to be too high. #280

Merged pull requests:

• Small fixes and tests improvements #347 (gpetretto)
• Fixture for run_check_cli that includes patching the consoles #346 (davidwaroquiers)
• CLI plugins #339 (gpetretto)
• Fix batch worker #334 (gpetretto)
• Optional jobstores #327 (gpetretto)
• Dw/test cli colors #324 (davidwaroquiers)
• Merge custom and automatic queries #321 (FabiPi3)
• Minor updates: color job list, allow resuming STOPPED jobs #316 (gpetretto)
• Include coverage from code executed remotely #255 (davidwaroquiers)

* This Changelog was automatically generated by github_changelog_generator

v0.1.7

Updates and bugfix

Changelog

v0.1.7 (2025-06-25)

Full Changelog

Implemented enhancements:

• SUGGESTION: --count option for jf flow list and jf job list #110

Closed issues:

• how to best cite jobflow-remote? #290
• Feature Request: New UX for setting job priority #287
• The long time interval between two jobs of a flow. #278
• The cores in the submit.sh is different from the one shown in the OUTCAR. #277
• Set a different work_dir from the one defined in the configuration file at runtime. #276
• Put CECAM slides in the docs? #273
• Where can we find information about the Slurm job metadata? #268
• How to dynamically change the worker configuration #267
• The run_dir text is often cutoff in the jf job info ID print-out #266
• Warn user to not use the same database for queue and output store #264
• How to solve the question: TypeError: 'dict' object is not callable #263
• a weird problem with job metadata #261
• How to update failed jobs input parameters ? #125

Merged pull requests:

• bugfix and test release workflow #313 (gpetretto)
• Update release.yml #310 (davidwaroquiers)
• Add --count option in CLI #307 (gpetretto)
• Fix testing, upgrade test requirements #302 (gpetretto)
• fix ignoring resolve_references in JobConfig #288 (gpetretto)
• CLI and other small updates #269 (gpetretto)
• Reset stored data field on rerun #260 (FabiPi3)
• Docker optimisations for CI #247 (ml-evs)

* This Changelog was automatically generated by github_changelog_generator

v0.1.6

Bug fix release: pin pymongo version to lower than 4.11 to avoid incompatibility with maggma

Changelog

v0.1.6 (2025-02-07)

Full Changelog

Closed issues:

• Replacing failed jobs and adding new jobs to a flow #250

Merged pull requests:

• Bug fix for Remote Host #253 (gpetretto)
• Hot fix to pin pymongo version to lower than 4.11. #251 (davidwaroquiers)

* This Changelog was automatically generated by github_changelog_generator

v0.1.5

Several updates, new features and bug fixes to improve user experience, configuration and maintenance.

Most notable changes:

• Added an upgrade to handle backward incompatible changes. From now on, after updating to a later version of jobflow-remote, users are suggested to run the jf admin upgrade command for all the projects that may be related.
• The batch submission mode can run multiple Jobs in parallel
• An experimental version of a GUI has been developed based on FastHTML. Start it with jf gui.
• A backup functionality for the queue database, accessible through jf backup in the CLI
• Reporting of the executed jobs over specific time intervals for Jobs and Flows: jf job report and jf flow report

Changelog

v0.1.5 (2025-01-29)

Full Changelog

Implemented enhancements:

• Optionally delay download after job end #210
• Add a backup feature #165

Closed issues:

• Print only with no wait #245
• Mismatched dumped taskdoc and database state #231
• Check status for simulation code in jf job list #227
• Successful Response not being written to jfremote_out.json #226
• jf admin upgrade causes a lot of errors #224
• QToolKit is currently pinned to development version #218
• Full-split : Remote Runner setup #214
• An Error occurred during the command execution: ValueError No daemon runner document. #211
• "Error executing jf project check --errors to connect to <cluster> (no additional info)" #208
• how to rerun a job but with improved memory #198
• Delete VASP files when rerunnning jobs? #197
• call jf from container #196
• how to limit number of jobs #195
• non-unique uuids as a problem in jobflow-remote #193
• Set projects_folder via env var #188
• Support for heterogenous computing resources? #184
• Jobflow remote logo #178
• How could I use SGE for job submission? #159
• Check there is not already a runner running #140
• How to add metadata to flows docs? #124
• Job submission fails based on job name #112
• preventing reset issues #102
• Missing docs: batch mode #96
• Does jobflow-remote support the pilot job model? #86

Merged pull requests:

• bump qtoolkit 0.1.6 #248 (gpetretto)
• Make JobDoc available at runtime #244 (gpetretto)
• Finalize gui implementation #243 (gpetretto)
• fix typing for submit_flow #242 (FabiPi3)
• Web GUI for jobflow remote #241 (fraricci)
• Allow switching off daemon if no running runner doc #240 (gpetretto)
• Logo #239 (gpetretto)
• [WIP] Save db dumps for test failures #237 (gpetretto)
• Various updates #234 (gpetretto)
• [WIP] test direct execution of docker commands and addition of PBS container #233 (gpetretto)
• Testing adding flags for codecov, separating unit and integration tests. #232 (davidwaroquiers)
• Option for stored data in jf job list #228 (FabiPi3)
• Delete job files when rerunning; customizable execution command #201 (gpetretto)
• Add sanitization option to host outputs #191 (gpetretto)
• Add backup functionality #190 (gpetretto)
• Require date to reset DB #189 (gpetretto)
• Allow list of jobs from scheduler using username #187 (gpetretto)
• Option to set Job priority #183 (gpetretto)
• New CLI functionalities: tree, report, job info #180 (gpetretto)
• Parallel batch submission #172 (gpetretto)
• Generalize integration tests to other queue systems (SGE) and test more Python versions #160 (ml-evs)
• Project check #158 (davidwaroquiers)
• WIP: Check only one runner #150 (davidwaroquiers)

* This Changelog was automatically generated by github_changelog_generator

v0.1.4

Bug fix release. See PR #177 for details. Prevents insertion of jobs/flows with the same uuid.
Users of previous versions are encouraged to upgrade and run jf admin index rebuild to regenerate the indexes in the queue collection.

Changelog

v0.1.4 (2024-09-13)

Full Changelog

Closed issues:

• Remote errors in interactive mode when larger workflows are run #174
• Add --query (or --worker) flag to jf job rerun/retry #170

Merged pull requests:

• Bugfix: improve collection indexes to prevent duplications #177 (gpetretto)
• Add --worker and --query option in CLI #171 (gpetretto)

* This Changelog was automatically generated by github_changelog_generator

v0.1.3

Linting based on ruff. New functionalities and bug fixes.

Changelog

v0.1.3 (2024-09-02)

Full Changelog

Closed issues:

• Cannot force delete more than 10 flows #169
• Issue with some db in DOWNLOAD state #161
• Jobs fail with error Remote error: file path/to/job_dir/remote_job_data.json for job xxx does not exist #157
• Correct setup of jfremote config project.yaml file for nested gateways #153
• Is there a way to visualize the workflow based on jobflow-remote info? #149
• Flow information is shared even when using different queue stores #144
• a suggestion to improve the documentation #143
• Delete job functionality #141
• Queue-out interactive mode #90
• Writing to home dir is crashing my submission. #81

Merged pull requests:

• Minor updates #156 (gpetretto)
• delete_job functionality #154 (gpetretto)
• Fix ruamel yaml dump #151 (davidwaroquiers)
• Add hyperlink for slurm and pbs resource keywords as a note in the documentation #148 (QuantumChemist)
• CLI add --pid option to jf job info #142 (janosh)
• Fix legacy ruff errors and enable corresponding rules for future linting #134 (janosh)

* This Changelog was automatically generated by github_changelog_generator

v0.1.2

Updates and bug fixes

Changelog

v0.1.2 (2024-06-26)

Full Changelog

Implemented enhancements:

• Allow setting resources for FAILED and COMPLETED jobs #119
• SUGGESTION: Catchall state for errors #111
• Flow information #19

Fixed bugs:

• Flow state stays COMPLETED when a job is rerun #118

Closed issues:

• Feature request: show more information about the flow(s) that will be deleted #130
• Feature request: optionally delete outputs when doing jf flow delete #129
• Feature request: support psutil 6.0.0 #128
• SUGGESTION: Add an example for querying results from the database #115

Merged pull requests:

• Small updates to JobController and CLI #132 (gpetretto)
• Migrate linting from flake8, isort, autoflake to ruff #122 (janosh)
• Minor updates #121 (gpetretto)
• Better error message on missing project name #120 (janosh)

* This Changelog was automatically generated by github_changelog_generator

v0.1.1

v0.1.1 (2024-03-20)

Full Changelog

Closed issues:

• Two projects, job submission #89
• Add guard for missing additional_store before executing job #76
• PyPI release checklist #53

Merged pull requests:

• Tests and updates #92 (gpetretto)
• Add options for remote JobStore #88 (gpetretto)
• Updates #87 (gpetretto)
• Interactive login for MFA #83 (gpetretto)
• Add testing and runtime checks for additional stores #59 (ml-evs)
• Remove errant __init__.py that prevents mypy from working #54 (ml-evs)

* This Changelog was automatically generated by github_changelog_generator

v0.1.0

This is the first public release of Jobflow Remote! Thank you to all those who have been testing the various previous iterations.

Although there are no breaking changes from the first public version (i.e., the publicizing of this GitHub repository), it is RECOMMENDED that existing users update to v0.1.0 as it will pull in the latest compatible versions of the dependencies, in particular QToolKit v0.1.3.

Changelog

v0.1.0 (2024-02-05)

Full Changelog

Closed issues:

• jf job list shows errors randomly #63
• Multiple lines to pre_run #61
• Add tests for Jobs with a function arg or kwarg #45
• Question about supported data types #44
• Error during retry #40
• 'REMOTE_ERROR' state for running job #36
• Targeted Python versions #33
• Jobflow dependency #30
• remote_job_data.json file missing #26
• Suggestion: improve -sdate option #24
• Job info for remote failures #20
• List of flows by state does not work #16
• Runner fails when default resources are provider per worker #14
• Cryptic failures when worker's workdir is missing #13
• Worker can be set to None leaving dead jobs #11
• Jobs with RESERVED state have datetime objects for last_updated field #8
• Command line "jf project" when there is no project #5
• .jfremote yaml test unsensitive to port change #4
• config issue #3

Merged pull requests:

• Add dev setup docs #57 (ml-evs)
• Add a test that runs with a given exec_config #56 (ml-evs)
• Pin dependency versions for testing, add PyPI release and enable dependabot #55 (ml-evs)
• Add integration test for job with a callable as a kwarg #51 (ml-evs)
• Add codecov upload #49 (ml-evs)
• Fix README badge #48 (ml-evs)
• Breaking changes: db_id type, cancelled state, deserialization and documentation #47 (gpetretto)
• Added GitHub release workflow and docs builds #46 (davidwaroquiers)
• [WIP] New job management system #37 (gpetretto)
• Add index to the folder name #35 (gpetretto)
• Linting fixes and CI config for multiple Python 3.9+ #34 (ml-evs)
• Add containerized integration tests for remote and local workers using slurm #32 (ml-evs)
• Use the qtoolkit and jobflow PyPI packages, add direct pydantic dep #31 (ml-evs)
• [WIP] pydantic2 updates #29 (gpetretto)
• Run CI on PRs to develop branch #28 (ml-evs)
• fix typo in cli help message #25 (FabiPi3)
• jf flow info and other cli updates #23 (gpetretto)
• Stopped states. #22 (davidwaroquiers)
• Fixed jf flow list. #18 (davidwaroquiers)
• Restore queries based on job id #17 (gpetretto)
• Check work_dir with project check CLI and enforce absolute paths #15 (ml-evs)
• Fix handling of null worker in submit_flow #12 (ml-evs)
• Added documentation structure. #9 (davidwaroquiers)
• A few minor tweaks from first use #7 (ml-evs)
• Added dependencies in pyproject.toml. #2 (davidwaroquiers)
• WIP Config #1 (davidwaroquiers)

* This Changelog was automatically generated by github_changelog_generator