1.12

The main feature of this release is the new high-level and low-level jobsub APIs (, that can be imported and used in experiment workflow code. For more information, please see the documentation here.

We have also created a jobsub command, that can be called with subcommands. The old `jobsub_*` command will still work exactly as before, but now the following are equivalent:

jobsub_submit -G experiment file:///bin/sleep 300

and

jobsub submit -G experiment file:///bin/sleep 300

Running the command jobsub with no arguments will print the help for all the jobsub subcommands.

For technical implementation details, please see PRs 594 and 609.

Features

• New jobsub higher-level API (#609)
• Jobsub subcomand parser and API interface (#594)
• Add a "did you mean" clause for invalid but close to right auth methods (e.g. "tokens" vs "token") (#601)
• Use Spack to find ifdh, and adding logging to sambegin.sh, samend.sh (#607)

Bugfixes

• Fix IFDH_VERSION behavior in job (#620)
• Fix pnfs dropbox case where -f --use-pnfs-dropbox would skip obtaining credentials (#634)

Other code improvements

• Make subparsers logic clearer (#614)
• Upgraded CI runner version to ubuntu-24.04 (#629)
• Updated PAT to fine-grained token for better security (#630)
• Make most unit tests pass without proxies (#638)

1.11.2

• Change in config so that DUNE logs go to different location that rest of logs

1.11.1

• Fix IFDH_VERSION behavior in job wrapper (#624)

1.11

Features

• This is the release that will change the default behavior of jobsub to no longer try to obtain X509 proxies for job submission and management
  • This default can be worked around by either (a) exporting JOBSUB_AUTH_METHODS="token,proxy", or (b) passing --auth-methods="token,proxy" to the jobsub_submit command. For example:

(a)

$ export JOBSUB_AUTH_METHODS="token,proxy"
$ jobsub_submit -G myexperiment file:///bin/true

(b)

$ jobsub_submit -G myexperiment --auth-methods="token,proxy" file:///bin/true

Bugfixes

• maxConcurrent should be an int, default of 0 (#618, #619)

Other code improvements

• None

1.10

Features

• Add --managed-token flag and environment variable (#597, #598)

Bugfixes

• None

Other code improvements

• None

1.9

Features

• Add --gpu flag to jobsub_submit that allows users to request GPUs (#588)

Bugfixes

• Add support for non-Analysis users in jobsub_cleanup_cred (#576)
• Fix CVMFS logs in failed job logs (#581)
• Fix RUNTIME for running jobs (#583)
• Fix jobsub fetchlog verbose (#589)

Other code improvements

• Upgraded test container image from alma 9.3-minimal to 9.4-minimal (#578)
• Update old defunct BLUEARC to new ceph mount in templates and tests (#580, #582)
• Drop support for SL7 (#586)
• Test bug (#587)

1.8.1

• If cigetcert fails for any reason, raise PermissionError (#572). This fixes an issue with the error message emitted by jobsub_submit when it tried to obtain a VOMS proxy. The message would sometimes mask the real underlying issue; for example if jobsub could not obtain the initial X509 proxy that is used to generate a VOMS proxy.

1.8

Features

• Update condor_vault_storer to reflect HTCondor 23 release changes (#563)
• Cleanup copies of weakened tokens after job submission (#565)

Bugfixes

• Make gfal operations for web sandbox creation check for BEARER_TOKEN (#562)
• When --global-pool flag is passed, make sure we save the BEARER_TOKEN_FILE and X509_USER_PROXY into the restored environment post-submission (#568)

Other code improvements

• Delete test file if it's already there (for example if a previous test run didn't finish unlinking the file (#566)
• Clear up escape warning in tests (#567)

1.7.2

• Increase dagbegin.cmd and dagend.cmd memory requests to 500 Mb (#558)

1.7.1

• Fixes a bug introduced in jobsub_lite 1.7 wrt POMS submissions (#557)