Resource requirement min max validation

[Stellatsuu]

Related to : #2163

• Added resource requirement minmax validation
• Added validation when running --validate
• Added related tests

[mr-c]

Thank you @Stellatsuu !

[codecov]

Codecov Report

❌ Patch coverage is 77.77778% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 55.48%. Comparing base (135a0c6) to head (c8d7d68).

Files with missing lines	Patch %	Lines
cwltool/checker.py	66.66%	1 Missing and 1 partial ⚠️
cwltool/process.py	50.00%	1 Missing and 1 partial ⚠️

❗ There is a different number of reports uploaded between BASE (135a0c6) and HEAD (c8d7d68). Click for more details.

HEAD has 6 uploads less than BASE

Flag	BASE (135a0c6)	HEAD (c8d7d68)
	17	11

Additional details and impacted files

@@             Coverage Diff             @@
##             main    #2179       +/-   ##
===========================================
- Coverage   85.12%   55.48%   -29.65%     
===========================================
  Files          46       46               
  Lines        8368     8383       +15     
  Branches     1956     1961        +5     
===========================================
- Hits         7123     4651     -2472     
- Misses        779     3230     +2451     
- Partials      466      502       +36     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[mr-c]

the CWL descriptions need inputs

[Stellatsuu]

It was a bad copypaste of mine, I only took a workflow from the test folder and modified it, forgot about the inputs.
Since we want to focus on requirements validation, I changed both workflows to be only helloworld without inputs or outputs but with invalid requirements.

Would it work like this? Or maybe I did not understand your review.

[mr-c]

I was explaining why the tests failed :-)

tests/wf/bad_resreq_mnmx_wf.cwl still requires inputs; you need to either provide them as a separate file, or also modify that CWL description to not need inputs provided (perhaps via default values).

[Stellatsuu]

Default inputs are required even in a workflow like this one where inputs and outputs are empty and not used?

#!/usr/bin/env cwl-runner
class: Workflow
cwlVersion: v1.2

requirements:
  ResourceRequirement:
      ramMin: 128
      ramMax: 64

inputs: []
outputs: []

steps:
  hello_world:
    requirements:
      ResourceRequirement:
        ramMin: 64
        ramMax: 128
    run:
        class: CommandLineTool
        baseCommand: [ "echo", "Hello World" ]
        inputs: [ ]
        outputs: [ ]
    out: [ ]
    in: [ ]

[Stellatsuu]

Would this be a correct workflow?

#!/usr/bin/env cwl-runner
class: Workflow
cwlVersion: v1.2

requirements:
  ResourceRequirement:
    ramMin: 128
    ramMax: 64

inputs:
  message: string?

outputs: []

steps:
  hello_world:
    requirements:
      ResourceRequirement:
        ramMin: 64
        ramMax: 128
    run:
      class: CommandLineTool
      baseCommand: echo
      inputs:
        message:
          type: string?
          default: "Hello World"
      outputs: []
      arguments:
        - $(inputs.message)
    in:
      message: message
    out: []

[Stellatsuu]

cwltool --disable-validate tests/wf/bad_resreq_mnmx_wf.cwl command works while it shouldn't, since the workflow resource requirement is wrong:

#!/usr/bin/env cwl-runner
class: Workflow
cwlVersion: v1.2

requirements: <-----
  ResourceRequirement:
      ramMin: 128
      ramMax: 64

inputs: []
outputs: []

steps:
  hello_world:
    requirements:
      ResourceRequirement:
        ramMin: 64
        ramMax: 128
    run:
        class: CommandLineTool
        baseCommand: [ "echo", "Hello World" ]
        inputs: [ ]
        outputs: [ ]
    out: [ ]
    in: [ ]

I looked at the code, and it seems that the main reason is that, evalResources is only called when the cwl object is not a Workflow:

def Process(...):
   def _init_job(...):
        ...
        if self.tool["class"] != "Workflow":
            builder.resources = self.evalResources(builder, runtime_context)
        return builder
   def evalResources(...):
        -validation is here-

How should I manage to check if the resource requirements of a Workflow are correct when calling --disable-validate?