Migrate a few more entries from EU #103
Conversation
| toolshed.g2.bx.psu.edu/repos/ecology/srs_preprocess_s2/srs_preprocess_s2/.*: | ||
| mem: 16 | ||
| toolshed.g2.bx.psu.edu/repos/ecology/wildlife_megadetector_huggingface/wildlife_megadetector_huggingface/.*: | ||
| gpus: 1 |
There was a problem hiding this comment.
Is it ok to specify here GPUs?
There was a problem hiding this comment.
If it doesn't work without I would say yes
| mem: 12 | ||
| mem: 24 | ||
| env: | ||
| EGGNOG_DBMEM: --dbmem |
There was a problem hiding this comment.
I have a related question here. @cat-bro do you know if on AU all those envs are passed into the containers without any extra magic?
| cores: 4 | ||
| mem: 30 | ||
| scheduling: | ||
| require: |
There was a problem hiding this comment.
I can remove that. But what is the preferred way to indicate that this tool can/should run in Singularity?
| - singularity | ||
| rules: | ||
| - if: input_size >= 0.01 | ||
| gpus: 1 |
There was a problem hiding this comment.
flag GPU here, if we don't want this in a shared DB.
| - if: input_size >= 0.01 | ||
| gpus: 1 | ||
| params: | ||
| singularity_run_extra_arguments: ' --nv ' |
There was a problem hiding this comment.
this is enabling Singularity containers to setup the containers for GPU/Nvidia ...
There was a problem hiding this comment.
My initial thought would be: keep this but remove the scheduling tag for singularity and add a comment? But in general running things in singularity is the recommended approach, or not?
| toolshed.g2.bx.psu.edu/repos/iuc/nanopolishcomp_eventaligncollapse/nanopolishcomp_eventaligncollapse/.*: | ||
| cores: 10 | ||
| mem: 12 | ||
| toolshed.g2.bx.psu.edu/repos/iuc/ncbi_fcs_gx/ncbi_fcs_gx/.*: |
There was a problem hiding this comment.
This is a very expensive and inefficient tool for VGP - TODO look at ORGs rule.
| toolshed.g2.bx.psu.edu/repos/iuc/pureclip/pureclip/.*: | ||
| cores: 2 | ||
| mem: 32 | ||
| # 4GB is enough for most of the runs as it seems |
There was a problem hiding this comment.
I had this in my notes, no idea why we have here so much more memory :(
There was a problem hiding this comment.
Yes, but one must take care of the edge cases otherwise people cannot run their jobs (if OOM killing is in place)
There was a problem hiding this comment.
I added a comment and put in the 30GB back in.
| cores: 12 | ||
| mem: 92 | ||
| - id: | ||
| if: input_size >= 1 |
There was a problem hiding this comment.
We are bailing out when we see large files and recommend Spades.
There was a problem hiding this comment.
This one is interesting. I don’t think this rule belongs in the shared db because it would prevent reproducible research in some cases. Apart from this there is no guarantee that all galaxies would have rnaspades installed.
There was a problem hiding this comment.
This one is interesting. I don’t think this rule belongs in the shared db because it would prevent reproducible research in some cases.
Mh, given the varying resource limits across our instances, I don't see a problem for reproducible research. Let's use the shared-db to coordinate the deprecation of resource-heavy legacy tools and align on more sustainable software choices.
There was a problem hiding this comment.
But ofc I'm happy to take this rule out. Imho ORG has something else as well.
bgruening
left a comment
bgruening
left a comment
There was a problem hiding this comment.
Thanks, I adopted this
| toolshed.g2.bx.psu.edu/repos/iuc/purge_dups/purge_dups/.*: | ||
| mem: 30.4 | ||
| cores: 1 | ||
| mem: 6 |
There was a problem hiding this comment.
I am frequently seeing max memory for jobs over 10GB, sometimes as high as 30GB.
There was a problem hiding this comment.
So we just deduce a rule here?
| mem: 6 | |
| # might be a good target for a memory rule | |
| mem: 30.4 |
No description provided.