The BCO format supports SHA1 checksums for URI objects. These checksums are particularly useful for intermediate output files since they are usually deleted once the run completes.
There is a Nextflow PR nextflow-io/nextflow#3802 to save the MD5 checksum of each output file to the task cache to facilitate better provenance reporting. The MD5 checksum is advantageous because the big three cloud providers automatically compute the MD5 checksum of an object when it is uploaded, so Nextflow can re-use these checksums and save some time.
Unfortunate that BCO only allows for SHA1. We would have to compute SHA1 in addition to the MD5 checksums already being computed. Need to think on this more.
The BCO format supports SHA1 checksums for URI objects. These checksums are particularly useful for intermediate output files since they are usually deleted once the run completes.
There is a Nextflow PR nextflow-io/nextflow#3802 to save the MD5 checksum of each output file to the task cache to facilitate better provenance reporting. The MD5 checksum is advantageous because the big three cloud providers automatically compute the MD5 checksum of an object when it is uploaded, so Nextflow can re-use these checksums and save some time.
Unfortunate that BCO only allows for SHA1. We would have to compute SHA1 in addition to the MD5 checksums already being computed. Need to think on this more.