-
Notifications
You must be signed in to change notification settings - Fork 1
Open
Labels
papertasks required for benchmarks paper submissiontasks required for benchmarks paper submission
Description
We had previously added multiple LINDI benchmarks to both make fair comparisons with some of the other streaming methods and measure performance in what would be a common use case.
LindiLocalJSONFileReadBenchmark.time_read_lindi_pynwb: measures the time to open a LINDI file with pynwb. Demos what would be the common use case but is slightly different than other read benchmarks in that the LINDI file has been downloaded locally.LindiDandiAPIDownloadBenchmark.time_download_lindi_dandi_api: measures the time to download a LINDI file. Adding this time to the read benchmark above may provide a more fair comparison with some of the other benchmarks tests.
Currently the methods ranking heat map only includes the time_read_lindi_pynwb benchmark, but @oruebel and I had discussed adding both read and download + read to the final figure for the paper. The download + read measurements could be calculated from the current results by combining the two benchmarks above, but I think an even simpler approach to avoid additional LINDI-specific data transformations would be to add another benchmark test that times the combined operations.
Metadata
Metadata
Assignees
Labels
papertasks required for benchmarks paper submissiontasks required for benchmarks paper submission