Skip to content

Add multiple LINDI benchmarks to method rankings #185

@stephprince

Description

@stephprince

We had previously added multiple LINDI benchmarks to both make fair comparisons with some of the other streaming methods and measure performance in what would be a common use case.

  • LindiLocalJSONFileReadBenchmark.time_read_lindi_pynwb: measures the time to open a LINDI file with pynwb. Demos what would be the common use case but is slightly different than other read benchmarks in that the LINDI file has been downloaded locally.
  • LindiDandiAPIDownloadBenchmark.time_download_lindi_dandi_api: measures the time to download a LINDI file. Adding this time to the read benchmark above may provide a more fair comparison with some of the other benchmarks tests.

Currently the methods ranking heat map only includes the time_read_lindi_pynwb benchmark, but @oruebel and I had discussed adding both read and download + read to the final figure for the paper. The download + read measurements could be calculated from the current results by combining the two benchmarks above, but I think an even simpler approach to avoid additional LINDI-specific data transformations would be to add another benchmark test that times the combined operations.

Metadata

Metadata

Assignees

No one assigned

    Labels

    papertasks required for benchmarks paper submission

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions