fix: divan walltime metric

[not-matthias]

Before:

After:

(benchmark has been renamed to sleep_100ms_with_custom_sample)

EDIT: Verified that it works on ruff:

• before: https://staging.codspeed.io/AvalancheHQ/ruff-variance-testing/runs/685d1edb7d35fb1ee0b0d8dd

Collected 1 benchmark suite(s) to run
Running ruff_benchmark ty_walltime
ty_walltime      fastest       │ slowest       │ median        │ mean          │ samples │ iters
╰─ small                       │               │               │               │         │
   ├─ altair     1.024 s       │ 1.147 s       │ 1.044 s       │ 1.072 s       │ 3       │ 6
   ├─ freqtrade  1.231 s       │ 1.842 s       │ 1.734 s       │ 1.602 s       │ 3       │ 6
   ╰─ pydantic   575.1 ms      │ 921.1 ms      │ 579.1 ms      │ 691.7 ms      │ 3       │ 6

• after: https://staging.codspeed.io/AvalancheHQ/ruff-variance-testing/runs/685d1f777d35fb1ee0b0d8de

ty_walltime      fastest       │ slowest       │ median        │ mean          │ samples │ iters
╰─ small                       │               │               │               │         │
   ├─ altair     699.6 ms      │ 892.7 ms      │ 708.6 ms      │ 766.9 ms      │ 3       │ 6
   ├─ freqtrade  1.2 s         │ 1.937 s       │ 1.282 s       │ 1.473 s       │ 3       │ 6
   ╰─ pydantic   1.067 s       │ 1.071 s       │ 1.068 s       │ 1.068 s       │ 3       │ 6

[codspeed-hq]

CodSpeed Instrumentation Performance Report

Merging #99 will improve performances by 3.26%

_{Comparing cod-1044-fix-divan-walltime-metric (f703a54) with main (b6a7be8)}

Summary

⚡ 1 improvements
✅ 162 untouched benchmarks
🆕 4 new benchmarks

Benchmarks breakdown

	Benchmark	BASE	HEAD	Change
🆕	sleep_100ms	N/A	566.1 ns	N/A
🆕	sleep_10ms	N/A	566.1 ns	N/A
🆕	sleep_1ms	N/A	566.1 ns	N/A
🆕	sleep_50ms	N/A	595.3 ns	N/A
⚡	mut_borrow	925 ns	895.8 ns	+3.26%

[codspeed-hq]

CodSpeed WallTime Performance Report

Merging #99 will degrade performances by 32.65%

_{Comparing cod-1044-fix-divan-walltime-metric (f703a54) with main (b6a7be8)}

Summary

⚡ 79 improvements
❌ 13 regressions
✅ 53 untouched benchmarks
🆕 9 new benchmarks

⚠️ Please fix the performance issues or acknowledge them on CodSpeed.

Benchmarks breakdown

	Benchmark	BASE	HEAD	Change
❌	build_vec	195 ns	209 ns	-6.7%
⚡	Iterative	21 ns	20 ns	+5%
⚡	Recursive	50.4 µs	45.6 µs	+10.44%
⚡	Iterative[20]	21 ns	20 ns	+5%
⚡	Iterative[21]	22 ns	21 ns	+4.76%
⚡	Recursive[20]	51 µs	45.6 µs	+11.75%
⚡	Recursive[21]	82.1 µs	73.8 µs	+11.34%
⚡	fibonacci_custom_measurement	407 ns	369 ns	+10.3%
❌	large_drop	167 µs	177.5 µs	-5.88%
❌	large_setup	11 ns	13 ns	-15.38%
❌	iter_with_setup	37 ns	40 ns	-7.5%
🆕	sleep_100ms	N/A	100.1 ms	N/A
🆕	sleep_10ms	N/A	10.1 ms	N/A
🆕	sleep_1ms	N/A	1.1 ms	N/A
🆕	sleep_50ms	N/A	50.1 ms	N/A
❌	from_elem[16384]	784 ns	819 ns	-4.27%
❌	from_elem[2048]	245 ns	253 ns	-3.16%
⚡	from_elem[4096]	327 ns	317 ns	+3.15%
⚡	from_elem_decimal[2048]	251 ns	242 ns	+3.72%
⚡	bench_array1[10]	5,157 ns	10 ns	×520
...	...	...	...	...

ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.

[adriencaccia]

EDIT: Verified that it works on ruff:

• before: staging.codspeed.io/AvalancheHQ/ruff-variance-testing/runs/685d1edb7d35fb1ee0b0d8dd

Collected 1 benchmark suite(s) to run
Running ruff_benchmark ty_walltime
ty_walltime      fastest       │ slowest       │ median        │ mean          │ samples │ iters
╰─ small                       │               │               │               │         │
   ├─ altair     1.024 s       │ 1.147 s       │ 1.044 s       │ 1.072 s       │ 3       │ 6
   ├─ freqtrade  1.231 s       │ 1.842 s       │ 1.734 s       │ 1.602 s       │ 3       │ 6
   ╰─ pydantic   575.1 ms      │ 921.1 ms      │ 579.1 ms      │ 691.7 ms      │ 3       │ 6

• after: staging.codspeed.io/AvalancheHQ/ruff-variance-testing/runs/685d1f777d35fb1ee0b0d8de

ty_walltime      fastest       │ slowest       │ median        │ mean          │ samples │ iters
╰─ small                       │               │               │               │         │
   ├─ altair     699.6 ms      │ 892.7 ms      │ 708.6 ms      │ 766.9 ms      │ 3       │ 6
   ├─ freqtrade  1.2 s         │ 1.937 s       │ 1.282 s       │ 1.473 s       │ 3       │ 6
   ╰─ pydantic   1.067 s       │ 1.071 s       │ 1.068 s       │ 1.068 s       │ 3       │ 6

There is an issue with the data in this comment, it does not correspond to what we have on the codspeed links you provided

[not-matthias]

EDIT: Verified that it works on ruff:

• before: staging.codspeed.io/AvalancheHQ/ruff-variance-testing/runs/685d1edb7d35fb1ee0b0d8dd

Collected 1 benchmark suite(s) to run
Running ruff_benchmark ty_walltime
ty_walltime      fastest       │ slowest       │ median        │ mean          │ samples │ iters
╰─ small                       │               │               │               │         │
   ├─ altair     1.024 s       │ 1.147 s       │ 1.044 s       │ 1.072 s       │ 3       │ 6
   ├─ freqtrade  1.231 s       │ 1.842 s       │ 1.734 s       │ 1.602 s       │ 3       │ 6
   ╰─ pydantic   575.1 ms      │ 921.1 ms      │ 579.1 ms      │ 691.7 ms      │ 3       │ 6

• after: staging.codspeed.io/AvalancheHQ/ruff-variance-testing/runs/685d1f777d35fb1ee0b0d8de

ty_walltime      fastest       │ slowest       │ median        │ mean          │ samples │ iters
╰─ small                       │               │               │               │         │
   ├─ altair     699.6 ms      │ 892.7 ms      │ 708.6 ms      │ 766.9 ms      │ 3       │ 6
   ├─ freqtrade  1.2 s         │ 1.937 s       │ 1.282 s       │ 1.473 s       │ 3       │ 6
   ╰─ pydantic   1.067 s       │ 1.071 s       │ 1.068 s       │ 1.068 s       │ 3       │ 6

There is an issue with the data in this comment, it does not correspond to what we have on the codspeed links you provided

The second one seems to be correct (since it uses the actual fixed times), but maybe I copied the wrong output for the first run. I'll do it again.

EDIT: new results for the 1st run - as you can see, the results differ which is due to the bug that we now fixed.

ty_walltime      fastest       │ slowest       │ median        │ mean          │ samples │ iters
╰─ small                       │               │               │               │         │
   ├─ altair     1.029 s       │ 1.127 s       │ 1.051 s       │ 1.069 s       │ 3       │ 6
   ├─ freqtrade  886.7 ms      │ 1.764 s       │ 1.674 s       │ 1.441 s       │ 3       │ 6
   ╰─ pydantic   540.4 ms      │ 1.019 s       │ 559.3 ms      │ 706.5 ms      │ 3       │ 6

https://staging.codspeed.io/AvalancheHQ/ruff-variance-testing/runs/685d6086d61ba36f94bb4d27

[art049]

looks good! The last thing needed as Adrien mentionned is having the same tests in criterion will ensure consitency

[not-matthias]

looks good! The last thing needed as Adrien mentionned is having the same tests in criterion will ensure consitency

They are already there. Looks like I forgot to resolve the comment. Will merge this now!