Releases: redis/redis-benchmarks-specification
v0.3.13 — fix preload-time blowup in multitenant-tagged specs
Cuts preload from ~28 min → ~15 sec by switching 5 SET preloads from -c 50 -t 4 -n allkeys (200 clients × 10M keys = 2B ops) to -c 1 -t 1 (exactly 10M ops, 1 SET per key). Test phases unchanged. Merged in #388.
v0.3.12 — multi-tenant tagged-keyname specs at 10M scale
Adds 7 benchmark specs that exercise a long-keyname multi-tenant access pattern not previously covered. Each key uses a unique hash-tag prefix plus a ~78-char structured suffix:
`{T:N}_metric_a_window_b_period_c_segment_d_aggregate_e_v1_v2_v3_v4_v5_v6`
This stresses ~6× more bytes hashed/compared per dict lookup than the default short `memtier-N` prefix used by sister specs.
Specs added
- `memtier_benchmark-multiple-hll-pfadd-10Mkeys-10B-values` — PFADD on 10M sparse HLLs
- `memtier_benchmark-10Mkeys-string-mget-80keys-100B-multitenant-tagged` — 80-key MGET batches
- `memtier_benchmark-10Mkeys-generic-expire-multitenant-tagged` — EXPIRE 7200 on long-keyname keys
- `memtier_benchmark-10Mkeys-string-incrby-multitenant-tagged` — INCRBY 1 on int-encoded keys
- `memtier_benchmark-10Mkeys-string-incrby-1000-multitenant-tagged` — INCRBY larger value variant
- `memtier_benchmark-10Mkeys-mixed-incrby-expire-multitenant-tagged` — mixed write workload (INCRBY:EXPIRE = 9:20 via `--command-ratio`)
- `memtier_benchmark-multiple-hll-pfcount-1key-10Mkeys-multitenant-tagged` — single-key PFCOUNT
All target `oss-standalone`. Used-memory at full scale: PFADD ~1.7 GB, MGET ~2.3 GB, EXPIRE ~2.3 GB, INCRBY ~1.25 GB, mixed ~1.25 GB, PFCOUNT ~1.7 GB. All request 8 GB resource headroom.
Merged in #387.
v0.3.11
What's new
session-caching specs: replace Lua preload with memtier preload_tool + add pipeline=16 variant (#386)
memtier_benchmark-playbook-session-caching-hash-100k-sessions.yml: drop theinit_luablock and usedbconfig.preload_toolwithmemtier_benchmark(--random-data+--data-size-range=200-300). Aligns with the standard memtier-preload idiom used elsewhere in the suite. Addscheck.keyspacelen: 100000sanity gate.- New
memtier_benchmark-playbook-session-caching-hash-100k-sessions-pipeline-16.yml: same workload with client--pipeline=16to amplify per-call overhead and exercise reply-buffer / output-list paths.
Full Changelog: v0.3.10...v0.3.11
v0.3.10
Add 3 string SET key value GET (no expire) memtier specs across EMBSTR + RAW value sizes:
memtier_benchmark-10Mkeys-string-set-with-get-option-10B(EMBSTR, ≤ 44 B)memtier_benchmark-1Mkeys-string-set-with-get-option-100B(RAW)memtier_benchmark-1Mkeys-string-set-with-get-option-1KB(RAW)
All three exercise the propagation argv-strip path in setGenericCommand() (src/t_string.c), which existing string benchmarks (separate SET / GET) did not cover.
Merged via #385.
v0.3.8
What's new
Adds 4 stream GC coverage specs (PR #382) to validate streamEntryIsReferenced() and streamCleanupEntryCGroupRefs() at larger consumer-group counts and across delete-strategy variants:
| Spec | CG count | Command mix |
|---|---|---|
stream-500-cgroups-xadd-xtrim-aggressive-gc |
500 | 70 % XADD + 30 % XTRIM MAXLEN ~ 1000 |
stream-5000-cgroups-xadd-xtrim-aggressive-gc |
5,000 | same |
stream-50-cgroups-xadd-xreadgroup-xdel |
50 | XADD (30%) + XDEL deterministic IDs (50%) + XREADGROUP (20%) |
stream-500-cgroups-xadd-xreadgroup-xdel |
500 | same |
Complements the existing stream-50-cgroups-xadd-xtrim-* specs by scaling CG count (10× / 100×) and adding the XDEL path (middle-of-stream delete triggering streamCleanupEntryCGroupRefs(), distinct from XTRIM's tail-trim).
Full Changelog: v0.3.7...v0.3.8
v0.3.7
Hotfix
- Add missing clientconfig.resources to EMBSTR validation specs (#381) — the 3 specs added in 0.3.5/0.3.6 were missing
resources:insideclientconfig, causingKeyError: 'resources'in the coordinator and aborting every run before benchmark data was collected.
Full Changelog: v0.3.6...v0.3.7
v0.3.6
Changes
- Add 10M-keys EMBSTR validation spec (#380) — DRAM-bound sibling of the 1M × 10B spec for robj-layout validation. At 10M keys, used_memory scales to ~660MB which no longer fits in L3 on any fleet CPU — measures the cache-miss cost of pointer chasing that the L3-resident 1M variant hides.
Full Changelog: v0.3.5...v0.3.6
v0.3.5
Changes
- Add two EMBSTR-layout validation specs (#379) — APPEND-path test (10B EMBSTR → 40B RAW conversion) and mixed EMBSTR/RAW GET pipeline-100 test. Both use
--random-data. Used to validate Valkey-#2516-style robj layout changes.
Full Changelog: v0.3.4...v0.3.5
v0.3.4
Hotfix
- fix(coordinator): correct compute_regression_table args + wrap in try/except (#378) — companion to #377/v0.3.3. Fixes a positional-arg mismatch (call was missing
tf_triggering_env_comparison, shifting every subsequent arg by one and routingto_dateinto thefrom_ts_msslot → TypeError in compare.py:1300 → per-test for-loop aborted → tests stuck intests_pending). Also wrapscompute_regression_tablein try/except so any future data-shape issue can't kill the stream.
Full Changelog: v0.3.3...v0.3.4
v0.3.3
Hotfix
- fix(coordinator): wrap per-test PR comment update in try/except (#377) — regression-ish: unwrapped
update_comment_if_needed()in the per-test GitHub update path would propagate any transient GH API exception through the for-loop and abort the entire stream. Observed on PRs #14934 and #15018 on 2026-04-22 — streams got stuck at 1/N forever. Fixed by wrapping theis_actionable_prblock the same way the siblingprepare_regression_commentpath was already wrapped.
Full Changelog: v0.3.2...v0.3.3