Skip to content

Conversation

@Akhil-Pathivada
Copy link
Contributor

Summary

This PR implements the feature requested in #638 - exposing the QPS-Latency tradeoff curve for streaming tests. Currently, streaming tests only report a single "Max QPS" value per stage, which provides limited visibility into performance characteristics for capacity planning.

What's New

Backend

  • Use HDR Histogram for memory-efficient latency tracking (~20KB fixed memory vs unbounded list growth)
  • Collect p99, p95, and average latency at each concurrency level per stage
  • Pass latency metrics through the pipeline to results

Frontend

  • New "Concurrent Search Performance" section on streaming results page
  • Single Stage View: Select a stage to see QPS vs Latency chart and detailed metrics table
  • Compare Stages View: Overlay multiple stages to visualize how performance evolves as data grows

Why This Matters

Production workloads typically target a specific latency SLA rather than maximum throughput. With this data, users can now determine:

  • What latency to expect at a specific QPS during ingestion
  • Where the optimal operating point is before latency spikes
  • How performance changes as data volume increases

Files Changed

File Change
mp_runner.py HDR Histogram + latency collection per concurrency
read_write_runner.py Pass latency data through pipeline
metric.py Add 5 new fields for per-stage latency data
concurrent_detail.py New - UI component for QPS-Latency visualization
streaming.py Integrate new UI section
pyproject.toml Add hdrhistogram>=0.10.1
requirements_py3.11.txt Add hdrhistogram>=0.10.1 for Docker

Screenshots

Screenshot 2025-12-08 at 15 25 57 Screenshot 2025-12-08 at 15 27 18

Closes #638

- Use HDR Histogram for memory-efficient latency tracking
- Collect p99, p95, avg latency at each concurrency level per stage
- Add Concurrent Search Performance UI with single-stage and multi-stage views
- Closes zilliztech#638
@Akhil-Pathivada Akhil-Pathivada force-pushed the feat/add-qps-latency-metrics-streaming branch from 6dddb5a to 8a598d5 Compare December 8, 2025 12:34
@Akhil-Pathivada Akhil-Pathivada marked this pull request as ready for review December 8, 2025 19:21
@Akhil-Pathivada
Copy link
Contributor Author

/assign @alwayslove2013

@alwayslove2013 alwayslove2013 self-requested a review December 10, 2025 08:44
@alwayslove2013 alwayslove2013 self-assigned this Dec 10, 2025
@sre-ci-robot
Copy link

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: Akhil-Pathivada, alwayslove2013
To complete the pull request process, please assign xuanyang-cn after the PR has been reviewed.
You can assign the PR to them by writing /assign @xuanyang-cn in a comment when ready.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@alwayslove2013 alwayslove2013 merged commit 3e6c993 into zilliztech:main Dec 19, 2025
4 checks passed
@Akhil-Pathivada Akhil-Pathivada deleted the feat/add-qps-latency-metrics-streaming branch December 19, 2025 09:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature Request] Add QPS-Latency tradeoff metrics for Streaming Tests

3 participants