Skip to content

Improve error reporting in final JSON results#29

Draft
cjonas9 wants to merge 10 commits intostepped-pacerfrom
improve-error-reporting
Draft

Improve error reporting in final JSON results#29
cjonas9 wants to merge 10 commits intostepped-pacerfrom
improve-error-reporting

Conversation

@cjonas9
Copy link
Copy Markdown
Contributor

@cjonas9 cjonas9 commented Apr 3, 2026

What

Following the stepped-pacer redesign (#23):

  • Adds a per-step-interval snapshot timeline to the final JSON results. Each snapshot captures target RPS, success/error counts, error rate (%), and latency percentiles (p50, p95, p99, p99.9) for that window only
  • Removes the redundant log file mirror (as it's now superseded by the richer JSON)
Visually, the results JSON now looks like:
{
  /*
  Old: cumulative summary data of the entire run
  */
  "start": ...,
  "end": ...,
  "duration_seconds": 123,
  "endpoints": {
    "getLedgers": {
       // summary for the entire run of performance, error breakdowns by type, etc.
      },

    /*
    NEW: data of how the RPC performed at each RPS phase for this endpoint
    */
      "timeline": [
        {
          "target_rps": 7.14, "success": 142, "errors": 0, "error_rate_pct": 0,
          "p50_ms": 165.631, "p95_ms": 537.087, "p99_ms": 900.095, "p99.9_ms": 935.423
        },
        {
          "target_rps": 14.29, "success": 277, "errors": 0, "error_rate_pct": 0,
          "p50_ms": 262.399, "p95_ms": 734.719, "p99_ms": 974.335, "p99.9_ms": 1098.751
        },
        ...,
        {
          "target_rps": 50, "success": 0, "errors": 116, "error_rate_pct": 100,
          "p50_ms": 15007.743, "p95_ms": 15007.743, "p99_ms": 15007.743, "p99.9_ms": 15007.743
        }
      ]
    }
  }
}

Why

The richer JSON gives operators a clear view of how the RPC degraded as load increased. Previously, the results JSON flattened all metrics into a single summary bucket per endpoint without any notion of an RPS-timeline, so there was no way to tell at what RPS the RPC buckled. This gives an operator running a load test a concise/minimal degradation curve (e.g. when errors started, how latency + error rates changed at each step) without requiring them to dig through through logs.

Known limitations

The intervals default to 5s when no ramp-up is provided as there's no natural "RPS phase" notion without a ramp-up stepping through various RPS levels.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant