Skip to content

BUG: aggregate() only tracks generatedTps peak; peak shows wrong metric when mode is "output" #29

Description

@devinoldenburg

Summary

aggregate() in tps.js tracks only one peak value: generatedTps. When the user configures metric: "output", the session peak displayed in the sidebar shows the per-message generated TPS peak, not the output TPS peak. This is misleading — the headline and average both use output TPS, but the peak is from a different metric.

Root Cause

File: plugins/tps/tps.js:201

if (s.generatedTps && s.generatedTps > peakTps) peakTps = s.generatedTps;

aggregate() only maintains peakTps from generatedTps. It never tracks outputTps peak.

File: plugins/tps/view.js:111

{ text: session.peakTps ? `  peak ${fmtRate(session.peakTps)}` : "", tone: "muted" },

The view unconditionally displays session.peakTps, which is always the generated TPS peak regardless of the configured metric.

Detailed Scenario

  1. User sets metric: "output" because they only care about visible output tokens, not reasoning tokens
  2. Session has two messages:
    • Message A: 500 output tokens over 10s = 50 output TPS, 0 reasoning → 50 generated TPS
    • Message B: 200 output tokens + 300 reasoning over 5s = 40 output TPS, 60 generated TPS
  3. Sidebar shows:
    • Headline: 40 tok/s (output TPS of last message) ✓
    • Average: pooled output TPS ✓
    • Peak: 60 ← WRONG. This is Message B's generated TPS. The output TPS peak is 50 (Message A).

Why This Matters

With reasoning-enabled models (Claude, o1, o3, etc.), reasoning tokens can dramatically inflate generatedTps while the user only sees outputTps in the headline. Having the peak come from a different denominator is inconsistent and confuses users who see "peak 200 tok/s" when their output TPS never exceeded 80.

Proposed Fix

aggregate() should track both peaks:

export function aggregate(statList) {
  // ...
  let peakGeneratedTps = 0;
  let peakOutputTps = 0;
  // ...
  for (const s of statList || []) {
    // ...
    if (s.generatedTps && s.generatedTps > peakGeneratedTps) peakGeneratedTps = s.generatedTps;
    if (s.outputTps && s.outputTps > peakOutputTps) peakOutputTps = s.outputTps;
    // ...
  }
  return {
    // ...
    peakGeneratedTps: peakGeneratedTps || null,
    peakOutputTps: peakOutputTps || null,
    // ... keep peakTps for backward compat? or remove it
  };
}

Then in view.js, pick the right peak based on cfg.metric:

const peakKey = cfg.metric === "output" ? "peakOutputTps" : "peakGeneratedTps";
// ...
{ text: session[peakKey] ? `  peak ${fmtRate(session[peakKey])}` : "", tone: "muted" },

Steps to Reproduce

  1. Configure metric: "output"
  2. Have a session with a message where reasoning tokens push generatedTps far above outputTps
  3. Observe the peak value — it matches generatedTps, not outputTps

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions