Optimize JSONL scanner chunk parsing to reduce CPU usage#392
Open
asonawalla wants to merge 1 commit intosteipete:mainfrom
Open
Optimize JSONL scanner chunk parsing to reduce CPU usage#392asonawalla wants to merge 1 commit intosteipete:mainfrom
asonawalla wants to merge 1 commit intosteipete:mainfrom
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR reduces CPU usage in the local cost-usage scanner by optimizing JSONL line parsing in
CostUsageJsonl.The previous scanner loop repeatedly removed bytes from the front of a growing buffer while splitting lines. On large Claude logs, that led to a lot of extra per-chunk work.
What changed
CostUsageJsonl.scanto process segment slices directly.Why
On large local Claude histories, stack samples showed scanner hotspots in the cost scan path (
CostUsageFetcher.loadTokenSnapshot->CostUsageScanner.scanClaudeRoot->CostUsageJsonl.scan).This change targets that hotspot by making newline splitting and carryover handling linear and lower-overhead for large files.
Validation
pnpm checkswift test(passed: 746 tests)Anecdotal runtime observation
After running this build locally for several hours, @asonawalla observed that CodexBar no longer appears among top CPU processes (previously it was frequently visible). This is anecdotal but consistent with the expected scanner improvement.
AI disclosure
This code and PR were generated with Codex and reviewed by @asonawalla