fix: address code review issues — parser-stage correctness, memory cap, O(N²) chunked fetch, and docs#355
Merged
Merged
Conversation
Fix 1 (correctness): Restore __error__ guard to parser-stage metric queries. VL native stats_query_range counts parse-failed lines while Loki excludes them. Queries with parser stages (| json, | logfmt, etc.) now route to the slow log-fetch path unless the caller explicitly opts in via " | drop __error__". The opt-in check uses origSpec.BaseQuery (inner LogQL pipeline without outer aggregation or range window brackets) so hasDropErrorOnlyPostParserStage correctly identifies the drop-error clause. The | math guard for grouped rate queries and a new pre-check for non-| math queries (count_over_time etc.) both apply this fast-path bypass before shouldUseManualRangeMetricCompat is called. Fix 2 (performance): Eliminate O(N²) copy amplification in coldBackwardChunkedFetch. Replaced prepend-into-growing-buffer (accumulated = append(chunkBody, accumulated...)) with a slice of chunks that is reversed once at the end and joined. Fix 3 (memory): Cap unbounded io.ReadAll in collectRangeMetricHits at 64 MB via the existing readBodyLimited helper to bound RSS on oversized VL error bodies. Fix 4 (documentation): Document optional tenant header behaviour in KNOWN_ISSUES.md. Fix 5 (documentation + code): Document multi-tenant serial fanout in KNOWN_ISSUES.md and add a TODO comment in multitenant.go referencing the tracking entry.
Contributor
PR Quality ReportCompared against base branch Coverage and tests
Compatibility
Performance smokeLower CPU cost (
State
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
__error__guard for parser-stage metric queries. VLstats_query_rangecounts parse-failed lines while Loki excludes them. Allrate/bytes_rate/count_over_time/bytes_over_timequeries with parser stages now route to the slow log-fetch path unless the caller explicitly adds| drop __error__to opt in to VL's count-all semantics. The opt-in check uses the inner LogQL base query (without outer aggregation or range window brackets) sohasDropErrorOnlyPostParserStagecorrectly identifies the clause.coldBackwardChunkedFetch. Replaced theappend(chunkBody, accumulated...)prepend loop with a slice of chunks reversed and joined once at the end.io.ReadAllincollectRangeMetricHitsat 64 MB via the existingreadBodyLimitedhelper to bound RSS on oversized VL error response bodies.KNOWN_ISSUES.md. Add aTODOcomment inmultitenant.goreferencing the tracking entry.Test Plan
go test ./internal/proxy/...passes (1638 tests, 0 failures)TestQueryRange_RateParserStageTumblingUsesSlowPathverifies parser-stage rate without| drop __error__uses slow pathTestQueryRange_RateParserStageDropErrTumblingUsesStatsQueryRangeverifies| drop __error__opt-in restores fast pathTestQueryRange_CountOverTimeParserUsesDirectStatsRangeverifies groupedcount_over_timewith| drop __error__, __error_details__usesstats_query_rangeTestCompatHelpers_ParseQuantileAndUnwrapErrorNameverifies updated unit assertions for parser-stage slow path