LUCENE-14611: Reduce memory pressure in TestTermInSetQuery.testDuel#16300
Open
lsh1215 wants to merge 5 commits into
Open
LUCENE-14611: Reduce memory pressure in TestTermInSetQuery.testDuel#16300lsh1215 wants to merge 5 commits into
lsh1215 wants to merge 5 commits into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This reduces avoidable memory pressure in
TestTermInSetQuery.testDuel.The test is checking query equivalence, not query-cache behavior. It still uses the randomized test-framework
newSearcher(reader), but disables query caching for this test so repeated equivalence checks do not retain cached doc-id sets across iterations.For the no-score comparison path, the test now collects matching doc IDs with the existing test-framework
FixedBitSetCollectorand compares the resulting bit sets. This avoids materializing full sortedTopDocswhen only doc-id equality matters.This is intentionally different from prior range-cap attempts such as #15084 and #15118: it does not reduce randomized input sizes or change the stress shape. Instead, it removes the pressure sources that are outside the behavior this test is meant to validate.
Local investigation
I could not reproduce the OOM on my machine with the original issue command, likely due to local environment differences. To make the failure mode visible locally, I used the same stress shape with the test JVM constrained to a 16m heap.
With the original code, that 16m squeeze reproduced:
Heap dump inspection,
jstat, and JFR heap profiling pointed to avoidable pressure from the randomized query cache retaining doc-id sets and from the no-score branch materializing sortedTopDocs.Verification
Ran with JDK 26.
I also re-ran the local 16m squeeze with the inner
testDuelloop temporarily raised from100to10000; with this fix applied, it completed successfully instead of reaching the Java heap OOM state observed with the original code.Fixes #14611.