chore: avoid redundant IndexDocument construction in heap emplace#152
chore: avoid redundant IndexDocument construction in heap emplace#152longway-code wants to merge 2 commits intoalibaba:mainfrom
Conversation
…e cannot improve top-k In IndexDocumentHeap::emplace, the parent ailego::Heap::emplace constructs the IndexDocument object before checking whether the candidate score beats the current heap front. For rejected candidates this construction is wasted. Add an early-exit guard: when the heap is full, skip the emplace call entirely if score >= front().score(). The short-circuit evaluation of || ensures front() is only accessed when the heap is non-empty. Benchmark (N=100k, dim=128, topk=10, 200 queries): ~14% latency reduction.
|
@greptile |
Greptile SummaryOptimizes
This optimization is semantically equivalent to the original implementation but avoids constructing Confidence Score: 5/5
Important Files Changed
Last reviewed commit: fe83ee8 |
|
hi @longway-code, thanks for your kindly contribution. As we see the fix, the condition to avoid emplace, is quite same as the emplace inner logical condition. We'll take a test to see if this can bring the mentioned performance improvement. |
|
Hi, @longway-code, a test we took by @iaojnh shows there's no remarkable improvement over current codes. Could you please provide more details about the test setup and datasets etc. ? |
Summary
IndexDocumentHeap::emplacenow checksscore < front().score()before calling intoailego::Heap, avoiding unnecessary object construction for candidates that would be rejected anyway.Performance
Tested locally, decreased ~15% end-to-end latency of
FlatSearcher::search_impl(N=100k, dim=128, topk=10, FP32 SquaredEuclidean, 200 queries averaged).All Tests Passsed