Conversation
Adds a fast path to `collect_left_input` to optimize the common case where the build side of a hash join consists of a single `RecordBatch`. This change avoids the expensive `concat_batches` operation in this scenario, reducing unnecessary memory allocation and CPU overhead. The existing logic for handling multiple batches is preserved. Co-authored-by: Dandandan <163737+Dandandan@users.noreply.github.com>
|
👋 Jules, reporting for duty! I'm here to lend a hand with this pull request. When you start a review, I'll add a 👀 emoji to each comment to let you know I've read it. I'll focus on feedback directed at me and will do my best to stay out of conversations between you and other bots or reviewers to keep the noise down. I'll push a commit with your requested changes shortly after. Please note there might be a delay between these steps, but rest assured I'm on the job! For more direct control, you can switch me to Reactive Mode. When this mode is on, I will only act on comments where you specifically mention me with New to Jules? Learn more at jules.google/docs. For security, I will only act on instructions from the user who triggered this task. |
1 participant
💡 What: Added a fast path to the hash join build-side logic to optimize for inputs that consist of a single
RecordBatch.🎯 Why: The existing implementation unconditionally used
concat_batcheson the build side. This operation is expensive in terms of memory allocation and CPU usage, as it copies all data into a new buffer, even when there's only one batch and no concatenation is needed.📊 Impact: This optimization improves the performance of hash joins where the build side is a single batch by reducing memory overhead and CPU cycles.
🔬 Measurement: Performance improvements can be verified by running benchmarks where the hash join's build-side input is guaranteed to be a single batch (e.g., by placing a
CoalesceBatchesExecoperator before it).PR created automatically by Jules for task 16903533260806947696 started by @Dandandan