Skip to content

Add rollup optimized for HashAggregation#122

Open
wenjiecn wants to merge 14 commits intobytedance:mainfrom
wenjiecn:rollup_optimized
Open

Add rollup optimized for HashAggregation#122
wenjiecn wants to merge 14 commits intobytedance:mainfrom
wenjiecn:rollup_optimized

Conversation

@wenjiecn
Copy link
Copy Markdown

What problem does this PR solve?

Issue Number: close #121

Type of Change

  • 🐛 Bug fix (non-breaking change which fixes an issue)
  • ✨ New feature (non-breaking change which adds functionality)
  • 🚀 Performance improvement (optimization)
  • ⚠️ Breaking change (fix or feature that would cause existing functionality to change)
  • 🔨 Refactoring (no logic changes)
  • 🔧 Build/CI or Infrastructure changes
  • 📝 Documentation only

Description

Create an aggNode in the intermediate phase in the agg operator, and use the aggNode to create three Intermediate groupingSet operators to process group data of groups 1 to 3. The original partial groupingSet operator processes data in group 0.
The total data input is reduced to reduce the HashTable operation time, which is optimized in the rollup scenario.

Performance Impact

  • No Impact: This change does not affect the critical path (e.g., build system, doc, error handling).

  • Positive Impact: I have run benchmarks.

    Click to view Benchmark Results

TPCDS99 1T results in velox.
q22
Before: 13.686s
After: 11.256s (+17.76%)
q67
Before: 152.416s
After: 136.983 (+10.13%)
```

  • Negative Impact: Explained below (e.g., trade-off for correctness).

Release Note

Please describe the changes in this PR

Release Note:

Release Note:
- Identifies the execution plan of the 1. expand->agg; 2. expand->projection->agg in the LocalPlanner and sets the flag of the rollup scenario to the operator.
- Add a flag to the expand operator to generate only data of group 0.
- Create an aggNode in the intermediate phase in the agg operator, and use the aggNode to create three Intermediate groupingSet operators to process group data of groups 1 to 3. The original partial groupingSet operator processes data in group 0.

Checklist (For Author)

  • I have added/updated unit tests (ctest).
  • I have verified the code with local build (Release/Debug).
  • I have run clang-format / linters.
  • (Optional) I have run Sanitizers (ASAN/TSAN) locally for complex C++ changes.
  • No need to test or manual test.

Breaking Changes

  • No

  • Yes (Description: ...)

    Click to view Breaking Changes
    Breaking Changes:
    - Description of the breaking change.
    - Possible solutions or workarounds.
    - Any other relevant information.
    

@CLAassistant
Copy link
Copy Markdown

CLAassistant commented Jan 15, 2026

CLA assistant check
All committers have signed the CLA.

Comment thread bolt/exec/LocalPlanner.cpp Outdated
Comment thread bolt/exec/LocalPlanner.cpp Outdated
@wenjiecn wenjiecn closed this Jan 22, 2026
@wenjiecn wenjiecn reopened this Jan 22, 2026
Copy link
Copy Markdown
Collaborator

@kexianda kexianda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@wenjiecn Thank you for contributing this PR to bolt.
It isn't a small PR, how about adding an UT for this?

Comment thread bolt/exec/Driver.h Outdated
Comment thread bolt/exec/HashAggregation.cpp Outdated
Comment thread bolt/exec/HashAggregation.cpp Outdated
@wenjiecn
Copy link
Copy Markdown
Author

@wenjiecn Thank you for contributing this PR to bolt. It isn't a small PR, how about adding an UT for this?

@kexianda Thank you for your review. All modifications are done.

@guhaiyan0221 guhaiyan0221 force-pushed the main branch 2 times, most recently from 906a3a9 to b88fc0e Compare March 4, 2026 15:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature] Add rollup optimized for HashAggregation

5 participants