Skip to content

optimize buffer build#77

Open
FelixYBW wants to merge 10 commits intooap-project:arrow-4.0.0-oapfrom
FelixYBW:arrow-4.0.0-oap
Open

optimize buffer build#77
FelixYBW wants to merge 10 commits intooap-project:arrow-4.0.0-oapfrom
FelixYBW:arrow-4.0.0-oap

Conversation

@FelixYBW
Copy link

There are 3 optimizations included:

  1. Added prefetch to buffer_build, assumption is if we append one value, we may append the second one soon
  2. Since most of the value isn't NULL, we reset the null bitmap buffer to 0xffffffff once we allocate it. Later we skip the bitset if value is true. Now we still need to update the bitlength, and one branch. Two more optimizations can be done: 1) skip memory allocation until first false value met. 2) in binary_builder etc, skip the bitmap append until first false value
  3. specify int32_t for typedbufferbuild, in which we use *(pos)=value, instead of memcpy

@github-actions
Copy link

Thanks for opening a pull request!

If this is not a minor PR. Could you open an issue for this pull request on JIRA? https://issues.apache.org/jira/browse/ARROW

Opening JIRAs ahead of time contributes to the Openness of the Apache Arrow project.

Then could you also rename pull request title in the following format?

ARROW-${JIRA_ID}: [${COMPONENT}] ${SUMMARY}

or

MINOR: [${COMPONENT}] ${SUMMARY}

See also:

binwei added 5 commits January 12, 2022 16:18
2. reset null bitmap to all true after allocation, skip bit update if it's true
…nto arrow-4.0.0-oap

Conflicts:
	cpp/src/arrow/buffer_builder.h
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant