Skip to content

Conversation

@fresh-borzoni
Copy link
Contributor

closes #191

Enables Arrow serialization for decimal and temporal types, providing feature parity with the Java client.

Fluss Type Arrow Type
DECIMAL(p, s) Decimal128(p, s)
DATE Date32
TIME Time32/Time64
TIMESTAMP Timestamp (all precisions)
TIMESTAMP_LTZ Timestamp (all precisions)

@fresh-borzoni
Copy link
Contributor Author

@luoyuxia @leekeiabstraction PTAL 🙏

Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Enables Apache Arrow serialization for Fluss DECIMAL and temporal types (DATE, TIME, TIMESTAMP, TIMESTAMP_LTZ) to match feature parity with the Java client.

Changes:

  • Added Arrow schema/type mappings for decimal and temporal Fluss types and made schema conversion fallible (Result).
  • Implemented Arrow appends for decimal/date/time/timestamps, including decimal rescaling/validation and timestamp unit handling.
  • Updated write/batch/scanner codepaths and tests to propagate the new fallible APIs.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
crates/fluss/src/row/datum.rs Adds datum accessors/conversions and Arrow append support for decimal + temporal types.
crates/fluss/src/record/arrow.rs Extends Arrow schema/type mapping and RecordBatch builder creation for decimal/temporal Arrow types; propagates errors instead of panicking.
crates/fluss/src/client/write/batch.rs Makes Arrow write batch construction fallible to reflect schema/builder validation.
crates/fluss/src/client/write/accumulator.rs Propagates new Result return from Arrow write batch creation.
crates/fluss/src/client/table/scanner.rs Propagates fallible Arrow schema conversion when initializing scanners/read contexts.
crates/fluss/src/client/table/log_fetch_buffer.rs Updates tests/helpers to handle fallible Arrow schema conversion and builder creation.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@fresh-borzoni fresh-borzoni force-pushed the temporal-decimals-arrow-support branch from f7efa2f to 2f12672 Compare January 22, 2026 01:45
@fresh-borzoni fresh-borzoni force-pushed the temporal-decimals-arrow-support branch from 2f12672 to 3bc9d15 Compare January 22, 2026 01:56
Copy link
Contributor

@leekeiabstraction leekeiabstraction left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TY for the PR! Left some comments

Copy link
Contributor

@leekeiabstraction leekeiabstraction left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added further question

Copy link
Contributor

@luoyuxia luoyuxia left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@fresh-borzoni Thanks for the pr. LGTM overall, only lefr minor comments.

@fresh-borzoni
Copy link
Contributor Author

@leekeiabstraction @luoyuxia
Thanks for the review. Addressed the comments. PTAL 🙏

Copy link
Contributor

@luoyuxia luoyuxia left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@fresh-borzoni Thanks. LGTM!

@luoyuxia luoyuxia merged commit ed646e2 into apache:main Jan 23, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support temporal and Decimal Types in Arrow writing

3 participants