Skip to content

Conversation

@fresh-borzoni
Copy link
Contributor

@fresh-borzoni fresh-borzoni commented Jan 23, 2026

Linked issue: close #167

Add Date, Time, Timestamp (NTZ/LTZ), and Decimal type support to Python bindings' append() method.

Brief change log

  • Add type converters for datetime.date, datetime.time, datetime.datetime, and decimal.Decimal in table.rs
  • Fix Arrow-to-Fluss type mapping with correct precision for Time32/Time64 and Timestamp types
  • Route Arrow Timestamp(unit, Some(tz)) → TimestampLtz, Timestamp(unit, None) → TimestampNtz
  • Fix write_pandas() to pass expected schema for proper type casting
  • Update example with new types demonstration

@fresh-borzoni fresh-borzoni force-pushed the python-arrow-decimals-temporal branch from 4fdd56c to e8de061 Compare January 23, 2026 18:07
@fresh-borzoni
Copy link
Contributor Author

@leekeiabstraction @luoyuxia PTAL 🙏

Copy link
Contributor

@leekeiabstraction leekeiabstraction left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the PR! Left some comments.

@fresh-borzoni
Copy link
Contributor Author

@leekeiabstraction Thank you for the review.
Addressed comments. PTAL 🙏

Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds support for temporal types (Date, Time, Timestamp NTZ/LTZ) and Decimal types to the Python bindings' append() method. It enables users to write these types directly from Python datetime objects and decimal.Decimal to Fluss tables.

Changes:

  • Added Arrow-to-Fluss type mapping with precision-aware conversions for Time32/Time64 and Timestamp types
  • Implemented Python-to-Datum converters for datetime.date, datetime.time, datetime.datetime (both naive and timezone-aware), pandas.Timestamp, and decimal.Decimal
  • Enhanced write_pandas() to pass expected schema for proper type casting
  • Updated example to demonstrate new type support

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

File Description
bindings/python/src/utils.rs Added precision-aware Arrow-to-Fluss type mappings for Time32/Time64 and Timestamp types, with timezone-based routing for TimestampLtz vs Timestamp NTZ
bindings/python/src/table.rs Implemented comprehensive type converters for temporal and decimal types, including time conversion constants, caching mechanisms, and helper functions for Python type conversions
bindings/python/example/example.py Updated example to demonstrate Date, Time, Timestamp, and Decimal type usage with both Arrow and row-level append operations
bindings/python/Cargo.toml Added bigdecimal 0.4 dependency for Decimal type support

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

// Total milliseconds (note: days can be negative for dates before epoch)
let total_micros = days * MICROS_PER_DAY + seconds * MICROS_PER_SECOND + microseconds;
let millis = total_micros / MICROS_PER_MILLI;
let nano_of_milli = ((total_micros % MICROS_PER_MILLI) * MICROS_PER_MILLI) as i32;
Copy link

Copilot AI Jan 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The constant MICROS_PER_MILLI is being used as both the modulo divisor and as a conversion factor from microseconds to nanoseconds. While numerically correct (both are 1000), this is semantically confusing. Consider using a more appropriately named constant for the conversion factor, such as NANOS_PER_MICRO, or use a literal 1000 for clarity. This would improve code readability and make the intent clearer.

Copilot uses AI. Check for mistakes.
hour as i8,
minute as i8,
second as i8,
microsecond as i32 * 1000,
Copy link

Copilot AI Jan 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For consistency with the rest of the code that uses named constants for time conversions (lines 29-35), this magic number should be replaced with a named constant. Consider using the existing NANOS_PER_MILLI constant divided by MICROS_PER_MILLI, or defining a new constant NANOS_PER_MICRO = 1000.

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add temporal and decimal type support to python row append() API

2 participants