Refactor ClickHouse date and datetime handlers and add array support #69
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This pull request refactors how ClickHouse column types are handled in the codebase, with a focus on improving date/time formatting and adding support for array types. The changes make date and datetime values more readable by formatting them as strings, and introduce a new handler for array types. Additionally, the logic for recognizing certain column types has been enhanced for better normalization.
This will fix issues related to date and array column parsing shown below:
Date and DateTime formatting improvements:
handleDateandhandleDateTimefunctions to formatDate,Date32,DateTime, andDateTime64columns as strings ("YYYY-MM-DD" for dates, and ISO format for datetimes), replacing the previous generichandleTimehandler. (internal/pkg/object/command/clickhouse/column_types.go) [1] [2]Support for array column types:
handleArrayfunction to properly handle ClickHouseArraycolumns, supporting both nullable and non-nullable arrays. (internal/pkg/object/command/clickhouse/column_types.go) [1] [2]Type normalization enhancements:
unwrapCHTypefunction to correctly identify and normalizeDateTime64,DateTime, andArraytypes based on their string representation. (internal/pkg/object/command/clickhouse/column_types.go)Test Results:
tested the new changes for all the datatype conversion in clickhouse plugin.
SQL:
Result: