Is your feature request related to a problem or challenge?
DataFusion can't represent declared range-partitioned on a scan because there currently is no logical representation.
Scan sort order has this capability with logical SortExpr, but range-partitioned scan output currently has no equivalent. datafusion_expr::Partitioning only supports hash, round-robin, and distribute-by.
Furthermore it is documented that that datafusion_expr::Partitioning is only used for introducing repartitions but this intends to extend the use case to be able to declare a source's output partitioning along with introducing repartitions.
Describe the solution you'd like
Add a logical range partitioning representation so range-partitioned sources can declare their output partitioning without depending on physical partitioning types. Execution, protobuf, and Substrait support can remain follow-up work.
Is your feature request related to a problem or challenge?
DataFusion can't represent declared range-partitioned on a scan because there currently is no logical representation.
Scan sort order has this capability with logical
SortExpr, but range-partitioned scan output currently has no equivalent.datafusion_expr::Partitioningonly supports hash, round-robin, and distribute-by.Furthermore it is documented that that
datafusion_expr::Partitioningis only used for introducing repartitions but this intends to extend the use case to be able to declare a source's output partitioning along with introducing repartitions.Describe the solution you'd like
Add a logical range partitioning representation so range-partitioned sources can declare their output partitioning without depending on physical partitioning types. Execution, protobuf, and Substrait support can remain follow-up work.