Related to #2723.
A lot of Spark SQL tests don't put their source data in a format that Comet accelerates reading from. Comet generally wants data in Parquet or Iceberg. A number of Spark SQL suites (e.g., UDFSuite) don't write to Parquet so their scans are just LocalTableScanExec, and Comet is not exercising UDF compatibility because the scan underneath isn't native. We can either do what #2723 suggests and enable converting to columnar, or try enabling LocalTableScanExec now that we have #2735. @andygrove tried the former in #2714, but the logs are gone now so I'm not sure how ugly it was. Starting with LocalTableScanExec might be a smaller blast radius?
Related to #2723.
A lot of Spark SQL tests don't put their source data in a format that Comet accelerates reading from. Comet generally wants data in Parquet or Iceberg. A number of Spark SQL suites (e.g., UDFSuite) don't write to Parquet so their scans are just
LocalTableScanExec, and Comet is not exercising UDF compatibility because the scan underneath isn't native. We can either do what #2723 suggests and enable converting to columnar, or try enablingLocalTableScanExecnow that we have #2735. @andygrove tried the former in #2714, but the logs are gone now so I'm not sure how ugly it was. Starting withLocalTableScanExecmight be a smaller blast radius?