Commit 05a3dca
feat: UDF support
* copy code from #67
* remove hacked RowEncoder.scala
* replace all schema(typeOf<R>()) to DataType.fromJson((schema(typeOf<R>()) as DataTypeWithClass).dt().json())
* add return udf data class test
* change test
* add in dataset test for calling the UDF-Wrapper
* add the same exception link
* refactor unWrapper
* add test for udf return a List
* make the test simpler
* add License
* add UDFRegister for 3.0
* remove useless import
* resolved deprecated method
* [experimental] add CatalystTypeConverters.scala for hacked it to implement UDF return data class
* [experimental] implement UDF return data class
* fix code inspection issue
* Adds suppre unused
Co-authored-by: can wang <can.wang@thoughtworks.com>
Co-authored-by: Pasha Finkelshteyn <pavel.finkelshtein@gmail.com>1 parent 9ef3809 commit 05a3dca
File tree
8 files changed
+3580
-1
lines changed- core/3.0
- src/main/scala/org/apache/spark/sql/catalyst
- kotlin-spark-api
- 2.4/src
- main/kotlin/org/jetbrains/kotlinx/spark/api
- test/kotlin/org/jetbrains/kotlinx/spark/api
- 3.0/src
- main/kotlin/org/jetbrains/kotlinx/spark/api
- test/kotlin/org/jetbrains/kotlinx/spark/api
8 files changed
+3580
-1
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
18 | 18 | | |
19 | 19 | | |
20 | 20 | | |
21 | | - | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
22 | 25 | | |
23 | 26 | | |
24 | 27 | | |
| |||
0 commit comments