Skip to content

Commit b43e3f1

Browse files
committed
docs: Updates versions in docs, adds info about reduceGroupsK
1 parent 9e3b467 commit b43e3f1

File tree

2 files changed

+14
-7
lines changed

2 files changed

+14
-7
lines changed

README.md

Lines changed: 11 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
# Kotlin for Apache® Spark™ [![Maven Central](https://img.shields.io/maven-central/v/org.jetbrains.kotlinx.spark/kotlin-spark-api-parent.svg?label=Maven%20Central)](https://search.maven.org/search?q=g:org.jetbrains.kotlinx.spark%20AND%20v:1.0.0-preview1) [![official JetBrains project](http://jb.gg/badges/incubator.svg)](https://confluence.jetbrains.com/display/ALL/JetBrains+on+GitHub)
1+
# Kotlin for Apache® Spark™ [![Maven Central](https://img.shields.io/maven-central/v/org.jetbrains.kotlinx.spark/kotlin-spark-api-parent.svg?label=Maven%20Central)](https://search.maven.org/search?q=g:org.jetbrains.kotlinx.spark%20AND%20v:1.0.1) [![official JetBrains project](http://jb.gg/badges/incubator.svg)](https://confluence.jetbrains.com/display/ALL/JetBrains+on+GitHub)
22

33

44
Your next API to work with [Apache Spark](https://spark.apache.org/).
@@ -21,6 +21,7 @@ We have opened a Spark Project Improvement Proposal: [Kotlin support for Apache
2121
- [withCached function](#withcached-function)
2222
- [toList and toArray](#tolist-and-toarray-methods)
2323
- [Column infix/operator functions](#column-infixoperator-functions)
24+
- [`reduceGroups`](#reducegroups)
2425
- [Examples](#examples)
2526
- [Reporting issues/Support](#reporting-issuessupport)
2627
- [Code of Conduct](#code-of-conduct)
@@ -30,9 +31,9 @@ We have opened a Spark Project Improvement Proposal: [Kotlin support for Apache
3031

3132
| Apache Spark | Scala | Kotlin for Apache Spark |
3233
|:------------:|:-----------:|:------------:|
33-
| 3.0.0+ | 2.12 | kotlin-spark-api-3.0.0:1.0.0-preview2 |
34-
| 2.4.1+ | 2.12 | kotlin-spark-api-2.4_2.12:1.0.0-preview2 |
35-
| 2.4.1+ | 2.11 | kotlin-spark-api-2.4_2.11:1.0.0-preview2 |
34+
| 3.0.0+ | 2.12 | kotlin-spark-api-3.0.0:1.0.1 |
35+
| 2.4.1+ | 2.12 | kotlin-spark-api-2.4_2.12:1.0.1 |
36+
| 2.4.1+ | 2.11 | kotlin-spark-api-2.4_2.11:1.0.1 |
3637

3738
## Releases
3839

@@ -189,6 +190,12 @@ val dataset: Dataset<YourClass> = ...
189190
val newDataset: Dataset<Pair<TypeA, TypeB>> = dataset.selectTyped(col(YourClass::colA), col(YourClass::colB))
190191
```
191192

193+
### `reduceGroups`
194+
195+
We had to implemet `reduceGroups` operator for Kotlin separately as `reduceGroupsK` function, because otherwise it caused resolution ambiguity between Kotlin, Scala and Java APIs, which was quite hard to solve.
196+
197+
We have a special example of work with this function in the [Groups example](https://github.com/JetBrains/kotlin-spark-api/edit/main/examples/src/main/kotlin/org/jetbrains/kotlinx/spark/examples/Group.kt).
198+
192199

193200
## Examples
194201

docs/quick-start-guide.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -73,7 +73,7 @@ Here's what the `pom.xml` looks like for this example:
7373
<dependency> <!-- Kotlin Spark API dependency -->
7474
<groupId>org.jetbrains.kotlinx.spark</groupId>
7575
<artifactId>kotlin-spark-api-3.0.0_2.12</artifactId>
76-
<version>1.0.0-preview1</version>
76+
<version>1.0.1</version>
7777
</dependency>
7878
<dependency> <!-- Spark dependency -->
7979
<groupId>org.apache.spark</groupId>
@@ -163,7 +163,7 @@ dependencies {
163163
// Kotlin stdlib
164164
implementation 'org.jetbrains.kotlin:kotlin-stdlib:1.4.0'
165165
// Kotlin Spark API
166-
implementation 'org.jetbrains.kotlinx.spark:kotlin-spark-api-3.0.0_2.12:1.0.0-preview1' // Apache Spark
166+
implementation 'org.jetbrains.kotlinx.spark:kotlin-spark-api-3.0.0_2.12:1.0.1' // Apache Spark
167167
compileOnly 'org.apache.spark:spark-sql_2.12:3.0.0'
168168
}
169169
@@ -198,7 +198,7 @@ dependencies {
198198
// Kotlin stdlib
199199
implementation ("org.jetbrains.kotlin:kotlin-stdlib:1.4.0")
200200
// Kotlin Spark API
201-
implementation ("org.jetbrains.kotlinx.spark:kotlin-spark-api-3.0.0_2.12:1.0.0-preview1")
201+
implementation ("org.jetbrains.kotlinx.spark:kotlin-spark-api-3.0.0_2.12:1.0.1")
202202
// Apache Spark
203203
compileOnly ("org.apache.spark:spark-sql_2.12:3.0.0")
204204
}

0 commit comments

Comments
 (0)