[FLINK-39640][hive] Bump Flink 1.20.0 to 1.20.3 and shade plugin to 3.6.0#40
[FLINK-39640][hive] Bump Flink 1.20.0 to 1.20.3 and shade plugin to 3.6.0#40jlalwani-amazon wants to merge 1 commit into
Conversation
jlalwani-amazon
force-pushed
the
flink-1.20.3-bump
branch
4 times, most recently
from
ddf0037 to
a48c7a6
Compare
….6.0 Flink 1.20.0 binary is no longer available on Apache download mirrors. Bump to 1.20.3 which is the current latest 1.20.x release. Flink 1.20.3 upgraded parquet-hadoop from 1.13.1 to 1.15.2 which includes multi-release JAR entries with Java 17/22 class files. The inherited maven-shade-plugin 3.2.4 cannot process these. Upgrade to 3.6.0 which supports multi-release JARs. Also fix pre-existing dependency convergence errors from Hive/Hadoop transitive dependencies by pinning versions in dependencyManagement, and update NOTICE file for the parquet version change.
jlalwani-amazon
force-pushed
the
flink-1.20.3-bump
branch
from
a48c7a6 to
1a54d50
Compare
| <version>0.13.0</version> | ||
| </dependency> | ||
| <!-- For dependency convergence: Hive/Hadoop transitive conflicts --> | ||
| <dependency> |
There was a problem hiding this comment.
A lot of the dependencies have active CVEs, Can you look into moving the packages to a api version to minimise them without breaking.
There was a problem hiding this comment.
These are dependency convergence pins to support 1.20.3 . These versions came from 1.20.3. Main has moved to 2.0-preview. I'm not sure if we want to support 1.20.3 in this connector. If not, then this PR is obsolete
There was a problem hiding this comment.
fyi the HTTP connector is going straight to supporting 2.2. I guess it depends on whetehr there is an appetite for a v1.20 version.
|
Do we still need to support Flink 1.20.x? I can base this off a branch if there is any need to support 1.20.x |
| <dependency> | ||
| <groupId>io.netty</groupId> | ||
| <artifactId>netty</artifactId> | ||
| <version>3.10.6.Final</version> | ||
| </dependency> | ||
| <dependency> |
There was a problem hiding this comment.
are you sure we need it?
| strategy: | ||
| matrix: | ||
| flink: [ 1.20.0 ] | ||
| flink: [ 1.20.3 ] |
There was a problem hiding this comment.
| flink: [ 1.20.3 ] | |
| flink: [ 1.20.4 ] |
4 participants
What is the purpose of the change
Fix broken CI: Flink 1.20.0 binary is no longer available on Apache download mirrors (only the latest patch release per minor version is hosted). Bump to 1.20.3.
Additionally, Flink 1.20.3 upgraded
parquet-hadoopfrom 1.13.1 to 1.15.2. The parquet 1.15.x line includes multi-release JAR entries with Java 17/22 class files underMETA-INF/versions/. The inheritedmaven-shade-plugin:3.2.4cannot process these, failing withUnsupported class file major version 61. Upgraded to 3.6.0.JIRA: FLINK-39640
Brief change log
flink.versionfrom 1.20.0 to 1.20.3maven-shade-pluginversion to 3.6.0 (multi-release JAR support)dependencyManagement)HiveServer2EndpointITCase(FLINK-31351): flaky test that times out on CI. The fix in the main Flink repo (flink-sql-gatewayOperationManager) reduced but did not eliminate the race condition on resource-constrained runners.Verifying this change
CI passed on fork: https://github.com/jlalwani-amazon/flink-connector-hive/actions
Does this pull request potentially affect one of the following parts?