Skip to content

Cascading-Flink not working with EMR Flink 1.3.2 #72

@mehikmat

Description

@mehikmat

I am thinking of using flink over EMR YARN. We have some hadoop jobs written in casdcading version 3.2.0 and hadoop 2.7.0.

Today, I tried to use latest cascading flink connector(ver 0.3.0) but got the folloing error while running OVER YARN:

Application application_1533715496882_0002 failed 1 times due to AM Container for appattempt_1533715496882_0002_000001 exited with exitCode: 31
For more detailed output, check application tracking page:http://ip-172-16-15-140.us-east-1.compute.internal:8088/cluster/app/application_1533715496882_0002Then, click on links to logs of each attempt.
Diagnostics: Exception from container-launch.
Container id: container_1533715496882_0002_01_000001
Exit code: 31
Stack trace: ExitCodeException exitCode=31:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:582)
at org.apache.hadoop.util.Shell.run(Shell.java:479)

It seems that current version of cascading flink supports only flink 1.0.3 and cascading 3.1.0 but EMR has Flink version 1.3.2.

So are we planing to upgrade flink version and casdcading version in the lastest casdcading flink...

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions