Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion dataflow/custom-containers/miniconda/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ FROM ubuntu:latest
WORKDIR /pipeline

# Set the entrypoint to Apache Beam SDK worker launcher.
COPY --from=apache/beam_python3.9_sdk:2.55.1 /opt/apache/beam /opt/apache/beam
COPY --from=apache/beam_python3.9_sdk:2.69.0 /opt/apache/beam /opt/apache/beam
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

While updating the Beam SDK base image to version 2.69.0, the corresponding apache-beam dependency in dataflow/custom-containers/miniconda/requirements.txt was not updated. It is currently pinned to 2.37.0.

A mismatch between the SDK version in the container and the version used to run the pipeline can lead to runtime errors. It's highly recommended to keep these versions in sync.

Please update dataflow/custom-containers/miniconda/requirements.txt to:

apache-beam[gcp]==2.69.0

ENTRYPOINT [ "/opt/apache/beam/boot" ]

# Copy the python installation from the builder stage.
Expand Down
2 changes: 1 addition & 1 deletion dataflow/custom-containers/minimal/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ FROM python:3.9-slim
WORKDIR /pipeline

# Set the entrypoint to Apache Beam SDK worker launcher.
COPY --from=apache/beam_python3.9_sdk:2.55.1 /opt/apache/beam /opt/apache/beam
COPY --from=apache/beam_python3.9_sdk:2.69.0 /opt/apache/beam /opt/apache/beam
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The Beam SDK base image is being updated to 2.69.0, but the apache-beam dependency in dataflow/custom-containers/minimal/requirements.txt is still at version 2.37.0.

To prevent potential runtime incompatibilities, it's important that the apache-beam package version matches the base image version.

Please update dataflow/custom-containers/minimal/requirements.txt to:

apache-beam[gcp]==2.69.0

ENTRYPOINT [ "/opt/apache/beam/boot" ]

# Install the requirements.
Expand Down