[REQUEST] Make images available in Amazon ECR Public Gallery

[kaweantonio]

Currently, the Docker images for the SageMaker Spark Container are not available in the Amazon ECR Public Gallery, which complicates the direct use of these images. I would like to request that the SageMaker Spark Container images be published in the ECR Public Gallery, having the images available publicly would greatly improve the development and deployment workflow for users of SageMaker Processing Jobs with Spark.

[evolate-tim]

This would be great.

Im also meet with this error while trying to fetch an image:
docker pull 330188676905.dkr.ecr.eu-north-1.amazonaws.com/sagemaker-spark-processing:3.5-cpu-py39-v1.0
Error response from daemon: Head "https://330188676905.dkr.ecr.eu-north-1.amazonaws.com/v2/sagemaker-spark-processing/manifests/3.5-cpu-py39-v1.0": no basic auth credentials

[Rishav-dev-star]

@kaweantonio @evolate-tim
are u able to resolve the above issue?

[mrpic-amzn]

having the images available publicly would greatly improve the development and deployment workflow for users of SageMaker Processing Jobs with Spark.

Can you elaborate on how development and deployment workflows would be improved? On each release, we publish the newly available tags in the private repos.

[kaweantonio]

Can you elaborate on how development and deployment workflows would be improved? On each release, we publish the newly available tags in the private repos.

At my workplace, we use a private repository to manage the packages we need to use in our pipelines (e.g. Docker images, NPM packages, Python libraries on PyPI, etc.). Since Sagemaker images are only available in private repositories in AWS accounts, we cannot replicate these images to our Artifactory.

If these images were available in AWS Public ECR Images (or DockerHub), our Artifactory would be able to replicate them instantly.

[timschill]

We face kinda the same issue as kaweantonio

[mrpic-amzn]

Unfortunately, due to prioritization this repository is basically in KTLO and might also be on the path for deprecation in the future. New releases will probably only be for security fixes.