Remove reliance on downloads.rapids.ai

[jameslamb]

Description

Across RAPIDS, we are actively working on removing use of downloads.rapids.ai to store build artifacts like conda packages and wheels.

For details, see https://github.com/rapidsai/build-infra/issues/237 (private issue, sorry).

There is no immediate plan to shut down downloads.rapids.ai altogether, but that is the eventual long-term plan. At this point, you should consider downloads.rapids.ai deprecated.

Benefits of this work

• protects against unanticipated disruptions

Acceptance Criteria

• nothing in this project relies on downloads.rapids.ai

Approach

Look for the uses of downloads.rapids.ai here:

git grep -E 'rapids\-.*\-s3.*'
git grep -E 'downloads\.rapids'

As of this writing, there was at least 1 place where packages were being manually aws cp'd up to downloads.rapids.ai:

MRC/ci/scripts/github/common.sh

Lines 58 to 60 in a30d865

	export DISPLAY_URL="https://downloads.rapids.ai/ci/mrc"
	export ARTIFACT_ENDPOINT="/pull-request/${PR_NUM}/${GIT_COMMIT}/${REAL_ARCH}/${BUILD_CC}"
	export ARTIFACT_URL="${S3_URL}${ARTIFACT_ENDPOINT}"

MRC/ci/scripts/github/common.sh

Line 65 in a30d865

export DISPLAY_ARTIFACT_URL="${DISPLAY_URL}${ARTIFACT_ENDPOINT}"

MRC/ci/scripts/github/common.sh

Lines 153 to 163 in a30d865

	function upload_artifact() {
	FILE_NAME=$1
	BASE_NAME=$(basename "${FILE_NAME}")
	rapids-logger "Uploading artifact: ${BASE_NAME}"
	if [[ "${LOCAL_CI}" == "1" ]]; then
	cp ${FILE_NAME} "${LOCAL_CI_TMP}/${BASE_NAME}"
	else
	aws s3 cp --only-show-errors "${FILE_NAME}" "${ARTIFACT_URL}/${BASE_NAME}"
	echo "- ${DISPLAY_ARTIFACT_URL}/${BASE_NAME}" >> ${GITHUB_STEP_SUMMARY}
	fi
	}

MRC/ci/scripts/github/conda.sh

Line 59 in a30d865

upload_artifact "${WORKSPACE_TMP}/conda-${REAL_ARCH}.tar.bz"

Across RAPIDS, we have been switching to the GitHub Actions artifact store, via a mix of the following:

• actions/upload-artifact GitHub action (code link)
• gha-tools shell scripts which interact with the GitHub API via the gh CLI (https://github.com/rapidsai/gha-tools)

I recommend you do something similar here.

Notes

Similar issue in Morpheus: nv-morpheus/Morpheus#2216