[data][ci] removing databuild from data CI#61508
Conversation
Signed-off-by: elliot-barn <elliot.barnwell@anyscale.com>
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request aims to remove the databuild CI configuration and replace its usages with datalbuild. The changes are mostly correct, but there is a critical issue where the datalbuild image is still being tagged as databuild. This will cause CI failures because tests will be unable to find the correctly named image. A fix is required in ci/docker/datal.build.wanda.yaml to update the image tag to align with the new build names.
| commands: | ||
| - bazel run //ci/ray_ci:test_in_docker -- //python/ray/data/... data | ||
| --build-name databuild-py3.12 --python-version 3.12 | ||
| --build-name datalbuild-py3.12 --python-version 3.12 |
There was a problem hiding this comment.
This change to --build-name datalbuild-py3.12 is correct in principle, but it will cause the CI to fail. The configuration in ci/docker/datal.build.wanda.yaml for this build still specifies the image tag as cr.ray.io/rayproject/databuild-py$PYTHON. This means the test runner will look for an image named datalbuild-py3.12 but won't find it, because the image will be tagged as databuild-py3.12.
To resolve this, the tags field in ci/docker/datal.build.wanda.yaml needs to be updated to cr.ray.io/rayproject/datalbuild-py$PYTHON.
This issue affects all test steps being updated to use datalbuild in this file.
There was a problem hiding this comment.
Nope. the tag on the datal.build.wanda.yaml images uses datalbuild-py$PYTHON
Link here: https://github.com/ray-project/ray/blob/master/ci/docker/datal.build.wanda.yaml#L1
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.
| PYTHON: "{{matrix}}" | ||
| tags: cibase | ||
|
|
||
| - name: databuild-multipy |
There was a problem hiding this comment.
Removed build step still referenced in core CI
High Severity
The databuild-multipy build step was removed from data.rayci.yml, but .buildkite/core.rayci.yml still references it at line ~182 with --build-name databuild-py3.12 and at line ~184 with depends_on: databuild-multipy. This creates a broken dependency — the core CI step depends on a build step that no longer exists, which will cause CI pipeline failures. These references in core.rayci.yml need to be updated to use datalbuild-py3.12 and datalbuild-multipy respectively.
| commands: | ||
| - bazel run //ci/ray_ci:test_in_docker -- //python/ray/data/... data | ||
| --build-name databuild-py3.12 --python-version 3.12 | ||
| --build-name datalbuild-py3.12 --python-version 3.12 |
| PYTHON: "{{matrix}}" | ||
| tags: cibase | ||
|
|
||
| - name: databuild-multipy |
aslonnie
left a comment
aslonnie
left a comment
There was a problem hiding this comment.
ping me for merging when the two issues/coomments are checked.
3 participants
databuild and datalbuild(latest) now are using the same set of dependencies.
removing databuild-py* and replacing with datalbuild-py CI images