Skip to content

[ci] upgrade CI test containers from Ubuntu 20.04 to 22.04#61533

Draft
andrew-anyscale wants to merge 4 commits intomasterfrom
andrew/revup/master/ubuntu-22-04-test
Draft

[ci] upgrade CI test containers from Ubuntu 20.04 to 22.04#61533
andrew-anyscale wants to merge 4 commits intomasterfrom
andrew/revup/master/ubuntu-22-04-test

Conversation

@andrew-anyscale
Copy link
Contributor

@andrew-anyscale andrew-anyscale commented Mar 5, 2026

Upgrades CI test container base images from Ubuntu 20.04 to 22.04 (jammy), including the mandatory clang-12 to clang-14 toolchain upgrade available in 22.04.

  • focal → jammy for all base images and wanda configs (base.test, base.gpu, base.cu128, forge wanda)
  • clang-12 → clang-14 in base.test, base.gpu, base.build Dockerfiles and install-llvm-binaries.sh
  • libncurses5 → libncurses6
  • libgl1-mesa-glx → libgl1 (dropped in 22.04)
  • netcat → netcat-openbsd (22.04 no longer provides the bare netcat metapackage)
  • MongoDB via official repo for jammy (8.0), replacing the obsolete ubuntu package
  • Java 8 update-alternatives: 22.04 defaults to Java 11, pin to Java 8 where needed
  • test_mongo.py: start mongod directly instead of via service (mongodb-org 8.0 has no SysV init script)
  • test_dashboard_profiler.py: skip py-spy --native on Linux (known issue, see [Dashboard] Getting stacktrace from the dashboard triggers "failed to get os threadid" when --native option is used #30566)
  • onnx 1.15.0 → 1.16.0 (compatibility with newer environment)
  • Simplified Podman doc (20.04 workaround removed)

Topic: ubuntu-22-04-test
Labels: draft

Signed-off-by: andrew andrew@anyscale.com

@andrew-anyscale
Copy link
Contributor Author

andrew-anyscale commented Mar 5, 2026

Reviews in this chain:
#61533 [ci] upgrade CI test containers from Ubuntu 20.04 to 22.04

@andrew-anyscale
Copy link
Contributor Author

andrew-anyscale commented Mar 5, 2026

# head base diff date summary
0 1dac0b9e 5270e8ce diff Mar 5 13:11 PM 19 files changed, 77 insertions(+), 45 deletions(-)
1 2f035d8c 5270e8ce diff Mar 5 13:12 PM 0 files changed
2 9de24404 5270e8ce diff Mar 5 13:38 PM 3 files changed, 13 insertions(+), 5 deletions(-)
3 4be924b4 5270e8ce diff Mar 5 14:01 PM 1 file changed, 3 insertions(+), 3 deletions(-)
4 fb42727f 5270e8ce diff Mar 6 9:17 AM 1 file changed, 3 insertions(+)
5 65100c8e 1f3a82ee diff Mar 6 12:37 PM 1 file changed, 3 insertions(+)

@andrew-anyscale andrew-anyscale changed the title [ci] Upgrade CI test containers from Ubuntu 20.04 to 22.04 [ci] upgrade CI test containers from Ubuntu 20.04 to 22.04 Mar 5, 2026
@andrew-anyscale andrew-anyscale force-pushed the andrew/revup/master/ubuntu-22-04-test branch from 1dac0b9 to 2f035d8 Compare March 5, 2026 21:12
Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request upgrades the CI test containers from Ubuntu 20.04 to 22.04, which is a significant and necessary infrastructure update. The changes across Dockerfiles, configuration files, and test code are mostly correct and well-justified. I have a few suggestions for improvement. First, for better compatibility, I recommend using LLVM binaries specifically built for Ubuntu 22.04. Second, in the documentation, it would be beneficial to retain installation instructions for Podman on Ubuntu 20.04, as it is a widely used LTS release. Lastly, I've suggested a small refactoring in a test file to improve resource management by ensuring a temporary directory is always cleaned up.

@andrew-anyscale andrew-anyscale force-pushed the andrew/revup/master/ubuntu-22-04-test branch from 2f035d8 to 9de2440 Compare March 5, 2026 21:38
@andrew-anyscale andrew-anyscale added the go add ONLY when ready to merge, run all tests label Mar 6, 2026
andrew-anyscale and others added 4 commits March 6, 2026 12:37
Upgrades CI test container base images from Ubuntu 20.04 to 22.04 (jammy), including the mandatory clang-12 to clang-14 toolchain upgrade available in 22.04.

- focal → jammy for all base images and wanda configs (base.test, base.gpu, base.cu128, forge wanda)
- clang-12 → clang-14 in base.test, base.gpu, base.build Dockerfiles and install-llvm-binaries.sh
- libncurses5 → libncurses6
- libgl1-mesa-glx → libgl1 (dropped in 22.04)
- netcat → netcat-openbsd (22.04 no longer provides the bare netcat metapackage)
- MongoDB via official repo for jammy (8.0), replacing the obsolete ubuntu package
- Java 8 update-alternatives: 22.04 defaults to Java 11, pin to Java 8 where needed
- test_mongo.py: start mongod directly instead of via service (mongodb-org 8.0 has no SysV init script)
- test_dashboard_profiler.py: skip py-spy --native on Linux (known issue, see #30566)
- onnx 1.15.0 → 1.16.0 (compatibility with newer environment)
- Simplified Podman doc (20.04 workaround removed)

Topic: ubuntu-22-04-test
Labels: draft

Signed-off-by: andrew <andrew@anyscale.com>
14.0.6 does not ship an ubuntu-22.04 tarball (404). No LLVM 14.x release
does. Use 14.0.0 with the ubuntu-18.04 tarball, which is glibc
backwards-compatible and works on Ubuntu 22.04.

Topic: ubuntu-22-04-test
Labels: draft

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: andrew <andrew@anyscale.com>
The base.test container was missing CC/CXX unlike base.build and
base.gpu. Set them explicitly so tools running inside the test
container (e.g. test_exported_symbols_unix which builds _raylet via
Bazel) use the correct clang-14 compiler.

Topic: ubuntu-22-04-test
Labels: draft

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: andrew <andrew@anyscale.com>
Include the C++ compiler in Bazel action fingerprints via
--action_env=CC and --action_env=CXX. This ensures cache entries
produced by one compiler version (e.g. clang-12 on Ubuntu 20.04)
are never reused by a different version (e.g. clang-14 on Ubuntu
22.04), preventing stale .cppmap and other C++ artifacts from
causing spurious build failures after a compiler upgrade.

Topic: ubuntu-22-04-test

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: andrew <andrew@anyscale.com>
@andrew-anyscale andrew-anyscale force-pushed the andrew/revup/master/ubuntu-22-04-test branch from fb42727 to 65100c8 Compare March 6, 2026 20:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci go add ONLY when ready to merge, run all tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant