Skip to content

[CI] Warm start CI before tests #565

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
xyao-nv wants to merge 3 commits into
base: main
Choose a base branch
from
Draft

[CI] Warm start CI before tests #565

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
4e473e9
warm start
xyao-nv Apr 9, 2026
577b001
Update warm_shader_cache.py
xyao-nv Apr 9, 2026
d409913
Merge branch 'main' into xyao/exp/ci_warm_start
xyao-nv Apr 10, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions .github/workflows/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -140,6 +140,10 @@ jobs:
# - All subprocess-spawning tests (each spawns their own SimulationApp)
#
# To restore sanity here is a goal for Isaac Lab Arena v0.3.

- name: Warm Kit shader cache
Copy link
Copy Markdown

@isaaclab-review-bot isaaclab-review-bot bot Apr 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider adding a step-level timeout-minutes: 10 here. The docstring says shader compilation can take 60-180s, but if Kit hangs (no GPU, license issues, etc.) you'd want the step to fail fast rather than burn the full job timeout.

run: /isaac-sim/python.sh scripts/warm_shader_cache.py

- name: Run in-process Newton tests
run: |
/isaac-sim/python.sh -m pytest -sv -m with_newton \
Expand Down Expand Up @@ -186,6 +190,9 @@ jobs:
- *git_lfs_step
- *install_project_step

- name: Warm Kit shader cache
run: /isaac-sim/python.sh scripts/warm_shader_cache.py

# Run the policy (GR00T) related tests.
- name: Run policy-related pytest
run: /isaac-sim/python.sh -m pytest -sv isaaclab_arena_gr00t/tests/
Expand Down
36 changes: 36 additions & 0 deletions scripts/warm_shader_cache.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
# Copyright (c) 2025-2026, The Isaac Lab Arena Project Developers (https://github.com/isaac-sim/IsaacLab-Arena/blob/main/CONTRIBUTORS.md).
# All rights reserved.
#
# SPDX-License-Identifier: Apache-2.0

"""Warm the Kit shader cache by launching and closing a SimulationApp.

On CI (fresh container, no cached shaders) the first SimulationApp startup
compiles hundreds of RTX/rendering shaders, taking 60-180 s on constrained
runners. Running this script once before tests ensures every subsequent
SimulationApp — persistent or subprocess-spawned — gets a warm-cache start.

Usage:
/isaac-sim/python.sh scripts/warm_shader_cache.py
"""

import argparse
import time

from isaaclab.app import AppLauncher


def main():
t0 = time.monotonic()
args = argparse.Namespace(headless=True, enable_cameras=True, visualizer=[])
launcher = AppLauncher(args)
Copy link
Copy Markdown

@isaaclab-review-bot isaaclab-review-bot bot Apr 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: Hand-rolling argparse.Namespace couples this to AppLauncher's internal expectations. If the launcher ever adds validation or required fields, this will break without a clear error.

Consider:

parser = argparse.ArgumentParser()
AppLauncher.add_app_launcher_args(parser)
args = parser.parse_args(["--headless", "--enable_cameras"])

This also self-documents the available flags.

Copy link
Copy Markdown

@isaaclab-review-bot isaaclab-review-bot bot Apr 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Worth a brief comment explaining why enable_cameras=True — the point is to trigger the RTX/rendering shader compilation path, not to actually use cameras. Future readers might otherwise wonder.

elapsed_launch = time.monotonic() - t0
print(f"[warm_shader_cache] AppLauncher ready in {elapsed_launch:.1f}s")

launcher.app.close()
elapsed_total = time.monotonic() - t0
print(f"[warm_shader_cache] Done in {elapsed_total:.1f}s")


if __name__ == "__main__":
main()
Loading