Skip to content

Add objective integration tests for all example_ws configs#534

Draft
rlpratt12 wants to merge 1 commit intomainfrom
claude/add-objective-integration-tests
Draft

Add objective integration tests for all example_ws configs#534
rlpratt12 wants to merge 1 commit intomainfrom
claude/add-objective-integration-tests

Conversation

@rlpratt12
Copy link

Summary

  • Add objectives_integration_test.py to 11 config packages (kitchen_sim, april_tag_sim, grinding_sim, dual_arm_sim, factory_sim, hangar_sim, kinova_sim, space_satellite_sim, space_satellite_sim_camera_cal, mock_sim, multi_arm_sim)
  • Each test auto-discovers objectives via SystemConfigParser and runs them with per-config skip lists for user-input/ML/GPU objectives
  • New objectives are automatically tested — if one requires skipping, CI fails and the PR author adds it to the skip list (fail-safe design)
  • Add CI matrix job to run each config's tests in parallel via the existing workspace_integration_test reusable workflow
  • Add missing ament_cmake_pytest + moveit_pro_test_utils test deps across all configs (including existing lab_sim)

Known TODOs

  • Skip lists are best-guess based on objective names/XML analysis — need to validate with actual CI runs
  • The existing integration-test-in-studio-container job still runs all tests including these, causing double-runs. Consider adding --packages-skip for config packages to that job
  • 600s timeout may need tuning for configs with many objectives (e.g. lab_sim has 80)
  • Cancel objectives sets are empty for new configs — looping objectives may need to be identified and added

Test plan

  • All 12 test files parse correctly with valid Python syntax
  • Skip list objective IDs verified against actual main_tree_to_execute XML attributes — all match
  • CMakeLists.txt and package.xml changes follow existing lab_sim pattern
  • Run integration tests for mock_sim (simplest, no skips) to validate end-to-end
  • Run integration tests for a config with skips (e.g. kitchen_sim) to validate skip behavior

🤖 Generated with Claude Code

Replicate the lab_sim objectives_integration_test.py pattern across all 11
remaining sim config packages: kitchen_sim, april_tag_sim, grinding_sim,
dual_arm_sim, factory_sim, hangar_sim, kinova_sim, space_satellite_sim,
space_satellite_sim_camera_cal, mock_sim, and multi_arm_sim.

Each test auto-discovers objectives via SystemConfigParser and runs them,
with per-config skip lists for objectives requiring user input or GPU/ML
inference. New objectives are automatically tested — if one requires
skipping, CI fails and the PR author adds it to the skip list.

Also adds a CI matrix job to run each config's tests in parallel, and
adds missing ament_cmake_pytest + moveit_pro_test_utils test deps
(including for the existing lab_sim test).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@rlpratt12 rlpratt12 requested review from JWhitleyWork and shaur-k and removed request for JWhitleyWork and shaur-k March 19, 2026 01:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant