Johnliu/pegasus integration (#295)

* Enabled Pegasus plugin on start-up.

* integrated dynamic extension enabling through ENV file

* PX4, MavLink, MAVROS connected

* fixed gitmodules after rebase

* Changed MAVLink port to be explicit for cross-service compatibility.

* simplified PX4 installation

* No longer launches MAVROS on sim side

* connects robot mavros nodes to sim px4 mavlink in px4-autopilot mode

* update local plan src to match new MAVROS ReliabilityPolicy

* made camera configs shared between robot and sim + created clock publisher in sim + added camera publishers in sim

* changed PegasusSim to fork for ROS_DOMAIN_ID management

* new config files README

* deleted old mavros_util file

* reenabled omni env pass

* fixed isaac-sim Dockerfile installation

* Change project version to latest commit with docker image change

* Fix issue where robot didn't build if wasn't built before

* Add comment to dockerfile

* Minor fixes from main branch

* Set workdir for isaac container to /isaac-sim

* Remaps image_raw images into image_rect images

* put in basic domain bridge file. Not integrated yet.

* domain bridge applied to 1 robot

* Removed image rectifier. Directly mapped sim raw to image_rect topics

* Added PegasusSim integration to use sim time

* hardcoded namespacing for tfs

* transferred PegasusSim ownership to castacks

* transferred PegasusSim ownership to castacks

* migrated to isaacsim 4.5.0 due to dependency issues

* Remove commented out clone of px4

* lidar with 1 robot

* domain bridge for 1 robot works

* updated PegasusSim lidar sensor config initialization

* Add option to launch standalone isaac sim gui

* Remove old code for scene loading

* Rename environment --> stage

* Only update image tag if docker-compose.yaml has a change to the build: key

* Enable change scene on command line; also enable python scene creation

* fixed depth subscription for disparity. Random walk semi-working.

* Add example scene with a fan

* Lower fans, easily see effect on drone takeoff

* Add standalone examples

* Add random primitive obstacles scene for Isaac Sim

- Created random_primitive_obstacles.py with customize_world function
- Generates 50 random primitive shapes (cubes, spheres, cylinders, cones, capsules)
- Includes mix of dynamic (physics-enabled) and visual-only objects
- Random positions, scales, orientations, and colors for variety
- Added DistantLight for better scene illumination
- Objects scattered throughout 20x20x8 meter scene bounds

Co-authored-by: openhands <openhands@all-hands.dev>

* Fixed default environment addition. Translates scene to full NVIDIA path

* Put stereo camera TFs in optical frame

* skeleton for MAVROS waiting for MAVLink to launch (not working)

* revert droan yaml

* reverted vdb_params yaml

* removed unecessary isaac_extensions folder

* Don't spawn isaac-sim-gui by default, only if explicitly specified

* Update clock to publish on world step to match isaac sim time. also add another test scene

* Silence macvo

* Move mavros startup to interface.launch.xml

* Imported Pegasus Ascent Extension

* Only update image tag if docker-compose.yaml has a change to the 'build:' key

* Make submodule init optional in airstack setup

* Renamed files to PegasusMultirotorNode

* Fixed PX4 Mavlink Backend launching

* Refactored spawning functionality into OgnPegasusMultirotorNode

* Individual Nodes for PX4 and ArduPilot backend-supported Multirotor Nodes

* dynamically sets ogn input macros

* switched default prim name to /World/Quadrotor

* Refactored ogn into PegasusSim Submodule

* Basic launch script to spawn a Drone prim with Ogn under it

* Updated Pegasus submodule

* Fix missing xacro

* Change to px4.launch instead of apm.launch

* Add tmux tpm plugin

* Ugh

* Setup dev container for isaac too

* Setup vscode debug breakpointing for isaac container

* Update user.config.json to autoload pegasus

* Remove python sensor instantiationg

* Add project wide settings.json

* Update with pegasus modules

* Change isaac sim dev container to open root

* Change to /World/stage

* Now working for drag and drop GUI!

* Remove not working bookmark

* Auto enable urdf plugins

* Make a folder for robot_descriptions for urdfs

* Progress, URDF working. Now need mavlink to work

* Comments

* Format code

* mavlink connection

* Try make things work

* Stuff takes off now and does fixed trajectory, yay

* Fix some launch issues

* Had to fix camera frame by 180deg rotate about X (following isaac docs)

* Frame updates

* Fix disabling autolaunch in vscode devcontainer

* Debugging droan local planner

* Temporarily delete CONFIG_MOUNT_PATH

* FF commit

* Formatting

* Delete

* Add gitignore

* Formatting

* Fix a timeout issue

* ADd cmake-tools

* Fix bug uninitialized variable if not ardupilot

* Rviz

* New nucleus server

* synced to initial omnigraph api implementation

* An example pegasus launch script

* Pegasus API updated

* provide two example scripts for scripted pegasus launching

* .env file now defines robot urdf description for launch

* cleaned up example one px4 vehicle script

* added .env flag to launch scripted or usd file for pegasus

* minor comment change for launch_scripts location

* updated docs for pegasus

* update user template to load needed isaac sim extensions for omni graph

* Reverted .env file to default Iris usd path

* deleted unneeded PX4 Module

* fixed isaacsim config bug, set docker image tag, switched to using new airlab-docker registry

---------

Co-authored-by: Andrew Jong <ajong@andrew.cmu.edu>
Co-authored-by: openhands <openhands@all-hands.dev>
Co-authored-by: Andrew Jong <andrewjong87@gmail.com>
Co-authored-by: John Keller <jkeller2@andrew.cmu.edu>

Author: 4 people

.devcontainer/gcs/devcontainer.json

@@ -17,5 +17,8 @@
         "ms-python.black-formatter"
       ]
     }
+  },
+  "containerEnv": {
+    "AUTOLAUNCH": "false"
   }
 }

.vscode/tasks.json .devcontainer/gcs/tasks.json.vscode/tasks.json renamed to .devcontainer/gcs/tasks.json

@@ -0,0 +1,26 @@
+// .devcontainer/devcontainer.json
+{
+    "name": "Isaac Sim Container",
+    "service": "isaac-sim",
+    "runServices": [
+        "isaac-sim"
+    ],
+    "dockerComposeFile": [
+        "../../docker-compose.yaml"
+    ],
+    "workspaceFolder": "/",
+    "customizations": {
+        "vscode": {
+            "extensions": [
+                "ms-python.python", // Install the Python extension
+                "ms-python.black-formatter"
+            ],
+            "settings": {
+                "python.defaultInterpreterPath": "/isaac-sim/python.sh" // Or your specific path
+            }
+        }
+    },
+    "containerEnv": {
+        "AUTOLAUNCH": "false"
+    }
+}

.devcontainer/isaac-sim/devcontainer.json

@@ -0,0 +1,20 @@
+{
+    "version": "2.0.0",
+    "configurations": [
+        {
+            "name": "Python: Current File",
+            "type": "debugpy",
+            "request": "launch",
+            "program": "${file}",
+            "console": "integratedTerminal",
+            "env": {
+                "RESOURCE_NAME": "IsaacSim"
+            },
+            "python": "/isaac-sim/kit/python/bin/python3",
+            "envFile": "/isaac-sim/.vscode/.standalone_examples.env",
+            "preLaunchTask": "setup_python_env"
+        },
+
+    ]
+}
+

.devcontainer/isaac-sim/launch.json

@@ -0,0 +1,14 @@
+{
+    // See https://go.microsoft.com/fwlink/?LinkId=733558
+    // for the documentation about the tasks.json format
+    "version": "2.0.0",
+    "tasks": [
+        {
+            "label": "setup_python_env",
+            "type": "shell",
+            "linux": {
+                "command": "export CARB_APP_PATH=/isaac-sim/kit && export ISAAC_PATH=/isaac-sim && export EXP_PATH=/isaac-sim/apps && source /isaac-sim/setup_python_env.sh && printenv >/isaac-sim/.vscode/.standalone_examples.env"
+            }
+        }
+    ]
+}

.devcontainer/isaac-sim/tasks.json

@@ -8,14 +8,17 @@
   ],
   "service": "robot",
   "workspaceFolder": "/home/robot/AirStack",
-  "initializeCommand": "export AUTOLAUNCH=false && echo 'You may close this window'",
   "customizations": {
     "vscode": {
       "extensions": [
         "Ranch-Hand-Robotics.rde-pack",
         "ms-vscode.cpptools",
-        "ms-python.black-formatter"
+        "ms-python.black-formatter",
+        "ms-vscode.cmake-tools"
       ],
     }
+  },
+  "containerEnv": {
+    "AUTOLAUNCH": "false"
   }
 }

.devcontainer/robot/devcontainer.json

@@ -0,0 +1,25 @@
+{
+    // `bws` is the same command across robot and gcs containers
+    "version": "2.0.0",
+    "tasks": [
+        {
+            "label": "Build ros_ws Debug",
+            "type": "shell",
+            "options": {
+                "cwd": "${workspaceFolder}"
+            },
+            "command": [
+                "source ${userHome}/.bashrc &&",
+                "bws --cmake-args '-DCMAKE_BUILD_TYPE=Debug'"
+            ],
+            "problemMatcher": [],
+            "group": {
+                "kind": "build",
+                "isDefault": true
+            }
+        },
+    ],
+    "configurations": [
+    
+    ]
+}

.devcontainer/robot/tasks.json

@@ -1,4 +1,5 @@
-# This is the main .env file for AirStack, which sets docker compose variables for variable interpolation.
+# This is the main .env file for AirStack, which sets DOCKER COMPOSE VARIABLES FOR VARIABLE INTERPOLATION IN docker-compose.yaml.
+# These variables do NOT get set within the robot container itself. For setting container environment variables, use env_file: attribute in docker-compose.yaml.
 # Standard Usage: airstack --env-file .env up
 
 # Warning: even though this file is organized into sections, all variables get propagated to all sub-level 
@@ -10,25 +11,32 @@ PROJECT_NAME="airstack"
 # If you've run ./airstack.sh setup, then this will auto-generate from the git commit hash every time a change is made 
 # to a Dockerfile or docker-compose.yaml file. Otherwise this can also be set explicitly to make a release version.
 # auto-generated from git commit hash
-DOCKER_IMAGE_TAG="0.14.4-alpha.2"
+DOCKER_IMAGE_TAG="0.15.0"
 # Can replace with your docker hub username
 PROJECT_DOCKER_REGISTRY="airlab-docker.andrew.cmu.edu/airstack"
 # ============================================
 
 # ================= Common ===================
-AUTOLAUNCH="true"  # If false, the docker-compose will automatically just spawn containers with no launch command.
+AUTOLAUNCH="true"  # If false, the docker-compose will just spawn idle docker containers with no launch command.
 NUM_ROBOTS="1"
+ROBOT_DEFINTIONS="/config/robot_definitions.yaml"  # Path to the robot definitions file, which is mounted at /config in containers.
 
-# ================ SIMULATION =================
-ISAAC_SIM_SCENE="omniverse://airlab-nucleus.andrew.cmu.edu/Projects/AirStack/AFCA/afca_v0.14.3.usd"
-PLAY_SIM_ON_START="true"
-# =============================================
+# ============== SIMULATION =====================
+ISAAC_SIM_GUI="omniverse://airlab-nucleus.andrew.cmu.edu/Library/Assets/Pegasus/iris_with_sensors.pegasus.robot.usd"
+# Set to "true" to launch Isaac Sim using a standalone Python script instead of USD file
+ISAAC_SIM_USE_STANDALONE_SCRIPT="false" # "true" or "false"
+# Script name (must be in /AirStack/simulation/isaac-sim/launch_scripts/)
+ISAAC_SIM_SCRIPT_NAME="example_one_px4_pegasus_launch_script.py"
+PLAY_SIM_ON_START="false" # Not supported in standalone script mode
+# ===============================================
 
 # ================= ROBOT =====================
 # See robot/docker/docker-compose.yaml for how these variables get propagated in 
-# the container's entry command.
+# the container's entry command. See robot's .bashrc for additional variables
 ROBOT_LAUNCH_PACKAGE="robot_bringup"
 ROBOT_LAUNCH_FILE="robot.launch.xml"
+ROBOT_DESCRIPTION_PACKAGE="iris_with_sensors_description"
+ROBOT_URDF_FILE="iris_with_sensors.pegasus.robot.urdf"
 
 # See robot-base-docker-compose.yaml for how these variables get propagated. 
 AUTONOMY_LAUNCH_PACKAGE="autonomy_bringup"
@@ -51,6 +59,10 @@ GLOBAL_LAUNCH_FILE="global.launch.xml"
 # --
 BEHAVIOR_LAUNCH_PACKAGE="behavior_bringup"
 BEHAVIOR_LAUNCH_FILE="behavior.launch.xml"
+
+# offboard API streaming out
+OFFBOARD_BASE_PORT=14540
+ONBOARD_BASE_PORT=14580
 # ===============================================
 
 # =========== GROUND CONTROL STATION ============

.env

@@ -10,3 +10,9 @@
 [submodule "common/ros_packages/rviz_behavior_tree_panel/xdot_cpp"]
 	path = common/ros_packages/rviz_behavior_tree_panel/xdot_cpp
 	url = https://github.com/castacks/xdot_cpp.git
+[submodule "simulation/isaac-sim/extensions/PegasusSimulator"]
+	path = simulation/isaac-sim/extensions/PegasusSimulator
+	url = https://github.com/castacks/PegasusSimulator-AirStack-Integration.git
+[submodule "robot/ros_ws/src/autonomy/0_interface/mavros"]
+	path = robot/ros_ws/src/autonomy/0_interface/mavros
+	url = https://github.com/castacks/mavros.git

.gitmodules

@@ -0,0 +1,2 @@
+# this gets mounted from each devcontainer
+tasks.json