From b3e595efc7bd951fe8673ba2529312a1d163a1a2 Mon Sep 17 00:00:00 2001 From: Qian Lin Date: Thu, 16 Apr 2026 01:48:49 +0800 Subject: [PATCH 1/3] Update doc to make env setting step clear --- .../locomanipulation/step_2_teleoperation.rst | 7 +++++++ .../step_2_teleoperation.rst | 7 ++++++- .../static_manipulation/step_2_teleoperation.rst | 7 ++++++- 3 files changed, 19 insertions(+), 2 deletions(-) diff --git a/docs/pages/example_workflows/locomanipulation/step_2_teleoperation.rst b/docs/pages/example_workflows/locomanipulation/step_2_teleoperation.rst index b23625315..9c20c0666 100644 --- a/docs/pages/example_workflows/locomanipulation/step_2_teleoperation.rst +++ b/docs/pages/example_workflows/locomanipulation/step_2_teleoperation.rst @@ -31,9 +31,16 @@ In another terminal, start the Arena Docker container and launch the teleop sess :docker_run_default: +Run the following command to activate IsaacTeleop CloudXR environment settings: + .. code-block:: bash source ~/.cloudxr/run/cloudxr.env + +Run the teleop script: + +.. code-block:: bash + python isaaclab_arena/scripts/imitation_learning/teleop.py \ --viz kit \ --device cpu \ diff --git a/docs/pages/example_workflows/sequential_static_manipulation/step_2_teleoperation.rst b/docs/pages/example_workflows/sequential_static_manipulation/step_2_teleoperation.rst index 4f3a22e52..54e25d627 100644 --- a/docs/pages/example_workflows/sequential_static_manipulation/step_2_teleoperation.rst +++ b/docs/pages/example_workflows/sequential_static_manipulation/step_2_teleoperation.rst @@ -83,11 +83,16 @@ In another terminal, start the Arena Docker container: :docker_run_default: -Run the recording script: +Run the following command to activate IsaacTeleop CloudXR environment settings: .. code-block:: bash source ~/.cloudxr/run/cloudxr.env + +Run the recording script: + +.. code-block:: bash + python isaaclab_arena/scripts/imitation_learning/record_demos.py \ --device cpu \ --viz kit \ diff --git a/docs/pages/example_workflows/static_manipulation/step_2_teleoperation.rst b/docs/pages/example_workflows/static_manipulation/step_2_teleoperation.rst index 7662c28bf..a28472b60 100644 --- a/docs/pages/example_workflows/static_manipulation/step_2_teleoperation.rst +++ b/docs/pages/example_workflows/static_manipulation/step_2_teleoperation.rst @@ -82,11 +82,16 @@ In another terminal, start the Arena Docker container: :docker_run_default: -Run the recording script: +Run the following command to activate IsaacTeleop CloudXR environment settings: .. code-block:: bash source ~/.cloudxr/run/cloudxr.env + +Run the recording script: + +.. code-block:: bash + python isaaclab_arena/scripts/imitation_learning/record_demos.py \ --device cpu \ --viz kit \ From 35b50341b134846fbb2a013a5b3fc813a0813446 Mon Sep 17 00:00:00 2001 From: Qian Lin Date: Thu, 16 Apr 2026 11:15:36 +0800 Subject: [PATCH 2/3] Add bullet points and note to emphasize the env sourcing order --- .../locomanipulation/step_2_teleoperation.rst | 81 +++++++++++-------- .../step_2_teleoperation.rst | 38 +++++---- .../step_2_teleoperation.rst | 34 ++++---- 3 files changed, 88 insertions(+), 65 deletions(-) diff --git a/docs/pages/example_workflows/locomanipulation/step_2_teleoperation.rst b/docs/pages/example_workflows/locomanipulation/step_2_teleoperation.rst index 9c20c0666..07bae8766 100644 --- a/docs/pages/example_workflows/locomanipulation/step_2_teleoperation.rst +++ b/docs/pages/example_workflows/locomanipulation/step_2_teleoperation.rst @@ -6,55 +6,58 @@ This workflow covers collecting demonstrations for the G1 loco-manipulation task Step 1: Start the CloudXR Runtime ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -On the host machine, configure the firewall to allow CloudXR traffic. The required ports depend on the client type. +#. On the host machine, configure the firewall to allow CloudXR traffic. The required ports depend on the client type. -.. code-block:: bash - - sudo ufw allow 49100/tcp # Signaling - sudo ufw allow 47998/udp # Media stream - sudo ufw allow 48322/tcp # Proxy (HTTPS mode only) + .. code-block:: bash + sudo ufw allow 49100/tcp # Signaling + sudo ufw allow 47998/udp # Media stream + sudo ufw allow 48322/tcp # Proxy (HTTPS mode only) -Start the CloudXR runtime from the Arena Docker container: +#. Start the CloudXR runtime from the Arena Docker container: -:docker_run_default: + :docker_run_default: -.. code-block:: bash + .. code-block:: bash - python -m isaacteleop.cloudxr + python -m isaacteleop.cloudxr Step 2: Start Arena Teleop ^^^^^^^^^^^^^^^^^^^^^^^^^^ -In another terminal, start the Arena Docker container and launch the teleop session to verify the pipeline: +#. In another terminal, start the Arena Docker container: -:docker_run_default: + :docker_run_default: -Run the following command to activate IsaacTeleop CloudXR environment settings: +#. Run the following command to activate IsaacTeleop CloudXR environment settings: -.. code-block:: bash + .. code-block:: bash - source ~/.cloudxr/run/cloudxr.env + source ~/.cloudxr/run/cloudxr.env -Run the teleop script: + .. important:: + **Order matters.** In the terminal where you will run Arena, ``source ~/.cloudxr/run/cloudxr.env`` *after* the CloudXR runtime from Step 1 is already running, + and *before* you start the Arena app. The Arena app must inherit the IsaacTeleop CloudXR environment variables. -.. code-block:: bash +#. Run the teleop script: - python isaaclab_arena/scripts/imitation_learning/teleop.py \ - --viz kit \ - --device cpu \ - galileo_g1_locomanip_pick_and_place \ - --teleop_device openxr + .. code-block:: bash -Start the session from the **XR** tab in the application window. + python isaaclab_arena/scripts/imitation_learning/teleop.py \ + --viz kit \ + --device cpu \ + galileo_g1_locomanip_pick_and_place \ + --teleop_device openxr -.. figure:: ../../../images/locomanip_arena_server.png - :width: 100% - :alt: Arena teleop with XR running (stereoscopic view and OpenXR settings) - :align: center +#. In the running application, start the session from the **XR** tab in the application window. + + .. figure:: ../../../images/locomanip_arena_server.png + :width: 100% + :alt: Arena teleop with XR running (stereoscopic view and OpenXR settings) + :align: center - Arena teleop session with XR running. Stereoscopic view (left) and OpenXR settings in the XR tab (right). + Arena teleop session with XR running. Stereoscopic view (left) and OpenXR settings in the XR tab (right). Step 3: Connect from Meta Quest 3 @@ -66,7 +69,6 @@ A strong wireless connection is essential for a high-quality streaming experienc #. Open the browser on your headset and navigate to ``_. - #. Enter the IP address of your Isaac Lab host machine in the **Server IP** field. #. Click the **Click https://:48322/ to accept cert** link that appears on the page. @@ -77,9 +79,9 @@ A strong wireless connection is essential for a high-quality streaming experienc #. **Teleoperation Controls**: -* **Left joystick**: Move the body forward/backward/left/right. -* **Right joystick**: Squat (down), rotate torso (left/right). -* **Controllers**: Move end-effector (EE) targets for the arms. + * **Left joystick**: Move the body forward/backward/left/right. + * **Right joystick**: Squat (down), rotate torso (left/right). + * **Controllers**: Move end-effector (EE) targets for the arms. .. note:: @@ -93,11 +95,20 @@ A strong wireless connection is essential for a high-quality streaming experienc Reducing render resolution from 1 (default) to 0.2. +Once you have verified the teleoperation pipeline, exit VR from the Quest 3 headset, and stop the Arena teleop app. Step 4: Record with Quest 3 ^^^^^^^^^^^^^^^^^^^^^^^^^^^ -#. **Recording**: When ready to collect data, run the recording script from the Arena container: +.. note:: + + Run the following command to activate IsaacTeleop CloudXR environment settings again if you are starting the recording app from a different terminal. + + .. code-block:: bash + + source ~/.cloudxr/run/cloudxr.env + +#. Run the recording script from the Arena container: .. code-block:: bash @@ -114,6 +125,10 @@ Step 4: Record with Quest 3 galileo_g1_locomanip_pick_and_place \ --teleop_device openxr +#. In the running application, start the session from the XR tab in the application window. + +#. Follow Step 3 to connect the Quest 3 headset again. + #. Complete the task for each demo. Reset between demos. The script saves successful runs to the HDF5 file above. .. hint:: diff --git a/docs/pages/example_workflows/sequential_static_manipulation/step_2_teleoperation.rst b/docs/pages/example_workflows/sequential_static_manipulation/step_2_teleoperation.rst index 54e25d627..4811c85fa 100644 --- a/docs/pages/example_workflows/sequential_static_manipulation/step_2_teleoperation.rst +++ b/docs/pages/example_workflows/sequential_static_manipulation/step_2_teleoperation.rst @@ -79,30 +79,34 @@ Step 1: Start the CloudXR Runtime Step 2: Start Recording ^^^^^^^^^^^^^^^^^^^^^^^ -In another terminal, start the Arena Docker container: +#. In another terminal, start the Arena Docker container: -:docker_run_default: + :docker_run_default: -Run the following command to activate IsaacTeleop CloudXR environment settings: +#. Run the following command to activate IsaacTeleop CloudXR environment settings: -.. code-block:: bash + .. code-block:: bash - source ~/.cloudxr/run/cloudxr.env + source ~/.cloudxr/run/cloudxr.env -Run the recording script: + .. important:: + **Order matters.** In the terminal where you will run Arena, ``source ~/.cloudxr/run/cloudxr.env`` *after* the CloudXR runtime from Step 1 is already running, + and *before* you start the Arena app. The Arena app must inherit the IsaacTeleop CloudXR environment variables. -.. code-block:: bash +#. Run the recording script: - python isaaclab_arena/scripts/imitation_learning/record_demos.py \ - --device cpu \ - --viz kit \ - --dataset_file $DATASET_DIR/ranch_bottle_into_fridge_recorded.hdf5 \ - --num_demos 10 \ - --num_success_steps 10 \ - put_item_in_fridge_and_close_door \ - --object ranch_dressing_hope_robolab \ - --embodiment gr1_pink \ - --teleop_device openxr + .. code-block:: bash + + python isaaclab_arena/scripts/imitation_learning/record_demos.py \ + --device cpu \ + --viz kit \ + --dataset_file $DATASET_DIR/ranch_bottle_into_fridge_recorded.hdf5 \ + --num_demos 10 \ + --num_success_steps 10 \ + put_item_in_fridge_and_close_door \ + --object ranch_dressing_hope_robolab \ + --embodiment gr1_pink \ + --teleop_device openxr Step 3: Connect XR Device and Record diff --git a/docs/pages/example_workflows/static_manipulation/step_2_teleoperation.rst b/docs/pages/example_workflows/static_manipulation/step_2_teleoperation.rst index a28472b60..29d67c486 100644 --- a/docs/pages/example_workflows/static_manipulation/step_2_teleoperation.rst +++ b/docs/pages/example_workflows/static_manipulation/step_2_teleoperation.rst @@ -78,28 +78,32 @@ Step 1: Start the CloudXR Runtime Step 2: Start Recording ^^^^^^^^^^^^^^^^^^^^^^^ -In another terminal, start the Arena Docker container: +#. In another terminal, start the Arena Docker container: -:docker_run_default: + :docker_run_default: -Run the following command to activate IsaacTeleop CloudXR environment settings: +#. Run the following command to activate IsaacTeleop CloudXR environment settings: -.. code-block:: bash + .. code-block:: bash - source ~/.cloudxr/run/cloudxr.env + source ~/.cloudxr/run/cloudxr.env -Run the recording script: + .. important:: + **Order matters.** In the terminal where you will run Arena, ``source ~/.cloudxr/run/cloudxr.env`` *after* the CloudXR runtime from Step 1 is already running, + and *before* you start the Arena app. The Arena app must inherit the IsaacTeleop CloudXR environment variables. -.. code-block:: bash +#. Run the recording script: - python isaaclab_arena/scripts/imitation_learning/record_demos.py \ - --device cpu \ - --viz kit \ - --dataset_file $DATASET_DIR/arena_gr1_manipulation_dataset_recorded.hdf5 \ - --num_demos 10 \ - --num_success_steps 2 \ - gr1_open_microwave \ - --teleop_device openxr + .. code-block:: bash + + python isaaclab_arena/scripts/imitation_learning/record_demos.py \ + --device cpu \ + --viz kit \ + --dataset_file $DATASET_DIR/arena_gr1_manipulation_dataset_recorded.hdf5 \ + --num_demos 10 \ + --num_success_steps 2 \ + gr1_open_microwave \ + --teleop_device openxr Step 3: Connect XR Device and Record From 67e78e5e27b47116ba417a6a722aec020a19b066 Mon Sep 17 00:00:00 2001 From: Qian Lin Date: Thu, 16 Apr 2026 11:22:03 +0800 Subject: [PATCH 3/3] Add step bullets for CloudXR runtime session to improve clarify --- .../step_2_teleoperation.rst | 76 +++++++++---------- .../step_2_teleoperation.rst | 75 +++++++++--------- 2 files changed, 72 insertions(+), 79 deletions(-) diff --git a/docs/pages/example_workflows/sequential_static_manipulation/step_2_teleoperation.rst b/docs/pages/example_workflows/sequential_static_manipulation/step_2_teleoperation.rst index 4811c85fa..3f7771806 100644 --- a/docs/pages/example_workflows/sequential_static_manipulation/step_2_teleoperation.rst +++ b/docs/pages/example_workflows/sequential_static_manipulation/step_2_teleoperation.rst @@ -6,75 +6,71 @@ This workflow covers collecting demonstrations using Isaac Teleop with an XR dev Step 1: Start the CloudXR Runtime ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - .. tab-set:: .. tab-item:: Meta Quest 3 / Pico 4 Ultra :selected: - On the host machine, configure the firewall to allow CloudXR traffic. - - .. code-block:: bash + #. On the host machine, configure the firewall to allow CloudXR traffic. - sudo ufw allow 49100/tcp # Signaling - sudo ufw allow 47998/udp # Media stream - sudo ufw allow 48322/tcp # Proxy (HTTPS mode only) + .. code-block:: bash + sudo ufw allow 49100/tcp # Signaling + sudo ufw allow 47998/udp # Media stream + sudo ufw allow 48322/tcp # Proxy (HTTPS mode only) - Start the CloudXR runtime from the Arena Docker container: + #. Start the CloudXR runtime from the Arena Docker container: - :docker_run_default: + :docker_run_default: - Create a CloudXR config to enable hand tracking: + #. Create a CloudXR config to enable hand tracking: - .. code-block:: bash + .. code-block:: bash - echo "NV_CXR_ENABLE_PUSH_DEVICES=0" > handtracking.env + echo "NV_CXR_ENABLE_PUSH_DEVICES=0" > handtracking.env + #. Start the CloudXR runtime with the customized config file: - Start the CloudXR runtime with the customized config file: + .. code-block:: bash - .. code-block:: bash - - python -m isaacteleop.cloudxr --cloudxr-env-config=handtracking.env + python -m isaacteleop.cloudxr --cloudxr-env-config=handtracking.env .. tab-item:: Apple Vision Pro - On the host machine, configure the firewall to allow CloudXR traffic. - - .. code-block:: bash + #. On the host machine, configure the firewall to allow CloudXR traffic. - # Signaling (use one based on connection mode) - sudo ufw allow 48010/tcp # Standard mode - sudo ufw allow 48322/tcp # Secure mode - # Video - sudo ufw allow 47998/udp - sudo ufw allow 48005/udp - sudo ufw allow 48008/udp - sudo ufw allow 48012/udp - # Input - sudo ufw allow 47999/udp - # Audio - sudo ufw allow 48000/udp - sudo ufw allow 48002/udp + .. code-block:: bash - Start the CloudXR runtime from the Arena Docker container: + # Signaling (use one based on connection mode) + sudo ufw allow 48010/tcp # Standard mode + sudo ufw allow 48322/tcp # Secure mode + # Video + sudo ufw allow 47998/udp + sudo ufw allow 48005/udp + sudo ufw allow 48008/udp + sudo ufw allow 48012/udp + # Input + sudo ufw allow 47999/udp + # Audio + sudo ufw allow 48000/udp + sudo ufw allow 48002/udp - :docker_run_default: + #. Start the CloudXR runtime from the Arena Docker container: - Create a customized config file with the following content: + :docker_run_default: - .. code-block:: bash + #. Create a customized config file with the following content: - printf '%s\n' 'NV_DEVICE_PROFILE=auto-native' 'NV_CXR_ENABLE_PUSH_DEVICES=0' > avp.env + .. code-block:: bash + printf '%s\n' 'NV_DEVICE_PROFILE=auto-native' 'NV_CXR_ENABLE_PUSH_DEVICES=0' > avp.env - Start the CloudXR runtime with the customized config file: + #. Start the CloudXR runtime with the customized config file: - .. code-block:: bash + .. code-block:: bash - python -m isaacteleop.cloudxr --cloudxr-env-config=avp.env + python -m isaacteleop.cloudxr --cloudxr-env-config=avp.env Step 2: Start Recording ^^^^^^^^^^^^^^^^^^^^^^^ diff --git a/docs/pages/example_workflows/static_manipulation/step_2_teleoperation.rst b/docs/pages/example_workflows/static_manipulation/step_2_teleoperation.rst index 29d67c486..21b08902a 100644 --- a/docs/pages/example_workflows/static_manipulation/step_2_teleoperation.rst +++ b/docs/pages/example_workflows/static_manipulation/step_2_teleoperation.rst @@ -11,69 +11,66 @@ Step 1: Start the CloudXR Runtime .. tab-item:: Meta Quest 3 / Pico 4 Ultra :selected: - On the host machine, configure the firewall to allow CloudXR traffic. + #. On the host machine, configure the firewall to allow CloudXR traffic. - .. code-block:: bash + .. code-block:: bash - sudo ufw allow 49100/tcp # Signaling - sudo ufw allow 47998/udp # Media stream - sudo ufw allow 48322/tcp # Proxy (HTTPS mode only) + sudo ufw allow 49100/tcp # Signaling + sudo ufw allow 47998/udp # Media stream + sudo ufw allow 48322/tcp # Proxy (HTTPS mode only) + #. Start the CloudXR runtime from the Arena Docker container: - Start the CloudXR runtime from the Arena Docker container: + :docker_run_default: - :docker_run_default: + #. Create a CloudXR config to enable hand tracking: - Create a CloudXR config to enable hand tracking: + .. code-block:: bash - .. code-block:: bash + echo "NV_CXR_ENABLE_PUSH_DEVICES=0" > handtracking.env - echo "NV_CXR_ENABLE_PUSH_DEVICES=0" > handtracking.env + #. Start the CloudXR runtime with the customized config file: + .. code-block:: bash - Start the CloudXR runtime with the customized config file: - - .. code-block:: bash - - python -m isaacteleop.cloudxr --cloudxr-env-config=handtracking.env + python -m isaacteleop.cloudxr --cloudxr-env-config=handtracking.env .. tab-item:: Apple Vision Pro - On the host machine, configure the firewall to allow CloudXR traffic. - - .. code-block:: bash + #. On the host machine, configure the firewall to allow CloudXR traffic. - # Signaling (use one based on connection mode) - sudo ufw allow 48010/tcp # Standard mode - sudo ufw allow 48322/tcp # Secure mode - # Video - sudo ufw allow 47998/udp - sudo ufw allow 48005/udp - sudo ufw allow 48008/udp - sudo ufw allow 48012/udp - # Input - sudo ufw allow 47999/udp - # Audio - sudo ufw allow 48000/udp - sudo ufw allow 48002/udp + .. code-block:: bash - Start the CloudXR runtime from the Arena Docker container: + # Signaling (use one based on connection mode) + sudo ufw allow 48010/tcp # Standard mode + sudo ufw allow 48322/tcp # Secure mode + # Video + sudo ufw allow 47998/udp + sudo ufw allow 48005/udp + sudo ufw allow 48008/udp + sudo ufw allow 48012/udp + # Input + sudo ufw allow 47999/udp + # Audio + sudo ufw allow 48000/udp + sudo ufw allow 48002/udp - :docker_run_default: + #. Start the CloudXR runtime from the Arena Docker container: - Create a customized config file with the following content: + :docker_run_default: - .. code-block:: bash + #. Create a customized config file with the following content: - printf '%s\n' 'NV_DEVICE_PROFILE=auto-native' 'NV_CXR_ENABLE_PUSH_DEVICES=0' > avp.env + .. code-block:: bash + printf '%s\n' 'NV_DEVICE_PROFILE=auto-native' 'NV_CXR_ENABLE_PUSH_DEVICES=0' > avp.env - Start the CloudXR runtime with the customized config file: + #. Start the CloudXR runtime with the customized config file: - .. code-block:: bash + .. code-block:: bash - python -m isaacteleop.cloudxr --cloudxr-env-config=avp.env + python -m isaacteleop.cloudxr --cloudxr-env-config=avp.env Step 2: Start Recording ^^^^^^^^^^^^^^^^^^^^^^^