raico_ZED2i

Repo for running ZED2i camera on Jetson AGX Orin

1. Download SDK for Jetson from here: https://www.stereolabs.com/en-gb/developers/release

ZED SDK for JetPack 6.1 and 6.2 (L4T 36.4) 5.1 (Jetson Orin, CUDA 12.6)

Requires email login, so files were downlaoded and then moved from Windows PC to Jetson with WinSCP.

2. Execution permission and run:

chmod +x ZED_SDK_Tegra_L4T36.4_v5.1.2.zstd.run
./ZED_SDK_Tegra_L4T36.4_v5.1.2.zstd.run

Please note that installing the 'libv4l-dev' apt package at any point on jetson will break the hardware encoding/decoding support

3. For ROS2 Installation follow this: https://www.stereolabs.com/docs/ros2

mkdir -p ~/ros2_ws_zed/src/
cd ~/ros2_ws_zed/src/ 
git clone https://github.com/stereolabs/zed-ros2-wrapper.git
cd ..
sudo apt update
rosdep install --from-paths src --ignore-src -r -y
colcon build --symlink-install --cmake-args=-DCMAKE_BUILD_TYPE=Release
source ros2_ws_zed/install/setup.bash

4. Testing

ros2 launch zed_wrapper zed_camera.launch.py camera_model:=zed2i
ros2 launch zed_wrapper zed_camera.launch.py camera_model:=zed2i publish_tf:=true publish_map_tf:=true pos_tracking.enable:=true mapping.enable:=true

For running octomap on WSL WSL: Enable mirrored

• create .wslconfig file in C:\Users\raico and write:

[wsl2]
networkingMode=mirrored
dnsTunneling=true

This makes WSL not use virtual ip but see 192.168.1.190 from the PC

ros2 run octomap_server octomap_server_node \
  --ros-args \
  -p frame_id:=odom \
  -r cloud_in:=/zed/zed_node/point_cloud/cloud_registered \
  -p resolution:=0.03 \
  -p publish_free_space:=true \
  -p height_map:=true \

or rtabmap from wsl (visual domoetry needs to be disabled)

ros2 launch rtabmap_launch rtabmap.launch.py \
  rgb_topic:=/zed/zed_node/rgb/color/rect/image \
  depth_topic:=/zed/zed_node/depth/depth_registered \
  camera_info_topic:=/zed/zed_node/rgb/color/rect/camera_info \
  odom_topic:=/zed/zed_node/odom \
  frame_id:=zed_camera_link \
  odom_frame_id:=odom \
  visual_odometry:=false

For running virtual kinova arm with moveit First git clone https://github.com/Kinovarobotics/ros2_kortex git checkout humble then build the thing, and William deletes some isaac stuff so it magically works

There are some changes that need to be made to ros2 kortex repo

Simulated robot needs to be moved to the same frame as camera

ros2 launch kinova_gen3_7dof_robotiq_2f_85_moveit_config robot.launch.py   robot_ip:=yyy.yyy.yyy.yyy   use_fake_hardware:=true
ros2 run tf2_ros static_transform_publisher \
0 0 0 0 0 0 \
world map

Inside /ros2_kortex/kortex_moveit_config/kinova_gen3_7dof_robotiq_2f_85_moveit_config/config a sensors_3d.yaml needs to be created. Online versions do not work witih humble and there are specifing formatting issues when those are used

sensors:
  - point_cloud_camera

point_cloud_camera:
    sensor_plugin: occupancy_map_monitor/PointCloudOctomapUpdater
    point_cloud_topic: /zed/zed_node/point_cloud/cloud_registered
    max_range: 5.0
    point_subsample: 1
    padding_offset: 0.1
    padding_scale: 1.0
    max_update_rate: 1.0
    filtered_cloud_topic: filtered_cloud

afterwards, in robot.launch.py add this:

from ament_index_python.packages import get_package_share_directory
import yaml
import os

def load_yaml(package_name, file_path):
    package_path = get_package_share_directory(package_name)
    full_path = os.path.join(package_path, file_path)
    with open(full_path, 'r') as f:
        return yaml.safe_load(f)

and in def launch_setup() add this:

 octomap_config = {
        'octomap_frame': 'world',  # if mobile robot, should be a fixed frame in the world
        'octomap_resolution': 0.05,
        'max_range': 5.0
    }

    octomap_updater_config = load_yaml('kinova_gen3_7dof_robotiq_2f_85_moveit_config', 'config/sensors_3d.yaml')


    move_group_node = Node(
        package="moveit_ros_move_group",
        executable="move_group",
        output="screen",
        parameters=[
            moveit_config.to_dict(),
            octomap_config,
            octomap_updater_config,
        ],
    )

For some reason both octamap_updated_config and octamap_config need to be passed to move_group_node