Skip to content

Integrating AgiBot X2 Ultra web UI keyboard teleop#2082

Open
Mly9 wants to merge 3 commits into
dimensionalOS:mainfrom
Mly9:mamadou/agibot-trial
Open

Integrating AgiBot X2 Ultra web UI keyboard teleop#2082
Mly9 wants to merge 3 commits into
dimensionalOS:mainfrom
Mly9:mamadou/agibot-trial

Conversation

@Mly9
Copy link
Copy Markdown

@Mly9 Mly9 commented May 14, 2026

Problem

AgiBot X2 Ultra humanoid robot had no DiMOS integration — no way to connect to it or
drive it via the web UI.

Closes DIM-XXX

Solution

Adds a ROS2 connection module (X2Connection) and a agibot-x2-basic blueprint for the
AgiBot X2 Ultra. The blueprint wires the web UI joystick's tele_cmd_vel output to the
robot's cmd_vel input via a remapping, using the existing WebsocketVisModule teleop path.
Input source registration with the robot's motion controller is handled automatically on
start.

How to Test

run: dimos run agibot-x2-basic
Then open the DiMOS web UI and use the joystick to drive the robot.

Contributor License Agreement

  • I have read and approved the CLA.

@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps Bot commented May 14, 2026

Greptile Summary

This PR adds DiMOS integration for the AgiBot X2 Ultra humanoid robot: a ROS2 connection module (X2Connection), a primitive vis-only blueprint, and an agibot-x2-basic blueprint that wires the web UI joystick's tele_cmd_vel output to the robot's cmd_vel input.

  • X2Connection subscribes to the X2 Ultra's ROS2 topics (RGB/depth camera, point cloud, LiDAR, IMU) and publishes velocity commands via McLocomotionVelocity; input source registration with the motion controller runs in a background thread on start.
  • The service endpoint name constant _SVC_INPUT_SOURCE contains _5F (URL-encoded _), which likely prevents the input source from ever registering — the robot would accept no velocity commands.
  • depth_camera_info: Out[CameraInfo] is declared as a module output but no ROS2 subscription feeds it, so it will silently emit nothing for any consumer that wires to it.

Confidence Score: 3/5

The blueprint and vis-module wiring are clean, but connection.py has two defects that affect core runtime behaviour.

The _SVC_INPUT_SOURCE constant contains what looks like a URL-percent-encoded underscore (_5F), mismatching the aimdk_msgs Python import. If wrong, the motion-controller registration loop will exhaust all 8 retries on every launch and the robot will not move. Separately, depth_camera_info: Out[CameraInfo] is wired up as a module output but no subscription populates it, so any consumer expecting depth camera intrinsics will silently receive nothing. Both issues are on the primary runtime path for this new integration.

dimos/robot/agibot/x2_ultra/connection.py — the service endpoint name and the unpopulated depth_camera_info output both need attention before this is usable on real hardware.

Important Files Changed

Filename Overview
dimos/robot/agibot/x2_ultra/connection.py Core ROS2 connection module with two logic bugs: a URL-encoded underscore in the input-source service name that will cause registration to silently time out, and a declared depth_camera_info output that is never populated. Also calls rclpy.shutdown() globally on stop.
dimos/robot/agibot/x2_ultra/blueprints/basic/agibot_x2_basic.py Clean blueprint wiring that routes the WebsocketVisModule joystick tele_cmd_vel to the X2Connection's cmd_vel input; follows existing blueprint composition patterns correctly.
dimos/robot/agibot/x2_ultra/blueprints/primitive/agibot_x2_primitive.py Adds a minimal vis-only blueprint with a rerun split layout (2D camera + 3D world); straightforward and follows existing vis_module patterns.
dimos/robot/all_blueprints.py Adds agibot-x2-basic blueprint and x2-connection module entries to the registry; ordering is correct and consistent with the rest of the file.
dimos/robot/agibot/x2_ultra/x2_ultra.urdf Adds a 1049-line URDF for the X2 Ultra; mesh references are relative (./meshes/), so they must exist alongside the URDF at runtime — no mesh assets are included in this PR.

Sequence Diagram

sequenceDiagram
    participant WebUI as Web UI Joystick
    participant WSVis as WebsocketVisModule
    participant X2C as X2Connection
    participant ROS2 as ROS2 (aimdk)

    WebUI->>WSVis: joystick input
    WSVis->>X2C: tele_cmd_vel to cmd_vel (remapped)
    X2C->>X2C: move(twist)
    X2C->>ROS2: publish McLocomotionVelocity

    Note over X2C,ROS2: On start()
    X2C->>ROS2: create_client(SetMcInputSource, _SVC_INPUT_SOURCE)
    ROS2-->>X2C: service_is_ready? (may time out if name is wrong)
    X2C->>ROS2: call_async(SetMcInputSource.Request)

    ROS2->>X2C: rgb_image topic
    X2C->>X2C: _on_rgb_image publishes color_image
    ROS2->>X2C: depth_image topic
    X2C->>X2C: _on_depth_image publishes depth_image
    ROS2->>X2C: lidar topic
    X2C->>X2C: _on_lidar publishes lidar
    ROS2->>X2C: imu topic
    X2C->>X2C: _on_imu publishes imu
Loading

Reviews (1): Last reviewed commit: "[autofix.ci] apply automated fixes" | Re-trigger Greptile

color_image: Out[Image]
camera_info: Out[CameraInfo]
depth_image: Out[Image]
depth_camera_info: Out[CameraInfo]
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 depth_camera_info output declared but never populated

depth_camera_info: Out[CameraInfo] is declared as a module output, but _start_ros never subscribes to a depth camera info topic and no callback publishes to it. Downstream consumers of this output (or any blueprint that wires to it) will receive nothing. The pattern in realsense/camera.py and zed/camera.py shows depth_camera_info is expected to carry real data — either subscribe to the depth camera info topic and add an _on_depth_camera_info callback, or remove the field declaration if it is intentionally unsupported on the X2 Ultra.

_TOPIC_LIDAR = "/aima/hal/sensor/lidar_chest_front/lidar_pointcloud"
_TOPIC_IMU = "/aima/hal/imu/chest/state"
_TOPIC_VELOCITY = "/aima/mc/locomotion/velocity"
_SVC_INPUT_SOURCE = "/aimdk_5Fmsgs/srv/SetMcInputSource"
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Suspicious URL-encoded underscore in service name constant

_SVC_INPUT_SOURCE contains _5F which is the URL-percent-encoding of an underscore (_). All other topic constants in this file use a plain underscore, and the Python module import is aimdk_msgs.srv (underscore). If the correct ROS2 service endpoint is /aimdk_msgs/srv/SetMcInputSource, the create_client call will never match the running service, service_is_ready() will always be False, and the 15-second timeout will fire on every start — leaving the robot unable to receive velocity commands. Please verify the exact service name against the AgiBot SDK and correct if needed.

Suggested change
_SVC_INPUT_SOURCE = "/aimdk_5Fmsgs/srv/SetMcInputSource"
_SVC_INPUT_SOURCE = "/aimdk_msgs/srv/SetMcInputSource"

Comment on lines +365 to +374
def _stop_ros(self) -> None:
if self._ros_node is not None:
try:
import rclpy

self._ros_node.destroy_node()
if rclpy.ok():
rclpy.shutdown()
except Exception as e:
logger.warning("X2Connection: error during ROS shutdown: %s", e)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 rclpy.shutdown() is a process-wide operation

rclpy.shutdown() terminates the global rclpy context — it affects every rclpy node in the process, not just this module's node. If any other module (current or future) creates a rclpy node in the same process, calling stop() on X2Connection would silently tear down those nodes too. Consider tracking a reference-count or using a shared context manager, and only call rclpy.shutdown() as a last resort in a process-exit hook rather than in per-module teardown.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant