Skip to content

CLAUDE.md

Latest commit

 

History

History
263 lines (196 loc) · 8.05 KB

CLAUDE.md

File metadata and controls

263 lines (196 loc) · 8.05 KB

Robot SDK - CLAUDE.md

Project Overview

robot_sdk is a comprehensive robotics software development kit targeting multiple robot platforms (AgiBot and Unitree). Built on ROS 2, it provides hardware abstraction, motion control, and interaction capabilities.

Supported Platforms:

  • AgiBot X2 series robots (pm01, se01, sa01, etc.)
  • Unitree robots
  • EngineAI protocol-compatible devices

Directory Structure

robot_sdk/
├── agibot/
│   ├── aimdk_msgs/          # ROS 2 message/service definitions (156 interface files)
│   └── aimdk_examples/      # Example applications and demos
├── engineai/
│   ├── interface_protocol/  # Low-level hardware interface protocol definitions
│   └── interface_example/   # Hardware control reference implementations
└── unitree/
    ├── unitree_sdk2/         # Unitree SDK (git submodule)
    └── unitree_sdk2_ament/  # ROS 2 ament wrapper

Tech Stack

  • ROS 2 (Humble/Foxy): Robot middleware and communication backbone
    • rclcpp: C++ client library
    • ament_cmake: Build system
    • rosidl: Message/service interface definition language
  • Languages: C++17 (primary), Python (launch scripts)
  • Third-party Libraries:
    • Eigen3: Linear algebra and matrix operations
    • yaml-cpp: YAML configuration parsing
    • OpenCV: Computer vision
    • ruckig: Trajectory planning
    • libcurl: HTTP networking
    • ncurses: Terminal keyboard input
    • MNN (Mobile Neural Network): ML inference for RL policies

Core Modules

1. agibot/aimdk_msgs — Message and Service Definitions

ROS 2 message/service interface package covering 14 protocol domains:

Directory Description
common/ Generic request/response/state header structures
mc/ Motion control (actions, gaits, velocity, preset motions)
hal/ Hardware abstraction (hands, joints, DCU monitoring)
interaction/ Interaction (TTS, video, expressions)
hds/ System diagnostics
event/, sm/, teleop_bridge/ Specialized protocols

Key Message Types:

  • Motion: McAction, McLocomotionVelocity, McPresetMotion
  • Hardware: HandCommand, JointCommand, HandState
  • Interaction: PlayTts, PlayEmoji, PlayVideo

2. agibot/aimdk_examples — Example Code

12+ executables organized by functional domain:

robot/ — High-level robot abstraction

  • robot.h / robot.cpp: aimdk::Robot class encapsulating all ROS 2 details
  • main.cpp: Complete usage workflow example
  • Key methods: SetAction, Move, SetHandPositions, Speak, ShowExpression, PresetMotion

mc/ — Motion control examples (10 files)

  • Action setting, input source management, gait control, velocity control
  • Keyboard input: keyboard.cpp

hal/ — Hardware abstraction layer examples (11 files)

  • Camera feedback (head, rear, stereo, RGBD)
  • Sensors (IMU, LiDAR)
  • Joint/hand/motor control

interaction/ — Interaction examples (5 files)

  • Text-to-speech (TTS), expressions, media playback, microphone input

3. engineai/interface_protocol — Low-level Protocol Definitions

Message Types (12):

  • MotorCommand / MotorState / MotorDebug
  • JointCommand / JointState
  • BodyVelCmd, GamepadKeys, ImuInfo, MotionState
  • Heartbeat, Tts, ParallelParserType

Service (1):

  • EnableMotor.srv: Motor enable/disable control

Key Data Structures:

  • JointCommand: 8 arrays (position, velocity, feed-forward torque, torque, stiffness, damping) + header
  • JointState: 3 arrays (position, velocity, torque) + header

4. engineai/interface_example — Hardware Control Reference Implementations

3 Executables:

Program Description
hold_joint_status Subscribes to joint state, publishes hold commands at 500Hz
joint_test_example Single-joint motion test, loads parameters from YAML
rl_basic_example Reinforcement learning control loop using MNN policy inference

Support Components:

  • components/message_handler.hpp: Unified subscription/publication interface with latest-message caching
  • parameter/rl_basic_param.h: RL parameter YAML loader
  • math/mnn_model.h: Neural network inference wrapper
  • math/rotation_matrix.h: IMU quaternion → Euler angle conversion
  • math/concatenate_vector.h: Vector concatenation utilities

Build

Requirements

  • ROS 2 Humble or Foxy
  • CMake 3.8+
  • C++17-compatible compiler
  • Third-party library path: /opt/engineai_robotics_third_party/

Build Commands

# Build all packages
colcon build

# Build specific packages
colcon build --packages-select aimdk_msgs aimdk_examples

# Enable keyboard input support (requires ncurses)
ENABLE_KEYBOARD=1 colcon build --packages-select aimdk_examples

# Enable/disable message generation (interface_protocol)
colcon build --packages-select interface_protocol --cmake-args -DGEN_MESSAGE=ON

Build Flags

All packages use -Wall -Wextra -Wpedantic with C++17 standard.

Running Examples

Environment Setup

# Set robot model (default: pm01)
export PRODUCT=pm01

# Set third-party library path
export LD_LIBRARY_PATH=/opt/engineai_robotics_third_party/lib:$LD_LIBRARY_PATH

# Source ROS 2 environment
source /opt/ros/humble/setup.bash
source install/setup.bash

Launch Examples

# AgiBot action example (two nodes)
ros2 launch aimdk_examples example.launch.py

# RL control example
ros2 launch interface_example rl_basic_example.launch.py

# Run individual example executables
ros2 run aimdk_examples set_mc_action
ros2 run aimdk_examples hand_control
ros2 run aimdk_examples play_tts
ros2 run interface_example hold_joint_status

Configuration Files

engineai/interface_example/config/pm01/
├── motor.yaml                          # Motor calibration, sign conventions, offsets
├── joint_test.yaml                     # Joint test parameters
├── pd_joint_test.yaml                  # PD control parameters
└── rl_basic/basic/rl_basic_param.yaml  # RL controller hyperparameters

Key rl_basic_param.yaml Parameters:

  • Policy file path (MNN model)
  • Observation/action dimensions
  • Active joint indices
  • Control frequency (dt)
  • Gait parameters (cycle time, transition time)
  • Observation/action scaling factors
  • Command scaling factors
  • IMU calibration bias

Architectural Patterns

Service-Oriented Architecture

  • State-changing commands use ROS 2 services
  • Async service calls with timeout handling
  • Service availability verified before calls (WaitServiceReady pattern)

Publish/Subscribe Pattern

  • Continuous data streams: joint state, IMU, gamepad input (≥500Hz)
  • Locomotion velocity topic: /aima/mc/locomotion/velocity
  • Hand command topic: /aima/hal/hoint/hand/command

High-Level API Wrapper

  • aimdk::Robot class abstracts all ROS 2 complexity
  • Service/topic names configurable via Options struct
  • Lifecycle: OnInit() → OnStart() → ... → OnShutdown()

Message Handler Pattern

  • example::MessageHandler: centralized subscription management
  • Latest message caching (shared_ptr)
  • Single access point for all sensor data

Configuration-Driven Control

  • YAML parameter loading
  • Config directory resolved dynamically by robot model
  • Multiple active joint subsets supported
  • Sim-to-real parameter scaling

Eigen-Based Math

  • Kinematics/dynamics vector and matrix operations
  • Quaternion to Euler angle conversion
  • IMU sensor fusion
  • RL observation history stacking

Topic and Service Naming Conventions

Type Pattern Example
MC services /aima/mc/<feature> /aima/mc/set_action
HAL topics /aima/hal/<device>/<type> /aima/hal/hoint/hand/command
Hardware feedback /hardware/<sensor> /hardware/motor_debug
Locomotion velocity /aima/mc/locomotion/velocity —

Git Submodules

# Initialize and fetch the Unitree SDK2 submodule
git submodule update --init --recursive

unitree_sdk2 source: https://github.com/unitreerobotics/unitree_sdk2.git