Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 11 additions & 8 deletions docs/concepts/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,7 @@ Behavior per state:
|---|---|---|
| **ZERO_TORQUE** | `humanoid_control/ZeroTorqueController` | 0 to all 5 cmd interfaces. Startup default, fault fallback. |
| **DAMPING** | `humanoid_control/DampingController` | `K=0`, `D=damping value`, `q_cmd=q_captured` — soft under gravity, resists velocity. |
| **STANDBY** | `humanoid_control/StandbyController` | Linear pose interpolation through a YAML sequence; ramps `K_p / K_d` on first segment. Publishes `StandbyState` with `is_finished`. |
| **STANDBY** | `humanoid_control/StandbyController` | Linear pose interpolation through a YAML sequence; ramps `K_p / K_d` on first segment. Publishes `StandbyState` with `is_finished`. Two spawned instances of this plugin — `standby_controller_a` (Pose A, `L1+A`) and `standby_controller_b` (Pose B, `L1+B`) — provide two independently selectable poses; from either you can START either policy. |
| **LOCOMOTION** | `humanoid_control/RLPolicyController` | In-process ONNX inference (System 0): packs observations, replays the `.mcap` motion reference, decodes + writes commands — all in the RT `update()`. Runs every learned policy (tracking / piano / locomotion); they differ only by the loaded `.onnx` + `.mcap`. |
| **REMOTE** | `humanoid_control/RemotePolicyController` | System 1/2 external-command ingress: subscribes `~/command` (`MITCommand` over DDS) from a *non*-real-time source (gravity-comp today, VLA / manipulation later) with arrival-time stale-command gating. |

Expand All @@ -94,14 +94,15 @@ plain `rclcpp::Node` that subscribes:

- `/joy` (gamepad intents; on by default — bringup hard-fails if `/dev/input/js*`
is missing unless you opt out with `enable_gamepad:=false`)
- `/standby_controller/state` (the `is_finished` gate for the two `START_*` intents)
- `/standby_controller_a/state` and `/standby_controller_b/state` (one per standby pose; the `is_finished` gate for the two `START_*` intents)
- `/safety_status` (the auto-DAMP trigger)

…and exposes five `std_srvs/Trigger` services so transitions can also be
…and exposes six `std_srvs/Trigger` services so transitions can also be
driven from the command line:

- `/humanoid_control/mode/damp`, `/humanoid_control/mode/load`, `/humanoid_control/mode/start_remote`,
`/humanoid_control/mode/start_locomotion`, `/humanoid_control/mode/quit`
- `/humanoid_control/mode/damp`, `/humanoid_control/mode/load_a`, `/humanoid_control/mode/load_b`,
`/humanoid_control/mode/start_remote`, `/humanoid_control/mode/start_locomotion`,
`/humanoid_control/mode/quit`

`/control_mode` is published at 50 Hz. The manager polls
`list_controllers` periodically (every 25 ticks = 500 ms) so controllers
Expand Down Expand Up @@ -234,8 +235,10 @@ Concrete examples:
STRICT switch to DAMPING. If DAMPING fails (e.g. command interfaces
unavailable), `mode_manager` falls back to ZERO_TORQUE.
- A `RemotePolicyController` whose Python publisher stalls for >100 ms
(`stale_command_timeout_ms` default) writes **passive commands** (zero
stiffness/damping) by default, or zero-order-holds the last command if
(`stale_command_timeout_ms` default) falls into a **damped hold** by
default (`stale_command_policy: passive` → zero stiffness, high damping
like DAMPING mode, holding the live joint position — the arms stay
damped, not limp), or zero-order-holds the last command if
`stale_command_policy: hold` is set. Staleness is measured against
**arrival time at the subscription callback**, not against
`MITCommand.header.stamp`, so publisher clock skew is irrelevant.
Expand All @@ -259,7 +262,7 @@ ships the piano-task-specific launches.

| Side | Machine | Launch | What lives here |
|---|---|---|---|
| **Robot** | Onboard computer (RT kernel, wired tether) | `humanoid_bringup_lite/launch/real.launch.py` (Humanoid Control) | `ros2_control_node`, `humanoid_devices_robstride` / `humanoid_devices_sito` hardware plugins, `joint_state_broadcaster`, the five FSM controllers (`zero_torque` / `damping` / `standby` / `rl_policy` / `remote_policy`), `mode_manager`, `joy_node`, `robot_state_publisher`, IMU driver |
| **Robot** | Onboard computer (RT kernel, wired tether) | `humanoid_bringup_lite/launch/real.launch.py` (Humanoid Control) | `ros2_control_node`, `humanoid_devices_robstride` / `humanoid_devices_sito` hardware plugins, `joint_state_broadcaster`, the six FSM controllers (`zero_torque` / `damping` / `standby_a` / `standby_b` / `rl_policy` / `remote_policy`), `mode_manager`, `joy_node`, `robot_state_publisher`, IMU driver |
| **Host** | Operator workstation | `humanoid_bringup_lite/launch/viz.launch.py` (Humanoid Control) | `viser_viz` *or* `rerun_viz` (selected by `viewer:=`) |
| **Robot** | Onboard computer | `humanoid_control_policy/launch/lite_policy.launch.py` (Humanoid Control) / `pianist_policy/launch/piano_policy.launch.py` (pianist_ros2) | Runs `prepare` (resolve ONNX, convert motion → `.mcap` + overlay) then loads `rl_policy_controller` into the local CM. Inference is in-process, so the `.onnx` / `.mcap` artifacts **and** the W&B / HF Hub / `onnxruntime` *prepare-time* deps live here. The RT path itself pulls none of them. |
| **Robot** | Onboard computer | `pianist_policy/launch/midi_keyboard_driver.launch.py` (pianist_ros2) | USB-MIDI keyboard driver → `/piano/key_state` (`std_msgs/Float32MultiArray`); feeds the on-robot controller's `key_pressed` extern term locally (loopback, does **not** cross the tether). |
Expand Down
44 changes: 28 additions & 16 deletions docs/concepts/five_mode_fsm.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ property.
|---|---|---|---|
| **ZERO_TORQUE** | `humanoid_control::ZeroTorqueController` | `0` to all 5 MIT interfaces on every joint | Startup default. Fault fallback when DAMPING can't be applied (e.g. state not yet valid). Robot is alive but inert. |
| **DAMPING** | `humanoid_control::DampingController` | `stiffness=0`, `damping=damping_value`, `position=captured`, `velocity=0`, `effort=0` | Compliant fail-safe. Robot stays soft under gravity but resists velocity. The state you pass through between operator-driven transitions. |
| **STANDBY** | `humanoid_control::StandbyController` | Interpolated `position` along a YAML pose sequence; `K_p`/`K_d` ramped 0→target during segment 0 | Animate the arms to the piano-ready pose with gain ramp-in. Publishes `~/state.is_finished` so transitions out are gated correctly. |
| **STANDBY** | `humanoid_control::StandbyController` | Interpolated `position` along a YAML pose sequence; `K_p`/`K_d` ramped 0→target during segment 0 | Animate the arms to a ready pose with gain ramp-in. Spawned as **two instances** — `standby_controller_a` (Pose A) and `standby_controller_b` (Pose B) — the same plugin with different YAML poses. Each publishes `~/state.is_finished` so transitions out are gated correctly. |
| **LOCOMOTION** | `humanoid_control::RLPolicyController` | In-process ONNX inference (low-latency, C++): packs obs, replays the `.mcap` motion reference, writes commands | **Every learned policy** — tracking, piano, locomotion. They differ only by the loaded `.onnx` + `.mcap`; the ONNX `task_type` selects the term set. This is the System 0 real-time path. |
| **REMOTE** | `humanoid_control::RemotePolicyController` | `MITCommand` consumed from `~/command` over DDS | System 1/2 external-command ingress: a *non*-real-time source publishes commands (gravity-comp today via `Lite-Gravity-Compensation`; VLA / manipulation later). Not used by the learned policies. |

Expand All @@ -49,7 +49,8 @@ current mode:

```
DAMP (X) : any state → DAMPING
LOAD (L1+A/B) : DAMPING → STANDBY
LOAD_A (L1+A) : DAMPING → STANDBY (standby_controller_a, Pose A)
LOAD_B (L1+B) : DAMPING → STANDBY (standby_controller_b, Pose B)
START_LOCOMOTION (R1+A) : STANDBY ∧ is_finished → LOCOMOTION
START_REMOTE (R1+B) : STANDBY ∧ is_finished → REMOTE
QUIT (BACK) : ZERO_TORQUE or DAMPING → rclcpp::shutdown()
Expand All @@ -58,7 +59,7 @@ QUIT (BACK) : ZERO_TORQUE or DAMPING → rclcpp::shutdown()
```

The same transitions are also exposed as `std_srvs/Trigger` services
under `/humanoid_control/mode/{damp,load,start_remote,start_locomotion,quit}`, so a
under `/humanoid_control/mode/{damp,load_a,load_b,start_remote,start_locomotion,quit}`, so a
keyboardless lab box can drive the FSM with `ros2 service call …`.

`mode_manager` publishes `/control_mode` (`humanoid_control_msgs/ControlMode`) at
Expand Down Expand Up @@ -91,19 +92,30 @@ back to ZERO_TORQUE and writes the failure reason into
See [Concepts → Safety pipeline](./safety_pipeline.md) for what each
flag means and which plugin sets it.

## Pairing convention for the START combos

`L1+A → R1+A` is the "locomotion" (local) path; `L1+B → R1+B` is the
"remote policy" path. **Both LOAD combos land in the same STANDBY** —
the A/B distinction is just operator UX so your thumb stays on one
column through `LOAD → START`. The FSM doesn't enforce it; you could
press `L1+A` then `R1+B` and the state machine would happily route
you DAMPING → STANDBY → REMOTE.

The reason for two combos at all is the **policy target is chosen at
runtime**, not by a launch arg. There's no `policy_mode` parameter on
`mode_manager` — the choice between `RemotePolicyController` and
`RLPolicyController` is the button that ends the START combo.
## Pose and policy are independent

The two LOAD combos select **which standby pose** to animate to; the two
START combos select **which policy** to run. The two axes are orthogonal:

- `L1+A` → LOAD_A loads `standby_controller_a` (Pose A); `L1+B` → LOAD_B
loads `standby_controller_b` (Pose B). The two instances are the same
`StandbyController` plugin (source `standby_controller.cpp`, unchanged)
configured with **different YAML poses**.
- From **either** standby pose you can start **either** policy: `R1+A` →
LOCOMOTION (`rl_policy_controller`), `R1+B` → REMOTE
(`remote_policy_controller`). There is no pairing — `L1+A` then `R1+B`
is exactly as valid as `L1+A` then `R1+A`.

The one thing you **cannot** do is switch directly from Pose A to Pose B
(or back): LOAD is admissible only from DAMPING (unchanged), so to change
pose you DAMP first, then LOAD the other pose. On `/control_mode` both
poses report `mode = STANDBY`; the `controller_name` field tells you which
one is active (`standby_controller_a` vs `standby_controller_b`).

The **policy target is still chosen at runtime**, not by a launch arg.
There's no `policy_mode` parameter on `mode_manager` — the choice between
`RemotePolicyController` and `RLPolicyController` is the button that ends
the START combo.

## What mode_manager is *not*

Expand Down
11 changes: 7 additions & 4 deletions docs/concepts/safety_pipeline.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,10 +70,13 @@ additionally has a **stale-command policy**: if the external source's
(default 100 ms = 5 ticks at 50 Hz, measured against arrival time at the
subscription callback — not against `MITCommand.header.stamp` — so
publisher clock skew is irrelevant), the controller writes a fallback
pattern rather than re-using the last command. Default `passive` → zero
stiffness / damping → motors go limp. Alternative `hold` → freeze at the
last commanded pose. Either way the controller stays alive and active;
the choice is whether to "fail compliant" or "fail rigid".
pattern rather than re-using the last command. Default `passive` → a
**damped hold**: zero stiffness, high damping (`kD = damping_scalar`,
default 6.0, matching DAMPING mode), holding the live joint position — the
arms stay damped, not limp. (The same damped hold also applies when REMOTE
is first entered, before any `MITCommand` has arrived.) Alternative `hold`
→ freeze at the last commanded pose. Either way the controller stays alive
and active; the choice is whether to "fail compliant" or "fail rigid".

`RLPolicyController` has no such command stream — it runs inference
in-process (System 0), so there is nothing to go stale. It guards
Expand Down
7 changes: 4 additions & 3 deletions docs/getting_started/lite_101.md
Original file line number Diff line number Diff line change
Expand Up @@ -91,7 +91,8 @@ Expected output (give or take):
joint_state_broadcaster joint_state_broadcaster/JointStateBroadcaster active
zero_torque_controller humanoid_control/ZeroTorqueController active
damping_controller humanoid_control/DampingController inactive
standby_controller humanoid_control/StandbyController inactive
standby_controller_a humanoid_control/StandbyController inactive
standby_controller_b humanoid_control/StandbyController inactive
rl_policy_controller humanoid_control/RLPolicyController inactive
remote_policy_controller humanoid_control/RemotePolicyController inactive
```
Expand All @@ -105,7 +106,7 @@ claims all 5 command interfaces on every joint and writes 0 to all of
them every tick. From an operator's perspective the robot is "alive but
inert".

The four other controllers are **loaded but inactive**. Loading them
The five other controllers are **loaded but inactive**. Loading them
runs their `on_configure` (params parsed, publishers/subscribers created)
without claiming the command interfaces. They sit ready to be activated
in a single service call.
Expand All @@ -117,7 +118,7 @@ ros2 topic list | grep -E "joint_states|control_mode|standby|safety"
# /control_mode
# /lite/joint_states
# /safety_status
# /standby_controller/state
# /standby_controller_a/state
```

`joint_state_broadcaster` is remapped at bringup so that it publishes on
Expand Down
8 changes: 7 additions & 1 deletion docs/how_to/add_new_joint.md
Original file line number Diff line number Diff line change
Expand Up @@ -119,7 +119,7 @@ length. Use the same K/D as a similar-class joint as a starting
point:

```yaml
standby_controller:
standby_controller_a:
ros__parameters:
joints: [...] # length 17 now
target_stiffness: [20, 20, ..., 20, 30, 30, 30] # was 14 entries, now 17
Expand All @@ -128,6 +128,12 @@ standby_controller:
pose_segment_1: [0.3, -1.0, ..., -0.3, 0, 0, 0] # piano-ready + neck-zero
```

STANDBY now has two poses, each a separate spawned instance of the same
plugin with its own params block — `standby_controller_a` (Pose A) and
`standby_controller_b` (Pose B). Both blocks own per-joint arrays, so
extend **both** the same way when you add a joint: the arrays in
`standby_controller_b:` must grow to the new length too.

## Step 4 — Calibrate

Once the URDF + YAML are updated and the build is clean, plug in the
Expand Down
2 changes: 1 addition & 1 deletion docs/how_to/first_real_bringup.md
Original file line number Diff line number Diff line change
Expand Up @@ -116,7 +116,7 @@ pixi shell
# 14 joints reporting at the configured update rate
ros2 control list_controllers
# Expect: joint_state_broadcaster (active), zero_torque_controller (active),
# damping_controller / standby_controller / remote_policy_controller (inactive)
# damping_controller / standby_controller_a / standby_controller_b / remote_policy_controller (inactive)

ros2 topic hz /lite/joint_states
# average rate: 50.0 (± 0.2)
Expand Down
4 changes: 2 additions & 2 deletions docs/how_to/promote_python_to_cpp.md
Original file line number Diff line number Diff line change
Expand Up @@ -101,8 +101,8 @@ Finally, drive the FSM to activate (third terminal, inside `pixi shell`):

```bash
ros2 service call /humanoid_control/mode/damp std_srvs/srv/Trigger
ros2 service call /humanoid_control/mode/load std_srvs/srv/Trigger
# wait for /standby_controller/state.is_finished == true
ros2 service call /humanoid_control/mode/load_a std_srvs/srv/Trigger # or load_b for Pose B
# wait for /standby_controller_a/state.is_finished == true (per pose)
ros2 service call /humanoid_control/mode/start_locomotion std_srvs/srv/Trigger
```

Expand Down
3 changes: 2 additions & 1 deletion docs/how_to/record_experiments.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,8 @@ explicitly:
ros2 bag record -s mcap -o tuning_$(date +%Y%m%d_%H%M%S) \
/lite/joint_states \
/control_mode \
/standby_controller/state \
/standby_controller_a/state \
/standby_controller_b/state \
/safety_status \
/joy \
/tf /tf_static
Expand Down
10 changes: 6 additions & 4 deletions docs/how_to/recover_from_fault.md
Original file line number Diff line number Diff line change
Expand Up @@ -141,11 +141,13 @@ walks back up:
# /safety_status is now OK (level=0)
# But mode_manager is still in DAMPING.
# Walk back via the gamepad:
# L1+A or L1+B → STANDBY
# R1+A → LOCOMOTION
# R1+B → REMOTE
# L1+A → STANDBY (Pose A)
# L1+B → STANDBY (Pose B)
# R1+A → LOCOMOTION
# R1+B → REMOTE
# Or via the std_srvs/Trigger services:
ros2 service call /humanoid_control/mode/load std_srvs/srv/Trigger
ros2 service call /humanoid_control/mode/load_a std_srvs/srv/Trigger
# (use /humanoid_control/mode/load_b for Pose B)
ros2 service call /humanoid_control/mode/start_remote std_srvs/srv/Trigger
```

Expand Down
26 changes: 19 additions & 7 deletions docs/how_to/switch_controllers_manually.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,36 +69,47 @@ oscillating.

### DAMPING → STANDBY

STANDBY has two poses, each a separately spawned instance of the same
plugin: `standby_controller_a` (Pose A) and `standby_controller_b`
(Pose B). Activate whichever pose you want:

```bash
ros2 control switch_controllers \
--deactivate damping_controller \
--activate standby_controller
--activate standby_controller_a
```

Use `--activate standby_controller_b` instead for Pose B.

**The motors will move.** Standby ramps `K_p` / `K_d` from 0 to the
target gains during segment 0, then interpolates to the piano-ready
pose during segment 1. Total runtime ~4 seconds. Support the arms or
have a clear workspace.

Watch `/standby_controller/state` for `is_finished: true`:
Watch the state topic for the pose you activated (one per instance) for
`is_finished: true`:

```bash
ros2 topic echo /standby_controller/state
ros2 topic echo /standby_controller_a/state
```

### STANDBY → REMOTE (or LOCOMOTION)

```bash
ros2 control switch_controllers \
--deactivate standby_controller \
--deactivate standby_controller_a \
--activate remote_policy_controller
```

(Deactivate whichever standby instance is active —
`standby_controller_a` or `standby_controller_b`.)

`remote_policy_controller` (`humanoid_control/RemotePolicyController`) is the
**System 1/2 external-command ingress**: it immediately starts looking
for `MITCommand` on `/remote_policy_controller/command`. Without a
publisher it'll trip its stale-command policy (`passive` by default
→ motors go limp) within 100 ms. To use this for real, start a
→ a damped hold: zero stiffness, high damping like DAMPING, holding
live position) within 100 ms. To use this for real, start a
non-real-time `MITCommand` source first — gravity compensation
(`Lite-Gravity-Compensation`) today, VLA / manipulation later. This
controller is **not** fed by any learned policy; learned policies run
Expand Down Expand Up @@ -135,7 +146,8 @@ ros2 control list_controllers
# damping_controller humanoid_control/DampingController active
# zero_torque_controller humanoid_control/ZeroTorqueController inactive
# joint_state_broadcaster joint_state_broadcaster/... active
# standby_controller humanoid_control/StandbyController inactive
# standby_controller_a humanoid_control/StandbyController inactive
# standby_controller_b humanoid_control/StandbyController inactive
# remote_policy_controller humanoid_control/RemotePolicyController inactive

# What hardware components are up?
Expand All @@ -160,7 +172,7 @@ ros2cs zero_torque_controller damping_controller
# Force a strict switch (fail if either controller is in the wrong state)
ros2 control switch_controllers \
--deactivate damping_controller \
--activate standby_controller \
--activate standby_controller_a \
--strict
```

Expand Down
2 changes: 1 addition & 1 deletion docs/how_to/talk_to_humanoid_control_from_python.md
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,7 @@ pub.write(zero_mit_command(state.name, damping=2.0))
| `JointState` | `/lite/joint_states` | reliable, depth 10 |
| `ControlMode` | `/control_mode` | reliable |
| `SafetyStatus` | `/safety_status` | reliable |
| `StandbyState` | `/standby_controller/state` | transient-local (latched) |
| `StandbyState` | `/standby_controller_a/state` (one per pose; `_b` for Pose B) | transient-local (latched) |

QoS reliability and durability **must** match the bringup for DDS to pair a
writer with a reader — the registry already encodes the matching values.
Expand Down
Loading
Loading