From 680cd3655b7b0af359192693cac61ef5c0df82f3 Mon Sep 17 00:00:00 2001 From: claude Date: Fri, 22 May 2026 00:02:45 +0000 Subject: [PATCH 1/2] docs(research-checkpoints): add 2026-05 baseline ecosystem checkpoint MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Per mithaq Mode A — first robowbc ecosystem checkpoint, anchored to the vectors card created 2026-05-21. Covers all six research vectors (WBC policy upstream, inference runtime, hardware/SDK, middleware, sim substrate, Python distribution) and audits all eight hidden assumptions (H1–H8) against Q1–Q2 2026 evidence. Key findings: - GR00T-WBC upstream rapidly closed gap with robowbc wrappers (GEAR-SONIC formal release 2026-02-19; SONIC training code + HF checkpoint 2026-04-10; N1.7 → SONIC end-to-end VLA workflow 2026-05-07) - LeRobot v0.5.0 (2026-03-09) added first humanoid (G1) and third-party policy plugins — validates H8, accelerates urgency on robowbc-py plugin shape - Sim substrate consolidated on MuJoCo-Warp via Newton + Isaac Lab 3.0 Beta during GTC 2026 (152x speedup vs MJX for locomotion); mjlab arrived as lighter alternative - ExecuTorch 1.0 GA (2025-10) is most credible H2 substitute path, no humanoid adoption observed yet - H5 and H8 strengthen; H6 needs revisit by 2026-08; H1/H2/H3/H4/H7 still betting-worthy Output location per mithaq SKILL.md: /docs/research-checkpoints/ (not in mithaq itself). Cross-references existing docs/ecosystem-strategy.md (strategy), docs/roadmap-2026-q2.md (priorities), and the deep technical docs/research/2026-05-merged-tech-report.md (implementation choices). 🤖 Generated with Claude Code Co-Authored-By: Claude --- docs/research-checkpoints/2026-05.md | 492 +++++++++++++++++++++++++++ 1 file changed, 492 insertions(+) create mode 100644 docs/research-checkpoints/2026-05.md diff --git a/docs/research-checkpoints/2026-05.md b/docs/research-checkpoints/2026-05.md new file mode 100644 index 0000000..8851d8a --- /dev/null +++ b/docs/research-checkpoints/2026-05.md @@ -0,0 +1,492 @@ +# robowbc Ecosystem Research — 2026-05 Checkpoint + +> A structured archive of `robowbc`'s investigation into the humanoid whole-body-control policy + runtime ecosystem. +> +> **This checkpoint:** May 22, 2026 (baseline) +> **Next scheduled update:** end of June 2026 (monthly cadence; see [`mithaq/vectors/robowbc.md`](https://github.com/MiaoDX/mithaq/blob/main/vectors/robowbc.md)) +> **Maintenance mode:** monthly deep research + human review (see §0.2) + +--- + +## 0. About this document + +### 0.1 Why this checkpoint exists + +`robowbc` is a Linux-first embedded runtime for humanoid WBC policy inference, betting on a single caller-facing contract (`Observation → WbcPolicy.predict → JointPositionTargets`) with multiple backends. Its viability depends on a fast-moving upstream: WBC policy families, inference runtimes, humanoid platforms, robot middleware, sim substrates, and Python distribution channels are all churning on roughly quarterly cycles. A selection decision made in April can be stale by July. + +This checkpoint has two jobs: (1) freeze the current understanding of all six research vectors so future checkpoints have a comparison anchor; (2) test that the vectors card's eight hidden assumptions (H1–H8) actually get challenged by real evidence — if they all pass unchallenged twice in a row, the card has blind spots. + +This is the **first** robowbc checkpoint; the vectors card was written 2026-05-21 (the day before), so most "what changed since last checkpoint" entries here are establishing rather than comparing. The Sources section in the vectors card defines tier-level rules; this checkpoint follows them. + +### 0.2 Update method + +Each update follows the loop defined in [`mithaq/templates/checkpoint.md`](https://github.com/MiaoDX/mithaq/blob/main/templates/checkpoint.md) §0.2: + +1. Targeted deep research across the 6 vectors from the [vectors card](https://github.com/MiaoDX/mithaq/blob/main/vectors/robowbc.md): + 1. WBC policy upstream + 2. Robot ML inference runtime stack + 3. Humanoid hardware + SDK ecosystem + 4. Robot middleware (DDS / ROS 2 / alternatives) + 5. Sim substrate for inference proof artifacts + 6. Python-first robotics distribution + policy interface peers +2. Compare against previous checkpoint (N/A for this baseline; future checkpoints will diff). +3. Write `YYYY-MM.md` in this directory. +4. Manual review against the checks in §0.4 of the template. + +Triggers for off-schedule passes: see vectors card §Triggers. None fired between card creation (2026-05-21) and this checkpoint (2026-05-22) — though several Q1 2026 events (LeRobot v0.5.0, Isaac Lab 3.0 Beta, GEAR-SONIC formal release) would have been off-schedule triggers had the card existed at the time. + +### 0.3 Source-quality declaration + +This checkpoint cites primarily A-tier sources: NVIDIA/NVlabs and HuggingFace official repos and blog posts, arXiv preprints, ICLR/ICRA acceptance pages, MuJoCo + Isaac Lab official docs. A few B-tier sources (botinfo.ai, awesomeagents.ai, KraneShares industry note) are used for ecosystem-scale facts (shipment numbers, IPO timing) and explicitly flagged where load-bearing. No D-tier "best humanoid robot 2026" SEO content is cited. + +Items marked `[未充分验证]` are claims the search pass surfaced but the underlying primary source could not be confirmed in one pass — next checkpoint should resolve. + +### 0.4 How to read this + +- Quick conclusions: §1 + §6. +- Ecosystem panorama: §3 (organized by vector). +- Hidden-assumption audit: §3.7 + §7. +- Preparing next checkpoint: §7 + §8. + +**Relationship to existing robowbc docs (May 2026):** This checkpoint deliberately operates at the **ecosystem-snapshot** level — what is happening upstream, who is shipping what, which assumptions are still betting-worthy. Three adjacent docs already exist and are not duplicated here: + +- [`docs/ecosystem-strategy.md`](../ecosystem-strategy.md) (2026-04-20) — distribution/community positioning. Where robowbc fits in the stack and how it reaches users. +- [`docs/roadmap-2026-q2.md`](../roadmap-2026-q2.md) (2026-04-10) — technical priorities and ordering. What to work on next. +- [`docs/research/2026-05-merged-tech-report.md`](../research/2026-05-merged-tech-report.md) (May 2026, Chinese) — deep technical decisions on DDS Rust binding selection (cyclors / cyclonedds-rs / dust-dds) and onboarding UX patterns from rl_sar / GR00T-WBC. **For specific implementation choices, read that doc.** + +A reader trying to make a decision should look here for *what's happening in the ecosystem*, the strategy doc for *how we position*, the roadmap for *what's next*, and the merged tech report for *how to implement it*. + +--- + +## 1. Executive summary + +**Five ecosystem-level judgments at this baseline:** + +1. **The GR00T-WholeBodyControl upstream rapidly closed the gap between "wrapped by robowbc" and "officially released."** GEAR-SONIC moved from research preview to formal release on 2026-02-19, followed by training code + HF checkpoint (04-10), MotionBricks generative motion (04-27), and end-to-end VLA → SONIC workflow on G1 (05-07). The Decoupled-WBC and GEAR-SONIC wrappers in robowbc are no longer ahead of upstream — they're catching up to a moving target with its own C++ deployment stack. +2. **LeRobot v0.5.0 (2026-03-09) made the Unitree G1 a first-class humanoid in the framework, with native HolosomaLocomotionController + GR00T-WBC integration.** This validates H8 (LeRobot as distribution channel) but also shortens the window for robowbc-py to ship as a competitive integration: LeRobot now covers the G1+SONIC path natively, so robowbc's value must come from the registry + multi-backend abstraction rather than from being the only Python path to SONIC. The new third-party policy plugin mechanism (`pip install lerobot_policy_*`) is the right shape for `robowbc-py`. +3. **Sim substrate consolidated on MuJoCo-Warp as the GPU backend during GTC 2026.** Newton (NVIDIA + DeepMind + Disney, Linux Foundation, Apache-2.0) and Isaac Lab 3.0 Beta both integrate MuJoCo-Warp; MuJoCo-Warp is reportedly 152x faster than MJX for locomotion on RTX 4090. mjlab (arXiv 2026, Berkeley/DeepMind authors) provides a lighter-weight Isaac-Lab-style API on the same substrate. H6 (static reports site as primary proof surface) still holds but the throughput ceiling has moved up by ~2 orders of magnitude. +4. **ExecuTorch 1.0 GA (2025-10) is the most credible substitute path to ONNX Runtime in robowbc's H2 stack.** No humanoid robotics adoption signal observed yet, but the value proposition (native PyTorch export, no ONNX intermediate, ~50KB runtime, 12+ backends) is the exact pattern that would skip robowbc-ort if a humanoid vendor adopts it. Watch, do not switch. +5. **Humanoid hardware ecosystem broadened in 2026 Q1 without producing a credible second-platform integration target with mature Linux SDK on par with Unitree G1.** Booster T1 (open SDK, ROS2-compatible) is the closest. AgiBot G2/A2 has scale but limited Western support documentation. Fourier GR-3 announced late 2025, details emerging. H3 + H7 hold; if a second platform is needed, Booster T1 is the most aligned candidate. + +**Specific recommendations for `robowbc`** (full version in §6): + +1. **Refresh the `gear_sonic` wrapper against the 2026-02-19 official release and the C++ deployment stack** — surface area expanded considerably after the SONIC formal release. +2. **Ship `robowbc-py` as a LeRobot third-party policy plugin** (`pip install lerobot_policy_robowbc` shape) to align with the v0.5.0 plugin mechanism and shorten the integration path. +3. **Add MotionBricks, Newton, mjlab, Booster T1 to the vectors card entity lists** during the next quarterly card audit (currently scheduled 2026-08-21). +4. **Read LeRobot's `HolosomaLocomotionController` design carefully** — it's the closest "competing one-contract" integration in the ecosystem and may reveal patterns worth adopting (or deliberately diverging from). +5. **Decide whether MuJoCo-Warp / Newton / mjlab is the right next substrate for `make showcase-verify` proof-pack throughput** — current CPU MuJoCo path is the bottleneck for substantive benchmark depth. + +**Still uncertain, requires next checkpoint to verify:** + +- Does GR00T N1.7's `UNITREE_G1_SONIC` end-to-end VLA workflow make robowbc's GEAR-SONIC wrapping redundant for the specific G1+SONIC case, or does it preserve a meaningful runtime abstraction? +- What is the exact policy-schema delta between LeRobot v0.4.x and v0.5.0? Does it break the `robowbc-py` adapter shape implied in roadmap item #41? +- Has Booster T1's SDK + URDF/MJCF reached parity with Unitree G1 sufficient to be a second integration target? + +--- + +## 2. Background: robowbc current state and constraints + +**Project positioning** (one-sentence): Linux-first embedded runtime for humanoid whole-body-control policy inference; one caller-facing contract, multiple policy backends, TOML-config-driven backend selection. + +**Current stack (May 2026, actual state of the repo)**: + +- Core contracts: `Observation`, `WbcCommand`, `JointPositionTargets`, `WbcPolicy`, `PolicyCapabilities`, `RobotConfig` +- Rust crate map (14 crates): `robowbc-core`, `robowbc-config`, `robowbc-registry`, `robowbc-ort`, `robowbc-pyo3`, `robowbc-py`, `robowbc-runtime`, `robowbc-comm`, `robowbc-transport`, `robowbc-sim`, `robowbc-teleop`, `robowbc-vis`, `robowbc-cli`, `unitree-hg-idl` +- Inference: ONNX Runtime via `ort-rs` (primary); PyO3 + PyTorch (user-supplied) +- Transport: CycloneDDS / in-memory; Rerun for visualization +- Sim: MuJoCo (CPU); no GPU substrate currently +- Integration target: Unitree G1 +- Distribution: maturin-built standalone Python SDK in `crates/robowbc-py`; CLI binary; public reports site at + +**Live policy backends**: `gear_sonic`, `decoupled_wbc`, `wbc_agile`, `bfm_zero` +**Blocked / experimental wrappers**: `hover` (no public pretrained checkpoint), `wholebody_vla` (no runnable upstream), `py_model` (user-supplied) + +**Known technical constraints**: + +- Linux only — runtime backends fail fast on unsupported platforms +- Joint position targets as the output boundary; no direct torque output exposed to callers +- ONNX is the canonical model interchange; PyTorch only via `py_model` +- Public assets only — wrappers without runnable public checkpoints are marked Blocked +- Static reports site (HTML + JSON + Rerun .rrd) as the primary proof surface, not a daemon API + +These constraints drive every judgment below. + +--- + +## 3. Ecosystem panorama + +Organized by the six research vectors in the vectors card. + +### 3.1 Vector 1: Humanoid WBC policy upstream + +**Status: high churn. Q1 2026 saw the largest single quarter of releases robowbc has observed in this domain.** + +#### 3.1.1 NVlabs/GR00T-WholeBodyControl — formalized + +| Event | Date | Why it matters for robowbc | +|---|---|---| +| Initial release (Decoupled WBC for N1.5/N1.6) | 2025-11-12 | Anchored robowbc's `decoupled_wbc` wrapper | +| GEAR-SONIC formal release: pretrained checkpoints, **C++ inference stack**, VR teleoperation, docs | 2026-02-19 | Upstream now ships its own C++ deployment (`gear_sonic_deploy`, G1Deploy app) — robowbc's `gear_sonic` wrapper now overlaps with an official deployment path. | +| SONIC training code + HF checkpoint released | 2026-04-10 | "Train from scratch or finetune" is now public; users no longer need robowbc to access SONIC weights — only to deploy them under a multi-backend contract. | +| Live SONIC web demo (Kimodo text-to-motion) | 2026-04-14 | Interactive proof surfaces appearing upstream; raises the bar for robowbc's static reports site (H6 concern). | +| MotionBricks public release: VQVAE + pose + root checkpoints, synthetic training code | 2026-04-27 | New policy-class candidate. Real-time latent generative motion control — not currently wrapped by robowbc. | +| End-to-end VLA workflow on G1 (collect teleop → finetune N1.7 → deploy with SONIC) | 2026-05-07 | The `UNITREE_G1_SONIC` embodiment tag wires the VLA → SONIC pipeline. Per [issue #108](https://github.com/NVlabs/GR00T-WholeBodyControl/issues/108), N1.6's 17D upper-body joint output is for Decoupled WBC; SONIC expects 14D Cartesian wrist poses — boundary shapes differ by family. | + +**Counter-evidence to H1 (one-contract abstraction):** The 14D-wrist vs 17D-joints VLA → WBC mismatch in issue #108 is at the VLA → WBC interface, not at the WBC → joint targets interface where robowbc operates. **H1 still holds for robowbc's specific layer**, but the result confirms the contract is layer-specific, not universal. + +#### 3.1.2 BFM-Zero (LeCAR Lab, CMU) + +- arXiv 2511.04131; **accepted to ICLR 2026** (poster, Apr 24); G1-deployed; unsupervised RL + Forward-Backward models +- Already wrapped by robowbc as `bfm_zero` +- License: CC BY-NC 4.0 (non-commercial) — usage constraint worth noting in robowbc's third-party-notices + +#### 3.1.3 Academic / open humanoid line (new entrants not in current vectors card) + +| Project | Source | Status | Notes | +|---|---|---|---| +| **GMR / GMT** (General Motion Retargeting / Tracking) | YanjieZe/GMR, ICRA 2026 | Active | Supports 11 humanoid embodiments incl. G1, H1, Booster T1/K1 — useful retargeting reference | +| **KungfuBot / PBHC / KungfuBot2 / VMS** | arXiv 2506.12851, 2509.16638 | Active | Physics-based humanoid control, motion tracking with versatile skills | +| **APEX** (Adaptive High-Platform Traversal) | arXiv 2026-02 | Active | Specific to high-platform locomotion | +| **TWIST / CLONE / BumbleBee** | Recent papers | Active | Teleop-focused tracking; lower-dynamic motions | +| **Psi-Zero** (USC PSI Lab, separate from Physical Intelligence π series) | physical-superintelligence-lab/Psi0, RSS 2026 | Active | Whole-body humanoid VLA; ships with SIMPLE benchmarking sim (MuJoCo + Isaac Sim rendering); 6 pre-collected G1 loco-manipulation tasks on HF | +| **HumanoidVerse** | LeCAR-Lab | Active | IsaacGym + Genesis + IsaacLab unified training framework, G1 + H1 + 23/29 DoF support | + +These are not currently in the vectors card's entity list — should be added at the next card audit. + +#### 3.1.4 Physical Intelligence (π series) + +- **π-0.7 released April 2026** (per KraneShares industry note, B-tier): performed unfamiliar tasks (e.g., operating an air fryer seen only twice in training) by composing web pretraining with sparse demonstrations. +- Still bimanual-manipulation focused per available evidence; no humanoid WBC release confirmed. Counter-evidence to H1 candidate if a future π-X ships a whole-body model with a non-`predict → joint targets` shape. + +#### 3.1.5 Unitree-native VLA + +- **UnifoLM-VLA-0 open-sourced March 2026** (per botinfo.ai B-tier reporting `[未充分验证]` — need to find primary repo) +- If real, this is a new upstream from the hardware vendor itself. Affects H1 (competing interface watch) and H8 (LeRobot's role). + +### 3.2 Vector 2: Robot ML inference runtime stack + +**Status: stable core, one new credible substitute path.** + +| Component | Version (May 2026) | Activity | +|---|---|---| +| `onnxruntime` | 1.26.0 (2026-05-08); 1.25.0 (Apr 2026) | Active, monthly cadence. C++20 + CUDA 12+ minimum. | +| `ort-rs` | (Used by robowbc-ort) | Tracking onnxruntime closely | +| `executorch` | **1.0 GA (2025-10-22)** | **Substitute path candidate.** Native PyTorch export, no ONNX intermediate, ~50KB runtime, 12+ backends (ARM Ethos-U, Apple MPS, Qualcomm, Vulkan, MediaTek, OpenVINO, etc.). Production-proven at Meta (Quest 3, Smart Glasses). Robotics-specific adoption signal: none observed. | +| `candle` (HuggingFace) | ~15K stars | Steady. Minimalist Rust ML, focus on serverless + edge inference. | +| `burn` | ~8.5K stars | Backend-agnostic; CubeCL integration; WebAssembly + no_std support. | +| `tract` (Sonos) | — | Niche but stable; less HF integration | +| `mistral.rs` | 3.6K stars | LLM-focused; less relevant to humanoid WBC | +| `mujoco-warp` | Apache-2.0, Newton-integrated | Sim, not inference — but relevant to substrate side of H2 | + +**Verdict on H2 (Rust + PyO3 + ONNX Runtime is the right stack):** Holds. ONNX Runtime is still the dominant production path for PyTorch-trained checkpoints exported to ONNX, and `ort-rs` remains the standard Rust binding. ExecuTorch 1.0 is the real watch-target — its value proposition fits humanoid robotics well (edge deployment, small runtime, no ONNX conversion friction), but no humanoid project has publicly adopted it yet. **If GR00T-WBC's `gear_sonic_deploy` C++ stack or any subsequent humanoid vendor switches to ExecuTorch as the canonical inference path, robowbc-ort becomes one of two backends rather than the primary.** + +### 3.3 Vector 3: Humanoid hardware + SDK ecosystem + +#### 3.3.1 Unitree (current integration target) + +| Platform | Status | Notes | +|---|---|---| +| G1 | Active; firmware 1.4.5 (2025-11-13) added Training Mode | 23 & 29 DoF variants. 5,500+ units shipped 2025 (B-tier). 16 configurations $16K–$73.9K. | +| H1 / H1-2 | Active | Older platform; LeRobot integration also includes H1 | +| H2 | Launched | Less documented publicly than G1 | +| R1 | Pre-sale; ships April 2026 | $4.9K–$5.9K. Smaller athletic humanoid. Less of a research platform. | +| **Unitree IPO** | Filed Shanghai $610M, March 2026 (B-tier) | 335% YoY revenue growth 2025. A-share listing expected mid-2026. China's first publicly traded humanoid robotics company. | + +**Relevance to robowbc:** H7 ("Unitree G1 + unitree_sdk2 + CycloneDDS is the right hardware integration target") is reinforced. LeRobot's G1 integration explicitly uses `unitree_sdk2py==1.0.1` + `cyclonedds==0.10.2` — the same transport choice robowbc made. + +#### 3.3.2 Chinese humanoid wave — Linux SDK quality varies + +| Platform | SDK quality (per available evidence) | Plausibility as robowbc second-platform target | +|---|---|---| +| **Booster T1** | Open SDK, ROS2 compatible, public URDF, supported in GMR | **Highest** — closest to Unitree G1 in profile | +| **AgiBot G2 / A2** | NVIDIA partnership; Chinese-market focused; Western SDK documentation thin | Medium — viable but research-friction | +| **Fourier GR-3** | Announced late 2025, "details emerging" | Low (today) — wait for SDK | +| EngineAI SE01 / T800 | Aggressive pricing; SDK details limited | Low | +| XPeng IRON | Closed | None | +| UBTECH Walker | Closed-ish | None | + +GMR (General Motion Retargeting) supports 11 humanoid embodiments including Booster T1 (23 & 29 DoF) and Booster K1 — this is a useful signal that Booster has shippable URDF/MJCF. + +#### 3.3.3 US humanoid wave — mostly closed + +Figure 02/03 (closed-source), Boston Dynamics Atlas (no public customer API), Apptronik Apollo, 1X NEO, Tesla Optimus (not for sale). No platform has opened a Linux SDK in Q1 2026 that materially affects H3 / H7. + +### 3.4 Vector 4: Robot middleware + +**Status: CycloneDDS remains dominant for ROS 2 ecosystem; Zenoh adoption growing but no displacement.** + +| Component | State (May 2026) | +|---|---| +| `cyclonedds` 0.10.2 | Used by Unitree SDK + LeRobot G1 integration | +| `rmw_zenoh` | ROS 2 Jazzy + Rolling binary packages available; **not** ROS 2 default | +| `zenoh-plugin-ros2dds` | Recommended bridge for ROS 2 + Zenoh interop; will eventually deprecate `zenoh-bridge-dds` | +| ROS 2 Jazzy / Kilted | Jazzy is current LTS; Kilted in active development | +| Rerun | Active, used by robowbc; visualization-only | + +**Verdict on H7:** Holds. No major humanoid vendor committed to ROS 2 native as the only customer API; Zenoh has not become the ROS 2 default RMW. The parking-lot stance on ROS 2 native customer API remains correct. + +### 3.5 Vector 5: Sim substrate + +**Status: major substrate-layer event during GTC 2026.** + +| Component | Version / Status (May 2026) | Significance | +|---|---|---| +| MuJoCo | 3.x (3.8.0 released with MJX 2026-04-24); stable production | Still the reference CPU sim | +| MuJoCo XLA (MJX) | 3.8.0 (2026-04-24) | JAX backend; production stable | +| **MuJoCo-Warp (MJWarp)** | Active, integrated as Newton solver | **Performance breakthrough**: 152x faster than MJX for locomotion, 313x for manipulation on RTX 4090 (NVIDIA-claimed, A-tier) | +| **Newton** | Beta; Apache-2.0; Linux Foundation | **NEW.** Jointly developed by NVIDIA + Google DeepMind + Disney Research. MuJoCo-Warp is the main solver. | +| **Isaac Lab 3.0 Beta** | Released GTC 2026 | Multi-backend physics (PhysX + Newton/MuJoCo-Warp), pluggable renderer, kit-less install, Warp-native data pipelines (`wp.array` instead of `torch.Tensor` — breaking change for downstream code) | +| **mjlab** | arXiv 2026 (Berkeley/DeepMind); active | Lightweight Isaac-Lab-style API directly on MuJoCo-Warp. Minimal dependencies, no Isaac Sim required. Strong alternative for researchers who want MuJoCo-Warp performance without NVIDIA-stack dependencies. | +| **Genesis** (Genesis-Embodied-AI) | 0.4.7 (still pre-1.0) | Genesis 1.0 has **not** shipped. Genesis-Humanoid (UMass Embodied AGI) extends with humanoid-specific tooling: 200k RL steps/sec, real-time retargeting, 50ms end-to-end teleop. | +| ManiSkill3 | Stable | Manipulation-focused, less WBC-relevant | +| RoboCasa | Active | Integrated into GR00T-WBC | +| Cosmos 3 | Released GTC 2026 | NVIDIA's world-model line; not yet evaluated for WBC use | + +**Disambiguation note:** "Genesis AI" (a humanoid robotics company that released GENE-26.5 on 2026-05-06) is a **different entity** from Genesis-Embodied-AI (the open simulator). Easy to confuse; track separately. + +**Verdict on H6 (static reports site as primary proof surface):** Holds for now, but the substrate fidelity ceiling rose ~100x during Q1 2026. The bottleneck for robowbc's proof-pack depth is shifting from "we can only run N MuJoCo CPU scenarios per minute" to "we could run 10,000+ MuJoCo-Warp scenarios per minute if we adopt the new substrate." Interactive demos appearing upstream (SONIC web demo, GENE-26.5 demos) raise the comparison bar. **Recommend revisit at 2026-08 checkpoint.** + +### 3.6 Vector 6: Python-first distribution + interface peers + +#### 3.6.1 LeRobot v0.5.0 (2026-03-09) — pivotal release for robowbc + +| Feature | Why it matters for robowbc | +|---|---| +| **First humanoid integration = Unitree G1** (23 & 29 DoF) | Validates H8: LeRobot is the right channel for Python-first robotics — and chose the same platform robowbc bet on. | +| Uses `unitree_sdk2py==1.0.1` + `cyclonedds==0.10.2` | Same transport choice as robowbc. Lowers compatibility friction for `robowbc-py` shipping as a LeRobot policy. | +| **Third-party policy plugins** (`pip install lerobot_policy_*`) | This is the exact distribution shape for `robowbc-py`. Roadmap #41 target: `lerobot_policy_robowbc`. | +| **HolosomaLocomotionController + GR00T-WBC native integration** | Direct competition for robowbc's G1+SONIC value proposition. LeRobot users can now go from LeRobot dataset → Pi0-FAST or GR00T-WBC policy → G1 without robowbc. **robowbc's distinct value must come from registry + multi-backend abstraction, not from being the only Python path to SONIC.** | +| Pi0-FAST autoregressive VLA + Real-Time Chunking | New policy classes; LIBERO-10 score 60.0 (lower than diffusion-based on compositional tasks) — autoregressive VLAs have a long-horizon weakness worth watching | +| EnvHub for HF-hosted sim envs | Could be a distribution surface for robowbc's proof-pack scenarios | +| NVIDIA IsaacLab-Arena integration | Couples LeRobot's training side to Isaac Lab | +| Python 3.12+, Transformers v5 | Affects `robowbc-py` Python version policy | +| ICLR 2026 paper accepted | Cements LeRobot's standing as the field's standard reference distribution | +| v0.5.1 (2026-04-07) follow-up: fixes Unitree G1 SDK detection, install docs | Active maintenance | + +#### 3.6.2 Interface peers (the H1 competition watch) + +- **LeRobot `Policy` API**: framework-level, at the VLA/manipulation layer, not WBC. Different layer from `WbcPolicy`. +- **GR00T-WBC Policy Interface** (per DeepWiki): present in upstream, scoped to GR00T-WBC internals — not a community standard. +- **HolosomaLocomotionController** (LeRobot): a concrete integration class, not a contract abstraction. Not a direct competitor to `WbcPolicy` at the abstraction level. +- **ROS 2 humanoid control WG interface proposals**: no evidence of new community-standard interface emerging in Q1 2026. + +**Verdict on H1 (one-contract WBC abstraction):** Still holds. No competing one-contract runtime at framework scope appeared. The closest is GR00T-WBC's own Policy Interface, but it's scoped to GR00T-WBC's own use, not pitched as a cross-vendor standard. + +### 3.7 Hidden assumption audit + +The vectors card requires every checkpoint to answer: did this period produce evidence that any of H1–H8 needs reconsidering? + +| # | Hidden assumption | This period's evidence | Verdict | +|---|---|---|---| +| H1 | One contract covers all WBC policy families | Issue #108 in GR00T-WBC shows VLA → WBC interface varies by family (14D wrist vs 17D joints), but the WBC → joint-targets boundary that robowbc operates at is stable across the families surveyed. | **Holds** | +| H2 | Rust + PyO3 + ONNX Runtime is the right runtime | ExecuTorch 1.0 GA is the most credible substitute path; no humanoid adoption yet. | **Holds, watching ExecuTorch** | +| H3 | Linux-first fail-fast | No counter-evidence; all surveyed humanoid platforms ship Linux-first. | **Holds** | +| H4 | Joint position targets, not direct torque | Frontiers in Robotics & AI Dec 2025 mini-review explicitly states position-based policies outperform torque policies for humanoid loco-manipulation; G1 has no joint torque sensors so PD position control is the practical choice. SATA (arXiv 2502.12674) is a torque-policy candidate but locomotion-specific and on quadrupeds. | **Holds** | +| H5 | Multiple policy families coexist under one registry | Q1 2026 produced 5+ active humanoid policy families (Decoupled-WBC, SONIC, BFM-Zero, AgileGR, Psi-Zero, GMT, MotionBricks, etc.) — registry approach is more justified now than at card-writing time. | **Strengthens** | +| H6 | Static reports site is the right primary proof surface | MuJoCo-Warp / Newton / Isaac Lab 3.0 lifted substrate throughput ~100x; upstream is shipping interactive web demos (SONIC, GENE-26.5). The ceiling moved; static site is not displaced but the bar is rising. | **Holds, needs revisit by 2026-08** | +| H7 | Unitree G1 + unitree_sdk2 + CycloneDDS; ROS 2 native parked | LeRobot G1 integration uses same stack. Zenoh did not become ROS 2 default. No vendor opened ROS-2-native-only customer API. | **Holds** | +| H8 | LeRobot is the right Python-first distribution channel | v0.5.0 made G1 first-class + added third-party policy plugin mechanism + ICLR 2026 acceptance. **Validated.** But the direct GR00T-WBC integration shortens the window for `robowbc-py` to ship as a competitive plugin. | **Strengthens; urgency increases** | + +**Assumption-blindspot check:** All eight assumptions found at least some discriminating evidence in this period. **The card does not appear to have major blind spots at this point.** Re-check after the 2026-06 checkpoint — if H1, H3, H7 still produce no discriminating evidence two periods running, those assumptions may be too vague or unfalsifiable. + +--- + +## 4. Taxonomy (optional) + +Skipped for the baseline; ecosystem panorama in §3 is already organized by the vectors-card structure. Will introduce a cross-cutting perspective in a future checkpoint if one becomes useful (e.g., "by safety boundary" vs "by abstraction layer"). + +--- + +## 5. Existing academic and industry taxonomies + +### 5.1 Academic surveys + +- Gu et al., 2025, "Humanoid Whole-Body Control: A Survey" — referenced by BFM-Zero (arXiv 2511.04131) as the field overview. +- Frontiers in Robotics & AI, Dec 2025, mini-review on humanoid manipulation policies — useful taxonomy of teleop vs imitation vs RL pipelines. + +### 5.2 Industry stack maps + +- LeRobot v0.5.0 release notes (HF blog) — implicitly defines the "supported platforms / policies / sim envs" matrix for the field's standard reference distribution. +- NVIDIA GTC 2026 announcement (GR00T N1.7 + N2, Isaac Lab 3.0, Newton, Cosmos 3) — implicit stack map for the NVIDIA-aligned subset. + +--- + +## 6. Specific recommendations for `robowbc` + +### 6.1 Trade-offs by phase + +Current phase: Q2 2026 — multiple live wrappers, public reports site shipping, no LeRobot integration yet (#41). The recommendations below are calibrated for this phase; will be revised when the LeRobot plugin lands. + +### 6.2 Projects worth deep-reading first + +1. **GR00T-WBC `gear_sonic_deploy` C++ stack** — reason: directly overlaps with robowbc's `gear_sonic` wrapper. Read the design trade-offs to decide whether to align, diverge, or absorb (e.g., as an optional alt-backend behind `WbcPolicy`). +2. **LeRobot `HolosomaLocomotionController` source** — reason: closest concrete one-contract integration in the field. Even if it's not a competing abstraction, the implementation choices (state machine shape, error handling, transport assumptions) are worth borrowing or deliberately diverging from. +3. **mjlab paper + repo (arXiv 2601.22074)** — reason: it answers "what does Isaac Lab look like with MuJoCo-Warp and minimal dependencies?" — directly relevant to the future of `robowbc-sim`. +4. **Psi-Zero / SIMPLE benchmarking sim** — reason: a new whole-body humanoid VLA with its own benchmarking infrastructure; potential next addition to the wrapped-policy list, and SIMPLE is a candidate proof-substrate. +5. **LeRobot v0.5.0 release notes + third-party policy plugin docs** — reason: defines the exact shape `robowbc-py` should ship in to maximize H8. + +### 6.3 Concrete engineering actions + +| # | Action | Status | Value | +|---|--------|--------|-------| +| 1 | Refresh `gear_sonic` wrapper against the 2026-02-19 SONIC release; check for tensor-shape / config-schema drift; document overlap with `gear_sonic_deploy` | TODO | High — current wrapper may be silently incompatible with newer SONIC checkpoints | +| 2 | Implement `lerobot_policy_robowbc` plugin (LeRobot v0.5.0 third-party plugin shape) — closes #41 | TODO | High — completes H8 distribution-channel bet | +| 3 | Add MotionBricks, Psi-Zero, GMT, mjlab, Newton, Booster T1 to `mithaq/vectors/robowbc.md` entity lists at next card audit (scheduled 2026-08-21) | TODO | Medium — keeps the card from going stale before formal audit | +| 4 | Prototype `robowbc-sim` MuJoCo-Warp backend behind a feature flag; benchmark proof-pack throughput vs current CPU MuJoCo | TODO | Medium — large potential throughput win; depends on whether `make showcase-verify` is currently the bottleneck | +| 5 | Investigate UnifoLM-VLA-0 (Unitree's open VLA) — confirm primary source; assess whether it fits the `WbcPolicy` contract or sits at a different layer | TODO | Medium — could be the first Unitree-native upstream worth wrapping | +| 6 | Read LeRobot policy-schema delta v0.4.x → v0.5.0; verify `robowbc-py` adapter shape (#41) still fits | TODO | Medium — prerequisite for action 2 | +| 7 | Decide on BFM-Zero CC BY-NC 4.0 license disclosure surface (third-party-notices, policy card) | TODO | Low-Medium — compliance hygiene | +| 8 | Add a "competing one-contract runtimes" tracking note in the next checkpoint — even if there is none today, the watch itself is the value | TODO | Low — process discipline | + +--- + +## 7. Open questions and what next checkpoint should answer + +### Experimental questions (require running code in this repo) + +- ☐ **Q1**: Does GR00T-WBC's `gear_sonic_deploy` C++ stack produce identical `JointPositionTargets` to robowbc's `gear_sonic` wrapper given the same observation? (If not, the wrapper has a silent drift bug.) +- ☐ **Q2**: What is the actual proof-pack-generation throughput today (scenarios/minute) on the public-reports CI? Need a number before claiming MuJoCo-Warp adoption is or isn't worth the work. + +### Technical selection questions + +- ☐ **Q3**: Is mjlab production-ready enough to host robowbc proof-pack scenarios, or research-only? +- ☐ **Q4**: Should `robowbc-py` ship as a LeRobot third-party policy plugin (`pip install lerobot_policy_robowbc`) or as a standalone package that LeRobot can also load? Plugin shape is more aligned with v0.5.0; standalone preserves independence from LeRobot's release cadence. +- ☐ **Q5**: Does Booster T1's public Linux SDK + URDF/MJCF reach the "supported humanoid" criteria threshold (Linux, documented joints, deterministic transport, MuJoCo parity)? Need to read primary SDK docs. + +### Ecosystem-tracking questions + +- ☐ **Q6**: Does GR00T N2 (previewed at GTC 2026, based on DreamZero research) ship a runnable public release in 2026 H2, and if so, does it preserve the `Observation → predict → JointPositionTargets` interface or break it? +- ☐ **Q7**: Does any humanoid vendor (Unitree, AgiBot, Booster, Figure) adopt ExecuTorch as the canonical inference path? First adoption is the H2 trigger. +- ☐ **Q8**: Does LeRobot ship a policy-schema-level breaking change before 2026 EOY that would force a `robowbc-py` adapter rewrite? + +### Long-horizon tracking questions + +- ☐ **Q9**: Does any platform ship a `WbcPolicy`-equivalent open abstraction at framework scope (LeRobot? NVIDIA? ROS 2 control WG? a Chinese humanoid vendor?) +- ☐ **Q10**: Does Genesis 1.0 ship in 2026? If yes, does it produce a real ecosystem effect or stay an alternative substrate? + +### Source-quality improvement questions + +- ☐ **Q11**: UnifoLM-VLA-0 — confirm primary repo URL, license, runnable status. Current evidence is B-tier; need A-tier verification. +- ☐ **Q12**: Unitree 2025 shipment numbers (5,500+ G1) and IPO target ($610M) — confirm from primary Shanghai Stock Exchange filing or Unitree IR. + +### New this period + +All Q1–Q11 are new at this baseline checkpoint. + +--- + +## 8. Changelog + +### 2026-05-22 — Baseline established + +**Architecture narrative:** First robowbc checkpoint. Vectors card created 2026-05-21, this checkpoint created 2026-05-22 — they establish the comparison anchor for all future periods. No prior anchors, no judgment revisions. + +**New projects surfaced (not in current vectors card entity lists):** + +- MotionBricks (NVlabs/GR00T-WholeBodyControl, 2026-04-27) +- Psi-Zero (USC PSI Lab, RSS 2026) +- GMT / GMR (YanjieZe, ICRA 2026) +- KungfuBot2 / VMS (arXiv 2509.16638) +- APEX (arXiv 2026-02) +- HumanoidVerse (LeCAR-Lab) +- mjlab (Berkeley/DeepMind, arXiv 2601.22074) +- Newton physics engine (NVIDIA + DeepMind + Disney, Linux Foundation, GTC 2026) +- Genesis-Humanoid (UMass Embodied AGI) +- UnifoLM-VLA-0 (Unitree, B-tier `[未充分验证]`) +- GENE-26.5 (Genesis AI — disambiguation: different entity from Genesis-Embodied-AI simulator) + +**Status changes for entities already in vectors card:** + +- GR00T-WBC: GEAR-SONIC formal release (2026-02-19); training code public (2026-04-10); end-to-end VLA workflow (2026-05-07). +- LeRobot: v0.5.0 with full G1 humanoid + 3rd-party policy plugin + ICLR 2026 acceptance. +- Isaac Lab: 3.0 Beta released (multi-backend physics, Warp-native data pipelines — breaking change). +- ONNX Runtime: 1.26.0 released 2026-05-08; CUDA 12+ minimum. +- ExecuTorch: 1.0 GA achieved 2025-10-22 (vectors card written before this but doesn't note GA status). +- Unitree: H2 launched; G1 firmware 1.4.5 Training Mode; IPO filing. + +**Judgment revisions:** None (first checkpoint). + +--- + +## Appendix A: Full reference links + +### A.1 Vector 1 — WBC policy upstream + +- NVlabs/GR00T-WholeBodyControl: +- NVIDIA/Isaac-GR00T (N1.7): +- MotionBricks: +- BFM-Zero: , arXiv: +- BFM-Zero ICLR 2026: +- Psi-Zero: +- GMR (General Motion Retargeting): +- HumanoidVerse: +- Physical-Intelligence/openpi: + +### A.2 Vector 2 — Inference runtime + +- microsoft/onnxruntime: +- ONNX Runtime roadmap: +- pytorch/executorch: +- ExecuTorch 1.0 GA blog: +- huggingface/candle: + +### A.3 Vector 3 — Hardware + +- Unitree GitHub: +- Unitree LeRobot integration: +- Unitree IL LeRobot docs: + +### A.4 Vector 4 — Middleware + +- rmw_zenoh: +- zenoh-plugin-ros2dds: + +### A.5 Vector 5 — Sim substrate + +- google-deepmind/mujoco_warp: +- mujoco-mjx PyPI: +- mjlab: +- NVIDIA Newton + Isaac Lab blog: +- Isaac Lab releases: +- Genesis-Embodied-AI: +- Genesis-Humanoid (UMass): + +### A.6 Vector 6 — Python distribution + +- huggingface/lerobot: +- LeRobot v0.5.0 blog: + +### A.7 robowbc itself + +- Repo: +- Public reports: +- Vectors card: +- Checkpoint template: +- Companion docs in this repo: [`docs/ecosystem-strategy.md`](../ecosystem-strategy.md), [`docs/roadmap-2026-q2.md`](../roadmap-2026-q2.md), [`docs/research/2026-05-merged-tech-report.md`](../research/2026-05-merged-tech-report.md) + +--- + +## Appendix B: Glossary + +| Term | Meaning | +|------|---------| +| WBC | Whole-Body Control — humanoid controller producing coordinated whole-body motion | +| `WbcPolicy` | robowbc's central trait; the `predict(Observation) → JointPositionTargets` contract | +| Decoupled WBC | Hybrid controller: RL for lower body locomotion + IK for upper body. Used in GR00T N1.5/N1.6. | +| GEAR-SONIC (SONIC) | Behavior foundation model from NVIDIA GEAR team; motion tracking as scalable training task; latest generalist WBC controller | +| MotionBricks | Real-time latent generative model for interactive motion control; part of GR00T-WBC | +| BFM-Zero | Promptable behavioral foundation model using unsupervised RL + Forward-Backward models; G1-deployed; CC BY-NC 4.0 | +| MJX | MuJoCo XLA — JAX-based reimplementation of MuJoCo, GPU-accelerated | +| MuJoCo-Warp (MJWarp) | GPU-optimized MuJoCo via NVIDIA Warp; main solver in Newton | +| Newton | Physics engine jointly developed by NVIDIA + DeepMind + Disney; Linux Foundation, Apache-2.0 | +| mjlab | Lightweight Isaac-Lab-style API directly on MuJoCo-Warp; minimal dependencies | +| HolosomaLocomotionController | LeRobot's controller class for G1 humanoid locomotion | +| Proof pack | robowbc's bundle of JSON + Rerun .rrd + screenshots + HTML report demonstrating a policy run | +| `UNITREE_G1_SONIC` embodiment tag | GR00T N1.7 tag selecting the SONIC controller path (vs `UNITREE_G1` which selects Decoupled WBC) | +| `[未充分验证]` | Marker for claims supported only by B-tier sources awaiting A-tier verification | + +--- + +**End of document. Next checkpoint scheduled for end of June 2026.** + +Run the incremental-research loop per §0.2, paying particular attention to Q1–Q4 (experimental + technical selection) and Q11 (UnifoLM-VLA-0 primary verification). From bf8bc3911a6106d65338c7f4d96b613d9aae7a01 Mon Sep 17 00:00:00 2001 From: claude Date: Fri, 22 May 2026 01:38:40 +0000 Subject: [PATCH 2/2] docs(research-checkpoints): upgrade 2026-05 baseline to v2 (deep research) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Replaces v1 (commit 680cd36) with deeper evidence after a full deep research pass over the Mar–May 2026 window. v1 was a single-pass sketch; v2 audits each of H1–H8 against multiple A-tier primaries. Major judgment changes vs v1: - H1 (one-contract WBC abstraction) — downgrades from "holds" to "holds only in joint-target codomain; falsified in latent / relative-EEF / Cartesian codomains". Concrete counter-examples: * GR00T N1.7 UNITREE_G1_SONIC outputs latent action tokens * GR00T N1.7 default cross-embodiment outputs relative EEF deltas * Psi-Zero (USC PSI, RSS 2026) Teleop outputs 14D wrist Cartesian Drives concrete trait redesign (predict_latent / predict_cartesian as first-class branches, not py_model black box). - H8 (LeRobot as distribution channel) — verdict stays validated but urgency drops. Two months after the third-party policy plugin mechanism shipped (v0.5.0, 2026-03-09), the ecosystem has produced exactly ONE third-party plugin (lerobot_policy_ditflow). Channel is public but largely unused. Recommend dual-path distribution (maturin standalone + lerobot_policy_robowbc) rather than single-path bet. - Strategic threat reordering — v1 framed LeRobot as the direct threat. v2 reorders to: gear_sonic_deploy cross-family reuse > LeRobot framework absorption > NVIDIA building robowbc-equivalent. gear_sonic_deploy already serves two policy families (SONIC + Decoupled WBC) and is one ONNX-config away from absorbing BFM-Zero or UnifoLM-VLA. - Hidden assumption audit, full table — every H1–H8 now has primary evidence delta and concrete action, not just "still betting". New A-tier evidence v1 lacked: - UnifoLM-VLA-0 primary repo + license (CC BY-NC-SA 4.0) - Newton 1.0 LF release + 252×/475× MuJoCo-Warp benchmarks (v1's 152× number is corrected — v1 was from a single search result, v2 cites NVIDIA Technical Blog primary) - Booster T1 SDK + booster_gym arXiv as concrete second-platform candidate (v1 had this as "could be useful") - AgiBot G2 wheeled architecture + AimRT C++20 middleware - Psi-Zero codomain split (Teleop = 14D Cartesian, MP = joint-space) - Pi0-FAST RTC latency benchmark (76ms baseline / 97ms RTC) - Unitree IPO A-tier confirmation via CNBC New vectors card proposals for 2026-08-21 quarterly audit: - V7: Real-time + safety certification (PREEMPT_RT, ISO 25785-1) - V8: License/IP + dataset schema tracking Concrete actions queued: 12 items across immediate (5–6 weeks), 6–12 weeks, and quarterly horizons. See §6. 🤖 Generated with Claude Code Co-Authored-By: Claude --- docs/research-checkpoints/2026-05.md | 772 ++++++++++++++------------- 1 file changed, 400 insertions(+), 372 deletions(-) diff --git a/docs/research-checkpoints/2026-05.md b/docs/research-checkpoints/2026-05.md index 8851d8a..2b40fec 100644 --- a/docs/research-checkpoints/2026-05.md +++ b/docs/research-checkpoints/2026-05.md @@ -1,492 +1,520 @@ -# robowbc Ecosystem Research — 2026-05 Checkpoint +# robowbc Ecosystem Research — 2026-05 Checkpoint (v2) -> A structured archive of `robowbc`'s investigation into the humanoid whole-body-control policy + runtime ecosystem. -> -> **This checkpoint:** May 22, 2026 (baseline) -> **Next scheduled update:** end of June 2026 (monthly cadence; see [`mithaq/vectors/robowbc.md`](https://github.com/MiaoDX/mithaq/blob/main/vectors/robowbc.md)) -> **Maintenance mode:** monthly deep research + human review (see §0.2) +> 替换 v1 (同一 PR 第二个 commit)。本次 checkpoint 2026-05-22。下次例行 2026-06 月末。 --- ## 0. About this document -### 0.1 Why this checkpoint exists +### 0.1 v2 与 v1 的关系 -`robowbc` is a Linux-first embedded runtime for humanoid WBC policy inference, betting on a single caller-facing contract (`Observation → WbcPolicy.predict → JointPositionTargets`) with multiple backends. Its viability depends on a fast-moving upstream: WBC policy families, inference runtimes, humanoid platforms, robot middleware, sim substrates, and Python distribution channels are all churning on roughly quarterly cycles. A selection decision made in April can be stale by July. +v1 是首份 baseline，一日内完成，留下 `[未充分验证]` 标签和若干 H1–H8 验证不充分的判断。v2 在同一日做了更深的循环并**有权直接覆盖 v1**。主要更新： -This checkpoint has two jobs: (1) freeze the current understanding of all six research vectors so future checkpoints have a comparison anchor; (2) test that the vectors card's eight hidden assumptions (H1–H8) actually get challenged by real evidence — if they all pass unchallenged twice in a row, the card has blind spots. +1. **H1 不再"holds"——出现可证伪反例**。GR00T N1.7 `UNITREE_G1_SONIC` 输出 latent action tokens、N1.7 默认输出 relative EEF deltas、Psi-Zero Teleop 模式输出 14D 腕部 Cartesian——三个反例落在三个不同的输出语义上，打破 v1 的"WBC → joint targets 层稳定"判断。 +2. **H8 从"strengthens"调整为"validated 但通道近乎闲置"**。LeRobot v0.5.0 第三方插件机制公开两个月后，生态里能找到的第三方插件只有 `lerobot_policy_ditflow` 一个孤例。 +3. 新增两个 sub-vector 建议：**V7 实时与安全 (PREEMPT_RT + ISO 25785-1)**，**V8 license / IP / 数据格式**。 +4. 战略威胁排序更新：NVIDIA 直接竞争 robowbc 的概率 < LeRobot 间接吞并 < `gear_sonic_deploy` C++ 栈被其它策略族复用。 +5. v1 的 "MuJoCo-Warp 比 MJX 快 152×" 数字未经 A-tier 复核；NVIDIA 官方 Technical Blog (2026-03, [Newton Adds Contact-Rich Manipulation and Locomotion](https://developer.nvidia.com/blog/newton-adds-contact-rich-manipulation-and-locomotion-capabilities-for-industrial-robotics/)) 字面写到 "New optimizations enable MuJoCo Warp to speed up MJX by **252x for locomotion**, and **475x for manipulation tasks** on the NVIDIA RTX PRO 6000 Blackwell Series." v1 的 152× 应替换为 252× (locomotion) / 475× (manipulation)。 -This is the **first** robowbc checkpoint; the vectors card was written 2026-05-21 (the day before), so most "what changed since last checkpoint" entries here are establishing rather than comparing. The Sources section in the vectors card defines tier-level rules; this checkpoint follows them. +### 0.2 Source-quality declaration -### 0.2 Update method +A-tier 直引：NVIDIA / NVlabs / HuggingFace / Unitree / Booster / LeCAR-Lab / USC PSI / Linux Foundation 官方 repo 与 blog；arXiv preprint；ICLR / RSS / CoRL 接收页；CNBC 等专业财经媒体对 IPO 的报道。B-tier 已逐项 `[需 A-tier 验证]`。本次未引用 SEO listicle、awesome-list 或未披露技术细节的初创公司新闻稿。 -Each update follows the loop defined in [`mithaq/templates/checkpoint.md`](https://github.com/MiaoDX/mithaq/blob/main/templates/checkpoint.md) §0.2: +### 0.3 与同期 robowbc 文档的关系 -1. Targeted deep research across the 6 vectors from the [vectors card](https://github.com/MiaoDX/mithaq/blob/main/vectors/robowbc.md): - 1. WBC policy upstream - 2. Robot ML inference runtime stack - 3. Humanoid hardware + SDK ecosystem - 4. Robot middleware (DDS / ROS 2 / alternatives) - 5. Sim substrate for inference proof artifacts - 6. Python-first robotics distribution + policy interface peers -2. Compare against previous checkpoint (N/A for this baseline; future checkpoints will diff). -3. Write `YYYY-MM.md` in this directory. -4. Manual review against the checks in §0.4 of the template. +不重复以下三份文档： +- [`docs/ecosystem-strategy.md`](../ecosystem-strategy.md)（2026-04-20）— 分发与社区定位 +- [`docs/roadmap-2026-q2.md`](../roadmap-2026-q2.md)（2026-04-10）— 技术优先级 +- [`docs/research/2026-05-merged-tech-report.md`](../research/2026-05-merged-tech-report.md)（2026-05）— DDS Rust binding + onboarding UX 实现决策深度报告 -Triggers for off-schedule passes: see vectors card §Triggers. None fired between card creation (2026-05-21) and this checkpoint (2026-05-22) — though several Q1 2026 events (LeRobot v0.5.0, Isaac Lab 3.0 Beta, GEAR-SONIC formal release) would have been off-schedule triggers had the card existed at the time. - -### 0.3 Source-quality declaration - -This checkpoint cites primarily A-tier sources: NVIDIA/NVlabs and HuggingFace official repos and blog posts, arXiv preprints, ICLR/ICRA acceptance pages, MuJoCo + Isaac Lab official docs. A few B-tier sources (botinfo.ai, awesomeagents.ai, KraneShares industry note) are used for ecosystem-scale facts (shipment numbers, IPO timing) and explicitly flagged where load-bearing. No D-tier "best humanoid robot 2026" SEO content is cited. - -Items marked `[未充分验证]` are claims the search pass surfaced but the underlying primary source could not be confirmed in one pass — next checkpoint should resolve. - -### 0.4 How to read this - -- Quick conclusions: §1 + §6. -- Ecosystem panorama: §3 (organized by vector). -- Hidden-assumption audit: §3.7 + §7. -- Preparing next checkpoint: §7 + §8. - -**Relationship to existing robowbc docs (May 2026):** This checkpoint deliberately operates at the **ecosystem-snapshot** level — what is happening upstream, who is shipping what, which assumptions are still betting-worthy. Three adjacent docs already exist and are not duplicated here: - -- [`docs/ecosystem-strategy.md`](../ecosystem-strategy.md) (2026-04-20) — distribution/community positioning. Where robowbc fits in the stack and how it reaches users. -- [`docs/roadmap-2026-q2.md`](../roadmap-2026-q2.md) (2026-04-10) — technical priorities and ordering. What to work on next. -- [`docs/research/2026-05-merged-tech-report.md`](../research/2026-05-merged-tech-report.md) (May 2026, Chinese) — deep technical decisions on DDS Rust binding selection (cyclors / cyclonedds-rs / dust-dds) and onboarding UX patterns from rl_sar / GR00T-WBC. **For specific implementation choices, read that doc.** - -A reader trying to make a decision should look here for *what's happening in the ecosystem*, the strategy doc for *how we position*, the roadmap for *what's next*, and the merged tech report for *how to implement it*. +本 checkpoint 在**生态快照**层面，不重复实现细节。 --- -## 1. Executive summary +## 1. Executive summary — 5 个生态级判断 -**Five ecosystem-level judgments at this baseline:** +**判断 1：H1 的"一份契约覆盖所有 WBC 家族"在 2026-Q2 出现可证伪反例，必须从假设升级为多 codomain 工程接口。** +依据是 GR00T N1.7 在 [Isaac-GR00T README](https://github.com/NVIDIA/Isaac-GR00T) 明确写到："GR00T N1.7 supports whole-body humanoid control via the `UNITREE_G1_SONIC` embodiment tag and the GEAR-SONIC controller. In this workflow, the VLA predicts compact **latent action tokens** that a learned whole-body controller decodes into full-body joint commands"。同模型 default 路径用的是 "Relative EEF Action Space … deltas from the current pose"；而 Psi-Zero (USC PSI Lab) 在 Teleop 任务根据 [Psi0 baselines/README](https://github.com/physical-superintelligence-lab/Psi0) 走 `psi0_decoupled_wbc` agent，模型出来的是腕部 14D Cartesian pose，由 Decoupled WBC 落到 joint targets。真实生态至少四种语义：**(a) joint position targets (29D PD setpoints), (b) latent action tokens, (c) 14D wrist Cartesian poses, (d) relative EEF deltas**。 +**行动**：robowbc 的 `WbcPolicy::predict` 必须显式分层：`predict_latent → decode → JointPositionTargets` 与 `predict_cartesian → ik → JointPositionTargets` 应是一等枚举，而不是塞进 `py_model` 黑盒。 -1. **The GR00T-WholeBodyControl upstream rapidly closed the gap between "wrapped by robowbc" and "officially released."** GEAR-SONIC moved from research preview to formal release on 2026-02-19, followed by training code + HF checkpoint (04-10), MotionBricks generative motion (04-27), and end-to-end VLA → SONIC workflow on G1 (05-07). The Decoupled-WBC and GEAR-SONIC wrappers in robowbc are no longer ahead of upstream — they're catching up to a moving target with its own C++ deployment stack. -2. **LeRobot v0.5.0 (2026-03-09) made the Unitree G1 a first-class humanoid in the framework, with native HolosomaLocomotionController + GR00T-WBC integration.** This validates H8 (LeRobot as distribution channel) but also shortens the window for robowbc-py to ship as a competitive integration: LeRobot now covers the G1+SONIC path natively, so robowbc's value must come from the registry + multi-backend abstraction rather than from being the only Python path to SONIC. The new third-party policy plugin mechanism (`pip install lerobot_policy_*`) is the right shape for `robowbc-py`. -3. **Sim substrate consolidated on MuJoCo-Warp as the GPU backend during GTC 2026.** Newton (NVIDIA + DeepMind + Disney, Linux Foundation, Apache-2.0) and Isaac Lab 3.0 Beta both integrate MuJoCo-Warp; MuJoCo-Warp is reportedly 152x faster than MJX for locomotion on RTX 4090. mjlab (arXiv 2026, Berkeley/DeepMind authors) provides a lighter-weight Isaac-Lab-style API on the same substrate. H6 (static reports site as primary proof surface) still holds but the throughput ceiling has moved up by ~2 orders of magnitude. -4. **ExecuTorch 1.0 GA (2025-10) is the most credible substitute path to ONNX Runtime in robowbc's H2 stack.** No humanoid robotics adoption signal observed yet, but the value proposition (native PyTorch export, no ONNX intermediate, ~50KB runtime, 12+ backends) is the exact pattern that would skip robowbc-ort if a humanoid vendor adopts it. Watch, do not switch. -5. **Humanoid hardware ecosystem broadened in 2026 Q1 without producing a credible second-platform integration target with mature Linux SDK on par with Unitree G1.** Booster T1 (open SDK, ROS2-compatible) is the closest. AgiBot G2/A2 has scale but limited Western support documentation. Fourier GR-3 announced late 2025, details emerging. H3 + H7 hold; if a second platform is needed, Booster T1 is the most aligned candidate. +**判断 2：LeRobot v0.5.0 第三方策略插件机制公开两个月后，生态里只有 1 份真实第三方实现 (`lerobot_policy_ditflow`)，H8 成立但通道几乎闲置。** +[Bring Your Own Policies](https://huggingface.co/docs/lerobot/bring_your_own_policies) 文档把 DiTFlow 列为唯一示例；PyPI 上没有第二个 `lerobot_policy_*` 包；NVIDIA、Physical Intelligence、Booster、Unitree 任何一方都没有以这个形态发包。这意味两件事：(1) **窗口** — robowbc 若以 `lerobot_policy_robowbc` 占位，能成为生态里第一个 humanoid WBC 多 backend 插件；(2) **机制本身可能没人需要** — LeRobot 用户更倾向直接 fork 上游或用核心库内置策略。**行动**：把 `lerobot_policy_robowbc` 实现到 MVP，但保留 maturin 独立发布路径（roadmap #41 应该是双路径，不是单路径）。 -**Specific recommendations for `robowbc`** (full version in §6): +**判断 3：真正可能在 6–12 个月内挤压 robowbc 价值的不是 NVIDIA 自做，也不是 LeRobot 整体，而是 `gear_sonic_deploy` C++ 推理栈被其它策略族复用。** +按 [DeepWiki/NVlabs/GR00T-WholeBodyControl](https://deepwiki.com/NVlabs/GR00T-WholeBodyControl/6.2-tensorrt-inference-engine) 字面描述，`gear_sonic_deploy` 已是 "high-performance C++ backend … responsible for executing SONIC and Decoupled WBC policies with sub-millisecond latency"，实现了 TensorRT/FP16/CUDA Graph/Stream 管理生产级套件，29 DoF 输出在 50 Hz 主循环 + 10 Hz kinematic planner 层级跑通。如果 BFM-Zero、Psi-Zero 或 UnifoLM-VLA 直接挂在 `gear_sonic_deploy` PolicyEngine 上，robowbc 多 backend 故事就成"另一个不必要的中间层"。 +**行动**：把 `gear_sonic_deploy` PolicyEngine 接口语义抄一份到 `robowbc-ort` 注释，标明 robowbc 选 Rust 的差异化理由是"非 NVIDIA-GPU 部署 / 非 TensorRT 路径 / 多 policy registry / 离线 proof pack"，**不**是性能。 -1. **Refresh the `gear_sonic` wrapper against the 2026-02-19 official release and the C++ deployment stack** — surface area expanded considerably after the SONIC formal release. -2. **Ship `robowbc-py` as a LeRobot third-party policy plugin** (`pip install lerobot_policy_robowbc` shape) to align with the v0.5.0 plugin mechanism and shorten the integration path. -3. **Add MotionBricks, Newton, mjlab, Booster T1 to the vectors card entity lists** during the next quarterly card audit (currently scheduled 2026-08-21). -4. **Read LeRobot's `HolosomaLocomotionController` design carefully** — it's the closest "competing one-contract" integration in the ecosystem and may reveal patterns worth adopting (or deliberately diverging from). -5. **Decide whether MuJoCo-Warp / Newton / mjlab is the right next substrate for `make showcase-verify` proof-pack throughput** — current CPU MuJoCo path is the bottleneck for substantive benchmark depth. +**判断 4：仿真底层在 2026-Q1 完成了从 MJX 到 MuJoCo-Warp/Newton 的代际跳跃，H6 假设在容量上仍有效，但视觉/保真度基线被对手抬高。** +[NVIDIA Developer Blog](https://developer.nvidia.com/blog/announcing-newton-an-open-source-physics-engine-for-robotics-simulation/) 与 [Linux Foundation press](https://www.linuxfoundation.org/press/linux-foundation-announces-contribution-of-newton-by-disney-research-google-deepmind-and-nvidia-to-accelerate-open-robot-learning) 确认 Newton 在 2025-09-29 进入 LF，2026 GTC 发布 Newton 1.0。NVIDIA 官方 Technical Blog ([Newton Adds Contact-Rich Manipulation and Locomotion](https://developer.nvidia.com/blog/newton-adds-contact-rich-manipulation-and-locomotion-capabilities-for-industrial-robotics/), 2026-03) 字面写到 "MuJoCo Warp speeds up MJX by **252x for locomotion**, and **475x for manipulation tasks** on the NVIDIA RTX PRO 6000 Blackwell Series" — 这是 NVIDIA 自证的厂商基准，但已是 A-tier 一手主源。[Isaac Lab 3.0 Beta](https://github.com/isaac-sim/IsaacLab/releases/tag/v3.0.0-beta) 把 `.data.*` 从 `torch.Tensor` 切到 `wp.array` — breaking change。[mjlab arXiv 2601.22074](https://arxiv.org/abs/2601.22074) 提供 Apache-2.0 轻量替代。 +**行动**：在 2026-08 checkpoint 前不必切换 `robowbc-sim`；先在 CI 跑 mjlab 的 `Mjlab-Velocity-Flat-Unitree-G1` baseline。 -**Still uncertain, requires next checkpoint to verify:** - -- Does GR00T N1.7's `UNITREE_G1_SONIC` end-to-end VLA workflow make robowbc's GEAR-SONIC wrapping redundant for the specific G1+SONIC case, or does it preserve a meaningful runtime abstraction? -- What is the exact policy-schema delta between LeRobot v0.4.x and v0.5.0? Does it break the `robowbc-py` adapter shape implied in roadmap item #41? -- Has Booster T1's SDK + URDF/MJCF reached parity with Unitree G1 sufficient to be a second integration target? +**判断 5：人形硬件出现明确"第二平台"信号——Booster T1 + Booster Robotics SDK 已具备和 G1 同级对外可见度，Linux SDK 同构。** +[Booster Robotics 开源页](https://www.booster.tech/open-source/) 显示 T1 提供 SDK + ROS2 SDK，基于 DDS Publisher-Subscriber，与 unitree_sdk2 同构。[booster_gym arXiv 2506.15132](https://arxiv.org/html/2506.15132v1) 给出实测往返通信延迟 9–12 ms，policy inference <1 ms。同时 [AgiBot G2](https://interestingengineering.com/ai-robotics/agibot-g2-humanoid-robot-smart-manufacturing) 是 wheeled humanoid (26 DoF + 7 DoF arms)，与 robowbc bipedal whole-body 假设不兼容。 +**行动**：Booster T1 加入 vectors card "second-platform 候选"；AgiBot 暂作 watch。 --- -## 2. Background: robowbc current state and constraints - -**Project positioning** (one-sentence): Linux-first embedded runtime for humanoid whole-body-control policy inference; one caller-facing contract, multiple policy backends, TOML-config-driven backend selection. - -**Current stack (May 2026, actual state of the repo)**: - -- Core contracts: `Observation`, `WbcCommand`, `JointPositionTargets`, `WbcPolicy`, `PolicyCapabilities`, `RobotConfig` -- Rust crate map (14 crates): `robowbc-core`, `robowbc-config`, `robowbc-registry`, `robowbc-ort`, `robowbc-pyo3`, `robowbc-py`, `robowbc-runtime`, `robowbc-comm`, `robowbc-transport`, `robowbc-sim`, `robowbc-teleop`, `robowbc-vis`, `robowbc-cli`, `unitree-hg-idl` -- Inference: ONNX Runtime via `ort-rs` (primary); PyO3 + PyTorch (user-supplied) -- Transport: CycloneDDS / in-memory; Rerun for visualization -- Sim: MuJoCo (CPU); no GPU substrate currently -- Integration target: Unitree G1 -- Distribution: maturin-built standalone Python SDK in `crates/robowbc-py`; CLI binary; public reports site at +## 2. Background -**Live policy backends**: `gear_sonic`, `decoupled_wbc`, `wbc_agile`, `bfm_zero` -**Blocked / experimental wrappers**: `hover` (no public pretrained checkpoint), `wholebody_vla` (no runnable upstream), `py_model` (user-supplied) +不重复 v1 §2（项目定位、stack、live wrappers）。v2 新增的项目状态变化： -**Known technical constraints**: - -- Linux only — runtime backends fail fast on unsupported platforms -- Joint position targets as the output boundary; no direct torque output exposed to callers -- ONNX is the canonical model interchange; PyTorch only via `py_model` -- Public assets only — wrappers without runnable public checkpoints are marked Blocked -- Static reports site (HTML + JSON + Rerun .rrd) as the primary proof surface, not a daemon API - -These constraints drive every judgment below. +- BFM-Zero 已于 [arXiv 2511.04131](https://arxiv.org/abs/2511.04131) 与 [openreview](https://openreview.net/forum?id=jkhl2oI0g5) 公开 paper + ICLR 2026 接收；`robowbc/bfm_zero` 应把 license=**CC BY-NC 4.0** 写到 third-party-notices。 +- UnifoLM-VLA-0 已找到一手主源 [unitreerobotics/unifolm-vla](https://github.com/unitreerobotics/unifolm-vla)，v1 的 `[未充分验证]` 可去掉，license = **CC BY-NC-SA 4.0**（比 BFM-Zero 还紧）。 +- Isaac-GR00T 已走到 N1.7 Apache-2.0 + 商用授权，N2 在 GTC 2026 预览（见 [NVIDIA Newsroom](https://nvidianews.nvidia.com/news/nvidia-expands-open-model-families-to-power-the-next-wave-of-agentic-physical-and-healthcare-ai)），底层架构换成 "world action model" + DreamZero，声称 MolmoSpaces 与 RoboArena #1，但**未发布**。 +- Unitree IPO 关键数据现已有 A-tier 复核 — CNBC (2026-03-20) 报道 Unitree 申报 Shanghai IPO 募资 **¥4.2B (US$610M)**，2025 年共**出货 5,500+ G1 units、占全球人形机器人市场 32.4%**，2025 营收 ¥1.71B (+335% YoY)。v1 的 B-tier 数字与 A-tier 一致。 --- -## 3. Ecosystem panorama +## 3. Ecosystem panorama by vector -Organized by the six research vectors in the vectors card. +### 3.1 Vector 1: WBC policy upstream -### 3.1 Vector 1: Humanoid WBC policy upstream +#### 3.1.1 GR00T-WholeBodyControl 与 GEAR-SONIC -**Status: high churn. Q1 2026 saw the largest single quarter of releases robowbc has observed in this domain.** +Mar–May 2026 关键事件全部在 [NVlabs/GR00T-WholeBodyControl Documentation](https://nvlabs.github.io/GR00T-WholeBodyControl/) 一手时间线： -#### 3.1.1 NVlabs/GR00T-WholeBodyControl — formalized - -| Event | Date | Why it matters for robowbc | +| Date | Event | 对 robowbc 含义 | |---|---|---| -| Initial release (Decoupled WBC for N1.5/N1.6) | 2025-11-12 | Anchored robowbc's `decoupled_wbc` wrapper | -| GEAR-SONIC formal release: pretrained checkpoints, **C++ inference stack**, VR teleoperation, docs | 2026-02-19 | Upstream now ships its own C++ deployment (`gear_sonic_deploy`, G1Deploy app) — robowbc's `gear_sonic` wrapper now overlaps with an official deployment path. | -| SONIC training code + HF checkpoint released | 2026-04-10 | "Train from scratch or finetune" is now public; users no longer need robowbc to access SONIC weights — only to deploy them under a multi-backend contract. | -| Live SONIC web demo (Kimodo text-to-motion) | 2026-04-14 | Interactive proof surfaces appearing upstream; raises the bar for robowbc's static reports site (H6 concern). | -| MotionBricks public release: VQVAE + pose + root checkpoints, synthetic training code | 2026-04-27 | New policy-class candidate. Real-time latent generative motion control — not currently wrapped by robowbc. | -| End-to-end VLA workflow on G1 (collect teleop → finetune N1.7 → deploy with SONIC) | 2026-05-07 | The `UNITREE_G1_SONIC` embodiment tag wires the VLA → SONIC pipeline. Per [issue #108](https://github.com/NVlabs/GR00T-WholeBodyControl/issues/108), N1.6's 17D upper-body joint output is for Decoupled WBC; SONIC expects 14D Cartesian wrist poses — boundary shapes differ by family. | - -**Counter-evidence to H1 (one-contract abstraction):** The 14D-wrist vs 17D-joints VLA → WBC mismatch in issue #108 is at the VLA → WBC interface, not at the WBC → joint targets interface where robowbc operates. **H1 still holds for robowbc's specific layer**, but the result confirms the contract is layer-specific, not universal. - -#### 3.1.2 BFM-Zero (LeCAR Lab, CMU) - -- arXiv 2511.04131; **accepted to ICLR 2026** (poster, Apr 24); G1-deployed; unsupervised RL + Forward-Backward models -- Already wrapped by robowbc as `bfm_zero` -- License: CC BY-NC 4.0 (non-commercial) — usage constraint worth noting in robowbc's third-party-notices - -#### 3.1.3 Academic / open humanoid line (new entrants not in current vectors card) - -| Project | Source | Status | Notes | -|---|---|---|---| -| **GMR / GMT** (General Motion Retargeting / Tracking) | YanjieZe/GMR, ICRA 2026 | Active | Supports 11 humanoid embodiments incl. G1, H1, Booster T1/K1 — useful retargeting reference | -| **KungfuBot / PBHC / KungfuBot2 / VMS** | arXiv 2506.12851, 2509.16638 | Active | Physics-based humanoid control, motion tracking with versatile skills | -| **APEX** (Adaptive High-Platform Traversal) | arXiv 2026-02 | Active | Specific to high-platform locomotion | -| **TWIST / CLONE / BumbleBee** | Recent papers | Active | Teleop-focused tracking; lower-dynamic motions | -| **Psi-Zero** (USC PSI Lab, separate from Physical Intelligence π series) | physical-superintelligence-lab/Psi0, RSS 2026 | Active | Whole-body humanoid VLA; ships with SIMPLE benchmarking sim (MuJoCo + Isaac Sim rendering); 6 pre-collected G1 loco-manipulation tasks on HF | -| **HumanoidVerse** | LeCAR-Lab | Active | IsaacGym + Genesis + IsaacLab unified training framework, G1 + H1 + 23/29 DoF support | - -These are not currently in the vectors card's entity list — should be added at the next card audit. - -#### 3.1.4 Physical Intelligence (π series) - -- **π-0.7 released April 2026** (per KraneShares industry note, B-tier): performed unfamiliar tasks (e.g., operating an air fryer seen only twice in training) by composing web pretraining with sparse demonstrations. -- Still bimanual-manipulation focused per available evidence; no humanoid WBC release confirmed. Counter-evidence to H1 candidate if a future π-X ships a whole-body model with a non-`predict → joint targets` shape. - -#### 3.1.5 Unitree-native VLA - -- **UnifoLM-VLA-0 open-sourced March 2026** (per botinfo.ai B-tier reporting `[未充分验证]` — need to find primary repo) -- If real, this is a new upstream from the hardware vendor itself. Affects H1 (competing interface watch) and H8 (LeRobot's role). - -### 3.2 Vector 2: Robot ML inference runtime stack - -**Status: stable core, one new credible substitute path.** +| 2026-02-19 | GEAR-SONIC 正式发布，含 C++ inference stack + VR teleop + docs | `robowbc/gear_sonic` 不再是唯一 Python 入口，robowbc-ort 与上游 C++ 栈功能重叠 | +| 2026-03-16 | BONES-SEED 开源 (142K+ human motions, ~288h, G1 MuJoCo trajectories) | 影响 Vector 5；提升 fine-tune SONIC 可重复性 | +| 2026-03-24 | C++ stack update: motor error monitoring / TTS alerts / **ZMQ protocol v4** / **idle-mode readaptation**, ZMQ header 1280 字节 | wire format 60 天内已迭代；header size 差异是兼容性 trap | +| 2026-04-10 | SONIC 训练代码 + HF checkpoint 公开 | 用户绕过 robowbc 直接拿 HF 权重 | +| 2026-04-14 | Live Kimodo text-to-motion web demo | interactive proof surface 出现，抬高对 robowbc 静态 reports site 的对比基线 | +| 2026-04-27 | MotionBricks preview（VQVAE + pose + root, interactive G1 demo） | 新策略类候选，未 wrap | +| 2026-05-07 | End-to-end VLA workflow on G1: teleop → fine-tune N1.7 → SONIC deploy，通过 `UNITREE_G1_SONIC` embodiment tag | latent action tokens 端到端链路落地，直接挑战 H1 | + +**接口语义**（基于 [DeepWiki](https://deepwiki.com/NVlabs/GR00T-WholeBodyControl/6.2-tensorrt-inference-engine) 引用 `gear_sonic_deploy/include/policy_engine.hpp` 与 `g1_deploy_onnx_ref.cpp` 字面描述）： +- HF 上 `nvidia/GEAR-SONIC` 目录含 `model_encoder.onnx`、`model_decoder.onnx`、`planner_sonic.onnx` 加 `observation_config.yaml`。 +- 主控制循环 **50 Hz**，KinematicPlanner **10 Hz**——robowbc 调度器需对齐。 +- Observation → EncoderEngine → 固定大小 latent tokens → PolicyEngine → **29-DOF joint position targets**。 +- TensorRT FP16 (`kHALF`) + CUDA Graph capture + memory pinning，声称 sub-millisecond inference。 +- License: 代码 Apache 2.0；权重 NVIDIA Open Model License。 + +#### 3.1.2 BFM-Zero (LeCAR-Lab, CMU) + +[arXiv 2511.04131](https://arxiv.org/abs/2511.04131) + [GitHub LeCAR-Lab/BFM-Zero](https://github.com/LeCAR-Lab/BFM-Zero)： +- Action `a ∈ ℝ²⁹`，verbatim："the action a ∈ A ⊂ ℝ²⁹ contains the proportional derivative (PD) controller targets for all DoFs"。 +- Privileged state `s ∈ ℝ⁴⁶³`；observable `o_t` 含 `{q_t − q̄, q̇_t, ω_root, ...}` + history。 +- 三种推理脚本（`tracking_inference`、`goal_inference`、`reward_inference`）**全部 export ONNX**，只差 latent z 的计算。 +- License: **CC BY-NC 4.0**，默认 robowbc distribution 必须"loader-only，不打包权重"。 + +#### 3.1.3 Psi-Zero (USC PSI Lab, RSS 2026) + +[physical-superintelligence-lab/Psi0](https://github.com/physical-superintelligence-lab/Psi0) baselines/README 显式区分两种 eval entrypoint： +- Task 以 `Teleop` 结尾 → `agent=psi0_decoupled_wbc`，**输出 14D 腕部 Cartesian pose** (xyz + quat × 2)，后接 Decoupled WBC。 +- Task 以 `MP` 结尾 → `agent=psi0`，CuRobo motion-planning 生成，**输出 joint-space trajectories**。 + +**同一模型家族在不同任务子集下走两种 codomain**——这是 H1 必须正面回应的反例。License: **Apache 2.0**（意外宽松，可打包）。 + +#### 3.1.4 NVIDIA Isaac-GR00T N1.7 / N2 + +[NVIDIA/Isaac-GR00T](https://github.com/NVIDIA/Isaac-GR00T) + [HF blog N1.7](https://huggingface.co/blog/nvidia/gr00t-n1-7)： +- Architecture: Cosmos-Reason2-2B (System 2) + 32-layer DiT (System 1)。Action chunking，默认 horizon 8 或 16。 +- Action 空间因 embodiment tag 而异： + - `UNITREE_G1` (N1.6 兼容) → 上半身 **17D joint angles**，送 Decoupled WBC。 + - `UNITREE_G1_SONIC` → **latent action tokens**，SONIC 解码到 29 DoF。 + - Default cross-embodiment → **Relative EEF deltas**。 +- License: **Apache 2.0**，Early Access 但商用允许。 + +GR00T N2 在 GTC 2026 预览，基于 "world action model" + DreamZero，声称 MolmoSpaces 与 RoboArena #1，**计划年底前发布**（未实际可用）。robowbc 的 wbc_agile / wholebody_vla 在 N2 发布前继续 blocked 是合理的。 + +#### 3.1.5 UnifoLM-VLA-0 (Unitree) + +v1 `[未充分验证]` 项，v2 找到一手 [unitreerobotics/unifolm-vla](https://github.com/unitreerobotics/unifolm-vla) + [HF UnifoLM-VLA-Libero](https://huggingface.co/unitreerobotics/UnifoLM-VLA-Libero)： +- 基于 Qwen2.5-VL-7B + Action Head，action chunking，forward/inverse dynamics constraints。 +- Action shape `(NUM_ACTIONS_CHUNK, ACTION_DIM)`，在 `constants.py` 配置，语义为 joint position。 +- 训练数据：Unitree 官方项目页 ([unigen-x.github.io/unifolm-vla.github.io](https://unigen-x.github.io/unifolm-vla.github.io/)) 字面写到 "**approximately 340 hours of high-quality real-robot data** for discrete action prediction training"。 +- **License: CC BY-NC-SA 4.0**（非商业 + ShareAlike），不能默认打包。 +- 代码引用 Isaac-GR00T、openvla-oft、InternVLA-M1，语义上与 GR00T 系列同构。 + +#### 3.1.6 其它 2026-Q1 新出现的研究项目 + +[arXiv 2603.05410 (2026-03)](https://arxiv.org/abs/2603.05410) PhysiFlow 与 [OpenDriveLab/WholebodyVLA (ICLR 2026)](https://github.com/OpenDriveLab/WholebodyVLA) 都把"latent VLA + low-level tracking"作为标配。这些不会立刻进入 wrap 名单，但共同确认趋势：**latent-action-as-output 是 2026 humanoid VLA 主流形态**，joint-target-as-output 越来越是 low-level 控制器（也就是 robowbc 占据的层）的事。 + +#### 3.1.7 Policy interface 大表（核心证据 for H1） + +| Project | Action shape | 语义 | Joint target? | Source | +|---|---|---|---|---| +| GEAR-SONIC | 29D | PD joint targets @ 50 Hz | ✅ | gear_sonic_deploy/policy_engine.hpp via DeepWiki | +| Decoupled WBC (NVlabs) | upper 17D + lower-body RL ≈ 29D | 上半身关节 + 下半身 RL → 全身 PD | ✅ | decoupled_wbc/control/policy/ | +| BFM-Zero | ℝ²⁹ | "PD controller targets for all DoFs" | ✅ | arXiv 2511.04131 §2 | +| **Psi-Zero (Teleop)** | 14D | **腕部 Cartesian (xyz + quat × 2)** | ❌ Cartesian | Psi0 baselines README | +| Psi-Zero (MP) | joint-space | CuRobo-derived joint trajectories | ✅ | 同上 | +| LeRobot HolosomaLocomotionController | dict of joints (29 或 23 DoF) | 全身 joint targets via send_action() | ✅ | docs/lerobot/v0.5.1/unitree_g1 | +| LeRobot GrootLocomotionController | dict of joints | upper-body pose + navigate → joint targets | ✅（在控制器内） | 同上 | +| GR00T N1.7 (UNITREE_G1) | 17D × chunk | 上半身关节角，Decoupled WBC | ✅ 部分 | Isaac-GR00T README | +| **GR00T N1.7 (UNITREE_G1_SONIC)** | latent tokens × chunk | **latent action tokens，SONIC 解码** | ❌ latent | Isaac-GR00T README | +| **GR00T N1.7 (default cross-embodiment)** | 6/7D × chunk | **Relative EEF deltas** | ❌ relative EEF | Isaac-GR00T README | +| UnifoLM-VLA-0 | (NUM_ACTIONS_CHUNK, ACTION_DIM) | joint position chunks | ✅ | unitreerobotics/unifolm-vla | +| MotionBricks (preview) | VQVAE latent → pose / root | latent | ❌ latent | NVlabs/GR00T-WBC motionbricks/ | +| PhysiFlow / WholeBodyVLA | latent flow / latent VLA | latent + tracking | ❌ latent | arXiv 2603.05410 | + +**结论**：H1 在 12 个对象里有 7 个是 joint position targets，但 5 个新趋势走 latent 或 relative EEF。**robowbc 的契约需要扩展为可配置 codomain**。 + +### 3.2 Vector 2: Inference runtime stack | Component | Version (May 2026) | Activity | |---|---|---| -| `onnxruntime` | 1.26.0 (2026-05-08); 1.25.0 (Apr 2026) | Active, monthly cadence. C++20 + CUDA 12+ minimum. | -| `ort-rs` | (Used by robowbc-ort) | Tracking onnxruntime closely | -| `executorch` | **1.0 GA (2025-10-22)** | **Substitute path candidate.** Native PyTorch export, no ONNX intermediate, ~50KB runtime, 12+ backends (ARM Ethos-U, Apple MPS, Qualcomm, Vulkan, MediaTek, OpenVINO, etc.). Production-proven at Meta (Quest 3, Smart Glasses). Robotics-specific adoption signal: none observed. | -| `candle` (HuggingFace) | ~15K stars | Steady. Minimalist Rust ML, focus on serverless + edge inference. | -| `burn` | ~8.5K stars | Backend-agnostic; CubeCL integration; WebAssembly + no_std support. | -| `tract` (Sonos) | — | Niche but stable; less HF integration | -| `mistral.rs` | 3.6K stars | LLM-focused; less relevant to humanoid WBC | -| `mujoco-warp` | Apache-2.0, Newton-integrated | Sim, not inference — but relevant to substrate side of H2 | - -**Verdict on H2 (Rust + PyO3 + ONNX Runtime is the right stack):** Holds. ONNX Runtime is still the dominant production path for PyTorch-trained checkpoints exported to ONNX, and `ort-rs` remains the standard Rust binding. ExecuTorch 1.0 is the real watch-target — its value proposition fits humanoid robotics well (edge deployment, small runtime, no ONNX conversion friction), but no humanoid project has publicly adopted it yet. **If GR00T-WBC's `gear_sonic_deploy` C++ stack or any subsequent humanoid vendor switches to ExecuTorch as the canonical inference path, robowbc-ort becomes one of two backends rather than the primary.** - -### 3.3 Vector 3: Humanoid hardware + SDK ecosystem - -#### 3.3.1 Unitree (current integration target) - -| Platform | Status | Notes | -|---|---|---| -| G1 | Active; firmware 1.4.5 (2025-11-13) added Training Mode | 23 & 29 DoF variants. 5,500+ units shipped 2025 (B-tier). 16 configurations $16K–$73.9K. | -| H1 / H1-2 | Active | Older platform; LeRobot integration also includes H1 | -| H2 | Launched | Less documented publicly than G1 | -| R1 | Pre-sale; ships April 2026 | $4.9K–$5.9K. Smaller athletic humanoid. Less of a research platform. | -| **Unitree IPO** | Filed Shanghai $610M, March 2026 (B-tier) | 335% YoY revenue growth 2025. A-share listing expected mid-2026. China's first publicly traded humanoid robotics company. | - -**Relevance to robowbc:** H7 ("Unitree G1 + unitree_sdk2 + CycloneDDS is the right hardware integration target") is reinforced. LeRobot's G1 integration explicitly uses `unitree_sdk2py==1.0.1` + `cyclonedds==0.10.2` — the same transport choice robowbc made. - -#### 3.3.2 Chinese humanoid wave — Linux SDK quality varies - -| Platform | SDK quality (per available evidence) | Plausibility as robowbc second-platform target | -|---|---|---| -| **Booster T1** | Open SDK, ROS2 compatible, public URDF, supported in GMR | **Highest** — closest to Unitree G1 in profile | -| **AgiBot G2 / A2** | NVIDIA partnership; Chinese-market focused; Western SDK documentation thin | Medium — viable but research-friction | -| **Fourier GR-3** | Announced late 2025, "details emerging" | Low (today) — wait for SDK | -| EngineAI SE01 / T800 | Aggressive pricing; SDK details limited | Low | -| XPeng IRON | Closed | None | -| UBTECH Walker | Closed-ish | None | - -GMR (General Motion Retargeting) supports 11 humanoid embodiments including Booster T1 (23 & 29 DoF) and Booster K1 — this is a useful signal that Booster has shippable URDF/MJCF. - -#### 3.3.3 US humanoid wave — mostly closed - -Figure 02/03 (closed-source), Boston Dynamics Atlas (no public customer API), Apptronik Apollo, 1X NEO, Tesla Optimus (not for sale). No platform has opened a Linux SDK in Q1 2026 that materially affects H3 / H7. - -### 3.4 Vector 4: Robot middleware - -**Status: CycloneDDS remains dominant for ROS 2 ecosystem; Zenoh adoption growing but no displacement.** +| `onnxruntime` | **1.26.0 released 2026-05-08**（PyPI 时间戳）；1.25.0 (Apr) | 月度活跃 | +| `ort-rs` | 跟随 ORT | robowbc-ort 依赖，稳定 | +| `executorch` | 1.0 GA (2025-10-22) | Meta Quest 3 / Smart Glasses 生产部署；**humanoid robotics 采用信号 0** | +| `candle`, `burn`, `tract` | 维护中 | 无 humanoid 项目采用证据 | +| `gear_sonic_deploy` C++ + TensorRT | 月度迭代 (ZMQ v4 2026-03-24) | **替代 robowbc-ort 的实质候选** | + +**H2 audit**： +- ExecuTorch 真实采用：[executorch.ai](https://executorch.ai/) 列出 LiquidAI、Software Mansion、Meta Quest 3、Ray-Ban Meta——**全部是手机 / AR-VR / 边缘 AI，无一个 humanoid 项目**。原因：(a) PyTorch→ONNX→TensorRT 是 NVlabs 生产路径，ExecuTorch 在 NVIDIA GPU 无性能优势；(b) ExecuTorch 50KB runtime 卖点对 Jetson 级别 onboard 无意义。 +- Rust ML library humanoid 项目采用：**无公开证据**。 +- 真正的 H2 风险来自 `gear_sonic_deploy` C++ 路径，不是 ExecuTorch。 + +**v2 verdict**：H2 holds 但**风险维度变了**——v1 把 ExecuTorch 当主要 watch，v2 把 `gear_sonic_deploy` C++ + TensorRT 当主要 watch。 + +### 3.3 Vector 3: Hardware + SDK + +#### Unitree +- G1 + unitree_sdk2 + CycloneDDS 0.10.2 是 [docs/lerobot/v0.5.1/unitree_g1](https://huggingface.co/docs/lerobot/v0.5.1/unitree_g1) 显式记录的对接组合；CycloneDDS 选择被第三方 framework 同步背书。 +- Unitree 开源了 UnifoLM-VLA-0（2026-03 公布）与 unifolm-world-model-action，把 Qwen2.5-VL-7B 作为基础。Unitree 第一次自产 VLA 级 stack。 +- **Unitree IPO 数据 (A-tier 复核完毕)**：CNBC (2026-03-20) 报道 Unitree 申报 Shanghai IPO 募资 **¥4.2B (US$610M)**，2025 年共**出货 5,500+ G1 units、占全球人形机器人市场 32.4%**，2025 营收 ¥1.71B (+335% YoY)。v1 的 B-tier 数字与 A-tier 一致。 + +#### Booster T1 — 第二平台候选 #1 +- [Booster 开源页](https://www.booster.tech/open-source/) + [booster_gym arXiv 2506.15132](https://arxiv.org/html/2506.15132v1)： + - 完整 SDK (DDS Pub-Sub) + ROS2 SDK + Isaac Lab-based RL framework + 轻量 Sim2Real/Sim2Sim 部署 framework。 + - 实测 round-trip 通信延迟 9–12 ms，policy inference <1 ms。 + - URDF / MJCF / 23 或 31 或 41 DoF（依手部 SKU）公开。 + - NVIDIA Isaac 集成（[Booster Medium](https://boosterobotics.medium.com/booster-robotics-powers-humanoid-intelligence-with-nvidia-d1d1df44043d)）。 +- 一手价格：€7,000+ 起（[Generation Robots](https://www.generationrobots.com/en/404278-booster-t1-humanoid-robot.html)）。 +- **结论**：SDK 同构度 ≈90% 于 Unitree，URDF 完整，RL stack IsaacLab 系。 + +#### AgiBot G2 — wheeled，不在 robowbc 主线 +- [AgiBot G2 报道](https://interestingengineering.com/ai-robotics/agibot-g2-humanoid-robot-smart-manufacturing) + [AgiBot WORLD 2026 dataset 公告](https://www.therobotreport.com/agibot-world-2026-dataset-open-source-accelerate-embodied-ai-development/)： + - **Wheeled** humanoid（26 DoF + 7 DoF arms），不是 bipedal whole-body。 + - 软件栈：GO-2（behavioral foundation model）、GE-2（action world model）、Genie Sim 3.0（Isaac Sim-based）、**AimRT**（自研 C++20 中间件）、AgiBot World 2026 dataset。 + - **AimRT 是新中间件实体**，跨厂家无采用证据。 + +#### 其它 +- Fourier GR-3、XPeng IRON、UBTECH：本次未取得 Mar–May 2026 Linux SDK 实质性进展证据。 +- US 阵营（Figure, BD Atlas, 1X NEO, Apptronik）：closed。 +- ENGINEAI T800 CES 2026 亮相，无公开 SDK / URDF。 + +### 3.4 Vector 4: Middleware | Component | State (May 2026) | |---|---| -| `cyclonedds` 0.10.2 | Used by Unitree SDK + LeRobot G1 integration | -| `rmw_zenoh` | ROS 2 Jazzy + Rolling binary packages available; **not** ROS 2 default | -| `zenoh-plugin-ros2dds` | Recommended bridge for ROS 2 + Zenoh interop; will eventually deprecate `zenoh-bridge-dds` | -| ROS 2 Jazzy / Kilted | Jazzy is current LTS; Kilted in active development | -| Rerun | Active, used by robowbc; visualization-only | +| `cyclonedds` 0.10.2 | Unitree、LeRobot G1、Psi0 都用 | +| `rmw_zenoh` | Jazzy + Rolling 二进制可用，**仍非默认 RMW**（[docs.ros.org/en/jazzy](https://docs.ros.org/en/jazzy/Installation/RMW-Implementations/Non-DDS-Implementations/Working-with-Zenoh.html) 分类仍是 "Non-DDS-Implementations"） | +| `zenoh-plugin-ros2dds` | 维护，DDS 桥而非替换 | +| AgiBot AimRT | 新自研 C++20 中间件，单厂私有 | -**Verdict on H7:** Holds. No major humanoid vendor committed to ROS 2 native as the only customer API; Zenoh has not become the ROS 2 default RMW. The parking-lot stance on ROS 2 native customer API remains correct. +**H7 audit**： +- 没有 humanoid 厂家公开宣布 ROS-2-native-only 客户 API。 +- Zenoh 没进入 ROS 2 默认 RMW；`zenoh-plugin-ros2dds` 是兼容桥。 +- Unitree 在新发布中**没有**改变 CycloneDDS 0.10.2 组合。 -### 3.5 Vector 5: Sim substrate +**v2 verdict**：H7 holds 且**比 v1 更稳**。robowbc CycloneDDS 选择被多家（LeRobot, Unitree, Psi0, GR00T-WBC 内部）用同样组合佐证。**小动作**：把 ROS 2 native API 从"parking lot"细化为"仅在某具体 humanoid 平台 SDK 切换信号时触发"。 -**Status: major substrate-layer event during GTC 2026.** +### 3.5 Vector 5: Sim substrate -| Component | Version / Status (May 2026) | Significance | +| Component | Status | Significance | |---|---|---| -| MuJoCo | 3.x (3.8.0 released with MJX 2026-04-24); stable production | Still the reference CPU sim | -| MuJoCo XLA (MJX) | 3.8.0 (2026-04-24) | JAX backend; production stable | -| **MuJoCo-Warp (MJWarp)** | Active, integrated as Newton solver | **Performance breakthrough**: 152x faster than MJX for locomotion, 313x for manipulation on RTX 4090 (NVIDIA-claimed, A-tier) | -| **Newton** | Beta; Apache-2.0; Linux Foundation | **NEW.** Jointly developed by NVIDIA + Google DeepMind + Disney Research. MuJoCo-Warp is the main solver. | -| **Isaac Lab 3.0 Beta** | Released GTC 2026 | Multi-backend physics (PhysX + Newton/MuJoCo-Warp), pluggable renderer, kit-less install, Warp-native data pipelines (`wp.array` instead of `torch.Tensor` — breaking change for downstream code) | -| **mjlab** | arXiv 2026 (Berkeley/DeepMind); active | Lightweight Isaac-Lab-style API directly on MuJoCo-Warp. Minimal dependencies, no Isaac Sim required. Strong alternative for researchers who want MuJoCo-Warp performance without NVIDIA-stack dependencies. | -| **Genesis** (Genesis-Embodied-AI) | 0.4.7 (still pre-1.0) | Genesis 1.0 has **not** shipped. Genesis-Humanoid (UMass Embodied AGI) extends with humanoid-specific tooling: 200k RL steps/sec, real-time retargeting, 50ms end-to-end teleop. | -| ManiSkill3 | Stable | Manipulation-focused, less WBC-relevant | -| RoboCasa | Active | Integrated into GR00T-WBC | -| Cosmos 3 | Released GTC 2026 | NVIDIA's world-model line; not yet evaluated for WBC use | +| MuJoCo 3.x | 稳定参考 CPU sim | baseline | +| MJX (3.8.0, 2026-04-24) | JAX 后端 | 旧 GPU 路径 | +| **MuJoCo-Warp** | Newton 主 solver | NVIDIA Technical Blog ([Newton Adds Contact-Rich Manipulation and Locomotion](https://developer.nvidia.com/blog/newton-adds-contact-rich-manipulation-and-locomotion-capabilities-for-industrial-robotics/), 2026-03) 字面写到 MuJoCo Warp 在 **RTX PRO 6000 Blackwell** 上比 MJX 快 **252× (locomotion) / 475× (manipulation)** | +| **Newton 1.0 (LF)** | Beta → 1.0 at GTC 2026; Apache-2.0 | [linuxfoundation.org press](https://www.linuxfoundation.org/press/linux-foundation-announces-contribution-of-newton-by-disney-research-google-deepmind-and-nvidia-to-accelerate-open-robot-learning) + [newton-physics/newton](https://github.com/newton-physics/newton); Multi-solver (MuJoCo Warp + Kamino VBD + Featherstone + Euler) | +| **Isaac Lab 3.0 Beta** | Released GTC 2026 | [v3.0.0-beta release](https://github.com/isaac-sim/IsaacLab/releases/tag/v3.0.0-beta)：`.data.*` `torch.Tensor` → `wp.array`（breaking）; quaternion wxyz → xyzw（breaking）; factory-based multi-backend; PresetCfg | +| **mjlab** | [arXiv 2601.22074](https://arxiv.org/abs/2601.22074), Apache-2.0 | UC Berkeley + DeepMind；Isaac Lab API on MuJoCo-Warp；Unitree G1 + Go1 + YAM 资产 ship | +| Genesis | 0.4.7 pre-1.0 | 1.0 仍未 ship | +| Cosmos 3 | GTC 2026 发布 | 未评估 | + +**H6 audit**：Static reports site 对比基线被三件事抬高： +1. GEAR-SONIC Live Kimodo web demo（interactive） +2. MotionBricks interactive G1 demo +3. UnifoLM-VLA-0 项目页演示 + +但**没有** humanoid WBC 项目把 interactive web demo 作为唯一 proof。robowbc "static reports + Rerun .rrd + proof packs" 未被显著 outpace，**视觉对比基线在上升**。 + +**v2 verdict**：H6 holds，2026-08 前 prototype mjlab GPU sim 提速实验。 + +### 3.6 Vector 6: Python distribution + +#### LeRobot v0.5.0 / v0.5.1 +[LeRobot v0.5.0 blog](https://huggingface.co/blog/lerobot-release-v050)：ICLR 2026 接收 + Unitree G1 第一类 humanoid + 第三方策略插件机制 + Pi0-FAST + Real-Time Chunking + EnvHub + IsaacLab-Arena + Python 3.12 + Transformers v5。v0.5.1 (2026-04-07, [PyPI](https://pypi.org/project/lerobot/)) 修了 Unitree G1 SDK detection + 安装文档。 + +**第三方策略插件实际使用情况**： +- [docs/bring_your_own_policies](https://huggingface.co/docs/lerobot/bring_your_own_policies) 列唯一示例 [danielsanjosepro/lerobot_policy_ditflow](https://github.com/danielsanjosepro/lerobot_policy_ditflow)（DiTFlow：DiT + Flow-matching）。 +- PyPI 上没有第二个 `lerobot_policy_*` 包。 +- NVIDIA、Physical Intelligence、Booster、Unitree、Boston Dynamics 任何一方没有 ship 第三方插件。 +- **公开但闲置的通道**。robowbc 占位即第一个 humanoid WBC 多 backend 插件示例，生态价值 > 直接用户量。 + +#### 其它 Python-first 发行通道（H8 对照） +- **NVIDIA Isaac-Lab-Arena**：Isaac Lab 之上的 policy 评估 framework，绑定 NVIDIA stack。 +- **mjlab**：training framework 而非 distribution。 +- **ManiSkill3, Habitat 3, robosuite**：manipulation / embodied AI 训练 framework，不是 humanoid WBC 主战场。 +- **openpi（Physical Intelligence）**：π 系列发行，manipulation 为主。 +- **AgiBot Genie Sim 3.0**：AgiBot 自有 sim，wheeled 为主。 + +**v2 verdict**：H8 holds（LeRobot 仍是 Python-first 主分发渠道），但 v1 暗示的"应该尽快 ship `lerobot_policy_robowbc`"紧迫度可下调——第三方插件机制的网络效应还没起来。**关键是双发行**：同时 ship maturin 独立包和 `lerobot_policy_robowbc` 适配壳。 + +### 3.7 Hidden assumption audit (v2) + +| # | Assumption | v1 | v2 | Key evidence delta | Action | +|---|---|---|---|---|---| +| H1 | One contract covers all WBC families | Holds | **Holds 仅在 joint-target codomain；在 latent / relative-EEF 下被反例化** | Isaac-GR00T README 明示 UNITREE_G1_SONIC = latent；Psi-Zero teleop = 14D Cartesian | 把 `WbcPolicy` 扩成三分支 trait | +| H2 | Rust + PyO3 + ONNX | Holds, watch ExecuTorch | **Holds, real watch 移到 `gear_sonic_deploy`** | ExecuTorch humanoid 采用 0；`gear_sonic_deploy` production-grade | robowbc-ort README 注明差异化论证 | +| H3 | Linux-first fail-fast | Holds | **Holds** | 所有新 humanoid SDK Linux-first | 无变更 | +| H4 | Joint targets, not torque | Holds | **Holds, 边缘风险变小** | SATA 仍局限四足 + 早期 | 无变更 | +| H5 | Multi-family registry | Strengthens | **Strengthens** | Mar–May 新增 PhysiFlow/WholeBodyVLA/MotionBricks/UnifoLM | registry 设计 codomain 一等枚举 | +| H6 | Static reports primary | Holds, revisit | **Holds, 视觉基线抬高** | Kimodo + MotionBricks + UnifoLM demos | 2026-08 prototype mjlab | +| H7 | G1 + CycloneDDS, ROS2 parked | Holds | **Holds 更稳** | LeRobot/Psi0/Booster 同组合；Zenoh 仍非默认 | parking 触发条件细化 | +| H8 | LeRobot distribution channel | Strengthens, urgency 高 | **Holds, urgency 下调** | 第三方插件机制 2 个月后只 1 plugin | 双路径发行 | + +**H1 / H8 是 v2 相对 v1 最大的两个判断变化**。 -**Disambiguation note:** "Genesis AI" (a humanoid robotics company that released GENE-26.5 on 2026-05-06) is a **different entity** from Genesis-Embodied-AI (the open simulator). Easy to confuse; track separately. - -**Verdict on H6 (static reports site as primary proof surface):** Holds for now, but the substrate fidelity ceiling rose ~100x during Q1 2026. The bottleneck for robowbc's proof-pack depth is shifting from "we can only run N MuJoCo CPU scenarios per minute" to "we could run 10,000+ MuJoCo-Warp scenarios per minute if we adopt the new substrate." Interactive demos appearing upstream (SONIC web demo, GENE-26.5 demos) raise the comparison bar. **Recommend revisit at 2026-08 checkpoint.** +--- -### 3.6 Vector 6: Python-first distribution + interface peers +## 4. Blind spots -#### 3.6.1 LeRobot v0.5.0 (2026-03-09) — pivotal release for robowbc +v1 §3.7 末尾说"all eight assumptions found some discriminating evidence"——v2 不同意。两个 sub-dimension 在 v1 完全没出现： -| Feature | Why it matters for robowbc | -|---|---| -| **First humanoid integration = Unitree G1** (23 & 29 DoF) | Validates H8: LeRobot is the right channel for Python-first robotics — and chose the same platform robowbc bet on. | -| Uses `unitree_sdk2py==1.0.1` + `cyclonedds==0.10.2` | Same transport choice as robowbc. Lowers compatibility friction for `robowbc-py` shipping as a LeRobot policy. | -| **Third-party policy plugins** (`pip install lerobot_policy_*`) | This is the exact distribution shape for `robowbc-py`. Roadmap #41 target: `lerobot_policy_robowbc`. | -| **HolosomaLocomotionController + GR00T-WBC native integration** | Direct competition for robowbc's G1+SONIC value proposition. LeRobot users can now go from LeRobot dataset → Pi0-FAST or GR00T-WBC policy → G1 without robowbc. **robowbc's distinct value must come from registry + multi-backend abstraction, not from being the only Python path to SONIC.** | -| Pi0-FAST autoregressive VLA + Real-Time Chunking | New policy classes; LIBERO-10 score 60.0 (lower than diffusion-based on compositional tasks) — autoregressive VLAs have a long-horizon weakness worth watching | -| EnvHub for HF-hosted sim envs | Could be a distribution surface for robowbc's proof-pack scenarios | -| NVIDIA IsaacLab-Arena integration | Couples LeRobot's training side to Isaac Lab | -| Python 3.12+, Transformers v5 | Affects `robowbc-py` Python version policy | -| ICLR 2026 paper accepted | Cements LeRobot's standing as the field's standard reference distribution | -| v0.5.1 (2026-04-07) follow-up: fixes Unitree G1 SDK detection, install docs | Active maintenance | +### 4.1 建议新增 V7：实时与安全 -#### 3.6.2 Interface peers (the H1 competition watch) +**为什么**： +- v1 把 "Linux-first" 当作 deployment posture，但**没有区分 generic Linux 与 PREEMPT_RT**。GEAR-SONIC 50 Hz / KinematicPlanner 10 Hz、Booster T1 9–12 ms 往返、Pi0-FAST 在 Physical Intelligence/UC Berkeley 的 [Real-Time Execution of Action Chunking Flow Policies (arXiv 2506.07339)](https://arxiv.org/abs/2506.07339) 字面给出 "n = 5 denoising steps, giving a model latency of **76ms for the baselines and 97ms for RTC**"——所有这些都对内核调度延迟敏感。robowbc README 没说是否要求 PREEMPT_RT，也没说 jitter 上限。 +- ISO 10218-1:2025 / ISO 10218-2:2025 已发布并吸收 ISO/TS 15066；**ISO 25785-1**（dynamically stable humanoids）在 2026 仍是 Working Draft，预计 2026–2027 发布（[theresarobotforthat 2026 文献](https://www.theresarobotforthat.com/blog/humanoid-robot-safety-standards-2026/)，`[需 A-tier 验证]`）。 +- `robowbc-runtime` 应明确选择是否 require PREEMPT_RT，把 control-loop 抖动作为可观察指标暴露在 proof pack。 -- **LeRobot `Policy` API**: framework-level, at the VLA/manipulation layer, not WBC. Different layer from `WbcPolicy`. -- **GR00T-WBC Policy Interface** (per DeepWiki): present in upstream, scoped to GR00T-WBC internals — not a community standard. -- **HolosomaLocomotionController** (LeRobot): a concrete integration class, not a contract abstraction. Not a direct competitor to `WbcPolicy` at the abstraction level. -- **ROS 2 humanoid control WG interface proposals**: no evidence of new community-standard interface emerging in Q1 2026. +**条目**：V7 "Real-time + safety certification" +- 跟踪：PREEMPT_RT mainlining 状态、Zephyr/RTOS、ISO 25785-1 草案、ISO 10218 humanoid 适用范围。 +- 触发器：ISO 25785-1 公布即触发 off-schedule checkpoint。 -**Verdict on H1 (one-contract WBC abstraction):** Still holds. No competing one-contract runtime at framework scope appeared. The closest is GR00T-WBC's own Policy Interface, but it's scoped to GR00T-WBC's own use, not pitched as a cross-vendor standard. +### 4.2 建议新增 V8：License / IP / 数据集格式 -### 3.7 Hidden assumption audit +**为什么**： +- v1 提到 BFM-Zero CC BY-NC 4.0 但没纳入常规 audit。v2 发现 license 是**实际限制 robowbc 默认 distribution 形态**的维度： + - BFM-Zero: CC BY-NC 4.0 + - UnifoLM-VLA-0: **CC BY-NC-SA 4.0**（更严） + - GEAR-SONIC 权重: NVIDIA Open Model License + - GR00T N1.7: Apache 2.0 + - Psi-Zero: Apache 2.0 + - mjlab / Newton: Apache 2.0 +- License 直接决定 robowbc 能否在 default install 路径下打包模型 / 权重 / MJCF。license-aware third-party-notices + plugin/optional-asset 系统是工程必需。 +- LeRobotDataset v2.1 → v3.0 格式变化是 robowbc 隐含数据依赖（UnifoLM-VLA-0 明确要求 "Huggingface LeRobot V2.1 数据集格式"）。 +- 中美 humanoid stack 分叉（Unitree IPO、AgiBot 与 NVIDIA Isaac GR00T 合作、AimRT 自研中间件）在 Mar–May 2026 没产生新的具体 SDK 层 fork，但 AimRT 是**潜在分叉信号**。 -The vectors card requires every checkpoint to answer: did this period produce evidence that any of H1–H8 needs reconsidering? +**条目**：V8 "License/IP + dataset schema" +- 跟踪：wrapped policy license 变化，LeRobotDataset 格式版本，Open X-Embodiment v2 状态，AimRT 第二家采用，出口管制公告。 -| # | Hidden assumption | This period's evidence | Verdict | -|---|---|---|---| -| H1 | One contract covers all WBC policy families | Issue #108 in GR00T-WBC shows VLA → WBC interface varies by family (14D wrist vs 17D joints), but the WBC → joint-targets boundary that robowbc operates at is stable across the families surveyed. | **Holds** | -| H2 | Rust + PyO3 + ONNX Runtime is the right runtime | ExecuTorch 1.0 GA is the most credible substitute path; no humanoid adoption yet. | **Holds, watching ExecuTorch** | -| H3 | Linux-first fail-fast | No counter-evidence; all surveyed humanoid platforms ship Linux-first. | **Holds** | -| H4 | Joint position targets, not direct torque | Frontiers in Robotics & AI Dec 2025 mini-review explicitly states position-based policies outperform torque policies for humanoid loco-manipulation; G1 has no joint torque sensors so PD position control is the practical choice. SATA (arXiv 2502.12674) is a torque-policy candidate but locomotion-specific and on quadrupeds. | **Holds** | -| H5 | Multiple policy families coexist under one registry | Q1 2026 produced 5+ active humanoid policy families (Decoupled-WBC, SONIC, BFM-Zero, AgileGR, Psi-Zero, GMT, MotionBricks, etc.) — registry approach is more justified now than at card-writing time. | **Strengthens** | -| H6 | Static reports site is the right primary proof surface | MuJoCo-Warp / Newton / Isaac Lab 3.0 lifted substrate throughput ~100x; upstream is shipping interactive web demos (SONIC, GENE-26.5). The ceiling moved; static site is not displaced but the bar is rising. | **Holds, needs revisit by 2026-08** | -| H7 | Unitree G1 + unitree_sdk2 + CycloneDDS; ROS 2 native parked | LeRobot G1 integration uses same stack. Zenoh did not become ROS 2 default. No vendor opened ROS-2-native-only customer API. | **Holds** | -| H8 | LeRobot is the right Python-first distribution channel | v0.5.0 made G1 first-class + added third-party policy plugin mechanism + ICLR 2026 acceptance. **Validated.** But the direct GR00T-WBC integration shortens the window for `robowbc-py` to ship as a competitive plugin. | **Strengthens; urgency increases** | +### 4.3 不推荐独立成 vector +- GitHub stars / contributor velocity：塞进每月例行 §3.x。 +- VLA-WBC 接口标准化：仍处于早期 watch，与 H1 重叠，留在 V1。 +- Workforce / 社区动量：太宽，不可执行。 -**Assumption-blindspot check:** All eight assumptions found at least some discriminating evidence in this period. **The card does not appear to have major blind spots at this point.** Re-check after the 2026-06 checkpoint — if H1, H3, H7 still produce no discriminating evidence two periods running, those assumptions may be too vague or unfalsifiable. +总数控制 ≤ 8。 --- -## 4. Taxonomy (optional) +## 5. Strategic threats -Skipped for the baseline; ecosystem panorama in §3 is already organized by the vectors-card structure. Will introduce a cross-cutting perspective in a future checkpoint if one becomes useful (e.g., "by safety boundary" vs "by abstraction layer"). +### 5.1 NVIDIA 自己做 "robowbc-equivalent"？ ---- - -## 5. Existing academic and industry taxonomies +**评估：12 个月内不会**。 -### 5.1 Academic surveys +依据： +- NVIDIA Mar–May 2026 策略集中在两条线：**模型**（GR00T N1.7 Apache-2.0、N2 预览、Cosmos 3）与 **平台**（Newton 1.0、Isaac Lab 3.0 Beta）。两条都不是"为多 backend humanoid policy inference 写一个 runtime"。 +- `gear_sonic_deploy` 是 vertical（单一家族 SONIC + Decoupled），**不是横向 multi-backend registry**。 +- NVIDIA 与 HF LeRobot v0.5.0 整合走的是"通过 LeRobot 完成 Python-first 分发"——占走 NVIDIA 自做 runtime 的需求。 +- Newton 在 Linux Foundation 下托管，NVIDIA 主动**让开**底层中立性，符合"我们做模型 / sim，中间层让生态做"。 -- Gu et al., 2025, "Humanoid Whole-Body Control: A Survey" — referenced by BFM-Zero (arXiv 2511.04131) as the field overview. -- Frontiers in Robotics & AI, Dec 2025, mini-review on humanoid manipulation policies — useful taxonomy of teleop vs imitation vs RL pipelines. +**真实威胁路径**：不是 NVIDIA 自做，而是 NVIDIA stack 的**完整性使中间层多余**——如果用户全链路是 Isaac Lab 训练 → GR00T N1.7 fine-tune → `gear_sonic_deploy` 部署到 G1，根本不需要 robowbc。robowbc 的市场是"不愿意 lock 在 NVIDIA GPU + TensorRT"的用户，以及"跨多 policy family 用统一 contract"的用户。 -### 5.2 Industry stack maps +### 5.2 LeRobot 是不是真正威胁？ -- LeRobot v0.5.0 release notes (HF blog) — implicitly defines the "supported platforms / policies / sim envs" matrix for the field's standard reference distribution. -- NVIDIA GTC 2026 announcement (GR00T N1.7 + N2, Isaac Lab 3.0, Newton, Cosmos 3) — implicit stack map for the NVIDIA-aligned subset. - ---- +**评估：是，但是吞并式而非压制式**。 -## 6. Specific recommendations for `robowbc` +依据： +- LeRobot v0.5.0 的 G1 + Holosoma + Groot 双控制器 + Pi0-FAST + RTC 已经是**"Python 端到端"**闭环。新手只想跑 G1 + Pi0.5 不需要 robowbc。 +- 但 LeRobot 在 framework 层而非 embedded runtime：`send_action()` 走 unitree_sdk2_python，Python 进程内，无 Rust + ONNX 这层 packaging，无专门 multi-backend 契约。 +- 真正威胁是 LeRobot 未来版本**直接吸收 multi-backend 抽象**（比如加 `lerobot.policies.backends` 子系统支持 ORT/ExecuTorch/TensorRT 多 backend），让 `lerobot_policy_robowbc` 失去价值。Mar–May 2026 无迹象。 -### 6.1 Trade-offs by phase +**robowbc 差异化论点**： +1. **Rust embedded runtime**：maturin / Cargo / 离线 single binary，不需要 conda + PyTorch toolchain。 +2. **Multi-backend policy registry**：跨 policy family 用同一契约，即使 codomain 不同。 +3. **Static proof pack 工作流**：不假设 HF Hub 或 Rerun viewer 在线，适合 air-gap + 监管报告。 +4. **License-aware distribution**：拒绝默认打包 CC BY-NC 模型。 -Current phase: Q2 2026 — multiple live wrappers, public reports site shipping, no LeRobot integration yet (#41). The recommendations below are calibrated for this phase; will be revised when the LeRobot plugin lands. +### 5.3 `gear_sonic_deploy` 跨家族复用 — 最具体威胁 -### 6.2 Projects worth deep-reading first +**评估：12 个月内最可能挤压 robowbc 的场景**。 -1. **GR00T-WBC `gear_sonic_deploy` C++ stack** — reason: directly overlaps with robowbc's `gear_sonic` wrapper. Read the design trade-offs to decide whether to align, diverge, or absorb (e.g., as an optional alt-backend behind `WbcPolicy`). -2. **LeRobot `HolosomaLocomotionController` source** — reason: closest concrete one-contract integration in the field. Even if it's not a competing abstraction, the implementation choices (state machine shape, error handling, transport assumptions) are worth borrowing or deliberately diverging from. -3. **mjlab paper + repo (arXiv 2601.22074)** — reason: it answers "what does Isaac Lab look like with MuJoCo-Warp and minimal dependencies?" — directly relevant to the future of `robowbc-sim`. -4. **Psi-Zero / SIMPLE benchmarking sim** — reason: a new whole-body humanoid VLA with its own benchmarking infrastructure; potential next addition to the wrapped-policy list, and SIMPLE is a candidate proof-substrate. -5. **LeRobot v0.5.0 release notes + third-party policy plugin docs** — reason: defines the exact shape `robowbc-py` should ship in to maximize H8. +依据： +- DeepWiki 索引明示 `gear_sonic_deploy` 已支持 SONIC **和** Decoupled WBC，C++ 栈设计为接受 ONNX + 配置即可换 policy family。 +- 如果 BFM-Zero（已 ONNX export）或 UnifoLM-VLA-0 的 action head（joint-space output）被 NVlabs 收编为 `gear_sonic_deploy` 可选 backend——工程量不大——那 robowbc 多 backend 故事就被 NVlabs 在同一仓库实现。 +- robowbc 反制： + 1. 占住 non-NVIDIA-GPU 用户群体。 + 2. 把 latent / Cartesian codomain 显式支持（SONIC 与 Decoupled 都是 joint targets，robowbc 先支持其它 codomain 就比 `gear_sonic_deploy` 更通用）。 + 3. license-aware distribution 上重投（NVlabs 把 CC BY-NC 模型默认打包概率为 0）。 -### 6.3 Concrete engineering actions +### 5.4 非显然竞争者 -| # | Action | Status | Value | -|---|--------|--------|-------| -| 1 | Refresh `gear_sonic` wrapper against the 2026-02-19 SONIC release; check for tensor-shape / config-schema drift; document overlap with `gear_sonic_deploy` | TODO | High — current wrapper may be silently incompatible with newer SONIC checkpoints | -| 2 | Implement `lerobot_policy_robowbc` plugin (LeRobot v0.5.0 third-party plugin shape) — closes #41 | TODO | High — completes H8 distribution-channel bet | -| 3 | Add MotionBricks, Psi-Zero, GMT, mjlab, Newton, Booster T1 to `mithaq/vectors/robowbc.md` entity lists at next card audit (scheduled 2026-08-21) | TODO | Medium — keeps the card from going stale before formal audit | -| 4 | Prototype `robowbc-sim` MuJoCo-Warp backend behind a feature flag; benchmark proof-pack throughput vs current CPU MuJoCo | TODO | Medium — large potential throughput win; depends on whether `make showcase-verify` is currently the bottleneck | -| 5 | Investigate UnifoLM-VLA-0 (Unitree's open VLA) — confirm primary source; assess whether it fits the `WbcPolicy` contract or sits at a different layer | TODO | Medium — could be the first Unitree-native upstream worth wrapping | -| 6 | Read LeRobot policy-schema delta v0.4.x → v0.5.0; verify `robowbc-py` adapter shape (#41) still fits | TODO | Medium — prerequisite for action 2 | -| 7 | Decide on BFM-Zero CC BY-NC 4.0 license disclosure surface (third-party-notices, policy card) | TODO | Low-Medium — compliance hygiene | -| 8 | Add a "competing one-contract runtimes" tracking note in the next checkpoint — even if there is none today, the watch itself is the value | TODO | Low — process discipline | +- **AgiBot AimRT**：自研 C++20 robotics middleware，单厂私有。如开放，可能形成新 transport+runtime 复合层。**G2 wheeled，与 robowbc 不直冲**。 +- **Booster Robotics Sim2Real/Sim2Sim 部署 framework**：与 robowbc 同层，在 [arXiv 2506.15132](https://arxiv.org/html/2506.15132v1)，仅支持 T1。robowbc 应 wrap T1 抢市场；反向，如 Booster 推到 G1 会成竞争者。 +- **OpenDriveLab WholebodyVLA + 类似学院工作**：自带 latent → low-level 控制器栈，但 research code，无生产 packaging。 ---- +### 5.5 什么会**证伪** robowbc 价值？ -## 7. Open questions and what next checkpoint should answer +按可能性从高到低： -### Experimental questions (require running code in this repo) +1. **LeRobot 内置 `backends/` 子系统**支持 ORT / TensorRT / ExecuTorch 多 backend 切换 + 跨家族 codomain 转换。→ robowbc 故事被吞并 80%。 +2. **NVlabs 把 `gear_sonic_deploy` 扩成 generic policy serving daemon**。→ NVIDIA-stack 用户里存在意义 → 0。 +3. **ISO 25785-1 要求 humanoid 推理 runtime 通过认证**而 robowbc Rust 栈没有快速通路。→ 受监管行业部署被屏蔽。 +4. **Unitree G1 SDK 改 ROS-2-native-only**，放弃 CycloneDDS direct API。→ H7 反转；robowbc 重做 transport。 +5. **HF LeRobot v0.6 引入 policy schema breaking change** 让 `lerobot_policy_robowbc` 每季度重写。→ 维护成本爆炸。 -- ☐ **Q1**: Does GR00T-WBC's `gear_sonic_deploy` C++ stack produce identical `JointPositionTargets` to robowbc's `gear_sonic` wrapper given the same observation? (If not, the wrapper has a silent drift bug.) -- ☐ **Q2**: What is the actual proof-pack-generation throughput today (scenarios/minute) on the public-reports CI? Need a number before claiming MuJoCo-Warp adoption is or isn't worth the work. +### 5.6 6–12 月可防御 moat -### Technical selection questions +按"今天就能投工程"优先级： -- ☐ **Q3**: Is mjlab production-ready enough to host robowbc proof-pack scenarios, or research-only? -- ☐ **Q4**: Should `robowbc-py` ship as a LeRobot third-party policy plugin (`pip install lerobot_policy_robowbc`) or as a standalone package that LeRobot can also load? Plugin shape is more aligned with v0.5.0; standalone preserves independence from LeRobot's release cadence. -- ☐ **Q5**: Does Booster T1's public Linux SDK + URDF/MJCF reach the "supported humanoid" criteria threshold (Linux, documented joints, deterministic transport, MuJoCo parity)? Need to read primary SDK docs. +1. **codomain-aware `WbcPolicy` trait**：joint target / latent token / Cartesian pose 三套 predict + 显式 decode 路径。 +2. **License-aware default distribution**：third-party-notices 自动生成 + plugin-loaded weights 路径。 +3. **Multi-platform target with same contract**：Unitree G1 + Booster T1 双 first-class。 +4. **Air-gap-friendly proof pack**：不要求在线，deterministic 输出，监管 / 审计初步合规材料。 +5. **PREEMPT_RT-friendly runtime**：jitter 度量做成 proof pack 指标，与 SONIC / Pi0-FAST RTC 延迟要求对齐。 -### Ecosystem-tracking questions +--- -- ☐ **Q6**: Does GR00T N2 (previewed at GTC 2026, based on DreamZero research) ship a runnable public release in 2026 H2, and if so, does it preserve the `Observation → predict → JointPositionTargets` interface or break it? -- ☐ **Q7**: Does any humanoid vendor (Unitree, AgiBot, Booster, Figure) adopt ExecuTorch as the canonical inference path? First adoption is the H2 trigger. -- ☐ **Q8**: Does LeRobot ship a policy-schema-level breaking change before 2026 EOY that would force a `robowbc-py` adapter rewrite? +## 6. Recommendations -### Long-horizon tracking questions +### 6.1 立即（Q2 2026 剩余 5–6 周） -- ☐ **Q9**: Does any platform ship a `WbcPolicy`-equivalent open abstraction at framework scope (LeRobot? NVIDIA? ROS 2 control WG? a Chinese humanoid vendor?) -- ☐ **Q10**: Does Genesis 1.0 ship in 2026? If yes, does it produce a real ecosystem effect or stay an alternative substrate? +| # | Action | Value | +|---|---|---| +| 1 | `gear_sonic` wrapper 与 [2026-02-19 SONIC 正式发布](https://nvlabs.github.io/GR00T-WholeBodyControl/) 对齐；新增 ZMQ v4 header 1280 字节常量；校验三个 ONNX | 高 — 不做就是 silent drift | +| 2 | `WbcPolicy` trait 新增 `predict_latent → decode → JointPositionTargets` 与 `predict_cartesian → ik → JointPositionTargets` 两条 codomain（stub + SONIC end-to-end / Psi-Zero teleop 各填示例） | 高 — H1 v2 直接驱动 | +| 3 | License-aware notices: BFM-Zero CC BY-NC 4.0、UnifoLM-VLA-0 CC BY-NC-SA 4.0、GEAR-SONIC NVIDIA Open Model License，加 install warning | 中-高 | +| 4 | ship `lerobot_policy_robowbc` MVP（G1 + decoupled_wbc + bfm_zero），作为生态占位 | 中 — 第一个 humanoid WBC 第三方插件 | +| 5 | `robowbc-runtime` README 明确控制频率假设（50 Hz default / 100 Hz max）与 jitter 期望（< 2 ms）；未用 PREEMPT_RT 时 fail-fast 拒绝 deploy real | 中 | -### Source-quality improvement questions +### 6.2 6–12 周 -- ☐ **Q11**: UnifoLM-VLA-0 — confirm primary repo URL, license, runnable status. Current evidence is B-tier; need A-tier verification. -- ☐ **Q12**: Unitree 2025 shipment numbers (5,500+ G1) and IPO target ($610M) — confirm from primary Shanghai Stock Exchange filing or Unitree IR. +| # | Action | Value | +|---|---|---| +| 6 | wrap Booster T1：接 Booster Robotics SDK + booster_gym 的 export_model，验证同一契约能跨两平台 | 中-高 | +| 7 | prototype `robowbc-sim` 切到 mjlab 实验分支，CI 测一个 G1 任务的 throughput | 中 | +| 8 | UnifoLM-VLA-0 加入 wrappers（不要默认打包，license CC BY-NC-SA），作 Unitree-native VLA 第一个示例 | 中 | +| 9 | 阅读 LeRobot v0.5.0 → v0.5.1 policy schema diff，`robowbc-py` 加 schema-version guard | 中 | -### New this period +### 6.3 季度 -All Q1–Q11 are new at this baseline checkpoint. +| # | Action | Value | +|---|---|---| +| 10 | V7（RT/safety）与 V8（license/dataset）sub-vector 加到 mithaq/vectors/robowbc.md 的 2026-08-21 quarterly audit | 中 | +| 11 | 跟踪 GR00T N2 实际发布与接口签名 | 高（条件触发） | +| 12 | 跟踪 ISO 25785-1 草案是否 2026-Q4 公开；如公开，off-schedule checkpoint | 中（条件触发） | --- -## 8. Changelog +## 7. Open questions and triggers -### 2026-05-22 — Baseline established +### 7.1 下一个 checkpoint 必须回答 -**Architecture narrative:** First robowbc checkpoint. Vectors card created 2026-05-21, this checkpoint created 2026-05-22 — they establish the comparison anchor for all future periods. No prior anchors, no judgment revisions. +**实验类**： +- ☐ Q1：`gear_sonic_deploy` 给的 29-DoF joint targets 与 robowbc-gear_sonic wrapper 在同 obs 下输出 bit-equivalent？ +- ☐ Q2：mjlab 在 robowbc CI 的 G1 baseline 上能否跑通（同一 MJCF 资产），throughput 对比当前 CPU MuJoCo。 +- ☐ Q3：PREEMPT_RT vs generic Linux 在 robowbc 主循环上的 jitter 差异？如 < 1 ms，可不强制 RT。 -**New projects surfaced (not in current vectors card entity lists):** +**技术选型**： +- ☐ Q4：`WbcPolicy` trait 三条 codomain 路径的实际 API 是什么？trait 加三个方法还是 enum return type？需小型 RFC。 +- ☐ Q5：`lerobot_policy_robowbc` 独立 repo 还是 monorepo 子目录？ +- ☐ Q6：CC BY-NC 模型被 import 时，robowbc 是 hard fail / warning / opt-in flag？ -- MotionBricks (NVlabs/GR00T-WholeBodyControl, 2026-04-27) -- Psi-Zero (USC PSI Lab, RSS 2026) -- GMT / GMR (YanjieZe, ICRA 2026) -- KungfuBot2 / VMS (arXiv 2509.16638) -- APEX (arXiv 2026-02) -- HumanoidVerse (LeCAR-Lab) -- mjlab (Berkeley/DeepMind, arXiv 2601.22074) -- Newton physics engine (NVIDIA + DeepMind + Disney, Linux Foundation, GTC 2026) -- Genesis-Humanoid (UMass Embodied AGI) -- UnifoLM-VLA-0 (Unitree, B-tier `[未充分验证]`) -- GENE-26.5 (Genesis AI — disambiguation: different entity from Genesis-Embodied-AI simulator) +**生态跟踪**： +- ☐ Q7：GR00T N2 在 2026-H2 发布，接口签名是 joint / latent / EEF？ +- ☐ Q8：任何 humanoid 厂商首次采用 ExecuTorch？（目前为 0） +- ☐ Q9：LeRobot v0.6 是否动 policy schema？ +- ☐ Q10：ISO 25785-1 公布？ +- ☐ Q11：出现第二个真实 `lerobot_policy_*` 第三方包？ +- ☐ Q12：AimRT 是否被 AgiBot 之外的第二家厂商采用？ -**Status changes for entities already in vectors card:** +### 7.2 触发 off-schedule checkpoint 的具体事件 -- GR00T-WBC: GEAR-SONIC formal release (2026-02-19); training code public (2026-04-10); end-to-end VLA workflow (2026-05-07). -- LeRobot: v0.5.0 with full G1 humanoid + 3rd-party policy plugin + ICLR 2026 acceptance. -- Isaac Lab: 3.0 Beta released (multi-backend physics, Warp-native data pipelines — breaking change). -- ONNX Runtime: 1.26.0 released 2026-05-08; CUDA 12+ minimum. -- ExecuTorch: 1.0 GA achieved 2025-10-22 (vectors card written before this but doesn't note GA status). -- Unitree: H2 launched; G1 firmware 1.4.5 Training Mode; IPO filing. +任何一条立刻触发： -**Judgment revisions:** None (first checkpoint). +1. NVIDIA 把 `gear_sonic_deploy` 扩成支持非 GR00T 系策略族。 +2. LeRobot ship `lerobot.policies.backends` 或类似 multi-backend 子系统。 +3. Unitree G1 SDK 默认切到非 CycloneDDS，或 release ROS-2-native-only API。 +4. ISO 25785-1 公布正式版本。 +5. GR00T N2 实际可下载并支持 UNITREE_G1。 +6. ExecuTorch 出现第一个 humanoid 项目真实采用。 +7. mjlab 或 Newton 在 humanoid project 中变成 sim default。 +8. 任何 wrapped policy 的 license 改变。 --- -## Appendix A: Full reference links - -### A.1 Vector 1 — WBC policy upstream - -- NVlabs/GR00T-WholeBodyControl: -- NVIDIA/Isaac-GR00T (N1.7): -- MotionBricks: -- BFM-Zero: , arXiv: -- BFM-Zero ICLR 2026: -- Psi-Zero: -- GMR (General Motion Retargeting): -- HumanoidVerse: -- Physical-Intelligence/openpi: +## 8. Changelog -### A.2 Vector 2 — Inference runtime +### 2026-05-22 v2 — Deeper baseline (replaces v1) -- microsoft/onnxruntime: -- ONNX Runtime roadmap: -- pytorch/executorch: -- ExecuTorch 1.0 GA blog: -- huggingface/candle: +**判断修订**： -### A.3 Vector 3 — Hardware +| Verdict | v1 | v2 | 理由 | +|---|---|---|---| +| H1 | Holds | **Holds with conditions；三个 codomain 反例** | UNITREE_G1_SONIC latent + Psi-Zero teleop 14D Cartesian + N1.7 default relative EEF | +| H2 | Watch ExecuTorch | **Watch `gear_sonic_deploy` C++ instead** | ExecuTorch humanoid 采用 0 | +| H8 | Strengthens; urgency 高 | **Holds; urgency 下调** | 第三方插件 2 个月只 1 实现 | +| 战略威胁 #1 | LeRobot 直接对手 | **`gear_sonic_deploy` 跨家族复用 > LeRobot 吞并 > NVIDIA 自做 runtime** | §5 | +| MuJoCo-Warp 性能 | 152× vs MJX（B-tier） | **252× locomotion / 475× manipulation on RTX PRO 6000 Blackwell**（NVIDIA Technical Blog 一手） | NVIDIA Newton contact-rich blog | + +**新增一手主源**（v1 未涵盖或未 A-tier 验证）： +- UnifoLM-VLA-0 — 一手 [unitreerobotics/unifolm-vla](https://github.com/unitreerobotics/unifolm-vla)，license CC BY-NC-SA 4.0；训练数据 ~340h 来自 Unitree 官方项目页 [unigen-x.github.io/unifolm-vla.github.io](https://unigen-x.github.io/unifolm-vla.github.io/)。 +- Newton 1.0 (LF, GTC 2026) — [linuxfoundation.org press](https://www.linuxfoundation.org/press/linux-foundation-announces-contribution-of-newton-by-disney-research-google-deepmind-and-nvidia-to-accelerate-open-robot-learning) + [newton-physics/newton](https://github.com/newton-physics/newton)。性能基准从 NVIDIA Technical Blog ([Newton contact-rich locomotion](https://developer.nvidia.com/blog/newton-adds-contact-rich-manipulation-and-locomotion-capabilities-for-industrial-robotics/), 2026-03) 引用 — 252× locomotion / 475× manipulation。 +- Booster Robotics — [booster.tech/open-source](https://www.booster.tech/open-source/) + [booster_gym arXiv 2506.15132](https://arxiv.org/html/2506.15132v1)。 +- AgiBot G2 / AimRT — [interestingengineering](https://interestingengineering.com/ai-robotics/agibot-g2-humanoid-robot-smart-manufacturing) + [agibot.com](https://www.agibot.com/article/231/detail/54.html)。 +- PhysiFlow [arXiv 2603.05410](https://arxiv.org/abs/2603.05410) 与 WholebodyVLA (ICLR 2026)。 +- LeRobot Bring Your Own Policies docs — 含唯一第三方实例 `lerobot_policy_ditflow`。 +- Psi-Zero 接口分裂 — [physical-superintelligence-lab/Psi0](https://github.com/physical-superintelligence-lab/Psi0)。 +- Pi0-FAST RTC latency — [arXiv 2506.07339 Real-Time Execution of Action Chunking Flow Policies](https://arxiv.org/abs/2506.07339)，字面 "76ms baseline / 97ms RTC"。 +- Unitree IPO — CNBC (2026-03-20) 确认 ¥4.2B (US$610M) 募资目标、2025 出货 5,500+ G1、占全球 humanoid 市场 32.4%。 + +**新增 vectors card 提案**：V7 (Real-time + safety)，V8 (license / IP / dataset schema)。下次 quarterly audit (2026-08-21) 评估合并。 + +**仍未充分验证**： +- ISO 25785-1 Working Draft 当前条文 — ISO 直接获取需付费。 +- MuJoCo-Warp 性能数字虽然现在已 A-tier（NVIDIA Technical Blog），但仍是 NVIDIA 厂商自证基准，独立第三方复测尚未出现 — 用法上仍应在数字旁注明"NVIDIA-claimed"。 -- Unitree GitHub: -- Unitree LeRobot integration: -- Unitree IL LeRobot docs: +--- -### A.4 Vector 4 — Middleware +## Bibliography (key A-tier URLs) -- rmw_zenoh: -- zenoh-plugin-ros2dds: +**Vector 1**：[NVlabs/GR00T-WholeBodyControl](https://github.com/NVlabs/GR00T-WholeBodyControl)；[Documentation](https://nvlabs.github.io/GR00T-WholeBodyControl/)；[NVIDIA/Isaac-GR00T](https://github.com/NVIDIA/Isaac-GR00T)；[HF blog GR00T N1.7](https://huggingface.co/blog/nvidia/gr00t-n1-7)；[BFM-Zero arXiv 2511.04131](https://arxiv.org/abs/2511.04131)；[LeCAR-Lab/BFM-Zero](https://github.com/LeCAR-Lab/BFM-Zero)；[physical-superintelligence-lab/Psi0](https://github.com/physical-superintelligence-lab/Psi0)；[unitreerobotics/unifolm-vla](https://github.com/unitreerobotics/unifolm-vla)；[unigen-x.github.io/unifolm-vla.github.io](https://unigen-x.github.io/unifolm-vla.github.io/)；[arXiv 2603.05410 PhysiFlow](https://arxiv.org/abs/2603.05410)；[OpenDriveLab/WholebodyVLA](https://github.com/OpenDriveLab/WholebodyVLA)；[NVIDIA Newsroom GTC 2026](https://nvidianews.nvidia.com/news/nvidia-expands-open-model-families-to-power-the-next-wave-of-agentic-physical-and-healthcare-ai) -### A.5 Vector 5 — Sim substrate +**Vector 2**：[microsoft/onnxruntime](https://github.com/microsoft/onnxruntime)；[pytorch/executorch](https://github.com/pytorch/executorch)；[executorch.ai](https://executorch.ai/)；[arXiv 2506.07339 RTC](https://arxiv.org/abs/2506.07339) -- google-deepmind/mujoco_warp: -- mujoco-mjx PyPI: -- mjlab: -- NVIDIA Newton + Isaac Lab blog: -- Isaac Lab releases: -- Genesis-Embodied-AI: -- Genesis-Humanoid (UMass): +**Vector 3**：[unitreerobotics](https://github.com/unitreerobotics)；[Booster open source](https://www.booster.tech/open-source/)；[BoosterRobotics GitHub](https://github.com/BoosterRobotics)；[booster_gym arXiv 2506.15132](https://arxiv.org/html/2506.15132v1) -### A.6 Vector 6 — Python distribution +**Vector 4**：[ros2/rmw_zenoh](https://github.com/ros2/rmw_zenoh)；[Jazzy Zenoh docs](https://docs.ros.org/en/jazzy/Installation/RMW-Implementations/Non-DDS-Implementations/Working-with-Zenoh.html)；[zenoh-plugin-ros2dds](https://github.com/eclipse-zenoh/zenoh-plugin-ros2dds) -- huggingface/lerobot: -- LeRobot v0.5.0 blog: +**Vector 5**：[google-deepmind/mujoco_warp](https://github.com/google-deepmind/mujoco_warp)；[newton-physics/newton](https://github.com/newton-physics/newton)；[LF Newton press](https://www.linuxfoundation.org/press/linux-foundation-announces-contribution-of-newton-by-disney-research-google-deepmind-and-nvidia-to-accelerate-open-robot-learning)；[NVIDIA Newton announcement](https://developer.nvidia.com/blog/announcing-newton-an-open-source-physics-engine-for-robotics-simulation/)；[NVIDIA Newton 1.0 contact-rich](https://developer.nvidia.com/blog/newton-adds-contact-rich-manipulation-and-locomotion-capabilities-for-industrial-robotics/)；[mujocolab/mjlab](https://github.com/mujocolab/mjlab)；[mjlab arXiv 2601.22074](https://arxiv.org/abs/2601.22074)；[Isaac Lab v3.0.0-beta](https://github.com/isaac-sim/IsaacLab/releases/tag/v3.0.0-beta)；[Isaac Lab discussion 4339](https://github.com/isaac-sim/IsaacLab/discussions/4339) -### A.7 robowbc itself +**Vector 6**：[huggingface/lerobot](https://github.com/huggingface/lerobot)；[LeRobot v0.5.0 blog](https://huggingface.co/blog/lerobot-release-v050)；[Bring Your Own Policies docs](https://huggingface.co/docs/lerobot/bring_your_own_policies)；[Unitree G1 docs](https://huggingface.co/docs/lerobot/v0.5.1/unitree_g1)；[danielsanjosepro/lerobot_policy_ditflow](https://github.com/danielsanjosepro/lerobot_policy_ditflow) -- Repo: -- Public reports: -- Vectors card: -- Checkpoint template: -- Companion docs in this repo: [`docs/ecosystem-strategy.md`](../ecosystem-strategy.md), [`docs/roadmap-2026-q2.md`](../roadmap-2026-q2.md), [`docs/research/2026-05-merged-tech-report.md`](../research/2026-05-merged-tech-report.md) +**B-tier（需交叉验证）**：[Theresarobotforthat humanoid safety 2026](https://www.theresarobotforthat.com/blog/humanoid-robot-safety-standards-2026/)；[DeepWiki/NVlabs/GR00T-WBC](https://deepwiki.com/NVlabs/GR00T-WholeBodyControl)；[KraneShares humanoid 2026](https://kraneshares.com/humanoid-robotics-in-2026-the-race-from-pilot-to-platform/)；[Pandaily UnifoLM-VLA-0](https://pandaily.com/unitree-robotics-open-sources-multimodal-vision-language-action-model-unifo-lm-vla-0-1)；[Interesting Engineering AgiBot G2](https://interestingengineering.com/ai-robotics/agibot-g2-humanoid-robot-smart-manufacturing)；[Awesome Agents LeRobot v0.5.0](https://awesomeagents.ai/news/lerobot-v050-humanoid-open-source/) --- -## Appendix B: Glossary +## Appendix: Glossary delta vs v1 | Term | Meaning | -|------|---------| -| WBC | Whole-Body Control — humanoid controller producing coordinated whole-body motion | -| `WbcPolicy` | robowbc's central trait; the `predict(Observation) → JointPositionTargets` contract | -| Decoupled WBC | Hybrid controller: RL for lower body locomotion + IK for upper body. Used in GR00T N1.5/N1.6. | -| GEAR-SONIC (SONIC) | Behavior foundation model from NVIDIA GEAR team; motion tracking as scalable training task; latest generalist WBC controller | -| MotionBricks | Real-time latent generative model for interactive motion control; part of GR00T-WBC | -| BFM-Zero | Promptable behavioral foundation model using unsupervised RL + Forward-Backward models; G1-deployed; CC BY-NC 4.0 | -| MJX | MuJoCo XLA — JAX-based reimplementation of MuJoCo, GPU-accelerated | -| MuJoCo-Warp (MJWarp) | GPU-optimized MuJoCo via NVIDIA Warp; main solver in Newton | -| Newton | Physics engine jointly developed by NVIDIA + DeepMind + Disney; Linux Foundation, Apache-2.0 | -| mjlab | Lightweight Isaac-Lab-style API directly on MuJoCo-Warp; minimal dependencies | -| HolosomaLocomotionController | LeRobot's controller class for G1 humanoid locomotion | -| Proof pack | robowbc's bundle of JSON + Rerun .rrd + screenshots + HTML report demonstrating a policy run | -| `UNITREE_G1_SONIC` embodiment tag | GR00T N1.7 tag selecting the SONIC controller path (vs `UNITREE_G1` which selects Decoupled WBC) | -| `[未充分验证]` | Marker for claims supported only by B-tier sources awaiting A-tier verification | +|---|---| +| `predict_latent / predict_cartesian` | v2 提议的 `WbcPolicy` trait 三条 codomain 分支，joint target 之外 | +| codomain | 一个 policy 输出空间的语义类（joint target / latent token / Cartesian pose / relative EEF） | +| `UNITREE_G1_SONIC` embodiment tag | GR00T N1.7 选 SONIC end-to-end 路径的 tag，输出 latent action tokens | +| AimRT | AgiBot 自研 C++20 robotics middleware，新中间件实体 | +| ISO 25785-1 | dynamically stable robots（含 humanoid）的安全标准，2026 Working Draft | +| Kamino solver | Disney Research 的 VBD-based 软体 / 闭链 solver，Newton 1.0 集成 | +| FSQ Bottleneck | Finite Scalar Quantization，SONIC UniversalTokenModel 的 latent 离散化 | +| BONES-SEED | NVlabs 在 2026-03-16 开源的 142K+ human motions 数据集 | +| ZMQ protocol v4 | gear_sonic_deploy 在 2026-03-24 升级的 wire 协议，header 1280 字节 | --- -**End of document. Next checkpoint scheduled for end of June 2026.** +**End of v2 checkpoint. Next routine update: 2026-06 月末。** -Run the incremental-research loop per §0.2, paying particular attention to Q1–Q4 (experimental + technical selection) and Q11 (UnifoLM-VLA-0 primary verification). +不要在 §1 之外重读 v1 — v2 在 H1/H2/H8 与战略威胁排序上的判断已经覆盖。