diff --git a/.claude/CLAUDE.md b/.claude/CLAUDE.md
index e959b70b..b80eb4d9 100644
--- a/.claude/CLAUDE.md
+++ b/.claude/CLAUDE.md
@@ -6,66 +6,16 @@ The **browser-native robotics dev environment** — vibe-code robots in a tab, r
**What's defensible.** Browser is the dev surface — no install, no SDK download. Browser-resident model serving — perception, detection, fiducial pose all client-side, no GPU server, no inference bill. Layered safety — firmware-bounded motors that the IDE-level planner (user code or Pip) can't bypass; ask-human is the terminal cascade rung (openpilot-panda pattern). Static-site deployable — no backend, no accounts, no data leaving the browser.
-**Directions worth pursuing:**
-- **Capability schema** — JSON manifest + chip handler + auto-rendered card. The IDE's plugin system; lets a user or Pip ship a new hardware capability without touching dashboard code.
-- **Multi-robot orchestration.** Two robots in the same room, scripts or Pip planning across them, no central server.
-
**Anti-drift guards.** Failure modes to refuse:
-- *"Yet another teleop dashboard"* — joystick-shaped UI for human pilots. The wedge is planning-shaped.
- *"Yet another fleet manager"* — server-resident cloud for N robots. Viam's space; ours is one operator running their own.
- *"The LLM does everything autonomously"* — Pip is one surface inside the IDE. User code is co-equal; both are bounded by the same firmware safety floor.
-# Developer reference
-
-`DEV.md` at the repo root is the canonical list of URL flags, `window.*` handles, IndexedDB stores, and common debug paths.
-
-# Project layout
-
-- `docs/` is the GitHub Pages publish root — the dashboard's static ES modules live here directly. (Repo-level docs like HARDWARE.md, SMOKE.md, etc. live at the root or inside subsystems, not in `docs/`.)
-- `docs/` is flat by design — file count is manageable, naming prefixes carry the subsystem boundary. Promote a subsystem to its own folder (like `capabilities/`) once it passes ~5 files whose internals shouldn't leak outside.
-
-# Subsystem map
-
-- **Pair layer** — `pairing.js`, `phones.js`, `mobile.js` + `phone.html`. Desktop ↔ phone WebRTC.
-- **Perception + detection** — `camera-frame.js`, `mediapipe.js` (closed-vocab COCO reflex, powers `watcher.js`), `aruco.js` (overhead ArUco → `entry.arucoPosition`, wired but unproven). Open-vocab queries route through `view_robot_frame` → Claude vision — no in-browser open-vocab model needed.
-- **Pip / assistant** — `assistant.js`, `claude.js`, `pip-tools.js`.
-- **Robot ops** — `ble.js`, `ops-response.js`, `capabilities/`.
-- **Robot lifecycle** — `prepare.js`, `recovery.js`, `pinout.js`.
-- **User code** — `scripts.js`. Mirrors the BLE capability surface; persisted in localStorage. See `USER-CODE.md`.
-- **App shell** — `app.js`, `dom.js`, `state.js`, `settings.js`, `log.js`, `auth.js`, `passwords.js`, `index.html`, `styles.css`, `icons.svg`.
-
-# Smoke testing
-
-Two layers, kept cheap:
-
-- `make smoke` — pure-function tests via `node --test tests/*.test.js`. Anything in `format.js` (and future pure helpers) earns a row in `tests/format.test.js`. Runs in <1 s.
-- `SMOKE.md` — manual checklist for architectural promises (lifecycle, render patterns, capability behavior, Pip flow, recovery).
-
-Pattern for new pure helpers: extract from `app.js` / cap runtime into `format.js`, import where used, add a test.
-
-`make install-hooks` wires `.githooks/` as `core.hooksPath`. Pre-commit runs `make smoke`, the gen-uuids drift check (when `protocol/uuids.json` is staged), and the sw.js VERSION stamp (when `docs/*` excluding firmware bins / sw.js is staged — folds the stamp into the user's commit so the dashboard "Reload to update" banner fires on the right commit instead of a CI follow-up). Bypassable with `--no-verify`; CI is the binding layer.
-
-# Comment discipline
-
-Default to no comments — every line is context cost in an AI-edited codebase.
-
-Keep when the comment carries WHY: hidden constraints, kernel/API gotchas, workarounds for past bugs, cross-file invariants ("must match `firmware/pi_robot/pi_robot.py`"), schema/wire-format examples. Cut when it restates WHAT: module preambles, narration, section banners, labels above obvious code.
-
-# Abstractions earn upstream consumers
-
-Before adding a logical layer, registry, wrapper, or routing decision, audit who outside its home module will use it. If only one module touches it, it's internal — bar for keeping it is high. The merge layer (item F) shipped with no consumers above the dashboard; deletion (R1) was clean precisely because nothing else depended on it. The cost of an unused abstraction isn't only the lines it adds — it's the explanatory comments, the cross-cutting params plumbed through siblings, and the bug-shaped negative space (the joypad-no-op was a child of the abstraction, not a coincidence). Audit before adding, not before deleting.
-
-# Dialog vs menu dismiss
-
-- **Menus + popovers** (robot-menu, avatar-menu, help popovers, Pip's `
`): outside-click + Escape dismiss.
-- **Dialogs**: × button or Escape only. Outside-click would nuke session state (recovery terminal, SD prep) for a tiny convenience win.
-
# Control-loop architecture
The "openpilot panda" pattern: safety enforced *below* the intelligent layer, not inside it.
- Firmware caps pulse duration and watchdog auto-stop. LLM-issued motion auto-stops at the end of the pulse window (4s on Pi, same on ESP32); the watchdog cuts persistent commands when the dashboard goes silent. The ultrasonic dist_cm clip stops pure-forward motion at walls regardless of who issued the move. The planner can't bypass these — not even via a malformed tool call.
-- Magnitude is *not* capped LLM-side anymore. Joypad and Pip share the same signed-byte range; the time-bound is what bounds a single bad decision. Earlier versions used `LLM_MAX_SPEED = 70` as an extra "reduced envelope for the planner" rung, but the duration cap already bounds the wrong-direction excursion, and the cap was making demos / Pip-driven motion artificially slow vs joypad without buying meaningful safety.
+- Magnitude is *not* capped LLM-side. Joypad and Pip share the same signed-byte range; the time-bound is what bounds a single bad decision. Earlier versions used `LLM_MAX_SPEED = 70` as an extra "reduced envelope for the planner" rung, but the duration cap already bounds the wrong-direction excursion, and the cap was making Pip-driven motion artificially slow vs joypad without buying meaningful safety.
- LLM-issued motion is pulse-bounded (`duration_ms` mandatory; firmware auto-stops). Persistent speed is reserved for human joystick control where there's a 20Hz+ decision loop.
- `ask_human_via_phone` is the terminal rung of the decision cascade — the planner asks to be overridden rather than waits for the operator to step in.
@@ -73,121 +23,55 @@ The "openpilot panda" pattern: safety enforced *below* the intelligent layer, no
Different model shapes are good at different jobs — distinct primitives, not interchangeable "AI". Past planner-layer attempts to paper over capability gaps with prompt-engineering have bitten us.
-**Detectors and perception:**
-
- **Closed-vocab reflex detector** (`mediapipe.js`, EfficientDet-Lite0 via MediaPipe Tasks API): 80 COCO classes, ~10–30 ms on GPU. Powers the per-robot Reflex card (`watcher.js`) and user-code `robot.watchFor` / `robot.detections`. Fire-once-and-disable shape — same terminal-rung pattern as `ask_human`. For backend-vision-capable Pip turns, `view_robot_frame` passes the raw frame straight to the planner — no caption step.
-
-**Unproven / experimental:** Overhead ArUco localization (`aruco.js`), YOLO26n closed-vocab detector (`yolo26.js`, opt-in via `/detector yolo26` — wired, not the default, no field-validation run logged). See `.claude/notes.md` "Wired but unproven." Keep out of user docs until validated. (Grounding DINO was previously the open-vocab fallback; deleted once Claude vision via `view_robot_frame` absorbed that role with scene reasoning the bbox-only detector couldn't do.)
-
-**Planners (Pip):**
-
- **Tool-using LLM via API** (`claude.js`): seconds-latency, multi-turn, tool-calling. Strong at goal decomposition, weak at closed-loop visual servo (2–5 s round-trip). Currently Claude; any tool-using LLM with the same tool surface fits here.
+- **Unproven / experimental**: Overhead ArUco localization (`aruco.js`), YOLO26n closed-vocab detector (`yolo26.js`, opt-in via `/detector yolo26`). See `.claude/exploration.md` → "Wired but unproven." Keep out of user docs until validated. Grounding DINO was deleted once Claude vision via `view_robot_frame` absorbed the open-vocab role with scene reasoning the bbox-only detector couldn't do.
# Transport channels
-Each transport has a distinct job:
+Pattern: control = BLE, observe = wifi/discover, recover = USB.
- **BLE** — control plane. Low latency, proximity-authenticated, lossy. Anything that sets motor speed, toggles an LED, commits state.
- **Typed ops over BLE** — structured verbs on a single characteristic (`get-log`, `get-config`, `restart-service`, `wifi-scan`, `wifi-join`). Each verb is a deliberate, reviewable decision instead of a real-shell transport.
- **WebRTC** — two distinct flows.
- *Phone ↔ desktop*: signaled via `wss://signal.neevs.io` (cross-network — operator may not be physically near the phone). Pair-ceremony authenticated (Ed25519 pubkey + signed pair-request). Carries camera frames, ask-human responses, robot-command relays.
- - *Robot ↔ desktop* (Pi or ESP32): signaled over the BLE `SIGNAL` characteristic — no internet rendezvous, no Mixed-Content/PNA gate. ESP32 handles signaling in-firmware; Pi forwards the offer to a local aiortc daemon (`pi_robot_rtc.py`) over a Unix socket and chunks the answer back via BLE notify. BLE pair = signal = auth. Carries OTA bundles, log tail, PTY shell (Pi), and camera video (BLE-signaled `camera-signal` char on Pi).
-- **Wifi-presence** — Pi exposes `.local:81/health` (pi_robot_health.py); dashboard probes it for the "on wifi" badge + service-crash detection. ESP32 retired its HTTP server in Phase 2.H — its presence shows up only when BLE-paired (wifi-status notify). No internet rendezvous for robot presence (signal.neevs.io stays for cross-network phone-pair only).
+ - *Robot ↔ desktop* (Pi or ESP32): signaled over the BLE `SIGNAL` characteristic — no internet rendezvous, no Mixed-Content/PNA gate. BLE pair = signal = auth. Carries OTA bundles, log tail, PTY shell (Pi), and camera video. See `firmware/esp32_robot_idf/WEBRTC.md` for the ESP32-side patches.
+- **Wifi-presence** — Pi exposes `.local:81/health`; dashboard probes it for the "on wifi" badge + service-crash detection. ESP32 presence shows up only when BLE-paired (wifi-status notify). No internet rendezvous for robot presence.
- **USB-CDC** — recovery plane. Last-resort serial console, runs as its own systemd unit so a `pi-robot.service` crash doesn't take recovery with it. Bounded by physical access.
-Pattern: control = BLE, observe = wifi/discover, recover = USB.
+# Connection-first init
-# ESP32 WebRTC: chip is the DTLS client, not the server
-
-Classic ESP32 streams WebRTC video to current Chrome via four coordinated
-patches that don't independently make sense — anyone debugging this stack
-needs to see them as one shape:
-
-1. **DTLS role: chip is CLIENT** (forced in `dtls_srtp_init` regardless of
- what libpeer's binary blob passes). libpeer always passes ROLE_SERVER,
- but mbedTLS's `ssl_parse_client_hello` can't reassemble Chrome's ~1413-
- byte fragmented ClientHello — bails immediately with `FEATURE_UNAVAILABLE`.
- As CLIENT, chip sends the (small, never-fragmented) ClientHello and
- Chrome handles whatever it receives. Chrome 124+ enforces this strictly.
-2. **DTLS cert is dashboard-supplied**, ECDSA P-256. The browser generates
- the keypair (WebCrypto) and self-signs an X.509 cert (@peculiar/x509),
- then pushes both PEMs over the SIGNAL char (opcodes 0x07/0x08/0x09) BEFORE
- the offer. Chip's `dtls_srtp_init` refuses to open WebRTC if nothing was
- supplied — chip-gen path was removed for ~9 KB flash saved (linker gc on
- mbedtls x509write_crt_* + ecp_gen_key). WebRTC standardized on ECDSA;
- current Chrome rejects RSA in DTLS-SRTP, so the dashboard cert is built
- ECDSA-only too.
-3. **All chip-quirk SDP rewriting lives in the dashboard** (webrtc-robot.js).
- The browser pre-strips TCP candidates from the offer (chip is UDP-only),
- pins offer MID to "0" so libpeer's hardcoded "0" in the answer matches,
- and flips `setup:passive`→`setup:active` on the incoming answer (libpeer
- always emits passive even though chip is actually CLIENT). Used to be
- three string-walking functions in webrtc_peer.c (`filter_sdp_for_chip`,
- `capture_offer_mid`, `rewrite_answer_mid`); centralizing made the chip
- an SDP-agnostic byte pipe.
-4. **mbedTLS Kconfig** must enable the WebRTC cipher set explicitly
- (DTLS_SRTP, ECDHE_ECDSA, ECDH_C, ECDSA_C, SECP256R1, GCM_C, SHA1_C,
- HKDF_C). IDF defaults are tuned for HTTPS-client and lack what DTLS-SRTP
- needs. X509_CREATE_C: not needed on v5 (dashboard does the cert
- creation, chip only parses); v6 path of esp_peer re-enables it for
- upstream cert helpers even though our flow stays dashboard-side.
-5. **PSRAM-default malloc** with `RESERVE_INTERNAL=32768` — mbedTLS context
- + libpeer SCTP/SRTP buffers go to PSRAM so the camera DMA's 32 KB
- contiguous internal block is always available mid-session.
-
-Removing any one of these reverts the chip to "DTLS handshake never
-completes" or "camera_acquire fails after WebRTC opens." Firmware-side
-constraints (DTLS role, mbedTLS Kconfig, PSRAM malloc) are documented in
-firmware/esp32_robot_idf/components/espressif__esp_peer/src/dtls_srtp.c
-and sdkconfig.defaults.esp32; dashboard-side constraints (cert push, SDP
-rewriting) in docs/webrtc-cert.js and docs/webrtc-robot.js.
-
-**Sunset path.** mbedTLS PR #10623 (3.6 backport of the fragmented DTLS-
-ClientHello reassembly fix, first released in 3.6.6 / 4.1.0, March 2026)
-collapses Patch 1 and the half of Patch 3 that exists because of it.
-ESP-IDF v5.5.4 (current pin) ships 3.6.5, v6.0.1 ships 4.0.0 — both
-pre-fix. espressif/esp-idf release/v5.5 (now on 3.6.6-idf) and
-release/v6.0 (now on 4.1.0-idf) have the fix on their HEAD branches; the
-next tagged release in either line is the trigger.
-
-Prefer v6.0.x. components/espressif__esp_peer/src/dtls_srtp_v6.c is
-pre-staged (CMake selects it on `IDF_VERSION_MAJOR >= 6`) and already
-encodes the post-sunset shape: role honored from cfg (no CLIENT
-override), HelloVerifyRequest cookies enabled, PSA crypto path. The
-cleanup on a v6.0.2 bump collapses to "delete the v5 dtls_srtp.c sibling
-and the IDF major-version CMake selector" rather than reverting patches
-in-place. The rest of the firmware migrates clean — NimBLE / WiFi /
-esp_netif / esp_http_server / LEDC / GPIO / NVS / esp_timer call sites
-all survive v6.0; exposure is `-Werror` flip + gnu23 default surfacing
-latent warnings.
-
-v5.5.5 is the fallback if v6.0.2 is slow. On v5.5.5, the manual cleanup
-is: revert chip-as-CLIENT in dtls_srtp.c (lines 75 and 161), restore
-HelloVerifyRequest cookies (line 95). In either case, drop the
-`setup:passive`→`setup:active` flip from docs/webrtc-robot.js. Patches
-2 (dashboard ECDSA cert), 4 (mbedTLS Kconfig) and 5 (PSRAM malloc) stay
-— those are WebRTC-spec or chip-shape, not mbedTLS-bug workarounds.
-
-**Opt-in via `CONFIG_BR_WEBRTC_ESP_PEER`** (main/Kconfig.projbuild, default
-y). Set =n to drop all WebRTC code — `select` chain removes the WebRTC-only
-mbedTLS bits, all call sites in webrtc_peer / app_main / gatt_svr / telemetry
-guard out with `#ifdef`, and the linker's `--gc-sections` strips libpeer.a
-from the image (~215 KB smaller binary). Useful for forks that only need
-HTTP MJPEG video. esp_peer always *registers* as a component (Kconfig values
-aren't visible to IDF's component-registration phase), but produces no live
-references when off, so the linker drops it.
+Connection infrastructure (BLE, WiFi, USB-CDC) initializes before capability infrastructure (camera, perception, motors). A robot whose BLE stays up with no camera is observable and actionable; the reverse is a brick. ESP32 example: NimBLE host init and `wifi_sta_init` run early in `app_main()` so radio drivers pre-allocate their buffers in fresh internal heap. Camera comes after; if it can't fit its 32 KB DMA buffer in what's left, it fails loudly and `fw_info` hides the cap so the dashboard adapts.
-# Connection-first init
+# Project layout
+
+`docs/` is the GitHub Pages publish root — static ES modules live there directly. Repo-level docs (HARDWARE.md, SMOKE.md, etc.) live at the root or inside subsystems, not in `docs/`.
+
+- **Root holds primitives, subsystems hold vocabularies.** `docs/` root is for (a) HTML entry points, (b) app-shell singletons (`app.js`, `state.js`, `dom.js`, `event-bus.js`, `log.js`, `settings.js`), (c) cross-cutting primitives imported by ≥3 subsystems (`format.js`, `error-capture.js`). Everything else presumed to belong in a subsystem folder.
+- **Promotion trigger: vocabulary closure.** Files belong in their own folder when they (1) share a naming prefix, (2) change together for the same reason, (3) expose ≤2 symbols outward. A 3-file sealed vocabulary (`pinout-*`) is more ready than a 6-file loose collection (`mobile-*`). When a prefix collects files that change for *different* reasons, split — don't folder.
-Connection infrastructure (BLE, WiFi, USB-CDC) initializes before capability infrastructure (camera, perception, motors). When constrained resources force a tradeoff, connection wins. A robot whose BLE stays up with no camera is observable and actionable; the reverse is a brick.
+# Comment discipline
-ESP32 example: NimBLE host init and `wifi_sta_init` run early in `app_main()` so radio drivers pre-allocate their buffers in fresh internal heap. Camera comes after; if it can't fit its 32 KB DMA buffer in what's left, it fails loudly via `camera_init_error()` and `fw_info` hides the cap so the dashboard adapts.
+Every line is context cost in an AI-edited codebase. Comments earn their place when they carry WHY: hidden constraints, kernel/API gotchas, workarounds for past bugs, cross-file invariants ("must match `firmware/pi_robot/pi_robot.py`"), schema/wire-format examples. Restatement (module preambles, narration, section banners, labels above obvious code) is the cut.
-# Unit preconditions belong in the script, not in `Condition*`
+# Abstractions earn upstream consumers
-`ConditionPathExists=`, `ConditionFileNotEmpty=`, etc. evaluate **once** at unit-start time and silently skip the unit when false — no retry, no log noise the operator can search for, no recovery without manual `systemctl start`. When the prerequisite is racy (asynchronous kernel-driver probes, hotplug events, network reachability, anything not synchronously guaranteed by an `After=` ordering), a missed check turns the unit invisibly inert until the next reboot, and even that may race the same way.
+Before adding a logical layer, registry, wrapper, or routing decision, audit who outside its home module will use it. If only one module touches it, it's internal. The cost of an unused abstraction isn't only the lines it adds — it's the explanatory comments, the cross-cutting params plumbed through siblings, and the bug-shaped negative space (a one-shot helper turns into a parameter on every sibling, then the sibling that forgot to use it ships a regression). Audit before adding, not before deleting.
-Pattern instead: drop the `Condition*` and wait inside the `ExecStart` script with a bounded poll loop. The script makes the timeout legible (logs a clear failure on exhaustion), the unit gets to use `Restart=on-failure` for self-healing, and a future contributor can read the wait-condition next to the work it gates. The `usb-gadget.service` → `usb-gadget-setup.sh` pair is the reference shape: 10 s poll for `/sys/class/udc` to populate, clean exit-1 with a message if dwc2 never publishes.
+# Dialog vs menu dismiss
-If the precondition really is synchronous and unambiguous (a config file the user wrote, the existence of a hardware feature already enumerated at boot), `Condition*` is fine. The line is "does this become true asynchronously after the unit's `After=` ordering?" — if yes, wait in the script.
+- **Menus + popovers** (robot-menu, avatar-menu, help popovers, Pip's `
`): outside-click + Escape dismiss.
+- **Dialogs**: × button or Escape only. Outside-click would nuke session state (recovery terminal, SD prep) for a tiny convenience win.
+# References
+
+- `DEV.md` — URL flags, `window.*` handles, IndexedDB stores, common debug paths.
+- `SMOKE.md` — manual checklist for architectural promises.
+- `USER-CODE.md` — surface that `scripts.js` exposes to user-authored code.
+- `HARDWARE.md` — wiring, board-specific knobs.
+- `.claude/direction.md` — what we're committing to close for the course pilot.
+- `.claude/exploration.md` — open architectural directions, design rationale, wired-but-unproven inventory, forks evaluated.
+- `.claude/field.md` — positioning analysis vs adjacent work.
+- `firmware/esp32_robot_idf/WEBRTC.md` — the four coordinated DTLS/SDP patches.
+- `firmware/pi_robot/SYSTEMD.md` — preconditions-belong-in-the-script pattern.
+- `make smoke` — pure-function tests (<1 s); `make install-hooks` wires pre-commit (`make smoke` + gen-uuids drift + sw.js VERSION stamp), bypassable with `--no-verify`, CI is the binding layer.
diff --git a/.claude/direction.md b/.claude/direction.md
index 6e22c4d5..9404252a 100644
--- a/.claude/direction.md
+++ b/.claude/direction.md
@@ -1,229 +1,34 @@
-# Architectural direction — better-robotics
+# Direction
-Long-horizon shape decisions. Unlike `working.md` (tactical pending), this file names structural moves the project is committing to. Updated when the shape of the system changes.
+What we're committing to close for the Fall 2026 course pilot. Open exploration lives in `exploration.md`; positioning research in `field.md`.
-## 1. Generic typed-characteristic runtime (in flight)
+## Ranked gaps
-**Claim.** Every capability today exists in ~3 places (browser module, Pi
-handler, ESP32 handler). 80% of those files are boilerplate isomorphic to
-the capability's TYPE, not its identity. A generic runtime keyed on type
-eliminates the boilerplate.
+1. **Live parameters get/set.** Learning step 2 — "tune parameters live without editing code." Capabilities are already structured; add a param characteristic + getter/setter + a tuner row per capability card. Cheapest gap to close.
-**The data already exists.** `fw-info.caps` declares typed schemas:
+2. **Sensor/motor hot-plug auto-discovery.** Learning step 3 — "add physical components." Today capabilities are firmware-declared, not detected at runtime. The "smart breadboard" promise. Tractable on ESP32 with an i2c scan + a discovery characteristic.
-```json
-{ "name": "led", "char": "…d92", "type": "toggle" }
-{ "name": "motors", "char": "…d99", "type": "signed-pair", "range": [-100, 100] }
-{ "name": "wifi", "chars": {...}, "type": "wifi-scan" }
-{ "name": "ota", "chars": {...}, "type": "bundle-ota" }
-{ "name": "camera", "chars": {...}, "type": "webrtc-installable" }
-{ "name": "ops", "char": "…d9c", "type": "command" }
-```
+3. **Pub/sub vocabulary for messaging.** Learning step 5 and the explicit ROS2-transition story. Could be a topic layer over BLE notify, with MQTT as the "once WiFi is on" tier. Earns its keep only if the ROS2-prep framing stays.
-**The runtime (browser side).** A per-type constructor `makeXxxCap(schema)`
-returns `{probe, cleanup, renderSection, wireActions, postRender?}`. Adding
-a capability of a known type = one schema entry + zero JS code.
+4. **Discovery graph view.** Cheap once (3) lands. Makes pub/sub legible — pedagogical payoff for low effort.
-**Firmware-side direction (farther out).** Pi and ESP32 firmware have
-identical ceremony: register char, parse read/write, notify on change,
-gate on config. A "typed char runtime" on firmware reads the capability
-declaration and handles generic typed chars with a small driver binding
-per capability (`{ on_write: fn, on_read: fn }`).
+5. **Simulation hook.** A `MockRobot` so a student can iterate scripts without a working kit on the table. Classroom-critical; not everyone has hardware ready every session.
-**Progress so far:**
-- fw-info.caps carries the typed schema (shipped)
-- Browser reads + stores `entry.capSchema` (shipped)
-- Each capability module exports its own `schema` for cross-check (shipped)
-- **First type migrated: `toggle` → LED** (this session)
-- Future types to migrate: `signed-pair`, `wifi-scan`, `bundle-ota`,
- `webrtc-installable`, `command`. Each is ~2–4 hours.
+## NFC tap-to-pair
-**Migration strategy.** Per-type, not per-capability. When we migrate
-`signed-pair`, both motors AND any future 2-axis input use the same
-runtime. The compound payoff is the Nth capability, not the first.
+The plan's original NFC role (handing the phone the puck's SoftAP creds) is dead post-BLE-first. Tags can still earn their keep as a *tap-to-pair-this-specific-robot* shortcut — collapses "scan → find robot-7 in a list of 12 → confirm" to a single tap.
-## 2. AI-maintained documentation (cheap, deferred)
+- **Tag content:** NDEF URL → `https://better-robotics.github.io/?pair=`. Dashboard reads `pair` from `location.search`, filters BLE scan to that device.
+- **Android Chrome:** tap → URL → filtered scan → confirm.
+- **iPhone:** iOS opens the URL but Web Bluetooth is unavailable. Workaround uses the existing phone↔desktop pair layer (`signal.neevs.io`, signed pair-request, `phone.html`): encode `phone.html?pair=`. Phone forwards `{type:"pair-robot", robotId}` over WebRTC; desktop surfaces a "Phone wants to pair robot-7 — click to confirm" banner. Desktop click is required because `navigator.bluetooth.requestDevice` needs a user gesture. Cross-network works for free.
+- **Bootstrap caveat:** first-ever use still needs the existing phone↔desktop pair ceremony.
-**Claim.** `README.md`, `HARDWARE.md`, `firmware/pi_robot/README.md`, and
-per-capability comments all describe what `fw-info.caps` + the code
-already know. They drift. An AI agent watching the schema + commit log
-can regenerate docs per release.
+~An afternoon to prototype. Concrete iOS+NFC demo defuses the "BLE-first leaves iPhones out" objection.
-**Scope.** ~2 days to wire a pre-commit generator plus a CI check that
-fails if docs aren't regenerated. Starts small: capability reference
-page auto-generated from the live schema. Expands to change-log
-summarization from commit messages.
+## Other gaps
-**Not urgent.** Doc drift isn't causing failures today. Worth doing
-when the project has contributors outside the core, or when we promise
-backward-compatibility guarantees that require accurate docs.
+- **Visual / block-based authoring tier.** Today: capability cards (drive motors, toggle LED) and `pip.ask` natural language — no block-editor surface for "when distance < 30cm, stop and turn right." XRP and MicroBlocks (see `.claude/field.md`) ship Blockly. Open question: do cards + Pip cover non-coder authoring, or is a drag-drop tier needed?
-## 3. Transparent-data-plane OTA (partially in flight)
+- **Inter-puck messaging.** Every message fans out through the browser as hub — no puck↔puck path. Tied to (3) but a separate architectural commitment.
-**Claim.** Every robot should have three OTA lanes with a clear fallback
-order. The dashboard picks the fastest available without user
-intervention. Iteration-loop speed is the core dev experience; "how fast
-does code get onto the robot" sets the tone for everything else.
-
-**The three lanes, decreasing friction:**
-
-1. **BLE-stream** — always works, no WiFi needed, no LAN co-location
- required. Baseline for every robot on every network.
- Today: `writeValueWithResponse` + ATT ack per 180-byte frame →
- 3-10 min for a 1.6 MB bin. Switching to
- `writeValueWithoutResponse` + software flow control over
- `ota-status` gets it to ~30 sec. **Not yet implemented.**
-
-2. **PNA direct to target robot** — dashboard fetches
- `http:///ota` straight from the browser. Chrome/Edge's
- Private Network Access (shipped 2022) gates the first request on a
- one-time user consent per origin. No TLS on the robot, no cert
- ceremony, no crypto IRAM pressure. ~1 sec for a 1.6 MB bin over
- LAN. Works whenever the dashboard and robot share a network.
- **Not yet implemented on ESP32** (Pi doesn't need this lane — BLE
- bundle OTA is already fast enough for Pi-sized updates).
-
-3. **Pi-as-gateway** — for multi-robot orchestration and offline-first
- classroom deployments. Pi runs an `aioquic` WebTransport server
- with a self-signed cert; dashboard uses `serverCertificateHashes`
- pinning (cert sha256 published in Pi's fw-info) to connect
- without PKI ceremony. Pi proxies raw TCP to the target ESP32 on
- the LAN. Same ~1 sec speed as PNA direct, with bonus orchestration
- surface (mesh multiple ESP32s, serve dashboard offline).
- **Not yet implemented.** Earns its slot when multi-robot coord or
- offline-first use cases land, not purely for OTA speed.
-
-**Why the three-lane shape is right:**
-- Lane 1 works on BLE only. No WiFi assumption.
-- Lane 2 works when browser and robot share a LAN. Most common case.
-- Lane 3 works when the fleet has a Pi (most Better Robotics fleets do).
-
-Dashboard tries fastest available, falls back automatically. User never
-picks a lane — it just updates as fast as the topology allows.
-
-**What's baked in vs what's not:**
-- BLE-stream as a baseline works today (for Pi bundle OTA; for ESP32
- single-binary OTA, the WithResponse variant is live and slow).
-- ESP32 already runs a raw `WiFiServer` (for MJPEG) — adding a `/ota`
- endpoint on the same task is near-zero new code on the firmware side.
-- Pi-as-gateway is purely additive to `pi_robot.py` — every Pi ships
- with it, no opt-in, just one more capability.
-- Dashboard-side lane selection: not yet written. Attempts lanes in
- order, falls back on timeout/error.
-
-**Sequencing:**
-1. BLE-WithoutResponse first (universal, smallest change).
-2. PNA + ESP32 `/ota` endpoint second (big bang for effort).
-3. Pi-as-gateway when its orchestration/offline story earns it.
-
-## 4. ESP32 build-as-a-service (bold, later)
-
-**Claim.** ESP32 firmware is purely deterministic from `{board, caps}`.
-Users currently install `arduino-cli` + core + toolchain to compile.
-If a service accepts a config and returns a signed `.bin`, the dashboard's
-"Flash firmware" button fetches a per-robot-config binary; no local dev
-environment is needed for adding capabilities.
-
-**Constraint.** The service has to be reliable enough that users aren't
-stuck if it's down. Either (a) same-origin build on GitHub Actions, or
-(b) a small hosted build service, or (c) in-browser compile via
-something like Wokwi's WebAssembly toolchain (the bold option).
-
-**The compound effect.** Combined with #1, adding an ESP32 capability
-becomes: declare schema, bind driver code in a capability driver DSL,
-click Flash. No C++, no toolchain, no linker flags.
-
-**Worth it when.** Project has contributors who want to add capabilities
-without learning the ESP32 toolchain. Today the audience is small enough
-that `make flash` is fine.
-
-## 5. Closed-loop visual control: draw-a-path (next, after overhead ArUco validates)
-
-**Claim.** Once an overhead camera + marker is established (the
-`aruco.js` work — overhead localization writing `entry.arucoPosition`
-per scan), the natural next layer is closed-loop control driven from
-that pose. Operator props the phone (or local webcam) overhead, finger-
-draws a path on the phone screen, the robot follows it. New sensor
-isn't needed; the pose primitive is already shipping.
-
-**The hard sub-problem isn't drawing or motor control — it's pose
-reliability.** Without knowing where the robot is each frame, the
-closed loop doesn't close and the robot drifts within seconds. The
-overhead ArUco surface gates this — until metric accuracy is
-validated against tape-measure ground truth (see `.claude/notes.md` →
-"Wired but unproven"), don't build the follower on top.
-
-**Right primitives, in order of load-bearing-ness:**
-- **Pose**: ArUco overhead, already shipped. Producer writes
- `entry.arucoPosition`; consumer (this work) must gate on
- `Date.now() - updatedAt` for staleness.
-- **Where compute lives**: dashboard runs detector + controller +
- emits pulse-bounded BLE motor writes. Phone is I/O. Robot
- unchanged. Same control-plane / data-plane split as everything else.
-- **Tech**: `js-aruco2` already in. Pure-pursuit controller in plain
- JS (~50 lines).
-- **Control loop budget**: detect (~15 ms) + plan (~1 ms) + BLE
- pulse (~50 ms) ≈ 70 ms / iter → ~14 Hz. Each iteration emits a
- short pulse (`duration_ms ≈ 100 ms`); firmware watchdog auto-stops
- if the next iter doesn't arrive. The existing pulse-bounded-motion
- + watchdog invariants are the safety floor — same discipline as
- Pip / user scripts.
-
-**Phases:**
-1. **Path source.** `
@@ -328,7 +328,7 @@
- JS that drives connected robots over BLE. Runs in this tab — nothing is uploaded. In scope: robot, robots, phones, pip, sleep, log, speak. Cmd/Ctrl-Enter to run. USER-CODE.md.
+ JS that drives connected robots over BLE. Runs in this tab — nothing is uploaded. In scope: robot, robots, phones, pip, sleep, log, speak. Cmd/Ctrl-Enter to run. USER-CODE.md.
diff --git a/docs/gamepad.js b/docs/input/gamepad.js
similarity index 93%
rename from docs/gamepad.js
rename to docs/input/gamepad.js
index eb691a1a..bbf57c58 100644
--- a/docs/gamepad.js
+++ b/docs/input/gamepad.js
@@ -1,8 +1,8 @@
// Polling stops when the last pad disconnects so idle cost is zero.
-import { $ } from "./dom.js";
-import { log } from "./log.js";
-import { state } from "./state.js";
-import { sendPairById } from "./capabilities/runtime/signed-pair.js";
+import { $ } from "../dom.js";
+import { log } from "../log.js";
+import { state } from "../state.js";
+import { sendPairById } from "../capabilities/runtime/signed-pair.js";
const GAMEPAD_DEADZONE = 0.10;
let _gamepadTargetId = null;
diff --git a/docs/joypad.js b/docs/input/joypad.js
similarity index 100%
rename from docs/joypad.js
rename to docs/input/joypad.js
diff --git a/docs/mobile-tilt-drive.js b/docs/input/mobile-tilt-drive.js
similarity index 99%
rename from docs/mobile-tilt-drive.js
rename to docs/input/mobile-tilt-drive.js
index 2216734a..1a8844d5 100644
--- a/docs/mobile-tilt-drive.js
+++ b/docs/input/mobile-tilt-drive.js
@@ -1,4 +1,4 @@
-import { $ } from "./dom.js";
+import { $ } from "../dom.js";
import { mix } from "./joypad.js";
// Phone-as-steering-wheel + on-screen throttle pedals. Rolling the phone
diff --git a/docs/log.js b/docs/log.js
index b4d369ed..7d6e6f85 100644
--- a/docs/log.js
+++ b/docs/log.js
@@ -1,6 +1,5 @@
import { $ } from "./dom.js";
-let _lastLogNode = null;
let _lastLogMsgNode = null;
let _lastLogNameNode = null;
let _lastLogKey = null;
@@ -90,7 +89,6 @@ export const log = (msg, name = "") => {
if (name && name === _lastLogName && _lastLogNameNode) {
_lastLogNameNode.classList.add("dup");
}
- _lastLogNode = line;
_lastLogMsgNode = msgSpan;
_lastLogNameNode = nameSpan;
_lastLogName = name;
diff --git a/docs/mobile.js b/docs/mobile.js
index e04449c9..d48f5307 100644
--- a/docs/mobile.js
+++ b/docs/mobile.js
@@ -1,17 +1,17 @@
import { $ } from "./dom.js";
-import { joinPairingRoom } from "./pairing.js";
-import { attachJoypad } from "./joypad.js";
+import { joinPairingRoom } from "./pair/pairing.js";
+import { attachJoypad } from "./input/joypad.js";
import { getMyPubkeyB64 } from "./signal-sdk/v1/peer-key.js";
import { makeTrustStore } from "./trust.js";
import {
setupServiceWorker, wireInstallMenuItem, wireCheckUpdatesMenuItem,
wireHardRefresh, wireDiagnosticsMenuItem, setReportIssueLink, readSwVersion,
} from "./app-menu.js";
-import { wireTiltDrive, stopTilt } from "./mobile-tilt-drive.js";
+import { wireTiltDrive, stopTilt } from "./input/mobile-tilt-drive.js";
import {
showReconnect, hideReconnect, wireReconnect, cameraUnavailableReason,
-} from "./mobile-qr-scan.js";
-import { startNearbyDiscovery, deviceLabel } from "./mobile-nearby-discovery.js";
+} from "./pair/mobile-qr-scan.js";
+import { startNearbyDiscovery, deviceLabel } from "./pair/mobile-nearby-discovery.js";
const _trust = makeTrustStore("better-robotics:trust:v1");
let _peer = null;
@@ -67,38 +67,54 @@ function wireStopButton() {
}
-// Wire: see askHuman() in phones.js. One ask on screen at a time; a second
-// replaces the first, prior resolves as skipped when its server-side timer
-// fires.
-function showAsk(msg) {
+// Shared phone-side dialog for ask-human and camera-share-request.
+// One dialog on screen at a time; a second showPhoneAskDialog call
+// replaces the first (the prior's pending response resolves through
+// the server-side timeout). `options` is either:
+// - array of strings → tappable answer buttons (each calls onRespond
+// with its label, once)
+// - array of {label, onClick} → custom click handler (e.g. for the
+// camera-share Share button that needs to run async work)
+// `freeText` enables the text input fallback when no options exist.
+function showPhoneAskDialog({ question, imageDataUrl, options, freeText, skipValue, onRespond }) {
const dialog = $("phone-ask-dialog");
const img = $("phone-ask-image");
const q = $("phone-ask-question");
const optsEl = $("phone-ask-options");
const free = $("phone-ask-free");
const freeInput = $("phone-ask-free-input");
-
- if (msg.imageDataUrl) { img.src = msg.imageDataUrl; img.hidden = false; }
- else { img.hidden = true; img.src = ""; }
- q.textContent = msg.question || "";
-
+ let responded = false;
+ const close = () => { if (!responded) { responded = true; dialog.close(); } };
const respond = (answer) => {
- _peer?.send({ type: "ask-reply", askId: msg.askId, answer });
+ if (responded) return;
+ responded = true;
+ onRespond(answer);
dialog.close();
};
+ if (imageDataUrl) { img.src = imageDataUrl; img.hidden = false; }
+ else { img.hidden = true; img.src = ""; }
+ q.textContent = question || "";
+
optsEl.innerHTML = "";
- if (Array.isArray(msg.options) && msg.options.length > 0) {
- free.hidden = true;
- for (const opt of msg.options) {
+ const hasOptions = Array.isArray(options) && options.length > 0;
+ if (hasOptions) {
+ for (const opt of options) {
const b = document.createElement("button");
b.type = "button";
b.className = "ask-option sm";
- b.textContent = String(opt);
- b.addEventListener("click", () => respond(String(opt)), { once: true });
+ if (typeof opt === "string") {
+ b.textContent = opt;
+ b.addEventListener("click", () => respond(opt), { once: true });
+ } else {
+ b.textContent = opt.label;
+ b.addEventListener("click", () => opt.onClick({ respond, close }), { once: true });
+ }
optsEl.appendChild(b);
}
- } else {
+ }
+
+ if (freeText && !hasOptions) {
free.hidden = false;
freeInput.value = "";
free.onsubmit = (e) => {
@@ -106,78 +122,58 @@ function showAsk(msg) {
const v = freeInput.value.trim();
if (v) respond(v);
};
+ } else {
+ free.hidden = true;
}
- $("phone-ask-skip").onclick = () => respond(null);
+ $("phone-ask-skip").onclick = () => respond(skipValue);
if (!dialog.open) dialog.showModal();
- // Autofocus the free input when there are no tappable options, so the
- // keyboard pops up immediately on mobile.
- if (free.hidden === false) setTimeout(() => freeInput.focus(), 50);
+ // Autofocus the free input so the soft keyboard pops up on mobile.
+ if (!free.hidden) setTimeout(() => freeInput.focus(), 50);
}
-// Desktop relayed a "please share your camera" prompt over the data
-// channel. Browsers won't let getUserMedia() run without a user gesture
-// in this tab; the Share button click below IS that gesture, so
-// toggleShareCamera() called synchronously from the handler can call
-// getUserMedia successfully. The handler awaits the share flow and
-// reports back so the desktop's startHelperCamera tool can resolve
+function showAsk(msg) {
+ showPhoneAskDialog({
+ question: msg.question,
+ imageDataUrl: msg.imageDataUrl,
+ options: msg.options,
+ freeText: true,
+ skipValue: null,
+ onRespond: (answer) => _peer?.send({ type: "ask-reply", askId: msg.askId, answer }),
+ });
+}
+
+// Browsers won't let getUserMedia() run without a user gesture in
+// this tab; the Share button click below IS that gesture. The handler
+// reports back so the desktop's startHelperCamera tool resolves
// instead of dead-ending on a string error.
-//
-// Reuses the phone-ask-dialog DOM rather than introducing a parallel
-// modal — same Share / Not now affordance shape as askHuman.
function showCameraShareRequest(msg) {
- const dialog = $("phone-ask-dialog");
- const img = $("phone-ask-image");
- const q = $("phone-ask-question");
- const optsEl = $("phone-ask-options");
- const free = $("phone-ask-free");
- let responded = false;
-
- img.hidden = true; img.src = "";
- q.textContent = "Pip wants to use this phone's camera. Share it?";
- free.hidden = true;
- optsEl.innerHTML = "";
-
- const respond = (result, error) => {
- if (responded) return;
- responded = true;
- _peer?.send({ type: "camera-share-result", requestId: msg.requestId, result, error });
- dialog.close();
- };
-
- const shareBtn = document.createElement("button");
- shareBtn.type = "button";
- shareBtn.className = "ask-option sm";
- shareBtn.textContent = _shareStream ? "Already sharing" : "Share camera";
- shareBtn.addEventListener("click", async () => {
- // Already-sharing fast path — desktop sometimes asks before its
- // onTrack handler has registered the stream we already sent.
- if (_shareStream) { respond("shared"); return; }
- // toggleShareCamera() awaits getUserMedia internally; the user
- // gesture from this click propagates through the first await per
- // the user-activation spec, so the permission dialog (if any) is
- // allowed to show. The {ok, error} return surfaces the real
- // permission / device error to the desktop instead of collapsing
- // every failure to "user dismissed".
- try {
- const res = await toggleShareCamera();
- if (res?.ok) respond("shared");
- else respond("error", res?.error || "getUserMedia returned no stream");
- } catch (err) {
- respond("error", err.message || String(err));
- }
- }, { once: true });
- optsEl.appendChild(shareBtn);
-
- const cancelBtn = document.createElement("button");
- cancelBtn.type = "button";
- cancelBtn.className = "ask-option sm";
- cancelBtn.textContent = "Not now";
- cancelBtn.addEventListener("click", () => respond("denied"), { once: true });
- optsEl.appendChild(cancelBtn);
-
- $("phone-ask-skip").onclick = () => respond("denied");
- if (!dialog.open) dialog.showModal();
+ const send = (result, error) => _peer?.send({
+ type: "camera-share-result", requestId: msg.requestId, result, error,
+ });
+ showPhoneAskDialog({
+ question: "Pip wants to use this phone's camera. Share it?",
+ skipValue: "denied",
+ onRespond: (answer) => send(answer ?? "denied"),
+ options: [
+ {
+ label: _shareStream ? "Already sharing" : "Share camera",
+ onClick: async ({ respond, close }) => {
+ // Desktop sometimes asks before its onTrack handler has
+ // registered the stream we already sent — short-circuit.
+ if (_shareStream) { respond("shared"); return; }
+ try {
+ const res = await toggleShareCamera();
+ if (res?.ok) respond("shared");
+ else { send("error", res?.error || "getUserMedia returned no stream"); close(); }
+ } catch (err) {
+ send("error", err.message || String(err)); close();
+ }
+ },
+ },
+ { label: "Not now", onClick: ({ respond }) => respond("denied") },
+ ],
+ });
}
// Pairing layer fires onTrack per track; both video tracks of one stream
@@ -185,18 +181,25 @@ function showCameraShareRequest(msg) {
function onPeerTrack(e) {
const v = $("phone-cam");
const section = $("phone-cam-section");
+ const waiting = $("phone-cam-waiting");
const stream = e.streams?.[0];
if (!stream) return;
if (v.srcObject !== stream) v.srcObject = stream;
section.hidden = false;
- // When the remote ends the track (laptop user clicked Stop), hide the
- // section so the phone doesn't show a frozen last frame as if it were live.
+ if (waiting) waiting.hidden = true;
+ // When the remote ends the track (laptop user clicked Stop), surface
+ // the "no stream" state instead of a frozen last frame. In operator-
+ // cam mode that means resurrecting the waiting overlay; in default
+ // mode it means hiding the whole section.
for (const t of stream.getTracks()) {
t.addEventListener("ended", () => {
- // If all tracks are ended, hide. Other tracks may still be live.
if (stream.getTracks().every(t2 => t2.readyState === "ended")) {
- section.hidden = true;
v.srcObject = null;
+ if (_currentScreenMode === "operator-cam") {
+ if (waiting) waiting.hidden = false;
+ } else {
+ section.hidden = true;
+ }
}
});
}
@@ -262,6 +265,7 @@ function renderCameraPicker() {
function onPeerMessage(msg) {
if (msg.type === "ask") { showAsk(msg); return; }
if (msg.type === "request-camera-share") { showCameraShareRequest(msg); return; }
+ if (msg.type === "screen-mode") { applyScreenMode(msg.mode, msg.robotLabel); return; }
if (msg.type === "available-sources") {
_availableSources.set(msg.robotId, {
sources: msg.sources || [], active: msg.active || null,
@@ -295,6 +299,72 @@ function onPeerMessage(msg) {
}
}
+// Phone-on-robot screen modes (set by the desktop via setPhoneScreenMode
+// when the operator mounts a phone via attachPhoneCameraTo):
+// "operator-cam" — fullscreen incoming video (the operator's face if
+// a local cam's "Send to phone" role is on; black otherwise).
+// "default" — normal operator companion UI.
+// In attached mode the sticky Stop button stays visible (semi-
+// transparent) so anyone in the room can still halt the robot. Desktop
+// owns the choice; the phone has no local override. Reset on peer.
+// onClose so a disconnect leaves the user with normal UI to reconnect.
+let _currentScreenMode = "default";
+function applyScreenMode(mode, robotLabel) {
+ const body = document.body;
+ const section = $("phone-cam-section");
+ const waiting = $("phone-cam-waiting");
+ const waitingName = $("phone-cam-waiting-name");
+ const v = $("phone-cam");
+ if (mode === _currentScreenMode) {
+ body.dataset.attachedTo = robotLabel || "";
+ if (waitingName) waitingName.textContent = robotLabel || "this robot";
+ return;
+ }
+ body.classList.remove("phone-mounted", "phone-attached");
+ delete body.dataset.attachedTo;
+ if (mode === "operator-cam") {
+ body.classList.add("phone-mounted", "phone-attached");
+ body.dataset.attachedTo = robotLabel || "";
+ // Surface the camera section even before a track lands so the screen
+ // isn't an unlabeled black void. Hide the section's normal "tap to
+ // switch source" affordance; the waiting overlay takes over.
+ if (section) section.hidden = false;
+ const hasStream = !!v?.srcObject;
+ if (waiting) waiting.hidden = hasStream;
+ if (waitingName) waitingName.textContent = robotLabel || "this robot";
+ } else {
+ // Leaving attached mode: the section's visibility goes back to
+ // "shown only when a stream is present" (onPeerTrack toggles it).
+ if (section && !v?.srcObject) section.hidden = true;
+ if (waiting) waiting.hidden = true;
+ }
+ _currentScreenMode = mode === "operator-cam" ? mode : "default";
+ // Keep the phone screen on while it's mounted on a robot — otherwise
+ // iOS dims and locks after ~30s of no tap, which breaks the operator-
+ // cam relay. Acquire unconditionally; iOS may ignore without a recent
+ // gesture, in which case the visibilitychange handler retries when
+ // the user returns.
+ if (_currentScreenMode === "default") releaseWakeLock();
+ else acquireWakeLock();
+}
+
+// Screen Wake Lock — held while the phone is attached to a robot.
+// Auto-released by the browser when the tab is backgrounded; we clear
+// the ref on visibility=hidden and re-acquire on visibility=visible if
+// still attached. iOS requires the request to land near a user gesture
+// for first-time acquire; pairing tap chain usually satisfies this.
+let _wakeLock = null;
+async function acquireWakeLock() {
+ if (_wakeLock || !("wakeLock" in navigator)) return;
+ try { _wakeLock = await navigator.wakeLock.request("screen"); }
+ catch { _wakeLock = null; }
+}
+async function releaseWakeLock() {
+ if (!_wakeLock) return;
+ try { await _wakeLock.release(); } catch {}
+ _wakeLock = null;
+}
+
function wireJoypad() {
const pad = $("phone-joypad");
const knob = pad?.querySelector(".joypad-knob");
@@ -317,6 +387,11 @@ function wireBackgroundStop() {
stopTilt();
_peer?.send({ type: "drive", l: 0, r: 0 });
_stopSharing();
+ // Browser auto-releases the wake lock on background; drop the ref
+ // so a re-acquire on return doesn't short-circuit.
+ _wakeLock = null;
+ } else if (_currentScreenMode !== "default") {
+ acquireWakeLock();
}
});
}
@@ -680,6 +755,10 @@ async function init() {
$("phone-cam-section").hidden = true;
_stopSharing();
$("phone-share").hidden = true;
+ // Exit attached-mode on disconnect so the user lands on normal UI
+ // to reconnect from. Desktop will re-send "attached" on reconnect
+ // if this phone was mounted (see phones.js phone-connect path).
+ applyScreenMode("default");
showReconnect("Lost the desktop. Scan a fresh QR to reconnect.");
startNearbyDiscovery();
});
diff --git a/docs/mobile-nearby-discovery.js b/docs/pair/mobile-nearby-discovery.js
similarity index 96%
rename from docs/mobile-nearby-discovery.js
rename to docs/pair/mobile-nearby-discovery.js
index f7742f59..998851aa 100644
--- a/docs/mobile-nearby-discovery.js
+++ b/docs/pair/mobile-nearby-discovery.js
@@ -1,7 +1,7 @@
-import { $ } from "./dom.js";
-import { discover } from "./signal-sdk/v1/discover.js";
-import { getMyPubkeyB64 } from "./signal-sdk/v1/peer-key.js";
-import { pairRequestClient } from "./signal-sdk/v1/pair-request.js";
+import { $ } from "../dom.js";
+import { discover } from "../signal-sdk/v1/discover.js";
+import { getMyPubkeyB64 } from "../signal-sdk/v1/peer-key.js";
+import { pairRequestClient } from "../signal-sdk/v1/pair-request.js";
// LAN discovery — request/accept flow.
//
diff --git a/docs/mobile-qr-scan.js b/docs/pair/mobile-qr-scan.js
similarity index 99%
rename from docs/mobile-qr-scan.js
rename to docs/pair/mobile-qr-scan.js
index 74b64ad6..da264dbd 100644
--- a/docs/mobile-qr-scan.js
+++ b/docs/pair/mobile-qr-scan.js
@@ -1,4 +1,4 @@
-import { $ } from "./dom.js";
+import { $ } from "../dom.js";
let _scanStream = null;
let _scanRaf = 0;
diff --git a/docs/pairing.js b/docs/pair/pairing.js
similarity index 99%
rename from docs/pairing.js
rename to docs/pair/pairing.js
index 00ae42bb..8a59e8ef 100644
--- a/docs/pairing.js
+++ b/docs/pair/pairing.js
@@ -2,8 +2,7 @@
// hard failure (channel closed and ICE restart didn't recover within the
// grace window) counts as "disconnected, rescan QR".
//
-// Signal protocol (~/Github/jonasneves/signal/src/server/room.js):
-// connect wss://signal.neevs.io/{room}/ws
+// Signal protocol (wss://signal.neevs.io/{room}/ws):
// send { type: "signal", peer: myPeerId, data: { offer|answer|ice } }
// recv { type: "state", peers: { peerId: lastSignal } } // once, on connect
// { type: "signal", peer: theirPeerId, data: {...} }
@@ -13,7 +12,7 @@
// fixed role key. The server's `state` snapshot recovers signals sent
// before late-joiners arrive; applied only when we're not already on a
// healthy connection.
-import { SIGNAL_WS, TURN_URL } from "./endpoints.js";
+import { SIGNAL_WS, TURN_URL } from "../endpoints.js";
// TURN proxy mints short-lived Cloudflare Realtime creds. STUN stays in
// line as a zero-roundtrip fallback so a degraded proxy (offline, rate-
// limited, mis-deployed) still gives us STUN-only pairing instead of nothing.
@@ -51,7 +50,7 @@ const QUEUE_MAX = 1000;
// as long as the dialog is — cleanup happens on dialog close.
const ICE_TIMEOUT_MS = 30000;
-import { parseCandidate, probeNetwork } from "./net-probe.js";
+import { parseCandidate, probeNetwork } from "../net-probe.js";
// Per-attempt diagnostic capture: every local + remote ICE candidate this
// side has seen during the most recent pair attempt. The Diagnostics
diff --git a/docs/phone-helpers.js b/docs/pair/phone-helpers.js
similarity index 94%
rename from docs/phone-helpers.js
rename to docs/pair/phone-helpers.js
index 820981f0..bd1c5131 100644
--- a/docs/phone-helpers.js
+++ b/docs/pair/phone-helpers.js
@@ -1,8 +1,9 @@
-import { $, escapeHtml } from "./dom.js";
+import { $, escapeHtml } from "../dom.js";
import { listPhones, setPhonesChangeHandler, notifyRobotStreamChange, requestPhoneCameraShare, setPhoneFeedStream } from "./phones.js";
-import { state } from "./state.js";
-import { settings, saveSettings } from "./settings.js";
-import { setOverheadSource, clearOverheadSource } from "./aruco.js";
+import { emit as busEmit, TOPICS } from "../event-bus.js";
+import { state } from "../state.js";
+import { settings, saveSettings } from "../settings.js";
+import { setOverheadSource, clearOverheadSource } from "../perception/aruco.js";
// Permanent print-marker affordance, rendered whenever a helper is the
// active overhead source. Single source of truth (no duplication into
@@ -51,6 +52,7 @@ export function initHelpers() {
if (navigator.mediaDevices?.addEventListener) {
navigator.mediaDevices.addEventListener("devicechange", () => enumerateLocalCameras());
}
+ wireDelegation();
render();
}
@@ -222,6 +224,7 @@ export function attachPhoneCameraTo(phoneId, robotId) {
}
if (!robotId) {
_phoneAttachments.delete(phoneId);
+ busEmit(TOPICS.PHONE_DETACHED, { phoneId });
} else {
if (settings.arucoOverheadPhoneId === phoneId) {
settings.arucoOverheadPhoneId = null;
@@ -230,6 +233,8 @@ export function attachPhoneCameraTo(phoneId, robotId) {
_phoneAttachments.set(phoneId, robotId);
const ps = _phoneStreams.get(phoneId);
if (ps?.stream) routeAttachedStream(phoneId, ps.stream);
+ const robot = state.devices.get(robotId);
+ busEmit(TOPICS.PHONE_ATTACHED, { phoneId, robotId, robotLabel: robot?.name || null });
}
render();
}
@@ -359,7 +364,7 @@ function renderPhoneCard(p) {
// robot card (see attachPhoneCameraTo callers in app.js). The mounted
// status surfaces in the meta line above.
const currentRole = isOverhead ? "overhead" : "operator";
- const picker = (live && !attachedRobot) ? `
+ const rolePicker = (live && !attachedRobot) ? `
` : "";
+ // Attached mode has one option (operator-cam) now that pip-face is
+ // extracted to its own repo; no picker needed.
+ const picker = rolePicker;
+
// Preview tile lives here whenever the stream isn't mounted on a robot.
// When overhead is designated, an SVG overlay paints detected markers
// on the same tile — no second video element, no duplicate decode.
@@ -680,21 +689,31 @@ function paintOverhead(helperId, { markers, frameCount, error }) {
}
}
+// One delegated change-listener on the helpers list so we don't re-
+// attach per-element handlers every time render() rebuilds the
+// innerHTML. Wired once from initHelpers; subsequent renders inherit.
+function wireDelegation() {
+ const list = $("helpers-list");
+ if (!list) return;
+ list.addEventListener("change", (e) => {
+ const sel = e.target.closest("select[data-action]");
+ if (!sel) return;
+ switch (sel.dataset.action) {
+ case "phone-role":
+ setPhoneRole(sel.dataset.phoneId, sel.value);
+ return;
+ case "local-role":
+ // Empty value = placeholder = no role active; setter takes "off".
+ setLocalCameraRole(sel.dataset.localId, sel.value || "off");
+ return;
+ }
+ });
+}
+
function wire() {
const list = $("helpers-list");
if (!list) return;
- list.querySelectorAll('[data-action="phone-role"]').forEach(sel => {
- sel.addEventListener("change", () => {
- setPhoneRole(sel.dataset.phoneId, sel.value);
- });
- });
- list.querySelectorAll('[data-action="local-role"]').forEach(sel => {
- sel.addEventListener("change", () => {
- // Empty value = placeholder = no role active; setter takes "off".
- setLocalCameraRole(sel.dataset.localId, sel.value || "off");
- });
- });
// Mount the live MediaStream into the freshly-rendered