XR Touchpad Integration: Scroll Mode, Inertia, and Multi-layer Input Pipeline

[EndlessJour9527]

Background

JSAR Runtime (Spatial Web Browser Engine) requires comprehensive touchpad input handling to align with modern XR devices. The specific scenario involves a device with a single-finger 2D touchpad surface (no pressure sensing), multi-finger capability (reserved for mode switching), and a pipeline involving Unity C# → Unity C++ plugin → shared memory (TrXRInputSource) → C++ client (XRInputSource) → V8 JS API. The design should:

• Default to Scroll mode: touchpad slide emits WheelEvent for DOM scrolling, with kinetic inertia enabled by default (decay constant ~300ms, config option)
• Only Scroll mode is enabled by default; cursor/hover modes reserved for future, togglable via two-finger gesture/long-press (feature flag)
• Multi-finger input reserved for fast mode switching, not multi-touch gestures by default
• Double-click/Long-press: pipeline and design placeholder provided, disabled by default (merge later)
• No pressure mapping; touchpad click = pressed/touched boolean
• Event synthesis respects 16ms throttle, clamp, zero-offset early exit for perf (see docs/internals/DOM_SCROLLING_PERFORMANCE.md)
• Should expose axes/buttons via WebXR Gamepads Module (XRInputSource.gamepad) for standards compliance
• Align input event timing with XR pose (targetRay) pipeline and shared memory updates

Solution Proposal

1. Input Flow

   • Unity C# (collect touchpad states: x, y, touching, pressed, finger-count)
   • Unity plugin API (C++): Write input state to shared memory (TrXRInputSource)
   • C++ client (XRInputSource): Read shared memory each frame; merge into XRInputSource.gamepad + event synthesis
   • V8 bindings: Expose gamepad on JS XRInputSource and synthesize WheelEvent to DOM

2. WebXR Compliance

   • Implement WebXR Gamepad mapping:
     • axes[0]/axes[1]: touchpad normalized coords [-1,1] with deadzone
     • buttons[0].touched = finger contact; pressed = physical click
     • No pressure value (value 0/1); multi-finger only for mode toggle
   • Double-click/long-press event recognition pipeline to be reserved, design spec aligned with standard DOM events

3. DOM Event Synthesis

   • Scrollable DOM target: touchpad slide emits WheelEvent (deltaX, deltaY) at 16ms intervals, with inertia (exponential decay τ=300ms)
   • Non-scroll target (future): pointermove/cursor event (behind flag)
   • Early exit for zero offset, clamp values, reuse perf infra

4. Config and Feature Flags

   • Defaults: Scroll mode only, inertia enabled, τ=300ms, scale=40px/unit
   • Cursor mode, double-click, long-press: placeholder in code, off by default
   • Runtime config: allow developer/UA to change mode/inertia

5. Testing and Fixtures

   • HTML test pages: vertical/horizontal scroll, live gamepad readout
   • Document perf with scroll event throttle, inertia curve, mode switch triggers

Related Docs/Code

• docs/internals/DOM_SCROLLING_PERFORMANCE.md: Scroll event throttle/clamp/early exit
• docs/contributing/programming_languages.md: Native C++ API pipeline, V8 bindings
• README: WebXR Inputs, Gamepad support status

Web Standards Reference & Implementation

• WebXR Gamepads Module Level 1
• WheelEvent
• Pointer Events Level 3
• Chromium: device/vr/public/cpp/xr_gamepad_data.h, blink/renderer/modules/xr/xr_input_source.*

Acceptance Criteria

• Device touchpad input integrated via Unity → C++ → shared memory → client → JS pipeline
• Scroll mode with inertia enabled and functional as default, throttle and performance profile matches spec
• WebXR Gamepad exposed correctly, DOM WheelEvent synthesized from input with correct timing
• Double-click/long-press/hover mode designed and pipelined for future merge but off by default
• Code, config, and docs updated; test pages for manual QA