Skip to content

Android: automatic brightness from local mic (LiteRT beat tracking) #804

@SlyWombat

Description

@SlyWombat

Feature

Android-app automatic master-brightness driven by the phone's local microphone, using LiteRT (TensorFlow Lite Runtime) on devices that support it to do real-time audio analysis. Map the dominant bass-beat envelope to visible brightness changes so the rig pulses with the music in the room without any external audio routing or DAW.

Behaviour

  • Operator opens Stage tab. The current brightness slider is replaced with a modal button that toggles "Auto Brightness" on/off (and shows current state — idle / listening / clipping / no-mic).
  • When enabled:
    • Mic stream is captured at the lowest viable sample rate for beat detection (e.g. 8 kHz mono, AGC off).
    • On supported hardware: an embedded LiteRT model (small CNN / spectral-flux beat tracker) runs on-device and emits a smoothed envelope every ~50 ms.
    • On unsupported hardware: fall back to a pure-DSP envelope follower (RMS + onset peak-pick on a band-passed low-frequency signal — kicks/bass).
    • Output is mapped to master brightness 0–255 with operator-tunable floor / ceiling / attack / release.
  • When disabled: brightness reverts to whatever the manual slider last commanded; the modal button still shows current value.

Settings page

New section Auto Brightness:

  • Enable/disable toggle (mirrors the Stage button).
  • Mode: LiteRT (auto-detect, greyed out if unsupported) / DSP fallback.
  • Mic source picker (built-in / Bluetooth / USB-C audio).
  • Sensitivity (gain compensation).
  • Floor / Ceiling brightness clamps.
  • Attack / Release smoothing (ms).
  • Test meter (live envelope bar + headroom indicator) so operator can dial it in before showtime.
  • Permission gate: requests RECORD_AUDIO runtime permission; if denied the feature is hard-disabled and the modal button shows a "tap to grant mic" state.

Stage page

  • Remove the master-brightness slider from the Stage tab.
  • In its place: modal button Auto Brightness showing icon + state + current envelope value as a thin progress bar fill.
  • Tapping opens a small modal (not the full Settings tab) with: Enable toggle, Sensitivity slider, Floor/Ceiling sliders, live meter. Done collapses it.
  • Manual brightness is moved to the Settings tab (or to the modal as a fallback when feature is off) — Stage stays focused on live show controls.

Constraints

  • Pure on-device — no audio leaves the phone.
  • Must coexist with Pointer / Controller orient streams (mic + sensors at the same time).
  • LiteRT model has to ship in the APK (size budget < 10 MB).
  • Latency target: < 80 ms mic-to-DMX.
  • No regressions when the feature is off — the rest of the Stage UI must behave identically to today.

Out of scope

  • Spectral / colour mapping (just brightness for v1).
  • Multi-band or tempo-locked sequencing — that's a separate effects-engine feature, not the master-brightness control.
  • Non-Android platforms (Windows SPA brightness slider stays as-is).

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions