Skip to content

Kiosk mode: camera motion/presence detection for screen wake#4497

Open
nstefanelli wants to merge 12 commits intohome-assistant:mainfrom
nstefanelli:kiosk-pr2-camera-detection
Open

Kiosk mode: camera motion/presence detection for screen wake#4497
nstefanelli wants to merge 12 commits intohome-assistant:mainfrom
nstefanelli:kiosk-pr2-camera-detection

Conversation

@nstefanelli
Copy link
Copy Markdown
Contributor

Summary

Adds camera-based motion and presence detection to kiosk mode (PR2 of kiosk series, per discussion #2403).

  • Motion detection: CoreImage pixel-diff at 5 fps with configurable sensitivity (low/medium/high)
  • Presence detection: Vision framework person/face detection with hysteresis (2-frame confirm, 10s absence timeout)
  • Screen wake: Configurable wake-on-motion and wake-on-presence triggers
  • Settings UI: New "Camera Detection" section with toggles, sensitivity picker
  • macCatalyst: Camera detection excluded on Mac

Builds on #4422 (PR1: core kiosk infrastructure).

Architecture

KioskCameraDetectionManager (coordinator)
├── KioskCameraMotionDetector (CoreImage pixel-diff, 5 fps, .low preset)
└── KioskPresenceDetector (Vision person/face detection, 3-10 fps, .medium preset)
         │
         ▼
KioskModeManager.wakeScreen(source:)
  • Frame processing runs on dedicated serial queues, not the main thread
  • State updates dispatched to MainActor via Combine publishers
  • Presence activity timer keeps screen awake while person is present

New files

File Lines Purpose
Camera/KioskCameraDetectionManager.swift ~200 Coordinator singleton
Camera/KioskCameraMotionDetector.swift ~250 CoreImage motion detection
Camera/KioskPresenceDetector.swift ~300 Vision presence/face detection
KioskCameraDetection.test.swift ~115 MotionSensitivity + settings tests

Test plan

  • MotionSensitivity enum: threshold values, Codable roundtrip, ordering
  • Camera settings: defaults, roundtrip, backwards compatibility with pre-camera JSON
  • Enable motion detection → wave hand → screen wakes from screensaver
  • Enable presence detection → walk up → screen wakes; walk away → screensaver resumes
  • Deny camera permission → detection doesn't start, no crash
  • Both detectors simultaneously → no conflict
  • macCatalyst build → camera section not visible

Related

Adds 11 kiosk.camera.* keys covering motion detection, presence detection,
face detection, sensitivity (low/medium/high), wake triggers, and footer copy.
Regenerates SwiftGen output with updated Strings.swift accessors under
L10n.Kiosk.Camera and L10n.Kiosk.Camera.Sensitivity.
…ings

Adds MotionSensitivity enum (low/medium/high with threshold values) and 6
camera detection properties to KioskSettings: cameraMotionEnabled,
cameraMotionSensitivity, wakeOnCameraMotion, cameraPresenceEnabled,
cameraFaceDetectionEnabled, wakeOnCameraPresence. Replaces synthesized
Codable with a custom init(from:) for backwards-compatible decoding so
existing persisted settings without camera fields decode without error.
- Make processFrame/calculateDifference nonisolated in motion detector
  so CIFilter and CIContext.render run on processingQueue, not MainActor
- Make processFrame nonisolated in presence detector so Vision requests
  run on processingQueue; settings read on MainActor then dispatched
- Mark previousFrame, ciContext, and Vision requests as nonisolated(unsafe)
  since they are only accessed from their respective processing queues
- Remove @published from errorMessage (internal debugging state, not UI)
- Replace PR-specific comments in tests with neutral wording
Custom init(from:) suppresses the synthesized default initializer.
Add explicit public init() so KioskSettings() continues to work.
Copilot AI review requested due to automatic review settings April 10, 2026 03:33
@codecov
Copy link
Copy Markdown

codecov bot commented Apr 10, 2026

Codecov Report

❌ Patch coverage is 0% with 3 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (main@295a2a7). Learn more about missing BASE report.

Files with missing lines Patch % Lines
Sources/Shared/Assets/Assets.swift 0.00% 3 Missing ⚠️
Additional details and impacted files
@@           Coverage Diff           @@
##             main    #4497   +/-   ##
=======================================
  Coverage        ?   42.58%           
=======================================
  Files           ?      274           
  Lines           ?    16230           
  Branches        ?        0           
=======================================
  Hits            ?     6912           
  Misses          ?     9318           
  Partials        ?        0           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant