12.1 (23) - HDR Pipeline overhaul#2
Open
NeoVectorX wants to merge 48 commits into
Open
Conversation
Complete feature release combining v11.2 and v12.0 improvements. Major Features: - SharePlay couch co-op multiplayer with host/guest PIN pairing - Gaze and touch control for both curved and flat display modes - Keyboard support in immersive and window modes - Reactive lighting modes (V1, V2, Starfield) with dynamic color matching - 3D SBS mode for flat display - 25+ new 360° environments - Microphone streaming with floating mute button - Custom resolution input and expanded resolution options Connection Stability: - Fix zombie window interference during mode switching - Fix connection hang during teardown (especially on headset removal) - Add graceful timeout handling for hung connections - Improve co-op connection flow and window management Bug Fixes & Polish: - Fix PS5 DualSense haptic rumble - Fix flat display headset resume - Fix cursor jitter in curved display - Memory leak fixes - Tutorial and user guide updates - Terminology improvements (Renderer → Display Mode, Screen Move → Screen Adjust) This release represents a major evolution of Neo Moonlight with experimental multiplayer capabilities and comprehensive input method support.
Updated project name and added new features in README.
Organized summary of changes since last GitHub upload: Stability & crash fixes - Fix connection crashes with ConnectionSerializer and Classic mode architecture refactor - Fix dual window opening and background thread UI access - Fix black screen on app reopen after backgrounding during stream - Fix SheetBridge crash and black screen on failed reconnect - Fix Resume button failing after visionOS reclaims immersive space - Fix dismissImmersiveSpace crash; Classic Display mode completion - Fix SwiftUI crash by correcting CoopSessionCoordinator singleton usage - Fix crash from force unwrapping nil VTDecompressionSession - Fix Int16 overflow in Task Manager button (bitPattern initializer) Decoder & rendering - CurvedDisplay mailbox→direct-push texture refactor; direct-push done - Async drawable present (addCompletedHandler) to avoid blocking decoder thread and OOM/watchdog crashes Classic Display mode - Apply aspect ratio lock before stream connects - Fix intermittent Resume button failure; resume button reliability - Add z-offset to center popups for proper floating appearance - Add rounded corners (0.6% corner radius) - Classic Display mode complete Mesh & performance (Curved/Flat) - Curved mesh 512→256, collision 256→64 to reduce geometry load and thermal throttling - Flat Display mesh 256→128 for smoother feel; mesh resolution 256 for Flat and Curved; jitter fix + 128 mesh Dark Controls & Dynamic menu - Add Dark Controls Mode toggle; fix Classic Display stability - Dark Controls (0.01 hidden opacity); Classic ignores dark mode; changelog and Curved/Dimming/CenterPreset/ControlComponents updates - Dynamic menu: smooth scale animation, pill only on expanded bar (no pill on center launcher); ellipsis icon; ZStack animation; settings and changelog updates Input, controls & features - Input lock: long press to disable hand/gaze on control buttons; lock popup text and messaging; width parameter for root CenterPresetPopup - Task Manager button (Ctrl+Shift+Esc); icon and overflow fix - Persist curvature preset and tilt angle across app restarts - Adjustable star distance presets for Starfield; Medium/Far preset tweaks - Controller battery indicator; co-op Leave button and state management fixes - Keyboard focus fix (no stealing from other visionOS apps in flat display) - Right-click instructions for Flat Display Default Controls UI & overlay - Stats overlay: reduce size in curved display; scaleEffect to prevent initial large font flash; sizing to match flat display - Brightness: remove opaque blending from brightness cycling; adjustable brightness in changelog - Lighting icons and User Guide updates Documentation & changelog - User guide and changelog: battery meter, co-op notes, input lock, sound stage, star distance, port testing, long-press input disable - Restore descriptive notes for settings - Changelog: Dynamic Controls Menu, performance optimizations Version - Build version 21 (CURRENT_PROJECT_VERSION)
Complete standards-correct, Vision Pro-optimized HDR pipeline across all three renderers: - Shaders.metal: new ShaderHDRParams/FullHDRParams structs; 6 ITU-spec YUV decode functions (BT.709/2020/601 × limited/full range, 8-bit and 10-bit); Apple's 1.961 SDR de-gamma; BT.709→P3 and BT.2020→P3 gamut matrices; Uchimura tone mapping operator with live EDR headroom ceiling; OLED toe lift for micro-OLED true-black shadow detail; luma-preserved grading (contrast/saturation/warmth); radial saturation falloff for Vision Pro optic chromatic aberration compensation. All four fragment shaders rebuilt on shared pipeline. - DrawableVideoDecoder.swift: fix force-PQ bug (reads actual transfer function metadata, pixel-format heuristic only as fallback); proper 10-bit and full-range detection via pixel format enum; live UIScreen EDR headroom uniform sent per-frame; updated struct layout (ShaderHDRParams/ShaderFullHDRParams) matching Metal exactly; inflight semaphore (value: 3) for GPU back-pressure stability. - MetalVideoDecoderRenderer.swift: mirror all Swift-side HDR detection fixes; live EDR headroom; updated ShaderHDRParams/FullHDRParams/ColorEnhancementUniforms matching Metal; pixel format helpers for 10-bit and full-range detection. - CurvedDisplayStreamView.swift + FlatDisplayStreamView.swift: set applyPostProcessToneMap = false on all video screen UnlitMaterial instances to prevent RealityKit's second tone-mapping pass on top of our already-correct output. - HDRSettings.swift: replace unused luminance/gamma/peakBrightness fields with pqExposure (Float, default 1.0, persisted under "hdrPqExposure"). - HDRControlPanel.swift: replace three unused Advanced sliders with single "HDR Exposure" slider (dial.medium.fill, range 0.5–2.0). - HDRParams struct: add pqExposure field; all call sites updated across Curved, Flat, RealityKit, and HDRTest stream views.
…ionals UIScreen.main is unavailable in visionOS. Use fixed EDR headroom value (2.0) for Vision Pro's micro-OLED display, with UIScreen fallback for other platforms.
The applyPostProcessToneMap property doesn't exist in this visionOS SDK version. Removed all references - our Metal shaders already handle tone mapping correctly, so RealityKit won't apply a second pass.
- Added "Custom" option to bitrate picker - When selected, displays text field for manual Mbps entry - Automatically converts Mbps to Kbps for internal use - Retains preset selection when switching back from custom - High bitrate warning applies to custom values too
- Remove $ from bitrateSelection (already a Binding) - Use .wrappedValue to access binding value - Extract resolution Binding to computed property to fix compiler timeout
- Reactive 1 (dim 2): Chromosphere edge bloom only; purple dome off; Reactive 2 (dim 10) solid reactive sphere without Chromaglow - Chromosphere halo mesh regenerated with screen curve/aspect; Z offset for flat vs curved; makeChromosphereMesh helpers in CurvedDisplayStreamView - CurvedDisplayStreamView+Mesh: drop duplicate generators; keep convertPositionToUV - Reactive 2 Digital Crown-style sphere envelope intro on preset select - Dimming picker: Reactive 1/2 labels and tap behavior; related stream/HDR/shader updates on branch
- Fix dark SDR when HDR off: gate 10-bit BT.2020 primaries/matrix fallback on hdrEnabled in DrawableVideoDecoder and MetalVideoDecoderRenderer - HDR settings, control panel, shader grading, and stream views (curved/flat/RealityKit/test)
Complete standards-correct, Vision Pro-optimized HDR pipeline across all three renderers: - Shaders.metal: new ShaderHDRParams/FullHDRParams structs; 6 ITU-spec YUV decode functions (BT.709/2020/601 × limited/full range, 8-bit and 10-bit); Apple's 1.961 SDR de-gamma; BT.709→P3 and BT.2020→P3 gamut matrices; Uchimura tone mapping operator with live EDR headroom ceiling; OLED toe lift for micro-OLED true-black shadow detail; luma-preserved grading (contrast/saturation/warmth); radial saturation falloff for Vision Pro optic chromatic aberration compensation. All four fragment shaders rebuilt on shared pipeline. - DrawableVideoDecoder.swift: fix force-PQ bug (reads actual transfer function metadata, pixel-format heuristic only as fallback); proper 10-bit and full-range detection via pixel format enum; live UIScreen EDR headroom uniform sent per-frame; updated struct layout (ShaderHDRParams/ShaderFullHDRParams) matching Metal exactly; inflight semaphore (value: 3) for GPU back-pressure stability. - MetalVideoDecoderRenderer.swift: mirror all Swift-side HDR detection fixes; live EDR headroom; updated ShaderHDRParams/FullHDRParams/ColorEnhancementUniforms matching Metal; pixel format helpers for 10-bit and full-range detection. - CurvedDisplayStreamView.swift + FlatDisplayStreamView.swift: set applyPostProcessToneMap = false on all video screen UnlitMaterial instances to prevent RealityKit's second tone-mapping pass on top of our already-correct output. - HDRSettings.swift: replace unused luminance/gamma/peakBrightness fields with pqExposure (Float, default 1.0, persisted under "hdrPqExposure"). - HDRControlPanel.swift: replace three unused Advanced sliders with single "HDR Exposure" slider (dial.medium.fill, range 0.5–2.0). - HDRParams struct: add pqExposure field; all call sites updated across Curved, Flat, RealityKit, and HDRTest stream views.
…ionals UIScreen.main is unavailable in visionOS. Use fixed EDR headroom value (2.0) for Vision Pro's micro-OLED display, with UIScreen fallback for other platforms.
The applyPostProcessToneMap property doesn't exist in this visionOS SDK version. Removed all references - our Metal shaders already handle tone mapping correctly, so RealityKit won't apply a second pass.
- Added "Custom" option to bitrate picker - When selected, displays text field for manual Mbps entry - Automatically converts Mbps to Kbps for internal use - Retains preset selection when switching back from custom - High bitrate warning applies to custom values too
- Remove $ from bitrateSelection (already a Binding) - Use .wrappedValue to access binding value - Extract resolution Binding to computed property to fix compiler timeout
- Reactive 1 (dim 2): Chromosphere edge bloom only; purple dome off; Reactive 2 (dim 10) solid reactive sphere without Chromaglow - Chromosphere halo mesh regenerated with screen curve/aspect; Z offset for flat vs curved; makeChromosphereMesh helpers in CurvedDisplayStreamView - CurvedDisplayStreamView+Mesh: drop duplicate generators; keep convertPositionToUV - Reactive 2 Digital Crown-style sphere envelope intro on preset select - Dimming picker: Reactive 1/2 labels and tap behavior; related stream/HDR/shader updates on branch
- Fix dark SDR when HDR off: gate 10-bit BT.2020 primaries/matrix fallback on hdrEnabled in DrawableVideoDecoder and MetalVideoDecoderRenderer - HDR settings, control panel, shader grading, and stream views (curved/flat/RealityKit/test)
When app HDR is off, derive bi-planar Metal formats from CVMetalHelpers again instead of forcing r8/rg8. Matches TestFlight-era behavior and MetalVideoDecoderRenderer; fixes crushed/dark 10-bit SDR.
The drawable is bgra8Unorm_sRGB when HDR is off, which expects gamma-encoded values. The YUV decode already outputs gamma-encoded RGB. Applying sdrToLinear() (pow 2.2) crushed the image because the sRGB drawable then encoded it again. Matches baseline/TestFlight behavior: pass decoded RGB through directly.
Add copyFragmentShader*_SDRLegacy (+ UIKit variants) matching pre-29104ca YUV expand, PQ branch, and applyVisionProGrading. RealityKit decoder and MetalVideoDecoderRenderer select legacy pipelines and buffer layouts when hdrEnabled is false; HDR keeps unified ShaderHDRParams / processFrame path.
…adroom scope Nested types inside draw(in:) with a member named edrHeadroom caused Swift to lose the outer let for later lines. Move MetalCopy* buffer layouts to the class, rename the EDR field to hdrEdrHeadroom, and keep a single edrHeadroom let in draw.
- Move 8798705-style YUV/grading helpers + LegacySDR* structs above HDR pipeline. - copyFragmentShaderHDR_EDR / _HEVC_EDR (RealityKit) and UIKit variants now ONLY run legacy expand + matrix + applyVisionProGrading (no decodeYUV/processFrame). - New symbols copyFragmentShaderHDR_HDRUnified* hold the unified PQ pipeline. - Decoder/UIKit pick *_HDRUnified only when hdrEnabled; SDR uses canonical names.
Align Sunshine/Moonlight HDR interpretation instead of relying on sliders: - AV1: map PQ (ST.2084), HLG, and linear in tcMap; attach mastering + CLL blobs to format description like HEVC - PQ when buffer omits transfer: fall back to CMFormatDescription Extension_TransferFunction - Tone map: pass maxCLL/maxFALL into ShaderHDRParams; hdrUchimuraShoulderP widens shoulder for modest CLL/desktop - YUV limited-range enumeration and format-desc fallback (DrawableVideoDecoder) retained - Also includes HDR UI/settings and immersive stream tweaks in this workspace snapshot
Persist variant in UserDefaults; drive chromaHaloScale + mesh rebuild. Curved display dimming picker matches Starfield hold-to-repeat UX.
…opy, SBS confirm size
…, changelog sections
Flat: present HDR via window ornament (avoids clipping and Metal bleed over RealityView). Center anchor with SwiftUI offset(z:) to match RK dialog depth like SBS. Remove hdrPanel RealityKit attachment and related setup. HDRControlPanel: attachmentLayoutScale and flat-only dimInactiveGradingControlsWhenReferenceHDR; curved uses defaults unchanged.
Re-enable decoder isReactiveDimmingEnabled and chromaHaloColorsUpdated guard for Reactive 1 alongside 10/12, so Replay of post-f588 commits cannot drop Chromosphere-linked sampling cadence.
- Decoder: mip-0 rim, half-res halo buffer, two rounds separable 13-tap blur, temporal on final V-pass; scratch textures and frame-mix tuning. - Shaders: chromaHaloRimFragment bezel-tight sampling (0.02–0.98), narrower rim kernel and milder anisotropy; rim + blur chain for smooth edge wash. - Curved: Chromosphere mesh 256²; ChromaHaloDownsample aligned with decoder. - Reactive UX / copy: reach tiers, picker, UpdatesView as applicable. User-confirmed: gradient taper good; edge color reads correctly after rim tweak.
- HDRControlPanel: With Reference HDR on (stream flag), keep full grading UI; replace orange slider accent with neutral chrome and disable drags/resets. - CurvedDisplayStreamView: Pass dimInactiveGradingControlsWhenReferenceHDR on the HDR attachment to match flat. Strengthen Reactive 1 Wide/Expanded/ Maximum top-bar and capsule opacity (tiered floors, min 0.75 when boosted).
…lish - Re-apply HDR grading from panel state when opening HDR (flat + curved). - Custom bitrate: confirm to apply, show selected Mbps, persist via AppStorage. - Dimming picker: Reactive 1 footer copy, footer icon alignment, Reactive 2 label glyph removed; remove Reactive 1 reach tier center popup (curved). - Changelog: Settings custom bitrate bullet for 12.1. - Xcode project file updates.
- Bitrate control uses Menu with Custom at top, presets below - Single-line capsule label for custom (NBSP + line limit) - Inline Mbps field: digits-only while typing, apply on Confirm - Remove auto-collapse debounce; widen digit field
- Add syncHDRSettingsForStreamStart() to ensure saved HDR panel values apply before decoder creation - Unify updateHDRParamsFromPanel() with stream start logic to prevent image snap when opening HDR panel - Add persistedFloat() helper to robustly read Float values from UserDefaults (handles Float/Double/NSNumber) - Fix issue where users had to click HDR button each time to apply their saved settings
Organized summary of changes since last GitHub upload (build 21): HDR Settings & Pipeline - Auto-apply saved HDR settings at stream start (no manual button click required) - Fix HDR settings "snap" when opening settings panel - Unified HDR application logic for custom preset (consistent between stream start and UI interaction) - Robust UserDefaults float persistence with type-safe helper function - HDR ornament and panel polish for flat/curved displays - Reference HDR parity across curved and flat display modes - Color preset HDR path fixes Chromosphere & Reactive Lighting - Chromosphere: Ambilight-style edge glow with ChromaHalo bloom system - Long-press cycles four Chromosphere halo reach tiers: Standard/Wide/Expanded/Maximum - Halo scale presets: 1.55x, 2.08x, 2.72x, 3.48x geometric margin - Ambient zone analyzer runs on dimLevel 2 (Chromosphere mode) - Separable blur with temporal smoothing for diffuse wall-wash effect UI/UX Improvements - Bitrate selection: Changed from Picker to Menu for better single-line layout - Custom bitrate: Added Confirm button to prevent interruption during typing - Custom bitrate: Moved "Custom..." option to top of dropdown menu - Custom bitrate: Widened entry field (56→88 pts) for easier input - Glass effect: Subtle transparency on main app UI (non-streaming views) - Picker translucency and lighting menu scale adjustments - CenterPresetPopup matches translucent menu chrome - Side-by-side (SBS) 3D confirmation size improvements Curved Display Polish - HDR Reference toast notifications for filter changes - Chromosphere toolbar chrome integration - Curved display HDR panel shared options - Improved curved display ornament positioning Version & Documentation - Build version 23 (CURRENT_PROJECT_VERSION) - Version 12.1 updates in changelog - Organized changelog sections
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Changes
HDR Settings & Pipeline
Reactive Lighting Retuned
UI/UX Improvements
Version