Skip to content

Fix/mediatek smart tv hevc audio optimizations#1577

Open
janblade wants to merge 4 commits into
moonlight-stream:masterfrom
janblade:fix/mediatek-smart-tv-hevc-audio-optimizations
Open

Fix/mediatek smart tv hevc audio optimizations#1577
janblade wants to merge 4 commits into
moonlight-stream:masterfrom
janblade:fix/mediatek-smart-tv-hevc-audio-optimizations

Conversation

@janblade

Copy link
Copy Markdown

Add MediaTek Smart TV SoC (omx.ms / MT5889) decoder support and display/audio improvements

Overview

This PR adds first-class support for MediaTek Smart TV SoCs that use the omx.ms
codec prefix (e.g. MT5889 in TCL C7xx, Hisense U8x, and similar Android TV 11+
devices). It also includes three independent improvements to frame rendering and
audio that benefit TV devices more broadly.


Commit 1 — Add support for MediaTek Smart TV SoC decoders (omx.ms / MT5889)

MediaCodecHelper.java

MediaTek Smart TV SoCs use the omx.ms codec prefix and are entirely distinct
from MediaTek mobile SoCs (omx.mtk). They were previously treated identically,
which incorrectly denied them several capabilities.

  • HEVC whitelistomx.ms decoders are whitelisted for HEVC on Android 11+
    (API 30+). The MT5889 properly declares HEVCProfileMain10HDR10 in its
    media_codecs.xml and is stable at 4K 60fps HDR10.

  • Direct submitomx.ms added to directSubmitPrefixes. These are
    fixed-function hardware decoders with low input-buffer latency, consistent with
    other hardware decoders already in that list.

  • Adaptive playbackomx.ms removed from blacklistedAdaptivePlaybackPrefixes.
    The original comment incorrectly grouped it with omx.mtk (mobile). TCL/MT5889
    devices declare <Feature name="adaptive-playback" /> in their codec XML and
    are stable with it enabled. The comment is updated to clarify that only
    omx.mtk / c2.mtk (mobile) require the blacklist.

  • HEVC RFIomx.ms added to refFrameInvalidationHevcPrefixes on
    Android 11+. Confirmed on TCL C735 (MT5889/Mali-G57). Unlike omx.mtk
    (which requires GPU-family gating due to GE8xxx hangs), omx.ms TV decoders
    do not require this restriction.

  • Low-latency pathmediaTeKTvDecoderPrefixes list added (omx.ms).
    The MT5889 ACodec layer requires both KEY_LOW_LATENCY (standard Android R
    API) and the vendor key vdec-lowlatency to fully engage low-latency
    I-frame decoding. The existing FEATURE_LowLatency early-return in
    setDecoderLowLatencyOptions() is bypassed for these decoders so both keys
    are always applied together across retry attempts.

  • Fused IDR fixdecoderSupportsFusedIdrFrame() now returns false for
    any decoder on the adaptive playback blacklist. Fused IDR submission requires
    the codec to have been configured with KEY_MAX_WIDTH/KEY_MAX_HEIGHT;
    blacklisted decoders are not configured that way, so submitting fused IDR
    buffers without adaptive playback would be incorrect. This is a generic
    correctness fix.

Tested on: TCL C735 (MT5889/Mali-G57, Android 11) via WiFi and Ethernet,
HEVC Main10 HDR at 4K 60fps.


Commit 2 — Enable Game Mode / ALLM via preferMinimalPostProcessing on Android 11+

Game.java

On Android 11+ (API 30), WindowManager.LayoutParams.preferMinimalPostProcessing
maps to HDMI Auto Low Latency Mode (ALLM) on connected displays. This
instructs the TV to switch to its Game Mode picture preset, bypassing MEMC
(motion smoothing / frame interpolation), post-sharpening, and other processing
pipelines that add display lag and can introduce periodic visual artefacts during
streaming.

Some TV vendor overlays (notably TCL/MediaTek) ignore the equivalent
AndroidManifest attribute; setting it programmatically on WindowManager LayoutParams at runtime is more reliable.

setAttributes() calls are also added in the display-mode selection paths to
ensure the flag is applied regardless of which branch is taken (mode change vs.
already-optimal mode).


Commit 3 — Fix MAX_SMOOTHNESS frame queue rendering and TV compositor timestamp

MediaCodecDecoderRenderer.java

  1. MAX_SMOOTHNESS / CAP_FPS frame queue drain — when multiple output buffers
    are queued, the previous code called releaseOutputBuffer(index, false) for
    all intermediate frames, silently discarding them. In MAX_SMOOTHNESS and
    CAP_FPS modes the intent is to never drop frames; intermediate frames are
    now released with timestamp=0 (immediate render) so every decoded frame
    reaches the display.

  2. Lowest-latency mode final frame timestamp — the previous code passed
    System.nanoTime() as the presentation timestamp. On smart TV SoCs (e.g.
    MediaTek MT5889), this causes SurfaceFlinger to queue frames against future
    VSync deadlines it cannot meet, leading to constant 2–3 fps drops and periodic
    compositor stalls. Passing timestamp=0 requests immediate release and avoids
    compositor-side queue build-up.


Commit 4 — Fix audio queue threshold and buffer selection with effects

AndroidAudioRenderer.java

  1. Audio queue threshold raised from 40 ms to 120 ms. At 40 ms the queue
    limit is hit during normal AudioTrack hardware buffering, causing constant
    packet drops that manifest as periodic audio stuttering. 120 ms safely absorbs
    the hardware buffer depth and brief network jitter without introducing
    perceptible latency.

  2. Skip buffer size index 2 when audio effects are active. With an equalizer
    or other effects chain enabled, standard-latency AudioTrack paths require a
    larger minimum buffer to avoid underruns under high CPU/GPU load (e.g. 4K AV1
    decode). Index 2 produces a size too small for reliable playback with effects
    active, so it is skipped in that case.


Testing

Device SoC OS Codec Resolution Network Result
TCL C735 MT5889 / Mali-G57 Android 11 HEVC Main10 HDR10 4K 60fps WiFi 5GHz ✅ No stutter
TCL C735 MT5889 / Mali-G57 Android 11 HEVC Main10 HDR10 4K 60fps Ethernet ✅ No stutter

Notes for reviewers

  • All omx.ms additions are gated on Build.VERSION.SDK_INT >= Build.VERSION_CODES.R
    (Android 11+) where confirmed stable. No changes affect devices below API 30.
  • The releaseOutputBuffer(0) change in commit 3 replaces System.nanoTime() only
    in the lowest-latency (non-balanced) code path. The Choreographer/balanced path is
    unchanged.
  • The root cause of the WiFi stutter on MT5889 was traced to NVENC single-frame VBV
    producing IDR frames too small (~41KB at 20Mbps/60fps) for the decoder at 4K.
    This is resolved server-side via nvenc_vbv_increase = 150 in Sunshine and is
    not related to the client-side changes in this PR.

janblade added 4 commits June 21, 2026 17:00
MediaTek Smart TV SoCs use the 'omx.ms' codec prefix (e.g. MT5889 in TCL C7xx
series). These are distinct from MediaTek mobile SoCs ('omx.mtk') and were
previously missing proper decoder support in several areas.

Changes:
- Whitelist omx.ms for HEVC decoding on Android 11+ (API 30+). The MT5889
  properly declares HEVC Main10 support in its media_codecs.xml.
- Enable direct submit for omx.ms. Fixed-function HW decoders with low
  input-buffer latency, matching the behaviour of other C2/HW decoders.
- Remove omx.ms from the adaptive playback blacklist. The original comment
  incorrectly grouped omx.ms with omx.mtk (mobile). TCL/MT5889 devices
  properly declare adaptive-playback support and are stable with it enabled.
- Enable HEVC RFI (reference frame invalidation) for omx.ms on Android 11+.
  Confirmed on TCL C735 (MT5889/Mali-G57); unlike omx.mtk this does not
  require GPU-family gating.
- Add mediaTeKTvDecoderPrefixes and a dedicated KEY_LOW_LATENCY +
  vdec-lowlatency code path. The MT5889 ACodec layer requires both
  KEY_LOW_LATENCY (standard Android R API) and the vendor 'vdec-lowlatency'
  key to fully engage low-latency I-frame decoding. The existing
  FEATURE_LowLatency early-return is bypassed for these decoders so both
  keys are always applied together.
- Fix decoderSupportsFusedIdrFrame() to return false for decoders on the
  adaptive playback blacklist. Fused IDR submission requires the codec to
  have been configured with KEY_MAX_WIDTH/KEY_MAX_HEIGHT; blacklisted
  decoders are not configured that way, so submitting fused IDR buffers
  would be incorrect.

Tested on TCL C735 (MT5889/Mali-G57, Android 11) via WiFi and Ethernet
with HEVC Main10 HDR at 4K 60fps.
…oid 11+

On Android 11+ (API 30), Window.preferMinimalPostProcessing maps to HDMI
Auto Low Latency Mode (ALLM) on connected displays. This instructs the TV
to switch to its Game Mode picture preset, bypassing MEMC (Motion
Smoothing / frame interpolation), post-sharpening, and other processing
that adds display lag and can introduce periodic visual artefacts during
streaming.

Some TV vendor overlays (notably TCL/MediaTek) ignore the equivalent
AndroidManifest attribute; setting it programmatically on WindowManager
LayoutParams at runtime is more reliable.

The setAttributes() calls in the display-mode selection paths are also
added to ensure the flag is applied to the window regardless of which
branch is taken (mode change vs. already-optimal mode).
…itor timestamp

Two related rendering fixes:

1. MAX_SMOOTHNESS / CAP_FPS frame queue drain: when multiple output buffers
   are queued, the previous code called releaseOutputBuffer(index, false)
   for all intermediate frames, silently discarding them. In MAX_SMOOTHNESS
   and CAP_FPS modes the intent is to never drop frames; intermediate frames
   are now released with timestamp=0 (immediate render) so every decoded
   frame reaches the display.

2. Lowest-latency mode final frame timestamp: the previous code passed
   System.nanoTime() as the presentation timestamp. On smart TV SoCs (e.g.
   MediaTek MT5889), this causes SurfaceFlinger to queue frames against
   future VSync deadlines it cannot meet, leading to constant 2-3 fps drops
   and periodic compositor stalls. Passing timestamp=0 requests immediate
   release and avoids compositor-side queue build-up.
…with effects

Two audio reliability fixes:

1. Audio queue threshold raised from 40 ms to 120 ms. At 40 ms the queue
   limit is hit during normal AudioTrack hardware buffering, causing constant
   packet drops that manifest as periodic audio stuttering. 120 ms safely
   absorbs the hardware buffer depth and brief network jitter without
   introducing perceptible latency.

2. Skip buffer size index 2 when audio effects processing is active. With an
   equalizer or other effects chain enabled, standard-latency AudioTrack paths
   require a larger minimum buffer to avoid underruns under high CPU/GPU load
   (e.g. 4K AV1 decode). Index 2 produces a size too small for reliable
   playback in that case.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant