Fix H.264 HW decode for AVC3 in-band streams (V4L2/m2m)#2034
Conversation
On platforms using V4L2 mem2mem H.264 decoding (e.g. Raspberry Pi with OSMC's custom FFmpeg), the decoder's h264_xd_copy() function rejects avcC extradata with 0 SPS/0 PPS entries, causing a fallback to software decode. This format is used by AVC3 in-band streams (e.g. BBC iPlayer DASH) where SPS/PPS are carried in-band rather than in the codec private data. Fix by: - Detecting avcC with 0 SPS in Session::UpdateStream() and FragmentedSampleReader::UpdateSampleDescription(), and clearing extradata so the V4L2 decoder skips h264_xd_copy() - Adding AVCCodecHandler::Transform() to convert NALU length-prefixed packets (avcC framing) to Annex B start-code framing at decode time - Setting the AnnexB transform flag for all H.264 AVC1-4 formats (with and without SPS), ensuring consistent packet-level conversion
|
Decision: APPROVE kodiai responseDecision: APPROVE Evidence:
Review Details
|
|
Development work should be done first on the main branch, and then, if accepted, in the older branches the way you explain it seems more like an issue that needs to be addressed at the higher level rather than here Also, what happens with same stream on FFmpeg direct add-on? should have same problem? |
|
sorry if i ping you @samnazarko but i dont know exactly who ping |
|
Honestly, I’d advise getting access to a stream. |
3 participants
Summary
On platforms using V4L2 mem2mem H.264 decoding (e.g. Raspberry Pi with OSMC's custom FFmpeg), the decoder's
h264_xd_copy()function rejects avcC extradata with 0 SPS/0 PPS entries, causing a fallback to software decode. This format is used by AVC3 in-band streams (e.g. BBC iPlayer DASH) where SPS/PPS are carried in-band rather than in the codec private data.Problem
AVC3 streams provide an avcC record like
01 64 00 1f ff e0 00— 7 bytes with 0 SPS and 0 PPS. The V4L2 mem2mem decoder's init function (h264_xd_copy) tries to parse SPS/PPS from this record and fails withAVERROR(EINVAL), causing the decoder to fall back to software decode.Fix
Three-part approach:
Clear extradata for 0-SPS avcC — In both
Session::UpdateStream()andFragmentedSampleReader::UpdateSampleDescription(), detect avcC records with 0 SPS and set extradata to null. This preventsh264_xd_copyfrom being called, allowing the V4L2 decoder to open successfully (it receives SPS/PPS from in-band NALUs instead).Convert avcC extradata to Annex B — For avcC records that do contain SPS, convert to Annex B format. This is already done for DRM streams (
SSD_ANNEXB_REQUIRED), but non-DRM AVC1-4 streams now also get this treatment, which is required for V4L2/m2mem compatibility.Packet-level Annex B transform — Add
AVCCodecHandler::Transform()override that replaces NALU length prefixes (1/2/4 byte avcC framing) with Annex B start codes (00 00 00 01). This is needed because when extradata is cleared or converted, the downstream decoder expects Annex B framed packets. TheTransform()virtual already exists inCodecHandlerand is called byFragmentedSampleReader::ReadSample().Changes
src/Session.cpp— RestructureUpdateStream()H.264 extradata handling: DRM+SPS → Annex B conversion; avcC with 0 SPS → clear extradata; avcC with SPS → Annex B conversion; non-avcC → pass through. Added error handling for failedAvcToAnnexb()calls.src/codechandler/AVCCodecHandler.h— AddTransform()override andSetAnnexBTransformNeeded()/m_needAnnexBTransformmembers.src/codechandler/AVCCodecHandler.cpp— ImplementTransform(): iterates buffer replacing NALU length prefixes with00 00 00 01start codes.src/samplereader/FragmentedSampleReader.cpp— Add AVC1-4 handling inUpdateSampleDescription(): 0-SPS → clear extradata + enable transform; with SPS →ExtraDataToAnnexB()+ enable transform; failed conversion → clear extradata + enable transform.Testing
Tested on Raspberry Pi 4 (OSMC 2026.05-1, kernel 5.15.92, Kodi 21.3 Omega) with BBC iPlayer DASH streams: