Description
Environment
- isaacsim.replicator.agent.core: 0.7.28+107.3.3
- FFmpeg: 6.1.1
Description
For NVENC annotators, RTSPWriter adds the following flags to the FFmpeg command:
-hwaccel cuda -hwaccel_output_format cuda -hwaccel_device <n>
These are decoder-side flags intended to keep decoded frames in GPU memory when decoding a compressed input stream. When the input is raw pixel data piped from stdin (as is the case here), there is no decoder so these flags are semantically incorrect.
When tested in isolation, FFmpeg with these flags and a raw pipe input selected the non-standard gbrp pixel format (HEVC Rext profile) rather than the standard yuv420p:
Stream #0:0: Video: hevc (Rext), gbrp(pc, gbr/unknown/unknown, progressive)
This did not cause a crash on the tested configuration (after adding -pix_fmt yuv420p on the output side), but may behave differently on other GPU configurations or FFmpeg versions.
Suggested fix
Remove -hwaccel cuda -hwaccel_output_format cuda -hwaccel_device from the command when the input is a raw pipe. If GPU-accelerated color space conversion is desired in future, it should be implemented via -vf filters rather than decode-side hwaccel flags.
Isaac Sim version
5.1.0
Operating System (OS)
Ubuntu 24.04.3 LTS
GPU Name
NVIDIA L40S
GPU Driver and CUDA versions
Driver 580.126.09, CUDA 13.0
Logs
No response
Additional information
No response
Description
Environment
Description
For NVENC annotators, RTSPWriter adds the following flags to the FFmpeg command:
These are decoder-side flags intended to keep decoded frames in GPU memory when decoding a compressed input stream. When the input is raw pixel data piped from stdin (as is the case here), there is no decoder so these flags are semantically incorrect.
When tested in isolation, FFmpeg with these flags and a raw pipe input selected the non-standard
gbrppixel format (HEVC Rext profile) rather than the standardyuv420p:This did not cause a crash on the tested configuration (after adding
-pix_fmt yuv420pon the output side), but may behave differently on other GPU configurations or FFmpeg versions.Suggested fix
Remove
-hwaccel cuda -hwaccel_output_format cuda -hwaccel_devicefrom the command when the input is a raw pipe. If GPU-accelerated color space conversion is desired in future, it should be implemented via-vffilters rather than decode-side hwaccel flags.Isaac Sim version
5.1.0
Operating System (OS)
Ubuntu 24.04.3 LTS
GPU Name
NVIDIA L40S
GPU Driver and CUDA versions
Driver 580.126.09, CUDA 13.0
Logs
No response
Additional information
No response