RTSPWriter does not recover after Stop/Play cycle, self.cameras cleared and never rebuilt

[h4kyu]

Description

Environment

• isaacsim.replicator.agent.core: 0.7.28+107.3.3

Description

After stopping and restarting the Isaac Sim timeline (pressing Stop then Play again), the RTSPWriter stream does not recover. All subsequent frames fail silently with no output to the RTSP stream.

Root cause

In context of rtsp.py source file, on_final_frame() is called when Stop is pressed. It kills the FFmpeg pipe and clears self.cameras. When Play is pressed again, attach() and _config_stream_cameras() are not called again, so self.cameras stays empty. Every subsequent write() call finds anno_streamer = None and drops the frame.

Confirmed via runtime diagnostic:

[DEBUG] frame 9300: anno_streamer is None
[DEBUG] frame 9400: anno_streamer is None
[DEBUG] frame 9500: anno_streamer is None

Current workaround

Re-run the writer setup script (re-call writer.attach()) after each Stop before pressing Play again. The following setup script was used:

# enable extensions
from isaacsim.core.utils.extensions import enable_extension
enable_extension("omni.replicator.core")
enable_extension("isaacsim.replicator.agent.core")
enable_extension("isaacsim.replicator.writers")
import isaacsim.replicator.agent.core.data_generation.writers.rtsp
print("[INFO] Extensions enabled.")

# setup
import omni.usd
import omni.replicator.core as rep
from pxr import UsdGeom

stage = omni.usd.get_context().get_stage()
camera_path = "/World/RTSP_Camera_01"
camera_prim = stage.GetPrimAtPath(camera_path)

if not camera_prim.IsValid():
    raise RuntimeError(f"Camera prim does not exist: {camera_path}")
if not camera_prim.IsA(UsdGeom.Camera):
    raise RuntimeError(f"Prim exists but is not a USD Camera: {camera_path}")

resolution = (1280, 720)
rtsp_base = "rtsp://127.0.0.1:8554/RTSPWriter"

render_product = rep.create.render_product(camera_path, resolution=resolution)

writer = rep.WriterRegistry.get("RTSPWriter")
writer.initialize(rtsp_stream_url=rtsp_base, rtsp_rgb=True, device=0)

writer.attach([render_product])
rep.orchestrator.set_capture_on_play(True)

Expected behavior

RTSPWriter should recover automatically when Play is pressed after a Stop,
either by registering a timeline event listener to re-call attach(), or
by preserving enough state in on_final_frame() to reconnect the pipe on
the next write() call.

Isaac Sim version

5.1.0

Operating System (OS)

Ubuntu 24.04.3 LTS

GPU Name

NVIDIA L40S

GPU Driver and CUDA versions

Driver 580.126.09, CUDA 13.0

Logs

No response

Additional information

No response

[PeterL-NV]

See #632 for more context on updates in Isaac Sim 6.0 on this. In short, please check if you still see the issue in 6.0.