Runtime spawned actors are visible in RGB but missing from depth and object_mask observations

[aohanah]

Hi SimWorld team,

I found that actors spawned at runtime via the Python API can be visible in the RGB (lit) camera view, but they do not appear in depth or object_mask.

Tested observation APIs

rgb = communicator.get_camera_observation(camera_id, "lit")
depth = communicator.get_camera_observation(camera_id, "depth")
mask = communicator.get_camera_observation(camera_id, "object_mask")

I also checked the raw depth endpoint:

vget /camera/{camera_id}/depth npy

Observation

Static scene geometry, such as buildings, trees, roads, and lamps, appears correctly in depth and object_mask.

However, runtime spawned actors such as:

• humanoid / pedestrian actors
• vehicle actors
• simple spawned objects, e.g. boxes

are visible in RGB but missing from depth and object_mask.

For example, a spawned actor placed in front of the camera is visible in lit, but the corresponding raw depth region appears to remain at background depth values, and the actor does not get a separate color in object_mask.

The object mask itself is functioning correctly for static scene assets.

Question

Is there an official way to make runtime spawned actors visible to the depth and object_mask sensors?

Specifically:

1. Does spawn_object / spawn_agent automatically register spawned actors for depth and object_mask?
2. Are there required UE component settings, such as Render in Depth Pass, Render CustomDepth Pass, or stencil/object-mask registration?
3. Is this expected to work from the Python API, or does it require UE/Blueprint-side modification?

Thanks!