Skip to content

Add CollectPointsWithUV for colored point clouds#7

Open
berkut0 wants to merge 2 commits intovvvv:masterfrom
berkut0:feature/pointcloud-uv
Open

Add CollectPointsWithUV for colored point clouds#7
berkut0 wants to merge 2 commits intovvvv:masterfrom
berkut0:feature/pointcloud-uv

Conversation

@berkut0
Copy link

@berkut0 berkut0 commented Jan 14, 2026

Problem

To create colored point clouds, it's necessary to sample the color camera image. However, since the color camera image doesn't align "one-to-one" with the depth sensor image, the Kinect SDK provides a special tool to get UV coordinates for each point. Previously, this tool was not exposed, making it impossible to get UV coordinates for point clouds. This was an oversight.

Solution

This PR adds CollectPointsWithUV method that generates UV coordinates alongside point positions, allowing users to color point clouds by sampling the color texture in shaders.

Changes

  • Added CollectPointsWithUV method in PointCloud.cs that returns SpreadBuilder<(Vector3 position, Vector2 uv)>
  • Added PointCloudWithUV node for use in VVVV Gamma
  • Added PointCloudWithUV (Reactive) node for reactive API

Technical Details

  • UV coordinates are normalized to 0..1 range (compatible with shaders)
  • Uses MapDepthFrameToColorSpaceUsingIntPtr from Kinect SDK
  • Dynamically gets color frame dimensions from KinectSensor.ColorFrameSource.FrameDescription
  • Invalid UV coordinates cause the entire point to be dropped (maintains synchronization)
  • Compatible with Observable pattern via HoldLatestCopy

@berkut0
Copy link
Author

berkut0 commented Jan 14, 2026

image

@joreg
Copy link
Member

joreg commented Jan 15, 2026

thanks for this already!
in order to help us review this: would you mind adding screenshots that show your case in the vl document? i'm also wondering if a change in a helppatch should be necessary? or a new helppatch?

@berkut0
Copy link
Author

berkut0 commented Jan 15, 2026

@joreg I have only used VL.FUSE and, to be honest, I have little idea how to use it in ‘vanilla’ Stride. Therefore, I think that if we make a help patch, it should be purely as a demonstration of the idea without any specific implementation (so as not to tie the patch to VL.FUSE). However, it's up to you.

image

kinect.zip

@joreg
Copy link
Member

joreg commented Jan 15, 2026

ok, but also i meant screenshots of the changes you did in the .vl doc

@berkut0
Copy link
Author

berkut0 commented Jan 15, 2026

image image image image

@joreg
Copy link
Member

joreg commented Feb 4, 2026

sorry for late..

  • a help without fuse would be cleaner indeed, but also i wouldn't mind it. so please add to this PR
  • what you mean by "probably should be renamed" in the screenshot?

@berkut0
Copy link
Author

berkut0 commented Feb 26, 2026

Help patch:

Screenshot_389

Changes to the main API:

Screenshot_388 Screenshot_387

@joreg
Copy link
Member

joreg commented Mar 10, 2026

thanks for the update. only one thing: in your help patch you're using pads as send/receive which is an anti-pattern as it means that in the first frame the readers are receiving an uninitialized value (as the warning suggests).

image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants