How to get SAM Generate Masks

[Yoow4]

Hi,
I would like to know more details about how to use SAM to get the segmentation masks on 2D frames and then map them into the 3D space via depth information.
When I use it, SAM sets the following parameters
mask_generator = SamAutomaticMaskGenerator( model=sam, points_per_side=64, pred_iou_thresh=0.86, box_nms_thresh= 0.7, stability_score_thresh=0.92, crop_n_layers=1, crop_n_points_downscale_factor=2, min_mask_region_area=200, # Requires open-cv to run post-processing )
But I can't get the same segmentation effect as you, how can I set the parameters and use the depth map to project to 3D space。
Hope to get your answer.
Thanks.

[yhyang-myron]

Hi, we get 3D point clouds with 2D RGB, depth and camera parameters at here.
And we process 2D masks at here.

[Yoow4]

Hi,
For the last step, when I run sam3d.py I get the.pth file.
I look at the pcd_ensemble function
with open(org_path) as f: segments = json.load(f)
It needs a.json file.

What should I do to get the.json file.

[flybiubiu]

Hi, For the last step, when I run sam3d.py I get the.pth file. I look at the pcd_ensemble function with open(org_path) as f: segments = json.load(f) It needs a.json file.

What should I do to get the.json file.

+1

[rayryeng]

Hi, For the last step, when I run sam3d.py I get the.pth file. I look at the pcd_ensemble function with open(org_path) as f: segments = json.load(f) It needs a.json file.

What should I do to get the.json file.

I made a comment about this here: #52 (comment) - you have to access a separate mesh segmentation application to create this JSON file. I provide detailed instructions on what to do and how to run it.