What if I don't have the Depth Image?

[briankim0825]

I'm trying to make a HHA image for other datasets that doesn't provide improved depth images such as SUNRGB-D, only raw ones. Can I get a recommendation of inpainting codes for improving raw Depth Image?

I also would like to ask would the result be far different if I use a raw depth image as an improved depth image to make a HHA image.

Thanks.

[JamesMcCullochDickens]

The NYUDv2 toolkit has a matlab script that has an inpainting algorithm based on the colorization approach of Levin, here:
https://cs.nyu.edu/~silberman/datasets/nyu_depth_v2.html

I have code in my github (can't remember the source tbh, not written by me) that has a python script of this approach.
https://github.com/JamesMcCullochDickens/CSI5340Project/blob/main/InPainting.py

There are probably much better/modern depth inpainting algorithms however, but if you are satisfied with this approach, not a bad place to start.

As for whether the HHA would be better with raw depth or improved depth, you can see in the SUN RGBD dataset the difference is pretty substantial imo.

[JamesMcCullochDickens]

For a more modern approach, see here:
https://github.com/ZJULearning/depthInpainting

[briankim0825]

The python file worked for me. Thanks a lot.

[JamesMcCullochDickens]

@briankim0825 I should add that the script is quite slow, and that if you want to make it faster you can either run it in matlab (the original NYUDv2 one), or if you don't have matlab you could re-write it in C++, since I believe its the loop that is the main problem, and I pretty much guarantee you will see huge speedups, only relevant really if you have lots of files. I should probably add a multi-threading/multi-processing component to it as well... will do eventually.