dynerf : fused.ply path inconsistency & imshow clipping warning

[Serendipity95]

Hi author,
First of all, thanks for open-sourcing this excellent DeGauss work! I've been trying to train the Neu3D dataset recently, but encountered two problems that confuse me a lot. I hope you can give me some guidance.

Problem 1: Missing fused.ply file
When I started the training, a FileNotFoundError popped up, indicating that the fused.ply file was missing in the /home/dongting/DeGauss/data/dynerf/coffee_martini directory.

The README mentions that we should follow the data preparation instructions of 4DGaussians for the Neu3D dataset. However, I noticed that the fused.ply generated for each scene in 4DGaussians is located at data/dynerf/XXX/colmap/dense/workspace/fused.ply , which is indeed inconsistent with the directory where DeGauss expects fused.ply to be located during training. Although you provide a preprocessed fused.ply for the Neu3D dataset via a Google Drive link, I still want to confirm:
Are the overall data preprocessing steps for the dynerf dataset in DeGauss completely consistent with those in 4DGaussians?
For the missing fused.ply, is downloading the one from your provided Google Drive link the only recommended way? Can we directly move/reuse the fused.ply generated by 4DGaussians (located in colmap/dense/workspace/) to the scene root directory required by DeGauss without issues?

Problem 2: Warning about clipping input data for imshow
During the training process, the console keeps outputting the warning: "Clipping input data to the valid range for imshow with RGB data ([0..1] for floats or [0..255] for integers)".

I wonder:
Is this warning related to the data preprocessing of the dynerf dataset? Did I miss certain preprocessing steps (such as image normalization) that caused the pixel value range to be abnormal?
Is there a simple way to eliminate this warning without affecting the training process?

I would be extremely grateful if you could answer these questions. Looking forward to your reply!

[BatFaceWayne]

Hi thanks for your interest in our work!

• regarding point 1, we could confirm that your obtained fused.ply from 4DGaussian is exactly the same as our processed one. You could copy it to the parent folder and run our code directly.

• Regarding point 2, this warning is a normal behavior that the render goes out of range. Please note this does not affect training but only happens when writing debuging images during traininig for visual feedback. To get rid of the warning, we enabled clipping before logging in our recent commit.

[Serendipity95]

Thank you for your prompt and helpful reply! All my previous questions have been addressed, but I have one more question I'd like to consult you about.

When using the coffee_martini dataset, I noticed slight changes in human heads, but the predicted masks do not seem to mark them as dynamic regions. The masks appear to prioritize detecting obvious motions (such as arm movements) and may lack sufficient sensitivity to slight motions— which also results in the human heads being quite blurry in the rendered images.

My Understanding (Potentially Inaccurate):
Based on the code, the masks are derived from the SH features of Gaussians (mask_dy = torch.sigmoid(shs_final[:, -1, 2])). During training, the following losses are used:

1. Reconstruction loss of synthetic images (primary supervision)

2. Foreground/background separation loss (applied in high-confidence regions where mask > 0.51)

3. Entropy loss (encouraging more definite masks)

My Questions:

1. Is this a design trade-off (prioritizing obvious motions) or can it be improved through parameter adjustments?
2. Are there recommended parameter tweaks for detecting slight motions?
3. If this issue indeed exists, are there any improvement suggestions or follow-up plans?

Please correct me if my observation is incorrect or if there is any deviation in my understanding of the code.

Thank you again for your excellent work and code sharing！

[BatFaceWayne]

Hi — this can occur in rare cases when the model settles into a local minimum specifically to coffee martini scene. I would suggest to try running it again; you may obtain images closer to those from our released checkpoints. https://huggingface.co/BatofGo/DeGauss_ckpts/tree/main

Your observation are mostly correct, we achieve dynamic-static decomposition based on thier ability to model and express the scene. And I would recommend to set main loss higher if you would like to detect more subtle motions(this is used for image datasets and egocentric videos in our experiment):

lambda_main_loss = 4

[Serendipity95]

Thanks a lot! The third run worked smoothly and I got the expected results. Appreciate your help!