Missing evaluation manifest files for Dynamic-RE10K test

[dmucby]

Great work and thanks for releasing the code for this paper.

I’m trying to run the evaluation, but I’m confused about the test split setup. The README mentions these files under data/dynamic_re10k/test/:

• dre10k_final_context_2.txt
• dre10k_final_context_2_view_idx.json
• wildrayzer_final_context_2.txt
• wildrayzer_final_context_2_view_idx.json

And the default inference config also seems to expect wildrayzer_final_context_2.txt and its corresponding view_idx json. I couldn’t find these files in the current public release, so I’m not sure what the intended way to reproduce the evaluation.

Could you please let me know whether these files will be released, or whether there is a recommended way to generate them from the released dataset?

Thanks a lot.

[XuweiyiChen]

Hi Mengbai,

Thank you very much for your interest. Apologies! The file was previously gitignored. The evaluation index is now available under data/dynamic_re10k/test. We will be releasing D-RE10K-iPhone and the complete evaluation index soon as well. Please stay tuned.

Best Regards,
Xuweiyi Chen

[dmucby]

Hi Xuweiyi,

Great work, and thank you for the quick reply.

I use the released code and checkpoint with the publicly available files for evaluation, but I only obtain a PSNR of 19.20, which is still below the 21.78 reported in the paper for the 2-view setting.

While checking the evaluation setup, I notice that the test configuration uses wildrayzer_final_context_2_view_idx.json rather than dre10k_final_context_2_view_idx.json. I wonder whether this file is missing from the public release. In addition, dre10k_final_context_2_view_idx.json seems to contain only 25 test scenes, which does not match the 75 test scenes reported in the paper.

I also want to ask about the “dynamic-view” input mentioned in the paper. From the released checkpoint and configs, it seems that the model uses a fixed number of input views. Could you clarify whether you train a separate model for each input-view setting, or whether the model is trained to handle a dynamic number of input views?

Thank you very much for your help.

Best regards,
Mengbai Cai

[XuweiyiChen]

Hi Mengbai,

Thanks for your interest!

The current JSON file corresponds to the 25 validation sequences mentioned in Sec. 3.1. The publicly released checkpoint is tailored for the demo, which I hope to extend to cover more in-the-wild cases, so it may not exactly reproduce the score reported in the paper. In fact, I found that an earlier checkpoint handles in-the-wild cases better. I will confirm this with my collaborator and release the corresponding checkpoints together with the test sequences soon. I will also clarify this in the README.

For dynamic-view, we train a separate model for each input-view setting because this gives the best evaluation performance in our experiments. We did try a VGGT-style encoder to support dynamic inputs, but it did not perform better than using separate models with our compute budget. I apologize for the confusion.

Best Regards,
Xuweiyi Chen