Mono2Stereo: A Benchmark and Empirical Study for Stereo Conversion

   

¹Songsong Yu | ² Yuxin Chen🌟 | ²Zhongang Qi✉️ | ³Zeke Xie | ¹Yifan Wang | ¹Lijun Wang✉️ | ²Ying Shan | ¹Huchuan Lu
¹Dalian University of Technology ²ARC Lab, Tencent PCG ³The Hong Kong University of Science and Technology (Guangzhou)
CVPR 2025📖 Project Lead🌟 Corresponding Authors ✉️

💡 With the rapid growth of 3D devices and a shortage of 3D content, stereo conversion is gaining attention. Recent studies have introduced pretrained Diffusion Models (DMs) for this task, but the lack of large-scale training data and comprehensive benchmarks has hindered optimal methodologies and accurate evaluation. To address these challenges, we introduce the Mono2Stereo dataset, providing high-quality training data and benchmarks. Our empirical studies reveal:
1. Existing metrics fail to focus on critical regions for stereo effects.
2.Mainstream methods face challenges in stereo effect degradation and image distortion.
We propose a new evaluation metric, Stereo Intersection-over-Union (Stereo IoU), which prioritizes disparity and correlates well with human judgments. Additionally, we introduce a strong baseline model that balances stereo effect and image quality.

📢 News

2025-03-16: Project page and inference code (this repository) are released.
2025-02-27: Accepted to CVPR 2025.

🛠️ Setup

The inference code was tested on:

• Python 3.8.20, CUDA 12.1

📦 Usage

Preparation
You can download our model weights to perform inference.

⚙️ Installation

Clone the repository (requires git):

git clone https://github.com/song2yu/Mono2Stereo.git
cd mono2stereo

First, you need to download the weights of depth anything v2-small to the 'depth/checkpoints/' folder, and also download the weights of the dual-condition baseline model (or from 🤗mono2stereo.ckpt) to the 'checkpoint/' folder.

create a Python native virtual environment and install dependencies into it:

conda create -n stereo python=3.8 -y
conda activate stereo
pip install -r requirements.txt

🏃🏻‍♂️‍➡️ Inference

python test.py

📊 Dataset
We provide the data processing code in data_process.py. The video data can be downloaded from this website.
We provide test data (or from 🤗mono2stereo-test.zip) for fair comparison. Additionally, we recommend using the Inria 3DMovies for model testing.

🎓 Citation

If you find this project useful, please consider citing:

@misc{yu2025mono2stereobenchmarkempiricalstudy,
      title={Mono2Stereo: A Benchmark and Empirical Study for Stereo Conversion}, 
      author={Songsong Yu and Yuxin Chen and Zhongang Qi and Zeke Xie and Yifan Wang and Lijun Wang and Ying Shan and Huchuan Lu},
      year={2025},
      eprint={2503.22262},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2503.22262}, 
}

🫂 Acknowledgement

We would like to express our sincere gratitude to the open-source projects depth anything and Marigold. This project is based on their code.