I need some help with installation (Windows 10)

[RudiRastelli]

Hi. First of all i wanna say thx for this project and all the work your putting into it.
I'm pretty new to this 2Dto3D conversion stuff and jumped into it just about 2 weeks ago with installing and running iw3 on linux and windows. Now i wanna try VisonDepth3D on Windows (maybe later on linux).

I've decided to try the recommended conda installation.... and here's where some questions arise...

Question 1
I don't have conda on my system currently so i've searched for an installer and i've found 3 versions.
Miniconda, Anaconda Distribution and Miniforge. I wanna use Miniconda... is this enough to install and run VisionDepth3D ?

Question 2
In "Step 1: Download the VisionDepth3D Program" i'm told to add the backwards warp model.
Does this mean to copy the downloaded file backward_warping_model.keras to install/path/VisionDepth3D/weights folder ?

Regards
Rudi

P.S.:
Just to mention:
If someone decides to use install method 1 he/she might get confused because it says to clone git which is already done when he/she followed step 1... and... there is no install_visiondepth3d.bat file

[VisionDepth]

Hello and thank you for your interest in VisionDepth3D, I will update the process on read me, you can use anaconda or mini conda either one should work as miniconda is just a compacted version, I ended up deleting the backwards warp logic in the code as it was not functioning properly so you don't need to download the backwards warp keras model anymore, I also took off the install.bat file as it was faulty from older versions, Since you are already using IW3 I recommend using that to generate your depthmap videos, since vd3d doesn't generate itself use iw3, there is a drop-down that lets you choose export disparity and that will create a depthmap frames folder, you can then use that frames folder and stitch the frames together to get your depthmap videos to use, I know sounds like a lot of work, working on integrating the process inside the pipeline, here is a link to a YouTube tutorial on how to install https://youtu.be/k9Hk8BTPJck?si=rDsq6WdcqxG0XjKG if there is any other questions don't hesitate to ask

[VisionDepth]

Hey i just created a Frame Stitch application to help with stitching the depth frames together, https://github.com/VisionDepth/DepthFrame-Stitch.git,

1. Windows (Using pip)

Step 1: Install Python

• Download and install Python 3.8+ from python.org.
• Ensure "Add Python to PATH" is selected during installation.

Step 2: Install Dependencies

• Open Command Prompt (cmd) and run:
• pip install -r requirements.txt

Step 3: Run the Application

• python DepthFrame-Stitch_GUI.py

2. Conda (Windows/Linux/macOS)

Step 1: Install Miniconda or Anaconda

• Download from conda.io.

Step 2: Create a Conda Environment

• Open Anaconda Prompt (Windows) or Terminal (Linux/macOS) and run:
• conda create --name depthframe python=3.9 -y
• conda activate depthframe

Step 3: Install Dependencies

• pip install -r requirements.txt

Step 4: Run the Application

• python DepthFrame-Stitch_GUI.py

3. Linux (Ubuntu/Debian-based)

Step 1: Install System Dependencies

• sudo apt update && sudo apt install python3 python3-pip python3-tk -y

Step 2: Install Python Packages

• pip install -r requirements.txt

###Step 3: Run the Application

• python3 DepthFrame-Stitch_GUI.py

Troubleshooting

Missing tkinter on Linux? Run:

• sudo apt install python3-tk -y
• Python not found? Add it to your system PATH or reinstall it.

Dependencies failing to install? Try:

• pip install --upgrade pip

[VisionDepth]

how to install visiondepth3d

Guide Sheet: Install

Installation Steps
This program runs on python 3.9 - 3.10,
openCV with Cuda recommended but opencv-cpu works,
ffmpeg installed in system and able to access system wide,
and pip and/or git installed on system

Step 1: Download the VisionDepth3D Program

• Download the VisionDepth3D.zip file from the official Github Repo.
• Extract the zip file to your desired folder (e.g., C:\User\VisionDepth3D)

Step 2: Install Required Dependencies

Open a terminal or command prompt or power shell and enter:

• in the prompt change directory to extracted zip folder location with cd path/to/visionDepth3D
• next type pip install -r requirements.txt
• it should install required dependancies
• once finished you can open the program with python VisionDepth3D.py

or alternatively you can clone the repo and use pip

• pip install git+https://github.com/VisionDepth/VisionDepth3D.git
• pip install -r requirements.txt

use anaconda or miniconda by setting up an environment (best method)

• open conda prompt
• enter cd Path/To/Visiondepth3D
• enter conda create --name VisionDepth3D python=3.10
• enter conda activate VisionDepth3D
• pip install -r requirements.txt
• enter python VisionDepth3D.py

[RudiRastelli]

Nice work man... will try the stitcher tomorrow.
Does it also scale the depthmaps ?
In IW3 i've played with some settings to get the depthmaps with same resolution as the source video, but with no luck so far. Am i missing something here ?

It would be nice to have this functionality directly in VisionDepth3D. This would allow the resolution and frame rate of the source video to be checked first and then the depthmap video could be generated accordingly.

Another useful feature for VisionDepth3D would be to have a start-stop-time setting so users can easily play with the other settings, check the result (eg of a small snippet of a movie) before rendering full length.

Last but not least... native support for amd gpus.
I've a 6800 XT and currently i use zluda as a workaround. In IW3 it works very good and stable.
When I have my first suitable depthmap video tomorrow I will also try zluda with the VisionDepth3D.

Good night
Rudi

[VisionDepth]

I'm definitely going to be adding a depthmap process in the future, in the depthframe-stitch program you can input the width and height to resize too and also the fps to match original video, if you generate a 3D conversion in visiondepth3d with an mkv file you can go to where the file is being created and you can watch it in real time, the gui for visiondepth3d also includes a suspend button to pause the render, a resume render button and cancel render button to stop and finalize a video if you are just testing short clips, just gave me an idea for a restart render button so you can adjust the settings and start over if you need, hopefully in the feature I will add a video scrubber to look at frames and adjust 3D depth in real time

[RudiRastelli]

I wanted to try the stitcher but there is no "Start Processing" button !?

Edit:
I've made the button visible by editing window size from 500x400 to 500x500.
It converted my depth frames just fine... good work !

Next will be testing VisionDepth3D in a few hours

Btw.... i've copied the Stitcher to my VisionDepth3D folder and added opencv-python and tk to requirements.txt for VisionDepth3D.

[RudiRastelli]

When I just wanted to test VisionDepth3D, I noticed something strange with the stitcher.

Let me explain:
I had used the start-stop values of IM3 to generate the deapthmaps required for VisionDepth3D for one minute of the source movie.

Now I needed this one-minute sequence from the movie itself, but I don't have it. (i know i can use other tools to get it)
But... What I do have is the rgb folder that IM3 created next to the depth folder, which contains all the images from the movie snippet.
So I had the obvious idea of using the stitcher to turn the images into the video I needed.
The stitcher did a good job, but... the video was played in black and white (!) in VLC.

I know that the stitcher is intended for processing deathmaps, but being able to stitch movie frames as well would be a 'nice to have'.

So the question is:
Is there a weighty reason for this black and white result, or is it possibly possible to change this behavior without sacrificing anything ?

[VisionDepth]

I'll change the behaviour of the script for RGB frame stitching and Depthframe stitching, thanks for catching that! Update coming soon

[VisionDepth]

new update for the frame stitcher out now
🔗 Here
Updates Include

• Detects if frames are RGB or Grayscale automatically
• Preserves color data if original frames are in RGB
• Ensures OpenCV writes correct video format (Fixes forced grayscale issue)
• Improved error handling for missing or corrupt frames
• Multithreaded frame processing for faster conversion 🚀
• Better UI experience with real-time progress updates 🎛️

[RudiRastelli]

WOW... that's pretty nice ! Works like a charme.
And more speed is always a good thing...

so... because of everthing works just fine the following might not be realy important... just a bunch of my test results and thoughts written here so it doesn't get lost.

Regarding speed.... does the spript check whether a call of the resize function is needed or not... eg when the resolution of the selected frames matches the values from the gui resolution setting ?
(May be the resize function returns immediately in such a case so it doesn't matter... speedwise.... but who knows without trying)

I've tried CUBIC vs LINEAR resize methods with a 30 sec clip i upscaled from 1080p to 2160p. To my surprise both needed aprox 1:25.
I've checked this several times just to make sure.
Another thing about resizing...
Whenever a downscale is nessassary AREA seems to be the best method to resize. At least when u believe the openCV manual, which also says CUBIC is "slow" ;)

Would be nice to select the frame rate from a bunch of presets, but it must also be possible enter a manual value.

Please repeat saying "VisionDepthFrame-Stitch" 10 times in a row as fast as possible and I'm sure you'll have one or even more knots in your tongue afterwards. ;)
How's about eg "VD Frame Stitcher" or something like that ? Something that rolls of your tongue instead of damaging it.
(Imagine a movie poster stating "Jason Statham is THE FRAME STITCHER"... haha... don't take me too seriously on that)

EDIT 1:
H264 encoding doesn't work for me in V1/V2. Progress bar runs very fast and output video is 0 bytes.... doesn't matter if i choose .mkv or .mp4., depth or rgb processing.
H264 encoding in VisonDepth3D works well.

EDIT 2:
Also creating depthmap video seems to be broken on my side with V2... V1 works.

[VisionDepth]

Yea I did notice the bug for H264 codec gonna work on that, I'll run some test with different depthmodels as it shouldn't matter, maybe the detect rgb is messing with the output thinking the different quality is messing with it I'll work on that, training a GAN model at the moment to incorporate true AI upscaling into the mix, I did think the name was too long was just trying to brand it visiondepth but yea VDFrame-Stitch works better thanks for pointing that out

[RudiRastelli]

Hi
Readme: " Download Backwards warp model"
Above u've told me that it is not needed for v1.... is it needed for v2 ?

[VisionDepth]

Yea it was reintroduced with the new update, you can download it Here make sure you download the whole backward_warp.keras file and put the downloaded file into the weights folder that came with the new update, if you run into any trouble with the dependencies let me know and we can pin point which ones are giving you trouble

[RudiRastelli]

Great Readme ! Reading it makes me wanna try ! Good Job !

But there's still room for some improvements i guess...

In the install section it's not totally clear for a noob that there are 2 ways to install. They are visually not clearly seperated from my point of view. How's about something like that...

• Install Method 1 (without virtual environment)
  OR
• Install Method 2 (with Conda virtual environment)

In Method 1 it should read:

• Scroll all the way up here on this webside, click the green "<> Code" button and selelect "Download ZIP"
• Copy the downloaded the zip file to where you want to install VisionDepth3D (eg 'c:\user')
• Right click zip file and select "Extract here" (this will create 'c:\user\VisionDepth3D-main'... rename it to "VisonDepth3D" if u like)

In Method 1 you can slightly simplify the instructions by replacing Step 2.1 and 2.2 with :

• Open VisionDepth3D directory in explorer and enter "cmd" in adress bar

In Method 1: How's about PyTorch install ? It should be explained very well the inexperienced user.
For example i had probs to find out which cuda version is installed on my system. Had to use "nvidia-smi" from the command prompt to find out. It's v12.8.
That leads to the next prob... there is no 'CUDA 12.8' button on the Pytorch webside currently.
A short search on the internet unveils that there is a 'CUDA 12.8' version in the nightly builds, even when it's not on the webside.
I had to change
"pip3 install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/cu126"
to
"pip3 install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/cu128"
In the end it is simple, but i'm pretty sure the inexperienced user will struggle here.

If i have to explain it i would do it like that:

• Run 'nvidia-smi' from command prompt and it will show your installed CUDA version on the 1st line of the output
• Visit "https://pytorch.org/get-started/locally/" with your webbrowser
• Scroll down a bit and set for example "Stable", "Windows", "Pip", "Phyton" and your CUDA version
• Copy command "pip3 install --pre ......" and press right mouse button in command window to paste it
• Press ENTER to install PyTorch for your system
  Note:
  If you have a CUDA version that is not listed on the PyTorch side switch to "Nightly" and when you've pasted the command change it like this for example:
  "..... download.pytorch.org/whl/nightly/cu126" -> "..... download.pytorch.org/whl/nightly/cu128"
  This will only work if PyTorch version for your CUDA is already in the nightlys... you have to try it with this changed command.

In Method 1 i missed the step how to start VisionDepth3D

In Method 2 you should add a 1st step

• Open (for example) 'c:\users' directory in explorer and enter "cmd" in address bar

In Method 2 i missed how to install "Backwards warp model"

May be it's a good idea to provide some batch files for starting... eg 'VisionDepth3D.bat' and 'VisionDepth3D(Conda)'

Will it also work with venv ? In that case i would line out a 3rd Install Method.

Could you pls add a lossless codec ?

[VisionDepth]

Thanks for the feedback again! I'll update the install process again with your suggestions,torch with cuda isn't necessary right now but will be when Depth Estimation gets added into the program, a bat file that installs everything is a great idea and will work on doing that, I use Cuda 11.8 for my Nvidia GPU due to Restraints on newer versions and compatibility with some older open source programs

[RudiRastelli]

a bat file that installs everything is a great idea

I didn't mean bat files for installing... i meant bat files for starting the app.
Don't make it to painless for the user... Playing with the command line at least a little has never hurt anyone.

Here's the start batch file I've made myself... pretty simple it starts the virtual environment and then the app.

VisionDepth3D.bat:

@echo off
call .\venv\Scripts\activate
pythonw -m VisionDepth3D.py
exit /b 0

Further i've made a desktop shortcut for VisionDepth3Dbat.
The only thing left is: a unique icon

I am already very excited about the depth estimation... can't wait to try it.

Regarding lossless codec... i've tried it with IW3 (codec=utvideo) and i got a 260GB(!!!) file for a 01:43:37 FHD video... lol
I think FFV1 codec is much better in terms of efficiency.
I avoid using lossy codecs on my way to the final result and at the last step i use Handbrake to finetune the x264 encoding.

[VisionDepth]

Ahhh gotcha! When I compile everything into a compact one click download I have an ICO image file I use for the icon just haven't been able to get that far with later version lmao, and I'll do more research on codecs and add more options as I just implemented simple codecs to use to get the program running some codecs need to be compiled with cv2.videowriter, once I can get the tabs to align perfectly with current gui which I'm currently struggling to do as you can see the tabs currently create a padding and moves everything over, when I just want the tabs over the video info box

[RudiRastelli]

... and i can see that u have a quest 2 ;)

[VisionDepth]

Yea I still haven't upgraded yet which is long over due as I got the headset two years ago lmao