Different optical flow image

[pachiko]

I'm using OpenCV 3.4.1 with your code, and I got these type of flow images.

Sometimes they even go black.

Training is impossible as I get NaN loss after awhile.

[pachiko]

Tried running a 2nd time. looks OK now, as it can learn and the loss doesn't explode. could be bad initialization. However, the optical flow image still looks different.