About cv2.canny

[Liqq1]

Hello! I have some questions about cv2.canny module.
What is the function of "cv2. Canny"module in the figure? This step is not in the MSRF-Net, and I don't see a place in the Shape Stream Block where I can input it.
I am looking forward hearing from you!

[amlarraz]

Hi!

cv2.Canny reefers to the canny function in opencv.
It is only neccesary for the "Shape Stream" which is feeded by x13, x23, x33, x44 and also with the original image with Canny applied. The objective of the shape Stream is to reproduce the canny image using the original canny image combined with the multiscale features. I think the shape stream block is an additional regulatization method. You can read more about this here.

In the code you can see Canny in the following lines:

• Dataset line 85 -87 and 91
• Model line 374 and 400
• Loss in class CombinedLoss (line 132)

You can also see that in the train, validation and test code.

[Liqq1]

Thank you for your reply～
After I looked at the code, I relearned it. I wonder if it can be understood as follows: MSRF-Net does not have a Canny operation in Figure 1, but it does in Figure 2. As a result, the network in Figure 2 will have an additional concat operation in Shape Stream.

[amlarraz]

Mmmmm I think the figure 1 doesn't have Canny operation because this operation isn't part of the model but part of the data to feed the model (and also to compute loss). Yep, maybe I shouldn't have added canny in the model architecture draw...

[Liqq1]

I mean that I think we can just use x13, X23, X33, x44 to get what we want without concatenating the edges of the original image；Maybe I was wrong about that:-(