I have noticed that every time when I reload the visual code, the memory usage seems to increase. It does decrease a bit after a few seconds, but not back to where the memory usage was before. Removing the visual doesn't help with this as well.
I am currently building my own visual based on this sample and are experiencing the same problem. I wonder whether the visual has cleaned up properly on reload or remove.