Make Kimera output throttle a parameter

[violetteavi]

Is your feature request related to a problem? Please describe.
Kimera throttles its output publishing rate. Currently, the rate is limited to a hardcoded 30Hz. I'm working with a dataset that operates at 60Hz, meaning that I will only ever publish half of the frames I see.

Describe the solution you'd like
I'd like a parameter passable through the .launch or .flag files that allows me to set this publishing rate myself. I plan to set it to 2x my input frequency so that it can catch up with any lags that occur.

Describe alternatives you've considered
I could just change the number in code and recompile each time, but that seems suboptimal.

Additional context
I'll be putting up a potential solution a little later.

[violetteavi]

The rate throttle in question is in the RosOnlineDataProvider here.

[ToniRV]

I’m about to push a ton of code to split publishing from subscription to dat, so everything can go as fast as possible

[violetteavi]

Great! This is not a processing speed issue, though. Even slowing down my rosbag 20x I still dropped half the frames-- the issue is with the rate limit on the Kimera output.

[ToniRV]

I think I found the reason why my use case is leaking so horribly-- Kimera throttles its output to 30Hz, so half of my frames are getting effectively leaked as the queue backs up.
Changing the publishing rate to 120Hz kept the RAM usage to a manageable 400MB throughout

Interesting 🤔 I don't think that is a memory leak though, but that the threadsafe queues are getting full. Can you switch on the verbosity in the ThreadsafeQueue.h? Checkout this line

The flag you need is:

--vmodule=ThreadsafeQueue=1

And let me know if you see that the queues are getting full.

The reason why I don't think it is a memory leak is that I don't see why or how what you said would happen:

half of my frames are getting effectively leaked as the queue backs up

What do you mean by the "queue backs up"?

[ToniRV]

Linking issue to #69

[violetteavi]

You're right-- "memory leak" is the wrong term here. It's not a leak, but a backlog in the queues as the output messages line up, but cannot be sent due to rate limiting (logs are below). This presents similar problems as a memory leak-- the backlog grows and grows, chewing up all available memory. The simple solution is to set the publishing rate > than the input framerate. This will prevent the backlog at the publishing point. This works because the publishing rate is completely arbitrary ahaha. It will be trickier when we run into rates that are not easily adjustable, like the Kimera processing rate.
OnlineOutpututQueueVerbose.txt
Of particular interest:

I0511 23:39:12.854346  8315 ThreadsafeQueue.h:270] Queue with id: Frame Rate Frontend output ROS is getting full, size: 317

I ran it with a 120 HZ publishing limit and the Frame Rate Frontend output ROS messages went away.

[violetteavi]

Note that this is no longer an issue because PR#72 removes the polling and replaces it with asynchronous publishing using callbacks.