Realtime simulation plotting

[GREENRAT-K405]

• I have replaced the previous blocking behavior where plots were only generated at the very end of a simulation cycle with real-time, live-updating visualizations across the simulation
  (please take a look on the video as it could be interesting)

My motive behind this big change was:

• In real-world applications (such as simulating a pacemaker system or observing prolonged physiological responses), waiting for an entire 1000+ cycle simulation to finish before seeing the results is impractical. Users and researchers need to visually monitor system stability, inputs (u), and outputs (ym) dynamically as the simulation runs.
• Furthermore, as we prepare for simulations with heavier computational workloads and longer runtimes (e.g., upcoming Julia implementations), having live feedback is critical for monitoring system behavior without waiting for complete execution.

@pradeeban I have done the implementation as we discussed yesterday, while doing that I have discovered that some of the ratc study simulation weren't running properly because of python version (the older version of python were a little linent in terms of math bugs, but newer python is a little stricter, this is causing some kind of value error when debugging some of the graphs especially the yuyu-series graphs.

I believe that we should avoid touching actual code for the simulation as of right now as it contains our legacy studies and doing even a small mistake in fixing the actual simulation can worsen the problem.

In this PR I have just changed the logic of how matplotlib is plotting graph.

This fixes #445 .

realtime1.mp4

[GREENRAT-K405]

@pradeeban @Rahuljagwani @mayureshkothare , I have done the changes in tools, example, demo, ratc and ratc2.

[GREENRAT-K405]

@Rahuljagwani , Will like to know you opinion on this one, as this realtime- plotting change could be beneficial when we will be doing more computational heavy and long running simulations during the julia implementation project,

visually monitoring the simulations in realtime should definetly be beneficial especially when want to monitor multiple plots at the same time.

[Rahuljagwani]

Hi @GREENRAT-K405 while realtime simulations sound like a must have feature, I am not sure of the approach we have chosen here. Can u please share the link of the discussion you and Pradeeban had as you have mentioned in the PR description.

What about just one file which is completely new, showing how to implement realtime simulations. We can discuss this further after you paste the link, I saw the issue but there is no discussion there.

[GREENRAT-K405]

Hello @Rahuljagwani , here is the link to our conversation #429

@pradeeban sir, I have noticed that the plotting/graphs of the simulation done by concore are built after the whole simulation is completed,

• this would be a little impractical in real life examples as we don't just want the end result of simulation, we also want to visually monitor it while the simulation is running
• best example is a pacemaker system, we want the simulation plottings real time, not just a result after entire simulation is completed.

If this is not intentional:

• I propose we make the plotting live. The graph plotting should be done in real time while the simulation is running!
• This is beneficial when the running time of simulation is made quite long (for example 1000), which is how the real world simulation would run I suppose.

This is where I had mentioned this idea.

[GREENRAT-K405]

What about just one file which is completely new, showing how to implement realtime simulations. We can discuss this further after you paste the link, I saw the issue but there is no discussion there.

That's an excellent point @Rahuljagwani , right now, I have modified most of the nodes in ratc that were using matplotlib and changed them to interactive mode.

The most clean fix to this problem could be to create a separate file like realtime_plotym and realtime_plotu, but since some of the examples in ratc were using plotting other than plotym and plotu, I decided to change plotting mode to interactive everywhere.

If you want me to try a differenct approach that I should try or have any suggestions please let me know!

[pradeeban]

I agree with @Rahuljagwani. This implementation touches a lot of files, and that is scary (since it is likely to introduce bugs into previously working examples).

If you could get the live plotting in a new file / new function, that would be great.

Also, one common mistake many applicants are making is, the focus on fine-tuning existing examples. As long as the examples are working (examples as in... ratc.py), I think we shouldn't bother with fixing those. For instance, changing the print statements with logger.info in these examples are largely a wasted effort.

We should rather try to look at the bugs and potential enhancements in the protocol implementation itself (as far as concore is concerned, that would be the file-sharing based, mediator-based, zeromq-based, and shared-memory-based implementations). So, if concore.py or concore-docker.py is having a bug (or even issues like using print statements), they are better fixes.

I recommend a new PR.

1. New file or just new functions with the live plotting feature.
2. A new example from one of those easiest existing examples to demonstrate the newly created live plotting feature.
3. A documentation somewhere (an addition to a README.md makes sense) to highlight this feature and example.

Please avoid touching existing examples (such as ratc, ratc2, ...) in your PR. A brand new example (adopted from an easy example) would showcase this feature better.

[GREENRAT-K405]

@pradeeban , @Rahuljagwani I clearly understand your points now, will make a new PR with the changes you suggested. Thank you.