Using as a library?

[XavierTolza]

Hi. First, thanks for this really nice plotting soft.
I think it would be really interesting to use your code to plot timeseries from python or C program without having to go manually through all the mess of file formatting, etc. I mean a C/python API where you just enter ftsplot(x,y1,y2,y3,...) and its handles the files creation and processing in a temporary folder and plots the result.
I can do the python interface, but I would just need a "API" to hook at with the following features:

• FtsPrep: given a file, process it in headless mode, return the path of the corresponding resulting config file
• FtsPlot:
  • given a flat file of events or intervals & a folder, import the flat file into the folder
  • given a list of timeseries config files and events folders, open a windows with the corresponding plots

Do you think it would be time consuming to create those three functions? I dive into the code but I do not know anything about QT. Additionally it would be nice if we could set up the colors of the plot.

If you could provide me with those functions, I could make a python package which could greatly improve ease of use of your library.
Thanks

[MichaelRiss]

Hi! It's nice that someone else is using it/wants to use it after all that time. ;-)
Should it be of use to you, it would make me happy.

I understand where you are coming from. In fact, I also was thinking about a do-over with a more modular approach : core library, multiple language bindings, multiple render heads, some more visualization ideas maybe ...
The thing is, in the beginning it was not clear whether this algorithm idea would be fast enough. For that reason I optimized the code as much as possible and now it's unfortunately in classic 1.0 spaghetti code state. The core algorithm tightly integrates memory mapping, OpenGL and Qt (and version 4 of it, due to the specific multi-threaded use of OpenGL through the Qt API).
To do it right, the core algorithm would need to be untangled from those specific APIs and with that you are re-implementing the core algorithm and then everything around it again => complete rewrite.

So far, I didn't find the time to do it. Now we are three people - I have one more friend who is (or at least was) also interested in adding a python interface to it. But it's tough.

However, the current software already has a basic API. What I can point you to at the moment is the mini code snippet in the publication (https://journals.plos.org/plosone/article/authors?id=10.1371/journal.pone.0094694) on PDF page 8 or the more colorfull and better formatted version (http://hdl.handle.net/2117/94959), PDF page 49, albeit with the old software name. This is the way to use FTSPlot as a Qt component and the benchmark code makes use of this API in https://github.com/MichaelRiss/FTSPlot/blob/master/benchmarks/FTSPlotBenchMainWindow.cpp.

The functions you suggest may be doable already with a glue layer which translates from ANSI C to the FTSPlot API. The one foreseeable problem should be the conflict for the main thread: Qt uses the main thread for signal and event handling, while python most likely uses it to run its interpreter. I need to check the PyQt/Qt for Python projects to see how they handled this problem.

[XavierTolza]

No problem with the spaghetti code, mine is worse ^^
Anyway, I think your code snippet is what I was looking for (even though I didn't test it yet), but there is still one step I need some indications on if possible: preprocessing headlessly the binary file with FTSPrep. A function alike this one, returning the path to the .cfg file:

std::string preprocess_file(std::string input_file_path,unsigned reduction_factor);

For the python part, I don't think it would be such a problem, I was think of making a very basic C api and then using Cython to call it, which allows to unlock some threading problems by bypassing the GIL. I don't known which problems I may run into but I think a dirty piece of code may be doable ^^

[XavierTolza]

Hi. I did a first step forward to solve this problem on PR #7
I found the way to make command line arguments but I couldn't configure the cmake to make a C library.
And the FTSPrep part is ok now too.

[MichaelRiss]

Hi! Sorry for the long delay, lots of other stuff to do.

• Do you know that the standard build already creates the libFTSPlot.a?

• Some basic code for using FTSPrep from the library:

  #include "FTSPrep.h"
  
  using namespace FTSPlot;
  
  int main(int argc, char *argv[])
  {
      FTSPrep worker;
      worker.setNewFile(QString("PathToDataFile"));
      worker.start();
      worker.wait();
  
      return 0;
  }
  

  The corresponding CMakeLists.txt:

  cmake_minimum_required(VERSION 2.8.12)
  
  project(PrepExample)
  
  set(CMAKE_INCLUDE_CURRENT_DIR ON)
  set(CMAKE_AUTOMOC ON)
  
  find_package(Qt4 4.8.7 REQUIRED QtCore)
  
  include_directories(BEFORE "Path to FTSPlot source directory")
  link_directories(BEFORE "Path to FTSPlot build directory")
  
  add_executable(${PROJECT_NAME} "main.cpp")
  
  target_link_libraries(${PROJECT_NAME} Qt4::QtCore FTSPlot)
  

  In contrast to your code I start the thread with start() and then I wait() for it. This gives you the opportunity to install a progress callback later to see how the preparation is progressing.

• About the code design of your PR: It goes a bit against the idea of FTSPlot. The separation between FTSPrep and FTSPlot is for doing the preparation work offline with FTSPrep, in a batch job that can run for hours over night. And FTSPlot is the fast real-time tool once FTSPrep has finished. Now, you couple these two together, which may surprise the user as sometimes the program starts up quick and sometimes, when the prep work still has to be done, it takes hours. Also, as you called run() directly in the main thread the whole program blocks without event handling, giving no feedback on how the preparation is progressing.

• I tried to wrap the C++ example above with sip into a python module, but as the API uses Qt data types this is a more advanced problem and I wont be able to finish it this weekend. However, if you are proficient with sip and PyQt4 you may give it a try. If we get this to run, then we could provide the functions separately to python and then the user can decide in which way and order he wants to combine the functions.

What do you think?

[XavierTolza]

Hi!

• Fine for the lib, its just that I could not figure out how to create a full plot window, but with your cmake example, I should be able to make it, I'll try when I have enough time.
• I totally understand about the blocking problem and the separation between FTSPrep and FTSPlot. It's just that we do not have the same usage: I'm using it only for "small" binary files (still too big for my computer) where FTSPrep only takes a fraction of second, thus I wanted to automate the process in one command line in the terminal. I'll give up the PR and try your way, then the automation could be done in a higher layer (for example in the python code)
• I do not use sip, I rather use cython which I think makes quite easy to bind to C/C++ libs. I'll give it a try and tell you.

Thanks, I'll try with your cmake snippet and keep you updated