File format for arbitrary array structs

[mmaelicke]

With #5 numpy arrays of 1D and 2D dims will be stored to .dat files. We then lack a standard format to store arrays of any shape, which can easily be opened in our Python, R and Matlab/Octave environments.

1. .mat is obviously easy in Matlab, but require extra packages in Python and R. In addition, a .mat can store more than one matrix, but we want to limit this use-case to only one.
2. We come up with our own txt/binary: a header storing the shape and then just a long list. This is extremely flexible, but needs custom implementation in all 3 environments
3. VTK files should be usable in most environments and can define multi-dimensional mesh. We would have to check if that can be implemented in R and octave without too many dependencies.
4. We use the good old netCDF. This has the advantage, that we could also set an attribute, describing that the .nc was written using toolbox-runner and any needed info to parse it in Matlab or R

@AlexDo1 what do you think? How should we implement nD-arrays?

[AlexDo1]

1. .mat is in my opinion too Matlab specific, this could possibly irritate users and then it would be logical that we create input files with Matlab, which we don't want I think.
2. A custom format would be okay, but this is also not intuitive for new users and would probably be harder to maintain.
3. I never heard of VTK files, but it looks like another pretty specific file format.
4. Using netCDF files is probably the best choice in my opinion, since we already work with these files. The disadvantage is that preprocessing the data (generating the netCDF) is more work than simply writing the matrix to a text file. But it would also be an advantage to be able to use the netCDF attributes, as you mentioned.

[mmaelicke]

In addition to 4:
The templates will need the netCDF reading capabilities anyway as described in #7. Thus, I agree that 4. is the best option.