Experiment planning (goal: data generation)

[MatthiasLen]

The implementation of algorithms and distortions is now in a good initial state. We need to move to planning and running experiments with the ultimate goal of data generation:

• what experiments do we run ?
• what data do we use?
• which distortions do we apply ?
• which hyper-parameters do we use for the distortion? see Hyperparameter Selection for Classic Reconstruction Algorithms #13

OLD Notes from discussion with @Andrewwango

[MatthiasLen]

@melanie-dohmen-bayer there is now functionality in place . how to proceed with planning experiments / data generation ?

[melanie-dohmen-bayer]

I will further work on this issue, when I find more time. My rough ideas are:

1. Select 2-5 raw k-space images from fast-MRI and Oasis dataset including different body parts and run over all available and possibly matching distortion- reconstruction combinations
   2a) The data generation script should already store the reconstructed images as 2D TIFF images (maybe select only a subset of the whole volumes) with a reasonable naming strategy and output folder structure, such that "reference" and different distortions thereof can be easily grouped together.
   2b) Maybe a separate data generation script for multiple pixel sizes will be needed, see issue Add Zero-filling distortion (Pixel Resolution) #31
2. Manually inspect results and confirm, if the distortions achieve the targeted effects and identify which combinations of distortion and reconstructions are not useful or redundant.
3. Select a larger set of cases from fast-MRI and Oasis and generate more images for a smaller set of distortion-reconstruction combinations.
4. Manually select cases from generated results, discuss in the group meeting and submit to the core group.

@Andrewwango /@MatthiasLen: Are there restrictions in the possible combinations of data input / distortions / reconstruction models and how could they be identified?

[MatthiasLen]

@melanie-dohmen-bayer i am not aware of anything specif at the moment (besides the functionallity that is not implemented jet ). I think it makes sense to go ahead and try :) in case something is not working we can fix it

[MatthiasLen]

Andrew will look end-2-end at recon x distortion x metric

[melanie-dohmen-bayer]

I propose to plan 3 experiment runs and 2 analysis steps and will create separate issues for each one:

Experiment Run 1)
Run all distortions and reconstruction algorithms on a minimum set of images (1 from each fast-MRI subdataset, that is supported and 1 from Oasis).
Analysis 1)
We then review and select combinations of data sources + distortions + reconstruction algorithms
Experiment Run 2)
For the pixel resolution distortion/ image property, we plan a separate set of experiments without further distortions and select an appropriate set of resolution parameters for each case.
Experiment Run 3)
Final run on much more cases, just on the selected combinations from Analysis 2
Analysis 2)
Final selection of image cases, consisting each of 1 reference and 2 modified versions displaying visible difference in one of the assigned image properties

[melanie-dohmen-bayer]

@MatthiasLen: With the newly create issues, can we close this, or do you propose changes to the plan?