separate data collection and analysis

[lukpueh]

The current design runs independent checks that each fetch their own data, extract what they need, return scalar results, and discard the rest.

This works for simple cases but doesn't scale well, when one check queries data, which could be used in multiple check results.

To improve flexibility, I suggest to revamp the check runner as "data collection" runner, where each "collector" independently (or inter-dependently; see #5) calls APIs, and feeds the queried data into a global flexible data model, for subsequent analysis and report generation.

The data analysis phase could follow a similar pattern as the data collection phase, where (initially only) independent analyzers run in parallel based on collected data. And new analyzers can be added easily.

[lukpueh]

Update:

To improve flexibility, I suggest to revamp the check runner as "data collection" runner, where each "collector" independently (or inter-dependently; see #5) calls APIs, and feeds the queried data into a global flexible data model, for subsequent analysis and report generation.

Implemented in #8, although "collectors" are still just called "checks".

The data analysis phase could follow a similar pattern as the data collection phase, where (initially only) independent analyzers run in parallel based on collected data. And new analyzers can be added easily.

I briefly tried having separate data "collectors" and "analyzers", but just ended up with a lot of duplicated code, without clear advantage. Moreover, I realized that there are scenarios, where "collected" data is "analyzed", and the result entails further "collection". This is easier in the current implementation, where there is no clear line between collection and analysis.

I suggest to leave this issue on the back burner and revisit when we have a better idea of the "report generation" phase of this tool.