Every plugin that supports graph X needs a series of integration tests that run Y variants of graph X through the plugin and compares to some reference implementation (you could also call this differential testing). This set of tests will be identical plugin to plugin, there will be many differentiated tests specific to particular plugins, but there will also be significant overlap.
Naturally, it would be better to define the test suite in one place. Keeping a central test suite will help avoid duplicate work, and will every time someone discovers a fiddley edge case constructable with the hipDNN API that a plugin needs to handle we can ensure that every plugin handles that well.
We can define the central suite as a library that plugins pull in, or given hipDNN's strong de-coupling between itself and plugins (hipDNN works with whatever .sos happen to end up in /lib/rocm/hipddn/plugins), as one central suite in TheRock that iterates over a parameterized set of loaded plugins. The latter option also allows us to test the final user facing hipDNN API that will be presented to users (i.e. hipDNN with the plugins that will build + distribute by default with ROCm).
Every plugin that supports graph X needs a series of integration tests that run Y variants of graph X through the plugin and compares to some reference implementation (you could also call this differential testing). This set of tests will be identical plugin to plugin, there will be many differentiated tests specific to particular plugins, but there will also be significant overlap.
Naturally, it would be better to define the test suite in one place. Keeping a central test suite will help avoid duplicate work, and will every time someone discovers a fiddley edge case constructable with the hipDNN API that a plugin needs to handle we can ensure that every plugin handles that well.
We can define the central suite as a library that plugins pull in, or given hipDNN's strong de-coupling between itself and plugins (hipDNN works with whatever
.sos happen to end up in/lib/rocm/hipddn/plugins), as one central suite in TheRock that iterates over a parameterized set of loaded plugins. The latter option also allows us to test the final user facing hipDNN API that will be presented to users (i.e. hipDNN with the plugins that will build + distribute by default with ROCm).