Revise NIT to quietly TAP by default #1404
Description
#1380 and #1383 introduced NIT (NUT Integration Testing) suite for running some dummy drivers, data server, and a few clients to test them, and the bindings from supported language.
Being an early effort dealing more with debugging the tests than with non-regressions, the currently merged NIT script is pretty noisy by itself, and the tools it calls spew even more to the console. This issue is about converting it to TAP standard (at least by default) so detailed logs go to file artifacts and console is only polluted by verdicts.
More reading:
- https://testanything.org/
- https://testanything.org/tap-version-14-specification.html
- https://www.gnu.org/software/automake/manual/html_node/Scripts_002dbased-Testsuites.html
- https://www.gnu.org/software/automake/manual/html_node/Using-the-TAP-test-protocol.html
- https://stackoverflow.com/questions/20440322/how-to-make-make-check-process-tap-output
- http://git.savannah.gnu.org/cgit/automake.git/tree/lib/tap-driver.sh
Also consider updating C/C++ NUT tests themselves to produce the TAP output as well for the points they check, see e.g. https://testanything.org/testing-with-tap/c-plus-plus.html (currently the GNU Automake TAP recipes handle the running of test programs each as a single test unit for us).
An aspect for NUT CI farm via Jenkins-Dynamatrix : Also extract the log files from (failed-only?) builds and visualize with https://github.com/jenkinsci/tap-plugin ; note that it already does some such extraction to provide clues for failed distcheck etc. investigations (but at this time there are issues to fix -- see e.g. networkupstools/jenkins-dynamatrix#38).