-
Notifications
You must be signed in to change notification settings - Fork 34
Open
Description
A proposed ci.yml refactoring splits flux-ubuntu tests into four separate jobs and runs them in parallel on separate nodes. Three jobs cover SPINDLE_DEBUG=[1-3] and the fourth does not set SPINDLE_DEBUG.
The run in question succeeded for SPINDLE_DEBUG=[1,3] and the no-debugging option, but failed on SPINDLE_DEBUG=2 with the error message:
Run docker exec node-1 bash -c 'cd Spindle-build/testsuite && SPINDLE_DEBUG=2 flux alloc --nodes=${workers} ./runTests --nodes=${workers} --tasks-per-node=3'
docker exec node-1 bash -c 'cd Spindle-build/testsuite && SPINDLE_DEBUG=2 flux alloc --nodes=${workers} ./runTests --nodes=${workers} --tasks-per-node=3'
shell: /usr/bin/bash -e {0}
Running: ./run_driver --dependency --push
60.237s: flux-shell[0]: stderr: spindleRunBE failed!
Error: The action 'Run spindle-flux-ubuntu-debug2 testsuite' has timed out after 7 minutes.
At that point the CI script should extract the log files, but based on the ls the script performs, no files were generated. Looking at the likely code path, LOGGING_FINI is not called during any of the error paths, so the log files were likely not flushed to disk before spindle exited.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels