[test_security] Secure communication tests are unstable causing CI to fail sometimes

[mikaelarguedas]

Bug report

Required Info:

• Operating System:
  • Ubuntu Focal
• Installation type:
  • source
• Version or commit hash:
  • master
• DDS implementation:
  • Fast-RTPS (static and dynamic)
• Client library (if applicable):
  • rclcpp

Since ros2/sros2#208 the end-to-end tests of sros2 are enabled on the CI of this repo (daily master branch test + PR testing).
While the code is located at https://github.com/ros2/system_tests/tree/master/test_security I'm reporting it here because that's the repo watched by he sros2 maintainer and the test failures are happening on the CI of this repo.

Some tests time out making CI fail, this seems unrelated to the changes being tested, some examples of this:

• Daily build: https://github.com/ros2/sros2/runs/710709221?check_suite_focus=true

41 - test_secure_publisher_subscriber__UnboundedSequences__rmw_fastrtps_dynamic_cpp__secure_comm_0 (Timeout)

• PR builds:
  • https://github.com/ros2/sros2/runs/722938561

19 - test_secure_publisher_subscriber__UnboundedSequences__rmw_fastrtps_cpp__secure_comm_1 (Timeout)

• https://github.com/ros2/sros2/pull/220/checks?check_run_id=723326846

30 - test_secure_publisher_subscriber__Empty__rmw_fastrtps_dynamic_cpp__secure_comm_0 (Timeout)

• https://github.com/ros2/sros2/pull/220/checks?check_run_id=723328796

30 - test_secure_publisher_subscriber__Empty__rmw_fastrtps_dynamic_cpp__secure_comm_0 (Timeout)

Looking at the ROS2 CI from ci.ros2.org over the last week it seems to fail very often but only on linux and only for fastrtps. It goes unnoticed because tests are ran multiple times:

Linux:

• May 30, 2020

  33 - test_secure_publisher_subscriber__Empty__rmw_fastrtps_cpp__secure_comm_3 (Timeout)
  43 - test_secure_publisher_subscriber__UnboundedSequences__rmw_fastrtps_cpp__secure_comm_2 (Timeout)

• May 29, 2020

  41 - test_secure_publisher_subscriber__UnboundedSequences__rmw_fastrtps_cpp__secure_comm_0 (Timeout)

• May 28, 2020

  43 - test_secure_publisher_subscriber__UnboundedSequences__rmw_fastrtps_cpp__secure_comm_2 (Timeout)

• May 27, 2020
  ✔️
• May 26, 2020

  43 - test_secure_publisher_subscriber__UnboundedSequences__rmw_fastrtps_cpp__secure_comm_2 (Timeout)

• May 24, 2020 (25 may did not run)
  ✔️

MacOS:

• All Connext tests fail (for a different reason) but all Fast-RTPS tests pass ✔️

Windows

• All tests pass ✔️

[kyrofa]

While the code is located at https://github.com/ros2/system_tests/tree/master/test_security I'm reporting it here because that's the repo watched by he sros2 maintainer and the test failures are happening on the CI of this repo.

I'm not sure it makes sense to have a system_tests issue logged here. It certainly effects sros2 but the fix won't happen here, it'll happen over there. I propose we open this issue against system_tests and assign @ros-security-wg over there like other issues. I don't have permission to transfer it, does anyone else?

[clalancette]

I don't have permission to transfer it, does anyone else?

I'll go ahead and do it.

[jacobperron]

I'm pretty sure this is a duplicate of #435, so I'll close this out. Please comment and/or re-open if I'm mistaken.

[mikaelarguedas]

I believe this issue was moved from sros2 to here to replace #435 that was a duplicate without much context.
If the content of #435 is now more up-to-date than the content of this one let's keep the other one