🧑‍🌾 `sros2.test.sros2.commands.security.verbs.test_generate_policy_no_nodes.test_generate_policy_no_nodes` failing consistently in Kilted CI

[Crola1702]

Generated by Generative AI

No response

Operating System:

Ubuntu Nomble

ROS version or commit hash:

Kilted

RMW implementation (if applicable):

No response

RMW Configuration (if applicable):

No response

Client library (if applicable):

No response

'ros2 doctor --report' output

No response

Steps to reproduce issue

1. Run a build in Kci debug or release preferably
2. See test fail

Expected behavior

Test should not fail

Actual behavior

Test is failing

Additional information

Test regression:

• sros2.test.sros2.commands.security.verbs.test_generate_policy_no_nodes.test_generate_policy_no_nodes

Reference builds:

• Kci__nightly-connext_ubuntu_noble_amd64#34
• Kci__nightly-cyclonedds_ubuntu_noble_amd64#29
• Kci__nightly-debug_ubuntu_noble_amd64#34
• Kci__nightly-release_ubuntu_noble_amd64#33

Log output:

=====================================================================================================================================================================================
FAIL: test_generate_policy_no_nodes.TestSROS2GeneratePolicyVerbWithNoNodes.test_generate_policy_no_nodes[rmw_connextdds, True]
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/ws/src/ros2/sros2/sros2/test/sros2/commands/security/verbs/test_generate_policy_no_nodes.py", line 62, in test_generate_policy_no_nodes
    assert gen_command.exit_code != launch_testing.asserts.EXIT_OK
AssertionError: assert 0 != 0
 +  where 0 = <launch_testing.tools.process.ProcessProxy object at 0x71058cd7a8d0>.exit_code
 +  and   0 = <module 'launch_testing.asserts' from '/tmp/ws/install_isolated/launch_testing/lib/python3.12/site-packages/launch_testing/asserts/__init__.py'>.EXIT_OK
 +    where <module 'launch_testing.asserts' from '/tmp/ws/install_isolated/launch_testing/lib/python3.12/site-packages/launch_testing/asserts/__init__.py'> = launch_testing.asserts


=====================================================================================================================================================================================
FAIL: test_generate_policy_no_nodes.TestSROS2GeneratePolicyVerbWithNoNodes.test_generate_policy_no_nodes[rmw_connextdds, False]
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/ws/src/ros2/sros2/sros2/test/sros2/commands/security/verbs/test_generate_policy_no_nodes.py", line 62, in test_generate_policy_no_nodes
    assert gen_command.exit_code != launch_testing.asserts.EXIT_OK
AssertionError: assert 0 != 0
 +  where 0 = <launch_testing.tools.process.ProcessProxy object at 0x71058d1d89e0>.exit_code
 +  and   0 = <module 'launch_testing.asserts' from '/tmp/ws/install_isolated/launch_testing/lib/python3.12/site-packages/launch_testing/asserts/__init__.py'>.EXIT_OK
 +    where <module 'launch_testing.asserts' from '/tmp/ws/install_isolated/launch_testing/lib/python3.12/site-packages/launch_testing/asserts/__init__.py'> = launch_testing.asserts

The test has been failing since the start of Kilted. And the test is consistent

Flaky ratio

job_name	last_fail	first_fail	build_count	failure_count	failure_percentage
Kci__nightly-release_ubuntu_noble_amd64	2025-08-20	2025-08-08	5	5	100.0
Kci__nightly-connext_ubuntu_noble_amd64	2025-08-19	2025-08-07	5	4	80.0
Kci__nightly-debug_ubuntu_noble_amd64	2025-08-16	2025-08-07	4	4	100.0
Kci__nightly-cyclonedds_ubuntu_noble_amd64	2025-08-07	2025-08-07	5	1	20.0

Last time it happen on Rolling was on Rci__nightly-release_ubuntu_noble_amd64#457 and [Rci__nightly-connext_ubuntu_noble_amd64#460] https://build.ros2.org/view/Rci/job/Rci__nightly-connext_ubuntu_noble_amd64/460/#showFailuresLink. However, there are no changes between the next Rolling builds that can determine if a fix was applied, or if a change solved it.

[ros-discourse]

This issue has been mentioned on Open Robotics Discourse. There might be relevant details there:

https://discourse.openrobotics.org/t/ros-pmc-minutes-for-august-26-2025/49759/1

[fujitatomoya]

this failure cannot be reproduced by my local environment.

the only thing i can see is that, test expects the NO other nodes are online, so that it expects the command fails with specific error message. but in the test CI environment, some nodes are still online or left over, that leads this test to pass the command ros2 security generate_policy xxx.

[Crola1702]

Flakiness for the last 15 days:

job_name	last_fail	first_fail	build_count	failure_count	failure_percentage
Kci__nightly-release_ubuntu_noble_amd64	2025-10-20	2025-10-11	5	4	80.0
nightly_linux_repeated	2025-10-19	2025-10-12	14	3	21.43
Kci__nightly-debug_ubuntu_noble_amd64	2025-10-19	2025-10-10	3	3	100.0
Kci__nightly-connext_ubuntu_noble_amd64	2025-10-19	2025-10-07	5	5	100.0
nightly_linux-aarch64_repeated	2025-10-17	2025-10-10	14	3	21.43
nightly_linux-rhel_repeated	2025-10-15	2025-10-13	15	2	13.33

See that it's happening now on repeated jobs.

Should we disable this test? @mjcarroll?

[ros-discourse]

This issue has been mentioned on Open Robotics Discourse. There might be relevant details there:

https://discourse.openrobotics.org/t/ros-pmc-minutes-for-november-11-2025/51100/1

[ros-discourse]

This issue has been mentioned on Open Robotics Discourse. There might be relevant details there:

https://discourse.openrobotics.org/t/ros-pmc-minutes-for-november-18-2025/51101/1

[ros-discourse]

This issue has been mentioned on Open Robotics Discourse. There might be relevant details there:

https://discourse.openrobotics.org/t/ros-pmc-minutes-for-november-25-2025/51279/1

[ros-discourse]

This issue has been mentioned on Open Robotics Discourse. There might be relevant details there:

https://discourse.openrobotics.org/t/ros-pmc-minutes-for-december-2-2025/51280/1