Robot disconnects briefly and instantly connects again

[Wingragon]

Affected ROS Driver version(s)

??

Used ROS distribution.

Other

Which combination of platform is the ROS driver running on.

Linux

How is the UR ROS Driver installed.

From binary packets

Which robot platform is the driver connected to.

UR E-series robot

Robot SW / URSim version(s)

Ur16e

How is the ROS driver used.

Through the robot teach pendant using External Control URCap

Issue details

Summary

Hi something on this robot doesn't work and i would be thankful for any and all help. If anymore data is needed like the Driver version it would be nice if someone could tell me where to find it, because this robot was setup long before my time and I'm just a student .

Issue details

After building a connection with robot and completing the handshake the connection to the reverse interface is dropped and instantly picked up again. This results in the robot stopping mid execution.

Steps to Reproduce

I execute this command ros2 launch ur_robot_driver ur_control.launch.py ur_type:=ur16e robot_ip:=192.168.20.20 launch_rviz:=false description_package:=ur_description in a terminal then wait until there is no more output and then start the urcap program on the tablet. After some time the error just pops up but i wasn't able to find any pattern.

Expected Behavior

I expect it to not lose connection because it did not do that until about 2-3 weeks ago.

Actual Behavior

Seen in log output.

Workaround Suggestion

Relevant log output

ros2 launch ur_robot_driver ur_control.launch.py ur_type:=ur16e robot_ip:=192.168.20.20 launch_rviz:=false description_package:=ur_description
[INFO] [launch]: All log files can be found below /home/autopilot/.ros/log/2026-03-04-15-59-54-980539-autopilot-9515
[INFO] [launch]: Default logging verbosity is set to INFO
[INFO] [dashboard_client-2]: process started with pid [9520]
[INFO] [controller_stopper_node-4]: process started with pid [9524]
[INFO] [ur_ros2_control_node-1]: process started with pid [9518]
[INFO] [robot_state_helper-3]: process started with pid [9522]
[INFO] [urscript_interface-5]: process started with pid [9526]
[INFO] [robot_state_publisher-6]: process started with pid [9528]
[INFO] [spawner-7]: process started with pid [9573]
[INFO] [spawner-8]: process started with pid [9582]
[ur_ros2_control_node-1] [WARN] [1772636395.729156605] [controller_manager]: [Deprecated] Passing the robot description parameter directly to the control_manager node is deprecated. Use '~/robot_description' topic from 'robot_state_publisher' instead.
[ur_ros2_control_node-1] text not specified in the tf_prefix tag
[ur_ros2_control_node-1] [INFO] [1772636395.729747886] [resource_manager]: Loading hardware 'ur16e' 
[ur_ros2_control_node-1] [INFO] [1772636395.736971912] [resource_manager]: Initialize hardware 'ur16e' 
[ur_ros2_control_node-1] [INFO] [1772636395.737145933] [resource_manager]: Successful initialization of hardware 'ur16e'
[ur_ros2_control_node-1] [INFO] [1772636395.737486679] [resource_manager]: 'configure' hardware 'ur16e' 
[ur_ros2_control_node-1] [INFO] [1772636395.737506616] [URPositionHardwareInterface]: Starting ...please wait...
[ur_ros2_control_node-1] [INFO] [1772636395.737528440] [URPositionHardwareInterface]: Initializing driver...
[ur_ros2_control_node-1] [WARN] [1772636395.749274266] [UR_Client_Library:]: Your system/user seems not to be setup for FIFO scheduling. We recommend using a lowlatency kernel with FIFO scheduling. See https://docs.universal-robots.com/Universal_Robots_ROS_Documentation/doc/ur_client_library/doc/real_time.html for details.
[robot_state_helper-3] [INFO] [1772636395.725459693] [UR_Client_Library:]: Starting primary client pipeline
[dashboard_client-2] [INFO] [1772636395.738208173] [UR_Client_Library:]: Starting primary client pipeline
[controller_stopper_node-4] [INFO] [1772636395.733650564] [Controller stopper]: Waiting for switch controller service to come up on controller_manager/switch_controller
[robot_state_publisher-6] [INFO] [1772636395.762088603] [robot_state_publisher]: got segment base
[robot_state_publisher-6] [INFO] [1772636395.762283745] [robot_state_publisher]: got segment base_link
[robot_state_publisher-6] [INFO] [1772636395.762304674] [robot_state_publisher]: got segment base_link_inertia
[robot_state_publisher-6] [INFO] [1772636395.762315330] [robot_state_publisher]: got segment flange
[robot_state_publisher-6] [INFO] [1772636395.762324547] [robot_state_publisher]: got segment forearm_link
[robot_state_publisher-6] [INFO] [1772636395.762332547] [robot_state_publisher]: got segment ft_frame
[robot_state_publisher-6] [INFO] [1772636395.762340355] [robot_state_publisher]: got segment shoulder_link
[robot_state_publisher-6] [INFO] [1772636395.762348035] [robot_state_publisher]: got segment tool0
[robot_state_publisher-6] [INFO] [1772636395.762355875] [robot_state_publisher]: got segment upper_arm_link
[robot_state_publisher-6] [INFO] [1772636395.762363620] [robot_state_publisher]: got segment world
[robot_state_publisher-6] [INFO] [1772636395.762371396] [robot_state_publisher]: got segment wrist_1_link
[robot_state_publisher-6] [INFO] [1772636395.762379172] [robot_state_publisher]: got segment wrist_2_link
[robot_state_publisher-6] [INFO] [1772636395.762386884] [robot_state_publisher]: got segment wrist_3_link
[ur_ros2_control_node-1] [INFO] [1772636395.793432970] [UR_Client_Library:]: Negotiated RTDE protocol version to 2.
[ur_ros2_control_node-1] [INFO] [1772636395.793961530] [UR_Client_Library:]: Setting up RTDE communication with frequency 500.000000
[ur_ros2_control_node-1] [INFO] [1772636395.803063660] [UR_Client_Library:]: Starting primary client pipeline
[ur_ros2_control_node-1] [INFO] [1772636395.803838372] [URPositionHardwareInterface]: Calibration checksum: 'calib_1935737212149401527'.
[robot_state_helper-3] [INFO] [1772636395.838764190] [UR_Client_Library:]: Logging an ErrorCodeMessage from the UR Controller Box: C210A0
[spawner-7] [INFO] [1772636396.305917095] [spawner_joint_state_broadcaster]: waiting for service /controller_manager/list_controllers to become available...
[spawner-8] [INFO] [1772636396.352277402] [spawner_joint_trajectory_controller]: waiting for service /controller_manager/list_controllers to become available...
[dashboard_client-2] [INFO] [1772636396.443635543] [UR_Client_Library:]: Connected: Universal Robots Dashboard Server
[dashboard_client-2] 
[ur_ros2_control_node-1] [WARN] [1772636396.816552141] [UR_Client_Library:]: Failed to read from stream, reconnecting in 1 seconds...
[ur_ros2_control_node-1] [ERROR] [1772636397.816851271] [UR_Client_Library:]: Pipeline producer overflowed! <PrimaryClient Pipeline>
[ur_ros2_control_node-1] [ERROR] [1772636398.804312931] [URPositionHardwareInterface]: The calibration parameters of the connected robot don't match the ones from the given kinematics config file. Please be aware that this can lead to critical inaccuracies of tcp positions. Use the ur_calibration tool to extract the correct calibration from the robot and pass that into the description. See [https://github.com/UniversalRobots/Universal_Robots_ROS2_Driver/blob/main/ur_calibration/README.md] for details.
[ur_ros2_control_node-1] [INFO] [1772636398.804763408] [URPositionHardwareInterface]: System successfully started!
[ur_ros2_control_node-1] [INFO] [1772636398.804837683] [resource_manager]: Successful 'configure' of hardware 'ur16e'
[ur_ros2_control_node-1] [INFO] [1772636398.804903509] [resource_manager]: 'activate' hardware 'ur16e' 
[ur_ros2_control_node-1] [INFO] [1772636398.804912725] [URPositionHardwareInterface]: Activating HW interface
[ur_ros2_control_node-1] [INFO] [1772636398.804920053] [resource_manager]: Successful 'activate' of hardware 'ur16e'
[controller_stopper_node-4] [INFO] [1772636398.811228371] [Controller stopper]: Service available
[controller_stopper_node-4] [INFO] [1772636398.811352503] [Controller stopper]: Waiting for list controllers service to come up on controller_manager/list_controllers
[controller_stopper_node-4] [INFO] [1772636398.811424473] [Controller stopper]: Service available
[ur_ros2_control_node-1] [WARN] [1772636398.815202475] [controller_manager]: Could not enable FIFO RT scheduling policy
[ur_ros2_control_node-1] [WARN] [1772636398.815637176] [UR_Client_Library:]: Your system/user seems not to be setup for FIFO scheduling. We recommend using a lowlatency kernel with FIFO scheduling. See https://docs.universal-robots.com/Universal_Robots_ROS_Documentation/doc/ur_client_library/doc/real_time.html for details.
[ur_ros2_control_node-1] [INFO] [1772636398.819405513] [controller_manager]: Loading controller 'joint_state_broadcaster'
[spawner-7] [INFO] [1772636398.834313546] [spawner_joint_state_broadcaster]: Loaded joint_state_broadcaster
[ur_ros2_control_node-1] [INFO] [1772636398.847212750] [controller_manager]: Configuring controller 'joint_state_broadcaster'
[ur_ros2_control_node-1] [INFO] [1772636398.847429909] [joint_state_broadcaster]: 'joints' or 'interfaces' parameter is empty. All available state interfaces will be published
[spawner-7] [INFO] [1772636398.861487132] [spawner_joint_state_broadcaster]: Configured and activated joint_state_broadcaster
[ur_ros2_control_node-1] [INFO] [1772636398.864181005] [controller_manager]: Loading controller 'io_and_status_controller'
[ur_ros2_control_node-1] [INFO] [1772636398.880016458] [controller_manager]: Loading controller 'joint_trajectory_controller'
[spawner-7] [INFO] [1772636398.881223278] [spawner_joint_state_broadcaster]: Loaded io_and_status_controller
[ur_ros2_control_node-1] [WARN] [1772636398.890683787] [joint_trajectory_controller]: [Deprecated]: "allow_nonzero_velocity_at_trajectory_end" is set to true. The default behavior will change to false.
[ur_ros2_control_node-1] [INFO] [1772636398.892130998] [controller_manager]: Configuring controller 'io_and_status_controller'
[spawner-8] [INFO] [1772636398.893453086] [spawner_joint_trajectory_controller]: Loaded joint_trajectory_controller
[ur_ros2_control_node-1] [INFO] [1772636398.895304470] [controller_manager]: Configuring controller 'joint_trajectory_controller'
[ur_ros2_control_node-1] [INFO] [1772636398.895642784] [joint_trajectory_controller]: No specific joint names are used for command interfaces. Using 'joints' parameter.
[ur_ros2_control_node-1] [INFO] [1772636398.895706306] [joint_trajectory_controller]: Command interfaces are [position] and state interfaces are [position velocity].
[ur_ros2_control_node-1] [INFO] [1772636398.895837318] [joint_trajectory_controller]: Using 'splines' interpolation method.
[ur_ros2_control_node-1] [INFO] [1772636398.896804227] [joint_trajectory_controller]: Controller state will be published at 100.00 Hz.
[ur_ros2_control_node-1] [INFO] [1772636398.899622328] [joint_trajectory_controller]: Action status changes will be monitored at 20.00 Hz.
[spawner-7] [INFO] [1772636398.913886629] [spawner_joint_state_broadcaster]: Configured and activated io_and_status_controller
[ur_ros2_control_node-1] [INFO] [1772636398.915937795] [controller_manager]: Loading controller 'forward_velocity_controller'
[spawner-8] [INFO] [1772636398.931413717] [spawner_joint_trajectory_controller]: Loaded forward_velocity_controller
[ur_ros2_control_node-1] [INFO] [1772636398.931637020] [controller_manager]: Loading controller 'speed_scaling_state_broadcaster'
[ur_ros2_control_node-1] [INFO] [1772636398.940068089] [speed_scaling_state_broadcaster]: Loading UR SpeedScalingStateBroadcaster with tf_prefix: 
[ur_ros2_control_node-1] [INFO] [1772636398.942054421] [controller_manager]: Configuring controller 'forward_velocity_controller'
[spawner-7] [INFO] [1772636398.943328219] [spawner_joint_state_broadcaster]: Loaded speed_scaling_state_broadcaster
[ur_ros2_control_node-1] [INFO] [1772636398.943860555] [forward_velocity_controller]: configure successful
[ur_ros2_control_node-1] [INFO] [1772636398.946041069] [controller_manager]: Configuring controller 'speed_scaling_state_broadcaster'
[ur_ros2_control_node-1] [INFO] [1772636398.946162129] [speed_scaling_state_broadcaster]: Publisher rate set to : 100.0 Hz
[ur_ros2_control_node-1] [INFO] [1772636398.952052770] [controller_manager]: Loading controller 'forward_position_controller'
[spawner-7] [INFO] [1772636398.961529055] [spawner_joint_state_broadcaster]: Configured and activated speed_scaling_state_broadcaster
[ur_ros2_control_node-1] [INFO] [1772636398.965307537] [controller_manager]: Loading controller 'force_torque_sensor_broadcaster'
[spawner-8] [INFO] [1772636398.966565303] [spawner_joint_trajectory_controller]: Loaded forward_position_controller
[robot_state_helper-3] [INFO] [1772636398.973047674] [robot_state_helper]: The robot is currently in safety mode NORMAL.
[robot_state_helper-3] [INFO] [1772636398.973072123] [robot_state_helper]: The robot is currently in mode RUNNING.
[ur_ros2_control_node-1] [INFO] [1772636398.974159004] [controller_manager]: Configuring controller 'forward_position_controller'
[ur_ros2_control_node-1] [INFO] [1772636398.974863057] [forward_position_controller]: configure successful
[spawner-7] [INFO] [1772636398.977601187] [spawner_joint_state_broadcaster]: Loaded force_torque_sensor_broadcaster
[ur_ros2_control_node-1] [INFO] [1772636398.978089042] [controller_manager]: Configuring controller 'force_torque_sensor_broadcaster'
[ur_ros2_control_node-1] [INFO] [1772636398.980279252] [controller_manager]: Loading controller 'force_mode_controller'
[spawner-8] [INFO] [1772636398.993115158] [spawner_joint_trajectory_controller]: Loaded force_mode_controller
[ur_ros2_control_node-1] [INFO] [1772636398.996125489] [controller_manager]: Configuring controller 'force_mode_controller'
[spawner-7] [INFO] [1772636398.997643134] [spawner_joint_state_broadcaster]: Configured and activated force_torque_sensor_broadcaster
[ur_ros2_control_node-1] [INFO] [1772636399.004410250] [controller_manager]: Loading controller 'tcp_pose_broadcaster'
[ur_ros2_control_node-1] [INFO] [1772636399.015980071] [controller_manager]: Loading controller 'passthrough_trajectory_controller'
[spawner-7] [INFO] [1772636399.017307407] [spawner_joint_state_broadcaster]: Loaded tcp_pose_broadcaster
[ur_ros2_control_node-1] [INFO] [1772636399.023125726] [controller_manager]: Configuring controller 'tcp_pose_broadcaster'
[spawner-8] [INFO] [1772636399.023670382] [spawner_joint_trajectory_controller]: Loaded passthrough_trajectory_controller
[ur_ros2_control_node-1] [INFO] [1772636399.028388860] [controller_manager]: Configuring controller 'passthrough_trajectory_controller'
[spawner-7] [INFO] [1772636399.036621940] [spawner_joint_state_broadcaster]: Configured and activated tcp_pose_broadcaster
[ur_ros2_control_node-1] [INFO] [1772636399.038523437] [controller_manager]: Loading controller 'freedrive_mode_controller'
[ur_ros2_control_node-1] [INFO] [1772636399.052153992] [controller_manager]: Loading controller 'ur_configuration_controller'
[spawner-8] [INFO] [1772636399.053269353] [spawner_joint_trajectory_controller]: Loaded freedrive_mode_controller
[ur_ros2_control_node-1] [INFO] [1772636399.060888559] [controller_manager]: Configuring controller 'freedrive_mode_controller'
[ur_ros2_control_node-1] [INFO] [1772636399.063930378] [controller_manager]: Configuring controller 'ur_configuration_controller'
[spawner-7] [INFO] [1772636399.063205012] [spawner_joint_state_broadcaster]: Loaded ur_configuration_controller
[spawner-7] [INFO] [1772636399.070866171] [spawner_joint_state_broadcaster]: Configured and activated ur_configuration_controller
[ur_ros2_control_node-1] [INFO] [1772636399.074143998] [controller_manager]: Loading controller 'scaled_joint_trajectory_controller'
[ur_ros2_control_node-1] [INFO] [1772636399.087736631] [scaled_joint_trajectory_controller]: Using scaling state from the hardware from interface speed_scaling/speed_scaling_factor.
[ur_ros2_control_node-1] [WARN] [1772636399.094609638] [scaled_joint_trajectory_controller]: [Deprecated]: "allow_nonzero_velocity_at_trajectory_end" is set to true. The default behavior will change to false.
[spawner-7] [INFO] [1772636399.100786496] [spawner_joint_state_broadcaster]: Loaded scaled_joint_trajectory_controller
[ur_ros2_control_node-1] [INFO] [1772636399.102435218] [controller_manager]: Configuring controller 'scaled_joint_trajectory_controller'
[ur_ros2_control_node-1] [INFO] [1772636399.102737243] [scaled_joint_trajectory_controller]: No specific joint names are used for command interfaces. Using 'joints' parameter.
[ur_ros2_control_node-1] [INFO] [1772636399.102776284] [scaled_joint_trajectory_controller]: Command interfaces are [position] and state interfaces are [position velocity].
[ur_ros2_control_node-1] [INFO] [1772636399.102805085] [scaled_joint_trajectory_controller]: Using 'splines' interpolation method.
[ur_ros2_control_node-1] [INFO] [1772636399.103833116] [scaled_joint_trajectory_controller]: Controller state will be published at 100.00 Hz.
[ur_ros2_control_node-1] [INFO] [1772636399.107569868] [scaled_joint_trajectory_controller]: Action status changes will be monitored at 20.00 Hz.
[spawner-7] [INFO] [1772636399.119247404] [spawner_joint_state_broadcaster]: Configured and activated scaled_joint_trajectory_controller
[ur_ros2_control_node-1] [INFO] [1772636399.235336787] [controller_manager]: Switch controller timeout is set to 0, using default 1s!
[INFO] [spawner-8]: process has finished cleanly [pid 9582]
[INFO] [spawner-7]: process has finished cleanly [pid 9573]
[ur_ros2_control_node-1] [INFO] [1772636401.744290738] [UR_Client_Library:]: Robot requested program
[ur_ros2_control_node-1] [INFO] [1772636401.744572635] [UR_Client_Library:]: Sent program to robot
[ur_ros2_control_node-1] [INFO] [1772636402.053868159] [UR_Client_Library:]: Robot connected to reverse interface. Ready to receive control commands.
[ur_ros2_control_node-1] [INFO] [1772636539.459539537] [UR_Client_Library:]: Connection to reverse interface dropped.
[ur_ros2_control_node-1] [INFO] [1772636539.462785490] [UR_Client_Library:]: Robot connected to reverse interface. Ready to receive control commands.
[ur_ros2_control_node-1] [INFO] [1772636606.278484491] [UR_Client_Library:]: Connection to reverse interface dropped.
[ur_ros2_control_node-1] [INFO] [1772636606.281364916] [UR_Client_Library:]: Robot connected to reverse interface. Ready to receive control commands.
[ur_ros2_control_node-1] [INFO] [1772636636.985774243] [UR_Client_Library:]: Connection to reverse interface dropped.
[ur_ros2_control_node-1] [INFO] [1772636636.998266766] [UR_Client_Library:]: Robot connected to reverse interface. Ready to receive control commands.
[ur_ros2_control_node-1] [INFO] [1772636698.734441891] [UR_Client_Library:]: Connection to reverse interface dropped.
[ur_ros2_control_node-1] [INFO] [1772636698.738351797] [UR_Client_Library:]: Robot connected to reverse interface. Ready to receive control commands.
[ur_ros2_control_node-1] [INFO] [1772636729.499548579] [UR_Client_Library:]: Connection to reverse interface dropped.
[ur_ros2_control_node-1] [INFO] [1772636729.502618029] [UR_Client_Library:]: Robot connected to reverse interface. Ready to receive control commands.
[ur_ros2_control_node-1] [INFO] [1772636760.409426892] [UR_Client_Library:]: Connection to reverse interface dropped.
[ur_ros2_control_node-1] [INFO] [1772636760.413189330] [UR_Client_Library:]: Robot connected to reverse interface. Ready to receive control commands.
[ur_ros2_control_node-1] [INFO] [1772637395.349361605] [UR_Client_Library:]: Connection to reverse interface dropped.
[ur_ros2_control_node-1] [INFO] [1772637395.353530942] [UR_Client_Library:]: Robot connected to reverse interface. Ready to receive control commands.

Accept Public visibility

• I agree to make this context public

[urfeex]

I think, that could happen, when there is an unstable connection between the ROS machine and the robot and you have a looping program on the robot.

The connection is dropped due to the bad connection, the PolyScope program finishes and restarts, effectively connecting to the reverse interface again.

A "bad connection" in that context could be

• WiFi (I hope, that's not the case)
• Poorly performing USB to Ethernet adapter (a common problem these days where many laptops don't have a physical ethernet port anymore)
• Connection going over a shared network switch with a lot of traffic on it
• Too much load on the ROS machine causing the control loop there to slow down significantly (This could potentially also include the ROS driver running in a VM)
• The ROS machine doesn't have direct access to the physical network device (e.g. because it is running in a VM / container using a more complex networking layer)

Do you think that one of these points could be applicable to your setup?

I would expect, that you see something like this in your robot's log:

I took that screenshot of a PolyScope X robot, so it would look different for you visually, but the log messages are the same.

[Wingragon]

Thanks for the quick response. I will check all of those when I'm working on Wednesday. I also just noticed that I accidentally posted in the ROS and not the ROS2 driver forum. I think the most likely answer would be the high machine load or connection with a lot of traffic. The machine is a very weak NUC, and there are quite a lot of robots and other devices in the network. I will check the logs and maybe change the machine if I can get another one.

Regarding the containers. Is working with containers in Docker a significant problem for the ROS2 driver? We are interested in switching our development into a Docker-based system because the setup on the NUC is used by many people and isn't always left in the best shape.
Thanks for the help :).

Edit:
I just remembered another group working on the same system told me it was working for them when they were working in the late evening, when no one else was there, so an overloaded network seems more likely.

[urfeex]

I'll move this over to the ROS 2 driver.

Is working with containers in Docker a significant problem for the ROS2 driver?

Containers usually aren't a problem. Depending on the specific network setup you use, a virtual network layer can add some latency to my knowledge. I haven't found a use case where performance was really affecting the driver, though. Best performance can be achieved if you use the host network in the container directly.

[Wingragon]

I just checked the logs, and they look very similar to the examples you sent. Today, the connection also produced the response "failed to read from stream" quite frequently, so I will assume it's network problems, either related to the PCs ethernet being bad or network overload.
Thanks for the help so far :).

[Wingragon]

Small update, I tried isolating the network to only the robot and the nuc because we don't really need internet access while diagnosing, but found that today the driver instantly produces the error "cannot read from stream" with the exponential time delay :(.

[Wingragon]

Another small update: The error has evolved into very fast-paced connection drops and recoveries, and after some amount followed by Failed to read from stream, reconnecting in .. seconds. The Failed error then stays and refreshes, while the " Connection Dropped " error does not show up again.

[urfeex]

What ROS distribution are you running on? We've done a bigger refactoring around the RTDE connection. This should remove the "Failed to read from stream" error.

The change has landed in ROS Jazzy already, Humble and Kilted still need a repository sync to make it available.

[Wingragon]

Hi again, thanks for all your help. We are running on Humble, but we think we have fixed the issue after someone looking over the system pointed out that even with everything except for robot and PC connected, we still got the error, and replaced the switch, which was 10 years old at that point. Sorry for not mentioning it; it only happened two days ago. So far, we haven't had any issues.

[urfeex]

Thanks for letting me know. I'll go ahead and close this issue, then.