Skip to content

lookup/insert race condition in BufferCore #895

@almaslov

Description

@almaslov

Generated by Generative AI

No response

Operating System:

Linux dev-server 5.15.0-113-generic #123-Ubuntu SMP Mon Jun 10 08:16:17 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

ROS version or commit hash:

rolling, humble

RMW implementation (if applicable):

no rmw involved

RMW Configuration (if applicable):

no rmw involved

Client library (if applicable):

No response

'ros2 doctor --report' output

   ACTION LIST
action                 : none
action server count    : 0
action client count    : 0

   ROS ENVIRONMENT
ROS environment variables        : ROS_AUTOMATIC_DISCOVERY_RANGE=SUBNET, ROS_DISTRO=rolling
rcutils environment variables    :
rmw environment variables        :

   NETWORK CONFIGURATION
inet         : 192.168.1.111
inet4        : ['127.0.0.1', '192.168.1.111']
netmask      : 255.255.255.255
device       : lo
flags        : UP,LOOPBACK,RUNNING
mtu          : 65536

   PACKAGE VERSIONS
tf_test                                   : latest=N/A, local=0.0.0
tf2                                       : latest=0.45.6, local=0.45.6
action_msgs                               : latest=2.4.3, local=2.4.2
ament_cmake                               : latest=2.8.5, local=2.8.4
ament_cmake_auto                          : latest=2.8.5, local=2.8.4
ament_cmake_copyright                     : latest=0.20.3, local=0.20.3
ament_cmake_core                          : latest=2.8.5, local=2.8.4
ament_cmake_cppcheck                      : latest=0.20.3, local=0.20.3
ament_cmake_cpplint                       : latest=0.20.3, local=0.20.3
ament_cmake_export_definitions            : latest=2.8.5, local=2.8.4
ament_cmake_export_dependencies           : latest=2.8.5, local=2.8.4
ament_cmake_export_include_directories    : latest=2.8.5, local=2.8.4
ament_cmake_export_libraries              : latest=2.8.5, local=2.8.4
ament_cmake_export_link_flags             : latest=2.8.5, local=2.8.4
ament_cmake_export_targets                : latest=2.8.5, local=2.8.4
ament_cmake_flake8                        : latest=0.20.3, local=0.20.3
ament_cmake_gen_version_h                 : latest=2.8.5, local=2.8.4
ament_cmake_gmock                         : latest=2.8.5, local=2.8.4
ament_cmake_gtest                         : latest=2.8.5, local=2.8.4
ament_cmake_include_directories           : latest=2.8.5, local=2.8.4
ament_cmake_libraries                     : latest=2.8.5, local=2.8.4
ament_cmake_lint_cmake                    : latest=0.20.3, local=0.20.3
ament_cmake_mypy                          : latest=0.20.3, local=0.20.3
ament_cmake_pep257                        : latest=0.20.3, local=0.20.3
ament_cmake_pytest                        : latest=2.8.5, local=2.8.4
ament_cmake_python                        : latest=2.8.5, local=2.8.4
ament_cmake_ros                           : latest=0.15.6, local=0.15.4
ament_cmake_ros_core                      : latest=0.15.6, local=0.15.4
ament_cmake_target_dependencies           : latest=2.8.5, local=2.8.4
ament_cmake_test                          : latest=2.8.5, local=2.8.4
ament_cmake_uncrustify                    : latest=0.20.3, local=0.20.3
ament_cmake_version                       : latest=2.8.5, local=2.8.4
ament_cmake_xmllint                       : latest=0.20.3, local=0.20.3
ament_copyright                           : latest=0.20.3, local=0.20.3
ament_cppcheck                            : latest=0.20.3, local=0.20.3
ament_cpplint                             : latest=0.20.3, local=0.20.3
ament_flake8                              : latest=0.20.3, local=0.20.3
ament_index_cpp                           : latest=1.13.1, local=1.12.1
ament_index_python                        : latest=1.13.1, local=1.12.1
ament_lint                                : latest=0.20.3, local=0.20.3
ament_lint_auto                           : latest=0.20.3, local=0.20.3
ament_lint_cmake                          : latest=0.20.3, local=0.20.3
ament_lint_common                         : latest=0.20.3, local=0.20.3
ament_mypy                                : latest=0.20.3, local=0.20.3
ament_package                             : latest=0.18.2, local=0.18.1
ament_pep257                              : latest=0.20.3, local=0.20.3
ament_uncrustify                          : latest=0.20.3, local=0.20.3
ament_xmllint                             : latest=0.20.3, local=0.20.3
builtin_interfaces                        : latest=2.4.3, local=2.4.2
class_loader                              : latest=2.9.1, local=2.9.1
common_interfaces                         : latest=5.9.1, local=5.9.1
composition_interfaces                    : latest=2.4.3, local=2.4.2
console_bridge_vendor                     : latest=1.9.1, local=1.9.1
diagnostic_msgs                           : latest=5.9.1, local=5.9.1
geometry_msgs                             : latest=5.9.1, local=5.9.1
launch                                    : latest=3.9.6, local=3.9.5
launch_ros                                : latest=0.29.6, local=0.29.4
launch_testing                            : latest=3.9.6, local=3.9.5
launch_testing_ament_cmake                : latest=3.9.6, local=3.9.5
launch_testing_ros                        : latest=0.29.6, local=0.29.4
launch_xml                                : latest=3.9.6, local=3.9.5
launch_yaml                               : latest=3.9.6, local=3.9.5
libstatistics_collector                   : latest=2.1.1, local=2.1.1
libyaml_vendor                            : latest=1.8.0, local=1.8.0
lifecycle_msgs                            : latest=2.4.3, local=2.4.2
nav_msgs                                  : latest=5.9.1, local=5.9.1
pluginlib                                 : latest=5.8.2, local=5.8.0
rcl                                       : latest=10.3.1, local=10.2.6
rcl_action                                : latest=10.3.1, local=10.2.6
rcl_interfaces                            : latest=2.4.3, local=2.4.2
rcl_lifecycle                             : latest=10.3.1, local=10.2.6
rcl_logging_interface                     : latest=3.4.0, local=3.3.2
rcl_logging_spdlog                        : latest=3.4.0, local=3.3.2
rcl_yaml_param_parser                     : latest=10.3.1, local=10.2.6
rclcpp                                    : latest=30.1.5, local=30.1.3
rclcpp_action                             : latest=30.1.5, local=30.1.3
rclcpp_components                         : latest=30.1.5, local=30.1.3
rclcpp_lifecycle                          : latest=30.1.5, local=30.1.3
rclpy                                     : latest=10.0.5, local=10.0.2
rcpputils                                 : latest=2.14.3, local=2.14.3
rcutils                                   : latest=7.0.5, local=7.0.4
rmw                                       : latest=7.9.1, local=7.9.1
rmw_dds_common                            : latest=6.0.0, local=4.0.2
rmw_fastrtps_cpp                          : latest=9.4.4, local=9.4.4
rmw_fastrtps_shared_cpp                   : latest=9.4.4, local=9.4.4
rmw_implementation                        : latest=3.1.4, local=3.1.3
rmw_implementation_cmake                  : latest=7.9.1, local=7.9.1
rmw_security_common                       : latest=7.9.1, local=7.9.1
rmw_test_fixture                          : latest=0.15.6, local=0.15.4
rmw_test_fixture_implementation           : latest=0.15.6, local=0.15.4
rmw_zenoh_cpp                             : latest=0.10.2, local=0.10.2
ros2action                                : latest=0.40.4, local=0.40.2
ros2cli                                   : latest=0.40.4, local=0.40.2
ros2cli_common_extensions                 : latest=0.5.2, local=0.5.1
ros2component                             : latest=0.40.4, local=0.40.2
ros2doctor                                : latest=0.40.4, local=0.40.2
ros2interface                             : latest=0.40.4, local=0.40.2
ros2launch                                : latest=0.29.6, local=0.29.4
ros2lifecycle                             : latest=0.40.4, local=0.40.2
ros2multicast                             : latest=0.40.4, local=0.40.2
ros2node                                  : latest=0.40.4, local=0.40.2
ros2param                                 : latest=0.40.4, local=0.40.2
ros2pkg                                   : latest=0.40.4, local=0.40.2
ros2run                                   : latest=0.40.4, local=0.40.2
ros2service                               : latest=0.40.4, local=0.40.2
ros2topic                                 : latest=0.40.4, local=0.40.2
ros_core                                  : latest=0.13.0, local=0.13.0
ros_environment                           : latest=4.4.1, local=4.4.1
ros_workspace                             : latest=1.0.3, local=1.0.3
rosgraph_msgs                             : latest=2.4.3, local=2.4.2
rosidl_adapter                            : latest=5.1.2, local=5.1.1
rosidl_cli                                : latest=5.1.2, local=5.1.1
rosidl_cmake                              : latest=5.1.2, local=5.1.1
rosidl_core_generators                    : latest=0.4.1, local=0.4.1
rosidl_core_runtime                       : latest=0.4.1, local=0.4.1
rosidl_default_generators                 : latest=1.8.1, local=1.8.1
rosidl_default_runtime                    : latest=1.8.1, local=1.8.1
rosidl_dynamic_typesupport                : latest=0.4.0, local=0.4.0
rosidl_dynamic_typesupport_fastrtps       : latest=0.5.1, local=0.5.1
rosidl_generator_c                        : latest=5.1.2, local=5.1.1
rosidl_generator_cpp                      : latest=5.1.2, local=5.1.1
rosidl_generator_py                       : latest=0.26.3, local=0.26.2
rosidl_generator_rs                       : latest=0.4.10, local=0.4.9
rosidl_generator_type_description         : latest=5.1.2, local=5.1.1
rosidl_parser                             : latest=5.1.2, local=5.1.1
rosidl_pycommon                           : latest=5.1.2, local=5.1.1
rosidl_runtime_c                          : latest=5.1.2, local=5.1.1
rosidl_runtime_cpp                        : latest=5.1.2, local=5.1.1
rosidl_runtime_py                         : latest=0.15.1, local=0.15.1
rosidl_typesupport_c                      : latest=3.4.1, local=3.4.1
rosidl_typesupport_cpp                    : latest=3.4.1, local=3.4.1
rosidl_typesupport_fastrtps_c             : latest=3.9.3, local=3.9.3
rosidl_typesupport_fastrtps_cpp           : latest=3.9.3, local=3.9.3
rosidl_typesupport_interface              : latest=5.1.2, local=5.1.1
rosidl_typesupport_introspection_c        : latest=5.1.2, local=5.1.1
rosidl_typesupport_introspection_cpp      : latest=5.1.2, local=5.1.1
rpyutils                                  : latest=0.7.1, local=0.7.1
sensor_msgs                               : latest=5.9.1, local=5.9.1
service_msgs                              : latest=2.4.3, local=2.4.2
shape_msgs                                : latest=5.9.1, local=5.9.1
spdlog_vendor                             : latest=1.8.0, local=1.8.0
sros2                                     : latest=0.16.3, local=0.16.2
sros2_cmake                               : latest=0.16.3, local=0.16.2
statistics_msgs                           : latest=2.4.3, local=2.4.2
std_msgs                                  : latest=5.9.1, local=5.9.1
std_srvs                                  : latest=5.9.1, local=5.9.1
stereo_msgs                               : latest=5.9.1, local=5.9.1
tracetools                                : latest=8.9.0, local=8.9.0
trajectory_msgs                           : latest=5.9.1, local=5.9.1
type_description_interfaces               : latest=2.4.3, local=2.4.2
uncrustify_vendor                         : latest=3.2.0, local=3.2.0
unique_identifier_msgs                    : latest=2.8.1, local=2.8.1
visualization_msgs                        : latest=5.9.1, local=5.9.1
zenoh_cpp_vendor                          : latest=0.10.2, local=0.10.2
eigen3_cmake_module                       : latest=0.5.1, local=0.5.1
geometry2                                 : latest=0.45.6, local=0.45.3
kdl_parser                                : latest=3.0.1, local=3.0.1
keyboard_handler                          : latest=0.5.0, local=0.5.0
liblz4_vendor                             : latest=0.33.0, local=0.33.0
mcap_vendor                               : latest=0.33.0, local=0.33.0
message_filters                           : latest=7.3.6, local=7.3.4
pybind11_vendor                           : latest=3.3.1, local=3.3.1
robot_state_publisher                     : latest=3.5.4, local=3.5.3
ros2bag                                   : latest=0.33.0, local=0.33.0
ros_base                                  : latest=0.13.0, local=0.13.0
rosbag2                                   : latest=0.33.0, local=0.33.0
rosbag2_compression                       : latest=0.33.0, local=0.33.0
rosbag2_compression_zstd                  : latest=0.33.0, local=0.33.0
rosbag2_cpp                               : latest=0.33.0, local=0.33.0
rosbag2_interfaces                        : latest=0.33.0, local=0.33.0
rosbag2_py                                : latest=0.33.0, local=0.33.0
rosbag2_storage                           : latest=0.33.0, local=0.33.0
rosbag2_storage_default_plugins           : latest=0.33.0, local=0.33.0
rosbag2_storage_mcap                      : latest=0.33.0, local=0.33.0
rosbag2_storage_sqlite3                   : latest=0.33.0, local=0.33.0
rosbag2_transport                         : latest=0.33.0, local=0.33.0
sensor_msgs_py                            : latest=5.9.1, local=5.9.1
sqlite3_vendor                            : latest=0.33.0, local=0.33.0
tf2_bullet                                : latest=0.45.6, local=0.45.3
tf2_eigen                                 : latest=0.45.6, local=0.45.3
tf2_eigen_kdl                             : latest=0.45.6, local=0.45.3
tf2_geometry_msgs                         : latest=0.45.6, local=0.45.3
tf2_kdl                                   : latest=0.45.6, local=0.45.3
tf2_msgs                                  : latest=0.45.6, local=0.45.3
tf2_py                                    : latest=0.45.6, local=0.45.3
tf2_ros                                   : latest=0.45.6, local=0.45.3
tf2_ros_py                                : latest=0.45.6, local=0.45.3
tf2_sensor_msgs                           : latest=0.45.6, local=0.45.3
tf2_tools                                 : latest=0.45.6, local=0.45.3
tinyxml2_vendor                           : latest=0.11.3, local=0.11.3
urdf                                      : latest=2.13.1, local=2.13.0
urdf_parser_plugin                        : latest=2.13.1, local=2.13.0
yaml_cpp_vendor                           : latest=9.2.0, local=9.2.0
zstd_vendor                               : latest=0.33.0, local=0.33.0

   PLATFORM INFORMATION
system           : Linux
platform info    : Linux-5.15.0-113-generic-x86_64-with-glibc2.39
release          : 5.15.0-113-generic
processor        : x86_64

   QOS COMPATIBILITY LIST
compatibility status    : No publisher/subscriber pairs found

   RMW MIDDLEWARE
middleware name    : rmw_fastrtps_cpp

   ROS 2 INFORMATION
distribution name      : rolling
distribution type      : ros2
distribution status    : rolling
release platforms      : {'debian': ['bookworm'], 'rhel': ['9'], 'ubuntu': ['noble']}

   SERVICE LIST
service          : none
service count    : 0
client count     : 0

   TOPIC LIST
topic               : none
publisher count     : 0
subscriber count    : 0

Steps to reproduce issue

  1. Run rolling docker image:
docker run -it --rm --net=host -w /wd ros:rolling bash
  1. Checkout tf2 package source:
git clone https://github.com/ros2/geometry2.git /wd/src/geometry2
  1. Create package with test executable:
ros2 pkg create --node-name tf_test --dependencies tf2 --build-type ament_cmake --destination-directory /wd/src tf_test
  1. Copy source code for executable from below: cat >/wd/src/tf_test/src/tf_test.cpp
  2. Enable sanitizers and build:
export LDFLAGS="-fsanitize=thread -fno-omit-frame-pointer"
export CXXFLAGS="-fsanitize=thread -fno-omit-frame-pointer"
colcon build --cmake-args -DCMAKE_BUILD_TYPE=Debug -DBUILD_TESTING=OFF --packages-select tf_test tf2
  1. Activate environment and run test:
source /wd/install/setup.sh
ros2 run tf_test tf_test

Test executable source code:

#include <chrono>
#include <iostream>
#include <thread>

#include <geometry_msgs/msg/transform_stamped.hpp>
#include <tf2/buffer_core.hpp>


using Buffer = tf2::BufferCore;


geometry_msgs::msg::TransformStamped makeTransform(
    const std::string& frame_id, const std::string& child_frame_id) {

  geometry_msgs::msg::TransformStamped transform;
  auto nanosec = std::chrono::system_clock::now().time_since_epoch().count();

  transform.header.stamp.sec = nanosec / 1000000000;
  transform.header.stamp.nanosec = nanosec % 1000000000;
  transform.header.frame_id = frame_id;
  transform.child_frame_id = child_frame_id;

  transform.transform.translation.x = 1.0;
  transform.transform.translation.y = 0.0;
  transform.transform.translation.z = 0.0;
  transform.transform.rotation.w = 1.0;

  return transform;
}

int main() {
  Buffer buffer;

  std::thread writer([&buffer]() {
      for (size_t i = 0; ; ++i) {
          auto transform = makeTransform("base", "frame_" + std::to_string(i));
          buffer.setTransform(transform, "authority");
      }
  });

  std::thread reader([&buffer]() {
    while (true) {
      buffer.canTransform("base", "frame_0", {});
    }
  });

  reader.join();
  writer.join();
  return 0;
}

Expected behavior

No data race detected

Actual behavior

Data race detected. Here is example of output:

WARNING: ThreadSanitizer: data race (pid=298)
  Write of size 8 at 0x721000025dc0 by thread T1 (mutexes: write M0):
    #0 std::_Hashtable<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, unsigned int>, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, unsigned int> >, std::__detail::_Select1st, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, false, true> >::_M_rehash_aux(unsigned long, std::integral_constant<bool, true>) /usr/include/c++/13/bits/hashtable.h:2606 (libtf2.so+0x71dcb) (BuildId: 83c73bae7e5ee3c9775f56776b87d01b3617d0fd)
    #1 std::_Hashtable<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, unsigned int>, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, unsigned int> >, std::__detail::_Select1st, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, false, true> >::_M_rehash(unsigned long, unsigned long const&) /usr/include/c++/13/bits/hashtable.h:2565 (libtf2.so+0x6e815) (BuildId: 83c73bae7e5ee3c9775f56776b87d01b3617d0fd)
    #2 std::_Hashtable<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, unsigned int>, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, unsigned int> >, std::__detail::_Select1st, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, false, true> >::_M_insert_unique_node(unsigned long, unsigned long, std::__detail::_Hash_node<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, unsigned int>, true>*, unsigned long) /usr/include/c++/13/bits/hashtable.h:2179 (libtf2.so+0x699d4) (BuildId: 83c73bae7e5ee3c9775f56776b87d01b3617d0fd)
    #3 std::__detail::_Map_base<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, unsigned int>, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, unsigned int> >, std::__detail::_Select1st, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, false, true>, true>::operator[](std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) /usr/include/c++/13/bits/hashtable_policy.h:825 (libtf2.so+0x66eb0) (BuildId: 83c73bae7e5ee3c9775f56776b87d01b3617d0fd)
    #4 std::unordered_map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, unsigned int, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, unsigned int> > >::operator[](std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) /usr/include/c++/13/bits/unordered_map.h:987 (libtf2.so+0x62d91) (BuildId: 83c73bae7e5ee3c9775f56776b87d01b3617d0fd)
    #5 tf2::BufferCore::lookupOrInsertFrameNumber(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) /wd/src/geometry2/tf2/src/buffer_core.cpp:1002 (libtf2.so+0x51b39) (BuildId: 83c73bae7e5ee3c9775f56776b87d01b3617d0fd)
    #6 tf2::BufferCore::setTransformImpl(tf2::Vector3 const&, tf2::Quaternion const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::chrono::time_point<std::chrono::_V2::system_clock, std::chrono::duration<long, std::ratio<1l, 1000000000l> > >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool) /wd/src/geometry2/tf2/src/buffer_core.cpp:273 (libtf2.so+0x4e723) (BuildId: 83c73bae7e5ee3c9775f56776b87d01b3617d0fd)
    #7 tf2::BufferCore::setTransform(geometry_msgs::msg::TransformStamped_<std::allocator<void> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool) /wd/src/geometry2/tf2/src/buffer_core.cpp:202 (libtf2.so+0x4d81c) (BuildId: 83c73bae7e5ee3c9775f56776b87d01b3617d0fd)
    #8 operator() /wd/src/tf_test/src/tf_test.cpp:37 (tf_test+0x39cf) (BuildId: d6f00023015426732b378134a805ffe10cc5b266)
    #9 __invoke_impl<void, main()::<lambda()> > /usr/include/c++/13/bits/invoke.h:61 (tf_test+0x491a) (BuildId: d6f00023015426732b378134a805ffe10cc5b266)
    #10 __invoke<main()::<lambda()> > /usr/include/c++/13/bits/invoke.h:96 (tf_test+0x4808) (BuildId: d6f00023015426732b378134a805ffe10cc5b266)
    #11 _M_invoke<0> /usr/include/c++/13/bits/std_thread.h:292 (tf_test+0x46ca) (BuildId: d6f00023015426732b378134a805ffe10cc5b266)
    #12 operator() /usr/include/c++/13/bits/std_thread.h:299 (tf_test+0x4616) (BuildId: d6f00023015426732b378134a805ffe10cc5b266)
    #13 _M_run /usr/include/c++/13/bits/std_thread.h:244 (tf_test+0x4582) (BuildId: d6f00023015426732b378134a805ffe10cc5b266)
    #14 <null> <null> (libstdc++.so.6+0xecdb3) (BuildId: ca77dae775ec87540acd7218fa990c40d1c94ab1)

  Previous read of size 8 at 0x721000025dc0 by thread T2:
    #0 std::_Hashtable<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, unsigned int>, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, unsigned int> >, std::__detail::_Select1st, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, false, true> >::_M_find_before_node(unsigned long, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, unsigned long) const /usr/include/c++/13/bits/hashtable.h:1958 (libtf2.so+0x6e399) (BuildId: 83c73bae7e5ee3c9775f56776b87d01b3617d0fd)
    #1 std::_Hashtable<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, unsigned int>, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, unsigned int> >, std::__detail::_Select1st, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, false, true> >::_M_find_node(unsigned long, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, unsigned long) const /usr/include/c++/13/bits/hashtable.h:815 (libtf2.so+0x69716) (BuildId: 83c73bae7e5ee3c9775f56776b87d01b3617d0fd)
    #2 std::_Hashtable<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, unsigned int>, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, unsigned int> >, std::__detail::_Select1st, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, false, true> >::find(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) const /usr/include/c++/13/bits/hashtable.h:1702 (libtf2.so+0x66aa1) (BuildId: 83c73bae7e5ee3c9775f56776b87d01b3617d0fd)
    #3 std::unordered_map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, unsigned int, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, unsigned int> > >::find(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) const /usr/include/c++/13/bits/unordered_map.h:887 (libtf2.so+0x62ba7) (BuildId: 83c73bae7e5ee3c9775f56776b87d01b3617d0fd)
    #4 tf2::BufferCore::lookupFrameNumber(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) const /wd/src/geometry2/tf2/src/buffer_core.cpp:985 (libtf2.so+0x519ae) (BuildId: 83c73bae7e5ee3c9775f56776b87d01b3617d0fd)
    #5 tf2::BufferCore::validateFrameId(char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*) const /wd/src/geometry2/tf2/src/buffer_core.cpp:124 (libtf2.so+0x4c9fb) (BuildId: 83c73bae7e5ee3c9775f56776b87d01b3617d0fd)
    #6 tf2::BufferCore::canTransform(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::chrono::time_point<std::chrono::_V2::system_clock, std::chrono::duration<long, std::ratio<1l, 1000000000l> > > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*) const /wd/src/geometry2/tf2/src/buffer_core.cpp:938 (libtf2.so+0x51760) (BuildId: 83c73bae7e5ee3c9775f56776b87d01b3617d0fd)
    #7 operator() /wd/src/tf_test/src/tf_test.cpp:43 (tf_test+0x3bc7) (BuildId: d6f00023015426732b378134a805ffe10cc5b266)
    #8 __invoke_impl<void, main()::<lambda()> > /usr/include/c++/13/bits/invoke.h:61 (tf_test+0x4891) (BuildId: d6f00023015426732b378134a805ffe10cc5b266)
    #9 __invoke<main()::<lambda()> > /usr/include/c++/13/bits/invoke.h:96 (tf_test+0x4769) (BuildId: d6f00023015426732b378134a805ffe10cc5b266)
    #10 _M_invoke<0> /usr/include/c++/13/bits/std_thread.h:292 (tf_test+0x4670) (BuildId: d6f00023015426732b378134a805ffe10cc5b266)
    #11 operator() /usr/include/c++/13/bits/std_thread.h:299 (tf_test+0x45cc) (BuildId: d6f00023015426732b378134a805ffe10cc5b266)
    #12 _M_run /usr/include/c++/13/bits/std_thread.h:244 (tf_test+0x4530) (BuildId: d6f00023015426732b378134a805ffe10cc5b266)
    #13 <null> <null> (libstdc++.so.6+0xecdb3) (BuildId: ca77dae775ec87540acd7218fa990c40d1c94ab1)

  Location is heap block of size 56 at 0x721000025dc0 allocated by thread T1:
    #0 operator new(unsigned long) ../../../../src/libsanitizer/tsan/tsan_new_delete.cpp:64 (libtsan.so.2+0xa6fae) (BuildId: 38097064631f7912bd33117a9c83d08b42e15571)
    #1 std::__new_allocator<std::__detail::_Hash_node<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, unsigned int>, true> >::allocate(unsigned long, void const*) /usr/include/c++/13/bits/new_allocator.h:151 (libtf2.so+0x7487e) (BuildId: 83c73bae7e5ee3c9775f56776b87d01b3617d0fd)
    #2 std::allocator_traits<std::allocator<std::__detail::_Hash_node<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, unsigned int>, true> > >::allocate(std::allocator<std::__detail::_Hash_node<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, unsigned int>, true> >&, unsigned long) /usr/include/c++/13/bits/alloc_traits.h:482 (libtf2.so+0x70057) (BuildId: 83c73bae7e5ee3c9775f56776b87d01b3617d0fd)
    #3 std::__detail::_Hash_node<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, unsigned int>, true>* std::__detail::_Hashtable_alloc<std::allocator<std::__detail::_Hash_node<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, unsigned int>, true> > >::_M_allocate_node<std::piecewise_construct_t const&, std::tuple<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&>, std::tuple<> >(std::piecewise_construct_t const&, std::tuple<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&>&&, std::tuple<>&&) /usr/include/c++/13/bits/hashtable_policy.h:1990 (libtf2.so+0x70057)
    #4 std::_Hashtable<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, unsigned int>, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, unsigned int> >, std::__detail::_Select1st, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, false, true> >::_Scoped_node::_Scoped_node<std::piecewise_construct_t const&, std::tuple<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&>, std::tuple<> >(std::__detail::_Hashtable_alloc<std::allocator<std::__detail::_Hash_node<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, unsigned int>, true> > >*, std::piecewise_construct_t const&, std::tuple<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&>&&, std::tuple<>&&) /usr/include/c++/13/bits/hashtable.h:307 (libtf2.so+0x6b690) (BuildId: 83c73bae7e5ee3c9775f56776b87d01b3617d0fd)
    #5 std::__detail::_Map_base<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, unsigned int>, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, unsigned int> >, std::__detail::_Select1st, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, false, true>, true>::operator[](std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) /usr/include/c++/13/bits/hashtable_policy.h:818 (libtf2.so+0x66e82) (BuildId: 83c73bae7e5ee3c9775f56776b87d01b3617d0fd)
    #6 std::unordered_map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, unsigned int, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, unsigned int> > >::operator[](std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) /usr/include/c++/13/bits/unordered_map.h:987 (libtf2.so+0x62d91) (BuildId: 83c73bae7e5ee3c9775f56776b87d01b3617d0fd)
    #7 tf2::BufferCore::lookupOrInsertFrameNumber(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) /wd/src/geometry2/tf2/src/buffer_core.cpp:1002 (libtf2.so+0x51b39) (BuildId: 83c73bae7e5ee3c9775f56776b87d01b3617d0fd)
    #8 tf2::BufferCore::setTransformImpl(tf2::Vector3 const&, tf2::Quaternion const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::chrono::time_point<std::chrono::_V2::system_clock, std::chrono::duration<long, std::ratio<1l, 1000000000l> > >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool) /wd/src/geometry2/tf2/src/buffer_core.cpp:273 (libtf2.so+0x4e723) (BuildId: 83c73bae7e5ee3c9775f56776b87d01b3617d0fd)
    #9 tf2::BufferCore::setTransform(geometry_msgs::msg::TransformStamped_<std::allocator<void> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool) /wd/src/geometry2/tf2/src/buffer_core.cpp:202 (libtf2.so+0x4d81c) (BuildId: 83c73bae7e5ee3c9775f56776b87d01b3617d0fd)
    #10 operator() /wd/src/tf_test/src/tf_test.cpp:37 (tf_test+0x39cf) (BuildId: d6f00023015426732b378134a805ffe10cc5b266)
    #11 __invoke_impl<void, main()::<lambda()> > /usr/include/c++/13/bits/invoke.h:61 (tf_test+0x491a) (BuildId: d6f00023015426732b378134a805ffe10cc5b266)
    #12 __invoke<main()::<lambda()> > /usr/include/c++/13/bits/invoke.h:96 (tf_test+0x4808) (BuildId: d6f00023015426732b378134a805ffe10cc5b266)
    #13 _M_invoke<0> /usr/include/c++/13/bits/std_thread.h:292 (tf_test+0x46ca) (BuildId: d6f00023015426732b378134a805ffe10cc5b266)
    #14 operator() /usr/include/c++/13/bits/std_thread.h:299 (tf_test+0x4616) (BuildId: d6f00023015426732b378134a805ffe10cc5b266)
    #15 _M_run /usr/include/c++/13/bits/std_thread.h:244 (tf_test+0x4582) (BuildId: d6f00023015426732b378134a805ffe10cc5b266)
    #16 <null> <null> (libstdc++.so.6+0xecdb3) (BuildId: ca77dae775ec87540acd7218fa990c40d1c94ab1)

  Mutex M0 (0x7ffd62933800) created at:
    #0 pthread_mutex_lock ../../../../src/libsanitizer/tsan/tsan_interceptors_posix.cpp:1341 (libtsan.so.2+0x59a13) (BuildId: 38097064631f7912bd33117a9c83d08b42e15571)
    #1 __gthread_mutex_lock(pthread_mutex_t*) <null> (libtf2.so+0x57f02) (BuildId: 83c73bae7e5ee3c9775f56776b87d01b3617d0fd)
    #2 std::mutex::lock() <null> (libtf2.so+0x58426) (BuildId: 83c73bae7e5ee3c9775f56776b87d01b3617d0fd)
    #3 std::unique_lock<std::mutex>::lock() /usr/include/c++/13/bits/unique_lock.h:141 (libtf2.so+0x650bb) (BuildId: 83c73bae7e5ee3c9775f56776b87d01b3617d0fd)
    #4 std::unique_lock<std::mutex>::unique_lock(std::mutex&) <null> (libtf2.so+0x5f8fe) (BuildId: 83c73bae7e5ee3c9775f56776b87d01b3617d0fd)
    #5 tf2::BufferCore::setTransformImpl(tf2::Vector3 const&, tf2::Quaternion const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::chrono::time_point<std::chrono::_V2::system_clock, std::chrono::duration<long, std::ratio<1l, 1000000000l> > >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool) /wd/src/geometry2/tf2/src/buffer_core.cpp:272 (libtf2.so+0x4e70a) (BuildId: 83c73bae7e5ee3c9775f56776b87d01b3617d0fd)
    #6 tf2::BufferCore::setTransform(geometry_msgs::msg::TransformStamped_<std::allocator<void> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool) /wd/src/geometry2/tf2/src/buffer_core.cpp:202 (libtf2.so+0x4d81c) (BuildId: 83c73bae7e5ee3c9775f56776b87d01b3617d0fd)
    #7 operator() /wd/src/tf_test/src/tf_test.cpp:37 (tf_test+0x39cf) (BuildId: d6f00023015426732b378134a805ffe10cc5b266)
    #8 __invoke_impl<void, main()::<lambda()> > /usr/include/c++/13/bits/invoke.h:61 (tf_test+0x491a) (BuildId: d6f00023015426732b378134a805ffe10cc5b266)
    #9 __invoke<main()::<lambda()> > /usr/include/c++/13/bits/invoke.h:96 (tf_test+0x4808) (BuildId: d6f00023015426732b378134a805ffe10cc5b266)
    #10 _M_invoke<0> /usr/include/c++/13/bits/std_thread.h:292 (tf_test+0x46ca) (BuildId: d6f00023015426732b378134a805ffe10cc5b266)
    #11 operator() /usr/include/c++/13/bits/std_thread.h:299 (tf_test+0x4616) (BuildId: d6f00023015426732b378134a805ffe10cc5b266)
    #12 _M_run /usr/include/c++/13/bits/std_thread.h:244 (tf_test+0x4582) (BuildId: d6f00023015426732b378134a805ffe10cc5b266)
    #13 <null> <null> (libstdc++.so.6+0xecdb3) (BuildId: ca77dae775ec87540acd7218fa990c40d1c94ab1)

  Thread T1 (tid=300, running) created by main thread at:
    #0 pthread_create ../../../../src/libsanitizer/tsan/tsan_interceptors_posix.cpp:1022 (libtsan.so.2+0x5ac1a) (BuildId: 38097064631f7912bd33117a9c83d08b42e15571)
    #1 std::thread::_M_start_thread(std::unique_ptr<std::thread::_State, std::default_delete<std::thread::_State> >, void (*)()) <null> (libstdc++.so.6+0xeceb0) (BuildId: ca77dae775ec87540acd7218fa990c40d1c94ab1)
    #2 main /wd/src/tf_test/src/tf_test.cpp:39 (tf_test+0x3cea) (BuildId: d6f00023015426732b378134a805ffe10cc5b266)

  Thread T2 (tid=301, running) created by main thread at:
    #0 pthread_create ../../../../src/libsanitizer/tsan/tsan_interceptors_posix.cpp:1022 (libtsan.so.2+0x5ac1a) (BuildId: 38097064631f7912bd33117a9c83d08b42e15571)
    #1 std::thread::_M_start_thread(std::unique_ptr<std::thread::_State, std::default_delete<std::thread::_State> >, void (*)()) <null> (libstdc++.so.6+0xeceb0) (BuildId: ca77dae775ec87540acd7218fa990c40d1c94ab1)
    #2 main /wd/src/tf_test/src/tf_test.cpp:45 (tf_test+0x3d20) (BuildId: d6f00023015426732b378134a805ffe10cc5b266)

SUMMARY: ThreadSanitizer: data race /usr/include/c++/13/bits/hashtable.h:2606 in std::_Hashtable<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, unsigned int>, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, unsigned int> >, std::__detail::_Select1st, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, false, true> >::_M_rehash_aux(unsigned long, std::integral_constant<bool, true>)

Additional information

The problem found in humble and reproduced in rolling.

Test executable inserts transforms with new frame_id in writer thread. Also it checks single transform in reader thread infinitely.
Looks like writer thread behaviour is quite rare in production systems. But still the problem causes segfault in our test cases occasionally.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions