Invalid read with Rice 4.11.0+

[ankane]

Hi, Valgrind is reporting an invalid read on garbage collection starting with Rice 4.11.0. I'm seeing this on CI with Numo.hpp.

I'm still trying to put together a minimal reproducible example, but wanted to see if you had any ideas.

Invalid read of size 8
  rb_vm_mark (vm.c:3311)
  rb_gc_mark_roots (gc.c:3030)
  mark_roots (default.c:4629)
  gc_marks_start (default.c:5815)
  gc_marks (default.c:5831)
  gc_start (default.c:6516)
  garbage_collect (default.c:6398)
  rb_gc_impl_start (default.c:6848)
  gc_start_internal (gc.c:3511)
  invoke_bf (vm_insnhelper.c:7538)
  vm_invoke_builtin (vm_insnhelper.c:7546)
  vm_exec_core (insns.def:1654)
  vm_exec_loop (vm.c:2825)
  rb_vm_exec (vm.c:2801)
  rb_vm_invoke_proc (vm.c:1965)
  rb_proc_call_kw (proc.c:1132)
  exec_end_procs_chain (eval_jump.c:105)
  rb_ec_exec_end_proc (eval_jump.c:121)
  rb_ec_teardown (eval.c:157)
  rb_ec_cleanup (eval.c:209)
  ruby_run_node (eval.c:321)
  rb_main (main.c:42)
  main (main.c:62)

[cfis]

So not a crash, just Valigrind being unhappy?

There were a lot of GC fixes in 4.11.0. I ran the test suite over and over with GC.stress on hunting down GC issues. Starting with that release, the Gitlab test runners (when using extconf.rb) enable GC stress.

You could try running with GC.stress=true and see what happens.

[ankane]

Yeah, no crash. Got a more detailed Valgrind trace with GC.stress.

Invalid read of size 8
  rb_vm_mark (vm.c:3311)
  rb_gc_mark_roots (gc.c:3030)
  mark_roots (default.c:4629)
  gc_marks_start (default.c:5815)
  gc_marks (default.c:5831)
  gc_start (default.c:6516)
  garbage_collect (default.c:6398)
  garbage_collect_with_gvl (default.c:6796)
  objspace_malloc_gc_stress (default.c:8039)
  objspace_malloc_gc_stress (default.c:8030)
  rb_gc_impl_malloc (default.c:8294)
  ruby_xmalloc_body (gc.c:5191)
  ruby_xmalloc (gc.c:5173)
  rb_gc_register_address (gc.c:3462)
 *void std::__invoke_impl<void, void (* const&)(unsigned long*), unsigned long*>(std::__invoke_other, void (* const&)(unsigned long*), unsigned long*&&) (at /home/ubuntu/code/lib/hello.so)
 *std::__invoke_result<void (* const&)(unsigned long*), unsigned long*>::type std::__invoke<void (* const&)(unsigned long*), unsigned long*>(void (* const&)(unsigned long*), unsigned long*&&) (at /home/ubuntu/code/lib/hello.so)
 *decltype(auto) std::__apply_impl<void (* const&)(unsigned long*), std::tuple<unsigned long*>, 0ul>(void (* const&)(unsigned long*), std::tuple<unsigned long*>&&, std::integer_sequence<unsigned long, 0ul>) (at /home/ubuntu/code/lib/hello.so)
 *decltype(auto) std::apply<void (* const&)(unsigned long*), std::tuple<unsigned long*> >(void (* const&)(unsigned long*), std::tuple<unsigned long*>&&) (at /home/ubuntu/code/lib/hello.so)
 *Rice::detail::NativeInvoker<void, void (*)(unsigned long*), std::tuple<unsigned long*> >::invoke() (at /home/ubuntu/code/lib/hello.so)
 *Rice::detail::protect<void (*)(unsigned long*), unsigned long*>(void (*)(unsigned long*), unsigned long*)::{lambda(unsigned long)#1}::operator()(unsigned long) const (at /home/ubuntu/code/lib/hello.so)
 *Rice::detail::protect<void (*)(unsigned long*), unsigned long*>(void (*)(unsigned long*), unsigned long*)::{lambda(unsigned long)#1}::_FUN(unsigned long) (at /home/ubuntu/code/lib/hello.so)
  rb_protect (eval.c:1127)
 *auto Rice::detail::protect<void (*)(unsigned long*), unsigned long*>(void (*)(unsigned long*), unsigned long*) (at /home/ubuntu/code/lib/hello.so)
 *Rice::detail::Anchor::Anchor(unsigned long) (at /home/ubuntu/code/lib/hello.so)
 *void std::_Construct<Rice::detail::Anchor, unsigned long&>(Rice::detail::Anchor*, unsigned long&) (at /home/ubuntu/code/lib/hello.so)
 *std::_Sp_counted_ptr_inplace<Rice::detail::Anchor, std::allocator<void>, (__gnu_cxx::_Lock_policy)2>::_Sp_counted_ptr_inplace<unsigned long&>(std::allocator<void>, unsigned long&) (at /home/ubuntu/code/lib/hello.so)
 *std::__shared_count<(__gnu_cxx::_Lock_policy)2>::__shared_count<Rice::detail::Anchor, std::allocator<void>, unsigned long&>(Rice::detail::Anchor*&, std::_Sp_alloc_shared_tag<std::allocator<void> >, unsigned long&) (at /home/ubuntu/code/lib/hello.so)
 *std::__shared_ptr<Rice::detail::Anchor, (__gnu_cxx::_Lock_policy)2>::__shared_ptr<std::allocator<void>, unsigned long&>(std::_Sp_alloc_shared_tag<std::allocator<void> >, unsigned long&) (at /home/ubuntu/code/lib/hello.so)
 *std::shared_ptr<Rice::detail::Anchor>::shared_ptr<std::allocator<void>, unsigned long&>(std::_Sp_alloc_shared_tag<std::allocator<void> >, unsigned long&) (at /home/ubuntu/code/lib/hello.so)
 *std::shared_ptr<std::enable_if<!std::is_array<Rice::detail::Anchor>::value, Rice::detail::Anchor>::type> std::make_shared<Rice::detail::Anchor, unsigned long&>(unsigned long&) (at /home/ubuntu/code/lib/hello.so)
 *Rice::Pin::Pin(unsigned long) (at /home/ubuntu/code/lib/hello.so)
 *Rice::Object::Object(unsigned long) (at /home/ubuntu/code/lib/hello.so)
 *Rice::Object Rice::Array::push<long const&>(long const&, bool) (at /home/ubuntu/code/lib/hello.so)
 *Init_hello::{lambda(numo::Int64)#1}::operator()(numo::Int64) const (at /home/ubuntu/code/lib/hello.so)
 *Rice::Array std::__invoke_impl<Rice::Array, Init_hello::{lambda(numo::Int64)#1}&, numo::Int64>(std::__invoke_other, Init_hello::{lambda(numo::Int64)#1}&, numo::Int64&&) (at /home/ubuntu/code/lib/hello.so)
 *std::__invoke_result<Init_hello::{lambda(numo::Int64)#1}&, numo::Int64>::type std::__invoke<Init_hello::{lambda(numo::Int64)#1}&, numo::Int64>(Init_hello::{lambda(numo::Int64)#1}&, numo::Int64&&) (at /home/ubuntu/code/lib/hello.so)
 *decltype(auto) std::__apply_impl<Init_hello::{lambda(numo::Int64)#1}&, std::tuple<numo::Int64>, 0ul>(Init_hello::{lambda(numo::Int64)#1}&, std::tuple<numo::Int64>&&, std::integer_sequence<unsigned long, 0ul>) (at /home/ubuntu/code/lib/hello.so)
 *decltype(auto) std::apply<Init_hello::{lambda(numo::Int64)#1}&, std::tuple<numo::Int64> >(Init_hello::{lambda(numo::Int64)#1}&, std::tuple<numo::Int64>&&) (at /home/ubuntu/code/lib/hello.so)
 *Rice::detail::NativeFunction<Init_hello::{lambda(numo::Int64)#1}, false>::invoke(std::tuple<numo::Int64>&&) (at /home/ubuntu/code/lib/hello.so)
 *Rice::detail::NativeFunction<Init_hello::{lambda(numo::Int64)#1}, false>::operator()(std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, unsigned long, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, unsigned long> > >&, unsigned long) (at /home/ubuntu/code/lib/hello.so)
 *Rice::detail::Native::resolve(int, unsigned long*, unsigned long)::{lambda()#1}::operator()() const (at /home/ubuntu/code/lib/hello.so)
 *auto Rice::detail::cpp_protect<Rice::detail::Native::resolve(int, unsigned long*, unsigned long)::{lambda()#1}>(Rice::detail::Native::resolve(int, unsigned long*, unsigned long)::{lambda()#1}&&) (at /home/ubuntu/code/lib/hello.so)
 *Rice::detail::Native::resolve(int, unsigned long*, unsigned long) (at /home/ubuntu/code/lib/hello.so)
  vm_call_cfunc_with_frame_ (vm_insnhelper.c:3902)
  vm_sendish.constprop.0 (vm_insnhelper.c:6124)
  vm_exec_core (insns.def:904)
  rb_vm_exec (vm.c:2798)
  vm_yield_with_cref (vm.c:1871)
  vm_yield (vm.c:1879)
  rb_yield_0 (vm_eval.c:1362)
  rb_yield (vm_eval.c:1378)
  rb_ary_each (array.c:2754)
  vm_call_cfunc_with_frame_ (vm_insnhelper.c:3902)
  vm_sendish.constprop.0 (vm_insnhelper.c:6124)
  vm_exec_core (insns.def:856)
  rb_vm_exec (vm.c:2798)
  vm_yield_with_cref (vm.c:1871)
  vm_yield (vm.c:1879)
  rb_yield_0 (vm_eval.c:1362)
  rb_yield (vm_eval.c:1378)
  rb_ary_collect (array.c:3767)
  vm_call_cfunc_with_frame_ (vm_insnhelper.c:3902)
  vm_sendish.constprop.0 (vm_insnhelper.c:6124)
  vm_exec_core (insns.def:856)

[cfis]

That was quite helpful. I pushed a fix to the dev branch, can you see if that works?

[ankane]

Hmm, still happening with that commit (CI logs and diff).

[cfis]

I didn't push updated include headers so the test used old code. I just pushed updated include headers.

[ankane]

Ah, missed that. Still seeing the issue with the latest commit (CI logs).

[cfis]

Bummer. Guess I will have to try and reproduce locally.