Cache miss fix + WIP #2

wildmaples · 2022-03-03T21:01:37Z

No description provided.

tenderlove · 2022-03-07T17:20:04Z

insns.def

 ()
 ()
 {
    VALUE recv = TOPN(vm_ci_argc(cd->ci));
-
+    rp(recv);


can you remove this line please?

…miss

Always look up instance variable buffers when iterating. It is possible for the instance variable buffer to change out from under the object during iteration, so we cannot cache the buffer on the stack. In the case of Bug #19095, the transient heap moved the buffer during iteration: ``` ; -> 1361 } 1362 } 1363 #endif 1364 miniruby`rb_obj_transient_heap_evacuate: -> 0x1006e5178 <+328>: b 0x1006e517c ; <+332> at variable.c:1362:1 0x1006e517c <+332>: ldp x29, x30, [sp, #0x50] 0x1006e5180 <+336>: add sp, sp, #0x60 0x1006e5184 <+340>: ret Target 0: (miniruby) stopped. (lldb) bt * thread #1, queue = 'com.apple.main-thread', stop reason = watchpoint 1 * frame #0: 0x00000001006e5178 miniruby`rb_obj_transient_heap_evacuate(obj=0x000000010d6b94b0, promote=1) at variable.c:1361:5 frame #1: 0x00000001006cb150 miniruby`transient_heap_block_evacuate(theap=0x0000000100b196c0, block=0x0000000107c00000) at transient_heap.c:734:17 frame #2: 0x00000001006c854c miniruby`transient_heap_evacuate(dmy=0x0000000000000000) at transient_heap.c:808:17 frame #3: 0x00000001007fe6c0 miniruby`rb_postponed_job_flush(vm=0x0000000104402900) at vm_trace.c:1773:21 frame #4: 0x0000000100637a84 miniruby`rb_threadptr_execute_interrupts(th=0x0000000103803bc0, blocking_timing=0) at thread.c:2316:13 frame #5: 0x000000010078b730 miniruby`rb_vm_check_ints(ec=0x00000001048038d0) at vm_core.h:2025:9 frame #6: 0x00000001006fbd10 miniruby`vm_pop_frame(ec=0x00000001048038d0, cfp=0x0000000104a04440, ep=0x0000000104904a28) at vm_insnhelper.c:422:5 frame ruby#7: 0x00000001006fbca0 miniruby`rb_vm_pop_frame(ec=0x00000001048038d0) at vm_insnhelper.c:431:5 frame ruby#8: 0x00000001007d6420 miniruby`vm_call0_cfunc_with_frame(ec=0x00000001048038d0, calling=0x000000016fdcc6a0, argv=0x0000000000000000) at vm_eval.c:153:9 frame ruby#9: 0x00000001007d44cc miniruby`vm_call0_cfunc(ec=0x00000001048038d0, calling=0x000000016fdcc6a0, argv=0x0000000000000000) at vm_eval.c:164:12 frame ruby#10: 0x0000000100766e80 miniruby`vm_call0_body(ec=0x00000001048038d0, calling=0x000000016fdcc6a0, argv=0x0000000000000000) at vm_eval.c:210:15 frame ruby#11: 0x00000001007d76f0 miniruby`vm_call0_cc(ec=0x00000001048038d0, recv=0x000000010d6b49d8, id=2769, argc=0, argv=0x0000000000000000, cc=0x000000010d6b2e58, kw_splat=0) at vm_eval.c:87:12 frame ruby#12: 0x0000000100769e48 miniruby`rb_funcallv_scope(recv=0x000000010d6b49d8, mid=2769, argc=0, argv=0x0000000000000000, scope=CALL_FCALL) at vm_eval.c:1051:16 frame ruby#13: 0x0000000100760a54 miniruby`rb_funcallv(recv=0x000000010d6b49d8, mid=2769, argc=0, argv=0x0000000000000000) at vm_eval.c:1066:12 frame ruby#14: 0x000000010037513c miniruby`rb_inspect(obj=0x000000010d6b49d8) at object.c:633:34 frame ruby#15: 0x000000010002c950 miniruby`inspect_ary(ary=0x000000010d6b4938, dummy=0x0000000000000000, recur=0) at array.c:3091:13 ``` In general though, any calls back out to the interpreter could change the IV buffer, so it's not safe to cache. [Bug #19095]

Always look up instance variable buffers when iterating. It is possible for the instance variable buffer to change out from under the object during iteration, so we cannot cache the buffer on the stack. In the case of Bug #19095, the transient heap moved the buffer during iteration: ``` Watchpoint 1 hit: old value: 0x0000000107c00df8 new value: 0x00000001032743c0 Process 31720 stopped * thread #1, queue = 'com.apple.main-thread', stop reason = watchpoint 1 frame #0: 0x00000001006e5178 miniruby`rb_obj_transient_heap_evacuate(obj=0x000000010d6b94b0, promote=1) at variable.c:1361:5 1358 } 1359 MEMCPY(new_ptr, old_ptr, VALUE, len); 1360 ROBJECT(obj)->as.heap.ivptr = new_ptr; -> 1361 } 1362 } 1363 #endif 1364 miniruby`rb_obj_transient_heap_evacuate: -> 0x1006e5178 <+328>: b 0x1006e517c ; <+332> at variable.c:1362:1 0x1006e517c <+332>: ldp x29, x30, [sp, #0x50] 0x1006e5180 <+336>: add sp, sp, #0x60 0x1006e5184 <+340>: ret Target 0: (miniruby) stopped. (lldb) bt * thread #1, queue = 'com.apple.main-thread', stop reason = watchpoint 1 * frame #0: 0x00000001006e5178 miniruby`rb_obj_transient_heap_evacuate(obj=0x000000010d6b94b0, promote=1) at variable.c:1361:5 frame #1: 0x00000001006cb150 miniruby`transient_heap_block_evacuate(theap=0x0000000100b196c0, block=0x0000000107c00000) at transient_heap.c:734:17 frame #2: 0x00000001006c854c miniruby`transient_heap_evacuate(dmy=0x0000000000000000) at transient_heap.c:808:17 frame #3: 0x00000001007fe6c0 miniruby`rb_postponed_job_flush(vm=0x0000000104402900) at vm_trace.c:1773:21 frame #4: 0x0000000100637a84 miniruby`rb_threadptr_execute_interrupts(th=0x0000000103803bc0, blocking_timing=0) at thread.c:2316:13 frame #5: 0x000000010078b730 miniruby`rb_vm_check_ints(ec=0x00000001048038d0) at vm_core.h:2025:9 frame #6: 0x00000001006fbd10 miniruby`vm_pop_frame(ec=0x00000001048038d0, cfp=0x0000000104a04440, ep=0x0000000104904a28) at vm_insnhelper.c:422:5 frame ruby#7: 0x00000001006fbca0 miniruby`rb_vm_pop_frame(ec=0x00000001048038d0) at vm_insnhelper.c:431:5 frame ruby#8: 0x00000001007d6420 miniruby`vm_call0_cfunc_with_frame(ec=0x00000001048038d0, calling=0x000000016fdcc6a0, argv=0x0000000000000000) at vm_eval.c:153:9 frame ruby#9: 0x00000001007d44cc miniruby`vm_call0_cfunc(ec=0x00000001048038d0, calling=0x000000016fdcc6a0, argv=0x0000000000000000) at vm_eval.c:164:12 frame ruby#10: 0x0000000100766e80 miniruby`vm_call0_body(ec=0x00000001048038d0, calling=0x000000016fdcc6a0, argv=0x0000000000000000) at vm_eval.c:210:15 frame ruby#11: 0x00000001007d76f0 miniruby`vm_call0_cc(ec=0x00000001048038d0, recv=0x000000010d6b49d8, id=2769, argc=0, argv=0x0000000000000000, cc=0x000000010d6b2e58, kw_splat=0) at vm_eval.c:87:12 frame ruby#12: 0x0000000100769e48 miniruby`rb_funcallv_scope(recv=0x000000010d6b49d8, mid=2769, argc=0, argv=0x0000000000000000, scope=CALL_FCALL) at vm_eval.c:1051:16 frame ruby#13: 0x0000000100760a54 miniruby`rb_funcallv(recv=0x000000010d6b49d8, mid=2769, argc=0, argv=0x0000000000000000) at vm_eval.c:1066:12 frame ruby#14: 0x000000010037513c miniruby`rb_inspect(obj=0x000000010d6b49d8) at object.c:633:34 frame ruby#15: 0x000000010002c950 miniruby`inspect_ary(ary=0x000000010d6b4938, dummy=0x0000000000000000, recur=0) at array.c:3091:13 frame ruby#16: 0x0000000100642020 miniruby`exec_recursive(func=(miniruby`inspect_ary at array.c:3084), obj=0x000000010d6b4938, pairid=0x0000000000000000, arg=0x0000000000000000, outer=0, mid=2769) at thread.c:5177:23 frame ruby#17: 0x00000001006412fc miniruby`rb_exec_recursive(func=(miniruby`inspect_ary at array.c:3084), obj=0x000000010d6b4938, arg=0x0000000000000000) at thread.c:5205:12 frame ruby#18: 0x00000001000127f0 miniruby`rb_ary_inspect(ary=0x000000010d6b4938) at array.c:3117:12 ``` In general though, any calls back out to the interpreter could change the IV buffer, so it's not safe to cache. [Bug #19095]

`gc_verify_internal_consistency` reports "found internal inconsistency" for "test_pattern_matching.rb". http://ci.rvm.jp/results/trunk-gc-asserts@ruby-sp2-docker/4501173 Ruby's parser manages objects by two different ways. 1. For parser * markable node holds objects * call `RB_OBJ_WRITTEN` with `p->ast` as parent * `mark_ast_value` marks objects 2. For ripper * unmarkable node, NODE_RIPPER/NODE_CDECL, holds objects * call `rb_ast_add_mark_object`. This function calls `rb_hash_aset` then `RB_OBJ_WRITTEN` is called with `mark_hash` as parent * `mark_hash` marks objects However in current pattern_matching implementation * markable node holds objects * call `rb_ast_add_mark_object` This commit fix it to be #2. This was inconsistency however always `mark_hash` is made young by `rb_ast_add_mark_object` call then objects are not collected.

[Bug #19793] Dummy frames are created at the top level when requiring another file. While requiring a file, it will try to convert using encodings. Some of these encodings will not respond to to_str. If method_missing is redefined on Object, then it will call method_missing and attempt raise an error. However, the iseq is invalid as it's a dummy frame so it will write an invalid iseq to the created NoMethodError. The following script crashes: ``` GC.stress = true class Object public :method_missing end File.write("/tmp/empty.rb", "") require "/tmp/empty.rb" ``` With the following backtrace: ``` frame #0: 0x00000001000fa8b8 miniruby`RVALUE_MARKED(obj=4308637824) at gc.c:1638:12 frame #1: 0x00000001000fb440 miniruby`RVALUE_BLACK_P(obj=4308637824) at gc.c:1763:12 frame #2: 0x00000001000facdc miniruby`gc_writebarrier_incremental(a=4308637824, b=4308332208, objspace=0x000000010180b000) at gc.c:8822:9 frame #3: 0x00000001000faad8 miniruby`rb_gc_writebarrier(a=4308637824, b=4308332208) at gc.c:8864:17 frame #4: 0x000000010016aff0 miniruby`rb_obj_written(a=4308637824, oldv=36, b=4308332208, filename="../iseq.c", line=1279) at gc.h:804:9 frame #5: 0x0000000100162a60 miniruby`rb_obj_write(a=4308637824, slot=0x0000000100d09888, b=4308332208, filename="../iseq.c", line=1279) at gc.h:837:5 frame #6: 0x0000000100165b0c miniruby`iseqw_new(iseq=0x0000000100d09880) at iseq.c:1279:9 frame ruby#7: 0x0000000100165a64 miniruby`rb_iseqw_new(iseq=0x0000000100d09880) at iseq.c:1289:12 frame ruby#8: 0x00000001000d8324 miniruby`name_err_init_attr(exc=4309777920, recv=4304780496, method=827660) at error.c:1830:35 frame ruby#9: 0x00000001000d1b80 miniruby`name_err_init(exc=4309777920, mesg=4308332496, recv=4304780496, method=827660) at error.c:1869:12 frame ruby#10: 0x00000001000d1bd4 miniruby`rb_nomethod_err_new(mesg=4308332496, recv=4304780496, method=827660, args=4308332448, priv=0) at error.c:1957:5 frame ruby#11: 0x000000010039049c miniruby`rb_make_no_method_exception(exc=4304914512, format=4308332496, obj=4304780496, argc=1, argv=0x000000016fdfab00, priv=0) at vm_eval.c:959:16 frame ruby#12: 0x00000001003b3274 miniruby`raise_method_missing(ec=0x0000000100b06f40, argc=1, argv=0x000000016fdfab00, obj=4304780496, last_call_status=MISSING_NOENTRY) at vm_eval.c:999:15 frame ruby#13: 0x00000001003945d4 miniruby`rb_method_missing(argc=1, argv=0x000000016fdfab00, obj=4304780496) at vm_eval.c:944:5 ... frame ruby#23: 0x000000010038f5e4 miniruby`rb_vm_call_kw(ec=0x0000000100b06f40, recv=4304780496, id=2865, argc=1, argv=0x000000016fdfab00, me=0x0000000100cbfcf0, kw_splat=0) at vm_eval.c:326:12 frame ruby#24: 0x00000001003c18e4 miniruby`call_method_entry(ec=0x0000000100b06f40, defined_class=4304927952, obj=4304780496, id=2865, cme=0x0000000100cbfcf0, argc=1, argv=0x000000016fdfab00, kw_splat=0) at vm_method.c:2720:20 frame ruby#25: 0x00000001003c440c miniruby`check_funcall_exec(v=6171896792) at vm_eval.c:589:12 frame ruby#26: 0x00000001000dec00 miniruby`rb_vrescue2(b_proc=(miniruby`check_funcall_exec at vm_eval.c:587), data1=6171896792, r_proc=(miniruby`check_funcall_failed at vm_eval.c:596), data2=6171896792, args="Pȗ") at eval.c:919:18 frame ruby#27: 0x00000001000deab0 miniruby`rb_rescue2(b_proc=(miniruby`check_funcall_exec at vm_eval.c:587), data1=6171896792, r_proc=(miniruby`check_funcall_failed at vm_eval.c:596), data2=6171896792) at eval.c:900:17 frame ruby#28: 0x000000010039008c miniruby`check_funcall_missing(ec=0x0000000100b06f40, klass=4304923536, recv=4304780496, mid=3233, argc=0, argv=0x0000000000000000, respond=-1, def=36, kw_splat=0) at vm_eval.c:666:15 frame ruby#29: 0x000000010038fa60 miniruby`rb_check_funcall_default_kw(recv=4304780496, mid=3233, argc=0, argv=0x0000000000000000, def=36, kw_splat=0) at vm_eval.c:703:21 frame ruby#30: 0x000000010038fb04 miniruby`rb_check_funcall(recv=4304780496, mid=3233, argc=0, argv=0x0000000000000000) at vm_eval.c:685:12 frame ruby#31: 0x00000001001c469c miniruby`convert_type_with_id(val=4304780496, tname="String", method=3233, raise=0, index=-1) at object.c:3061:15 frame ruby#32: 0x00000001001c4a4c miniruby`rb_check_convert_type_with_id(val=4304780496, type=5, tname="String", method=3233) at object.c:3153:9 frame ruby#33: 0x00000001002d59f8 miniruby`rb_check_string_type(str=4304780496) at string.c:2571:11 frame ruby#34: 0x000000010014b7b0 miniruby`io_encoding_set(fptr=0x0000000100d09ca0, v1=4304780496, v2=4, opt=4) at io.c:11655:19 frame ruby#35: 0x0000000100139a58 miniruby`rb_io_set_encoding(argc=1, argv=0x000000016fdfb450, io=4308334032) at io.c:13497:5 frame ruby#36: 0x00000001003c0004 miniruby`ractor_safe_call_cfunc_m1(recv=4308334032, argc=1, argv=0x000000016fdfb450, func=(miniruby`rb_io_set_encoding at io.c:13487)) at vm_insnhelper.c:3271:12 ... frame ruby#43: 0x0000000100390b08 miniruby`rb_funcall(recv=4308334032, mid=16593, n=1) at vm_eval.c:1137:12 frame ruby#44: 0x00000001002a43d8 miniruby`load_file_internal(argp_v=6171899936) at ruby.c:2500:5 ... ```

to avoid deadlock ```ruby r = Ractor.new do obj = Thread.new{} Ractor.yield obj rescue => e e.message end p r.take ``` ``` (lldb) bt * thread #1, name = 'miniruby', stop reason = signal SIGSTOP * frame #0: 0x0000ffff44881410 libpthread.so.0`__lll_lock_wait + 88 frame #1: 0x0000ffff4487a078 libpthread.so.0`__pthread_mutex_lock + 232 frame #2: 0x0000aaab617c0980 miniruby`rb_native_mutex_lock(lock=<unavailable>) at thread_pthread.c:109:14 frame #3: 0x0000aaab617c1d58 miniruby`ubf_event_waiting [inlined] thread_sched_lock_(th=0x0000aaab9df82980, file=<unavailable>, line=46, sched=0x0000aaab9dec79b8) at thread_pthread.c:351:5 frame #4: 0x0000aaab617c1d50 miniruby`ubf_event_waiting(ptr=0x0000aaab9df82980) at thread_pthread_mn.c:46:5 frame #5: 0x0000aaab617c6020 miniruby`rb_threadptr_interrupt [inlined] rb_threadptr_interrupt_common(trap=0, th=0x0000aaab9df82980) at thread.c:352:25 frame #6: 0x0000aaab617c5fec miniruby`rb_threadptr_interrupt(th=0x0000aaab9df82980) at thread.c:365:5 frame ruby#7: 0x0000aaab617379b0 miniruby`rb_ractor_terminate_all at ractor.c:2364:13 frame ruby#8: 0x0000aaab6173797c miniruby`rb_ractor_terminate_all at ractor.c:2383:17 frame ruby#9: 0x0000aaab61737958 miniruby`rb_ractor_terminate_all [inlined] ractor_terminal_interrupt_all(vm=0x0000aaab9dea3320) at ractor.c:2375:1 frame ruby#10: 0x0000aaab61737950 miniruby`rb_ractor_terminate_all at ractor.c:2424:13 frame ruby#11: 0x0000aaab6164f108 miniruby`rb_ec_cleanup(ec=0x0000aaab9dea5900, ex=RUBY_TAG_NONE) at eval.c:239:9 frame ruby#12: 0x0000aaab6164fa3c miniruby`ruby_run_node(n=0x0000ffff417ed178) at eval.c:328:12 frame ruby#13: 0x0000aaab615a5ab0 miniruby`main at main.c:39:12 frame ruby#14: 0x0000aaab615a5a98 miniruby`main(argc=<unavailable>, argv=<unavailable>) at main.c:58:12 frame ruby#15: 0x0000ffff44714b2c libc.so.6`__libc_start_main + 228 frame ruby#16: 0x0000aaab615a5b0c miniruby`_start + 52 (lldb) thread select 3 * thread #3, name = 'bootstraptest.*', stop reason = signal SIGSTOP frame #0: 0x0000ffff448813ec libpthread.so.0`__lll_lock_wait + 52 libpthread.so.0`__lll_lock_wait: -> 0xffff448813ec <+52>: svc #0 0xffff448813f0 <+56>: eor w20, w20, #0x80 0xffff448813f4 <+60>: sxtw x20, w20 0xffff448813f8 <+64>: b 0xffff44881414 ; <+92> (lldb) bt * thread #3, name = 'bootstraptest.*', stop reason = signal SIGSTOP * frame #0: 0x0000ffff448813ec libpthread.so.0`__lll_lock_wait + 52 frame #1: 0x0000ffff4487a078 libpthread.so.0`__pthread_mutex_lock + 232 frame #2: 0x0000aaab617c0980 miniruby`rb_native_mutex_lock(lock=<unavailable>) at thread_pthread.c:109:14 frame #3: 0x0000aaab61823d68 miniruby`rb_vm_lock_enter_body [inlined] vm_lock_enter(no_barrier=false, lev=0x0000ffff215bfbe4, locked=false, vm=0x0000aaab9dea3320, cr=0x0000aaab9dec7890) at vm_sync.c:57:9 frame #4: 0x0000aaab61823d60 miniruby`rb_vm_lock_enter_body(lev=0x0000ffff215bfbe4) at vm_sync.c:119:9 frame #5: 0x0000aaab617c1b30 miniruby`thread_sched_setup_running_threads [inlined] rb_vm_lock_enter(file=<unavailable>, line=597, lev=0x0000ffff215bfbe4) at vm_sync.h:75:9 frame #6: 0x0000aaab617c1b14 miniruby`thread_sched_setup_running_threads(vm=0x0000aaab9dea3320, add_th=0x0000aaab9df82980, del_th=<unavailable>, add_timeslice_th=0x0000000000000000, cr=<unavailable>, sched=<unavailable>, sched=<unavailable>) at thread_pthread.c:597:9 frame ruby#7: 0x0000aaab617c29b4 miniruby`thread_sched_wait_running_turn at thread_pthread.c:614:5 frame ruby#8: 0x0000aaab617c298c miniruby`thread_sched_wait_running_turn(sched=0x0000aaab9dec79b8, th=0x0000aaab9df82980, can_direct_transfer=true) at thread_pthread.c:868:9 frame ruby#9: 0x0000aaab617c6f0c miniruby`thread_sched_wait_events(sched=0x0000aaab9dec79b8, th=0x0000aaab9df82980, fd=<unavailable>, events=<unavailable>, rel=<unavailable>) at thread_pthread_mn.c:90:17 frame ruby#10: 0x0000aaab617c7354 miniruby`rb_thread_terminate_all at thread_pthread.c 8000 :3248:13 frame ruby#11: 0x0000aaab617c733c miniruby`rb_thread_terminate_all(th=0x0000aaab9df82980) at thread.c:466:13 frame ruby#12: 0x0000aaab617c7a64 miniruby`thread_start_func_2(th=0x0000aaab9df82980, stack_start=<unavailable>) at thread.c:713:9 frame ruby#13: 0x0000aaab617c7d1c miniruby`co_start [inlined] call_thread_start_func_2(th=0x0000aaab9df82980) at thread_pthread.c:2165:5 frame ruby#14: 0x0000aaab617c7cd0 miniruby`co_start(from=<unavailable>, self=0x0000aaab9df0f760) at thread_pthread_mn.c:421:9 ```

When Ruby is built with ASAN and RUBY_FREE_AT_EXIT is enabled, the following error occurs: READ of size 8 at 0x74c666610020 thread T0 #0 0x593b6712ecc6 in RB_BUILTIN_TYPE include/ruby/internal/value_type.h:191:30 #1 0x593b6712ecc6 in rb_gc_impl_shutdown_free_objects gc_impl.c:3208:17 #2 0x593b6749a62e in ruby_vm_destruct vm.c:3133:17

During compilation, we write keyword default values into the iseq, so we should mark it to ensure it does not get GC'd. This might fix issues on ASAN like http://ci.rvm.jp/logfiles/brlog.trunk_asan.20240927-194923 ==805257==ERROR: AddressSanitizer: use-after-poison on address 0x7b7e5e3e2828 at pc 0x5e09ac4822f8 bp 0x7ffde56b0140 sp 0x7ffde56b0138 READ of size 8 at 0x7b7e5e3e2828 thread T0 #0 0x5e09ac4822f7 in RB_BUILTIN_TYPE include/ruby/internal/value_type.h:191:30 #1 0x5e09ac4822f7 in rbimpl_RB_TYPE_P_fastpath include/ruby/internal/value_type.h:352:19 #2 0x5e09ac4822f7 in gc_mark gc/default.c:4488:9 #3 0x5e09ac51011e in rb_iseq_mark_and_move iseq.c:361:17 #4 0x5e09ac4b85c4 in rb_imemo_mark_and_move imemo.c:386:9 #5 0x5e09ac467544 in rb_gc_mark_children gc.c:2508:9 #6 0x5e09ac482c24 in gc_mark_children gc/default.c:4673:5 ruby#7 0x5e09ac482c24 in gc_mark_stacked_objects gc/default.c:4694:9 ruby#8 0x5e09ac482c24 in gc_mark_stacked_objects_all gc/default.c:4732:12 ruby#9 0x5e09ac48c7f9 in gc_marks_rest gc/default.c:5755:9 ruby#10 0x5e09ac48c7f9 in gc_marks gc/default.c:5870:9 ruby#11 0x5e09ac48c7f9 in gc_start gc/default.c:6517:13

fill_lines is passed -1 for offset, which causes it to read the -1 index of traces. This is not valid memory as -1 is reading before the trace global variable in rb_print_backtrace. This code comes from commit 99d1f5f, where there used to be special handling for the -1 index. We can see this error in ASAN: ==71037==ERROR: AddressSanitizer: global-buffer-overflow on address 0x00010157abf8 at pc 0x00010116f3b8 bp 0x00016f92c3b0 sp 0x00016f92c3a8 READ of size 8 at 0x00010157abf8 thread T0 #0 0x10116f3b4 in debug_info_read addr2line.c:1945 #1 0x10116cc90 in fill_lines addr2line.c:2497 #2 0x101169dbc in rb_dump_backtrace_with_lines addr2line.c:2635 #3 0x100e56788 in rb_print_backtrace vm_dump.c:825 #4 0x100e56db4 in rb_vm_bugreport vm_dump.c:1155 #5 0x100734dc4 in rb_bug_without_die error.c:1085 #6 0x100734ae4 in rb_bug error.c:109

[Bug #20921] When we create a cache entry for a constant, the following sequence of events could happen: - vm_track_constant_cache is called to insert a constant cache. - In vm_track_constant_cache, we first look up the ST table for the ID of the constant. Assume the ST table exists because another iseq also holds a cache entry for this ID. - We then insert into this ST table with the iseq_inline_constant_cache. - However, while inserting into this ST table, it allocates memory, which could trigger a GC. Assume that it does trigger a GC. - The GC frees the one and only other iseq that holds a cache entry for this ID. - In remove_from_constant_cache, it will appear that the ST table is now empty because there are no more iseq with cache entries for this ID, so we free the ST table. - We complete GC and continue our st_insert. However, this ST table has been freed so we now have a use-after-free. This issue is very hard to reproduce, because it requires that the GC runs at a very specific time. However, we can make it show up by applying this patch which runs GC right before the st_insert to mimic the st_insert triggering a GC: diff --git a/vm_insnhelper.c b/vm_insnhelper.c index 3cb23f0..a93998136a 100644 --- a/vm_insnhelper.c +++ b/vm_insnhelper.c @@ -6338,6 +6338,10 @@ vm_track_constant_cache(ID id, void *ic) rb_id_table_insert(const_cache, id, (VALUE)ics); } + if (id == rb_intern("MyConstant")) rb_gc(); + st_insert(ics, (st_data_t) ic, (st_data_t) Qtrue); } And if we run this script: Object.const_set("MyConstant", "Hello!") my_proc = eval("-> { MyConstant }") my_proc.call my_proc = eval("-> { MyConstant }") my_proc.call We can see that ASAN outputs a use-after-free error: ==36540==ERROR: AddressSanitizer: heap-use-after-free on address 0x606000049528 at pc 0x000102f3ceac bp 0x00016d607a70 sp 0x00016d607a68 READ of size 8 at 0x606000049528 thread T0 #0 0x102f3cea8 in do_hash st.c:321 #1 0x102f3ddd0 in rb_st_insert st.c:1132 #2 0x103140700 in vm_track_constant_cache vm_insnhelper.c:6345 #3 0x1030b91d8 in vm_ic_track_const_chain vm_insnhelper.c:6356 #4 0x1030b8cf8 in rb_vm_opt_getconstant_path vm_insnhelper.c:6424 #5 0x1030bc1e0 in vm_exec_core insns.def:263 #6 0x1030b55fc in rb_vm_exec vm.c:2585 ruby#7 0x1030fe0ac in rb_iseq_eval_main vm.c:2851 ruby#8 0x102a82588 in rb_ec_exec_node eval.c:281 ruby#9 0x102a81fe0 in ruby_run_node eval.c:319 ruby#10 0x1027f3db4 in rb_main main.c:43 ruby#11 0x1027f3bd4 in main main.c:68 ruby#12 0x183900270 (<unknown module>) 0x606000049528 is located 8 bytes inside of 56-byte region [0x606000049520,0x606000049558) freed by thread T0 here: #0 0x104174d40 in free+0x98 (libclang_rt.asan_osx_dynamic.dylib:arm64e+0x54d40) #1 0x102ada89c in rb_gc_impl_free default.c:8183 #2 0x102ada7dc in ruby_sized_xfree gc.c:4507 #3 0x102ac4d34 in ruby_xfree gc.c:4518 #4 0x102f3cb34 in rb_st_free_table st.c:663 #5 0x102bd52d8 in remove_from_constant_cache iseq.c:119 #6 0x102bbe2cc in iseq_clear_ic_references iseq.c:153 ruby#7 0x102bbd2a0 in rb_iseq_free iseq.c:166 ruby#8 0x102b32ed0 in rb_imemo_free imemo.c:564 ruby#9 0x102ac4b44 in rb_gc_obj_free gc.c:1407 ruby#10 0x102af4290 in gc_sweep_plane default.c:3546 ruby#11 0x102af3bdc in gc_sweep_page default.c:3634 ruby#12 0x102aeb140 in gc_sweep_step default.c:3906 ruby#13 0x102aeadf0 in gc_sweep_rest default.c:3978 ruby#14 0x102ae4714 in gc_sweep default.c:4155 ruby#15 0x102af8474 in gc_start default.c:6484 ruby#16 0x102afbe30 in garbage_collect default.c:6363 ruby#17 0x102ad37f0 in rb_gc_impl_start default.c:6816 ruby#18 0x102ad3634 in rb_gc gc.c:3624 ruby#19 0x1031406ec in vm_track_constant_cache vm_insnhelper.c:6342 #20 0x1030b91d8 in vm_ic_track_const_chain vm_insnhelper.c:6356 ruby#21 0x1030b8cf8 in rb_vm_opt_getconstant_path vm_insnhelper.c:6424 ruby#22 0x1030bc1e0 in vm_exec_core insns.def:263 ruby#23 0x1030b55fc in rb_vm_exec vm.c:2585 ruby#24 0x1030fe0ac in rb_iseq_eval_main vm.c:2851 ruby#25 0x102a82588 in rb_ec_exec_node eval.c:281 ruby#26 0x102a81fe0 in ruby_run_node eval.c:319 ruby#27 0x1027f3db4 in rb_main main.c:43 ruby#28 0x1027f3bd4 in main main.c:68 ruby#29 0x183900270 (<unknown module>) previously allocated by thread T0 here: #0 0x104174c04 in malloc+0x94 (libclang_rt.asan_osx_dynamic.dylib:arm64e+0x54c04) #1 0x102ada0ec in rb_gc_impl_malloc default.c:8198 #2 0x102acee44 in ruby_xmalloc gc.c:4438 #3 0x102f3c85c in rb_st_init_table_with_size st.c:571 #4 0x102f3c900 in rb_st_init_table st.c:600 #5 0x102f3c920 in rb_st_init_numtable st.c:608 #6 0x103140698 in vm_track_constant_cache vm_insnhelper.c:6337 ruby#7 0x1030b91d8 in vm_ic_track_const_chain vm_insnhelper.c:6356 ruby#8 0x1030b8cf8 in rb_vm_opt_getconstant_path vm_insnhelper.c:6424 ruby#9 0x1030bc1e0 in vm_exec_core insns.def:263 ruby#10 0x1030b55fc in rb_vm_exec vm.c:2585 ruby#11 0x1030fe0ac in rb_iseq_eval_main vm.c:2851 ruby#12 0x102a82588 in rb_ec_exec_node eval.c:281 ruby#13 0x102a81fe0 in ruby_run_node eval.c:319 ruby#14 0x1027f3db4 in rb_main main.c:43 ruby#15 0x1027f3bd4 in main main.c:68 ruby#16 0x183900270 (<unknown module>) This commit fixes this bug by adding a inserting_constant_cache_id field to the VM, which stores the ID that is currently being inserted and, in remove_from_constant_cache, we don't free the ST table for ID equal to this one. Co-Authored-By: Alan Wu <alanwu@ruby-lang.org>

…uby#13231) This change addresses the following ASAN error: ``` ==36597==ERROR: AddressSanitizer: heap-use-after-free on address 0x512000396ba8 at pc 0x7fcad5cbad9f bp 0x7fff19739af0 sp 0x7fff19739ae8 WRITE of size 8 at 0x512000396ba8 thread T0 [643/756] 36600=optparse/test_summary #0 0x7fcad5cbad9e in free_fast_fallback_getaddrinfo_entry /home/runner/work/ruby-dev-builder/ruby-dev-builder/ext/socket/raddrinfo.c:3046:22 #1 0x7fcad5c9fb48 in fast_fallback_inetsock_cleanup /home/runner/work/ruby-dev-builder/ruby-dev-builder/ext/socket/ipsocket.c:1179:17 #2 0x7fcadf3b611a in rb_ensure /home/runner/work/ruby-dev-builder/ruby-dev-builder/eval.c:1081:5 #3 0x7fcad5c9b44b in rsock_init_inetsock /home/runner/work/ruby-dev-builder/ruby-dev-builder/ext/socket/ipsocket.c:1289:20 #4 0x7fcad5ca22b8 in tcp_init /home/runner/work/ruby-dev-builder/ruby-dev-builder/ext/socket/tcpsocket.c:76:12 #5 0x7fcadf83ba70 in vm_call0_cfunc_with_frame /home/runner/work/ruby-dev-builder/ruby-dev-builder/./vm_eval.c:164:15 ... ``` A `struct fast_fallback_getaddrinfo_shared` is shared between the main thread and two child threads. This struct contains an array of `fast_fallback_getaddrinfo_entry`. `fast_fallback_getaddrinfo_entry` and `fast_fallback_getaddrinfo_shared` were freed separately, and if `fast_fallback_getaddrinfo_shared` was freed first and then an attempt was made to free a `fast_fallback_getaddrinfo_entry`, a `heap-use-after-free` could occur. This change avoids that possibility by separating the deallocation of the addrinfo memory held by `fast_fallback_getaddrinfo_entry` from the access and lifecycle of the `fast_fallback_getaddrinfo_entry` itself.

[Bug #18119] When we create classes, it pushes the class to the subclass list of the superclass. This access needs to be synchronized because multiple Ractors may be creating classes with the same superclass, which would cause race conditions and cause the linked list to be corrupted. For example, we can reproduce with this script crashing: workers = (0...8).map do Ractor.new do loop do 100.times.map { Class.new } Ractor.yield nil end end end 100.times { Ractor.select(*workers) } With ASAN enabled, we can see that there are use-after-free errors: ==176013==ERROR: AddressSanitizer: heap-use-after-free on address 0x5030000974f0 at pc 0x62f9e56f892d bp 0x7a503f1ffd90 sp 0x7a503f1ffd88 WRITE of size 8 at 0x5030000974f0 thread T4 #0 0x62f9e56f892c in rb_class_remove_from_super_subclasses class.c:149:24 #1 0x62f9e58c9dd2 in rb_gc_obj_free gc.c:1262:9 #2 0x62f9e58f6e19 in gc_sweep_plane gc/default/default.c:3450:21 #3 0x62f9e58f686a in gc_sweep_page gc/default/default.c:3535:13 #4 0x62f9e58f12b4 in gc_sweep_step gc/default/default.c:3810:9 #5 0x62f9e58ed2a7 in gc_sweep gc/default/default.c:4058:13 #6 0x62f9e58fac93 in gc_start gc/default/default.c:6402:13 ruby#7 0x62f9e58e8b69 in heap_prepare gc/default/default.c:2032:13 ruby#8 0x62f9e58e8b69 in heap_next_free_page gc/default/default.c:2255:9 ruby#9 0x62f9e58e8b69 in newobj_cache_miss gc/default/default.c:2362:38 ... 0x5030000974f0 is located 16 bytes inside of 24-byte region [0x5030000974e0,0x5030000974f8) freed by thread T4 here: #0 0x62f9e562f28a in free (miniruby+0x1fd28a) (BuildId: 5ad6d9e7cec8318df6726ea5ce34d3c76d0d0233) #1 0x62f9e58ca2ab in rb_gc_impl_free gc/default/default.c:8102:9 #2 0x62f9e58ca2ab in ruby_sized_xfree gc.c:5029:13 #3 0x62f9e58ca2ab in ruby_xfree gc.c:5040:5 #4 0x62f9e56f88e6 in rb_class_remove_from_super_subclasses class.c:152:9 #5 0x62f9e58c9dd2 in rb_gc_obj_free gc.c:1262:9 #6 0x62f9e58f6e19 in gc_sweep_plane gc/default/default.c:3450:21 ruby#7 0x62f9e58f686a in gc_sweep_page gc/default/default.c:3535:13 ruby#8 0x62f9e58f12b4 in gc_sweep_step gc/default/default.c:3810:9 ruby#9 0x62f9e58ed2a7 in gc_sweep gc/default/default.c:4058:13 ... previously allocated by thread T5 here: #0 0x62f9e562f70d in calloc (miniruby+0x1fd70d) (BuildId: 5ad6d9e7cec8318df6726ea5ce34d3c76d0d0233) #1 0x62f9e58c8e1a in calloc1 gc/default/default.c:1472:12 #2 0x62f9e58c8e1a in rb_gc_impl_calloc gc/default/default.c:8138:5 #3 0x62f9e58c8e1a in ruby_xcalloc_body gc.c:4964:12 #4 0x62f9e58c8e1a in ruby_xcalloc gc.c:4958:34 #5 0x62f9e56f906e in push_subclass_entry_to_list class.c:88:13 #6 0x62f9e56f906e in rb_class_subclass_add class.c:111:38 ruby#7 0x62f9e56f906e in RCLASS_SET_SUPER internal/class.h:257:9 ruby#8 0x62f9e56fca7a in make_metaclass class.c:786:5 ruby#9 0x62f9e59db982 in rb_class_initialize object.c:2101:5

In commit d42b9ff, an optimization was introduced that can speed up Regexp#match by 15% when it matches with strings of different encodings. This optimization, however, does not work across ractors. To fix this, we only use the optimization if no ractors have been started. In the future, we could use atomics for the reference counting if we find it's needed and if it's more performant. The backtrace of the misbehaving native thread: ``` * frame #0: 0x0000000189c94388 libsystem_kernel.dylib`__pthread_kill + 8 frame #1: 0x0000000189ccd88c libsystem_pthread.dylib`pthread_kill + 296 frame #2: 0x0000000189bd6c60 libsystem_c.dylib`abort + 124 frame #3: 0x0000000189adb174 libsystem_malloc.dylib`malloc_vreport + 892 frame #4: 0x0000000189adec90 libsystem_malloc.dylib`malloc_report + 64 frame #5: 0x0000000189ae321c libsystem_malloc.dylib`___BUG_IN_CLIENT_OF_LIBMALLOC_POINTER_BEING_FREED_WAS_NOT_ALLOCATED + 32 frame #6: 0x00000001001c3be4 ruby`onig_free_body(reg=0x000000012d84b660) at regcomp.c:5663:5 frame ruby#7: 0x00000001001ba828 ruby`rb_reg_prepare_re(re=4748462304, str=4748451168) at re.c:1680:13 frame ruby#8: 0x00000001001bac58 ruby`rb_reg_onig_match(re=4748462304, str=4748451168, match=(ruby`reg_onig_search [inlined] rbimpl_RB_TYPE_P_fastpath at value_type.h:349:14 ruby`reg_onig_search [inlined] rbimpl_rstring_getmem at rstring.h:391:5 ruby`reg_onig_search at re.c:1781:5), args=0x000000013824b168, regs=0x000000013824b150) at re.c:1708:20 frame ruby#9: 0x00000001001baefc ruby`rb_reg_search_set_match(re=4748462304, str=4748451168, pos=<unavailable>, reverse=0, set_backref_str=1, set_match=0x0000000000000000) at re.c:1809:27 frame ruby#10: 0x00000001001bae80 ruby`rb_reg_search0(re=<unavailable>, str=<unavailable>, pos=<unavailable>, reverse=<unavailable>, set_backref_str=<unavailable>, match=<unavailable>) at re.c:1861:12 [artificial] frame ruby#11: 0x0000000100230b90 ruby`rb_pat_search0(pat=<unavailable>, str=<unavailable>, pos=<unavailable>, set_backref_str=<unavailable>, match=<unavailable>) at string.c:6619:16 [artificial] frame ruby#12: 0x00000001002287f4 ruby`rb_str_sub_bang [inlined] rb_pat_search(pat=4748462304, str=4748451168, pos=0, set_backref_str=1) at string.c:6626:12 frame ruby#13: 0x00000001002287dc ruby`rb_str_sub_bang(argc=1, argv=0x00000001381280d0, str=4748451168) at string.c:6668:11 frame ruby#14: 0x000000010022826c ruby`rb_str_sub ``` You can reproduce this by running: ``` RUBY_TESTOPTS="--name=/test_str_capitalize/" make test-all TESTS=test/ruby/test_m17n.comb ``` However, you need to run it with multiple ractors at once. Co-authored-by: jhawthorn <john@hawthorn.email>

`name` is used via `RSTRING_PTR` within rb_str_catf, which may allocate and thus potentially trigger GC. Although `name` is still referenced by a local variable, the compiler might optimize away the reference before the GC sees it, especially under aggressive optimization or when debugging tools like ASAN are used. This patch adds an explicit `RB_GC_GUARD` to ensure `name` is kept alive until after the last use. While it's not certain this is the root cause of the following observed use-after-poison ASAN error, I believe this fix is indeed needed and hopefully a likely candidate for preventing the error. ``` ==1960369==ERROR: AddressSanitizer: use-after-poison on address 0x7ec6a00f1d88 at pc 0x5fb5bcafcf2e bp 0x7ffcc1178cb0 sp 0x7ffcc1178470 READ of size 61 at 0x7ec6a00f1d88 thread T0 #0 0x5fb5bcafcf2d in __asan_memcpy (/tmp/ruby/build/trunk_asan/ruby+0x204f2d) (BuildId: 6d92c84a27b87cfd253c38eeb552593f215ffb3d) #1 0x5fb5bcde1fa5 in memcpy /usr/include/x86_64-linux-gnu/bits/string_fortified.h:29:10 #2 0x5fb5bcde1fa5 in ruby_nonempty_memcpy /tmp/ruby/src/trunk_asan/include/ruby/internal/memory.h:758:16 #3 0x5fb5bcde1fa5 in ruby__sfvwrite /tmp/ruby/src/trunk_asan/sprintf.c:1083:9 #4 0x5fb5bcde1521 in BSD__sprint /tmp/ruby/src/trunk_asan/vsnprintf.c:318:8 #5 0x5fb5bcde0fbc in BSD_vfprintf /tmp/ruby/src/trunk_asan/vsnprintf.c:1215:3 #6 0x5fb5bcdde4b1 in ruby_vsprintf0 /tmp/ruby/src/trunk_asan/sprintf.c:1164:5 ruby#7 0x5fb5bcddd648 in rb_str_vcatf /tmp/ruby/src/trunk_asan/sprintf.c:1234:5 ruby#8 0x5fb5bcddd648 in rb_str_catf /tmp/ruby/src/trunk_asan/sprintf.c:1245:11 ruby#9 0x5fb5bcf97c67 in location_format /tmp/ruby/src/trunk_asan/vm_backtrace.c:462:9 ruby#10 0x5fb5bcf97c67 in location_to_str /tmp/ruby/src/trunk_asan/vm_backtrace.c:493:12 ruby#11 0x5fb5bcf90a37 in location_to_str_dmyarg /tmp/ruby/src/trunk_asan/vm_backtrace.c:795:12 ruby#12 0x5fb5bcf90a37 in backtrace_collect /tmp/ruby/src/trunk_asan/vm_backtrace.c:786:28 ruby#13 0x5fb5bcf90a37 in backtrace_to_str_ary /tmp/ruby/src/trunk_asan/vm_backtrace.c:804:9 ruby#14 0x5fb5bcf90a37 in rb_backtrace_to_str_ary /tmp/ruby/src/trunk_asan/vm_backtrace.c:816:9 ruby#15 0x5fb5bd335b25 in exc_backtrace /tmp/ruby/src/trunk_asan/error.c:1904:15 ruby#16 0x5fb5bd335b25 in rb_get_backtrace /tmp/ruby/src/trunk_asan/error.c:1924:16 ``` https://ci.rvm.jp/results/trunk_asan@ruby-sp1/5810304

This change addresses the following ASAN error: ``` ==1973462==ERROR: AddressSanitizer: heap-use-after-free on address 0x5110002117dc at pc 0x749c307c8a65 bp 0x7ffc3af331d0 sp 0x7ffc3af331c8 READ of size 4 at 0x5110002117dc thread T0 #0 0x749c307c8a64 in rb_getaddrinfo /tmp/ruby/src/trunk_asan/ext/socket/raddrinfo.c:564:14 #1 0x749c307c8a64 in rsock_getaddrinfo /tmp/ruby/src/trunk_asan/ext/socket/raddrinfo.c:1008:21 #2 0x749c307cac48 in rsock_addrinfo /tmp/ruby/src/trunk_asan/ext/socket/raddrinfo.c:1049:12 #3 0x749c307b10ae in init_inetsock_internal /tmp/ruby/src/trunk_asan/ext/socket/ipsocket.c:62:23 #4 0x562c5b2e327e in rb_ensure /tmp/ruby/src/trunk_asan/eval.c:1080:18 #5 0x749c307aafd4 in rsock_init_inetsock /tmp/ruby/src/trunk_asan/ext/socket/ipsocket.c:1318:12 #6 0x749c307b3b78 in tcp_svr_init /tmp/ruby/src/trunk_asan/ext/socket/tcpserver.c:39:12 ``` Fixed to avoid accessing memory that has already been freed after calling `free_getaddrinfo_arg`.

This is notably faster: no need to hash indices. Before: ``` plum% samply record ~/.rubies/ruby-zjit/bin/ruby --zjit benchmarks/getivar.rb ruby 3.5.0dev (2025-07-10T14:40:49Z master 51252ef) +ZJIT dev +PRISM [arm64-darwin24] itr: time #1: 5311ms #2: 49ms #3: 49ms #4: 48ms ``` After: ``` plum% samply record ~/.rubies/ruby-zjit/bin/ruby --zjit benchmarks/getivar.rb ruby 3.5.0dev (2025-07-10T15:09:06Z mb-benchmark-compile 42ffd3c) +ZJIT dev +PRISM [arm64-darwin24] itr: time #1: 1332ms #2: 49ms #3: 48ms #4: 48ms ```

Add new label for jump_if_cache_miss

33b80a4

tenderlove reviewed Mar 7, 2022

View reviewed changes

WIP: Problem with cache being cleared prematurely resulting in cache …

1b9458d

…miss

wildmaples force-pushed the cache-miss branch from dac157e to 1b9458d Compare March 8, 2022 17:04

Add two simple tests for inlining

82cc74b

tenderlove merged commit 5cac90d into tenderlove:iseq-inline Mar 8, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Cache miss fix + WIP #2

Cache miss fix + WIP #2

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Cache miss fix + WIP #2

Cache miss fix + WIP #2

Uh oh!

Conversation

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!