Initialize VM stack if VM_CHECK_MODE

XrXr · XrXr · commit def7023ee4a3 · 2024-03-15T19:15:58.000-04:00
Lately there has been a few flaky YJIT CI failures where a new Ruby thread is finding the canary on the VM stack. For example: https://github.com/ruby/ruby/actions/runs/8287357784/job/22679508482#step:14:109 After checking a local rr recording, it's clear that the canary was written there when YJIT was using a temporary malloc region, and then later handed to the new Ruby thread. Previously, the VM stack was uninitialized, so it can have stale values in it, like the canary. Though unlikely, this can happen without YJIT too. Initialize the stack if we're spawning canaries.
diff --git a/vm.c b/vm.c
@@ -3560,6 +3560,10 @@ rb_ec_initialize_vm_stack(rb_execution_context_t *ec, VALUE *stack, size_t size)
 {
     rb_ec_set_vm_stack(ec, stack, size);
 
+#if VM_CHECK_MODE > 0
+    MEMZERO(stack, VALUE, size); // malloc memory could have the VM canary in it
+#endif
+
     ec->cfp = (void *)(ec->vm_stack + ec->vm_stack_size);
 
     vm_push_frame(ec,

Original file line number	Diff line number	Diff line change
`@@ -3560,6 +3560,10 @@ rb_ec_initialize_vm_stack(rb_execution_context_t ec, VALUE stack, size_t size)`
`3560`	`3560`	`{`
`3561`	`3561`	`rb_ec_set_vm_stack(ec, stack, size);`
`3562`	`3562`
	`3563`	`+#if VM_CHECK_MODE > 0`
	`3564`	`+ MEMZERO(stack, VALUE, size); // malloc memory could have the VM canary in it`
	`3565`	`+#endif`
	`3566`	`+`
`3563`	`3567`	`ec->cfp = (void *)(ec->vm_stack + ec->vm_stack_size);`
`3564`	`3568`
`3565`	`3569`	`vm_push_frame(ec,`