If a gem is required circular, and there are unresolved specs depending
on it, we may end up in an activation conflict.
The solution is to not try to activate unresolved gems when requiring a
default gem, regardless of it having already been activated or not.
https://github.com/rubygems/rubygems/commit/3b2b8f4e3e
The following conditions must be met:
* A default gem is required.
* A previous require left some gems unresolved, and those dependencies
themselves depend on the default gem.
In this case, rubygems require will first activate the default version
of the gem, then try to activate another unresolved version of the
default gem that conflicts with the first activation.
The solution is, if we are in the middle of requiring a default gem,
skip this step, because we have already activated it successfully.
https://github.com/rubygems/rubygems/commit/8cd5608db5
Co-authored-by: Stan Hu <stanhu@gmail.com>
We don't have support for keyword splat anywhere, but we tried to
compile these anyways in case of `invokeblock`. This led to bad things
happening such as passing the wrong value and passing a hash into
rb_yjit_array_len(), which raised in the middle of compilation.
[Bug #20192]
ASAN leaves a pointer to the fake frame on the stack; we can use the
__asan_addr_is_in_fake_stack API to work out the extent of the fake
stack and thus mark any VALUEs contained therein.
[Bug #20001]
__has_feature is a clang-ism, and GCC has a different way to tell if
sanitizers are enabled. For this reason, I don't want to spray
__has_feature all over the codebase for other places where conditional
compilation based on sanitizers is required.
[Bug #20001]
Where a local variable is used as part of the stack bounds detection, it
has to actually be on the stack. ASAN can put local variable on "fake
stacks", however, with addresses in different memory mappings. This
completely destroys the stack bounds calculation, and can lead to e.g.
things not getting GC marked on the machine stack or stackoverflow
checks that always fail.
The __asan_addr_is_in_fake_stack helper can be used to get the _real_
stack address of such variables, and thus perform the stack size
calculation properly
[Bug #20001]
This commit changes how stack extents are calculated for both the main
thread and other threads. Ruby uses the address of a local variable as
part of the calculation for machine stack extents:
* pthreads uses it as a lower-bound on the start of the stack, because
glibc (and maybe other libcs) can store its own data on the stack
before calling into user code on thread creation.
* win32 uses it as an argument to VirtualQuery, which gets the extent of
the memory mapping which contains the variable
However, the local being used for this is actually too low (too close to
the leaf function call) in both the main thread case and the new thread
case.
In the main thread case, we have the `INIT_STACK` macro, which is used
for pthreads to set the `native_main_thread->stack_start` value. This
value is correctly captured at the very top level of the program (in
main.c). However, this is _not_ what's used to set the execution context
machine stack (`th->ec->machine_stack.stack_start`); that gets set as
part of a call to `ruby_thread_init_stack` in `Init_BareVM`, using the
address of a local variable allocated _inside_ `Init_BareVM`. This is
too low; we need to use a local allocated closer to the top of the
program.
In the new thread case, the lolcal is allocated inside
`native_thread_init_stack`, which is, again, too low.
In both cases, this means that we might have VALUEs lying outside the
bounds of `th->ec->machine.stack_{start,end}`, which won't be marked
correctly by the GC machinery.
To fix this,
* In the main thread case: We already have `INIT_STACK` at the right
level, so just pass that local var to `ruby_thread_init_stack`.
* In the new thread case: Allocate the local one level above the call to
`native_thread_init_stack` in `call_thread_start_func2`.
[Bug #20001]
fix
pm_scope_node_destroy frees the scope node after we're done using it to
make sure that the index_lookup_table is not leaked.
For example:
10.times do
100_000.times do
RubyVM::InstructionSequence.compile_prism("begin; 1; rescue; 2; end")
end
puts `ps -o rss= -p #{$$}`
end
Before:
33056
50304
67776
84544
101520
118448
135712
152352
169136
186656
After:
15264
15296
15408
17040
17152
17152
18320
18352
18400
18608
The temporary array conditions_labels is heap allocated and never freed.
We can use alloca instead to stack allocate it so that we don't need to
manually free it.
For example:
code = "case; #{100.times.map { "when #{it}; " }.join}; end"
10.times do
10_000.times do
RubyVM::InstructionSequence.compile_prism(code)
end
puts `ps -o rss= -p #{$$}`
end
Before:
21376
30304
38800
47184
55456
64192
72288
80400
89040
97104
After:
13088
13632
13760
14016
14688
14992
15120
15232
15744
15744
rb_iseq_compile_prism_node calls both rb_translate_prism and iseq_setup.
Both of these functions call iseq_set_sequence. This means that the first
iseq_set_sequence will leak because the iseq will be overwritten.
For example:
10.times do
100_000.times do
RubyVM::InstructionSequence.compile_prism("")
end
puts `ps -o rss= -p #{$$}`
end
Before:
20528
27328
33840
40208
46400
52960
59168
65600
71888
78352
After:
13696
13712
13712
13712
13712
14352
14352
14992
14992
14992
Hiroshi SHIBATA
git
dependabot[bot]
David Rodríguez
Stan Lo
NAITOH Jun
Sutou Kouhei
Alan Wu
KJ Tsanaktsidis
KJ Tsanaktsidis
Olle Jonsson
Peter Zhu
Matt Valentine-House
Takashi Kokubun
Robert Schulze
Burdette Lamar