This patch rewrites Ractor synchronization mechanism, send/receive
and take/yield.
* API
* Ractor::Selector is introduced for lightweight waiting
for many ractors.
* Data structure
* remove `struct rb_ractor_waiting_list` and use
`struct rb_ractor_queue takers_queue` to manage takers.
* remove `rb_ractor_t::yield_atexit` and use
`rb_ractor_t::sync::will_basket::type` to check the will.
* add `rb_ractor_basket::p.take` to represent a taking ractor.
* Synchronization protocol
* For the Ractor local GC, `take` can not make a copy object
directly so ask to generate the copy from the yielding ractor.
* The following steps shows what `r1.take` does on `r0`.
* step1: (r0) register `r0` into `r1`'s takers.
* step2: (r0) check `r1`'s status and wakeup r0 if `r1` is waiting
for yielding a value.
* step3: (r0) sleep until `r1` wakes up `r0`.
* The following steps shows what `Ractor.yield(v)` on `r1`.
* step1: (r1) check first takers of `r1` and if there is (`r0`),
make a copy object of `v` and pass it to `r0` and
wakes up `r0`.
* step2: (r1) if there is no taker ractors, sleep until
another ractor try to take.
If the previous instruction is not a leaf instruction, then the PC was
incremented before the instruction was ran (meaning the currently
executing instruction is actually the previous instruction), so we
should not increment the PC otherwise we will calculate the source
line for the next instruction.
This bug can be reproduced in the following script:
```
require "objspace"
ObjectSpace.trace_object_allocations_start
a =
1.0 / 0.0
p [ObjectSpace.allocation_sourceline(a), ObjectSpace.allocation_sourcefile(a)]
```
Which outputs: [4, "test.rb"]
This is incorrect because the object was allocated on line 10 and not
line 4. The behaviour is correct when we use a leaf instruction (e.g.
if we replaced `1.0 / 0.0` with `"hello"`), then the output is:
[10, "test.rb"].
[Bug #19456]
Right now the attached object is stored as an instance variable
and all the call sites that either get or set it have to know how it's
stored.
It's preferable to hide this implementation detail behind accessors
so that it is easier to change how it's stored.
Rust 1.58.0 unfortunately doesn't provide facilities to control symbol
visibility/presence, but we care about controlling the list of
symbols exported from libruby-static.a and libruby.so.
This commit uses `ld -r` to make a single object out of rustc's
staticlib output, libyjit.a. This moves libyjit.a out of MAINLIBS and adds
libyjit.o into COMMONOBJS, which obviates the code for merging libyjit.a
into libruby-static.a. The odd appearance of libyjit.a in SOLIBS is also
gone.
To filter out symbols we do not want to export on ELF platforms, we use
objcopy after the partial link. On darwin, we supply a symbol list to
the linker which takes care of hiding unprefixed symbols.
[Bug #19255]
Co-authored-by: Nobuyoshi Nakada <nobu@ruby-lang.org>
This reduces the code size of libyjit.a by a lot. On darwin it went from
23 MiB to 12 MiB for me. I chose ThinLTO over fat LTO for the relatively
fast build time; in case we need to debug release-build-only problems
it won't be painful.
Ignoring the following messages:
```
(snip)
-e 'load "spec/bundler/support/bundle.rb"' -- install --gemfile=tool/bundler/dev_gems.rb
Using rake 13.0.6
Using bundler 2.5.0.dev
Using diff-lcs 1.5.0
Using parallel 1.22.1
Using parallel_tests 2.32.0
Using power_assert 2.0.2
Using rb_sys 0.9.52
Using rspec-support 3.12.0
Using rspec-core 3.12.0
Using rspec-expectations 3.12.0
Using rspec-mocks 3.12.1
Using test-unit 3.5.5
Using uri 0.12.0
Using webrick 1.7.0
Bundle complete! 11 Gemfile dependencies, 14 gems now installed.
Gems in the groups 'lint' and 'doc' were not installed.
Use `bundle info [gemname]` to see where a bundled gem is installed.
```
With --enable-yjit, you see an annoying warning like this:
warning: Missing auto-load script at offset 0 in section .debug_gdb_scripts
of file /home/k0kubun/src/github.com/ruby/ruby/.ruby/miniruby.
Use `info auto-load python-scripts [REGEXP]' to list them.
Using `rust-gdb` instead fixes it. I use this like `make gdb GDB=rust-gdb`.
If baseruby is available (and its version is different from one being
built) when compiling ruby, tool/outdate-bundled-gems.rb (which is
invoked by `make install`) wrongly deletes debug.so and rbs_extension.so
in .bundle/extension/*.
This leads to a broken installation of ruby which lacks the libraries,
which may make rubygems show the following warnings (in some additional
complex conditions):
```
$ irb
Ignoring debug-1.7.1 because its extensions are not built. Try: gem pristine debug --version 1.7.1
Ignoring rbs-2.8.2 because its extensions are not built. Try: gem pristine rbs --version 2.8.2
irb(main):001:0>
```
According to some committers, tool/outdate-bundled-gems.rb is introduced
for fixing a build issue, but the detail is not recorded. The issue
seems to occur only when debug gem or rbs gem is updated, so it is
difficult to fix the script so soon.
Tentatively, this change stops invoking the script by default.
This should be backported to ruby_3_2.
Fixes [Bug #19271]
I noticed this while running test_yjit with --mjit-call-threshold=1,
which redefines `Integer#<`. When Ruby is monkey-patched,
MJIT itself could be broken.
Similarly, Ruby scripts could break MJIT in many different ways. I
prepared the same set of hooks as YJIT so that we could possibly
override it and disable it on those moments. Every constant under
RubyVM::MJIT is private and thus it's an unsupported behavior though.
SIZE_POOL_COUNT is a GC macro, it should belong in gc.h and not shape.h.
SIZE_POOL_COUNT doesn't depend on shape.h so we can have shape.h depend
on gc.h.
Co-Authored-By: Matt Valentine-House <matt@eightbitraptor.com>
* YJIT: Generate debug info in release builds
They are helpful in case we need to do core dump debugging.
* Remove Cirrus DOC skip rule
The syntax for this is weird, and escaping [ and ] cause parse failures.
Cirrus' docs said to surround with .*, but then that seems to skip
everything. Revert e0a4205eb7 for now.
I see several arguments in doing so.
First they use a non trivial amount of memory, so for various memory
profiling/mapping tools it is relevant to have visibility of the space
occupied by shapes.
Then, some pathological code can create a tons of shape, so it is
valuable to have a way to have a way to observe shapes without having
to compile Ruby with `SHAPE_DEBUG=1`.
And additionally it's likely much faster to dump then this way than
to use `RubyVM::Shape`.
There are however a few open questions:
- Shapes can't respect the `since:` argument. Not sure what to do when
it is provided. Would probably make sense to not dump them.
- Maybe it would make more sense to have a separate `ObjectSpace.dump_shapes`?
- Maybe instead `dump_all` should take a `shapes: false` argument?
Additionally, `ObjectSpace.dump_shapes` is added for the use case of
debugging the evolution of the shape tree.
Cases like this:
```ruby
obj = Object.new
loop do
obj.instance_variable_set(:@foo, 1)
obj.remove_instance_variable(:@foo)
end
```
can cause us to use many more shapes than we want (and even run out).
This commit changes the code such that when an instance variable is
removed, we'll walk up the shape tree, find the shape, then rebuild any
child nodes that happened to be below the "targetted for removal" IV.
This also requires moving any instance variables so that indexes derived
from the shape tree will work correctly.
Co-Authored-By: Jemma Issroff <jemmaissroff@gmail.com>
Co-authored-by: John Hawthorn <jhawthorn@github.com>
Koichi Sasada
Matt Valentine-House
Peter Zhu
Kaíque Kandy Koga
Nobuyoshi Nakada
Jean Boussier
Takashi Kokubun
Alan Wu
Hiroshi SHIBATA
Yusuke Endoh
Benoit Daloze
Aaron Patterson
Daniel Colson