btest can't be used for testing RJIT because RJIT doesn't work on
miniruby. However, btest-ruby is not necessarily useful for testing RJIT
because both the runner could crash as well as the target.
TBH I'm not sure why we want to use RUNRUBY instead of BOOTSTRAPRUBY on
btest-ruby. However, to achieve what I want to do while keeping the
current behavior, I'm just introducing a new target.
These classes don't belong in gc.c as they're not actually part of the
GC. This commit refactors the code by moving all the code into a
weakmap.c file.
We used to require MJIT is supported when YJIT is supported. However,
now that RJIT dropped some platforms that YJIT supports, it no longer
makes sense. We should be able to enable only YJIT, and vice versa.
This patch rewrites Ractor synchronization mechanism, send/receive
and take/yield.
* API
* Ractor::Selector is introduced for lightweight waiting
for many ractors.
* Data structure
* remove `struct rb_ractor_waiting_list` and use
`struct rb_ractor_queue takers_queue` to manage takers.
* remove `rb_ractor_t::yield_atexit` and use
`rb_ractor_t::sync::will_basket::type` to check the will.
* add `rb_ractor_basket::p.take` to represent a taking ractor.
* Synchronization protocol
* For the Ractor local GC, `take` can not make a copy object
directly so ask to generate the copy from the yielding ractor.
* The following steps shows what `r1.take` does on `r0`.
* step1: (r0) register `r0` into `r1`'s takers.
* step2: (r0) check `r1`'s status and wakeup r0 if `r1` is waiting
for yielding a value.
* step3: (r0) sleep until `r1` wakes up `r0`.
* The following steps shows what `Ractor.yield(v)` on `r1`.
* step1: (r1) check first takers of `r1` and if there is (`r0`),
make a copy object of `v` and pass it to `r0` and
wakes up `r0`.
* step2: (r1) if there is no taker ractors, sleep until
another ractor try to take.
If the previous instruction is not a leaf instruction, then the PC was
incremented before the instruction was ran (meaning the currently
executing instruction is actually the previous instruction), so we
should not increment the PC otherwise we will calculate the source
line for the next instruction.
This bug can be reproduced in the following script:
```
require "objspace"
ObjectSpace.trace_object_allocations_start
a =
1.0 / 0.0
p [ObjectSpace.allocation_sourceline(a), ObjectSpace.allocation_sourcefile(a)]
```
Which outputs: [4, "test.rb"]
This is incorrect because the object was allocated on line 10 and not
line 4. The behaviour is correct when we use a leaf instruction (e.g.
if we replaced `1.0 / 0.0` with `"hello"`), then the output is:
[10, "test.rb"].
[Bug #19456]
Right now the attached object is stored as an instance variable
and all the call sites that either get or set it have to know how it's
stored.
It's preferable to hide this implementation detail behind accessors
so that it is easier to change how it's stored.
Rust 1.58.0 unfortunately doesn't provide facilities to control symbol
visibility/presence, but we care about controlling the list of
symbols exported from libruby-static.a and libruby.so.
This commit uses `ld -r` to make a single object out of rustc's
staticlib output, libyjit.a. This moves libyjit.a out of MAINLIBS and adds
libyjit.o into COMMONOBJS, which obviates the code for merging libyjit.a
into libruby-static.a. The odd appearance of libyjit.a in SOLIBS is also
gone.
To filter out symbols we do not want to export on ELF platforms, we use
objcopy after the partial link. On darwin, we supply a symbol list to
the linker which takes care of hiding unprefixed symbols.
[Bug #19255]
Co-authored-by: Nobuyoshi Nakada <nobu@ruby-lang.org>
This reduces the code size of libyjit.a by a lot. On darwin it went from
23 MiB to 12 MiB for me. I chose ThinLTO over fat LTO for the relatively
fast build time; in case we need to debug release-build-only problems
it won't be painful.
Ignoring the following messages:
```
(snip)
-e 'load "spec/bundler/support/bundle.rb"' -- install --gemfile=tool/bundler/dev_gems.rb
Using rake 13.0.6
Using bundler 2.5.0.dev
Using diff-lcs 1.5.0
Using parallel 1.22.1
Using parallel_tests 2.32.0
Using power_assert 2.0.2
Using rb_sys 0.9.52
Using rspec-support 3.12.0
Using rspec-core 3.12.0
Using rspec-expectations 3.12.0
Using rspec-mocks 3.12.1
Using test-unit 3.5.5
Using uri 0.12.0
Using webrick 1.7.0
Bundle complete! 11 Gemfile dependencies, 14 gems now installed.
Gems in the groups 'lint' and 'doc' were not installed.
Use `bundle info [gemname]` to see where a bundled gem is installed.
```
With --enable-yjit, you see an annoying warning like this:
warning: Missing auto-load script at offset 0 in section .debug_gdb_scripts
of file /home/k0kubun/src/github.com/ruby/ruby/.ruby/miniruby.
Use `info auto-load python-scripts [REGEXP]' to list them.
Using `rust-gdb` instead fixes it. I use this like `make gdb GDB=rust-gdb`.
If baseruby is available (and its version is different from one being
built) when compiling ruby, tool/outdate-bundled-gems.rb (which is
invoked by `make install`) wrongly deletes debug.so and rbs_extension.so
in .bundle/extension/*.
This leads to a broken installation of ruby which lacks the libraries,
which may make rubygems show the following warnings (in some additional
complex conditions):
```
$ irb
Ignoring debug-1.7.1 because its extensions are not built. Try: gem pristine debug --version 1.7.1
Ignoring rbs-2.8.2 because its extensions are not built. Try: gem pristine rbs --version 2.8.2
irb(main):001:0>
```
According to some committers, tool/outdate-bundled-gems.rb is introduced
for fixing a build issue, but the detail is not recorded. The issue
seems to occur only when debug gem or rbs gem is updated, so it is
difficult to fix the script so soon.
Tentatively, this change stops invoking the script by default.
This should be backported to ruby_3_2.
Fixes [Bug #19271]
I noticed this while running test_yjit with --mjit-call-threshold=1,
which redefines `Integer#<`. When Ruby is monkey-patched,
MJIT itself could be broken.
Similarly, Ruby scripts could break MJIT in many different ways. I
prepared the same set of hooks as YJIT so that we could possibly
override it and disable it on those moments. Every constant under
RubyVM::MJIT is private and thus it's an unsupported behavior though.
Matt Valentine-House
Takashi Kokubun
Peter Zhu
Koichi Sasada
Kaíque Kandy Koga
Nobuyoshi Nakada
Jean Boussier
Alan Wu
Hiroshi SHIBATA
Yusuke Endoh