We crashed in some edge cases due to the recent change to not compile
encoded iseqs that are larger than `u16::MAX`.
- Match the C signature of rb_yjit_constant_ic_update() and clamp down
to `IseqIdx` size
- Return failure instead of panicking with `unwrap()` in codegen when
the iseq is too large
Co-authored-by: Maxime Chevalier-Boisvert <maxime.chevalierboisvert@shopify.com>
Co-authored-by: Noah Gibbs <noah.gibbs@shopify.com>
So by itself, this shouldn't have been a correctness issue, but we
also pop the stack for block_args. Doing stack manipulation like that
and then side-exiting causes issues. So, while this fixes the
immediate failure, we have a bigger issue with block_args popping and
then exiting that we need to deal with.
This reverts commit 5d0a1ffafa.
This commit is causing sequel in yjit-bench to raise with this stack trace:
```
sequel-5.64.0/lib/sequel/dataset/sql.rb:266:in `literal': wrong argument type Array (expected Proc) (TypeError)
from sequel-5.64.0/lib/sequel/database/misc.rb:269:in `literal'
from sequel-5.64.0/lib/sequel/adapters/shared/sqlite.rb:314:in `column_definition_default_sql'
from sequel-5.64.0/lib/sequel/database/schema_methods.rb:564:in `block in column_definition_sql'
from sequel-5.64.0/lib/sequel/database/schema_methods.rb:564:in `each'
from sequel-5.64.0/lib/sequel/database/schema_methods.rb:564:in `column_definition_sql'
from sequel-5.64.0/lib/sequel/database/schema_methods.rb:634:in `block in column_list_sql'
from sequel-5.64.0/lib/sequel/database/schema_methods.rb:634:in `map'
from sequel-5.64.0/lib/sequel/database/schema_methods.rb:634:in `column_list_sql'
from sequel-5.64.0/lib/sequel/database/schema_methods.rb:753:in `create_table_sql'
from sequel-5.64.0/lib/sequel/adapters/shared/sqlite.rb:348:in `create_table_sql'
from sequel-5.64.0/lib/sequel/database/schema_methods.rb:702:in `create_table_from_generator'
from sequel-5.64.0/lib/sequel/database/schema_methods.rb:203:in `create_table'
from benchmarks/sequel/benchmark.rb:19:in `<main>'
```
* YJIT: Rest and block_arg support
* Update bootstraptest/test_yjit.rb
---------
Co-authored-by: Maxime Chevalier-Boisvert <maximechevalierb@gmail.com>
If you have a method that takes rest arguments and a splat call that
happens to line up perfectly with that rest, you can just dupe the
array rather than move anything around. We still have to dupe, because
people could have a custom to_a method or something like that which
means it is hard to guarantee we have exclusive access to that array.
Example:
```ruby
def foo(a, b, *rest)
end
foo(1, 2, *[3, 4])
```
This patch rewrites Ractor synchronization mechanism, send/receive
and take/yield.
* API
* Ractor::Selector is introduced for lightweight waiting
for many ractors.
* Data structure
* remove `struct rb_ractor_waiting_list` and use
`struct rb_ractor_queue takers_queue` to manage takers.
* remove `rb_ractor_t::yield_atexit` and use
`rb_ractor_t::sync::will_basket::type` to check the will.
* add `rb_ractor_basket::p.take` to represent a taking ractor.
* Synchronization protocol
* For the Ractor local GC, `take` can not make a copy object
directly so ask to generate the copy from the yielding ractor.
* The following steps shows what `r1.take` does on `r0`.
* step1: (r0) register `r0` into `r1`'s takers.
* step2: (r0) check `r1`'s status and wakeup r0 if `r1` is waiting
for yielding a value.
* step3: (r0) sleep until `r1` wakes up `r0`.
* The following steps shows what `Ractor.yield(v)` on `r1`.
* step1: (r1) check first takers of `r1` and if there is (`r0`),
make a copy object of `v` and pass it to `r0` and
wakes up `r0`.
* step2: (r1) if there is no taker ractors, sleep until
another ractor try to take.
Related to:
https://github.com/ruby/ruby/pull/7377
Previously it was believed that there was a problem with a combination
of cfuncs + splat + send, but it turns out the same issue happened
without send. For example `Integer.sqrt(1, *[])`. The issue was
happened not because of send, but because of setting the wrong argc
when we don't need to splat any args.
`make test-spec` revealed this issue after applying an unrelated bug
fix. A crashing case is included, though I suspect there are other
scenarios where it misbehaves. Don't compile for now.
Note that this is *not* an issue on the 3.2.x series; it has
`send_args_splat_non_iseq` which already rejects all splats to cfuncs,
including sends with splats.
Support invokesuper in a block on YJIT
invokesuper previously side exited when it is in a block. To make sure we're compiling the correct method in super, we now use the local environment pointer (LEP) to get the method, which will work in a block.
Co-authored-by: John Hawthorn <john@hawthorn.email>
YJIT: Implement splat for cfuncs. Split exit cases
This also implements a new check for ruby2keywords as the last
argument of a splat. This does mean that we generate more code, but in
actual benchmarks where we gained speed from this (binarytrees) I
don't see any significant slow down. I did have to struggle here with
the register allocator to find code that didn't allocate too many
registers. It's a bit hard when everything is implicit. But I think I
got to the minimal amount of copying and stuff given our current
allocation strategy.
This reverts commit 650a20a3e1.
Now that 3.2.0 is released, let's disable flaky tests. Koichi said he'll
rework Ractor implementation for this, and it has not been done yet.
I noticed this while running test_yjit with --mjit-call-threshold=1,
which redefines `Integer#<`. When Ruby is monkey-patched,
MJIT itself could be broken.
Similarly, Ruby scripts could break MJIT in many different ways. I
prepared the same set of hooks as YJIT so that we could possibly
override it and disable it on those moments. Every constant under
RubyVM::MJIT is private and thus it's an unsupported behavior though.
This would be still flaky, but we want to make sure there's no
YJIT-specific issue when Ruby 3.2 is released. We might skip it again
after the release.
* YJIT: Make case-when optimization respect === redefinition
Even when a fixnum key is in the dispatch hash, if there is a case such
that its basic operations for === is redefined, we need to fall back to
checking each case like the interpreter. Semantically we're always
checking each case by calling === in order, it's just that this is not
observable when basic operations are intact.
When all the keys are fixnums, though, we can do the optimization we're
doing right now. Check for this condition.
* Update yjit/src/cruby_bindings.inc.rs
Co-authored-by: Takashi Kokubun <takashikkbn@gmail.com>
Co-authored-by: Takashi Kokubun <takashikkbn@gmail.com>
Previously we essentially never freed block even after invalidation.
Their reference count never reached zero for a couple of reasons:
1. `Branch::block` formed a cycle with the block holding the branch
2. Strong count on a branch that has ever contained a stub never
reached 0 because we increment the `.clone()` call for
`BranchRef::into_raw()` didn't have a matching decrement.
It's not safe to immediately deallocate blocks during
invalidation since `branch_stub_hit()` can end up
running with a branch pointer from an invalidated branch.
To plug the leaks, we wait until code GC or global invalidation and
deallocate the blocks for iseqs that are definitely not running.
@casperisfine reporting a bug in this gist https://gist.github.com/casperisfine/d59e297fba38eb3905a3d7152b9e9350
After investigating I found it was caused by a combination of send and a c_func that we have overwritten in the JIT. For send calls, we need to do some stack manipulation before making the call. Because of the way exits works, we need to do that stack manipulation at the last possible moment. In this case, we weren't doing that stack manipulation at all. Unfortunately, with how the code is structured there isn't a great place to do that stack manipulation for our overridden C funcs.
Each overridden C func can return a boolean stating that it shouldn't be used. We would need to do the stack manipulation after all of those checks are done. We could pass a lambda(?) or separate out the logic for "can I run this override" from "now generate the code for it". Since we are coming up on a release, I went with the path of least resistence and just decided to not use these overrides if we are in a send call.
We definitely should revist this in the future.
Follow-up for 2b8191bdad. Since that
commit, we stopped doing code invalidation the second time the call and
return events are enabled. We need to do it every time these events are
enabled because we might have generated code while these events are
disabled.
Also rename locals and edit comments to make it more clear that the iseq
rewrite code path only happens the first time a particular iseq trace
event is enabled.
The internal location in ractor.rb is not usefull at all.
```
$ ruby -e 'Ractor.new {}'
<internal:ractor>:267: warning: Ractor is experimental, ...
```
* YJIT: fix a parameter name
* YJIT: add support for calling bmethods
This commit adds support for the VM_METHOD_TYPE_BMETHOD method type in
YJIT. You can get these type of methods from facilities like
Kernel#define_singleton_method and Module#define_method.
Even though the body of these methods are blocks, the parameter setup
for them is exactly the same as VM_METHOD_TYPE_ISEQ, so we can reuse
the same logic in gen_send_iseq(). You can see this from how
vm_call_bmethod() eventually calls setup_parameters_complex() with
arg_setup_method.
Bmethods do need their frame environment to be setup differently. We
handle this by allowing callers of gen_send_iseq() to control the iseq,
the frame flag, and the prev_ep. The `prev_ep` goes into the same
location as the block handler would go into in an iseq method frame.
Co-authored-by: John Hawthorn <john@hawthorn.email>
Co-authored-by: John Hawthorn <john@hawthorn.email>
Object Shapes is used for accessing instance variables and representing the
"frozenness" of objects. Object instances have a "shape" and the shape
represents some attributes of the object (currently which instance variables are
set and the "frozenness"). Shapes form a tree data structure, and when a new
instance variable is set on an object, that object "transitions" to a new shape
in the shape tree. Each shape has an ID that is used for caching. The shape
structure is independent of class, so objects of different types can have the
same shape.
For example:
```ruby
class Foo
def initialize
# Starts with shape id 0
@a = 1 # transitions to shape id 1
@b = 1 # transitions to shape id 2
end
end
class Bar
def initialize
# Starts with shape id 0
@a = 1 # transitions to shape id 1
@b = 1 # transitions to shape id 2
end
end
foo = Foo.new # `foo` has shape id 2
bar = Bar.new # `bar` has shape id 2
```
Both `foo` and `bar` instances have the same shape because they both set
instance variables of the same name in the same order.
This technique can help to improve inline cache hits as well as generate more
efficient machine code in JIT compilers.
This commit also adds some methods for debugging shapes on objects. See
`RubyVM::Shape` for more details.
For more context on Object Shapes, see [Feature: #18776]
Co-Authored-By: Aaron Patterson <tenderlove@ruby-lang.org>
Co-Authored-By: Eileen M. Uchitelle <eileencodes@gmail.com>
Co-Authored-By: John Hawthorn <john@hawthorn.email>
Object Shapes is used for accessing instance variables and representing the
"frozenness" of objects. Object instances have a "shape" and the shape
represents some attributes of the object (currently which instance variables are
set and the "frozenness"). Shapes form a tree data structure, and when a new
instance variable is set on an object, that object "transitions" to a new shape
in the shape tree. Each shape has an ID that is used for caching. The shape
structure is independent of class, so objects of different types can have the
same shape.
For example:
```ruby
class Foo
def initialize
# Starts with shape id 0
@a = 1 # transitions to shape id 1
@b = 1 # transitions to shape id 2
end
end
class Bar
def initialize
# Starts with shape id 0
@a = 1 # transitions to shape id 1
@b = 1 # transitions to shape id 2
end
end
foo = Foo.new # `foo` has shape id 2
bar = Bar.new # `bar` has shape id 2
```
Both `foo` and `bar` instances have the same shape because they both set
instance variables of the same name in the same order.
This technique can help to improve inline cache hits as well as generate more
efficient machine code in JIT compilers.
This commit also adds some methods for debugging shapes on objects. See
`RubyVM::Shape` for more details.
For more context on Object Shapes, see [Feature: #18776]
Co-Authored-By: Aaron Patterson <tenderlove@ruby-lang.org>
Co-Authored-By: Eileen M. Uchitelle <eileencodes@gmail.com>
Co-Authored-By: John Hawthorn <john@hawthorn.email>
* Create code generation func
* Make rb_vm_concat_array available to use in Rust
* Map opcode to code gen func
* Implement code gen for concatarray
* Add test for concatarray
* Use new asm backend
* Add comment to C func wrapper
`YJIT.simulate_oom!` used to leave one byte of space in the code block,
so our test didn't expose a problem with asserting that the write
position is in bounds in `CodeBlock::set_pos`. We do the following when
patching code:
1. save current write position
2. seek to middle of the code block and patch
3. restore old write position
The bounds check fails on (3) when the code block is already filled up.
Leaving one byte of space also meant that when we write that byte, we
need to fill the entire code region with trapping instruction in
`VirtualMem`, which made the OOM tests unnecessarily slow.
Remove the incorrect bounds check and stop leaving space in the code
block when simulating OOM.
* Port gen_send_iseq to the new backend IR
* Replace occurrences of 8 by SIZEOF_VALUE
Co-authored-by: Maxime Chevalier-Boisvert <maximechevalierb@gmail.com>
Co-authored-by: Maxime Chevalier-Boisvert <maxime.chevalierboisvert@shopify.com>
* Port gen_send_cfunc to the new backend
* Remove an obsoleted test
* Add more cfunc tests
* Use csel_e instead and more into()
Co-authored-by: Maxime Chevalier-Boisvert <maximechevalierb@gmail.com>
* Add a missing lea for build_kwargs
* Split cfunc test cases
Co-authored-by: Maxime Chevalier-Boisvert <maximechevalierb@gmail.com>
Teach getblockparamproxy to handle the no-block case without exiting
Co-authored-by: John Hawthorn <john@hawthorn.email>
Co-authored-by: John Hawthorn <john@hawthorn.email>
The test in [1] was removed because it stopped working when we limited
the power of Kernel#binding in [2]. However, the underlying issue could
still be reproduced using blocks. Add back a regression test.
I tested the test by commenting out the fix from [1].
[1]: 54c91042ed
[2]: 343ea9967e
In December 2021, we opened an [issue] to solicit feedback regarding the
porting of the YJIT codebase from C99 to Rust. There were some
reservations, but this project was given the go ahead by Ruby core
developers and Matz. Since then, we have successfully completed the port
of YJIT to Rust.
The new Rust version of YJIT has reached parity with the C version, in
that it passes all the CRuby tests, is able to run all of the YJIT
benchmarks, and performs similarly to the C version (because it works
the same way and largely generates the same machine code). We've even
incorporated some design improvements, such as a more fine-grained
constant invalidation mechanism which we expect will make a big
difference in Ruby on Rails applications.
Because we want to be careful, YJIT is guarded behind a configure
option:
```shell
./configure --enable-yjit # Build YJIT in release mode
./configure --enable-yjit=dev # Build YJIT in dev/debug mode
```
By default, YJIT does not get compiled and cargo/rustc is not required.
If YJIT is built in dev mode, then `cargo` is used to fetch development
dependencies, but when building in release, `cargo` is not required,
only `rustc`. At the moment YJIT requires Rust 1.60.0 or newer.
The YJIT command-line options remain mostly unchanged, and more details
about the build process are documented in `doc/yjit/yjit.md`.
The CI tests have been updated and do not take any more resources than
before.
The development history of the Rust port is available at the following
commit for interested parties:
1fd9573d8b
Our hope is that Rust YJIT will be compiled and included as a part of
system packages and compiled binaries of the Ruby 3.2 release. We do not
anticipate any major problems as Rust is well supported on every
platform which YJIT supports, but to make sure that this process works
smoothly, we would like to reach out to those who take care of building
systems packages before the 3.2 release is shipped and resolve any
issues that may come up.
[issue]: https://bugs.ruby-lang.org/issues/18481
Co-authored-by: Maxime Chevalier-Boisvert <maximechevalierb@gmail.com>
Co-authored-by: Noah Gibbs <the.codefolio.guy@gmail.com>
Co-authored-by: Kevin Newton <kddnewton@gmail.com>
Check whether the current or previous frame is a Ruby frame in
call_trace_func and rb_tracearg_binding before attempting to
create a binding for the frame.
Fixes [Bug #18487]
Co-authored-by: Alan Wu <XrXr@users.noreply.github.com>
This commit reintroduces finer-grained constant cache invalidation.
After 8008fb7 got merged, it was causing issues on token-threaded
builds (such as on Windows).
The issue was that when you're iterating through instruction sequences
and using the translator functions to get back the instruction structs,
you're either using `rb_vm_insn_null_translator` or
`rb_vm_insn_addr2insn2` depending if it's a direct-threading build.
`rb_vm_insn_addr2insn2` does some normalization to always return to
you the non-trace version of whatever instruction you're looking at.
`rb_vm_insn_null_translator` does not do that normalization.
This means that when you're looping through the instructions if you're
trying to do an opcode comparison, it can change depending on the type
of threading that you're using. This can be very confusing. So, this
commit creates a new translator function
`rb_vm_insn_normalizing_translator` to always return the non-trace
version so that opcode comparisons don't have to worry about different
configurations.
[Feature #18589]
This reverts commits for [Feature #18589]:
* 8008fb7352
"Update formatting per feedback"
* 8f6eaca2e1
"Delete ID from constant cache table if it becomes empty on ISEQ free"
* 629908586b
"Finer-grained inline constant cache invalidation"
MSWin builds on AppVeyor have been crashing since the merger.
Check whether the current or previous frame is a Ruby frame in
call_trace_func before attempting to create a binding for the frame.
Fixes [Bug #18487]
Co-authored-by: Alan Wu <XrXr@users.noreply.github.com>
Current behavior - caches depend on a global counter. All constant mutations cause caches to be invalidated.
```ruby
class A
B = 1
end
def foo
A::B # inline cache depends on global counter
end
foo # populate inline cache
foo # hit inline cache
C = 1 # global counter increments, all caches are invalidated
foo # misses inline cache due to `C = 1`
```
Proposed behavior - caches depend on name components. Only constant mutations with corresponding names will invalidate the cache.
```ruby
class A
B = 1
end
def foo
A::B # inline cache depends constants named "A" and "B"
end
foo # populate inline cache
foo # hit inline cache
C = 1 # caches that depend on the name "C" are invalidated
foo # hits inline cache because IC only depends on "A" and "B"
```
Examples of breaking the new cache:
```ruby
module C
# Breaks `foo` cache because "A" constant is set and the cache in foo depends
# on "A" and "B"
class A; end
end
B = 1
```
We expect the new cache scheme to be invalidated less often because names aren't frequently reused. With the cache being invalidated less, we can rely on its stability more to keep our constant references fast and reduce the need to throw away generated code in YJIT.
Backticks method invokes `/bin/sh` when the command contains
quotes, and `sh` clears some environment variables set in
runruby.rb to search the built shared library.
* `-j` option for concurrent test with threads
* `-jN` uses N threads
* `-j` uses nproc/2 threads
* Introduce `BT` struct to manage configurations
* Introduce `Assertion` to manage all assertions
* Remove all toplevel instance variables
* Show elapsed seconds at last
```
$ time make btest
...
real 0m37.319s
user 0m26.221s
sys 0m16.534s
$ time make btest TESTOPTS=-j
...
real 0m11.812s
user 0m36.667s
sys 0m21.872s
```
Jimmy Miller
Ole Friis Østergaard
Alan Wu
Peter Zhu
Takashi Kokubun
Nobuyoshi Nakada
Koichi Sasada
Maxime Chevalier-Boisvert
Maple Ong
Benoit Daloze
Matthew Draper
Jemma Issroff
Aaron Patterson
John Hawthorn
Hiroshi SHIBATA
Maple Ong
Zack Deveau
Matt Valentine-House
Noah Gibbs (and/or Benchmark CI)
Alan Wu
Jeremy Evans
Kevin Newton