`iv_count` is a misleading name because when IVs are unset, the new
shape doesn't decrement this value. `next_iv_count` is an accurate, and
more descriptive name.
Previously, we found the current page by rounding the current pointer to
the closest smaller page size. This is incorrect because pages are
relative to the start of the address we reserve. For example, if the
starting address is 12KiB modulo the 16KiB page size, once we have more
than 4KiB of code, calculating with the address would incorrectly give
us page 1 when we're actually still on page 0.
Previously, I can reproduce crashes with:
make btest RUN_OPTS=--yjit-code-page-size=32
on ARM64 macOS, where system page sizes are 16KiB.
This patch makes sure that we're not accidentally reading rb_num_t
instruction arguments as VALUE and accidentally baking them into
code and marking them. Some of these are simply moving the cast earlier,
but some of these avoid potential problems for flag and ID arguments.
Follow-up for 39f7eddec4.
The expandarray instruction interpreters its arguments as rb_num_t.
YJIT was treating the num argument as a VALUE previously and when
it has a certain bit pattern, it can look like a GC pointer. The
argument is not a pointer, so YJIT crashed when trying to mark those
pointers.
This bug existed previously, but our test suite didn't expose it until
f55212bce9. TestArgf#test_to_io has a
line like:
a1, a2, a3, a4, a5, a6, a7, a8 = array
Which maps to an expandarray with an argument of 8. Qnil happened to
be defined as 8, which masked the issue.
Fix it by not using the argument as a VALUE.
Previously, enabling only "disasm" didn't actually build. Since these
two features are closely related and we don't really use one without the
other, let's simplify and merge the two features together.
The commented out instance of free_block() is left over from the port.
The addition in gen_single_block() was a place we missed. The new block
is allocated in the same function and could have invariants associated
with it even though there is no space to hold all the code.
* Implement optimize send in yjit
This successfully makes all our benchmarks exit way less for optimize send reasons.
It makes some benchmarks faster, but not by as much as I'd like. I think this implementation
works, but there are definitely more optimial arrangements. For example, what if we compiled
send to a jump table? That seems like perhaps the most optimal we could do, but not obvious (to me)
how to implement give our current setup.
Co-authored-by: Alan Wu <XrXr@users.noreply.github.com>
* Attempt at fixing the issues raised by @XrXr
* fix allowlist
* returns 0 instead of nil when not found
* remove comment about encoding exception
* Fix up c changes
* Update assert
Co-authored-by: Alan Wu <XrXr@users.noreply.github.com>
* get rid of unneeded code and fix the flags
* Apply suggestions from code review
Co-authored-by: Alan Wu <XrXr@users.noreply.github.com>
* rename and fix typo
Co-authored-by: Alan Wu <XrXr@users.noreply.github.com>
Prior to this commit, we were reading and writing ivar index and
shape ID in inline caches in two separate instructions when
getting and setting ivars. This meant there was a race condition
with ractors and these caches where one ractor could change
a value in the cache while another was still reading from it.
This commit instead reads and writes shape ID and ivar index to
inline caches atomically so there is no longer a race condition.
Co-Authored-By: Aaron Patterson <tenderlove@ruby-lang.org>
Co-Authored-By: John Hawthorn <john@hawthorn.email>
We set the PC in branch_stub_hit(), which only makes sense if we're
running with the intended iseq for the stub. We ran into an issue caught
by this while tweaking code layout.
For logical instructions such as AND, there is a constraint that the N
part of the bitmask immediate must be 0. We weren't respecting this
condition previously and were silently emitting undefined instructions.
Check for this condition in the assembler and tweak the backend to
correctly detect whether a number could be encoded as an immediate in a
32 bit logical instruction. Due to the nature of the immediate encoding,
the same numeric value encodes differently depending on the size of
the register the instruction works on.
We currently don't have cases where we use 32 bit immediates but we ran
into this encoding issue during development.
* YJIT: fix a parameter name
* YJIT: add support for calling bmethods
This commit adds support for the VM_METHOD_TYPE_BMETHOD method type in
YJIT. You can get these type of methods from facilities like
Kernel#define_singleton_method and Module#define_method.
Even though the body of these methods are blocks, the parameter setup
for them is exactly the same as VM_METHOD_TYPE_ISEQ, so we can reuse
the same logic in gen_send_iseq(). You can see this from how
vm_call_bmethod() eventually calls setup_parameters_complex() with
arg_setup_method.
Bmethods do need their frame environment to be setup differently. We
handle this by allowing callers of gen_send_iseq() to control the iseq,
the frame flag, and the prev_ep. The `prev_ep` goes into the same
location as the block handler would go into in an iseq method frame.
Co-authored-by: John Hawthorn <john@hawthorn.email>
Co-authored-by: John Hawthorn <john@hawthorn.email>
Object Shapes is used for accessing instance variables and representing the
"frozenness" of objects. Object instances have a "shape" and the shape
represents some attributes of the object (currently which instance variables are
set and the "frozenness"). Shapes form a tree data structure, and when a new
instance variable is set on an object, that object "transitions" to a new shape
in the shape tree. Each shape has an ID that is used for caching. The shape
structure is independent of class, so objects of different types can have the
same shape.
For example:
```ruby
class Foo
def initialize
# Starts with shape id 0
@a = 1 # transitions to shape id 1
@b = 1 # transitions to shape id 2
end
end
class Bar
def initialize
# Starts with shape id 0
@a = 1 # transitions to shape id 1
@b = 1 # transitions to shape id 2
end
end
foo = Foo.new # `foo` has shape id 2
bar = Bar.new # `bar` has shape id 2
```
Both `foo` and `bar` instances have the same shape because they both set
instance variables of the same name in the same order.
This technique can help to improve inline cache hits as well as generate more
efficient machine code in JIT compilers.
This commit also adds some methods for debugging shapes on objects. See
`RubyVM::Shape` for more details.
For more context on Object Shapes, see [Feature: #18776]
Co-Authored-By: Aaron Patterson <tenderlove@ruby-lang.org>
Co-Authored-By: Eileen M. Uchitelle <eileencodes@gmail.com>
Co-Authored-By: John Hawthorn <john@hawthorn.email>
* Change IncrCounter lowering on AArch64
Previously we were using LDADDAL which is not available on
Graviton 1 chips. Instead, we're going to use an exclusive
load/store group through the LDAXR/STLXR instructions.
* Update yjit/src/backend/arm64/mod.rs
Co-authored-by: Maxime Chevalier-Boisvert <maximechevalierb@gmail.com>
Object Shapes is used for accessing instance variables and representing the
"frozenness" of objects. Object instances have a "shape" and the shape
represents some attributes of the object (currently which instance variables are
set and the "frozenness"). Shapes form a tree data structure, and when a new
instance variable is set on an object, that object "transitions" to a new shape
in the shape tree. Each shape has an ID that is used for caching. The shape
structure is independent of class, so objects of different types can have the
same shape.
For example:
```ruby
class Foo
def initialize
# Starts with shape id 0
@a = 1 # transitions to shape id 1
@b = 1 # transitions to shape id 2
end
end
class Bar
def initialize
# Starts with shape id 0
@a = 1 # transitions to shape id 1
@b = 1 # transitions to shape id 2
end
end
foo = Foo.new # `foo` has shape id 2
bar = Bar.new # `bar` has shape id 2
```
Both `foo` and `bar` instances have the same shape because they both set
instance variables of the same name in the same order.
This technique can help to improve inline cache hits as well as generate more
efficient machine code in JIT compilers.
This commit also adds some methods for debugging shapes on objects. See
`RubyVM::Shape` for more details.
For more context on Object Shapes, see [Feature: #18776]
Co-Authored-By: Aaron Patterson <tenderlove@ruby-lang.org>
Co-Authored-By: Eileen M. Uchitelle <eileencodes@gmail.com>
Co-Authored-By: John Hawthorn <john@hawthorn.email>
* YJIT: Test Rust 1.58.1 as well on Cirrus
* YJIT: Avoid using a Rust 1.60.0 feature
* YJIT: Use autoconf to detect support
* YJIT: We actually need to run it
for checking it properly
* YJIT: Try cfg!(target_feature = "lse")
* Revert "YJIT: Try cfg!(target_feature = "lse")"
This reverts commit 4e2a9ca9a9c83052c23b5e205c91bdf79e88342e.
* YJIT: Add --features stats only when it works
* Update configure.ac
Co-authored-by: Maxime Chevalier-Boisvert <maximechevalierb@gmail.com>
This refactors the "push frame" operation common to both gen_send_iseq
and gen_send_cfunc into its own method. This allows that logic to live
in one place.
* Guard --yjit-stats behind #[cfg(feature = "stats")]
* Only ask for --yjit-stats with dev builds on cirrus CI
* Revert "Only ask for --yjit-stats with dev builds on cirrus CI"
This reverts commit cfb5ddfa4b9394ca240447eee02637788435b02a.
* Make it so the --yjit-stats option works for non-release builds
* Revert accidental changes
This add support for bmake, which should allow building with
`configure --enable-yjit` for the BSDs. Tested on FreeBSD 13 and
on macOS with `configure MAKE=bmake` on a case-sensitive file system.
It works by including a fragment into the Makefile through the configure
script, similar to common.mk. It uses the always rebuild approach to
keep build system changes minimal.
* YJIT: Show --yjit-stats of railsbench on CI
* YJIT: Use --enable-yjit=dev to see ratio_in_yjit
* YJIT: Show master GitHub URL for quick comparison
* YJIT: Avoid making CI red by a yjit-bench failure
Jemma Issroff
Alan Wu
Nobuyoshi Nakada
Takashi Kokubun
Jimmy Miller
Aaron Patterson
John Hawthorn
Maxime Chevalier-Boisvert
Kevin Newton