The code path for leave that returns to the interpreter
(gen_leave() -> yjit_gen_leave_exit()) used to have the logic:
```
cfp->sp++;
cfp->sp[-1] = return_val;
cfp->sp--;
return return_val;
```
The SP changes it made was unnecessary and this change removes it.
After this change, `leave` doesn't adjust the `cfp->sp` of the caller
and only writes `cfp->sp[0]`. To accomodate this in the JIT-to-JIT
return case, return stubs have an `sp_offset` of 1.
The change removes sp adjustment from the JIT-to-JIT return case, too,
making it more efficient. Also, since the C method case of `send`
has an `sp_offset` of 1 after the call, this change enables block
version sharing.
Previously checktype only supported heap objects, however it's not
uncommon to receive an immediate, for example when string interpolating
a Symbol or Integer.
This change fixes some cases where YJIT fails to fire tracing events.
Most of the situations YJIT did not handle correctly involves enabling
tracing while running inside generated code.
A new operation to invalidate all generated code is added, which uses
patching to make generated code exit at the next VM instruction
boundary. A new routine called `jit_prepare_routine_call()` is
introduced to facilitate this and should be used when generating code
that could allocate, or could otherwise use `RB_VM_LOCK_ENTER()`.
The `c_return` event is fired in the middle of an instruction as opposed
to at an instruction boundary, so it requires special handling. C method
call return points are patched to go to a fucntion which does everything
the interpreter does, including firing the `c_return` event. The
generated code for C method calls normally does not fire the event.
Invalided code should not change after patching so the exits are not
clobbered. A new variable is introduced to track the region of code that
should not change.
RUBY_DEBUG have a very significant performance overhead. Enough that
YJIT with RUBY_DEBUG is noticeably slower than the interpreter without
RUBY_DEBUG.
This makes it hard to collect yjit-stats in production environments.
By allowing to collect JIT statistics without the RUBy_DEBUG overhead,
I hope to make such use cases smoother.
The FIXME is there so we remember to investigate why insns clears the
temporary array. Is this necessary? If it's not we can remove it from
both.
Co-authored-by: Aaron Patterson <tenderlove@ruby-lang.org>
Adds yjit support for setting global variables.
Co-authored-by: Aaron Patterson <tenderlove@ruby-lang.org>
Co-authored-by: John Hawthorn <john@hawthorn.email>
Methods with optional parameters don't always start executing at the
first PC, but we compile all methods assuming that they do. This commit
adds a guard to ensure that we're actually starting at the first PC for
methods with optional params
Always using `ret` to return to the interpreter means that we never have
to check the VM_FRAME_FLAG_FINISH flag.
In the case that we return `Qundef`, the interpreter will execute the
cfp. We can take advantage of this by setting the PC to the instruction
we can't handle, and let the interpreter pick up the ball from there.
If we return a value other than Qundef, the interpreter will take that
value as the "return value" from the JIT and push that to the SP of the
caller
The leave instruction puts the return value on the top of the calling
frame's stack. YJIT does the same thing for leave instructions.
However, when we're returning back to the interpreter, the leave
instruction _should not_ put the return value on the top of the stack,
but put it in RAX and use RET. This commit pops the last value from the
stack pointer and puts it in RAX so that the interpreter is happy with
SP.
In jit_guard_known_klass whenever we encounter a new class should
recompile the current instruction.
However, previously once jit_guard_known_klass had guarded for a heap
object it would not recompile for any immediate (special const) objects
arriving afterwards and would take a plain side-exit instead of a chain
guard.
This commit uses jit_chain_guard inside jit_guard_known_klass instead of
the plain side exit, so that we can recompile for any special constants
arriving afterwards.
We recently added the ability to optimize a known cfunc with custom code
generation for it.
Previously we performed this lookup with a switch statement on the
address of the c function being called. This commit swaps that out for a
hash lookup on the method definition. For now I've kept this limited
this to cfuncs, but it wouldn't take significant changes to make this
work for other method types.
This implemenation is similar to how the interpreter keeps track of
which BOPs (basic operations) are redefined
This has a few advantages:
- Doesn't the C function's symbol to be exported (they're often static)
- This could support VM_METHOD_TYPE_OPTIMIZED in the future.
- This could support VM_METHOD_TYPE_ISEQ in the future. Kernel#class
would be a good candidate for this since to yjit it will just be a
constant push as we already know the class through BBV.
- Slightly cleaner to declare
- Less tightly coupled to each method's implementation
And a couple minor trade-offs:
- The looser coupling could be seen as a disadvantage (I don't think so,
- If a cfunc is defined multiple times we would need to declare it on
each definition. ex. BasicObject#== and BasicObject#equal?. This is
rare compared to using an alias.
As an optimization, multiple objects could share the same singleton
class. The optimization introduced in 6698e43393
wasn't handling this correctly so was generating guards that never pass
for the inputs we defer compilation to wait for. After generating
identical code multiple times and failing, the call site is falsely
recognized as megamorphic and it side exits. See disassembly for the
following before this commit:
def foo(obj)
obj.itself
end
o = Object.new.singleton_class
foo(o)
puts YJIT.disasm(method(:foo))
See also: comment in rb_singleton_class_clone_and_attach().
Singleton classes should only ever be attached to one object. This means
that checking for the object should be the same as checking for the
class. This should be slightly faster by avoiding one memory acccess as
well as allowing us to skip checking if the receiver is a heap object.
This will be most common for calling class methods.
jit_rb_obj_not() wants to access the type information of the receiver,
but we were discarding type info of locals before jit_rb_obj_not() runs
unncessarily.
There are also cases we are unncessarily discarding local type info. For
example, ivar reader and setter methods can never change local
variables.
This commit improves opt_not by making it correct when TrueClass#!
and/or FalseClass#! is defined and genearting better code when the
receiver is a heap object.
guard_known_class() can now handle true, false, and nil, and we
introduce a codegen function reimplementing rb_obj_not(), used when we
know we are calling into rb_obj_not().
Co-authored-by: Maxime Chevalier-Boisvert <maximechevalierb@gmail.com>
Co-authored-by: Noah Gibbs <the.codefolio.guy@gmail.com>
invokebuiltin_delegate can be run in place of
invokebuiltin_delegate_leave because there is always a leave instruction
afterwards (the interpreter takes advantage of this as well when
rewriting iseqs for tracing).
invokebuiltin_delegate is a special version of invokebuiltin used for
sending a contiguous subset of the current method's locals.
In some cases YJIT would already handle this for trivial cases it could
be inlined, implementing this OP allows it to work when the method isn't
inlinable (not marked as 'inline', does more than just call, not called
from yjit, etc).
This commit implements the topn instruction
Co-Authored-By: Maxime Chevalier-Boisvert <maxime.chevalierboisvert@shopify.com>
Co-Authored-By: Noah Gibbs <noah.gibbs@shopify.com>
* Use builtin_inline_p to skip a frame of C methods
* Fix bugs in primitive cfunc call code
* Remove if (push_frame) {}
* Remove if (push_frame) {}
* Push Aaron's fix to avoid hardcoding insn lengths
Co-authored-by: Takashi Kokubun <takashikkbn@gmail.com>
* Implement gen_newarray
* Implement newhash for n=0
* Add yjit tests for newhash/newarray
* Fix integer size warning on clang
* Save PC and SP in newhash and newarray
Co-authored-by: Maxime Chevalier-Boisvert <maximechevalierb@gmail.com>
Previously this could crash on Nokogiri when JITing the getivar
instruction because we would attempt to treat Nokogiri::XML::Document's
T_DATA as a T_OBJECT in calling rb_iv_index_tbl_lookup.
This commit also checks for T_OBJECT at compile time and emits the
rb_ivar_get fallback in that case.
Co-authored-by: HParker <HParker@github.com>
Co-authored-by: Dinah Shi <dinahshi@github.com>
Co-authored-by: HParker <HParker@github.com>
Co-authored-by: Dinah Shi <dinahshi@github.com>
* Implement getblockparamproxy
* Parallel runner: wait for timeout thread to terminate after killing
Or else the leak cheaker could sees the thread as running and cause test
failures in test-tool.
* Add a comment, use jne
* Comment about where 0x3 comes from
* Implement send with blocks
Not that much extra work compared to `opt_send_without_block`.
Moved the stack over flow check because it could've exited after changes
are made to cfp.
* rename oswb counters
* Might as well implement sending block to cfuncs
* Disable sending blocks to cfuncs for now
* Reconstruct interpreter sp before calling into cfuncs
In case the callee cfunc calls a method or delegates to a block.
This also has the side benefit of letting call sites that sometimes are
iseq calls and sometimes cfunc call share the same successor.
* only sync with interpreter sp when passing a block
Co-authored-by: Maxime Chevalier-Boisvert <maximechevalierb@gmail.com>
Co-authored-by: Aaron Patterson <aaron.patterson@shopify.com>
* Use rb_ivar_get() for general case of getivar
Pretty straight forward. Buys about 1% coverage on railsbench.
* Update yjit_codegen.c
Co-authored-by: Maxime Chevalier-Boisvert <maximechevalierb@gmail.com>
* Implement calls to methods with simple optional params
* Remove unnecessary MJIT_STATIC
See comment for MJIT_STATIC. I added it not knowing whether it's
required because the function next to it has it. Don't use it and wait
for problems to come up instead.
* Better naming, some comments
* Count bailing on kw only iseqs
On railsbench:
```
opt_send_without_block exit reasons:
bmethod 59729 (27.7%)
optimized_method 59137 (27.5%)
iseq_complex_callee 41362 (19.2%)
alias_method 33346 (15.5%)
callsite_not_simple 19170 ( 8.9%)
iseq_only_keywords 1300 ( 0.6%)
kw_splat 1299 ( 0.6%)
cfunc_ruby_array_varg 18 ( 0.0%)
```
Make sure `opt_getinlinecache` is in a block all on its own, and
invalidate it from the interpreter when `opt_setinlinecache`.
It will recompile with a filled cache the second time around.
This lets YJIT runs well when the IC for constant is cold.
Introduce a new macro `ADD_COMMENT(cb, comment)` that records a comment
for the current write position in the code block.
Co-authored-by: Maxime Chevalier-Boisvert <maximechevalierb@gmail.com>
Co-authored-by: Aaron Patterson <aaron.patterson@shopify.com>
We didn't need the private check before because we were lifting from the
interpreter's cache, and the interpreter only caches when visibility
checks go through.
* YJIT: unify exits. Patch iseqs only when necessary
This fixes the gotcha that returning YJIT_CANT_COPMILE for an
instruction at entry position leading to infinite loop.
Also, iseq patching is only done only when necessary, which should make
most exits faster.
* Now that exits are the same, return YJIT_CANT_COMPILE
Lazily compile out a chain of checks for different known classes and
whether `self` embeds its ivars or not.
* Remove trailing whitespaces
* Get proper addresss in Capstone disassembly
* Lowercase address in Capstone disassembly
Capstone uses lowercase for jump targets in generated listings. Let's
match it.
* Use the same successor in getivar guard chains
Cuts down on duplication
* Address reviews
* Fix copypasta error
* Add a comment
When a BOP is redefined, the BOP redefinition callback will invalidate
any blocks that depend on BOPS. This allows us to eliminate runtime
checks for BOP redefinition.
Alan Wu
John Hawthorn
Maxime Chevalier-Boisvert
Jean Boussier
eileencodes
Aaron Patterson
Kevin Newton
Kevin Deisz
Maxime Chevalier-Boisvert
Jose Narvaez