Граф коммитов

622 Коммитов

Автор SHA1 Сообщение Дата
Nobuyoshi Nakada 010d92e93d
Adjust indent [ci skip] 2022-04-02 14:54:37 +09:00
Kevin Newton 6068da8937 Finer-grained constant cache invalidation (take 2)
This commit reintroduces finer-grained constant cache invalidation.
After 8008fb7 got merged, it was causing issues on token-threaded
builds (such as on Windows).

The issue was that when you're iterating through instruction sequences
and using the translator functions to get back the instruction structs,
you're either using `rb_vm_insn_null_translator` or
`rb_vm_insn_addr2insn2` depending if it's a direct-threading build.
`rb_vm_insn_addr2insn2` does some normalization to always return to
you the non-trace version of whatever instruction you're looking at.
`rb_vm_insn_null_translator` does not do that normalization.

This means that when you're looping through the instructions if you're
trying to do an opcode comparison, it can change depending on the type
of threading that you're using. This can be very confusing. So, this
commit creates a new translator function
`rb_vm_insn_normalizing_translator` to always return the non-trace
version so that opcode comparisons don't have to worry about different
configurations.

[Feature #18589]
2022-04-01 14:48:22 -04:00
Nobuyoshi Nakada 69967ee64e
Revert "Finer-grained inline constant cache invalidation"
This reverts commits for [Feature #18589]:
* 8008fb7352
  "Update formatting per feedback"
* 8f6eaca2e1
  "Delete ID from constant cache table if it becomes empty on ISEQ free"
* 629908586b
  "Finer-grained inline constant cache invalidation"

MSWin builds on AppVeyor have been crashing since the merger.
2022-03-25 20:29:09 +09:00
Kevin Newton 8f6eaca2e1 Delete ID from constant cache table if it becomes empty on ISEQ free
Co-authored-by: John Hawthorn <john@hawthorn.email>
2022-03-24 09:14:38 -07:00
Kevin Newton 629908586b Finer-grained inline constant cache invalidation
Current behavior - caches depend on a global counter. All constant mutations cause caches to be invalidated.

```ruby
class A
  B = 1
end

def foo
  A::B # inline cache depends on global counter
end

foo # populate inline cache
foo # hit inline cache

C = 1 # global counter increments, all caches are invalidated

foo # misses inline cache due to `C = 1`
```

Proposed behavior - caches depend on name components. Only constant mutations with corresponding names will invalidate the cache.

```ruby
class A
  B = 1
end

def foo
  A::B # inline cache depends constants named "A" and "B"
end

foo # populate inline cache
foo # hit inline cache

C = 1 # caches that depend on the name "C" are invalidated

foo # hits inline cache because IC only depends on "A" and "B"
```

Examples of breaking the new cache:

```ruby
module C
  # Breaks `foo` cache because "A" constant is set and the cache in foo depends
  # on "A" and "B"
  class A; end
end

B = 1
```

We expect the new cache scheme to be invalidated less often because names aren't frequently reused. With the cache being invalidated less, we can rely on its stability more to keep our constant references fast and reduce the need to throw away generated code in YJIT.
2022-03-24 09:14:38 -07:00
Peter Zhu 5f10bd634f Add ISEQ_BODY macro
Use ISEQ_BODY macro to get the rb_iseq_constant_body of the ISeq. Using
this macro will make it easier for us to change the allocation strategy
of rb_iseq_constant_body when using Variable Width Allocation.
2022-03-24 10:03:51 -04:00
Nobuyoshi Nakada 8f3a36fb6e
Fix indents [ci skip] 2022-02-03 11:21:41 +09:00
Jemma Issroff 2913a2f5cf Treat TS_ICVARC cache as separate from TS_IVC cache 2022-02-02 09:20:34 -08:00
Koichi Sasada 6bef1ac628 `rb_iseq_update_references()` cares `script_lines`
and it fixes compaction issue:
http://rubyci.s3.amazonaws.com/freebsd12/ruby-master/log/20211218T203001Z.fail.html.gz
2021-12-19 06:15:22 +09:00
Koichi Sasada dd29ba0764 `iseq_type_sym()` -> `iseq_type_id()`
`iseq_type_sym()` returns `ID` (surprisingly!) so rename it
to `iseq_type_id()`.
2021-12-19 05:35:16 +09:00
Koichi Sasada 89a02d8932 add `rb_iseq_type()` to return iseq type in Symbol
It is shorthand `ISeq#to_a[9]`.
2021-12-19 05:16:29 +09:00
Koichi Sasada 2e6e2fd9da fix local TP memory leak
It free `rb_hook_list_t` itself if needed. To recognize the
need, this patch introduced `rb_hook_list_t::is_local` flag.

This patch is succession of https://github.com/ruby/ruby/pull/4652
2021-12-15 02:31:58 +09:00
Yusuke Endoh 8613c0c675 Introduce an option "--dump=insns_without_opt" for debugging purposes 2021-12-13 10:29:08 +09:00
Alan Wu 9121e57a5f Rework tracing for blocks running as methods
The main impetus for this change is to fix [Bug #13392]. Previously, we
fired the "return" TracePoint event after popping the stack frame for
the block running as method (BMETHOD). This gave undesirable source
location outputs as the return event normally fires right before the
frame going away.

The iseq for each block can run both as a block and as a method. To
accommodate that, this commit makes vm_trace() fire call/return events for
instructions that have b_call/b_return events attached when the iseq is
running as a BMETHOD. The logic for rewriting to "trace_*" instruction
is tweaked so that when the user listens to call/return events,
instructions with b_call/b_return become trace variants.

To continue to provide the return value for non-local returns done using
the "return" or "break" keyword inside BMETHODs, the stack unwinding
code is tweaked. b_return events now provide the same return value as
return events for these non-local cases. A pre-existing test deemed not
providing a return value for these b_return events as a limitation.

This commit removes the checks for call/return TracePoint events that
happen when calling into BMETHODs when no TracePoints are active.
Technically, migrating just the return event is enough to fix the bug,
but migrating both call and return removes our reliance on
`VM_FRAME_FLAG_FINISH` and re-entering the interpreter when the caller
is already in the interpreter.
2021-12-01 17:42:33 -05:00
Koichi Sasada 7ec1fc37f4 add `VM_CALLCACHE_ON_STACK`
check if iseq refers to on stack CC (it shouldn't).
2021-11-17 22:21:42 +09:00
Koichi Sasada b1b73936c1 `Primitive.mandatory_only?` for fast path
Compare with the C methods, A built-in methods written in Ruby is
slower if only mandatory parameters are given because it needs to
check the argumens and fill default values for optional and keyword
parameters (C methods can check the number of parameters with `argc`,
so there are no overhead). Passing mandatory arguments are common
(optional arguments are exceptional, in many cases) so it is important
to provide the fast path for such common cases.

`Primitive.mandatory_only?` is a special builtin function used with
`if` expression like that:

```ruby
  def self.at(time, subsec = false, unit = :microsecond, in: nil)
    if Primitive.mandatory_only?
      Primitive.time_s_at1(time)
    else
      Primitive.time_s_at(time, subsec, unit, Primitive.arg!(:in))
    end
  end
```

and it makes two ISeq,

```
  def self.at(time, subsec = false, unit = :microsecond, in: nil)
    Primitive.time_s_at(time, subsec, unit, Primitive.arg!(:in))
  end

  def self.at(time)
    Primitive.time_s_at1(time)
  end
```

and (2) is pointed by (1). Note that `Primitive.mandatory_only?`
should be used only in a condition of an `if` statement and the
`if` statement should be equal to the methdo body (you can not
put any expression before and after the `if` statement).

A method entry with `mandatory_only?` (`Time.at` on the above case)
is marked as `iseq_overload`. When the method will be dispatch only
with mandatory arguments (`Time.at(0)` for example), make another
method entry with ISeq (2) as mandatory only method entry and it
will be cached in an inline method cache.

The idea is similar discussed in https://bugs.ruby-lang.org/issues/16254
but it only checks mandatory parameters or more, because many cases
only mandatory parameters are given. If we find other cases (optional
or keyword parameters are used frequently and it hurts performance),
we can extend the feature.
2021-11-15 15:58:56 +09:00
Koichi Sasada cbf2078a25 need to mark script_lines 2021-10-29 02:00:06 +09:00
Koichi Sasada 7185c00fcc freeze (make shareable) script_lines 2021-10-21 16:17:39 +09:00
Koichi Sasada c7550537f1 `RubyVM.keep_script_lines`
`RubyVM.keep_script_lines` enables to keep script lines
for each ISeq and AST. This feature is for debugger/REPL
support.

```ruby
RubyVM.keep_script_lines = true
RubyVM::keep_script_lines = true

eval("def foo = nil\ndef bar = nil")
pp RubyVM::InstructionSequence.of(method(:foo)).script_lines
```
2021-10-21 16:17:39 +09:00
Alan Wu 506c037a16 Cleanup iseq.c minimize diff with upstream
Most of these are vestiges of our old setup where we hack into the
interpreter loop.
2021-10-20 18:19:42 -04:00
Alan Wu bd876c243a TracePoint support
This change fixes some cases where YJIT fails to fire tracing events.
Most of the situations YJIT did not handle correctly involves enabling
tracing while running inside generated code.

A new operation to invalidate all generated code is added, which uses
patching to make generated code exit at the next VM instruction
boundary. A new routine called `jit_prepare_routine_call()` is
introduced to facilitate this and should be used when generating code
that could allocate, or could otherwise use `RB_VM_LOCK_ENTER()`.

The `c_return` event is fired in the middle of an instruction as opposed
to at an instruction boundary, so it requires special handling. C method
call return points are patched to go to a fucntion which does everything
the interpreter does, including firing the `c_return` event. The
generated code for C method calls normally does not fire the event.

Invalided code should not change after patching so the exits are not
clobbered. A new variable is introduced to track the region of code that
should not change.
2021-10-20 18:19:39 -04:00
Aaron Patterson 71cef74432 Clear JIT code when tracepoints get enabled
Clear out any JIT code on iseqs when tracepoints get enabled.  We can't
handle tracepoints right now, so we'll just try to recompile later.
2021-10-20 18:19:38 -04:00
Jose Narvaez 4e2eb7695e Yet Another Ruby JIT!
Renaming uJIT to YJIT. AKA s/ujit/yjit/g.
2021-10-20 18:19:31 -04:00
Alan Wu 03af6ad995 Should fix builds without mjit support 2021-10-20 18:19:29 -04:00
Alan Wu c02517bacb Tie lifetime of uJIT blocks to iseqs
* Tie lifetime of uJIT blocks to iseqs

   Blocks weren't being freed when iseqs are collected.

* Add rb_dary. Use it for method dependency table

* Keep track of blocks per iseq

  Remove global version_tbl

* Block version bookkeeping fix

* dary -> darray

* free ujit_blocks

* comment about size of ujit_blocks
2021-10-20 18:19:29 -04:00
Maxime Chevalier-Boisvert 2e561ff255 Restore interpreter regs in ujit hook. Implement leave bytecode. 2021-10-20 18:19:28 -04:00
Alan Wu 11c1daea17 Add to the MicroJIT scraper an example that passes ec 2021-10-20 18:19:25 -04:00
Alan Wu 93f6ac39f2 MicroJIT: Don't compile trace instructions 2021-10-20 18:19:25 -04:00
Alan Wu 040cfdb139 Small build fixes 2021-10-20 18:19:24 -04:00
Alan Wu 7d1546ece5 Try to fix Windows build 2021-10-20 18:19:24 -04:00
Alan Wu 265c5ca8b1 Avoid triggering GC while translating threaded code 2021-10-20 18:19:23 -04:00
Maxime Chevalier-Boisvert 3739588811 Remove PC argument from ujit instructions 2021-10-20 18:19:23 -04:00
Maxime Chevalier-Boisvert 26fecc7236 Removed native_pop_code, ported call with label 2021-10-20 18:19:23 -04:00
Alan Wu 4929ba0a5c Generate multiple copies of native code for `pop`
Insert generated addresses into st_table for mapping native code
addresses back to info about VM instructions. Export `encoded_insn_data`
to do this. Also some style fixes.
2021-10-20 18:19:23 -04:00
Maxime Chevalier-Boisvert 251531bdf0 Reimplement Alan's pop instruction with the new assembler 2021-10-20 18:19:23 -04:00
Maxime Chevalier-Boisvert efcaa49a7b Added sub instruction, 32-bit registers, more tests 2021-10-20 18:19:23 -04:00
Alan Wu 16c5ce863c Yeah, this actually works! 2021-10-20 18:19:22 -04:00
Alan Wu cec197696f Add example handler for ujit and scrape it from vm.o 2021-10-20 18:19:22 -04:00
S.H dc9112cf10
Using NIL_P macro instead of `== Qnil` 2021-10-03 22:34:45 +09:00
S-H-GAMELINKS bdd6d8746f Replace RBOOL macro 2021-09-05 23:01:27 +09:00
Jeremy Evans 2d98593bf5 Support tracing of attr_reader and attr_writer
In vm_call_method_each_type, check for c_call and c_return events before
dispatching to vm_call_ivar and vm_call_attrset.  With this approach, the
call cache will still dispatch directly to those functions, so this
change will only decrease performance for the first (uncached) call, and
even then, the performance decrease is very minimal.

This approach requires that we clear the call caches when tracing is
enabled or disabled.  The approach currently switches all vm_call_ivar
and vm_call_attrset call caches to vm_call_general any time tracing is
enabled or disabled. So it could theoretically result in a slowdown for
code that constantly enables or disables tracing.

This approach does not handle targeted tracepoints, but from my testing,
c_call and c_return events are not supported for targeted tracepoints,
so that shouldn't matter.

This includes a benchmark showing the performance decrease is minimal
if detectable at all.

Fixes [Bug #16383]
Fixes [Bug #10470]

Co-authored-by: Takashi Kokubun <takashikkbn@gmail.com>
2021-08-29 07:23:39 -07:00
Nobuyoshi Nakada 4c3140d60f
Add keyrest to ruby2_keywords parameters [Bug #18011] 2021-08-03 10:56:50 +09:00
Jeremy Evans 64ac984129 Make RubyVM::AbstractSyntaxTree.of raise for method/proc created in eval
This changes Thread::Location::Backtrace#absolute_path to return
nil for methods/procs defined in eval.  If the realpath of an iseq
is nil, that indicates it was defined in eval, in which case you
cannot use RubyVM::AbstractSyntaxTree.of.

Fixes [Bug #16983]

Co-authored-by: Koichi Sasada <ko1@atdot.net>
2021-07-29 13:51:03 -07:00
Yusuke Endoh 68e1dc5172 iseq.c: Make ast_line_count return 0 when syntax error occurred
This broke coverage CI

```
  1) Failure:
TestRequire#test_load_syntax_error [/home/runner/work/actions/actions/ruby/test/ruby/test_require.rb:228]:
Exception(SyntaxError) with message matches to /unexpected/.
[SyntaxError] exception expected, not #<TypeError: no implicit conversion of false into Integer>.
```
https://github.com/ruby/actions/runs/2914743968?check_suite_focus=true
2021-06-26 00:15:16 +09:00
Yusuke Endoh 0a36cab1b5 Enable USE_ISEQ_NODE_ID by default
... which is formally called EXPERIMENTAL_ISEQ_NODE_ID.

See also ff69ef27b0.

https://bugs.ruby-lang.org/issues/17930
2021-06-18 03:35:38 +09:00
Yusuke Endoh dfba87cd62 Make it possible to get AST::Node from Thread::Backtrace::Location
RubyVM::AST.of(Thread::Backtrace::Location) returns a node that
corresponds to the location. Typically, the node is a method call, but
not always.

This change also includes iseq's dump/load support of node_ids for each
instructions.
2021-06-18 03:35:38 +09:00
Yusuke Endoh fb01411ae8 node.h: Reduce struct size to fit with Ruby object size (five VALUEs)
by merging `rb_ast_body_t#line_count` and `#script_lines`.

Fortunately `line_count == RARRAY_LEN(script_lines)` was always
satisfied. When script_lines is saved, it has an array of lines, and
when not saved, it has a Fixnum that represents the old line_count.
2021-06-18 02:34:27 +09:00
Takashi Kokubun 070caf54d2
Refactor rb_vm_insn_addr2insn calls
It's been a way too much amount of ifdefs.
2021-06-02 01:16:50 -07:00
Yusuke Endoh ff69ef27b0 compile.c: Pass node instead of nd_line(node) to ADD_INSN* functions
... then, new_insn_core extracts nd_line(node).

Also, if a macro "EXPERIMENTAL_ISEQ_NODE_ID" is defined, this changeset
keeps nd_node_id(node) for each instruction. This is intended for
TypeProf to identify what AST::Node corresponds to each instruction.

This patch is originally authored by @yui-knk for showing which column a
NoMethodError occurred.

https://github.com/ruby/ruby/compare/master...yui-knk:feature/node_id

Co-Authored-By: Yuichiro Kaneko <yui-knk@ruby-lang.org>
2021-05-07 17:02:15 +09:00
Aaron Patterson 8359821870 Use rb_fstring for "defined" strings.
We can take advantage of fstrings to de-duplicate the defined strings.
This means we don't need to keep the list of defined strings on the VM
(or register them as mark objects)
2021-03-17 10:55:37 -07:00