Moved this hack mark to an argument to `compile_hash`.
> Bad Hack: temporarily mark hash node with flag so
> compile_hash can compile call differently.
Formerly, branch coverage measurement counters are generated for each
compilation traverse of the AST. However, ensure clause node is
traversed twice; one is for normal-exit case (the resulted bytecode is
embedded in its outer scope), and the other is for exceptional case (the
resulted bytecode is used in catch table). Two branch coverage counters
are generated for the two cases, but it is not desired.
This changeset revamps the internal representation of branch coverage
measurement. Branch coverage counters are generated only at the first
visit of a branch node. Visiting the same node reuses the
already-generated counter, so double counting is avoided.
This makes:
```ruby
args = [1, 2, -> {}]; foo(*args, &args.pop)
```
call `foo` with 1, 2, and the lambda, in addition to passing the
lambda as a block. This is different from the previous behavior,
which passed the lambda as a block but not as a regular argument,
which goes against the expected left-to-right evaluation order.
This is how Ruby already compiled arguments if using leading
arguments, trailing arguments, or keywords in the same call.
This works by disabling the optimization that skipped duplicating
the array during the splat (splatarray instruction argument
switches from false to true). In the above example, the splat
call duplicates the array. I've tested and cases where a
local variable or symbol are used do not duplicate the array,
so I don't expect this to decrease the performance of most Ruby
programs. However, programs such as:
```ruby
foo(*args, &bar)
```
could see a decrease in performance, if `bar` is a method call
and not a local variable.
This is not a perfect solution, there are ways to get around
this:
```ruby
args = Struct.new(:a).new([:x, :y])
def args.to_a; a; end
def args.to_proc; a.pop; ->{}; end
foo(*args, &args)
# calls foo with 1 argument (:x)
# not 2 arguments (:x and :y)
```
A perfect solution would require completely disabling the
optimization.
Fixes [Bug #16504]
Fixes [Bug #16500]
These crashes are due to alignment issues, casting ADJUST to INSN
and then accessing after the end of the ADJUST. These patches
come from Stefan Sperling <stsp@apache.org>, who reported the
issue.
The GC will not disassemble incomplete instruction sequences. So it is
important that when instructions are being assembled, any objects the
instructions point at should not be moved.
This patch implements a fixed width array that pins its references.
When the instructions are done being assembled, the pinning array goes
away and the objects inside the iseqs are allowed to move.
With compiling `CPDEBUG >= 2`, `rb_iseq_disasm` segfaults if this
table has not been created. Also `ibf_load_iseq_each` calls
`rb_iseq_insns_info_encode_positions`.
Accessing past the end of an array is technically UB. Use C99 flexible
array member instead to avoid the UB and simplify allocation size
calculation.
See also: DCL38-C in the SEI CERT C Coding Standard
Previously, passing a keyword splat to a method always allocated
a hash on the caller side, and accepting arbitrary keywords in
a method allocated a separate hash on the callee side. Passing
explicit keywords to a method that accepted a keyword splat
did not allocate a hash on the caller side, but resulted in two
hashes allocated on the callee side.
This commit makes passing a single keyword splat to a method not
allocate a hash on the caller side. Passing multiple keyword
splats or a mix of explicit keywords and a keyword splat still
generates a hash on the caller side. On the callee side,
if arbitrary keywords are not accepted, it does not allocate a
hash. If arbitrary keywords are accepted, it will allocate a
hash, but this commit uses a callinfo flag to indicate whether
the caller already allocated a hash, and if so, the callee can
use the passed hash without duplicating it. So this commit
should make it so that a maximum of a single hash is allocated
during method calls.
To set the callinfo flag appropriately, method call argument
compilation checks if only a single keyword splat is given.
If only one keyword splat is given, the VM_CALL_KW_SPLAT_MUT
callinfo flag is not set, since in that case the keyword
splat is passed directly and not mutable. If more than one
splat is used, a new hash needs to be generated on the caller
side, and in that case the callinfo flag is set, indicating
the keyword splat is mutable by the callee.
In compile_hash, used for both hash and keyword argument
compilation, if compiling keyword arguments and only a
single keyword splat is used, pass the argument directly.
On the caller side, in vm_args.c, the callinfo flag needs to
be recognized and handled. Because the keyword splat
argument may not be a hash, it needs to be converted to a
hash first if not. Then, unless the callinfo flag is set,
the hash needs to be duplicated. The temporary copy of the
callinfo flag, kw_flag, is updated if a hash was duplicated,
to prevent the need to duplicate it again. If we are
converting to a hash or duplicating a hash, we need to update
the argument array, which can including duplicating the
positional splat array if one was passed. CALLER_SETUP_ARG
and a couple other places needs to be modified to handle
similar issues for other types of calls.
This includes fairly comprehensive tests for different ways
keywords are handled internally, checking that you get equal
results but that keyword splats on the caller side result in
distinct objects for keyword rest parameters.
Included are benchmarks for keyword argument calls.
Brief results when compiled without optimization:
def kw(a: 1) a end
def kws(**kw) kw end
h = {a: 1}
kw(a: 1) # about same
kw(**h) # 2.37x faster
kws(a: 1) # 1.30x faster
kws(**h) # 2.19x faster
kw(a: 1, **h) # 1.03x slower
kw(**h, **h) # about same
kws(a: 1, **h) # 1.16x faster
kws(**h, **h) # 1.14x faster
Previously, method call keyword splats and hash keyword splats
were compiled exactly the same. This is because parse-wise, they
operate on indentical nodes when it comes to compiling the **{}.
Fix this by using an ugly hack of temporarily modifying the
nd_brace flag in the method call keyword splat case. Inside
compile_hash, only optimize the **{} case for hashes where the
nd_brace flag has been modified to reflect we are in the method
call keyword splat case and it is safe to do so.
Since compile_keyword_args is only called in one place, move the
keyword_node_p call out of that method to the single caller to
avoid duplicating the code.
This patch contains several ideas:
(1) Disposable inline method cache (IMC) for race-free inline method cache
* Making call-cache (CC) as a RVALUE (GC target object) and allocate new
CC on cache miss.
* This technique allows race-free access from parallel processing
elements like RCU.
(2) Introduce per-Class method cache (pCMC)
* Instead of fixed-size global method cache (GMC), pCMC allows flexible
cache size.
* Caching CCs reduces CC allocation and allow sharing CC's fast-path
between same call-info (CI) call-sites.
(3) Invalidate an inline method cache by invalidating corresponding method
entries (MEs)
* Instead of using class serials, we set "invalidated" flag for method
entry itself to represent cache invalidation.
* Compare with using class serials, the impact of method modification
(add/overwrite/delete) is small.
* Updating class serials invalidate all method caches of the class and
sub-classes.
* Proposed approach only invalidate the method cache of only one ME.
See [Feature #16614] for more details.
Now, rb_call_info contains how to call the method with tuple of
(mid, orig_argc, flags, kwarg). Most of cases, kwarg == NULL and
mid+argc+flags only requires 64bits. So this patch packed
rb_call_info to VALUE (1 word) on such cases. If we can not
represent it in VALUE, then use imemo_callinfo which contains
conventional callinfo (rb_callinfo, renamed from rb_call_info).
iseq->body->ci_kw_size is removed because all of callinfo is VALUE
size (packed ci or a pointer to imemo_callinfo).
To access ci information, we need to use these functions:
vm_ci_mid(ci), _flag(ci), _argc(ci), _kwarg(ci).
struct rb_call_info_kw_arg is renamed to rb_callinfo_kwarg.
rb_funcallv_with_cc() and rb_method_basic_definition_p_with_cc()
is temporary removed because cd->ci should be marked.
This behavior was deprecated in 2.7 and scheduled to be removed
in 3.0.
Calling yield in a class definition outside a method is now a
SyntaxError instead of a LocalJumpError, as well.
Nobuyoshi Nakada
Kazuki Tsujimoto
Vladimir Dementyev
Takashi Kokubun
Yusuke Endoh
Jeremy Evans
Koichi Sasada
Aaron Patterson
卜部昌平
Alan Wu
Masataka Pocke Kuwabara
NagayamaRyoga