Setting up the fake array is a bit more expensive than would be
expected because `rb_ary_freeze` does a lot of checks and lookup
a shape transition.
If we assume fake arrays will always be frozen, we can precompute
the flags state and just assign it.
Use plain paragraphs instead of a nested unordered list.
Remove some examples, including examples that produce warnings in
verbose mode or raise exceptions.
Explicitly document the common pitfall of using an expression as a
default value instead of using a block.
Improves activerecord by about 1% on the interpreter:
```
before: ruby 3.4.0dev (2024-07-03T18:40:10Z master f88841b8f3) [arm64-darwin23]
after: ruby 3.4.0dev (2024-07-03T18:41:14Z ruby-map 6c0df4eb32) [arm64-darwin23]
------------ ----------- ---------- ---------- ---------- ------------- ------------
bench before (ms) stddev (%) after (ms) stddev (%) after 1st itr before/after
activerecord 235.2 0.8 233.6 0.7 1.01 1.01
------------ ----------- ---------- ---------- ---------- ------------- ------------
Legend:
- after 1st itr: ratio of before/after time for the first benchmarking iteration.
- before/after: ratio of before/after time. Higher is better for after. Above 1 represents a speedup.
```
Improves YJIT by about 4%:
```
before: ruby 3.4.0dev (2024-07-03T18:40:10Z master f88841b8f3) +YJIT [arm64-darwin23]
after: ruby 3.4.0dev (2024-07-03T18:41:14Z ruby-map 6c0df4eb32) +YJIT [arm64-darwin23]
------------ ----------- ---------- ---------- ---------- ------------- ------------
bench before (ms) stddev (%) after (ms) stddev (%) after 1st itr before/after
activerecord 142.1 1.2 137.0 0.6 1.00 1.04
------------ ----------- ---------- ---------- ---------- ------------- ------------
Legend:
- after 1st itr: ratio of before/after time for the first benchmarking iteration.
- before/after: ratio of before/after time. Higher is better for after. Above 1 represents a speedup.
```
This speeds up the mail benchmark by about 7% on the interpreter:
```
before: ruby 3.4.0dev (2024-07-03T17:01:41Z master f4b313f733) [arm64-darwin23]
after: ruby 3.4.0dev (2024-07-03T17:45:50Z ruby-select de282cacd5) [arm64-darwin23]
----- ----------- ---------- ---------- ---------- ------------- ------------
bench before (ms) stddev (%) after (ms) stddev (%) after 1st itr before/after
mail 72.9 0.8 68.2 1.0 1.02 1.07
----- ----------- ---------- ---------- ---------- ------------- ------------
Legend:
- after 1st itr: ratio of before/after time for the first benchmarking iteration.
- before/after: ratio of before/after time. Higher is better for after. Above 1 represents a speedup.
```
YJIT is about 13% faster:
```
before: ruby 3.4.0dev (2024-07-03T17:01:41Z master f4b313f733) +YJIT [arm64-darwin23]
after: ruby 3.4.0dev (2024-07-03T17:45:50Z ruby-select de282cacd5) +YJIT [arm64-darwin23]
----- ----------- ---------- ---------- ---------- ------------- ------------
bench before (ms) stddev (%) after (ms) stddev (%) after 1st itr before/after
mail 51.0 0.8 45.2 0.6 1.00 1.13
----- ----------- ---------- ---------- ---------- ------------- ------------
Legend:
- after 1st itr: ratio of before/after time for the first benchmarking iteration.
- before/after: ratio of before/after time. Higher is better for after. Above 1 represents a speedup.
```
Following [Feature #20589] it can happen that we change the
capacity of a frozen array, so these assertions no longer make
sense.
Normally we don't hit them because `Array#freeze` shrinks the
array, but if somehow the Array was frozen using `Object#freeze`
then we may shrink it after it was frozen.
While working on a separate issue we found that in some cases
`ary_heap_realloc` was being called on frozen arrays. To fix this, this
change does the following:
1) Updates `rb_ary_freeze` to assert the type is an array, return if
already frozen, and shrink the capacity if it is not embedded, shared
or a shared root.
2) Replaces `rb_obj_freeze` with `rb_ary_freeze` when the object is
always an array.
3) In `ary_heap_realloc`, ensure the new capa is set with
`ARY_SET_CAPA`. Previously the change in capa was not set.
4) Adds an assertion to `ary_heap_realloc` that the array is not frozen.
Some of this work was originally done in
https://github.com/ruby/ruby/pull/2640, referencing this issue
https://bugs.ruby-lang.org/issues/16291. There didn't appear to be any
objections to this PR, it appears to have simply lost traction.
The original PR made changes to arrays and strings at the same time,
this PR only does arrays. Also it was old enough that rather than revive
that branch I've made a new one. I added Lourens as co-author in addtion
to Aaron who helped me with this patch.
The original PR made this change for performance reasons, and while
that's still true for this PR, the goal of this PR is to avoid
calling `ary_heap_realloc` on frozen arrays. The capacity should be
shrunk _before_ the array is frozen, not after.
Co-authored-by: Aaron Patterson <tenderlove@ruby-lang.org>
Co-Authored-By: methodmissing <lourens@methodmissing.com>
Ref: https://github.com/ruby/ruby/pull/10872
These should be the last internal uses of the old `Data` API
inside Ruby itself. Some use remain in a couple default gems.
In cases where `rb_ary_sort_bang` is called with a block and
tmp is an embedded array, we need to account for the block
potentially impacting the capacity of ary.
ex:
```
var_0 = (1..70).to_a
var_0.sort! do |var_0_block_129, var_1_block_129|
var_0.pop
var_1_block_129 <=> var_0_block_129
end.shift(3)
```
The above example can put the array into a corrupted state
resulting in a heap buffer overflow and possible segfault:
```
ERROR: AddressSanitizer: heap-buffer-overflow on address [...]
WRITE of size 560 at 0x60b0000034f0 thread T0 [...]
```
This commit adds a conditional to determine when the capacity
of ary has been modified by the provided block. If this is
the case, ensure that the capacity of ary is adjusted to
handle at minimum the len of tmp.
I discovered the problem in `compile.c` from a failing
TestIseqLoad#test_stressful_roundtrip test with ASAN enabled. The other
two changes in array.c and string.c I found by auditing similar usages
of DATA_PTR in the codebase.
[Bug #20402]
assert does not print the bug report, only the file and line number of
the assertion that failed. RUBY_ASSERT prints the full bug report, which
makes it much easier to debug.
BurdetteLamar
Jean Boussier
Jeremy Evans
Aaron Patterson
eileencodes
Nobuyoshi Nakada
Matt Valentine-House
Zack Deveau
KJ Tsanaktsidis
Peter Zhu
Takashi Kokubun
Edwing123
Akshay Birajdar