Граф коммитов

374 Коммитов

Автор SHA1 Сообщение Дата
Peter Zhu fc043a2e61 Remove write barrier for non-Ruby object
enumerator's meth attribute is an ID, which is not a Ruby object, so we
should not apply the write barrier on it.
2024-09-23 15:51:16 -04:00
Jeremy Evans e7dda08617 Do not set Enumerator::Lazy#zip to use packed format
Enumerator#zip yields a single array, not multiple arguments,
so Enumerator::Lazy#zip should do the same.

Fixes [#20623]
2024-07-20 18:16:42 -07:00
eileencodes d25b74b32c Resize arrays in `rb_ary_freeze` and use it for freezing arrays
While working on a separate issue we found that in some cases
`ary_heap_realloc` was being called on frozen arrays. To fix this, this
change does the following:

1) Updates `rb_ary_freeze` to assert the type is an array, return if
already frozen, and shrink the capacity if it is not embedded, shared
or a shared root.
2) Replaces `rb_obj_freeze` with `rb_ary_freeze` when the object is
always an array.
3) In `ary_heap_realloc`, ensure the new capa is set with
`ARY_SET_CAPA`. Previously the change in capa was not set.
4) Adds an assertion to `ary_heap_realloc` that the array is not frozen.

Some of this work was originally done in
https://github.com/ruby/ruby/pull/2640, referencing this issue
https://bugs.ruby-lang.org/issues/16291. There didn't appear to be any
objections to this PR, it appears to have simply lost traction.

The original PR made changes to arrays and strings at the same time,
this PR only does arrays. Also it was old enough that rather than revive
that branch I've made a new one. I added Lourens as co-author in addtion
to Aaron who helped me with this patch.

The original PR made this change for performance reasons, and while
that's still true for this PR, the goal of this PR is to avoid
calling `ary_heap_realloc` on frozen arrays. The capacity should be
shrunk _before_ the array is frozen, not after.

Co-authored-by: Aaron Patterson <tenderlove@ruby-lang.org>
Co-Authored-By: methodmissing <lourens@methodmissing.com>
2024-07-02 10:34:23 -07:00
Nobuyoshi Nakada 29110fe18d
[Bug #20325] `Enumerator.product.size` is 0 if any size is 0 2024-04-16 16:13:19 +09:00
Jean Boussier d4f3dcf4df Refactor VM root modules
This `st_table` is used to both mark and pin classes
defined from the C API. But `vm->mark_object_ary` already
does both much more efficiently.

Currently a Ruby process starts with 252 rooted classes,
which uses `7224B` in an `st_table` or `2016B` in an `RArray`.

So a baseline of 5kB saved, but since `mark_object_ary` is
preallocated with `1024` slots but only use `405` of them,
it's a net `7kB` save.

`vm->mark_object_ary` is also being refactored.

Prior to this changes, `mark_object_ary` was a regular `RArray`, but
since this allows for references to be moved, it was marked a second
time from `rb_vm_mark()` to pin these objects.

This has the detrimental effect of marking these references on every
minors even though it's a mostly append only list.

But using a custom TypedData we can save from having to mark
all the references on minor GC runs.

Addtionally, immediate values are now ignored and not appended
to `vm->mark_object_ary` as it's just wasted space.
2024-03-06 15:33:43 -05:00
Peter Zhu 5af64ff7db Fix Enumerator#with_index for GC compaction
enumerator_block_call was not safe for compaction because the Array
backing the argv was not pinned, so it could get moved during compaction
which would make argv point to somewhere else.

The test crashes when RGENGC_CHECK_MODE is turned on:

    TestEnumerator#test_with_index_under_gc_compact_stress
    check_rvalue_consistency: 0x55db0b399450 is not a Ruby object.
    test/ruby/test_enumerator.rb:133: [BUG] check_rvalue_consistency_force: there is 1 errors.
    ruby 3.3.0dev (2023-12-23T23:00:27Z master 50bf437341) [x86_64-linux]
    -- Control frame information -----------------------------------------------
    c:0034 p:---- s:0192 e:000187 CFUNC  :with_index
    c:0033 p:---- s:0185 e:000184 CFUNC  :each
    c:0032 p:---- s:0182 e:000181 CFUNC  :to_a
    c:0031 p:0055 s:0178 e:000175 BLOCK  test/ruby/test_enumerator.rb:133
    c:0030 p:0024 s:0172 e:000171 METHOD tool/lib/envutil.rb:242
    c:0029 p:0024 s:0167 e:000166 METHOD tool/lib/envutil.rb:251
    c:0028 p:0005 s:0160 e:000159 METHOD test/ruby/test_enumerator.rb:131
    ...
    -- C level backtrace information -------------------------------------------
    build/ruby(rb_print_backtrace+0x14) [0x55db0b1deb21] vm_dump.c:820
    build/ruby(rb_vm_bugreport) vm_dump.c:1151
    build/ruby(bug_report_end+0x0) [0x55db0b3a53a6] error.c:1042
    build/ruby(rb_bug_without_die) error.c:1042
    build/ruby(die+0x0) [0x55db0afc77c2] error.c:1050
    build/ruby(rb_bug) error.c:1052
    build/ruby(gc_move+0x0) [0x55db0afbada0] gc.c:1714
    build/ruby(check_rvalue_consistency+0xa) [0x55db0afef0c3] gc.c:1729
    build/ruby(is_markable_object) gc.c:4769
    build/ruby(gc_mark_stack_values) gc.c:6595
    build/ruby(rb_gc_mark_vm_stack_values) gc.c:6605
    build/ruby(rb_execution_context_mark+0x39) [0x55db0b1d8589] vm.c:3309
    build/ruby(thread_mark+0x15) [0x55db0b1a9805] vm.c:3381
    build/ruby(gc_mark_stacked_objects+0x6d) [0x55db0aff2c3d] gc.c:7564
    build/ruby(gc_mark_stacked_objects_all) gc.c:7602
    build/ruby(gc_marks_rest) gc.c:8797
    build/ruby(gc_marks+0xd) [0x55db0aff43d5] gc.c:8855
    build/ruby(gc_start) gc.c:9608
    build/ruby(rb_multi_ractor_p+0x0) [0x55db0aff5463] gc.c:9489
    build/ruby(rb_vm_lock_leave) vm_sync.h:92
    build/ruby(garbage_collect) gc.c:9491
    build/ruby(newobj_slowpath+0xcb) [0x55db0aff57ab] gc.c:2871
    build/ruby(newobj_slowpath_wb_protected) gc.c:2895
    build/ruby(newobj_of0+0x24) [0x55db0aff59e4] gc.c:2937
    build/ruby(newobj_of) gc.c:2947
    build/ruby(rb_wb_protected_newobj_of) gc.c:2962
    build/ruby(ary_alloc_embed+0x10) [0x55db0b2f3e40] array.c:668
    build/ruby(ary_new) array.c:709
    build/ruby(rb_ary_tmp_new_from_values) array.c:759
    build/ruby(rb_ary_new_from_values) array.c:771
    build/ruby(args_copy+0x18) [0x55db0b1bbb88] vm_args.c:158
2023-12-24 16:22:34 -05:00
Nobuyoshi Nakada 0cdad3b92a Add `RUBY_REFERENCES`
Instead of `RUBY_REFERENCES_START` and `RUBY_REFERENCES_END`, so that
auto-indent works well.
2023-11-30 21:39:28 +09:00
Nobuyoshi Nakada 30f7b7a053 Prefix `REF_EDGE` and `REFS_LIST_PTR` with `RUBY_`
Also move `struct` so that `typedef`-ed names can be used.
2023-11-30 21:39:28 +09:00
Peter Zhu db7f3064a8 Implement proc_entry on VWA 2023-11-29 22:51:13 -05:00
Peter Zhu 40e67cb20e Implement Write Barriers on proc_entry 2023-11-29 22:51:13 -05:00
Peter Zhu 705a3c69d7 Add RUBY_TYPED_FREE_IMMEDIATELY flag to proc_entry 2023-11-29 22:51:13 -05:00
Peter Zhu 2e4a0a4d90 Implement Write Barriers on Enumerator::Producer 2023-11-27 08:29:35 -05:00
Peter Zhu 2dadd17c78 Implement Write Barriers on Enumerator::Generator 2023-11-27 08:29:35 -05:00
Peter Zhu d7165d88ec Implement Write Barriers on Enumerator::Yielder 2023-11-27 08:29:35 -05:00
Peter Zhu 315240e73b Implement Write Barriers on Enumerator 2023-11-23 13:23:29 -05:00
Matt Valentine-House cdc578ee00 Remove unneccesary defines in enumerator 2023-11-22 10:36:23 +00:00
Matt Valentine-House a3f9a98c29 Remove unneccesary memsize functions in enumerator 2023-11-22 10:36:23 +00:00
Matt Valentine-House 1e075a734f VWA Embed the rest of the data objects in Enumerator 2023-11-22 10:36:23 +00:00
Peter Zhu a182b2c5e1 Implement Enumerator objects on VWA
This commit implements Enumerator objects on VWA. This speeds allocations
and decreases memory usage.

```
require "benchmark"

ary = []

puts(Benchmark.measure do
  10_000_000.times do
    u = ary.to_enum
  end
end)

puts `ps -o rss= -p #{$$}`
```

Before:

```
  1.500774   0.002717   1.503491 (  1.506791)
 18512
```

After:

```
  0.892580   0.002539   0.895119 (  0.897642)
 16480
```
2023-11-20 18:59:01 -05:00
Hiroya Fujinami 3e64cf60b5
Fix [Bug #19632]: Disable external iterator for frozen enumerator (#7791)
* Fix [Bug #19632]: Disable external iterator for frozen enumerator

Currently, methods to manipulate an external iterator like `#next`
and `#feed` can be called even if a receiver of an enumerator is
frozen. However, these methods change the state of an external
iterator in an enumerator. Therefore, it seems a BUG to me, and
these methods should raise FrozenError if the receiver is frozen.

This fixed the following methods to raise FrozenError if the receiver is
frozen.

- `Enumerator#next`
- `Enumerator#next_values`
- `Enumerator#peek`
- `Enumerator#peek_values`
- `Enumerator#feed`
- `Enumerator#rewind`

* Fix a typo in the document

Thanks @Maumagnaguagno.
2023-10-25 16:32:25 +09:00
Marcelo Pereira f15123c34c Fix stack trace for rescued StopIteration 2023-07-15 15:24:43 +09:00
Jeremy Evans a14915ca4b Do not have Enumeratory::Lazy#zip mark result as packed
Fixes [Bug #19569]
2023-06-24 08:41:32 -07:00
Matt Valentine-House 22b349294b Implement declarative references for enumerator 2023-03-17 19:20:40 +00:00
Benoit Daloze d93d786338 Try to fix RDoc markup for Enumerator 2022-12-26 15:28:37 +01:00
Akinori MUSHA cc4c28ec2e Make Enumerartor.product return nil when called with a block 2022-12-21 19:13:15 +09:00
Akinori MUSHA 308ccbaeb2 Make product consistently yield an array of N elements instead of N arguments
Inconsistency pointed out by @mame:

```
>> Enumerator.product([1], [2], [3]).to_a
=> [[1, 2, 3]]
>> Enumerator.product([1], [2]).to_a
=> [[1, 2]]
>> Enumerator.product([1]).to_a
=> [1]
>> Enumerator.product().to_a
=> [nil]
```

Got fixed as follows:

```
>> Enumerator.product([1], [2], [3]).to_a
=> [[1, 2, 3]]
>> Enumerator.product([1], [2]).to_a
=> [[1, 2]]
>> Enumerator.product([1]).to_a
=> [[1]]
>> Enumerator.product().to_a
=> [[]]
```

This was due to the nature of the N-argument funcall in Ruby.
2022-12-21 18:19:19 +09:00
Benoit Daloze 33debffdd3 Use "Fiber storage variables" consistently 2022-12-20 23:05:56 +01:00
Benoit Daloze 4495dea153 Improve documentation for fiber-scoped variables
* Especially around Enumerator.
2022-12-20 23:05:56 +01:00
Benoit Daloze 45175962a6 Never use the storage of another Fiber, that violates the whole design
* See https://bugs.ruby-lang.org/issues/19078#note-30
2022-12-20 19:32:23 +01:00
Akinori MUSHA ad18d1297e Reject keyword arguments given to Enumerator::Product.new
The use of keyword arguments should be reserved for future extension.
2022-12-16 13:32:13 +09:00
Samuel Williams 0436f1e15a
Introduce `Fiber#storage` for inheritable fiber-scoped variables. (#6612) 2022-12-01 23:00:33 +13:00
Nobuyoshi Nakada 64c8ed272f `remain` no longer starts with 0 2022-11-25 16:11:17 +09:00
Nobuyoshi Nakada ffc6c5d056 [Bug #18971] Add precheck to enumerator 2022-11-25 16:11:17 +09:00
S-H-GAMELINKS 1f4f6c9832 Using UNDEF_P macro 2022-11-16 18:58:33 +09:00
Jeremy Evans cfb9624460
Fix Array#[] with ArithmeticSequence with negative steps (#5739)
* Fix Array#[] with ArithmeticSequence with negative steps

Previously, Array#[] when called with an ArithmeticSequence
with a negative step did not handle all cases correctly,
especially cases involving infinite ranges, inverted ranges,
and/or exclusive ends.

Fixes [Bug #18247]

* Add Array#slice tests for ArithmeticSequence with negative step to test_array

Add tests of rb_arithmetic_sequence_beg_len_step C-API function.

* Fix ext/-test-/arith_seq/beg_len_step/depend

* Rename local variables

* Fix a variable name

Co-authored-by: Kenta Murata <3959+mrkn@users.noreply.github.com>
2022-08-11 19:16:49 +09:00
Nobuyoshi Nakada 58c8b6e862
Adjust styles [ci skip] 2022-08-06 10:13:20 +09:00
Akinori MUSHA 1a73a6cdd2
Implement Enumerator::Product and Enumerator.product [Feature #18685] 2022-07-30 20:05:14 +09:00
Takashi Kokubun 5b21e94beb Expand tabs [ci skip]
[Misc #18891]
2022-07-21 09:42:04 -07:00
Nobuyoshi Nakada f7ffe9dbde
Introduce `RBIMPL_NONNULL_ARG` macro
Runtime assertion for the argument declared as non-null.
This macro does nothing if `RBIMPL_ATTR_NONNULL` is effective,
otherwise asserts that the argument is non-null.
2021-09-27 14:47:52 +09:00
Nobuyoshi Nakada 8b48b57fd8
Comparing nonnull argument to NULL is useless 2021-09-23 16:55:35 +09:00
S-H-GAMELINKS 83a5e2bb5c Using RB_FLOAT_TYPE_P macro 2021-09-12 11:16:31 +09:00
S-H-GAMELINKS bdd6d8746f Replace RBOOL macro 2021-09-05 23:01:27 +09:00
Nobuyoshi Nakada e4f891ce8d
Adjust styles [ci skip]
* --braces-after-func-def-line
* --dont-cuddle-else
* --procnames-start-lines
* --space-after-for
* --space-after-if
* --space-after-while
2021-06-17 10:13:40 +09:00
Jeremy Evans fd8991f797 Fix lazy enumerator with index size
Fixes [Bug #17889]
2021-05-27 14:17:32 -07:00
Jeremy Evans 9c31fb6114 Fix documentation for Enumerator::Lazy#with_index
If a block is given, it returns a lazy enumerator that will iterate
over the block, it doesn't iterate over the block immediately.

Fixes [Bug #17789]
2021-04-09 10:44:07 -07:00
Jeremy Evans 68d028578a Undef Enumerator::Chain#{feed,next,next_values,peek,peek_values}
Previously these methods were defined but raised TypeError, which
seems worse.
2021-03-06 13:56:16 -08:00
Jeremy Evans e1d16a9e56 Make Enumerator#{+,chain} create lazy chain if any included enumerator is lazy
Implements [Feature #17347]
2021-03-06 13:56:16 -08:00
Jeremy Evans bf40fe9fed Fix calling enumerator methods such as with_index on Enumerator::Chain
This previously raised a TypeError.  Wrap the Enumerator::Chain in
an Enumerator to work around the problem.

Fixes [Bug #17216]
2021-03-06 13:56:16 -08:00
Masataka Pocke Kuwabara 391ee3ee3a Replace `Kernel.#open` with `URI.open` in doc
Because `Kernel.#open` no longer opens URI since Ruby 3.0.
2021-01-08 07:31:27 -08:00
zverok b8d33df1d9 Add Enumerable#compact and Enumerator::Lazy#compact 2021-01-02 17:27:24 +09:00