Граф коммитов

82481 Коммитов

Автор SHA1 Сообщение Дата
NARUSE, Yui 577f9c7a83
Backport 37ed86fd3c (#10248)
merge revision(s) 37ed86fd3c798e298fad9db6e7df1f3f45e1e03b: [Backport #--ticket=20161]

	Fix memory leak in regexp grapheme clusters

	[Bug #20161]

	The cc->mbuf gets overwritten, so we need to free it to not leak memory.

	For example:

	    str = "hello world".encode(Encoding::UTF_32LE)

	    10.times do
	      1_000.times do
	        str.grapheme_clusters
	      end

	      puts `ps -o rss= -p #{$$}`
	    end

	Before:

	    15536
	    15760
	    15920
	    16144
	    16304
	    16480
	    16640
	    16784
	    17008
	    17280

	After:

	    15584
	    15584
	    15760
	    15824
	    15888
	    15888
	    15888
	    15888
	    16048
	    16112
	---
	 regparse.c | 3 ++-
	 1 file changed, 2 insertions(+), 1 deletion(-)
2024-03-14 07:53:14 +00:00
NARUSE, Yui ade02f3c89 merge revision(s) 051a874325c177e040301878069c2b28f5d06ce6: [Backport #20096]
Fix memory overread in registry.rb

	The terminator is not actually getting filled in; we're simply passing
	(two) bytes of empty memory as the NUL terminator. This can lead to
	garbage characters getting written to registry values.

	Fix this by explicitly putting a WCHAR_NUL character into the string to
	be sent to the registry API, like we do in the MULTI_SZ case.

	[Bug #20096]
	---
	 ext/win32/lib/win32/registry.rb | 7 ++-----
	 1 file changed, 2 insertions(+), 5 deletions(-)
2024-03-04 11:57:04 +09:00
Takashi Kokubun 3f0e3ede02
YJIT: Fix exits on splatkw instruction (#9715)
[[Bug #20214]](https://bugs.ruby-lang.org/issues/20214)
2024-02-05 14:51:16 +00:00
Hiroshi SHIBATA ac526abcd6
Merge RubyGems 3.5.5 and Bundler 2.5.5 (#9676)
* Merge RubyGems-3.5.4 and Bundler-2.5.4

* Merge RubyGems-3.5.5 and Bundler-2.5.5

* Make tests play with upstream Ruby tests

CI broke in https://github.com/ruby/ruby/pull/9604 because if any Ruby
tests run `require 'net/http'`, they will pollute the
`$LOADED_FEATURES` for the RubyGems tests. We can fix this by renaming
the test default gem from `net-http` to `my-http`.

See https://github.com/rubygems/rubygems/pull/7379#issuecomment-1901241299
for more details.

---------

Co-authored-by: Stan Hu <stanhu@gmail.com>
2024-02-05 14:51:04 +00:00
Yuta Saito 7f97e3540c
[Backport 3.3] [Bug #20085] Use consistent default options for `-mbranch-protection` (#9385)
[Bug #20085] Use consistent default options for `-mbranch-protection`

We need to use the same options for both C compiler and assembler
when `-mbranch-protection` is guessed by configure. Otherwise,
`coroutine/arm64/Context.{h,S}` will use incompatible PAC strategies.
2024-02-05 14:50:20 +00:00
NARUSE, Yui 288d84d827
merge revision(s) 2554c5d3b8738a248cedb2fea96dfab9fbe19417: [Backport #20231] (#9831)
Don't wait in `io_binwrite_string` if not necessary. (#9792)

	---
	 io.c | 8 +++-----
	 1 file changed, 3 insertions(+), 5 deletions(-)
2024-02-05 08:41:56 +00:00
NARUSE, Yui d7dc57a545
merge revision(s) c5cf4d4e129f64cb69aaf0a829aed068ef1943c4: [Backport#19542] (#9829)
merge revision(s) c5cf4d4e129f64cb69aaf0a829aed068ef1943c4: [Backport #19542]

	Improve behavioural consistency of unallocated (zero length)
	 `IO::Buffer`. (#9532)

	This makes the behaviour of IO::Buffer.new(0) and IO::Buffer.new.slice(0, 0) consistent.

	Fixes https://bugs.ruby-lang.org/issues/19542 and https://bugs.ruby-lang.org/issues/18805.
	---
	 io_buffer.c                 | 14 ++++++--------
	 test/ruby/test_io_buffer.rb | 35 +++++++++++++++++++++++++++++++++++
	 2 files changed, 41 insertions(+), 8 deletions(-)
2024-02-04 13:54:36 +00:00
Takashi Kokubun a065b68bd8
Backport #9415 to ruby_3_3 (#9424)
YJIT: Let RubyVM::YJIT.enable respect --yjit-stats
2024-02-04 05:37:33 +00:00
Maxime Chevalier-Boisvert ba16b34098
YJIT: reduce default exec mem size to 48MiB (#9692)
* YJIT: reduce default exec mem size to 48MiB based

Based on user feedback from @jhawthorn and others.
Better for small and memory-constrained deployments.

NOTE: This commit should be included in the next Ruby 3.3.x point
release. @xrxr should we tag someone specific?

* YJIT: Update yjit.md about mem size (#9687)

---------

Co-authored-by: Takashi Kokubun <takashikkbn@gmail.com>
2024-02-04 05:36:34 +00:00
NARUSE, Yui ce6863a0cf
merge revision(s) 18573b8d054f655e3e8b24902985bf4028f88810: [Backport #20178] (#9822)
Avoid reading unused lvars in Primitive.cexpr

	Previously on builds with optimizations disabled, this could result in
	an out of bounds read. When we had all of:
	* built with -O0
	* Leaf builtin
	* Primitive.mandatory_only
	* "no args builtin", called by vm_call_single_noarg_inline_builti
	* The stack is escaped to the heap via binding or a proc

	This is because mk_builtin_loader generated reads for all locals
	regardless of whether they were used and in the case we generated a
	mandatory_only iseq that would include more variables than were actually
	available.

	On optimized builds, the invalid accesses would be optimized away, and
	this also was often unnoticed as the invalid access would just hit
	another part of the stack unless it had been escaped to the heap.

	The fix here is imperfect, as this could have false positives, but since
	Primitive.cexpr! is only available within the cruby codebase itself
	that's probably fine as a proper fix would be much more challenging (the
	only false positives we found were in rjit.rb).

	Fixes [Bug #20178]

	Co-authored-by: Adam Hess <HParker@github.com>
	---
	 bootstraptest/test_method.rb | 9 +++++++++
	 tool/mk_builtin_loader.rb    | 6 ++++++
	 2 files changed, 15 insertions(+)
2024-02-04 04:13:15 +00:00
NARUSE, Yui 53d4e9c4bb
merge revision(s) 1bd98c820da46a05328d2d53b8f748f28e7ee8f7: [Backport #20172] (#9798)
Remove setaffinity of pthread for getaddrinfo

	It looks like `sched_getcpu(3)` returns a strange number on some
	(virtual?) environments.

	I decided to remove the setaffinity mechanism because the performance
	does not appear to degrade on a quick benchmark even if removed.

	[Bug #20172]
	---
	 ext/socket/extconf.rb  |  2 --
	 ext/socket/raddrinfo.c | 48 ++++--------------------------------------------
	 2 files changed, 4 insertions(+), 46 deletions(-)
2024-02-03 15:47:32 +00:00
NARUSE, Yui 4506461072
Fix test session reuse but expire (#9824)
* Show OpenSSL version in the error message of assert_equal

* OpenSSL 3.2.1 30 Jan 2024 is also broken
2024-02-03 13:35:44 +00:00
NARUSE, Yui 3fb51b93d2
merge revision(s) 82b57d7bfeefd717c10f7a5a3484aca6b3e708a3: [Backport… (#9795)
merge revision(s) 82b57d7bfeefd717c10f7a5a3484aca6b3e708a3: [Backport #20162]

	Fix memory leak when duplicating too complex object

	[Bug #20162]

	Creating a ST table then calling st_replace leaks memory because the
	st_replace overwrites the ST table without freeing any of the existing
	memory. This commit changes it to use st_copy instead.

	For example:

	    RubyVM::Shape.exhaust_shapes

	    o = Object.new
	    o.instance_variable_set(:@a, 0)

	    10.times do
	      100_000.times { o.dup }

	      puts `ps -o rss= -p #{$$}`
	    end

	Before:

	    23264
	    33600
	    42672
	    52160
	    61600
	    71728
	    81056
	    90528
	    100560
	    109840

	After:

	    14752
	    14816
	    15584
	    15584
	    15664
	    15664
	    15664
	    15664
	    15664
	    15664
	---
	 object.c                 |  3 +--
	 test/ruby/test_shapes.rb | 13 +++++++++++++
	 2 files changed, 14 insertions(+), 2 deletions(-)
2024-02-01 07:19:48 +00:00
NARUSE, Yui 7231fc5baa
merge revision(s) 597955a,8b65d15: [Backport #20173] (#9794)
Fix to work match cache with peek next optimization (#9459)

	---
	 regexec.c                | 3 ++-
	 test/ruby/test_regexp.rb | 9 +++++++++
	 2 files changed, 11 insertions(+), 1 deletion(-)

	Fix test case for `test_match_cache_with_peek_optimization` (#9466)

	---
	 test/ruby/test_regexp.rb | 2 +-
	 1 file changed, 1 insertion(+), 1 deletion(-)
2024-02-01 07:13:50 +00:00
Hiroshi SHIBATA 57d8d6e58a
Backport bundled_gems.rb for Ruby 3.3 (#9457)
racc is extracted at Ruby 3.3, not 3.4
2024-02-01 04:34:32 +00:00
KJ Tsanaktsidis 920c17dc94
Backport #20157 to Ruby 3.3 (#9428)
* Fix GC.measure_total_time regression

Commit 93ac7405b8 introduced a regression
where measurements would still be taken after setting
GC.measure_total_time = false.

Fixes [Bug #20157]

* Add test case for GC.measure_total_time

---------

Co-authored-by: Rian McGuire <rian@rian.id.au>
2024-02-01 04:33:30 +00:00
NARUSE, Yui 2886564279 merge revision(s) 6aacbd690ccde53f9b97c6673482cb11df3f2955: [Backport #20149]
Free pthread_attr after setting up the thread

	[bug #20149]
	---
	 ext/socket/raddrinfo.c | 12 +++++++++++-
	 1 file changed, 11 insertions(+), 1 deletion(-)
2024-02-01 10:39:54 +09:00
NARUSE, Yui aeffb5e21d merge revision(s) 6c252912af4981f016a9abdb4c1689307a4f1d2f: [Backport #20145]
Memory leak when duplicating identhash

	[Bug #20145]

	Before this commit, both copy_compare_by_id and hash_copy will create a
	copy of the ST table, so the ST table created in copy_compare_by_id will
	be leaked.

	    h = { 1 => 2 }.compare_by_identity

	    10.times do
	      1_000_000.times do
	        h.select { false }
	      end

	      puts `ps -o rss= -p #{$$}`
	    end

	Before:

	    110736
	    204352
	    300272
	    395520
	    460704
	    476736
	    542000
	    604704
	    682624
	    770528

	After:

	    15504
	    16048
	    16144
	    16256
	    16320
	    16320
	    16752
	    16752
	    16752
	    16752
	---
	 hash.c                 | 10 +++++++++-
	 test/ruby/test_hash.rb | 10 ++++++++++
	 2 files changed, 19 insertions(+), 1 deletion(-)
2024-02-01 09:08:06 +09:00
NARUSE, Yui f585171a6b merge revision(s) e12d4c654e3cb7a4473014610bc3bae41aaf811e: [Backport #20104]
Don't create T_MATCH object if /regexp/.match(string) doesn't match

	Fixes [Bug #20104]
	---
	 re.c                     |  9 ++++++---
	 test/ruby/test_regexp.rb | 12 ++++++++++++
	 tool/lib/envutil.rb      |  8 ++++++++
	 3 files changed, 26 insertions(+), 3 deletions(-)
2024-01-30 20:29:02 +09:00
NARUSE, Yui f8f0d342e4 merge revision(s) 3d19409637de1462b6790d2a92344bf0a10d8c52: [Backport #20090]
Use index for referring to symbols in `args` rule instead of named
	 references

	In `args: args ',' arg_splat`, `args` is not unique name.
	Currently the associated rule is interpreted as
	`$$ = rest_arg_append(p, $$, $3, &@$);`.
	The action works as expected because `$$` is initialized with
	`$1` before each action is executed.
	However it's misleading then change to use index.
	---
	 parse.y | 4 ++--
	 1 file changed, 2 insertions(+), 2 deletions(-)
2024-01-30 18:57:00 +09:00
NARUSE, Yui 9f18cbd796 Revert "merge revision(s) bc002971b6ad483dbf69b8a275c44412bb6ab954: [Backport #20094]"
This reverts commit d4b780e84e.
2024-01-30 18:00:47 +09:00
NARUSE, Yui d4b780e84e merge revision(s) bc002971b6ad483dbf69b8a275c44412bb6ab954: [Backport #20094]
[Bug #20094] Distinguish `begin` and parentheses

	---
	 compile.c                    |  1 +
	 parse.y                      | 36 +++++++++++++++++++++---------------
	 test/ruby/test_whileuntil.rb | 18 ++++++++++++++++++
	 3 files changed, 40 insertions(+), 15 deletions(-)
2024-01-30 17:41:31 +09:00
NARUSE, Yui 5f3dfa1c27 merge revision(s) d8702ddbfbe8cc7fc601a9a4d19842ef9c2b76c1: [Backport #20083]
Fix [Bug #20083]: correct a cache point size for atomic groups
	 (#9367)

	---
	 regexec.c                | 2 +-
	 test/ruby/test_regexp.rb | 8 ++++++++
	 2 files changed, 9 insertions(+), 1 deletion(-)
2024-01-30 10:31:15 +09:00
NARUSE, Yui 818b4ea9b1 merge revision(s) e5a4f757bdf5dc3d8c329ddd268432f9ecc7bff6: [Backport #20086]
Fix Window private file mapping unlink EACCES issue. (#9358)

	* Don't return early.

	* Add missing `mapping` assignment.

	* Make debug logs conditional.
	---
	 io_buffer.c                 | 18 ++++++++++++------
	 test/ruby/test_io_buffer.rb | 32 ++++++++++++++------------------
	 2 files changed, 26 insertions(+), 24 deletions(-)
2024-01-30 10:24:31 +09:00
Hiroshi SHIBATA 634d4e29ef
Update net-* gems for Ruby 3.3 (#9418)
* Bump up net-ftp to 0.3.4

* Bump up net-smtp to 0.4.0.1

* Bump up net-imap to 0.4.9.1

* [ruby/net-http] Renew test certificates

The private key is replaced with a public known test key published at
[RFC 9500].

Also lifetime has been extended to 10 years from 4 years.

[RFC 9500]: https://www.rfc-editor.org/rfc/rfc9500.html

https://github.com/ruby/net-http/commit/4ab6c4a500

* Bump up net-http to 0.4.1

---------

Co-authored-by: Sorah Fukumori <her@sorah.jp>
2024-01-05 22:54:57 +09:00
NARUSE, Yui 5124f9ac75 v3.3.0 2023-12-25 14:59:38 +09:00
Nobuyoshi Nakada a44a59dbaf
[DOC] Add NEWS for [Feature #19370] 2023-12-25 14:52:06 +09:00
Nobuyoshi Nakada a9f0961831 [Feature #19370] Prohibit nesting anonymous parameter forwarding 2023-12-25 14:44:04 +09:00
Nobuyoshi Nakada b641b7e640
Move non-portable pragmas to rubyspec.h
In the extension libraries in spec/ruby/optional/capi, do not care
about deprecated declarations.
2023-12-25 14:11:49 +09:00
Hiroshi SHIBATA 863ded45a1
Typofix under bootstraptest, spec and yjit directories 2023-12-25 13:50:23 +09:00
Nobuyoshi Nakada 2cdbeb29e6
Do not leave test file
Run this test separately because something seems remained unreleased
on Windows.
2023-12-25 13:01:55 +09:00
Peter Zhu b4efa4b700 Don't copy RUBY_FL_PROMOTED flag in rb_obj_setup
RUBY_FL_PROMOTED is used by the garbage collector to track when an
object becomes promoted to the old generation. rb_obj_setup must not
copy that flag over because then it may become out-of-sync with the age
of the object.

This fixes a bug in Method#clone where the cloned Method object may get
RUBY_FL_PROMOTED incorrectly set.
2023-12-24 22:13:49 -05:00
Hiroshi SHIBATA 1b5f3dd6a1
Removed empty sections 2023-12-25 12:04:28 +09:00
Hiroshi SHIBATA 27ead9907d
Typofix under lib and test, tool directories 2023-12-25 11:32:42 +09:00
Peter Zhu 7002e77694 Fix Symbol#inspect for GC compaction
The test fails when RGENGC_CHECK_MODE is turned on:

    1) Failure:
    TestSymbol#test_inspect_under_gc_compact_stress [test/ruby/test_symbol.rb:123]:
    <":testing"> expected but was
    <":\x00\x00\x00\x00\x00\x00\x00">.
2023-12-24 21:29:40 -05:00
Hiroshi SHIBATA e233730846
Partly reverted c903cddf55
These are intentional words
2023-12-25 11:22:12 +09:00
Hiroshi SHIBATA c903cddf55
Typofix under doc directory 2023-12-25 11:17:51 +09:00
Peter Zhu 70618a48f7 Fix off-by-one error for declarative marking
The for loops for marking and reference updating declaratively marked
TypedData objects did not mark/reference update the very last element.

When RGENGC_CHECK_MODE is turned on, this caused the test in Enumerator
to fail with:

    tool/lib/test/unit/testcase.rb:173:in `rescue in run': failed to allocate memory (NoMemoryError)
2023-12-24 20:37:59 -05:00
Samuel Williams 260bf60e52
Correctly release the underlying file mapping. (#9340)
* Avoiding using `Tempfile` which was retaining the file preventing it from unlinking.
2023-12-25 14:20:53 +13:00
Peter Zhu 5af64ff7db Fix Enumerator#with_index for GC compaction
enumerator_block_call was not safe for compaction because the Array
backing the argv was not pinned, so it could get moved during compaction
which would make argv point to somewhere else.

The test crashes when RGENGC_CHECK_MODE is turned on:

    TestEnumerator#test_with_index_under_gc_compact_stress
    check_rvalue_consistency: 0x55db0b399450 is not a Ruby object.
    test/ruby/test_enumerator.rb:133: [BUG] check_rvalue_consistency_force: there is 1 errors.
    ruby 3.3.0dev (2023-12-23T23:00:27Z master 50bf437341) [x86_64-linux]
    -- Control frame information -----------------------------------------------
    c:0034 p:---- s:0192 e:000187 CFUNC  :with_index
    c:0033 p:---- s:0185 e:000184 CFUNC  :each
    c:0032 p:---- s:0182 e:000181 CFUNC  :to_a
    c:0031 p:0055 s:0178 e:000175 BLOCK  test/ruby/test_enumerator.rb:133
    c:0030 p:0024 s:0172 e:000171 METHOD tool/lib/envutil.rb:242
    c:0029 p:0024 s:0167 e:000166 METHOD tool/lib/envutil.rb:251
    c:0028 p:0005 s:0160 e:000159 METHOD test/ruby/test_enumerator.rb:131
    ...
    -- C level backtrace information -------------------------------------------
    build/ruby(rb_print_backtrace+0x14) [0x55db0b1deb21] vm_dump.c:820
    build/ruby(rb_vm_bugreport) vm_dump.c:1151
    build/ruby(bug_report_end+0x0) [0x55db0b3a53a6] error.c:1042
    build/ruby(rb_bug_without_die) error.c:1042
    build/ruby(die+0x0) [0x55db0afc77c2] error.c:1050
    build/ruby(rb_bug) error.c:1052
    build/ruby(gc_move+0x0) [0x55db0afbada0] gc.c:1714
    build/ruby(check_rvalue_consistency+0xa) [0x55db0afef0c3] gc.c:1729
    build/ruby(is_markable_object) gc.c:4769
    build/ruby(gc_mark_stack_values) gc.c:6595
    build/ruby(rb_gc_mark_vm_stack_values) gc.c:6605
    build/ruby(rb_execution_context_mark+0x39) [0x55db0b1d8589] vm.c:3309
    build/ruby(thread_mark+0x15) [0x55db0b1a9805] vm.c:3381
    build/ruby(gc_mark_stacked_objects+0x6d) [0x55db0aff2c3d] gc.c:7564
    build/ruby(gc_mark_stacked_objects_all) gc.c:7602
    build/ruby(gc_marks_rest) gc.c:8797
    build/ruby(gc_marks+0xd) [0x55db0aff43d5] gc.c:8855
    build/ruby(gc_start) gc.c:9608
    build/ruby(rb_multi_ractor_p+0x0) [0x55db0aff5463] gc.c:9489
    build/ruby(rb_vm_lock_leave) vm_sync.h:92
    build/ruby(garbage_collect) gc.c:9491
    build/ruby(newobj_slowpath+0xcb) [0x55db0aff57ab] gc.c:2871
    build/ruby(newobj_slowpath_wb_protected) gc.c:2895
    build/ruby(newobj_of0+0x24) [0x55db0aff59e4] gc.c:2937
    build/ruby(newobj_of) gc.c:2947
    build/ruby(rb_wb_protected_newobj_of) gc.c:2962
    build/ruby(ary_alloc_embed+0x10) [0x55db0b2f3e40] array.c:668
    build/ruby(ary_new) array.c:709
    build/ruby(rb_ary_tmp_new_from_values) array.c:759
    build/ruby(rb_ary_new_from_values) array.c:771
    build/ruby(args_copy+0x18) [0x55db0b1bbb88] vm_args.c:158
2023-12-24 16:22:34 -05:00
BurdetteLamar 86ef38194b RDoc for complex.c 2023-12-24 14:56:06 -05:00
Peter Zhu f0efeddd41 Fix Regexp#inspect for GC compaction
rb_reg_desc was not safe for GC compaction because it took in the C
string and length but not the backing String object so it get moved
during compaction. This commit changes rb_reg_desc to use the string
from the Regexp object.

The test fails when RGENGC_CHECK_MODE is turned on:

    TestRegexp#test_inspect_under_gc_compact_stress [test/ruby/test_regexp.rb:474]:
    <"(?-mix:\\/)|"> expected but was
    <"/\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00/">.
2023-12-24 11:04:41 -05:00
BurdetteLamar 8ad8803bb4 RDoc for Complex 2023-12-24 11:00:30 -05:00
Peter Zhu 42442ed789 Fix Regexp#match for GC compaction
The test fails when RGENGC_CHECK_MODE is turned on:

    TestRegexp#test_match_under_gc_compact_stress:
    NoMethodError: undefined method `match' for nil
        test_regexp.rb:878:in `block in test_match_under_gc_compact_stress'
2023-12-24 09:03:55 -05:00
Samuel Williams 37753f163e
IO::Buffer improvements and documentation. (#9329)
* Restore experimental warnings.

* Documentation and code structure improvements.

* Improved validation of flags, clarified documentation of argument handling.

* Remove inconsistent use of `Example:` and add example to `null?`.

* Expose `IO::Buffer#private?` and add test.
2023-12-25 02:03:36 +13:00
Nobuyoshi Nakada 61289d9405
Initialize loop variable of `ccan_list_for_each`
On platforms where `typeof` is unsupported, `ccan_container_off_var`
calculates the offset of member by pointer subtraction.  Although this
will result in the compile-time invariant value regardless the pointer
value, the loop variable will be used before assignment and may cause
an using uninitialized variable warning.
2023-12-24 19:34:25 +09:00
git 86893b28f7 Update bundled gems list as of 2023-12-24 2023-12-24 06:58:23 +00:00
Koichi Sasada a87ae242bb Use noinline version of accessing current ec
On universal.arm64e-darwin22 with clang 15.0.0 (I didn't check
details yet) accessing `ruby_current_ec` directly causes
assertion violation `VM_ASSERT(ec == rb_current_ec_noinline())`
on `rb_current_execution_context()`, maybe because TLS accessing
issue.
2023-12-24 15:13:33 +09:00
Koichi Sasada 541371e286 accept `RB_WAITFD_IN | RB_WAITFD_OUT` for waiting events
Assrsion was `events == RB_WAITFD_IN || events == RB_WAITFD_OUT`
but it should accept `RB_WAITFD_IN | RB_WAITFD_OUT`.
2023-12-24 14:53:46 +09:00
Nobuyoshi Nakada 0fef890b4f
[DOC] Fix markup in HTML
Inside HTML is not markdown.
2023-12-24 13:59:34 +09:00