On mark we check whether a callcache has been invalidated and if it has
we replace it with the empty callcache, rb_vm_empty_cc(). However we
also consider the empty callcache to not be active, and so previously
would overwrite it with itself.
These additional writes are problematic because they may force
Copy-on-Write to occur on the memory page, increasing system memory use.
Fix regex timeout double-free after stack_double
As of 10574857ce, it's possible to crash
on a double free due to `stk_alloc` AKA `msa->stack_p` being freed
twice, once at the end of match_at and a second time in `FREE_MATCH_ARG`
in the parent caller.
Fixes [Bug #20886]
Fix update_coderange for binary strings
Although a binary (aka ASCII-8BIT) string will never have a broken
coderange, it still has to differentiate between "valid" and "7bit".
On Ruby 3.4/trunk this problem is masked because we now clear the
coderange more agressively in rb_str_resize, and we happened to always
be strinking this string, but we should not assume that.
On Ruby 3.3 this created strings where `ascii_only?` was true in cases
it shouldn't be as well as other problems.
Fixes [Bug #20883]
Co-authored-by: Daniel Colson <danieljamescolson@gmail.com>
Co-authored-by: Matthew Draper <matthew@trebex.net>
These test failures first appeared on irrelevant changes. It probably
came from changes in GitHub Actions instead of CRuby's.
Until we figure out how to fix these tests, let's skip them to make the
CI usable.
[Bug #20853] Fix Proc#hash to not change after compaction
The hash value of a Proc must remain constant after a compaction, otherwise
it may not work as the key in a hash table.
Fix method caching bug when including/prepend module A that prepends module B
Fix by always adding the generated iclass to the subclasses list,
otherwise the method cache for the iclass is not cleared when
the method in the module is overwritten.
Fixes [Bug #20716]
[Bug #20704] Win32: Fix chdir to non-ASCII path
On Windows, `chdir` in compilers' runtime libraries uses the active
code page, but command line arguments in ruby are always UTF-8, since
commit:33ea2646b98adb49ae2e1781753bf22d33729ac0.
[win32/registry] Fallback to UTF-8 for unknown codepages
There are some codepages like cp708 for which no ruby encoding exists:
$ ruby -e "Encoding.find('cp708')"
Traceback (most recent call last):
1: from -e:1:in `<main>'
-e:1:in `find': unknown encoding name - cp708 (ArgumentError)
win32/registry uses ENCODING to transcode error messages and expand environment
variables from UTF-16LE, so using UTF-8 seems like the best choice and is better
than a hard failure.
This should resolve [Bug #13831]
If one thread is reading and another closes that socket, the close
blocks waiting for the read to abort cleanly. This ensures that Ruby is
totally done with the file descriptor _BEFORE_ we tell the OS to close
and potentially re-use it.
When the read is correctly terminated, the close should be unblocked.
That currently works if closing is happening on a thread, but if it's
happening on a fiber with a fiber scheduler, it does NOT work.
This patch ensures that if the close happened in a fiber scheduled
thread, that the scheduler is notified that the fiber is unblocked.
[Bug #20723]
Fixes [Bug #20718]
Allow objects that are not of type `RTypedData` to use the default free function, as `RTYPEDDATA_EMBEDDED_P` can return a false positive when casting non-`RTypedData` objects
Fix use-after-free in WeakKeyMap#clear
[Bug #20691]
If the WeakKeyMap has been marked but sweeping hasn't started yet and we
cann WeakKeyMap#clear, then there could be a use-after-free because we do
not call rb_gc_remove_weak to remove the key from the GC.
For example, the following code triggers use-after-free errors in Valgrind:
map = ObjectSpace::WeakKeyMap.new
1_000.times do
1_000.times do
map[Object.new] = nil
end
map.clear
end
Output from Valgrind:
==61230== Invalid read of size 8
==61230== at 0x25CAF8: gc_update_weak_references (default.c:5593)
==61230== by 0x25CAF8: gc_marks_finish (default.c:5641)
==61230== by 0x26031C: gc_marks_continue (default.c:5987)
==61230== by 0x26031C: gc_continue (default.c:2255)
==61230== by 0x2605FC: newobj_cache_miss (default.c:2589)
==61230== by 0x26111F: newobj_alloc (default.c:2622)
==61230== by 0x26111F: rb_gc_impl_new_obj (default.c:2701)
==61230== by 0x26111F: newobj_of (gc.c:890)
==61230== by 0x26111F: rb_wb_protected_newobj_of (gc.c:917)
==61230== by 0x2DE218: rb_class_allocate_instance (object.c:131)
==61230== by 0x2E32A8: class_call_alloc_func (object.c:2141)
==61230== by 0x2E32A8: rb_class_alloc (object.c:2113)
==61230== by 0x2E32A8: rb_class_new_instance_pass_kw (object.c:2172)
==61230== by 0x4296BC: vm_call_cfunc_with_frame_ (vm_insnhelper.c:3788)
==61230== by 0x44A9CD: vm_sendish (vm_insnhelper.c:5955)
==61230== by 0x44A9CD: vm_exec_core (insns.def:898)
==61230== by 0x43A0E4: rb_vm_exec (vm.c:2564)
==61230== by 0x2341B4: rb_ec_exec_node (eval.c:281)
==61230== by 0x236258: ruby_run_node (eval.c:319)
==61230== by 0x15D665: rb_main (main.c:43)
==61230== by 0x15D665: main (main.c:62)
==61230== Address 0x2159cb00 is 0 bytes inside a block of size 8 free'd
==61230== at 0x4849B2C: free (vg_replace_malloc.c:989)
==61230== by 0x248EF1: rb_gc_impl_free (default.c:8512)
==61230== by 0x248EF1: rb_gc_impl_free (default.c:8493)
==61230== by 0x248EF1: ruby_sized_xfree.constprop.0 (gc.c:4178)
==61230== by 0x4627EC: wkmap_free_table_i (weakmap.c:652)
==61230== by 0x3A54AF: apply_functor (st.c:1633)
==61230== by 0x3A54AF: st_general_foreach (st.c:1543)
==61230== by 0x3A54AF: rb_st_foreach (st.c:1640)
==61230== by 0x46203C: wkmap_clear (weakmap.c:973)
==61230== by 0x4296BC: vm_call_cfunc_with_frame_ (vm_insnhelper.c:3788)
==61230== by 0x44A9CD: vm_sendish (vm_insnhelper.c:5955)
==61230== by 0x44A9CD: vm_exec_core (insns.def:898)
==61230== by 0x43A0E4: rb_vm_exec (vm.c:2564)
==61230== by 0x2341B4: rb_ec_exec_node (eval.c:281)
==61230== by 0x236258: ruby_run_node (eval.c:319)
==61230== by 0x15D665: rb_main (main.c:43)
==61230== by 0x15D665: main (main.c:62)
==61230== Block was alloc'd at
==61230== at 0x484680F: malloc (vg_replace_malloc.c:446)
==61230== by 0x25C68E: rb_gc_impl_malloc (default.c:8527)
==61230== by 0x4622E9: wkmap_aset_replace (weakmap.c:817)
==61230== by 0x3A4D02: rb_st_update (st.c:1487)
==61230== by 0x4623E4: wkmap_aset (weakmap.c:854)
==61230== by 0x4296BC: vm_call_cfunc_with_frame_ (vm_insnhelper.c:3788)
==61230== by 0x44A9CD: vm_sendish (vm_insnhelper.c:5955)
==61230== by 0x44A9CD: vm_exec_core (insns.def:898)
==61230== by 0x43A0E4: rb_vm_exec (vm.c:2564)
==61230== by 0x2341B4: rb_ec_exec_node (eval.c:281)
==61230== by 0x236258: ruby_run_node (eval.c:319)
==61230== by 0x15D665: rb_main (main.c:43)
==61230== by 0x15D665: main (main.c:62)
==61230==
==61230== Invalid write of size 8
==61230== at 0x25CB3B: gc_update_weak_references (default.c:5598)
==61230== by 0x25CB3B: gc_marks_finish (default.c:5641)
==61230== by 0x26031C: gc_marks_continue (default.c:5987)
==61230== by 0x26031C: gc_continue (default.c:2255)
==61230== by 0x2605FC: newobj_cache_miss (default.c:2589)
==61230== by 0x26111F: newobj_alloc (default.c:2622)
==61230== by 0x26111F: rb_gc_impl_new_obj (default.c:2701)
==61230== by 0x26111F: newobj_of (gc.c:890)
==61230== by 0x26111F: rb_wb_protected_newobj_of (gc.c:917)
==61230== by 0x2DE218: rb_class_allocate_instance (object.c:131)
==61230== by 0x2E32A8: class_call_alloc_func (object.c:2141)
==61230== by 0x2E32A8: rb_class_alloc (object.c:2113)
==61230== by 0x2E32A8: rb_class_new_instance_pass_kw (object.c:2172)
==61230== by 0x4296BC: vm_call_cfunc_with_frame_ (vm_insnhelper.c:3788)
==61230== by 0x44A9CD: vm_sendish (vm_insnhelper.c:5955)
==61230== by 0x44A9CD: vm_exec_core (insns.def:898)
==61230== by 0x43A0E4: rb_vm_exec (vm.c:2564)
==61230== by 0x2341B4: rb_ec_exec_node (eval.c:281)
==61230== by 0x236258: ruby_run_node (eval.c:319)
==61230== by 0x15D665: rb_main (main.c:43)
==61230== by 0x15D665: main (main.c:62)
==61230== Address 0x2159cb00 is 0 bytes inside a block of size 8 free'd
==61230== at 0x4849B2C: free (vg_replace_malloc.c:989)
==61230== by 0x248EF1: rb_gc_impl_free (default.c:8512)
==61230== by 0x248EF1: rb_gc_impl_free (default.c:8493)
==61230== by 0x248EF1: ruby_sized_xfree.constprop.0 (gc.c:4178)
==61230== by 0x4627EC: wkmap_free_table_i (weakmap.c:652)
==61230== by 0x3A54AF: apply_functor (st.c:1633)
==61230== by 0x3A54AF: st_general_foreach (st.c:1543)
==61230== by 0x3A54AF: rb_st_foreach (st.c:1640)
==61230== by 0x46203C: wkmap_clear (weakmap.c:973)
==61230== by 0x4296BC: vm_call_cfunc_with_frame_ (vm_insnhelper.c:3788)
==61230== by 0x44A9CD: vm_sendish (vm_insnhelper.c:5955)
==61230== by 0x44A9CD: vm_exec_core (insns.def:898)
==61230== by 0x43A0E4: rb_vm_exec (vm.c:2564)
==61230== by 0x2341B4: rb_ec_exec_node (eval.c:281)
==61230== by 0x236258: ruby_run_node (eval.c:319)
==61230== by 0x15D665: rb_main (main.c:43)
==61230== by 0x15D665: main (main.c:62)
==61230== Block was alloc'd at
==61230== at 0x484680F: malloc (vg_replace_malloc.c:446)
==61230== by 0x25C68E: rb_gc_impl_malloc (default.c:8527)
==61230== by 0x4622E9: wkmap_aset_replace (weakmap.c:817)
==61230== by 0x3A4D02: rb_st_update (st.c:1487)
==61230== by 0x4623E4: wkmap_aset (weakmap.c:854)
==61230== by 0x4296BC: vm_call_cfunc_with_frame_ (vm_insnhelper.c:3788)
==61230== by 0x44A9CD: vm_sendish (vm_insnhelper.c:5955)
==61230== by 0x44A9CD: vm_exec_core (insns.def:898)
==61230== by 0x43A0E4: rb_vm_exec (vm.c:2564)
==61230== by 0x2341B4: rb_ec_exec_node (eval.c:281)
==61230== by 0x236258: ruby_run_node (eval.c:319)
==61230== by 0x15D665: rb_main (main.c:43)
==61230== by 0x15D665: main (main.c:62)
Co-authored-by: Jean Boussier <byroot@ruby-lang.org>
* Add struct weakmap_entry for WeakMap entries
* Refactor wmap_foreach to pass weakmap_entry
* Use wmap_foreach for wmap_mark
* Refactor wmap_compact to use wmap_foreach
* Remove wmap_free_entry
* Fix WeakMap use-after-free
[Bug #20688]
We cannot free the weakmap_entry before the ST_DELETE because it could
hash the key which would read the weakmap_entry and would cause a
use-after-free. Instead, we store the entry and free it on the next
iteration.
For example, the following script triggers a use-after-free in Valgrind:
weakmap = ObjectSpace::WeakMap.new
10_000.times { weakmap[Object.new] = Object.new }
==25795== Invalid read of size 8
==25795== at 0x462297: wmap_cmp (weakmap.c:165)
==25795== by 0x3A2B1C: find_table_bin_ind (st.c:930)
==25795== by 0x3A5EAA: st_general_foreach (st.c:1599)
==25795== by 0x3A5EAA: rb_st_foreach (st.c:1640)
==25795== by 0x25C991: gc_mark_children (default.c:4870)
==25795== by 0x25C991: gc_marks_wb_unprotected_objects_plane (default.c:5565)
==25795== by 0x25C991: rgengc_rememberset_mark_plane (default.c:5557)
==25795== by 0x25C991: rgengc_rememberset_mark (default.c:6233)
==25795== by 0x25C991: gc_marks_start (default.c:6057)
==25795== by 0x25C991: gc_marks (default.c:6077)
==25795== by 0x25C991: gc_start (default.c:6723)
==25795== by 0x260F96: heap_prepare (default.c:2282)
==25795== by 0x260F96: heap_next_free_page (default.c:2489)
==25795== by 0x260F96: newobj_cache_miss (default.c:2598)
==25795== by 0x26197F: newobj_alloc (default.c:2622)
==25795== by 0x26197F: rb_gc_impl_new_obj (default.c:2701)
==25795== by 0x26197F: newobj_of (gc.c:890)
==25795== by 0x26197F: rb_wb_protected_newobj_of (gc.c:917)
==25795== by 0x2DEA88: rb_class_allocate_instance (object.c:131)
==25795== by 0x2E3B18: class_call_alloc_func (object.c:2141)
==25795== by 0x2E3B18: rb_class_alloc (object.c:2113)
==25795== by 0x2E3B18: rb_class_new_instance_pass_kw (object.c:2172)
==25795== by 0x429DDC: vm_call_cfunc_with_frame_ (vm_insnhelper.c:3786)
==25795== by 0x44B08D: vm_sendish (vm_insnhelper.c:5953)
==25795== by 0x44B08D: vm_exec_core (insns.def:898)
==25795== by 0x43A7A4: rb_vm_exec (vm.c:2564)
==25795== by 0x234914: rb_ec_exec_node (eval.c:281)
==25795== Address 0x21603710 is 0 bytes inside a block of size 16 free'd
==25795== at 0x4849B2C: free (vg_replace_malloc.c:989)
==25795== by 0x249651: rb_gc_impl_free (default.c:8527)
==25795== by 0x249651: rb_gc_impl_free (default.c:8508)
==25795== by 0x249651: ruby_sized_xfree.constprop.0 (gc.c:4178)
==25795== by 0x4626EC: ruby_sized_xfree_inlined (gc.h:277)
==25795== by 0x4626EC: wmap_free_entry (weakmap.c:45)
==25795== by 0x4626EC: wmap_mark_weak_table_i (weakmap.c:61)
==25795== by 0x3A5CEF: apply_functor (st.c:1633)
==25795== by 0x3A5CEF: st_general_foreach (st.c:1543)
==25795== by 0x3A5CEF: rb_st_foreach (st.c:1640)
==25795== by 0x25C991: gc_mark_children (default.c:4870)
==25795== by 0x25C991: gc_marks_wb_unprotected_objects_plane (default.c:5565)
==25795== by 0x25C991: rgengc_rememberset_mark_plane (default.c:5557)
==25795== by 0x25C991: rgengc_rememberset_mark (default.c:6233)
==25795== by 0x25C991: gc_marks_start (default.c:6057)
==25795== by 0x25C991: gc_marks (default.c:6077)
==25795== by 0x25C991: gc_start (default.c:6723)
==25795== by 0x260F96: heap_prepare (default.c:2282)
==25795== by 0x260F96: heap_next_free_page (default.c:2489)
==25795== by 0x260F96: newobj_cache_miss (default.c:2598)
==25795== by 0x26197F: newobj_alloc (default.c:2622)
==25795== by 0x26197F: rb_gc_impl_new_obj (default.c:2701)
==25795== by 0x26197F: newobj_of (gc.c:890)
==25795== by 0x26197F: rb_wb_protected_newobj_of (gc.c:917)
==25795== by 0x2DEA88: rb_class_allocate_instance (object.c:131)
==25795== by 0x2E3B18: class_call_alloc_func (object.c:2141)
==25795== by 0x2E3B18: rb_class_alloc (object.c:2113)
==25795== by 0x2E3B18: rb_class_new_instance_pass_kw (object.c:2172)
==25795== by 0x429DDC: vm_call_cfunc_with_frame_ (vm_insnhelper.c:3786)
==25795== by 0x44B08D: vm_sendish (vm_insnhelper.c:5953)
==25795== by 0x44B08D: vm_exec_core (insns.def:898)
==25795== by 0x43A7A4: rb_vm_exec (vm.c:2564)
==25795== Block was alloc'd at
==25795== at 0x484680F: malloc (vg_replace_malloc.c:446)
==25795== by 0x25CE9E: rb_gc_impl_malloc (default.c:8542)
==25795== by 0x462A39: wmap_aset_replace (weakmap.c:423)
==25795== by 0x3A5542: rb_st_update (st.c:1487)
==25795== by 0x462B8E: wmap_aset (weakmap.c:452)
==25795== by 0x429DDC: vm_call_cfunc_with_frame_ (vm_insnhelper.c:3786)
==25795== by 0x44B08D: vm_sendish (vm_insnhelper.c:5953)
==25795== by 0x44B08D: vm_exec_core (insns.def:898)
==25795== by 0x43A7A4: rb_vm_exec (vm.c:2564)
==25795== by 0x234914: rb_ec_exec_node (eval.c:281)
==25795== by 0x2369B8: ruby_run_node (eval.c:319)
==25795== by 0x15D675: rb_main (main.c:43)
==25795== by 0x15D675: main (main.c:62)
* Fix use-after-free for WeakKeyMap
[Bug #20688]
We cannot free the key before the ST_DELETE because it could hash the
key which would read the key and would cause a use-after-free. Instead,
we store the key and free it on the next iteration.
* Make sure to always use the right `warn`
* lib/bundled_gems.rb: more reliable caller detection
The `2` skipped frames went out of sync and now it should be `3`.
Rather than just update the offset, we can implement a way that
is adaptative as long as all require decorators are also called require.
Also we should compute the corresponding `uplevel` otherwise the
warning will still point decorators.
Co-authored-by: "Hiroshi SHIBATA" <hsbt@ruby-lang.org>
* Warn ostruct for Ruby 3.5
* Warn pstore for Ruby 3.5
* Mark rdoc as bundled gems at Ruby 3.5
* Warn to use win32ole without Gemfile for Ruby 3.5
* EXACT list is mostly same as SINCE list on bundled gems.
* Mark to warn fiddle as bundled gems for Ruby 3.5
* Mark to warn logger as bundled gems for Ruby 3.5
* We should use uplevel:2 in another case.
Like the following scenario with bootsnap, that frames are same or smaller than frame_to_skip(=3).
---
"/Users/hsbt/.local/share/rbenv/versions/3.3-dev/lib/ruby/3.3.0/bundled_gems.rb:69:in `block (2 levels) in replace_require'"
"/Users/hsbt/.local/share/gem/gems/bootsnap-1.18.4/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:30:in `require'"
"test_warn_bootsnap.rb:11:in `<main>'"
---
* Delete unnecessary rubocop disable comment
* Show correct script name with sub-feature case
* Skip to show script name with using ruby -r option
* Don't show script name when bundle exec and call ruby script directly.
* Pick word fix from 34adc07372
---------
Co-authored-by: David Rodríguez <deivid.rodriguez@riseup.net>
Co-authored-by: Jean Boussier <jean.boussier@gmail.com>
Co-authored-by: Kentaro Takeyama <75117116+obregonia1@users.noreply.github.com>
[Bug #20633] Re-initialize vm->ractor.sched.lock after fork
Previously under certain conditions it was possible to encounter a
deadlock in the forked child process if ractor.sched.lock was held.
Co-authored-by: Nathan Froyd <froydnj@gmail.com>
[Bug #20668]
The `dest` node is assumed to be a `CDECL`, so overwriting it with
a `LIT` cause a crash on the next iteration.
Co-authored-by: Jean Boussier <jean.boussier@gmail.com>
* Merge RubyGems-3.5.12 and Bundler-2.5.12
* Merge RubyGems-3.5.13 and Bundler-2.5.13
* Merge RubyGems-3.5.14 and Bundler-2.5.14
* Merge RubyGems-3.5.15 and Bundler-2.5.15
* Merge RubyGems-3.5.16 and Bundler-2.5.16
John Hawthorn
Takashi Kokubun
Jonathan Calvert
Alex Myers
Hiroshi SHIBATA
Nobuyoshi Nakada
KJ Tsanaktsidis
Peter Zhu
Jean byroot Boussier