* YJIT: Specialize `String#[]` (`String#slice`) with fixnum arguments
String#[] is in the top few C calls of several YJIT benchmarks:
liquid-compile rubocop mail sudoku
This speeds up these benchmarks by 1-2%.
* YJIT: Try harder to get type info for `String#[]`
In the large generated code of the mail gem the context doesn't have
the type info. In that case if we peek at the stack and add a guard
we can still apply the specialization
and it speeds up the mail benchmark by 5%.
Co-authored-by: Maxime Chevalier-Boisvert <maxime.chevalierboisvert@shopify.com>
Co-authored-by: Takashi Kokubun (k0kubun) <takashikkbn@gmail.com>
---------
Co-authored-by: Maxime Chevalier-Boisvert <maxime.chevalierboisvert@shopify.com>
Co-authored-by: Takashi Kokubun (k0kubun) <takashikkbn@gmail.com>
* Use FL_USER0 for ELTS_SHARED
This makes space in RString for two bits for chilled strings.
* Mark strings returned by `Symbol#to_s` as chilled
[Feature #20350]
`STR_CHILLED` now spans on two user flags. If one bit is set it
marks a chilled string literal, if it's the other it marks a
`Symbol#to_s` chilled string.
Since it's not possible, and doesn't make much sense to include
debug info when `--debug-frozen-string-literal` is set, we can't
include allocation source, but we can safely include the symbol
name in the warning message, making it much easier to find the source
of the issue.
Co-Authored-By: Étienne Barrié <etienne.barrie@gmail.com>
---------
Co-authored-by: Étienne Barrié <etienne.barrie@gmail.com>
Co-authored-by: Jean Boussier <jean.boussier@gmail.com>
Since `str_do_hash` will most likely scan the string to
compute the coderange, we might as well copy it over in the
interned string in case it's useful later.
While profiling msgpack-ruby I noticed a very substantial amout of time
spent in `rb_enc_associate_index`, called by `rb_utf8_str_new`.
On that benchmark, `rb_utf8_str_new` is 33% of the total runtime,
in big part because it cause GC to trigger often, but even then
`5.3%` of the total runtime is spent in `rb_enc_associate_index`
called by `rb_utf8_str_new`.
After closer inspection, it appears that it's performing a lot of
safety check we can assert we don't need, and other extra useless
operations, because strings are first created and filled as ASCII-8BIT
and then later reassociated to the desired encoding.
By directly allocating the string with the right encoding, it allow
to skip a lot of duplicated and useless operations.
After this change, the time spent in `rb_utf8_str_new` is down
to `28.4%` of total runtime, and most of that is GC.
I don't want to make bundled_gems.rb more complex and complicate.
Revert "Fixed warning condition with LoadError"
This reverts commit 3a9e48b9a4.
Revert "Only warn fiddle as optional dependency"
This reverts commit ff3f61556f.
Revert "Only `warn` about bundled gems when require succeeds"
This reverts commit a70adce1ce.
[Bug #20892]
Until the introduction of that method, it was impossible for a
Module name not to be valid JSON, hence it wasn't going through
the slower escaping function.
This assumption no longer hold.
* This is notably necessary on TruffleRuby, which is updating to Ruby 3.3 which introduces Prism as a default gem.
* Using the existing path is not an option as it would end up in truffleruby/lib/build/libprism.so and
"truffleruby/lib/include/#{header}" which are not good places for such files.
https://github.com/ruby/prism/commit/5d16473e69
Parsing the regexp /\A{/ causes uses an uninitialized value because it
tries to parse it as a range quantifier, so it reads the character after
the closing curly bracket. This is using uninitialized values because
prism strings are not null terminated. This can be seen in the Valgrind
output:
==834710== Conditional jump or move depends on uninitialised value(s)
==834710== at 0x5DA010: pm_regexp_parse_range_quantifier (regexp.c:163)
==834710== by 0x5DA010: pm_regexp_parse_quantifier (regexp.c:243)
==834710== by 0x5DAD69: pm_regexp_parse_expression (regexp.c:738)
==834710== by 0x5DAD69: pm_regexp_parse_pattern (regexp.c:761)
==834710== by 0x5DAD69: pm_regexp_parse (regexp.c:773)
==834710== by 0x5A2EE7: parse_regular_expression_named_captures (prism.c:20886)
==834710== by 0x5A2EE7: parse_expression_infix (prism.c:21388)
==834710== by 0x5A5FA5: parse_expression (prism.c:21804)
==834710== by 0x5A64F3: parse_statements (prism.c:13858)
==834710== by 0x5A9730: parse_program (prism.c:22011)
==834710== by 0x576F0D: parse_input_success_p (extension.c:1062)
==834710== by 0x576F0D: parse_success_p (extension.c:1084)
This commit adds checks for the end of the string to
pm_regexp_parse_range_quantifier.
https://github.com/ruby/prism/commit/be6cbc23ef
with a TCPSoerver that is only listening
to avoid AssertionFailedError on Ubuntu.
---
The tests such as
`TestNetHTTP_v1_2_chunked#test_timeout_during_non_chunked_streamed_HTTP_session_write`
expect to raise a `Net::WriteTimeout` due to a failure in writing to the server.
However, on Ubuntu environments,
the server immediately returns a "Connection Refused" in such cases.
The socket created with `TCPSocket.new` that supports HEv2 catches this immediately
and raises a `Net::OpenTimeout`.
As a result, these tests fail due to raising a different exception than expected.
This PR adds `Net::OpenTimeout` asexceptions to avoid these test failures.
As of 10574857ce, it's possible to crash
on a double free due to `stk_alloc` AKA `msa->stack_p` being freed
twice, once at the end of match_at and a second time in `FREE_MATCH_ARG`
in the parent caller.
Fixes [Bug #20886]
Useful for core dumps. It used to not work because:
(gdb) p !""
evaluation of this expression requires the target program to be active
(gdb) p 0 == ""
evaluation of this expression requires the target program to be active
Right now attempting to pretty print a BasicObject or any other
object lacking a few core Object methods will result in an error
```
Error: test_basic_object(PPTestModule::PPInspectTest): NoMethodError: undefined method `is_a?' for an instance of BasicObject
lib/pp.rb:192:in `pp'
lib/pp.rb:97:in `block in pp'
lib/pp.rb:158:in `guard_inspect_key'
lib/pp.rb:97:in `pp'
test/test_pp.rb:131:in `test_basic_object'
128:
129: def test_basic_object
130: a = BasicObject.new
=> 131: assert_match(/\A#<BasicObject:0x[\da-f]+>\n\z/, PP.pp(a, ''.dup))
132: end
133: end
134:
```
With some fairly small changes we can fallback to `Object#inspect`
which is better than an error.
https://github.com/ruby/pp/commit/4e9f6c2de0
[Bug #20808]
The previous implementation assumed all members are accessible,
but it's possible for users to change the visibility of members or
to entirely remove the accessor.
https://github.com/ruby/pp/commit/fb19501434
* Introduction of Happy Eyeballs Version 2 (RFC8305) in TCPSocket.new
This is an implementation of Happy Eyeballs version 2 (RFC 8305) in `TCPSocket.new`.
See https://github.com/ruby/ruby/pull/11653
1. Background
Prior to this implementation, I implemented Happy Eyeballs Version 2 (HEv2) for `Socket.tcp` in https://github.com/ruby/ruby/pull/9374.
HEv2 is an algorithm defined in [RFC 8305](https://datatracker.ietf.org/doc/html/rfc8305), aimed at improving network connectivity.
For more details on the specific cases that HEv2 helps, please refer to https://bugs.ruby-lang.org/issues/20108.
2. Proposal & Outcome
This proposal implements the same HEv2 algorithm in `TCPSocket.new`.
Since `TCPSocket.new` is used more widely than `Socket.tcp`, this change is expected to broaden the impact of HEv2's benefits.
Like `Socket.tcp`, I have also added `fast_fallback` keyword argument to `TCPSocket.new`.
This option is set to true by default, enabling the HEv2 functionality.
However, users can explicitly set it to false to disable HEv2 and use the previous behavior of `TCPSocket.new`.
It should be noted that HEv2 is enabled only in environments where pthreads are available.
This specification follows the approach taken in https://bugs.ruby-lang.org/issues/19965 , where name resolution can be interrupted.
(In environments where pthreads are not available, the `fast_fallback` option is ignored.)
3. Performance
Below is the benchmark of 100 requests to `www.ruby-lang.org` with the fast_fallback option set to true and false, respectively.
While there is a slight performance degradation when HEv2 is enabled, the degradation is smaller compared to that seen in `Socket.tcp`.
```
~/s/build ❯❯❯ ../install/bin/ruby ../ruby/test.rb
Rehearsal --------------------------------------------------------
fast_fallback: true 0.017588 0.097045 0.114633 ( 1.460664)
fast_fallback: false 0.014033 0.078984 0.093017 ( 1.413951)
----------------------------------------------- total: 0.207650sec
user system total real
fast_fallback: true 0.020891 0.124054 0.144945 ( 1.473816)
fast_fallback: false 0.018392 0.110852 0.129244 ( 1.466014)
```
* Update debug prints
Co-authored-by: Nobuyoshi Nakada <nobu.nakada@gmail.com>
* Remove debug prints
* misc
* Disable HEv2 in Win
* Raise resolution error with hostname resolution
* Fix to handle errors
* Remove warnings
* Errors that do not need to be handled
* misc
* Improve doc
* Fix bug on cancellation
* Avoid EAI_ADDRFAMILY for resolving IPv6
* Follow upstream
* misc
* Refactor connection_attempt_fds management
- Introduced allocate_connection_attempt_fds and reallocate_connection_attempt_fds for improved memory allocation of connection_attempt_fds
- Added remove_connection_attempt_fd to resize connection_attempt_fds dynamically.
- Simplified the in_progress_fds function to only check the size of connection_attempt_fds.
* Rename do_pthread_create to raddrinfo_pthread_create to avoid conflicting
---------
Co-authored-by: Nobuyoshi Nakada <nobu.nakada@gmail.com>
I was looking at some crash reports and noticed that many have a line
like the following for YJIT code memory:
<addr>-<addr> r-xp 00000000 00:00 0 [heap]
I guess YJIT confused the kernel into thinking this region is from
sbrk(). While this seems to have no consequences beyond mislabeling,
it's still a little concerning.
Probe downwards instead.
When we run with RUBY_FREE_AT_EXIT, there's a false-positive memory leak
reported in YJIT because the METHOD_CODEGEN_TABLE is never freed. This
commit adds rb_yjit_free_at_exit that is called at shutdown when
RUBY_FREE_AT_EXIT is set.
Reported memory leak:
==699816== 1,104 bytes in 1 blocks are possibly lost in loss record 1 of 1
==699816== at 0x484680F: malloc (vg_replace_malloc.c:446)
==699816== by 0x155B3E: UnknownInlinedFun (unix.rs:14)
==699816== by 0x155B3E: UnknownInlinedFun (stats.rs:36)
==699816== by 0x155B3E: UnknownInlinedFun (stats.rs:27)
==699816== by 0x155B3E: alloc (alloc.rs:98)
==699816== by 0x155B3E: alloc_impl (alloc.rs:181)
==699816== by 0x155B3E: allocate (alloc.rs:241)
==699816== by 0x155B3E: do_alloc<alloc::alloc::Global> (alloc.rs:15)
==699816== by 0x155B3E: new_uninitialized<alloc::alloc::Global> (mod.rs:1750)
==699816== by 0x155B3E: fallible_with_capacity<alloc::alloc::Global> (mod.rs:1788)
==699816== by 0x155B3E: prepare_resize<alloc::alloc::Global> (mod.rs:2864)
==699816== by 0x155B3E: resize_inner<alloc::alloc::Global> (mod.rs:3060)
==699816== by 0x155B3E: reserve_rehash_inner<alloc::alloc::Global> (mod.rs:2950)
==699816== by 0x155B3E: hashbrown::raw::RawTable<T,A>::reserve_rehash (mod.rs:1231)
==699816== by 0x5BC39F: UnknownInlinedFun (mod.rs:1179)
==699816== by 0x5BC39F: find_or_find_insert_slot<(usize, fn(&mut yjit::codegen::JITState, &mut yjit::backend::ir::Assembler, *const yjit::cruby::autogened::rb_callinfo, *const yjit::cruby::autogened::rb_callable_method_entry_struct, core::option::Option<yjit::codegen::BlockHandler>, i32, core::option::Option<yjit::cruby::VALUE>) -> bool), alloc::alloc::Global, hashbrown::map::equivalent_key::{closure_env#0}<usize, usize, fn(&mut yjit::codegen::JITState, &mut yjit::backend::ir::Assembler, *const yjit::cruby::autogened::rb_callinfo, *const yjit::cruby::autogened::rb_callable_method_entry_struct, core::option::Option<yjit::codegen::BlockHandler>, i32, core::option::Option<yjit::cruby::VALUE>) -> bool>, hashbrown::map::make_hasher::{closure_env#0}<usize, fn(&mut yjit::codegen::JITState, &mut yjit::backend::ir::Assembler, *const yjit::cruby::autogened::rb_callinfo, *const yjit::cruby::autogened::rb_callable_method_entry_struct, core::option::Option<yjit::codegen::BlockHandler>, i32, core::option::Option<yjit::cruby::VALUE>) -> bool, std:#️⃣:random::RandomState>> (mod.rs:1413)
==699816== by 0x5BC39F: hashbrown::map::HashMap<K,V,S,A>::insert (map.rs:1754)
==699816== by 0x57C5C6: insert<usize, fn(&mut yjit::codegen::JITState, &mut yjit::backend::ir::Assembler, *const yjit::cruby::autogened::rb_callinfo, *const yjit::cruby::autogened::rb_callable_method_entry_struct, core::option::Option<yjit::codegen::BlockHandler>, i32, core::option::Option<yjit::cruby::VALUE>) -> bool, std:#️⃣:random::RandomState> (map.rs:1104)
==699816== by 0x57C5C6: yjit::codegen::reg_method_codegen (codegen.rs:10521)
==699816== by 0x57C295: yjit::codegen::yjit_reg_method_codegen_fns (codegen.rs:10464)
==699816== by 0x5C6B07: rb_yjit_init (yjit.rs:40)
==699816== by 0x393723: ruby_opt_init (ruby.c:1820)
==699816== by 0x393723: ruby_opt_init (ruby.c:1767)
==699816== by 0x3957D4: prism_script (ruby.c:2215)
==699816== by 0x3957D4: process_options (ruby.c:2538)
==699816== by 0x396065: ruby_process_options (ruby.c:3166)
==699816== by 0x236E56: ruby_options (eval.c:117)
==699816== by 0x15BAED: rb_main (main.c:43)
==699816== by 0x15BAED: main (main.c:62)
After this patch, there are no more memory leaks reported when running
RUBY_FREE_AT_EXIT with Valgrind on an empty Ruby script:
$ RUBY_FREE_AT_EXIT=1 valgrind --leak-check=full ruby -e ""
...
==700357== HEAP SUMMARY:
==700357== in use at exit: 0 bytes in 0 blocks
==700357== total heap usage: 36,559 allocs, 36,559 frees, 6,064,783 bytes allocated
==700357==
==700357== All heap blocks were freed -- no leaks are possible
Takashi Kokubun
Randy Stauner
Alexis Bernard
Jean byroot Boussier
Jean Boussier
David Rodríguez
Hiroshi SHIBATA
HoneyryderChuck
卜部昌平
Orien Madgwick
dependabot[bot]
Benoit Daloze
Peter Zhu
Misaki Shioi
License Update
Nobuyoshi Nakada
John Hawthorn
git
Alan Wu
Alan Wu
BurdetteLamar
tomoya ishida