Tabs were expanded because the file did not have any tab indentation in unedited lines.
Please update your editor config, and use misc/expand_tabs.rb in the pre-commit hook.
It makes testing for JSON errors very tedious. You either have
to use a Regexp or to regularly update all your assertions
when JSON is upgraded.
https://github.com/flori/json/commit/de9eb1d28e
rb_ary_tmp_new suggests that the array is temporary in some way, but
that's not true, it just creates an array that's hidden and not on the
transient heap. This commit renames it to rb_ary_hidden_new.
The `rb_profile_frames` API did not skip the two dummy frames that
each thread has at its beginning. This was unlike `backtrace_each` and
`rb_ec_parcial_backtrace_object`, which do skip them.
This does not seem to be a problem for non-main thread frames,
because both `VM_FRAME_RUBYFRAME_P(cfp)` and
`rb_vm_frame_method_entry(cfp)` are NULL for them.
BUT, on the main thread `VM_FRAME_RUBYFRAME_P(cfp)` was true
and thus the dummy thread was still included in the output of
`rb_profile_frames`.
I've now made `rb_profile_frames` skip this extra frame (like
`backtrace_each` and friends), as well as add a test that asserts
the size and contents of `rb_profile_frames`.
Fixes [Bug #18907] (<https://bugs.ruby-lang.org/issues/18907>)
Sidekiq has a method named `❨╯°□°❩╯︵┻━┻`which corrupts
heap dumps.
Normally we could just dump is as is since it's valid UTF-8 and need
no escaping. But our code to escape control characters isn't UTF-8
aware so it's more complicated than it seems.
Ultimately since the overwhelming majority of method names are
pure ASCII, it's not a big loss to just skip it.
When configuring with `--disable-rpath` and `--static-linked-ext` (e.g.
building for WASI), `extmk.rb` doesn't build exts under bundled gems,
and `.bundle/gems/#{gemname}-#{ver}` are not created due to no call of
`extmake`.
b249178398 starts creating symlink at
`.bundle/gems/#{gemname}-#{ver}/lib`, but the parent dir is not created,
so configuration aginst debug and rbs gems were failed.
The default gems have not been installed yet in the build directory,
bundled gems depending on them can not work. As those dependencies
should be usable there even without rubygems, make temporary gemspec
files without the dependencies, and use them in the build directory.
Brings a dozen call-seq schemas into compliance with the doc guide.
Adds links to section "Argument start" where needed.
Revises (minorly) ::today.
Otherwise, does not disturb existing text.
https://github.com/ruby/date/commit/9aec11df50
On Windows with OpenSSL 3, the gem fails to compile with the following
error message:
ruby/src/ext/openssl/extconf.rb:188: undefined method \`sub!' for nil:NilClass
This is because $warnflags is nil.
[Bug #18900]
Thread#join and a few other codepaths are using native sleep as
a way to suspend the current thread. So we should call the relevant
hook when this happen, otherwise some thread may transition
directly from `RESUMED` to `READY`.
I suspect that some shared pages are invalidated because
some static string don't have their coderange set eagerly.
So the first time they are scanned, the entire memory page is
invalidated.
Being able to see the coderange in `ObjectSpace` would help debug
this.
And in addition `dump` currently call `is_broken_string()` and `is_ascii_string()`
which both end up scanning the string and assigning coderange. I think it's
undesirable as `dump` should be read only.
[Feature #18339]
After experimenting with the initial version of the API I figured there is a need
for an exit event to cleanup instrumentation data. e.g. if you record data in a
{thread_id -> data} table, you need to free associated data when a thread goes away.
I tried to build Ruby on a system without libyaml today and realized
that my attempt from <https://github.com/ruby/psych/pull/557> doesn't
fix the error in <https://github.com/ruby/psych/issues/552>. I still got
the same `LoadError` from `digest` which stopped the build.
Since `LoadError` is not a `StandardError`, a plain `rescue` doesn't catch
it. Catch `LoadError` explicitly instead and reduce the scope of the
`begin` block.
I tested this change in a Ruby build on macOS without libyaml installed
and confirmed that `make` continues with a warning instead of aborting:
*** Following extensions are not compiled:
psych:
Could not be configured. It will not be installed.
...
This should address <https://bugs.ruby-lang.org/issues/18790>.
https://github.com/ruby/psych/commit/251289ba83
Previously, because opt_aref and opt_aset don't push a frame, when they
would call rb_hash to determine the hash value of the key, the initial
level of recursion would incorrectly use the method id at the top of the
stack instead of "hash".
This commit replaces rb_exec_recursive_outer with
rb_exec_recursive_outer_mid, which takes an explicit method id, so that
we can make the hash calculation behave consistently.
rb_exec_recursive_outer was documented as being internal, so I believe
this should be okay to change.
Ref: https://bugs.ruby-lang.org/issues/18339
Design:
- This tries to minimize the overhead when no hook is registered.
It should only incur an extra unsynchronized boolean check.
- The hook list is protected with a read-write lock as to cause
contention when some hooks are registered.
- The hooks MUST be thread safe, and MUST NOT call into Ruby as they
are executed outside the GVL.
- It's simply a noop on Windows.
API:
```
rb_internal_thread_event_hook_t * rb_internal_thread_add_event_hook(rb_internal_thread_event_callback callback, rb_event_flag_t internal_event, void *user_data);
bool rb_internal_thread_remove_event_hook(rb_internal_thread_event_hook_t * hook);
```
You can subscribe to 3 events:
- READY: called right before attempting to acquire the GVL
- RESUMED: called right after successfully acquiring the GVL
- SUSPENDED: called right after releasing the GVL.
The hooks MUST be threadsafe, as they are executed outside of the GVL, they also MUST NOT call any Ruby API.
Try to mirror IO behavior, where chomp takes out the entire paragraph
separators between entries, but does not chomp a single line separator
at the end of the string.
Partially Fixes [Bug #18768]
https://github.com/ruby/stringio/commit/a83ddbb7f0
Previously, this could result in an infinite loop. Always update
the e pointer in this case, setting w when chomping so the chomped
data is not included in the output.
Fixes [Bug #18769]
https://github.com/ruby/stringio/commit/4bf64d5130
In December 2021, we opened an [issue] to solicit feedback regarding the
porting of the YJIT codebase from C99 to Rust. There were some
reservations, but this project was given the go ahead by Ruby core
developers and Matz. Since then, we have successfully completed the port
of YJIT to Rust.
The new Rust version of YJIT has reached parity with the C version, in
that it passes all the CRuby tests, is able to run all of the YJIT
benchmarks, and performs similarly to the C version (because it works
the same way and largely generates the same machine code). We've even
incorporated some design improvements, such as a more fine-grained
constant invalidation mechanism which we expect will make a big
difference in Ruby on Rails applications.
Because we want to be careful, YJIT is guarded behind a configure
option:
```shell
./configure --enable-yjit # Build YJIT in release mode
./configure --enable-yjit=dev # Build YJIT in dev/debug mode
```
By default, YJIT does not get compiled and cargo/rustc is not required.
If YJIT is built in dev mode, then `cargo` is used to fetch development
dependencies, but when building in release, `cargo` is not required,
only `rustc`. At the moment YJIT requires Rust 1.60.0 or newer.
The YJIT command-line options remain mostly unchanged, and more details
about the build process are documented in `doc/yjit/yjit.md`.
The CI tests have been updated and do not take any more resources than
before.
The development history of the Rust port is available at the following
commit for interested parties:
1fd9573d8b
Our hope is that Rust YJIT will be compiled and included as a part of
system packages and compiled binaries of the Ruby 3.2 release. We do not
anticipate any major problems as Rust is well supported on every
platform which YJIT supports, but to make sure that this process works
smoothly, we would like to reach out to those who take care of building
systems packages before the 3.2 release is shipped and resolve any
issues that may come up.
[issue]: https://bugs.ruby-lang.org/issues/18481
Co-authored-by: Maxime Chevalier-Boisvert <maximechevalierb@gmail.com>
Co-authored-by: Noah Gibbs <the.codefolio.guy@gmail.com>
Co-authored-by: Kevin Newton <kddnewton@gmail.com>
WHen libyaml is not installed, make fails with the following cryptic
message:
```
gmake[2]: Entering directory '/home/chkbuild/chkbuild-crossruby/tmp/build/20220325T045825Z/ruby/ext/psych'
gmake[2]: *** No rule to make target 'yaml/yaml.h', needed by 'psych.o'. Stop.
gmake[2]: Leaving directory '/home/chkbuild/chkbuild-crossruby/tmp/build/20220325T045825Z/ruby/ext/psych'
```
I think it should give up building psych with a clear message.
Header file include/ruby/internal/abi.h contains RUBY_ABI_VERSION which
is the ABI version. This value should be bumped whenever an ABI
incompatible change is introduced.
When loading dynamic libraries, Ruby will compare its own
`ruby_abi_version` and the `ruby_abi_version` of the loaded library. If
these two values don't match it will raise a `LoadError`. This feature
can also be turned off by setting the environment variable
`RUBY_RUBY_ABI_CHECK=0`.
This feature will prevent cases where previously installed native gems
fail in unexpected ways due to incompatibility of changes in header
files. This will force the developer to recompile their gems to use the
same header files as the built Ruby.
In Ruby, the ABI version is exposed through
`RbConfig::CONFIG["ruby_abi_version"]`.
Inside ObjectSpace.reachable_objects_from we keep an internal identhash
in order to de-duplicate reachable objects when wrapping them as
InternalObject. Previously this hash was not hidden, making it possible
to leak references to those internal objects to Ruby if using
ObjectSpace.each_object.
This commit solves this by hiding the hash. To simplify collection of
values, we instead now just use the hash as a set of visited objects,
and collect an Array (not hidden) of values to be returned.
The gem doesn't even install on old rubies, but since the gemspec claims
it's supported, `gem install pathname` will try to install it and print
an error.
This commit doesn't fix the above issue. The only way to fix it would be
to restore support and release a new version that actually supports old
rubies. However, such a change has been proposed and ignored for a long
time.
So this issue proposes to leave that broken but at least bring the
gemspec manifest and the CI matrix in sync to hopefully avoid this issue
from happening again in the future.
https://github.com/ruby/pathname/commit/3ee010b538
It does not seem needed, and it's causing issues on Windows when
uninstalling `strscan`, because strscan's shared library being used when
RubyGems tries to remove it (because its loaded through Psych, which
RubyGems uses for loading configuration).
https://github.com/ruby/psych/commit/3911356ec1
* Enhanced RDoc for BigDecimal
* Update ext/bigdecimal/bigdecimal.c
Remove the instance number of `Float::DIG`.
* Update ext/bigdecimal/bigdecimal.c
Add BigDecimal call-seq without ndigits.
* Update ext/bigdecimal/bigdecimal.c
Replace the word sum with value or result in the description of BigDecimal().
* Update ext/bigdecimal/bigdecimal.c
Remove the instance value of Float::DIG.
* Update ext/bigdecimal/bigdecimal.c
Fix mis-description of precision
* Update ext/bigdecimal/bigdecimal.c
Fix the description of precision determination
* Update ext/bigdecimal/bigdecimal.c
Add the description of the precision in the Rational case.
https://github.com/ruby/bigdecimal/commit/acabb132a4
Co-authored-by: Kenta Murata <3959+mrkn@users.noreply.github.com>
OpenSSL 3.0's EVP_PKEY_get0() returns NULL for provider-backed pkeys.
This causes segfault because it was supposed to never return NULL
before.
We can't check the existence of public key components in this way on
OpenSSL 3.0. Let's just skip it for now.
https://github.com/ruby/openssl/commit/ccdb6f7bfa
The entire ENGINE API is deprecated in OpenSSL 3.0 in favor of the new
"Provider" concept.
OpenSSL::Engine will not be defined when compiled with OpenSSL 3.0.
We would need a way to interact with providers from Ruby programs, but
since the concept is completely different from the ENGINE API, it will
not be through the current OpenSSL::Engine interface.
https://github.com/ruby/openssl/commit/69a27d8de4
OpenSSL 3.0 made EVP_PKEY immutable. This means we can only have a const
pointer of the low level struct and the following methods can no longer
be provided when linked against OpenSSL 3.0:
- OpenSSL::PKey::RSA#set_key
- OpenSSL::PKey::RSA#set_factors
- OpenSSL::PKey::RSA#set_crt_params
- OpenSSL::PKey::DSA#set_pqg
- OpenSSL::PKey::DSA#set_key
- OpenSSL::PKey::DH#set_pqg
- OpenSSL::PKey::DH#set_key
- OpenSSL::PKey::EC#group=
- OpenSSL::PKey::EC#private_key=
- OpenSSL::PKey::EC#public_key=
There is no direct replacement for this functionality at the moment.
I plan to introduce a wrapper around EVP_PKEY_fromdata(), which takes
all key components at once to construct an EVP_PKEY.
https://github.com/ruby/openssl/commit/6848d2d969
OpenSSL::PKey::EC#generate_key! will not work on OpenSSL 3.0 because
keys are made immutable. Users should use OpenSSL::PKey.generate_key
instead.
https://github.com/ruby/openssl/commit/5e2e66cce8
OpenSSL::PKey::DH#generate_key! will not work on OpenSSL 3.0 because
keys are made immutable. Users should use OpenSSL::PKey.generate_key
instead.
https://github.com/ruby/openssl/commit/8ee6a582c7
DH#set_key will not work on OpenSSL 3.0 because keys are immutable.
For now, let's reimplement DH#compute_key by manually constructing a
DER-encoded SubjectPublicKeyInfo structure and feeding it to
OpenSSL::PKey.read.
Eventually, we should implement a new method around EVP_PKEY_fromdata()
and use it instead.
https://github.com/ruby/openssl/commit/46ca47060c
Allocate an EVP_PKEY when the content is ready: when #initialize
or #initialize_copy is called, rather than when a T_DATA is allocated.
This is more natural because the lower level API has been deprecated
and an EVP_PKEY is becoming the minimum unit of handling keys.
https://github.com/ruby/openssl/commit/74f6c61756
OpenSSL 3.0 has rewritten routines to load pkeys (PEM_read_bio_* and
d2i_* functions) around the newly introduced OSSL_DECODER API.
This comes with a slight behavior change. They now decrypt and parse
each encountered PEM block, then check the kind of the block. This used
to be the reverse: they checked the PEM header to see the kind, and then
decrypted the content. This means that the password callback may now be
called repeatedly.
Let's use the OSSL_DECODER API directly on OpenSSL 3.0 so that the
return value from the password callback will be reused automatically.
https://github.com/ruby/openssl/commit/a84ea531bb
This `NODE` type was used in pre-YARV implementation, to improve
the performance of assignment to dynamic local variable defined at
the innermost scope. It has no longer any actual difference with
`NODE_DASGN`, except for the node dump.
The old code of IRB still uses this method. The warning is noisy on
rails console.
In principle, Ruby 3.1 deprecates nothing, so let's avoid the
deprecation for the while.
I think It is not so hard to continue to maintain it as it is a trivial
shim.
https://github.com/ruby/ruby/pull/5093
The last element in the `@buf` may be either an array or an `Elem`. In the case it is an `Elem` we iterate over every element, when we do not need to. This check guards that case by ensuring that we only iterate over an array of elements.
When Zlib::Inflate#inflate or Zlib::Deflate#deflate is called
recursively inside the block, a crash can occur because of an
use-after-free bug.
https://github.com/ruby/zlib/commit/50fb8a0338
After 5680c38c75, postponed job APIs now
expect to be called on native threads not managed by Ruby and handles
getting a NULL execution context. However, in debug builds the change
runs into an assertion failure with GET_EC() which asserts that EC is
non-NULL. Avoid the assertion failure by passing `false` for `expect_ec`
instead as the intention is to handle when there is no EC.
Add a test from John Crepezzi and John Hawthorn to exercise this
situation.
See GH-4108
See GH-5094
[Bug #17573]
Co-authored-by: John Hawthorn <john@hawthorn.email>
Co-authored-by: John Crepezzi <john.crepezzi@gmail.com>
An almost universal convention for gems is to expose `Namespace::VERSION`
which makes it mcuh easier when debugging etc.
Many gems extracted from ruby don't do this, even though it would be even more
useful because they ship with ruby, so it's less clear which version it is.
https://github.com/ruby/date/commit/fef7ec18d8
http://rubyci.s3.amazonaws.com/ubuntu1804/ruby-master/log/20211117T033003Z.log.html.gz
```
installing default gems from ext: /home/chkbuild/chkbuild/tmp/build/20211117T033003Z/lib/ruby/gems/3.1.0
/home/chkbuild/chkbuild/tmp/build/20211117T033003Z/ruby/ext/digest/lib/digest/version.rb:4: warning: already initialized constant Digest::VERSION
/home/chkbuild/chkbuild/tmp/build/20211117T033003Z/ruby/.ext/common/digest/version.rb:4: warning: previous definition of VERSION was here
```
This hack is copied from ext/psych/psych.gemspec
Reverts 2eb3841e9c
because it fails on "update-deps" check in the ruby/ruby CI.
https://github.com/ruby/ruby/runs/4230891140?check_suite_focus=true
```
diff --git a/ext/io/wait/depend b/ext/io/wait/depend
index 7b314b9..449e9fe 100644
--- a/ext/io/wait/depend
+++ b/ext/io/wait/depend
...
```
Maybe now it does not work on Ruby 2.6. This file must be changed for
each Ruby version. I have no good idea to fix this issue.
`Date.parse` now raises an ArgumentError when a given date string is
longer than 128. You can configure the limit by giving `limit` keyword
arguments like `Date.parse(str, limit: 1000)`. If you pass `limit: nil`,
the limit is disabled.
Not only `Date.parse` but also the following methods are changed.
* Date._parse
* Date.parse
* DateTime.parse
* Date._iso8601
* Date.iso8601
* DateTime.iso8601
* Date._rfc3339
* Date.rfc3339
* DateTime.rfc3339
* Date._xmlschema
* Date.xmlschema
* DateTime.xmlschema
* Date._rfc2822
* Date.rfc2822
* DateTime.rfc2822
* Date._rfc822
* Date.rfc822
* DateTime.rfc822
* Date._jisx0301
* Date.jisx0301
* DateTime.jisx0301
https://github.com/ruby/date/commit/3959accef8
OpenSSL::SSL::SSLSocket allowed #read and #write to be called before an
SSL/TLS handshake is completed. They passed unencrypted data to the
underlying socket.
This behavior is very odd to have in this library. A verbose mode
warning "SSL session is not started yet" was emitted whenever this
happened. It also didn't behave well with OpenSSL::Buffering. Let's
just get rid of it.
Fixes: https://github.com/ruby/openssl/issues/9https://github.com/ruby/openssl/commit/bf780748b3
BN_pseudo_rand() and BN_pseudo_rand_range() are deprecated in
OpenSSL 3.0. Since they are identical to their non-'pseudo' version
anyway, let's make them alias.
https://github.com/ruby/openssl/commit/2d34e85ddf
It converts the internal representation of the point object to the
affine coordinate system. However, it had no real use case because the
difference in the internal representation has not been visible from
Ruby/OpenSSL at all.
EC_POINT_make_affine() is marked as deprecated in OpenSSL 3.0.
https://github.com/ruby/openssl/commit/e2cc81fef7
OpenSSL 3.0 renamed EVP_MD_CTX_pkey_ctx() to include "get" in the
function name. Adjust compatibility macro so that we can use the new
function name for all OpenSSL 1.0.2-3.0.
https://github.com/ruby/openssl/commit/c106d888c6
The function was renamed in OpenSSL 3.0 due to the change of the
lifetime of EVP_MD objects. They are no longer necessarily statically
allocated and can be reference-counted -- when an EVP_MD_CTX is free'd,
the associated EVP_MD can also become inaccessible.
Currently Ruby/OpenSSL only handles builtin algorithms, so no special
handling is needed except for adapting to the rename.
https://github.com/ruby/openssl/commit/0a253027e6
Use SSL_get_rbio() instead of SSL_get_fd(). SSL_get_fd() internally
calls SSL_get_rbio() and it's enough for our purpose.
In OpenSSL 3.0, SSL_get_fd() leaves an entry in the OpenSSL error queue
if BIO has not been set up yet, and we would have to clean it up.
https://github.com/ruby/openssl/commit/e95ee24867
If we use the same version as the default strscan gem in Ruby, "gem
install" doesn't extract .gem. It fails "gem install" because "gem
install" can't find ext/strscan/ to be built.
https://github.com/ruby/strscan/commit/3ceafa6cdc
SSLSocket#connect eventually calls `GetOpenFile` in order to get the
underlying file descriptor for the IO object passed in on
initialization. `GetOpenFile` assumes that the Ruby object passed in is
a T_FILE object and just casts it to a T_FILE without any checks. If
you pass an object that *isn't* a T_FILE to that function, the program
will segv.
Since we assume the IO object is a file in the `connect` method, this
commit adds a `CheckType` in the initialize method to ensure that the IO
object is actually a T_FILE. If the object *isn't* a T_FILE, this class
will segv on `connect`, so I think this is a backwards compatible
change.
https://github.com/ruby/openssl/commit/919fa44ec2
* Tie lifetime of uJIT blocks to iseqs
Blocks weren't being freed when iseqs are collected.
* Add rb_dary. Use it for method dependency table
* Keep track of blocks per iseq
Remove global version_tbl
* Block version bookkeeping fix
* dary -> darray
* free ujit_blocks
* comment about size of ujit_blocks
Drop support for Ruby 2.3, 2.4, and 2.5.
As of 2021-10, Ruby 2.6 is the oldest version that still receives
security fixes from the Ruby core team, so it doesn't make much sense
to keep code for those ancient versions.
https://github.com/ruby/openssl/commit/3436bd040d
On the server side, the serialized list of protocols is stored in
SSL_CTX as a String object reference. We utilize a hidden instance
variable to prevent it from being GC'ed, but this is not enough because
it can also be relocated by GC.compact.
https://github.com/ruby/openssl/commit/5eb68ba778
We store the reverse reference to the Ruby object in the OpenSSL
struct for use from OpenSSL callback functions. To prevent the Ruby
object from being relocated by GC.compact, we must "pin" it by calling
rb_gc_mark().
https://github.com/ruby/openssl/commit/a6ba9f894f
We store the reverse reference to the Ruby object in the OpenSSL
struct for use from OpenSSL callback functions. To prevent the Ruby
object from being relocated by GC.compact, we must "pin" it by calling
rb_gc_mark().
https://github.com/ruby/openssl/commit/022b7ceada
Similarly to SSLSocket#syswrite, the blocking SSLSocket#sysread allows
context switches. We must prevent other threads from modifying the
string buffer.
We can use rb_str_locktmp() and rb_str_unlocktmp() to temporarily
prohibit modification of the string.
https://github.com/ruby/openssl/commit/d38274949f
Provide a wrapper of SSL_set0_tmp_dh_pkey()/SSL_CTX_set_tmp_dh(), which
sets the DH parameters used for ephemeral DH key exchange.
SSLContext#tmp_dh_callback= already exists for this purpose, as a
wrapper around SSL_CTX_set_tmp_dh_callback(), but it is considered
obsolete and the OpenSSL API is deprecated for future removal. There is
no practical use case where an application needs to use different DH
parameters nowadays. This was originally introduced to support export
grade ciphers.
RDoc for #tmp_dh_callback= is updated to recommend the new #tmp_dh=.
Note that current versions of OpenSSL support automatic ECDHE curve
selection which is enabled by default. SSLContext#tmp_dh= should only be
necessary if you must allow ancient clients which don't support ECDHE.
https://github.com/ruby/openssl/commit/aa43da4f04
In AIX, altzone exists in the standard library but is not declared
in time.h. By 524513be39, have_var
and try_var in mkmf recognizes a variable that exists in a library
even when it is not declared. As a result, in AIX, HAVE_ALTZONE
is defined, but compile fails due to the lack of the declaration.
%v is supposed to be the VMS date, and VMS date format uses an
uppercase month.
Ruby 1.8 used an uppercase month for %v, but the behavior was
changed without explanation in r31672.
Time#strftime still uses an uppercase month for %v, so this change
makes Date#strftime consistent with Time#strftime.
Fixes [Bug #13810]
https://github.com/ruby/date/commit/56c489fd7e
Just append OpenSSL error reason to the given message string
object, which would be alreadly formatted.
Suppress -Wformat-security warning in `ossl_tsfac_create_ts`.
https://github.com/ruby/openssl/commit/11b1d8a6b8
This prevents early collection of the array. The GC doesn't see the
array on the stack when Ruby is compiled with optimizations enabled
Thanks @jhaberman for the test case
[ruby-core:105099] [Bug #18140]