I would like to check if a symbol is defined before trying to access it.
Some symbols aren't available on all platforms, so instead of raising an
exception, I want to check if it's defined first.
Today we have to do:
```ruby
begin
addr = Fiddle::Handle.sym("something")
# do something
rescue Fiddle::DLError
end
```
I want to write this:
```ruby
if Fiddle::Handle.sym_defined?("something")
addr = Fiddle::Handle.sym("something")
# do something
end
```
https://github.com/ruby/fiddle/commit/9d3371de13
Co-authored-by: Sutou Kouhei <kou@clear-code.com>
This helps to reduce repetition in code. Instead of doing "TYPE_*"
everywhere, you can do `include Fiddle::Types`, and write the type name
directly.
This PR is to help reduce repetition when writing Fiddle code. Right now
we have to type `TYPE_` everywhere, and you also have to include all of
`Fiddle` to access `TYPE_*` constants. With this change, you can just
include `Fiddle::Types` and it will shorten your code and also you only
have to include those constants.
Here is an example before:
```ruby
require "fiddle"
module MMAP
# All Fiddle constants included
include Fiddle
def self.make_function name, args, ret
ptr = Handle::DEFAULT[name]
func = Function.new ptr, args, ret, name: name
define_singleton_method name, &func.to_proc
end
make_function "munmap", [TYPE_VOIDP, # addr
TYPE_SIZE_T], # len
TYPE_INT
make_function "mmap", [TYPE_VOIDP,
TYPE_SIZE_T,
TYPE_INT,
TYPE_INT,
TYPE_INT,
TYPE_INT], TYPE_VOIDP
make_function "mprotect", [TYPE_VOIDP, TYPE_SIZE_T, TYPE_INT], TYPE_INT
end
```
After:
```ruby
require "fiddle"
module MMAP
# Only type names included
include Fiddle::Types
def self.make_function name, args, ret
ptr = Fiddle::Handle::DEFAULT[name]
func = Fiddle::Function.new ptr, args, ret, name: name
define_singleton_method name, &func.to_proc
end
make_function "munmap", [VOIDP, # addr
SIZE_T], # len
INT
make_function "mmap", [VOIDP, SIZE_T, INT, INT, INT, INT], VOIDP
make_function "mprotect", [VOIDP, SIZE_T, INT], INT
end
```
We only need to import the type names, and you don't have to type
`TYPE_` over and over. I think this makes Fiddle code easier to read.
https://github.com/ruby/fiddle/commit/49fa7233e5
Co-authored-by: Sutou Kouhei <kou@clear-code.com>
This commit adds constants for unsigned values. Currently we can use `-`
to mean "unsigned", but I think having a specific name makes Fiddle more
user friendly. This commit continues to support `-`, but introduces
negative constants with "unsigned" names
I think this will help to eliminate [this
code](3a56bf0bcc/lib/mjit/c_type.rb (L31-L38))
https://github.com/ruby/fiddle/commit/2bef0f1082
Co-authored-by: Sutou Kouhei <kou@clear-code.com>
Moves Expect library doc into io.c.
Changes certain links to local sections, now pointing to sections in doc/io_streams.rdoc.
Removes local sections now superseded by sections in doc/io_streams.rdoc.
Object Shapes is used for accessing instance variables and representing the
"frozenness" of objects. Object instances have a "shape" and the shape
represents some attributes of the object (currently which instance variables are
set and the "frozenness"). Shapes form a tree data structure, and when a new
instance variable is set on an object, that object "transitions" to a new shape
in the shape tree. Each shape has an ID that is used for caching. The shape
structure is independent of class, so objects of different types can have the
same shape.
For example:
```ruby
class Foo
def initialize
# Starts with shape id 0
@a = 1 # transitions to shape id 1
@b = 1 # transitions to shape id 2
end
end
class Bar
def initialize
# Starts with shape id 0
@a = 1 # transitions to shape id 1
@b = 1 # transitions to shape id 2
end
end
foo = Foo.new # `foo` has shape id 2
bar = Bar.new # `bar` has shape id 2
```
Both `foo` and `bar` instances have the same shape because they both set
instance variables of the same name in the same order.
This technique can help to improve inline cache hits as well as generate more
efficient machine code in JIT compilers.
This commit also adds some methods for debugging shapes on objects. See
`RubyVM::Shape` for more details.
For more context on Object Shapes, see [Feature: #18776]
Co-Authored-By: Aaron Patterson <tenderlove@ruby-lang.org>
Co-Authored-By: Eileen M. Uchitelle <eileencodes@gmail.com>
Co-Authored-By: John Hawthorn <john@hawthorn.email>
Object Shapes is used for accessing instance variables and representing the
"frozenness" of objects. Object instances have a "shape" and the shape
represents some attributes of the object (currently which instance variables are
set and the "frozenness"). Shapes form a tree data structure, and when a new
instance variable is set on an object, that object "transitions" to a new shape
in the shape tree. Each shape has an ID that is used for caching. The shape
structure is independent of class, so objects of different types can have the
same shape.
For example:
```ruby
class Foo
def initialize
# Starts with shape id 0
@a = 1 # transitions to shape id 1
@b = 1 # transitions to shape id 2
end
end
class Bar
def initialize
# Starts with shape id 0
@a = 1 # transitions to shape id 1
@b = 1 # transitions to shape id 2
end
end
foo = Foo.new # `foo` has shape id 2
bar = Bar.new # `bar` has shape id 2
```
Both `foo` and `bar` instances have the same shape because they both set
instance variables of the same name in the same order.
This technique can help to improve inline cache hits as well as generate more
efficient machine code in JIT compilers.
This commit also adds some methods for debugging shapes on objects. See
`RubyVM::Shape` for more details.
For more context on Object Shapes, see [Feature: #18776]
Co-Authored-By: Aaron Patterson <tenderlove@ruby-lang.org>
Co-Authored-By: Eileen M. Uchitelle <eileencodes@gmail.com>
Co-Authored-By: John Hawthorn <john@hawthorn.email>
getenv is a very basic function that has been in stdlib.h since
ISO/IEC 9899:1990. There is absolutely zero need for us to redeclare.
pty.c already includes stdlib.h out of the box so we need nothing.
The reason why this was commented out was because of gperf 3.0 vs 3.1
differences (see [Feature #13883]). Five years passed, I am pretty
confident that we can drop support of old versions here.
Ditto for uniname2ctype_p(), onig_jis_property(), and zonetab().
Fixes id.h error during updating ripper.c by `make after-update`.
While it used to update id.h in the build directory, but was trying to
update ripper.c in the source directory. In principle, files in the
source directory can or should not depend on files in the build
directory.
icc now seems to provide libutil.so that is not related to pty.
This extconf.rb wrongly finds it and adds `-lutil`, but `ruby -rpty`
fails because it cannot find libutil.so on the runtime.
http://rubyci.s3.amazonaws.com/icc-x64/ruby-master/log/20220815T210005Z.fail.html.gz
```
Exception raised:
<#<LoadError: libutil.so: cannot open shared object file: No such file or directory - /home/chkbuild/chkbuild/tmp/build/20220815T210005Z/ruby/.ext/x86_64-linux/pty.so>>
```
This change makes extconf.rb check libutil only on OpenBSD.
As commented in include/ruby/internal/abi.h, since teeny versions of
Ruby should guarantee ABI compatibility, `RUBY_ABI_VERSION` has no role
in released versions of Ruby.
* Fix Array#[] with ArithmeticSequence with negative steps
Previously, Array#[] when called with an ArithmeticSequence
with a negative step did not handle all cases correctly,
especially cases involving infinite ranges, inverted ranges,
and/or exclusive ends.
Fixes [Bug #18247]
* Add Array#slice tests for ArithmeticSequence with negative step to test_array
Add tests of rb_arithmetic_sequence_beg_len_step C-API function.
* Fix ext/-test-/arith_seq/beg_len_step/depend
* Rename local variables
* Fix a variable name
Co-authored-by: Kenta Murata <3959+mrkn@users.noreply.github.com>
Adding `ruby` to `PREP` causes the following circular dependencies
because `PREP` is used as a prerequisite by some targets required to
build `ruby` target itself.
```
make: Circular .rbconfig.time <- ruby dependency dropped.
make: Circular builtin_binary.inc <- ruby dependency dropped.
make: Circular ext/extinit.c <- ruby dependency dropped.
make: Circular ruby <- ruby dependency dropped.
```
Adding a new Make variable like `EXTPREP` only for exts may be also
reasonable, but it would introduce another complexity into our build
system. `-bundle_loader` doesn't care that link-time and run-time
loader executables are different as long as bound symbols are provided,
so it's ok to resolve from miniruby to simplify our build.
For the macOS -bundle_loader linker option, we need a path to the
Ruby exectuable. $(RUBY) is not necessarily a path since it could
be a command line invocation. That happens during build with
runruby.rb and can happen post installation if the user passes
the --ruby option to a extconf.rb. Use $(bindir) to locate
the executable instead.
Before installation, $(bindir) doesn't exist, so we need to be
able to override $(BUILTRUBY) in such situations so test-spec
and bundled extensions could build. Use a new mkmf global,
$builtruby, to do this; set it in fake.rb and in extmk.rb.
Our build system is quite complex...
ld64 shipped with Xcode 14 emits a warning when using `-undefined
dynamic_lookup`.
```
ld: warning: -undefined dynamic_lookup may not work with chained fixups
```
Actually, `-undefined dynamic_lookup` doesn't work when:
1. Link a *shared library* with the option
2. Link it with a program that uses the chained-fixup introduced from
macOS 12 and iOS 15
because `-undefined dynamic_lookup` uses lazy-bindings and they won't be
bound while dyld fixes-up by traversing chained-fixup info.
However, we build exts as *bundles* and they are loaded only through
`dlopen`, so it's safe to use `-undefined dynamic_lookup` in theory.
So the warning produced by ld64 is false-positive, and it results
failure of option checking in configuration. Therefore, it would be an
option to ignore the warning during our configuration.
On the other hand, `-undefined dynamic_lookup` is already deprecated on
all darwin platforms except for macOS, so it's good time to get rid of
the option. ld64 also provides `-bundle_loader <executable>` option,
which allows to resolve symbols defined in the executable symtab while
linking. It behaves almost the same with `-undefined dynamic_lookup`,
but it makes the following changes:
1. Require that unresolved symbols among input objects must be defined
in the executable.
2. Lazy symbol binding will lookup only the symtab of the bundle loader
executable. (`-undefined dynamic_lookup` lookups all symtab as flat
namespace)
This patch adds `-bundle_loader $(RUBY)` when non-EXTSTATIC
configuration by assuming ruby executable can be linked before building
exts.
See "New Features" subsection under "Linking" section for chained fixup
https://developer.apple.com/documentation/xcode-release-notes/xcode-13-release-notes
Treats:
::httpdate
#to_date
#to_time
#to_datetime
In behalf of ::httpdate, I've introduced section "Argument limit" that can be pointed to by various Date methods (similar to existing section "Argument start"). This will involve 8 already-enhanced methods plus 8 more not yet done.
https://github.com/ruby/date/commit/00326ff99c
Tabs were expanded because the file did not have any tab indentation in unedited lines.
Please update your editor config, and use misc/expand_tabs.rb in the pre-commit hook.
Tabs were expanded because the file did not have any tab indentation in unedited lines.
Please update your editor config, and use misc/expand_tabs.rb in the pre-commit hook.
It makes testing for JSON errors very tedious. You either have
to use a Regexp or to regularly update all your assertions
when JSON is upgraded.
https://github.com/flori/json/commit/de9eb1d28e
rb_ary_tmp_new suggests that the array is temporary in some way, but
that's not true, it just creates an array that's hidden and not on the
transient heap. This commit renames it to rb_ary_hidden_new.
The `rb_profile_frames` API did not skip the two dummy frames that
each thread has at its beginning. This was unlike `backtrace_each` and
`rb_ec_parcial_backtrace_object`, which do skip them.
This does not seem to be a problem for non-main thread frames,
because both `VM_FRAME_RUBYFRAME_P(cfp)` and
`rb_vm_frame_method_entry(cfp)` are NULL for them.
BUT, on the main thread `VM_FRAME_RUBYFRAME_P(cfp)` was true
and thus the dummy thread was still included in the output of
`rb_profile_frames`.
I've now made `rb_profile_frames` skip this extra frame (like
`backtrace_each` and friends), as well as add a test that asserts
the size and contents of `rb_profile_frames`.
Fixes [Bug #18907] (<https://bugs.ruby-lang.org/issues/18907>)
Sidekiq has a method named `❨╯°□°❩╯︵┻━┻`which corrupts
heap dumps.
Normally we could just dump is as is since it's valid UTF-8 and need
no escaping. But our code to escape control characters isn't UTF-8
aware so it's more complicated than it seems.
Ultimately since the overwhelming majority of method names are
pure ASCII, it's not a big loss to just skip it.
When configuring with `--disable-rpath` and `--static-linked-ext` (e.g.
building for WASI), `extmk.rb` doesn't build exts under bundled gems,
and `.bundle/gems/#{gemname}-#{ver}` are not created due to no call of
`extmake`.
b249178398 starts creating symlink at
`.bundle/gems/#{gemname}-#{ver}/lib`, but the parent dir is not created,
so configuration aginst debug and rbs gems were failed.
The default gems have not been installed yet in the build directory,
bundled gems depending on them can not work. As those dependencies
should be usable there even without rubygems, make temporary gemspec
files without the dependencies, and use them in the build directory.
Brings a dozen call-seq schemas into compliance with the doc guide.
Adds links to section "Argument start" where needed.
Revises (minorly) ::today.
Otherwise, does not disturb existing text.
https://github.com/ruby/date/commit/9aec11df50
On Windows with OpenSSL 3, the gem fails to compile with the following
error message:
ruby/src/ext/openssl/extconf.rb:188: undefined method \`sub!' for nil:NilClass
This is because $warnflags is nil.
[Bug #18900]
Thread#join and a few other codepaths are using native sleep as
a way to suspend the current thread. So we should call the relevant
hook when this happen, otherwise some thread may transition
directly from `RESUMED` to `READY`.
I suspect that some shared pages are invalidated because
some static string don't have their coderange set eagerly.
So the first time they are scanned, the entire memory page is
invalidated.
Being able to see the coderange in `ObjectSpace` would help debug
this.
And in addition `dump` currently call `is_broken_string()` and `is_ascii_string()`
which both end up scanning the string and assigning coderange. I think it's
undesirable as `dump` should be read only.
[Feature #18339]
After experimenting with the initial version of the API I figured there is a need
for an exit event to cleanup instrumentation data. e.g. if you record data in a
{thread_id -> data} table, you need to free associated data when a thread goes away.
I tried to build Ruby on a system without libyaml today and realized
that my attempt from <https://github.com/ruby/psych/pull/557> doesn't
fix the error in <https://github.com/ruby/psych/issues/552>. I still got
the same `LoadError` from `digest` which stopped the build.
Since `LoadError` is not a `StandardError`, a plain `rescue` doesn't catch
it. Catch `LoadError` explicitly instead and reduce the scope of the
`begin` block.
I tested this change in a Ruby build on macOS without libyaml installed
and confirmed that `make` continues with a warning instead of aborting:
*** Following extensions are not compiled:
psych:
Could not be configured. It will not be installed.
...
This should address <https://bugs.ruby-lang.org/issues/18790>.
https://github.com/ruby/psych/commit/251289ba83
Previously, because opt_aref and opt_aset don't push a frame, when they
would call rb_hash to determine the hash value of the key, the initial
level of recursion would incorrectly use the method id at the top of the
stack instead of "hash".
This commit replaces rb_exec_recursive_outer with
rb_exec_recursive_outer_mid, which takes an explicit method id, so that
we can make the hash calculation behave consistently.
rb_exec_recursive_outer was documented as being internal, so I believe
this should be okay to change.
Ref: https://bugs.ruby-lang.org/issues/18339
Design:
- This tries to minimize the overhead when no hook is registered.
It should only incur an extra unsynchronized boolean check.
- The hook list is protected with a read-write lock as to cause
contention when some hooks are registered.
- The hooks MUST be thread safe, and MUST NOT call into Ruby as they
are executed outside the GVL.
- It's simply a noop on Windows.
API:
```
rb_internal_thread_event_hook_t * rb_internal_thread_add_event_hook(rb_internal_thread_event_callback callback, rb_event_flag_t internal_event, void *user_data);
bool rb_internal_thread_remove_event_hook(rb_internal_thread_event_hook_t * hook);
```
You can subscribe to 3 events:
- READY: called right before attempting to acquire the GVL
- RESUMED: called right after successfully acquiring the GVL
- SUSPENDED: called right after releasing the GVL.
The hooks MUST be threadsafe, as they are executed outside of the GVL, they also MUST NOT call any Ruby API.
Try to mirror IO behavior, where chomp takes out the entire paragraph
separators between entries, but does not chomp a single line separator
at the end of the string.
Partially Fixes [Bug #18768]
https://github.com/ruby/stringio/commit/a83ddbb7f0
Previously, this could result in an infinite loop. Always update
the e pointer in this case, setting w when chomping so the chomped
data is not included in the output.
Fixes [Bug #18769]
https://github.com/ruby/stringio/commit/4bf64d5130
In December 2021, we opened an [issue] to solicit feedback regarding the
porting of the YJIT codebase from C99 to Rust. There were some
reservations, but this project was given the go ahead by Ruby core
developers and Matz. Since then, we have successfully completed the port
of YJIT to Rust.
The new Rust version of YJIT has reached parity with the C version, in
that it passes all the CRuby tests, is able to run all of the YJIT
benchmarks, and performs similarly to the C version (because it works
the same way and largely generates the same machine code). We've even
incorporated some design improvements, such as a more fine-grained
constant invalidation mechanism which we expect will make a big
difference in Ruby on Rails applications.
Because we want to be careful, YJIT is guarded behind a configure
option:
```shell
./configure --enable-yjit # Build YJIT in release mode
./configure --enable-yjit=dev # Build YJIT in dev/debug mode
```
By default, YJIT does not get compiled and cargo/rustc is not required.
If YJIT is built in dev mode, then `cargo` is used to fetch development
dependencies, but when building in release, `cargo` is not required,
only `rustc`. At the moment YJIT requires Rust 1.60.0 or newer.
The YJIT command-line options remain mostly unchanged, and more details
about the build process are documented in `doc/yjit/yjit.md`.
The CI tests have been updated and do not take any more resources than
before.
The development history of the Rust port is available at the following
commit for interested parties:
1fd9573d8b
Our hope is that Rust YJIT will be compiled and included as a part of
system packages and compiled binaries of the Ruby 3.2 release. We do not
anticipate any major problems as Rust is well supported on every
platform which YJIT supports, but to make sure that this process works
smoothly, we would like to reach out to those who take care of building
systems packages before the 3.2 release is shipped and resolve any
issues that may come up.
[issue]: https://bugs.ruby-lang.org/issues/18481
Co-authored-by: Maxime Chevalier-Boisvert <maximechevalierb@gmail.com>
Co-authored-by: Noah Gibbs <the.codefolio.guy@gmail.com>
Co-authored-by: Kevin Newton <kddnewton@gmail.com>
WHen libyaml is not installed, make fails with the following cryptic
message:
```
gmake[2]: Entering directory '/home/chkbuild/chkbuild-crossruby/tmp/build/20220325T045825Z/ruby/ext/psych'
gmake[2]: *** No rule to make target 'yaml/yaml.h', needed by 'psych.o'. Stop.
gmake[2]: Leaving directory '/home/chkbuild/chkbuild-crossruby/tmp/build/20220325T045825Z/ruby/ext/psych'
```
I think it should give up building psych with a clear message.
Samuel Williams
Hiroshi SHIBATA
Sutou Kouhei
Aaron Patterson
Nobuyoshi Nakada
Burdette Lamar
Jemma Issroff
Maciej Rzasa
Chad Wilson
David Carlier
卜部昌平
Peter Zhu
Takashi Kokubun
Yusuke Endoh
Jean Boussier
Jeremy Evans
Alexander Momchilov
Yuta Saito
Alan Wu
Thomas Winsnes
Koichi Sasada
git
Ivo Anjo
Jean byroot Boussier
Karl Anderson
Jarek Prokop
twkmd12
Stefan Kaes
John Hawthorn
Florian Frank
Andrew Bromwich
MSP-Greg
Alan Wu
Kazuhiro NISHIYAMA
Tanaka Akira
Akinori MUSHA
Stan Lo
xtkoba
Charles Oliver Nutter