(https://github.com/ruby/erb/pull/29)
Typically, strpbrk(3) is optimized pretty well with SIMD instructions.
Just using it makes this as fast as a SIMD-based implementation for the
no-escape case.
Not utilizing this for escaped cases because memory allocation would be
a more significant bottleneck for many strings anyway. Also, there'll be
some overhead in calling a C function (strpbrk) many times because we're
not using SIMD instructions directly. So using strpbrk all the time
might not necessarily be faster.
So different timestamps for different paths will be used. Extentions
paths in bundled gems contain `ruby_version`, which includes the ABI
version, and the same timestamp file for different paths resulted in
build failures when it changed.
- This callback is invoked when TLS key material is generated or
received, in order to allow applications to store this keying material
for debugging purposes.
- It is invoked with an `SSLSocket` and a string containing the key
material in the format used by NSS for its SSLKEYLOGFILE debugging
output.
- This commit adds the Ruby binding `keylog_cb` and the related tests
- It is only compatible with OpenSSL >= 1.1.1. Even if LibreSSL implements
`SSL_CTX_set_keylog_callback()` from v3.4.2, it does nothing (see
648d39f0f0)
https://github.com/ruby/openssl/commit/3b63232cf1
Scan through the input for a private key, then fallback to generic
decoder.
OpenSSL 3.0's OSSL_DECODER supports encoded key parameters. The PEM
header "-----BEGIN EC PARAMETERS-----" is used by one of such encoding
formats. While this is useful for OpenSSL::PKey::PKey, an edge case has
been discovered.
The openssl CLI command line "openssl ecparam -genkey" prints two PEM
blocks in a row, one for EC parameters and another for the private key.
Feeding the whole output into OSSL_DECODER results in only the first PEM
block, the key parameters, being decoded. Previously, ruby/openssl did
not support decoding key parameters and it would decode the private key
PEM block instead.
While the new behavior is technically correct, "openssl ecparam -genkey"
is so widely used that ruby/openssl does not want to break existing
applications.
Fixes https://github.com/ruby/openssl/pull/535https://github.com/ruby/openssl/commit/d486c82833
Current OpenSSL 3.0.x release has a regression with zero-length MAC
keys. While this issue should be fixed in a future release of OpenSSL,
we can use EVP_PKEY_new_raw_private_key() in place of the problematic
EVP_PKEY_new_mac_key() to avoid the issue. OpenSSL 3.0's man page
recommends using it regardless:
> EVP_PKEY_new_mac_key() works in the same way as
> EVP_PKEY_new_raw_private_key(). New applications should use
> EVP_PKEY_new_raw_private_key() instead.
Fixes https://github.com/ruby/openssl/issues/369#issuecomment-1224912710https://github.com/ruby/openssl/commit/4293f18b1f
X509at_delete_attr() in OpenSSL master puts an error queue entry if
there is no attribute left to delete. We must either clear the error
queue, or try not to call it when the list is already empty.
https://github.com/ruby/openssl/commit/a0c878481f
X509_STORE_get_ex_new_index() is a macro, so passing just its name to
have_func() doesn't detect it. Pass an example call instead.
https://github.com/ruby/openssl/commit/8d264d3e60
Co-authored-by: Nobuyoshi Nakada <nobu@ruby-lang.org>
While building with a custom build of OpenSSL, I noticed in mkmf.log
that all the feature detection checks are done using a program lacking
an OpenSSL header include. `mkmf` retries using a fallback program when
this fails, but that means all the `have_func` calls compile twice when
compiling once should suffice. Example log without this commit:
have_func: checking for X509_STORE_CTX_get0_cert()... -------------------- yes
DYLD_FALLBACK_LIBRARY_PATH=.:../.. "clang -o conftest ...
conftest.c:14:57: error: use of undeclared identifier 'X509_STORE_CTX_get0_cert'
int t(void) { void ((*volatile p)()); p = (void ((*)()))X509_STORE_CTX_get0_cert; return !p; }
^
1 error generated.
checked program was:
/* begin */
1: #include "ruby.h"
2:
3: /*top*/
4: extern int t(void);
5: int main(int argc, char **argv)
6: {
7: if (argc > 1000000) {
8: int (* volatile tp)(void)=(int (*)(void))&t;
9: printf("%d", (*tp)());
10: }
11:
12: return !!argv[argc];
13: }
14: int t(void) { void ((*volatile p)()); p = (void ((*)()))X509_STORE_CTX_get0_cert; return !p; }
/* end */
DYLD_FALLBACK_LIBRARY_PATH=.:../.. "clang -o conftest ...
checked program was:
/* begin */
1: #include "ruby.h"
2:
3: /*top*/
4: extern int t(void);
5: int main(int argc, char **argv)
6: {
7: if (argc > 1000000) {
8: int (* volatile tp)(void)=(int (*)(void))&t;
9: printf("%d", (*tp)());
10: }
11:
12: return !!argv[argc];
13: }
14: extern void X509_STORE_CTX_get0_cert();
15: int t(void) { X509_STORE_CTX_get0_cert(); return 0; }
/* end */
The second compilation succeeds.
Specify the header for each checked function.
https://github.com/ruby/openssl/commit/34ae7d92d0
openssl has to support older versions of Ruby. Undo the change in
ext/openssl/ossl_pkey_ec.c by commit efb91ff19b ("Rename
rb_ary_tmp_new to rb_ary_hidden_new", 2022-07-25).
By this change, syntax error is recovered smaller units.
In the case below, "DEFN :bar" is same level with "CLASS :Foo"
now.
```
module Z
class Foo
foo.
end
def bar
end
end
```
[Feature #19013]
I would like to check if a symbol is defined before trying to access it.
Some symbols aren't available on all platforms, so instead of raising an
exception, I want to check if it's defined first.
Today we have to do:
```ruby
begin
addr = Fiddle::Handle.sym("something")
# do something
rescue Fiddle::DLError
end
```
I want to write this:
```ruby
if Fiddle::Handle.sym_defined?("something")
addr = Fiddle::Handle.sym("something")
# do something
end
```
https://github.com/ruby/fiddle/commit/9d3371de13
Co-authored-by: Sutou Kouhei <kou@clear-code.com>
This helps to reduce repetition in code. Instead of doing "TYPE_*"
everywhere, you can do `include Fiddle::Types`, and write the type name
directly.
This PR is to help reduce repetition when writing Fiddle code. Right now
we have to type `TYPE_` everywhere, and you also have to include all of
`Fiddle` to access `TYPE_*` constants. With this change, you can just
include `Fiddle::Types` and it will shorten your code and also you only
have to include those constants.
Here is an example before:
```ruby
require "fiddle"
module MMAP
# All Fiddle constants included
include Fiddle
def self.make_function name, args, ret
ptr = Handle::DEFAULT[name]
func = Function.new ptr, args, ret, name: name
define_singleton_method name, &func.to_proc
end
make_function "munmap", [TYPE_VOIDP, # addr
TYPE_SIZE_T], # len
TYPE_INT
make_function "mmap", [TYPE_VOIDP,
TYPE_SIZE_T,
TYPE_INT,
TYPE_INT,
TYPE_INT,
TYPE_INT], TYPE_VOIDP
make_function "mprotect", [TYPE_VOIDP, TYPE_SIZE_T, TYPE_INT], TYPE_INT
end
```
After:
```ruby
require "fiddle"
module MMAP
# Only type names included
include Fiddle::Types
def self.make_function name, args, ret
ptr = Fiddle::Handle::DEFAULT[name]
func = Fiddle::Function.new ptr, args, ret, name: name
define_singleton_method name, &func.to_proc
end
make_function "munmap", [VOIDP, # addr
SIZE_T], # len
INT
make_function "mmap", [VOIDP, SIZE_T, INT, INT, INT, INT], VOIDP
make_function "mprotect", [VOIDP, SIZE_T, INT], INT
end
```
We only need to import the type names, and you don't have to type
`TYPE_` over and over. I think this makes Fiddle code easier to read.
https://github.com/ruby/fiddle/commit/49fa7233e5
Co-authored-by: Sutou Kouhei <kou@clear-code.com>
This commit adds constants for unsigned values. Currently we can use `-`
to mean "unsigned", but I think having a specific name makes Fiddle more
user friendly. This commit continues to support `-`, but introduces
negative constants with "unsigned" names
I think this will help to eliminate [this
code](3a56bf0bcc/lib/mjit/c_type.rb (L31-L38))
https://github.com/ruby/fiddle/commit/2bef0f1082
Co-authored-by: Sutou Kouhei <kou@clear-code.com>
Moves Expect library doc into io.c.
Changes certain links to local sections, now pointing to sections in doc/io_streams.rdoc.
Removes local sections now superseded by sections in doc/io_streams.rdoc.
Object Shapes is used for accessing instance variables and representing the
"frozenness" of objects. Object instances have a "shape" and the shape
represents some attributes of the object (currently which instance variables are
set and the "frozenness"). Shapes form a tree data structure, and when a new
instance variable is set on an object, that object "transitions" to a new shape
in the shape tree. Each shape has an ID that is used for caching. The shape
structure is independent of class, so objects of different types can have the
same shape.
For example:
```ruby
class Foo
def initialize
# Starts with shape id 0
@a = 1 # transitions to shape id 1
@b = 1 # transitions to shape id 2
end
end
class Bar
def initialize
# Starts with shape id 0
@a = 1 # transitions to shape id 1
@b = 1 # transitions to shape id 2
end
end
foo = Foo.new # `foo` has shape id 2
bar = Bar.new # `bar` has shape id 2
```
Both `foo` and `bar` instances have the same shape because they both set
instance variables of the same name in the same order.
This technique can help to improve inline cache hits as well as generate more
efficient machine code in JIT compilers.
This commit also adds some methods for debugging shapes on objects. See
`RubyVM::Shape` for more details.
For more context on Object Shapes, see [Feature: #18776]
Co-Authored-By: Aaron Patterson <tenderlove@ruby-lang.org>
Co-Authored-By: Eileen M. Uchitelle <eileencodes@gmail.com>
Co-Authored-By: John Hawthorn <john@hawthorn.email>
Object Shapes is used for accessing instance variables and representing the
"frozenness" of objects. Object instances have a "shape" and the shape
represents some attributes of the object (currently which instance variables are
set and the "frozenness"). Shapes form a tree data structure, and when a new
instance variable is set on an object, that object "transitions" to a new shape
in the shape tree. Each shape has an ID that is used for caching. The shape
structure is independent of class, so objects of different types can have the
same shape.
For example:
```ruby
class Foo
def initialize
# Starts with shape id 0
@a = 1 # transitions to shape id 1
@b = 1 # transitions to shape id 2
end
end
class Bar
def initialize
# Starts with shape id 0
@a = 1 # transitions to shape id 1
@b = 1 # transitions to shape id 2
end
end
foo = Foo.new # `foo` has shape id 2
bar = Bar.new # `bar` has shape id 2
```
Both `foo` and `bar` instances have the same shape because they both set
instance variables of the same name in the same order.
This technique can help to improve inline cache hits as well as generate more
efficient machine code in JIT compilers.
This commit also adds some methods for debugging shapes on objects. See
`RubyVM::Shape` for more details.
For more context on Object Shapes, see [Feature: #18776]
Co-Authored-By: Aaron Patterson <tenderlove@ruby-lang.org>
Co-Authored-By: Eileen M. Uchitelle <eileencodes@gmail.com>
Co-Authored-By: John Hawthorn <john@hawthorn.email>
getenv is a very basic function that has been in stdlib.h since
ISO/IEC 9899:1990. There is absolutely zero need for us to redeclare.
pty.c already includes stdlib.h out of the box so we need nothing.
The reason why this was commented out was because of gperf 3.0 vs 3.1
differences (see [Feature #13883]). Five years passed, I am pretty
confident that we can drop support of old versions here.
Ditto for uniname2ctype_p(), onig_jis_property(), and zonetab().
Fixes id.h error during updating ripper.c by `make after-update`.
While it used to update id.h in the build directory, but was trying to
update ripper.c in the source directory. In principle, files in the
source directory can or should not depend on files in the build
directory.
icc now seems to provide libutil.so that is not related to pty.
This extconf.rb wrongly finds it and adds `-lutil`, but `ruby -rpty`
fails because it cannot find libutil.so on the runtime.
http://rubyci.s3.amazonaws.com/icc-x64/ruby-master/log/20220815T210005Z.fail.html.gz
```
Exception raised:
<#<LoadError: libutil.so: cannot open shared object file: No such file or directory - /home/chkbuild/chkbuild/tmp/build/20220815T210005Z/ruby/.ext/x86_64-linux/pty.so>>
```
This change makes extconf.rb check libutil only on OpenBSD.
As commented in include/ruby/internal/abi.h, since teeny versions of
Ruby should guarantee ABI compatibility, `RUBY_ABI_VERSION` has no role
in released versions of Ruby.
* Fix Array#[] with ArithmeticSequence with negative steps
Previously, Array#[] when called with an ArithmeticSequence
with a negative step did not handle all cases correctly,
especially cases involving infinite ranges, inverted ranges,
and/or exclusive ends.
Fixes [Bug #18247]
* Add Array#slice tests for ArithmeticSequence with negative step to test_array
Add tests of rb_arithmetic_sequence_beg_len_step C-API function.
* Fix ext/-test-/arith_seq/beg_len_step/depend
* Rename local variables
* Fix a variable name
Co-authored-by: Kenta Murata <3959+mrkn@users.noreply.github.com>
Adding `ruby` to `PREP` causes the following circular dependencies
because `PREP` is used as a prerequisite by some targets required to
build `ruby` target itself.
```
make: Circular .rbconfig.time <- ruby dependency dropped.
make: Circular builtin_binary.inc <- ruby dependency dropped.
make: Circular ext/extinit.c <- ruby dependency dropped.
make: Circular ruby <- ruby dependency dropped.
```
Adding a new Make variable like `EXTPREP` only for exts may be also
reasonable, but it would introduce another complexity into our build
system. `-bundle_loader` doesn't care that link-time and run-time
loader executables are different as long as bound symbols are provided,
so it's ok to resolve from miniruby to simplify our build.
For the macOS -bundle_loader linker option, we need a path to the
Ruby exectuable. $(RUBY) is not necessarily a path since it could
be a command line invocation. That happens during build with
runruby.rb and can happen post installation if the user passes
the --ruby option to a extconf.rb. Use $(bindir) to locate
the executable instead.
Before installation, $(bindir) doesn't exist, so we need to be
able to override $(BUILTRUBY) in such situations so test-spec
and bundled extensions could build. Use a new mkmf global,
$builtruby, to do this; set it in fake.rb and in extmk.rb.
Our build system is quite complex...
ld64 shipped with Xcode 14 emits a warning when using `-undefined
dynamic_lookup`.
```
ld: warning: -undefined dynamic_lookup may not work with chained fixups
```
Actually, `-undefined dynamic_lookup` doesn't work when:
1. Link a *shared library* with the option
2. Link it with a program that uses the chained-fixup introduced from
macOS 12 and iOS 15
because `-undefined dynamic_lookup` uses lazy-bindings and they won't be
bound while dyld fixes-up by traversing chained-fixup info.
However, we build exts as *bundles* and they are loaded only through
`dlopen`, so it's safe to use `-undefined dynamic_lookup` in theory.
So the warning produced by ld64 is false-positive, and it results
failure of option checking in configuration. Therefore, it would be an
option to ignore the warning during our configuration.
On the other hand, `-undefined dynamic_lookup` is already deprecated on
all darwin platforms except for macOS, so it's good time to get rid of
the option. ld64 also provides `-bundle_loader <executable>` option,
which allows to resolve symbols defined in the executable symtab while
linking. It behaves almost the same with `-undefined dynamic_lookup`,
but it makes the following changes:
1. Require that unresolved symbols among input objects must be defined
in the executable.
2. Lazy symbol binding will lookup only the symtab of the bundle loader
executable. (`-undefined dynamic_lookup` lookups all symtab as flat
namespace)
This patch adds `-bundle_loader $(RUBY)` when non-EXTSTATIC
configuration by assuming ruby executable can be linked before building
exts.
See "New Features" subsection under "Linking" section for chained fixup
https://developer.apple.com/documentation/xcode-release-notes/xcode-13-release-notes
Treats:
::httpdate
#to_date
#to_time
#to_datetime
In behalf of ::httpdate, I've introduced section "Argument limit" that can be pointed to by various Date methods (similar to existing section "Argument start"). This will involve 8 already-enhanced methods plus 8 more not yet done.
https://github.com/ruby/date/commit/00326ff99c
Tabs were expanded because the file did not have any tab indentation in unedited lines.
Please update your editor config, and use misc/expand_tabs.rb in the pre-commit hook.
Tabs were expanded because the file did not have any tab indentation in unedited lines.
Please update your editor config, and use misc/expand_tabs.rb in the pre-commit hook.
It makes testing for JSON errors very tedious. You either have
to use a Regexp or to regularly update all your assertions
when JSON is upgraded.
https://github.com/flori/json/commit/de9eb1d28e
rb_ary_tmp_new suggests that the array is temporary in some way, but
that's not true, it just creates an array that's hidden and not on the
transient heap. This commit renames it to rb_ary_hidden_new.
The `rb_profile_frames` API did not skip the two dummy frames that
each thread has at its beginning. This was unlike `backtrace_each` and
`rb_ec_parcial_backtrace_object`, which do skip them.
This does not seem to be a problem for non-main thread frames,
because both `VM_FRAME_RUBYFRAME_P(cfp)` and
`rb_vm_frame_method_entry(cfp)` are NULL for them.
BUT, on the main thread `VM_FRAME_RUBYFRAME_P(cfp)` was true
and thus the dummy thread was still included in the output of
`rb_profile_frames`.
I've now made `rb_profile_frames` skip this extra frame (like
`backtrace_each` and friends), as well as add a test that asserts
the size and contents of `rb_profile_frames`.
Fixes [Bug #18907] (<https://bugs.ruby-lang.org/issues/18907>)
Sidekiq has a method named `❨╯°□°❩╯︵┻━┻`which corrupts
heap dumps.
Normally we could just dump is as is since it's valid UTF-8 and need
no escaping. But our code to escape control characters isn't UTF-8
aware so it's more complicated than it seems.
Ultimately since the overwhelming majority of method names are
pure ASCII, it's not a big loss to just skip it.
When configuring with `--disable-rpath` and `--static-linked-ext` (e.g.
building for WASI), `extmk.rb` doesn't build exts under bundled gems,
and `.bundle/gems/#{gemname}-#{ver}` are not created due to no call of
`extmake`.
b249178398 starts creating symlink at
`.bundle/gems/#{gemname}-#{ver}/lib`, but the parent dir is not created,
so configuration aginst debug and rbs gems were failed.
The default gems have not been installed yet in the build directory,
bundled gems depending on them can not work. As those dependencies
should be usable there even without rubygems, make temporary gemspec
files without the dependencies, and use them in the build directory.
Takashi Kokubun
Nobuyoshi Nakada
Peter Zhu
Burdette Lamar
Jemma Issroff
Aaron Patterson
Sutou Kouhei
Ben Toews
Bart de Water
Samuel Williams
Christophe De La Fuente
Kazuki Yamaguchi
madblobfish
Nobuhiro IMAI
Alan Wu
yui-knk
Hiroshi SHIBATA
Maciej Rzasa
Chad Wilson
David Carlier
卜部昌平
Yusuke Endoh
Jean Boussier
Jeremy Evans
Alexander Momchilov
Yuta Saito
Thomas Winsnes
Koichi Sasada
git
Ivo Anjo
Jean byroot Boussier
Karl Anderson