Always test only the scripts just under “lib", and just under
child directories which has not the same name script in the upper
level; instead of random sampling from whole libraries.
Previously, Kernel#lambda returned a non-lambda proc when given a
non-literal block and issued a warning under the `:deprecated` category.
With this change, Kernel#lambda will always return a lambda proc, if it
returns without raising.
Due to interactions with block passing optimizations, we previously had
two separate code paths for detecting whether Kernel#lambda got a
literal block. This change allows us to remove one path, the hack done
with rb_control_frame_t::block_code introduced in 85a337f for supporting
situations where Kernel#lambda returned a non-lambda proc.
[Feature #19777]
Co-authored-by: Takashi Kokubun <takashikkbn@gmail.com>
These are replacements for regular expressions when they are used
alone as the predicate of a conditional. That's because they are
significantly different from a regular expression because they are
not evaluated for truthyness, but instead evaluated as a match
against the last line read by an IO object.
https://github.com/ruby/yarp/commit/0f1c7780e8
The flags integer is an implementation detail. We want people to
use the query methods to access the individual fields so we are
freed from having to maintain a specific order. As such, this
commit changes the Ruby API to mark all flags fields as private
attr_readers.
The only one that has a clear use case is returning the set of
options given to regular expressions, to mirror the Regexp#options
API. So, to support this use case, this commit introduces
RegularExpressionNode#options and InterpolatedRegularExpressionNode#options.
These APIs provide back the same integer so that they can be used
interchangeably.
https://github.com/ruby/yarp/commit/4e6d5dd99f
This test creates a lot of Objects held in an array, and a set of weak
references to them using WeakMap. It then clears the array and frees it
and asserts that all the weak references to it are also gone.
This test is failing because one of the dummy objects in our weakmap is
ending up on the stack, and so is being marked, even though we thought
that we'd removed the only reference to it.
This behaviour has changed since this commit:
5b5ae3d9e0
which rewrites `Integer#times` from C into Ruby. This change is somehow
causing the last object we append to our array to consistently end up on
the stack during GC.
This commit fixes the specific weakmap test by using an enumerator and
each, instead of `Integer#times`, and thus avoids having our last object
created end up on the stack.
The parser now passes around `yp_diagnostic_id_t` for diagnostic
messages instead of character strings, and we rely on the function
`diagnostic_message()` to resolve that to a string.
In addition, many messages were edited so that the parser expresses
coordinate ideas in similar form [1] using consistent voice and
typographic conventions.
Closes https://github.com/ruby/yarp/pull/1379, and makes progress on #941.
[1] Strunk & White rule 19
https://github.com/ruby/yarp/commit/0b6dd85bf1
Before this commit, constants in the constant pool were assumed to
be slices of the source string. This works in _almost_ all cases.
There are times, however, when a string needs to be synthesized.
This can occur when passing in locals that need to be scoped through
eval, or when generating method names like `foo=`.
After this commit, there is a single bit `owned` boolean on
constants in the pool that indicates whether or not it is a slice
of the source string. If it is not, it is assumed to be allocated
memory that should be freed by the constant pool when the constant
pool is freed.
When serializing, the most significant bit in the location of the
contents of the constant indicates whether or not it is owned.
When it is, instead of 4 bytes for the source offset and 4 bytes
for the length it is instead 4 bytes for the buffer offset and 4
bytes the length. The contents of the owned constants are embedded
into the buffer after the constant pool itself.
https://github.com/ruby/yarp/commit/461c047365
`ppc64le` appears to be struggling with this test due to timeout. Let's see
if reducing the number of iterations can help improve the test performance.
This test sometimes fails with:
```
1) Failure:
TestProcess#test_warmup_frees_pages [test/ruby/test_process.rb:2750]:
<202> expected but was
<203>.
```
Check that OpenSSL::PKey::EC.builtin_curves returns an array in the
expected format.
Similarly to OpenSSL::Cipher.ciphers, OpenSSL::PKey::EC.builtin_curves
returns a list of known named curves rather than actually usable ones.
https://github.com/ruby/openssl/issues/671 found that the list may
include unapproved (and thus unusable) curves when the FIPS module is
loaded.
https://github.com/ruby/openssl/commit/c53cbabe00
The test case test_error_data utilizes the error message generated by
X509V3_EXT_nconf_nid(). The next commit will use X509V3_EXT_nconf(),
which generates a slightly different error message. Let's adapt the
check to it.
https://github.com/ruby/openssl/commit/9cdfa3a4d1
instead of looking of NIDs and then using X509V3_EXT_nconf_nid,
instead just pass strings to X509V3_EXT_nconf, which has all the logic for
processing dealing with generic extensions
also process the oid through ln2nid() to retain compatibility.
[rhe: tweaked commit message and added a test case]
https://github.com/ruby/openssl/commit/9f15741331
OpenSSL::X509::ExtensionFactory#create_ext and #create_extensions
accepts both sn (short names) and ln (long names) for registered OIDs.
This is different from the behavior of the openssl command-line utility
which accepts only sn in openssl.cnf keys.
Add a test case to check this.
https://github.com/ruby/openssl/commit/91ae46c8d7
Remove the OSSL_DEBUG flag and OpenSSL.mem_check_start which is only
compiled when the flag is given. They are meant purely for development
of Ruby/OpenSSL.
OpenSSL.mem_check_start helped us find memory leak bugs in past, but
it is no longer working with the recent OpenSSL versions. Let's just
remove it now.
https://github.com/ruby/openssl/commit/8c7a6a17e2
This is a tradeoff that I think is worth it. Right now we have a
location list that tracks the location of each of the block locals.
Instead, I'd like to make that a node list that has a proper node
in each spot in the list. In doing so, we eliminate the need to have
a location list at all, making it simpler on all of the various
consumers as we have one fewer field type. There should be minimal
memory implications here since this syntax is exceedingly rare.
https://github.com/ruby/yarp/commit/04d329ddf0
```
/home/chkbuild/chkbuild/tmp/build/20230905T063003Z/ruby/test/yarp/compiler_test.rb:16: warning: ambiguous first argument; put parentheses or a space even after `+' operator
/home/chkbuild/chkbuild/tmp/build/20230905T063003Z/ruby/test/yarp/compiler_test.rb:17: warning: ambiguous first argument; put parentheses or a space even after `-' operator
/home/chkbuild/chkbuild/tmp/build/20230905T063003Z/ruby/test/yarp/compiler_test.rb:28: warning: ambiguous first argument; put parentheses or a space even after `+' operator
/home/chkbuild/chkbuild/tmp/build/20230905T063003Z/ruby/test/yarp/compiler_test.rb:29: warning: ambiguous first argument; put parentheses or a space even after `-' operator
```
http://rubyci.s3.amazonaws.com/debian10/ruby-master/log/20230905T063003Z.log.html.gz
```
[ 3463/25719] JSONGeneratorTest#test_string_ext_included_calls_super/home/chkbuild/chkbuild/tmp/build/20230905T063003Z/ruby/test/json/json_generator_test.rb:411: warning: method redefined; discarding old included
/home/chkbuild/chkbuild/tmp/build/20230905T063003Z/ruby/test/json/json_generator_test.rb:399: warning: previous definition of included was here
```
http://rubyci.s3.amazonaws.com/debian10/ruby-master/log/20230905T063003Z.log.html.gz
This allows RDoc to better generate documentation for methods
following the Ruby core documentation guide (which omits aliases
in call-seq in most cases). This makes documentation for methods
defined in C more similar to methods defined in Ruby. For methods
defined in Ruby, the method description of the aliased method is
already not used (you have to explicitly document the alias to
use it).
Internally, this adds AnyMethod#has_call_seq? and #skip_description?,
and updates Darkfish to:
* only show the method name if there is a call-seq for the method,
but the call-seq omits the method
* to omit the method description if the method is an alias or has
aliases and has a call-seq that does not include the method
See discussion in https://github.com/ruby/ruby/pull/7316 for
details.
https://github.com/ruby/rdoc/commit/e3688de49b
This test sometimes fails with:
```
1) Failure:
TestProcess#test_warmup_frees_pages [test/ruby/test_process.rb:2751]:
<0> expected but was
<1>.
```
I'm not sure why, so add some debug info.
The test sometimes fails with:
```
1) Failure:
TestProcess#test_warmup_frees_pages [test/ruby/test_process.rb:2749]:
<0> expected but was
<1>.
```
I think there's a page with an object that needs finalization, so run
GC to clear that object.
The test sometimes fails with:
```
1) Failure:
TestProcess#test_warmup_run_major_gc_and_compact [test/ruby/test_process.rb:2712]:
<2> expected but was
<3>.
```
Essentially, this change updates `yp_unescape_calculate_difference` to
not create syntax errors, and we rely entirely on
`yp_unescape_manipulate_string` to report syntax errors.
To do that, this PR adds another (!) parameter to `unescape`:
`yp_list_t *error_list`. When present, `unescape` reports syntax
errors (and otherwise does not).
However, an edge case that needed to be addressed is reporting syntax
errors in this case:
?\u{1234 2345}
In a string context, it's possible to have multiple codepoints by
doing something like `"\u{1234 2345}"`; however, in the character
literal context, this is a syntax error -- only a single codepoint is
allowed.
Unfortunately, when `yp_unescape_manipulate_string` is called, there's
nothing to indicate that we are in a "character literal" context and
that only a single codepoint is valid.
To make this work, this PR:
- introduces a new static utility function in yarp.c,
`yp_char_literal_node_create_and_unescape`, which is called when
we're parsing `YP_TOKEN_CHARACTER_LITERAL`
- introduces a new (unexported) function,
`yp_unescape_manipulate_char_literal` which does the same thing as
`yp_unescape_manipulate_string` but tells `unescape` that only a
single codepoint is expected
https://github.com/ruby/yarp/commit/f6a65840b5
* Consistent with ClassVariableWriteNode, ConstantWriteNode, InstanceVariableWriteNode, GlobalVariableWriteNode.
* Fixes desugaring of local variable with operators.
https://github.com/ruby/yarp/commit/9a66737775
* Consistent with ClassVariableWriteNode, ConstantWriteNode, InstanceVariableWriteNode, LocalVariableWriteNode.
* Fixes desugaring of global variable with operators.
https://github.com/ruby/yarp/commit/fb5a53fc0b
file
(https://github.com/ruby/yarp/pull/1371)
* refactor: move EOF check into yp_unescape_calculate_difference
parser_lex is a bit more readable when we can rely on that behavior
* fix: octal and hex digits at the end of a file
Previously this resulted in invalid memory access.
* fix: unicode strings at the end of a file
Previously this resulted in invalid memory access.
* Unterminated curly-bracket unicode is a syntax error
https://github.com/ruby/yarp/commit/21cf11acb5
OpenBSD and Solaris behave differently here.
Linux does deliver the empty packet, which is questionable
as it's undistinguishable from a closed connection.
It seems that OpenBSD and Solaris simply drop it.
We could test the platform before doing the assertion, but
it would likely be fragile, and the entire web recommend
to not ever send an empty packet, so the value of this
assertion is low.
The old algorithm could calculate an undercount for the initial pages
due to two issues:
1. It did not take into account that some heap pages will have one less
slot due to alignment. It assumed that every heap page would be able
to be fully filled with slots. Pages that are unaligned with the slot
size will lose one slot. The new algorithm assumes that every page
will be unaligned.
2. It performed integer division, which truncates down. This means that
the number of pages might not actually satisfy the number of slots.
This can cause the heap to grow in `gc_sweep_finish_size_pool` after
allocating all of the allocatable pages because the total number of
slots would be less than the initial configured number of slots.
This commit changes RUBY_GC_HEAP_INIT_SIZE_{40,80,160,320,640}_SLOTS to
RUBY_GC_HEAP_{0,1,2,3,4}_INIT_SLOTS. This is easier to use because the
user does not need to determine the slot sizes (which can vary between
32 and 64 bit systems). They now just use the heap names
(`GC.stat_heap.keys`).
Also, a similar test and fix for interpolated regular expressions.
This snippet:
<<-A.g//,
A
/{/, ''\
previously created a regular expression node with inverted start and
end:
RegularExpressionNode(14...13)((14...15), (15...21), (12...13), ", ''", 0),
which failed an assertion during serialization.
After this change:
RegularExpressionNode(12...15)((14...15), (15...21), (12...13), ", ''", 0),
Found by the fuzzer.
https://github.com/ruby/yarp/commit/5fef572f95
The snippet added in this commit previously resulted in a CallNode
with inverted start and end locations:
> AssocNode(15...13)(
> CallNode(15...13)(
StringNode(15...17)((15...16), (16...16), (16...17), ""),
nil,
(12...13),
nil,
ArgumentsNode(12...13)([MissingNode(12...13)()]),
nil,
nil,
0,
"/"
),
MissingNode(13...13)(),
(13...13)
),
which failed an assertion during serialization.
After this change, it looks better:
> AssocNode(12...13)(
> CallNode(12...17)(
StringNode(15...17)((15...16), (16...16), (16...17), ""),
nil,
(12...13),
nil,
ArgumentsNode(12...13)([MissingNode(12...13)()]),
nil,
nil,
0,
"/"
),
MissingNode(13...13)(),
(13...13)
),
Found by the fuzzer.
https://github.com/ruby/yarp/commit/040aa63ad6
The presence of the heredocs in this snippet with invalid syntax:
for <<A + <<B
A
B
causes the MissingNode to have a location after other nodes in the
list, resulting in a StatementsNode with inverted start and end
locations:
[ForNode(0...14)(
MultiWriteNode(4...7)([InterpolatedStringNode(4...7)((4...7), [], (14...16))], nil, nil, nil, nil),
MissingNode(16...16)(),
> StatementsNode(16...14)(
[MissingNode(16...16)(), InterpolatedStringNode(10...13)((10...13), [], (16...18)), MissingNode(13...14)()]
),
(0...3),
(16...16),
nil,
(14...14)
)]
which failed an assertion during serialization.
With this fix, the node's locations are:
[ForNode(0...14)(
MultiWriteNode(4...7)([InterpolatedStringNode(4...7)((4...7), [], (14...16))], nil, nil, nil, nil),
MissingNode(16...16)(),
> StatementsNode(10...16)(
[MissingNode(16...16)(), InterpolatedStringNode(10...13)((10...13), [], (16...18)), MissingNode(13...14)()]
),
(0...3),
(16...16),
nil,
(14...14)
)]
Found by the fuzzer.
https://github.com/ruby/yarp/commit/09bcedc05e
Previously this resulted in invalid memory access as well as a
cascading failed assertion:
src/enc/yp_unicode.c:2224: yp_utf_8_codepoint: Assertion `n >= 1' failed.
Found by the fuzzer.
https://github.com/ruby/yarp/commit/a34c534440
Class variables, global variables, constants, and constant paths
should actually desugar to `defined?` instead of just reading the
value.
https://github.com/ruby/yarp/commit/551a59b876
[Bug #19012]
man recvmsg(2) states:
> Return Value
> These calls return the number of bytes received, or -1 if an error occurred.
> The return value will be 0 when the peer has performed an orderly shutdown.
Not too sure how one is supposed to make the difference between a packet of
size 0 and a closed connection.
If initial slots is set, then during a minor GC, if we have allocatable
pages but the heap is mostly full, then we will set `grow_heap` to true
since `total_slots` does not count allocatable pages so it will be less
than `init_slots`. This can cause `allocatable_pages` to grow to much
higher than desired since it will appear that the heap is mostly full.
Two fixes were necessary:
- ensure we are handling newlines correctly
- accept two consecutive string tokens without a separator
https://github.com/ruby/yarp/commit/4e707937cb
Co-authored-by: Kevin Newton <kddnewton@gmail.com>
Specific files are named earlier in the block, and we now have the
ability to skip just the lex matching, or skip ripper entirely (for
files that don't parse).
https://github.com/ruby/yarp/commit/dcd3806dca
This commit adds `free_empty_pages` which frees all empty heap pages and
moves the number of pages freed to the allocatable pages counter. This
is used in Process.warmup to improve performance because page
invalidation from copy-on-write is slower than allocating a new page.
This should make it easier on the sync to determine what changed
and hopefully result in fewer merge conflicts that have to be
manually resolved.
https://github.com/ruby/yarp/commit/17d82afbfc
Hiroshi SHIBATA
Nobuyoshi Nakada
Kevin Newton
Alan Wu
Jemma Issroff
Benoit Daloze
negi0109
Tim Morgan
Vinicius Stock
Matt Valentine-House
Takashi Kokubun
Mike Dalessio
Samuel Williams
Andy Waite
Peter Zhu
Kazuki Yamaguchi
Michael Richardson
Mau Magnaguagno
Yusuke Endoh
yui-knk
Jeremy Evans
theo-squadracer
ywenc
John Hawthorn
Jean Boussier
Nathan Froyd
Jean Boussier
Stan Lo
tomoya ishida
Sutou Kouhei
Summer ☀️
Kouhei Yanagita
Jun Aruga