Otherwise we get
```
✗ rake TEST=test/rubygems/test_project_sanity.rb
Loaded suite /Users/deivid/.gem/ruby/3.2.0/gems/rake-13.0.6/lib/rake/rake_test_loader
Started
E
============================================================================================================================================================================================================
Error: test_manifest_is_up_to_date(TestProjectSanity):
RuntimeError: There was an error running `rake check_manifest`: /Users/deivid/.asdf/installs/ruby/3.2.1/lib/ruby/site_ruby/3.2.0/rubygems.rb:263:in `find_spec_for_exe': can't find gem rake (>= 0.a) with executable rake (Gem::GemNotFoundException)
from /Users/deivid/.asdf/installs/ruby/3.2.1/lib/ruby/site_ruby/3.2.0/rubygems.rb:282:in `activate_bin_path'
from /Users/deivid/.asdf/installs/ruby/3.2.1/bin/rake:25:in `<main>'
/Users/deivid/Code/rubygems/rubygems/test/rubygems/test_project_sanity.rb:27:in `test_manifest_is_up_to_date'
24:
25: raise "Expected Manifest.txt to be up to date, but it's not. Run `rake update_manifest` to sync it."
26: else
=> 27: raise "There was an error running `rake check_manifest`: #{out}"
28: end
29: end
30: end
============================================================================================================================================================================================================
.
Finished in 0.188192 seconds.
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
2 tests, 1 assertions, 0 failures, 1 errors, 0 pendings, 0 omissions, 0 notifications
50% passed
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
10.63 tests/s, 5.31 assertions/s
rake aborted!
```
https://github.com/rubygems/rubygems/commit/29829933a6
This reverts commit cae4342dd5.
This is failing a lot of CIs and nobody is actively looking into fixing
it. Let me revert this until we have a solution to it.
This is useful for crash triaging. It also helps to hint extension
developers about the misuse of `rb_thread_call_without_gvl()`.
Example:
$ ./miniruby -e 'Ractor.new{Ractor.receive};
Thread.new{sleep}; Process.kill:SEGV,Process.pid'
<snip>
-- Threading information ---------------------------------------------------
Total ractor count: 2
Ruby thread count for this ractor: 2
On `f(*a, **kw)` method calls, a rest keyword parameter is identically
same Hash object is passed and it should make `#dup`ed Hahs.
fix https://bugs.ruby-lang.org/issues/19526
[Bug #19529]
The fix for [Bug #19529] in commit 548086b contained a bug that crashes
on the following script:
```
wm = ObjectSpace::WeakMap.new
obj = Object.new
100.times do
wm[Object.new] = obj
GC.start
end
GC.compact
```
[Bug #19529]
`rb_gc_update_tbl_refs` can't be used on `w->obj2wmap` because it's
not a `VALUE -> VALUE` table, but a `VALUE -> VALUE *` table, so
we need some dedicated iterator.
* Revert "Remove special handling of `SIGCHLD`. (#7482)"
This reverts commit 44a0711eab.
* Revert "Remove prototypes for functions that are no longer used. (#7497)"
This reverts commit 4dce12bead.
* Revert "Remove SIGCHLD `waidpid`. (#7476)"
This reverts commit 1658e7d966.
* Fix change to rjit variable name.
This makes the behavior of classes and modules when there are too many instance variables match the behavior of objects with too many instance variables.
When a Ractor is created whilst a tracepoint for
RUBY_INTERNAL_EVENT_NEWOBJ is active, the interpreter crashes. This is
because during the early setup of the Ractor, the stdio objects are
created, which allocates Ruby objects, which fires the tracepoint.
However, the tracepoint machinery tries to dereference the control frame
(ec->cfp->pc), which isn't set up yet and so crashes with a null pointer
dereference.
Fix this by not firing GC tracepoints if cfp isn't yet set up.
We need to zero out the whole slot when running the newobj hook for a
newly allocated class because the slot could be filled with garbage,
which would cause a crash if a GC runs inside of the newobj hook.
For example, the following script crashes:
```
require "objspace"
GC.stress = true
ObjectSpace.trace_object_allocations {
100.times do
Class.new
end
}
```
[Bug #19482]
TarReader#each previously implemented a partial version of seek.
This code moved to Entry#seek for use from TarReader#each.
Entry#close now returns nil instead of true, like IO#close.
Closing an Entry now seeks to the end of the Entry, seeking past
any remaining zero byte tar file padding and moving the io to the
correcty position to read the next file in the archive.
Uses seek for Entry#rewind and #pos=, fixing the tar->gzip->tar nested
rewind that would break previous to this change.
Add Entry.open that behaves more like File.open.
https://github.com/rubygems/rubygems/commit/f5149565d5
includes word break characters
(https://github.com/ruby/irb/pull/523)
* Improve method completion for string and regexp that includes word break characters
* Remove completion-test's assert_not_include because candidates no longer include every possible methods
* Add comment about string's method completion regexp
Co-authored-by: Stan Lo <stan001212@gmail.com>
* Add comment about regexp's method completion regexp
Co-authored-by: Stan Lo <stan001212@gmail.com>
---------
https://github.com/ruby/irb/commit/aa8128c533
Co-authored-by: Stan Lo <stan001212@gmail.com>
(https://github.com/ruby/irb/pull/529)
Consider completion for this example: `foo.bar.b`
Without type information, it is hard to know the return value of the `bar`
method, so the current implementation interates through `ObjectSpace` to
get all possible candidates for the second method.
In small projects, the performance and accuracy are acceptable. But in
bigger projects, the performance is unacceptable and the accuracy is mostly
poor.
So this commit drops the support for chained methods' completion.
```
/path/to/rubygems/test/rubygems/test_gem_commands_exec_command.rb:745: warning: ambiguity between regexp and two divisions: wrap regexp in parentheses or add a space after `/' operator
```
https://github.com/rubygems/rubygems/commit/e6a538b5c8
If the main object of the context has `#delete` method, the following
warning is printed.
```
irb: warn: can't alias delete from irb_delete.
```
https://github.com/ruby/irb/commit/00b39be61f
string
(https://github.com/ruby/irb/pull/528)
* Handle long inspect and control characters in prompt string
* Add constants for prompt truncate length, omission and replace pattern
* Simply compare string instead of regexp in prompt truncation test
This test skipped sometimes due to failure to load 'rake/packagetask'.
This is due to manipulation of $LOAD_PATH by other rubygems tests. If
rake is loaded before any rubygems tests run, then it works fine.
To reproduce the skipping behavior:
$ make test-all TESTOPTS="-j6 --test-order=sorted test/rubygems/test_*.rb"
Previously, YJIT failed to put the stack into the correct shape when
`BasicObject#send` calls an alias method for the send method itself.
This can manifest as strange `NoMethodError`s in the final non-send
receiver, as [seen][1] with the kt-paperclip gem. I also found a case
where it makes YJIT fail the stack size assertion while compiling
`leave`.
YJIT's `BasicObject#__send__` implementation already rejects sends to
`send`, but didn't detect sends to aliases of `send`. Adjust the
detection and reject these cases.
Fixes [Bug #19464]
[1]: https://github.com/Shopify/yjit/issues/306
`IRB::Inspector#inspect_value` errors
(https://github.com/ruby/irb/pull/511)
**Before**
```
irb(main):001:0> c = Cat.new "foo"
(Object doesn't support #inspect)
=>
```
**After**
```
irb(main):001:0> c = Cat.new "foo"
An error occurred when inspecting the object: #<NoMethodError: undefined method `is_a?' for foo:Cat
if obj.is_a?(String)
^^^^^^>
Result of Kernel#inspect: #<Cat:0x0000000109090d80 @name="foo">
=>
```
duplicated line.
(https://github.com/ruby/reline/pull/460)
* whole_lines should consider prev_line_index, and must not duplicate last_line
* Add test for lines passed to dynamic prompt proc
* Refactor whole_lines parameters used in test helper
* Remove whole_line's arguments
Previously, when Block::entry_exit is requested from any instruction
that is not the first one in the block, we generated the exit with an
incorrect PC. We should always be using the PC for the entry of the
block for Block::entry_exit.
It was a simple typo. The bug was [introduced][1] while we were
refactoring to use the current backend. Later, we had a chance to spot
this issue while [preparing][2] to enable unused variable warnings, but
didn't spot the issue.
Fixes [Bug #19463]
[1]: 27fcab995e
[2]: 31461c7e0e
If the previous instruction is not a leaf instruction, then the PC was
incremented before the instruction was ran (meaning the currently
executing instruction is actually the previous instruction), so we
should not increment the PC otherwise we will calculate the source
line for the next instruction.
This bug can be reproduced in the following script:
```
require "objspace"
ObjectSpace.trace_object_allocations_start
a =
1.0 / 0.0
p [ObjectSpace.allocation_sourceline(a), ObjectSpace.allocation_sourcefile(a)]
```
Which outputs: [4, "test.rb"]
This is incorrect because the object was allocated on line 10 and not
line 4. The behaviour is correct when we use a leaf instruction (e.g.
if we replaced `1.0 / 0.0` with `"hello"`), then the output is:
[10, "test.rb"].
[Bug #19456]
(https://github.com/ruby/irb/pull/520)
* Remove redundant completion test
The test case was introduced to guard an old implementation, which relied
on `Module#name`. Commit:
8827d18274
However, the current implementation has avoided calling `Module#name`
completely, so the test case is no longer necessary. Commit:
88311ce3c8
* Remove unnecessary pend
(https://github.com/ruby/irb/pull/484)
* Improve assert_indenting helper
Instead of putting assertions inside the `auto_indent` block, we
can just make `auto_indent` return the calculated space count, and use
it for assertion outside of the `auto_indent` block call.
This simplifies the setup code and makes the intention easier to
understand.
* Introduce assert_row_indenting helper
1. Helper users shouldn't need to write 2 assertions for the current and
the next line's indentation.
2. With this new approach, we can generate clearer error message for
both cases:
When the current line's space count doesn't match
```
Incorrect spaces calculation for line:
```
> def each_top_level_statement
```
All lines:
```
def each_top_level_statement
```
<0> expected but was
<nil>
```
When the next line's space count doesn't match
```
Incorrect spaces calculation for line after the current line:
```
def each_top_level_statement
>
```
All lines:
```
def each_top_level_statement
```
<3> expected but was
<2>
```
* Replace assert_indenting with assert_row_indenting
There is a `time` key in GC.stat that gives us the total time spent in
GC. However, we don't know what proportion of the time is spent between
marking and sweeping. This makes it difficult to tune the GC as we're
not sure where to focus our efforts on.
This PR adds keys `marking_time` and `sweeping_time` to GC.stat for the
time spent marking and sweeping, in milliseconds.
[Feature #19437]
(https://github.com/ruby/strscan/pull/58)
`string` returns the original string after `scan` is called. Current
test doesn't check this behavior and now it's covered.
I run a 32-bit (x86) userspace on a 64-bit kernel to save memory
and this test fails for the same reason it does on pure 32-bit
platforms.
Followup-to: 6cf7c0a48f (test/readline/test_readline.rb: skip a test on i686-linux, 2021-11-09)
We saw SEGVs due to this when running with StackProf, which needs a
correct PC for RUBY_INTERNAL_EVENT_NEWOBJ, the same event used for
ObjectSpace allocation tracing.
[Bug #19444]
Don't move the timer_thread to ThreadGroup::Default, when it's
created in an enclosed ThreadGroup.
Prevents the exception: "add" can't move from the enclosed thread group"
https://github.com/ruby/timeout/commit/eb889d2c8b
This patch is follo-up of 0a82bfe.
Without this patch, if env is escaped (Proc'ed), strange svar
can be touched.
This patch tracks escaped env and use it.
When a class with a class variable is cloned we need to also copy the
cvar cache table from the original table to the clone. I found this bug
while working on fixing [Bug #19379]. While this does not fix that bug
directly it is still a required change to fix another bug revealed by
the fix in https://github.com/ruby/ruby/pull/7265
This needs to be backported to 3.2.x and 3.1.x.
Co-authored-by: Aaron Patterson <tenderlove@ruby-lang.org>
(https://github.com/ruby/irb/pull/514)
* Improve encoding error test case
The test input IRB currently uses happen to hit a compatibility bug in
TruffleRuby, which has been documented in
https://github.com/oracle/truffleruby/issues/2848
Although it'll eventually be fixed, we can make the test case support TruffleRuby
now by tweaking it just a little bit.
Co-authored-by: Kevin Menard <kevin@nirvdrum.com>
* Remove redundant TruffleRuby omits/pends
Co-authored-by: Kevin Menard <kevin@nirvdrum.com>
* Use a different way to test warning emission
The test case was added in d08ef68d2d
to verify that IRB emits Ruby warning as expected.
But the subject it uses relies on CRuby's regexp engine, which isn't always
used in other language implementations, like TruffleRuby. That's why we
ended up skipping TruffleRuby in this test case.
Since the test isn't about regexp itself, we can change the testing subject
and just remove the special condition for TruffleRuby.
Co-authored-by: Kevin Menard <kevin@nirvdrum.com>
---------
https://github.com/ruby/irb/commit/6fdf4f3e97
Co-authored-by: Kevin Menard <kevin@nirvdrum.com>
[Bug #19415]
If multiple threads attemps to load the same file concurrently
it's not a circular dependency issue.
So we check that the existing ThreadShield is owner by the current
fiber before warning about circular dependencies.
The old RUBY_GC_HEAP_INIT_SLOTS isn't really usable anymore as
it initalize all the pools by the same factor, but it's unlikely
that pools will need similar sizes.
In production our 40B pool is 5 to 6 times bigger than our 80B pool.
[Bug #19415]
If multiple threads attemps to load the same file concurrently
it's not a circular dependency issue.
So we check that the existing ThreadShield is owner by the current
fiber before warning about circular dependencies.
Create SHAPE_MAX_NUM_IVS (currently 50) and limit all shapes of
T_OBJECTS to that number of IVs. When a shape with a T_OBJECT has more than 50 IVs, fall back to the
obj_too_complex shape which uses hash lookup for ivs.
Note that a previous version of this commit
78fcc9847a was reverted in
88f2b94065 because it did not account for
non-T_OBJECTS
Fixes the same issue at https://github.com/ruby/ruby/pull/5417
`ruby` is not always available in certain build environments and
configure options (e.g. --program-suffix)
This patch tries to choose an appropriate command line for spawning a
fresh Ruby process, based on EnvUtil implementation in ruby/ruby's test
suite.
Plus when this library is directly mirrored into ruby/ruby, prefer EnvUtil
available there over the implementation in this library's test suite.
https://github.com/ruby/reline/commit/278327d2e9
Added more tests for some of the other behavior as well.
Tests were missing for readpartial with a buffer, and reading
remaining bytes after a partial read, using StringIO as reference.
https://github.com/rubygems/rubygems/commit/d600a657b1
There's a memory leak in ObjectSpace::WeakMap due to not freeing
the `struct weakmap`. It can be seen in the following script:
```
100.times do
10000.times do
ObjectSpace::WeakMap.new
end
# Output the Resident Set Size (memory usage, in KB) of the current Ruby process
puts `ps -o rss= -p #{$$}`
end
```
the final copying of the extension into place has been slimmed
down, reflecting that it only needs to copy a single file, rather
than replicating the more involved process used for a C ext
this also refactors #build so that #cargo_crate_name only needs
to be called once, and hopefully the build flow is easier to
understand
https://github.com/rubygems/rubygems/commit/5a0d7f2e6c
This was broken in ec3542229b. That commit
didn't handle cases where extended mode was turned on/off inside the
regexp. There are two ways to turn extended mode on/off:
```
/(?-x:#y)#z
/x =~ '#y'
/(?-x)#y(?x)#z
/x =~ '#y'
```
These can be nested inside the same regexp:
```
/(?-x:(?x)#x
(?-x)#y)#z
/x =~ '#y'
```
As you can probably imagine, this makes handling these regexps
somewhat complex. Due to the nesting inside portions of regexps,
the unassign_nonascii function needs to be recursive. In
recursive mode, it needs to track both opening and closing
parentheses, similar to how it already tracked opening and
closing brackets for character classes.
When scanning the regexp and coming to `(?` not followed by `#`,
scan for options, and use `x` and `i` to determine whether to
turn on or off extended mode. For `:`, indicting only the
current regexp section should have the extended mode
switched, recurse with the extended mode set or unset. For `)`,
indicating the remainder of the regexp (or current regexp portion
if already recursing) should turn extended mode on or off, just
change the extended mode flag and keep scanning.
While testing this, I noticed that `a`, `d`, and `u` are accepted
as options, in addition to `i`, `m`, and `x`, but I can't see
where those options are documented. I'm not sure whether or not
handling `a`, `d`, and `u` as options is a bug.
Fixes [Bug #19379]
[Bug #19390]
We shouldn't check the string length when skipping zeros, as the
string might only contains zero characters, resulting in an empty string.
bytesplice(index, length, str, str_index, str_length) -> string
bytesplice(range, str, str_range) -> string
In these forms, the content of +self+ is replaced by str.byteslice(str_index, str_length) or str.byteslice(str_range); however the substring of +str+ is not allocated as a new string.
In Feature #19314, we concluded that the return value of String#bytesplice
should be changed from the source string to the receiver, because the source
string is useless and confusing when extra arguments are added.
This change should be included in Ruby 3.2.1.
Some background for this refactor:
1. Through a RubyLex instance's lifetime, the context passed to its methods
should be the same.
Given that `Context` is only initialised in `Irb#initialize`,
this should be true.
2. When `RubyLex` is initialised, the context object should be accessible.
This is also true in all 3 of `RubyLex.new`'s invocations.
With the above observations, we should be able to store the context in `RubyLex`
as an instance variable. And doing so will make `RubyLex`'s instance methods
easier to use and maintain.
https://github.com/ruby/irb/commit/5c8d3df2df
On the cfunc methods, if a splat argument is given, all array elements
are expanded on the VM stack and it can cause SystemStackError.
The idea to avoid it is making a hidden array to contain all parameters
and use this array as an argv.
This patch is reviesed version of https://github.com/ruby/ruby/pull/6816
The main change is all changes are closed around calling cfunc logic.
Fixes [Bug #4040]
Co-authored-by: Jeremy Evans <code@jeremyevans.net>
(https://github.com/ruby/irb/pull/498)
When the main object is frozen, `IRB` wraps a `SimpleDelegator` around it.
But because `SimpleDelegator` doesn't delegate private methods, methods like
`require_relative` or `const_get` would cause error, which are needed for
lazily loading commands.
This commit works around this limitation by avoiding those private method calls
when setting up command execution.
sequence) to check_multiline_prompt
(https://github.com/ruby/reline/pull/458)
* pass unmodified lines to check_multiline_prompt
* Add test to check that output modified by output_modifier_proc is not passed to prompt_proc
(https://github.com/ruby/irb/pull/504)
* Simplify `RubyLex#set_prompt`
It's optional argument is never used by any caller.
* Remove the optional `p` argument from `RubyLex#set_input`
The argument is only used in a test case, which can be easily replaced by
a block argument.
```
{"address":"0x7f8c03e9fcf0", "type":"STRING", "shape_id":10, "slot_size":40, "class":"0x7f8c00dbed98", "frozen":true, "embedded":true, "fstring":true, "bytesize":5, "value":"TEST2", "encoding":"US-ASCII", "coderange":"7bit", "memsize":40, "flags":{"wb_protected":true}}
{"address":"0x7f8c03e9ffc0", "type":"STRING", "shape_id":0, "slot_size":40, "class":"0x7f8c00dbed98", "embedded":true, "bytesize":5, "value":"TEST2", "encoding":"US-ASCII", "coderange":"7bit", "memsize":40, "flags":{"wb_protected":true}}
{"address":"0x7f8c03e487c0", "type":"STRING", "shape_id":0, "slot_size":40, "class":"0x7f8c00dbed98", "embedded":true, "bytesize":5, "value":"TEST2", "encoding":"UTF-8", "coderange":"unknown", "file":"-", "line":4, "method":"dump_my_heap_please", "generation":1, "memsize":40, "flags":{"wb_protected":true}}
1) Failure:
TestObjSpace#test_dump_all [/tmp/ruby/src/trunk-gc-asserts/test/objspace/test_objspace.rb:622]:
number of strings.
<2> expected but was
<3>.
```
This failure only occurred on a ruby built with `DEFS=\"-DRGENGC_CHECK_MODE=2\""`
and only on a specific machine (Docker container) and difficult to reproduce,
so skip this failure to check other failures.
Previously, we did not update `cfp->sp` before calling the C function of
ISEQs marked with `Primitive.attr! "inline"` (leaf builtins). This
caused the GC to miss temporary values on the stack in case the function
allocates and triggers a GC run. Right now, there is only a few leaf
builtins in numeric.rb on Integer methods such as `Integer#~`. Since
these methods only allocate when operating on big numbers, we missed
this issue.
Fix by saving PC and SP before calling the functions -- our usual
protocol for calling C functions that may allocate on the GC heap.
[Bug #19316]
The reference updating code for strings is not re-embedding strings
because the code is incorrectly wrapped inside of a
`if (STR_SHARED_P(obj))` clause. Shared strings can't be re-embedded
so this ends up being a no-op. This means that strings can be moved to a
large size pool during compaction, but won't be re-embedded, which would
waste the space.
(https://github.com/ruby/irb/pull/491)
Currently, IRB has 35 test cases for 10 non-debugging commands, with the
rest 10 commands untested. So in the long-term, it could have around 70
test cases for all existing commands.
With this number of test cases, I think it's easier to manage them by grouping
them in classes (by command).
trace_arg->id is the ID of the original method of an aliased method. If
the original method is removed, then the lookup will fail. We should use
trace_arg->called_id instead, which is the ID of the aliased method.
Fixes [Bug #19305]
Previously if any of the tests that move objects between size pools
failed to move anything, then the call to stats.dig would return `nil`
which would then cause assert_operator to error.
This should be a test Failure, rather than an Error so this commit uses
a default value of 0 if stats.dig fails to find a key.
Also refactor object movement tests to use stats.dig, rather than :[]
This was already the behavior when a single `'external:internal'`
encoding specifier string was passed. This makes the behavior
consistent for the case where separate external and internal
encoding specifiers are provided.
While here, fix the IO#set_encoding method documentation to
state that either the first or second argument can be a string
with an encoding name, and describe the behavior when the
external encoding is binary.
Fixes [Bug #18899]
There is an integer underflow when the environment variable
RUBY_GC_HEAP_INIT_SLOTS is less than the number of slots currently
in the Ruby heap.
[Bug #19284]
It looks like tool/sync_default_gems.rb is not capable of cherry-picking
commits from ruby/irb. I just executed `tool/sync_default_gems.rb irb`
to fix the sync status.
I'm not sure if what's the cause. It could be related to some diff that
doesn't exist in ruby/ruby, or it might be related to non-linear history
due to merge commits. For next time, I'd like to at least exclude the
second possibility, so I disabled merge commits in ruby/irb.
It's not an intentional behavior of the command and it behaves
differently in different environments. So checking it actually brings
more problem than benefit.
https://github.com/ruby/irb/commit/b3203bc784
I noticed this while running test_yjit with --mjit-call-threshold=1,
which redefines `Integer#<`. When Ruby is monkey-patched,
MJIT itself could be broken.
Similarly, Ruby scripts could break MJIT in many different ways. I
prepared the same set of hooks as YJIT so that we could possibly
override it and disable it on those moments. Every constant under
RubyVM::MJIT is private and thus it's an unsupported behavior though.
Frozen arrays should not move from heap allocated to embedded because
frozen arrays could be shared roots for other (shared) arrays. If the
frozen array moves from heap allocated to embedded it would cause issues
since the shared array would no longer know where to set the pointer
in the shared root.
The behavior of EVP_PKEY_public_check changed between OpenSSL 1.1.1
and 3.0 so that it no longer validates the private key. Instead, private
keys can be validated through EVP_PKEY_private_check and
EVP_PKEY_pairwise_check.
[ky: simplified condition to use either EVP_PKEY_check() or
EVP_PKEY_public_check().]
https://github.com/ruby/openssl/commit/e38a63ab3d