The default gems have not been installed yet in the build directory,
bundled gems depending on them can not work. As those dependencies
should be usable there even without rubygems, make temporary gemspec
files without the dependencies, and use them in the build directory.
Since extension libraries can not be built in the source directory,
rubygems warns gems have extension libraries as the extensions are not
built. To order to suppress this warnings, extract such gemspec files
under each gem directories instead of the common `specifications`
directory.
Since a few commits ago, we no longer call `Gem::Specification.reset`
after each invocation of `Gem::Installer#install`. This means we don't
call it when the default Bundler is installed during `gem update
--system`. This causes no issues until the end of the upgrade process
when:
* The previous default Bundler spec is removed from disk.
* All specification stubs are turned into real specifications by loading
them from disk. But the one for Bundler no longer exists so
materializes to `nil` and regenerating binstubs crashes like this:
```
Bundler 2.4.0.dev installed
RubyGems 3.4.0.dev installed
Regenerating binstubs
ERROR: While executing gem ... (NoMethodError)
undefined method `platform' for nil:NilClass
/Users/deivid/Code/rubygems/rubygems/lib/rubygems/commands/pristine_command.rb:116:in `block in execute'
/Users/deivid/Code/rubygems/rubygems/lib/rubygems/specification.rb:981:in `block in each'
/Users/deivid/Code/rubygems/rubygems/lib/rubygems/specification.rb:980:in `each'
/Users/deivid/Code/rubygems/rubygems/lib/rubygems/specification.rb:980:in `each'
/Users/deivid/Code/rubygems/rubygems/lib/rubygems/commands/pristine_command.rb:116:in `map'
/Users/deivid/Code/rubygems/rubygems/lib/rubygems/commands/pristine_command.rb:116:in `each'
/Users/deivid/Code/rubygems/rubygems/lib/rubygems/commands/pristine_command.rb:116:in `select'
/Users/deivid/Code/rubygems/rubygems/lib/rubygems/commands/pristine_command.rb:116:in `execute'
/Users/deivid/Code/rubygems/rubygems/lib/rubygems/command.rb:323:in `invoke_with_build_args'
/Users/deivid/Code/rubygems/rubygems/lib/rubygems/command.rb:301:in `invoke'
/Users/deivid/Code/rubygems/rubygems/lib/rubygems/commands/setup_command.rb:604:in `regenerate_binstubs'
/Users/deivid/Code/rubygems/rubygems/lib/rubygems/commands/setup_command.rb:183:in `execute'
/Users/deivid/Code/rubygems/rubygems/lib/rubygems/command.rb:323:in `invoke_with_build_args'
/Users/deivid/Code/rubygems/rubygems/lib/rubygems/command_manager.rb:185:in `process_args'
/Users/deivid/Code/rubygems/rubygems/lib/rubygems/command_manager.rb:149:in `run'
/Users/deivid/Code/rubygems/rubygems/lib/rubygems/gem_runner.rb:51:in `run'
setup.rb:33:in `<main>'
```
We fix it by more carefully managing the removal of the previous default
Bundler gem.
https://github.com/rubygems/rubygems/commit/9989f6d5af
We saw the following failure:
```
TestThreadInstrumentation#test_thread_instrumentation [/tmp/ruby/v3/src/trunk-random3/test/-ext-/thread/test_instrumentation_api.rb:25]:
Expected 0..3 to include 4.
```
Which shouldn't happen unless somehow there was a leaked thread.
An error message is primarily rendered in a terminal emulator, but is
also shown in a browser by converting it to a HTML fragment.
However, the conversion would be unreasonably difficult if the message
includes any escape sequence (such as cursor move or screen clear).
This change adds a guideline about escape sequences in
`Exception#detailed_message`:
* Use widely-supported escape sequences: bold, underline, and basic
eight foreground colors (except white and black).
* Make the message readable if all escape sequences are ignored.
Splits certain guidelines for singleton and instance method.
Calls for instance method to not prefix anything (like RDoc itself for a Ruby-coded instance method); e.g.:
count -> integer, not array.count,.
<=> other -> integer or nil, not hash <=> other -> integer or nil.
Groups previous guidelines into Arguments, Block, Return types, Aliases.
`rb_thread_wait_for_single_fd` needs to mutate the `waiting_fds` list
that is stored on the VM. We need to delete the FD from the list before
returning, and deleting from the list requires a VM lock (because the
list is a global).
[Bug #18816] [ruby-core:108771]
Co-Authored-By: Alan Wu <alanwu@ruby-lang.org>
In a small script the speed of this feature isn't really noticeable but
on Rails it's very noticeable how slow this can be. This PR aims to
speed up two parts of the functionality.
1) The Rust exit recording code
Instead of adding all samples as we see them to the yjit_raw_samples and
yjit_line_samples, we can increment the counter on the ones we've seen
before. This will be faster on traces where we are hitting the same
stack often. In a crude measurement of booting just the active record
base test (`test/cases/base_test.rb`) we found that this improved the
speed by 1 second.
This also results in a smaller marshal dump file which sped up the test
boot time by 4 seconds with trace exits on.
2) The Ruby parsing code
Previously we were allocating new arrays using `shift` and
`each_with_index`. This change avoids allocating new arrays by using an
index. This change saves us the most amount of time, gaining 11 seconds.
Before this change the test boot time took 62 seconds, after it took 47
seconds. This is still too long but it's a step closer to faster
functionality. Next we're going to tackle allowing you to collect trace
exits for a specific instruction. There is also some potential slowness
in the GC code that I'd like to take a second look at.
Co-authored-by: Aaron Patterson <tenderlove@ruby-lang.org>
Co-authored-by: Aaron Patterson <tenderlove@ruby-lang.org>
This commit enables Arrays to move between size pools during compaction.
This can occur if the array is mutated such that it would fit in a
different size pool when embedded.
The move is carried out in two stages:
1. The RVALUE is moved to a destination heap during object movement
phase of compaction
2. The array data is re-embedded and the original buffer free'd if
required. This happens during the update references step
At that commit, I fixed a wrong conditional expression that was always
true. However, that seemed to have caused a regression. [Bug #18906]
This change removes the condition to make the code always enabled.
It had been enabled until that commit, albeit unintentionally, and even
if it is enabled it only consumes a tiny bit of memory, so I believe it
is harmless. [Bug #18906]
In order to reliably test compaction we need to be able to move objects
between size pools.
In order for this to happen there must be pages in a size pool into
which we can allocate.
The existing implementation of `double_heap` only doubled the existing
number of pages in the heap, so if a size pool had a low number of pages
(or 0) it's not guaranteed that enough space will be created to move
objects into that size pool.
This commit deprecates the `double_heap` option and replaces it with
`expand_heap` instead.
expand heap will expand each heap by enough pages to hold a number of
slots defined by `GC_HEAP_INIT_SLOTS` or by `heap->total_pags` whichever
is larger.
If both `double_heap` and `expand_heap` are present, a deprecation
warning will be shown for `double_heap` and the `expand_heap` behaviour
will take precedence
Given that this is an API intended for debugging and testing GC
compaction I'm not concerned about the extra memory usage or time taken
to create the pages. However, for completeness:
Running the following `test.rb` and using `time` on my Macbook Pro shows
the following memory usage and time impact:
pp "RSS (kb): #{`ps -o rss #{Process.pid}`.lines.last.to_i}"
GC.verify_compaction_references(double_heap: true, toward: :empty)
pp "RSS (kb): #{`ps -o rss #{Process.pid}`.lines.last.to_i}"
❯ time make run
./miniruby -I./lib -I. -I.ext/common -r./arm64-darwin21-fake ./test.rb
"RSS (kb): 24000"
<internal:gc>:251: warning: double_heap is deprecated and will be removed
"RSS (kb): 25232"
________________________________________________________
Executed in 124.37 millis fish external
usr time 82.22 millis 0.09 millis 82.12 millis
sys time 28.76 millis 2.61 millis 26.15 millis
❯ time make run
./miniruby -I./lib -I. -I.ext/common -r./arm64-darwin21-fake ./test.rb
"RSS (kb): 24000"
"RSS (kb): 49040"
________________________________________________________
Executed in 150.13 millis fish external
usr time 103.32 millis 0.10 millis 103.22 millis
sys time 35.73 millis 2.59 millis 33.14 millis
Nobuyoshi Nakada
David Rodríguez
Jean Boussier
git
Burdette Lamar
Yusuke Endoh
Koichi Sasada
Hiroshi SHIBATA
Aaron Patterson
Eileen M. Uchitelle
Kevin Backhouse
Matt Valentine-House
Takashi Kokubun
dependabot[bot]
st0012