Add tests for pre, move more of the setup into a helper method, and
restructure tests.
There seem to be five considerations for these tests (level, pre, strict,
locked, and whether the current version is a prerelease version, though
the last one overlaps with pre and didn't seem to behave how I expected
under test). Rather than write out the 16 (/32 if the last consideration
is real) combinations, I wrote most with independent tests for each
value. The existing combined tests were maintained (level vs strict)
because these seem the most interrelated.
https://github.com/rubygems/rubygems/commit/74c23a91b2
This is useful for passing directory file descriptors over UNIX
sockets or to child processes to avoid TOCTOU vulnerabilities.
The implementation follows the Dir.chdir code.
This will raise NotImplementedError on platforms not supporting
both fchdir and dirfd.
Implements [Feature #19347]
The documentation states it returns a copy of self with nil value
entries removed. However, the previous behavior was creating a
plain new hash with non-nil values copied into it. This change
aligns the behavior with the documentation.
Fixes [Bug #19113]
This was already copied for non-empty hashes. As Hash.ruby2_keywords_hash
copies default values, it should also copy the compare_by_identity flag.
Partially Fixes [Bug #19113]
It wasn't copied for empty hashes, and Hash.[] doesn't copy the
default value, so copying the compare_by_identity flag does not
make sense.
Partially Fixes [Bug #19113]
I've never seen this error in real life, and if it was happening, I
think it's either some server side issue that would need to be fixed or
some transient issue. We should move away from the full index, since
it's slow, so let's stop recommending it.
The debug message suggests retrying using `--full-index`, but the retry
is happening automatically. Just log that we are falling back to the
full index, like we do with other errors.
Since Bundler 2.4, we will try to checkout any branch specified in the
Gemfile, while until Bundler 2.3 we would directly checkout the locked
revision.
This should not make any difference in most situations, but in some edge
cases, like if the branch specified in the `Gemfile` has been renamed,
but the locked revision still exist, it causes an error now while before
it would update the lockfile without issues.
I debated which behavior was best, since I was not sure. But my
conclusion is that if the situation does not require expiring the
lockfile source in favor of the Gemfile source, we should use the locked
revision directly and proceed happily. So I restored Bundler 2.3
behavior.
I think this is consistent with how yanked gems are handled, for example.
Of course, if explicitly updating the git source itself, or all gems, we
will still get any errors like missing branches related to the git source.
This was working fine for direct dependencies using
`force_ruby_platform` explicitly through Gemfile, but not for indirect
dependencies. In general, indirect dependencies do not have this
property set, but in truffleruby this is different and the default value
is to have it set.
This should be a very rare edge case, however, it does happen when using
a .dev version of Bundler because in that case, that's the only version
that the resolver considers, and it should not be ignored.
We could've special cased this specifically for Bundler, but I think it
does make sense for every gem.
Currently, the --no-install option to `bundle package` is totally
ignored for git sources. This can have very strange effects if you have:
- a git-sourced gem,
- with native extensions,
- whose extconf.rb script depends on another gem,
- which is installed from Rubygems in the gemfile.
In that circumstance, `bundle package --no-install --all` will download
the Rubygems dependencies to `vendor/cache` but NOT install them. It
will also check out the git gems to `vendor/cache` (good), and attempt
to build their native extensions (bad!).
The native extension build will fail because the extconf.rb script crashes,
since the dependency it needs is missing.
I implemented a fix for this in `source/git.rb`, since this is analogous
to what's happening in `source/rubygems.rb`. I do admit though the whole
thing is a little strange though - an "install" method that.... proceeds
to look at a global flag to not install anything.
Add test to confirm cache respects the --no-install flag
https://github.com/rubygems/rubygems/commit/5a77d1c397
Co-authored-by: KJ Tsanaktsidis <kj@kjtsanaktsidis.id.au>
If the original `BUNDLE_GEMFILE` is different from the default, then the
suggestion wouldn't work as is.
Before:
```
$ util/rubocop
Could not find rubocop-1.30.1 in locally installed gems
Run `bundle install` to install missing gems.
$ rubygems git:(better-cmd-suggestion) ✗ bundle install
Could not locate Gemfile
```
After:
```
$ util/rubocop
Could not find rubocop-1.30.1 in locally installed gems
Run `bundle install --gemfile /path/to/rubygems/bundler/tool/bundler/lint_gems.rb` to install missing gems.
$ bundle install --gemfile /path/to/rubygems/bundler/tool/bundler/lint_gems.rb
Fetching gem metadata from https://rubygems.org/.........
Using ast 2.4.2
Using bundler 2.4.7
Using parser 3.1.2.0
Using rainbow 3.1.1
Using parallel 1.22.1
Using regexp_parser 2.5.0
Using rubocop-ast 1.18.0
Using rexml 3.2.5
Using ruby-progressbar 1.11.0
Using unicode-display_width 2.1.0
Fetching rubocop 1.30.1
Installing rubocop 1.30.1
Using rubocop-performance 1.14.2
Bundle complete! 2 Gemfile dependencies, 12 gems now installed.
Use `bundle info [gemname]` to see where a bundled gem is installed.
$ util/rubocop
Inspecting 345 files
.........................................................................................................................................................................................................................................................................................................................................................
345 files inspected, no offenses detected
```
https://github.com/rubygems/rubygems/commit/bf1320d805
Following up on https://github.com/rubygems/rubygems/pull/6355, which
turned a crash into a nicer error message, this commit auto-heals the
corrupt lockfile instead.
In this particular case (a corrupt Gemfile.lock with missing
dependencies) the LazySpecification will not have accurate dependency
information, we have to materialize the SpecSet to determine there are
missing dependencies. We've already got a way to handle this, via
`SpecSet#incomplete_specs`, but it wasn't quite working for this case
because we'd get to `@incomplete_specs += lookup[name]` and
`lookup[name]` would be empty for the dependency.
With this commit we catch it a bit earlier, marking the parent spec
containing the missing dependency as incomplete.
https://github.com/rubygems/rubygems/commit/486ecb8f20
[Bug #19439]
The instance variables were restore on the Regexp source,
not the regexp itself.
Unfortunately we have a bit of a chicken and egg problem.
The source holds the encoding, and the encoding need to be set on
the source to be able to instantiate the Regexp.
So the instance variables have to be read on the `source`.
To correct this we transfert the instance variables after
instantiating the Regexp.
The only way to avoid this would be to read the instance variable
twice and rewind.
* Replaces the wording of "is forbidden" with "cannot be used"
* Fixes the method signature of VersionRange::Empty#eql?
https://github.com/rubygems/rubygems/commit/8c6b3f130b
Co-authored-by: Daniel Colson <danieljamescolson@gmail.com>
This test is no longer passing:
```
1)
BigDecimal#remainder returns NaN if Infinity is involved FAILED
Expected Infinity.nan?
to be truthy but was false
/home/runner/work/ruby/ruby/src/spec/ruby/library/bigdecimal/remainder_spec.rb:58:in `block (2 levels) in <top (required)>'
/home/runner/work/ruby/ruby/src/spec/ruby/library/bigdecimal/remainder_spec.rb:4:in `<top (required)>'
```
https://github.com/ruby/bigdecimal/pull/243
I did a bad thing (script that edits the Gemfile.lock directly) and
ended up with a Gemfile.lock that was completely missing some indirect
dependencies. While this is my fault and an error is reasonable, I
noticed that the error got progressively less friendly in recent
versions of bundler.
Something similar came up in https://github.com/rubygems/rubygems/issues/6210,
and this commit would have helped with that case as well
(although we've already handled this a different way with #6219).
Details:
---
Back on Bundler 2.2.23, a corrupt lockfile like this would cause a helpful error:
```
Unable to find a spec satisfying minitest (>= 5.1) in the set. Perhaps the lockfile is corrupted?
```
Bundler 2.3.26 gave a helpful warning:
```
Warning:
Your lockfile was created by an old Bundler that left some things out.
Because of the missing DEPENDENCIES, we can only install gems one at a time,
instead of installing 16 at a time.
You can fix this by adding the missing gems to your Gemfile, running bundle
install, and then removing the gems from your Gemfile.
The missing gems are:
* minitest depended upon by activesupport
```
But then continued on and crashed while trying to report the unmet
dependency:
```
--- ERROR REPORT TEMPLATE -------------------------------------------------------
NoMethodError: undefined method `full_name' for nil:NilClass
lib/bundler/installer/parallel_installer.rb:127:in `block (2 levels) in check_for_unmet_dependencies'
...
```
Bundler 2.4.0 and up crash as above when jobs=1, but crash
even harder when run in parallel:
```
--- ERROR REPORT TEMPLATE -------------------------------------------------------
fatal: No live threads left. Deadlock?
3 threads, 3 sleeps current:0x00007fa6b6704660 main thread:0x00007fa6b6704660
* #<Thread:0x000000010833b130 sleep_forever>
rb_thread_t:0x00007fa6b6704660 native:0x0000000108985600 int:0
* #<Thread:0x0000000108dea630@Parallel Installer Worker #0 tmp/1/gems/system/gems/bundler-2.5.0.dev/lib/bundler/worker.rb:90 sleep_forever>
rb_thread_t:0x00007fa6b67f67c0 native:0x0000700009a62000 int:0
* #<Thread:0x0000000108dea4a0@Parallel Installer Worker #1 tmp/1/gems/system/gems/bundler-2.5.0.dev/lib/bundler/worker.rb:90 sleep_forever>
rb_thread_t:0x00007fa6b67f63c0 native:0x0000700009c65000 int:0
<internal:thread_sync>:18:in `pop'
tmp/1/gems/system/gems/bundler-2.5.0.dev/lib/bundler/worker.rb:42:in `deq'
...
```
Changes
---
This commit fixes the confusing thread deadlock crash by detecting if
dependencies are missing such that we'll never be able to enqueue. When
that happens we treat it as a failure so the install can finish.
That gets us back to the `NoMethodError`, which this commit fixes by
using a different warning in the case where no spec is found.
https://github.com/rubygems/rubygems/commit/d73001a21d
`trying to manually editing` doesn't seem quite grammatically
correct. We could change it to `trying to manually edit` (is that a
split infinitive?), but I don't think `trying to` adds much here so
I've removed it instead so `editing` is the verb.
For the list of dependencies, the wording before this commit seemed to
reverse the dependency. "B, depended on A" sounds like B depends on A
(or did in the past but doesn't anymore?), but that's not correct. I
think there's a missing word: "B, depended on by A", but I find "B,
dependency of A" a bit nicer.
https://github.com/rubygems/rubygems/commit/49a31257e3
[Bug #19415]
If multiple threads attemps to load the same file concurrently
it's not a circular dependency issue.
So we check that the existing ThreadShield is owner by the current
fiber before warning about circular dependencies.
[Bug #19415]
If multiple threads attemps to load the same file concurrently
it's not a circular dependency issue.
So we check that the existing ThreadShield is owner by the current
fiber before warning about circular dependencies.
The previous code loads bundler's gemspec which does not include the generated
gemspec file, and thus the test was passing where it should indeed fail.
With this change, the test properly fails now.
https://github.com/rubygems/rubygems/commit/2098ea0d75
Given an existing application using native gems (e.g., nokogiri)
And a lockfile generated with a stable ruby version
When we test the application against ruby-head and `bundle install`
Then bundler should fall back to the generic ruby platform gem
Note that this test has been passing since 45931ac9
https://github.com/rubygems/rubygems/commit/0ecc6de378
Prior to this commit `bundle binstubs --standalone --all` would output a
warning about not being able to generate a standalone binstub for
bundler.
This warning predates the `--all` option, and I don't think it makes
sense in this context. The warning makes good sense when explicitly
trying to generate a bundler standalone binstub with `bundle binstubs
bundler --standalone`, since that command won't do what the user might
have expected. But `--all` is not specifically asking for bundler, and
having it report each time that the bundler binstubs could not be
generated does not seem particularly helpful. The only way to make that
warning go away would be to stop using `--standalone --all`.
This commit skips the warning when running with the `--all` option.
https://github.com/rubygems/rubygems/commit/e6a72e19eb
This commit moves the classpath (and tmp_classpath) from instance
variables to the rb_classext_t. This improves performance as we no
longer need to set an instance variable when assigning a classpath to
a class.
I benchmarked with the following script:
```ruby
name = :MyClass
puts(Benchmark.measure do
10_000_000.times do |i|
Object.const_set(name, Class.new)
Object.send(:remove_const, name)
end
end)
```
Before this patch:
```
5.440119 0.025264 5.465383 ( 5.467105)
```
After this patch:
```
4.889646 0.028325 4.917971 ( 4.942678)
```
Running the file with shebang has a few issues.
* shebang is an OS dependent feature. Many modern UNIX-like OSes
support it, but not all, e.g., Windows.
* `env` command may not be in `/usr/bin`.
* "ruby" command may not be "ruby", when `--program-suffix` or other
configuration option is used.
https://github.com/ruby/syntax_suggest/commit/2edf241055
This was already the behavior when a single `'external:internal'`
encoding specifier string was passed. This makes the behavior
consistent for the case where separate external and internal
encoding specifiers are provided.
While here, fix the IO#set_encoding method documentation to
state that either the first or second argument can be a string
with an encoding name, and describe the behavior when the
external encoding is binary.
Fixes [Bug #18899]
HOMEPATH is set to "\WINDOWS\system32" when running per "runas" session.
This directory is not writable by ordinary users, leading to errors with many ruby tools.
Also config files in the home directory are not recognized.
Still keeping HOME at first which is not used by native Windows,
but by ruby specs and by MSYS2 environment.
Dir.home returns an UTF-8 string since ruby-3.0, but the actual
encoding of the bytes was CP_ACP or CP_OEMCP.
That led to invalid bytes when calling Dir.home with an unicode
username.
Previously, only certain values of the 3rd argument triggered a
deprecation warning.
First step for fix for bug #18797. Support for the 3rd argument
will be removed after the release of Ruby 3.2.
Fix minor fallout discovered by the tests.
Co-authored-by: Nobuyoshi Nakada <nobu@ruby-lang.org>
With `GemVersionPromoter#sort_versions` being so simple, we no longer
need to reach into the class's internals to make private methods public
in order to effectively test. We can just allow both cases to go through
the main method.
https://github.com/rubygems/rubygems/commit/6cbe891003
We have a check for a corrupt lockfile right before installing. However,
the check accounted for locked specs not satisfying locked dependencies,
but not for locked specs missing for some locked dependencies.
Instead of fixing this check, I decided to remove it in favor of
automatically detecting the situation and re-resolve to automatically
fix the lockfile rather than printing a warning but leave the problem
there.
https://github.com/rubygems/rubygems/commit/4a7a584252
bundle lock --update can do everything that bundle update can do, but
it doesn't actually install gems. This is especially useful for
generating a lockfile on a machine that doesn't have the libraries
available to be able to build native extensions.
But, there was no parallel for bundle update --bundler. So let's add
one.
https://github.com/rubygems/rubygems/commit/7fc00bd2a5
Cases like this:
```ruby
obj = Object.new
loop do
obj.instance_variable_set(:@foo, 1)
obj.remove_instance_variable(:@foo)
end
```
can cause us to use many more shapes than we want (and even run out).
This commit changes the code such that when an instance variable is
removed, we'll walk up the shape tree, find the shape, then rebuild any
child nodes that happened to be below the "targetted for removal" IV.
This also requires moving any instance variables so that indexes derived
from the shape tree will work correctly.
Co-Authored-By: Jemma Issroff <jemmaissroff@gmail.com>
Co-authored-by: John Hawthorn <jhawthorn@github.com>
It can be configured by `--with-os-version-style=TYPE` option, and
just copies from theApple's installation as the default. We don't
know why it is major only.
UnboundMethod records caller's class, like `D` or `E` on the
following case:
```ruby
class C
def foo = :foo
end
class D < C
end
class E < C
end
d = D.instance_method(:foo)
e = E.instance_method(:foo)
```
But `d` and `e` only refers `C#foo` so that UnboundMethod doesn't
record `D` or `E`. This behavior changes the following methods:
* `UnboundMethod#inspect` (doesn't show caller's class)
* `UnboundMethod#==` (`d == e` for example)
fix https://bugs.ruby-lang.org/issues/18798
Due to a problem with ripper we do not recognize `break` as invalid code. It's confusing that "Syntax OK" is output in that case.
When there's no syntax error, the algorithm should not say anything. The exception is in the CLI and that's for compatibility with `ruby -wc`
```
$ cat /tmp/break.rb
break
⛄️ 3.1.2 🚀 /Users/rschneeman/Documents/projects/syntax_suggest (schneems/no-syntax-not-okay-break)
$ ruby -wc /tmp/break.rb
Syntax OK
```
> Note that this is invalid, running this code will raise a Syntax error.
```
$ exe/syntax_suggest /tmp/break.rb
Syntax OK
```
Close https://github.com/ruby/syntax_suggest/pull/157https://github.com/ruby/syntax_suggest/commit/d7bd8f03a2
It's questionable whether we want to allow rstrip to work for strings
where the broken coderange occurs before the trailing whitespace and
not after, but this approach is probably simpler, and I don't think
users should expect string operations like rstrip to work on broken
strings.
In some cases, this changes rstrip to raise
Encoding::CompatibilityError instead of ArgumentError. However, as
the problem is related to an encoding issue in the receiver, and due
not due to an issue with an argument, I think
Encoding::CompatibilityError is the more appropriate error.
Fixes [Bug #18931]
Ruby interpreter loads some special gems at startup: did_you_mean,
error_highlight, and syntax_suggest. These gems are loaded before
`bundler/setup` is loaded by `RUBYOPT=-rbundler/setup`.
So, the versions of the gems are not controllable by Gemfile.
This change will `require "bundler/setup"` in rubygems.rb (i.e., before
the special gems are loaded). Now `bundle exec` sets an environment
variable `BUNDLER_SETUP`, and rubygems requires the variable if defined.
See also: https://bugs.ruby-lang.org/issues/19089https://github.com/rubygems/rubygems/commit/963cb65a2d
Not only powerpc64le, also s390x and arm32 seem failing too. These
failures are probably caused by filesystem settings on Travis, but
unrelated to CPUs.
`Complex.polar` accepts Complex values as arguments for the polar form as long
as the value of the complex has no imaginary part (ie it is 'real'). In
`f_complex_polar` this is handled by extracting the real part of the arguments.
However in the case `polar` is called with only a single argument, the absolute
value (abs), then the Complex is created without applying a check on the type
of abs, meaning it is possible to create a Complex where the real part is itself
an instance of a Complex. This change removes the short circuit for the single
argument case meaning the real part extraction is performed correctly
(by f_complex_polar).
Also adds an example to `spec/ruby/core/complex/polar_spec.rb` to check that
the real part of a complex argument is correctly extracted and used in the
resulting Complex real and imaginary parts.
Implements [Feature #12084]
Returns the object for which the receiver is the singleton class, or
raises TypeError if the receiver is not a singleton class.
[Feature #18982]
Instead of introducing an `exception: false` argument to have `non_block`
return nil rather than raise, we can clearly document that a timeout of 0
immediately returns.
The code is refactored a bit to avoid doing a time calculation in
such case.
We sometimes check assertions on lockfile contents, which involves
comparing a reasonably long string. Sometimes RSpec is not able to show
the part of the string that's actually different, making it hard to
figure out the issue.
Configuring this setting should fix the issue in most cases.
https://github.com/rubygems/rubygems/commit/5ad8ee499e
This allows the file to be created without copying permissions
from Bundler's installation source. The previous behaviour was
noticed after installing Ruby through brew, and using bundle
init, which yielded a read-only Gemfile.
https://github.com/rubygems/rubygems/commit/839a06851d
Object Shapes is used for accessing instance variables and representing the
"frozenness" of objects. Object instances have a "shape" and the shape
represents some attributes of the object (currently which instance variables are
set and the "frozenness"). Shapes form a tree data structure, and when a new
instance variable is set on an object, that object "transitions" to a new shape
in the shape tree. Each shape has an ID that is used for caching. The shape
structure is independent of class, so objects of different types can have the
same shape.
For example:
```ruby
class Foo
def initialize
# Starts with shape id 0
@a = 1 # transitions to shape id 1
@b = 1 # transitions to shape id 2
end
end
class Bar
def initialize
# Starts with shape id 0
@a = 1 # transitions to shape id 1
@b = 1 # transitions to shape id 2
end
end
foo = Foo.new # `foo` has shape id 2
bar = Bar.new # `bar` has shape id 2
```
Both `foo` and `bar` instances have the same shape because they both set
instance variables of the same name in the same order.
This technique can help to improve inline cache hits as well as generate more
efficient machine code in JIT compilers.
This commit also adds some methods for debugging shapes on objects. See
`RubyVM::Shape` for more details.
For more context on Object Shapes, see [Feature: #18776]
Co-Authored-By: Aaron Patterson <tenderlove@ruby-lang.org>
Co-Authored-By: Eileen M. Uchitelle <eileencodes@gmail.com>
Co-Authored-By: John Hawthorn <john@hawthorn.email>
Object Shapes is used for accessing instance variables and representing the
"frozenness" of objects. Object instances have a "shape" and the shape
represents some attributes of the object (currently which instance variables are
set and the "frozenness"). Shapes form a tree data structure, and when a new
instance variable is set on an object, that object "transitions" to a new shape
in the shape tree. Each shape has an ID that is used for caching. The shape
structure is independent of class, so objects of different types can have the
same shape.
For example:
```ruby
class Foo
def initialize
# Starts with shape id 0
@a = 1 # transitions to shape id 1
@b = 1 # transitions to shape id 2
end
end
class Bar
def initialize
# Starts with shape id 0
@a = 1 # transitions to shape id 1
@b = 1 # transitions to shape id 2
end
end
foo = Foo.new # `foo` has shape id 2
bar = Bar.new # `bar` has shape id 2
```
Both `foo` and `bar` instances have the same shape because they both set
instance variables of the same name in the same order.
This technique can help to improve inline cache hits as well as generate more
efficient machine code in JIT compilers.
This commit also adds some methods for debugging shapes on objects. See
`RubyVM::Shape` for more details.
For more context on Object Shapes, see [Feature: #18776]
Co-Authored-By: Aaron Patterson <tenderlove@ruby-lang.org>
Co-Authored-By: Eileen M. Uchitelle <eileencodes@gmail.com>
Co-Authored-By: John Hawthorn <john@hawthorn.email>
After recent musl support was added, Bundler started hanging in musl
platforms. I identified the issue where valid candidates were being
filtered out because their platform was specified as a string, and thus
`Gem::Platform.match_spec?` which under the hood ends up calling
`Gem::Platform#===` would return `nil`, because it does not support
comparing platforms to strings.
In particular, `Bundler::EndpointSpecification`'s platform coming from
the API was not instantiated as a `Gem::Platform`, hence the issue.
Also, this spec surfaced another issue where a bug corrected in
`Gem::Platform#match_platforms` had not been yet backported to Bundler.
So this commit also backports that to get the spec green across RubyGems
versions.
Finally, the fix in `Bundler::EndpointSpecification` made a realworld
spec start failing. This spec was faking out `rails-4.2.7.1` requirement
on Bundler in the `Gemfile.lock` file to be `>= 1.17, < 3` when the real
requirement is `>= 1.17, < 2`. Due to the bug in
`Bundler::EndpointSpecification`, the real requirement provided by the
compact index API (recorded with VCR) was being ignored, and the
`Gemfile.lock` fake requirement was being used, which made the spec
pass. This is all expected, and to fix the issue I changed the spec to
be really realworld and don't fake any Bundler requirements.
https://github.com/rubygems/rubygems/commit/faf4ef46bc
Michael Siegfried
Jeremy Evans
David Rodríguez
Nobuyoshi Nakada
Hiroshi SHIBATA
Jean Boussier
Ian Ker-Seymer
Samuel Williams
Julie Haehn
Takashi Kokubun
johnnyshields
Tony Hsu
Ellen Marie Dash
Daniel Colson
Benoit Daloze
Mercedes Bernard
John Hawthorn
Yusuke Endoh
Jean byroot Boussier
Akira Matsuda
Mike Dalessio
Tim Bates
Peter Zhu
git
yoka
Kenta Murata
Lars Kanis
Mark Burns
Josef Šimánek
Michael Siegfried
Alyssa Ross
Aaron Patterson
Aaron Patterson
Martin Dürst
Koichi Sasada
schneems
MSP-Greg
Bo Anderson
Stephen Ierodiaconou
Ufuk Kayserilioglu
Jemma Issroff
Jean Boussier
Victor Gama
Victor Shepelev
Vít Ondruch
Chris Salzberg
Yuta Saito