This changes the CompactIndexClient to store etags received from the
compact index in separate files rather than relying on the MD5 checksum
of the file as the etag.
Smoothes the upgrade from md5 etags to opaque by generating them when no
etag file exists. This should reduce the initial impact of changing the
caching behavior by reducing cache misses when the MD5 etag is the same.
Eventually, the MD5 behavior should be retired and the etag should be
considered completely opaque with no assumption that MD5 would match.
```
==> memprof.after.txt <==
Total allocated: 1.13 MB (2352 objects)
Total retained: 10.08 kB (78 objects)
==> memprof.before.txt <==
Total allocated: 46.27 MB (38439 objects)
Total retained: 9.94 kB (75 objects)
```
Yes, we were allocating 45MB of arrays in `dependencies_installed?`,
it was accidentally cubic.
https://github.com/rubygems/rubygems/commit/13ab874388
If a platform specific variant would not match the current Ruby, we would still be
considering it compatible with the initial resolution and adding its
platform to the lockfile, but we would later fail to materialize it for
installation due to not really being compatible.
Fix is to only add platforms for variants that are also compatible with
current Ruby and RubyGems versions.
https://github.com/rubygems/rubygems/commit/75d1290843
Gem::RemoteFetcher uses Gem::Request, which adds the RubyGems UA.
Gem::RemoteFetcher is used to download gems, as well as the full index.
We would like the bundler UA to be used whenever bundler is making
requests.
This PR also avoids unsafely mutating the headers hash on the shared
`Gem::RemoteFetcher.fetcher` instance, which could cause corruption or
incorrect headers when making parallel requests. Instead, we create one
remote fetcher per rubygems remote, which is similar to the connection
segregation bundler is already doing
https://github.com/rubygems/rubygems/commit/f0e8dacdec
If a Gemfile duplicates a development dependency also defined in a local
gemspec with a different requirement, the requirement in the local
gemspec will be silently ignored.
This surprised me.
I think we should either:
* Make sure both requirements are considered, like it happens for
runtime dependencies (I added a spec to illustrate the current behavior
here).
* Add a warning that the requirement in the gemspec will be ignored.
I think the former is slightly preferable, but it may cause some
bundle's that previously resolve to no longer resolver.
I went with the latter but the more I think about it, the more this
seems like it should behave like the former.
https://github.com/rubygems/rubygems/commit/ad6843972f
Since we started locking the specific platform in the lockfile, that has
created an annoying situation for users that don't develop on Linux.
They will create a lockfile on their machines, locking their local
platform, for example, darwin. But then that lockfile won't work
automatically when deploying to Heroku for example, because the lockfile
is frozen and the Linux platform is not included.
There's the chance though that resolving against two platforms (Linux +
the local platform) won't succeed while resolving for just the current
platform will. So, instead, we check other platform specific variants
available for the resolution we initially found, and lock those
platforms and specs too if they satisfy the resolution.
This is only done when generating new lockfiles from scratch, existing
lockfiles should keep working as before, and it's only done for "ruby
platforms", i.e., not Java or Windows which have their own complexities,
and so are excluded.
With this change, we expect that MacOS users can bundle locally and
deploy to Heroku without needing to do anything special.
https://github.com/rubygems/rubygems/commit/5f24f06bc5
Daily Bundler CI against ruby-head is failing because ruby-head now
uses bigdecimal 3.1.5, so that gets locked by this spec.
This change should make the test stable until bigdecimal 99.1.5 is
bundled with Ruby :)
https://github.com/rubygems/rubygems/commit/830326041f
Since #6945 the extension dir changed to Gem::BasicSpecification's implementation, we didn't hook that in rubygems_ext.rb. So for universal rubies, we ended up using the universal platform name when installing, but arch replaced platform name when checking. This lead to native extensions can never be correctly installed on universal rubies.
Hook Gem::BasicSpecifications so the behavior is consistent on installing and checking.
https://github.com/rubygems/rubygems/commit/8d699ed096
Comparing file paths as strings may not work well for some reasons,
symlink, relative `__FILE__`, etc.
Some alternatives are possible: comparing with `File.realpath`, or
with `File.identical?`, it should be most robust to escape the target
string contained within this file itself.
Instead, don't check that at all and proceed. If something fails to be
written inside GEM_HOME, we'll eventually fail with a proper permissions
error.
In addition to that, the writable bit in GEM_HOME is not even reliable,
because only the immediate parent is actually checked when writing. For
example,
```
$ mkdir -p foo/bar
$ chmod -w foo
$ touch foo/bar/baz # writes without issue
```
https://github.com/rubygems/rubygems/commit/4bced7ac73
Enable the test commented out in ruby/ruby@d0f5dc9eac.
Extracted from GH-7033, that is for initialization at start up time
and this test is unrelated to it.
Improve error reporting for checksums, raises a new error class.
Solve for multi-source checksum errors.
Add CHECKSUMS to tool/bundler/(dev|standard|rubocop)26_gems.rb
https://github.com/rubygems/rubygems/commit/26ceee0e76
Co-authored-by: Samuel Giddins <segiddins@segiddins.me>
This gets the specs passing, and handles the fact that we expect
checkums to be pinned only to a particular source
This also avoids reading in .gem files during lockfile generation,
instead allowing us to query the source for each resolved gem to grab
the checksum
Finally, this opens up a route to having user-stored checksum databases,
similar to how other package managers do this!
Add checksums to dev lockfiles
Handle full name conflicts from different original_platforms when adding checksums to store from compact index
Specs passing on Bundler 3
https://github.com/rubygems/rubygems/commit/86c7084e1c
1. Use the checksum provided by the server if provided: provides security
knowing if the gem you downloaded matches the gem on the server
2. Calculate the checksum from the gem on disk: provides security knowing
if the gem has changed between installs
3. In some cases, neither is possible in which case we don't put anything
in the checksum and we maintain functionality as it is today
Add the checksums to specs in the index if we already have them
Prior to checksums, we didn't lose any information when overwriting specs
in the index with stubs. But now when we overwrite EndpointSpecifications
or RemoteSpecifications with more generic specs, we could lose checksum
info. This manually sets checksum info so we keep it in the index.
https://github.com/rubygems/rubygems/commit/de00a4f153
Benoit Daloze
Josef Šimánek
David Rodríguez
Samuel Giddins
Eric Mueller
Bo Anderson
Nobuyoshi Nakada
Jean Boussier
Peter Zhu
Han Young
Hiroshi SHIBATA
Yusuke Endoh
Cody Cutrer
Lars Kanis
Martin Emde
Martin Emde
Mercedes Bernard