Fallback errors are not an error, but an
informational message.
This changes those errors to be logged
as "Info" instead of "Error".
After this patch, debug logs look like this;
DEBU[0050] Calling GET /_ping
DEBU[0050] Calling POST /v1.27/images/create?fromImage=localhost%3A5000%2Ffoo&tag=latest
DEBU[0050] Trying to pull localhost:5000/foo from https://localhost:5000 v2
WARN[0050] Error getting v2 registry: Get https://localhost:5000/v2/: http: server gave HTTP response to HTTPS client
INFO[0050] Attempting next endpoint for pull after error: Get https://localhost:5000/v2/: http: server gave HTTP response to HTTPS client
DEBU[0050] Trying to pull localhost:5000/foo from http://localhost:5000 v2
INFO[0050] Attempting next endpoint for pull after error: manifest unknown: manifest unknown
DEBU[0050] Trying to pull localhost:5000/foo from https://localhost:5000 v1
DEBU[0050] attempting v1 ping for registry endpoint https://localhost:5000/v1/
DEBU[0050] Fallback from error: Get https://localhost:5000/v1/_ping: http: server gave HTTP response to HTTPS client
INFO[0050] Attempting next endpoint for pull after error: Get https://localhost:5000/v1/_ping: http: server gave HTTP response to HTTPS client
DEBU[0050] Trying to pull localhost:5000/foo from http://localhost:5000 v1
DEBU[0050] [registry] Calling GET http://localhost:5000/v1/repositories/foo/images
ERRO[0050] Not continuing with pull after error: Error: image foo:latest not found
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Remove forked reference package. Use normalized named values
everywhere and familiar functions to convert back to familiar
strings for UX and storage compatibility.
Enforce that the source repository in the distribution metadata
is always a normalized string, ignore invalid values which are not.
Update distribution tests to use normalized values.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Use resolving to repo info as the split point between the
legitimate reference package and forked reference package.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
As noted by #30083, the new strict checking of mediatypes misses some
cases where earlier bugs caused nonstandard mediatypes to be stored in
manifests. Two of the known cases are text/html and application/json,
which were returned by certain registries and stored by earlier versions
of Docker. Add special cases for text/html and application/json.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
This patch fixed below 4 types of code line
1. Remove unnecessary variable assignment
2. Use variables declaration instead of explicit initial zero value
3. Change variable name to underbar when variable not used
4. Add erro check and return for ignored error
Signed-off-by: Daehyeok Mun <daehyeok@gmail.com>
When storeLayer.Parent returns the parent layer, it needs to use the same logic as Get where it wraps in a describablyStoreLayer if the layer is describable. Otherwise, on Windows, this can result in pushing the foreign layers, which is not supposed to be allowed.
This fixes https://github.com/docker/docker/issues/30080.
Signed-off-by: Stefan J. Wernli <swernli@microsoft.com>
The `digest` data type, used throughout docker for image verification
and identity, has been broken out into `opencontainers/go-digest`. This
PR updates the dependencies and moves uses over to the new type.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
- Make it possible to define a shorter waiting time of httputils
- Make a small hack to reduce the waiting time on distribution/xfer
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Signed-off-by: Ke Li <kel@splunk.com>
Add missing changes
Signed-off-by: Ke Li <kel@splunk.com>
User errors.New to create error
Signed-off-by: Ke Li <kel@splunk.com>
Move plugins to shared distribution stack with images.
Create immutable plugin config that matches schema2 requirements.
Ensure data being pushed is same as pulled/created.
Store distribution artifacts in a blobstore.
Run init layer setup for every plugin start.
Fix breakouts from unsafe file accesses.
Add support for `docker plugin install --alias`
Uses normalized references for default names to avoid collisions when using default hosts/tags.
Some refactoring of the plugin manager to support the change, like removing the singleton manager and adding manager config struct.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Signed-off-by: liwenqi <vikilwq@zju.edu.cn>
update some files in the folder of distribution/xfer
Signed-off-by: liwenqi <vikilwq@zju.edu.cn>
correct again
Signed-off-by: liwenqi <vikilwq@zju.edu.cn>
Move configurations into a single file.
Abstract download manager in pull config.
Add supports for schema2 only and schema2 type checking.
Add interface for providing push layers.
Abstract image store to generically handle configurations.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
When building a manifest during a push operation, all layers must have
an associated descriptor. If a layer is missing a descriptor, that leads
to a panic.
A break inside a switch in layerAlreadyExists meant to break from the
loop surrounding the switch, but instead breaks from the switch. This
causes the loop to continue, and can overwrite the descriptor with an
empty one, leading to the panic.
Also, fix layerAlreadyExists not to abort the push when a speculative
stat on a candidate layer digest fails with an error. This could happen
in situations like a potential cross-repository mount where the user
does not have permission to access the source repository.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Modified newV1DependencyImage and generateDependencyImages not to return
errors instead of always return nil.
Signed-off-by: Daehyeok Mun <daehyeok@gmail.com>
Move some of the optional parameters of CreateRWLayer() in a struct
called CreateRWLayerOpts. This will make it easy to add more options
arguments without having to change signature of CreateRWLayer().
Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
Translate pull errors to provide a more consistent and user friendly
error message.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Remove the following comment in pullV2Tag:
// NOTE: not using TagService.Get, since it uses HEAD requests
// against the manifests endpoint, which are not supported by
// all registry versions.
This is actually not an issue, because TagService.Get does a fallback to
GET if HEAD fails. It has done this ever since TagService was added to
the distribution API, so this comment was probably based on an early
version of TagService before it was merged, or was always a
misunderstanding.
However, we continue to use ManifestService.Get instead because it
saves a round trip. The manifest can be retrieved directly instead of
resolving the digest first.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Allow built images to be squash to scratch.
Squashing does not destroy any images or layers, and preserves the
build cache.
Introduce a new CLI argument --squash to docker build
Introduce a new param to the build API endpoint `squash`
Once the build is complete, docker creates a new image loading the diffs
from each layer into a single new layer and references all the parent's
layers.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Classify blobs into three categories based on size.
Use a very limited number of mount attempts and no existence check for
small blobs. Use more attempts for bigger blobs.
Also remember blob associations during layer existence check.
Blob digests are now checked in the target repository from newest to
latest. If the blob exists and the metadata entry does not, it will be
created. If the blob is not found, the metadata entry will be removed.
Signed-off-by: Michal Minář <miminar@redhat.com>
Don't fallback back immediately to blob upload if the cross-repo mount
fails and layer upload is initiated by registry.
Instead cancel the upload and re-try cross-repo mount from different
source repository before doing full re-upload.
Signed-off-by: Michal Minář <miminar@redhat.com>
to avoid unnecessary blob re-uploads.
Cross repository mount from particular repo will most probably fail if
the user pushing to the registry is not the same as the one who pulled
or pushed to the source repo.
This PR attempts first to cross-repo mount from the source repositories
associated with the pusher's auth config. Then it falls back to other
repositories sorted from the most similar to the target repo to the
least.
It also prevents metadata deletion in cases where cross-repo mount fails
and the auth config hashes differ.
Signed-off-by: Michal Minář <miminar@redhat.com>
Sebastiaan van Stijn
Vincent Demeester
Derek McGowan
Darren Stahl
Matt Moore
Aaron Lehmann
Sebastiaan van Stijn
Aaron.L.Xu
Daehyeok Mun
Stefan J. Wernli
Stephen J Day
Brian Goff
yuexiao-wang
Ke Li
Tonis Tiigi
liwenqi
Anusha Ragunathan
Victor Vieux
Victor Vieux
Alexander Morozov
Vivek Goyal
Kenfe-Mickael Laventure
Michal Minář
John Starks