Several improvements to error handling:
- Introduce ImageConfigPullError type, wrapping errors related to
downloading the image configuration blob in schema2. This allows for a
more descriptive error message to be seen by the end user.
- Change some logrus.Debugf calls that display errors to logrus.Errorf.
Add log lines in the push/pull fallback cases to make sure the errors
leading to the fallback are shown.
- Move error-related types and functions which are only used by the
distribution package out of the registry package.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
The daemon uses two similar filename extensions to identify different
kinds of certificates. ".crt" files are interpreted as CA certificates,
and ".cert" files are interprted as client certificates. If a CA
certificate is accidentally given the extension ".cert", it will lead to
the following error message:
Missing key ca.key for certificate ca.cert
To make this slightly less confusing, clarify the error message with a
note that CA certificates should use the extension ".crt".
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
- Use the ones provided by docker/go-connections, they are a drop in replacement.
- Remove pkg/sockets from docker.
- Keep pkg/tlsconfig because libnetwork still needs it and there is a
circular dependency issue.
Signed-off-by: David Calavera <david.calavera@gmail.com>
This is a followup to #18839. That PR relaxed the fallback logic so that
if a manifest doesn't exist on v2, or the user is unauthorized to access
it, we try again with the v1 protocol. A similar special case is needed
for "pull all tags" (docker pull -a). If the v2 registry doesn't
recognize the repository, or doesn't allow the user to access it, we
should fall back to v1 and try to pull all tags from the v1 registry.
Conversely, if the v2 registry does allow us to list the tags, there
should be no fallback, even if there are errors pulling those tags.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
PR #18590 caused compatibility issues with registries such as gcr.io
which support both the v1 and v2 protocols, but do not provide the same
set of images over both protocols. After #18590, pulls from these
registries would never use the v1 protocol, because of the
Docker-Distribution-Api-Version header indicating that v2 was supported.
Fix the problem by making an exception for the case where a manifest is
not found. This should allow fallback to v1 in case that image is
exposed over the v1 protocol but not the v2 protocol.
This avoids the overly aggressive fallback behavior before #18590 which
would allow protocol fallback after almost any error, but restores
interoperability with mixed v1/v2 registry setups.
Fixes#18832
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
ContinueOnError assumes that something of type errcode.Errors contains
at least one error. This is generally true, but might not be true if the
remote registry returns an empty error body or invalid JSON. Add the
bounds check, and in the case where it fails, allow fallbacks to v1.
Fixes#18481
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
This fix avoids overwritting the previous error messages, ensures the client gets the correct error messages and not just the most recent message during the pull request.
For this `var lastErr` replaced with a slice which acts as a temp place holder for the list of returned error messages for every attempt.
The slice is later joined and returned to the caller function after searching for the image with diffirent versions(v2,v1,v0).
Updated the code with check for no space left on device error occurance and prevent the
daemon on falling back to v1,v0.
Incorporated the comments from @calavera, @RichardScothern, @cpuguy83
Signed-off-by: Anil Belur <askb23@gmail.com>
This updates the vendored docker/distribution to the current master
branch.
Note the following changes:
- The manifest package was split into manifest/schema1. Most references
to the manifest package in the engine needed to be updated to use
schema1 instead.
- Validation functions in api/v2 were replaced by the
distribution/reference package. The engine code has been updated to
use the reference package for validation where necessary. A future PR
will change the engine to use the types defined in
distribution/reference more comprehensively.
- The reference package explicitly allows double _ characters in
repository names. registry_test.go was updated for this.
- TestPullFailsWithAlteredManifest was corrupting the manifest JSON, now
that the schema1 package unmarshals the correct payload. The test is
being changed to modify the JSON without affecting its length, which
allows the pull to succeed to the point where digest validation
happens.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
This reverts commit d5cd032a86.
Commit caused issues on systems with case-insensitive filesystems.
Revert for now
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
- Move autogen/dockerversion to version
- Update autogen and "builds" to use this package and a build flag
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Add a daemon flag to control this behaviour. Add a warning message when pulling
an image from a v1 registry. The default order of pull is slightly altered
with this changset.
Previously it was:
https v2, https v1, http v2, http v1
now it is:
https v2, http v2, https v1, http v1
Prevent login to v1 registries by explicitly setting the version before ping to
prevent fallback to v1.
Add unit tests for v2 only mode. Create a mock server that can register
handlers for various endpoints. Assert no v1 endpoints are hit with legacy
registries disabled for the following commands: pull, push, build, run and
login. Assert the opposite when legacy registries are not disabled.
Signed-off-by: Richard Scothern <richard.scothern@gmail.com>
This patch fixes a bug where a user specifies a v1 mirror for
--registry-mirror and pull an image from the Hub.
It used to not fallback because of an unexpected error returned when
trying to JSON marshal nginx output.
We now ensure that any unexpected error falls back to the next endpoint
in the list.
Signed-off-by: Tibor Vass <tibor@docker.com>
Currently login and search do not load per registry certificates.
This is a regression caused by the last refactor since this was recently fixed.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Add a trusted flag to force the cli to resolve a tag into a digest via the notary trust library and pull by digest.
On push the flag the trust flag will indicate the digest and size of a manifest should be signed and push to a notary server.
If a tag is given, the cli will resolve the tag into a digest and pull by digest.
After pulling, if a tag is given the cli makes a request to tag the image.
Use certificate directory for notary requests
Read certificates using same logic used by daemon for registry requests.
Catch JSON syntax errors from Notary client
When an uncaught error occurs in Notary it may show up in Docker as a JSON syntax error, causing a confusing error message to the user.
Provide a generic error when a JSON syntax error occurs.
Catch expiration errors and wrap in additional context.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
* Add godoc documentation where it was missing
* Change identifier names that don't match Go style, such as INDEX_NAME
* Rename RegistryInfo to PingResult, which more accurately describes
what this structure is for. It also has the benefit of making the name
not stutter if used outside the package.
Updates #14756
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
The registry client's TLS configuration used the
default cipher list, including RC4. This change
copies the default cipher list from Golang 1.4 and
removes RC4 from that list. RC4 ciphers are considered
weak and vulnerable to a number of attacks.
Uses the tlsconfig package to define allowed ciphers.
Signed-off-by: Eric Windisch <eric@windisch.us>
It should not print to STDOUT so that it only prints the debugTransport
output if there was an error in one of the registry tests.
Signed-off-by: Tibor Vass <tibor@docker.com>
This patch removes the need for requestFactories and decorators
by implementing http.RoundTripper transports instead.
It refactors some challenging-to-read code.
NewSession now takes an *http.Client that can already have a
custom Transport, it will add its own auth transport by wrapping
it.
The idea is that callers of http.Client should not bother
setting custom headers for every handler but instead it should
be transparent to the callers of a same context.
This patch is needed for future refactorings of registry,
namely refactoring of the v1 client code.
Signed-off-by: Tibor Vass <tibor@docker.com>
Passing RepositoryInfo to ResolveAuthConfig, pullRepository, and pushRepository
Moving --registry-mirror configuration to registry config
Created resolve_repository job
Repo names with 'index.docker.io' or 'docker.io' are now synonymous with omitting an index name.
Adding test for RepositoryInfo
Adding tests for opts.StringSetOpts and registry.ValidateMirror
Fixing search term use of repoInfo
Adding integration tests for registry mirror configuration
Normalizing LookupImage image name to match LocalName parsing rules
Normalizing repository LocalName to avoid multiple references to an official image
Removing errorOut use in tests
Removing TODO comment
gofmt changes
golint comments cleanup. renaming RegistryOptions => registry.Options, and RegistryServiceConfig => registry.ServiceConfig
Splitting out builtins.Registry and registry.NewService calls
Stray whitespace cleanup
Moving integration tests for Mirrors and InsecureRegistries into TestNewIndexInfo unit test
Factoring out ValidateRepositoryName from NewRepositoryInfo
Removing unused IndexServerURL
Allowing json marshaling of ServiceConfig. Exposing ServiceConfig in /info
Switching to CamelCase for json marshaling
PR cleanup; removing 'Is' prefix from boolean members. Removing unneeded json tags.
Removing non-cleanup related fix for 'localhost:[port]' in splitReposName
Merge fixes for gh9735
Fixing integration test
Reapplying #9754
Adding comment on config.IndexConfigs use from isSecureIndex
Remove unused error return value from isSecureIndex
Signed-off-by: Don Kjer <don.kjer@gmail.com>
Adding back comment in isSecureIndex
Signed-off-by: Don Kjer <don.kjer@gmail.com>
Fix issue with restoring the tag store and setting static configuration
from the daemon. i.e. the field on the TagStore struct must be made
internal or the json.Unmarshal in restore will overwrite the insecure
registries to be an empty struct.
Signed-off-by: Michael Crosby <michael@docker.com>
Conflicts:
graph/pull.go
graph/push.go
graph/tags.go
David Calavera
Aaron Lehmann
Tibor Vass
Daniel Nephin
Phil Estes
Michal Gebauer
Alexander Morozov
Anil Belur
Vincent Demeester
Jess Frazelle
Brian Goff
Richard Scothern
John Howard
Tibor Vass
Derek McGowan
Eric Windisch
Antonio Murdaca
Antonio Murdaca
Arnaud Porterie
James Lal
Rik Nijessen
Don Kjer
Matthew Riley
unclejack
Vincent Batts
Tibor Vass
Michael Crosby