* Add implementation for GetNetworkConfiguration
Previously the NMAgent client did not have support for the
GetNetworkConfiguration API call. This adds it and appropriate coverage.
* Refactor retry loops to use shared function
The cancellable retry was common enough that it made sense to extract it
to a separate BackoffRetry function in internal. This made its
functionality easier to test and reduced the number of tests necessary
for each new endpoint
* Slight re-org
The client had enough extra stuff in it that it made sense to start
separating things into different files
* Add retries for Unauthorized responses
In the original logic, unauthorized responses are treated as temporary
for a specific period of time. This makes the nmagent.Error consider
Unauthorized responses as temporary for a configurable time. Given that
BackoffRetry cares only whether or not an error is temporary, this
naturally causes them to be retried.
Additional coverage was added for these scenarios as well.
* Add a WireserverTransport
This deals with all the quirks of proxying requests to NMAgent through
Wireserver, without spreading that concern through the NMAgent client
itself.
* Reorganize the nmagent internal package
The wireserver transport became big enough to warrant its own file
* Use WireserverTransport
This required some changes to the test so that the WireserverTransport
middleware could take effect always
* Add PutNetworkContainer method to NMAgent client
This is another API that must be implemented
* Switch NMAgent client port to uint16
Ports are uint16s by definition.
* Add missing body close and context propagation
* Add DeleteNetworkContainer endpoint
* Move internal imports to another section
It's a bit clearer when internal imports are isolated into one section,
standard library imports in another, then finally external imports in
another section.
* Additional Validation / Retry improvements
This is a bit of a rollup commit, including some additional validation
logic for some nmagent requests and also some improvements to the
internal retry logic. The retry logic is now a struct, and the client
depends only on an interface for retrying. This is to accommodate the
existing retry package (which was unknown).
The internal Retrier was enhanced to add a configurable Cooldown
function with strategies for Fixed backoff, Exponential, and a Max
limitation.
* Move GetNetworkConfig request params to a struct
This follows the pattern established in other API calls. It moves
validation to the request itself and also leaves the responsibility for
constructing paths to the request.
* Add Validation and Path to put request
To be consistent with the other request types, this adds Validate and
Path methods to the PutNetworkContainerRequest
* Introduce Request and Option
Enough was common among building requests and validating them that it
made sense to formalize it as an interface of its own. This allowed
centralizing the construction of HTTP requests in the nmagent.Client. As
such, it made adding TLS disablement trivial. Since there is some
optional behavior that can be configured with the nmagent.Client,
nmagent.Option has been introduced to handle this in a clean manner.
* Add additional error documentation
The NMAgent documentation contains some additional documentation as to
the meaning of particular HTTP Status codes. Since we have this
information, it makes sense to enhance the nmagent.Error so it can
explain what the problem is.
* Fix issue with cooldown remembering state
Previously, cooldown functions were able to retain state across
invocations of the "Do" method of the retrier. This adds an additional
layer of functions to allow the Retrier to purge the accumulated state
* Move Validation to reflection-based helper
The validation logic for each struct was repetitive and it didn't help
answer the common question "what fields are required by this request
struct." This adds a "validate" struct tag that can be used to annotate
fields within the request struct and mark them as required.
It's still possible to do arbitrary validation within the Validate
method of each request, but the common things like "is this field a zero
value?" are abstracted into the internal helper. This also serves as
documentation to future readers, making it easier to use the package.
* Housekeeping: file renaming
nmagent.go was really focused on the nmagent.Error type, so it made
sense to rename the file to be more revealing. The same goes for
internal.go and internal_test.go. Both of those were focused on retry
logic.
Also added a quick note explaining why client_helpers_test.go exists,
since it can be a little subtle to those new to the language.
* Remove Vim fold markers
While this is nice for vim users, @ramiro-gamarra rightly pointed out
that this is a maintenance burden for non-vim users with little benefit.
Removing these to reduce the overhead.
* Set default scheme to http for nmagent client
In practice, most communication for the nmagent client occurs over HTTP
because it is intra-node traffic. While this is a useful option to have,
the default should be useful for the common use case.
* Change retry functions to return durations
It was somewhat limiting that cooldown functions themselves would block.
What was really interesting about them is that they calculated some
time.Duration. Since passing 0 to time.Sleep causes it to return
immediately, this has no impact on the AsFastAsPossible strategy
Also improved some documentation and added a few examples at the request
of @aegal
* Rename imports
The imports were incorrect because this client was moved from another
module.
* Duplicate the request in wireserver transport
Upon closer reading of the RoundTripper documentation, it's clear that
RoundTrippers should not modify the request. While effort has been made
to reset any mutations, this is still, technically, modifying the
request. Instead, this duplicates the request immediately and re-uses
the context that was provided to it.
* Drain and close http ResponseBodies
It's not entirely clear whether this is needed or not. The documentation
for http.(*Client).Do indicates that this is necessary, but
experimentation in the community has found that this is maybe not 100%
necessary (calling `Close` on the Body appears to be enough).
The only harm that can come from this is if Wireserver hands back
enormous responses, which is not the case--these responses are fairly
small.
* Capture unexpected content from Wireserver
During certain error cases, Wireserver may return XML. This XML is
useful in debugging, so we want to capture it in the error and surface
it appropriately. It's unclear whether Wireserver notes the
Content-Type, so we use Go's content type detection to figure out what
the type of the response is and clean it up to pass along to the NMAgent
Client. This also introduces a new ContentError which semantically
represents the situation where we were given a content type that we
didn't expect.
* Don't return a response with an error in RoundTrip
The http.Client complains if you return a non-nil response and an error
as well. This fixes one instance where that was happening.
* Remove extra vim folding marks
These were intended to be removed in another commit, but there were some
stragglers.
* Replace fmt.Errorf with errors.Wrap
Even though fmt.Errorf provides an official error-wrapping solution for
Go, we have made the decision to use errors.Wrap for its stack
collection support. This integrates well with Uber's Zap logger, which
we also plan to integrate.
* Use Config struct instead of functional Options
We determined that a Config struct would be more obvious than the
functional options in a debugging scenario.
* Remove validation struct tags
The validation struct tags were deemed too magical and thus removed in
favor of straight-line validation logic.
* Address Linter Feedback
The linter flagged many items here because it wasn't being run locally
during development. This addresses all of the feedback.
* Remove the UnauthorizedGracePeriod
NMAgent only defines 102 processing as a temporary status. It's up to
consumers of the client to determine whether an unauthorized status
means that it should be retried or not.
* Add error source to NMA error
One of the problems with using the WireserverTransport to modify the
http status code is that it obscures the source of those errors. Should
there be an issue with NMAgent or Wireserver, it will be difficult (or
impossible) to figure out which is which. The error itself should tell
you, and WireserverTransport knows which component is responsible. This
adds a header to the HTTP response and uses that to communicate the
responsible party. This is then wired into the outgoing error so that
clients can take appropriate action.
* Remove leftover unauthorizedGracePeriod
These blocks escaped notice when the rest of the UnauthorizedGracePeriod
logic was removed from the nmagent client.
* Remove extra validation tag
This validation tag wasn't noticed when the validation struct tags were
removed in a previous commit.
* Add the body to the nmagent.Error
When errors are returned, it's useful to have the body available for
inspection during debugging efforts. This captures the returned body and
makes it available in the nmagent.Error. It's also printed when the
error is converted to its string representation.
* Remove VirtualNetworkID
This was redundant, since VNetID covered the same key. It's actually
unclear what would happen in this circumstance if this remained, but
since it's incorrect this removes it.
* Add StatusCode to error
Clients still want to be able to communicate the status code in logs, so
this includes the StatusCode there as well.
* Add GreKey field to PutNetworkContainerRequest
In looking at usages, this `greKey` field is undocumented but critical
for certain use cases. This adds it so that it remains supported.
* Add periods at the end of all docstrings
Docstrings should have punctuation since they're documentation. This
adds punctuation to every docstring that is exported (and some that
aren't).
* Remove unused Option type
This was leftover from a previous cleanup commit.
* Change `error` to a function
The `nmagent.(*Client).error` method wasn't actually using any part of
`*Client`. Therefore it should be a function. Since we can't use `error`
as a function name because it's a reserved keyword, we're throwing back
to the Perl days and calling this one `die`.
Timothy J. Raymond