The test suite was flaking in CI due to the rate limiter having enough
time to refill its bucket. Decrease the rate of refilling from 25 per
second to 5 per second, which increases the time taken to permit an
additional request from 40ms to 200ms.
Change-Id: I29fb9c692648f4462d190c9f63a7066e335ae957
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/246037
Run-TryBot: Shaquille Que <shaquille@golang.org>
TryBot-Result: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
Add a metric for paths that work in godoc.org but not in pkg.go.dev.
Change-Id: I0290cb143cd41355c427dba6115be9bf1daadd42
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/245100
Reviewed-by: Andrew Bonventre <andybons@golang.org>
Reviewed-by: Julie Qiu <julie@golang.org>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
Run-TryBot: Andrew Bonventre <andybons@golang.org>
TryBot-Result: kokoro <noreply+kokoro@google.com>
The quota server now checks for authentication using a known
header, to see if requests from the source should be rate-limited.
This replaces an existing implementation, where the quota server checked
for the "Referer", since this is header is usually not set by requests
that do not originate from a browser.
This functionality is used to allow the teeproxy to forward requests to
pkg.go.dev.
Change-Id: Ic1eac95c3047219842b561f2cee3b0547fe36c5c
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/243283
Reviewed-by: Jonathan Amsterdam <jba@google.com>
The teeproxy currently allows forwarding to a host specified in each
request coming from godoc.org. Use environment variables instead to
specify the target host/s for each request.
Also add a new tag to aggregate metrics by host.
Change-Id: Ifcba0f227375801f7dd154d736a84583bbbdd827
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/242097
Reviewed-by: Jonathan Amsterdam <jba@google.com>
The teeproxy currently uses a database flag to determine whether to
forward requests to pkg.go.dev. Use an environment variable instead.
Change-Id: If036184131512611fab477ac769edfbeee7a61a9
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/241819
Reviewed-by: Julie Qiu <julie@golang.org>
recordTeeProxyMetric was using gddoLatency to calculate latency
for pkg.go.dev, which is now fixed.
For golang/go#39843
Change-Id: Id826d626b7409d75cf650df6b2191200f9a9dbe4
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/241600
Reviewed-by: Shaquille Que <shaquille@golang.org>
Add metrics for request latencies and counts from godoc.org and to
pkg.go.dev, as well as total server metrics.
A custom HTTP status code is added to denote that the circuit breaker
rejected the request. This status code is necessary because the metrics
are aggregated by status code, and we need to differentiate a rate
limited request vs a request rejected by the circuit breaker.
Fixesgolang/go#39843
Change-Id: I16aa34cf59794e1d226b5a77fbafca34da47ff81
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/239818
Reviewed-by: Julie Qiu <julie@golang.org>
Previously, NewSet and NewContext required the caller to construct a set
that was passed in. The functions now accept experiment names, to make
them easier to use.
Change-Id: Id306902910d51483c48a6a3bc8205cf484694d6a
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/240857
Reviewed-by: Jonathan Amsterdam <jba@google.com>
Add rate limiting and circuit breaker to the teeproxy to prevent
requests from godoc.org from bringing pkg.go.dev down.
An additional package, internal/breaker, is added to implement the
circuit breaker.
Fixesgolang/go#39702
Change-Id: Ifeda9ab7e2dbeea990f53c85cdedc840ff418845
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/238298
Reviewed-by: Julie Qiu <julie@golang.org>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
Use shadowing via ":=", which clearly signals we're not storing the
error value in a variable to be used at a later time. This way, one
knows that the error value is scoped to the if statement, so there is
less to keep in one's head.
Change-Id: Ibfb5efc92e9500a525ca58ad341d59ced0581fd9
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/238043
Reviewed-by: Julie Qiu <julie@golang.org>
A teeproxy service is add to tee traffic from godoc.org.
23683d71bb
These requests are used to collect data about requests to godoc.org, and
ensure that all incoming traffic to godoc.org can be handled
appropriately by pkg.go.dev.
Updates b/157405502
Change-Id: Ia696457a3867ba0cc883c6ebfd38b45d671e160f
Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/754075
CI-Result: Cloud Build <devtools-proctor-result-processor@system.gserviceaccount.com>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
Shaquille Que
Julie Qiu
shaquilleq
Daniel Martí