This change adds a handler which was mistakenly removed. The handler facilitates
local development.
Updates golang/go#54735
Change-Id: Ib1e8ca1ea9a3ee6471192e794975bbd5bf74428e
Reviewed-on: https://go-review.googlesource.com/c/build/+/430695
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Carlos Amedee <carlos@golang.org>
Auto-Submit: Carlos Amedee <carlos@golang.org>
Reviewed-by: Jenny Rakoczy <jenny@golang.org>
Reviewed-by: Carlos Amedee <carlos@golang.org>
This removes the original API used to instantiate remote buildlet and
gomote instances. This also removes the original gomote SSH access
implementation. This API has been replaced with a GRPC based API. The
SSH functionality has been replaced with an implementation which uses
certificate authentication.
Gomote client changes will follow this CL once in-flight changes have been submitted.
Updates golang/go#47521Fixesgolang/go#54735
Change-Id: Idfd7b95eb7ed64637e74c3c0149e0bff7027f507
Reviewed-on: https://go-review.googlesource.com/c/build/+/430279
Run-TryBot: Carlos Amedee <carlos@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Remote buildlets should not be destroyed by the pool implementations for
instances which have been created on clouds. This change adds a check
to ensure that the pool implementations can check for remote buildlets
for instances which are managed by the session pools. Those sessions will
be destroyed by the session pool.
The buildlet client accessors for instance names have been changed so
that they are no longer GCE specific.
The EC2 session pool implementation has been changed so that the
instance name is set in the buildlet client when an EC2 instance is
created.
Updates golang/go#54735
Updates golang/go#54696
Change-Id: Iae3c86cc47b4b79e3287d581dc042dcf2714f7c9
Reviewed-on: https://go-review.googlesource.com/c/build/+/426015
Reviewed-by: Carlos Amedee <carlos@golang.org>
Auto-Submit: Carlos Amedee <carlos@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Run-TryBot: Carlos Amedee <carlos@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
This was fully disabled for golang/go#25108: PreferContainersOnCOS is true in all
environments. (Note that the comment on that field is wrong AFAICT.)
Notable changes:
- basepin images are created in VMZones[0], which in prod is
us-central1-a rather than -f.
- I updated the logic for automatically determining the GCE zone/region.
I don't know when this is used and didn't bother testing it.
Change-Id: I45908383001ccdf21454cdf17abb0863abe822af
Reviewed-on: https://go-review.googlesource.com/c/build/+/422956
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Run-TryBot: Heschi Kreinick <heschi@google.com>
Auto-Submit: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
hostPathHandler used hijackResponseWriter, which was unnecessary for
html endpoints, and also broke http/2 connections in local development.
Change-Id: I62795ce67e3163a447fab01e1f31ada7b72ba0b7
Reviewed-on: https://go-review.googlesource.com/c/build/+/419978
Run-TryBot: Jenny Rakoczy <jenny@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Jenny Rakoczy <jenny@golang.org>
This moves templates from string literals to their own files, embedded
in the coordinator.
Change-Id: I1e305ce4130d0e411d7a012e881d814af0a94f34
Reviewed-on: https://go-review.googlesource.com/c/build/+/419974
Reviewed-by: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Jenny Rakoczy <jenny@golang.org>
Auto-Submit: Jenny Rakoczy <jenny@golang.org>
This adds a UI for viewing what builds are in each queue.
For golang/go#48857
Change-Id: I025d01510833cc9c0d23556d7f90a62119eb39fb
Reviewed-on: https://go-review.googlesource.com/c/build/+/419429
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Run-TryBot: Jenny Rakoczy <jenny@golang.org>
Auto-Submit: Jenny Rakoczy <jenny@golang.org>
Populates a new User field on SchedItem which will keep track of either
the gomote user or the commit author email for trybots.
Sets relui related builds to the highest priority, PriorityUrgent.
Fixesgolang/go#48857
Change-Id: I8930a0e8c6bdb1becd454e953779f42f3885fbef
Reviewed-on: https://go-review.googlesource.com/c/build/+/419428
Run-TryBot: Jenny Rakoczy <jenny@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Adds a priority queue per-cpu-quota for buildlets. The scheduler
prioritization mechanism was unaware of per-CPU-type quotas. This
replaces it with a queue per-quota implemented instead on the pool.
With this change, this should correctly prioritize gomotes that are
waiting over other builds.
Previously, when waiting on a gomote (such as for a releaselet), other
smaller host types could win the quota race before a larger releaselet
would be created. This was particularly problematic during minor
releases with a number of security fixes, which causes a very large
number of concurrent builds due to all of the backports being committed
shortly before the release.
reverse buildlets: This change replaces waiters and waitChan with a
per-hostType queue for prioritizing remote buildlet requests.
EC2 pool: This replaces quota management in the EC2 pool with
QuotaQueue. Some minor changes were necessary to deal with bookkeeping
across different instance types. A1 metal instances, for example, have a
separate hard-limit that we want to enforce, even though they also
consume the normal CPU quota.
Moved internal/coordinator/log to avoid an import cycle.
Moved SchedItem to internal/coordinator/pool/queue, where it can live
next to the Quota code that relies on it heavily.
For golang/go#48857
Change-Id: Ic4039cf425d47f46e2cc6678b4b578338c653143
Reviewed-on: https://go-review.googlesource.com/c/build/+/419036
Auto-Submit: Jenny Rakoczy <jenny@golang.org>
Run-TryBot: Jenny Rakoczy <jenny@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Carlos Amedee <carlos@golang.org>
I'm enabling UBA on go-build-log and go-build-snapshots, at which point
setting per-object ACLs will fail.
For golang/go#53890.
Change-Id: I9fe0d423146daedcc71b88ba03d035404279bb58
Reviewed-on: https://go-review.googlesource.com/c/build/+/418793
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Run-TryBot: Heschi Kreinick <heschi@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
The README suggests a command to run coordinator locally in dev mode.
Update it to take into account that coordinator uses internal/https for
HTTP(S) serving now and no longer always serves https://localhost:8119.
While here, document that the -env flag isn't needed when -mode=dev is
being used. Fix and simplify the /try-dev page: it was using dev build
constraint, so not tested via CI and broke. (I considered just deleting
it, but it was simple enough to keep and it adds value to dev mode.)
Change-Id: I4ffaecd07e5f1321f55bc18f42281d48f2433f30
Reviewed-on: https://go-review.googlesource.com/c/build/+/416215
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Alex Rakoczy <jenny@golang.org>
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
When we updated coordinator to use internal/https, the /status/ handlers
were accidentally missed and left serving on the now unused default mux.
Change-Id: Ie858dfa2b321a4c39f2ee4e96b2d0cb4d94b5511
Reviewed-on: https://go-review.googlesource.com/c/build/+/416214
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Alex Rakoczy <jenny@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
This was accidentally left unformatted in CL 408696.
For golang/go#52653.
Change-Id: Ia47ef27e5f6a7b328f0b51b3378ef823ffcb03a9
Reviewed-on: https://go-review.googlesource.com/c/build/+/410242
Run-TryBot: Michael Pratt <mpratt@google.com>
Auto-Submit: Michael Pratt <mpratt@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
We may know of more than one issue affecting a builder. In some
contexts (such as the build dashboard) we only have enough room to
display one of those, but in other contexts (such as SlowBot failure
messages) it may be useful to see all of them as a list.
Moreover, this way we don't have to remember to check for other known
issues when removing an issue that has been resolved.
For golang/go#52653.
Change-Id: I9da42a3bf99ad24ec4b3ba9c4e6ce214b9f9cac7
Reviewed-on: https://go-review.googlesource.com/c/build/+/408696
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Alex Rakoczy <alex@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Bryan Mills <bcmills@google.com>
Run-TryBot: Bryan Mills <bcmills@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
This adds the gomote instances to the status page presented at
farmer.golang.org.
Updates golang/go#52594
For golang/go#47521
Change-Id: I29c73262031fc95cc85cdb43734da49149c958b3
Reviewed-on: https://go-review.googlesource.com/c/build/+/405258
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Run-TryBot: Carlos Amedee <carlos@golang.org>
Auto-Submit: Carlos Amedee <carlos@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Alex Rakoczy <alex@golang.org>
This change moves the SSH handlers into the internal packages. It also
adds the handler which will use the session pool instead of the remote
buildlets.
Updates golang/go#52594
For golang/go#47521
Change-Id: I7e99fdbb16e0f80a871696cec79a9b638354e662
Reviewed-on: https://go-review.googlesource.com/c/build/+/405257
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Carlos Amedee <carlos@golang.org>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Run-TryBot: Carlos Amedee <carlos@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
This change adds OpenSSH certificate authentication to the SSH Server.
The change also:
- Moves the public key authentication logic into the remote package.
- Enables a chain of authentication methods such that both public
key authentication and certificate authentication are used.
- Usees a dynamically created CA certificate for certificate
authentication. Each time the server is started, a new certificate
is created.
Fixesgolang/go#52594
For golang/go#47521
Change-Id: I8198d436b53844357af01510dcf8f2fc67af83b4
Reviewed-on: https://go-review.googlesource.com/c/build/+/405256
Auto-Submit: Carlos Amedee <carlos@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Reviewed-by: Alex Rakoczy <alex@golang.org>
Run-TryBot: Carlos Amedee <carlos@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
This change refactors the SSH Server used by the coordinator. Portions
of the server have been moved into the internal/coordinator/remote
package and are being prepared for the addition of a different
authentication scheme.
Updates golang/go#52594
For golang/go#47521
Change-Id: Ib1e961ea6d27c861f787068d237a02a47b6b0a2c
Reviewed-on: https://go-review.googlesource.com/c/build/+/405255
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Run-TryBot: Carlos Amedee <carlos@golang.org>
Reviewed-by: Alex Rakoczy <alex@golang.org>
Auto-Submit: Carlos Amedee <carlos@golang.org>
We had a lot of flexibility over timeouts, making their maintenance
harder. Consolidate it to a single timeout in the pool package, and
modify it from 45 minutes to 2 hours.
There's room for improvement in how we maintain this timeout,
but I'm leaving that for future work (with a tracking issue).
Fixesgolang/go#52591.
Updates golang/go#52929.
Updates golang/go#49666.
Updates golang/go#42699.
Change-Id: I2ad92648d89a714397bd8b0e1ec490fc9f6d6790
Reviewed-on: https://go-review.googlesource.com/c/build/+/406216
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
In CL 342712, functionality was added to build x-repo builders using
slowbots. However, based on the existing precedent this CL limited this
functionality to trybots. There is no reason for this restriction, so
relax the condition to allow any post-submit builder.
Fixesgolang/go#52667
Change-Id: Ica54291605982680a5e69a36a27ac96b79bd5346
Reviewed-on: https://go-review.googlesource.com/c/build/+/403675
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Robert Findley <rfindley@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
Gofmt to update doc comments to the new formatting.
For golang/go#51082.
Change-Id: Ie063c2fd890641b551c0f304fee5fa19121f1b38
Reviewed-on: https://go-review.googlesource.com/c/build/+/399555
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Russ Cox <rsc@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
This change adds the upload file endpoint which will be used by the
gomote clients to upload files to GCS before they are retrieved by a
gomote instance. The endpoint generates a signed URL and associated
fields which must be used in the upload.
For golang/go#47521
Updates golang/go#48742
Change-Id: Id85a55b41b8211b3aae8c2e30245a0b71ecfa238
Reviewed-on: https://go-review.googlesource.com/c/build/+/397595
Trust: Carlos Amedee <carlos@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
In service of getting the right branch to performance monitoring data,
this change stops using the confusingly named (and confusingly set)
goBranch field of buildStatus in favor of always passing a commitDetail
to newBuild with a valid branch. This means that all uses of
commitDetail pass a non-nil commitDetail, so this change also makes it
passed by value.
This change also does some additional quality-of-life refactoring.
Namely, it lifts the parsing of commit times into the commitDetail,
allowing us to just embed a commitDetail into the buildStatus struct.
It also removes dead code related to staging that would have to change
as a result of this CL.
This change is almost a no-op, except for performance monitoring
use-cases. In all cases where goBranch was used, the equivalent
RevBranch is now populated and used instead. For performance monitoring
however, by using RevBranch instead of goBranch, we guarantee that we
always have a branch and it's the one we actually want.
Conveniently, the branch is now always available to buildStatus when it
wasn't before, which may be useful in the future.
For golang/go#48803.
Change-Id: I8ad3f115b1b2546dbf0848649965b2c720f73ead
Reviewed-on: https://go-review.googlesource.com/c/build/+/395174
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Trust: Michael Knyszek <mknyszek@google.com>
This change adds the implementation for the sign SSH key endpoint.
The endpoint accepts a public SSH key and signs it with the gomote
server's certificate authority. The certificate added to the public
key will be used to validate if the certificate authority was used
to sign the certificate. It will also be used to determine if
the requestor has rights to initiate and SSH session with the
gomote instance being requested. This is part of a shift to
OpenSSH certificate authentication in the gomote SSH server.
For golang/go#47521
Updates golang/go#48742
Change-Id: I427b34c7f006ae20f5643322dc0754bf7a82e5f1
Reviewed-on: https://go-review.googlesource.com/c/build/+/391516
Trust: Carlos Amedee <carlos@golang.org>
Run-TryBot: Carlos Amedee <carlos@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Benchmarking code needs more information on the commit time for the main
repo revision. However, it seems silly to *just* plumb that. So, let's
plumb down both repo and subrepo commit times and add a helper to
determine which one to use at the point of reference.
At *that* point, it seems silly to only do this for commit times and not
for the branch. Let's err on plumbing additional details for the moment
because they're sure to come in handy for identifying benchmark run
differences down the line. This change does the same thing for the
branch.
Change-Id: I38e97e25b271cbd36d7e9bb2f765211c8555cb7d
Reviewed-on: https://go-review.googlesource.com/c/build/+/393640
Trust: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Alex Rakoczy <alex@golang.org>
The build repository currently uses two diffrent GRPC implementations.
The go4.org one and the official one. This change attempts to converge
on a single version of GRPC for the build repository.
Change-Id: I3c19c1159646cbbff8fa28c8f1738cb468b6be8c
Reviewed-on: https://go-review.googlesource.com/c/build/+/361454
Reviewed-by: Alex Rakoczy <alex@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Trust: Carlos Amedee <carlos@golang.org>
Run-TryBot: Carlos Amedee <carlos@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Explicitly define a ServeMux and routes that we want to expose to
prevent dependencies from registering unwanted handlers.
Fixesgolang/go#50894
Change-Id: Ic083b346efd6e5ad33dd6d5a45c2a63e5c61aa0e
Reviewed-on: https://go-review.googlesource.com/c/build/+/381737
Trust: Alex Rakoczy <alex@golang.org>
Run-TryBot: Alex Rakoczy <alex@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Carlos Amedee <carlos@golang.org>
This change adds the option to skip the validation of the audience
field in JWT tokens. We understand that validating the JWT token is
enough to know that the packet came from a valid source.
Updates golang/go#48742
Change-Id: I77c79e4b7f7273c3fb74f088c402c0a55efc3169
Reviewed-on: https://go-review.googlesource.com/c/build/+/378574
Trust: Carlos Amedee <carlos@golang.org>
Run-TryBot: Carlos Amedee <carlos@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
This change implements the endpoint to create gomote instances for the
gomote GRPC service. In the process of implementing creates, various
other changes were needed:
- Refactoring the remote session pool.
- Extending the fake schedule used for testing.
Updates golang/go#48742
Change-Id: I0c74e38539428d028917200ccd6bd0c58fa14801
Reviewed-on: https://go-review.googlesource.com/c/build/+/370662
Trust: Carlos Amedee <carlos@golang.org>
Run-TryBot: Carlos Amedee <carlos@golang.org>
Trust: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
This change introduces the use of a buildlet client interface in
existing functions. This is being introduced in order to facilitate
testing.
Change-Id: I41cd5a372fc31edbd9bcba1859cdf84308360174
Reviewed-on: https://go-review.googlesource.com/c/build/+/371818
Trust: Carlos Amedee <carlos@golang.org>
Run-TryBot: Carlos Amedee <carlos@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Alex Rakoczy <alex@golang.org>
This moves the coordinator scheduler into a package. The span has also
been moved into the schedule package. It also adds a mostly
uimplemented fake scheduler.
Updates golang/go#38337
Updates golang/go#48742
Change-Id: I980241e8e8ba2acafa38f732fe480e66d9d3a4f3
Reviewed-on: https://go-review.googlesource.com/c/build/+/368676
Trust: Carlos Amedee <carlos@golang.org>
Run-TryBot: Carlos Amedee <carlos@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
This change moves the function which returns the appropriate pool for
the configuration passed in into the pool package. This work is being
done as part of a project to break the coordinator into seperate parts.
Updates golang/go#48742
Updates golang/go#38337
Change-Id: Ie5b3fc2da6534fca6e55ba6bb710db5e206efe00
Reviewed-on: https://go-review.googlesource.com/c/build/+/368674
Trust: Carlos Amedee <carlos@golang.org>
Run-TryBot: Carlos Amedee <carlos@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Reviewed-by: Alex Rakoczy <alex@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
This change ensures the buildenv is set when the coordinator is
running in production.
Updates golang/go#47521
Change-Id: Ibd1a31609f5e85ac6445bad5daec5222a06b13e4
Reviewed-on: https://go-review.googlesource.com/c/build/+/367554
Trust: Carlos Amedee <carlos@golang.org>
Run-TryBot: Carlos Amedee <carlos@golang.org>
Reviewed-by: Alexander Rakoczy <alex@golang.org>
This change:
- Adds a simple GRPC gomote server.
- Updates the documentation for the audiance required for IAP authentication.
- Adds a field for the backend service id in the build enviornment package.
- Creates middleware for the GRPC server use in the existing HTTP servers.
Updates golang/go#47521
Updates golang/go#48742
Change-Id: I2a56e39b96bf1b429f807f79c58aee3f72a45a33
Reviewed-on: https://go-review.googlesource.com/c/build/+/361098
Trust: Carlos Amedee <carlos@golang.org>
Run-TryBot: Carlos Amedee <carlos@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Alexander Rakoczy <alex@golang.org>
coordinator.go has grown to about 4100 lines and has become difficult to
navigate. The largest related chunk of code is buildStatus, which we now
organize into its own file, which should help navigation a little.
Also, clean up a couple comments, deperecated, and unused code along the
way.
Change-Id: I83508a915a63074cf6ec44b4c21229389377e1a5
Reviewed-on: https://go-review.googlesource.com/c/build/+/359434
Trust: Alexander Rakoczy <alex@golang.org>
Run-TryBot: Alexander Rakoczy <alex@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Carlos Amedee <carlos@golang.org>
These make it possible to tell what was run, as well as a convenience
field stating whether this was a post-submit build or a trybot run.
For golang/go#49207
Change-Id: Iba979bcfd5a3bbdc11e2df0b8de4094cc7212356
Reviewed-on: https://go-review.googlesource.com/c/build/+/361874
Trust: Michael Pratt <mpratt@google.com>
Run-TryBot: Michael Pratt <mpratt@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Alexander Rakoczy <alex@golang.org>
The performance test output contains benchfmt-formatted benchmark
results. Upload the output wholesale to perfdata.golang.org for
long-term storage and analysis.
The results for now are a bit rough, as the output may also contain
unrelated output that lines that look like benchfmt. For example. "go:
downloading github.com/BurntSushi/toml v0.3.1" adds a "go" label with
the value "downloading ...". In the future, we will ideally filter these
a bit better (perhaps in x/benchmarks/cmd/bench).
For golang/go#49207
Change-Id: Ifd2512c93902a74f9040db0f9d0c600348fc1849
Reviewed-on: https://go-review.googlesource.com/c/build/+/361418
Reviewed-by: Alexander Rakoczy <alex@golang.org>
Trust: Michael Pratt <mpratt@google.com>
Run-TryBot: Michael Pratt <mpratt@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Add initial support for running the performance tests from x/benchmarks.
Since there are a variety of different test suites (some `go test`,
bent, etc), x/benchmarks provides a basic wrapper command,
golang.org/x/benchmarks/cmd/bench which know the minute details. The
coordinator just needs to run that one command.
This build mode is limited to builds of x/benchmarks on builders with
RunBench set to true. Currently there are none, a future CL will add the
initial such linux-amd64 builder.
For golang/go#49207
Change-Id: Ie006ec4a3757a5c2fed0925a3f9eb91edeaa5224
Reviewed-on: https://go-review.googlesource.com/c/build/+/361417
Trust: Michael Pratt <mpratt@google.com>
Run-TryBot: Michael Pratt <mpratt@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Alexander Rakoczy <alex@golang.org>
Running benchmarks has been disabled since 2018. Remove all the code to
keep things more maintainable and understandable.
We will be adding new benchmarking support soon, and may reuse some of
this code, but don't want half-working code adding confusion.
For golang/go#49207.
Change-Id: I11d52b0315bed4d91651c162af11853895012868
Reviewed-on: https://go-review.googlesource.com/c/build/+/354315
Trust: Michael Pratt <mpratt@google.com>
Run-TryBot: Michael Pratt <mpratt@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Reviewed-by: Alexander Rakoczy <alex@golang.org>
We're gearing up to put most everything behind Cloud load balancers, at
which point Autocert serving will be unnecessary. To support HTTP/2 we
still need to serve TLS, but we can do it using a self-signed
certificate.
Unify the configuration of all that stuff behind a single set of flags
and clean up all the copied and tweaked code I could find.
The self-signed serving is dormant for now; I'll activate it in a later
change.
I didn't understand the need to tweak keepalive intervals everywhere;
keepalives have been on by default since 1.13. So I dropped them.
Change-Id: I0016211f1032a6c64375aad9e34e4e2e62861c5b
Reviewed-on: https://go-review.googlesource.com/c/build/+/358654
Trust: Heschi Kreinick <heschi@google.com>
Trust: Alexander Rakoczy <alex@golang.org>
Reviewed-by: Alexander Rakoczy <alex@golang.org>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Run-TryBot: Alexander Rakoczy <alex@golang.org>
Following:
https://cloud.google.com/kubernetes-engine/docs/concepts/ingresshttps://cloud.google.com/kubernetes-engine/docs/how-to/managed-certs
Create an Ingress with HTTPS redirects enabled.
For each of build, dev, gerritbot, maintner:
- Add three certs: the new Google-managed cert, plus the two old LE certs.
- Add an ingress rule for the host, and a new internal Service it maps to.
- Disable our own HTTPS redirects, since the ingress terminates HTTPS.
They will be accessible via HTTP for the duration of the migration;
they don't serve anything sensitive anyway.
dev-test.golang.org is already rolled out using only a Google-managed
cert, so you can try HTTP and HTTPS directly there. I've also verified
that the LE certs are served:
curl -v --resolve dev.golang.org:443:34.149.223.5 https://dev.golang.org/
Change-Id: I20f3c92ffb00dbd9066822f748508013b68d5483
Reviewed-on: https://go-review.googlesource.com/c/build/+/355670
Trust: Heschi Kreinick <heschi@google.com>
Trust: Alexander Rakoczy <alex@golang.org>
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Alexander Rakoczy <alex@golang.org>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Start the main findWorkLoop in dev mode, to discover work to run from
the dashboard. In dev mode, was also replace the linux-amd64 builder
config with one using host-linux-amd64-localdev reverse buildlets.
This provides a complete lifecycle to test out builds with a local dev
coordinator and buildlet.
For golang/go#48803
Change-Id: I8ade6c8bccf3bc51437ca9e7d11c232753fe7465
Reviewed-on: https://go-review.googlesource.com/c/build/+/354638
Trust: Michael Pratt <mpratt@google.com>
Run-TryBot: Michael Pratt <mpratt@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Alexander Rakoczy <alex@golang.org>
Though it is a good default, it is cumbersome for dev mode to listen
only on localhost when developing over SSH. Add a -listen flag to allow
overriding this with any address.
For golang/go#48803
Change-Id: If01a0b44926a33f2aa01548319508166592936e3
Reviewed-on: https://go-review.googlesource.com/c/build/+/354637
Trust: Michael Pratt <mpratt@google.com>
Run-TryBot: Michael Pratt <mpratt@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Alexander Rakoczy <alex@golang.org>
In dev mode, the coordinator uses hostPathHandler, which redirects
/reverse and /revdial to /farmer.golang.org/reverse, etc.
Establishing a revdial connection chokes on this redirect, as there it
expects to complete the protocol switch in a single request.
Add rudimentary redirect support via a helper so this works in dev mode.
Note that linkRewriter must implement http.Hijacker as
revdial.ConnHandler type-asserts the http.ResponseWriter to a Hijacker.
For golang/go#48803
Change-Id: I191fa6ff17bbd334203430f3c1f2c5e03db407ff
Reviewed-on: https://go-review.googlesource.com/c/build/+/354630
Trust: Michael Pratt <mpratt@google.com>
Run-TryBot: Michael Pratt <mpratt@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Alexander Rakoczy <alex@golang.org>
All of the subrepos we test use modules, so GOPATH test mode here is
unused and can be deleted.
Fixesgolang/go#48875
Change-Id: Id5143e9fcad7fc91e8e0f63dcca35f5661be5a03
Reviewed-on: https://go-review.googlesource.com/c/build/+/355089
Trust: Michael Pratt <mpratt@google.com>
Run-TryBot: Michael Pratt <mpratt@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
This is the last remaining revdialv2 reference, missed by
https://golang.org/cl/208598.
Change-Id: I2ebf774b0e7c1e8d34e40b9768f8924248f077f0
Reviewed-on: https://go-review.googlesource.com/c/build/+/354290
Trust: Michael Pratt <mpratt@google.com>
Run-TryBot: Michael Pratt <mpratt@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Carlos Amedee
Heschi Kreinick
Alex Rakoczy
Dmitri Shuralyov
Michael Pratt
Bryan C. Mills
Robert Findley
Russ Cox
Michael Anthony Knyszek