This adds the gomote instances to the status page presented at
farmer.golang.org.
Updates golang/go#52594
For golang/go#47521
Change-Id: I29c73262031fc95cc85cdb43734da49149c958b3
Reviewed-on: https://go-review.googlesource.com/c/build/+/405258
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Run-TryBot: Carlos Amedee <carlos@golang.org>
Auto-Submit: Carlos Amedee <carlos@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Alex Rakoczy <alex@golang.org>
This change moves the SSH handlers into the internal packages. It also
adds the handler which will use the session pool instead of the remote
buildlets.
Updates golang/go#52594
For golang/go#47521
Change-Id: I7e99fdbb16e0f80a871696cec79a9b638354e662
Reviewed-on: https://go-review.googlesource.com/c/build/+/405257
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Carlos Amedee <carlos@golang.org>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Run-TryBot: Carlos Amedee <carlos@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
This change adds OpenSSH certificate authentication to the SSH Server.
The change also:
- Moves the public key authentication logic into the remote package.
- Enables a chain of authentication methods such that both public
key authentication and certificate authentication are used.
- Usees a dynamically created CA certificate for certificate
authentication. Each time the server is started, a new certificate
is created.
Fixesgolang/go#52594
For golang/go#47521
Change-Id: I8198d436b53844357af01510dcf8f2fc67af83b4
Reviewed-on: https://go-review.googlesource.com/c/build/+/405256
Auto-Submit: Carlos Amedee <carlos@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Reviewed-by: Alex Rakoczy <alex@golang.org>
Run-TryBot: Carlos Amedee <carlos@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
This change refactors the SSH Server used by the coordinator. Portions
of the server have been moved into the internal/coordinator/remote
package and are being prepared for the addition of a different
authentication scheme.
Updates golang/go#52594
For golang/go#47521
Change-Id: Ib1e961ea6d27c861f787068d237a02a47b6b0a2c
Reviewed-on: https://go-review.googlesource.com/c/build/+/405255
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Run-TryBot: Carlos Amedee <carlos@golang.org>
Reviewed-by: Alex Rakoczy <alex@golang.org>
Auto-Submit: Carlos Amedee <carlos@golang.org>
We had a lot of flexibility over timeouts, making their maintenance
harder. Consolidate it to a single timeout in the pool package, and
modify it from 45 minutes to 2 hours.
There's room for improvement in how we maintain this timeout,
but I'm leaving that for future work (with a tracking issue).
Fixesgolang/go#52591.
Updates golang/go#52929.
Updates golang/go#49666.
Updates golang/go#42699.
Change-Id: I2ad92648d89a714397bd8b0e1ec490fc9f6d6790
Reviewed-on: https://go-review.googlesource.com/c/build/+/406216
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Some reverse builders may be further away geographically, so waiting
only 5 seconds (a time chosen in 2015) before concluding the buildlet
is unhealthy seems short. Give them a bit more time to accommodate for
round-trip network trip and processing latency.
Change-Id: I707c430538b2f19115e9b9dd78ec6aaeb64c4270
Reviewed-on: https://go-review.googlesource.com/c/build/+/406215
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
By now there are no reverse builders that still use revdial v1,
all are version 23 or newer (where revdial v2 support is added).
Most of the code was dropped in 2019 (in CL 208598),
this CL deletes some remaining obsolete bookkeeping.
Updates golang/go#31639.
Change-Id: Icedd48f9342ea88287c5d580f085785c57846a65
Reviewed-on: https://go-review.googlesource.com/c/build/+/406214
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Gofmt to update doc comments to the new formatting.
For golang/go#51082.
Change-Id: Ie063c2fd890641b551c0f304fee5fa19121f1b38
Reviewed-on: https://go-review.googlesource.com/c/build/+/399555
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Russ Cox <rsc@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Generated with go run update-readmes.go.
Also capitalize the word "Package" in 3 package comments
and update releaselet test not to fail on README.md file.
Change-Id: I031b15ec570dd089b1931b79dcb1f8e6694e7e62
Reviewed-on: https://go-review.googlesource.com/c/build/+/394520
Reviewed-by: Carlos Amedee <carlos@golang.org>
Trust: Carlos Amedee <carlos@golang.org>
Trust: Emmanuel Odeke <emmanuel@orijtech.com>
Trust: Dmitri Shuralyov <dmitshur@golang.org>
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
This change adds the implementation for the sign SSH key endpoint.
The endpoint accepts a public SSH key and signs it with the gomote
server's certificate authority. The certificate added to the public
key will be used to validate if the certificate authority was used
to sign the certificate. It will also be used to determine if
the requestor has rights to initiate and SSH session with the
gomote instance being requested. This is part of a shift to
OpenSSH certificate authentication in the gomote SSH server.
For golang/go#47521
Updates golang/go#48742
Change-Id: I427b34c7f006ae20f5643322dc0754bf7a82e5f1
Reviewed-on: https://go-review.googlesource.com/c/build/+/391516
Trust: Carlos Amedee <carlos@golang.org>
Run-TryBot: Carlos Amedee <carlos@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Debian jessie is out of LTS support. This change removes
the Debian jessie based builders:
- use linux-x86-stretch for the misc-compile builders.
- linux-x86-jessie host has been removed.
- documentation for the linux-x86-clang builders have been updated.
- documentation throughout the repo has been updated.
For golang/go#47721
Change-Id: Ia2e916ea94fe8e6adb8f5a56d6c7319817de75e2
Reviewed-on: https://go-review.googlesource.com/c/build/+/383154
Trust: Carlos Amedee <carlos@golang.org>
Run-TryBot: Carlos Amedee <carlos@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
Trust: Bryan Mills <bcmills@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
CL 207841 removed the last use of this option. Additionally, the
scheduler makes setting this option from somewhere difficult as the
scheduler constructs a new background context to pass to the pool. As a
result, this option is unlikely to be used going forward.
Change-Id: I1aed89e72f35fc01b3ec9bf9f164f92b32143ffe
Reviewed-on: https://go-review.googlesource.com/c/build/+/378335
Trust: Michael Pratt <mpratt@google.com>
Run-TryBot: Michael Pratt <mpratt@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Alex Rakoczy <alex@golang.org>
Allow individual host configurations to override the VM delete timeout
if they are using for longer than normal builds.
For golang/go#49207.
Change-Id: I9c5c80e5ee7dac2375cff17c64871ae2211f6309
Reviewed-on: https://go-review.googlesource.com/c/build/+/378334
Trust: Michael Pratt <mpratt@google.com>
Run-TryBot: Michael Pratt <mpratt@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Alex Rakoczy <alex@golang.org>
This change adds the implementation for gomote instance alive. This
endpoint enables the caller to check if an instance is alive. If the
instance is alive, it will extend the gomote timeout time. A renew
timeout method has been added to the session pool. This differs from
the existing keep alive method in that it is an single call to renew
the timeout for an instance instead of a continuous renewal of the
timeout tied to the lifetime of a context.
Updated golang/go#48742
Change-Id: I3b3462407d9f4a02c4e2cea0f14950c8c9f21060
Reviewed-on: https://go-review.googlesource.com/c/build/+/374115
Trust: Carlos Amedee <carlos@golang.org>
Run-TryBot: Carlos Amedee <carlos@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
This change implements the endpoint to create gomote instances for the
gomote GRPC service. In the process of implementing creates, various
other changes were needed:
- Refactoring the remote session pool.
- Extending the fake schedule used for testing.
Updates golang/go#48742
Change-Id: I0c74e38539428d028917200ccd6bd0c58fa14801
Reviewed-on: https://go-review.googlesource.com/c/build/+/370662
Trust: Carlos Amedee <carlos@golang.org>
Run-TryBot: Carlos Amedee <carlos@golang.org>
Trust: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
This change introduces the use of a buildlet client interface in
existing functions. This is being introduced in order to facilitate
testing.
Change-Id: I41cd5a372fc31edbd9bcba1859cdf84308360174
Reviewed-on: https://go-review.googlesource.com/c/build/+/371818
Trust: Carlos Amedee <carlos@golang.org>
Run-TryBot: Carlos Amedee <carlos@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Alex Rakoczy <alex@golang.org>
This change removes the go1.13 and go1.16 build constraints since they are no
longer needed.
Change-Id: I4428d4bc6df7d02d788dbdf6a5329dbd0d5e095b
Reviewed-on: https://go-review.googlesource.com/c/build/+/371454
Trust: Carlos Amedee <carlos@golang.org>
Run-TryBot: Carlos Amedee <carlos@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
This moves the coordinator scheduler into a package. The span has also
been moved into the schedule package. It also adds a mostly
uimplemented fake scheduler.
Updates golang/go#38337
Updates golang/go#48742
Change-Id: I980241e8e8ba2acafa38f732fe480e66d9d3a4f3
Reviewed-on: https://go-review.googlesource.com/c/build/+/368676
Trust: Carlos Amedee <carlos@golang.org>
Run-TryBot: Carlos Amedee <carlos@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
This change moves the function which returns the appropriate pool for
the configuration passed in into the pool package. This work is being
done as part of a project to break the coordinator into seperate parts.
Updates golang/go#48742
Updates golang/go#38337
Change-Id: Ie5b3fc2da6534fca6e55ba6bb710db5e206efe00
Reviewed-on: https://go-review.googlesource.com/c/build/+/368674
Trust: Carlos Amedee <carlos@golang.org>
Run-TryBot: Carlos Amedee <carlos@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Reviewed-by: Alex Rakoczy <alex@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
This tracks CPU quota when scheduling instances, and shows quota use in
the coordinator UI.
Change-Id: I60f2382657757a4a1dd1952327934f8f6a44e414
Reviewed-on: https://go-review.googlesource.com/c/build/+/367555
Trust: Alexander Rakoczy <alex@golang.org>
Run-TryBot: Alexander Rakoczy <alex@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Carlos Amedee <carlos@golang.org>
This change refactors the existing remote buildlet session management
implementation. It implements logic which was originally found in the
HTTP handlers and moves it into a session manager. This change also
introduces a fake buildlet client intended to be used for testing when
additional functionality is added to the session pool.
Updates golang/go#48735
Change-Id: Ib7d66cefb9847cf2659e7049f707d78536fbbbea
Reviewed-on: https://go-review.googlesource.com/c/build/+/361097
Trust: Carlos Amedee <carlos@golang.org>
Run-TryBot: Carlos Amedee <carlos@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Reviewed-by: Alexander Rakoczy <alex@golang.org>
This change copies the remote buildlet session management logic from
the cmd/coordinator/remote.go file. This copies the existing
implementation without including the HTTP and SSH handlers which also
contain small portions of logic. This is the first step before
refactoring the implementation which will be used in the GRPC server.
Updates golang/go#48735
Change-Id: Idb26616ddb5494a8f96f8b66369e20f19f2871dc
Reviewed-on: https://go-review.googlesource.com/c/build/+/361096
Trust: Carlos Amedee <carlos@golang.org>
Run-TryBot: Carlos Amedee <carlos@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Alexander Rakoczy <alex@golang.org>
Show our quotas on the farmer homepage making it easier to see when
we're at capacity.
Change-Id: I8c6820298914b320c4165e0ec7d9d9d9f1d7a5c4
Reviewed-on: https://go-review.googlesource.com/c/build/+/361657
Trust: Alexander Rakoczy <alex@golang.org>
Run-TryBot: Alexander Rakoczy <alex@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
We currently use E2, C2, and N2 instances on GCE. C2 and N2 instances
have their own quotas, which are accounted separately from the CPUS
quotas.
This could probably be cleaned up to keep track of all CPU quotas and
handle more instance types, but this should work for the time being.
See: https://cloud.google.com/compute/quotas#cpu_quota
For golang/go#49207
Change-Id: Ida1e8de3c857560637095d57e972bca7222284ed
Reviewed-on: https://go-review.googlesource.com/c/build/+/361656
Trust: Alexander Rakoczy <alex@golang.org>
Run-TryBot: Alexander Rakoczy <alex@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
This replaces the broken test in internal/coordinator/pool.
For golang/go#48803
Change-Id: I7bd9265bba555562ffa7d59169a9c8792ed97d3c
Reviewed-on: https://go-review.googlesource.com/c/build/+/354629
Trust: Michael Pratt <mpratt@google.com>
Run-TryBot: Michael Pratt <mpratt@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Alexander Rakoczy <alex@golang.org>
We should have similar performance for these machines, and increased
memory availability in some areas.
This is also compliant with an internal policy change.
Change-Id: I07a452a07431322d6fdd4edbb98c5914045d34ed
Reviewed-on: https://go-review.googlesource.com/c/build/+/354757
Trust: Alexander Rakoczy <alex@golang.org>
Run-TryBot: Alexander Rakoczy <alex@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Resubmission of https://golang.org/cl/354642 now that we have our NAT
config sorted. Hopefully.
We're hitting our external IP quota. Now that we have Cloud NAT set up,
we don't need the external IPs at all. Disable them and stop checking
for them.
Change-Id: I5091ebacd79eb0bddec394f35eb9c7368a699ac4
Reviewed-on: https://go-review.googlesource.com/c/build/+/354756
Trust: Heschi Kreinick <heschi@google.com>
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Alexander Rakoczy <alex@golang.org>
This reverts commit 7e966852b8 (https://golang.org/cl/354642).
Reason for revert: breaks internet access in -longtest builders somehow
Change-Id: I5344e0c8cec5bf93a23c2c3bf592215843e129de
Reviewed-on: https://go-review.googlesource.com/c/build/+/354753
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
We're hitting our external IP quota. Now that we have Cloud NAT set up,
we don't need the external IPs at all. Disable them and stop checking
for them.
(I don't understand why the code wanted to use an external IP when TLS is
enabled, so I ripped it out. If there's a reason I'm missing let me
know.)
Change-Id: I3e66b2ed984d936aa0f58d3e7178ab7356d73ce1
Reviewed-on: https://go-review.googlesource.com/c/build/+/354642
Trust: Heschi Kreinick <heschi@google.com>
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Alexander Rakoczy <alex@golang.org>
Turns out that to query for a regional Kubernetes cluster you need to
use a different API. Require an explicit location to create a client and
use that new API.
Also, modern Kubernetes clusters won't have client cert auth enabled.
Use OAuth instead.
Finally, don't require that the buildlets cluster be in the same zone as
the coordinator.
For golang/go#48408.
Change-Id: Ic3f9525b9bffa89d779e684c8ea1be116d3f983f
Reviewed-on: https://go-review.googlesource.com/c/build/+/350754
Trust: Heschi Kreinick <heschi@google.com>
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Alexander Rakoczy <alex@golang.org>
I noticed some unused-looking fields in the Environment type. gcpinit
looks dead to me, so delete it. Correct me if I'm wrong.
For golang/go#48408.
Change-Id: Ie0732177ada8c291f2bddc1b64a6b242a521d5bc
Reviewed-on: https://go-review.googlesource.com/c/build/+/350113
Trust: Heschi Kreinick <heschi@google.com>
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Alexander Rakoczy <alex@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Our makefiles install xb from the repository, so this has to be checked
in before I do anything else, and nobody else should deploy anything
until I'm done.
Because the new cluster is an Autopilot cluster, it lives in a region
(us-central1) and that's what you pass to the kubectl command, etc.
Move Region/Zone into the individual KubeConfigs and use the correct
ones as appropriate.
For golang/go#48408.
Change-Id: Iceacfe68305a3744aa87ce0fef777b977a252586
Reviewed-on: https://go-review.googlesource.com/c/build/+/350137
Trust: Heschi Kreinick <heschi@google.com>
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Create a new service account, and move the deployment over to the prod
namespace.
The network metadata entry we look for isn't available under Workload
Identity
(https://cloud.google.com/kubernetes-engine/docs/how-to/workload-identity#gke_mds)
so use the hostname instead, which works fine.
For golang/go#48263.
Change-Id: I91ef091de3e0a923b4a96c56a7f8e7c9e614be8f
Reviewed-on: https://go-review.googlesource.com/c/build/+/349950
Trust: Heschi Kreinick <heschi@google.com>
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Rather than using a service account key, enable Workload Identity for
gitmirror. This involved moving into a non-default namespace, so the
coordinator's monitoring logic needs updating too.
Change-Id: I196fae0825f806248571aae6fb125d8c0917934d
Reviewed-on: https://go-review.googlesource.com/c/build/+/347410
Trust: Heschi Kreinick <heschi@google.com>
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Alexander Rakoczy <alex@golang.org>
Reviewed-by: Carlos Amedee <carlos@golang.org>
This change allows the ledger to track how many a1.metal instances
have either been created or are in the process of being created. A
hard limit of 1 has been set for a1.metal limits while testing is
being performed.
For golang/go#42604
Change-Id: I5bcb3a65407af07d225caf2884877ce040ee011b
Reviewed-on: https://go-review.googlesource.com/c/build/+/322855
Trust: Carlos Amedee <carlos@golang.org>
Run-TryBot: Carlos Amedee <carlos@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Alexander Rakoczy <alex@golang.org>
Replace low-level Stackdriver monitoring API usage for OpenCensus
with a Stackdriver exporter. To benefit local development, expose
metrics at an /metrics endpoint (to be picked up with Prometheus).
This makes it much easier to add new metrics, to test them locally,
and brings our metrics solution in sync with what's currently in
use in x/playground (see CL 302769). It's expected to be preferable
to migrate to OpenTelemetry in the future when a good migration path
becomes available, and both x/build and x/playground can be updated
at that time.
This CL is based on work in CL 229679 and CL 138522.
For golang/go#26779.
For golang/go#44406.
For golang/go#17104.
Co-authored-by: Alexander Rakoczy <alex@golang.org>
Co-authored-by: Emmanuel T Odeke <emmanuel@orijtech.com>
Change-Id: Iad45730feace471db1668e828b7c9775377be8a9
Reviewed-on: https://go-review.googlesource.com/c/build/+/303669
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Alexander Rakoczy <alex@golang.org>
Reviewed-by: Emmanuel Odeke <emmanuel@orijtech.com>
Make all our package sources use Go 1.17 gofmt format
(adding //go:build lines).
Not strictly necessary but will avoid spurious changes
as files are edited.
Part of //go:build change (#41184).
See https://golang.org/design/draft-gobuild
Change-Id: I8020e066756b0cfa99ed1cb804b17e127e4151b9
Reviewed-on: https://go-review.googlesource.com/c/build/+/294417
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
Reviewed-by: Jason A. Donenfeld <Jason@zx2c4.com>
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
This adds a polling function that is tested and can be reused by
callers in the x/build repository. This was suggested in the review of
CL 247907. The polling function replaces the pollers used in the EC2 buildlet pool.
For golang/go#36841
Change-Id: I120ceb83e2740f0bdc5ee2423e0edd3ad727bf4b
Reviewed-on: https://go-review.googlesource.com/c/build/+/255358
Trust: Carlos Amedee <carlos@golang.org>
Run-TryBot: Carlos Amedee <carlos@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Alexander Rakoczy <alex@golang.org>
Before destroying an untracked EC2 instance, verify that it is a
buildlet. Non buildlets will be destroyed manually as they are most
likely instances created while an operator is debugging a problem.
For golang/go#36841
Change-Id: Id0f65192ce4e72f9ecc495dd3e94750724091b51
Reviewed-on: https://go-review.googlesource.com/c/build/+/249121
Run-TryBot: Carlos Amedee <carlos@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
The EC2 buildlet pool added by this commit will manage the lifecycle
of buildlets running on EC2. EC2 VMs will only be created for the
ARM64 architecture. As VMs are requested, the pool will ensure that
there are enough resources for the VM to be created and keep track of
the VMs created. Once a VM is destroyed, the resources will be made
available for other VMs. This pool will only keep instances as they
are needed.
Updates golang/go#36841
Updates golang/go#38337
Change-Id: Ic777485c0b0a69ec13726c58b49e9fdc1df4777e
Reviewed-on: https://go-review.googlesource.com/c/build/+/247907
Run-TryBot: Carlos Amedee <carlos@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Alexander Rakoczy <alex@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
This change adds a ledger to the coordinator pool package. The ledger
is used to store a record of EC2 instance types created and their
attributes. It also keeps a record of resource usage for EC2 instances
in order to ensure that we do not exceed the resource limits for the account.
Updates golang/go#36841
Change-Id: I6a0afdb31c8e3a83634e7c1fc8b2b733b7a50c01
Reviewed-on: https://go-review.googlesource.com/c/build/+/247906
Run-TryBot: Carlos Amedee <carlos@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Set gceMode earlier in InitGCE. Its value is used by some of the code
that runs inside InitGCE.
Don't try to run gcePool.pollQuotaLoop in dev mode. Make the code more
clear and consistent about this and createBasepinDisks calls.
Merge oAuthHTTPClient into the "Initialized by InitGCE" var block above.
Remove initGCECalled, it has become unused.
For golang/go#34744.
For golang/go#38337.
Change-Id: Ic47870fa9aed0eded0cfdf14e18e63c1acda4554
Reviewed-on: https://go-review.googlesource.com/c/build/+/244398
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Reviewed-by: Alexander Rakoczy <alex@golang.org>
The logic for naming buildlets appears in each particular buildlet
pool implementation. This change creates a function which contains
the buildlet naming logic. This is being added before the addition of
a new buildlet pool.
Updates golang/go#36841
Updates golang/go#38337
Change-Id: I8c03f9b513efde14414bcc6d823f3cf1a59c8f70
Reviewed-on: https://go-review.googlesource.com/c/build/+/243337
Run-TryBot: Carlos Amedee <carlos@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Andrew Bonventre <andybons@golang.org>
Reviewed-by: Alexander Rakoczy <alex@golang.org>
This is a set in a series of steps which will move everything buildlet
pool related into a pool package.
Updates golang/go#36841
Updates golang/go#38337
Change-Id: Ic7a0ccd7838345036df2e72b13084070541cb63c
Reviewed-on: https://go-review.googlesource.com/c/build/+/227769
Run-TryBot: Carlos Amedee <carlos@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Alexander Rakoczy <alex@golang.org>
This is a set in a series of steps which will move everything buildlet
pool related into a pool package.
Updates golang/go#36841
Change-Id: I8efb1f94c7b929be559004d9f455bca0370c7800
Reviewed-on: https://go-review.googlesource.com/c/build/+/227768
Run-TryBot: Carlos Amedee <carlos@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Alexander Rakoczy <alex@golang.org>
This CL creates the internal/coordinator/pool package intended to
contain all buildlet pool implementations. In order to keep this
change small and carefully discover where the interactions are
between the gce buildlet pool and the rest of the coordinator
are, this change only moves the gce buildlet over to the new
package.
The next steps will be to move the rest of the buildlet pools
over to this package. After that we will restructure the
implementations themselves in order to increase test coverage
and increase the ease of testing.
Updates golang/go#36841
Updates golang/go#38337
Change-Id: If82ae1b584bd77c697aa84fadf9011c9e79fa409
Reviewed-on: https://go-review.googlesource.com/c/build/+/227141
Run-TryBot: Carlos Amedee <carlos@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Alexander Rakoczy <alex@golang.org>
Carlos Amedee
Dmitri Shuralyov
Russ Cox
cuishuang
Michael Pratt
Alexander Rakoczy
Heschi Kreinick