Граф коммитов

39 Коммитов

Автор SHA1 Сообщение Дата
Oleksandr Redko f03e733dd2 all: correct typo and grammar issues in comments
Changes applied only to Go comments.

Change-Id: I0019ebe895186aed16daf252d7817c5e095c4b94
Reviewed-on: https://go-review.googlesource.com/c/build/+/469456
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Run-TryBot: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-02-21 15:14:29 +00:00
cui fliter 459bd2670d all: fix some comments
Change-Id: I15769c4302e8dd1d649e0eaa9b16099f0eda603b
Reviewed-on: https://go-review.googlesource.com/c/build/+/464239
Run-TryBot: Ian Lance Taylor <iant@google.com>
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
2023-01-31 14:57:29 +00:00
Carlos Amedee e8c8ca12ab all: add license headers
This change adds license headers to files which are missing them.
A modified version of the addlicense tool was used to add these headers:
https://github.com/google/addlicense

The command issued was:
addlicense -c "Go Authors" -l bsd *

Change-Id: Id6fd17c587f99b3ff0ef5f5f35befcfe5aaeef72
Reviewed-on: https://go-review.googlesource.com/c/build/+/409614
Run-TryBot: Carlos Amedee <carlos@golang.org>
Auto-Submit: Carlos Amedee <carlos@golang.org>
Reviewed-by: Carlos Amedee <carlos@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Alex Rakoczy <alex@golang.org>
2022-06-01 18:19:15 +00:00
Heschi Kreinick d4dc690249 kubernetes/gke: skip tests unless specifically requested
Just running on GCE is not enough for them to work.

Updates golang/go#28543.

Change-Id: I79a0702f9c2dfaf256a872557836258ba2ab4d0d
Reviewed-on: https://go-review.googlesource.com/c/build/+/408956
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Run-TryBot: Heschi Kreinick <heschi@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-05-27 22:03:25 +00:00
Russ Cox 993f2c6f22 all: gofmt
Gofmt to update doc comments to the new formatting.

For golang/go#51082.

Change-Id: Ie063c2fd890641b551c0f304fee5fa19121f1b38
Reviewed-on: https://go-review.googlesource.com/c/build/+/399555
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Russ Cox <rsc@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
2022-04-12 02:01:23 +00:00
Heschi Kreinick 35ff7b4a63 cmd/coordinator: modernize Kubernetes access
Turns out that to query for a regional Kubernetes cluster you need to
use a different API. Require an explicit location to create a client and
use that new API.

Also, modern Kubernetes clusters won't have client cert auth enabled.
Use OAuth instead.

Finally, don't require that the buildlets cluster be in the same zone as
the coordinator.

For golang/go#48408.

Change-Id: Ic3f9525b9bffa89d779e684c8ea1be116d3f983f
Reviewed-on: https://go-review.googlesource.com/c/build/+/350754
Trust: Heschi Kreinick <heschi@google.com>
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Alexander Rakoczy <alex@golang.org>
2021-09-17 21:59:33 +00:00
Heschi Kreinick a9d7de15d2 all: use Workload Identity for gitmirror, move to "prod" namespace
Rather than using a service account key, enable Workload Identity for
gitmirror. This involved moving into a non-default namespace, so the
coordinator's monitoring logic needs updating too.

Change-Id: I196fae0825f806248571aae6fb125d8c0917934d
Reviewed-on: https://go-review.googlesource.com/c/build/+/347410
Trust: Heschi Kreinick <heschi@google.com>
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Alexander Rakoczy <alex@golang.org>
Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-09-02 21:43:28 +00:00
Julie Qiu a56ae9428e all: add or change GoDoc badge to pkg.go.dev with update-readmes.go
update-readmes.go is updated to add a pkg.go.dev badge instead of a
godoc.org badge.

The GoDoc badge is added or changed in all READMEs by running
update-readmes.go

For golang/go#42251

Change-Id: I0c49df26a6c695f938c98c9a7b26325d991b3cfb
Reviewed-on: https://go-review.googlesource.com/c/build/+/276032
Trust: Julie Qiu <julie@golang.org>
Run-TryBot: Julie Qiu <julie@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2020-12-15 04:52:31 +00:00
Dmitri Shuralyov d3fb66653c kubernetes/gke: skip 4 tests on builders
The current GKE tests require to be run on GCE and with Application
Default Credentials that have at least the container.clusters.list
permission, at least one GKE cluster, and possibly more.

The builders that run these tests don't have sufficient permissions,
which means these tests never pass. Skip these tests, they can be
re-enabled if/when we decide they're worth running automatically and
make it possible to do so. Until then, they can be run manually.

Updates golang/go#28543
Updates golang/go#11811

Change-Id: Ib76f9d4f93ece2b922c099a21dec4ceefb45b546
Reviewed-on: https://go-review.googlesource.com/c/build/+/215299
Reviewed-by: Alexander Rakoczy <alex@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Alexander Rakoczy <alex@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2020-01-17 21:25:17 +00:00
Brad Fitzpatrick 366373d709 kubernetes/gke: skip API tests in short mode
Updates golang/go#30612

Change-Id: I752c57f8631025f1784951e701340d2c21ee277c
Reviewed-on: https://go-review.googlesource.com/c/build/+/165777
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-03-06 16:35:24 +00:00
Brad Fitzpatrick a6dd626c4c all: use "context" instead of golang.org/x/net/context
Change-Id: I106c81a16eb5b39ff38c6f896095a27f597b3f8d
Reviewed-on: https://go-review.googlesource.com/99016
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2018-03-06 23:26:52 +00:00
Brad Fitzpatrick 73f88a6d4c all: add README.md files where missing, and tool to keep them updated
Change-Id: I385171c415bf168c04c6c3a7a996bff88964af84
Reviewed-on: https://go-review.googlesource.com/52856
Reviewed-by: Andrew Bonventre <andybons@golang.org>
2017-08-02 22:17:52 +00:00
Kevin Burke 674c8cf7e8 kubernetes/gke: skip GKE tests if no metadata present
Currently the x/build test suite fails on Travis CI, since Travis
runs on GKE but doesn't have any of the correct metadata set up. We
should skip these tests in that environment like we do when we are not
running on GKE.

Change-Id: Ib99ad1e9f33ce9f05aaf690653ab1bf23718ce2b
Reviewed-on: https://go-review.googlesource.com/37942
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2017-03-09 21:06:35 +00:00
Brad Fitzpatrick c5562d003a kubernetes: fix, test GetNodes
Broken in my cleanup golang.org/cl/36692

Change-Id: I5f82314b8ff703cf2c894771fbc0eb16834fc47b
Reviewed-on: https://go-review.googlesource.com/37216
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2017-02-17 22:01:21 +00:00
Kevin Burke b7a944e366 all: fix vet errors
It would be nice to run these alongside the tests. There is only one
vet error left in the x/build tree, an unreachable return next to a
TODO.

Change-Id: Ic31d96cdee5bb0f6b9795237ff3b743a4b76e3cf
Reviewed-on: https://go-review.googlesource.com/36898
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2017-02-12 17:35:52 +00:00
Brad Fitzpatrick 7449a82969 kubernetes: fix typos
From golang.org/cl/36692

Change-Id: Icc63edd28b687cf30c222a3125371ad912bc6da7
Reviewed-on: https://go-review.googlesource.com/36802
Reviewed-by: Kevin Burke <kev@inburke.com>
Reviewed-by: Matt Layher <mdlayher@gmail.com>
2017-02-11 20:05:12 +00:00
Brad Fitzpatrick b30f5067b7 kubernetes: cleanup, and add DialServicePort
And moves Dial methods to *kubernetes.Client, instead of a separate
Dialer type.

Updates golang/go#18817

Change-Id: I045ac48441b9139cb0e01ceb4969b29cf5e72507
Reviewed-on: https://go-review.googlesource.com/36692
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2017-02-09 23:20:38 +00:00
Brad Fitzpatrick adc161a243 Add gke package, add kubenetes.Dialer type.
Updates golang/go#18817

Change-Id: Ifee53384486b0692899b77be2eaa42ca9006ef8e
Reviewed-on: https://go-review.googlesource.com/36016
Reviewed-by: Chris Broadfoot <cbro@golang.org>
2017-01-31 21:33:25 +00:00
shawnps 72ebb1e3e7 all: fix typos
Change-Id: I16c20d6eb746a3ec81021c2a367d74c258437019
Reviewed-on: https://go-review.googlesource.com/34921
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2017-01-07 18:01:14 +00:00
Brad Fitzpatrick 71265acedb all: adjust things for upgrade from GKE 1.2 to GKE 1.4
We hit GKE bugs and changes when upgrading from GKE 1.2 to 1.4.

The main issue is that Kubernetes does't reserve CPU or memory for
itself on nodes, so things were OOMing and getting killed. And when
Docker or Kubernetes got killed themselves, they were wedging and not
recovering.

So we're going to run a daemonset (POD on all nodes) to reserve space
for Kubernetes for it. That's not in this CL.

But this CL got us limping along and was already in production. It
doubles resource RAM usage for jobs, so fewer things schedule per node.
While we're at it, let jobs use more CPU if it's available.

Also, disable auto-scaling. It was off before by hand. Force it off
programatically too. And make the node count 5, like it was by hand.

Also, force un-graceful pod deletes, since GKE 1.3 or something
introduced a graceful-vs-ungraceful distinction, which we weren't
handling previously and therefore pods never were being deleted.

Change-Id: I3606e4e2e92c496d8194503d510921bd1614d34e
Reviewed-on: https://go-review.googlesource.com/33490
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2016-12-01 23:48:21 +00:00
Johan Euphrosine ce69a9ff6d kubernetes: add basic tests
Change-Id: Ifa4283bc60717229ccf858f5ff7f4119ccfb28d6
Reviewed-on: https://go-review.googlesource.com/25164
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2016-08-05 01:16:14 +00:00
Brad Fitzpatrick 304492cf2e kuberentes: roll back recent Kubernetes client changes
https://golang.org/cl/21765 was not tested and not adequately reviewed
and doesn't work at all. Roll it back for now, since the blind fix in
https://golang.org/cl/21765 helped a bit but wasn't enough.

Change-Id: I781fac04d019fa5e6120ec398e13671b89d802c5
Reviewed-on: https://go-review.googlesource.com/25043
Reviewed-by: Johan Euphrosine <proppy@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2016-07-19 02:49:29 +00:00
Brad Fitzpatrick 49a940a217 kubernetes: fix response status code checking
Fixes bugs from https://golang.org/cl/21765

Change-Id: Ifb68b3a6e8b18d5ce61e8fc01b010cc55f3afa00
Reviewed-on: https://go-review.googlesource.com/25042
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2016-07-19 02:02:17 +00:00
Johan Euphrosine f46fbc254a build/kubernetes: add secret API methods
- add Get and Create secret
- refactor client
- fix example

Change-Id: Ia5c6f17e86cb01aa8093ec058630b40ee61fa74f
Reviewed-on: https://go-review.googlesource.com/21765
Reviewed-by: Evan Brown <evanbrown@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2016-07-01 18:31:07 +00:00
Andrew Gerrand f88094097f kubernetes/api: change import path for inf package
The author moved their repository.

Change-Id: I9944c666aa772651c99cecb91b92e5ab7fce3e8b
Reviewed-on: https://go-review.googlesource.com/23676
Reviewed-by: Andrew Gerrand <adg@golang.org>
2016-06-06 01:38:33 +00:00
Brad Fitzpatrick 4a3f4a8c16 kubernetes: avoid the WatchPod streaming API
Poll every 5 seconds instead.

Change-Id: I0a8a30fee1118b2aecf4b3cbbf342e07afc27602
Reviewed-on: https://go-review.googlesource.com/22820
Reviewed-by: Evan Brown <evanbrown@google.com>
2016-05-05 20:21:37 +00:00
Brad Fitzpatrick c6ced0a944 kubernetes: be more defensive in WatchPod
Change-Id: I0fe9bfb9f0a440a33ff8eedf00940f53b2b7dc87
Reviewed-on: https://go-review.googlesource.com/22789
Reviewed-by: Evan Brown <evanbrown@google.com>
2016-05-04 23:35:15 +00:00
Brad Fitzpatrick 90f7179840 kubernetes: fix potential goroutine leak
Change-Id: I2e9343c7441db43fe7004d877ee0409d29cb9a23
Reviewed-on: https://go-review.googlesource.com/21764
Reviewed-by: Evan Brown <evanbrown@google.com>
2016-04-08 19:09:22 +00:00
Evan Brown 34ff1d9bc8 all: kubernetes builder autoscaling
Improvements to support rapid scheduling of many build jobs:

- Retry logic in Kubernetes client to handle sporadic connection
  closes from their API server under heavy load

- Cluster autoscaler scales on default CPU utilization metric

- Debug mode allows scheduling multiple builds to test scaling

- Account for scheduled vs. provisioned resources in a cluster
  and use that information to estimate when a build's pod
  will be scheduled and in running state

- Use estimated scheduled time to set context timeout

- Track pod lifecycle (requested time, estimated available time,
  actual available time, terminate time, etc)

Change-Id: I14d6c5e01af0970dbb3390a29d1ee5c43049fff8
Reviewed-on: https://go-review.googlesource.com/19524
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2016-04-07 17:22:57 +00:00
Evan Brown cfb4f34505 kubernetes: improve error message if pod create fails
Updates golang/go#14112

Change-Id: I53e42647e5a8b22cfb70b78fc4571ff78dd23289
Reviewed-on: https://go-review.googlesource.com/18984
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2016-01-27 22:43:28 +00:00
Brad Fitzpatrick 5f62335c08 cmd/coordinator: fix two crashes
Change-Id: I5ad75b5cd92ba4199642fb2650c7891bbecbed70
Reviewed-on: https://go-review.googlesource.com/17131
Reviewed-by: Evan Brown <evanbrown@google.com>
2015-12-03 18:57:28 +00:00
Evan Brown 2e452e1be8 all: improve Kubernetes API client
* pod creation returns an *api.Pod
* WatchPod replaces WatchPodStatus

Updates golang/go#12546

Change-Id: I34bb6e0d994e552b41a8082cc4672a663ce961a3
Reviewed-on: https://go-review.googlesource.com/17100
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2015-11-20 17:56:59 +00:00
Evan Brown 83f9748046 all: display pool status and delete failed/old pods
* status page shows kube pool details
* pods created by the coorindator are tracked
* pods that fail to create are deleted
* pods older than delete-at are deleted
* pods created by a different coordinator are deleted

Updates golang/go#12546

Change-Id: I4c4f8ff906962b4a014a66d0a9d490ff17710d62
Reviewed-on: https://go-review.googlesource.com/16101
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2015-11-05 11:33:24 +00:00
Matthew Dempsky d5f422f474 all: fix vet warnings
Change-Id: Ia4519d9b050136b4d46dc8361f546e3c4205a9cd
Reviewed-on: https://go-review.googlesource.com/16288
Reviewed-by: Dave Cheney <dave@cheney.net>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2015-10-25 04:14:39 +00:00
Evan Brown 956434c23a all: improved monitoring of buildlet pods after creation
* Replaced cancel with context.Context
* StartPod can be canceled
* Wait for buildlet to come online, but fail fast if pod fails first
* Support timeout waiting for pod to leave pending phase
* Use Kubernetes watch API (long poll)

Updates golang/go#12546

Change-Id: I792a3b8fed615362a0290feee7de0c2cefe43c0e
Reviewed-on: https://go-review.googlesource.com/15285
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2015-10-15 23:45:50 +00:00
Evan Brown c51d4e0590 all: more work on Kubernetes buildlet pool
Updates golang/go#12546

Change-Id: Iac31c237fed7973ec2bebace5248904fe3e3692d
Reviewed-on: https://go-review.googlesource.com/14396
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2015-09-08 23:10:58 +00:00
Brad Fitzpatrick 1f41f3c1a6 kubernetes/api: new package with API type definitions only
This is a copy of Kubernetes' API types.go, with a million some
unnecessary dependencies removed.

Change-Id: I3b79256f37b2d022c8307afdc40782caeb5a560c
Reviewed-on: https://go-review.googlesource.com/14390
Reviewed-by: Evan Brown <evanbrown@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2015-09-08 21:50:55 +00:00
Evan Brown 94367e21d9 kubernetes: update to v1 API
Change-Id: Ia3fe94e944ac80a58eadaa13487a2e14494600d4
Reviewed-on: https://go-review.googlesource.com/14384
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2015-09-08 18:43:58 +00:00
Johan Euphrosine 093044aa43 kubernetes: add minimal kubernetes API client
Change-Id: I4ca7c27084d5f80f116b8461d0e6769b8bd99ceb
Reviewed-on: https://go-review.googlesource.com/9981
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2015-06-12 14:50:32 +00:00