Граф коммитов

1371 Коммитов

Автор SHA1 Сообщение Дата
Julie Qiu 7c7e97c9cc cmd/gopherbot: label x/vuln issues
Automatically label all x/vuln issues with the label "x/vuln".

Change-Id: I4f0c6faeddd1a9da410d07cb91c6e6cc0eaaf765
Reviewed-on: https://go-review.googlesource.com/c/build/+/360715
Trust: Julie Qiu <julie@golang.org>
Run-TryBot: Julie Qiu <julie@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2021-11-02 20:17:36 +00:00
Dmitri Shuralyov 4e92a35446 cmd/releasebot: add modes for making a release tweet programmatically
This allows exercising (and benefiting from improvements offered by)
the programmatic tweet creation tasks as soon as the next minor release.
The same task will be added and used by relui when possible.

For golang/go#40279.
Updates golang/go#47403.

Change-Id: I8b47f86489e753bea6a767990ff98dd525e668df
Reviewed-on: https://go-review.googlesource.com/c/build/+/358899
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Dmitri Shuralyov <dmitshur@golang.org>
Trust: Alexander Rakoczy <alex@golang.org>
Reviewed-by: Alexander Rakoczy <alex@golang.org>
2021-11-02 15:09:53 +00:00
Dmitri Shuralyov 46f6bfb6da internal/task: add task to tweet about Go releases
This CL adds an internal API for making release tweets of various kinds,
including the templates used to generate the tweet text and tweet image.

It does not have code for using the Twitter API. CL 358898 will do that.

For golang/go#40279.
For golang/go#47403.

Change-Id: Ic0cb2f5f1e59c0fd932aed7c5c4a88decc5a3d8d
Reviewed-on: https://go-review.googlesource.com/c/build/+/358897
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Dmitri Shuralyov <dmitshur@golang.org>
Trust: Alexander Rakoczy <alex@golang.org>
Reviewed-by: Alexander Rakoczy <alex@golang.org>
Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-11-02 15:00:47 +00:00
Bryan C. Mills 6d1c239cf4 cmd/buildlet: override the last PATH entry, not the first
If both the process environment and the "env" post field included
entries for "PATH", the setPathEnv helper function would erroneously
use the first one instead of the last. (In current versions of Go,
os/exec always uses the last entry for each variable.)

This change removes much of the complexity of setPathEnv, pushing some
of the less obvious parts of the logic (skipping empty slices and
suppressing no-op changes) to the caller side.

For golang/go#31567

Change-Id: I7460bf65c61073a71f0ea11d4d69a16f3e9b7c16
Reviewed-on: https://go-review.googlesource.com/c/build/+/354754
Trust: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2021-11-01 18:49:42 +00:00
Heschi Kreinick 79bb74b4be all: disable unnecessary serving methods
We only need self-signed HTTPS on many services now.

For golang/go#49191.

Change-Id: I523a98b738f9cca7aeba57f7f6f66c199d99b788
Reviewed-on: https://go-review.googlesource.com/c/build/+/359480
Trust: Heschi Kreinick <heschi@google.com>
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-11-01 17:11:18 +00:00
Heschi Kreinick e6783b0bbe cmd/coordinator,cmd/gerritbot: use HTTP/2 between LB and app
Also disassociate LE certs we don't need any more.

For golang/go#49191.

Change-Id: I74acf2f2f52fbf91670d27d91112136450f81944
Reviewed-on: https://go-review.googlesource.com/c/build/+/359479
Trust: Heschi Kreinick <heschi@google.com>
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-11-01 17:11:14 +00:00
Heschi Kreinick 109ac0ba59 devapp,maintnerd: use HTTP/2 between LB and app
Enable self-signed HTTPS, and configure the Service and Ingress to do
HTTP/2 over that port.

Also fix various silly mistakes and typos that I discovered while
rolling out the previous change.

For #49191.

Change-Id: If4f308d0e79a94a480ba97b27b5a503cf2aeff2e
Reviewed-on: https://go-review.googlesource.com/c/build/+/358914
Trust: Heschi Kreinick <heschi@google.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Reviewed-by: Alexander Rakoczy <alex@golang.org>
2021-10-27 20:00:55 +00:00
Heschi Kreinick db77efa219 all: unify HTTP/HTTPS serving
We're gearing up to put most everything behind Cloud load balancers, at
which point Autocert serving will be unnecessary. To support HTTP/2 we
still need to serve TLS, but we can do it using a self-signed
certificate.

Unify the configuration of all that stuff behind a single set of flags
and clean up all the copied and tweaked code I could find.

The self-signed serving is dormant for now; I'll activate it in a later
change.

I didn't understand the need to tweak keepalive intervals everywhere;
keepalives have been on by default since 1.13. So I dropped them.

Change-Id: I0016211f1032a6c64375aad9e34e4e2e62861c5b
Reviewed-on: https://go-review.googlesource.com/c/build/+/358654
Trust: Heschi Kreinick <heschi@google.com>
Trust: Alexander Rakoczy <alex@golang.org>
Reviewed-by: Alexander Rakoczy <alex@golang.org>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Run-TryBot: Alexander Rakoczy <alex@golang.org>
2021-10-27 16:35:18 +00:00
Heschi Kreinick 9c47891a04 all: prepare to migrate to Ingress
Following:
https://cloud.google.com/kubernetes-engine/docs/concepts/ingress
https://cloud.google.com/kubernetes-engine/docs/how-to/managed-certs

Create an Ingress with HTTPS redirects enabled.

For each of build, dev, gerritbot, maintner:
 - Add three certs: the new Google-managed cert, plus the two old LE certs.
 - Add an ingress rule for the host, and a new internal Service it maps to.
 - Disable our own HTTPS redirects, since the ingress terminates HTTPS.
   They will be accessible via HTTP for the duration of the migration;
   they don't serve anything sensitive anyway.

dev-test.golang.org is already rolled out using only a Google-managed
cert, so you can try HTTP and HTTPS directly there. I've also verified
that the LE certs are served:

curl -v --resolve dev.golang.org:443:34.149.223.5 https://dev.golang.org/

Change-Id: I20f3c92ffb00dbd9066822f748508013b68d5483
Reviewed-on: https://go-review.googlesource.com/c/build/+/355670
Trust: Heschi Kreinick <heschi@google.com>
Trust: Alexander Rakoczy <alex@golang.org>
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Alexander Rakoczy <alex@golang.org>
Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-10-21 19:14:11 +00:00
Alexander Rakoczy 19d8baf2fd cmd/relui,internal/relui: resume workflows
Resume all unfinished workflows on relui startup. Moves most workflow
running logic to a new Worker type.

This also adds a couple new columns to the workflow to record output and
whether a workflow has completed. In order to actually resume workflows,
we'll have to either add functionality to unmark them as "finished"
(restarting manually, which seems reasonable), or not marking a workflow
finished if it terminates with context.Cancelled.

For golang/go#47401

Change-Id: I3e0ed021d7a47fb125f1034df83dc3c6d95887f8
Reviewed-on: https://go-review.googlesource.com/c/build/+/353553
Trust: Alexander Rakoczy <alex@golang.org>
Run-TryBot: Alexander Rakoczy <alex@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
2021-10-20 19:39:46 +00:00
Michael Pratt 3025285794 cmd/coordinator: find work in dev mode
Start the main findWorkLoop in dev mode, to discover work to run from
the dashboard. In dev mode, was also replace the linux-amd64 builder
config with one using host-linux-amd64-localdev reverse buildlets.

This provides a complete lifecycle to test out builds with a local dev
coordinator and buildlet.

For golang/go#48803

Change-Id: I8ade6c8bccf3bc51437ca9e7d11c232753fe7465
Reviewed-on: https://go-review.googlesource.com/c/build/+/354638
Trust: Michael Pratt <mpratt@google.com>
Run-TryBot: Michael Pratt <mpratt@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Alexander Rakoczy <alex@golang.org>
2021-10-15 19:50:59 +00:00
Michael Pratt d59bfcb8a8 cmd/coordinator: make listen address configurable
Though it is a good default, it is cumbersome for dev mode to listen
only on localhost when developing over SSH. Add a -listen flag to allow
overriding this with any address.

For golang/go#48803

Change-Id: If01a0b44926a33f2aa01548319508166592936e3
Reviewed-on: https://go-review.googlesource.com/c/build/+/354637
Trust: Michael Pratt <mpratt@google.com>
Run-TryBot: Michael Pratt <mpratt@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Alexander Rakoczy <alex@golang.org>
2021-10-15 19:50:53 +00:00
Michael Pratt d5abf980d5 all: handle revdial redirects on connect
In dev mode, the coordinator uses hostPathHandler, which redirects
/reverse and /revdial to /farmer.golang.org/reverse, etc.

Establishing a revdial connection chokes on this redirect, as there it
expects to complete the protocol switch in a single request.

Add rudimentary redirect support via a helper so this works in dev mode.

Note that linkRewriter must implement http.Hijacker as
revdial.ConnHandler type-asserts the http.ResponseWriter to a Hijacker.

For golang/go#48803

Change-Id: I191fa6ff17bbd334203430f3c1f2c5e03db407ff
Reviewed-on: https://go-review.googlesource.com/c/build/+/354630
Trust: Michael Pratt <mpratt@google.com>
Run-TryBot: Michael Pratt <mpratt@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Alexander Rakoczy <alex@golang.org>
2021-10-15 19:50:46 +00:00
Michael Pratt d4249f0024 cmd/buildlet: add revdial test
This replaces the broken test in internal/coordinator/pool.

For golang/go#48803

Change-Id: I7bd9265bba555562ffa7d59169a9c8792ed97d3c
Reviewed-on: https://go-review.googlesource.com/c/build/+/354629
Trust: Michael Pratt <mpratt@google.com>
Run-TryBot: Michael Pratt <mpratt@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Alexander Rakoczy <alex@golang.org>
2021-10-15 19:50:15 +00:00
Dmitri Shuralyov 5747e451b3 cmd/releasebot: create GitHub milestones for next release
The current release process relies on humans to create GitHub milestones
for future Go releases. releasebot includes automated checks that detect
when that step is forgotten. CL 294249 included one of those checks, and
its commit message mentioned:

	(Future release process improvements may include automatically making
	the milestone. That is better suited to be in scope of golang/go#40279.)

This is the release process improvement that automates making milestones.

For golang/go#40279.

Change-Id: I8e02aff6714a5cf2de4ee4bcfe98aaf68abb0cd4
Reviewed-on: https://go-review.googlesource.com/c/build/+/354758
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Reviewed-by: Alexander Rakoczy <alex@golang.org>
Trust: Dmitri Shuralyov <dmitshur@golang.org>
2021-10-12 17:24:08 +00:00
Dmitri Shuralyov 70d1a997b4 cmd/buildlet: add Makefile target for darwin/arm64
This target updates the buildlet binary used by builders that
test the darwin/arm64 port (i.e., darwin-arm64-11_0-toothrot).

Updates golang/go#39782.

Change-Id: I9247c822a7a103bcc016e81664c67b2fbaa32e03
Reviewed-on: https://go-review.googlesource.com/c/build/+/355312
Trust: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Reviewed-by: Alexander Rakoczy <alex@golang.org>
2021-10-12 16:49:48 +00:00
Michael Pratt ec8f347677 cmd/coordinator,dashboard: drop subrepo GOPATH testing support
All of the subrepos we test use modules, so GOPATH test mode here is
unused and can be deleted.

Fixes golang/go#48875

Change-Id: Id5143e9fcad7fc91e8e0f63dcca35f5661be5a03
Reviewed-on: https://go-review.googlesource.com/c/build/+/355089
Trust: Michael Pratt <mpratt@google.com>
Run-TryBot: Michael Pratt <mpratt@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2021-10-11 17:33:26 +00:00
Bryan C. Mills ecfecf7bea all: consistently set PWD to match Dir for subprocesses
Also consistently deduplicate Env entries, mainly to reduce confusion
when reading logs.

This change is probably larger than what is strictly needed to fix the
issue, but it seemed simpler than trying to figure out which of the
many calls to exec.Command in the module are actually relevant.
(It also provides some useful case studies for possible additions to
the Go standard library.)

For golang/go#33598

Change-Id: Ia2bce4549e4a71b56fb497e3df093f79fbcf7f29
Reviewed-on: https://go-review.googlesource.com/c/build/+/353549
Trust: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2021-10-08 15:15:56 +00:00
Bryan C. Mills c2503f1d9a cmd/fetchlogs: link each subrepo Go revision to a unique subdirectory
Prior to this change, we would link only one (arbitrary) Go revision
for each subrepo revision, causing greplogs to under-report subrepo
failures.

Change-Id: Id36360005283e85d2c0077958d5839901db921f8
Reviewed-on: https://go-review.googlesource.com/c/build/+/340170
Trust: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Robert Findley <rfindley@google.com>
2021-10-07 20:03:34 +00:00
Michael Pratt 6bc77f1bbc cmd/coordinator: drop special revdial package import name
This is the last remaining revdialv2 reference, missed by
https://golang.org/cl/208598.

Change-Id: I2ebf774b0e7c1e8d34e40b9768f8924248f077f0
Reviewed-on: https://go-review.googlesource.com/c/build/+/354290
Trust: Michael Pratt <mpratt@google.com>
Run-TryBot: Michael Pratt <mpratt@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2021-10-07 15:27:17 +00:00
Michael Pratt 86303821b7 cmd/coordinator: don't snapshot in dev mode
In dev mode, we have no GCS storage client, so attempting to write a
snapshot will panic.

For golang/go#48803

Change-Id: Id37264ebd765f914a55acf2fd18274020850331f
Reviewed-on: https://go-review.googlesource.com/c/build/+/353909
Trust: Michael Pratt <mpratt@google.com>
Run-TryBot: Michael Pratt <mpratt@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2021-10-06 16:08:49 +00:00
Alexander Rakoczy 1409edded2 cmd/relui: use migrations for sqlc schema
Use migrations instead of maintaining a schema.sql. Having a schema dump
can be useful, but we can instead go with our migrations to avoid
confusion.

For golang/go#47401

Change-Id: I426ffd46e17d3aa381d5bc0d517c8529961702f0
Reviewed-on: https://go-review.googlesource.com/c/build/+/353552
Trust: Alexander Rakoczy <alex@golang.org>
Run-TryBot: Alexander Rakoczy <alex@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2021-10-04 20:04:57 +00:00
Alexander Rakoczy 80b982a6bd cmd/relui: add migrate-down-up flag
Adds a migrate-down-up flag that re-runs the down and up steps of the
last migration. This is useful when developing migrations, making it
easier to make changes.

For golang/go#47401

Change-Id: I21369bc6792220a02641109edf0f628f9dc18981
Reviewed-on: https://go-review.googlesource.com/c/build/+/353409
Trust: Alexander Rakoczy <alex@golang.org>
Run-TryBot: Alexander Rakoczy <alex@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
2021-10-01 05:16:33 +00:00
Alexander Rakoczy 71603fa585 internal/relui: persist logs to database
Save log output from workflow tasks in the database. Render logs in the
UI alongside each task. This will almost certainly be a performance
problem if we have many logs until we implement pagination.

For golang/go#47401

Change-Id: I4ddc3d7845e4559cc636b3b7972e0adefe9fcec4
Reviewed-on: https://go-review.googlesource.com/c/build/+/353170
Trust: Alexander Rakoczy <alex@golang.org>
Run-TryBot: Alexander Rakoczy <alex@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
2021-09-30 18:32:41 +00:00
Alexander Rakoczy 7959cfa261 cmd/relui: document database migrations
Add some missing documentation about creating and running database
migrations.

For golang/go#47401

Change-Id: I42ca1af43b7097c7913cb3c1ae40c70425ae8977
Reviewed-on: https://go-review.googlesource.com/c/build/+/353229
Trust: Alexander Rakoczy <alex@golang.org>
Run-TryBot: Alexander Rakoczy <alex@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
2021-09-30 15:46:36 +00:00
Dmitri Shuralyov a82df1ba57 internal/task, cmd/releasebot: add mail-dl-cl task to releasebot CLI
This change adds a release task that exercises the ability to create
a Gerrit CL via the Gerrit API, which will play a part in release
automation.

The task will eventually be exposed via relui's web UI when it is ready,
and it has a signature we expect to be able to use. To be able to use it
sooner, it's exposed via releasebot's CLI.

For golang.org/issue/38075.

Change-Id: I05ff078d7fcc25b1c2ac7001b094f3a3fee6b3a1
Reviewed-on: https://go-review.googlesource.com/c/build/+/350631
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Reviewed-by: Alexander Rakoczy <alex@golang.org>
Trust: Dmitri Shuralyov <dmitshur@golang.org>
2021-09-29 20:37:17 +00:00
Carlos Amedee c354d4cdcf cmd/coordinator/internal/metrics, cmd/coordinator: add GKE specific metrics config
This change modifies the monitored resource type used to report
metrics to the recommended k8s_container. This change was made because
a recent upgrade to the Kubernetes cluster the infrastructure is
deployed on modified the metadata endpoints available to the
coordinator.

Fixes golang/go#48581

Change-Id: Ie31d0f5f1a6abac53c44474191a248f11eeed5ea
Reviewed-on: https://go-review.googlesource.com/c/build/+/352895
Trust: Carlos Amedee <carlos@golang.org>
Run-TryBot: Carlos Amedee <carlos@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
2021-09-29 18:39:58 +00:00
Carlos Amedee 54222ee0cf cmd/coordinator: add metrics for gomote usage
This change introduces some metrics collection around gomote usage. It
records:
- gomote creates and the associated builder types.
- gomote ssh and the success of the call.
- gomote RDP.

Updates golang/go#47521
Fixes golang/go#48579

Change-Id: I5dfa04862254de0ceae747d0328918480d11db7c
Reviewed-on: https://go-review.googlesource.com/c/build/+/352809
Trust: Carlos Amedee <carlos@golang.org>
Trust: Dmitri Shuralyov <dmitshur@golang.org>
Run-TryBot: Carlos Amedee <carlos@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Alexander Rakoczy <alex@golang.org>
2021-09-29 01:47:27 +00:00
Carlos Amedee a5e20a6883 internal/secret, cmd/coordinator: move SSH private key to secret manager
This change moves the gomote SSH private key to GCP Secret Manager.

Fixes golang/go#48423

Change-Id: I5354fddd2069ef5a0b1c49fb2a889c8a183c6fb4
Reviewed-on: https://go-review.googlesource.com/c/build/+/352849
Trust: Carlos Amedee <carlos@golang.org>
Run-TryBot: Carlos Amedee <carlos@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Alexander Rakoczy <alex@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2021-09-28 22:20:26 +00:00
Alexander Rakoczy 4c1ebefe74 cmd/relui,internal/relui: cleanup go:build comments
These are no longer necessary as embed is in all supported verions of
Go.

For golang/go#47401

Change-Id: I3b128a28028d0ca15088d71202984c8b8ef4469c
Reviewed-on: https://go-review.googlesource.com/c/build/+/352569
Trust: Alexander Rakoczy <alex@golang.org>
Run-TryBot: Alexander Rakoczy <alex@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
2021-09-28 16:34:42 +00:00
Alexander Rakoczy cbb5112b70 cmd/relui,internal/relui: create and run workflows
Create and run workflows, storing state in a postgres database. Generate
database queries using sqlc, which allows the author to have complete
control over the query structure.

The basic structure of creating and running a workflow works, but
there is still much work to do in terms of selecting which workflow to
create, recording logs, maintaining a list of currently running
workflows, and recording workflow-level results.

Task persistence needs to be updated, as the current implementation
cannot be resumed due to losing the type during marshalling.

For golang/go#47401

Change-Id: I9ccddf117d023f70568de655c482820a1152d9cb
Reviewed-on: https://go-review.googlesource.com/c/build/+/350449
Trust: Alexander Rakoczy <alex@golang.org>
Run-TryBot: Alexander Rakoczy <alex@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
2021-09-28 15:24:53 +00:00
Heschi Kreinick 35ff7b4a63 cmd/coordinator: modernize Kubernetes access
Turns out that to query for a regional Kubernetes cluster you need to
use a different API. Require an explicit location to create a client and
use that new API.

Also, modern Kubernetes clusters won't have client cert auth enabled.
Use OAuth instead.

Finally, don't require that the buildlets cluster be in the same zone as
the coordinator.

For golang/go#48408.

Change-Id: Ic3f9525b9bffa89d779e684c8ea1be116d3f983f
Reviewed-on: https://go-review.googlesource.com/c/build/+/350754
Trust: Heschi Kreinick <heschi@google.com>
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Alexander Rakoczy <alex@golang.org>
2021-09-17 21:59:33 +00:00
Heschi Kreinick 93c2ea3f19 all: update resource requests
The new "services" cluster is stricter about resource requirements. I'm
not sure exactly why but I don't think it matters. Set ones that work.

In particular, anything that uses maintner needs a bunch of disk and
ram.

For golang/go#48408.

Change-Id: I5eacd22eb6ae177bd82faedb83b3489fafa98811
Reviewed-on: https://go-review.googlesource.com/c/build/+/350753
Trust: Heschi Kreinick <heschi@google.com>
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Alexander Rakoczy <alex@golang.org>
2021-09-17 21:59:01 +00:00
Dmitri Shuralyov 8a7ec5201b cmd/releasebot: improve "about to tag this commit, ok?" prompt UX
Accept "Y" in addition to "y", and don't print a verbose stack trace.
This change is based on past experience described in golang/go#36948.

Fixes golang/go#36948.

Change-Id: I1d79dc18ed66b174b84b2861d8baf9bb193a8f66
Reviewed-on: https://go-review.googlesource.com/c/build/+/350629
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
2021-09-17 19:47:14 +00:00
Alexander Rakoczy d0268397ad cmd/relui,internal/relui: add database migrations
This change adds golang-migrate to manage database migrations for relui.
relui will run all database migrations on start, and return an error for
any failures.

The current developer tooling is managed through the Makefile. For ease
of use, common tasks will be moved to a script in a future CL.

For golang/go#47401

Change-Id: I678c3b21d7e89d7d5a66305f66075f02cc2f3284
Reviewed-on: https://go-review.googlesource.com/c/build/+/349951
Trust: Alexander Rakoczy <alex@golang.org>
Run-TryBot: Alexander Rakoczy <alex@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
2021-09-17 16:15:38 +00:00
Xiaodong Liu 5eaa6e07e6 dashboard: add linux-loong64-3a5000 builder
Updates golang/go#46229

Change-Id: Ib48e1e64d96a07d5c5282d78935124faf75c89d7
Reviewed-on: https://go-review.googlesource.com/c/build/+/347309
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Cherry Mui <cherryyz@google.com>
2021-09-16 16:15:49 +00:00
Heschi Kreinick e4e0662004 all: remove node selector
For golang/go#48408.

Change-Id: I5709cb4278d8c754508d9f4bf76849592b6f9855
Reviewed-on: https://go-review.googlesource.com/c/build/+/350114
Trust: Heschi Kreinick <heschi@google.com>
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2021-09-15 20:30:55 +00:00
Heschi Kreinick 861218b9e3 cmd/gcpinit: delete
I noticed some unused-looking fields in the Environment type. gcpinit
looks dead to me, so delete it. Correct me if I'm wrong.

For golang/go#48408.

Change-Id: Ie0732177ada8c291f2bddc1b64a6b242a521d5bc
Reviewed-on: https://go-review.googlesource.com/c/build/+/350113
Trust: Heschi Kreinick <heschi@google.com>
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Alexander Rakoczy <alex@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2021-09-15 20:30:51 +00:00
Heschi Kreinick c88fdc01d5 all: move services to new Kubernetes cluster
Our makefiles install xb from the repository, so this has to be checked
in before I do anything else, and nobody else should deploy anything
until I'm done.

Because the new cluster is an Autopilot cluster, it lives in a region
(us-central1) and that's what you pass to the kubectl command, etc.
Move Region/Zone into the individual KubeConfigs and use the correct
ones as appropriate.

For golang/go#48408.

Change-Id: Iceacfe68305a3744aa87ce0fef777b977a252586
Reviewed-on: https://go-review.googlesource.com/c/build/+/350137
Trust: Heschi Kreinick <heschi@google.com>
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2021-09-15 20:30:45 +00:00
Heschi Kreinick fc41926a43 cmd/coordinator: move to Workload Identity
Create a new service account, and move the deployment over to the prod
namespace.

The network metadata entry we look for isn't available under Workload
Identity
(https://cloud.google.com/kubernetes-engine/docs/how-to/workload-identity#gke_mds)
so use the hostname instead, which works fine.

For golang/go#48263.

Change-Id: I91ef091de3e0a923b4a96c56a7f8e7c9e614be8f
Reviewed-on: https://go-review.googlesource.com/c/build/+/349950
Trust: Heschi Kreinick <heschi@google.com>
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2021-09-14 19:27:30 +00:00
Julie Qiu 1c53e0647f cmd/gopherbot: set pkgsite milestones
Automatically set milestones for pkgsite issues.

Change-Id: Ib736e33313abad7bfdb44d27cbbf62c33300944d
Reviewed-on: https://go-review.googlesource.com/c/build/+/348971
Trust: Julie Qiu <julie@golang.org>
Run-TryBot: Julie Qiu <julie@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2021-09-13 16:38:46 +00:00
Alexander Rakoczy 15e3ac06db cmd/coordinator: fix gomote ssh connection closing
gomote ssh connections immediately close with a "Connection closed"
error. The coordinator logs indicate an error in our call to pty.Start,
which is used to relay the tty between ssh connections. When the
coordinator was updated to Go 1.16 from Go 1.13 in July, it likely
caused this issue (see creack/pty#96).

Fixes golang/go#48329

Change-Id: Ieec5d2220c6c5ef4c53de594acbdd598a1bafb6d
Reviewed-on: https://go-review.googlesource.com/c/build/+/349092
Trust: Alexander Rakoczy <alex@golang.org>
Run-TryBot: Alexander Rakoczy <alex@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
2021-09-10 20:46:07 +00:00
Alexander Rakoczy 62419b0ffa cmd/coordinator: update Dockerfile to Go 1.17
This updates the coordinator to the latest supported version of Go. The
current version of Go labeled golang:1.17 is 1.17.1.

Use go mod download instead of manually trying to install dependencies
because it's easier to maintain and fast enough for our current needs.
Also, use the correct label syntax.

For golang/go#48329

Change-Id: I3db030624c1be12008705f374b269bc92e60ab25
Reviewed-on: https://go-review.googlesource.com/c/build/+/349091
Trust: Alexander Rakoczy <alex@golang.org>
Run-TryBot: Alexander Rakoczy <alex@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
2021-09-10 20:46:02 +00:00
Dmitri Shuralyov c4b8abe9ac cmd/gopherbot: move to Workload Identity
Create a new service account, and move the deployment over to the prod
namespace. Also update the build image to Go 1.17.

For golang/go#48263.

Change-Id: I9c7776b294ee78c2745670f805ec70cab1ae4573
Reviewed-on: https://go-review.googlesource.com/c/build/+/349056
Trust: Dmitri Shuralyov <dmitshur@golang.org>
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
2021-09-10 19:08:05 +00:00
Heschi Kreinick 71eba72053 cmd/pubsubhelper: move to Workload Identity
Create a new service account, and move the deployment over to the prod
namespace.

Update build image to Go 1.17, and use a GCS bucket for autocert cache
storage rather than a PD.

For golang/go#48263.

Change-Id: I33db02695f08ebdf9ef8b958bfd0b81567931b73
Reviewed-on: https://go-review.googlesource.com/c/build/+/348434
Trust: Heschi Kreinick <heschi@google.com>
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2021-09-10 17:17:53 +00:00
Dmitri Shuralyov 8757fe40fc cmd/releasebot: set trust when mailing VERSION file CL
The CL is generated and trusted. Save release managers the extra click
of adding Trust+1 to it before submission. It's already available as a
flag in modern versions of git-codereview.

Updates golang/go#38075.

Change-Id: I6a3296a9e19f308af07e5e2daef7e943c0f52956
Reviewed-on: https://go-review.googlesource.com/c/build/+/348435
Trust: Dmitri Shuralyov <dmitshur@golang.org>
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Heschi Kreinick <heschi@google.com>
2021-09-10 17:17:38 +00:00
Heschi Kreinick d4579c2b5c cmd/gerritbot: move to Workload Identity
Create a new service account, and move the deployment over to the prod
namespace.

Also fix AutoCert configuration so we can serve our happy little home
page.

Fixes golang/go#37377.
For golang/go#48263.

Change-Id: I9d0a5e49db53c0224379f448b49c9b679d59d23b
Reviewed-on: https://go-review.googlesource.com/c/build/+/348433
Trust: Heschi Kreinick <heschi@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2021-09-08 20:32:04 +00:00
Heschi Kreinick f1273632d1 cmd/gitmirror: push to prod namespace
Overlooked this -- the set-image command doesn't use a YAML file so
needs to specify the namespace explicitly.

Change-Id: I3602dbb31afc27df6087207b57b081975024021d
Reviewed-on: https://go-review.googlesource.com/c/build/+/347411
Trust: Heschi Kreinick <heschi@google.com>
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2021-09-08 19:24:18 +00:00
Heschi Kreinick a9d7de15d2 all: use Workload Identity for gitmirror, move to "prod" namespace
Rather than using a service account key, enable Workload Identity for
gitmirror. This involved moving into a non-default namespace, so the
coordinator's monitoring logic needs updating too.

Change-Id: I196fae0825f806248571aae6fb125d8c0917934d
Reviewed-on: https://go-review.googlesource.com/c/build/+/347410
Trust: Heschi Kreinick <heschi@google.com>
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Alexander Rakoczy <alex@golang.org>
Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-09-02 21:43:28 +00:00
Alexander Rakoczy 3734795929 cmd/relui: move implementation to internal/relui
This makes it explicit that the code is not intended to be used outside
this module.

For golang/go#47401

Change-Id: I6c7ca7402b451dbcfac39e504d5fc767811c973d
Reviewed-on: https://go-review.googlesource.com/c/build/+/347292
Trust: Alexander Rakoczy <alex@golang.org>
Run-TryBot: Alexander Rakoczy <alex@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
2021-09-02 21:31:40 +00:00