The windows-amd64-2008 is the same OS as windows-amd64-gce but is an
auto-generated image.
TODO: 386 auto-generated Windows builders, and then maybe we'll move
TryBots to Windows Server 2016. One step at a time. This should be a
no-op. I verified performance is the same.
Updates golang/go#17513
Change-Id: I34984db14b87d03771e15465978b1687df6895f7
Reviewed-on: https://go-review.googlesource.com/41611
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
We've been logging event spans to datastore for years, but I'd lost
this CL and just found it back. This does two things: syncs the
datastore logs to BigQuery, and starts to use the from-BigQuery timing
info in the coordinator for scheduling sharded tests.
The plan was to have a job occasionally do a BigQuery query and write
out the results to a CSV file on GCS. The code to read that CSV file
is in this CL, but that code path is disabled, so this CL should be a
no-op.
A future change will periodically do the query and write the CSV file,
and then we can start using the new code path and remove the static
map of expected test durations.
Updates golang/go#12669
Change-Id: Ibe5b41d6a3009c2ade8ab728fa1cad646788e621
Reviewed-on: https://go-review.googlesource.com/30716
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Benchmarks are treated as unit tests and distributed to the test
helpers, which allows them to fit in our 5m trybot budget.
Currently we only run the go1 and x/benchmarks. Running package
benchmarks is a TODO.
This feature is disabled by default, and is enabled by the
"farmer-run-bench" project attribute.
Updates golang/go#19178
Updates golang/go#19871
Change-Id: I9c3a14da60c3662e7e2cb4e71953060915cc4364
Reviewed-on: https://go-review.googlesource.com/38306
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
You can trigger this by creating a Corpus and only adding Gerrit
repositories to it. These call parseGithubRefs, but c.github.repos
is never initialized, so that function panics when you try to get
c.github.repos.
I'm a little worried I've triggered two of these recently; maybe it's
worth it to initialize the data structures earlier instead of
lazy-loading them.
Change-Id: I8f80c0947d320aef0dd9391586c306460ac77eea
Reviewed-on: https://go-review.googlesource.com/41305
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
We print the entire reponse from Github when we log messages about how
many comments were found, which seems unnecessary - the most useful
bit of information is the number of comments found. Also print the
issue number we're fetching comments for.
Change-Id: I5eca0ea1e4ea737642a535b61a8dd65838b44773
Reviewed-on: https://go-review.googlesource.com/41301
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Previously processGithubIssueMutation called initGithub, but
processGithubMutation did not, so if your on-disk mutation file had
a GithubMutation before a GithubIssueMutation, c.github would be nil
and Initialize() would panic.
Call initGithub from processGithubMutation and add a test to protect
against regressions.
Change-Id: Ie706fa04cb8ea87c2e0259dbee024c0005f0523f
Reviewed-on: https://go-review.googlesource.com/41298
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
The new untar package will be shared by the buildlet later.
Change-Id: I7a33bf46e5cb6b7e5abd59613bb943039c84aa08
Reviewed-on: https://go-review.googlesource.com/41131
Reviewed-by: Jeff Johnson <jrjohnson@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This adds indexing of Github mentions to maintner, then adds gopherbot
functionality to say "CL https://golang.org/cl/NNNN mentions this issue."
on Github when a Gerrit CL references it.
Also the start of the cherry-pick milestone bot which needs the same
Github issue reference tracking. But that part's only barely started
and still disabled.
Fixesgolang/go#18818
Updates golang/go#19776
Change-Id: Ie5f7f6845317a7c39cc36d7397c7539bf99c3f92
Reviewed-on: https://go-review.googlesource.com/39352
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This ends support for Go 1.6.
Change-Id: If7765a15add556a5a7a881b24b1a38d96ff10841
Reviewed-on: https://go-review.googlesource.com/40938
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
devapp now supports 3 modes of operation:
- App Engine Standard. This supports Go 1.6 and requires a modified
local go-github to continue working. This app is built with the
"appengine" build tag.
- App Engine Flex. This builds Go 1.8 in a Docker container, using the
build tag "appenginevm", and works with the Google Cloud platform.
- Normal. This uses an in-memory datastore, and does not
support user login/logout.
To get this working I copied appengine.go to noappengine.go and
modified the calls until they were working. You can view a running app
at https://devapp-161505.appspot.com.
Change-Id: I8ea018e63baf2dafb44150f7eee419e09065ba2c
Reviewed-on: https://go-review.googlesource.com/38161
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
It's starting to pollute the package, and seems separable. I also plan
to use it more from elsewhere in upcoming CLs.
Updates golang/go#19866
Change-Id: I7b2add37f74ed42c2f78939924f19d8322179823
Reviewed-on: https://go-review.googlesource.com/40868
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Previously we printed duration to seven decimal points after the
second, which isn't helpful or necessary to determine how long a
build took. Instead, round the duration to the nearest tenth of a
second (if the build took more than 10 seconds), the nearest hundredth
of a second (if it took 1-10 seconds), or the nearest tenth of a
millisecond (if it took less than a second), which should be more than
enough precision and is much easier to read.
Change-Id: I1c29d4a81335bf9ee3cddda0a341d3f321e82f6b
Reviewed-on: https://go-review.googlesource.com/40855
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Replace oauth2.NoContext (deprecated) with context.Background(),
which has been available for two consecutive releases.
Add more cloud.google.com/go packages to the cmd/coordinator
Dockerfile to fix an error building cmd/coordinator. A dependency is
not present in master of cloud.google.com/go, but was present in the
older revision, and was not getting checked out correctly during the
"go get" step. In addition, we were failing to fetch dependencies for
some packages that coordinator depends on. I added instructions for
hopefully doing this more systematically in the future.
Fix the gitmirror Dockerfile which has the same problem.
Change-Id: Id6c2220482350a686b87742ec7915c457a689e52
Reviewed-on: https://go-review.googlesource.com/40852
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Adds an initial set of github handles for builder owners and displays a link in
coordinator if set.
Updates golang/go#19848
Change-Id: I85aa98a48b8a094482b22021b32b465af5313bbe
Reviewed-on: https://go-review.googlesource.com/40654
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
host-linux-arm64-linaro is identical to host-linux-arm64-packet, at
least for now.
And remove the old arm64-linaro support. We took down those machines
and are reimaging them.
Change-Id: Ieef462debc9356aadd828863561cc161fd10d8d1
Reviewed-on: https://go-review.googlesource.com/40653
Reviewed-by: Sarah Adams <shadams@google.com>
The snapshot code had an old workaround that's no longer relevant. Remove it.
Also, add a BuildConfig.SkipSnapshot bool, and use it for the slow
mips builders.
Fixesgolang/go#19953
Change-Id: I114bb0a524184eaaae5be4715ce63f6adc519c2e
Reviewed-on: https://go-review.googlesource.com/40505
Reviewed-by: Sarah Adams <shadams@google.com>
Because almost all the BuildConfig methods indirectly call AllScript,
add in a little defense against accident creation of BuildConfig zero
values (of the value type, not nil pointers). This can happen by doing
a lookup in a map[string]dashboard.BuildConfig and then calling a
method on it. We should probably move everything to pointer values of
BuildConfig. There aren't many value types remaining.
This paranoia isn't a result of any bug yet, but rather than the result
of discussion on CL 40477.
Change-Id: Ic273d4de28ac8930b56bc232bf77a9ba39f99fad
Reviewed-on: https://go-review.googlesource.com/40497
Reviewed-by: Herbie Ong <herbie@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Adds a Dockerfile and Makefile so maintnerd can be deployed to GKE.
Updates golang/go#19866
Change-Id: I83d8d409c2acab3022c2b74516157d32515fe28b
Reviewed-on: https://go-review.googlesource.com/40478
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Optimized image to be smaller as well.
Before:
REPOSITORY TAG IMAGE ID CREATED SIZE
go-builders/linux-x86-clang latest b08175ba2529 About a minute ago 1.01GB
After:
REPOSITORY TAG IMAGE ID CREATED SIZE
go-builders/linux-x86-clang latest 1545f177bd2f About a minute ago 543MB
Change-Id: I3e0d6007aed3231215feb653e2a11f29a01a5e45
Reviewed-on: https://go-review.googlesource.com/40472
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Cleanup and optimize Dockerfiles for making images smaller.
Before:
REPOSITORY TAG IMAGE ID CREATED SIZE
go-builders/linux-x86-nacl latest c814d8cff592 3 seconds ago 1.16GB
After:
REPOSITORY TAG IMAGE ID CREATED SIZE
go-builders/linux-x86-nacl latest 5cf064519947 5 seconds ago 726MB
Change-Id: Ie61ddfad53df9dd0e6971532c92f59f18953171a
Reviewed-on: https://go-review.googlesource.com/40473
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
The NetBSD builders were supposed to all be skipped by
buildRev.skipBuild, but there was another code path generating work
(the bootstrap-a-new-builder-column code) that wasn't using skipBuild,
which meant when the NetBSD column totally disappeared (as we wanted),
the builder tried to bring it back to life, doing a bunch of hanging
builds.
Change-Id: I976016a485e241b46f9cf8bb8b371e1038bb5f8c
Reviewed-on: https://go-review.googlesource.com/40470
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Cleanup and optimize Dockerfiles for making images smaller.
Before:
REPOSITORY TAG IMAGE ID CREATED SIZE
go-builders/linux-x86-sid latest 9b9e88b7655c 55 seconds ago 1.12GB
go-builders/linux-x86-std-kube latest 91a5a7a8cb1f 8 seconds ago 986MB
After:
REPOSITORY TAG IMAGE ID CREATED SIZE
go-builders/linux-x86-sid latest aab9e758925e 7 seconds ago 731MB
go-builders/linux-x86-std-kube latest 8309123c2745 2 minutes ago 577MB
Change-Id: I613896faf0d298448995952d265b1729cd03392c
Reviewed-on: https://go-review.googlesource.com/40409
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
And update it. What we used to call "sid" was probably ancient.
Change-Id: I76c1aad60eb655513119eaad0550ba5e0df3e9c5
Reviewed-on: https://go-review.googlesource.com/40399
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
The watcher is dead. It was renamed to gitmirror, but its
"watcher-world" environment is no longer relevant, since it's deployed
as a Kubernetes container now.
And the linux-x86-std VM configuration is old. We only use
linux-x86-std-kube on Kubernetes now.
Remove some gccgo stuff too, since that bitrot ages ago.
Change-Id: I74038961dee818b70a5e33c949a64a210bb1abb7
Reviewed-on: https://go-review.googlesource.com/40398
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Currently all builds start and think they're running, but most are
just fighting over a mutex to grab a builder. That will be fixed, but
in the meantime it's nice to see what's actually working vs what's
waiting on e.g. arm5 hardware which won't be available for hours.
This is a baby step towards more monitoring. Currently this is just HTML
output, but the same data could be exported via JSON or something else later
for graphing.
Updates golang/go#19178 (add a buildlet scheduler)
Updates golang/go#15760 (monitor everything)
Change-Id: I36e16ea0919afe8023fe7fedd981f2e857f0d6df
Reviewed-on: https://go-review.googlesource.com/40397
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
-- move policy of which builders are trybots out of coordinator
and into dashboard/builders.go.
-- move some GCE-specific code from coordinator.go to gce.go.
-- rename an old "watcher" reference to "gitmirror"
-- add some docs
-- actually add the Alpine builder, missing from https://golang.org/cl/33890Fixesgolang/go#17891
Change-Id: Ia63671ca09aec322ed57b3663e0ac5042cdc56f2
Reviewed-on: https://go-review.googlesource.com/40395
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
The server has 96 cores and 128 GB of RAM, so run 20 builders on it.
This adds the Dockerfile each builder runs in, and the machinery to
run 20 copies of the Docker container.
Also reduce some logspam in the buildlet's shutdown.
(The dashboard/builders.go entry for this builder was added previously
in https://golang.org/cl/39851)
Fixesgolang/go#19929
Change-Id: I8537eee52c002dc9efcadcfb7e78b3a5db07ae44
Reviewed-on: https://go-review.googlesource.com/40392
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Map iteration is random, so it's possible for two simultaneous calls
to getStatus to deadlock due to the use of defer to unlock.
Change-Id: Ib2c0b4122bd5ea17fde5e4c60da1a853a12ed2fd
Reviewed-on: https://go-review.googlesource.com/40301
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Brad Fitzpatrick
Quentin Smith
Kevin Burke
shawnps
Jess Frazelle
Sarah Adams