Also use iterator.Done instead of datastore.Done.
Change-Id: I3948ade043ff9bbe5c4eb0abc1e51963f740b608
Reviewed-on: https://go-review.googlesource.com/32445
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Chris Broadfoot <cbro@golang.org>
Normal Go policy is to support the past two released Go version, which
is currently Go 1.6 and Go 1.7.
Change-Id: I8db9495582d40e82ceb4ccf52ea22ec3969f6ae5
Reviewed-on: https://go-review.googlesource.com/31767
Reviewed-by: Quentin Smith <quentin@golang.org>
This new Plan 9 image implements a "close" message in the TCP
connection control file. The "close" message is analogous to
the "hangup" message, but closes the TCP connection gracefully,
using a FIN message instead of a RST.
Change-Id: Id5652ca399d38d78e6f5616a43b2ba88a8a4c7e7
Reviewed-on: https://go-review.googlesource.com/31272
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
The Makefile used to create the Docker image used by the Kubernetes builder
creates an image with a different name than the dashboard expects. This
change updates the dashboard to pull the image created by the Makefile.
Change-Id: Idcf8f5867d9995205c6c8482a7e8bfa1b63d9551
Reviewed-on: https://go-review.googlesource.com/28416
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Saves 4.5 minutes or so by using fast x86 machines to build the ARM
build instead of running make.bash on Scaleway ARM machines.
We still run the tests on ARM, and have a separate builder only
running make.bash on ARM (see prior golang.org/cl/29670)
Fixesgolang/go#17105
Updates golang/go#17104
Change-Id: I1cb7b0e5b1cc8b644195f262328884ed3aff120a
Reviewed-on: https://go-review.googlesource.com/29677
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This imports https://swtch.com/tmp/dash.html and makes it render for any
release. Initially, the only graph is the # of open issues by milestone
over the course of the release.
The dashboard is not currently tracking label history, which is needed
to draw the second graph on that page.
Change-Id: I9bd031f8709701b304e18208ae3c972bdfe3b276
Reviewed-on: https://go-review.googlesource.com/30012
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
perfrun fetches the latest N successful builds from the dashboard and
runs their go1 benchmarks, saving logs on disk
This is the start of a daemon that will automatically run the benchmarks
for every successful build and archive the logs to Cloud Storage.
Change-Id: I5dfd715788919abaf22f71a206f2682eceeaadcc
Reviewed-on: https://go-review.googlesource.com/29399
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
The far superior linux distro of champions.
Updates golang/go#17105
Change-Id: I5ea0cd2361753f61bb74bf3d4dea6c181f1427fa
Reviewed-on: https://go-review.googlesource.com/29687
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Moved arm to armel, if anyone wants that in the future.
Change-Id: I57db5c8dff12c02cd2ceb4bd657d5fa63040985c
Reviewed-on: https://go-review.googlesource.com/29684
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Moved cross-compile builders to env/crosscompile
Tested it works:
$ readelf --headers go/bin/go
ELF Header:
Magic: 7f 45 4c 46 01 01 01 00 00 00 00 00 00 00 00 00
Class: ELF32
Data: 2's complement, little endian
Version: 1 (current)
OS/ABI: UNIX - System V
ABI Version: 0
Type: EXEC (Executable file)
Machine: ARM
Version: 0x1
Entry point address: 0xc3a80
Start of program headers: 52 (bytes into file)
Start of section headers: 276 (bytes into file)
Flags: 0x5000002, has entry point, Version5 EABI
Size of this header: 52 (bytes)
Size of program headers: 32 (bytes)
Number of program headers: 7
Size of section headers: 40 (bytes)
Number of section headers: 23
Section header string table index: 7
Change-Id: I3db71fa954575c29bb0a12a07c9bf84c4c96d88e
Reviewed-on: https://go-review.googlesource.com/29679
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This change removes the code that puts Kubernetes cluster metrics to
Stackdriver. These metrics were replaced by built-in CPU % metrics
and are no longer used.
Change-Id: I8cbd92bbb00ef323f008b20fd66b0fffd160f260
Reviewed-on: https://go-review.googlesource.com/29673
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This is a new builder in prep for the change to the "linux-arm"
builder where the GOARCH=arm make.bash will be cross-compiled from a
Kubernetes container on fast hardware.
Updates golang/go#17105 (cross-compile ARM builders' make.bash)
Updates golang/go#17104 (5 minute trybots)
Change-Id: Icfd2644d77639f731151abe54839322960418254
Reviewed-on: https://go-review.googlesource.com/29670
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Our builders are named of the form "GOOS-GOARCH" or
"GOOS-GOARCH-suffix".
Over time we've grown many builders. This CL doesn't change
that. Builders continue to be named and operate as before.
Previously the build configuration file (dashboard/builders.go) made
each builder type ("linux-amd64-race", etc) define how to create a
host running a buildlet of that type, even though many builders had
identical host configs. For example, these builders all share the same
host type (a Kubernetes container):
linux-amd64
linux-amd64-race
linux-386
linux-386-387
And these are the same host type (a GCE VM):
windows-amd64-gce
windows-amd64-race
windows-386-gce
This CL creates a new concept of a "hostType" which defines how
the buildlet is created (Kube, GCE, Reverse, and how), and then each
builder itself references a host type.
Users never see the hostType. (except perhaps in gomote list output)
But they at least never need to care about them.
Reverse buildlets now can only be one hostType at a time, which
simplifies things. We were no longer using multiple roles per machine
once moving to VMs for OS X.
gomote continues to operate as it did previously but its underlying
protocol changed and clients will need to be updated. As a new
feature, gomote now has a new flag to let you reuse a buildlet host
connection for different builder rules if they share the same
underlying host type. But users can ignore that.
This CL is a long-standing TODO (previously attempted and aborted) and
will make many things easier and faster, including the linux-arm
cross-compilation effort, and keeping pre-warmed buildlets of VM types
ready to go.
Updates golang/go#17104
Change-Id: Iad8387f48680424a8441e878a2f4762bf79ea4d2
Reviewed-on: https://go-review.googlesource.com/29551
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Also, don't start obtaining test sharding buildlets early if they're
reverse buildlets. Reverse buildlets are either immediately available,
or they're not. No point monopolizing them earlier than needed.
Updates golang/go#17104
Change-Id: If5a0bbd0c59b55750adfeeaa8d0f81cdbcc8ad48
Reviewed-on: https://go-review.googlesource.com/29473
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
It's been there for the past two releases. We can assume it's present.
Change-Id: I94ac107d4c52ea823d42a63ced571ff4c81532a2
Reviewed-on: https://go-review.googlesource.com/29471
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
We're now on flash instead of spinning disks, which means
our datastore name changes of course.
Change-Id: I74d4d7d4c79d69024bd316319ede85ec45564f50
Reviewed-on: https://go-review.googlesource.com/29235
Reviewed-by: Quentin Smith <quentin@golang.org>
Go's gzip is faster than using a child process nowadays.
Also, this wasn't safe on Windows. The Windows buildlets hadn't been
deployed in over a year. When I deployed a new version (which included
the pargzip package), they started failing to gzip snapshots, breaking
test sharding. Trybots succeeded, but only after ~970 seconds, running
all the tests on the original make.bat machine.
Change-Id: I1d144b82e40285fee0b3019a27ee0ce2f3844d5a
Reviewed-on: https://go-review.googlesource.com/29432
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Some builders (OpenBSD, FreeBSD, Plan9 at least?) have their buildlet
process's stdout/stderr hooked up to their serial console.
The log line for each untarred Go1.4 + Go src tarball going to the
serial console added just shy of 1 minute (!!) to the build time. Now
it takes 3 seconds. (Or 12 seconds before change to use an async+noatime
filesystem on the BSDs)
Updates golang/go#17104
Change-Id: I1e6f00bcca955ead26b279a79729e50319384593
Reviewed-on: https://go-review.googlesource.com/29430
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
OpenBSD is deployed and tested. FreeBSD has only been tested by hand,
but this CL doesn't fail the buildlet if the remount fails. It only
logs either way.
Updates golang/go#17104
Change-Id: Ia9662b42ae8305ad9eaa4292c94fa3194cc26b11
Reviewed-on: https://go-review.googlesource.com/29238
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
If the coordinator has a better guess as to how long various tests
take, then it can do better critical path scheduling and reduce the
overall time the sharded tests take to complete.
This table still needs to die and be based on recent empirical data,
but at least it's more accurate now, after a long delay in being
updated.
Update is from https://github.com/golang/go/issues/17104#issuecomment-247123458
Updates golang/go#17104
Change-Id: I115aad23fbdb0cde1b196e71a4131fbe36480cc0
Reviewed-on: https://go-review.googlesource.com/29167
Reviewed-by: David Crawshaw <crawshaw@golang.org>
I'm aiming to have trybot runs finish in under 5 minutes.
This CL removes openbsd-386-gce58 and freebsd-386-gce101 from the trybot set.
openbsd-386-gce58 is the slowest builder. It has an average speed of
722 seconds (and 95 percentile of 923 seconds) over the past week, and
that's sharded over 4 machines. Too slow. It's not worth the resources
to keep it as a trybot. It hasn't caught any interesting bugs. This
builder will still run, but not as a pre-submit trybot.
freebsd-386-gce101 is not slow, but we're removing it to shift its
resources to shard other builders wider.
The coordinator now supports varying the build sharding width based on
whether a build is for a trybot or not. This CL defines separate
numbers for each, sharding builds wider as needed for some trybots.
freebsd-amd64-gce101 goes from 4 to 5 machines in try runs, and down
to 3 when not in try runs.
linux-amd64-race gets one more machine during try runs, and one fewer
in regular runs.
linux-arm goes from 7 machines always, to 3 or 8, depending on whether
it's a try run.
openbsd-amd64-58 goes from 4 to 3 or 6.
windows-amd64-gce goes from 4 to 2 or 6.
windows-amd64-race goes from 4 to 2 or 6.
darwin-amd64-10_11 goes from 3 to 3 or 4.
I'll see how these do over the next few days and readjust as needed.
Also in this CL: fix the constants for the expected duration of
make.bash, which impact when we schedule the creation of test sharding
helper buildlets. We were creating them too early before, wasting
resources.
Change-Id: I38a9b24841e196f1eb668de058c49af8c1d1c64f
Reviewed-on: https://go-review.googlesource.com/29116
Reviewed-by: Quentin Smith <quentin@golang.org>
Currently this tool just syncs build data from Datastore to Bigquery.
It'll do more later.
Updates golang/go#12669
Change-Id: Ic544cf112ea8699c86943c32f8887d31c81daf79
Reviewed-on: https://go-review.googlesource.com/29112
Reviewed-by: Quentin Smith <quentin@golang.org>
Now that the stats dashboard is (mostly) working, link it from the
homepage.
Change-Id: I5fa1c2f241151d809c2d9a6cf8f664dda14cf8b3
Reviewed-on: https://go-review.googlesource.com/28975
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
ReverseProxy.ServeHTTP expects a server *http.Request.
The gomote proxy in the coordinator was making a client *http.Request
(which has different semantics for ContentLength depending on whether
it's client or server) and that was confusing the ReverseProxy code.
Set the next (server) request's ContentLength explicitly to the
original incoming Request.ContentLength. It was zero before, which
used to be safely okay, but not safe as of
https://go-review.googlesource.com/28412
Change-Id: I96d5c2b992d223372f9a01d268d3a419ccdc0d95
Reviewed-on: https://go-review.googlesource.com/28751
Reviewed-by: Chris Broadfoot <cbro@golang.org>
The favicon is hotlinked from golang.org/favicon.ico, which is now
larger.
Resize it to 16x16.
Fixesgolang/go#17022.
Change-Id: Id74eb3bd8c855a042d033664359f04089d7857ff
Reviewed-on: https://go-review.googlesource.com/28770
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Currently it's just a package main. It might split in two later. It
might stop shelling out to govc and use the underlying API directly
too, but it's hairy.
Easier to work like this for now.
Updates golang/go#9495
Change-Id: I0d2e19abcb5114ab7fe2e2c543d14e50897d4cbb
Reviewed-on: https://go-review.googlesource.com/28584
Reviewed-by: Quentin Smith <quentin@golang.org>
Brad Fitzpatrick
Jonathan Amsterdam
Dmitry Vyukov
Jess Frazelle
David du Colombier
Russ Cox
Quentin Smith
Evan Brown
Joel Sing
Chris Broadfoot