This is a quick & hacky version of our eventual sharding plan. Later
we want to run make.bash once, snapshot it, and then intelligently
schedule all the tests out over N machines (starting with the built
snapshot), giving each the next test to run as they become idle.
And then we want to stitch together all of their output.
But this CL just makes 6 ARM builders, each of which runs make.bash (4
minutes) and then each which runs a statically-configured subset of
the tests. Based on measurements, each should run between 8.2 and 10.6
minutes. We can break up some of the larger test cases later into more
shardable pieces (in particular: "test").
Update golang/go#10029
Change-Id: Ib3e008b61ce6f28d799ae298c8467bf62d89e5d4
Reviewed-on: https://go-review.googlesource.com/10056
Reviewed-by: Andrew Gerrand <adg@golang.org>
Make all-compiler a trybot-only builder. This will be used for
temporary sharded ARM trybots.
Updates golang/go#10029
Change-Id: I6cc3188887c2f24edef97d0a9354e39553a15896
Reviewed-on: https://go-review.googlesource.com/10051
Reviewed-by: Andrew Gerrand <adg@golang.org>
It's slow.
This has been running for a week but I forgot to mail the CL.
Change-Id: Ibe0646e18ffb23eb0b10d9fc0c18864bfba08a44
Reviewed-on: https://go-review.googlesource.com/9853
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Tested by running a local coordinator on the darwin-arm64 builder.
Change-Id: Id38faaa3275d6535b897f3c65df6d01ed9c1847e
Reviewed-on: https://go-review.googlesource.com/9459
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Also introduces more dev mode manual testing machinary. It is now
possible to start a coordinator and buildlet on a local machine and
hit localhost:8119/dosomework/buildername to rebuild the latest
revision from build.golang.org.
No support for VMs on reverse buildlets yet. That will have to wait
for some future work.
Change-Id: I590a9bc1779add030bde8dbf496eb6814ef57126
Reviewed-on: https://go-review.googlesource.com/9248
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Also, I rebuilt the Windows image with the 30GB base image. I named it
-v2 during manual testing with gomote, and I'm keeping it like that
now, so update dashboard/builders.go too.
Fixesgolang/go#10071
Change-Id: I30029310cbf61fb21ef80063f9822cb90ce843c0
Reviewed-on: https://go-review.googlesource.com/7914
Reviewed-by: David Crawshaw <crawshaw@golang.org>
Also, make Windows use regular disks for now, since its image is so
large (100 GB) and we only have 2TB of SSD quota.
This is all very conservative and paranoid for now until I figure out
what part of the coordinator was misbehaving.
Change-Id: Icead5c07cf706c2cfc4d1dd66a108649429018ac
Reviewed-on: https://go-review.googlesource.com/7910
Reviewed-by: David Crawshaw <crawshaw@golang.org>
Everything is in VMs now.
Change-Id: I2fc28207eb84a7971236efa39a4684ec682d315a
Reviewed-on: https://go-review.googlesource.com/5360
Reviewed-by: Andrew Gerrand <adg@golang.org>
Otherwise it unnecessarily does:
"Building compilers and Go bootstrap tool for host, linux/amd64"
Change-Id: I051e4fe42030fbf5452372a0a6a42391383d53e3
Reviewed-on: https://go-review.googlesource.com/4871
Reviewed-by: Dave Cheney <dave@cheney.net>
The release command will replace misc/makerelease from the main repo.
The contents of makerelease make their way into this program.
Also add GorootFinal and MakeScript methods to dashboard.BuildConfig.
Change-Id: Id20f3f7a85a21f040d8cf47e438d299d57360556
Reviewed-on: https://go-review.googlesource.com/3693
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
We still use Dockerfiles to describe them, but then a new tool
(docker2boot) converts them into VM images suitable for booting on
GCE, running the buildlet like all the other operating systems. Things
are easier if everything acts the same way.
Note that since we're no longer moving around Docker images, the image
size and the layer accumulation cruft no longer matters. We can now
have Dockerfile lines like "RUN rm -rf /usr/share/doc" and it actually
results in a smaller VM image, since we just "docker export" the files
out of it to create the VM image.
This doesn't yet convert the clang, sid, or nacl builders. The
coordinator still runs those under Docker directly. A future change
will convert those to VM images as well.
Change-Id: Iedb136ae3daf888c955eb843bdcc9a638d08f5e9
Reviewed-on: https://go-review.googlesource.com/3341
Reviewed-by: Andrew Gerrand <adg@golang.org>
I ran an OpenBSD 1.4.1 build by hand with gomote, ran gomote gettar to
get the built tree, and put it on GCS.
This CL then adds the URL of that Go 1.4 pre-built tree to the OpenBSD
builder's configuration, and modifies the coordinator to tell the
buildlet to pull extract the tarball from that URL to the "go1.4"
directory before the build starts.
Aside: I'm taking advantage of the build repo's new freedom in
depending on outside libraries and using Camlistore's syncutil.Group.
Change-Id: I2d6e416d2220d2d34c630fed50344bf2de4ae710
Reviewed-on: https://go-review.googlesource.com/3114
Reviewed-by: Andrew Gerrand <adg@golang.org>
Also:
- Move the watcher to cmd/watcher (somehow this was missed earlier).
- Move dashboard package from the repo root to its own directory.
- Update docker build scripts. (Although not yet the version hashes in
the Dockerfiles; this leaves the docker builds broken, but they were
already broken after moving the builder to cmd/builder. They'll be
fixed in a followup CL after this one is submitted.)
Change-Id: I29a9758da1f3c60446e3ce18174c0df26e4d8325
Reviewed-on: https://go-review.googlesource.com/3077
Reviewed-by: Andrew Gerrand <adg@golang.org>
Brad Fitzpatrick
Shenghou Ma
David Crawshaw
Andrew Gerrand
Bill Thiede