Previously, a builder was considered to be "SlowBot" whenever it was
requested via the TRY= syntax, and it wasn't already a part of the
default set of trybot builders.
This change makes it so that a builder is considered a SlowBot if
it was explicitly requested via the TRY= syntax, even if it was
already going to run anyway.
The goal behind this change is to improve the experience of using
SlowBots, by making them more useful and predictable.
The UI currently reports that "SlowBots are starting ..." whenever
there is a non-zero amount of SlowBots. So if a user requested
TRY=linux-amd64, they'd likely see "TryBots are starting ...",
which can be misunderstood to mean that SlowBots are not working.
Additionally, if a user requested three builders via TRY= syntax,
but one of them happened to be a part of the default try set, then
the message at the end would be "Extra slowbot builds that ran:"
including only 2 builders. At that point, it's hard to tell whether
they got the TRY= syntax wrong for one of the builders, or if it was
omitted because it's always run anyway.
A future usability improvement in golang.org/issue/38279 is to make
SlowBots not skip any tests that normal trybots may skip for faster
results. This change is in preparation for that.
For golang/go#38279.
For golang/go#34501.
Change-Id: Idb7f1bcb1694fe72f85237976995a94f273c7c16
Reviewed-on: https://go-review.googlesource.com/c/build/+/227778
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Alexander Rakoczy <alex@golang.org>
This implements bullets 4 and 5 from the plan to unify the dashboard &
coordinator:
https://github.com/golang/go/issues/34744#issuecomment-563398753
Previously the coordinator would POST to the dashboard regularly, for
each active build, to say "I'm still working on this build! Write that
to memcache!". And then if somebody loaded https://build.golang.org/
it would do a big memcache multi-get to populate the little blue
gopher links to their status pages on the coordinator.
This instead turns it around. We no longer POST from the coordinator
to the dashboard, and we no longer use any memcache (which also means
migrating to the App Engine Go 1.13 runtime is easier, which drops
the built-in memcache support). Instead, the dashboard now does a GET
to the coordinator to get the list of active builds.
This also adds test coverage, which we didn't have before.
Updates golang/go#34744
Change-Id: I97a486ec362a7a00d29076c81a88d6417b138c1b
Reviewed-on: https://go-review.googlesource.com/c/build/+/210838
Reviewed-by: Alexander Rakoczy <alex@golang.org>
Run-TryBot: Alexander Rakoczy <alex@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Historically, the build.golang.org was the entire build system, and it
maintained a parallel copy of that git history in its datastore. It
was always buggy and incomplete and things like force pushes were
scary because the datastore mirror could get out of sync. It was also
a lot of code to support that sync.
This changes build.golang.org to instead get the git history from
maintnerd, and then we can remove all the HTTP handlers around
updating it, and can remove all the gitmirror code to call it to
maintain it.
Now build.golang.org only keeps build results, keyed on the commit
hash. It's much less code, but is still an App Engine app for now.
(but it's getting small enough, that porting it to
cloud.google.com/go/datastore is looking very simple)
This also adds a new "repos" package to unify the configuration of the
various Go repos. There were incomplete & redundant copies all over
the place.
Updates golang/go#34744Fixesgolang/go#35828Fixesgolang/go#31236 (New branch=mixed support adds this when desired)
Fixesgolang/go#35944
Change-Id: Ifb39417287df3dea052ba8510566d80b4bc75d51
Reviewed-on: https://go-review.googlesource.com/c/build/+/208697
Reviewed-by: Bryan C. Mills <bcmills@google.com>
If somebody wants to "gomote create" a reverse buildlet that's busy
doing a post-submit build, the coordinator tries to find an active
non-trybot build and call (*buildStatus).cancelBuild() on it, to make
it clean up and return itself to the coordinator, so the scheduler
will get it soon and give it to the gomote user.
Fixesgolang/go#35714
Change-Id: I417aead083cf3520058b3fd4d626fc385f25984e
Reviewed-on: https://go-review.googlesource.com/c/build/+/209757
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Alexander Rakoczy <alex@golang.org>
Also modernizes some code in the coordinator.
Updates golang/go#35354 (or fixes. But we could return more info.)
Change-Id: Ifc1aa85ca217a0932e388ec5d36ef0737b90c63d
Reviewed-on: https://go-review.googlesource.com/c/build/+/207841
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
This parses TRY= comments to opt-in to slower/difference trybots.
This needs some docs/UI work yet.
Updates golang/go#34501
Change-Id: I13a835520d7ac341bc86139b0a2118235bc83e60
Reviewed-on: https://go-review.googlesource.com/c/build/+/201338
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
The new BuildConfig.MinimumGoVersion field specifies the minimum
Go version the builder is allowed to use. It's useful when some
of the builders are too new, and do not support all of the supported
Go releases (e.g., openbsd-amd64-64 and freebsd-amd64-12_0 currently
require Go 1.11 and don't work on Go 1.10).
It only needs to be set when a builder doesn't support all supported
Go releases, since we don't typically test unsupported Go releases.
To allow cmd/coordinator to use this field and filter out work it
receives from maintner/maintnerd's GoFindTryWork RPC call,
we add a GoVersion slice to apipb.GerritTryWorkItem,
and populate it in maintapi.apiService.GoFindTryWork method.
For trybots on the Go repo, the GoVersion field is determined from
the branch name. For "release-branch.goX.Y" branches, it parses out
the major-minor Go version from the branch name. For master and
other branches, it assumes the latest Go release.
For trybots on subrepos, we already have the Go version information
available, so use it directly.
Afterwards, all that's left is to modify newTrySet in cmd/coordinator
to make use of BuildConfig.MinimumGoVersion and work.GoVersion to skip
builders that are too new for the Go version that needs to be tested.
Fixesgolang/go#29265
Change-Id: I50b01830647e33e37e9eb8b89e0f2518812fa44f
Reviewed-on: https://go-review.googlesource.com/c/155463
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
We want to pass this down to the coordinator so each
dashboard.BuildConfig can have its own policy on which branches to
build. For example, we'll have NetBSD skip building subrepos for Go
versions that are too old.
Updates golang/go#23102
Change-Id: I40d35c9c5a6d4455772430c169dbea054929545e
Reviewed-on: https://go-review.googlesource.com/83536
Reviewed-by: Andrew Bonventre <andybons@golang.org>
Since farmer.golang.org has a valid certificate in production now, we
should use that certificate for retrieving data.
Change-Id: I3685d84953ea11e6ac09b4692fa27c73538a565c
Reviewed-on: https://go-review.googlesource.com/39290
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Will be used for dynamic creation/destruction of Mac VMs in subsequent CL.
Updates golang/go#9495 (Mac virtualization)
Updates golang/go#15760 (monitoring)
Change-Id: I48b17589b258d5d742bad5a3ddae18de98778149
Reviewed-on: https://go-review.googlesource.com/37457
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Most of these are documentation fixes.
Change-Id: If100cc2ef0ee0b8b47a1e96e4c51a2be8caf5842
Reviewed-on: https://go-review.googlesource.com/22366
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This expands the JSON dashboard output to include everything that's
visible on the HTML dashboard (which includes all of the obviously
meaningful fields from the Commit objects).
Change-Id: Ib0437bec276a78ca9efd57fcc16041e40ccd3d4c
Reviewed-on: https://go-review.googlesource.com/16632
Reviewed-by: Andrew Gerrand <adg@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This adds an RFC 3339 commit date for each revision represented in the
mode=json output of the dashboard. This is shown on the HTML version
of the dashboard, so it's already readily available, and its useful
when scraping the dashboard.
Change-Id: I7683ab76505b3dcd2395f54ec6f904a4a8d56617
Reviewed-on: https://go-review.googlesource.com/9012
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Dmitri Shuralyov
Brad Fitzpatrick
Andrew Bonventre
Kevin Burke
Andrew Gerrand
Austin Clements