go
/
build
зеркало из https://github.com/golang/build.git
1
0
Форкнуть 0
Код Задачи Пакеты Проекты Релизы Вики Активность
build/dashboard/builders.go

2954 строки
102 KiB
Go
Исходник Обычный вид История

dashboard: move builder configuration out of coordinator into a package The new package will be used by the buildlet command line tool in the future. Also use the metadata package in the coordinator for the master build key, since it was already depending on that package. Might as well use it. Change-Id: I6ad7593787ebba364090951ccbaecb4aa38e1dd6 Reviewed-on: https://go-review.googlesource.com/2920 Reviewed-by: Andrew Gerrand <adg@golang.org>
2015-01-15 23:46:22 +03:00
// Copyright 2015 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// Package dashboard contains shared configuration and logic used by various
// pieces of the Go continuous build system.
package dashboard
buildlet, cmd/coordinator: GCE quota accounting, fixes And deal with Preemptible resource exhaustion errors. And change all-compile to misc-compile and only do the builders not covered otherwise (Fixes #11073) And make the watcher serve git source. And cache and singleflight fetching of git source. And a million other things. Fixes golang/go#11073 Change-Id: I0f45610f0c6a06bd0c8ba9632b8624e00aeb52fc Reviewed-on: https://go-review.googlesource.com/10750 Reviewed-by: Andrew Gerrand <adg@golang.org>
2015-06-05 04:25:50 +03:00
import (
all: split builder config into builder & host configs Our builders are named of the form "GOOS-GOARCH" or "GOOS-GOARCH-suffix". Over time we've grown many builders. This CL doesn't change that. Builders continue to be named and operate as before. Previously the build configuration file (dashboard/builders.go) made each builder type ("linux-amd64-race", etc) define how to create a host running a buildlet of that type, even though many builders had identical host configs. For example, these builders all share the same host type (a Kubernetes container): linux-amd64 linux-amd64-race linux-386 linux-386-387 And these are the same host type (a GCE VM): windows-amd64-gce windows-amd64-race windows-386-gce This CL creates a new concept of a "hostType" which defines how the buildlet is created (Kube, GCE, Reverse, and how), and then each builder itself references a host type. Users never see the hostType. (except perhaps in gomote list output) But they at least never need to care about them. Reverse buildlets now can only be one hostType at a time, which simplifies things. We were no longer using multiple roles per machine once moving to VMs for OS X. gomote continues to operate as it did previously but its underlying protocol changed and clients will need to be updated. As a new feature, gomote now has a new flag to let you reuse a buildlet host connection for different builder rules if they share the same underlying host type. But users can ignore that. This CL is a long-standing TODO (previously attempted and aborted) and will make many things easier and faster, including the linux-arm cross-compilation effort, and keeping pre-warmed buildlets of VM types ready to go. Updates golang/go#17104 Change-Id: Iad8387f48680424a8441e878a2f4762bf79ea4d2 Reviewed-on: https://go-review.googlesource.com/29551 Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2016-09-22 00:27:37 +03:00
"fmt"
cmd/coordinator, cmd/buildlet, cmd/gomote: add SSH support This adds an SSH server to farmer.golang.org on port 2222 that proxies SSH connections to users' gomote-created buildlet instances. For example: $ gomote create openbsd-amd64-60 user-bradfitz-openbsd-amd64-60-1 $ gomote ssh user-bradfitz-openbsd-amd64-60-1 Warning: Permanently added '[localhost]:33351' (ECDSA) to the list of known hosts. OpenBSD 6.0 (GENERIC.MP) golang/go#2319: Tue Jul 26 13:00:43 MDT 2016 Welcome to OpenBSD: The proactively secure Unix-like operating system. Please use the sendbug(1) utility to report bugs in the system. Before reporting a bug, please try to reproduce it with the latest version of the code. With bug reports, please try to ensure that enough information to reproduce the problem is enclosed, and if a known fix for it exists, include that as well. $ As before, if the coordinator process is restarted (or crashes, is evicted, etc), all gomote instances die. Not yet supported: * scp (help wanted) * not all host types are configured. most are. some will need slight config tweaks to the Docker image (e.g. adding openssh-server) Supports currently: * linux-amd64 (host type shared by 386, nacl) * linux-arm * linux-arm64 * darwin * freebsd * openbsd * plan9-386 * windows Implementation details: * the ssh server process listens on port 2222 in the coordinator (farmer.golang.org), which is behind a GKE TCP load balancer. * the ssh server library is github.com/gliderlabs/ssh * authentication is done via Github users' public keys. It's assumed that gomote user == github user. But there's a mapping in the code for known exceptions. * we can't give out access to this too widely. too many things are accessible from within the host environment if you look in the right places. Details omitted. But the Go team and other trusted gomote users can use this. * the buildlet binary has a new /connect-ssh handler that acts like a CONNECT request but instead of taking an explicit host:port, just says "give me your machine's SSH connection". The buildlet can also start sshd if needed for the environment. The /connect-ssh handler also installs the coordinator's public key. * a new buildlet client library method "ConnectSSH" hits the /connect-ssh handler and returns a net.Conn. * the coordinator's ssh.Handler is just running the OpenSSH ssh client. * because the OpenSSH ssh child process can't connect to a net.Conn, an emphemeral localhost port is created on the coordinator to proxy between the ssh client and the net.Conn returned by ConnectSSH. * The /connect-ssh handler requires http.Hijacker, which requires fully compliant net.Conn implementations as of Go 1.8. So I needed to flesh out revdial too, testing it with the golang.org/x/net/nettest package. * plan9 doesn't have an ssh server, so we use 0intro's new conterm program (drawterm without GUI support) to connect to plan9 from the coordinator ssh proxy instead of using the OpenSSH ssh client binary. * windows doesn't have an ssh server, so we enable the telnet service and the coordinator ssh proxy uses telnet instead on the backend on the private network. (There is a Windows ssh server but only in new versions.) Happy debugging over ssh! Fixes golang/go#19956 Change-Id: I80a62064c5f85af1f195f980c862ba29af4015f0 Reviewed-on: https://go-review.googlesource.com/50750 Reviewed-by: Herbie Ong <herbie@google.com> Reviewed-by: Jessie Frazelle <me@jessfraz.com> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2017-07-22 22:15:56 +03:00
"os"
all: misc cleanup, docs, actually add Alpine builder -- move policy of which builders are trybots out of coordinator and into dashboard/builders.go. -- move some GCE-specific code from coordinator.go to gce.go. -- rename an old "watcher" reference to "gitmirror" -- add some docs -- actually add the Alpine builder, missing from https://golang.org/cl/33890 Fixes golang/go#17891 Change-Id: Ia63671ca09aec322ed57b3663e0ac5042cdc56f2 Reviewed-on: https://go-review.googlesource.com/40395 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2017-04-12 03:35:37 +03:00
"sort"
buildlet, cmd/coordinator: GCE quota accounting, fixes And deal with Preemptible resource exhaustion errors. And change all-compile to misc-compile and only do the builders not covered otherwise (Fixes #11073) And make the watcher serve git source. And cache and singleflight fetching of git source. And a million other things. Fixes golang/go#11073 Change-Id: I0f45610f0c6a06bd0c8ba9632b8624e00aeb52fc Reviewed-on: https://go-review.googlesource.com/10750 Reviewed-by: Andrew Gerrand <adg@golang.org>
2015-06-05 04:25:50 +03:00
"strconv"
"strings"
cmd/coordinator, dashboard: scale test exec timeouts by GO_TEST_TIMEOUT_SCALE So slow builders can finish their cgo tests, etc. Updates golang/go#29177 etc Change-Id: If5ee8d2201dc53ab506d4f5e31406d6aae240352 Reviewed-on: https://go-review.googlesource.com/c/162540 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-02-14 05:18:06 +03:00
"time"
all: consolidate configuration for coordinator and gomote buildenv.Environment type defines configuration options: - Coordinator uses the GCE project name to lookup config. A custom config name can be provided at runtime to override. - The conventional prod and stage project names ('symbolic-datum-552' and 'go-dashboard-dev') map to prod and staging configuration structs. - Production and staging status is explicitly defined in configuration. - GCS bucket names for buildlet, logs, and snapshots are configurable. Change-Id: I7e6d7874eb0bdfe35dbdd5fcf6212ab50d576b88 Reviewed-on: https://go-review.googlesource.com/19502 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2016-02-15 02:59:59 +03:00
"golang.org/x/build/buildenv"
dashboard, cmd/coordinator: support multiple builder owners Prefer displaying GitHub usernames for convenience of pinging in GitHub issues, since all current builder owners have them. Keep it possible to use an email or something else if no GitHub account exists. Update the owner of the Corellium builders to be individual GitHub account, so that it can be pinged in an issue. The GitHub organization is already mentioned in the builder notes. Also switch to html/template (instead of text/template) for rendering the builders page. Fixes golang/go#49596. Change-Id: I89a96eb8f7a3057bc1f3aaee16abc9c776c45ee7 Reviewed-on: https://go-review.googlesource.com/c/build/+/363983 Trust: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-11-16 20:42:29 +03:00
"golang.org/x/build/internal/gophers"
dashboard: unify post-submit & trybot policy in builders.go, not coordinator This removes a bunch of TODOs about scattered policy and moves it all into builders.go. While here, * add a bunch of tests * unexport some things * rename some things * document some things * adjust FreeBSD policy as function of branch (per Go 1.12 being last to support FreeBSD 10.x, and to unblock CL 165801) * set GO_BUILDER_SET_GOPROXY=coordinator for reverse buildlets, which is necessary to remove the oauth2 & build special cases in the config * change Elias Naur's mobile builder to how I think he wants it (he was fighting the old system) * add $HOME on the Solaris smartos builder, which was missing & causing tests to fail lately * make the (currently failing) longtest builder have GOPROXY set * remove an allocation in version.ParseReleaseBranch Fixes golang/go#9603 Updates golang/go#14594 Change-Id: I50a23ad7cdf478c95b14bee9b3931ba361baacfa Reviewed-on: https://go-review.googlesource.com/c/build/+/166218 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-03-07 20:44:41 +03:00
"golang.org/x/build/maintner/maintnerd/maintapi/version"
dashboard, cmd/coordinator, maintner/maintnerd: add support for BuildConfig.MinimumGoVersion field The new BuildConfig.MinimumGoVersion field specifies the minimum Go version the builder is allowed to use. It's useful when some of the builders are too new, and do not support all of the supported Go releases (e.g., openbsd-amd64-64 and freebsd-amd64-12_0 currently require Go 1.11 and don't work on Go 1.10). It only needs to be set when a builder doesn't support all supported Go releases, since we don't typically test unsupported Go releases. To allow cmd/coordinator to use this field and filter out work it receives from maintner/maintnerd's GoFindTryWork RPC call, we add a GoVersion slice to apipb.GerritTryWorkItem, and populate it in maintapi.apiService.GoFindTryWork method. For trybots on the Go repo, the GoVersion field is determined from the branch name. For "release-branch.goX.Y" branches, it parses out the major-minor Go version from the branch name. For master and other branches, it assumes the latest Go release. For trybots on subrepos, we already have the Go version information available, so use it directly. Afterwards, all that's left is to modify newTrySet in cmd/coordinator to make use of BuildConfig.MinimumGoVersion and work.GoVersion to skip builders that are too new for the Go version that needs to be tested. Fixes golang/go#29265 Change-Id: I50b01830647e33e37e9eb8b89e0f2518812fa44f Reviewed-on: https://go-review.googlesource.com/c/155463 Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-01-12 00:44:01 +03:00
"golang.org/x/build/types"
buildlet, cmd/coordinator: GCE quota accounting, fixes And deal with Preemptible resource exhaustion errors. And change all-compile to misc-compile and only do the builders not covered otherwise (Fixes #11073) And make the watcher serve git source. And cache and singleflight fetching of git source. And a million other things. Fixes golang/go#11073 Change-Id: I0f45610f0c6a06bd0c8ba9632b8624e00aeb52fc Reviewed-on: https://go-review.googlesource.com/10750 Reviewed-by: Andrew Gerrand <adg@golang.org>
2015-06-05 04:25:50 +03:00
)
dashboard: move builder configuration out of coordinator into a package The new package will be used by the buildlet command line tool in the future. Also use the metadata package in the coordinator for the master build key, since it was already depending on that package. Might as well use it. Change-Id: I6ad7593787ebba364090951ccbaecb4aa38e1dd6 Reviewed-on: https://go-review.googlesource.com/2920 Reviewed-by: Andrew Gerrand <adg@golang.org>
2015-01-15 23:46:22 +03:00
cmd/coordinator: add SlowBots support, opt-in to different slow trybots This parses TRY= comments to opt-in to slower/difference trybots. This needs some docs/UI work yet. Updates golang/go#34501 Change-Id: I13a835520d7ac341bc86139b0a2118235bc83e60 Reviewed-on: https://go-review.googlesource.com/c/build/+/201338 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-10-16 09:02:29 +03:00
// slowBotAliases maps short names from TRY= comments to which builder to run.
//
// TODO: we'll likely expand this, or move it, or change the matching
// syntax entirely. This is a first draft.
var slowBotAliases = map[string]string{
// Known missing builders:
dashboard,cmd/release: replace windows-arm64-aws with windows-arm64-10 This change removes the windows-arm64-aws buildlet and replaces all usages with the windows-arm64-10 buildlet. - Remove the known issue from the windows-arm64-10 buildlet, which has been passing reliably. - Enable tests during releases, and add a slowbot alias, as it's a much faster buildlet. Updates golang/go#46502 For golang/go#42604 Change-Id: Ib9f7c3d5391b01e303b43bbdad030b3f94147c5d Reviewed-on: https://go-review.googlesource.com/c/build/+/331670 Trust: Alexander Rakoczy <alex@golang.org> Run-TryBot: Alexander Rakoczy <alex@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-06-29 18:40:03 +03:00
"ios-amd64": "", // There is no builder for the iOS Simulator. See issues 42100 and 42177.
cmd/coordinator: add SlowBots support, opt-in to different slow trybots This parses TRY= comments to opt-in to slower/difference trybots. This needs some docs/UI work yet. Updates golang/go#34501 Change-Id: I13a835520d7ac341bc86139b0a2118235bc83e60 Reviewed-on: https://go-review.googlesource.com/c/build/+/201338 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-10-16 09:02:29 +03:00
dashboard: improve ppc64 slowbot aliases This improves the selection of slowbot aliases for ppc64. Updates golang/go#42067 Change-Id: I180fe2f49a1e44c422e64c311e601dd93b07f7a7 Reviewed-on: https://go-review.googlesource.com/c/build/+/301769 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Trust: Carlos Amedee <carlos@golang.org>
2021-03-15 17:39:52 +03:00
"386": "linux-386",
"aix": "aix-ppc64",
"amd64": "linux-amd64",
"android": "android-amd64-emu",
"android-386": "android-386-emu",
"android-amd64": "android-amd64-emu",
"android-arm": "android-arm-corellium",
"android-arm64": "android-arm64-corellium",
dashboard: add linux-arm-aws to trybots This change adds linux-arm-aws to the set of trybots. The related changes include: - Renaming the linux-arm builder to linux-arm-scaleway. - Adding slowbot aliases which use linux-arm-aws as the default builder for linux-arm builds. Fixes golang/go#35628 Fixes golang/go#40872 Updates golang/go#45064 Change-Id: Ib49f50e03fc4f766015fdd78080b804c2784a398 Reviewed-on: https://go-review.googlesource.com/c/build/+/303230 Trust: Carlos Amedee <carlos@golang.org> Run-TryBot: Carlos Amedee <carlos@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2021-03-19 20:54:17 +03:00
"arm": "linux-arm-aws",
dashboard: add linux-arm64-aws to set of trybots This change adds linux-arm64-aws to the set of trybots. It also removes linux-arm and linux-arm64 from the list of builds attempted in the misc-compile-linuxarm builder. Updates golang/go#36170 Fixes golang/go#45065 Change-Id: If25fed08e35b2a91c9c9dbbf31701ff5464dc913 Reviewed-on: https://go-review.googlesource.com/c/build/+/303232 Trust: Carlos Amedee <carlos@golang.org> Run-TryBot: Carlos Amedee <carlos@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2021-03-19 21:53:06 +03:00
"arm64": "linux-arm64-aws",
buildlet,dashboard: tweaks to Darwin, OpenBSD, and Linux builders - Add more Monterey builders and remove the known issue: it's stable. Also use it for race and nocgo. Update slowbot aliases to point to it. - Don't test 1.16 on OpenBSD 7.0, it segfaults. - Pin the Linux (ContainerOS) builders to an older version to avoid the pprof kernel bug. For golang/go#49065, golang/go#49149, and golang/go#48977. Change-Id: Ibec2fa735183ec65e5066c7c752ac356b7360550 Reviewed-on: https://go-review.googlesource.com/c/build/+/365777 Trust: Heschi Kreinick <heschi@google.com> Run-TryBot: Heschi Kreinick <heschi@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org>
2021-11-19 22:46:52 +03:00
"darwin": "darwin-amd64-12_0",
"darwin-amd64": "darwin-amd64-12_0",
dashboard: add darwin-arm64-12_0 builder Upgrades our arm64-11_0 builder. For golang/go#49149 Change-Id: I517cc5a999cea0cbc96ee2a6d3901840e2473b6a Reviewed-on: https://go-review.googlesource.com/c/build/+/365736 Trust: Alexander Rakoczy <alex@golang.org> Run-TryBot: Alexander Rakoczy <alex@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2021-11-18 19:31:07 +03:00
"darwin-arm64": "darwin-arm64-12_0-toothrot",
dashboard: improve ppc64 slowbot aliases This improves the selection of slowbot aliases for ppc64. Updates golang/go#42067 Change-Id: I180fe2f49a1e44c422e64c311e601dd93b07f7a7 Reviewed-on: https://go-review.googlesource.com/c/build/+/301769 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Trust: Carlos Amedee <carlos@golang.org>
2021-03-15 17:39:52 +03:00
"ios-arm64": "ios-arm64-corellium",
"dragonfly": "dragonfly-amd64",
dashboard: update freebsd-amd64-race builder host This change updates the version of FreeBSD used for the freebsd-amd64-race builder to 13.0. The slowbot aliases have been updated to use FreeBSD 13.0. Updates golang/go#46272 Updates golang/go#45903 Change-Id: I1092f9e6bf66a9f429b463dce5d0d5647d44eec8 Reviewed-on: https://go-review.googlesource.com/c/build/+/365616 Trust: Carlos Amedee <carlos@golang.org> Run-TryBot: Carlos Amedee <carlos@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2021-11-19 18:11:34 +03:00
"freebsd": "freebsd-amd64-13_0",
"freebsd-386": "freebsd-386-13_0",
"freebsd-amd64": "freebsd-amd64-13_0",
dashboard: improve ppc64 slowbot aliases This improves the selection of slowbot aliases for ppc64. Updates golang/go#42067 Change-Id: I180fe2f49a1e44c422e64c311e601dd93b07f7a7 Reviewed-on: https://go-review.googlesource.com/c/build/+/301769 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Trust: Carlos Amedee <carlos@golang.org>
2021-03-15 17:39:52 +03:00
"freebsd-arm": "freebsd-arm-paulzhol",
"freebsd-arm64": "freebsd-arm64-dmgk",
"illumos": "illumos-amd64",
"ios": "ios-arm64-corellium",
"js": "js-wasm",
"linux": "linux-amd64",
dashboard: add linux-arm-aws to trybots This change adds linux-arm-aws to the set of trybots. The related changes include: - Renaming the linux-arm builder to linux-arm-scaleway. - Adding slowbot aliases which use linux-arm-aws as the default builder for linux-arm builds. Fixes golang/go#35628 Fixes golang/go#40872 Updates golang/go#45064 Change-Id: Ib49f50e03fc4f766015fdd78080b804c2784a398 Reviewed-on: https://go-review.googlesource.com/c/build/+/303230 Trust: Carlos Amedee <carlos@golang.org> Run-TryBot: Carlos Amedee <carlos@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2021-03-19 20:54:17 +03:00
"linux-arm": "linux-arm-aws",
dashboard: add linux-arm64-aws to set of trybots This change adds linux-arm64-aws to the set of trybots. It also removes linux-arm and linux-arm64 from the list of builds attempted in the misc-compile-linuxarm builder. Updates golang/go#36170 Fixes golang/go#45065 Change-Id: If25fed08e35b2a91c9c9dbbf31701ff5464dc913 Reviewed-on: https://go-review.googlesource.com/c/build/+/303232 Trust: Carlos Amedee <carlos@golang.org> Run-TryBot: Carlos Amedee <carlos@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2021-03-19 21:53:06 +03:00
"linux-arm64": "linux-arm64-aws",
dashboard: add linux-loong64-3a5000 builder Updates golang/go#46229 Change-Id: Ib48e1e64d96a07d5c5282d78935124faf75c89d7 Reviewed-on: https://go-review.googlesource.com/c/build/+/347309 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org> Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Trust: Cherry Mui <cherryyz@google.com>
2021-09-02 10:51:28 +03:00
"linux-loong64": "linux-loong64-3a5000",
dashboard: improve ppc64 slowbot aliases This improves the selection of slowbot aliases for ppc64. Updates golang/go#42067 Change-Id: I180fe2f49a1e44c422e64c311e601dd93b07f7a7 Reviewed-on: https://go-review.googlesource.com/c/build/+/301769 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Trust: Carlos Amedee <carlos@golang.org>
2021-03-15 17:39:52 +03:00
"linux-mips": "linux-mips-rtrk",
"linux-mips64": "linux-mips64-rtrk",
"linux-mips64le": "linux-mips64le-mengzhuo",
"linux-mipsle": "linux-mipsle-rtrk",
"linux-ppc64": "linux-ppc64-buildlet",
"linux-ppc64le": "linux-ppc64le-buildlet",
"linux-ppc64le-power9": "linux-ppc64le-power9osu",
"linux-riscv64": "linux-riscv64-jsing",
"linux-s390x": "linux-s390x-ibm",
"longtest": "linux-amd64-longtest",
"mac": "darwin-amd64-10_14",
"macos": "darwin-amd64-10_14",
"mips": "linux-mips-rtrk",
"mips64": "linux-mips64-rtrk",
"mips64le": "linux-mips64le-mengzhuo",
"mipsle": "linux-mipsle-rtrk",
"netbsd": "netbsd-amd64-9_0",
"netbsd-386": "netbsd-386-9_0",
"netbsd-amd64": "netbsd-amd64-9_0",
"netbsd-arm": "netbsd-arm-bsiegert",
"netbsd-arm64": "netbsd-arm64-bsiegert",
"nocgo": "linux-amd64-nocgo",
"openbsd": "openbsd-amd64-68",
"openbsd-386": "openbsd-386-68",
"openbsd-amd64": "openbsd-amd64-68",
"openbsd-arm": "openbsd-arm-jsing",
"openbsd-arm64": "openbsd-arm64-jsing",
"openbsd-mips64": "openbsd-mips64-jsing",
"plan9": "plan9-arm",
"plan9-386": "plan9-386-0intro",
dashboard: rename plan9-amd64-9front builder to plan9-386-0intro The builder name was misleading, because plan9-amd64-9front hasn't been running 9front anymore since 2016, but is running the 9k kernel from Bell Labs. Change-Id: I6929583b784d076a86b818649c64b821a538ed1c Reviewed-on: https://go-review.googlesource.com/c/build/+/307409 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Trust: Carlos Amedee <carlos@golang.org> Trust: David du Colombier <0intro@gmail.com> Run-TryBot: Carlos Amedee <carlos@golang.org> TryBot-Result: Go Bot <gobot@golang.org>
2021-04-05 21:42:05 +03:00
"plan9-amd64": "plan9-amd64-0intro",
dashboard: improve ppc64 slowbot aliases This improves the selection of slowbot aliases for ppc64. Updates golang/go#42067 Change-Id: I180fe2f49a1e44c422e64c311e601dd93b07f7a7 Reviewed-on: https://go-review.googlesource.com/c/build/+/301769 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Trust: Carlos Amedee <carlos@golang.org>
2021-03-15 17:39:52 +03:00
"ppc64": "linux-ppc64-buildlet",
"ppc64le": "linux-ppc64le-buildlet",
"ppc64lep9": "linux-ppc64le-power9osu",
"riscv64": "linux-riscv64-jsing",
"s390x": "linux-s390x-ibm",
"solaris": "solaris-amd64-oraclerel",
"solaris-amd64": "solaris-amd64-oraclerel",
"wasm": "js-wasm",
"windows": "windows-amd64-2016",
dashboard: do not sunset windows 2008 CL 363474 added a 2012 builder, which is good, but it also took a major step in sunsetting Windows 7 support by removing all 2008 builders from trybots and for Go 1.18+. This will surely bitrot Windows 7 support and force us to remove support for it, which isn't okay. Rather than line up dominoes to force our hand on policy, such huge policy changes with sweeping ramifications should be properly decided. So, this commit keeps the 2008 builders in the running, alongside the 2012 ones. Updates golang/go#47845. Updates golang/go#49490. Change-Id: I574f51ea22f638d2f9b09fcd00fdfc3097631771 Reviewed-on: https://go-review.googlesource.com/c/build/+/366656 Trust: Jason A. Donenfeld <Jason@zx2c4.com> Trust: Carlos Amedee <carlos@golang.org> Run-TryBot: Jason A. Donenfeld <Jason@zx2c4.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-11-23 23:55:31 +03:00
"windows-386": "windows-386-2008",
dashboard: improve ppc64 slowbot aliases This improves the selection of slowbot aliases for ppc64. Updates golang/go#42067 Change-Id: I180fe2f49a1e44c422e64c311e601dd93b07f7a7 Reviewed-on: https://go-review.googlesource.com/c/build/+/301769 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Trust: Carlos Amedee <carlos@golang.org>
2021-03-15 17:39:52 +03:00
"windows-amd64": "windows-amd64-2016",
"windows-arm": "windows-arm-zx2c4",
dashboard,cmd/release: replace windows-arm64-aws with windows-arm64-10 This change removes the windows-arm64-aws buildlet and replaces all usages with the windows-arm64-10 buildlet. - Remove the known issue from the windows-arm64-10 buildlet, which has been passing reliably. - Enable tests during releases, and add a slowbot alias, as it's a much faster buildlet. Updates golang/go#46502 For golang/go#42604 Change-Id: Ib9f7c3d5391b01e303b43bbdad030b3f94147c5d Reviewed-on: https://go-review.googlesource.com/c/build/+/331670 Trust: Alexander Rakoczy <alex@golang.org> Run-TryBot: Alexander Rakoczy <alex@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-06-29 18:40:03 +03:00
"windows-arm64": "windows-arm64-10",
cmd/coordinator: add SlowBots support, opt-in to different slow trybots This parses TRY= comments to opt-in to slower/difference trybots. This needs some docs/UI work yet. Updates golang/go#34501 Change-Id: I13a835520d7ac341bc86139b0a2118235bc83e60 Reviewed-on: https://go-review.googlesource.com/c/build/+/201338 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-10-16 09:02:29 +03:00
}
dashboard: move builder configuration out of coordinator into a package The new package will be used by the buildlet command line tool in the future. Also use the metadata package in the coordinator for the master build key, since it was already depending on that package. Might as well use it. Change-Id: I6ad7593787ebba364090951ccbaecb4aa38e1dd6 Reviewed-on: https://go-review.googlesource.com/2920 Reviewed-by: Andrew Gerrand <adg@golang.org>
2015-01-15 23:46:22 +03:00
// Builders are the different build configurations.
// The keys are like "darwin-amd64" or "linux-386-387".
// This map should not be modified by other packages.
all: misc cleanup, docs, actually add Alpine builder -- move policy of which builders are trybots out of coordinator and into dashboard/builders.go. -- move some GCE-specific code from coordinator.go to gce.go. -- rename an old "watcher" reference to "gitmirror" -- add some docs -- actually add the Alpine builder, missing from https://golang.org/cl/33890 Fixes golang/go#17891 Change-Id: Ia63671ca09aec322ed57b3663e0ac5042cdc56f2 Reviewed-on: https://go-review.googlesource.com/40395 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2017-04-12 03:35:37 +03:00
// Initialization happens below, via calls to addBuilder.
all: change dashboard.BuildConfig to be a pointer type, like HostConfig BuildConfig grew way too large and has too many slices & such to be a value type. Make it a pointer type, which matches HostConfig. Change-Id: Ie625bece9d6d8c1ec6cff26e77416bd9b1f256d8 Reviewed-on: https://go-review.googlesource.com/c/145077 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2018-10-26 20:52:25 +03:00
var Builders = map[string]*BuildConfig{}
dashboard: move builder configuration out of coordinator into a package The new package will be used by the buildlet command line tool in the future. Also use the metadata package in the coordinator for the master build key, since it was already depending on that package. Might as well use it. Change-Id: I6ad7593787ebba364090951ccbaecb4aa38e1dd6 Reviewed-on: https://go-review.googlesource.com/2920 Reviewed-by: Andrew Gerrand <adg@golang.org>
2015-01-15 23:46:22 +03:00
all: split builder config into builder & host configs Our builders are named of the form "GOOS-GOARCH" or "GOOS-GOARCH-suffix". Over time we've grown many builders. This CL doesn't change that. Builders continue to be named and operate as before. Previously the build configuration file (dashboard/builders.go) made each builder type ("linux-amd64-race", etc) define how to create a host running a buildlet of that type, even though many builders had identical host configs. For example, these builders all share the same host type (a Kubernetes container): linux-amd64 linux-amd64-race linux-386 linux-386-387 And these are the same host type (a GCE VM): windows-amd64-gce windows-amd64-race windows-386-gce This CL creates a new concept of a "hostType" which defines how the buildlet is created (Kube, GCE, Reverse, and how), and then each builder itself references a host type. Users never see the hostType. (except perhaps in gomote list output) But they at least never need to care about them. Reverse buildlets now can only be one hostType at a time, which simplifies things. We were no longer using multiple roles per machine once moving to VMs for OS X. gomote continues to operate as it did previously but its underlying protocol changed and clients will need to be updated. As a new feature, gomote now has a new flag to let you reuse a buildlet host connection for different builder rules if they share the same underlying host type. But users can ignore that. This CL is a long-standing TODO (previously attempted and aborted) and will make many things easier and faster, including the linux-arm cross-compilation effort, and keeping pre-warmed buildlets of VM types ready to go. Updates golang/go#17104 Change-Id: Iad8387f48680424a8441e878a2f4762bf79ea4d2 Reviewed-on: https://go-review.googlesource.com/29551 Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2016-09-22 00:27:37 +03:00
// Hosts contains the names and configs of all the types of
// buildlets. They can be VMs, containers, or dedicated machines.
var Hosts = map[string]*HostConfig{
env/linux-x86-bullseye, dashboard: add Debian Bullseye builder This change adds the host-linux-bullseye host as well as the configuration for the host. The linux-amd64-bullseye builder has been added in this change as well. Fixes golang/go#47720 Change-Id: I607f937133a0db3ed6a841f53dc7fef5e310a5a4 Reviewed-on: https://go-review.googlesource.com/c/build/+/342630 Trust: Carlos Amedee <carlos@golang.org> Run-TryBot: Carlos Amedee <carlos@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org>
2021-08-16 22:35:39 +03:00
"host-linux-bullseye": &HostConfig{
Notes: "Debian Bullseye",
ContainerImage: "linux-x86-bullseye:latest",
buildletURLTmpl: "https://storage.googleapis.com/$BUCKET/buildlet.linux-amd64",
env: []string{"GOROOT_BOOTSTRAP=/go1.4"},
SSHUsername: "root",
},
env/linux-x86-buster: add Debian Buster builder This change adds a Debian Buster builder. Fixes golang/go#42410 Change-Id: If11ebe6df3bad88aac40b0a6f841edc5fb3a5b4a Reviewed-on: https://go-review.googlesource.com/c/build/+/267937 Trust: Carlos Amedee <carlos@golang.org> Run-TryBot: Carlos Amedee <carlos@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2020-11-02 23:11:10 +03:00
"host-linux-buster": &HostConfig{
Notes: "Debian Buster",
ContainerImage: "linux-x86-buster:latest",
buildletURLTmpl: "https://storage.googleapis.com/$BUCKET/buildlet.linux-amd64",
env: []string{"GOROOT_BOOTSTRAP=/go1.4"},
SSHUsername: "root",
},
dashboard, env: add Debian Stretch builder, rename Jessie builder Updates golang/go#25386 Change-Id: I65e3dda63e5ea86fa94480e3305556b73afe121a Reviewed-on: https://go-review.googlesource.com/113059 Reviewed-by: Bryan C. Mills <bcmills@google.com>
2018-05-14 20:22:45 +03:00
"host-linux-jessie": &HostConfig{
all: use Container-Optimized VMs instead of Kubernetes for buildlet containers Fixes golang/go#25108 Change-Id: I084669b52b699700ed26a7fdd890d9205a8b9dc9 Reviewed-on: https://go-review.googlesource.com/111267 Reviewed-by: Andrew Bonventre <andybons@golang.org>
2018-05-04 05:42:17 +03:00
Notes: "Debian Jessie, our standard Linux container image.",
dashboard, env: add Debian Stretch builder, rename Jessie builder Updates golang/go#25386 Change-Id: I65e3dda63e5ea86fa94480e3305556b73afe121a Reviewed-on: https://go-review.googlesource.com/113059 Reviewed-by: Bryan C. Mills <bcmills@google.com>
2018-05-14 20:22:45 +03:00
ContainerImage: "linux-x86-jessie:latest",
buildletURLTmpl: "http://storage.googleapis.com/$BUCKET/buildlet.linux-amd64",
env: []string{"GOROOT_BOOTSTRAP=/go1.4"},
SSHUsername: "root",
},
"host-linux-stretch": &HostConfig{
Notes: "Debian Stretch",
ContainerImage: "linux-x86-stretch:latest",
all: migrate from n1 to e2 instances We should have similar performance for these machines, and increased memory availability in some areas. This is also compliant with an internal policy change. Change-Id: I07a452a07431322d6fdd4edbb98c5914045d34ed Reviewed-on: https://go-review.googlesource.com/c/build/+/354757 Trust: Alexander Rakoczy <alex@golang.org> Run-TryBot: Alexander Rakoczy <alex@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Heschi Kreinick <heschi@google.com> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-10-09 00:03:24 +03:00
machineType: "e2-standard-4", // 4 vCPUs, 16 GB mem
dashboard: increase timeout scale for longtest builders, add more CPU Updates golang/go#25886 Change-Id: I5168e291ab77cbd3843bdc39e319a68dfa65aedd Reviewed-on: https://go-review.googlesource.com/c/build/+/167638 Reviewed-by: Russ Cox <rsc@golang.org>
2019-03-14 19:06:57 +03:00
buildletURLTmpl: "http://storage.googleapis.com/$BUCKET/buildlet.linux-amd64",
env: []string{"GOROOT_BOOTSTRAP=/go1.4"},
SSHUsername: "root",
},
"host-linux-stretch-morecpu": &HostConfig{
all: migrate from n1 to e2 instances We should have similar performance for these machines, and increased memory availability in some areas. This is also compliant with an internal policy change. Change-Id: I07a452a07431322d6fdd4edbb98c5914045d34ed Reviewed-on: https://go-review.googlesource.com/c/build/+/354757 Trust: Alexander Rakoczy <alex@golang.org> Run-TryBot: Alexander Rakoczy <alex@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Heschi Kreinick <heschi@google.com> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-10-09 00:03:24 +03:00
Notes: "Debian Stretch, but on e2-highcpu-16",
dashboard: increase timeout scale for longtest builders, add more CPU Updates golang/go#25886 Change-Id: I5168e291ab77cbd3843bdc39e319a68dfa65aedd Reviewed-on: https://go-review.googlesource.com/c/build/+/167638 Reviewed-by: Russ Cox <rsc@golang.org>
2019-03-14 19:06:57 +03:00
ContainerImage: "linux-x86-stretch:latest",
all: migrate from n1 to e2 instances We should have similar performance for these machines, and increased memory availability in some areas. This is also compliant with an internal policy change. Change-Id: I07a452a07431322d6fdd4edbb98c5914045d34ed Reviewed-on: https://go-review.googlesource.com/c/build/+/354757 Trust: Alexander Rakoczy <alex@golang.org> Run-TryBot: Alexander Rakoczy <alex@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Heschi Kreinick <heschi@google.com> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-10-09 00:03:24 +03:00
machineType: "e2-highcpu-16", // 16 vCPUs, 16 GB mem
all: split builder config into builder & host configs Our builders are named of the form "GOOS-GOARCH" or "GOOS-GOARCH-suffix". Over time we've grown many builders. This CL doesn't change that. Builders continue to be named and operate as before. Previously the build configuration file (dashboard/builders.go) made each builder type ("linux-amd64-race", etc) define how to create a host running a buildlet of that type, even though many builders had identical host configs. For example, these builders all share the same host type (a Kubernetes container): linux-amd64 linux-amd64-race linux-386 linux-386-387 And these are the same host type (a GCE VM): windows-amd64-gce windows-amd64-race windows-386-gce This CL creates a new concept of a "hostType" which defines how the buildlet is created (Kube, GCE, Reverse, and how), and then each builder itself references a host type. Users never see the hostType. (except perhaps in gomote list output) But they at least never need to care about them. Reverse buildlets now can only be one hostType at a time, which simplifies things. We were no longer using multiple roles per machine once moving to VMs for OS X. gomote continues to operate as it did previously but its underlying protocol changed and clients will need to be updated. As a new feature, gomote now has a new flag to let you reuse a buildlet host connection for different builder rules if they share the same underlying host type. But users can ignore that. This CL is a long-standing TODO (previously attempted and aborted) and will make many things easier and faster, including the linux-arm cross-compilation effort, and keeping pre-warmed buildlets of VM types ready to go. Updates golang/go#17104 Change-Id: Iad8387f48680424a8441e878a2f4762bf79ea4d2 Reviewed-on: https://go-review.googlesource.com/29551 Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2016-09-22 00:27:37 +03:00
buildletURLTmpl: "http://storage.googleapis.com/$BUCKET/buildlet.linux-amd64",
env: []string{"GOROOT_BOOTSTRAP=/go1.4"},
cmd/coordinator, cmd/buildlet, cmd/gomote: add SSH support This adds an SSH server to farmer.golang.org on port 2222 that proxies SSH connections to users' gomote-created buildlet instances. For example: $ gomote create openbsd-amd64-60 user-bradfitz-openbsd-amd64-60-1 $ gomote ssh user-bradfitz-openbsd-amd64-60-1 Warning: Permanently added '[localhost]:33351' (ECDSA) to the list of known hosts. OpenBSD 6.0 (GENERIC.MP) golang/go#2319: Tue Jul 26 13:00:43 MDT 2016 Welcome to OpenBSD: The proactively secure Unix-like operating system. Please use the sendbug(1) utility to report bugs in the system. Before reporting a bug, please try to reproduce it with the latest version of the code. With bug reports, please try to ensure that enough information to reproduce the problem is enclosed, and if a known fix for it exists, include that as well. $ As before, if the coordinator process is restarted (or crashes, is evicted, etc), all gomote instances die. Not yet supported: * scp (help wanted) * not all host types are configured. most are. some will need slight config tweaks to the Docker image (e.g. adding openssh-server) Supports currently: * linux-amd64 (host type shared by 386, nacl) * linux-arm * linux-arm64 * darwin * freebsd * openbsd * plan9-386 * windows Implementation details: * the ssh server process listens on port 2222 in the coordinator (farmer.golang.org), which is behind a GKE TCP load balancer. * the ssh server library is github.com/gliderlabs/ssh * authentication is done via Github users' public keys. It's assumed that gomote user == github user. But there's a mapping in the code for known exceptions. * we can't give out access to this too widely. too many things are accessible from within the host environment if you look in the right places. Details omitted. But the Go team and other trusted gomote users can use this. * the buildlet binary has a new /connect-ssh handler that acts like a CONNECT request but instead of taking an explicit host:port, just says "give me your machine's SSH connection". The buildlet can also start sshd if needed for the environment. The /connect-ssh handler also installs the coordinator's public key. * a new buildlet client library method "ConnectSSH" hits the /connect-ssh handler and returns a net.Conn. * the coordinator's ssh.Handler is just running the OpenSSH ssh client. * because the OpenSSH ssh child process can't connect to a net.Conn, an emphemeral localhost port is created on the coordinator to proxy between the ssh client and the net.Conn returned by ConnectSSH. * The /connect-ssh handler requires http.Hijacker, which requires fully compliant net.Conn implementations as of Go 1.8. So I needed to flesh out revdial too, testing it with the golang.org/x/net/nettest package. * plan9 doesn't have an ssh server, so we use 0intro's new conterm program (drawterm without GUI support) to connect to plan9 from the coordinator ssh proxy instead of using the OpenSSH ssh client binary. * windows doesn't have an ssh server, so we enable the telnet service and the coordinator ssh proxy uses telnet instead on the backend on the private network. (There is a Windows ssh server but only in new versions.) Happy debugging over ssh! Fixes golang/go#19956 Change-Id: I80a62064c5f85af1f195f980c862ba29af4015f0 Reviewed-on: https://go-review.googlesource.com/50750 Reviewed-by: Herbie Ong <herbie@google.com> Reviewed-by: Jessie Frazelle <me@jessfraz.com> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2017-07-22 22:15:56 +03:00
SSHUsername: "root",
all: split builder config into builder & host configs Our builders are named of the form "GOOS-GOARCH" or "GOOS-GOARCH-suffix". Over time we've grown many builders. This CL doesn't change that. Builders continue to be named and operate as before. Previously the build configuration file (dashboard/builders.go) made each builder type ("linux-amd64-race", etc) define how to create a host running a buildlet of that type, even though many builders had identical host configs. For example, these builders all share the same host type (a Kubernetes container): linux-amd64 linux-amd64-race linux-386 linux-386-387 And these are the same host type (a GCE VM): windows-amd64-gce windows-amd64-race windows-386-gce This CL creates a new concept of a "hostType" which defines how the buildlet is created (Kube, GCE, Reverse, and how), and then each builder itself references a host type. Users never see the hostType. (except perhaps in gomote list output) But they at least never need to care about them. Reverse buildlets now can only be one hostType at a time, which simplifies things. We were no longer using multiple roles per machine once moving to VMs for OS X. gomote continues to operate as it did previously but its underlying protocol changed and clients will need to be updated. As a new feature, gomote now has a new flag to let you reuse a buildlet host connection for different builder rules if they share the same underlying host type. But users can ignore that. This CL is a long-standing TODO (previously attempted and aborted) and will make many things easier and faster, including the linux-arm cross-compilation effort, and keeping pre-warmed buildlets of VM types ready to go. Updates golang/go#17104 Change-Id: Iad8387f48680424a8441e878a2f4762bf79ea4d2 Reviewed-on: https://go-review.googlesource.com/29551 Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2016-09-22 00:27:37 +03:00
},
dashboard, buildlet: add a disabled builder with nested virt, for testing This adds a linux-amd64 COS builder that should be just like our existing linux-amd64 COS builder except that it's using a forked image that has the VMX license bit enabled for nested virtualization. (GCE appears to be using the license mechanism as some sort of opt-in mechanism for features that aren't yet GA; might go away?) Once this is in, it won't do any new builds as regular+trybot builders are disabled. But it means I can then use gomote + debugnewvm to work on preparing the other four image types. Updates golang/go#15581 (solaris) Updates golang/go#23060 (dragonfly) Updates golang/go#30262 (riscv) Updates golang/go#30267 (fuchsia) Updates golang/go#23824 (android) Change-Id: Ic55f17eea17908dba7f58618d8cd162a2ed9b015 Reviewed-on: https://go-review.googlesource.com/c/162959 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-02-16 01:05:42 +03:00
"host-linux-stretch-vmx": &HostConfig{
Notes: "Debian Stretch w/ Nested Virtualization (VMX CPU bit) enabled, for testing",
ContainerImage: "linux-x86-stretch:latest",
dashboard: use n2 instances where necessary e2 and n2d instances do not support specifying a MinCPUVersion or NestedVirt. For golang/go#48969 Change-Id: I0c4b359b29cd24302b869748014f848a9b7b5e39 Reviewed-on: https://go-review.googlesource.com/c/build/+/356071 Trust: Alexander Rakoczy <alex@golang.org> Run-TryBot: Alexander Rakoczy <alex@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-10-15 04:54:32 +03:00
machineType: "n2-highcpu-4", // e2 instances do not support MinCPUPlatform or NestedVirt.
dashboard, buildlet: add a disabled builder with nested virt, for testing This adds a linux-amd64 COS builder that should be just like our existing linux-amd64 COS builder except that it's using a forked image that has the VMX license bit enabled for nested virtualization. (GCE appears to be using the license mechanism as some sort of opt-in mechanism for features that aren't yet GA; might go away?) Once this is in, it won't do any new builds as regular+trybot builders are disabled. But it means I can then use gomote + debugnewvm to work on preparing the other four image types. Updates golang/go#15581 (solaris) Updates golang/go#23060 (dragonfly) Updates golang/go#30262 (riscv) Updates golang/go#30267 (fuchsia) Updates golang/go#23824 (android) Change-Id: Ic55f17eea17908dba7f58618d8cd162a2ed9b015 Reviewed-on: https://go-review.googlesource.com/c/162959 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-02-16 01:05:42 +03:00
NestedVirt: true,
buildletURLTmpl: "http://storage.googleapis.com/$BUCKET/buildlet.linux-amd64",
env: []string{"GOROOT_BOOTSTRAP=/go1.4"},
SSHUsername: "root",
},
cmd/coordinator: cross-compile ARM build on Kubernetes Saves 4.5 minutes or so by using fast x86 machines to build the ARM build instead of running make.bash on Scaleway ARM machines. We still run the tests on ARM, and have a separate builder only running make.bash on ARM (see prior golang.org/cl/29670) Fixes golang/go#17105 Updates golang/go#17104 Change-Id: I1cb7b0e5b1cc8b644195f262328884ed3aff120a Reviewed-on: https://go-review.googlesource.com/29677 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2016-09-23 23:44:43 +03:00
"host-linux-armhf-cross": &HostConfig{
env/crosscompile: update the three cross compile envs from Jessie to Buster The earlier fix to remove IN_KUBERNETES in buildlet/stage0/stage0.go in CL 201739 caused Kubernetes-based cross compilation builds on fail to start up because I forgot to rebuild & push those container images. But when I went to re-push them, I found they're too old (Debian Jessie) and don't build anymore. So update them to Debian Buster. While I'm here, remove the Debian release name from their image name, as this won't be the last time we need to upgrade them. Unlike real builders, the OS release isn't the point of these containers, as they don't run tests. They exist only to do builds, so the OS isn't interesting or worth calling out. Updates golang/go#34956 Change-Id: Iee4691a821421b161a9333b4441f83ff45d1ffec Reviewed-on: https://go-review.googlesource.com/c/build/+/202021 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-10-19 07:02:04 +03:00
Notes: "Debian with armhf cross-compiler, built from env/crosscompile/linux-armhf",
ContainerImage: "linux-armhf-cross:latest",
cmd/coordinator: cross-compile ARM build on Kubernetes Saves 4.5 minutes or so by using fast x86 machines to build the ARM build instead of running make.bash on Scaleway ARM machines. We still run the tests on ARM, and have a separate builder only running make.bash on ARM (see prior golang.org/cl/29670) Fixes golang/go#17105 Updates golang/go#17104 Change-Id: I1cb7b0e5b1cc8b644195f262328884ed3aff120a Reviewed-on: https://go-review.googlesource.com/29677 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2016-09-23 23:44:43 +03:00
buildletURLTmpl: "http://storage.googleapis.com/$BUCKET/buildlet.linux-amd64",
env: []string{"GOROOT_BOOTSTRAP=/go1.4"},
},
cmd/coordinator, dashboard: cross compile linux-arm-arm5spacemonkey Change-Id: I3a228627ec8236d0d10bb9a1a61443c81493145d Reviewed-on: https://go-review.googlesource.com/38460 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2017-03-23 18:46:39 +03:00
"host-linux-armel-cross": &HostConfig{
env/crosscompile: update the three cross compile envs from Jessie to Buster The earlier fix to remove IN_KUBERNETES in buildlet/stage0/stage0.go in CL 201739 caused Kubernetes-based cross compilation builds on fail to start up because I forgot to rebuild & push those container images. But when I went to re-push them, I found they're too old (Debian Jessie) and don't build anymore. So update them to Debian Buster. While I'm here, remove the Debian release name from their image name, as this won't be the last time we need to upgrade them. Unlike real builders, the OS release isn't the point of these containers, as they don't run tests. They exist only to do builds, so the OS isn't interesting or worth calling out. Updates golang/go#34956 Change-Id: Iee4691a821421b161a9333b4441f83ff45d1ffec Reviewed-on: https://go-review.googlesource.com/c/build/+/202021 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-10-19 07:02:04 +03:00
Notes: "Debian with armel cross-compiler, from env/crosscompile/linux-armel",
ContainerImage: "linux-armel-cross:latest",
cmd/coordinator, dashboard: cross compile linux-arm-arm5spacemonkey Change-Id: I3a228627ec8236d0d10bb9a1a61443c81493145d Reviewed-on: https://go-review.googlesource.com/38460 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2017-03-23 18:46:39 +03:00
buildletURLTmpl: "http://storage.googleapis.com/$BUCKET/buildlet.linux-amd64",
env: []string{"GOROOT_BOOTSTRAP=/go1.4"},
},
cmd/coordinator, cmd/buildlet, cmd/gomote: add SSH support This adds an SSH server to farmer.golang.org on port 2222 that proxies SSH connections to users' gomote-created buildlet instances. For example: $ gomote create openbsd-amd64-60 user-bradfitz-openbsd-amd64-60-1 $ gomote ssh user-bradfitz-openbsd-amd64-60-1 Warning: Permanently added '[localhost]:33351' (ECDSA) to the list of known hosts. OpenBSD 6.0 (GENERIC.MP) golang/go#2319: Tue Jul 26 13:00:43 MDT 2016 Welcome to OpenBSD: The proactively secure Unix-like operating system. Please use the sendbug(1) utility to report bugs in the system. Before reporting a bug, please try to reproduce it with the latest version of the code. With bug reports, please try to ensure that enough information to reproduce the problem is enclosed, and if a known fix for it exists, include that as well. $ As before, if the coordinator process is restarted (or crashes, is evicted, etc), all gomote instances die. Not yet supported: * scp (help wanted) * not all host types are configured. most are. some will need slight config tweaks to the Docker image (e.g. adding openssh-server) Supports currently: * linux-amd64 (host type shared by 386, nacl) * linux-arm * linux-arm64 * darwin * freebsd * openbsd * plan9-386 * windows Implementation details: * the ssh server process listens on port 2222 in the coordinator (farmer.golang.org), which is behind a GKE TCP load balancer. * the ssh server library is github.com/gliderlabs/ssh * authentication is done via Github users' public keys. It's assumed that gomote user == github user. But there's a mapping in the code for known exceptions. * we can't give out access to this too widely. too many things are accessible from within the host environment if you look in the right places. Details omitted. But the Go team and other trusted gomote users can use this. * the buildlet binary has a new /connect-ssh handler that acts like a CONNECT request but instead of taking an explicit host:port, just says "give me your machine's SSH connection". The buildlet can also start sshd if needed for the environment. The /connect-ssh handler also installs the coordinator's public key. * a new buildlet client library method "ConnectSSH" hits the /connect-ssh handler and returns a net.Conn. * the coordinator's ssh.Handler is just running the OpenSSH ssh client. * because the OpenSSH ssh child process can't connect to a net.Conn, an emphemeral localhost port is created on the coordinator to proxy between the ssh client and the net.Conn returned by ConnectSSH. * The /connect-ssh handler requires http.Hijacker, which requires fully compliant net.Conn implementations as of Go 1.8. So I needed to flesh out revdial too, testing it with the golang.org/x/net/nettest package. * plan9 doesn't have an ssh server, so we use 0intro's new conterm program (drawterm without GUI support) to connect to plan9 from the coordinator ssh proxy instead of using the OpenSSH ssh client binary. * windows doesn't have an ssh server, so we enable the telnet service and the coordinator ssh proxy uses telnet instead on the backend on the private network. (There is a Windows ssh server but only in new versions.) Happy debugging over ssh! Fixes golang/go#19956 Change-Id: I80a62064c5f85af1f195f980c862ba29af4015f0 Reviewed-on: https://go-review.googlesource.com/50750 Reviewed-by: Herbie Ong <herbie@google.com> Reviewed-by: Jessie Frazelle <me@jessfraz.com> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2017-07-22 22:15:56 +03:00
"host-linux-amd64-localdev": &HostConfig{
IsReverse: true,
ExpectNum: 0,
Notes: "for localhost development of buildlets/gomote/coordinator only",
SSHUsername: os.Getenv("USER"),
},
env/js-wasm, dashboard: add start of a js-wasm builder Updates golang/go#18892 Change-Id: I4b16afe8f0ce74f6ecd022f8bca14ada94ab308f Reviewed-on: https://go-review.googlesource.com/112736 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-05-11 08:54:32 +03:00
"host-js-wasm": &HostConfig{
Notes: "Container with node.js for testing js/wasm.",
ContainerImage: "js-wasm:latest",
buildletURLTmpl: "http://storage.googleapis.com/$BUCKET/buildlet.linux-amd64",
env: []string{"GOROOT_BOOTSTRAP=/go1.4"},
dashboard, env, cmd/buildlet/testssh: fix gomote ssh for a number of buidlers And add a testssh tool to validate that SSH is working. Updates golang/go#32430 Change-Id: I5182419fa4db31b598f7a02412fb4ecc4060a796 Reviewed-on: https://go-review.googlesource.com/c/build/+/207459 Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-11-15 23:31:23 +03:00
SSHUsername: "root",
env/js-wasm, dashboard: add start of a js-wasm builder Updates golang/go#18892 Change-Id: I4b16afe8f0ce74f6ecd022f8bca14ada94ab308f Reviewed-on: https://go-review.googlesource.com/112736 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-05-11 08:54:32 +03:00
},
dashboard: remove misleading -kube suffixes from host types They used to run on Kubernetes (and still could) but they're just a container and we run them on COS nowadays. Change-Id: I21a6db90efc5071997c6d6dcae1a42a7c28afa3a Reviewed-on: https://go-review.googlesource.com/c/build/+/209957 Reviewed-by: Alexander Rakoczy <alex@golang.org> Run-TryBot: Alexander Rakoczy <alex@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-12-05 00:27:28 +03:00
"host-s390x-cross": &HostConfig{
all: use Container-Optimized VMs instead of Kubernetes for buildlet containers Fixes golang/go#25108 Change-Id: I084669b52b699700ed26a7fdd890d9205a8b9dc9 Reviewed-on: https://go-review.googlesource.com/111267 Reviewed-by: Andrew Bonventre <andybons@golang.org>
2018-05-04 05:42:17 +03:00
Notes: "Container with s390x cross-compiler.",
env/crosscompile: update the three cross compile envs from Jessie to Buster The earlier fix to remove IN_KUBERNETES in buildlet/stage0/stage0.go in CL 201739 caused Kubernetes-based cross compilation builds on fail to start up because I forgot to rebuild & push those container images. But when I went to re-push them, I found they're too old (Debian Jessie) and don't build anymore. So update them to Debian Buster. While I'm here, remove the Debian release name from their image name, as this won't be the last time we need to upgrade them. Unlike real builders, the OS release isn't the point of these containers, as they don't run tests. They exist only to do builds, so the OS isn't interesting or worth calling out. Updates golang/go#34956 Change-Id: Iee4691a821421b161a9333b4441f83ff45d1ffec Reviewed-on: https://go-review.googlesource.com/c/build/+/202021 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-10-19 07:02:04 +03:00
ContainerImage: "linux-s390x-cross:latest",
all: split builder config into builder & host configs Our builders are named of the form "GOOS-GOARCH" or "GOOS-GOARCH-suffix". Over time we've grown many builders. This CL doesn't change that. Builders continue to be named and operate as before. Previously the build configuration file (dashboard/builders.go) made each builder type ("linux-amd64-race", etc) define how to create a host running a buildlet of that type, even though many builders had identical host configs. For example, these builders all share the same host type (a Kubernetes container): linux-amd64 linux-amd64-race linux-386 linux-386-387 And these are the same host type (a GCE VM): windows-amd64-gce windows-amd64-race windows-386-gce This CL creates a new concept of a "hostType" which defines how the buildlet is created (Kube, GCE, Reverse, and how), and then each builder itself references a host type. Users never see the hostType. (except perhaps in gomote list output) But they at least never need to care about them. Reverse buildlets now can only be one hostType at a time, which simplifies things. We were no longer using multiple roles per machine once moving to VMs for OS X. gomote continues to operate as it did previously but its underlying protocol changed and clients will need to be updated. As a new feature, gomote now has a new flag to let you reuse a buildlet host connection for different builder rules if they share the same underlying host type. But users can ignore that. This CL is a long-standing TODO (previously attempted and aborted) and will make many things easier and faster, including the linux-arm cross-compilation effort, and keeping pre-warmed buildlets of VM types ready to go. Updates golang/go#17104 Change-Id: Iad8387f48680424a8441e878a2f4762bf79ea4d2 Reviewed-on: https://go-review.googlesource.com/29551 Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2016-09-22 00:27:37 +03:00
buildletURLTmpl: "https://storage.googleapis.com/$BUCKET/buildlet.linux-amd64",
env: []string{"GOROOT_BOOTSTRAP=/go1.4"},
},
env: add alpine linux builder Adds flag to upload to build binary statically. Updates golang/go#17891 Change-Id: If19b2e2cf9a84fa2b4cc6fdf55d8add5b54abff7 Reviewed-on: https://go-review.googlesource.com/33890 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2016-12-02 22:14:12 +03:00
"host-linux-x86-alpine": &HostConfig{
all: use Container-Optimized VMs instead of Kubernetes for buildlet containers Fixes golang/go#25108 Change-Id: I084669b52b699700ed26a7fdd890d9205a8b9dc9 Reviewed-on: https://go-review.googlesource.com/111267 Reviewed-by: Andrew Bonventre <andybons@golang.org>
2018-05-04 05:42:17 +03:00
Notes: "Alpine container",
all: rename Kube to Container and IsGCE to IsVM Once containers run on COS instead of Kubernetes, one name (Kube*) is wrong and the other (GCE) is ambiguous. So rename them now to be more specific. No behavior changes. Just renaming in this step, to reduce size of next CL. Updates golang/go#25108 Change-Id: Ib09eb682ef74acbbf6ed50b46074f834ef5e0c0b Reviewed-on: https://go-review.googlesource.com/111639 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-05-05 19:36:05 +03:00
ContainerImage: "linux-x86-alpine:latest",
dashboard, env, cmd/buildlet/testssh: fix gomote ssh for a number of buidlers And add a testssh tool to validate that SSH is working. Updates golang/go#32430 Change-Id: I5182419fa4db31b598f7a02412fb4ecc4060a796 Reviewed-on: https://go-review.googlesource.com/c/build/+/207459 Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-11-15 23:31:23 +03:00
buildletURLTmpl: "https://storage.googleapis.com/$BUCKET/buildlet.linux-amd64-static",
env: add alpine linux builder Adds flag to upload to build binary statically. Updates golang/go#17891 Change-Id: If19b2e2cf9a84fa2b4cc6fdf55d8add5b54abff7 Reviewed-on: https://go-review.googlesource.com/33890 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2016-12-02 22:14:12 +03:00
env: []string{"GOROOT_BOOTSTRAP=/usr/lib/go"},
dashboard, env, cmd/buildlet/testssh: fix gomote ssh for a number of buidlers And add a testssh tool to validate that SSH is working. Updates golang/go#32430 Change-Id: I5182419fa4db31b598f7a02412fb4ecc4060a796 Reviewed-on: https://go-review.googlesource.com/c/build/+/207459 Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-11-15 23:31:23 +03:00
SSHUsername: "root",
env: add alpine linux builder Adds flag to upload to build binary statically. Updates golang/go#17891 Change-Id: If19b2e2cf9a84fa2b4cc6fdf55d8add5b54abff7 Reviewed-on: https://go-review.googlesource.com/33890 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2016-12-02 22:14:12 +03:00
},
all: split builder config into builder & host configs Our builders are named of the form "GOOS-GOARCH" or "GOOS-GOARCH-suffix". Over time we've grown many builders. This CL doesn't change that. Builders continue to be named and operate as before. Previously the build configuration file (dashboard/builders.go) made each builder type ("linux-amd64-race", etc) define how to create a host running a buildlet of that type, even though many builders had identical host configs. For example, these builders all share the same host type (a Kubernetes container): linux-amd64 linux-amd64-race linux-386 linux-386-387 And these are the same host type (a GCE VM): windows-amd64-gce windows-amd64-race windows-386-gce This CL creates a new concept of a "hostType" which defines how the buildlet is created (Kube, GCE, Reverse, and how), and then each builder itself references a host type. Users never see the hostType. (except perhaps in gomote list output) But they at least never need to care about them. Reverse buildlets now can only be one hostType at a time, which simplifies things. We were no longer using multiple roles per machine once moving to VMs for OS X. gomote continues to operate as it did previously but its underlying protocol changed and clients will need to be updated. As a new feature, gomote now has a new flag to let you reuse a buildlet host connection for different builder rules if they share the same underlying host type. But users can ignore that. This CL is a long-standing TODO (previously attempted and aborted) and will make many things easier and faster, including the linux-arm cross-compilation effort, and keeping pre-warmed buildlets of VM types ready to go. Updates golang/go#17104 Change-Id: Iad8387f48680424a8441e878a2f4762bf79ea4d2 Reviewed-on: https://go-review.googlesource.com/29551 Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2016-09-22 00:27:37 +03:00
"host-linux-clang": &HostConfig{
all: use Container-Optimized VMs instead of Kubernetes for buildlet containers Fixes golang/go#25108 Change-Id: I084669b52b699700ed26a7fdd890d9205a8b9dc9 Reviewed-on: https://go-review.googlesource.com/111267 Reviewed-by: Andrew Bonventre <andybons@golang.org>
2018-05-04 05:42:17 +03:00
Notes: "Container with clang.",
all: rename Kube to Container and IsGCE to IsVM Once containers run on COS instead of Kubernetes, one name (Kube*) is wrong and the other (GCE) is ambiguous. So rename them now to be more specific. No behavior changes. Just renaming in this step, to reduce size of next CL. Updates golang/go#25108 Change-Id: Ib09eb682ef74acbbf6ed50b46074f834ef5e0c0b Reviewed-on: https://go-review.googlesource.com/111639 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-05-05 19:36:05 +03:00
ContainerImage: "linux-x86-clang:latest",
all: split builder config into builder & host configs Our builders are named of the form "GOOS-GOARCH" or "GOOS-GOARCH-suffix". Over time we've grown many builders. This CL doesn't change that. Builders continue to be named and operate as before. Previously the build configuration file (dashboard/builders.go) made each builder type ("linux-amd64-race", etc) define how to create a host running a buildlet of that type, even though many builders had identical host configs. For example, these builders all share the same host type (a Kubernetes container): linux-amd64 linux-amd64-race linux-386 linux-386-387 And these are the same host type (a GCE VM): windows-amd64-gce windows-amd64-race windows-386-gce This CL creates a new concept of a "hostType" which defines how the buildlet is created (Kube, GCE, Reverse, and how), and then each builder itself references a host type. Users never see the hostType. (except perhaps in gomote list output) But they at least never need to care about them. Reverse buildlets now can only be one hostType at a time, which simplifies things. We were no longer using multiple roles per machine once moving to VMs for OS X. gomote continues to operate as it did previously but its underlying protocol changed and clients will need to be updated. As a new feature, gomote now has a new flag to let you reuse a buildlet host connection for different builder rules if they share the same underlying host type. But users can ignore that. This CL is a long-standing TODO (previously attempted and aborted) and will make many things easier and faster, including the linux-arm cross-compilation effort, and keeping pre-warmed buildlets of VM types ready to go. Updates golang/go#17104 Change-Id: Iad8387f48680424a8441e878a2f4762bf79ea4d2 Reviewed-on: https://go-review.googlesource.com/29551 Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2016-09-22 00:27:37 +03:00
buildletURLTmpl: "http://storage.googleapis.com/$BUCKET/buildlet.linux-amd64",
env: []string{"GOROOT_BOOTSTRAP=/go1.4"},
dashboard, env, cmd/buildlet/testssh: fix gomote ssh for a number of buidlers And add a testssh tool to validate that SSH is working. Updates golang/go#32430 Change-Id: I5182419fa4db31b598f7a02412fb4ecc4060a796 Reviewed-on: https://go-review.googlesource.com/c/build/+/207459 Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-11-15 23:31:23 +03:00
SSHUsername: "root",
all: split builder config into builder & host configs Our builders are named of the form "GOOS-GOARCH" or "GOOS-GOARCH-suffix". Over time we've grown many builders. This CL doesn't change that. Builders continue to be named and operate as before. Previously the build configuration file (dashboard/builders.go) made each builder type ("linux-amd64-race", etc) define how to create a host running a buildlet of that type, even though many builders had identical host configs. For example, these builders all share the same host type (a Kubernetes container): linux-amd64 linux-amd64-race linux-386 linux-386-387 And these are the same host type (a GCE VM): windows-amd64-gce windows-amd64-race windows-386-gce This CL creates a new concept of a "hostType" which defines how the buildlet is created (Kube, GCE, Reverse, and how), and then each builder itself references a host type. Users never see the hostType. (except perhaps in gomote list output) But they at least never need to care about them. Reverse buildlets now can only be one hostType at a time, which simplifies things. We were no longer using multiple roles per machine once moving to VMs for OS X. gomote continues to operate as it did previously but its underlying protocol changed and clients will need to be updated. As a new feature, gomote now has a new flag to let you reuse a buildlet host connection for different builder rules if they share the same underlying host type. But users can ignore that. This CL is a long-standing TODO (previously attempted and aborted) and will make many things easier and faster, including the linux-arm cross-compilation effort, and keeping pre-warmed buildlets of VM types ready to go. Updates golang/go#17104 Change-Id: Iad8387f48680424a8441e878a2f4762bf79ea4d2 Reviewed-on: https://go-review.googlesource.com/29551 Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2016-09-22 00:27:37 +03:00
},
"host-linux-sid": &HostConfig{
env/linux-x86-sid: convert from GCE VM to Kubernetes And update it. What we used to call "sid" was probably ancient. Change-Id: I76c1aad60eb655513119eaad0550ba5e0df3e9c5 Reviewed-on: https://go-review.googlesource.com/40399 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2017-04-12 04:54:09 +03:00
Notes: "Debian sid, updated occasionally.",
all: rename Kube to Container and IsGCE to IsVM Once containers run on COS instead of Kubernetes, one name (Kube*) is wrong and the other (GCE) is ambiguous. So rename them now to be more specific. No behavior changes. Just renaming in this step, to reduce size of next CL. Updates golang/go#25108 Change-Id: Ib09eb682ef74acbbf6ed50b46074f834ef5e0c0b Reviewed-on: https://go-review.googlesource.com/111639 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-05-05 19:36:05 +03:00
ContainerImage: "linux-x86-sid:latest",
all: split builder config into builder & host configs Our builders are named of the form "GOOS-GOARCH" or "GOOS-GOARCH-suffix". Over time we've grown many builders. This CL doesn't change that. Builders continue to be named and operate as before. Previously the build configuration file (dashboard/builders.go) made each builder type ("linux-amd64-race", etc) define how to create a host running a buildlet of that type, even though many builders had identical host configs. For example, these builders all share the same host type (a Kubernetes container): linux-amd64 linux-amd64-race linux-386 linux-386-387 And these are the same host type (a GCE VM): windows-amd64-gce windows-amd64-race windows-386-gce This CL creates a new concept of a "hostType" which defines how the buildlet is created (Kube, GCE, Reverse, and how), and then each builder itself references a host type. Users never see the hostType. (except perhaps in gomote list output) But they at least never need to care about them. Reverse buildlets now can only be one hostType at a time, which simplifies things. We were no longer using multiple roles per machine once moving to VMs for OS X. gomote continues to operate as it did previously but its underlying protocol changed and clients will need to be updated. As a new feature, gomote now has a new flag to let you reuse a buildlet host connection for different builder rules if they share the same underlying host type. But users can ignore that. This CL is a long-standing TODO (previously attempted and aborted) and will make many things easier and faster, including the linux-arm cross-compilation effort, and keeping pre-warmed buildlets of VM types ready to go. Updates golang/go#17104 Change-Id: Iad8387f48680424a8441e878a2f4762bf79ea4d2 Reviewed-on: https://go-review.googlesource.com/29551 Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2016-09-22 00:27:37 +03:00
buildletURLTmpl: "http://storage.googleapis.com/$BUCKET/buildlet.linux-amd64",
env: []string{"GOROOT_BOOTSTRAP=/go1.4"},
dashboard, env, cmd/buildlet/testssh: fix gomote ssh for a number of buidlers And add a testssh tool to validate that SSH is working. Updates golang/go#32430 Change-Id: I5182419fa4db31b598f7a02412fb4ecc4060a796 Reviewed-on: https://go-review.googlesource.com/c/build/+/207459 Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-11-15 23:31:23 +03:00
SSHUsername: "root",
all: split builder config into builder & host configs Our builders are named of the form "GOOS-GOARCH" or "GOOS-GOARCH-suffix". Over time we've grown many builders. This CL doesn't change that. Builders continue to be named and operate as before. Previously the build configuration file (dashboard/builders.go) made each builder type ("linux-amd64-race", etc) define how to create a host running a buildlet of that type, even though many builders had identical host configs. For example, these builders all share the same host type (a Kubernetes container): linux-amd64 linux-amd64-race linux-386 linux-386-387 And these are the same host type (a GCE VM): windows-amd64-gce windows-amd64-race windows-386-gce This CL creates a new concept of a "hostType" which defines how the buildlet is created (Kube, GCE, Reverse, and how), and then each builder itself references a host type. Users never see the hostType. (except perhaps in gomote list output) But they at least never need to care about them. Reverse buildlets now can only be one hostType at a time, which simplifies things. We were no longer using multiple roles per machine once moving to VMs for OS X. gomote continues to operate as it did previously but its underlying protocol changed and clients will need to be updated. As a new feature, gomote now has a new flag to let you reuse a buildlet host connection for different builder rules if they share the same underlying host type. But users can ignore that. This CL is a long-standing TODO (previously attempted and aborted) and will make many things easier and faster, including the linux-arm cross-compilation effort, and keeping pre-warmed buildlets of VM types ready to go. Updates golang/go#17104 Change-Id: Iad8387f48680424a8441e878a2f4762bf79ea4d2 Reviewed-on: https://go-review.googlesource.com/29551 Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2016-09-22 00:27:37 +03:00
},
env/linux-x86-fedora: add Fedora builder Not using a Fedora kernel yet, but better than nothing. Updates golang/go#26796 Change-Id: I228442bf6606696407a8d3fbd098df59df7353e1 Reviewed-on: https://go-review.googlesource.com/c/build/+/175999 Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-05-08 19:25:38 +03:00
"host-linux-fedora": &HostConfig{
Notes: "Fedora 30",
ContainerImage: "linux-x86-fedora:latest",
buildletURLTmpl: "http://storage.googleapis.com/$BUCKET/buildlet.linux-amd64",
env: []string{"GOROOT_BOOTSTRAP=/goboot"},
dashboard, env, cmd/buildlet/testssh: fix gomote ssh for a number of buidlers And add a testssh tool to validate that SSH is working. Updates golang/go#32430 Change-Id: I5182419fa4db31b598f7a02412fb4ecc4060a796 Reviewed-on: https://go-review.googlesource.com/c/build/+/207459 Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-11-15 23:31:23 +03:00
SSHUsername: "root",
env/linux-x86-fedora: add Fedora builder Not using a Fedora kernel yet, but better than nothing. Updates golang/go#26796 Change-Id: I228442bf6606696407a8d3fbd098df59df7353e1 Reviewed-on: https://go-review.googlesource.com/c/build/+/175999 Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-05-08 19:25:38 +03:00
},
dashboard: add linux-riscv64-jsing builder (reverse) Also use this as the riscv64 slowbot, since linux-riscv64-unleashed has been MIA for quite some time. For golang/go#27532 Change-Id: I01309a66b96d4c35dbc9d1e08f55f5357b0acba0 Reviewed-on: https://go-review.googlesource.com/c/build/+/257917 Trust: Joel Sing <joel@sing.id.au> Run-TryBot: Joel Sing <joel@sing.id.au> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2020-09-27 21:25:41 +03:00
"host-linux-riscv64-joelsing": &HostConfig{
dashboard, cmd/coordinator: support multiple builder owners Prefer displaying GitHub usernames for convenience of pinging in GitHub issues, since all current builder owners have them. Keep it possible to use an email or something else if no GitHub account exists. Update the owner of the Corellium builders to be individual GitHub account, so that it can be pinged in an issue. The GitHub organization is already mentioned in the builder notes. Also switch to html/template (instead of text/template) for rendering the builders page. Fixes golang/go#49596. Change-Id: I89a96eb8f7a3057bc1f3aaee16abc9c776c45ee7 Reviewed-on: https://go-review.googlesource.com/c/build/+/363983 Trust: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-11-16 20:42:29 +03:00
Notes: "SiFive HiFive Unleashed RISC-V board. 8 GB RAM, 4 cores.",
IsReverse: true,
ExpectNum: 1,
Owners: []*gophers.Person{gh("4a6f656c")},
env: []string{"GOROOT_BOOTSTRAP=/usr/local/goboot"},
dashboard: add linux-riscv64-jsing builder (reverse) Also use this as the riscv64 slowbot, since linux-riscv64-unleashed has been MIA for quite some time. For golang/go#27532 Change-Id: I01309a66b96d4c35dbc9d1e08f55f5357b0acba0 Reviewed-on: https://go-review.googlesource.com/c/build/+/257917 Trust: Joel Sing <joel@sing.id.au> Run-TryBot: Joel Sing <joel@sing.id.au> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2020-09-27 21:25:41 +03:00
},
dashboard: add linux-riscv64-unmatched This adds a reverse buildlet for linux riscv64 running under Hifive Unmatched. For golang/go#46804. Change-Id: If5ed8b47205cecebcb90a4d1b03c463e1eb466c7 Reviewed-on: https://go-review.googlesource.com/c/build/+/329489 Trust: Meng Zhuo <mzh@golangcn.org> Run-TryBot: Meng Zhuo <mzh@golangcn.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2021-08-27 07:26:14 +03:00
"host-linux-riscv64-unmatched": &HostConfig{
dashboard, cmd/coordinator: support multiple builder owners Prefer displaying GitHub usernames for convenience of pinging in GitHub issues, since all current builder owners have them. Keep it possible to use an email or something else if no GitHub account exists. Update the owner of the Corellium builders to be individual GitHub account, so that it can be pinged in an issue. The GitHub organization is already mentioned in the builder notes. Also switch to html/template (instead of text/template) for rendering the builders page. Fixes golang/go#49596. Change-Id: I89a96eb8f7a3057bc1f3aaee16abc9c776c45ee7 Reviewed-on: https://go-review.googlesource.com/c/build/+/363983 Trust: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-11-16 20:42:29 +03:00
Notes: "SiFive HiFive Unmatched RISC-V board. 16 GB RAM, 4 cores.",
IsReverse: true,
ExpectNum: 1,
Owners: []*gophers.Person{gh("mengzhuo")},
env: []string{"GOROOT_BOOTSTRAP=/usr/local/goboot"},
dashboard: add linux-riscv64-unmatched This adds a reverse buildlet for linux riscv64 running under Hifive Unmatched. For golang/go#46804. Change-Id: If5ed8b47205cecebcb90a4d1b03c463e1eb466c7 Reviewed-on: https://go-review.googlesource.com/c/build/+/329489 Trust: Meng Zhuo <mzh@golangcn.org> Run-TryBot: Meng Zhuo <mzh@golangcn.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2021-08-27 07:26:14 +03:00
},
dashboard, cmd/coordinator: add linux-riscv64 builder, fix bug from CL 205608 Updates golang/go#27532 Updates golang/go#30262 Change-Id: If1593ab6e5653a9dce625f25a16bf8a082796592 Reviewed-on: https://go-review.googlesource.com/c/build/+/206639 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-11-12 09:19:21 +03:00
"host-linux-riscv64-unleashed": &HostConfig{
dashboard, cmd/coordinator: support multiple builder owners Prefer displaying GitHub usernames for convenience of pinging in GitHub issues, since all current builder owners have them. Keep it possible to use an email or something else if no GitHub account exists. Update the owner of the Corellium builders to be individual GitHub account, so that it can be pinged in an issue. The GitHub organization is already mentioned in the builder notes. Also switch to html/template (instead of text/template) for rendering the builders page. Fixes golang/go#49596. Change-Id: I89a96eb8f7a3057bc1f3aaee16abc9c776c45ee7 Reviewed-on: https://go-review.googlesource.com/c/build/+/363983 Trust: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-11-16 20:42:29 +03:00
Notes: "SiFive HiFive Unleashed RISC-V board. 8 GB RAM, 4 cores.",
IsReverse: true,
dashboard: update ExpectNum for host-linux-riscv64-unleashed to 0 Take a small step to bring the dashboard towards the current state of the builder. For golang/go#49325. Change-Id: I4a2aa4efcb374ad6d2affa167e0eba2c55f52e50 Reviewed-on: https://go-review.googlesource.com/c/build/+/365695 Trust: Dmitri Shuralyov <dmitshur@golang.org> Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Heschi Kreinick <heschi@google.com>
2021-11-19 21:00:01 +03:00
ExpectNum: 0, // See golang.org/issue/49325.
dashboard, cmd/coordinator: support multiple builder owners Prefer displaying GitHub usernames for convenience of pinging in GitHub issues, since all current builder owners have them. Keep it possible to use an email or something else if no GitHub account exists. Update the owner of the Corellium builders to be individual GitHub account, so that it can be pinged in an issue. The GitHub organization is already mentioned in the builder notes. Also switch to html/template (instead of text/template) for rendering the builders page. Fixes golang/go#49596. Change-Id: I89a96eb8f7a3057bc1f3aaee16abc9c776c45ee7 Reviewed-on: https://go-review.googlesource.com/c/build/+/363983 Trust: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-11-16 20:42:29 +03:00
Owners: []*gophers.Person{gh("bradfitz")}, // at home
env: []string{"GOROOT_BOOTSTRAP=/usr/local/goboot"},
dashboard, cmd/coordinator: add linux-riscv64 builder, fix bug from CL 205608 Updates golang/go#27532 Updates golang/go#30262 Change-Id: If1593ab6e5653a9dce625f25a16bf8a082796592 Reviewed-on: https://go-review.googlesource.com/c/build/+/206639 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-11-12 09:19:21 +03:00
},
dashboard: add OpenBSD 6.8 builders This change builds on top of CL 268297, using the updated env script to build OpenBSD 6.8 images, and adding new builders that use them. This has been tested with cmd/debugnewvm on a recent Go tip commit, as well as Go 1.15.4 and 1.14.11 commits. It passed, although there were also some failures seen. Once the builder is added, we'll know the test results for a wider set of commits, and can decide how to proceed from there. Minimize the configuration to a bare minimum. Specifically, remove MinCPUPlatform since setting it "Intel Skylake" was found not to have a noticeable effect on the speed of the build compared to the zero value: # With MinCPUPlatform = "Intel Skylake": $ time debugnewvm -run-build=openbsd-amd64-68 -rev=go1.15.4 [...] real 17m8.308s # With a zero MinCPUPlatform value: $ time debugnewvm -run-build=openbsd-amd64-68 -rev=go1.15.4 [...] real 16m28.518s For golang/go#35712. Updates golang/go#42426. Change-Id: I517a85738a034a3a4074488ccecd4408a91a4a07 Reviewed-on: https://go-review.googlesource.com/c/build/+/269577 Trust: Dmitri Shuralyov <dmitshur@golang.org> Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
2020-11-12 20:27:32 +03:00
"host-openbsd-amd64-68": &HostConfig{
dashboard: update OpenBSD 6.8 images with 009_exit syspatch, take 2 These images were regenerated after the fix in CL 278732. For golang/go#35712. Updates golang/go#42542. Updates golang/go#42426. Change-Id: Ib9c9dc316e4f68e7fed2cafe5400942a94ba8fd2 Reviewed-on: https://go-review.googlesource.com/c/build/+/279134 Trust: Dmitri Shuralyov <dmitshur@golang.org> Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Joel Sing <joel@sing.id.au>
2020-12-18 07:53:39 +03:00
VMImage: "openbsd-amd64-68-v3", // v3 adds 009_exit syspatch; see golang.org/cl/278732.
all: migrate from n1 to e2 instances We should have similar performance for these machines, and increased memory availability in some areas. This is also compliant with an internal policy change. Change-Id: I07a452a07431322d6fdd4edbb98c5914045d34ed Reviewed-on: https://go-review.googlesource.com/c/build/+/354757 Trust: Alexander Rakoczy <alex@golang.org> Run-TryBot: Alexander Rakoczy <alex@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Heschi Kreinick <heschi@google.com> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-10-09 00:03:24 +03:00
machineType: "e2-highcpu-4",
dashboard: add OpenBSD 6.8 builders This change builds on top of CL 268297, using the updated env script to build OpenBSD 6.8 images, and adding new builders that use them. This has been tested with cmd/debugnewvm on a recent Go tip commit, as well as Go 1.15.4 and 1.14.11 commits. It passed, although there were also some failures seen. Once the builder is added, we'll know the test results for a wider set of commits, and can decide how to proceed from there. Minimize the configuration to a bare minimum. Specifically, remove MinCPUPlatform since setting it "Intel Skylake" was found not to have a noticeable effect on the speed of the build compared to the zero value: # With MinCPUPlatform = "Intel Skylake": $ time debugnewvm -run-build=openbsd-amd64-68 -rev=go1.15.4 [...] real 17m8.308s # With a zero MinCPUPlatform value: $ time debugnewvm -run-build=openbsd-amd64-68 -rev=go1.15.4 [...] real 16m28.518s For golang/go#35712. Updates golang/go#42426. Change-Id: I517a85738a034a3a4074488ccecd4408a91a4a07 Reviewed-on: https://go-review.googlesource.com/c/build/+/269577 Trust: Dmitri Shuralyov <dmitshur@golang.org> Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
2020-11-12 20:27:32 +03:00
buildletURLTmpl: "https://storage.googleapis.com/$BUCKET/buildlet.openbsd-amd64",
goBootstrapURLTmpl: "https://storage.googleapis.com/$BUCKET/gobootstrap-openbsd-amd64-go1_12.tar.gz",
dashboard: update OpenBSD 6.8 images with 009_exit syspatch For golang/go#35712. Updates golang/go#42542. Updates golang/go#42426. Change-Id: Ifbdd025b8d1708e715ba312c438f391e1aeaeff8 Reviewed-on: https://go-review.googlesource.com/c/build/+/277115 Trust: Dmitri Shuralyov <dmitshur@golang.org> Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Joel Sing <joel@sing.id.au>
2020-12-11 00:17:58 +03:00
Notes: "OpenBSD 6.8 (with 009_exit syspatch); GCE VM is built from script in build/env/openbsd-amd64",
dashboard: add OpenBSD 6.8 builders This change builds on top of CL 268297, using the updated env script to build OpenBSD 6.8 images, and adding new builders that use them. This has been tested with cmd/debugnewvm on a recent Go tip commit, as well as Go 1.15.4 and 1.14.11 commits. It passed, although there were also some failures seen. Once the builder is added, we'll know the test results for a wider set of commits, and can decide how to proceed from there. Minimize the configuration to a bare minimum. Specifically, remove MinCPUPlatform since setting it "Intel Skylake" was found not to have a noticeable effect on the speed of the build compared to the zero value: # With MinCPUPlatform = "Intel Skylake": $ time debugnewvm -run-build=openbsd-amd64-68 -rev=go1.15.4 [...] real 17m8.308s # With a zero MinCPUPlatform value: $ time debugnewvm -run-build=openbsd-amd64-68 -rev=go1.15.4 [...] real 16m28.518s For golang/go#35712. Updates golang/go#42426. Change-Id: I517a85738a034a3a4074488ccecd4408a91a4a07 Reviewed-on: https://go-review.googlesource.com/c/build/+/269577 Trust: Dmitri Shuralyov <dmitshur@golang.org> Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
2020-11-12 20:27:32 +03:00
SSHUsername: "gopher",
},
"host-openbsd-386-68": &HostConfig{
dashboard: update OpenBSD 6.8 images with 009_exit syspatch, take 2 These images were regenerated after the fix in CL 278732. For golang/go#35712. Updates golang/go#42542. Updates golang/go#42426. Change-Id: Ib9c9dc316e4f68e7fed2cafe5400942a94ba8fd2 Reviewed-on: https://go-review.googlesource.com/c/build/+/279134 Trust: Dmitri Shuralyov <dmitshur@golang.org> Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Joel Sing <joel@sing.id.au>
2020-12-18 07:53:39 +03:00
VMImage: "openbsd-386-68-v3", // v3 adds 009_exit syspatch; see golang.org/cl/278732.
all: migrate from n1 to e2 instances We should have similar performance for these machines, and increased memory availability in some areas. This is also compliant with an internal policy change. Change-Id: I07a452a07431322d6fdd4edbb98c5914045d34ed Reviewed-on: https://go-review.googlesource.com/c/build/+/354757 Trust: Alexander Rakoczy <alex@golang.org> Run-TryBot: Alexander Rakoczy <alex@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Heschi Kreinick <heschi@google.com> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-10-09 00:03:24 +03:00
machineType: "e2-highcpu-4",
dashboard: add OpenBSD 6.8 builders This change builds on top of CL 268297, using the updated env script to build OpenBSD 6.8 images, and adding new builders that use them. This has been tested with cmd/debugnewvm on a recent Go tip commit, as well as Go 1.15.4 and 1.14.11 commits. It passed, although there were also some failures seen. Once the builder is added, we'll know the test results for a wider set of commits, and can decide how to proceed from there. Minimize the configuration to a bare minimum. Specifically, remove MinCPUPlatform since setting it "Intel Skylake" was found not to have a noticeable effect on the speed of the build compared to the zero value: # With MinCPUPlatform = "Intel Skylake": $ time debugnewvm -run-build=openbsd-amd64-68 -rev=go1.15.4 [...] real 17m8.308s # With a zero MinCPUPlatform value: $ time debugnewvm -run-build=openbsd-amd64-68 -rev=go1.15.4 [...] real 16m28.518s For golang/go#35712. Updates golang/go#42426. Change-Id: I517a85738a034a3a4074488ccecd4408a91a4a07 Reviewed-on: https://go-review.googlesource.com/c/build/+/269577 Trust: Dmitri Shuralyov <dmitshur@golang.org> Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
2020-11-12 20:27:32 +03:00
buildletURLTmpl: "https://storage.googleapis.com/$BUCKET/buildlet.openbsd-386",
goBootstrapURLTmpl: "https://storage.googleapis.com/$BUCKET/gobootstrap-openbsd-386-go1_12.tar.gz",
dashboard: update OpenBSD 6.8 images with 009_exit syspatch For golang/go#35712. Updates golang/go#42542. Updates golang/go#42426. Change-Id: Ifbdd025b8d1708e715ba312c438f391e1aeaeff8 Reviewed-on: https://go-review.googlesource.com/c/build/+/277115 Trust: Dmitri Shuralyov <dmitshur@golang.org> Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Joel Sing <joel@sing.id.au>
2020-12-11 00:17:58 +03:00
Notes: "OpenBSD 6.8 (with 009_exit syspatch); GCE VM is built from script in build/env/openbsd-386",
dashboard: add OpenBSD 6.8 builders This change builds on top of CL 268297, using the updated env script to build OpenBSD 6.8 images, and adding new builders that use them. This has been tested with cmd/debugnewvm on a recent Go tip commit, as well as Go 1.15.4 and 1.14.11 commits. It passed, although there were also some failures seen. Once the builder is added, we'll know the test results for a wider set of commits, and can decide how to proceed from there. Minimize the configuration to a bare minimum. Specifically, remove MinCPUPlatform since setting it "Intel Skylake" was found not to have a noticeable effect on the speed of the build compared to the zero value: # With MinCPUPlatform = "Intel Skylake": $ time debugnewvm -run-build=openbsd-amd64-68 -rev=go1.15.4 [...] real 17m8.308s # With a zero MinCPUPlatform value: $ time debugnewvm -run-build=openbsd-amd64-68 -rev=go1.15.4 [...] real 16m28.518s For golang/go#35712. Updates golang/go#42426. Change-Id: I517a85738a034a3a4074488ccecd4408a91a4a07 Reviewed-on: https://go-review.googlesource.com/c/build/+/269577 Trust: Dmitri Shuralyov <dmitshur@golang.org> Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
2020-11-12 20:27:32 +03:00
SSHUsername: "gopher",
},
env,dashboard: add OpenBSD 7.0 all.bash takes a long time on these builders. I'll keep an eye on them to see if it causes trouble. For golang/go#48977. Change-Id: I0239a93c004157855968dab397e68abe38f5f439 Reviewed-on: https://go-review.googlesource.com/c/build/+/364094 Trust: Heschi Kreinick <heschi@google.com> Run-TryBot: Heschi Kreinick <heschi@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-11-15 23:02:19 +03:00
"host-openbsd-amd64-70": &HostConfig{
VMImage: "openbsd-amd64-70",
machineType: "e2-highcpu-4",
buildletURLTmpl: "https://storage.googleapis.com/$BUCKET/buildlet.openbsd-amd64",
goBootstrapURLTmpl: "https://storage.googleapis.com/$BUCKET/gobootstrap-openbsd-amd64-go1_12.tar.gz",
Notes: "OpenBSD 7.0; GCE VM is built from script in build/env/openbsd-amd64",
SSHUsername: "gopher",
},
dashboard: add n1 hosts for OpenBSD and NetBSD We recently migrated in CL 354757 from n1 to e2 hosts around the same time issues started appearing with these builders. n1 hosts are notably different in that they are older generation CPUs with hyperthreading disabled. For golang/go#49209 Change-Id: I8b6822fbd4910602f61d504b34580282c97c0ca7 Reviewed-on: https://go-review.googlesource.com/c/build/+/365776 Trust: Alexander Rakoczy <alex@golang.org> Run-TryBot: Alexander Rakoczy <alex@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2021-11-19 22:22:11 +03:00
"host-openbsd-amd64-70-n1": &HostConfig{
VMImage: "openbsd-amd64-70",
machineType: "n1-highcpu-4",
buildletURLTmpl: "https://storage.googleapis.com/$BUCKET/buildlet.openbsd-amd64",
goBootstrapURLTmpl: "https://storage.googleapis.com/$BUCKET/gobootstrap-openbsd-amd64-go1_12.tar.gz",
Notes: "OpenBSD 7.0; GCE VM is built from script in build/env/openbsd-amd64. n1-highcpu host.",
SSHUsername: "gopher",
},
env,dashboard: add OpenBSD 7.0 all.bash takes a long time on these builders. I'll keep an eye on them to see if it causes trouble. For golang/go#48977. Change-Id: I0239a93c004157855968dab397e68abe38f5f439 Reviewed-on: https://go-review.googlesource.com/c/build/+/364094 Trust: Heschi Kreinick <heschi@google.com> Run-TryBot: Heschi Kreinick <heschi@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-11-15 23:02:19 +03:00
"host-openbsd-386-70": &HostConfig{
VMImage: "openbsd-386-70",
machineType: "e2-highcpu-4",
buildletURLTmpl: "https://storage.googleapis.com/$BUCKET/buildlet.openbsd-386",
goBootstrapURLTmpl: "https://storage.googleapis.com/$BUCKET/gobootstrap-openbsd-386-go1_12.tar.gz",
Notes: "OpenBSD 7.0; GCE VM is built from script in build/env/openbsd-386",
SSHUsername: "gopher",
},
dashboard: add n1 hosts for OpenBSD and NetBSD We recently migrated in CL 354757 from n1 to e2 hosts around the same time issues started appearing with these builders. n1 hosts are notably different in that they are older generation CPUs with hyperthreading disabled. For golang/go#49209 Change-Id: I8b6822fbd4910602f61d504b34580282c97c0ca7 Reviewed-on: https://go-review.googlesource.com/c/build/+/365776 Trust: Alexander Rakoczy <alex@golang.org> Run-TryBot: Alexander Rakoczy <alex@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2021-11-19 22:22:11 +03:00
"host-openbsd-386-70-n1": &HostConfig{
VMImage: "openbsd-386-70",
machineType: "n1-highcpu-4",
buildletURLTmpl: "https://storage.googleapis.com/$BUCKET/buildlet.openbsd-386",
goBootstrapURLTmpl: "https://storage.googleapis.com/$BUCKET/gobootstrap-openbsd-386-go1_12.tar.gz",
Notes: "OpenBSD 7.0; GCE VM is built from script in build/env/openbsd-386. n1-highcpu host.",
SSHUsername: "gopher",
},
dashboard: add openbsd-arm-jsing builder This will allow the openbsd-arm builder to be migrated to a buildlet. Change-Id: If38788daf2d1e7f237ebdcce384a098e08bffb0b Reviewed-on: https://go-review.googlesource.com/c/build/+/192598 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-08-30 20:09:05 +03:00
"host-openbsd-arm-joelsing": &HostConfig{
dashboard, cmd/coordinator: support multiple builder owners Prefer displaying GitHub usernames for convenience of pinging in GitHub issues, since all current builder owners have them. Keep it possible to use an email or something else if no GitHub account exists. Update the owner of the Corellium builders to be individual GitHub account, so that it can be pinged in an issue. The GitHub organization is already mentioned in the builder notes. Also switch to html/template (instead of text/template) for rendering the builders page. Fixes golang/go#49596. Change-Id: I89a96eb8f7a3057bc1f3aaee16abc9c776c45ee7 Reviewed-on: https://go-review.googlesource.com/c/build/+/363983 Trust: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-11-16 20:42:29 +03:00
IsReverse: true,
ExpectNum: 1,
env: []string{"GOROOT_BOOTSTRAP=/usr/local/go"},
Owners: []*gophers.Person{gh("4a6f656c")},
dashboard: add openbsd-arm-jsing builder This will allow the openbsd-arm builder to be migrated to a buildlet. Change-Id: If38788daf2d1e7f237ebdcce384a098e08bffb0b Reviewed-on: https://go-review.googlesource.com/c/build/+/192598 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-08-30 20:09:05 +03:00
},
dashboard: add openbsd-arm64-jsing builder (reverse) Update golang/go#31656 Fixes golang/go#34035 Change-Id: I7e3d20be148ad5111efeb78702352da14274938e Reviewed-on: https://go-review.googlesource.com/c/build/+/250117 Trust: Joel Sing <joel@sing.id.au> Run-TryBot: Joel Sing <joel@sing.id.au> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2020-08-20 21:42:01 +03:00
"host-openbsd-arm64-joelsing": &HostConfig{
dashboard, cmd/coordinator: support multiple builder owners Prefer displaying GitHub usernames for convenience of pinging in GitHub issues, since all current builder owners have them. Keep it possible to use an email or something else if no GitHub account exists. Update the owner of the Corellium builders to be individual GitHub account, so that it can be pinged in an issue. The GitHub organization is already mentioned in the builder notes. Also switch to html/template (instead of text/template) for rendering the builders page. Fixes golang/go#49596. Change-Id: I89a96eb8f7a3057bc1f3aaee16abc9c776c45ee7 Reviewed-on: https://go-review.googlesource.com/c/build/+/363983 Trust: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-11-16 20:42:29 +03:00
IsReverse: true,
ExpectNum: 1,
env: []string{"GOROOT_BOOTSTRAP=/usr/local/go"},
Owners: []*gophers.Person{gh("4a6f656c")},
dashboard: add openbsd-arm64-jsing builder (reverse) Update golang/go#31656 Fixes golang/go#34035 Change-Id: I7e3d20be148ad5111efeb78702352da14274938e Reviewed-on: https://go-review.googlesource.com/c/build/+/250117 Trust: Joel Sing <joel@sing.id.au> Run-TryBot: Joel Sing <joel@sing.id.au> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2020-08-20 21:42:01 +03:00
},
dashboard: add openbsd-mips64-jsing builder (reverse) Update golang/go#40995 Change-Id: Ie733438ab1823d65109e684a6a5e3098cdfdb252 Reviewed-on: https://go-review.googlesource.com/c/build/+/250118 Trust: Joel Sing <joel@sing.id.au> Run-TryBot: Joel Sing <joel@sing.id.au> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2020-08-23 19:02:40 +03:00
"host-openbsd-mips64-joelsing": &HostConfig{
dashboard, cmd/coordinator: support multiple builder owners Prefer displaying GitHub usernames for convenience of pinging in GitHub issues, since all current builder owners have them. Keep it possible to use an email or something else if no GitHub account exists. Update the owner of the Corellium builders to be individual GitHub account, so that it can be pinged in an issue. The GitHub organization is already mentioned in the builder notes. Also switch to html/template (instead of text/template) for rendering the builders page. Fixes golang/go#49596. Change-Id: I89a96eb8f7a3057bc1f3aaee16abc9c776c45ee7 Reviewed-on: https://go-review.googlesource.com/c/build/+/363983 Trust: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-11-16 20:42:29 +03:00
IsReverse: true,
ExpectNum: 1,
env: []string{"GOROOT_BOOTSTRAP=/usr/local/go"},
Owners: []*gophers.Person{gh("4a6f656c")},
dashboard: add openbsd-mips64-jsing builder (reverse) Update golang/go#40995 Change-Id: Ie733438ab1823d65109e684a6a5e3098cdfdb252 Reviewed-on: https://go-review.googlesource.com/c/build/+/250118 Trust: Joel Sing <joel@sing.id.au> Run-TryBot: Joel Sing <joel@sing.id.au> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2020-08-23 19:02:40 +03:00
},
dashboard: add FreeBSD 11.2 and 10.4 builders Not yet the defaults or trybots until we see they're working regularly for a few days and compare timings, etc. Updates golang/go#25289 Change-Id: I4baed066b5f6d9f01e5454b716cacda4986620d0 Reviewed-on: https://go-review.googlesource.com/138258 Reviewed-by: Yuval Pavel Zholkover <paulzhol@gmail.com> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2018-09-28 02:43:25 +03:00
"host-freebsd-11_2": &HostConfig{
VMImage: "freebsd-amd64-112",
Notes: "FreeBSD 11.2; GCE VM is built from script in build/env/freebsd-amd64",
all: migrate from n1 to e2 instances We should have similar performance for these machines, and increased memory availability in some areas. This is also compliant with an internal policy change. Change-Id: I07a452a07431322d6fdd4edbb98c5914045d34ed Reviewed-on: https://go-review.googlesource.com/c/build/+/354757 Trust: Alexander Rakoczy <alex@golang.org> Run-TryBot: Alexander Rakoczy <alex@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Heschi Kreinick <heschi@google.com> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-10-09 00:03:24 +03:00
machineType: "e2-highcpu-4",
dashboard: add FreeBSD 11.2 and 10.4 builders Not yet the defaults or trybots until we see they're working regularly for a few days and compare timings, etc. Updates golang/go#25289 Change-Id: I4baed066b5f6d9f01e5454b716cacda4986620d0 Reviewed-on: https://go-review.googlesource.com/138258 Reviewed-by: Yuval Pavel Zholkover <paulzhol@gmail.com> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2018-09-28 02:43:25 +03:00
buildletURLTmpl: "https://storage.googleapis.com/$BUCKET/buildlet.freebsd-amd64",
goBootstrapURLTmpl: "https://storage.googleapis.com/$BUCKET/go1.4-freebsd-amd64.tar.gz",
SSHUsername: "gopher",
},
dashboard: add amd64 FreeBSD 11.4 and 12.2 builders This change adds builders for amd64 FreeBSD 11.4 and 12.2. These builders are labeled with a known issue until they have been tested. They will not be used in trybots until the known issue label has been removed. Updates golang/go#44431 Change-Id: I902f2f6bd887135bf247feb1be0e0fd49f732614 Reviewed-on: https://go-review.googlesource.com/c/build/+/311869 Trust: Carlos Amedee <carlos@golang.org> Run-TryBot: Carlos Amedee <carlos@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2021-04-20 20:43:05 +03:00
"host-freebsd-11_4": &HostConfig{
VMImage: "freebsd-amd64-114",
Notes: "FreeBSD 11.4; GCE VM is built from script in build/env/freebsd-amd64",
all: migrate from n1 to e2 instances We should have similar performance for these machines, and increased memory availability in some areas. This is also compliant with an internal policy change. Change-Id: I07a452a07431322d6fdd4edbb98c5914045d34ed Reviewed-on: https://go-review.googlesource.com/c/build/+/354757 Trust: Alexander Rakoczy <alex@golang.org> Run-TryBot: Alexander Rakoczy <alex@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Heschi Kreinick <heschi@google.com> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-10-09 00:03:24 +03:00
machineType: "e2-highcpu-4",
dashboard: add amd64 FreeBSD 11.4 and 12.2 builders This change adds builders for amd64 FreeBSD 11.4 and 12.2. These builders are labeled with a known issue until they have been tested. They will not be used in trybots until the known issue label has been removed. Updates golang/go#44431 Change-Id: I902f2f6bd887135bf247feb1be0e0fd49f732614 Reviewed-on: https://go-review.googlesource.com/c/build/+/311869 Trust: Carlos Amedee <carlos@golang.org> Run-TryBot: Carlos Amedee <carlos@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2021-04-20 20:43:05 +03:00
buildletURLTmpl: "https://storage.googleapis.com/$BUCKET/buildlet.freebsd-amd64",
goBootstrapURLTmpl: "https://storage.googleapis.com/$BUCKET/go1.4-freebsd-amd64.tar.gz",
SSHUsername: "gopher",
},
"host-freebsd-12_2": &HostConfig{
VMImage: "freebsd-amd64-122",
Notes: "FreeBSD 12.2; GCE VM is built from script in build/env/freebsd-amd64",
all: migrate from n1 to e2 instances We should have similar performance for these machines, and increased memory availability in some areas. This is also compliant with an internal policy change. Change-Id: I07a452a07431322d6fdd4edbb98c5914045d34ed Reviewed-on: https://go-review.googlesource.com/c/build/+/354757 Trust: Alexander Rakoczy <alex@golang.org> Run-TryBot: Alexander Rakoczy <alex@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Heschi Kreinick <heschi@google.com> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-10-09 00:03:24 +03:00
machineType: "e2-highcpu-4",
dashboard: add amd64 FreeBSD 11.4 and 12.2 builders This change adds builders for amd64 FreeBSD 11.4 and 12.2. These builders are labeled with a known issue until they have been tested. They will not be used in trybots until the known issue label has been removed. Updates golang/go#44431 Change-Id: I902f2f6bd887135bf247feb1be0e0fd49f732614 Reviewed-on: https://go-review.googlesource.com/c/build/+/311869 Trust: Carlos Amedee <carlos@golang.org> Run-TryBot: Carlos Amedee <carlos@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2021-04-20 20:43:05 +03:00
buildletURLTmpl: "https://storage.googleapis.com/$BUCKET/buildlet.freebsd-amd64",
goBootstrapURLTmpl: "https://storage.googleapis.com/$BUCKET/go1.4-freebsd-amd64.tar.gz",
SSHUsername: "gopher",
},
dashboard: add freebsd-[amd64|386]-13_0 builders This change adds builders for FreeBSD 13.0. Updates golang/go#46272 Fixes golang/go#45903 Change-Id: I92f68f8d5eeee2d3bae8864dbfe5ee61b93ea677 Reviewed-on: https://go-review.googlesource.com/c/build/+/365314 Trust: Carlos Amedee <carlos@golang.org> Run-TryBot: Carlos Amedee <carlos@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2021-11-19 00:33:40 +03:00
"host-freebsd-13_0": &HostConfig{
VMImage: "freebsd-amd64-130",
Notes: "FreeBSD 13.0; GCE VM is built from script in build/env/freebsd-amd64",
machineType: "e2-highcpu-4",
buildletURLTmpl: "https://storage.googleapis.com/$BUCKET/buildlet.freebsd-amd64",
goBootstrapURLTmpl: "https://storage.googleapis.com/$BUCKET/go1.4-freebsd-amd64.tar.gz",
SSHUsername: "gopher",
},
dashboard: update freebsd-amd64-race builder host This change updates the version of FreeBSD used for the freebsd-amd64-race builder to 13.0. The slowbot aliases have been updated to use FreeBSD 13.0. Updates golang/go#46272 Updates golang/go#45903 Change-Id: I1092f9e6bf66a9f429b463dce5d0d5647d44eec8 Reviewed-on: https://go-review.googlesource.com/c/build/+/365616 Trust: Carlos Amedee <carlos@golang.org> Run-TryBot: Carlos Amedee <carlos@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2021-11-19 18:11:34 +03:00
"host-freebsd-13_0-big": &HostConfig{
VMImage: "freebsd-amd64-130",
Notes: "Same as host-freebsd-13_0, but on e2-highcpu-16",
all: migrate from n1 to e2 instances We should have similar performance for these machines, and increased memory availability in some areas. This is also compliant with an internal policy change. Change-Id: I07a452a07431322d6fdd4edbb98c5914045d34ed Reviewed-on: https://go-review.googlesource.com/c/build/+/354757 Trust: Alexander Rakoczy <alex@golang.org> Run-TryBot: Alexander Rakoczy <alex@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Heschi Kreinick <heschi@google.com> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-10-09 00:03:24 +03:00
machineType: "e2-highcpu-16", // 16 vCPUs, 16 GB mem
dashboard, cmd/release: remove what's unused as of Go 1.16 release We can drop the 32-bit and 10.11 macOS builders, they're unused. Remove FreeBSD 11.1 builder, it's unused. Update freebsd-amd64-race to use the latest available FreeBSD builder. Simplify build policy and comments, removing anything that mentioned Go version 1.14 or older, since they're not relevant anymore. Drop Go 1.14 support from minSupportedMacOSVersion, and add support for the future Go 1.17 release, which will increase minimum macOS version requirement to macOS 10.13. Fixes golang/go#40562. Change-Id: I7beeec0952a516afe99a5154b22c7f25ceeb3da1 Reviewed-on: https://go-review.googlesource.com/c/build/+/293771 Trust: Dmitri Shuralyov <dmitshur@golang.org> Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org>
2021-02-19 20:17:28 +03:00
buildletURLTmpl: "https://storage.googleapis.com/$BUCKET/buildlet.freebsd-amd64",
goBootstrapURLTmpl: "https://storage.googleapis.com/$BUCKET/go1.4-freebsd-amd64.tar.gz",
SSHUsername: "gopher",
},
dashboard, env/netbsd-{amd64,386}: add NetBSD 9.0 builders Update NetBSD amd64 and 386 builders to the latest NetBSD 9.0 release, using the latest versions of anita (2.3) and pkgsrc packages (2019Q4). Also increase the image disk size from 4 GB to 16 GB. The motivation is to help tests that require more disk space and large dependencies. Additional adjustments to the image building script will need to be made to make use of increased disk space for the /tmp volume, as it is currently being mounted in RAM with size equal to 25% of memory. Remove the manual invocation of the mozilla-rootcerts install command. It no longer works as is in NetBSD 9.0. The mozilla-rootcerts-openssl package is added instead, which successfully performs that same task. Hopefully this will be a more robust approach for the future. Continue to use an explicit URL for the NetBSD release download, and remove unused find_latest_release. The plan is to roll out these builders first (without trybots), gain more confidence, enable trybots, and then remove NetBSD 8.0 builders. For golang/go#37244. For golang/go#30839. For golang/go#31858. For golang/go#31726. For golang/go#38283. Change-Id: Id41278eaebe64cfbcf7b1ab9bfd64e550ee0bbf1 Reviewed-on: https://go-review.googlesource.com/c/build/+/226977 Reviewed-by: Benny Siegert <bsiegert@gmail.com> Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-04-02 15:55:30 +03:00
"host-netbsd-amd64-9_0": &HostConfig{
VMImage: "netbsd-amd64-9-0-2019q4",
Notes: "NetBSD 9.0; GCE VM is built from script in build/env/netbsd-amd64",
all: migrate from n1 to e2 instances We should have similar performance for these machines, and increased memory availability in some areas. This is also compliant with an internal policy change. Change-Id: I07a452a07431322d6fdd4edbb98c5914045d34ed Reviewed-on: https://go-review.googlesource.com/c/build/+/354757 Trust: Alexander Rakoczy <alex@golang.org> Run-TryBot: Alexander Rakoczy <alex@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Heschi Kreinick <heschi@google.com> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-10-09 00:03:24 +03:00
machineType: "e2-highcpu-4",
dashboard, env/netbsd-{amd64,386}: add NetBSD 9.0 builders Update NetBSD amd64 and 386 builders to the latest NetBSD 9.0 release, using the latest versions of anita (2.3) and pkgsrc packages (2019Q4). Also increase the image disk size from 4 GB to 16 GB. The motivation is to help tests that require more disk space and large dependencies. Additional adjustments to the image building script will need to be made to make use of increased disk space for the /tmp volume, as it is currently being mounted in RAM with size equal to 25% of memory. Remove the manual invocation of the mozilla-rootcerts install command. It no longer works as is in NetBSD 9.0. The mozilla-rootcerts-openssl package is added instead, which successfully performs that same task. Hopefully this will be a more robust approach for the future. Continue to use an explicit URL for the NetBSD release download, and remove unused find_latest_release. The plan is to roll out these builders first (without trybots), gain more confidence, enable trybots, and then remove NetBSD 8.0 builders. For golang/go#37244. For golang/go#30839. For golang/go#31858. For golang/go#31726. For golang/go#38283. Change-Id: Id41278eaebe64cfbcf7b1ab9bfd64e550ee0bbf1 Reviewed-on: https://go-review.googlesource.com/c/build/+/226977 Reviewed-by: Benny Siegert <bsiegert@gmail.com> Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-04-02 15:55:30 +03:00
buildletURLTmpl: "https://storage.googleapis.com/$BUCKET/buildlet.netbsd-amd64",
goBootstrapURLTmpl: "https://storage.googleapis.com/$BUCKET/gobootstrap-netbsd-amd64-2da6b33.tar.gz",
SSHUsername: "root",
},
dashboard: add n1 hosts for OpenBSD and NetBSD We recently migrated in CL 354757 from n1 to e2 hosts around the same time issues started appearing with these builders. n1 hosts are notably different in that they are older generation CPUs with hyperthreading disabled. For golang/go#49209 Change-Id: I8b6822fbd4910602f61d504b34580282c97c0ca7 Reviewed-on: https://go-review.googlesource.com/c/build/+/365776 Trust: Alexander Rakoczy <alex@golang.org> Run-TryBot: Alexander Rakoczy <alex@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2021-11-19 22:22:11 +03:00
"host-netbsd-amd64-9_0-n1": &HostConfig{
VMImage: "netbsd-amd64-9-0-2019q4",
Notes: "NetBSD 9.0; GCE VM is built from script in build/env/netbsd-amd64. n1-highcpu host.",
machineType: "n1-highcpu-4",
buildletURLTmpl: "https://storage.googleapis.com/$BUCKET/buildlet.netbsd-amd64",
goBootstrapURLTmpl: "https://storage.googleapis.com/$BUCKET/gobootstrap-netbsd-amd64-2da6b33.tar.gz",
SSHUsername: "root",
},
dashboard, env/netbsd-{amd64,386}: add NetBSD 9.0 builders Update NetBSD amd64 and 386 builders to the latest NetBSD 9.0 release, using the latest versions of anita (2.3) and pkgsrc packages (2019Q4). Also increase the image disk size from 4 GB to 16 GB. The motivation is to help tests that require more disk space and large dependencies. Additional adjustments to the image building script will need to be made to make use of increased disk space for the /tmp volume, as it is currently being mounted in RAM with size equal to 25% of memory. Remove the manual invocation of the mozilla-rootcerts install command. It no longer works as is in NetBSD 9.0. The mozilla-rootcerts-openssl package is added instead, which successfully performs that same task. Hopefully this will be a more robust approach for the future. Continue to use an explicit URL for the NetBSD release download, and remove unused find_latest_release. The plan is to roll out these builders first (without trybots), gain more confidence, enable trybots, and then remove NetBSD 8.0 builders. For golang/go#37244. For golang/go#30839. For golang/go#31858. For golang/go#31726. For golang/go#38283. Change-Id: Id41278eaebe64cfbcf7b1ab9bfd64e550ee0bbf1 Reviewed-on: https://go-review.googlesource.com/c/build/+/226977 Reviewed-by: Benny Siegert <bsiegert@gmail.com> Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-04-02 15:55:30 +03:00
"host-netbsd-386-9_0": &HostConfig{
VMImage: "netbsd-i386-9-0-2019q4",
Notes: "NetBSD 9.0; GCE VM is built from script in build/env/netbsd-386",
all: migrate from n1 to e2 instances We should have similar performance for these machines, and increased memory availability in some areas. This is also compliant with an internal policy change. Change-Id: I07a452a07431322d6fdd4edbb98c5914045d34ed Reviewed-on: https://go-review.googlesource.com/c/build/+/354757 Trust: Alexander Rakoczy <alex@golang.org> Run-TryBot: Alexander Rakoczy <alex@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Heschi Kreinick <heschi@google.com> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-10-09 00:03:24 +03:00
machineType: "e2-highcpu-4",
dashboard, env/netbsd-{amd64,386}: add NetBSD 9.0 builders Update NetBSD amd64 and 386 builders to the latest NetBSD 9.0 release, using the latest versions of anita (2.3) and pkgsrc packages (2019Q4). Also increase the image disk size from 4 GB to 16 GB. The motivation is to help tests that require more disk space and large dependencies. Additional adjustments to the image building script will need to be made to make use of increased disk space for the /tmp volume, as it is currently being mounted in RAM with size equal to 25% of memory. Remove the manual invocation of the mozilla-rootcerts install command. It no longer works as is in NetBSD 9.0. The mozilla-rootcerts-openssl package is added instead, which successfully performs that same task. Hopefully this will be a more robust approach for the future. Continue to use an explicit URL for the NetBSD release download, and remove unused find_latest_release. The plan is to roll out these builders first (without trybots), gain more confidence, enable trybots, and then remove NetBSD 8.0 builders. For golang/go#37244. For golang/go#30839. For golang/go#31858. For golang/go#31726. For golang/go#38283. Change-Id: Id41278eaebe64cfbcf7b1ab9bfd64e550ee0bbf1 Reviewed-on: https://go-review.googlesource.com/c/build/+/226977 Reviewed-by: Benny Siegert <bsiegert@gmail.com> Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-04-02 15:55:30 +03:00
buildletURLTmpl: "https://storage.googleapis.com/$BUCKET/buildlet.netbsd-386",
goBootstrapURLTmpl: "https://storage.googleapis.com/$BUCKET/gobootstrap-netbsd-386-0b3b511.tar.gz",
SSHUsername: "root",
},
dashboard: add n1 hosts for OpenBSD and NetBSD We recently migrated in CL 354757 from n1 to e2 hosts around the same time issues started appearing with these builders. n1 hosts are notably different in that they are older generation CPUs with hyperthreading disabled. For golang/go#49209 Change-Id: I8b6822fbd4910602f61d504b34580282c97c0ca7 Reviewed-on: https://go-review.googlesource.com/c/build/+/365776 Trust: Alexander Rakoczy <alex@golang.org> Run-TryBot: Alexander Rakoczy <alex@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2021-11-19 22:22:11 +03:00
"host-netbsd-386-9_0-n1": &HostConfig{
VMImage: "netbsd-i386-9-0-2019q4",
Notes: "NetBSD 9.0; GCE VM is built from script in build/env/netbsd-386. n1-highcpu host.",
machineType: "n1-highcpu-4",
buildletURLTmpl: "https://storage.googleapis.com/$BUCKET/buildlet.netbsd-386",
goBootstrapURLTmpl: "https://storage.googleapis.com/$BUCKET/gobootstrap-netbsd-386-0b3b511.tar.gz",
SSHUsername: "root",
},
dashboard: add netbsd-arm-bsiegert This seems to be the last builder still using the old builder binary and not buildlet. It needs an entry in here to switch to buildlet and stage0. Change-Id: If2fdd2298ab88d56b6850061050caf27ca7c8983 Reviewed-on: https://go-review.googlesource.com/c/build/+/168061 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-03-19 16:28:04 +03:00
"host-netbsd-arm-bsiegert": &HostConfig{
dashboard, cmd/coordinator: support multiple builder owners Prefer displaying GitHub usernames for convenience of pinging in GitHub issues, since all current builder owners have them. Keep it possible to use an email or something else if no GitHub account exists. Update the owner of the Corellium builders to be individual GitHub account, so that it can be pinged in an issue. The GitHub organization is already mentioned in the builder notes. Also switch to html/template (instead of text/template) for rendering the builders page. Fixes golang/go#49596. Change-Id: I89a96eb8f7a3057bc1f3aaee16abc9c776c45ee7 Reviewed-on: https://go-review.googlesource.com/c/build/+/363983 Trust: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-11-16 20:42:29 +03:00
IsReverse: true,
ExpectNum: 1,
env: []string{"GOROOT_BOOTSTRAP=/usr/pkg/go112"},
Owners: []*gophers.Person{gh("bsiegert")},
dashboard: add netbsd-arm-bsiegert This seems to be the last builder still using the old builder binary and not buildlet. It needs an entry in here to switch to buildlet and stage0. Change-Id: If2fdd2298ab88d56b6850061050caf27ca7c8983 Reviewed-on: https://go-review.googlesource.com/c/build/+/168061 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-03-19 16:28:04 +03:00
},
dashboard: add netbsd-arm64-bsiegert builder (reverse) Update golang/go#34036 Change-Id: Ic92093e764fa163d97a2f125a4534581adf217da Reviewed-on: https://go-review.googlesource.com/c/build/+/249137 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2020-08-18 21:42:30 +03:00
"host-netbsd-arm64-bsiegert": &HostConfig{
dashboard, cmd/coordinator: support multiple builder owners Prefer displaying GitHub usernames for convenience of pinging in GitHub issues, since all current builder owners have them. Keep it possible to use an email or something else if no GitHub account exists. Update the owner of the Corellium builders to be individual GitHub account, so that it can be pinged in an issue. The GitHub organization is already mentioned in the builder notes. Also switch to html/template (instead of text/template) for rendering the builders page. Fixes golang/go#49596. Change-Id: I89a96eb8f7a3057bc1f3aaee16abc9c776c45ee7 Reviewed-on: https://go-review.googlesource.com/c/build/+/363983 Trust: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-11-16 20:42:29 +03:00
IsReverse: true,
ExpectNum: 1,
env: []string{"GOROOT_BOOTSTRAP=/usr/pkg/go114"},
Owners: []*gophers.Person{gh("bsiegert")},
dashboard: add netbsd-arm64-bsiegert builder (reverse) Update golang/go#34036 Change-Id: Ic92093e764fa163d97a2f125a4534581adf217da Reviewed-on: https://go-review.googlesource.com/c/build/+/249137 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2020-08-18 21:42:30 +03:00
},
dashboard: rename DragonFlyBSD host type, change owners Change-Id: Iff3de20c4bb6a017f204602542652f8533b599e7 Reviewed-on: https://go-review.googlesource.com/c/build/+/205460 Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-11-05 21:55:17 +03:00
"host-dragonfly-amd64-master": &HostConfig{
IsReverse: true,
ExpectNum: 1,
Notes: "DragonFly BSD master, run by DragonFly team",
env: []string{"GOROOT_BOOTSTRAP=/usr/local/go"},
dashboard: enable SSH access in DragonFly BSD builders Fixes golang/go#38796. Change-Id: If84c7ba39fe96ce0b93d818ae9f163338d954976 GitHub-Last-Rev: 3548bbc15722b14d819c46f5dabef0ae4a7455b7 GitHub-Pull-Request: golang/build#29 Reviewed-on: https://go-review.googlesource.com/c/build/+/232337 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2020-05-05 22:18:12 +03:00
SSHUsername: "root",
dashboard, cmd/coordinator: support multiple builder owners Prefer displaying GitHub usernames for convenience of pinging in GitHub issues, since all current builder owners have them. Keep it possible to use an email or something else if no GitHub account exists. Update the owner of the Corellium builders to be individual GitHub account, so that it can be pinged in an issue. The GitHub organization is already mentioned in the builder notes. Also switch to html/template (instead of text/template) for rendering the builders page. Fixes golang/go#49596. Change-Id: I89a96eb8f7a3057bc1f3aaee16abc9c776c45ee7 Reviewed-on: https://go-review.googlesource.com/c/build/+/363983 Trust: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-11-16 20:42:29 +03:00
Owners: []*gophers.Person{gh("tuxillo")},
dashboard: update dragonfly-amd64 and freebsd-arm to be buildlet-based Updates golang/go#21191 Change-Id: I636204e9e4cbe77e34b2bbad65312fb6f2ace2dc Reviewed-on: https://go-review.googlesource.com/53490 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2017-08-06 01:56:18 +03:00
},
"host-freebsd-arm-paulzhol": &HostConfig{
dashboard, cmd/coordinator: support multiple builder owners Prefer displaying GitHub usernames for convenience of pinging in GitHub issues, since all current builder owners have them. Keep it possible to use an email or something else if no GitHub account exists. Update the owner of the Corellium builders to be individual GitHub account, so that it can be pinged in an issue. The GitHub organization is already mentioned in the builder notes. Also switch to html/template (instead of text/template) for rendering the builders page. Fixes golang/go#49596. Change-Id: I89a96eb8f7a3057bc1f3aaee16abc9c776c45ee7 Reviewed-on: https://go-review.googlesource.com/c/build/+/363983 Trust: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-11-16 20:42:29 +03:00
IsReverse: true,
ExpectNum: 1,
Notes: "Cubiboard2 1Gb RAM dual-core Cortex-A7 (Allwinner A20), FreeBSD 11.1-RELEASE",
env: []string{"GOROOT_BOOTSTRAP=/usr/home/paulzhol/go1.4"},
Owners: []*gophers.Person{gh("paulzhol")},
dashboard: update dragonfly-amd64 and freebsd-arm to be buildlet-based Updates golang/go#21191 Change-Id: I636204e9e4cbe77e34b2bbad65312fb6f2ace2dc Reviewed-on: https://go-review.googlesource.com/53490 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2017-08-06 01:56:18 +03:00
},
dashboard: add FreeBSD arm64 builder Updates golang/go#24715 Change-Id: I8eb3f6372d69718b6f453e5415cdf16aa00ccae6 Reviewed-on: https://go-review.googlesource.com/c/build/+/198597 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-10-03 15:56:19 +03:00
"host-freebsd-arm64-dmgk": &HostConfig{
dashboard, cmd/coordinator: support multiple builder owners Prefer displaying GitHub usernames for convenience of pinging in GitHub issues, since all current builder owners have them. Keep it possible to use an email or something else if no GitHub account exists. Update the owner of the Corellium builders to be individual GitHub account, so that it can be pinged in an issue. The GitHub organization is already mentioned in the builder notes. Also switch to html/template (instead of text/template) for rendering the builders page. Fixes golang/go#49596. Change-Id: I89a96eb8f7a3057bc1f3aaee16abc9c776c45ee7 Reviewed-on: https://go-review.googlesource.com/c/build/+/363983 Trust: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-11-16 20:42:29 +03:00
IsReverse: true,
ExpectNum: 1,
Notes: "AWS EC2 a1.large 2 vCPU 4GiB RAM, FreeBSD 12.1-STABLE",
env: []string{"GOROOT_BOOTSTRAP=/usr/home/builder/gobootstrap"},
Owners: []*gophers.Person{gh("dmgk")},
dashboard: add FreeBSD arm64 builder Updates golang/go#24715 Change-Id: I8eb3f6372d69718b6f453e5415cdf16aa00ccae6 Reviewed-on: https://go-review.googlesource.com/c/build/+/198597 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-10-03 15:56:19 +03:00
},
dashboard: convert plan9-arm, plan9-amd64-9front, nacl-arm to buildlets Updates golang/go#21191 Change-Id: I577dad51d490d7e281ccde9fa7d82bb1bee95bd9 Reviewed-on: https://go-review.googlesource.com/53417 Reviewed-by: David du Colombier <0intro@gmail.com>
2017-08-06 18:57:06 +03:00
"host-plan9-arm-0intro": &HostConfig{
dashboard, cmd/coordinator: support multiple builder owners Prefer displaying GitHub usernames for convenience of pinging in GitHub issues, since all current builder owners have them. Keep it possible to use an email or something else if no GitHub account exists. Update the owner of the Corellium builders to be individual GitHub account, so that it can be pinged in an issue. The GitHub organization is already mentioned in the builder notes. Also switch to html/template (instead of text/template) for rendering the builders page. Fixes golang/go#49596. Change-Id: I89a96eb8f7a3057bc1f3aaee16abc9c776c45ee7 Reviewed-on: https://go-review.googlesource.com/c/build/+/363983 Trust: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-11-16 20:42:29 +03:00
IsReverse: true,
ExpectNum: 1,
Notes: "Raspberry Pi 3 Model B, Plan 9 from Bell Labs",
Owners: []*gophers.Person{gh("0intro")},
dashboard: convert plan9-arm, plan9-amd64-9front, nacl-arm to buildlets Updates golang/go#21191 Change-Id: I577dad51d490d7e281ccde9fa7d82bb1bee95bd9 Reviewed-on: https://go-review.googlesource.com/53417 Reviewed-by: David du Colombier <0intro@gmail.com>
2017-08-06 18:57:06 +03:00
},
"host-plan9-amd64-0intro": &HostConfig{
dashboard, cmd/coordinator: support multiple builder owners Prefer displaying GitHub usernames for convenience of pinging in GitHub issues, since all current builder owners have them. Keep it possible to use an email or something else if no GitHub account exists. Update the owner of the Corellium builders to be individual GitHub account, so that it can be pinged in an issue. The GitHub organization is already mentioned in the builder notes. Also switch to html/template (instead of text/template) for rendering the builders page. Fixes golang/go#49596. Change-Id: I89a96eb8f7a3057bc1f3aaee16abc9c776c45ee7 Reviewed-on: https://go-review.googlesource.com/c/build/+/363983 Trust: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-11-16 20:42:29 +03:00
IsReverse: true,
ExpectNum: 1,
Notes: "QEMU VM, Plan 9 from Bell Labs, 9k kernel",
Owners: []*gophers.Person{gh("0intro")},
dashboard: convert plan9-arm, plan9-amd64-9front, nacl-arm to buildlets Updates golang/go#21191 Change-Id: I577dad51d490d7e281ccde9fa7d82bb1bee95bd9 Reviewed-on: https://go-review.googlesource.com/53417 Reviewed-by: David du Colombier <0intro@gmail.com>
2017-08-06 18:57:06 +03:00
},
dashboard: add plan9-386-0intro builder This change adds the plan9-386-0intro builder configuration. Updates golang/go#29801. Change-Id: I5f64f4592a499eb1691498805fd0dfa5082c33d5 Reviewed-on: https://go-review.googlesource.com/c/build/+/175901 Run-TryBot: David du Colombier <0intro@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-05-09 19:52:40 +03:00
"host-plan9-386-0intro": &HostConfig{
dashboard, cmd/coordinator: support multiple builder owners Prefer displaying GitHub usernames for convenience of pinging in GitHub issues, since all current builder owners have them. Keep it possible to use an email or something else if no GitHub account exists. Update the owner of the Corellium builders to be individual GitHub account, so that it can be pinged in an issue. The GitHub organization is already mentioned in the builder notes. Also switch to html/template (instead of text/template) for rendering the builders page. Fixes golang/go#49596. Change-Id: I89a96eb8f7a3057bc1f3aaee16abc9c776c45ee7 Reviewed-on: https://go-review.googlesource.com/c/build/+/363983 Trust: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-11-16 20:42:29 +03:00
IsReverse: true,
ExpectNum: 1,
Notes: "QEMU VM, Plan 9 from Bell Labs",
Owners: []*gophers.Person{gh("0intro")},
dashboard: add plan9-386-0intro builder This change adds the plan9-386-0intro builder configuration. Updates golang/go#29801. Change-Id: I5f64f4592a499eb1691498805fd0dfa5082c33d5 Reviewed-on: https://go-review.googlesource.com/c/build/+/175901 Run-TryBot: David du Colombier <0intro@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-05-09 19:52:40 +03:00
},
all: split builder config into builder & host configs Our builders are named of the form "GOOS-GOARCH" or "GOOS-GOARCH-suffix". Over time we've grown many builders. This CL doesn't change that. Builders continue to be named and operate as before. Previously the build configuration file (dashboard/builders.go) made each builder type ("linux-amd64-race", etc) define how to create a host running a buildlet of that type, even though many builders had identical host configs. For example, these builders all share the same host type (a Kubernetes container): linux-amd64 linux-amd64-race linux-386 linux-386-387 And these are the same host type (a GCE VM): windows-amd64-gce windows-amd64-race windows-386-gce This CL creates a new concept of a "hostType" which defines how the buildlet is created (Kube, GCE, Reverse, and how), and then each builder itself references a host type. Users never see the hostType. (except perhaps in gomote list output) But they at least never need to care about them. Reverse buildlets now can only be one hostType at a time, which simplifies things. We were no longer using multiple roles per machine once moving to VMs for OS X. gomote continues to operate as it did previously but its underlying protocol changed and clients will need to be updated. As a new feature, gomote now has a new flag to let you reuse a buildlet host connection for different builder rules if they share the same underlying host type. But users can ignore that. This CL is a long-standing TODO (previously attempted and aborted) and will make many things easier and faster, including the linux-arm cross-compilation effort, and keeping pre-warmed buildlets of VM types ready to go. Updates golang/go#17104 Change-Id: Iad8387f48680424a8441e878a2f4762bf79ea4d2 Reviewed-on: https://go-review.googlesource.com/29551 Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2016-09-22 00:27:37 +03:00
"host-plan9-386-gce": &HostConfig{
env/plan9: revert SMP changes, stop running cmd/go tests Revert "env: enable SMP in Plan 9 image" This reverts commit 6478d3e7bdff4c8d810ba44f970ca4b0bfc92405. Revert "dashboard: bump plan9 GCE image" This reverts commit ce208996bf954a87d68789bb6a9000256f016c77. Updates golang/go#29822 Change-Id: I47e75d135e092ee9d5a80c66e91c1048c67cd5f6 Reviewed-on: https://go-review.googlesource.com/c/158542 Reviewed-by: Bryan C. Mills <bcmills@google.com> Reviewed-by: David du Colombier <0intro@gmail.com>
2019-01-19 01:19:58 +03:00
VMImage: "plan9-386-v7",
all: split builder config into builder & host configs Our builders are named of the form "GOOS-GOARCH" or "GOOS-GOARCH-suffix". Over time we've grown many builders. This CL doesn't change that. Builders continue to be named and operate as before. Previously the build configuration file (dashboard/builders.go) made each builder type ("linux-amd64-race", etc) define how to create a host running a buildlet of that type, even though many builders had identical host configs. For example, these builders all share the same host type (a Kubernetes container): linux-amd64 linux-amd64-race linux-386 linux-386-387 And these are the same host type (a GCE VM): windows-amd64-gce windows-amd64-race windows-386-gce This CL creates a new concept of a "hostType" which defines how the buildlet is created (Kube, GCE, Reverse, and how), and then each builder itself references a host type. Users never see the hostType. (except perhaps in gomote list output) But they at least never need to care about them. Reverse buildlets now can only be one hostType at a time, which simplifies things. We were no longer using multiple roles per machine once moving to VMs for OS X. gomote continues to operate as it did previously but its underlying protocol changed and clients will need to be updated. As a new feature, gomote now has a new flag to let you reuse a buildlet host connection for different builder rules if they share the same underlying host type. But users can ignore that. This CL is a long-standing TODO (previously attempted and aborted) and will make many things easier and faster, including the linux-arm cross-compilation effort, and keeping pre-warmed buildlets of VM types ready to go. Updates golang/go#17104 Change-Id: Iad8387f48680424a8441e878a2f4762bf79ea4d2 Reviewed-on: https://go-review.googlesource.com/29551 Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2016-09-22 00:27:37 +03:00
Notes: "Plan 9 from 0intro; GCE VM is built from script in build/env/plan9-386",
buildletURLTmpl: "http://storage.googleapis.com/$BUCKET/buildlet.plan9-386",
goBootstrapURLTmpl: "https://storage.googleapis.com/$BUCKET/gobootstrap-plan9-386.tar.gz",
all: migrate from n1 to e2 instances We should have similar performance for these machines, and increased memory availability in some areas. This is also compliant with an internal policy change. Change-Id: I07a452a07431322d6fdd4edbb98c5914045d34ed Reviewed-on: https://go-review.googlesource.com/c/build/+/354757 Trust: Alexander Rakoczy <alex@golang.org> Run-TryBot: Alexander Rakoczy <alex@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Heschi Kreinick <heschi@google.com> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-10-09 00:03:24 +03:00
machineType: "e2-highcpu-4",
env: []string{"GO_TEST_TIMEOUT_SCALE=3"},
all: split builder config into builder & host configs Our builders are named of the form "GOOS-GOARCH" or "GOOS-GOARCH-suffix". Over time we've grown many builders. This CL doesn't change that. Builders continue to be named and operate as before. Previously the build configuration file (dashboard/builders.go) made each builder type ("linux-amd64-race", etc) define how to create a host running a buildlet of that type, even though many builders had identical host configs. For example, these builders all share the same host type (a Kubernetes container): linux-amd64 linux-amd64-race linux-386 linux-386-387 And these are the same host type (a GCE VM): windows-amd64-gce windows-amd64-race windows-386-gce This CL creates a new concept of a "hostType" which defines how the buildlet is created (Kube, GCE, Reverse, and how), and then each builder itself references a host type. Users never see the hostType. (except perhaps in gomote list output) But they at least never need to care about them. Reverse buildlets now can only be one hostType at a time, which simplifies things. We were no longer using multiple roles per machine once moving to VMs for OS X. gomote continues to operate as it did previously but its underlying protocol changed and clients will need to be updated. As a new feature, gomote now has a new flag to let you reuse a buildlet host connection for different builder rules if they share the same underlying host type. But users can ignore that. This CL is a long-standing TODO (previously attempted and aborted) and will make many things easier and faster, including the linux-arm cross-compilation effort, and keeping pre-warmed buildlets of VM types ready to go. Updates golang/go#17104 Change-Id: Iad8387f48680424a8441e878a2f4762bf79ea4d2 Reviewed-on: https://go-review.googlesource.com/29551 Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2016-09-22 00:27:37 +03:00
},
dashboard: add configs for new windows 2008, 2012, 2016 builders See also: https://golang.org/cl/41142 Updates golang/go#17513 Change-Id: Ie743ae4604e65892e28423ed0af008450b647197 Reviewed-on: https://go-review.googlesource.com/41393 Reviewed-by: Jeff Johnson <jrjohnson@google.com> Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
2017-04-21 22:35:28 +03:00
"host-windows-amd64-2008": &HostConfig{
env/windows: disable windows defender on builders Windows Defender scans all new files written to disk which slows down build performance. - Uninstall Windows Defender as part of image building - Refactor/Cleanup registry section for consistency - Promote new image version to dashboard (built in staging/prod) fixes golang/go#26055 Change-Id: I3e16b9a9581824c20abed5e8ffae1efd46c6dd09 Reviewed-on: https://go-review.googlesource.com/121937 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-07-02 20:49:55 +03:00
VMImage: "windows-amd64-server-2008r2-v7",
all: migrate from n1 to e2 instances We should have similar performance for these machines, and increased memory availability in some areas. This is also compliant with an internal policy change. Change-Id: I07a452a07431322d6fdd4edbb98c5914045d34ed Reviewed-on: https://go-review.googlesource.com/c/build/+/354757 Trust: Alexander Rakoczy <alex@golang.org> Run-TryBot: Alexander Rakoczy <alex@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Heschi Kreinick <heschi@google.com> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-10-09 00:03:24 +03:00
machineType: "e2-highcpu-4", // 4 vCPUs, 4 GB mem
dashboard: upsize machine type for windows-amd64-{longtest,race} builders Start using n1-highcpu-8 machine type instead of n1-highcpu-4 for windows-amd64-longtest and windows-amd64-race builders. Continue to use the existing n1-highcpu-4 machine type for all other windows/386 and windows/amd64 builders for now. The primary motivation of this change is to increase amount of RAM available on the longtest builder from 3.6 GB to 7.2 GB, because it is consistenly failing due to being out of memory. Also fix minor typos, and increase consistency in comments and notes. Updates golang/go#33951 Change-Id: Iac5da67fa584059842850a9c8deda98c242741f2 Reviewed-on: https://go-review.googlesource.com/c/build/+/208500 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org>
2019-11-23 02:01:25 +03:00
buildletURLTmpl: "http://storage.googleapis.com/$BUCKET/buildlet.windows-amd64",
goBootstrapURLTmpl: "https://storage.googleapis.com/$BUCKET/go1.4-windows-amd64.tar.gz",
SSHUsername: "gopher",
},
dashboard: add configs for new windows 2008, 2012, 2016 builders See also: https://golang.org/cl/41142 Updates golang/go#17513 Change-Id: Ie743ae4604e65892e28423ed0af008450b647197 Reviewed-on: https://go-review.googlesource.com/41393 Reviewed-by: Jeff Johnson <jrjohnson@google.com> Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
2017-04-21 22:35:28 +03:00
"host-windows-amd64-2012": &HostConfig{
env/windows: disable windows defender on builders Windows Defender scans all new files written to disk which slows down build performance. - Uninstall Windows Defender as part of image building - Refactor/Cleanup registry section for consistency - Promote new image version to dashboard (built in staging/prod) fixes golang/go#26055 Change-Id: I3e16b9a9581824c20abed5e8ffae1efd46c6dd09 Reviewed-on: https://go-review.googlesource.com/121937 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-07-02 20:49:55 +03:00
VMImage: "windows-amd64-server-2012r2-v7",
all: migrate from n1 to e2 instances We should have similar performance for these machines, and increased memory availability in some areas. This is also compliant with an internal policy change. Change-Id: I07a452a07431322d6fdd4edbb98c5914045d34ed Reviewed-on: https://go-review.googlesource.com/c/build/+/354757 Trust: Alexander Rakoczy <alex@golang.org> Run-TryBot: Alexander Rakoczy <alex@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Heschi Kreinick <heschi@google.com> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-10-09 00:03:24 +03:00
machineType: "e2-highcpu-4", // 4 vCPUs, 4 GB mem
dashboard: add configs for new windows 2008, 2012, 2016 builders See also: https://golang.org/cl/41142 Updates golang/go#17513 Change-Id: Ie743ae4604e65892e28423ed0af008450b647197 Reviewed-on: https://go-review.googlesource.com/41393 Reviewed-by: Jeff Johnson <jrjohnson@google.com> Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
2017-04-21 22:35:28 +03:00
buildletURLTmpl: "http://storage.googleapis.com/$BUCKET/buildlet.windows-amd64",
goBootstrapURLTmpl: "https://storage.googleapis.com/$BUCKET/go1.4-windows-amd64.tar.gz",
dashboard,cmd/buildlet,cmd/coordinator: public key auth on windows Enable public key auth for windows buildlets. Previously the username/password was hardcoded. This change enables the `/connect-ssh` http endpoint on the buildlet to setup authorized_keys and uses the coordinator to proxy ssh traffic to the buildlet. Validation: - run local coordinator (with patches to dev env to act like GCE) with changes - build windows buildlet, post to own bucket - execute: gomote -localdev create windows-amd64-2012 ssh -p 2222 user-jrjohnson-windows-amd64-2012-0@127.0.0.1 Fixes golang/go#21247 Change-Id: If7adc7e6296a071d50412d9fa9d46ecd9e6cc3f3 Reviewed-on: https://go-review.googlesource.com/52350 Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
2017-08-01 02:55:28 +03:00
SSHUsername: "gopher",
dashboard: add configs for new windows 2008, 2012, 2016 builders See also: https://golang.org/cl/41142 Updates golang/go#17513 Change-Id: Ie743ae4604e65892e28423ed0af008450b647197 Reviewed-on: https://go-review.googlesource.com/41393 Reviewed-by: Jeff Johnson <jrjohnson@google.com> Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
2017-04-21 22:35:28 +03:00
},
"host-windows-amd64-2016": &HostConfig{
env/windows: disable windows defender on builders Windows Defender scans all new files written to disk which slows down build performance. - Uninstall Windows Defender as part of image building - Refactor/Cleanup registry section for consistency - Promote new image version to dashboard (built in staging/prod) fixes golang/go#26055 Change-Id: I3e16b9a9581824c20abed5e8ffae1efd46c6dd09 Reviewed-on: https://go-review.googlesource.com/121937 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-07-02 20:49:55 +03:00
VMImage: "windows-amd64-server-2016-v7",
all: migrate from n1 to e2 instances We should have similar performance for these machines, and increased memory availability in some areas. This is also compliant with an internal policy change. Change-Id: I07a452a07431322d6fdd4edbb98c5914045d34ed Reviewed-on: https://go-review.googlesource.com/c/build/+/354757 Trust: Alexander Rakoczy <alex@golang.org> Run-TryBot: Alexander Rakoczy <alex@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Heschi Kreinick <heschi@google.com> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-10-09 00:03:24 +03:00
machineType: "e2-highcpu-4", // 4 vCPUs, 4 GB mem
dashboard: upsize machine type for windows-amd64-{longtest,race} builders Start using n1-highcpu-8 machine type instead of n1-highcpu-4 for windows-amd64-longtest and windows-amd64-race builders. Continue to use the existing n1-highcpu-4 machine type for all other windows/386 and windows/amd64 builders for now. The primary motivation of this change is to increase amount of RAM available on the longtest builder from 3.6 GB to 7.2 GB, because it is consistenly failing due to being out of memory. Also fix minor typos, and increase consistency in comments and notes. Updates golang/go#33951 Change-Id: Iac5da67fa584059842850a9c8deda98c242741f2 Reviewed-on: https://go-review.googlesource.com/c/build/+/208500 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org>
2019-11-23 02:01:25 +03:00
buildletURLTmpl: "http://storage.googleapis.com/$BUCKET/buildlet.windows-amd64",
goBootstrapURLTmpl: "https://storage.googleapis.com/$BUCKET/go1.4-windows-amd64.tar.gz",
SSHUsername: "gopher",
},
"host-windows-amd64-2016-big": &HostConfig{
all: migrate from n1 to e2 instances We should have similar performance for these machines, and increased memory availability in some areas. This is also compliant with an internal policy change. Change-Id: I07a452a07431322d6fdd4edbb98c5914045d34ed Reviewed-on: https://go-review.googlesource.com/c/build/+/354757 Trust: Alexander Rakoczy <alex@golang.org> Run-TryBot: Alexander Rakoczy <alex@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Heschi Kreinick <heschi@google.com> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-10-09 00:03:24 +03:00
Notes: "Same as host-windows-amd64-2016, but on e2-highcpu-16",
dashboard: upsize machine type for windows-amd64-{longtest,race} builders Start using n1-highcpu-8 machine type instead of n1-highcpu-4 for windows-amd64-longtest and windows-amd64-race builders. Continue to use the existing n1-highcpu-4 machine type for all other windows/386 and windows/amd64 builders for now. The primary motivation of this change is to increase amount of RAM available on the longtest builder from 3.6 GB to 7.2 GB, because it is consistenly failing due to being out of memory. Also fix minor typos, and increase consistency in comments and notes. Updates golang/go#33951 Change-Id: Iac5da67fa584059842850a9c8deda98c242741f2 Reviewed-on: https://go-review.googlesource.com/c/build/+/208500 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org>
2019-11-23 02:01:25 +03:00
VMImage: "windows-amd64-server-2016-v7",
all: migrate from n1 to e2 instances We should have similar performance for these machines, and increased memory availability in some areas. This is also compliant with an internal policy change. Change-Id: I07a452a07431322d6fdd4edbb98c5914045d34ed Reviewed-on: https://go-review.googlesource.com/c/build/+/354757 Trust: Alexander Rakoczy <alex@golang.org> Run-TryBot: Alexander Rakoczy <alex@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Heschi Kreinick <heschi@google.com> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-10-09 00:03:24 +03:00
machineType: "e2-highcpu-16", // 16 vCPUs, 16 GB mem
dashboard: add configs for new windows 2008, 2012, 2016 builders See also: https://golang.org/cl/41142 Updates golang/go#17513 Change-Id: Ie743ae4604e65892e28423ed0af008450b647197 Reviewed-on: https://go-review.googlesource.com/41393 Reviewed-by: Jeff Johnson <jrjohnson@google.com> Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
2017-04-21 22:35:28 +03:00
buildletURLTmpl: "http://storage.googleapis.com/$BUCKET/buildlet.windows-amd64",
goBootstrapURLTmpl: "https://storage.googleapis.com/$BUCKET/go1.4-windows-amd64.tar.gz",
dashboard,cmd/buildlet,cmd/coordinator: public key auth on windows Enable public key auth for windows buildlets. Previously the username/password was hardcoded. This change enables the `/connect-ssh` http endpoint on the buildlet to setup authorized_keys and uses the coordinator to proxy ssh traffic to the buildlet. Validation: - run local coordinator (with patches to dev env to act like GCE) with changes - build windows buildlet, post to own bucket - execute: gomote -localdev create windows-amd64-2012 ssh -p 2222 user-jrjohnson-windows-amd64-2012-0@127.0.0.1 Fixes golang/go#21247 Change-Id: If7adc7e6296a071d50412d9fa9d46ecd9e6cc3f3 Reviewed-on: https://go-review.googlesource.com/52350 Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
2017-08-01 02:55:28 +03:00
SSHUsername: "gopher",
dashboard: add configs for new windows 2008, 2012, 2016 builders See also: https://golang.org/cl/41142 Updates golang/go#17513 Change-Id: Ie743ae4604e65892e28423ed0af008450b647197 Reviewed-on: https://go-review.googlesource.com/41393 Reviewed-by: Jeff Johnson <jrjohnson@google.com> Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
2017-04-21 22:35:28 +03:00
},
dashboard: add new Windows ARM builder This partially reverts CL 229559, while adjusting it to use a new builder. The builder will be used now for windows/arm and later for windows/arm64. Updates golang/go#42604. Updates golang/go#38607. Updates golang/go#36439. Change-Id: I1ab661a6585e64de8196955c077715b32dc732b8 Reviewed-on: https://go-review.googlesource.com/c/build/+/272106 Run-TryBot: Jason A. Donenfeld <Jason@zx2c4.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Trust: Alex Brainman <alex.brainman@gmail.com> Trust: Jason A. Donenfeld <Jason@zx2c4.com>
2020-11-20 20:35:33 +03:00
"host-windows-arm64-zx2c4": &HostConfig{
dashboard, cmd/coordinator: support multiple builder owners Prefer displaying GitHub usernames for convenience of pinging in GitHub issues, since all current builder owners have them. Keep it possible to use an email or something else if no GitHub account exists. Update the owner of the Corellium builders to be individual GitHub account, so that it can be pinged in an issue. The GitHub organization is already mentioned in the builder notes. Also switch to html/template (instead of text/template) for rendering the builders page. Fixes golang/go#49596. Change-Id: I89a96eb8f7a3057bc1f3aaee16abc9c776c45ee7 Reviewed-on: https://go-review.googlesource.com/c/build/+/363983 Trust: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-11-16 20:42:29 +03:00
IsReverse: true,
ExpectNum: 1,
Owners: []*gophers.Person{gh("zx2c4")},
env: []string{"GOROOT_BOOTSTRAP=C:\\Program Files (Arm)\\Go"},
dashboard: add new Windows ARM builder This partially reverts CL 229559, while adjusting it to use a new builder. The builder will be used now for windows/arm and later for windows/arm64. Updates golang/go#42604. Updates golang/go#38607. Updates golang/go#36439. Change-Id: I1ab661a6585e64de8196955c077715b32dc732b8 Reviewed-on: https://go-review.googlesource.com/c/build/+/272106 Run-TryBot: Jason A. Donenfeld <Jason@zx2c4.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Trust: Alex Brainman <alex.brainman@gmail.com> Trust: Jason A. Donenfeld <Jason@zx2c4.com>
2020-11-20 20:35:33 +03:00
},
dashboard: add windows-arm64-10 reverse buildlet This adds a reverse buildlet for Windows ARM64 running under qemu on M1 Mac Minis. The VM currently boots, but more changes are needed to stage0 and buildlet in order for the builder to run. For golang/go#42604. Change-Id: Ie86e596c0f166d2ca8c075c0c971fcd1e34e940f Reviewed-on: https://go-review.googlesource.com/c/build/+/328311 Trust: Alexander Rakoczy <alex@golang.org> Run-TryBot: Alexander Rakoczy <alex@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Heschi Kreinick <heschi@google.com> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-06-15 23:42:51 +03:00
"host-windows-arm64-mini": &HostConfig{
Notes: "macOS hosting Windows 10 in qemu with HVM acceleration.",
buildletURLTmpl: "http://storage.googleapis.com/$BUCKET/buildlet.windows-arm64",
goBootstrapURLTmpl: "https://storage.googleapis.com/$BUCKET/gobootstrap-windows-arm64-f22ec5.tar.gz",
IsReverse: true,
dashboard: add windows-arm-10 builder This adds a windows-arm builder using host-windows-arm64-mini hosts. The new BuildConfig has a KnownIssue until we can verify that the builders are building correctly. For golang/go#47019 Change-Id: I0505d4dc367dbb687290facbddc718eb13567025 Reviewed-on: https://go-review.googlesource.com/c/build/+/344990 Trust: Alexander Rakoczy <alex@golang.org> Run-TryBot: Alexander Rakoczy <alex@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-08-25 22:50:51 +03:00
ExpectNum: 7,
dashboard: add windows-arm64-10 reverse buildlet This adds a reverse buildlet for Windows ARM64 running under qemu on M1 Mac Minis. The VM currently boots, but more changes are needed to stage0 and buildlet in order for the builder to run. For golang/go#42604. Change-Id: Ie86e596c0f166d2ca8c075c0c971fcd1e34e940f Reviewed-on: https://go-review.googlesource.com/c/build/+/328311 Trust: Alexander Rakoczy <alex@golang.org> Run-TryBot: Alexander Rakoczy <alex@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Heschi Kreinick <heschi@google.com> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-06-15 23:42:51 +03:00
},
build: add macoS Sierra builder Change-Id: I10a7a9e44a158fdce5287a9b0b065b35d11fb502 Reviewed-on: https://go-review.googlesource.com/35470 Reviewed-by: Chris Broadfoot <cbro@golang.org>
2017-01-19 01:45:12 +03:00
"host-darwin-10_12": &HostConfig{
IsReverse: true,
buildlet,dashboard: tweaks to Darwin, OpenBSD, and Linux builders - Add more Monterey builders and remove the known issue: it's stable. Also use it for race and nocgo. Update slowbot aliases to point to it. - Don't test 1.16 on OpenBSD 7.0, it segfaults. - Pin the Linux (ContainerOS) builders to an older version to avoid the pprof kernel bug. For golang/go#49065, golang/go#49149, and golang/go#48977. Change-Id: Ibec2fa735183ec65e5066c7c752ac356b7360550 Reviewed-on: https://go-review.googlesource.com/c/build/+/365777 Trust: Heschi Kreinick <heschi@google.com> Run-TryBot: Heschi Kreinick <heschi@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org>
2021-11-19 22:46:52 +03:00
ExpectNum: 2,
build: add macoS Sierra builder Change-Id: I10a7a9e44a158fdce5287a9b0b065b35d11fb502 Reviewed-on: https://go-review.googlesource.com/35470 Reviewed-by: Chris Broadfoot <cbro@golang.org>
2017-01-19 01:45:12 +03:00
Notes: "MacStadium OS X 10.12 VM under VMWare ESXi",
env: []string{
"GOROOT_BOOTSTRAP=/Users/gopher/go1.4",
},
cmd/coordinator, cmd/buildlet, cmd/gomote: add SSH support This adds an SSH server to farmer.golang.org on port 2222 that proxies SSH connections to users' gomote-created buildlet instances. For example: $ gomote create openbsd-amd64-60 user-bradfitz-openbsd-amd64-60-1 $ gomote ssh user-bradfitz-openbsd-amd64-60-1 Warning: Permanently added '[localhost]:33351' (ECDSA) to the list of known hosts. OpenBSD 6.0 (GENERIC.MP) golang/go#2319: Tue Jul 26 13:00:43 MDT 2016 Welcome to OpenBSD: The proactively secure Unix-like operating system. Please use the sendbug(1) utility to report bugs in the system. Before reporting a bug, please try to reproduce it with the latest version of the code. With bug reports, please try to ensure that enough information to reproduce the problem is enclosed, and if a known fix for it exists, include that as well. $ As before, if the coordinator process is restarted (or crashes, is evicted, etc), all gomote instances die. Not yet supported: * scp (help wanted) * not all host types are configured. most are. some will need slight config tweaks to the Docker image (e.g. adding openssh-server) Supports currently: * linux-amd64 (host type shared by 386, nacl) * linux-arm * linux-arm64 * darwin * freebsd * openbsd * plan9-386 * windows Implementation details: * the ssh server process listens on port 2222 in the coordinator (farmer.golang.org), which is behind a GKE TCP load balancer. * the ssh server library is github.com/gliderlabs/ssh * authentication is done via Github users' public keys. It's assumed that gomote user == github user. But there's a mapping in the code for known exceptions. * we can't give out access to this too widely. too many things are accessible from within the host environment if you look in the right places. Details omitted. But the Go team and other trusted gomote users can use this. * the buildlet binary has a new /connect-ssh handler that acts like a CONNECT request but instead of taking an explicit host:port, just says "give me your machine's SSH connection". The buildlet can also start sshd if needed for the environment. The /connect-ssh handler also installs the coordinator's public key. * a new buildlet client library method "ConnectSSH" hits the /connect-ssh handler and returns a net.Conn. * the coordinator's ssh.Handler is just running the OpenSSH ssh client. * because the OpenSSH ssh child process can't connect to a net.Conn, an emphemeral localhost port is created on the coordinator to proxy between the ssh client and the net.Conn returned by ConnectSSH. * The /connect-ssh handler requires http.Hijacker, which requires fully compliant net.Conn implementations as of Go 1.8. So I needed to flesh out revdial too, testing it with the golang.org/x/net/nettest package. * plan9 doesn't have an ssh server, so we use 0intro's new conterm program (drawterm without GUI support) to connect to plan9 from the coordinator ssh proxy instead of using the OpenSSH ssh client binary. * windows doesn't have an ssh server, so we enable the telnet service and the coordinator ssh proxy uses telnet instead on the backend on the private network. (There is a Windows ssh server but only in new versions.) Happy debugging over ssh! Fixes golang/go#19956 Change-Id: I80a62064c5f85af1f195f980c862ba29af4015f0 Reviewed-on: https://go-review.googlesource.com/50750 Reviewed-by: Herbie Ong <herbie@google.com> Reviewed-by: Jessie Frazelle <me@jessfraz.com> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2017-07-22 22:15:56 +03:00
SSHUsername: "gopher",
HermeticReverse: true, // we destroy the VM when done & let cmd/makemac recreate
build: add macoS Sierra builder Change-Id: I10a7a9e44a158fdce5287a9b0b065b35d11fb502 Reviewed-on: https://go-review.googlesource.com/35470 Reviewed-by: Chris Broadfoot <cbro@golang.org>
2017-01-19 01:45:12 +03:00
},
all: add a macOS 10.14 (Mojave) builder Fixes golang/go#27806 Change-Id: I576f563acb2c50cd0456cd4e6a1271b9aa59c9df Reviewed-on: https://go-review.googlesource.com/c/build/+/169498 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-03-27 02:04:31 +03:00
"host-darwin-10_14": &HostConfig{
IsReverse: true,
buildlet,dashboard: tweaks to Darwin, OpenBSD, and Linux builders - Add more Monterey builders and remove the known issue: it's stable. Also use it for race and nocgo. Update slowbot aliases to point to it. - Don't test 1.16 on OpenBSD 7.0, it segfaults. - Pin the Linux (ContainerOS) builders to an older version to avoid the pprof kernel bug. For golang/go#49065, golang/go#49149, and golang/go#48977. Change-Id: Ibec2fa735183ec65e5066c7c752ac356b7360550 Reviewed-on: https://go-review.googlesource.com/c/build/+/365777 Trust: Heschi Kreinick <heschi@google.com> Run-TryBot: Heschi Kreinick <heschi@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org>
2021-11-19 22:46:52 +03:00
ExpectNum: 2,
all: add a macOS 10.14 (Mojave) builder Fixes golang/go#27806 Change-Id: I576f563acb2c50cd0456cd4e6a1271b9aa59c9df Reviewed-on: https://go-review.googlesource.com/c/build/+/169498 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-03-27 02:04:31 +03:00
Notes: "MacStadium macOS Mojave (10.14) VM under VMWare ESXi",
env: []string{
"GOROOT_BOOTSTRAP=/Users/gopher/goboot", // Go 1.12.1
},
SSHUsername: "gopher",
HermeticReverse: true, // we destroy the VM when done & let cmd/makemac recreate
},
dashboard, cmd/makemac: add macOS 10.15 Catalina builder Added the macOS 10.15 Catalina builder to the dashboard. Catalina builders should use the Mojave guestid in vSphere 6.7 update 3 because the Catalina guestid is not available in this version of vSphere. A Catalina builder will run on this version of vSphere with the Mojave guestid. Modified the expected number of vm instances running on Macstadium due to the addition of macOS 10.15 builders. Updates golang/go#34748 Change-Id: If14f6e088afc05dd1783460f21907e9345e3c338 Reviewed-on: https://go-review.googlesource.com/c/build/+/207937 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-11-19 17:55:01 +03:00
"host-darwin-10_15": &HostConfig{
IsReverse: true,
buildlet,dashboard: tweaks to Darwin, OpenBSD, and Linux builders - Add more Monterey builders and remove the known issue: it's stable. Also use it for race and nocgo. Update slowbot aliases to point to it. - Don't test 1.16 on OpenBSD 7.0, it segfaults. - Pin the Linux (ContainerOS) builders to an older version to avoid the pprof kernel bug. For golang/go#49065, golang/go#49149, and golang/go#48977. Change-Id: Ibec2fa735183ec65e5066c7c752ac356b7360550 Reviewed-on: https://go-review.googlesource.com/c/build/+/365777 Trust: Heschi Kreinick <heschi@google.com> Run-TryBot: Heschi Kreinick <heschi@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org>
2021-11-19 22:46:52 +03:00
ExpectNum: 3,
dashboard, cmd/makemac: add macOS 10.15 Catalina builder Added the macOS 10.15 Catalina builder to the dashboard. Catalina builders should use the Mojave guestid in vSphere 6.7 update 3 because the Catalina guestid is not available in this version of vSphere. A Catalina builder will run on this version of vSphere with the Mojave guestid. Modified the expected number of vm instances running on Macstadium due to the addition of macOS 10.15 builders. Updates golang/go#34748 Change-Id: If14f6e088afc05dd1783460f21907e9345e3c338 Reviewed-on: https://go-review.googlesource.com/c/build/+/207937 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-11-19 17:55:01 +03:00
Notes: "MacStadium macOS Catalina (10.15) VM under VMWare ESXi",
env: []string{
"GOROOT_BOOTSTRAP=/Users/gopher/goboot", // Go 1.12.1
},
SSHUsername: "gopher",
HermeticReverse: true, // we destroy the VM when done & let cmd/makemac recreate
},
dashboard: add macOS 11.0 Big Sur builder This change adds macOS 11.0 Big Sur builder to the dashboard. It will not spawn an instance on the older MacStadium cluster which we are migrating away from. It will be marked with a known issue until until we have migrated away from the old cluster and makemac has been updated to process the new macOS versioning convention. Updates golang/go#40762 Updates golang/go#40757 Updates golang/go#40761 Change-Id: I24ce615a2c0c55dd75beed4f821119d4058ad8af Reviewed-on: https://go-review.googlesource.com/c/build/+/288192 Trust: Carlos Amedee <carlos@golang.org> Run-TryBot: Carlos Amedee <carlos@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2021-01-29 23:47:13 +03:00
"host-darwin-amd64-11_0": &HostConfig{
IsReverse: true,
buildlet,dashboard: tweaks to Darwin, OpenBSD, and Linux builders - Add more Monterey builders and remove the known issue: it's stable. Also use it for race and nocgo. Update slowbot aliases to point to it. - Don't test 1.16 on OpenBSD 7.0, it segfaults. - Pin the Linux (ContainerOS) builders to an older version to avoid the pprof kernel bug. For golang/go#49065, golang/go#49149, and golang/go#48977. Change-Id: Ibec2fa735183ec65e5066c7c752ac356b7360550 Reviewed-on: https://go-review.googlesource.com/c/build/+/365777 Trust: Heschi Kreinick <heschi@google.com> Run-TryBot: Heschi Kreinick <heschi@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org>
2021-11-19 22:46:52 +03:00
ExpectNum: 4,
dashboard: add macOS 11.0 Big Sur builder This change adds macOS 11.0 Big Sur builder to the dashboard. It will not spawn an instance on the older MacStadium cluster which we are migrating away from. It will be marked with a known issue until until we have migrated away from the old cluster and makemac has been updated to process the new macOS versioning convention. Updates golang/go#40762 Updates golang/go#40757 Updates golang/go#40761 Change-Id: I24ce615a2c0c55dd75beed4f821119d4058ad8af Reviewed-on: https://go-review.googlesource.com/c/build/+/288192 Trust: Carlos Amedee <carlos@golang.org> Run-TryBot: Carlos Amedee <carlos@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2021-01-29 23:47:13 +03:00
Notes: "MacStadium macOS Big Sur (11.0) VM under VMWare ESXi",
env: []string{
"GOROOT_BOOTSTRAP=/Users/gopher/goboot", // Go 1.13.4
},
SSHUsername: "gopher",
HermeticReverse: true, // we destroy the VM when done & let cmd/makemac recreate
},
all: add macOS 12 AMD64 builders I'm not 100% sure they'll work, so just one for now. I've updated the documentation according to my very shaky understanding. For golang/go#49149. Change-Id: Ic8f3b7fe96a1a6c18c0940a4aa2d08ec9dc2138b Reviewed-on: https://go-review.googlesource.com/c/build/+/365194 Trust: Heschi Kreinick <heschi@google.com> Run-TryBot: Heschi Kreinick <heschi@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-11-17 01:09:45 +03:00
"host-darwin-amd64-12_0": &HostConfig{
IsReverse: true,
buildlet,dashboard: tweaks to Darwin, OpenBSD, and Linux builders - Add more Monterey builders and remove the known issue: it's stable. Also use it for race and nocgo. Update slowbot aliases to point to it. - Don't test 1.16 on OpenBSD 7.0, it segfaults. - Pin the Linux (ContainerOS) builders to an older version to avoid the pprof kernel bug. For golang/go#49065, golang/go#49149, and golang/go#48977. Change-Id: Ibec2fa735183ec65e5066c7c752ac356b7360550 Reviewed-on: https://go-review.googlesource.com/c/build/+/365777 Trust: Heschi Kreinick <heschi@google.com> Run-TryBot: Heschi Kreinick <heschi@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org>
2021-11-19 22:46:52 +03:00
ExpectNum: 5,
all: add macOS 12 AMD64 builders I'm not 100% sure they'll work, so just one for now. I've updated the documentation according to my very shaky understanding. For golang/go#49149. Change-Id: Ic8f3b7fe96a1a6c18c0940a4aa2d08ec9dc2138b Reviewed-on: https://go-review.googlesource.com/c/build/+/365194 Trust: Heschi Kreinick <heschi@google.com> Run-TryBot: Heschi Kreinick <heschi@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-11-17 01:09:45 +03:00
Notes: "MacStadium macOS Monterey (12.0) VM under VMWare ESXi",
env: []string{
"GOROOT_BOOTSTRAP=/Users/gopher/goboot", // Go 1.17.3
dashboard: move MallocNanoZone to host configuration We need this on every build and I forgot to put it on the new ones. Move it to the host where it's not forgettable. Hopefully fixes golang/go#49723. Change-Id: I9263f8aa9d847250bb438f59c04fe48bcfd5b1da Reviewed-on: https://go-review.googlesource.com/c/build/+/366136 Trust: Heschi Kreinick <heschi@google.com> Run-TryBot: Heschi Kreinick <heschi@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com>
2021-11-22 20:55:03 +03:00
"MallocNanoZone=0", // golang.org/issue/49138
all: add macOS 12 AMD64 builders I'm not 100% sure they'll work, so just one for now. I've updated the documentation according to my very shaky understanding. For golang/go#49149. Change-Id: Ic8f3b7fe96a1a6c18c0940a4aa2d08ec9dc2138b Reviewed-on: https://go-review.googlesource.com/c/build/+/365194 Trust: Heschi Kreinick <heschi@google.com> Run-TryBot: Heschi Kreinick <heschi@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-11-17 01:09:45 +03:00
},
SSHUsername: "gopher",
HermeticReverse: true, // we destroy the VM when done & let cmd/makemac recreate
},
dashboard: add new darwin-arm64-11_0-toothrot builder These builders are run by the Go team. They are non-hermetic. For golang/go#39782 Change-Id: Ie402faf9a92ff6c667c8da6fc0dac8528331df48 Reviewed-on: https://go-review.googlesource.com/c/build/+/278432 Trust: Alexander Rakoczy <alex@golang.org> Run-TryBot: Alexander Rakoczy <alex@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
2020-12-15 20:58:50 +03:00
"host-darwin-arm64-11_0-toothrot": &HostConfig{
IsReverse: true,
Notes: "macOS Big Sur (11.0) ARM64 (M1). Mac mini",
dashboard: add darwin-arm64-12_0 builder Upgrades our arm64-11_0 builder. For golang/go#49149 Change-Id: I517cc5a999cea0cbc96ee2a6d3901840e2473b6a Reviewed-on: https://go-review.googlesource.com/c/build/+/365736 Trust: Alexander Rakoczy <alex@golang.org> Run-TryBot: Alexander Rakoczy <alex@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2021-11-18 19:31:07 +03:00
ExpectNum: 1,
env: []string{
"GOROOT_BOOTSTRAP=/Users/gopher/goboot",
},
SSHUsername: "gopher",
},
"host-darwin-arm64-12_0-toothrot": &HostConfig{
IsReverse: true,
ExpectNum: 1,
Notes: "macOS Big Sur (12.0) ARM64 (M1). Mac mini",
dashboard: add new darwin-arm64-11_0-toothrot builder These builders are run by the Go team. They are non-hermetic. For golang/go#39782 Change-Id: Ie402faf9a92ff6c667c8da6fc0dac8528331df48 Reviewed-on: https://go-review.googlesource.com/c/build/+/278432 Trust: Alexander Rakoczy <alex@golang.org> Run-TryBot: Alexander Rakoczy <alex@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
2020-12-15 20:58:50 +03:00
env: []string{
"GOROOT_BOOTSTRAP=/Users/gopher/goboot",
dashboard: set MallocNanoZone on ARM64 too Accidentally removed in the fix for an accidental removal because I am in too much of a hurry today. Change-Id: I67a5518f36b051c2e99c0d3cabed3d61ce3401e9 Reviewed-on: https://go-review.googlesource.com/c/build/+/366234 Trust: Heschi Kreinick <heschi@google.com> Run-TryBot: Heschi Kreinick <heschi@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org> TryBot-Result: Go Bot <gobot@golang.org>
2021-11-22 21:47:03 +03:00
"MallocNanoZone=0", // golang.org/issue/49138
dashboard: add new darwin-arm64-11_0-toothrot builder These builders are run by the Go team. They are non-hermetic. For golang/go#39782 Change-Id: Ie402faf9a92ff6c667c8da6fc0dac8528331df48 Reviewed-on: https://go-review.googlesource.com/c/build/+/278432 Trust: Alexander Rakoczy <alex@golang.org> Run-TryBot: Alexander Rakoczy <alex@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
2020-12-15 20:58:50 +03:00
},
SSHUsername: "gopher",
},
all: split builder config into builder & host configs Our builders are named of the form "GOOS-GOARCH" or "GOOS-GOARCH-suffix". Over time we've grown many builders. This CL doesn't change that. Builders continue to be named and operate as before. Previously the build configuration file (dashboard/builders.go) made each builder type ("linux-amd64-race", etc) define how to create a host running a buildlet of that type, even though many builders had identical host configs. For example, these builders all share the same host type (a Kubernetes container): linux-amd64 linux-amd64-race linux-386 linux-386-387 And these are the same host type (a GCE VM): windows-amd64-gce windows-amd64-race windows-386-gce This CL creates a new concept of a "hostType" which defines how the buildlet is created (Kube, GCE, Reverse, and how), and then each builder itself references a host type. Users never see the hostType. (except perhaps in gomote list output) But they at least never need to care about them. Reverse buildlets now can only be one hostType at a time, which simplifies things. We were no longer using multiple roles per machine once moving to VMs for OS X. gomote continues to operate as it did previously but its underlying protocol changed and clients will need to be updated. As a new feature, gomote now has a new flag to let you reuse a buildlet host connection for different builder rules if they share the same underlying host type. But users can ignore that. This CL is a long-standing TODO (previously attempted and aborted) and will make many things easier and faster, including the linux-arm cross-compilation effort, and keeping pre-warmed buildlets of VM types ready to go. Updates golang/go#17104 Change-Id: Iad8387f48680424a8441e878a2f4762bf79ea4d2 Reviewed-on: https://go-review.googlesource.com/29551 Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2016-09-22 00:27:37 +03:00
"host-linux-s390x": &HostConfig{
dashboard, cmd/coordinator: support multiple builder owners Prefer displaying GitHub usernames for convenience of pinging in GitHub issues, since all current builder owners have them. Keep it possible to use an email or something else if no GitHub account exists. Update the owner of the Corellium builders to be individual GitHub account, so that it can be pinged in an issue. The GitHub organization is already mentioned in the builder notes. Also switch to html/template (instead of text/template) for rendering the builders page. Fixes golang/go#49596. Change-Id: I89a96eb8f7a3057bc1f3aaee16abc9c776c45ee7 Reviewed-on: https://go-review.googlesource.com/c/build/+/363983 Trust: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-11-16 20:42:29 +03:00
Notes: "run by IBM",
dashboard: add s390x builder maintainers Also set ExpectNum 2, since there are 2 known reverse builders. Updates golang/go#49557. Change-Id: I639f88701b62004619b8dbeacda79cb2469d1595 Reviewed-on: https://go-review.googlesource.com/c/build/+/363984 Reviewed-by: Heschi Kreinick <heschi@google.com> Trust: Dmitri Shuralyov <dmitshur@golang.org>
2021-11-16 20:42:29 +03:00
Owners: []*gophers.Person{gh("ruixin-bao"), gh("jonathan-albrecht-ibm"), gophers.GetPerson("Cindy Lee")}, // See https://groups.google.com/g/golang-dev/c/obUDaYbaxXw/m/5sMgfDYVAAAJ.
dashboard, cmd/coordinator: support multiple builder owners Prefer displaying GitHub usernames for convenience of pinging in GitHub issues, since all current builder owners have them. Keep it possible to use an email or something else if no GitHub account exists. Update the owner of the Corellium builders to be individual GitHub account, so that it can be pinged in an issue. The GitHub organization is already mentioned in the builder notes. Also switch to html/template (instead of text/template) for rendering the builders page. Fixes golang/go#49596. Change-Id: I89a96eb8f7a3057bc1f3aaee16abc9c776c45ee7 Reviewed-on: https://go-review.googlesource.com/c/build/+/363983 Trust: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-11-16 20:42:29 +03:00
IsReverse: true,
dashboard: add s390x builder maintainers Also set ExpectNum 2, since there are 2 known reverse builders. Updates golang/go#49557. Change-Id: I639f88701b62004619b8dbeacda79cb2469d1595 Reviewed-on: https://go-review.googlesource.com/c/build/+/363984 Reviewed-by: Heschi Kreinick <heschi@google.com> Trust: Dmitri Shuralyov <dmitshur@golang.org>
2021-11-16 20:42:29 +03:00
ExpectNum: 2, // See https://github.com/golang/go/issues/49557#issuecomment-969148789.
dashboard, cmd/coordinator: support multiple builder owners Prefer displaying GitHub usernames for convenience of pinging in GitHub issues, since all current builder owners have them. Keep it possible to use an email or something else if no GitHub account exists. Update the owner of the Corellium builders to be individual GitHub account, so that it can be pinged in an issue. The GitHub organization is already mentioned in the builder notes. Also switch to html/template (instead of text/template) for rendering the builders page. Fixes golang/go#49596. Change-Id: I89a96eb8f7a3057bc1f3aaee16abc9c776c45ee7 Reviewed-on: https://go-review.googlesource.com/c/build/+/363983 Trust: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-11-16 20:42:29 +03:00
env: []string{"GOROOT_BOOTSTRAP=/var/buildlet/go-linux-s390x-bootstrap"},
all: split builder config into builder & host configs Our builders are named of the form "GOOS-GOARCH" or "GOOS-GOARCH-suffix". Over time we've grown many builders. This CL doesn't change that. Builders continue to be named and operate as before. Previously the build configuration file (dashboard/builders.go) made each builder type ("linux-amd64-race", etc) define how to create a host running a buildlet of that type, even though many builders had identical host configs. For example, these builders all share the same host type (a Kubernetes container): linux-amd64 linux-amd64-race linux-386 linux-386-387 And these are the same host type (a GCE VM): windows-amd64-gce windows-amd64-race windows-386-gce This CL creates a new concept of a "hostType" which defines how the buildlet is created (Kube, GCE, Reverse, and how), and then each builder itself references a host type. Users never see the hostType. (except perhaps in gomote list output) But they at least never need to care about them. Reverse buildlets now can only be one hostType at a time, which simplifies things. We were no longer using multiple roles per machine once moving to VMs for OS X. gomote continues to operate as it did previously but its underlying protocol changed and clients will need to be updated. As a new feature, gomote now has a new flag to let you reuse a buildlet host connection for different builder rules if they share the same underlying host type. But users can ignore that. This CL is a long-standing TODO (previously attempted and aborted) and will make many things easier and faster, including the linux-arm cross-compilation effort, and keeping pre-warmed buildlets of VM types ready to go. Updates golang/go#17104 Change-Id: Iad8387f48680424a8441e878a2f4762bf79ea4d2 Reviewed-on: https://go-review.googlesource.com/29551 Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2016-09-22 00:27:37 +03:00
},
all: add ppc64 (big endian) builders Change-Id: I0fddda33179555a50b6f03dcff13338544b742c3 Reviewed-on: https://go-review.googlesource.com/30599 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2016-10-06 23:30:58 +03:00
"host-linux-ppc64-osu": &HostConfig{
cmd/coordinator, cmd/buildlet, cmd/gomote: add SSH support This adds an SSH server to farmer.golang.org on port 2222 that proxies SSH connections to users' gomote-created buildlet instances. For example: $ gomote create openbsd-amd64-60 user-bradfitz-openbsd-amd64-60-1 $ gomote ssh user-bradfitz-openbsd-amd64-60-1 Warning: Permanently added '[localhost]:33351' (ECDSA) to the list of known hosts. OpenBSD 6.0 (GENERIC.MP) golang/go#2319: Tue Jul 26 13:00:43 MDT 2016 Welcome to OpenBSD: The proactively secure Unix-like operating system. Please use the sendbug(1) utility to report bugs in the system. Before reporting a bug, please try to reproduce it with the latest version of the code. With bug reports, please try to ensure that enough information to reproduce the problem is enclosed, and if a known fix for it exists, include that as well. $ As before, if the coordinator process is restarted (or crashes, is evicted, etc), all gomote instances die. Not yet supported: * scp (help wanted) * not all host types are configured. most are. some will need slight config tweaks to the Docker image (e.g. adding openssh-server) Supports currently: * linux-amd64 (host type shared by 386, nacl) * linux-arm * linux-arm64 * darwin * freebsd * openbsd * plan9-386 * windows Implementation details: * the ssh server process listens on port 2222 in the coordinator (farmer.golang.org), which is behind a GKE TCP load balancer. * the ssh server library is github.com/gliderlabs/ssh * authentication is done via Github users' public keys. It's assumed that gomote user == github user. But there's a mapping in the code for known exceptions. * we can't give out access to this too widely. too many things are accessible from within the host environment if you look in the right places. Details omitted. But the Go team and other trusted gomote users can use this. * the buildlet binary has a new /connect-ssh handler that acts like a CONNECT request but instead of taking an explicit host:port, just says "give me your machine's SSH connection". The buildlet can also start sshd if needed for the environment. The /connect-ssh handler also installs the coordinator's public key. * a new buildlet client library method "ConnectSSH" hits the /connect-ssh handler and returns a net.Conn. * the coordinator's ssh.Handler is just running the OpenSSH ssh client. * because the OpenSSH ssh child process can't connect to a net.Conn, an emphemeral localhost port is created on the coordinator to proxy between the ssh client and the net.Conn returned by ConnectSSH. * The /connect-ssh handler requires http.Hijacker, which requires fully compliant net.Conn implementations as of Go 1.8. So I needed to flesh out revdial too, testing it with the golang.org/x/net/nettest package. * plan9 doesn't have an ssh server, so we use 0intro's new conterm program (drawterm without GUI support) to connect to plan9 from the coordinator ssh proxy instead of using the OpenSSH ssh client binary. * windows doesn't have an ssh server, so we enable the telnet service and the coordinator ssh proxy uses telnet instead on the backend on the private network. (There is a Windows ssh server but only in new versions.) Happy debugging over ssh! Fixes golang/go#19956 Change-Id: I80a62064c5f85af1f195f980c862ba29af4015f0 Reviewed-on: https://go-review.googlesource.com/50750 Reviewed-by: Herbie Ong <herbie@google.com> Reviewed-by: Jessie Frazelle <me@jessfraz.com> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2017-07-22 22:15:56 +03:00
Notes: "Debian jessie; run by Go team on osuosl.org",
IsReverse: true,
ExpectNum: 5,
env: []string{"GOROOT_BOOTSTRAP=/usr/local/go-bootstrap"},
dashboard: fix ssh username for ppc64 builders Updates golang/go#34739 Change-Id: I9e9bcf66a4e3939a68178b344672a1bab2698bb4 Reviewed-on: https://go-review.googlesource.com/c/build/+/200377 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-10-10 20:44:44 +03:00
SSHUsername: "root",
all: update linux-ppc64le builders Fixes golang/go#21189 Updates golang/go#21260 Updates golang/go#34830 (ppc64 big endian remains) Change-Id: I7ff38edb519bf4f34cd05bb795b6a41f72b10640 Reviewed-on: https://go-review.googlesource.com/c/build/+/201637 Reviewed-by: Carlos Eduardo Seo <cseo@linux.vnet.ibm.com> Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-10-17 08:08:40 +03:00
HermeticReverse: false, // TODO: run in chroots with overlayfs? https://github.com/golang/go/issues/34830#issuecomment-543386764
all: add ppc64 (big endian) builders Change-Id: I0fddda33179555a50b6f03dcff13338544b742c3 Reviewed-on: https://go-review.googlesource.com/30599 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2016-10-06 23:30:58 +03:00
},
all: split builder config into builder & host configs Our builders are named of the form "GOOS-GOARCH" or "GOOS-GOARCH-suffix". Over time we've grown many builders. This CL doesn't change that. Builders continue to be named and operate as before. Previously the build configuration file (dashboard/builders.go) made each builder type ("linux-amd64-race", etc) define how to create a host running a buildlet of that type, even though many builders had identical host configs. For example, these builders all share the same host type (a Kubernetes container): linux-amd64 linux-amd64-race linux-386 linux-386-387 And these are the same host type (a GCE VM): windows-amd64-gce windows-amd64-race windows-386-gce This CL creates a new concept of a "hostType" which defines how the buildlet is created (Kube, GCE, Reverse, and how), and then each builder itself references a host type. Users never see the hostType. (except perhaps in gomote list output) But they at least never need to care about them. Reverse buildlets now can only be one hostType at a time, which simplifies things. We were no longer using multiple roles per machine once moving to VMs for OS X. gomote continues to operate as it did previously but its underlying protocol changed and clients will need to be updated. As a new feature, gomote now has a new flag to let you reuse a buildlet host connection for different builder rules if they share the same underlying host type. But users can ignore that. This CL is a long-standing TODO (previously attempted and aborted) and will make many things easier and faster, including the linux-arm cross-compilation effort, and keeping pre-warmed buildlets of VM types ready to go. Updates golang/go#17104 Change-Id: Iad8387f48680424a8441e878a2f4762bf79ea4d2 Reviewed-on: https://go-review.googlesource.com/29551 Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2016-09-22 00:27:37 +03:00
"host-linux-ppc64le-osu": &HostConfig{
all: update linux-ppc64le builders Fixes golang/go#21189 Updates golang/go#21260 Updates golang/go#34830 (ppc64 big endian remains) Change-Id: I7ff38edb519bf4f34cd05bb795b6a41f72b10640 Reviewed-on: https://go-review.googlesource.com/c/build/+/201637 Reviewed-by: Carlos Eduardo Seo <cseo@linux.vnet.ibm.com> Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-10-17 08:08:40 +03:00
Notes: "Debian Buster; run by Go team on osuosl.org; see x/build/env/linux-ppc64le/osuosl",
cmd/coordinator, cmd/buildlet, cmd/gomote: add SSH support This adds an SSH server to farmer.golang.org on port 2222 that proxies SSH connections to users' gomote-created buildlet instances. For example: $ gomote create openbsd-amd64-60 user-bradfitz-openbsd-amd64-60-1 $ gomote ssh user-bradfitz-openbsd-amd64-60-1 Warning: Permanently added '[localhost]:33351' (ECDSA) to the list of known hosts. OpenBSD 6.0 (GENERIC.MP) golang/go#2319: Tue Jul 26 13:00:43 MDT 2016 Welcome to OpenBSD: The proactively secure Unix-like operating system. Please use the sendbug(1) utility to report bugs in the system. Before reporting a bug, please try to reproduce it with the latest version of the code. With bug reports, please try to ensure that enough information to reproduce the problem is enclosed, and if a known fix for it exists, include that as well. $ As before, if the coordinator process is restarted (or crashes, is evicted, etc), all gomote instances die. Not yet supported: * scp (help wanted) * not all host types are configured. most are. some will need slight config tweaks to the Docker image (e.g. adding openssh-server) Supports currently: * linux-amd64 (host type shared by 386, nacl) * linux-arm * linux-arm64 * darwin * freebsd * openbsd * plan9-386 * windows Implementation details: * the ssh server process listens on port 2222 in the coordinator (farmer.golang.org), which is behind a GKE TCP load balancer. * the ssh server library is github.com/gliderlabs/ssh * authentication is done via Github users' public keys. It's assumed that gomote user == github user. But there's a mapping in the code for known exceptions. * we can't give out access to this too widely. too many things are accessible from within the host environment if you look in the right places. Details omitted. But the Go team and other trusted gomote users can use this. * the buildlet binary has a new /connect-ssh handler that acts like a CONNECT request but instead of taking an explicit host:port, just says "give me your machine's SSH connection". The buildlet can also start sshd if needed for the environment. The /connect-ssh handler also installs the coordinator's public key. * a new buildlet client library method "ConnectSSH" hits the /connect-ssh handler and returns a net.Conn. * the coordinator's ssh.Handler is just running the OpenSSH ssh client. * because the OpenSSH ssh child process can't connect to a net.Conn, an emphemeral localhost port is created on the coordinator to proxy between the ssh client and the net.Conn returned by ConnectSSH. * The /connect-ssh handler requires http.Hijacker, which requires fully compliant net.Conn implementations as of Go 1.8. So I needed to flesh out revdial too, testing it with the golang.org/x/net/nettest package. * plan9 doesn't have an ssh server, so we use 0intro's new conterm program (drawterm without GUI support) to connect to plan9 from the coordinator ssh proxy instead of using the OpenSSH ssh client binary. * windows doesn't have an ssh server, so we enable the telnet service and the coordinator ssh proxy uses telnet instead on the backend on the private network. (There is a Windows ssh server but only in new versions.) Happy debugging over ssh! Fixes golang/go#19956 Change-Id: I80a62064c5f85af1f195f980c862ba29af4015f0 Reviewed-on: https://go-review.googlesource.com/50750 Reviewed-by: Herbie Ong <herbie@google.com> Reviewed-by: Jessie Frazelle <me@jessfraz.com> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2017-07-22 22:15:56 +03:00
IsReverse: true,
ExpectNum: 5,
env: []string{"GOROOT_BOOTSTRAP=/usr/local/go-bootstrap"},
dashboard: fix ssh username for ppc64 builders Updates golang/go#34739 Change-Id: I9e9bcf66a4e3939a68178b344672a1bab2698bb4 Reviewed-on: https://go-review.googlesource.com/c/build/+/200377 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-10-10 20:44:44 +03:00
SSHUsername: "root",
all: update linux-ppc64le builders Fixes golang/go#21189 Updates golang/go#21260 Updates golang/go#34830 (ppc64 big endian remains) Change-Id: I7ff38edb519bf4f34cd05bb795b6a41f72b10640 Reviewed-on: https://go-review.googlesource.com/c/build/+/201637 Reviewed-by: Carlos Eduardo Seo <cseo@linux.vnet.ibm.com> Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-10-17 08:08:40 +03:00
HermeticReverse: true,
all: split builder config into builder & host configs Our builders are named of the form "GOOS-GOARCH" or "GOOS-GOARCH-suffix". Over time we've grown many builders. This CL doesn't change that. Builders continue to be named and operate as before. Previously the build configuration file (dashboard/builders.go) made each builder type ("linux-amd64-race", etc) define how to create a host running a buildlet of that type, even though many builders had identical host configs. For example, these builders all share the same host type (a Kubernetes container): linux-amd64 linux-amd64-race linux-386 linux-386-387 And these are the same host type (a GCE VM): windows-amd64-gce windows-amd64-race windows-386-gce This CL creates a new concept of a "hostType" which defines how the buildlet is created (Kube, GCE, Reverse, and how), and then each builder itself references a host type. Users never see the hostType. (except perhaps in gomote list output) But they at least never need to care about them. Reverse buildlets now can only be one hostType at a time, which simplifies things. We were no longer using multiple roles per machine once moving to VMs for OS X. gomote continues to operate as it did previously but its underlying protocol changed and clients will need to be updated. As a new feature, gomote now has a new flag to let you reuse a buildlet host connection for different builder rules if they share the same underlying host type. But users can ignore that. This CL is a long-standing TODO (previously attempted and aborted) and will make many things easier and faster, including the linux-arm cross-compilation effort, and keeping pre-warmed buildlets of VM types ready to go. Updates golang/go#17104 Change-Id: Iad8387f48680424a8441e878a2f4762bf79ea4d2 Reviewed-on: https://go-review.googlesource.com/29551 Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2016-09-22 00:27:37 +03:00
},
dashboard: add linux-ppc64le POWER9 builder This change adds an extra linux-ppc64le builder running on POWER9. Fixes golang/go#30138 Change-Id: I23bdfc116bb89c6d96be330c2c87827c47705b6b Reviewed-on: https://go-review.googlesource.com/c/163757 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-02-25 23:05:12 +03:00
"host-linux-ppc64le-power9-osu": &HostConfig{
all: update linux-ppc64le builders Fixes golang/go#21189 Updates golang/go#21260 Updates golang/go#34830 (ppc64 big endian remains) Change-Id: I7ff38edb519bf4f34cd05bb795b6a41f72b10640 Reviewed-on: https://go-review.googlesource.com/c/build/+/201637 Reviewed-by: Carlos Eduardo Seo <cseo@linux.vnet.ibm.com> Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-10-17 08:08:40 +03:00
Notes: "Debian Buster; run by Go team on osuosl.org; see x/build/env/linux-ppc64le/osuosl",
dashboard: add linux-ppc64le POWER9 builder This change adds an extra linux-ppc64le builder running on POWER9. Fixes golang/go#30138 Change-Id: I23bdfc116bb89c6d96be330c2c87827c47705b6b Reviewed-on: https://go-review.googlesource.com/c/163757 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-02-25 23:05:12 +03:00
IsReverse: true,
dashboard: fix POWER9 builder code generation When we moved the ppc64le builders to Docker, GOPPC64=power9 was not set for the POWER9 builders. This variable has to be passed to the environment so the compiler can generate POWER9 instructions. Fixes golang/go#36802 Change-Id: I13e99eb316e2126302e26ee771aaf8bbfbbc1c8c Reviewed-on: https://go-review.googlesource.com/c/build/+/216577 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2020-01-27 21:28:06 +03:00
env: []string{"GOROOT_BOOTSTRAP=/usr/local/go-bootstrap", "GOPPC64=power9"},
dashboard: fix ssh username for ppc64le-power9 builder Updates golang/go#34739 Change-Id: I6499ff6a0e1885fad3a5f5ba7c1fa4e0113e716c Reviewed-on: https://go-review.googlesource.com/c/build/+/200398 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-10-10 22:48:15 +03:00
SSHUsername: "root",
all: update linux-ppc64le builders Fixes golang/go#21189 Updates golang/go#21260 Updates golang/go#34830 (ppc64 big endian remains) Change-Id: I7ff38edb519bf4f34cd05bb795b6a41f72b10640 Reviewed-on: https://go-review.googlesource.com/c/build/+/201637 Reviewed-by: Carlos Eduardo Seo <cseo@linux.vnet.ibm.com> Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-10-17 08:08:40 +03:00
HermeticReverse: true,
dashboard: add linux-ppc64le POWER9 builder This change adds an extra linux-ppc64le builder running on POWER9. Fixes golang/go#30138 Change-Id: I23bdfc116bb89c6d96be330c2c87827c47705b6b Reviewed-on: https://go-review.googlesource.com/c/163757 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-02-25 23:05:12 +03:00
},
build: update NetBSD from 7.0 to 7.1, disable builds, add arm64 packet builder Change-Id: I73b8d6e6ccaeca3b932238448ddcceead34105b4 Reviewed-on: https://go-review.googlesource.com/39851 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2017-04-07 01:16:28 +03:00
"host-linux-arm64-packet": &HostConfig{
env/linux-arm64: update container image from Debian Jessie to Buster Fixes golang/go#34905 Change-Id: Ib467d31b158b5ad3f4f993959fdf1a25ac91b8b1 Reviewed-on: https://go-review.googlesource.com/c/build/+/201080 Reviewed-by: Keith Randall <khr@golang.org>
2019-10-15 07:44:49 +03:00
Notes: "On 96 core packet.net host (Xenial) in Docker containers (Debian Buster); run by Go team. See x/build/env/linux-arm64/packet",
cmd/coordinator, cmd/buildlet, cmd/gomote: add SSH support This adds an SSH server to farmer.golang.org on port 2222 that proxies SSH connections to users' gomote-created buildlet instances. For example: $ gomote create openbsd-amd64-60 user-bradfitz-openbsd-amd64-60-1 $ gomote ssh user-bradfitz-openbsd-amd64-60-1 Warning: Permanently added '[localhost]:33351' (ECDSA) to the list of known hosts. OpenBSD 6.0 (GENERIC.MP) golang/go#2319: Tue Jul 26 13:00:43 MDT 2016 Welcome to OpenBSD: The proactively secure Unix-like operating system. Please use the sendbug(1) utility to report bugs in the system. Before reporting a bug, please try to reproduce it with the latest version of the code. With bug reports, please try to ensure that enough information to reproduce the problem is enclosed, and if a known fix for it exists, include that as well. $ As before, if the coordinator process is restarted (or crashes, is evicted, etc), all gomote instances die. Not yet supported: * scp (help wanted) * not all host types are configured. most are. some will need slight config tweaks to the Docker image (e.g. adding openssh-server) Supports currently: * linux-amd64 (host type shared by 386, nacl) * linux-arm * linux-arm64 * darwin * freebsd * openbsd * plan9-386 * windows Implementation details: * the ssh server process listens on port 2222 in the coordinator (farmer.golang.org), which is behind a GKE TCP load balancer. * the ssh server library is github.com/gliderlabs/ssh * authentication is done via Github users' public keys. It's assumed that gomote user == github user. But there's a mapping in the code for known exceptions. * we can't give out access to this too widely. too many things are accessible from within the host environment if you look in the right places. Details omitted. But the Go team and other trusted gomote users can use this. * the buildlet binary has a new /connect-ssh handler that acts like a CONNECT request but instead of taking an explicit host:port, just says "give me your machine's SSH connection". The buildlet can also start sshd if needed for the environment. The /connect-ssh handler also installs the coordinator's public key. * a new buildlet client library method "ConnectSSH" hits the /connect-ssh handler and returns a net.Conn. * the coordinator's ssh.Handler is just running the OpenSSH ssh client. * because the OpenSSH ssh child process can't connect to a net.Conn, an emphemeral localhost port is created on the coordinator to proxy between the ssh client and the net.Conn returned by ConnectSSH. * The /connect-ssh handler requires http.Hijacker, which requires fully compliant net.Conn implementations as of Go 1.8. So I needed to flesh out revdial too, testing it with the golang.org/x/net/nettest package. * plan9 doesn't have an ssh server, so we use 0intro's new conterm program (drawterm without GUI support) to connect to plan9 from the coordinator ssh proxy instead of using the OpenSSH ssh client binary. * windows doesn't have an ssh server, so we enable the telnet service and the coordinator ssh proxy uses telnet instead on the backend on the private network. (There is a Windows ssh server but only in new versions.) Happy debugging over ssh! Fixes golang/go#19956 Change-Id: I80a62064c5f85af1f195f980c862ba29af4015f0 Reviewed-on: https://go-review.googlesource.com/50750 Reviewed-by: Herbie Ong <herbie@google.com> Reviewed-by: Jessie Frazelle <me@jessfraz.com> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2017-07-22 22:15:56 +03:00
IsReverse: true,
HermeticReverse: true,
dashboard: restrict CPU usage on packet builders The packet arm64 build box has 96 cores, and easily gets overloaded with multiple builds running simultaneously. Restrict each container to fewer cores, and decrease the number of workers. In order to not lose some of the coverage we got from packet as a high-CPU builder, we will need to create new builder types going forward to ensure we don't lose coverage. See the issue for details. Updates golang/go#36170 Change-Id: I16478fa05258bc14ac59a1e708133ffcc4c5a664 Reviewed-on: https://go-review.googlesource.com/c/build/+/211579 Run-TryBot: Alexander Rakoczy <alex@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-12-17 01:14:27 +03:00
ExpectNum: 8,
cmd/coordinator, cmd/buildlet, cmd/gomote: add SSH support This adds an SSH server to farmer.golang.org on port 2222 that proxies SSH connections to users' gomote-created buildlet instances. For example: $ gomote create openbsd-amd64-60 user-bradfitz-openbsd-amd64-60-1 $ gomote ssh user-bradfitz-openbsd-amd64-60-1 Warning: Permanently added '[localhost]:33351' (ECDSA) to the list of known hosts. OpenBSD 6.0 (GENERIC.MP) golang/go#2319: Tue Jul 26 13:00:43 MDT 2016 Welcome to OpenBSD: The proactively secure Unix-like operating system. Please use the sendbug(1) utility to report bugs in the system. Before reporting a bug, please try to reproduce it with the latest version of the code. With bug reports, please try to ensure that enough information to reproduce the problem is enclosed, and if a known fix for it exists, include that as well. $ As before, if the coordinator process is restarted (or crashes, is evicted, etc), all gomote instances die. Not yet supported: * scp (help wanted) * not all host types are configured. most are. some will need slight config tweaks to the Docker image (e.g. adding openssh-server) Supports currently: * linux-amd64 (host type shared by 386, nacl) * linux-arm * linux-arm64 * darwin * freebsd * openbsd * plan9-386 * windows Implementation details: * the ssh server process listens on port 2222 in the coordinator (farmer.golang.org), which is behind a GKE TCP load balancer. * the ssh server library is github.com/gliderlabs/ssh * authentication is done via Github users' public keys. It's assumed that gomote user == github user. But there's a mapping in the code for known exceptions. * we can't give out access to this too widely. too many things are accessible from within the host environment if you look in the right places. Details omitted. But the Go team and other trusted gomote users can use this. * the buildlet binary has a new /connect-ssh handler that acts like a CONNECT request but instead of taking an explicit host:port, just says "give me your machine's SSH connection". The buildlet can also start sshd if needed for the environment. The /connect-ssh handler also installs the coordinator's public key. * a new buildlet client library method "ConnectSSH" hits the /connect-ssh handler and returns a net.Conn. * the coordinator's ssh.Handler is just running the OpenSSH ssh client. * because the OpenSSH ssh child process can't connect to a net.Conn, an emphemeral localhost port is created on the coordinator to proxy between the ssh client and the net.Conn returned by ConnectSSH. * The /connect-ssh handler requires http.Hijacker, which requires fully compliant net.Conn implementations as of Go 1.8. So I needed to flesh out revdial too, testing it with the golang.org/x/net/nettest package. * plan9 doesn't have an ssh server, so we use 0intro's new conterm program (drawterm without GUI support) to connect to plan9 from the coordinator ssh proxy instead of using the OpenSSH ssh client binary. * windows doesn't have an ssh server, so we enable the telnet service and the coordinator ssh proxy uses telnet instead on the backend on the private network. (There is a Windows ssh server but only in new versions.) Happy debugging over ssh! Fixes golang/go#19956 Change-Id: I80a62064c5f85af1f195f980c862ba29af4015f0 Reviewed-on: https://go-review.googlesource.com/50750 Reviewed-by: Herbie Ong <herbie@google.com> Reviewed-by: Jessie Frazelle <me@jessfraz.com> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2017-07-22 22:15:56 +03:00
env: []string{"GOROOT_BOOTSTRAP=/usr/local/go-bootstrap"},
SSHUsername: "root",
build: update NetBSD from 7.0 to 7.1, disable builds, add arm64 packet builder Change-Id: I73b8d6e6ccaeca3b932238448ddcceead34105b4 Reviewed-on: https://go-review.googlesource.com/39851 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2017-04-07 01:16:28 +03:00
},
buildlet: add an AWS buildlet client This change adds an AWS buildlet client which allows us to create EC2 instances on AWS. With this change we have also moved a portion of the gce creation logic into a helper function which allows multiple clients to use it. Metadata for the instances are stored in the user data fields. The creation of a buildlet pool and modifications to rundocker buildlet be made in order to enable this change. Updates golang/go#36841 Change-Id: Ice03e1520513d51a02b9d66542e00012453bf0d9 Reviewed-on: https://go-review.googlesource.com/c/build/+/232077 Run-TryBot: Carlos Amedee <carlos@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-05-04 20:13:17 +03:00
"host-linux-arm64-aws": &HostConfig{
dashboard, buildenv: label EC2 instances This change adds a boolean which identified when an instance is an EC2 instance. Various EC2 related variables have been renamed to be more consistent. Updates golang/go#36841 Change-Id: Ief385bf9d41e320ebd7d31bc0108e8053f337afd Reviewed-on: https://go-review.googlesource.com/c/build/+/233801 Run-TryBot: Carlos Amedee <carlos@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-05-13 20:27:15 +03:00
Notes: "Debian Buster, EC2 arm64 instance. See x/build/env/linux-arm64/aws",
dashboard, env/linux-arm64/aws: update EC2 ami search query This change updates which source ami is used as a base for the creation of the linux-arm64-aws builder. Packer will now search for the latest version of the arm64 Debian image to use as the source ami. For golang/go#36841 Change-Id: I036dbaf7ee65d4e7028cd6a0cdf6710bf898fdd2 Reviewed-on: https://go-review.googlesource.com/c/build/+/266697 Trust: Carlos Amedee <carlos@golang.org> Run-TryBot: Carlos Amedee <carlos@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2020-10-29 00:23:17 +03:00
VMImage: "ami-03089323a1d38e652",
env/linux-arm64/aws, dashboard: update linux-arm64 builder image This updates the instruction for creating a VM image and a buildlet container image. The instructions will provide a better description for the steps required to create the images. The VM image has been updated to no longer require sudo when the docker command is executed. The host configuration has been updated with a new VM type and VM image. The rundockerbuildlet service configuration has been modified: * The memory limit has been removed because rundockerbuildlet will now use all of the available memory for buildlets running on AWS. * The service will report as running even after rundockerbuildlet has exited. Rundockerbuildlet will initiate a single buildlet and then exit. * The service will not be restarted after rundockerbuildlet has exited. Changes made to rundockerbuildlet which required these changes have been made in CL 234114. Updates golang/go#36841 Change-Id: I3ff51e414b11a2b1427cbd7c6e24e9135fc2a5c3 Reviewed-on: https://go-review.googlesource.com/c/build/+/236297 Run-TryBot: Carlos Amedee <carlos@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-05-20 00:26:14 +03:00
ContainerImage: "gobuilder-arm64-aws:latest",
machineType: "m6g.xlarge",
dashboard, buildenv: label EC2 instances This change adds a boolean which identified when an instance is an EC2 instance. Various EC2 related variables have been renamed to be more consistent. Updates golang/go#36841 Change-Id: Ief385bf9d41e320ebd7d31bc0108e8053f337afd Reviewed-on: https://go-review.googlesource.com/c/build/+/233801 Run-TryBot: Carlos Amedee <carlos@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-05-13 20:27:15 +03:00
isEC2: true,
buildlet: add an AWS buildlet client This change adds an AWS buildlet client which allows us to create EC2 instances on AWS. With this change we have also moved a portion of the gce creation logic into a helper function which allows multiple clients to use it. Metadata for the instances are stored in the user data fields. The creation of a buildlet pool and modifications to rundocker buildlet be made in order to enable this change. Updates golang/go#36841 Change-Id: Ice03e1520513d51a02b9d66542e00012453bf0d9 Reviewed-on: https://go-review.googlesource.com/c/build/+/232077 Run-TryBot: Carlos Amedee <carlos@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-05-04 20:13:17 +03:00
env: []string{"GOROOT_BOOTSTRAP=/usr/local/go-bootstrap"},
env/linux-arm/aws, dashboard: add linux-arm-aws builder This change adds the linux-arm-aws builder. This builder will be started on a EC2 instance which will spawn a single instance of the container. After the builder work is done, the instance will be destroyed. Updates golang/go#41867 Change-Id: I09d911c2f99d1dff8fe02feb458457e07e7d0cdd Reviewed-on: https://go-review.googlesource.com/c/build/+/260797 Trust: Carlos Amedee <carlos@golang.org> Run-TryBot: Carlos Amedee <carlos@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-10-08 23:49:48 +03:00
buildletURLTmpl: "http://storage.googleapis.com/$BUCKET/buildlet.linux-arm64",
SSHUsername: "root",
},
"host-linux-arm-aws": &HostConfig{
Notes: "Debian Buster, EC2 arm instance. See x/build/env/linux-arm/aws",
VMImage: "ami-07409163bccd5ac4d",
ContainerImage: "gobuilder-arm-aws:latest",
machineType: "m6g.xlarge",
isEC2: true,
env: []string{"GOROOT_BOOTSTRAP=/usr/local/go-bootstrap"},
buildletURLTmpl: "http://storage.googleapis.com/$BUCKET/buildlet.linux-arm",
buildlet: use TLS certificates for SSH connections This change adds the use of TLS certificates when connecting to a remote buildlet via port 443. This change only applies to the ConnectSSH, all other remote buildlet connections already use the TLS certificate. Fixes golang/go#41697 Change-Id: Ibc3e85edb562c42c9da2b1025f4b291b4a88deaa Reviewed-on: https://go-review.googlesource.com/c/build/+/258097 Trust: Carlos Amedee <carlos@golang.org> Run-TryBot: Carlos Amedee <carlos@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2020-09-28 22:42:57 +03:00
SSHUsername: "root",
buildlet: add an AWS buildlet client This change adds an AWS buildlet client which allows us to create EC2 instances on AWS. With this change we have also moved a portion of the gce creation logic into a helper function which allows multiple clients to use it. Metadata for the instances are stored in the user data fields. The creation of a buildlet pool and modifications to rundocker buildlet be made in order to enable this change. Updates golang/go#36841 Change-Id: Ice03e1520513d51a02b9d66542e00012453bf0d9 Reviewed-on: https://go-review.googlesource.com/c/build/+/232077 Run-TryBot: Carlos Amedee <carlos@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-05-04 20:13:17 +03:00
},
dashboard: add interim illumos builder While the work to make illumos a first class GCE guest is completed, use this interim zone provided by an illumos community member to run illumos builds. Updates golang/go#15581 Change-Id: I1784847e5407894d01ce0aadf489b38d7e5c1924 Reviewed-on: https://go-review.googlesource.com/c/build/+/201597 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-10-17 11:48:13 +03:00
"host-illumos-amd64-jclulow": &HostConfig{
Notes: "SmartOS base64@19.1.0 zone",
dashboard, cmd/coordinator: support multiple builder owners Prefer displaying GitHub usernames for convenience of pinging in GitHub issues, since all current builder owners have them. Keep it possible to use an email or something else if no GitHub account exists. Update the owner of the Corellium builders to be individual GitHub account, so that it can be pinged in an issue. The GitHub organization is already mentioned in the builder notes. Also switch to html/template (instead of text/template) for rendering the builders page. Fixes golang/go#49596. Change-Id: I89a96eb8f7a3057bc1f3aaee16abc9c776c45ee7 Reviewed-on: https://go-review.googlesource.com/c/build/+/363983 Trust: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-11-16 20:42:29 +03:00
Owners: []*gophers.Person{gh("jclulow")},
dashboard: add interim illumos builder While the work to make illumos a first class GCE guest is completed, use this interim zone provided by an illumos community member to run illumos builds. Updates golang/go#15581 Change-Id: I1784847e5407894d01ce0aadf489b38d7e5c1924 Reviewed-on: https://go-review.googlesource.com/c/build/+/201597 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-10-17 11:48:13 +03:00
IsReverse: true,
ExpectNum: 1,
SSHUsername: "gobuild",
},
x/build: add Solaris amd64 release builder to dashboard Fixes golang/go#20949 Change-Id: I4f85ee76783a1240a2ebd4a2a1f167ab849fcebe Reviewed-on: https://go-review.googlesource.com/47811 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2017-07-07 23:41:20 +03:00
"host-solaris-oracle-amd64-oraclerel": &HostConfig{
dashboard, cmd/coordinator: support multiple builder owners Prefer displaying GitHub usernames for convenience of pinging in GitHub issues, since all current builder owners have them. Keep it possible to use an email or something else if no GitHub account exists. Update the owner of the Corellium builders to be individual GitHub account, so that it can be pinged in an issue. The GitHub organization is already mentioned in the builder notes. Also switch to html/template (instead of text/template) for rendering the builders page. Fixes golang/go#49596. Change-Id: I89a96eb8f7a3057bc1f3aaee16abc9c776c45ee7 Reviewed-on: https://go-review.googlesource.com/c/build/+/363983 Trust: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-11-16 20:42:29 +03:00
Notes: "Oracle Solaris amd64 Release System",
Owners: []*gophers.Person{gh("rorth")}, // https://github.com/golang/go/issues/15581#issuecomment-550368581
IsReverse: true,
ExpectNum: 1,
env: []string{"GOROOT_BOOTSTRAP=/opt/golang/go-solaris-amd64-bootstrap"},
dashboard: add Oracle Solaris development builder Fixes golang/go#15072 Change-Id: I85677636a0f7f8b6a7b2405bb0669c39cc1da7f0 Reviewed-on: https://go-review.googlesource.com/46831 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2017-06-27 20:11:15 +03:00
},
dashboard: add linux-loong64-3a5000 builder Updates golang/go#46229 Change-Id: Ib48e1e64d96a07d5c5282d78935124faf75c89d7 Reviewed-on: https://go-review.googlesource.com/c/build/+/347309 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org> Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Trust: Cherry Mui <cherryyz@google.com>
2021-09-02 10:51:28 +03:00
"host-linux-loong64-3a5000": &HostConfig{
dashboard, cmd/coordinator: support multiple builder owners Prefer displaying GitHub usernames for convenience of pinging in GitHub issues, since all current builder owners have them. Keep it possible to use an email or something else if no GitHub account exists. Update the owner of the Corellium builders to be individual GitHub account, so that it can be pinged in an issue. The GitHub organization is already mentioned in the builder notes. Also switch to html/template (instead of text/template) for rendering the builders page. Fixes golang/go#49596. Change-Id: I89a96eb8f7a3057bc1f3aaee16abc9c776c45ee7 Reviewed-on: https://go-review.googlesource.com/c/build/+/363983 Trust: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-11-16 20:42:29 +03:00
Notes: "Loongson 3A5000 Box hosted by Loongson; loong64 is the short name of LoongArch 64 bit version",
Owners: []*gophers.Person{gh("XiaodongLoong")},
IsReverse: true,
ExpectNum: 5,
dashboard: add linux-loong64-3a5000 builder Updates golang/go#46229 Change-Id: Ib48e1e64d96a07d5c5282d78935124faf75c89d7 Reviewed-on: https://go-review.googlesource.com/c/build/+/347309 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org> Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Trust: Cherry Mui <cherryyz@google.com>
2021-09-02 10:51:28 +03:00
env: []string{
"GOROOT_BOOTSTRAP=/usr/lib/go-1.15",
},
},
dashboard: add linux-mipsle-mengzhuo builder Updates golang/go#31217 Change-Id: I7bda4f3ddf7dc76bdd974726747621f69a7acfa0 Reviewed-on: https://go-review.googlesource.com/c/build/+/191577 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-09-10 02:21:11 +03:00
"host-linux-mipsle-mengzhuo": &HostConfig{
dashboard, cmd/coordinator: support multiple builder owners Prefer displaying GitHub usernames for convenience of pinging in GitHub issues, since all current builder owners have them. Keep it possible to use an email or something else if no GitHub account exists. Update the owner of the Corellium builders to be individual GitHub account, so that it can be pinged in an issue. The GitHub organization is already mentioned in the builder notes. Also switch to html/template (instead of text/template) for rendering the builders page. Fixes golang/go#49596. Change-Id: I89a96eb8f7a3057bc1f3aaee16abc9c776c45ee7 Reviewed-on: https://go-review.googlesource.com/c/build/+/363983 Trust: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-11-16 20:42:29 +03:00
Notes: "Loongson 3A Box hosted by Meng Zhuo; actually MIPS64le despite the name",
Owners: []*gophers.Person{gh("mengzhuo")},
IsReverse: true,
ExpectNum: 1,
dashboard: add linux-mipsle-mengzhuo builder Updates golang/go#31217 Change-Id: I7bda4f3ddf7dc76bdd974726747621f69a7acfa0 Reviewed-on: https://go-review.googlesource.com/c/build/+/191577 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-09-10 02:21:11 +03:00
env: []string{
"GOROOT_BOOTSTRAP=/usr/lib/golang",
"GOMIPS64=hardfloat",
},
},
dashboard: add MIPS builders at RT-RK Updates golang/go#31217 Change-Id: I3c947fa1dcd0f1d938ffa4dfe565a35b2ccc3533 Reviewed-on: https://go-review.googlesource.com/c/build/+/204043 Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-10-29 23:35:07 +03:00
"host-linux-mips64le-rtrk": &HostConfig{
dashboard, cmd/coordinator: support multiple builder owners Prefer displaying GitHub usernames for convenience of pinging in GitHub issues, since all current builder owners have them. Keep it possible to use an email or something else if no GitHub account exists. Update the owner of the Corellium builders to be individual GitHub account, so that it can be pinged in an issue. The GitHub organization is already mentioned in the builder notes. Also switch to html/template (instead of text/template) for rendering the builders page. Fixes golang/go#49596. Change-Id: I89a96eb8f7a3057bc1f3aaee16abc9c776c45ee7 Reviewed-on: https://go-review.googlesource.com/c/build/+/363983 Trust: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-11-16 20:42:29 +03:00
Notes: "cavium,rhino_utm8 board hosted at RT-RK.com; quad-core cpu, 8GB of ram and 240GB ssd disks.",
Owners: []*gophers.Person{gh("bogojevic"), gh("milanknezevic")}, // See https://github.com/golang/go/issues/31217#issuecomment-547004892.
IsReverse: true,
ExpectNum: 1,
dashboard: add MIPS builders at RT-RK Updates golang/go#31217 Change-Id: I3c947fa1dcd0f1d938ffa4dfe565a35b2ccc3533 Reviewed-on: https://go-review.googlesource.com/c/build/+/204043 Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-10-29 23:35:07 +03:00
env: []string{
"GOROOT_BOOTSTRAP=/usr/local/go-bootstrap",
},
},
"host-linux-mips64-rtrk": &HostConfig{
dashboard, cmd/coordinator: support multiple builder owners Prefer displaying GitHub usernames for convenience of pinging in GitHub issues, since all current builder owners have them. Keep it possible to use an email or something else if no GitHub account exists. Update the owner of the Corellium builders to be individual GitHub account, so that it can be pinged in an issue. The GitHub organization is already mentioned in the builder notes. Also switch to html/template (instead of text/template) for rendering the builders page. Fixes golang/go#49596. Change-Id: I89a96eb8f7a3057bc1f3aaee16abc9c776c45ee7 Reviewed-on: https://go-review.googlesource.com/c/build/+/363983 Trust: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-11-16 20:42:29 +03:00
Notes: "cavium,rhino_utm8 board hosted at RT-RK.com; quad-core cpu, 8GB of ram and 240GB ssd disks.",
Owners: []*gophers.Person{gh("bogojevic"), gh("milanknezevic")}, // See https://github.com/golang/go/issues/31217#issuecomment-547004892.
IsReverse: true,
ExpectNum: 1,
dashboard: add MIPS builders at RT-RK Updates golang/go#31217 Change-Id: I3c947fa1dcd0f1d938ffa4dfe565a35b2ccc3533 Reviewed-on: https://go-review.googlesource.com/c/build/+/204043 Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-10-29 23:35:07 +03:00
env: []string{
"GOROOT_BOOTSTRAP=/usr/local/go-bootstrap",
},
},
dashboard,env: move iOS builders to GOOS=ios Updates golang/go#41610 Change-Id: Ie8cb3294da5566e09fc06c01e679b8b773a439cf Reviewed-on: https://go-review.googlesource.com/c/build/+/259358 Run-TryBot: Elias Naur <mail@eliasnaur.com> TryBot-Result: Go Bot <gobot@golang.org> Trust: Elias Naur <mail@eliasnaur.com> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2020-10-03 12:37:07 +03:00
"host-ios-arm64-corellium-ios": &HostConfig{
dashboard, cmd/coordinator: support multiple builder owners Prefer displaying GitHub usernames for convenience of pinging in GitHub issues, since all current builder owners have them. Keep it possible to use an email or something else if no GitHub account exists. Update the owner of the Corellium builders to be individual GitHub account, so that it can be pinged in an issue. The GitHub organization is already mentioned in the builder notes. Also switch to html/template (instead of text/template) for rendering the builders page. Fixes golang/go#49596. Change-Id: I89a96eb8f7a3057bc1f3aaee16abc9c776c45ee7 Reviewed-on: https://go-review.googlesource.com/c/build/+/363983 Trust: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-11-16 20:42:29 +03:00
Notes: "Virtual iOS devices hosted by Zenly on Corellium; see issues 31722 and 40523",
dashboard: remove known issue for iOS and Android builders The old known issue has been resolved: the builders have been restarted and are back. That uncovered what appears to be a recent regression, reported as golang/go#49616. Also add a new builder owner based on conversation at https://groups.google.com/g/golang-dev/c/oiuIE7qrWp0. Updates golang/go#48772. Updates golang/go#49048. Updates golang/go#49616. Change-Id: I7a6a89b7fb088373a70bd3496ad2091ec7a3d79f Reviewed-on: https://go-review.googlesource.com/c/build/+/363985 Trust: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org>
2021-11-16 20:42:29 +03:00
Owners: []*gophers.Person{gh("steeve"), gh("changkun")}, // See https://groups.google.com/g/golang-dev/c/oiuIE7qrWp0.
dashboard, cmd/coordinator: support multiple builder owners Prefer displaying GitHub usernames for convenience of pinging in GitHub issues, since all current builder owners have them. Keep it possible to use an email or something else if no GitHub account exists. Update the owner of the Corellium builders to be individual GitHub account, so that it can be pinged in an issue. The GitHub organization is already mentioned in the builder notes. Also switch to html/template (instead of text/template) for rendering the builders page. Fixes golang/go#49596. Change-Id: I89a96eb8f7a3057bc1f3aaee16abc9c776c45ee7 Reviewed-on: https://go-review.googlesource.com/c/build/+/363983 Trust: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-11-16 20:42:29 +03:00
IsReverse: true,
ExpectNum: 3,
dashboard: add Zenly's virtual iOS builders Fixes golang/go#31722 Change-Id: Ia7f0d3f2e1a25000c42a10ff58d7e91d384b1219 Reviewed-on: https://go-review.googlesource.com/c/build/+/174797 Run-TryBot: Elias Naur <mail@eliasnaur.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-05-01 21:33:26 +03:00
env: []string{
dashboard: update host-ios-arm64-corellium-ios to match its previous env In CL 353549 I refactored environment-handling in the buildlet such that an explicit per-host GOROOT_BOOTSTRAP environment configured in dashboard/builders.go overrides the latent GOROOT_BOOTSTRAP value on the host. (The entry in dashboard/builders.go may omit GOROOT_BOOTSTRAP entirely to defer to the host environment.) That change caught out a previously-unnoticed configuration mismatch for the host-ios-arm64-corellium-ios builder: its entry in dashboard/builders.go indicated "GOROOT_BOOTSTRAP=/var/mobile/go-ios-arm64-bootstrap", but (as shown in previous builder logs¹) it was actually using "GOROOT_BOOTSTRAP=/var/root/go-ios-arm64-bootstrap". The latter empirically exists on the builder, while the former does not. ¹https://build.golang.org/log/4da918256e30f3442be7885019dc2d17bc718e0d For golang/go#33598. Change-Id: I09e5f83b46f23d5a4744c752fb66fcd4b33e4cff Reviewed-on: https://go-review.googlesource.com/c/build/+/354752 Trust: Bryan C. Mills <bcmills@google.com> Run-TryBot: Bryan C. Mills <bcmills@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2021-10-08 20:49:33 +03:00
"GOROOT_BOOTSTRAP=/var/root/go-ios-arm64-bootstrap",
dashboard: add Zenly's virtual iOS builders Fixes golang/go#31722 Change-Id: Ia7f0d3f2e1a25000c42a10ff58d7e91d384b1219 Reviewed-on: https://go-review.googlesource.com/c/build/+/174797 Run-TryBot: Elias Naur <mail@eliasnaur.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-05-01 21:33:26 +03:00
},
},
dashboard: add Zenly's virtual Android builders Change-Id: Id0be99f343a819c19b0004dc4160175c9bb0ace9 Reviewed-on: https://go-review.googlesource.com/c/build/+/174798 Run-TryBot: Elias Naur <mail@eliasnaur.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-05-01 23:28:43 +03:00
"host-android-arm64-corellium-android": &HostConfig{
dashboard, cmd/coordinator: support multiple builder owners Prefer displaying GitHub usernames for convenience of pinging in GitHub issues, since all current builder owners have them. Keep it possible to use an email or something else if no GitHub account exists. Update the owner of the Corellium builders to be individual GitHub account, so that it can be pinged in an issue. The GitHub organization is already mentioned in the builder notes. Also switch to html/template (instead of text/template) for rendering the builders page. Fixes golang/go#49596. Change-Id: I89a96eb8f7a3057bc1f3aaee16abc9c776c45ee7 Reviewed-on: https://go-review.googlesource.com/c/build/+/363983 Trust: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-11-16 20:42:29 +03:00
Notes: "Virtual Android devices hosted by Zenly on Corellium; see issues 31722 and 40523",
dashboard: remove known issue for iOS and Android builders The old known issue has been resolved: the builders have been restarted and are back. That uncovered what appears to be a recent regression, reported as golang/go#49616. Also add a new builder owner based on conversation at https://groups.google.com/g/golang-dev/c/oiuIE7qrWp0. Updates golang/go#48772. Updates golang/go#49048. Updates golang/go#49616. Change-Id: I7a6a89b7fb088373a70bd3496ad2091ec7a3d79f Reviewed-on: https://go-review.googlesource.com/c/build/+/363985 Trust: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org>
2021-11-16 20:42:29 +03:00
Owners: []*gophers.Person{gh("steeve"), gh("changkun")}, // See https://groups.google.com/g/golang-dev/c/oiuIE7qrWp0.
dashboard, cmd/coordinator: support multiple builder owners Prefer displaying GitHub usernames for convenience of pinging in GitHub issues, since all current builder owners have them. Keep it possible to use an email or something else if no GitHub account exists. Update the owner of the Corellium builders to be individual GitHub account, so that it can be pinged in an issue. The GitHub organization is already mentioned in the builder notes. Also switch to html/template (instead of text/template) for rendering the builders page. Fixes golang/go#49596. Change-Id: I89a96eb8f7a3057bc1f3aaee16abc9c776c45ee7 Reviewed-on: https://go-review.googlesource.com/c/build/+/363983 Trust: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-11-16 20:42:29 +03:00
IsReverse: true,
ExpectNum: 3,
dashboard: add Zenly's virtual Android builders Change-Id: Id0be99f343a819c19b0004dc4160175c9bb0ace9 Reviewed-on: https://go-review.googlesource.com/c/build/+/174798 Run-TryBot: Elias Naur <mail@eliasnaur.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-05-01 23:28:43 +03:00
env: []string{
"GOROOT_BOOTSTRAP=/data/data/com.termux/files/home/go-android-arm64-bootstrap",
},
},
dashboard: add aix-ppc64 builder This is a starting point for adding an aix-ppc64 reverse builder. There can be further adjustments made to it as needed. Updates golang/go#27160. Change-Id: I8a61adbe21f1a08265df9a206d94bb04e880a508 Reviewed-on: https://go-review.googlesource.com/131335 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-08-24 21:41:05 +03:00
"host-aix-ppc64-osuosl": &HostConfig{
dashboard, cmd/coordinator: support multiple builder owners Prefer displaying GitHub usernames for convenience of pinging in GitHub issues, since all current builder owners have them. Keep it possible to use an email or something else if no GitHub account exists. Update the owner of the Corellium builders to be individual GitHub account, so that it can be pinged in an issue. The GitHub organization is already mentioned in the builder notes. Also switch to html/template (instead of text/template) for rendering the builders page. Fixes golang/go#49596. Change-Id: I89a96eb8f7a3057bc1f3aaee16abc9c776c45ee7 Reviewed-on: https://go-review.googlesource.com/c/build/+/363983 Trust: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-11-16 20:42:29 +03:00
Notes: "AIX 7.2 VM on OSU; run by Tony Reix",
Owners: []*gophers.Person{gh("trex58")},
IsReverse: true,
ExpectNum: 1,
env: []string{"GOROOT_BOOTSTRAP=/opt/freeware/lib/golang"},
dashboard: add aix-ppc64 builder This is a starting point for adding an aix-ppc64 reverse builder. There can be further adjustments made to it as needed. Updates golang/go#27160. Change-Id: I8a61adbe21f1a08265df9a206d94bb04e880a508 Reviewed-on: https://go-review.googlesource.com/131335 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-08-24 21:41:05 +03:00
},
env/android-amd64-emu: add new Android emulator container + VM image Now that we can do nested virtualization on GCE, that means we can run the Android emulator (which requires KVM) on GCE and at least get fast trybots and such for android-amd64. Updates golang/go#23824 Change-Id: I0da38c7fa0f15492230a31291d2921ba72f2151d Reviewed-on: https://go-review.googlesource.com/c/163738 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-02-25 23:35:29 +03:00
"host-android-amd64-emu": &HostConfig{
Notes: "Debian Buster w/ Android SDK + emulator (use nested virt)",
env/android-amd64-emu: enable gomobile device tests By adding gradle and Android build tools to PATH, setting ANDROID_HOME, and adding the OpenAL headers to CGO_CFLAGS. Change-Id: Ifeaa1b82f4108afc3787eee4449b3317185bbaec Reviewed-on: https://go-review.googlesource.com/c/build/+/164658 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-03-01 13:21:20 +03:00
ContainerImage: "android-amd64-emu:bff27c0c9263",
env/android-amd64-emu: add new Android emulator container + VM image Now that we can do nested virtualization on GCE, that means we can run the Android emulator (which requires KVM) on GCE and at least get fast trybots and such for android-amd64. Updates golang/go#23824 Change-Id: I0da38c7fa0f15492230a31291d2921ba72f2151d Reviewed-on: https://go-review.googlesource.com/c/163738 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-02-25 23:35:29 +03:00
KonletVMImage: "android-amd64-emu",
NestedVirt: true,
buildletURLTmpl: "http://storage.googleapis.com/$BUCKET/buildlet.linux-amd64",
env: []string{"GOROOT_BOOTSTRAP=/go1.4"},
SSHUsername: "root",
},
dashboard: add new linux-amd64-wsl builder Updates golang/go#17365 Change-Id: I90b9c1c8f82ae9164deacea36ff5dafd008eef03 Reviewed-on: https://go-review.googlesource.com/c/build/+/272266 Trust: Meng Zhuo <mzh@golangcn.org> Run-TryBot: Meng Zhuo <mzh@golangcn.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
2020-11-24 19:44:38 +03:00
"host-linux-amd64-wsl": &HostConfig{
dashboard, cmd/coordinator: support multiple builder owners Prefer displaying GitHub usernames for convenience of pinging in GitHub issues, since all current builder owners have them. Keep it possible to use an email or something else if no GitHub account exists. Update the owner of the Corellium builders to be individual GitHub account, so that it can be pinged in an issue. The GitHub organization is already mentioned in the builder notes. Also switch to html/template (instead of text/template) for rendering the builders page. Fixes golang/go#49596. Change-Id: I89a96eb8f7a3057bc1f3aaee16abc9c776c45ee7 Reviewed-on: https://go-review.googlesource.com/c/build/+/363983 Trust: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-11-16 20:42:29 +03:00
Notes: "Windows 10 WSL2 Ubuntu",
Owners: []*gophers.Person{gh("mengzhuo")},
IsReverse: true,
ExpectNum: 2,
env: []string{"GOROOT_BOOTSTRAP=/usr/lib/go"},
dashboard: add new linux-amd64-wsl builder Updates golang/go#17365 Change-Id: I90b9c1c8f82ae9164deacea36ff5dafd008eef03 Reviewed-on: https://go-review.googlesource.com/c/build/+/272266 Trust: Meng Zhuo <mzh@golangcn.org> Run-TryBot: Meng Zhuo <mzh@golangcn.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
2020-11-24 19:44:38 +03:00
},
dashboard: add linux-amd64-perf host and builder Add a new builder to run the x/benchmarks performance tests on linux-amd64. For now, this runs on a GCE C2 instance type, as these instances have well-defined, consistent CPUs and other server architecture components. In basic noise testing, even standard VMs of this type appear to be fairly low noise. As we gain experience with actual monitoring, we may change this to a sole-tenant VM type or even a dedicated machine if necessary. For golang/go#49207 Change-Id: I17eaeeb5349af925249940bebd5b860a2579e6df Reviewed-on: https://go-review.googlesource.com/c/build/+/354311 Trust: Michael Pratt <mpratt@google.com> Run-TryBot: Michael Pratt <mpratt@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org>
2021-10-01 21:59:33 +03:00
"host-linux-amd64-perf": &HostConfig{
Notes: "Cascade Lake performance testing machines",
dashboard: SkipSnapshot for linux-amd64-perf Since this builder doesn't build the go repo, it will be waiting forever for a snapshot. Instead, just build Go for each run. For golang/go#49207 Change-Id: I34a73b507278db402c478b4f5956633996772aae Reviewed-on: https://go-review.googlesource.com/c/build/+/361754 Trust: Alexander Rakoczy <alex@golang.org> Run-TryBot: Alexander Rakoczy <alex@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2021-11-05 22:01:27 +03:00
machineType: "c2-standard-8", // C2 has precisely defined, consistent server architecture.
dashboard: add linux-amd64-perf host and builder Add a new builder to run the x/benchmarks performance tests on linux-amd64. For now, this runs on a GCE C2 instance type, as these instances have well-defined, consistent CPUs and other server architecture components. In basic noise testing, even standard VMs of this type appear to be fairly low noise. As we gain experience with actual monitoring, we may change this to a sole-tenant VM type or even a dedicated machine if necessary. For golang/go#49207 Change-Id: I17eaeeb5349af925249940bebd5b860a2579e6df Reviewed-on: https://go-review.googlesource.com/c/build/+/354311 Trust: Michael Pratt <mpratt@google.com> Run-TryBot: Michael Pratt <mpratt@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org>
2021-10-01 21:59:33 +03:00
ContainerImage: "linux-x86-bullseye:latest",
buildletURLTmpl: "https://storage.googleapis.com/$BUCKET/buildlet.linux-amd64",
env: []string{"GOROOT_BOOTSTRAP=/go1.4"},
SSHUsername: "root",
},
all: split builder config into builder & host configs Our builders are named of the form "GOOS-GOARCH" or "GOOS-GOARCH-suffix". Over time we've grown many builders. This CL doesn't change that. Builders continue to be named and operate as before. Previously the build configuration file (dashboard/builders.go) made each builder type ("linux-amd64-race", etc) define how to create a host running a buildlet of that type, even though many builders had identical host configs. For example, these builders all share the same host type (a Kubernetes container): linux-amd64 linux-amd64-race linux-386 linux-386-387 And these are the same host type (a GCE VM): windows-amd64-gce windows-amd64-race windows-386-gce This CL creates a new concept of a "hostType" which defines how the buildlet is created (Kube, GCE, Reverse, and how), and then each builder itself references a host type. Users never see the hostType. (except perhaps in gomote list output) But they at least never need to care about them. Reverse buildlets now can only be one hostType at a time, which simplifies things. We were no longer using multiple roles per machine once moving to VMs for OS X. gomote continues to operate as it did previously but its underlying protocol changed and clients will need to be updated. As a new feature, gomote now has a new flag to let you reuse a buildlet host connection for different builder rules if they share the same underlying host type. But users can ignore that. This CL is a long-standing TODO (previously attempted and aborted) and will make many things easier and faster, including the linux-arm cross-compilation effort, and keeping pre-warmed buildlets of VM types ready to go. Updates golang/go#17104 Change-Id: Iad8387f48680424a8441e878a2f4762bf79ea4d2 Reviewed-on: https://go-review.googlesource.com/29551 Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2016-09-22 00:27:37 +03:00
}
dashboard, cmd/coordinator: move cross compilation config into dashboard package The dashboard package has been gradually absorbing all configuration, keeping cmd/coordinator only about mechanics. But we'd missed this cross compilation bit in earlier cleanups. Move it, and add some tests & docs while I'm at it. Also, disable the linux-arm-nativemake builder by default. It's redundant with linux-arm. They used to be more different when we always cross compiled for linux-arm, but I guess that got disabled at some point. At this poin it serves no purpose except as an opt-in slowbot if the user really wants to run make.bash on scaleway. Change-Id: Iac683ccf4d30214330480f9db39f131263b1bf8a Reviewed-on: https://go-review.googlesource.com/c/build/+/202020 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-10-19 05:49:53 +03:00
// CrossCompileConfig describes how to cross-compile a build on a
// faster host.
type CrossCompileConfig struct {
// CompileHostType is the host type to use for compilation
CompileHostType string
// CCForTarget is the CC_FOR_TARGET environment variable.
CCForTarget string
// GOARM is any GOARM= environment variable.
GOARM string
// AlwaysCrossCompile controls whether this builder always
// cross compiles. Otherwise it's only done for trybot runs.
AlwaysCrossCompile bool
}
dashboard, cmd/coordinator: support multiple builder owners Prefer displaying GitHub usernames for convenience of pinging in GitHub issues, since all current builder owners have them. Keep it possible to use an email or something else if no GitHub account exists. Update the owner of the Corellium builders to be individual GitHub account, so that it can be pinged in an issue. The GitHub organization is already mentioned in the builder notes. Also switch to html/template (instead of text/template) for rendering the builders page. Fixes golang/go#49596. Change-Id: I89a96eb8f7a3057bc1f3aaee16abc9c776c45ee7 Reviewed-on: https://go-review.googlesource.com/c/build/+/363983 Trust: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-11-16 20:42:29 +03:00
func gh(githubUsername string) *gophers.Person {
p := gophers.GetPerson("@" + githubUsername)
if p == nil {
panic("person with GitHub username " + githubUsername + " does not exist in the golang.org/x/build/internal/gophers package")
}
return p
}
all: split builder config into builder & host configs Our builders are named of the form "GOOS-GOARCH" or "GOOS-GOARCH-suffix". Over time we've grown many builders. This CL doesn't change that. Builders continue to be named and operate as before. Previously the build configuration file (dashboard/builders.go) made each builder type ("linux-amd64-race", etc) define how to create a host running a buildlet of that type, even though many builders had identical host configs. For example, these builders all share the same host type (a Kubernetes container): linux-amd64 linux-amd64-race linux-386 linux-386-387 And these are the same host type (a GCE VM): windows-amd64-gce windows-amd64-race windows-386-gce This CL creates a new concept of a "hostType" which defines how the buildlet is created (Kube, GCE, Reverse, and how), and then each builder itself references a host type. Users never see the hostType. (except perhaps in gomote list output) But they at least never need to care about them. Reverse buildlets now can only be one hostType at a time, which simplifies things. We were no longer using multiple roles per machine once moving to VMs for OS X. gomote continues to operate as it did previously but its underlying protocol changed and clients will need to be updated. As a new feature, gomote now has a new flag to let you reuse a buildlet host connection for different builder rules if they share the same underlying host type. But users can ignore that. This CL is a long-standing TODO (previously attempted and aborted) and will make many things easier and faster, including the linux-arm cross-compilation effort, and keeping pre-warmed buildlets of VM types ready to go. Updates golang/go#17104 Change-Id: Iad8387f48680424a8441e878a2f4762bf79ea4d2 Reviewed-on: https://go-review.googlesource.com/29551 Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2016-09-22 00:27:37 +03:00
func init() {
for key, c := range Hosts {
if key == "" {
panic("empty string key in Hosts")
}
if c.HostType == "" {
c.HostType = key
}
if c.HostType != key {
panic(fmt.Sprintf("HostType %q != key %q", c.HostType, key))
}
nSet := 0
dashboard: add windows-arm64 builder Add the windows-arm64 builder to the dashboard and coordinator with a known issue until we can verify builds are succeeding regularly. Also, tweak the logic around validating ec2 configurations. The arm64 buildlet sets VMImage but not ContainerImage, unlike other ec2 buildlets. This should work for both. For golang/go#42604 Change-Id: I7176b8f6bba837830a0565e7713c9e9a0eb76e90 Reviewed-on: https://go-review.googlesource.com/c/build/+/322656 Trust: Alexander Rakoczy <alex@golang.org> Run-TryBot: Alexander Rakoczy <alex@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-05-26 04:47:41 +03:00
if c.VMImage != "" {
all: split builder config into builder & host configs Our builders are named of the form "GOOS-GOARCH" or "GOOS-GOARCH-suffix". Over time we've grown many builders. This CL doesn't change that. Builders continue to be named and operate as before. Previously the build configuration file (dashboard/builders.go) made each builder type ("linux-amd64-race", etc) define how to create a host running a buildlet of that type, even though many builders had identical host configs. For example, these builders all share the same host type (a Kubernetes container): linux-amd64 linux-amd64-race linux-386 linux-386-387 And these are the same host type (a GCE VM): windows-amd64-gce windows-amd64-race windows-386-gce This CL creates a new concept of a "hostType" which defines how the buildlet is created (Kube, GCE, Reverse, and how), and then each builder itself references a host type. Users never see the hostType. (except perhaps in gomote list output) But they at least never need to care about them. Reverse buildlets now can only be one hostType at a time, which simplifies things. We were no longer using multiple roles per machine once moving to VMs for OS X. gomote continues to operate as it did previously but its underlying protocol changed and clients will need to be updated. As a new feature, gomote now has a new flag to let you reuse a buildlet host connection for different builder rules if they share the same underlying host type. But users can ignore that. This CL is a long-standing TODO (previously attempted and aborted) and will make many things easier and faster, including the linux-arm cross-compilation effort, and keeping pre-warmed buildlets of VM types ready to go. Updates golang/go#17104 Change-Id: Iad8387f48680424a8441e878a2f4762bf79ea4d2 Reviewed-on: https://go-review.googlesource.com/29551 Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2016-09-22 00:27:37 +03:00
nSet++
}
dashboard: add windows-arm64 builder Add the windows-arm64 builder to the dashboard and coordinator with a known issue until we can verify builds are succeeding regularly. Also, tweak the logic around validating ec2 configurations. The arm64 buildlet sets VMImage but not ContainerImage, unlike other ec2 buildlets. This should work for both. For golang/go#42604 Change-Id: I7176b8f6bba837830a0565e7713c9e9a0eb76e90 Reviewed-on: https://go-review.googlesource.com/c/build/+/322656 Trust: Alexander Rakoczy <alex@golang.org> Run-TryBot: Alexander Rakoczy <alex@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-05-26 04:47:41 +03:00
if c.ContainerImage != "" && !c.isEC2 {
all: split builder config into builder & host configs Our builders are named of the form "GOOS-GOARCH" or "GOOS-GOARCH-suffix". Over time we've grown many builders. This CL doesn't change that. Builders continue to be named and operate as before. Previously the build configuration file (dashboard/builders.go) made each builder type ("linux-amd64-race", etc) define how to create a host running a buildlet of that type, even though many builders had identical host configs. For example, these builders all share the same host type (a Kubernetes container): linux-amd64 linux-amd64-race linux-386 linux-386-387 And these are the same host type (a GCE VM): windows-amd64-gce windows-amd64-race windows-386-gce This CL creates a new concept of a "hostType" which defines how the buildlet is created (Kube, GCE, Reverse, and how), and then each builder itself references a host type. Users never see the hostType. (except perhaps in gomote list output) But they at least never need to care about them. Reverse buildlets now can only be one hostType at a time, which simplifies things. We were no longer using multiple roles per machine once moving to VMs for OS X. gomote continues to operate as it did previously but its underlying protocol changed and clients will need to be updated. As a new feature, gomote now has a new flag to let you reuse a buildlet host connection for different builder rules if they share the same underlying host type. But users can ignore that. This CL is a long-standing TODO (previously attempted and aborted) and will make many things easier and faster, including the linux-arm cross-compilation effort, and keeping pre-warmed buildlets of VM types ready to go. Updates golang/go#17104 Change-Id: Iad8387f48680424a8441e878a2f4762bf79ea4d2 Reviewed-on: https://go-review.googlesource.com/29551 Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2016-09-22 00:27:37 +03:00
nSet++
}
if c.IsReverse {
nSet++
}
if nSet != 1 {
all: rename Kube to Container and IsGCE to IsVM Once containers run on COS instead of Kubernetes, one name (Kube*) is wrong and the other (GCE) is ambiguous. So rename them now to be more specific. No behavior changes. Just renaming in this step, to reduce size of next CL. Updates golang/go#25108 Change-Id: Ib09eb682ef74acbbf6ed50b46074f834ef5e0c0b Reviewed-on: https://go-review.googlesource.com/111639 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-05-05 19:36:05 +03:00
panic(fmt.Sprintf("exactly one of VMImage, ContainerImage, IsReverse must be set for host %q; got %v", key, nSet))
all: split builder config into builder & host configs Our builders are named of the form "GOOS-GOARCH" or "GOOS-GOARCH-suffix". Over time we've grown many builders. This CL doesn't change that. Builders continue to be named and operate as before. Previously the build configuration file (dashboard/builders.go) made each builder type ("linux-amd64-race", etc) define how to create a host running a buildlet of that type, even though many builders had identical host configs. For example, these builders all share the same host type (a Kubernetes container): linux-amd64 linux-amd64-race linux-386 linux-386-387 And these are the same host type (a GCE VM): windows-amd64-gce windows-amd64-race windows-386-gce This CL creates a new concept of a "hostType" which defines how the buildlet is created (Kube, GCE, Reverse, and how), and then each builder itself references a host type. Users never see the hostType. (except perhaps in gomote list output) But they at least never need to care about them. Reverse buildlets now can only be one hostType at a time, which simplifies things. We were no longer using multiple roles per machine once moving to VMs for OS X. gomote continues to operate as it did previously but its underlying protocol changed and clients will need to be updated. As a new feature, gomote now has a new flag to let you reuse a buildlet host connection for different builder rules if they share the same underlying host type. But users can ignore that. This CL is a long-standing TODO (previously attempted and aborted) and will make many things easier and faster, including the linux-arm cross-compilation effort, and keeping pre-warmed buildlets of VM types ready to go. Updates golang/go#17104 Change-Id: Iad8387f48680424a8441e878a2f4762bf79ea4d2 Reviewed-on: https://go-review.googlesource.com/29551 Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2016-09-22 00:27:37 +03:00
}
all: rename Kube to Container and IsGCE to IsVM Once containers run on COS instead of Kubernetes, one name (Kube*) is wrong and the other (GCE) is ambiguous. So rename them now to be more specific. No behavior changes. Just renaming in this step, to reduce size of next CL. Updates golang/go#25108 Change-Id: Ib09eb682ef74acbbf6ed50b46074f834ef5e0c0b Reviewed-on: https://go-review.googlesource.com/111639 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-05-05 19:36:05 +03:00
if c.buildletURLTmpl == "" && (c.VMImage != "" || c.ContainerImage != "") {
all: split builder config into builder & host configs Our builders are named of the form "GOOS-GOARCH" or "GOOS-GOARCH-suffix". Over time we've grown many builders. This CL doesn't change that. Builders continue to be named and operate as before. Previously the build configuration file (dashboard/builders.go) made each builder type ("linux-amd64-race", etc) define how to create a host running a buildlet of that type, even though many builders had identical host configs. For example, these builders all share the same host type (a Kubernetes container): linux-amd64 linux-amd64-race linux-386 linux-386-387 And these are the same host type (a GCE VM): windows-amd64-gce windows-amd64-race windows-386-gce This CL creates a new concept of a "hostType" which defines how the buildlet is created (Kube, GCE, Reverse, and how), and then each builder itself references a host type. Users never see the hostType. (except perhaps in gomote list output) But they at least never need to care about them. Reverse buildlets now can only be one hostType at a time, which simplifies things. We were no longer using multiple roles per machine once moving to VMs for OS X. gomote continues to operate as it did previously but its underlying protocol changed and clients will need to be updated. As a new feature, gomote now has a new flag to let you reuse a buildlet host connection for different builder rules if they share the same underlying host type. But users can ignore that. This CL is a long-standing TODO (previously attempted and aborted) and will make many things easier and faster, including the linux-arm cross-compilation effort, and keeping pre-warmed buildlets of VM types ready to go. Updates golang/go#17104 Change-Id: Iad8387f48680424a8441e878a2f4762bf79ea4d2 Reviewed-on: https://go-review.googlesource.com/29551 Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2016-09-22 00:27:37 +03:00
panic(fmt.Sprintf("missing buildletURLTmpl for host type %q", key))
}
}
}
// A HostConfig describes the available ways to obtain buildlets of
dashboard, app/appengine: set a known issue for staticlockranking builder The linux-amd64-staticlockranking builder was added pre-emptively so that it could be tested and used during the development of a CL that implements a new feature. It's expected to fail until that work is completed. Mark it as a builder with a known issue, golang/go#38029. The new BuildConfig.KnownIssue field can be used by builders being added in the future so that if they fail at first, it doesn't take away time for people looking at build.golang.org, which we always aim to have as green as possible¹. ¹ https://groups.google.com/d/msg/golang-dev/y0yM_Xi665Q/hUpBiUPiCAAJ Fixes golang/go#38283 For golang/go#38029 For golang/go#37937 For golang/go#11811 Change-Id: Iba1606c7021ffa7e67ddbaae2fc6b65f4e46ab34 Reviewed-on: https://go-review.googlesource.com/c/build/+/227379 Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-04-07 00:34:17 +03:00
// different types. Some host configs can serve multiple
dashboard, env: add Debian Stretch builder, rename Jessie builder Updates golang/go#25386 Change-Id: I65e3dda63e5ea86fa94480e3305556b73afe121a Reviewed-on: https://go-review.googlesource.com/113059 Reviewed-by: Bryan C. Mills <bcmills@google.com>
2018-05-14 20:22:45 +03:00
// builders. For example, a host config of "host-linux-jessie" can
all: split builder config into builder & host configs Our builders are named of the form "GOOS-GOARCH" or "GOOS-GOARCH-suffix". Over time we've grown many builders. This CL doesn't change that. Builders continue to be named and operate as before. Previously the build configuration file (dashboard/builders.go) made each builder type ("linux-amd64-race", etc) define how to create a host running a buildlet of that type, even though many builders had identical host configs. For example, these builders all share the same host type (a Kubernetes container): linux-amd64 linux-amd64-race linux-386 linux-386-387 And these are the same host type (a GCE VM): windows-amd64-gce windows-amd64-race windows-386-gce This CL creates a new concept of a "hostType" which defines how the buildlet is created (Kube, GCE, Reverse, and how), and then each builder itself references a host type. Users never see the hostType. (except perhaps in gomote list output) But they at least never need to care about them. Reverse buildlets now can only be one hostType at a time, which simplifies things. We were no longer using multiple roles per machine once moving to VMs for OS X. gomote continues to operate as it did previously but its underlying protocol changed and clients will need to be updated. As a new feature, gomote now has a new flag to let you reuse a buildlet host connection for different builder rules if they share the same underlying host type. But users can ignore that. This CL is a long-standing TODO (previously attempted and aborted) and will make many things easier and faster, including the linux-arm cross-compilation effort, and keeping pre-warmed buildlets of VM types ready to go. Updates golang/go#17104 Change-Id: Iad8387f48680424a8441e878a2f4762bf79ea4d2 Reviewed-on: https://go-review.googlesource.com/29551 Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2016-09-22 00:27:37 +03:00
// serve linux-amd64, linux-amd64-race, linux-386, linux-386-387, etc.
type HostConfig struct {
// HostType is the unique name of this host config. It is also
// the key in the Hosts map.
HostType string
// buildletURLTmpl is the URL "template" ($BUCKET is auto-expanded)
// for the URL to the buildlet binary.
all: use Container-Optimized VMs instead of Kubernetes for buildlet containers Fixes golang/go#25108 Change-Id: I084669b52b699700ed26a7fdd890d9205a8b9dc9 Reviewed-on: https://go-review.googlesource.com/111267 Reviewed-by: Andrew Bonventre <andybons@golang.org>
2018-05-04 05:42:17 +03:00
// This field is required for VM and Container builders. It's not
all: split builder config into builder & host configs Our builders are named of the form "GOOS-GOARCH" or "GOOS-GOARCH-suffix". Over time we've grown many builders. This CL doesn't change that. Builders continue to be named and operate as before. Previously the build configuration file (dashboard/builders.go) made each builder type ("linux-amd64-race", etc) define how to create a host running a buildlet of that type, even though many builders had identical host configs. For example, these builders all share the same host type (a Kubernetes container): linux-amd64 linux-amd64-race linux-386 linux-386-387 And these are the same host type (a GCE VM): windows-amd64-gce windows-amd64-race windows-386-gce This CL creates a new concept of a "hostType" which defines how the buildlet is created (Kube, GCE, Reverse, and how), and then each builder itself references a host type. Users never see the hostType. (except perhaps in gomote list output) But they at least never need to care about them. Reverse buildlets now can only be one hostType at a time, which simplifies things. We were no longer using multiple roles per machine once moving to VMs for OS X. gomote continues to operate as it did previously but its underlying protocol changed and clients will need to be updated. As a new feature, gomote now has a new flag to let you reuse a buildlet host connection for different builder rules if they share the same underlying host type. But users can ignore that. This CL is a long-standing TODO (previously attempted and aborted) and will make many things easier and faster, including the linux-arm cross-compilation effort, and keeping pre-warmed buildlets of VM types ready to go. Updates golang/go#17104 Change-Id: Iad8387f48680424a8441e878a2f4762bf79ea4d2 Reviewed-on: https://go-review.googlesource.com/29551 Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2016-09-22 00:27:37 +03:00
// needed for reverse buildlets because in that case, the buildlets
// are already running and their stage0 should know how to update it
// it automatically.
buildletURLTmpl string
dashboard: add host-linux-arm64-aws builder Add the host-linux-arm64-aws builder. The HostConfig IsVM method has been modified to support the concept that an EC2 VM can be created which runs the buildlet binary in a container. This configuration would have both a VM image and a container image set in the host configuration. For golang/go#36841 Change-Id: I092b48b1dda24d24bcd0b10dff0b2126f5f43017 Reviewed-on: https://go-review.googlesource.com/c/build/+/249118 Run-TryBot: Carlos Amedee <carlos@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2020-08-18 20:32:05 +03:00
// Exactly 1 of these must be set (with the exception of EC2 instances).
// An EC2 instance may run a container inside a VM. In that case, a VMImage
// and ContainerImage will both be set.
all: rename Kube to Container and IsGCE to IsVM Once containers run on COS instead of Kubernetes, one name (Kube*) is wrong and the other (GCE) is ambiguous. So rename them now to be more specific. No behavior changes. Just renaming in this step, to reduce size of next CL. Updates golang/go#25108 Change-Id: Ib09eb682ef74acbbf6ed50b46074f834ef5e0c0b Reviewed-on: https://go-review.googlesource.com/111639 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-05-05 19:36:05 +03:00
VMImage string // e.g. "openbsd-amd64-60"
ContainerImage string // e.g. "linux-buildlet-std:latest" (suffix after "gcr.io/<PROJ>/")
IsReverse bool // if true, only use the reverse buildlet pool
all: split builder config into builder & host configs Our builders are named of the form "GOOS-GOARCH" or "GOOS-GOARCH-suffix". Over time we've grown many builders. This CL doesn't change that. Builders continue to be named and operate as before. Previously the build configuration file (dashboard/builders.go) made each builder type ("linux-amd64-race", etc) define how to create a host running a buildlet of that type, even though many builders had identical host configs. For example, these builders all share the same host type (a Kubernetes container): linux-amd64 linux-amd64-race linux-386 linux-386-387 And these are the same host type (a GCE VM): windows-amd64-gce windows-amd64-race windows-386-gce This CL creates a new concept of a "hostType" which defines how the buildlet is created (Kube, GCE, Reverse, and how), and then each builder itself references a host type. Users never see the hostType. (except perhaps in gomote list output) But they at least never need to care about them. Reverse buildlets now can only be one hostType at a time, which simplifies things. We were no longer using multiple roles per machine once moving to VMs for OS X. gomote continues to operate as it did previously but its underlying protocol changed and clients will need to be updated. As a new feature, gomote now has a new flag to let you reuse a buildlet host connection for different builder rules if they share the same underlying host type. But users can ignore that. This CL is a long-standing TODO (previously attempted and aborted) and will make many things easier and faster, including the linux-arm cross-compilation effort, and keeping pre-warmed buildlets of VM types ready to go. Updates golang/go#17104 Change-Id: Iad8387f48680424a8441e878a2f4762bf79ea4d2 Reviewed-on: https://go-review.googlesource.com/29551 Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2016-09-22 00:27:37 +03:00
dashboard: add linux-amd64-perf host and builder Add a new builder to run the x/benchmarks performance tests on linux-amd64. For now, this runs on a GCE C2 instance type, as these instances have well-defined, consistent CPUs and other server architecture components. In basic noise testing, even standard VMs of this type appear to be fairly low noise. As we gain experience with actual monitoring, we may change this to a sole-tenant VM type or even a dedicated machine if necessary. For golang/go#49207 Change-Id: I17eaeeb5349af925249940bebd5b860a2579e6df Reviewed-on: https://go-review.googlesource.com/c/build/+/354311 Trust: Michael Pratt <mpratt@google.com> Run-TryBot: Michael Pratt <mpratt@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org>
2021-10-01 21:59:33 +03:00
// GCE options, if VMImage != "" || ContainerImage != ""
buildlet, dashboard: add min CPU platform knob, require Skylake for OpenBSD I thought this would be enough for OpenBSD to select the TSC on its own without being forced to (as in CL 160319), but apparently it is not: https://github.com/golang/go/issues/29223#issuecomment-459035482 So it seems like we want both this CL and CL 160319. Updates golang/go#29223 Change-Id: I0a092d62881d8dcce0ef1129d8d32d8f4025b6ac Reviewed-on: https://go-review.googlesource.com/c/160457 Reviewed-by: Andrew Bonventre <andybons@golang.org>
2019-01-30 20:13:07 +03:00
machineType string // optional GCE instance type
RegularDisk bool // if true, use spinning disk instead of SSD
MinCPUPlatform string // optional; https://cloud.google.com/compute/docs/instances/specify-min-cpu-platform
all: split builder config into builder & host configs Our builders are named of the form "GOOS-GOARCH" or "GOOS-GOARCH-suffix". Over time we've grown many builders. This CL doesn't change that. Builders continue to be named and operate as before. Previously the build configuration file (dashboard/builders.go) made each builder type ("linux-amd64-race", etc) define how to create a host running a buildlet of that type, even though many builders had identical host configs. For example, these builders all share the same host type (a Kubernetes container): linux-amd64 linux-amd64-race linux-386 linux-386-387 And these are the same host type (a GCE VM): windows-amd64-gce windows-amd64-race windows-386-gce This CL creates a new concept of a "hostType" which defines how the buildlet is created (Kube, GCE, Reverse, and how), and then each builder itself references a host type. Users never see the hostType. (except perhaps in gomote list output) But they at least never need to care about them. Reverse buildlets now can only be one hostType at a time, which simplifies things. We were no longer using multiple roles per machine once moving to VMs for OS X. gomote continues to operate as it did previously but its underlying protocol changed and clients will need to be updated. As a new feature, gomote now has a new flag to let you reuse a buildlet host connection for different builder rules if they share the same underlying host type. But users can ignore that. This CL is a long-standing TODO (previously attempted and aborted) and will make many things easier and faster, including the linux-arm cross-compilation effort, and keeping pre-warmed buildlets of VM types ready to go. Updates golang/go#17104 Change-Id: Iad8387f48680424a8441e878a2f4762bf79ea4d2 Reviewed-on: https://go-review.googlesource.com/29551 Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2016-09-22 00:27:37 +03:00
dashboard, buildenv: label EC2 instances This change adds a boolean which identified when an instance is an EC2 instance. Various EC2 related variables have been renamed to be more consistent. Updates golang/go#36841 Change-Id: Ief385bf9d41e320ebd7d31bc0108e8053f337afd Reviewed-on: https://go-review.googlesource.com/c/build/+/233801 Run-TryBot: Carlos Amedee <carlos@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-05-13 20:27:15 +03:00
// EC2 options
isEC2 bool // if true, the instance is configured to run on EC2
cmd/coordinator: clean up reverse buildlet code, export status JSON Will be used for dynamic creation/destruction of Mac VMs in subsequent CL. Updates golang/go#9495 (Mac virtualization) Updates golang/go#15760 (monitoring) Change-Id: I48b17589b258d5d742bad5a3ddae18de98778149 Reviewed-on: https://go-review.googlesource.com/37457 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2017-02-24 20:50:49 +03:00
// ReverseOptions:
cmd/coordinator, cmd/buildlet, cmd/gomote: add SSH support This adds an SSH server to farmer.golang.org on port 2222 that proxies SSH connections to users' gomote-created buildlet instances. For example: $ gomote create openbsd-amd64-60 user-bradfitz-openbsd-amd64-60-1 $ gomote ssh user-bradfitz-openbsd-amd64-60-1 Warning: Permanently added '[localhost]:33351' (ECDSA) to the list of known hosts. OpenBSD 6.0 (GENERIC.MP) golang/go#2319: Tue Jul 26 13:00:43 MDT 2016 Welcome to OpenBSD: The proactively secure Unix-like operating system. Please use the sendbug(1) utility to report bugs in the system. Before reporting a bug, please try to reproduce it with the latest version of the code. With bug reports, please try to ensure that enough information to reproduce the problem is enclosed, and if a known fix for it exists, include that as well. $ As before, if the coordinator process is restarted (or crashes, is evicted, etc), all gomote instances die. Not yet supported: * scp (help wanted) * not all host types are configured. most are. some will need slight config tweaks to the Docker image (e.g. adding openssh-server) Supports currently: * linux-amd64 (host type shared by 386, nacl) * linux-arm * linux-arm64 * darwin * freebsd * openbsd * plan9-386 * windows Implementation details: * the ssh server process listens on port 2222 in the coordinator (farmer.golang.org), which is behind a GKE TCP load balancer. * the ssh server library is github.com/gliderlabs/ssh * authentication is done via Github users' public keys. It's assumed that gomote user == github user. But there's a mapping in the code for known exceptions. * we can't give out access to this too widely. too many things are accessible from within the host environment if you look in the right places. Details omitted. But the Go team and other trusted gomote users can use this. * the buildlet binary has a new /connect-ssh handler that acts like a CONNECT request but instead of taking an explicit host:port, just says "give me your machine's SSH connection". The buildlet can also start sshd if needed for the environment. The /connect-ssh handler also installs the coordinator's public key. * a new buildlet client library method "ConnectSSH" hits the /connect-ssh handler and returns a net.Conn. * the coordinator's ssh.Handler is just running the OpenSSH ssh client. * because the OpenSSH ssh child process can't connect to a net.Conn, an emphemeral localhost port is created on the coordinator to proxy between the ssh client and the net.Conn returned by ConnectSSH. * The /connect-ssh handler requires http.Hijacker, which requires fully compliant net.Conn implementations as of Go 1.8. So I needed to flesh out revdial too, testing it with the golang.org/x/net/nettest package. * plan9 doesn't have an ssh server, so we use 0intro's new conterm program (drawterm without GUI support) to connect to plan9 from the coordinator ssh proxy instead of using the OpenSSH ssh client binary. * windows doesn't have an ssh server, so we enable the telnet service and the coordinator ssh proxy uses telnet instead on the backend on the private network. (There is a Windows ssh server but only in new versions.) Happy debugging over ssh! Fixes golang/go#19956 Change-Id: I80a62064c5f85af1f195f980c862ba29af4015f0 Reviewed-on: https://go-review.googlesource.com/50750 Reviewed-by: Herbie Ong <herbie@google.com> Reviewed-by: Jessie Frazelle <me@jessfraz.com> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2017-07-22 22:15:56 +03:00
ExpectNum int // expected number of reverse buildlets of this type
HermeticReverse bool // whether reverse buildlet has fresh env per conn
cmd/coordinator: clean up reverse buildlet code, export status JSON Will be used for dynamic creation/destruction of Mac VMs in subsequent CL. Updates golang/go#9495 (Mac virtualization) Updates golang/go#15760 (monitoring) Change-Id: I48b17589b258d5d742bad5a3ddae18de98778149 Reviewed-on: https://go-review.googlesource.com/37457 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2017-02-24 20:50:49 +03:00
dashboard, buildlet: add a disabled builder with nested virt, for testing This adds a linux-amd64 COS builder that should be just like our existing linux-amd64 COS builder except that it's using a forked image that has the VMX license bit enabled for nested virtualization. (GCE appears to be using the license mechanism as some sort of opt-in mechanism for features that aren't yet GA; might go away?) Once this is in, it won't do any new builds as regular+trybot builders are disabled. But it means I can then use gomote + debugnewvm to work on preparing the other four image types. Updates golang/go#15581 (solaris) Updates golang/go#23060 (dragonfly) Updates golang/go#30262 (riscv) Updates golang/go#30267 (fuchsia) Updates golang/go#23824 (android) Change-Id: Ic55f17eea17908dba7f58618d8cd162a2ed9b015 Reviewed-on: https://go-review.googlesource.com/c/162959 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-02-16 01:05:42 +03:00
// Container image options, if ContainerImage != "":
dashboard: use n2 instances where necessary e2 and n2d instances do not support specifying a MinCPUVersion or NestedVirt. For golang/go#48969 Change-Id: I0c4b359b29cd24302b869748014f848a9b7b5e39 Reviewed-on: https://go-review.googlesource.com/c/build/+/356071 Trust: Alexander Rakoczy <alex@golang.org> Run-TryBot: Alexander Rakoczy <alex@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-10-15 04:54:32 +03:00
NestedVirt bool // container requires VMX nested virtualization. e2 and n2d instances are not supported.
env/linux-x86-vmx: add new Debian host that's like Container-Optimized OS + vmx This adds scripts to create a new builder host image that acts like Container-Optimized OS (has docker, runs konlet on startup) but with a Debian 9 kernel + userspace that permits KVM for nested virtualization. Updates golang/go#15581 (solaris) Updates golang/go#23060 (dragonfly) Updates golang/go#30262 (riscv) Updates golang/go#30267 (fuchsia) Updates golang/go#23824 (android) Change-Id: Ib1d3a250556703856083c222be2a70c4e8d91884 Reviewed-on: https://go-review.googlesource.com/c/163301 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-02-22 00:10:54 +03:00
KonletVMImage string // optional VM image (containing konlet) to use instead of default
dashboard, buildlet: add a disabled builder with nested virt, for testing This adds a linux-amd64 COS builder that should be just like our existing linux-amd64 COS builder except that it's using a forked image that has the VMX license bit enabled for nested virtualization. (GCE appears to be using the license mechanism as some sort of opt-in mechanism for features that aren't yet GA; might go away?) Once this is in, it won't do any new builds as regular+trybot builders are disabled. But it means I can then use gomote + debugnewvm to work on preparing the other four image types. Updates golang/go#15581 (solaris) Updates golang/go#23060 (dragonfly) Updates golang/go#30262 (riscv) Updates golang/go#30267 (fuchsia) Updates golang/go#23824 (android) Change-Id: Ic55f17eea17908dba7f58618d8cd162a2ed9b015 Reviewed-on: https://go-review.googlesource.com/c/162959 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-02-16 01:05:42 +03:00
all: split builder config into builder & host configs Our builders are named of the form "GOOS-GOARCH" or "GOOS-GOARCH-suffix". Over time we've grown many builders. This CL doesn't change that. Builders continue to be named and operate as before. Previously the build configuration file (dashboard/builders.go) made each builder type ("linux-amd64-race", etc) define how to create a host running a buildlet of that type, even though many builders had identical host configs. For example, these builders all share the same host type (a Kubernetes container): linux-amd64 linux-amd64-race linux-386 linux-386-387 And these are the same host type (a GCE VM): windows-amd64-gce windows-amd64-race windows-386-gce This CL creates a new concept of a "hostType" which defines how the buildlet is created (Kube, GCE, Reverse, and how), and then each builder itself references a host type. Users never see the hostType. (except perhaps in gomote list output) But they at least never need to care about them. Reverse buildlets now can only be one hostType at a time, which simplifies things. We were no longer using multiple roles per machine once moving to VMs for OS X. gomote continues to operate as it did previously but its underlying protocol changed and clients will need to be updated. As a new feature, gomote now has a new flag to let you reuse a buildlet host connection for different builder rules if they share the same underlying host type. But users can ignore that. This CL is a long-standing TODO (previously attempted and aborted) and will make many things easier and faster, including the linux-arm cross-compilation effort, and keeping pre-warmed buildlets of VM types ready to go. Updates golang/go#17104 Change-Id: Iad8387f48680424a8441e878a2f4762bf79ea4d2 Reviewed-on: https://go-review.googlesource.com/29551 Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2016-09-22 00:27:37 +03:00
// Optional base env. GOROOT_BOOTSTRAP should go here if the buildlet
// has Go 1.4+ baked in somewhere.
env []string
dashboard: rename Go14URL to goBootstrapURLTmpl and provide accessor Also update all users, and rename another template field. Includes changes to coordinator, gomote, and release. Change-Id: I1c4408eadbcb83d61063a910dfa18cc395952bc2 Reviewed-on: https://go-review.googlesource.com/20976 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2016-03-22 01:05:52 +03:00
// These template URLs may contain $BUCKET which is expanded to the
// relevant Cloud Storage bucket as specified by the build environment.
goBootstrapURLTmpl string // optional URL to a built Go 1.4+ tar.gz
dashboard: move builder configuration out of coordinator into a package The new package will be used by the buildlet command line tool in the future. Also use the metadata package in the coordinator for the master build key, since it was already depending on that package. Might as well use it. Change-Id: I6ad7593787ebba364090951ccbaecb4aa38e1dd6 Reviewed-on: https://go-review.googlesource.com/2920 Reviewed-by: Andrew Gerrand <adg@golang.org>
2015-01-15 23:46:22 +03:00
dashboard, cmd/coordinator: support multiple builder owners Prefer displaying GitHub usernames for convenience of pinging in GitHub issues, since all current builder owners have them. Keep it possible to use an email or something else if no GitHub account exists. Update the owner of the Corellium builders to be individual GitHub account, so that it can be pinged in an issue. The GitHub organization is already mentioned in the builder notes. Also switch to html/template (instead of text/template) for rendering the builders page. Fixes golang/go#49596. Change-Id: I89a96eb8f7a3057bc1f3aaee16abc9c776c45ee7 Reviewed-on: https://go-review.googlesource.com/c/build/+/363983 Trust: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-11-16 20:42:29 +03:00
Owners []*gophers.Person // owners; empty means golang-dev
Notes string // notes for humans
all: split builder config into builder & host configs Our builders are named of the form "GOOS-GOARCH" or "GOOS-GOARCH-suffix". Over time we've grown many builders. This CL doesn't change that. Builders continue to be named and operate as before. Previously the build configuration file (dashboard/builders.go) made each builder type ("linux-amd64-race", etc) define how to create a host running a buildlet of that type, even though many builders had identical host configs. For example, these builders all share the same host type (a Kubernetes container): linux-amd64 linux-amd64-race linux-386 linux-386-387 And these are the same host type (a GCE VM): windows-amd64-gce windows-amd64-race windows-386-gce This CL creates a new concept of a "hostType" which defines how the buildlet is created (Kube, GCE, Reverse, and how), and then each builder itself references a host type. Users never see the hostType. (except perhaps in gomote list output) But they at least never need to care about them. Reverse buildlets now can only be one hostType at a time, which simplifies things. We were no longer using multiple roles per machine once moving to VMs for OS X. gomote continues to operate as it did previously but its underlying protocol changed and clients will need to be updated. As a new feature, gomote now has a new flag to let you reuse a buildlet host connection for different builder rules if they share the same underlying host type. But users can ignore that. This CL is a long-standing TODO (previously attempted and aborted) and will make many things easier and faster, including the linux-arm cross-compilation effort, and keeping pre-warmed buildlets of VM types ready to go. Updates golang/go#17104 Change-Id: Iad8387f48680424a8441e878a2f4762bf79ea4d2 Reviewed-on: https://go-review.googlesource.com/29551 Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2016-09-22 00:27:37 +03:00
cmd/coordinator, cmd/buildlet, cmd/gomote: add SSH support This adds an SSH server to farmer.golang.org on port 2222 that proxies SSH connections to users' gomote-created buildlet instances. For example: $ gomote create openbsd-amd64-60 user-bradfitz-openbsd-amd64-60-1 $ gomote ssh user-bradfitz-openbsd-amd64-60-1 Warning: Permanently added '[localhost]:33351' (ECDSA) to the list of known hosts. OpenBSD 6.0 (GENERIC.MP) golang/go#2319: Tue Jul 26 13:00:43 MDT 2016 Welcome to OpenBSD: The proactively secure Unix-like operating system. Please use the sendbug(1) utility to report bugs in the system. Before reporting a bug, please try to reproduce it with the latest version of the code. With bug reports, please try to ensure that enough information to reproduce the problem is enclosed, and if a known fix for it exists, include that as well. $ As before, if the coordinator process is restarted (or crashes, is evicted, etc), all gomote instances die. Not yet supported: * scp (help wanted) * not all host types are configured. most are. some will need slight config tweaks to the Docker image (e.g. adding openssh-server) Supports currently: * linux-amd64 (host type shared by 386, nacl) * linux-arm * linux-arm64 * darwin * freebsd * openbsd * plan9-386 * windows Implementation details: * the ssh server process listens on port 2222 in the coordinator (farmer.golang.org), which is behind a GKE TCP load balancer. * the ssh server library is github.com/gliderlabs/ssh * authentication is done via Github users' public keys. It's assumed that gomote user == github user. But there's a mapping in the code for known exceptions. * we can't give out access to this too widely. too many things are accessible from within the host environment if you look in the right places. Details omitted. But the Go team and other trusted gomote users can use this. * the buildlet binary has a new /connect-ssh handler that acts like a CONNECT request but instead of taking an explicit host:port, just says "give me your machine's SSH connection". The buildlet can also start sshd if needed for the environment. The /connect-ssh handler also installs the coordinator's public key. * a new buildlet client library method "ConnectSSH" hits the /connect-ssh handler and returns a net.Conn. * the coordinator's ssh.Handler is just running the OpenSSH ssh client. * because the OpenSSH ssh child process can't connect to a net.Conn, an emphemeral localhost port is created on the coordinator to proxy between the ssh client and the net.Conn returned by ConnectSSH. * The /connect-ssh handler requires http.Hijacker, which requires fully compliant net.Conn implementations as of Go 1.8. So I needed to flesh out revdial too, testing it with the golang.org/x/net/nettest package. * plan9 doesn't have an ssh server, so we use 0intro's new conterm program (drawterm without GUI support) to connect to plan9 from the coordinator ssh proxy instead of using the OpenSSH ssh client binary. * windows doesn't have an ssh server, so we enable the telnet service and the coordinator ssh proxy uses telnet instead on the backend on the private network. (There is a Windows ssh server but only in new versions.) Happy debugging over ssh! Fixes golang/go#19956 Change-Id: I80a62064c5f85af1f195f980c862ba29af4015f0 Reviewed-on: https://go-review.googlesource.com/50750 Reviewed-by: Herbie Ong <herbie@google.com> Reviewed-by: Jessie Frazelle <me@jessfraz.com> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2017-07-22 22:15:56 +03:00
SSHUsername string // username to ssh as, empty means not supported
all: split builder config into builder & host configs Our builders are named of the form "GOOS-GOARCH" or "GOOS-GOARCH-suffix". Over time we've grown many builders. This CL doesn't change that. Builders continue to be named and operate as before. Previously the build configuration file (dashboard/builders.go) made each builder type ("linux-amd64-race", etc) define how to create a host running a buildlet of that type, even though many builders had identical host configs. For example, these builders all share the same host type (a Kubernetes container): linux-amd64 linux-amd64-race linux-386 linux-386-387 And these are the same host type (a GCE VM): windows-amd64-gce windows-amd64-race windows-386-gce This CL creates a new concept of a "hostType" which defines how the buildlet is created (Kube, GCE, Reverse, and how), and then each builder itself references a host type. Users never see the hostType. (except perhaps in gomote list output) But they at least never need to care about them. Reverse buildlets now can only be one hostType at a time, which simplifies things. We were no longer using multiple roles per machine once moving to VMs for OS X. gomote continues to operate as it did previously but its underlying protocol changed and clients will need to be updated. As a new feature, gomote now has a new flag to let you reuse a buildlet host connection for different builder rules if they share the same underlying host type. But users can ignore that. This CL is a long-standing TODO (previously attempted and aborted) and will make many things easier and faster, including the linux-arm cross-compilation effort, and keeping pre-warmed buildlets of VM types ready to go. Updates golang/go#17104 Change-Id: Iad8387f48680424a8441e878a2f4762bf79ea4d2 Reviewed-on: https://go-review.googlesource.com/29551 Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2016-09-22 00:27:37 +03:00
}
// A BuildConfig describes how to run a builder.
type BuildConfig struct {
// Name is the unique name of the builder, in the form of
// "GOOS-GOARCH" or "GOOS-GOARCH-suffix". For example,
// "darwin-386", "linux-386-387", "linux-amd64-race". Some
// suffixes are well-known and carry special meaning, such as
// "-race".
Name string
// HostType is the required key into the Hosts map, describing
// the type of host this build will run on.
dashboard, env: add Debian Stretch builder, rename Jessie builder Updates golang/go#25386 Change-Id: I65e3dda63e5ea86fa94480e3305556b73afe121a Reviewed-on: https://go-review.googlesource.com/113059 Reviewed-by: Bryan C. Mills <bcmills@google.com>
2018-05-14 20:22:45 +03:00
// For example, "host-linux-jessie".
all: split builder config into builder & host configs Our builders are named of the form "GOOS-GOARCH" or "GOOS-GOARCH-suffix". Over time we've grown many builders. This CL doesn't change that. Builders continue to be named and operate as before. Previously the build configuration file (dashboard/builders.go) made each builder type ("linux-amd64-race", etc) define how to create a host running a buildlet of that type, even though many builders had identical host configs. For example, these builders all share the same host type (a Kubernetes container): linux-amd64 linux-amd64-race linux-386 linux-386-387 And these are the same host type (a GCE VM): windows-amd64-gce windows-amd64-race windows-386-gce This CL creates a new concept of a "hostType" which defines how the buildlet is created (Kube, GCE, Reverse, and how), and then each builder itself references a host type. Users never see the hostType. (except perhaps in gomote list output) But they at least never need to care about them. Reverse buildlets now can only be one hostType at a time, which simplifies things. We were no longer using multiple roles per machine once moving to VMs for OS X. gomote continues to operate as it did previously but its underlying protocol changed and clients will need to be updated. As a new feature, gomote now has a new flag to let you reuse a buildlet host connection for different builder rules if they share the same underlying host type. But users can ignore that. This CL is a long-standing TODO (previously attempted and aborted) and will make many things easier and faster, including the linux-arm cross-compilation effort, and keeping pre-warmed buildlets of VM types ready to go. Updates golang/go#17104 Change-Id: Iad8387f48680424a8441e878a2f4762bf79ea4d2 Reviewed-on: https://go-review.googlesource.com/29551 Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2016-09-22 00:27:37 +03:00
HostType string
dashboard, app/appengine: set a known issue for staticlockranking builder The linux-amd64-staticlockranking builder was added pre-emptively so that it could be tested and used during the development of a CL that implements a new feature. It's expected to fail until that work is completed. Mark it as a builder with a known issue, golang/go#38029. The new BuildConfig.KnownIssue field can be used by builders being added in the future so that if they fail at first, it doesn't take away time for people looking at build.golang.org, which we always aim to have as green as possible¹. ¹ https://groups.google.com/d/msg/golang-dev/y0yM_Xi665Q/hUpBiUPiCAAJ Fixes golang/go#38283 For golang/go#38029 For golang/go#37937 For golang/go#11811 Change-Id: Iba1606c7021ffa7e67ddbaae2fc6b65f4e46ab34 Reviewed-on: https://go-review.googlesource.com/c/build/+/227379 Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-04-07 00:34:17 +03:00
// KnownIssue is a non-zero golang.org/issue/nnn number for a builder
dashboard: create misc-compile-ios builder Create the builder as a post-submit builder with a non-zero KnownIssue value first. Once we know it works well, we'll have the confidence to be able to safely promote it to a normal TryBot, without the risk of causing disruption for contributors. For golang/go#41610. Change-Id: If871addf325b7728109afbe094b72d40e27ceec1 Reviewed-on: https://go-review.googlesource.com/c/build/+/257617 Trust: Dmitri Shuralyov <dmitshur@golang.org> Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Andrew Bonventre <andybons@golang.org>
2020-09-25 18:52:04 +03:00
// that may fail due to a known issue, such as because it is a new
// builder still in development/testing, or because the feature
dashboard, app/appengine: set a known issue for staticlockranking builder The linux-amd64-staticlockranking builder was added pre-emptively so that it could be tested and used during the development of a CL that implements a new feature. It's expected to fail until that work is completed. Mark it as a builder with a known issue, golang/go#38029. The new BuildConfig.KnownIssue field can be used by builders being added in the future so that if they fail at first, it doesn't take away time for people looking at build.golang.org, which we always aim to have as green as possible¹. ¹ https://groups.google.com/d/msg/golang-dev/y0yM_Xi665Q/hUpBiUPiCAAJ Fixes golang/go#38283 For golang/go#38029 For golang/go#37937 For golang/go#11811 Change-Id: Iba1606c7021ffa7e67ddbaae2fc6b65f4e46ab34 Reviewed-on: https://go-review.googlesource.com/c/build/+/227379 Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-04-07 00:34:17 +03:00
// or port that it's meant to test hasn't been added yet, etc.
//
// A non-zero value here means that failures on this builder should not
// be considered a serious regression and don't need investigation beyond
// what is already in scope of the listed issue.
KnownIssue int
all: split builder config into builder & host configs Our builders are named of the form "GOOS-GOARCH" or "GOOS-GOARCH-suffix". Over time we've grown many builders. This CL doesn't change that. Builders continue to be named and operate as before. Previously the build configuration file (dashboard/builders.go) made each builder type ("linux-amd64-race", etc) define how to create a host running a buildlet of that type, even though many builders had identical host configs. For example, these builders all share the same host type (a Kubernetes container): linux-amd64 linux-amd64-race linux-386 linux-386-387 And these are the same host type (a GCE VM): windows-amd64-gce windows-amd64-race windows-386-gce This CL creates a new concept of a "hostType" which defines how the buildlet is created (Kube, GCE, Reverse, and how), and then each builder itself references a host type. Users never see the hostType. (except perhaps in gomote list output) But they at least never need to care about them. Reverse buildlets now can only be one hostType at a time, which simplifies things. We were no longer using multiple roles per machine once moving to VMs for OS X. gomote continues to operate as it did previously but its underlying protocol changed and clients will need to be updated. As a new feature, gomote now has a new flag to let you reuse a buildlet host connection for different builder rules if they share the same underlying host type. But users can ignore that. This CL is a long-standing TODO (previously attempted and aborted) and will make many things easier and faster, including the linux-arm cross-compilation effort, and keeping pre-warmed buildlets of VM types ready to go. Updates golang/go#17104 Change-Id: Iad8387f48680424a8441e878a2f4762bf79ea4d2 Reviewed-on: https://go-review.googlesource.com/29551 Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2016-09-22 00:27:37 +03:00
Notes string // notes for humans
dashboard: unify post-submit & trybot policy in builders.go, not coordinator This removes a bunch of TODOs about scattered policy and moves it all into builders.go. While here, * add a bunch of tests * unexport some things * rename some things * document some things * adjust FreeBSD policy as function of branch (per Go 1.12 being last to support FreeBSD 10.x, and to unblock CL 165801) * set GO_BUILDER_SET_GOPROXY=coordinator for reverse buildlets, which is necessary to remove the oauth2 & build special cases in the config * change Elias Naur's mobile builder to how I think he wants it (he was fighting the old system) * add $HOME on the Solaris smartos builder, which was missing & causing tests to fail lately * make the (currently failing) longtest builder have GOPROXY set * remove an allocation in version.ParseReleaseBranch Fixes golang/go#9603 Updates golang/go#14594 Change-Id: I50a23ad7cdf478c95b14bee9b3931ba361baacfa Reviewed-on: https://go-review.googlesource.com/c/build/+/166218 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-03-07 20:44:41 +03:00
// tryBot optionally specifies a policy func for whether trybots are enabled.
// nil means off. Even if tryBot returns true, BuildConfig.BuildsRepo must also
// return true. See the implementation of BuildConfig.BuildsRepoTryBot.
// The proj is "go", "net", etc. The branch is proj's branch.
// The goBranch is the same as branch for proj "go", else it's the go branch
// ("master, "release-branch.go1.12", etc).
tryBot func(proj, branch, goBranch string) bool
tryOnly bool // only used for trybots, and not regular builds
CompileOnly bool // if true, compile tests, but don't run them
FlakyNet bool // network tests are flaky (try anyway, but ignore some failures)
dashboard: document and clarify defaultBuildsRepoPolicy The defaultBuildsRepoPolicy function was very important in that any changes to it would affect many builders, but it was not documented well. This refactor attempts to improve the situation. It turned out better to also rename defaultBuildsRepoPolicy to buildRepoByDefault and limit its scope to report whether a given repo should be built by default. The branch and goBranch parameters were never used, and thus misleading. If they're needed in the future, we can easily re-add them. The name defaultBuildsRepoPolicy made it sound as if it was the entire "builds repo" policy, bun reality most of the policy is implemented in the BuildConfig.buildsRepoAtAll method and is non-configurable. Only a small part of the policy was in defaultBuildsRepoPolicy. For golang/go#39156. Change-Id: Ie7c6a68bd624056362dc7d964bd8d640f665b35a Reviewed-on: https://go-review.googlesource.com/c/build/+/240687 Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-07-01 21:45:10 +03:00
// buildsRepo optionally specifies whether this builder does
// builds (of any type) for the given repo ("go", "net", etc.)
// and its branch ("master", "release-branch.go1.12", "dev.link", etc.).
// goBranch is the branch of "go" to build against. If repo == "go",
// goBranch == branch.
dashboard: add linux-amd64-staticlockranking builder This is a new linux/amd64 GCE builder for the purpose of testing the GOEXPERIMENT=staticlockranking configuration which is landing to master branch soon. It runs only on the main Go repository master branch and release branches 1.15 and newer. No golang.org/x repos. Not included in trybots by default, but can be requested via SlowBots by saying TRY=linux-amd64-staticlockranking. Also expand documentation of BuildConfig.buildsRepo field to make it more clear how to use it. A common mistake for custom buildsRepo functions that are added is to create a custom policy with a minor adjustment and forgetting to preserve the rest of the default build repo policy. Fixes golang/go#37937 Change-Id: Ic56f82e9ede30cb6fe8f6cc7d0282d6a0a12c1b8 Reviewed-on: https://go-review.googlesource.com/c/build/+/224078 Reviewed-by: Robert Findley <rfindley@google.com> Reviewed-by: Alexander Rakoczy <alex@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org> Run-TryBot: Alexander Rakoczy <alex@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2020-03-19 17:34:11 +03:00
//
dashboard: document and clarify defaultBuildsRepoPolicy The defaultBuildsRepoPolicy function was very important in that any changes to it would affect many builders, but it was not documented well. This refactor attempts to improve the situation. It turned out better to also rename defaultBuildsRepoPolicy to buildRepoByDefault and limit its scope to report whether a given repo should be built by default. The branch and goBranch parameters were never used, and thus misleading. If they're needed in the future, we can easily re-add them. The name defaultBuildsRepoPolicy made it sound as if it was the entire "builds repo" policy, bun reality most of the policy is implemented in the BuildConfig.buildsRepoAtAll method and is non-configurable. Only a small part of the policy was in defaultBuildsRepoPolicy. For golang/go#39156. Change-Id: Ie7c6a68bd624056362dc7d964bd8d640f665b35a Reviewed-on: https://go-review.googlesource.com/c/build/+/240687 Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-07-01 21:45:10 +03:00
// If nil, a default set of repos as reported by buildRepoByDefault
// is built. See buildsRepoAtAll method for details.
dashboard: add linux-amd64-staticlockranking builder This is a new linux/amd64 GCE builder for the purpose of testing the GOEXPERIMENT=staticlockranking configuration which is landing to master branch soon. It runs only on the main Go repository master branch and release branches 1.15 and newer. No golang.org/x repos. Not included in trybots by default, but can be requested via SlowBots by saying TRY=linux-amd64-staticlockranking. Also expand documentation of BuildConfig.buildsRepo field to make it more clear how to use it. A common mistake for custom buildsRepo functions that are added is to create a custom policy with a minor adjustment and forgetting to preserve the rest of the default build repo policy. Fixes golang/go#37937 Change-Id: Ic56f82e9ede30cb6fe8f6cc7d0282d6a0a12c1b8 Reviewed-on: https://go-review.googlesource.com/c/build/+/224078 Reviewed-by: Robert Findley <rfindley@google.com> Reviewed-by: Alexander Rakoczy <alex@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org> Run-TryBot: Alexander Rakoczy <alex@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2020-03-19 17:34:11 +03:00
//
// To implement a minor change to the default policy, create a
dashboard: document and clarify defaultBuildsRepoPolicy The defaultBuildsRepoPolicy function was very important in that any changes to it would affect many builders, but it was not documented well. This refactor attempts to improve the situation. It turned out better to also rename defaultBuildsRepoPolicy to buildRepoByDefault and limit its scope to report whether a given repo should be built by default. The branch and goBranch parameters were never used, and thus misleading. If they're needed in the future, we can easily re-add them. The name defaultBuildsRepoPolicy made it sound as if it was the entire "builds repo" policy, bun reality most of the policy is implemented in the BuildConfig.buildsRepoAtAll method and is non-configurable. Only a small part of the policy was in defaultBuildsRepoPolicy. For golang/go#39156. Change-Id: Ie7c6a68bd624056362dc7d964bd8d640f665b35a Reviewed-on: https://go-review.googlesource.com/c/build/+/240687 Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-07-01 21:45:10 +03:00
// function that uses buildRepoByDefault. For example:
dashboard: add linux-amd64-staticlockranking builder This is a new linux/amd64 GCE builder for the purpose of testing the GOEXPERIMENT=staticlockranking configuration which is landing to master branch soon. It runs only on the main Go repository master branch and release branches 1.15 and newer. No golang.org/x repos. Not included in trybots by default, but can be requested via SlowBots by saying TRY=linux-amd64-staticlockranking. Also expand documentation of BuildConfig.buildsRepo field to make it more clear how to use it. A common mistake for custom buildsRepo functions that are added is to create a custom policy with a minor adjustment and forgetting to preserve the rest of the default build repo policy. Fixes golang/go#37937 Change-Id: Ic56f82e9ede30cb6fe8f6cc7d0282d6a0a12c1b8 Reviewed-on: https://go-review.googlesource.com/c/build/+/224078 Reviewed-by: Robert Findley <rfindley@google.com> Reviewed-by: Alexander Rakoczy <alex@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org> Run-TryBot: Alexander Rakoczy <alex@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2020-03-19 17:34:11 +03:00
//
// buildsRepo: func(repo, branch, goBranch string) bool {
dashboard: document and clarify defaultBuildsRepoPolicy The defaultBuildsRepoPolicy function was very important in that any changes to it would affect many builders, but it was not documented well. This refactor attempts to improve the situation. It turned out better to also rename defaultBuildsRepoPolicy to buildRepoByDefault and limit its scope to report whether a given repo should be built by default. The branch and goBranch parameters were never used, and thus misleading. If they're needed in the future, we can easily re-add them. The name defaultBuildsRepoPolicy made it sound as if it was the entire "builds repo" policy, bun reality most of the policy is implemented in the BuildConfig.buildsRepoAtAll method and is non-configurable. Only a small part of the policy was in defaultBuildsRepoPolicy. For golang/go#39156. Change-Id: Ie7c6a68bd624056362dc7d964bd8d640f665b35a Reviewed-on: https://go-review.googlesource.com/c/build/+/240687 Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-07-01 21:45:10 +03:00
// b := buildRepoByDefault(repo)
dashboard: add linux-amd64-staticlockranking builder This is a new linux/amd64 GCE builder for the purpose of testing the GOEXPERIMENT=staticlockranking configuration which is landing to master branch soon. It runs only on the main Go repository master branch and release branches 1.15 and newer. No golang.org/x repos. Not included in trybots by default, but can be requested via SlowBots by saying TRY=linux-amd64-staticlockranking. Also expand documentation of BuildConfig.buildsRepo field to make it more clear how to use it. A common mistake for custom buildsRepo functions that are added is to create a custom policy with a minor adjustment and forgetting to preserve the rest of the default build repo policy. Fixes golang/go#37937 Change-Id: Ic56f82e9ede30cb6fe8f6cc7d0282d6a0a12c1b8 Reviewed-on: https://go-review.googlesource.com/c/build/+/224078 Reviewed-by: Robert Findley <rfindley@google.com> Reviewed-by: Alexander Rakoczy <alex@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org> Run-TryBot: Alexander Rakoczy <alex@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2020-03-19 17:34:11 +03:00
// // ... modify b from the default value as needed ...
// return b
// }
//
dashboard: unify post-submit & trybot policy in builders.go, not coordinator This removes a bunch of TODOs about scattered policy and moves it all into builders.go. While here, * add a bunch of tests * unexport some things * rename some things * document some things * adjust FreeBSD policy as function of branch (per Go 1.12 being last to support FreeBSD 10.x, and to unblock CL 165801) * set GO_BUILDER_SET_GOPROXY=coordinator for reverse buildlets, which is necessary to remove the oauth2 & build special cases in the config * change Elias Naur's mobile builder to how I think he wants it (he was fighting the old system) * add $HOME on the Solaris smartos builder, which was missing & causing tests to fail lately * make the (currently failing) longtest builder have GOPROXY set * remove an allocation in version.ParseReleaseBranch Fixes golang/go#9603 Updates golang/go#14594 Change-Id: I50a23ad7cdf478c95b14bee9b3931ba361baacfa Reviewed-on: https://go-review.googlesource.com/c/build/+/166218 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-03-07 20:44:41 +03:00
buildsRepo func(repo, branch, goBranch string) bool
dashboard: add a GOOS=linux builder & trybot for Android Updates golang/go#30482 Change-Id: I198dd40dfdb86b20e681c373141754680351e82f Reviewed-on: https://go-review.googlesource.com/c/164740 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Elias Naur <mail@eliasnaur.com>
2019-03-01 19:54:25 +03:00
cmd/coordinator: run performance tests from x/benchmarks Add initial support for running the performance tests from x/benchmarks. Since there are a variety of different test suites (some `go test`, bent, etc), x/benchmarks provides a basic wrapper command, golang.org/x/benchmarks/cmd/bench which know the minute details. The coordinator just needs to run that one command. This build mode is limited to builds of x/benchmarks on builders with RunBench set to true. Currently there are none, a future CL will add the initial such linux-amd64 builder. For golang/go#49207 Change-Id: Ie006ec4a3757a5c2fed0925a3f9eb91edeaa5224 Reviewed-on: https://go-review.googlesource.com/c/build/+/361417 Trust: Michael Pratt <mpratt@google.com> Run-TryBot: Michael Pratt <mpratt@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org>
2021-10-11 20:25:58 +03:00
// RunBench enables benchmarking of the toolchain using x/benchmarks.
// This only applies when building x/benchmarks.
RunBench bool
dashboard, cmd/coordinator, maintner/maintnerd: add support for BuildConfig.MinimumGoVersion field The new BuildConfig.MinimumGoVersion field specifies the minimum Go version the builder is allowed to use. It's useful when some of the builders are too new, and do not support all of the supported Go releases (e.g., openbsd-amd64-64 and freebsd-amd64-12_0 currently require Go 1.11 and don't work on Go 1.10). It only needs to be set when a builder doesn't support all supported Go releases, since we don't typically test unsupported Go releases. To allow cmd/coordinator to use this field and filter out work it receives from maintner/maintnerd's GoFindTryWork RPC call, we add a GoVersion slice to apipb.GerritTryWorkItem, and populate it in maintapi.apiService.GoFindTryWork method. For trybots on the Go repo, the GoVersion field is determined from the branch name. For "release-branch.goX.Y" branches, it parses out the major-minor Go version from the branch name. For master and other branches, it assumes the latest Go release. For trybots on subrepos, we already have the Go version information available, so use it directly. Afterwards, all that's left is to modify newTrySet in cmd/coordinator to make use of BuildConfig.MinimumGoVersion and work.GoVersion to skip builders that are too new for the Go version that needs to be tested. Fixes golang/go#29265 Change-Id: I50b01830647e33e37e9eb8b89e0f2518812fa44f Reviewed-on: https://go-review.googlesource.com/c/155463 Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-01-12 00:44:01 +03:00
// MinimumGoVersion optionally specifies the minimum Go version
// this builder is allowed to use. It can be useful for skipping
// builders that are too new and no longer support some supported
// Go versions. It doesn't need to be set for builders that support
// all supported Go versions.
//
// Note: This field currently has effect on trybot runs only.
dashboard: unify post-submit & trybot policy in builders.go, not coordinator This removes a bunch of TODOs about scattered policy and moves it all into builders.go. While here, * add a bunch of tests * unexport some things * rename some things * document some things * adjust FreeBSD policy as function of branch (per Go 1.12 being last to support FreeBSD 10.x, and to unblock CL 165801) * set GO_BUILDER_SET_GOPROXY=coordinator for reverse buildlets, which is necessary to remove the oauth2 & build special cases in the config * change Elias Naur's mobile builder to how I think he wants it (he was fighting the old system) * add $HOME on the Solaris smartos builder, which was missing & causing tests to fail lately * make the (currently failing) longtest builder have GOPROXY set * remove an allocation in version.ParseReleaseBranch Fixes golang/go#9603 Updates golang/go#14594 Change-Id: I50a23ad7cdf478c95b14bee9b3931ba361baacfa Reviewed-on: https://go-review.googlesource.com/c/build/+/166218 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-03-07 20:44:41 +03:00
//
// TODO: unexport this and make buildsRepoAtAll return false on too-old
// of repos. The callers in coordinator will need updating.
dashboard, cmd/coordinator, maintner/maintnerd: add support for BuildConfig.MinimumGoVersion field The new BuildConfig.MinimumGoVersion field specifies the minimum Go version the builder is allowed to use. It's useful when some of the builders are too new, and do not support all of the supported Go releases (e.g., openbsd-amd64-64 and freebsd-amd64-12_0 currently require Go 1.11 and don't work on Go 1.10). It only needs to be set when a builder doesn't support all supported Go releases, since we don't typically test unsupported Go releases. To allow cmd/coordinator to use this field and filter out work it receives from maintner/maintnerd's GoFindTryWork RPC call, we add a GoVersion slice to apipb.GerritTryWorkItem, and populate it in maintapi.apiService.GoFindTryWork method. For trybots on the Go repo, the GoVersion field is determined from the branch name. For "release-branch.goX.Y" branches, it parses out the major-minor Go version from the branch name. For master and other branches, it assumes the latest Go release. For trybots on subrepos, we already have the Go version information available, so use it directly. Afterwards, all that's left is to modify newTrySet in cmd/coordinator to make use of BuildConfig.MinimumGoVersion and work.GoVersion to skip builders that are too new for the Go version that needs to be tested. Fixes golang/go#29265 Change-Id: I50b01830647e33e37e9eb8b89e0f2518812fa44f Reviewed-on: https://go-review.googlesource.com/c/155463 Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-01-12 00:44:01 +03:00
MinimumGoVersion types.MajorMinor
dashboard, cmd/coordinator: clean up snapshot code, add snapshot opt-out The snapshot code had an old workaround that's no longer relevant. Remove it. Also, add a BuildConfig.SkipSnapshot bool, and use it for the slow mips builders. Fixes golang/go#19953 Change-Id: I114bb0a524184eaaae5be4715ce63f6adc519c2e Reviewed-on: https://go-review.googlesource.com/40505 Reviewed-by: Sarah Adams <shadams@google.com>
2017-04-13 07:34:42 +03:00
// SkipSnapshot, if true, means to not fetch a tarball
// snapshot of the world post-make.bash from the buildlet (and
// thus to not write it to Google Cloud Storage). This is
// incompatible with sharded tests, and should only be used
// for very slow builders or networks, unable to transfer
// the tarball in under ~5 minutes.
SkipSnapshot bool
dashboard: add a make.bash-only builder on real ARM hardware This is a new builder in prep for the change to the "linux-arm" builder where the GOARCH=arm make.bash will be cross-compiled from a Kubernetes container on fast hardware. Updates golang/go#17105 (cross-compile ARM builders' make.bash) Updates golang/go#17104 (5 minute trybots) Change-Id: Icfd2644d77639f731151abe54839322960418254 Reviewed-on: https://go-review.googlesource.com/29670 Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2016-09-23 02:29:31 +03:00
// StopAfterMake causes the build to stop after the make
// script completes, returning its result as the result of the
// whole build. It does not run or compile any of the tests,
// nor does it write a snapshot of the world to cloud
// storage. This option is only supported for builders whose
// BuildConfig.SplitMakeRun returns true.
StopAfterMake bool
dashboard: unify post-submit & trybot policy in builders.go, not coordinator This removes a bunch of TODOs about scattered policy and moves it all into builders.go. While here, * add a bunch of tests * unexport some things * rename some things * document some things * adjust FreeBSD policy as function of branch (per Go 1.12 being last to support FreeBSD 10.x, and to unblock CL 165801) * set GO_BUILDER_SET_GOPROXY=coordinator for reverse buildlets, which is necessary to remove the oauth2 & build special cases in the config * change Elias Naur's mobile builder to how I think he wants it (he was fighting the old system) * add $HOME on the Solaris smartos builder, which was missing & causing tests to fail lately * make the (currently failing) longtest builder have GOPROXY set * remove an allocation in version.ParseReleaseBranch Fixes golang/go#9603 Updates golang/go#14594 Change-Id: I50a23ad7cdf478c95b14bee9b3931ba361baacfa Reviewed-on: https://go-review.googlesource.com/c/build/+/166218 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-03-07 20:44:41 +03:00
// needsGoProxy is whether this builder should have GOPROXY set.
// Currently this is only for the longtest builder, which needs
// to run cmd/go tests fetching from the network.
needsGoProxy bool
dashboard, cmd/coordinator: add -racecompile builder Change-Id: Idc5756f82081de6b5dfd17036f2bfc9cf11c5659 Reviewed-on: https://go-review.googlesource.com/41414 Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
2017-04-22 02:41:12 +03:00
// InstallRacePackages controls which packages to "go install
// -race <pkgs>" after running make.bash (or equivalent). If
// the builder ends in "-race", the default if non-nil is just
// "std".
InstallRacePackages []string
cmd/coordinator: move git ancestor checking before buildlet creation, generalize This also adds GoDeps to the builders config file and adds a dependency for the ssacheck builder, which currently fails on older release branches with: > Failed on linux-amd64-ssacheck: > https://storage.googleapis.com/go-build-log/1255edaf/linux-amd64-ssacheck_18f7e015.log > > ... > compile: unknown debug key -d dclstack Change-Id: I4ad0a298705feed56f467f3c8d9488f8be2e4af5 Reviewed-on: https://go-review.googlesource.com/43774 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2017-05-20 00:23:14 +03:00
// GoDeps is a list of of git sha1 commits that must be in the
// commit to be tested's history. If absent, this builder is
// not run for that commit.
GoDeps []string
dashboard, cmd/coordinator: move cross compilation config into dashboard package The dashboard package has been gradually absorbing all configuration, keeping cmd/coordinator only about mechanics. But we'd missed this cross compilation bit in earlier cleanups. Move it, and add some tests & docs while I'm at it. Also, disable the linux-arm-nativemake builder by default. It's redundant with linux-arm. They used to be more different when we always cross compiled for linux-arm, but I guess that got disabled at some point. At this poin it serves no purpose except as an opt-in slowbot if the user really wants to run make.bash on scaleway. Change-Id: Iac683ccf4d30214330480f9db39f131263b1bf8a Reviewed-on: https://go-review.googlesource.com/c/build/+/202020 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-10-19 05:49:53 +03:00
// CrossCompileConfig optionally specifies whether and how
// this build is cross compiled.
CrossCompileConfig *CrossCompileConfig
dashboard: refactor mechanism for adjusting cmd/dist test policy Previously, the behavior of ShouldRunDistTest could be overridden by an individual builder by setting a the shouldRunDistTest field to a custom policy implementation. A pitfall of this approach is that it requires each custom policy to reimplement the default cmd/dist test policy as well. If that isn't done when attempting to make a small adjustment to the default policy, the result is that the default policy is ignored. The fasterTrybots policy, as the name suggests, exists to skip some tests in trybot mode. Due to the problem above, it was causing some builders to run more tests in trybot mode than they would if it weren't applied. For example, the freebsd-amd64-12_0 builder should have been skipping the "api" test in trybot mode, but wasn't. This refactor fixes that by modifying the approach for how individual builders can customize the cmd/dist test behavior. Now, a function distTestAdjust is called with the first parameter specifying the default policy decision. That function can adjust the decision as it sees fit, but otherwise it preserves its initial value in situations where no change is intended to be made. For golang/go#38279. Change-Id: I220248f51abfc8e77195ae0c9134a17f7cac9bc2 Reviewed-on: https://go-review.googlesource.com/c/build/+/227777 Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-04-09 19:47:03 +03:00
// distTestAdjust optionally specifies a function that can
// adjust the cmd/dist test policy for this builder.
//
// The BuildConfig.ShouldRunDistTest method implements the
// default cmd/dist test policy, and then calls distTestAdjust
// to adjust that decision further, if distTestAdjust is not nil.
//
// The initial value of the run parameter is what the default
// policy said. The returned value from distTestAdjust is what
// BuildConfig.ShouldRunDistTest reports to the caller.
//
// For example:
//
cmd/coordinator, dashboard: don't skip cmd/dist tests for SlowBots When a user explicitly requests a SlowBot builder via the TRY= syntax, it is a signal that they're willing to wait longer for test results. It's more useful to not skip any tests and give the user confidence that the SlowBot run will catch all issues that would otherwise be caught only by post-submit builders after the CL is submitted. This is done by considering a TryBot that isn't an explicit SlowBot to be a "normal TryBot", and the ShouldRunDistTest policy function is provided a parameter specifying whether it's a normal TryBot (non-SlowBot) run. Tests are only skipped during normal TryBot runs, not SlowBot runs. Fixes golang/go#38279. For golang/go#34501. Change-Id: Ia11fd51c71f9de1089b6b6f4c027014893f224b9 Reviewed-on: https://go-review.googlesource.com/c/build/+/227779 Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-04-09 23:24:30 +03:00
// distTestAdjust: func(run bool, distTest string, isNormalTry bool) bool {
dashboard: refactor mechanism for adjusting cmd/dist test policy Previously, the behavior of ShouldRunDistTest could be overridden by an individual builder by setting a the shouldRunDistTest field to a custom policy implementation. A pitfall of this approach is that it requires each custom policy to reimplement the default cmd/dist test policy as well. If that isn't done when attempting to make a small adjustment to the default policy, the result is that the default policy is ignored. The fasterTrybots policy, as the name suggests, exists to skip some tests in trybot mode. Due to the problem above, it was causing some builders to run more tests in trybot mode than they would if it weren't applied. For example, the freebsd-amd64-12_0 builder should have been skipping the "api" test in trybot mode, but wasn't. This refactor fixes that by modifying the approach for how individual builders can customize the cmd/dist test behavior. Now, a function distTestAdjust is called with the first parameter specifying the default policy decision. That function can adjust the decision as it sees fit, but otherwise it preserves its initial value in situations where no change is intended to be made. For golang/go#38279. Change-Id: I220248f51abfc8e77195ae0c9134a17f7cac9bc2 Reviewed-on: https://go-review.googlesource.com/c/build/+/227777 Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-04-09 19:47:03 +03:00
// // ... modify run from the initial value as needed ...
// return run
// }
//
cmd/coordinator, dashboard: don't skip cmd/dist tests for SlowBots When a user explicitly requests a SlowBot builder via the TRY= syntax, it is a signal that they're willing to wait longer for test results. It's more useful to not skip any tests and give the user confidence that the SlowBot run will catch all issues that would otherwise be caught only by post-submit builders after the CL is submitted. This is done by considering a TryBot that isn't an explicit SlowBot to be a "normal TryBot", and the ShouldRunDistTest policy function is provided a parameter specifying whether it's a normal TryBot (non-SlowBot) run. Tests are only skipped during normal TryBot runs, not SlowBot runs. Fixes golang/go#38279. For golang/go#34501. Change-Id: Ia11fd51c71f9de1089b6b6f4c027014893f224b9 Reviewed-on: https://go-review.googlesource.com/c/build/+/227779 Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-04-09 23:24:30 +03:00
distTestAdjust func(run bool, distTest string, isNormalTry bool) bool
cmd/coordinator, dashboard: add mac race builder, skip redundant trybot tests This adds a darwin-amd64-race builder. Of the four platforms that support the race detector, this was the only one missing a builder. To make up for the extra work, skip some redundant tests that the Mac (and other) builds do. The test directory is really slow, and doesn't have platform-specific code. We have amd64 & 386 test coverage for that directory via the Linux builders, which is fine. No need to do it on some of the heavier VM builders, or for trybots. Fixes golang/go#17674 Change-Id: I888119b3fdd28884b87a59d78746ab1fef6960c7 Reviewed-on: https://go-review.googlesource.com/82395 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2017-12-07 03:40:40 +03:00
cmd/coordinator, dashboard: remove some trybots, shard others wider I'm aiming to have trybot runs finish in under 5 minutes. This CL removes openbsd-386-gce58 and freebsd-386-gce101 from the trybot set. openbsd-386-gce58 is the slowest builder. It has an average speed of 722 seconds (and 95 percentile of 923 seconds) over the past week, and that's sharded over 4 machines. Too slow. It's not worth the resources to keep it as a trybot. It hasn't caught any interesting bugs. This builder will still run, but not as a pre-submit trybot. freebsd-386-gce101 is not slow, but we're removing it to shift its resources to shard other builders wider. The coordinator now supports varying the build sharding width based on whether a build is for a trybot or not. This CL defines separate numbers for each, sharding builds wider as needed for some trybots. freebsd-amd64-gce101 goes from 4 to 5 machines in try runs, and down to 3 when not in try runs. linux-amd64-race gets one more machine during try runs, and one fewer in regular runs. linux-arm goes from 7 machines always, to 3 or 8, depending on whether it's a try run. openbsd-amd64-58 goes from 4 to 3 or 6. windows-amd64-gce goes from 4 to 2 or 6. windows-amd64-race goes from 4 to 2 or 6. darwin-amd64-10_11 goes from 3 to 3 or 4. I'll see how these do over the next few days and readjust as needed. Also in this CL: fix the constants for the expected duration of make.bash, which impact when we schedule the creation of test sharding helper buildlets. We were creating them too early before, wasting resources. Change-Id: I38a9b24841e196f1eb668de058c49af8c1d1c64f Reviewed-on: https://go-review.googlesource.com/29116 Reviewed-by: Quentin Smith <quentin@golang.org>
2016-09-14 01:45:48 +03:00
// numTestHelpers is the number of _additional_ buildlets
// past the first one to help out with sharded tests.
dashboard: try to speed up pre-submit longtest builders The longtest builders are currently primarily post-submit builders, where it's okay for them to be as slow as they need to be in order to provide additional test coverage. In this context, whether they take 40 minutes or 50 makes little difference. The longtest builders are also sometimes requested via SlowBots for changes that are riskier than usual, or otherwise desire additional coverage beyond the normal TryBots. They're also always enabled for CLs to release branches. In such contexts, speeding up SlowBot runs from 40 minutes to 20 or less would be appreciated and in turn help people use longtest SlowBots more frequently. Longtest builders are already configured to use sharded tests. Configure them to use additional helpers to speed up test execution. Try out 3, 5, and 9 helpers to see how much it helps before settling. For golang/go#37439. Change-Id: I425bc0257b7a54bb32c0eb1719fea7ba3f4fd461 Reviewed-on: https://go-review.googlesource.com/c/build/+/268037 Trust: Dmitri Shuralyov <dmitshur@golang.org> Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
2020-11-06 15:55:36 +03:00
// For TryBots and SlowBots, the numTryHelpers value is used,
// unless it's zero, in which case numTestHelpers is used.
cmd/coordinator, dashboard: remove some trybots, shard others wider I'm aiming to have trybot runs finish in under 5 minutes. This CL removes openbsd-386-gce58 and freebsd-386-gce101 from the trybot set. openbsd-386-gce58 is the slowest builder. It has an average speed of 722 seconds (and 95 percentile of 923 seconds) over the past week, and that's sharded over 4 machines. Too slow. It's not worth the resources to keep it as a trybot. It hasn't caught any interesting bugs. This builder will still run, but not as a pre-submit trybot. freebsd-386-gce101 is not slow, but we're removing it to shift its resources to shard other builders wider. The coordinator now supports varying the build sharding width based on whether a build is for a trybot or not. This CL defines separate numbers for each, sharding builds wider as needed for some trybots. freebsd-amd64-gce101 goes from 4 to 5 machines in try runs, and down to 3 when not in try runs. linux-amd64-race gets one more machine during try runs, and one fewer in regular runs. linux-arm goes from 7 machines always, to 3 or 8, depending on whether it's a try run. openbsd-amd64-58 goes from 4 to 3 or 6. windows-amd64-gce goes from 4 to 2 or 6. windows-amd64-race goes from 4 to 2 or 6. darwin-amd64-10_11 goes from 3 to 3 or 4. I'll see how these do over the next few days and readjust as needed. Also in this CL: fix the constants for the expected duration of make.bash, which impact when we schedule the creation of test sharding helper buildlets. We were creating them too early before, wasting resources. Change-Id: I38a9b24841e196f1eb668de058c49af8c1d1c64f Reviewed-on: https://go-review.googlesource.com/29116 Reviewed-by: Quentin Smith <quentin@golang.org>
2016-09-14 01:45:48 +03:00
numTestHelpers int
dashboard: try to speed up pre-submit longtest builders The longtest builders are currently primarily post-submit builders, where it's okay for them to be as slow as they need to be in order to provide additional test coverage. In this context, whether they take 40 minutes or 50 makes little difference. The longtest builders are also sometimes requested via SlowBots for changes that are riskier than usual, or otherwise desire additional coverage beyond the normal TryBots. They're also always enabled for CLs to release branches. In such contexts, speeding up SlowBot runs from 40 minutes to 20 or less would be appreciated and in turn help people use longtest SlowBots more frequently. Longtest builders are already configured to use sharded tests. Configure them to use additional helpers to speed up test execution. Try out 3, 5, and 9 helpers to see how much it helps before settling. For golang/go#37439. Change-Id: I425bc0257b7a54bb32c0eb1719fea7ba3f4fd461 Reviewed-on: https://go-review.googlesource.com/c/build/+/268037 Trust: Dmitri Shuralyov <dmitshur@golang.org> Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
2020-11-06 15:55:36 +03:00
numTryTestHelpers int // For TryBots/SlowBots. If 0, numTestHelpers is used.
cmd/coordinator: test sharding, better status, logs Also gomote updates which came about during the process of developing and debugging this. Change-Id: Ia53d674118a6b99bcdda7062d3b7161279b6ad52 Reviewed-on: https://go-review.googlesource.com/10463 Reviewed-by: Andrew Gerrand <adg@golang.org>
2015-05-28 07:51:25 +03:00
buildlet, cmd/coordinator: GCE quota accounting, fixes And deal with Preemptible resource exhaustion errors. And change all-compile to misc-compile and only do the builders not covered otherwise (Fixes #11073) And make the watcher serve git source. And cache and singleflight fetching of git source. And a million other things. Fixes golang/go#11073 Change-Id: I0f45610f0c6a06bd0c8ba9632b8624e00aeb52fc Reviewed-on: https://go-review.googlesource.com/10750 Reviewed-by: Andrew Gerrand <adg@golang.org>
2015-06-05 04:25:50 +03:00
env []string // extra environment ("key=value") pairs
allScriptArgs []string
cmd/coordinator, dashboard: scale test exec timeouts by GO_TEST_TIMEOUT_SCALE So slow builders can finish their cgo tests, etc. Updates golang/go#29177 etc Change-Id: If5ee8d2201dc53ab506d4f5e31406d6aae240352 Reviewed-on: https://go-review.googlesource.com/c/162540 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-02-14 05:18:06 +03:00
testHostConf *HostConfig // override HostConfig for testing, at least for now
dashboard: move builder configuration out of coordinator into a package The new package will be used by the buildlet command line tool in the future. Also use the metadata package in the coordinator for the master build key, since it was already depending on that package. Might as well use it. Change-Id: I6ad7593787ebba364090951ccbaecb4aa38e1dd6 Reviewed-on: https://go-review.googlesource.com/2920 Reviewed-by: Andrew Gerrand <adg@golang.org>
2015-01-15 23:46:22 +03:00
}
dashboard: unify post-submit & trybot policy in builders.go, not coordinator This removes a bunch of TODOs about scattered policy and moves it all into builders.go. While here, * add a bunch of tests * unexport some things * rename some things * document some things * adjust FreeBSD policy as function of branch (per Go 1.12 being last to support FreeBSD 10.x, and to unblock CL 165801) * set GO_BUILDER_SET_GOPROXY=coordinator for reverse buildlets, which is necessary to remove the oauth2 & build special cases in the config * change Elias Naur's mobile builder to how I think he wants it (he was fighting the old system) * add $HOME on the Solaris smartos builder, which was missing & causing tests to fail lately * make the (currently failing) longtest builder have GOPROXY set * remove an allocation in version.ParseReleaseBranch Fixes golang/go#9603 Updates golang/go#14594 Change-Id: I50a23ad7cdf478c95b14bee9b3931ba361baacfa Reviewed-on: https://go-review.googlesource.com/c/build/+/166218 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-03-07 20:44:41 +03:00
// Env returns the environment variables this builder should run with.
cmd/coordinator: set GO_BUILDER_NAME env var when exec'ing on buildlet This will allow tests to identify that they are running on the continuous build infrastructure and run extra and/or different tests. Fixes golang/go#11346 Change-Id: Iafe25babc764b7e012c05f0b62331b11690bf194 Reviewed-on: https://go-review.googlesource.com/13120 Reviewed-by: Russ Cox <rsc@golang.org>
2015-08-04 05:19:02 +03:00
func (c *BuildConfig) Env() []string {
dashboard: mark some builders as having flaky networks Follow-up to https://golang.org/cl/22852 Change-Id: I5f0cd25248a8892b7d1b0f14e87b12b8b9a8da3a Reviewed-on: https://go-review.googlesource.com/22883 Reviewed-by: Ian Lance Taylor <iant@golang.org>
2016-05-06 21:55:26 +03:00
env := []string{"GO_BUILDER_NAME=" + c.Name}
if c.FlakyNet {
env = append(env, "GO_BUILDER_FLAKY_NET=1")
}
dashboard: use a more consistent definition for longtest builders Previously, it was possible to define a builder whose IsLongTest method would report positively, such that it'd test golang.org/x repos in long test mode, but the main Go repository in short test mode. This would be the case for any builder with "-longtest" suffix if it did not manually include GO_TEST_SHORT=0 in its environment configuration. It's unlikely we would want to do that intentionally, so this refactor makes such misconfiguration less likely by inserting the GO_TEST_SHORT environment variable assignment into the output from Env automatically. Now, a longtest builder is defined in one consistent way: it must have a "-longtest" suffix, so that the IsLongTest method reports positively. For golang/go#39054. For golang/go#29252. For golang/go#12508. Change-Id: Ic24df7b3b7de7db728bba6dc6ad4dd38a2e61e82 Reviewed-on: https://go-review.googlesource.com/c/build/+/233901 Reviewed-by: Carlos Amedee <carlos@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-05-13 21:30:25 +03:00
if c.IsLongTest() {
// Set a private hook in cmd/dist to run main Go repository tests
// without the default -short flag. See golang.org/issue/12508.
env = append(env, "GO_TEST_SHORT=0")
}
cmd/coordinator: finish the scheduler code, at least mostly Optimizations and tuning remain, but this should be tons better than what we had before (random). Updates golang/go#19178 Change-Id: Idb483a4c4209a012814322cc8b37b966ee4681de Reviewed-on: https://go-review.googlesource.com/c/build/+/205078 Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-10-30 06:59:13 +03:00
env = append(env, c.HostConfig().env...)
dashboard: mark some builders as having flaky networks Follow-up to https://golang.org/cl/22852 Change-Id: I5f0cd25248a8892b7d1b0f14e87b12b8b9a8da3a Reviewed-on: https://go-review.googlesource.com/22883 Reviewed-by: Ian Lance Taylor <iant@golang.org>
2016-05-06 21:55:26 +03:00
return append(env, c.env...)
cmd/coordinator: set GO_BUILDER_NAME env var when exec'ing on buildlet This will allow tests to identify that they are running on the continuous build infrastructure and run extra and/or different tests. Fixes golang/go#11346 Change-Id: Iafe25babc764b7e012c05f0b62331b11690bf194 Reviewed-on: https://go-review.googlesource.com/13120 Reviewed-by: Russ Cox <rsc@golang.org>
2015-08-04 05:19:02 +03:00
}
all: add env support, freebsd-386, freebsd-amd64-race, re-enable plan9 Update golang/go#9491 Update golang/go#8639 Change-Id: I1f43c751777f10a6d5870ca9bbb8c2a3430189bf Reviewed-on: https://go-review.googlesource.com/3170 Reviewed-by: Andrew Gerrand <adg@golang.org>
2015-01-22 02:15:48 +03:00
dashboard: unify post-submit & trybot policy in builders.go, not coordinator This removes a bunch of TODOs about scattered policy and moves it all into builders.go. While here, * add a bunch of tests * unexport some things * rename some things * document some things * adjust FreeBSD policy as function of branch (per Go 1.12 being last to support FreeBSD 10.x, and to unblock CL 165801) * set GO_BUILDER_SET_GOPROXY=coordinator for reverse buildlets, which is necessary to remove the oauth2 & build special cases in the config * change Elias Naur's mobile builder to how I think he wants it (he was fighting the old system) * add $HOME on the Solaris smartos builder, which was missing & causing tests to fail lately * make the (currently failing) longtest builder have GOPROXY set * remove an allocation in version.ParseReleaseBranch Fixes golang/go#9603 Updates golang/go#14594 Change-Id: I50a23ad7cdf478c95b14bee9b3931ba361baacfa Reviewed-on: https://go-review.googlesource.com/c/build/+/166218 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-03-07 20:44:41 +03:00
// ModulesEnv returns the extra module-specific environment variables
// to append to this builder as a function of the repo being built
// ("go", "oauth2", "net", etc).
func (c *BuildConfig) ModulesEnv(repo string) (env []string) {
dashboard: set the correct module proxy for EC2 builders This change sets the default module proxy address for EC2 builders. EC2 builders should have the same proxy address set as reverse builders. Fixes golang/go#40914 Change-Id: I4de2ed178170f60fdbe4e2f9aa6f024bb43ee020 Reviewed-on: https://go-review.googlesource.com/c/build/+/249577 Run-TryBot: Carlos Amedee <carlos@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2020-08-20 18:53:58 +03:00
// EC2 and reverse builders should set the public module proxy
// address instead of the internal proxy.
if (c.HostConfig().isEC2 || c.IsReverse()) && repo != "go" {
cmd/coordinator: use proxy.golang.org instead of private Athens Fixes golang/go#31770 Change-Id: Ia21b3c835b3692351d83e00b9a8cb73db2103192 Reviewed-on: https://go-review.googlesource.com/c/build/+/175057 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-05-02 23:32:40 +03:00
env = append(env, "GOPROXY=https://proxy.golang.org")
dashboard: unify post-submit & trybot policy in builders.go, not coordinator This removes a bunch of TODOs about scattered policy and moves it all into builders.go. While here, * add a bunch of tests * unexport some things * rename some things * document some things * adjust FreeBSD policy as function of branch (per Go 1.12 being last to support FreeBSD 10.x, and to unblock CL 165801) * set GO_BUILDER_SET_GOPROXY=coordinator for reverse buildlets, which is necessary to remove the oauth2 & build special cases in the config * change Elias Naur's mobile builder to how I think he wants it (he was fighting the old system) * add $HOME on the Solaris smartos builder, which was missing & causing tests to fail lately * make the (currently failing) longtest builder have GOPROXY set * remove an allocation in version.ParseReleaseBranch Fixes golang/go#9603 Updates golang/go#14594 Change-Id: I50a23ad7cdf478c95b14bee9b3931ba361baacfa Reviewed-on: https://go-review.googlesource.com/c/build/+/166218 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-03-07 20:44:41 +03:00
}
switch repo {
dashboard: set GOPROXY=off for the "go" repo to catch more network usage Updates golang/go#30612 Updates golang/go#30760 Change-Id: I296f4f7a163c1bdd59e839229c0961b21f26da89 Reviewed-on: https://go-review.googlesource.com/c/build/+/167039 Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-03-13 07:44:17 +03:00
case "go":
if !c.OutboundNetworkAllowed() {
env = append(env, "GOPROXY=off")
}
dashboard: set GO111MODULE=on for x/perf CL 191378 removed some +build appengine tags from ./analysis/appengine, so those packages now require GO111MODULE=on in order to resolve the dependency on google.golang.org/appengine. Fixes golang/go#33952 Change-Id: I20c9ba3f42d488265726fda9b1c790e48a1433b3 Reviewed-on: https://go-review.googlesource.com/c/build/+/192333 Run-TryBot: Bryan C. Mills <bcmills@google.com> Reviewed-by: Alexander Rakoczy <alex@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-08-29 21:26:25 +03:00
case "oauth2", "build", "perf", "website":
dashboard: unify post-submit & trybot policy in builders.go, not coordinator This removes a bunch of TODOs about scattered policy and moves it all into builders.go. While here, * add a bunch of tests * unexport some things * rename some things * document some things * adjust FreeBSD policy as function of branch (per Go 1.12 being last to support FreeBSD 10.x, and to unblock CL 165801) * set GO_BUILDER_SET_GOPROXY=coordinator for reverse buildlets, which is necessary to remove the oauth2 & build special cases in the config * change Elias Naur's mobile builder to how I think he wants it (he was fighting the old system) * add $HOME on the Solaris smartos builder, which was missing & causing tests to fail lately * make the (currently failing) longtest builder have GOPROXY set * remove an allocation in version.ParseReleaseBranch Fixes golang/go#9603 Updates golang/go#14594 Change-Id: I50a23ad7cdf478c95b14bee9b3931ba361baacfa Reviewed-on: https://go-review.googlesource.com/c/build/+/166218 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-03-07 20:44:41 +03:00
env = append(env, "GO111MODULE=on")
}
dashboard, cmd/coordinator: add controls to skip subrepo dirs in GOPATH mode This change adds controls to the dashboard package that makes it possible to skip certain directories in subrepos from being tested in GOPATH mode. It uses this control mechanism to skip the gopls directory in x/tools from being tested in GOPATH mode. That directory and everything inside is developed with support for module mode only. All other packages in x/tools continue to be supported both in module and GOPATH modes for the current and previous releases, as per release policy¹. ¹ https://golang.org/doc/devel/release.html#policy Fixes golang/go#34190 Change-Id: Icdb4d0e1419f69c84c9a3f9a33727a09a35599f4 Reviewed-on: https://go-review.googlesource.com/c/build/+/194397 Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-09-09 23:48:08 +03:00
return env
}
cmd/coordinator: finish the scheduler code, at least mostly Optimizations and tuning remain, but this should be tons better than what we had before (random). Updates golang/go#19178 Change-Id: Idb483a4c4209a012814322cc8b37b966ee4681de Reviewed-on: https://go-review.googlesource.com/c/build/+/205078 Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-10-30 06:59:13 +03:00
func (c *BuildConfig) IsReverse() bool { return c.HostConfig().IsReverse }
all: split builder config into builder & host configs Our builders are named of the form "GOOS-GOARCH" or "GOOS-GOARCH-suffix". Over time we've grown many builders. This CL doesn't change that. Builders continue to be named and operate as before. Previously the build configuration file (dashboard/builders.go) made each builder type ("linux-amd64-race", etc) define how to create a host running a buildlet of that type, even though many builders had identical host configs. For example, these builders all share the same host type (a Kubernetes container): linux-amd64 linux-amd64-race linux-386 linux-386-387 And these are the same host type (a GCE VM): windows-amd64-gce windows-amd64-race windows-386-gce This CL creates a new concept of a "hostType" which defines how the buildlet is created (Kube, GCE, Reverse, and how), and then each builder itself references a host type. Users never see the hostType. (except perhaps in gomote list output) But they at least never need to care about them. Reverse buildlets now can only be one hostType at a time, which simplifies things. We were no longer using multiple roles per machine once moving to VMs for OS X. gomote continues to operate as it did previously but its underlying protocol changed and clients will need to be updated. As a new feature, gomote now has a new flag to let you reuse a buildlet host connection for different builder rules if they share the same underlying host type. But users can ignore that. This CL is a long-standing TODO (previously attempted and aborted) and will make many things easier and faster, including the linux-arm cross-compilation effort, and keeping pre-warmed buildlets of VM types ready to go. Updates golang/go#17104 Change-Id: Iad8387f48680424a8441e878a2f4762bf79ea4d2 Reviewed-on: https://go-review.googlesource.com/29551 Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2016-09-22 00:27:37 +03:00
cmd/coordinator: finish the scheduler code, at least mostly Optimizations and tuning remain, but this should be tons better than what we had before (random). Updates golang/go#19178 Change-Id: Idb483a4c4209a012814322cc8b37b966ee4681de Reviewed-on: https://go-review.googlesource.com/c/build/+/205078 Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-10-30 06:59:13 +03:00
func (c *BuildConfig) IsContainer() bool { return c.HostConfig().IsContainer() }
all: rename Kube to Container and IsGCE to IsVM Once containers run on COS instead of Kubernetes, one name (Kube*) is wrong and the other (GCE) is ambiguous. So rename them now to be more specific. No behavior changes. Just renaming in this step, to reduce size of next CL. Updates golang/go#25108 Change-Id: Ib09eb682ef74acbbf6ed50b46074f834ef5e0c0b Reviewed-on: https://go-review.googlesource.com/111639 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-05-05 19:36:05 +03:00
func (c *HostConfig) IsContainer() bool { return c.ContainerImage != "" }
all: split builder config into builder & host configs Our builders are named of the form "GOOS-GOARCH" or "GOOS-GOARCH-suffix". Over time we've grown many builders. This CL doesn't change that. Builders continue to be named and operate as before. Previously the build configuration file (dashboard/builders.go) made each builder type ("linux-amd64-race", etc) define how to create a host running a buildlet of that type, even though many builders had identical host configs. For example, these builders all share the same host type (a Kubernetes container): linux-amd64 linux-amd64-race linux-386 linux-386-387 And these are the same host type (a GCE VM): windows-amd64-gce windows-amd64-race windows-386-gce This CL creates a new concept of a "hostType" which defines how the buildlet is created (Kube, GCE, Reverse, and how), and then each builder itself references a host type. Users never see the hostType. (except perhaps in gomote list output) But they at least never need to care about them. Reverse buildlets now can only be one hostType at a time, which simplifies things. We were no longer using multiple roles per machine once moving to VMs for OS X. gomote continues to operate as it did previously but its underlying protocol changed and clients will need to be updated. As a new feature, gomote now has a new flag to let you reuse a buildlet host connection for different builder rules if they share the same underlying host type. But users can ignore that. This CL is a long-standing TODO (previously attempted and aborted) and will make many things easier and faster, including the linux-arm cross-compilation effort, and keeping pre-warmed buildlets of VM types ready to go. Updates golang/go#17104 Change-Id: Iad8387f48680424a8441e878a2f4762bf79ea4d2 Reviewed-on: https://go-review.googlesource.com/29551 Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2016-09-22 00:27:37 +03:00
cmd/coordinator: finish the scheduler code, at least mostly Optimizations and tuning remain, but this should be tons better than what we had before (random). Updates golang/go#19178 Change-Id: Idb483a4c4209a012814322cc8b37b966ee4681de Reviewed-on: https://go-review.googlesource.com/c/build/+/205078 Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-10-30 06:59:13 +03:00
func (c *BuildConfig) IsVM() bool { return c.HostConfig().IsVM() }
dashboard: add host-linux-arm64-aws builder Add the host-linux-arm64-aws builder. The HostConfig IsVM method has been modified to support the concept that an EC2 VM can be created which runs the buildlet binary in a container. This configuration would have both a VM image and a container image set in the host configuration. For golang/go#36841 Change-Id: I092b48b1dda24d24bcd0b10dff0b2126f5f43017 Reviewed-on: https://go-review.googlesource.com/c/build/+/249118 Run-TryBot: Carlos Amedee <carlos@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2020-08-18 20:32:05 +03:00
// IsVM reports whether the instance running the job is ultimately a VM. Hosts where
// a VM is used only to initiate a container are considered a container, not a VM.
// EC2 instances may be configured to run in containers that are running
// on custom AMIs.
func (c *HostConfig) IsVM() bool {
if c.isEC2 {
return c.VMImage != "" && c.ContainerImage == ""
}
return c.VMImage != ""
}
all: split builder config into builder & host configs Our builders are named of the form "GOOS-GOARCH" or "GOOS-GOARCH-suffix". Over time we've grown many builders. This CL doesn't change that. Builders continue to be named and operate as before. Previously the build configuration file (dashboard/builders.go) made each builder type ("linux-amd64-race", etc) define how to create a host running a buildlet of that type, even though many builders had identical host configs. For example, these builders all share the same host type (a Kubernetes container): linux-amd64 linux-amd64-race linux-386 linux-386-387 And these are the same host type (a GCE VM): windows-amd64-gce windows-amd64-race windows-386-gce This CL creates a new concept of a "hostType" which defines how the buildlet is created (Kube, GCE, Reverse, and how), and then each builder itself references a host type. Users never see the hostType. (except perhaps in gomote list output) But they at least never need to care about them. Reverse buildlets now can only be one hostType at a time, which simplifies things. We were no longer using multiple roles per machine once moving to VMs for OS X. gomote continues to operate as it did previously but its underlying protocol changed and clients will need to be updated. As a new feature, gomote now has a new flag to let you reuse a buildlet host connection for different builder rules if they share the same underlying host type. But users can ignore that. This CL is a long-standing TODO (previously attempted and aborted) and will make many things easier and faster, including the linux-arm cross-compilation effort, and keeping pre-warmed buildlets of VM types ready to go. Updates golang/go#17104 Change-Id: Iad8387f48680424a8441e878a2f4762bf79ea4d2 Reviewed-on: https://go-review.googlesource.com/29551 Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2016-09-22 00:27:37 +03:00
dashboard: create buildlet client package, move coordinator code into it Operation Packification, step 2 of tons. Eventually the buildlet client binary will use this stuff now. Change-Id: I4cf5f3e6beb9e56bdc795ed513ce6daaf61425e3 Reviewed-on: https://go-review.googlesource.com/2921 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2015-01-16 03:29:16 +03:00
func (c *BuildConfig) GOOS() string { return c.Name[:strings.Index(c.Name, "-")] }
func (c *BuildConfig) GOARCH() string {
arch := c.Name[strings.Index(c.Name, "-")+1:]
i := strings.Index(arch, "-")
if i == -1 {
return arch
}
return arch[:i]
}
cmd/coordinator: add SlowBots support, opt-in to different slow trybots This parses TRY= comments to opt-in to slower/difference trybots. This needs some docs/UI work yet. Updates golang/go#34501 Change-Id: I13a835520d7ac341bc86139b0a2118235bc83e60 Reviewed-on: https://go-review.googlesource.com/c/build/+/201338 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-10-16 09:02:29 +03:00
// MatchesSlowBotTerm reports whether some provided term from a
// TRY=... comment on a Run-TryBot+1 vote on Gerrit should match this
// build config.
func (c *BuildConfig) MatchesSlowBotTerm(term string) bool {
return term != "" && (term == c.Name || slowBotAliases[term] == c.Name)
}
cmd/coordinator: test sharding, better status, logs Also gomote updates which came about during the process of developing and debugging this. Change-Id: Ia53d674118a6b99bcdda7062d3b7161279b6ad52 Reviewed-on: https://go-review.googlesource.com/10463 Reviewed-by: Andrew Gerrand <adg@golang.org>
2015-05-28 07:51:25 +03:00
// FilePathJoin is mostly like filepath.Join (without the cleaning) except
// it uses the path separator of c.GOOS instead of the host system's.
func (c *BuildConfig) FilePathJoin(x ...string) string {
if c.GOOS() == "windows" {
return strings.Join(x, "\\")
}
return strings.Join(x, "/")
}
cmd/coordinator, dashboard: scale test exec timeouts by GO_TEST_TIMEOUT_SCALE So slow builders can finish their cgo tests, etc. Updates golang/go#29177 etc Change-Id: If5ee8d2201dc53ab506d4f5e31406d6aae240352 Reviewed-on: https://go-review.googlesource.com/c/162540 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-02-14 05:18:06 +03:00
// DistTestsExecTimeout returns how long the coordinator should wait
// for a cmd/dist test execution to run the provided dist test names.
func (c *BuildConfig) DistTestsExecTimeout(distTests []string) time.Duration {
// TODO: consider using distTests? We never did before, but
// now we have the TestStats in the coordinator. Pass in a
// *buildstats.TestStats and use historical data times some
// fudge factor? For now just use the old 20 minute limit
// we've used since 2014, but scale it by the
// GO_TEST_TIMEOUT_SCALE for the super slow builders which
// struggle with, say, the cgo tests. (which should be broken
// up into separate dist tests or shards, like the test/ dir
// was)
d := 20 * time.Minute
d *= time.Duration(c.timeoutScale())
return d
}
// timeoutScale returns this builder's GO_TEST_TIMEOUT_SCALE value, or 1.
func (c *BuildConfig) timeoutScale() int {
const pfx = "GO_TEST_TIMEOUT_SCALE="
cmd/coordinator: finish the scheduler code, at least mostly Optimizations and tuning remain, but this should be tons better than what we had before (random). Updates golang/go#19178 Change-Id: Idb483a4c4209a012814322cc8b37b966ee4681de Reviewed-on: https://go-review.googlesource.com/c/build/+/205078 Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-10-30 06:59:13 +03:00
for _, env := range [][]string{c.env, c.HostConfig().env} {
cmd/coordinator, dashboard: scale test exec timeouts by GO_TEST_TIMEOUT_SCALE So slow builders can finish their cgo tests, etc. Updates golang/go#29177 etc Change-Id: If5ee8d2201dc53ab506d4f5e31406d6aae240352 Reviewed-on: https://go-review.googlesource.com/c/162540 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-02-14 05:18:06 +03:00
for _, kv := range env {
if strings.HasPrefix(kv, pfx) {
if n, err := strconv.Atoi(kv[len(pfx):]); err == nil && n > 0 {
return n
}
}
}
}
return 1
}
cmd/coordinator: finish the scheduler code, at least mostly Optimizations and tuning remain, but this should be tons better than what we had before (random). Updates golang/go#19178 Change-Id: Idb483a4c4209a012814322cc8b37b966ee4681de Reviewed-on: https://go-review.googlesource.com/c/build/+/205078 Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-10-30 06:59:13 +03:00
// HostConfig returns the host configuration of c.
func (c *BuildConfig) HostConfig() *HostConfig {
cmd/coordinator, dashboard: scale test exec timeouts by GO_TEST_TIMEOUT_SCALE So slow builders can finish their cgo tests, etc. Updates golang/go#29177 etc Change-Id: If5ee8d2201dc53ab506d4f5e31406d6aae240352 Reviewed-on: https://go-review.googlesource.com/c/162540 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-02-14 05:18:06 +03:00
if c.testHostConf != nil {
return c.testHostConf
}
all: split builder config into builder & host configs Our builders are named of the form "GOOS-GOARCH" or "GOOS-GOARCH-suffix". Over time we've grown many builders. This CL doesn't change that. Builders continue to be named and operate as before. Previously the build configuration file (dashboard/builders.go) made each builder type ("linux-amd64-race", etc) define how to create a host running a buildlet of that type, even though many builders had identical host configs. For example, these builders all share the same host type (a Kubernetes container): linux-amd64 linux-amd64-race linux-386 linux-386-387 And these are the same host type (a GCE VM): windows-amd64-gce windows-amd64-race windows-386-gce This CL creates a new concept of a "hostType" which defines how the buildlet is created (Kube, GCE, Reverse, and how), and then each builder itself references a host type. Users never see the hostType. (except perhaps in gomote list output) But they at least never need to care about them. Reverse buildlets now can only be one hostType at a time, which simplifies things. We were no longer using multiple roles per machine once moving to VMs for OS X. gomote continues to operate as it did previously but its underlying protocol changed and clients will need to be updated. As a new feature, gomote now has a new flag to let you reuse a buildlet host connection for different builder rules if they share the same underlying host type. But users can ignore that. This CL is a long-standing TODO (previously attempted and aborted) and will make many things easier and faster, including the linux-arm cross-compilation effort, and keeping pre-warmed buildlets of VM types ready to go. Updates golang/go#17104 Change-Id: Iad8387f48680424a8441e878a2f4762bf79ea4d2 Reviewed-on: https://go-review.googlesource.com/29551 Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2016-09-22 00:27:37 +03:00
if c, ok := Hosts[c.HostType]; ok {
return c
}
panic(fmt.Sprintf("missing buildlet config for buildlet %q", c.Name))
}
dashboard: fix docs of BuildsRepoPostSubmit, GoBootstrapURL There were some copy-paste mistakes affecting the documentation of these two identifiers. This change corrects that. Change-Id: Ic619f19c4a659a9357c816ff7fb6b0cd7b3bccd1 Reviewed-on: https://go-review.googlesource.com/c/build/+/169437 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-03-26 20:45:16 +03:00
// GoBootstrapURL returns the URL of a built Go 1.4+ tar.gz for the
// build configuration type c, or empty string if there isn't one.
dashboard: rename Go14URL to goBootstrapURLTmpl and provide accessor Also update all users, and rename another template field. Includes changes to coordinator, gomote, and release. Change-Id: I1c4408eadbcb83d61063a910dfa18cc395952bc2 Reviewed-on: https://go-review.googlesource.com/20976 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2016-03-22 01:05:52 +03:00
func (c *BuildConfig) GoBootstrapURL(e *buildenv.Environment) string {
cmd/coordinator: finish the scheduler code, at least mostly Optimizations and tuning remain, but this should be tons better than what we had before (random). Updates golang/go#19178 Change-Id: Idb483a4c4209a012814322cc8b37b966ee4681de Reviewed-on: https://go-review.googlesource.com/c/build/+/205078 Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-10-30 06:59:13 +03:00
return strings.Replace(c.HostConfig().goBootstrapURLTmpl, "$BUCKET", e.BuildletBucket, 1)
dashboard: rename Go14URL to goBootstrapURLTmpl and provide accessor Also update all users, and rename another template field. Includes changes to coordinator, gomote, and release. Change-Id: I1c4408eadbcb83d61063a910dfa18cc395952bc2 Reviewed-on: https://go-review.googlesource.com/20976 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2016-03-22 01:05:52 +03:00
}
build: convert most Linux builders to be VM-based We still use Dockerfiles to describe them, but then a new tool (docker2boot) converts them into VM images suitable for booting on GCE, running the buildlet like all the other operating systems. Things are easier if everything acts the same way. Note that since we're no longer moving around Docker images, the image size and the layer accumulation cruft no longer matters. We can now have Dockerfile lines like "RUN rm -rf /usr/share/doc" and it actually results in a smaller VM image, since we just "docker export" the files out of it to create the VM image. This doesn't yet convert the clang, sid, or nacl builders. The coordinator still runs those under Docker directly. A future change will convert those to VM images as well. Change-Id: Iedb136ae3daf888c955eb843bdcc9a638d08f5e9 Reviewed-on: https://go-review.googlesource.com/3341 Reviewed-by: Andrew Gerrand <adg@golang.org>
2015-01-28 01:22:21 +03:00
// BuildletBinaryURL returns the public URL of this builder's buildlet.
all: split builder config into builder & host configs Our builders are named of the form "GOOS-GOARCH" or "GOOS-GOARCH-suffix". Over time we've grown many builders. This CL doesn't change that. Builders continue to be named and operate as before. Previously the build configuration file (dashboard/builders.go) made each builder type ("linux-amd64-race", etc) define how to create a host running a buildlet of that type, even though many builders had identical host configs. For example, these builders all share the same host type (a Kubernetes container): linux-amd64 linux-amd64-race linux-386 linux-386-387 And these are the same host type (a GCE VM): windows-amd64-gce windows-amd64-race windows-386-gce This CL creates a new concept of a "hostType" which defines how the buildlet is created (Kube, GCE, Reverse, and how), and then each builder itself references a host type. Users never see the hostType. (except perhaps in gomote list output) But they at least never need to care about them. Reverse buildlets now can only be one hostType at a time, which simplifies things. We were no longer using multiple roles per machine once moving to VMs for OS X. gomote continues to operate as it did previously but its underlying protocol changed and clients will need to be updated. As a new feature, gomote now has a new flag to let you reuse a buildlet host connection for different builder rules if they share the same underlying host type. But users can ignore that. This CL is a long-standing TODO (previously attempted and aborted) and will make many things easier and faster, including the linux-arm cross-compilation effort, and keeping pre-warmed buildlets of VM types ready to go. Updates golang/go#17104 Change-Id: Iad8387f48680424a8441e878a2f4762bf79ea4d2 Reviewed-on: https://go-review.googlesource.com/29551 Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2016-09-22 00:27:37 +03:00
func (c *HostConfig) BuildletBinaryURL(e *buildenv.Environment) string {
dashboard: rename Go14URL to goBootstrapURLTmpl and provide accessor Also update all users, and rename another template field. Includes changes to coordinator, gomote, and release. Change-Id: I1c4408eadbcb83d61063a910dfa18cc395952bc2 Reviewed-on: https://go-review.googlesource.com/20976 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2016-03-22 01:05:52 +03:00
tmpl := c.buildletURLTmpl
all: consolidate configuration for coordinator and gomote buildenv.Environment type defines configuration options: - Coordinator uses the GCE project name to lookup config. A custom config name can be provided at runtime to override. - The conventional prod and stage project names ('symbolic-datum-552' and 'go-dashboard-dev') map to prod and staging configuration structs. - Production and staging status is explicitly defined in configuration. - GCS bucket names for buildlet, logs, and snapshots are configurable. Change-Id: I7e6d7874eb0bdfe35dbdd5fcf6212ab50d576b88 Reviewed-on: https://go-review.googlesource.com/19502 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2016-02-15 02:59:59 +03:00
return strings.Replace(tmpl, "$BUCKET", e.BuildletBucket, 1)
build: convert most Linux builders to be VM-based We still use Dockerfiles to describe them, but then a new tool (docker2boot) converts them into VM images suitable for booting on GCE, running the buildlet like all the other operating systems. Things are easier if everything acts the same way. Note that since we're no longer moving around Docker images, the image size and the layer accumulation cruft no longer matters. We can now have Dockerfile lines like "RUN rm -rf /usr/share/doc" and it actually results in a smaller VM image, since we just "docker export" the files out of it to create the VM image. This doesn't yet convert the clang, sid, or nacl builders. The coordinator still runs those under Docker directly. A future change will convert those to VM images as well. Change-Id: Iedb136ae3daf888c955eb843bdcc9a638d08f5e9 Reviewed-on: https://go-review.googlesource.com/3341 Reviewed-by: Andrew Gerrand <adg@golang.org>
2015-01-28 01:22:21 +03:00
}
cmd/coordinator: shard linux race builder Fixes golang/go#11074 Change-Id: I9401122e4d7febd3e38a80be89fcb2e03969f383 Reviewed-on: https://go-review.googlesource.com/11168 Reviewed-by: Andrew Gerrand <adg@golang.org>
2015-06-17 18:22:14 +03:00
func (c *BuildConfig) IsRace() bool {
return strings.HasSuffix(c.Name, "-race")
}
dashboard: use a more consistent definition for longtest builders Previously, it was possible to define a builder whose IsLongTest method would report positively, such that it'd test golang.org/x repos in long test mode, but the main Go repository in short test mode. This would be the case for any builder with "-longtest" suffix if it did not manually include GO_TEST_SHORT=0 in its environment configuration. It's unlikely we would want to do that intentionally, so this refactor makes such misconfiguration less likely by inserting the GO_TEST_SHORT environment variable assignment into the output from Env automatically. Now, a longtest builder is defined in one consistent way: it must have a "-longtest" suffix, so that the IsLongTest method reports positively. For golang/go#39054. For golang/go#29252. For golang/go#12508. Change-Id: Ic24df7b3b7de7db728bba6dc6ad4dd38a2e61e82 Reviewed-on: https://go-review.googlesource.com/c/build/+/233901 Reviewed-by: Carlos Amedee <carlos@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-05-13 21:30:25 +03:00
// IsLongTest reports whether this is a longtest builder.
// A longtest builder runs tests without the -short flag.
//
// A builder is considered to be a longtest builder
// if and only if its name ends with "-longtest".
dashboard, cmd/coordinator: various fixes * add support for longtest builders for subrepos. * fix race builders for subrepos (they weren't passing the -race flag). * adjust the policy for the js-wasm builders to build fewer subrepos where it'll never work or isn't worth it. * fix the android emu builders which disappeared because an empty string was being passed to buildsRepoAtAll. In some places in the code an empty string for goBranch for the "go" repo meant the same as branch, but I forgot that in the new code, so an old caller was confusing the new config hooks. Rather than make all policy funcs be aware of both ways, the new code in this CL now maps an empty string to the same as the repo's branch when the repo == "go". Adds a test too. * fix some outdated comments. Change-Id: Icf3fb85e5542a4d314443b59d02517b306ef46b7 Reviewed-on: https://go-review.googlesource.com/c/build/+/166897 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-03-11 21:49:56 +03:00
func (c *BuildConfig) IsLongTest() bool {
return strings.HasSuffix(c.Name, "-longtest")
}
dashboard: set GOPROXY=off for the "go" repo to catch more network usage Updates golang/go#30612 Updates golang/go#30760 Change-Id: I296f4f7a163c1bdd59e839229c0961b21f26da89 Reviewed-on: https://go-review.googlesource.com/c/build/+/167039 Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-03-13 07:44:17 +03:00
// OutboundNetworkAllowed reports whether this builder should be
// allowed to make outbound network requests. This is only enforced
// on some builders. (Currently most Linux ones)
func (c *BuildConfig) OutboundNetworkAllowed() bool {
dashboard: remove vetall, update misc-compile builders Updates golang/go#31916 Change-Id: I38c08955bdb4ff2b0963d5c91c6e8f78267b8004 Reviewed-on: https://go-review.googlesource.com/c/build/+/176599 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-05-10 22:28:40 +03:00
return c.IsLongTest()
dashboard: set GOPROXY=off for the "go" repo to catch more network usage Updates golang/go#30612 Updates golang/go#30760 Change-Id: I296f4f7a163c1bdd59e839229c0961b21f26da89 Reviewed-on: https://go-review.googlesource.com/c/build/+/167039 Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-03-13 07:44:17 +03:00
}
dashboard, cmd/coordinator: add -racecompile builder Change-Id: Idc5756f82081de6b5dfd17036f2bfc9cf11c5659 Reviewed-on: https://go-review.googlesource.com/41414 Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
2017-04-22 02:41:12 +03:00
func (c *BuildConfig) GoInstallRacePackages() []string {
if c.InstallRacePackages != nil {
return append([]string(nil), c.InstallRacePackages...)
}
if c.IsRace() {
return []string{"std"}
}
return nil
}
dashboard: move buildlet exec code from coordinator to client package Change-Id: I778ac78ed02be9f67436ec045a3816dfc24afda3 Reviewed-on: https://go-review.googlesource.com/2923 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2015-01-16 20:54:03 +03:00
// AllScript returns the relative path to the operating system's script to
// do the build and run its standard set of tests.
// Example values are "src/all.bash", "src/all.bat", "src/all.rc".
func (c *BuildConfig) AllScript() string {
dashboard: add sanity check Because almost all the BuildConfig methods indirectly call AllScript, add in a little defense against accident creation of BuildConfig zero values (of the value type, not nil pointers). This can happen by doing a lookup in a map[string]dashboard.BuildConfig and then calling a method on it. We should probably move everything to pointer values of BuildConfig. There aren't many value types remaining. This paranoia isn't a result of any bug yet, but rather than the result of discussion on CL 40477. Change-Id: Ic273d4de28ac8930b56bc232bf77a9ba39f99fad Reviewed-on: https://go-review.googlesource.com/40497 Reviewed-by: Herbie Ong <herbie@google.com> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2017-04-13 00:37:13 +03:00
if c.Name == "" {
panic("bogus BuildConfig")
}
cmd/coordinator: shard linux race builder Fixes golang/go#11074 Change-Id: I9401122e4d7febd3e38a80be89fcb2e03969f383 Reviewed-on: https://go-review.googlesource.com/11168 Reviewed-by: Andrew Gerrand <adg@golang.org>
2015-06-17 18:22:14 +03:00
if c.IsRace() {
all: add env support, freebsd-386, freebsd-amd64-race, re-enable plan9 Update golang/go#9491 Update golang/go#8639 Change-Id: I1f43c751777f10a6d5870ca9bbb8c2a3430189bf Reviewed-on: https://go-review.googlesource.com/3170 Reviewed-by: Andrew Gerrand <adg@golang.org>
2015-01-22 02:15:48 +03:00
if strings.HasPrefix(c.Name, "windows-") {
return "src/race.bat"
}
return "src/race.bash"
}
dashboard: move buildlet exec code from coordinator to client package Change-Id: I778ac78ed02be9f67436ec045a3816dfc24afda3 Reviewed-on: https://go-review.googlesource.com/2923 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2015-01-16 20:54:03 +03:00
if strings.HasPrefix(c.Name, "windows-") {
return "src/all.bat"
}
if strings.HasPrefix(c.Name, "plan9-") {
return "src/all.rc"
}
cmd/coordinator: shard the misc-compile builder over 4 builders Now that it's on Kubernetes, it's fast and cheap to shard it wider. Change-Id: I2858f4c417ef26661575a348fbfc47f4c25d81a2 Reviewed-on: https://go-review.googlesource.com/22795 Reviewed-by: Andrew Gerrand <adg@golang.org>
2016-05-05 03:42:49 +03:00
if strings.HasPrefix(c.Name, "misc-compile") {
dashboard: add all-compile builder Runs buildall.bash only. Change-Id: I1a68d54569ccb511d956c5b181e463503353d4d4 Reviewed-on: https://go-review.googlesource.com/9512 Reviewed-by: David Crawshaw <crawshaw@golang.org>
2015-04-30 00:44:46 +03:00
return "src/buildall.bash"
}
dashboard: move buildlet exec code from coordinator to client package Change-Id: I778ac78ed02be9f67436ec045a3816dfc24afda3 Reviewed-on: https://go-review.googlesource.com/2923 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2015-01-16 20:54:03 +03:00
return "src/all.bash"
}
dashboard, cmd/coordinator: only split make & run for builders where it'll work Change-Id: I1fdc27ea4adcc83fbd736d2bd1a34a07ca36a50c Reviewed-on: https://go-review.googlesource.com/10423 Reviewed-by: Andrew Gerrand <adg@golang.org>
2015-05-27 21:51:27 +03:00
// SplitMakeRun reports whether the coordinator should first compile
// (using c.MakeScript), then snapshot, then run the tests (ideally
dashboard: unify post-submit & trybot policy in builders.go, not coordinator This removes a bunch of TODOs about scattered policy and moves it all into builders.go. While here, * add a bunch of tests * unexport some things * rename some things * document some things * adjust FreeBSD policy as function of branch (per Go 1.12 being last to support FreeBSD 10.x, and to unblock CL 165801) * set GO_BUILDER_SET_GOPROXY=coordinator for reverse buildlets, which is necessary to remove the oauth2 & build special cases in the config * change Elias Naur's mobile builder to how I think he wants it (he was fighting the old system) * add $HOME on the Solaris smartos builder, which was missing & causing tests to fail lately * make the (currently failing) longtest builder have GOPROXY set * remove an allocation in version.ParseReleaseBranch Fixes golang/go#9603 Updates golang/go#14594 Change-Id: I50a23ad7cdf478c95b14bee9b3931ba361baacfa Reviewed-on: https://go-review.googlesource.com/c/build/+/166218 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-03-07 20:44:41 +03:00
// sharded) using cmd/dist test.
dashboard, cmd/coordinator: only split make & run for builders where it'll work Change-Id: I1fdc27ea4adcc83fbd736d2bd1a34a07ca36a50c Reviewed-on: https://go-review.googlesource.com/10423 Reviewed-by: Andrew Gerrand <adg@golang.org>
2015-05-27 21:51:27 +03:00
// Eventually this function should always return true (and then be deleted)
// but for now we've only set up the scripts and verified that the main
// configurations work.
func (c *BuildConfig) SplitMakeRun() bool {
switch c.AllScript() {
build: move Linux and nacl builders to Kubernetes, add ssacheck trybot And shard the nacl builders. Fixes golang/go#15242 Fixes golang/go#12546 Change-Id: I3311678dbb64a76026f75f5691cffac0f25d976b Reviewed-on: https://go-review.googlesource.com/22780 Reviewed-by: Andrew Gerrand <adg@golang.org> Reviewed-by: Ross Light <light@google.com>
2016-05-04 20:47:12 +03:00
case "src/all.bash", "src/all.bat",
"src/race.bash", "src/race.bat",
dashboard, env/linux-x86-nacl: remove nacl builders and supporting files Go 1.13 was the last release with nacl, and it's not supported by now. For golang/go#30439. Change-Id: Ic690aa388ab4505d5e755bbe40fed850a7300c69 Reviewed-on: https://go-review.googlesource.com/c/build/+/260200 Trust: Dmitri Shuralyov <dmitshur@golang.org> Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
2020-10-07 18:12:00 +03:00
"src/all.rc":
dashboard, cmd/coordinator: only split make & run for builders where it'll work Change-Id: I1fdc27ea4adcc83fbd736d2bd1a34a07ca36a50c Reviewed-on: https://go-review.googlesource.com/10423 Reviewed-by: Andrew Gerrand <adg@golang.org>
2015-05-27 21:51:27 +03:00
// These we've verified to work.
return true
}
dashboard: remove iostest special case There are no tethered iOS builders left. If we add some in the future, they should use all.bash directly. Change-Id: Ieef7454af706c7e99c0d10fc37a337fb2c3ba130 Reviewed-on: https://go-review.googlesource.com/c/build/+/259357 Run-TryBot: Elias Naur <mail@eliasnaur.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Trust: Elias Naur <mail@eliasnaur.com>
2020-10-03 12:22:43 +03:00
// TODO(bradfitz): buildall.bash should really just be N small container
// jobs instead of a "buildall.bash". Then we can delete this whole method.
dashboard, cmd/coordinator: only split make & run for builders where it'll work Change-Id: I1fdc27ea4adcc83fbd736d2bd1a34a07ca36a50c Reviewed-on: https://go-review.googlesource.com/10423 Reviewed-by: Andrew Gerrand <adg@golang.org>
2015-05-27 21:51:27 +03:00
return false
}
dashboard: unify post-submit & trybot policy in builders.go, not coordinator This removes a bunch of TODOs about scattered policy and moves it all into builders.go. While here, * add a bunch of tests * unexport some things * rename some things * document some things * adjust FreeBSD policy as function of branch (per Go 1.12 being last to support FreeBSD 10.x, and to unblock CL 165801) * set GO_BUILDER_SET_GOPROXY=coordinator for reverse buildlets, which is necessary to remove the oauth2 & build special cases in the config * change Elias Naur's mobile builder to how I think he wants it (he was fighting the old system) * add $HOME on the Solaris smartos builder, which was missing & causing tests to fail lately * make the (currently failing) longtest builder have GOPROXY set * remove an allocation in version.ParseReleaseBranch Fixes golang/go#9603 Updates golang/go#14594 Change-Id: I50a23ad7cdf478c95b14bee9b3931ba361baacfa Reviewed-on: https://go-review.googlesource.com/c/build/+/166218 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-03-07 20:44:41 +03:00
func (c *BuildConfig) IsTryOnly() bool { return c.tryOnly }
func (c *BuildConfig) NeedsGoProxy() bool { return c.needsGoProxy }
dashboard: fix docs of BuildsRepoPostSubmit, GoBootstrapURL There were some copy-paste mistakes affecting the documentation of these two identifiers. This change corrects that. Change-Id: Ic619f19c4a659a9357c816ff7fb6b0cd7b3bccd1 Reviewed-on: https://go-review.googlesource.com/c/build/+/169437 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-03-26 20:45:16 +03:00
// BuildsRepoPostSubmit reports whether the build configuration type c
dashboard: unify post-submit & trybot policy in builders.go, not coordinator This removes a bunch of TODOs about scattered policy and moves it all into builders.go. While here, * add a bunch of tests * unexport some things * rename some things * document some things * adjust FreeBSD policy as function of branch (per Go 1.12 being last to support FreeBSD 10.x, and to unblock CL 165801) * set GO_BUILDER_SET_GOPROXY=coordinator for reverse buildlets, which is necessary to remove the oauth2 & build special cases in the config * change Elias Naur's mobile builder to how I think he wants it (he was fighting the old system) * add $HOME on the Solaris smartos builder, which was missing & causing tests to fail lately * make the (currently failing) longtest builder have GOPROXY set * remove an allocation in version.ParseReleaseBranch Fixes golang/go#9603 Updates golang/go#14594 Change-Id: I50a23ad7cdf478c95b14bee9b3931ba361baacfa Reviewed-on: https://go-review.googlesource.com/c/build/+/166218 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-03-07 20:44:41 +03:00
// should build the given repo ("go", "net", etc) and branch
// ("master", "release-branch.go1.12") as a post-submit build
// that shows up on https://build.golang.org/.
func (c *BuildConfig) BuildsRepoPostSubmit(repo, branch, goBranch string) bool {
if c.tryOnly {
dashboard: build sub-repos on more builders Change-Id: I57c12f427448281d71b8507ed950038209134143 Reviewed-on: https://go-review.googlesource.com/10915 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2015-06-11 20:01:24 +03:00
return false
}
dashboard: unify post-submit & trybot policy in builders.go, not coordinator This removes a bunch of TODOs about scattered policy and moves it all into builders.go. While here, * add a bunch of tests * unexport some things * rename some things * document some things * adjust FreeBSD policy as function of branch (per Go 1.12 being last to support FreeBSD 10.x, and to unblock CL 165801) * set GO_BUILDER_SET_GOPROXY=coordinator for reverse buildlets, which is necessary to remove the oauth2 & build special cases in the config * change Elias Naur's mobile builder to how I think he wants it (he was fighting the old system) * add $HOME on the Solaris smartos builder, which was missing & causing tests to fail lately * make the (currently failing) longtest builder have GOPROXY set * remove an allocation in version.ParseReleaseBranch Fixes golang/go#9603 Updates golang/go#14594 Change-Id: I50a23ad7cdf478c95b14bee9b3931ba361baacfa Reviewed-on: https://go-review.googlesource.com/c/build/+/166218 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-03-07 20:44:41 +03:00
return c.buildsRepoAtAll(repo, branch, goBranch)
cmd/coordinator: build sub-repositories Fixes golang/go#9506 Change-Id: Iebfb52386b5ba3df7d058f266e227d634d210ec6 Reviewed-on: https://go-review.googlesource.com/10732 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2015-06-05 02:45:17 +03:00
}
dashboard: unify post-submit & trybot policy in builders.go, not coordinator This removes a bunch of TODOs about scattered policy and moves it all into builders.go. While here, * add a bunch of tests * unexport some things * rename some things * document some things * adjust FreeBSD policy as function of branch (per Go 1.12 being last to support FreeBSD 10.x, and to unblock CL 165801) * set GO_BUILDER_SET_GOPROXY=coordinator for reverse buildlets, which is necessary to remove the oauth2 & build special cases in the config * change Elias Naur's mobile builder to how I think he wants it (he was fighting the old system) * add $HOME on the Solaris smartos builder, which was missing & causing tests to fail lately * make the (currently failing) longtest builder have GOPROXY set * remove an allocation in version.ParseReleaseBranch Fixes golang/go#9603 Updates golang/go#14594 Change-Id: I50a23ad7cdf478c95b14bee9b3931ba361baacfa Reviewed-on: https://go-review.googlesource.com/c/build/+/166218 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-03-07 20:44:41 +03:00
// BuildsRepoTryBot reports whether the build configuration type c
// should build the given repo ("go", "net", etc) and branch
// ("master", "release-branch.go1.12") as a trybot.
func (c *BuildConfig) BuildsRepoTryBot(repo, branch, goBranch string) bool {
return c.tryBot != nil && c.tryBot(repo, branch, goBranch) && c.buildsRepoAtAll(repo, branch, goBranch)
dashboard, cmd/coordinator: move more config out of coordinator into dashboard And with the increased flexibility, enable trybots for more repos for the "sys" project. Details: https://github.com/golang/go/issues/24893#issuecomment-433517519 Fixes golang/go#24893 Change-Id: I89ce6f219be986201332f8fad11c774bdd6246da Reviewed-on: https://go-review.googlesource.com/c/145157 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2018-10-26 22:21:58 +03:00
}
cmd/coordinator: add a JSON mode to the /builders handler And add a test that indirectly verifies that the BuildConfig and HostConfig are JSON serializable. They weren't due to an exported func. But that exported func shouldn't be exported, so unexport it and move more policy into dashboard/builders.go. (There's been a number of recent cleanup CLs to move all policy into dashboard/builders.go instead of sprinkled all over the coordinator) A future CL will use this JSON in gomote create. Updates golang/go#30929 Change-Id: I726eaf6a4f3eeaab27d31e2642cb7642111ccd67 Reviewed-on: https://go-review.googlesource.com/c/build/+/168341 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-03-19 20:17:23 +03:00
// ShouldRunDistTest reports whether the named cmd/dist test should be
cmd/coordinator, dashboard: don't skip cmd/dist tests for SlowBots When a user explicitly requests a SlowBot builder via the TRY= syntax, it is a signal that they're willing to wait longer for test results. It's more useful to not skip any tests and give the user confidence that the SlowBot run will catch all issues that would otherwise be caught only by post-submit builders after the CL is submitted. This is done by considering a TryBot that isn't an explicit SlowBot to be a "normal TryBot", and the ShouldRunDistTest policy function is provided a parameter specifying whether it's a normal TryBot (non-SlowBot) run. Tests are only skipped during normal TryBot runs, not SlowBot runs. Fixes golang/go#38279. For golang/go#34501. Change-Id: Ia11fd51c71f9de1089b6b6f4c027014893f224b9 Reviewed-on: https://go-review.googlesource.com/c/build/+/227779 Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-04-09 23:24:30 +03:00
// run for this build config. The isNormalTry parameter is whether this
// is for a normal TryBot (non-SlowBot) run.
cmd/coordinator: add a JSON mode to the /builders handler And add a test that indirectly verifies that the BuildConfig and HostConfig are JSON serializable. They weren't due to an exported func. But that exported func shouldn't be exported, so unexport it and move more policy into dashboard/builders.go. (There's been a number of recent cleanup CLs to move all policy into dashboard/builders.go instead of sprinkled all over the coordinator) A future CL will use this JSON in gomote create. Updates golang/go#30929 Change-Id: I726eaf6a4f3eeaab27d31e2642cb7642111ccd67 Reviewed-on: https://go-review.googlesource.com/c/build/+/168341 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-03-19 20:17:23 +03:00
//
cmd/coordinator, dashboard: don't skip cmd/dist tests for SlowBots When a user explicitly requests a SlowBot builder via the TRY= syntax, it is a signal that they're willing to wait longer for test results. It's more useful to not skip any tests and give the user confidence that the SlowBot run will catch all issues that would otherwise be caught only by post-submit builders after the CL is submitted. This is done by considering a TryBot that isn't an explicit SlowBot to be a "normal TryBot", and the ShouldRunDistTest policy function is provided a parameter specifying whether it's a normal TryBot (non-SlowBot) run. Tests are only skipped during normal TryBot runs, not SlowBot runs. Fixes golang/go#38279. For golang/go#34501. Change-Id: Ia11fd51c71f9de1089b6b6f4c027014893f224b9 Reviewed-on: https://go-review.googlesource.com/c/build/+/227779 Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-04-09 23:24:30 +03:00
// In general, this returns true. When in normal trybot mode,
// some slow portable tests are only run on the fastest builder.
dashboard: refactor mechanism for adjusting cmd/dist test policy Previously, the behavior of ShouldRunDistTest could be overridden by an individual builder by setting a the shouldRunDistTest field to a custom policy implementation. A pitfall of this approach is that it requires each custom policy to reimplement the default cmd/dist test policy as well. If that isn't done when attempting to make a small adjustment to the default policy, the result is that the default policy is ignored. The fasterTrybots policy, as the name suggests, exists to skip some tests in trybot mode. Due to the problem above, it was causing some builders to run more tests in trybot mode than they would if it weren't applied. For example, the freebsd-amd64-12_0 builder should have been skipping the "api" test in trybot mode, but wasn't. This refactor fixes that by modifying the approach for how individual builders can customize the cmd/dist test behavior. Now, a function distTestAdjust is called with the first parameter specifying the default policy decision. That function can adjust the decision as it sees fit, but otherwise it preserves its initial value in situations where no change is intended to be made. For golang/go#38279. Change-Id: I220248f51abfc8e77195ae0c9134a17f7cac9bc2 Reviewed-on: https://go-review.googlesource.com/c/build/+/227777 Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-04-09 19:47:03 +03:00
//
// Individual builders can adjust this policy to fit their needs.
cmd/coordinator, dashboard: don't skip cmd/dist tests for SlowBots When a user explicitly requests a SlowBot builder via the TRY= syntax, it is a signal that they're willing to wait longer for test results. It's more useful to not skip any tests and give the user confidence that the SlowBot run will catch all issues that would otherwise be caught only by post-submit builders after the CL is submitted. This is done by considering a TryBot that isn't an explicit SlowBot to be a "normal TryBot", and the ShouldRunDistTest policy function is provided a parameter specifying whether it's a normal TryBot (non-SlowBot) run. Tests are only skipped during normal TryBot runs, not SlowBot runs. Fixes golang/go#38279. For golang/go#34501. Change-Id: Ia11fd51c71f9de1089b6b6f4c027014893f224b9 Reviewed-on: https://go-review.googlesource.com/c/build/+/227779 Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-04-09 23:24:30 +03:00
func (c *BuildConfig) ShouldRunDistTest(distTest string, isNormalTry bool) bool {
dashboard: refactor mechanism for adjusting cmd/dist test policy Previously, the behavior of ShouldRunDistTest could be overridden by an individual builder by setting a the shouldRunDistTest field to a custom policy implementation. A pitfall of this approach is that it requires each custom policy to reimplement the default cmd/dist test policy as well. If that isn't done when attempting to make a small adjustment to the default policy, the result is that the default policy is ignored. The fasterTrybots policy, as the name suggests, exists to skip some tests in trybot mode. Due to the problem above, it was causing some builders to run more tests in trybot mode than they would if it weren't applied. For example, the freebsd-amd64-12_0 builder should have been skipping the "api" test in trybot mode, but wasn't. This refactor fixes that by modifying the approach for how individual builders can customize the cmd/dist test behavior. Now, a function distTestAdjust is called with the first parameter specifying the default policy decision. That function can adjust the decision as it sees fit, but otherwise it preserves its initial value in situations where no change is intended to be made. For golang/go#38279. Change-Id: I220248f51abfc8e77195ae0c9134a17f7cac9bc2 Reviewed-on: https://go-review.googlesource.com/c/build/+/227777 Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-04-09 19:47:03 +03:00
run := true
// This section implements the default cmd/dist test policy.
// Any changes here will affect test coverage on all builders.
cmd/coordinator, dashboard: don't skip cmd/dist tests for SlowBots When a user explicitly requests a SlowBot builder via the TRY= syntax, it is a signal that they're willing to wait longer for test results. It's more useful to not skip any tests and give the user confidence that the SlowBot run will catch all issues that would otherwise be caught only by post-submit builders after the CL is submitted. This is done by considering a TryBot that isn't an explicit SlowBot to be a "normal TryBot", and the ShouldRunDistTest policy function is provided a parameter specifying whether it's a normal TryBot (non-SlowBot) run. Tests are only skipped during normal TryBot runs, not SlowBot runs. Fixes golang/go#38279. For golang/go#34501. Change-Id: Ia11fd51c71f9de1089b6b6f4c027014893f224b9 Reviewed-on: https://go-review.googlesource.com/c/build/+/227779 Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-04-09 23:24:30 +03:00
if isNormalTry {
dashboard: refactor mechanism for adjusting cmd/dist test policy Previously, the behavior of ShouldRunDistTest could be overridden by an individual builder by setting a the shouldRunDistTest field to a custom policy implementation. A pitfall of this approach is that it requires each custom policy to reimplement the default cmd/dist test policy as well. If that isn't done when attempting to make a small adjustment to the default policy, the result is that the default policy is ignored. The fasterTrybots policy, as the name suggests, exists to skip some tests in trybot mode. Due to the problem above, it was causing some builders to run more tests in trybot mode than they would if it weren't applied. For example, the freebsd-amd64-12_0 builder should have been skipping the "api" test in trybot mode, but wasn't. This refactor fixes that by modifying the approach for how individual builders can customize the cmd/dist test behavior. Now, a function distTestAdjust is called with the first parameter specifying the default policy decision. That function can adjust the decision as it sees fit, but otherwise it preserves its initial value in situations where no change is intended to be made. For golang/go#38279. Change-Id: I220248f51abfc8e77195ae0c9134a17f7cac9bc2 Reviewed-on: https://go-review.googlesource.com/c/build/+/227777 Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-04-09 19:47:03 +03:00
slowPortableTest := distTest == "api" // Whether a test is slow and has the same behavior everywhere.
fastestBuilder := c.Name == "linux-amd64"
if slowPortableTest && !fastestBuilder {
// Don't run the test on this builder.
run = false
}
cmd/coordinator: add a JSON mode to the /builders handler And add a test that indirectly verifies that the BuildConfig and HostConfig are JSON serializable. They weren't due to an exported func. But that exported func shouldn't be exported, so unexport it and move more policy into dashboard/builders.go. (There's been a number of recent cleanup CLs to move all policy into dashboard/builders.go instead of sprinkled all over the coordinator) A future CL will use this JSON in gomote create. Updates golang/go#30929 Change-Id: I726eaf6a4f3eeaab27d31e2642cb7642111ccd67 Reviewed-on: https://go-review.googlesource.com/c/build/+/168341 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-03-19 20:17:23 +03:00
}
dashboard: refactor mechanism for adjusting cmd/dist test policy Previously, the behavior of ShouldRunDistTest could be overridden by an individual builder by setting a the shouldRunDistTest field to a custom policy implementation. A pitfall of this approach is that it requires each custom policy to reimplement the default cmd/dist test policy as well. If that isn't done when attempting to make a small adjustment to the default policy, the result is that the default policy is ignored. The fasterTrybots policy, as the name suggests, exists to skip some tests in trybot mode. Due to the problem above, it was causing some builders to run more tests in trybot mode than they would if it weren't applied. For example, the freebsd-amd64-12_0 builder should have been skipping the "api" test in trybot mode, but wasn't. This refactor fixes that by modifying the approach for how individual builders can customize the cmd/dist test behavior. Now, a function distTestAdjust is called with the first parameter specifying the default policy decision. That function can adjust the decision as it sees fit, but otherwise it preserves its initial value in situations where no change is intended to be made. For golang/go#38279. Change-Id: I220248f51abfc8e77195ae0c9134a17f7cac9bc2 Reviewed-on: https://go-review.googlesource.com/c/build/+/227777 Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-04-09 19:47:03 +03:00
// Let individual builders adjust the cmd/dist test policy.
if c.distTestAdjust != nil {
cmd/coordinator, dashboard: don't skip cmd/dist tests for SlowBots When a user explicitly requests a SlowBot builder via the TRY= syntax, it is a signal that they're willing to wait longer for test results. It's more useful to not skip any tests and give the user confidence that the SlowBot run will catch all issues that would otherwise be caught only by post-submit builders after the CL is submitted. This is done by considering a TryBot that isn't an explicit SlowBot to be a "normal TryBot", and the ShouldRunDistTest policy function is provided a parameter specifying whether it's a normal TryBot (non-SlowBot) run. Tests are only skipped during normal TryBot runs, not SlowBot runs. Fixes golang/go#38279. For golang/go#34501. Change-Id: Ia11fd51c71f9de1089b6b6f4c027014893f224b9 Reviewed-on: https://go-review.googlesource.com/c/build/+/227779 Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-04-09 23:24:30 +03:00
run = c.distTestAdjust(run, distTest, isNormalTry)
cmd/coordinator: add a JSON mode to the /builders handler And add a test that indirectly verifies that the BuildConfig and HostConfig are JSON serializable. They weren't due to an exported func. But that exported func shouldn't be exported, so unexport it and move more policy into dashboard/builders.go. (There's been a number of recent cleanup CLs to move all policy into dashboard/builders.go instead of sprinkled all over the coordinator) A future CL will use this JSON in gomote create. Updates golang/go#30929 Change-Id: I726eaf6a4f3eeaab27d31e2642cb7642111ccd67 Reviewed-on: https://go-review.googlesource.com/c/build/+/168341 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-03-19 20:17:23 +03:00
}
dashboard: refactor mechanism for adjusting cmd/dist test policy Previously, the behavior of ShouldRunDistTest could be overridden by an individual builder by setting a the shouldRunDistTest field to a custom policy implementation. A pitfall of this approach is that it requires each custom policy to reimplement the default cmd/dist test policy as well. If that isn't done when attempting to make a small adjustment to the default policy, the result is that the default policy is ignored. The fasterTrybots policy, as the name suggests, exists to skip some tests in trybot mode. Due to the problem above, it was causing some builders to run more tests in trybot mode than they would if it weren't applied. For example, the freebsd-amd64-12_0 builder should have been skipping the "api" test in trybot mode, but wasn't. This refactor fixes that by modifying the approach for how individual builders can customize the cmd/dist test behavior. Now, a function distTestAdjust is called with the first parameter specifying the default policy decision. That function can adjust the decision as it sees fit, but otherwise it preserves its initial value in situations where no change is intended to be made. For golang/go#38279. Change-Id: I220248f51abfc8e77195ae0c9134a17f7cac9bc2 Reviewed-on: https://go-review.googlesource.com/c/build/+/227777 Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-04-09 19:47:03 +03:00
return run
cmd/coordinator: add a JSON mode to the /builders handler And add a test that indirectly verifies that the BuildConfig and HostConfig are JSON serializable. They weren't due to an exported func. But that exported func shouldn't be exported, so unexport it and move more policy into dashboard/builders.go. (There's been a number of recent cleanup CLs to move all policy into dashboard/builders.go instead of sprinkled all over the coordinator) A future CL will use this JSON in gomote create. Updates golang/go#30929 Change-Id: I726eaf6a4f3eeaab27d31e2642cb7642111ccd67 Reviewed-on: https://go-review.googlesource.com/c/build/+/168341 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-03-19 20:17:23 +03:00
}
dashboard: unify post-submit & trybot policy in builders.go, not coordinator This removes a bunch of TODOs about scattered policy and moves it all into builders.go. While here, * add a bunch of tests * unexport some things * rename some things * document some things * adjust FreeBSD policy as function of branch (per Go 1.12 being last to support FreeBSD 10.x, and to unblock CL 165801) * set GO_BUILDER_SET_GOPROXY=coordinator for reverse buildlets, which is necessary to remove the oauth2 & build special cases in the config * change Elias Naur's mobile builder to how I think he wants it (he was fighting the old system) * add $HOME on the Solaris smartos builder, which was missing & causing tests to fail lately * make the (currently failing) longtest builder have GOPROXY set * remove an allocation in version.ParseReleaseBranch Fixes golang/go#9603 Updates golang/go#14594 Change-Id: I50a23ad7cdf478c95b14bee9b3931ba361baacfa Reviewed-on: https://go-review.googlesource.com/c/build/+/166218 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-03-07 20:44:41 +03:00
// buildsRepoAtAll reports whether we should do builds of the provided
// repo ("go", "sys", "net", etc). This applies to both post-submit
// and trybot builds. Use BuildsRepoPostSubmit for only post-submit
// or BuildsRepoTryBot for trybots.
dashboard, cmd/coordinator: various fixes * add support for longtest builders for subrepos. * fix race builders for subrepos (they weren't passing the -race flag). * adjust the policy for the js-wasm builders to build fewer subrepos where it'll never work or isn't worth it. * fix the android emu builders which disappeared because an empty string was being passed to buildsRepoAtAll. In some places in the code an empty string for goBranch for the "go" repo meant the same as branch, but I forgot that in the new code, so an old caller was confusing the new config hooks. Rather than make all policy funcs be aware of both ways, the new code in this CL now maps an empty string to the same as the repo's branch when the repo == "go". Adds a test too. * fix some outdated comments. Change-Id: Icf3fb85e5542a4d314443b59d02517b306ef46b7 Reviewed-on: https://go-review.googlesource.com/c/build/+/166897 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-03-11 21:49:56 +03:00
//
// The branch is the branch of repo ("master",
// "release-branch.go1.12", etc); it is required. The goBranch is the
// branch of Go itself. It's required if repo != "go". When repo ==
// "go", the goBranch defaults to the value of branch.
dashboard: unify post-submit & trybot policy in builders.go, not coordinator This removes a bunch of TODOs about scattered policy and moves it all into builders.go. While here, * add a bunch of tests * unexport some things * rename some things * document some things * adjust FreeBSD policy as function of branch (per Go 1.12 being last to support FreeBSD 10.x, and to unblock CL 165801) * set GO_BUILDER_SET_GOPROXY=coordinator for reverse buildlets, which is necessary to remove the oauth2 & build special cases in the config * change Elias Naur's mobile builder to how I think he wants it (he was fighting the old system) * add $HOME on the Solaris smartos builder, which was missing & causing tests to fail lately * make the (currently failing) longtest builder have GOPROXY set * remove an allocation in version.ParseReleaseBranch Fixes golang/go#9603 Updates golang/go#14594 Change-Id: I50a23ad7cdf478c95b14bee9b3931ba361baacfa Reviewed-on: https://go-review.googlesource.com/c/build/+/166218 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-03-07 20:44:41 +03:00
func (c *BuildConfig) buildsRepoAtAll(repo, branch, goBranch string) bool {
dashboard, cmd/coordinator: various fixes * add support for longtest builders for subrepos. * fix race builders for subrepos (they weren't passing the -race flag). * adjust the policy for the js-wasm builders to build fewer subrepos where it'll never work or isn't worth it. * fix the android emu builders which disappeared because an empty string was being passed to buildsRepoAtAll. In some places in the code an empty string for goBranch for the "go" repo meant the same as branch, but I forgot that in the new code, so an old caller was confusing the new config hooks. Rather than make all policy funcs be aware of both ways, the new code in this CL now maps an empty string to the same as the repo's branch when the repo == "go". Adds a test too. * fix some outdated comments. Change-Id: Icf3fb85e5542a4d314443b59d02517b306ef46b7 Reviewed-on: https://go-review.googlesource.com/c/build/+/166897 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-03-11 21:49:56 +03:00
if goBranch == "" {
if repo == "go" {
goBranch = branch
} else {
panic("missing goBranch")
}
}
if branch == "" {
panic("missing branch")
}
if repo == "" {
panic("missing repo")
}
dashboard: skip Android build on older releases Also do some cleanup of no longer applicable policy. Fixes golang/go#30485 Change-Id: Id921ea5e7f60dda097be9ebba052a09349ea000a Reviewed-on: https://go-review.googlesource.com/c/164681 Reviewed-by: Elias Naur <mail@eliasnaur.com>
2019-03-01 02:20:41 +03:00
// Don't build old branches.
const minGo1x = 11
dashboard: require at least Go 1.12 for aix builder Fixes golang/go#30799 Change-Id: I0b3b53b715e2dc00f10548f74443224190003818 Reviewed-on: https://go-review.googlesource.com/c/build/+/167357 Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-03-13 11:29:15 +03:00
for _, b := range []string{branch, goBranch} {
if bmaj, bmin, ok := version.ParseReleaseBranch(b); ok {
if bmaj != 1 || bmin < minGo1x {
return false
}
bmm := types.MajorMinor{bmaj, bmin}
if bmm.Less(c.MinimumGoVersion) {
return false
}
dashboard: don't test the exp repo against release branches Exp is experimental. It's hard enough keeping it passing on master. We're not going to fight release branches as well. If somebody wants to see test coverage against three versions of Go, they can graduate their code out of experimental. Change-Id: I4695ce479c47bd56a4a58dba05d367d3de020c58 Reviewed-on: https://go-review.googlesource.com/c/build/+/170518 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-04-03 07:45:29 +03:00
if repo == "exp" {
// Don't test exp against release branches; it's experimental.
return false
}
dashboard: require at least Go 1.12 for aix builder Fixes golang/go#30799 Change-Id: I0b3b53b715e2dc00f10548f74443224190003818 Reviewed-on: https://go-review.googlesource.com/c/build/+/167357 Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-03-13 11:29:15 +03:00
}
cmd/coordinator: move skip branch policy into dashboard This change takes care of a TODO comment. It makes the builder configuration more centralized and contained in the dashboard package. Invert it for consistency with BuildConfig.BuildRepo method. Updates golang/go#26791 Change-Id: I46368adadb85f2ec730da4fc0abe5fd6a112a7c7 Reviewed-on: https://go-review.googlesource.com/c/149738 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-11-15 08:55:22 +03:00
}
dashboard: skip Android build on older releases Also do some cleanup of no longer applicable policy. Fixes golang/go#30485 Change-Id: Id921ea5e7f60dda097be9ebba052a09349ea000a Reviewed-on: https://go-review.googlesource.com/c/164681 Reviewed-by: Elias Naur <mail@eliasnaur.com>
2019-03-01 02:20:41 +03:00
dashboard: unify post-submit & trybot policy in builders.go, not coordinator This removes a bunch of TODOs about scattered policy and moves it all into builders.go. While here, * add a bunch of tests * unexport some things * rename some things * document some things * adjust FreeBSD policy as function of branch (per Go 1.12 being last to support FreeBSD 10.x, and to unblock CL 165801) * set GO_BUILDER_SET_GOPROXY=coordinator for reverse buildlets, which is necessary to remove the oauth2 & build special cases in the config * change Elias Naur's mobile builder to how I think he wants it (he was fighting the old system) * add $HOME on the Solaris smartos builder, which was missing & causing tests to fail lately * make the (currently failing) longtest builder have GOPROXY set * remove an allocation in version.ParseReleaseBranch Fixes golang/go#9603 Updates golang/go#14594 Change-Id: I50a23ad7cdf478c95b14bee9b3931ba361baacfa Reviewed-on: https://go-review.googlesource.com/c/build/+/166218 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-03-07 20:44:41 +03:00
// Build dev.boringcrypto branches only on linux/amd64 and windows/386 (see golang.org/issue/26791).
if repo == "go" && (branch == "dev.boringcrypto" || strings.HasPrefix(branch, "dev.boringcrypto.")) {
dashboard: do not sunset windows 2008 CL 363474 added a 2012 builder, which is good, but it also took a major step in sunsetting Windows 7 support by removing all 2008 builders from trybots and for Go 1.18+. This will surely bitrot Windows 7 support and force us to remove support for it, which isn't okay. Rather than line up dominoes to force our hand on policy, such huge policy changes with sweeping ramifications should be properly decided. So, this commit keeps the 2008 builders in the running, alongside the 2012 ones. Updates golang/go#47845. Updates golang/go#49490. Change-Id: I574f51ea22f638d2f9b09fcd00fdfc3097631771 Reviewed-on: https://go-review.googlesource.com/c/build/+/366656 Trust: Jason A. Donenfeld <Jason@zx2c4.com> Trust: Carlos Amedee <carlos@golang.org> Run-TryBot: Jason A. Donenfeld <Jason@zx2c4.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-11-23 23:55:31 +03:00
if c.Name != "linux-amd64" && c.Name != "windows-386-2012" && c.Name != "windows-386-2008" {
cmd/coordinator: move skip branch policy into dashboard This change takes care of a TODO comment. It makes the builder configuration more centralized and contained in the dashboard package. Invert it for consistency with BuildConfig.BuildRepo method. Updates golang/go#26791 Change-Id: I46368adadb85f2ec730da4fc0abe5fd6a112a7c7 Reviewed-on: https://go-review.googlesource.com/c/149738 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-11-15 08:55:22 +03:00
return false
}
}
dashboard: disable more builders Make exp off by default, as it barely passes anywhere. Lock in the places where it does pass. Also disable x/net master combined with Go 1.11 on FreeBSD 32-bit. It doesn't work on our builders and it'll never be fixed (we're not going to make changes to Go 1.11) and there's resistance to skipping those tests (my CL 170777), so I'm just turning off the builder. That's not ideal, but it's better than people getting used to red on the dashboard. Fixes golang/go#31221 Change-Id: Id0228e4b0ee73715a0c2da84f68dcfc989a48aaf Reviewed-on: https://go-review.googlesource.com/c/build/+/170782 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-04-05 08:26:05 +03:00
if repo != "go" && !c.SplitMakeRun() {
return false
}
dashboard: unify post-submit & trybot policy in builders.go, not coordinator This removes a bunch of TODOs about scattered policy and moves it all into builders.go. While here, * add a bunch of tests * unexport some things * rename some things * document some things * adjust FreeBSD policy as function of branch (per Go 1.12 being last to support FreeBSD 10.x, and to unblock CL 165801) * set GO_BUILDER_SET_GOPROXY=coordinator for reverse buildlets, which is necessary to remove the oauth2 & build special cases in the config * change Elias Naur's mobile builder to how I think he wants it (he was fighting the old system) * add $HOME on the Solaris smartos builder, which was missing & causing tests to fail lately * make the (currently failing) longtest builder have GOPROXY set * remove an allocation in version.ParseReleaseBranch Fixes golang/go#9603 Updates golang/go#14594 Change-Id: I50a23ad7cdf478c95b14bee9b3931ba361baacfa Reviewed-on: https://go-review.googlesource.com/c/build/+/166218 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-03-07 20:44:41 +03:00
if p := c.buildsRepo; p != nil {
return p(repo, branch, goBranch)
}
dashboard: document and clarify defaultBuildsRepoPolicy The defaultBuildsRepoPolicy function was very important in that any changes to it would affect many builders, but it was not documented well. This refactor attempts to improve the situation. It turned out better to also rename defaultBuildsRepoPolicy to buildRepoByDefault and limit its scope to report whether a given repo should be built by default. The branch and goBranch parameters were never used, and thus misleading. If they're needed in the future, we can easily re-add them. The name defaultBuildsRepoPolicy made it sound as if it was the entire "builds repo" policy, bun reality most of the policy is implemented in the BuildConfig.buildsRepoAtAll method and is non-configurable. Only a small part of the policy was in defaultBuildsRepoPolicy. For golang/go#39156. Change-Id: Ie7c6a68bd624056362dc7d964bd8d640f665b35a Reviewed-on: https://go-review.googlesource.com/c/build/+/240687 Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-07-01 21:45:10 +03:00
return buildRepoByDefault(repo)
dashboard: disable more builders Make exp off by default, as it barely passes anywhere. Lock in the places where it does pass. Also disable x/net master combined with Go 1.11 on FreeBSD 32-bit. It doesn't work on our builders and it'll never be fixed (we're not going to make changes to Go 1.11) and there's resistance to skipping those tests (my CL 170777), so I'm just turning off the builder. That's not ideal, but it's better than people getting used to red on the dashboard. Fixes golang/go#31221 Change-Id: Id0228e4b0ee73715a0c2da84f68dcfc989a48aaf Reviewed-on: https://go-review.googlesource.com/c/build/+/170782 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-04-05 08:26:05 +03:00
}
dashboard: document and clarify defaultBuildsRepoPolicy The defaultBuildsRepoPolicy function was very important in that any changes to it would affect many builders, but it was not documented well. This refactor attempts to improve the situation. It turned out better to also rename defaultBuildsRepoPolicy to buildRepoByDefault and limit its scope to report whether a given repo should be built by default. The branch and goBranch parameters were never used, and thus misleading. If they're needed in the future, we can easily re-add them. The name defaultBuildsRepoPolicy made it sound as if it was the entire "builds repo" policy, bun reality most of the policy is implemented in the BuildConfig.buildsRepoAtAll method and is non-configurable. Only a small part of the policy was in defaultBuildsRepoPolicy. For golang/go#39156. Change-Id: Ie7c6a68bd624056362dc7d964bd8d640f665b35a Reviewed-on: https://go-review.googlesource.com/c/build/+/240687 Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-07-01 21:45:10 +03:00
// buildRepoByDefault reports whether builders should do builds
// for the given repo ("go", "net", etc.) by default.
//
// It's used directly by BuildConfig.buildsRepoAtAll method for
// builders with a nil BuildConfig.buildsRepo value.
// It's also used by many builders that provide a custom build
// repo policy (a non-nil BuildConfig.buildsRepo value) as part
// of making the decision of whether to build a given repo.
//
// As a result, it effectively implements the default build repo policy.
// Any changes here will affect repo coverage of many builders.
func buildRepoByDefault(repo string) bool {
dashboard: add some "exp" trybots back, clean up some config Fixes golang/go#30781 Change-Id: I2672ca2949837c3cfe8fa6bd8011d1b9b43cca03 Reviewed-on: https://go-review.googlesource.com/c/build/+/167640 Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-03-14 23:35:32 +03:00
switch repo {
case "go":
dashboard: document and clarify defaultBuildsRepoPolicy The defaultBuildsRepoPolicy function was very important in that any changes to it would affect many builders, but it was not documented well. This refactor attempts to improve the situation. It turned out better to also rename defaultBuildsRepoPolicy to buildRepoByDefault and limit its scope to report whether a given repo should be built by default. The branch and goBranch parameters were never used, and thus misleading. If they're needed in the future, we can easily re-add them. The name defaultBuildsRepoPolicy made it sound as if it was the entire "builds repo" policy, bun reality most of the policy is implemented in the BuildConfig.buildsRepoAtAll method and is non-configurable. Only a small part of the policy was in defaultBuildsRepoPolicy. For golang/go#39156. Change-Id: Ie7c6a68bd624056362dc7d964bd8d640f665b35a Reviewed-on: https://go-review.googlesource.com/c/build/+/240687 Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-07-01 21:45:10 +03:00
// Build the main Go repository by default.
dashboard: add some "exp" trybots back, clean up some config Fixes golang/go#30781 Change-Id: I2672ca2949837c3cfe8fa6bd8011d1b9b43cca03 Reviewed-on: https://go-review.googlesource.com/c/build/+/167640 Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-03-14 23:35:32 +03:00
return true
dashboard: don't build x/build on so many builders It was kinda overkill. Change-Id: Ied7fc42aa59a1ed18cc1da20e51c8ed386d84629 Reviewed-on: https://go-review.googlesource.com/c/build/+/210378 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-12-07 08:14:28 +03:00
case "mobile", "exp", "build":
dashboard: document and clarify defaultBuildsRepoPolicy The defaultBuildsRepoPolicy function was very important in that any changes to it would affect many builders, but it was not documented well. This refactor attempts to improve the situation. It turned out better to also rename defaultBuildsRepoPolicy to buildRepoByDefault and limit its scope to report whether a given repo should be built by default. The branch and goBranch parameters were never used, and thus misleading. If they're needed in the future, we can easily re-add them. The name defaultBuildsRepoPolicy made it sound as if it was the entire "builds repo" policy, bun reality most of the policy is implemented in the BuildConfig.buildsRepoAtAll method and is non-configurable. Only a small part of the policy was in defaultBuildsRepoPolicy. For golang/go#39156. Change-Id: Ie7c6a68bd624056362dc7d964bd8d640f665b35a Reviewed-on: https://go-review.googlesource.com/c/build/+/240687 Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-07-01 21:45:10 +03:00
// Don't build x/mobile, x/exp, x/build by default.
//
// Builders need to explicitly opt-in to build these repos.
dashboard: update some build policy Mostly some regressions (building too much) from the massive refactoring in CL 166218, but also some minor changes. See test updates. Change-Id: Ie30f51b425367f18a49764ac971e262131959bc8 Reviewed-on: https://go-review.googlesource.com/c/build/+/166557 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-03-11 07:20:46 +03:00
return false
dashboard: document and clarify defaultBuildsRepoPolicy The defaultBuildsRepoPolicy function was very important in that any changes to it would affect many builders, but it was not documented well. This refactor attempts to improve the situation. It turned out better to also rename defaultBuildsRepoPolicy to buildRepoByDefault and limit its scope to report whether a given repo should be built by default. The branch and goBranch parameters were never used, and thus misleading. If they're needed in the future, we can easily re-add them. The name defaultBuildsRepoPolicy made it sound as if it was the entire "builds repo" policy, bun reality most of the policy is implemented in the BuildConfig.buildsRepoAtAll method and is non-configurable. Only a small part of the policy was in defaultBuildsRepoPolicy. For golang/go#39156. Change-Id: Ie7c6a68bd624056362dc7d964bd8d640f665b35a Reviewed-on: https://go-review.googlesource.com/c/build/+/240687 Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-07-01 21:45:10 +03:00
default:
// Build all other golang.org/x repositories by default.
return true
dashboard: update some build policy Mostly some regressions (building too much) from the massive refactoring in CL 166218, but also some minor changes. See test updates. Change-Id: Ie30f51b425367f18a49764ac971e262131959bc8 Reviewed-on: https://go-review.googlesource.com/c/build/+/166557 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-03-11 07:20:46 +03:00
}
cmd/coordinator: move skip branch policy into dashboard This change takes care of a TODO comment. It makes the builder configuration more centralized and contained in the dashboard package. Invert it for consistency with BuildConfig.BuildRepo method. Updates golang/go#26791 Change-Id: I46368adadb85f2ec730da4fc0abe5fd6a112a7c7 Reviewed-on: https://go-review.googlesource.com/c/149738 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-11-15 08:55:22 +03:00
}
dashboard: disable more builders Make exp off by default, as it barely passes anywhere. Lock in the places where it does pass. Also disable x/net master combined with Go 1.11 on FreeBSD 32-bit. It doesn't work on our builders and it'll never be fixed (we're not going to make changes to Go 1.11) and there's resistance to skipping those tests (my CL 170777), so I'm just turning off the builder. That's not ideal, but it's better than people getting used to red on the dashboard. Fixes golang/go#31221 Change-Id: Id0228e4b0ee73715a0c2da84f68dcfc989a48aaf Reviewed-on: https://go-review.googlesource.com/c/build/+/170782 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-04-05 08:26:05 +03:00
func defaultPlusExp(repo, branch, goBranch string) bool {
if repo == "exp" {
return true
}
dashboard: document and clarify defaultBuildsRepoPolicy The defaultBuildsRepoPolicy function was very important in that any changes to it would affect many builders, but it was not documented well. This refactor attempts to improve the situation. It turned out better to also rename defaultBuildsRepoPolicy to buildRepoByDefault and limit its scope to report whether a given repo should be built by default. The branch and goBranch parameters were never used, and thus misleading. If they're needed in the future, we can easily re-add them. The name defaultBuildsRepoPolicy made it sound as if it was the entire "builds repo" policy, bun reality most of the policy is implemented in the BuildConfig.buildsRepoAtAll method and is non-configurable. Only a small part of the policy was in defaultBuildsRepoPolicy. For golang/go#39156. Change-Id: Ie7c6a68bd624056362dc7d964bd8d640f665b35a Reviewed-on: https://go-review.googlesource.com/c/build/+/240687 Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-07-01 21:45:10 +03:00
return buildRepoByDefault(repo)
dashboard: disable more builders Make exp off by default, as it barely passes anywhere. Lock in the places where it does pass. Also disable x/net master combined with Go 1.11 on FreeBSD 32-bit. It doesn't work on our builders and it'll never be fixed (we're not going to make changes to Go 1.11) and there's resistance to skipping those tests (my CL 170777), so I'm just turning off the builder. That's not ideal, but it's better than people getting used to red on the dashboard. Fixes golang/go#31221 Change-Id: Id0228e4b0ee73715a0c2da84f68dcfc989a48aaf Reviewed-on: https://go-review.googlesource.com/c/build/+/170782 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-04-05 08:26:05 +03:00
}
dashboard: don't build x/build on so many builders It was kinda overkill. Change-Id: Ied7fc42aa59a1ed18cc1da20e51c8ed386d84629 Reviewed-on: https://go-review.googlesource.com/c/build/+/210378 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-12-07 08:14:28 +03:00
func defaultPlusExpBuild(repo, branch, goBranch string) bool {
if repo == "exp" || repo == "build" {
return true
}
dashboard: document and clarify defaultBuildsRepoPolicy The defaultBuildsRepoPolicy function was very important in that any changes to it would affect many builders, but it was not documented well. This refactor attempts to improve the situation. It turned out better to also rename defaultBuildsRepoPolicy to buildRepoByDefault and limit its scope to report whether a given repo should be built by default. The branch and goBranch parameters were never used, and thus misleading. If they're needed in the future, we can easily re-add them. The name defaultBuildsRepoPolicy made it sound as if it was the entire "builds repo" policy, bun reality most of the policy is implemented in the BuildConfig.buildsRepoAtAll method and is non-configurable. Only a small part of the policy was in defaultBuildsRepoPolicy. For golang/go#39156. Change-Id: Ie7c6a68bd624056362dc7d964bd8d640f665b35a Reviewed-on: https://go-review.googlesource.com/c/build/+/240687 Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-07-01 21:45:10 +03:00
return buildRepoByDefault(repo)
dashboard: don't build x/build on so many builders It was kinda overkill. Change-Id: Ied7fc42aa59a1ed18cc1da20e51c8ed386d84629 Reviewed-on: https://go-review.googlesource.com/c/build/+/210378 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-12-07 08:14:28 +03:00
}
cmd/coordinator, dashboard: split build into 'make' and 'run' steps Change-Id: Ib60a601ef21163d5e0d1fed3a2609d9b719c616a Reviewed-on: https://go-review.googlesource.com/10073 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2015-05-21 07:02:38 +03:00
// AllScriptArgs returns the set of arguments that should be passed to the
dashboard: add darwin-arm/arm64 buildlets Tested by running a local coordinator on the darwin-arm64 builder. Change-Id: Id38faaa3275d6535b897f3c65df6d01ed9c1847e Reviewed-on: https://go-review.googlesource.com/9459 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2015-04-29 15:54:19 +03:00
// all.bash-equivalent script. Usually empty.
func (c *BuildConfig) AllScriptArgs() []string {
buildlet, cmd/coordinator: GCE quota accounting, fixes And deal with Preemptible resource exhaustion errors. And change all-compile to misc-compile and only do the builders not covered otherwise (Fixes #11073) And make the watcher serve git source. And cache and singleflight fetching of git source. And a million other things. Fixes golang/go#11073 Change-Id: I0f45610f0c6a06bd0c8ba9632b8624e00aeb52fc Reviewed-on: https://go-review.googlesource.com/10750 Reviewed-by: Andrew Gerrand <adg@golang.org>
2015-06-05 04:25:50 +03:00
return append([]string(nil), c.allScriptArgs...)
dashboard: add darwin-arm/arm64 buildlets Tested by running a local coordinator on the darwin-arm64 builder. Change-Id: Id38faaa3275d6535b897f3c65df6d01ed9c1847e Reviewed-on: https://go-review.googlesource.com/9459 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2015-04-29 15:54:19 +03:00
}
cmd/release: add new program to build releases The release command will replace misc/makerelease from the main repo. The contents of makerelease make their way into this program. Also add GorootFinal and MakeScript methods to dashboard.BuildConfig. Change-Id: Id20f3f7a85a21f040d8cf47e438d299d57360556 Reviewed-on: https://go-review.googlesource.com/3693 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2015-02-02 15:05:01 +03:00
// MakeScript returns the relative path to the operating system's script to
// do the build.
// Example values are "src/make.bash", "src/make.bat", "src/make.rc".
func (c *BuildConfig) MakeScript() string {
if strings.HasPrefix(c.Name, "windows-") {
return "src/make.bat"
}
if strings.HasPrefix(c.Name, "plan9-") {
return "src/make.rc"
}
return "src/make.bash"
}
cmd/coordinator, dashboard: split build into 'make' and 'run' steps Change-Id: Ib60a601ef21163d5e0d1fed3a2609d9b719c616a Reviewed-on: https://go-review.googlesource.com/10073 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2015-05-21 07:02:38 +03:00
// MakeScriptArgs returns the set of arguments that should be passed to the
// make.bash-equivalent script. Usually empty.
func (c *BuildConfig) MakeScriptArgs() []string {
return c.AllScriptArgs()
}
cmd/release: add new program to build releases The release command will replace misc/makerelease from the main repo. The contents of makerelease make their way into this program. Also add GorootFinal and MakeScript methods to dashboard.BuildConfig. Change-Id: Id20f3f7a85a21f040d8cf47e438d299d57360556 Reviewed-on: https://go-review.googlesource.com/3693 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2015-02-02 15:05:01 +03:00
// GorootFinal returns the default install location for
// releases for this platform.
func (c *BuildConfig) GorootFinal() string {
if strings.HasPrefix(c.Name, "windows-") {
return "c:\\go"
}
return "/usr/local/go"
}
dashboard: move builder configuration out of coordinator into a package The new package will be used by the buildlet command line tool in the future. Also use the metadata package in the coordinator for the master build key, since it was already depending on that package. Might as well use it. Change-Id: I6ad7593787ebba364090951ccbaecb4aa38e1dd6 Reviewed-on: https://go-review.googlesource.com/2920 Reviewed-by: Andrew Gerrand <adg@golang.org>
2015-01-15 23:46:22 +03:00
// MachineType returns the GCE machine type to use for this builder.
all: split builder config into builder & host configs Our builders are named of the form "GOOS-GOARCH" or "GOOS-GOARCH-suffix". Over time we've grown many builders. This CL doesn't change that. Builders continue to be named and operate as before. Previously the build configuration file (dashboard/builders.go) made each builder type ("linux-amd64-race", etc) define how to create a host running a buildlet of that type, even though many builders had identical host configs. For example, these builders all share the same host type (a Kubernetes container): linux-amd64 linux-amd64-race linux-386 linux-386-387 And these are the same host type (a GCE VM): windows-amd64-gce windows-amd64-race windows-386-gce This CL creates a new concept of a "hostType" which defines how the buildlet is created (Kube, GCE, Reverse, and how), and then each builder itself references a host type. Users never see the hostType. (except perhaps in gomote list output) But they at least never need to care about them. Reverse buildlets now can only be one hostType at a time, which simplifies things. We were no longer using multiple roles per machine once moving to VMs for OS X. gomote continues to operate as it did previously but its underlying protocol changed and clients will need to be updated. As a new feature, gomote now has a new flag to let you reuse a buildlet host connection for different builder rules if they share the same underlying host type. But users can ignore that. This CL is a long-standing TODO (previously attempted and aborted) and will make many things easier and faster, including the linux-arm cross-compilation effort, and keeping pre-warmed buildlets of VM types ready to go. Updates golang/go#17104 Change-Id: Iad8387f48680424a8441e878a2f4762bf79ea4d2 Reviewed-on: https://go-review.googlesource.com/29551 Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2016-09-22 00:27:37 +03:00
func (c *HostConfig) MachineType() string {
dashboard: move builder configuration out of coordinator into a package The new package will be used by the buildlet command line tool in the future. Also use the metadata package in the coordinator for the master build key, since it was already depending on that package. Might as well use it. Change-Id: I6ad7593787ebba364090951ccbaecb4aa38e1dd6 Reviewed-on: https://go-review.googlesource.com/2920 Reviewed-by: Andrew Gerrand <adg@golang.org>
2015-01-15 23:46:22 +03:00
if v := c.machineType; v != "" {
return v
}
dashboard: use n2 instances where necessary e2 and n2d instances do not support specifying a MinCPUVersion or NestedVirt. For golang/go#48969 Change-Id: I0c4b359b29cd24302b869748014f848a9b7b5e39 Reviewed-on: https://go-review.googlesource.com/c/build/+/356071 Trust: Alexander Rakoczy <alex@golang.org> Run-TryBot: Alexander Rakoczy <alex@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-10-15 04:54:32 +03:00
if c.NestedVirt {
// e2 instances do not support nested virtualization, but n2
// instances do.
return "n2-standard-4" // 4 vCPUs, 16 GB mem
}
all: use Container-Optimized VMs instead of Kubernetes for buildlet containers Fixes golang/go#25108 Change-Id: I084669b52b699700ed26a7fdd890d9205a8b9dc9 Reviewed-on: https://go-review.googlesource.com/111267 Reviewed-by: Andrew Bonventre <andybons@golang.org>
2018-05-04 05:42:17 +03:00
if c.IsContainer() {
// Set a higher default machine size for containers,
// so their /workdir tmpfs can be larger. The COS
// image has no swap, so we want to make sure the
// /workdir fits completely in memory.
all: migrate from n1 to e2 instances We should have similar performance for these machines, and increased memory availability in some areas. This is also compliant with an internal policy change. Change-Id: I07a452a07431322d6fdd4edbb98c5914045d34ed Reviewed-on: https://go-review.googlesource.com/c/build/+/354757 Trust: Alexander Rakoczy <alex@golang.org> Run-TryBot: Alexander Rakoczy <alex@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Heschi Kreinick <heschi@google.com> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-10-09 00:03:24 +03:00
return "e2-standard-4" // 4 vCPUs, 16 GB mem
all: use Container-Optimized VMs instead of Kubernetes for buildlet containers Fixes golang/go#25108 Change-Id: I084669b52b699700ed26a7fdd890d9205a8b9dc9 Reviewed-on: https://go-review.googlesource.com/111267 Reviewed-by: Andrew Bonventre <andybons@golang.org>
2018-05-04 05:42:17 +03:00
}
all: migrate from n1 to e2 instances We should have similar performance for these machines, and increased memory availability in some areas. This is also compliant with an internal policy change. Change-Id: I07a452a07431322d6fdd4edbb98c5914045d34ed Reviewed-on: https://go-review.googlesource.com/c/build/+/354757 Trust: Alexander Rakoczy <alex@golang.org> Run-TryBot: Alexander Rakoczy <alex@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Heschi Kreinick <heschi@google.com> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-10-09 00:03:24 +03:00
return "e2-highcpu-2"
dashboard: move builder configuration out of coordinator into a package The new package will be used by the buildlet command line tool in the future. Also use the metadata package in the coordinator for the master build key, since it was already depending on that package. Might as well use it. Change-Id: I6ad7593787ebba364090951ccbaecb4aa38e1dd6 Reviewed-on: https://go-review.googlesource.com/2920 Reviewed-by: Andrew Gerrand <adg@golang.org>
2015-01-15 23:46:22 +03:00
}
dashboard, buildenv: label EC2 instances This change adds a boolean which identified when an instance is an EC2 instance. Various EC2 related variables have been renamed to be more consistent. Updates golang/go#36841 Change-Id: Ief385bf9d41e320ebd7d31bc0108e8053f337afd Reviewed-on: https://go-review.googlesource.com/c/build/+/233801 Run-TryBot: Carlos Amedee <carlos@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-05-13 20:27:15 +03:00
// IsEC2 returns true if the machine type is an EC2 arm64 type.
func (c *HostConfig) IsEC2() bool {
return c.isEC2
}
cmd/coordinator: fix the /builders page, add Host info Change-Id: I8828d62918e29c732592d82f49c48a8c3442c7f3 Reviewed-on: https://go-review.googlesource.com/29674 Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2016-09-23 03:06:28 +03:00
// PoolName returns a short summary of the builder's host type for the
all: upgrade farmer.golang.org links to https Since farmer.golang.org has a valid certificate in production now, we should use that certificate for retrieving data. Change-Id: I3685d84953ea11e6ac09b4692fa27c73538a565c Reviewed-on: https://go-review.googlesource.com/39290 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2017-04-03 17:51:28 +03:00
// https://farmer.golang.org/builders page.
cmd/coordinator: fix the /builders page, add Host info Change-Id: I8828d62918e29c732592d82f49c48a8c3442c7f3 Reviewed-on: https://go-review.googlesource.com/29674 Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2016-09-23 03:06:28 +03:00
func (c *HostConfig) PoolName() string {
switch {
case c.IsReverse:
return "Reverse (dedicated machine/VM)"
dashboard, buildenv: label EC2 instances This change adds a boolean which identified when an instance is an EC2 instance. Various EC2 related variables have been renamed to be more consistent. Updates golang/go#36841 Change-Id: Ief385bf9d41e320ebd7d31bc0108e8053f337afd Reviewed-on: https://go-review.googlesource.com/c/build/+/233801 Run-TryBot: Carlos Amedee <carlos@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-05-13 20:27:15 +03:00
case c.IsEC2():
env/linux-arm64/aws, dashboard: update linux-arm64 builder image This updates the instruction for creating a VM image and a buildlet container image. The instructions will provide a better description for the steps required to create the images. The VM image has been updated to no longer require sudo when the docker command is executed. The host configuration has been updated with a new VM type and VM image. The rundockerbuildlet service configuration has been modified: * The memory limit has been removed because rundockerbuildlet will now use all of the available memory for buildlets running on AWS. * The service will report as running even after rundockerbuildlet has exited. Rundockerbuildlet will initiate a single buildlet and then exit. * The service will not be restarted after rundockerbuildlet has exited. Changes made to rundockerbuildlet which required these changes have been made in CL 234114. Updates golang/go#36841 Change-Id: I3ff51e414b11a2b1427cbd7c6e24e9135fc2a5c3 Reviewed-on: https://go-review.googlesource.com/c/build/+/236297 Run-TryBot: Carlos Amedee <carlos@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-05-20 00:26:14 +03:00
return "EC2 VM Container"
all: rename Kube to Container and IsGCE to IsVM Once containers run on COS instead of Kubernetes, one name (Kube*) is wrong and the other (GCE) is ambiguous. So rename them now to be more specific. No behavior changes. Just renaming in this step, to reduce size of next CL. Updates golang/go#25108 Change-Id: Ib09eb682ef74acbbf6ed50b46074f834ef5e0c0b Reviewed-on: https://go-review.googlesource.com/111639 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-05-05 19:36:05 +03:00
case c.IsVM():
cmd/coordinator: fix the /builders page, add Host info Change-Id: I8828d62918e29c732592d82f49c48a8c3442c7f3 Reviewed-on: https://go-review.googlesource.com/29674 Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2016-09-23 03:06:28 +03:00
return "GCE VM"
all: rename Kube to Container and IsGCE to IsVM Once containers run on COS instead of Kubernetes, one name (Kube*) is wrong and the other (GCE) is ambiguous. So rename them now to be more specific. No behavior changes. Just renaming in this step, to reduce size of next CL. Updates golang/go#25108 Change-Id: Ib09eb682ef74acbbf6ed50b46074f834ef5e0c0b Reviewed-on: https://go-review.googlesource.com/111639 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-05-05 19:36:05 +03:00
case c.IsContainer():
all: use Container-Optimized VMs instead of Kubernetes for buildlet containers Fixes golang/go#25108 Change-Id: I084669b52b699700ed26a7fdd890d9205a8b9dc9 Reviewed-on: https://go-review.googlesource.com/111267 Reviewed-by: Andrew Bonventre <andybons@golang.org>
2018-05-04 05:42:17 +03:00
return "Container"
cmd/coordinator: fix the /builders page, add Host info Change-Id: I8828d62918e29c732592d82f49c48a8c3442c7f3 Reviewed-on: https://go-review.googlesource.com/29674 Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2016-09-23 03:06:28 +03:00
}
cmd/coordinator, cmd/buildlet, cmd/gomote: add SSH support This adds an SSH server to farmer.golang.org on port 2222 that proxies SSH connections to users' gomote-created buildlet instances. For example: $ gomote create openbsd-amd64-60 user-bradfitz-openbsd-amd64-60-1 $ gomote ssh user-bradfitz-openbsd-amd64-60-1 Warning: Permanently added '[localhost]:33351' (ECDSA) to the list of known hosts. OpenBSD 6.0 (GENERIC.MP) golang/go#2319: Tue Jul 26 13:00:43 MDT 2016 Welcome to OpenBSD: The proactively secure Unix-like operating system. Please use the sendbug(1) utility to report bugs in the system. Before reporting a bug, please try to reproduce it with the latest version of the code. With bug reports, please try to ensure that enough information to reproduce the problem is enclosed, and if a known fix for it exists, include that as well. $ As before, if the coordinator process is restarted (or crashes, is evicted, etc), all gomote instances die. Not yet supported: * scp (help wanted) * not all host types are configured. most are. some will need slight config tweaks to the Docker image (e.g. adding openssh-server) Supports currently: * linux-amd64 (host type shared by 386, nacl) * linux-arm * linux-arm64 * darwin * freebsd * openbsd * plan9-386 * windows Implementation details: * the ssh server process listens on port 2222 in the coordinator (farmer.golang.org), which is behind a GKE TCP load balancer. * the ssh server library is github.com/gliderlabs/ssh * authentication is done via Github users' public keys. It's assumed that gomote user == github user. But there's a mapping in the code for known exceptions. * we can't give out access to this too widely. too many things are accessible from within the host environment if you look in the right places. Details omitted. But the Go team and other trusted gomote users can use this. * the buildlet binary has a new /connect-ssh handler that acts like a CONNECT request but instead of taking an explicit host:port, just says "give me your machine's SSH connection". The buildlet can also start sshd if needed for the environment. The /connect-ssh handler also installs the coordinator's public key. * a new buildlet client library method "ConnectSSH" hits the /connect-ssh handler and returns a net.Conn. * the coordinator's ssh.Handler is just running the OpenSSH ssh client. * because the OpenSSH ssh child process can't connect to a net.Conn, an emphemeral localhost port is created on the coordinator to proxy between the ssh client and the net.Conn returned by ConnectSSH. * The /connect-ssh handler requires http.Hijacker, which requires fully compliant net.Conn implementations as of Go 1.8. So I needed to flesh out revdial too, testing it with the golang.org/x/net/nettest package. * plan9 doesn't have an ssh server, so we use 0intro's new conterm program (drawterm without GUI support) to connect to plan9 from the coordinator ssh proxy instead of using the OpenSSH ssh client binary. * windows doesn't have an ssh server, so we enable the telnet service and the coordinator ssh proxy uses telnet instead on the backend on the private network. (There is a Windows ssh server but only in new versions.) Happy debugging over ssh! Fixes golang/go#19956 Change-Id: I80a62064c5f85af1f195f980c862ba29af4015f0 Reviewed-on: https://go-review.googlesource.com/50750 Reviewed-by: Herbie Ong <herbie@google.com> Reviewed-by: Jessie Frazelle <me@jessfraz.com> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2017-07-22 22:15:56 +03:00
panic("unknown builder type")
}
env/linux-x86-vmx: add new Debian host that's like Container-Optimized OS + vmx This adds scripts to create a new builder host image that acts like Container-Optimized OS (has docker, runs konlet on startup) but with a Debian 9 kernel + userspace that permits KVM for nested virtualization. Updates golang/go#15581 (solaris) Updates golang/go#23060 (dragonfly) Updates golang/go#30262 (riscv) Updates golang/go#30267 (fuchsia) Updates golang/go#23824 (android) Change-Id: Ib1d3a250556703856083c222be2a70c4e8d91884 Reviewed-on: https://go-review.googlesource.com/c/163301 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-02-22 00:10:54 +03:00
// ContainerVMImage returns the base VM name (not the fully qualified
// URL resource name of the VM) that starts the konlet program that
dashboard, buildlet, cmd/debugnewvm: clean up HostConfig.ContainerVMImage The debugnewvm command was printing out the wrong VM name, since the old method had also assumed (but not documented) that it only applied to nested virtualization containers. But remove that requirement and make the empty string mean unspecified instead (which we currently mean to use Container-Optimized OS) role and then use its new definition. Change-Id: Ieca138285aa567b1c24d585c5aa180f8a1534154 Reviewed-on: https://go-review.googlesource.com/c/build/+/177919 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-05-17 23:08:13 +03:00
// pulls & runs a container.
// The empty string means that no particular VM image is required
// and the caller can run this container in any host.
//
// This method is only applicable when c.IsContainer() is true.
env/linux-x86-vmx: add new Debian host that's like Container-Optimized OS + vmx This adds scripts to create a new builder host image that acts like Container-Optimized OS (has docker, runs konlet on startup) but with a Debian 9 kernel + userspace that permits KVM for nested virtualization. Updates golang/go#15581 (solaris) Updates golang/go#23060 (dragonfly) Updates golang/go#30262 (riscv) Updates golang/go#30267 (fuchsia) Updates golang/go#23824 (android) Change-Id: Ib1d3a250556703856083c222be2a70c4e8d91884 Reviewed-on: https://go-review.googlesource.com/c/163301 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-02-22 00:10:54 +03:00
func (c *HostConfig) ContainerVMImage() string {
if c.KonletVMImage != "" {
return c.KonletVMImage
}
dashboard, buildlet, cmd/debugnewvm: clean up HostConfig.ContainerVMImage The debugnewvm command was printing out the wrong VM name, since the old method had also assumed (but not documented) that it only applied to nested virtualization containers. But remove that requirement and make the empty string mean unspecified instead (which we currently mean to use Container-Optimized OS) role and then use its new definition. Change-Id: Ieca138285aa567b1c24d585c5aa180f8a1534154 Reviewed-on: https://go-review.googlesource.com/c/build/+/177919 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-05-17 23:08:13 +03:00
if c.NestedVirt {
return "debian-stretch-vmx"
}
env/linux-arm64/aws, dashboard: update linux-arm64 builder image This updates the instruction for creating a VM image and a buildlet container image. The instructions will provide a better description for the steps required to create the images. The VM image has been updated to no longer require sudo when the docker command is executed. The host configuration has been updated with a new VM type and VM image. The rundockerbuildlet service configuration has been modified: * The memory limit has been removed because rundockerbuildlet will now use all of the available memory for buildlets running on AWS. * The service will report as running even after rundockerbuildlet has exited. Rundockerbuildlet will initiate a single buildlet and then exit. * The service will not be restarted after rundockerbuildlet has exited. Changes made to rundockerbuildlet which required these changes have been made in CL 234114. Updates golang/go#36841 Change-Id: I3ff51e414b11a2b1427cbd7c6e24e9135fc2a5c3 Reviewed-on: https://go-review.googlesource.com/c/build/+/236297 Run-TryBot: Carlos Amedee <carlos@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-05-20 00:26:14 +03:00
if c.isEC2 && c.ContainerImage != "" {
return fmt.Sprintf("gcr.io/%s/%s", buildenv.Production.ProjectName, c.ContainerImage)
}
dashboard, buildlet, cmd/debugnewvm: clean up HostConfig.ContainerVMImage The debugnewvm command was printing out the wrong VM name, since the old method had also assumed (but not documented) that it only applied to nested virtualization containers. But remove that requirement and make the empty string mean unspecified instead (which we currently mean to use Container-Optimized OS) role and then use its new definition. Change-Id: Ieca138285aa567b1c24d585c5aa180f8a1534154 Reviewed-on: https://go-review.googlesource.com/c/build/+/177919 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-05-17 23:08:13 +03:00
return ""
env/linux-x86-vmx: add new Debian host that's like Container-Optimized OS + vmx This adds scripts to create a new builder host image that acts like Container-Optimized OS (has docker, runs konlet on startup) but with a Debian 9 kernel + userspace that permits KVM for nested virtualization. Updates golang/go#15581 (solaris) Updates golang/go#23060 (dragonfly) Updates golang/go#30262 (riscv) Updates golang/go#30267 (fuchsia) Updates golang/go#23824 (android) Change-Id: Ib1d3a250556703856083c222be2a70c4e8d91884 Reviewed-on: https://go-review.googlesource.com/c/163301 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-02-22 00:10:54 +03:00
}
cmd/coordinator, cmd/buildlet, cmd/gomote: add SSH support This adds an SSH server to farmer.golang.org on port 2222 that proxies SSH connections to users' gomote-created buildlet instances. For example: $ gomote create openbsd-amd64-60 user-bradfitz-openbsd-amd64-60-1 $ gomote ssh user-bradfitz-openbsd-amd64-60-1 Warning: Permanently added '[localhost]:33351' (ECDSA) to the list of known hosts. OpenBSD 6.0 (GENERIC.MP) golang/go#2319: Tue Jul 26 13:00:43 MDT 2016 Welcome to OpenBSD: The proactively secure Unix-like operating system. Please use the sendbug(1) utility to report bugs in the system. Before reporting a bug, please try to reproduce it with the latest version of the code. With bug reports, please try to ensure that enough information to reproduce the problem is enclosed, and if a known fix for it exists, include that as well. $ As before, if the coordinator process is restarted (or crashes, is evicted, etc), all gomote instances die. Not yet supported: * scp (help wanted) * not all host types are configured. most are. some will need slight config tweaks to the Docker image (e.g. adding openssh-server) Supports currently: * linux-amd64 (host type shared by 386, nacl) * linux-arm * linux-arm64 * darwin * freebsd * openbsd * plan9-386 * windows Implementation details: * the ssh server process listens on port 2222 in the coordinator (farmer.golang.org), which is behind a GKE TCP load balancer. * the ssh server library is github.com/gliderlabs/ssh * authentication is done via Github users' public keys. It's assumed that gomote user == github user. But there's a mapping in the code for known exceptions. * we can't give out access to this too widely. too many things are accessible from within the host environment if you look in the right places. Details omitted. But the Go team and other trusted gomote users can use this. * the buildlet binary has a new /connect-ssh handler that acts like a CONNECT request but instead of taking an explicit host:port, just says "give me your machine's SSH connection". The buildlet can also start sshd if needed for the environment. The /connect-ssh handler also installs the coordinator's public key. * a new buildlet client library method "ConnectSSH" hits the /connect-ssh handler and returns a net.Conn. * the coordinator's ssh.Handler is just running the OpenSSH ssh client. * because the OpenSSH ssh child process can't connect to a net.Conn, an emphemeral localhost port is created on the coordinator to proxy between the ssh client and the net.Conn returned by ConnectSSH. * The /connect-ssh handler requires http.Hijacker, which requires fully compliant net.Conn implementations as of Go 1.8. So I needed to flesh out revdial too, testing it with the golang.org/x/net/nettest package. * plan9 doesn't have an ssh server, so we use 0intro's new conterm program (drawterm without GUI support) to connect to plan9 from the coordinator ssh proxy instead of using the OpenSSH ssh client binary. * windows doesn't have an ssh server, so we enable the telnet service and the coordinator ssh proxy uses telnet instead on the backend on the private network. (There is a Windows ssh server but only in new versions.) Happy debugging over ssh! Fixes golang/go#19956 Change-Id: I80a62064c5f85af1f195f980c862ba29af4015f0 Reviewed-on: https://go-review.googlesource.com/50750 Reviewed-by: Herbie Ong <herbie@google.com> Reviewed-by: Jessie Frazelle <me@jessfraz.com> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2017-07-22 22:15:56 +03:00
// IsHermetic reports whether this host config gets a fresh
// environment (including /usr, /var, etc) for each execution. This is
// true for VMs, GKE, and reverse buildlets running their containers
// running in Docker, but false on some reverse buildlets.
func (c *HostConfig) IsHermetic() bool {
switch {
case c.IsReverse:
return c.HermeticReverse
dashboard, buildenv: label EC2 instances This change adds a boolean which identified when an instance is an EC2 instance. Various EC2 related variables have been renamed to be more consistent. Updates golang/go#36841 Change-Id: Ief385bf9d41e320ebd7d31bc0108e8053f337afd Reviewed-on: https://go-review.googlesource.com/c/build/+/233801 Run-TryBot: Carlos Amedee <carlos@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-05-13 20:27:15 +03:00
case c.IsEC2():
return true
all: rename Kube to Container and IsGCE to IsVM Once containers run on COS instead of Kubernetes, one name (Kube*) is wrong and the other (GCE) is ambiguous. So rename them now to be more specific. No behavior changes. Just renaming in this step, to reduce size of next CL. Updates golang/go#25108 Change-Id: Ib09eb682ef74acbbf6ed50b46074f834ef5e0c0b Reviewed-on: https://go-review.googlesource.com/111639 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-05-05 19:36:05 +03:00
case c.IsVM():
cmd/coordinator, cmd/buildlet, cmd/gomote: add SSH support This adds an SSH server to farmer.golang.org on port 2222 that proxies SSH connections to users' gomote-created buildlet instances. For example: $ gomote create openbsd-amd64-60 user-bradfitz-openbsd-amd64-60-1 $ gomote ssh user-bradfitz-openbsd-amd64-60-1 Warning: Permanently added '[localhost]:33351' (ECDSA) to the list of known hosts. OpenBSD 6.0 (GENERIC.MP) golang/go#2319: Tue Jul 26 13:00:43 MDT 2016 Welcome to OpenBSD: The proactively secure Unix-like operating system. Please use the sendbug(1) utility to report bugs in the system. Before reporting a bug, please try to reproduce it with the latest version of the code. With bug reports, please try to ensure that enough information to reproduce the problem is enclosed, and if a known fix for it exists, include that as well. $ As before, if the coordinator process is restarted (or crashes, is evicted, etc), all gomote instances die. Not yet supported: * scp (help wanted) * not all host types are configured. most are. some will need slight config tweaks to the Docker image (e.g. adding openssh-server) Supports currently: * linux-amd64 (host type shared by 386, nacl) * linux-arm * linux-arm64 * darwin * freebsd * openbsd * plan9-386 * windows Implementation details: * the ssh server process listens on port 2222 in the coordinator (farmer.golang.org), which is behind a GKE TCP load balancer. * the ssh server library is github.com/gliderlabs/ssh * authentication is done via Github users' public keys. It's assumed that gomote user == github user. But there's a mapping in the code for known exceptions. * we can't give out access to this too widely. too many things are accessible from within the host environment if you look in the right places. Details omitted. But the Go team and other trusted gomote users can use this. * the buildlet binary has a new /connect-ssh handler that acts like a CONNECT request but instead of taking an explicit host:port, just says "give me your machine's SSH connection". The buildlet can also start sshd if needed for the environment. The /connect-ssh handler also installs the coordinator's public key. * a new buildlet client library method "ConnectSSH" hits the /connect-ssh handler and returns a net.Conn. * the coordinator's ssh.Handler is just running the OpenSSH ssh client. * because the OpenSSH ssh child process can't connect to a net.Conn, an emphemeral localhost port is created on the coordinator to proxy between the ssh client and the net.Conn returned by ConnectSSH. * The /connect-ssh handler requires http.Hijacker, which requires fully compliant net.Conn implementations as of Go 1.8. So I needed to flesh out revdial too, testing it with the golang.org/x/net/nettest package. * plan9 doesn't have an ssh server, so we use 0intro's new conterm program (drawterm without GUI support) to connect to plan9 from the coordinator ssh proxy instead of using the OpenSSH ssh client binary. * windows doesn't have an ssh server, so we enable the telnet service and the coordinator ssh proxy uses telnet instead on the backend on the private network. (There is a Windows ssh server but only in new versions.) Happy debugging over ssh! Fixes golang/go#19956 Change-Id: I80a62064c5f85af1f195f980c862ba29af4015f0 Reviewed-on: https://go-review.googlesource.com/50750 Reviewed-by: Herbie Ong <herbie@google.com> Reviewed-by: Jessie Frazelle <me@jessfraz.com> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2017-07-22 22:15:56 +03:00
return true
all: rename Kube to Container and IsGCE to IsVM Once containers run on COS instead of Kubernetes, one name (Kube*) is wrong and the other (GCE) is ambiguous. So rename them now to be more specific. No behavior changes. Just renaming in this step, to reduce size of next CL. Updates golang/go#25108 Change-Id: Ib09eb682ef74acbbf6ed50b46074f834ef5e0c0b Reviewed-on: https://go-review.googlesource.com/111639 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-05-05 19:36:05 +03:00
case c.IsContainer():
cmd/coordinator, cmd/buildlet, cmd/gomote: add SSH support This adds an SSH server to farmer.golang.org on port 2222 that proxies SSH connections to users' gomote-created buildlet instances. For example: $ gomote create openbsd-amd64-60 user-bradfitz-openbsd-amd64-60-1 $ gomote ssh user-bradfitz-openbsd-amd64-60-1 Warning: Permanently added '[localhost]:33351' (ECDSA) to the list of known hosts. OpenBSD 6.0 (GENERIC.MP) golang/go#2319: Tue Jul 26 13:00:43 MDT 2016 Welcome to OpenBSD: The proactively secure Unix-like operating system. Please use the sendbug(1) utility to report bugs in the system. Before reporting a bug, please try to reproduce it with the latest version of the code. With bug reports, please try to ensure that enough information to reproduce the problem is enclosed, and if a known fix for it exists, include that as well. $ As before, if the coordinator process is restarted (or crashes, is evicted, etc), all gomote instances die. Not yet supported: * scp (help wanted) * not all host types are configured. most are. some will need slight config tweaks to the Docker image (e.g. adding openssh-server) Supports currently: * linux-amd64 (host type shared by 386, nacl) * linux-arm * linux-arm64 * darwin * freebsd * openbsd * plan9-386 * windows Implementation details: * the ssh server process listens on port 2222 in the coordinator (farmer.golang.org), which is behind a GKE TCP load balancer. * the ssh server library is github.com/gliderlabs/ssh * authentication is done via Github users' public keys. It's assumed that gomote user == github user. But there's a mapping in the code for known exceptions. * we can't give out access to this too widely. too many things are accessible from within the host environment if you look in the right places. Details omitted. But the Go team and other trusted gomote users can use this. * the buildlet binary has a new /connect-ssh handler that acts like a CONNECT request but instead of taking an explicit host:port, just says "give me your machine's SSH connection". The buildlet can also start sshd if needed for the environment. The /connect-ssh handler also installs the coordinator's public key. * a new buildlet client library method "ConnectSSH" hits the /connect-ssh handler and returns a net.Conn. * the coordinator's ssh.Handler is just running the OpenSSH ssh client. * because the OpenSSH ssh child process can't connect to a net.Conn, an emphemeral localhost port is created on the coordinator to proxy between the ssh client and the net.Conn returned by ConnectSSH. * The /connect-ssh handler requires http.Hijacker, which requires fully compliant net.Conn implementations as of Go 1.8. So I needed to flesh out revdial too, testing it with the golang.org/x/net/nettest package. * plan9 doesn't have an ssh server, so we use 0intro's new conterm program (drawterm without GUI support) to connect to plan9 from the coordinator ssh proxy instead of using the OpenSSH ssh client binary. * windows doesn't have an ssh server, so we enable the telnet service and the coordinator ssh proxy uses telnet instead on the backend on the private network. (There is a Windows ssh server but only in new versions.) Happy debugging over ssh! Fixes golang/go#19956 Change-Id: I80a62064c5f85af1f195f980c862ba29af4015f0 Reviewed-on: https://go-review.googlesource.com/50750 Reviewed-by: Herbie Ong <herbie@google.com> Reviewed-by: Jessie Frazelle <me@jessfraz.com> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2017-07-22 22:15:56 +03:00
return true
}
panic("unknown builder type")
cmd/coordinator: fix the /builders page, add Host info Change-Id: I8828d62918e29c732592d82f49c48a8c3442c7f3 Reviewed-on: https://go-review.googlesource.com/29674 Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2016-09-23 03:06:28 +03:00
}
buildlet, cmd/coordinator: GCE quota accounting, fixes And deal with Preemptible resource exhaustion errors. And change all-compile to misc-compile and only do the builders not covered otherwise (Fixes #11073) And make the watcher serve git source. And cache and singleflight fetching of git source. And a million other things. Fixes golang/go#11073 Change-Id: I0f45610f0c6a06bd0c8ba9632b8624e00aeb52fc Reviewed-on: https://go-review.googlesource.com/10750 Reviewed-by: Andrew Gerrand <adg@golang.org>
2015-06-05 04:25:50 +03:00
// GCENumCPU reports the number of GCE CPUs this buildlet requires.
all: split builder config into builder & host configs Our builders are named of the form "GOOS-GOARCH" or "GOOS-GOARCH-suffix". Over time we've grown many builders. This CL doesn't change that. Builders continue to be named and operate as before. Previously the build configuration file (dashboard/builders.go) made each builder type ("linux-amd64-race", etc) define how to create a host running a buildlet of that type, even though many builders had identical host configs. For example, these builders all share the same host type (a Kubernetes container): linux-amd64 linux-amd64-race linux-386 linux-386-387 And these are the same host type (a GCE VM): windows-amd64-gce windows-amd64-race windows-386-gce This CL creates a new concept of a "hostType" which defines how the buildlet is created (Kube, GCE, Reverse, and how), and then each builder itself references a host type. Users never see the hostType. (except perhaps in gomote list output) But they at least never need to care about them. Reverse buildlets now can only be one hostType at a time, which simplifies things. We were no longer using multiple roles per machine once moving to VMs for OS X. gomote continues to operate as it did previously but its underlying protocol changed and clients will need to be updated. As a new feature, gomote now has a new flag to let you reuse a buildlet host connection for different builder rules if they share the same underlying host type. But users can ignore that. This CL is a long-standing TODO (previously attempted and aborted) and will make many things easier and faster, including the linux-arm cross-compilation effort, and keeping pre-warmed buildlets of VM types ready to go. Updates golang/go#17104 Change-Id: Iad8387f48680424a8441e878a2f4762bf79ea4d2 Reviewed-on: https://go-review.googlesource.com/29551 Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2016-09-22 00:27:37 +03:00
func (c *HostConfig) GCENumCPU() int {
buildlet, cmd/coordinator: GCE quota accounting, fixes And deal with Preemptible resource exhaustion errors. And change all-compile to misc-compile and only do the builders not covered otherwise (Fixes #11073) And make the watcher serve git source. And cache and singleflight fetching of git source. And a million other things. Fixes golang/go#11073 Change-Id: I0f45610f0c6a06bd0c8ba9632b8624e00aeb52fc Reviewed-on: https://go-review.googlesource.com/10750 Reviewed-by: Andrew Gerrand <adg@golang.org>
2015-06-05 04:25:50 +03:00
t := c.MachineType()
n, _ := strconv.Atoi(t[strings.LastIndex(t, "-")+1:])
return n
}
dashboard: try to speed up pre-submit longtest builders The longtest builders are currently primarily post-submit builders, where it's okay for them to be as slow as they need to be in order to provide additional test coverage. In this context, whether they take 40 minutes or 50 makes little difference. The longtest builders are also sometimes requested via SlowBots for changes that are riskier than usual, or otherwise desire additional coverage beyond the normal TryBots. They're also always enabled for CLs to release branches. In such contexts, speeding up SlowBot runs from 40 minutes to 20 or less would be appreciated and in turn help people use longtest SlowBots more frequently. Longtest builders are already configured to use sharded tests. Configure them to use additional helpers to speed up test execution. Try out 3, 5, and 9 helpers to see how much it helps before settling. For golang/go#37439. Change-Id: I425bc0257b7a54bb32c0eb1719fea7ba3f4fd461 Reviewed-on: https://go-review.googlesource.com/c/build/+/268037 Trust: Dmitri Shuralyov <dmitshur@golang.org> Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
2020-11-06 15:55:36 +03:00
// NumTestHelpers reports how many additional buildlets
// past the first one to help out with sharded tests.
//
// isTry specifies whether it's for a pre-submit test
// run (a TryBot or SlowBot) where speed matters more.
cmd/coordinator, dashboard: remove some trybots, shard others wider I'm aiming to have trybot runs finish in under 5 minutes. This CL removes openbsd-386-gce58 and freebsd-386-gce101 from the trybot set. openbsd-386-gce58 is the slowest builder. It has an average speed of 722 seconds (and 95 percentile of 923 seconds) over the past week, and that's sharded over 4 machines. Too slow. It's not worth the resources to keep it as a trybot. It hasn't caught any interesting bugs. This builder will still run, but not as a pre-submit trybot. freebsd-386-gce101 is not slow, but we're removing it to shift its resources to shard other builders wider. The coordinator now supports varying the build sharding width based on whether a build is for a trybot or not. This CL defines separate numbers for each, sharding builds wider as needed for some trybots. freebsd-amd64-gce101 goes from 4 to 5 machines in try runs, and down to 3 when not in try runs. linux-amd64-race gets one more machine during try runs, and one fewer in regular runs. linux-arm goes from 7 machines always, to 3 or 8, depending on whether it's a try run. openbsd-amd64-58 goes from 4 to 3 or 6. windows-amd64-gce goes from 4 to 2 or 6. windows-amd64-race goes from 4 to 2 or 6. darwin-amd64-10_11 goes from 3 to 3 or 4. I'll see how these do over the next few days and readjust as needed. Also in this CL: fix the constants for the expected duration of make.bash, which impact when we schedule the creation of test sharding helper buildlets. We were creating them too early before, wasting resources. Change-Id: I38a9b24841e196f1eb668de058c49af8c1d1c64f Reviewed-on: https://go-review.googlesource.com/29116 Reviewed-by: Quentin Smith <quentin@golang.org>
2016-09-14 01:45:48 +03:00
func (c *BuildConfig) NumTestHelpers(isTry bool) int {
if isTry && c.numTryTestHelpers != 0 {
return c.numTryTestHelpers
}
return c.numTestHelpers
}
dashboard, cmd/coordinator: move more config out of coordinator into dashboard And with the increased flexibility, enable trybots for more repos for the "sys" project. Details: https://github.com/golang/go/issues/24893#issuecomment-433517519 Fixes golang/go#24893 Change-Id: I89ce6f219be986201332f8fad11c774bdd6246da Reviewed-on: https://go-review.googlesource.com/c/145157 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2018-10-26 22:21:58 +03:00
// defaultTrySet returns a trybot policy function that reports whether
// a project should use trybots. All the default projects are included,
// plus any given in extraProj.
dashboard: unify post-submit & trybot policy in builders.go, not coordinator This removes a bunch of TODOs about scattered policy and moves it all into builders.go. While here, * add a bunch of tests * unexport some things * rename some things * document some things * adjust FreeBSD policy as function of branch (per Go 1.12 being last to support FreeBSD 10.x, and to unblock CL 165801) * set GO_BUILDER_SET_GOPROXY=coordinator for reverse buildlets, which is necessary to remove the oauth2 & build special cases in the config * change Elias Naur's mobile builder to how I think he wants it (he was fighting the old system) * add $HOME on the Solaris smartos builder, which was missing & causing tests to fail lately * make the (currently failing) longtest builder have GOPROXY set * remove an allocation in version.ParseReleaseBranch Fixes golang/go#9603 Updates golang/go#14594 Change-Id: I50a23ad7cdf478c95b14bee9b3931ba361baacfa Reviewed-on: https://go-review.googlesource.com/c/build/+/166218 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-03-07 20:44:41 +03:00
func defaultTrySet(extraProj ...string) func(proj, branch, goBranch string) bool {
return func(proj, branch, goBranch string) bool {
dashboard, cmd/coordinator: move more config out of coordinator into dashboard And with the increased flexibility, enable trybots for more repos for the "sys" project. Details: https://github.com/golang/go/issues/24893#issuecomment-433517519 Fixes golang/go#24893 Change-Id: I89ce6f219be986201332f8fad11c774bdd6246da Reviewed-on: https://go-review.googlesource.com/c/145157 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2018-10-26 22:21:58 +03:00
if proj == "go" {
return true
}
for _, p := range extraProj {
if proj == p {
return true
}
}
switch proj {
dashboard: add some "exp" trybots back, clean up some config Fixes golang/go#30781 Change-Id: I2672ca2949837c3cfe8fa6bd8011d1b9b43cca03 Reviewed-on: https://go-review.googlesource.com/c/build/+/167640 Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-03-14 23:35:32 +03:00
case "grpc-review":
dashboard, cmd/coordinator: move more config out of coordinator into dashboard And with the increased flexibility, enable trybots for more repos for the "sys" project. Details: https://github.com/golang/go/issues/24893#issuecomment-433517519 Fixes golang/go#24893 Change-Id: I89ce6f219be986201332f8fad11c774bdd6246da Reviewed-on: https://go-review.googlesource.com/c/145157 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2018-10-26 22:21:58 +03:00
return false
}
return true
}
}
// explicitTrySet returns a trybot policy function that reports
// whether a project should use trybots. Only the provided projects in
// projs are enabled.
dashboard: unify post-submit & trybot policy in builders.go, not coordinator This removes a bunch of TODOs about scattered policy and moves it all into builders.go. While here, * add a bunch of tests * unexport some things * rename some things * document some things * adjust FreeBSD policy as function of branch (per Go 1.12 being last to support FreeBSD 10.x, and to unblock CL 165801) * set GO_BUILDER_SET_GOPROXY=coordinator for reverse buildlets, which is necessary to remove the oauth2 & build special cases in the config * change Elias Naur's mobile builder to how I think he wants it (he was fighting the old system) * add $HOME on the Solaris smartos builder, which was missing & causing tests to fail lately * make the (currently failing) longtest builder have GOPROXY set * remove an allocation in version.ParseReleaseBranch Fixes golang/go#9603 Updates golang/go#14594 Change-Id: I50a23ad7cdf478c95b14bee9b3931ba361baacfa Reviewed-on: https://go-review.googlesource.com/c/build/+/166218 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-03-07 20:44:41 +03:00
func explicitTrySet(projs ...string) func(proj, branch, goBranch string) bool {
return func(proj, branch, goBranch string) bool {
dashboard, cmd/coordinator: move more config out of coordinator into dashboard And with the increased flexibility, enable trybots for more repos for the "sys" project. Details: https://github.com/golang/go/issues/24893#issuecomment-433517519 Fixes golang/go#24893 Change-Id: I89ce6f219be986201332f8fad11c774bdd6246da Reviewed-on: https://go-review.googlesource.com/c/145157 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2018-10-26 22:21:58 +03:00
for _, p := range projs {
if proj == p {
return true
}
}
return false
}
}
dashboard: move builder configuration out of coordinator into a package The new package will be used by the buildlet command line tool in the future. Also use the metadata package in the coordinator for the master build key, since it was already depending on that package. Might as well use it. Change-Id: I6ad7593787ebba364090951ccbaecb4aa38e1dd6 Reviewed-on: https://go-review.googlesource.com/2920 Reviewed-by: Andrew Gerrand <adg@golang.org>
2015-01-15 23:46:22 +03:00
func init() {
dashboard: add FreeBSD 11.2 and 10.4 builders Not yet the defaults or trybots until we see they're working regularly for a few days and compare timings, etc. Updates golang/go#25289 Change-Id: I4baed066b5f6d9f01e5454b716cacda4986620d0 Reviewed-on: https://go-review.googlesource.com/138258 Reviewed-by: Yuval Pavel Zholkover <paulzhol@gmail.com> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2018-09-28 02:43:25 +03:00
addBuilder(BuildConfig{
Name: "freebsd-amd64-11_2",
HostType: "host-freebsd-11_2",
dashboard, cmd/coordinator: move more config out of coordinator into dashboard And with the increased flexibility, enable trybots for more repos for the "sys" project. Details: https://github.com/golang/go/issues/24893#issuecomment-433517519 Fixes golang/go#24893 Change-Id: I89ce6f219be986201332f8fad11c774bdd6246da Reviewed-on: https://go-review.googlesource.com/c/145157 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2018-10-26 22:21:58 +03:00
tryBot: explicitTrySet("sys"),
dashboard: refactor mechanism for adjusting cmd/dist test policy Previously, the behavior of ShouldRunDistTest could be overridden by an individual builder by setting a the shouldRunDistTest field to a custom policy implementation. A pitfall of this approach is that it requires each custom policy to reimplement the default cmd/dist test policy as well. If that isn't done when attempting to make a small adjustment to the default policy, the result is that the default policy is ignored. The fasterTrybots policy, as the name suggests, exists to skip some tests in trybot mode. Due to the problem above, it was causing some builders to run more tests in trybot mode than they would if it weren't applied. For example, the freebsd-amd64-12_0 builder should have been skipping the "api" test in trybot mode, but wasn't. This refactor fixes that by modifying the approach for how individual builders can customize the cmd/dist test behavior. Now, a function distTestAdjust is called with the first parameter specifying the default policy decision. That function can adjust the decision as it sees fit, but otherwise it preserves its initial value in situations where no change is intended to be made. For golang/go#38279. Change-Id: I220248f51abfc8e77195ae0c9134a17f7cac9bc2 Reviewed-on: https://go-review.googlesource.com/c/build/+/227777 Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-04-09 19:47:03 +03:00
distTestAdjust: fasterTrybots,
dashboard: add FreeBSD 11.2 and 10.4 builders Not yet the defaults or trybots until we see they're working regularly for a few days and compare timings, etc. Updates golang/go#25289 Change-Id: I4baed066b5f6d9f01e5454b716cacda4986620d0 Reviewed-on: https://go-review.googlesource.com/138258 Reviewed-by: Yuval Pavel Zholkover <paulzhol@gmail.com> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2018-09-28 02:43:25 +03:00
numTryTestHelpers: 4,
dashboard: make FreeBSD 11.4 and 12.2 builders the default With this change: - The default FreeBSD builders used on master are 11.4 and 12.2. - The FreeBSD 12.0 builders have been removed. - The FreeBSD 11.2 builders have been set to run on at most Go 1.16. They will continue to be used for Go 1.16.x and 1.15.x minor releases. - The freebsd-amd64-race builder has been updated to use the FreeBSD 12.2 host. - The slowbot aliases for FreeBSD have been updated to use 12.2. Fixes golang/go#44431 Update golang/go#45727 Update golang/go#40561 Change-Id: Ib558fca65c2de1916b4633711d3e320c390bd2b2 Reviewed-on: https://go-review.googlesource.com/c/build/+/313070 Trust: Carlos Amedee <carlos@golang.org> Run-TryBot: Carlos Amedee <carlos@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2021-04-23 18:33:51 +03:00
buildsRepo: func(repo, branch, goBranch string) bool {
// This builder is still used by Go 1.16 and 1.15,
// so keep it around a bit longer. See golang.org/issue/45727.
// Test relevant Go versions so that we're better informed.
return atMostGo1(goBranch, 16) && buildRepoByDefault(repo)
},
dashboard: add FreeBSD 11.2 and 10.4 builders Not yet the defaults or trybots until we see they're working regularly for a few days and compare timings, etc. Updates golang/go#25289 Change-Id: I4baed066b5f6d9f01e5454b716cacda4986620d0 Reviewed-on: https://go-review.googlesource.com/138258 Reviewed-by: Yuval Pavel Zholkover <paulzhol@gmail.com> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2018-09-28 02:43:25 +03:00
})
dashboard: add amd64 FreeBSD 11.4 and 12.2 builders This change adds builders for amd64 FreeBSD 11.4 and 12.2. These builders are labeled with a known issue until they have been tested. They will not be used in trybots until the known issue label has been removed. Updates golang/go#44431 Change-Id: I902f2f6bd887135bf247feb1be0e0fd49f732614 Reviewed-on: https://go-review.googlesource.com/c/build/+/311869 Trust: Carlos Amedee <carlos@golang.org> Run-TryBot: Carlos Amedee <carlos@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2021-04-20 20:43:05 +03:00
addBuilder(BuildConfig{
Name: "freebsd-amd64-11_4",
HostType: "host-freebsd-11_4",
dashboard: make FreeBSD 11.4 and 12.2 builders the default With this change: - The default FreeBSD builders used on master are 11.4 and 12.2. - The FreeBSD 12.0 builders have been removed. - The FreeBSD 11.2 builders have been set to run on at most Go 1.16. They will continue to be used for Go 1.16.x and 1.15.x minor releases. - The freebsd-amd64-race builder has been updated to use the FreeBSD 12.2 host. - The slowbot aliases for FreeBSD have been updated to use 12.2. Fixes golang/go#44431 Update golang/go#45727 Update golang/go#40561 Change-Id: Ib558fca65c2de1916b4633711d3e320c390bd2b2 Reviewed-on: https://go-review.googlesource.com/c/build/+/313070 Trust: Carlos Amedee <carlos@golang.org> Run-TryBot: Carlos Amedee <carlos@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2021-04-23 18:33:51 +03:00
tryBot: explicitTrySet("sys"),
distTestAdjust: fasterTrybots,
dashboard: add amd64 FreeBSD 11.4 and 12.2 builders This change adds builders for amd64 FreeBSD 11.4 and 12.2. These builders are labeled with a known issue until they have been tested. They will not be used in trybots until the known issue label has been removed. Updates golang/go#44431 Change-Id: I902f2f6bd887135bf247feb1be0e0fd49f732614 Reviewed-on: https://go-review.googlesource.com/c/build/+/311869 Trust: Carlos Amedee <carlos@golang.org> Run-TryBot: Carlos Amedee <carlos@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2021-04-20 20:43:05 +03:00
numTryTestHelpers: 4,
})
dashboard: add FreeBSD 12.0-alpha8 builder Change-Id: I4f0278e41d801ae0f4358d73fcc2e17456091af6 Reviewed-on: https://go-review.googlesource.com/c/140179 Reviewed-by: Yuval Pavel Zholkover <paulzhol@gmail.com> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2018-10-05 23:57:29 +03:00
addBuilder(BuildConfig{
dashboard: make FreeBSD 11.4 and 12.2 builders the default With this change: - The default FreeBSD builders used on master are 11.4 and 12.2. - The FreeBSD 12.0 builders have been removed. - The FreeBSD 11.2 builders have been set to run on at most Go 1.16. They will continue to be used for Go 1.16.x and 1.15.x minor releases. - The freebsd-amd64-race builder has been updated to use the FreeBSD 12.2 host. - The slowbot aliases for FreeBSD have been updated to use 12.2. Fixes golang/go#44431 Update golang/go#45727 Update golang/go#40561 Change-Id: Ib558fca65c2de1916b4633711d3e320c390bd2b2 Reviewed-on: https://go-review.googlesource.com/c/build/+/313070 Trust: Carlos Amedee <carlos@golang.org> Run-TryBot: Carlos Amedee <carlos@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2021-04-23 18:33:51 +03:00
Name: "freebsd-amd64-12_2",
HostType: "host-freebsd-12_2",
tryBot: defaultTrySet("sys"),
dashboard: unify post-submit & trybot policy in builders.go, not coordinator This removes a bunch of TODOs about scattered policy and moves it all into builders.go. While here, * add a bunch of tests * unexport some things * rename some things * document some things * adjust FreeBSD policy as function of branch (per Go 1.12 being last to support FreeBSD 10.x, and to unblock CL 165801) * set GO_BUILDER_SET_GOPROXY=coordinator for reverse buildlets, which is necessary to remove the oauth2 & build special cases in the config * change Elias Naur's mobile builder to how I think he wants it (he was fighting the old system) * add $HOME on the Solaris smartos builder, which was missing & causing tests to fail lately * make the (currently failing) longtest builder have GOPROXY set * remove an allocation in version.ParseReleaseBranch Fixes golang/go#9603 Updates golang/go#14594 Change-Id: I50a23ad7cdf478c95b14bee9b3931ba361baacfa Reviewed-on: https://go-review.googlesource.com/c/build/+/166218 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-03-07 20:44:41 +03:00
dashboard: refactor mechanism for adjusting cmd/dist test policy Previously, the behavior of ShouldRunDistTest could be overridden by an individual builder by setting a the shouldRunDistTest field to a custom policy implementation. A pitfall of this approach is that it requires each custom policy to reimplement the default cmd/dist test policy as well. If that isn't done when attempting to make a small adjustment to the default policy, the result is that the default policy is ignored. The fasterTrybots policy, as the name suggests, exists to skip some tests in trybot mode. Due to the problem above, it was causing some builders to run more tests in trybot mode than they would if it weren't applied. For example, the freebsd-amd64-12_0 builder should have been skipping the "api" test in trybot mode, but wasn't. This refactor fixes that by modifying the approach for how individual builders can customize the cmd/dist test behavior. Now, a function distTestAdjust is called with the first parameter specifying the default policy decision. That function can adjust the decision as it sees fit, but otherwise it preserves its initial value in situations where no change is intended to be made. For golang/go#38279. Change-Id: I220248f51abfc8e77195ae0c9134a17f7cac9bc2 Reviewed-on: https://go-review.googlesource.com/c/build/+/227777 Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-04-09 19:47:03 +03:00
distTestAdjust: fasterTrybots, // If changing this policy, update TestShouldRunDistTest accordingly.
dashboard: add FreeBSD 12.0-alpha8 builder Change-Id: I4f0278e41d801ae0f4358d73fcc2e17456091af6 Reviewed-on: https://go-review.googlesource.com/c/140179 Reviewed-by: Yuval Pavel Zholkover <paulzhol@gmail.com> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2018-10-05 23:57:29 +03:00
numTryTestHelpers: 4,
})
dashboard: add amd64 FreeBSD 11.4 and 12.2 builders This change adds builders for amd64 FreeBSD 11.4 and 12.2. These builders are labeled with a known issue until they have been tested. They will not be used in trybots until the known issue label has been removed. Updates golang/go#44431 Change-Id: I902f2f6bd887135bf247feb1be0e0fd49f732614 Reviewed-on: https://go-review.googlesource.com/c/build/+/311869 Trust: Carlos Amedee <carlos@golang.org> Run-TryBot: Carlos Amedee <carlos@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2021-04-20 20:43:05 +03:00
addBuilder(BuildConfig{
Name: "freebsd-386-12_2",
HostType: "host-freebsd-12_2",
env: []string{"GOARCH=386", "GOHOSTARCH=386"},
dashboard: make FreeBSD 11.4 and 12.2 builders the default With this change: - The default FreeBSD builders used on master are 11.4 and 12.2. - The FreeBSD 12.0 builders have been removed. - The FreeBSD 11.2 builders have been set to run on at most Go 1.16. They will continue to be used for Go 1.16.x and 1.15.x minor releases. - The freebsd-amd64-race builder has been updated to use the FreeBSD 12.2 host. - The slowbot aliases for FreeBSD have been updated to use 12.2. Fixes golang/go#44431 Update golang/go#45727 Update golang/go#40561 Change-Id: Ib558fca65c2de1916b4633711d3e320c390bd2b2 Reviewed-on: https://go-review.googlesource.com/c/build/+/313070 Trust: Carlos Amedee <carlos@golang.org> Run-TryBot: Carlos Amedee <carlos@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2021-04-23 18:33:51 +03:00
distTestAdjust: fasterTrybots,
dashboard: add amd64 FreeBSD 11.4 and 12.2 builders This change adds builders for amd64 FreeBSD 11.4 and 12.2. These builders are labeled with a known issue until they have been tested. They will not be used in trybots until the known issue label has been removed. Updates golang/go#44431 Change-Id: I902f2f6bd887135bf247feb1be0e0fd49f732614 Reviewed-on: https://go-review.googlesource.com/c/build/+/311869 Trust: Carlos Amedee <carlos@golang.org> Run-TryBot: Carlos Amedee <carlos@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2021-04-20 20:43:05 +03:00
numTryTestHelpers: 4,
})
all: add env support, freebsd-386, freebsd-amd64-race, re-enable plan9 Update golang/go#9491 Update golang/go#8639 Change-Id: I1f43c751777f10a6d5870ca9bbb8c2a3430189bf Reviewed-on: https://go-review.googlesource.com/3170 Reviewed-by: Andrew Gerrand <adg@golang.org>
2015-01-22 02:15:48 +03:00
addBuilder(BuildConfig{
cmd/coordinator: delete useScheduler knob & old pre-scheduler hacks, code, TODOs Updates golang/go#19178 Change-Id: Id4d8b016c41f57bfaba5ee4ab046285607047493 Reviewed-on: https://go-review.googlesource.com/c/build/+/207178 Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-11-14 18:43:33 +03:00
Name: "freebsd-amd64-race",
dashboard: update freebsd-amd64-race builder host This change updates the version of FreeBSD used for the freebsd-amd64-race builder to 13.0. The slowbot aliases have been updated to use FreeBSD 13.0. Updates golang/go#46272 Updates golang/go#45903 Change-Id: I1092f9e6bf66a9f429b463dce5d0d5647d44eec8 Reviewed-on: https://go-review.googlesource.com/c/build/+/365616 Trust: Carlos Amedee <carlos@golang.org> Run-TryBot: Carlos Amedee <carlos@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2021-11-19 18:11:34 +03:00
HostType: "host-freebsd-13_0-big",
dashboard: add FreeBSD 11 builders Fixes golang/go#19097 Change-Id: I29725ef511f03e10d93561889b1a4a00eb959177 Reviewed-on: https://go-review.googlesource.com/37498 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2017-02-27 19:05:25 +03:00
})
dashboard: add freebsd-[amd64|386]-13_0 builders This change adds builders for FreeBSD 13.0. Updates golang/go#46272 Fixes golang/go#45903 Change-Id: I92f68f8d5eeee2d3bae8864dbfe5ee61b93ea677 Reviewed-on: https://go-review.googlesource.com/c/build/+/365314 Trust: Carlos Amedee <carlos@golang.org> Run-TryBot: Carlos Amedee <carlos@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2021-11-19 00:33:40 +03:00
addBuilder(BuildConfig{
Name: "freebsd-amd64-13_0",
HostType: "host-freebsd-13_0",
tryBot: explicitTrySet("sys"),
distTestAdjust: fasterTrybots, // If changing this policy, update TestShouldRunDistTest accordingly.
numTryTestHelpers: 4,
})
addBuilder(BuildConfig{
Name: "freebsd-386-13_0",
HostType: "host-freebsd-13_0",
tryBot: explicitTrySet("sys"),
env: []string{"GOARCH=386", "GOHOSTARCH=386"},
distTestAdjust: fasterTrybots,
numTryTestHelpers: 4,
})
dashboard: add FreeBSD 11.2 and 10.4 builders Not yet the defaults or trybots until we see they're working regularly for a few days and compare timings, etc. Updates golang/go#25289 Change-Id: I4baed066b5f6d9f01e5454b716cacda4986620d0 Reviewed-on: https://go-review.googlesource.com/138258 Reviewed-by: Yuval Pavel Zholkover <paulzhol@gmail.com> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2018-09-28 02:43:25 +03:00
addBuilder(BuildConfig{
dashboard: refactor mechanism for adjusting cmd/dist test policy Previously, the behavior of ShouldRunDistTest could be overridden by an individual builder by setting a the shouldRunDistTest field to a custom policy implementation. A pitfall of this approach is that it requires each custom policy to reimplement the default cmd/dist test policy as well. If that isn't done when attempting to make a small adjustment to the default policy, the result is that the default policy is ignored. The fasterTrybots policy, as the name suggests, exists to skip some tests in trybot mode. Due to the problem above, it was causing some builders to run more tests in trybot mode than they would if it weren't applied. For example, the freebsd-amd64-12_0 builder should have been skipping the "api" test in trybot mode, but wasn't. This refactor fixes that by modifying the approach for how individual builders can customize the cmd/dist test behavior. Now, a function distTestAdjust is called with the first parameter specifying the default policy decision. That function can adjust the decision as it sees fit, but otherwise it preserves its initial value in situations where no change is intended to be made. For golang/go#38279. Change-Id: I220248f51abfc8e77195ae0c9134a17f7cac9bc2 Reviewed-on: https://go-review.googlesource.com/c/build/+/227777 Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-04-09 19:47:03 +03:00
Name: "freebsd-386-11_2",
HostType: "host-freebsd-11_2",
distTestAdjust: noTestDirAndNoReboot,
tryBot: explicitTrySet("sys"),
env: []string{"GOARCH=386", "GOHOSTARCH=386"},
dashboard: make FreeBSD 11.4 and 12.2 builders the default With this change: - The default FreeBSD builders used on master are 11.4 and 12.2. - The FreeBSD 12.0 builders have been removed. - The FreeBSD 11.2 builders have been set to run on at most Go 1.16. They will continue to be used for Go 1.16.x and 1.15.x minor releases. - The freebsd-amd64-race builder has been updated to use the FreeBSD 12.2 host. - The slowbot aliases for FreeBSD have been updated to use 12.2. Fixes golang/go#44431 Update golang/go#45727 Update golang/go#40561 Change-Id: Ib558fca65c2de1916b4633711d3e320c390bd2b2 Reviewed-on: https://go-review.googlesource.com/c/build/+/313070 Trust: Carlos Amedee <carlos@golang.org> Run-TryBot: Carlos Amedee <carlos@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2021-04-23 18:33:51 +03:00
buildsRepo: func(repo, branch, goBranch string) bool {
// This builder is still used by Go 1.16 and 1.15,
// so keep it around a bit longer. See golang.org/issue/45727.
// Test relevant Go versions so that we're better informed.
return atMostGo1(goBranch, 16) && buildRepoByDefault(repo)
},
dashboard: add FreeBSD 11.2 and 10.4 builders Not yet the defaults or trybots until we see they're working regularly for a few days and compare timings, etc. Updates golang/go#25289 Change-Id: I4baed066b5f6d9f01e5454b716cacda4986620d0 Reviewed-on: https://go-review.googlesource.com/138258 Reviewed-by: Yuval Pavel Zholkover <paulzhol@gmail.com> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2018-09-28 02:43:25 +03:00
})
dashboard: add amd64 FreeBSD 11.4 and 12.2 builders This change adds builders for amd64 FreeBSD 11.4 and 12.2. These builders are labeled with a known issue until they have been tested. They will not be used in trybots until the known issue label has been removed. Updates golang/go#44431 Change-Id: I902f2f6bd887135bf247feb1be0e0fd49f732614 Reviewed-on: https://go-review.googlesource.com/c/build/+/311869 Trust: Carlos Amedee <carlos@golang.org> Run-TryBot: Carlos Amedee <carlos@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2021-04-20 20:43:05 +03:00
addBuilder(BuildConfig{
Name: "freebsd-386-11_4",
HostType: "host-freebsd-11_4",
distTestAdjust: noTestDirAndNoReboot,
dashboard: make FreeBSD 11.4 and 12.2 builders the default With this change: - The default FreeBSD builders used on master are 11.4 and 12.2. - The FreeBSD 12.0 builders have been removed. - The FreeBSD 11.2 builders have been set to run on at most Go 1.16. They will continue to be used for Go 1.16.x and 1.15.x minor releases. - The freebsd-amd64-race builder has been updated to use the FreeBSD 12.2 host. - The slowbot aliases for FreeBSD have been updated to use 12.2. Fixes golang/go#44431 Update golang/go#45727 Update golang/go#40561 Change-Id: Ib558fca65c2de1916b4633711d3e320c390bd2b2 Reviewed-on: https://go-review.googlesource.com/c/build/+/313070 Trust: Carlos Amedee <carlos@golang.org> Run-TryBot: Carlos Amedee <carlos@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2021-04-23 18:33:51 +03:00
tryBot: explicitTrySet("sys"),
dashboard: add amd64 FreeBSD 11.4 and 12.2 builders This change adds builders for amd64 FreeBSD 11.4 and 12.2. These builders are labeled with a known issue until they have been tested. They will not be used in trybots until the known issue label has been removed. Updates golang/go#44431 Change-Id: I902f2f6bd887135bf247feb1be0e0fd49f732614 Reviewed-on: https://go-review.googlesource.com/c/build/+/311869 Trust: Carlos Amedee <carlos@golang.org> Run-TryBot: Carlos Amedee <carlos@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2021-04-20 20:43:05 +03:00
env: []string{"GOARCH=386", "GOHOSTARCH=386"},
})
build: convert most Linux builders to be VM-based We still use Dockerfiles to describe them, but then a new tool (docker2boot) converts them into VM images suitable for booting on GCE, running the buildlet like all the other operating systems. Things are easier if everything acts the same way. Note that since we're no longer moving around Docker images, the image size and the layer accumulation cruft no longer matters. We can now have Dockerfile lines like "RUN rm -rf /usr/share/doc" and it actually results in a smaller VM image, since we just "docker export" the files out of it to create the VM image. This doesn't yet convert the clang, sid, or nacl builders. The coordinator still runs those under Docker directly. A future change will convert those to VM images as well. Change-Id: Iedb136ae3daf888c955eb843bdcc9a638d08f5e9 Reviewed-on: https://go-review.googlesource.com/3341 Reviewed-by: Andrew Gerrand <adg@golang.org>
2015-01-28 01:22:21 +03:00
addBuilder(BuildConfig{
dashboard: refactor mechanism for adjusting cmd/dist test policy Previously, the behavior of ShouldRunDistTest could be overridden by an individual builder by setting a the shouldRunDistTest field to a custom policy implementation. A pitfall of this approach is that it requires each custom policy to reimplement the default cmd/dist test policy as well. If that isn't done when attempting to make a small adjustment to the default policy, the result is that the default policy is ignored. The fasterTrybots policy, as the name suggests, exists to skip some tests in trybot mode. Due to the problem above, it was causing some builders to run more tests in trybot mode than they would if it weren't applied. For example, the freebsd-amd64-12_0 builder should have been skipping the "api" test in trybot mode, but wasn't. This refactor fixes that by modifying the approach for how individual builders can customize the cmd/dist test behavior. Now, a function distTestAdjust is called with the first parameter specifying the default policy decision. That function can adjust the decision as it sees fit, but otherwise it preserves its initial value in situations where no change is intended to be made. For golang/go#38279. Change-Id: I220248f51abfc8e77195ae0c9134a17f7cac9bc2 Reviewed-on: https://go-review.googlesource.com/c/build/+/227777 Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-04-09 19:47:03 +03:00
Name: "linux-386",
dashboard: update Debian version for 386 builders This change updates various linux-386 builders after the release of bullseye: - linux-386 has been updated to bullseye. - linux-386-jessie has had the notes updated. - linux-386-buster has been added. Updates golang/go#47721 Change-Id: Iab0048fcad4d51839013074a51ad37b424d3a7af Reviewed-on: https://go-review.googlesource.com/c/build/+/343632 Trust: Carlos Amedee <carlos@golang.org> Run-TryBot: Carlos Amedee <carlos@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2021-08-19 00:45:09 +03:00
HostType: "host-linux-bullseye",
dashboard: refactor mechanism for adjusting cmd/dist test policy Previously, the behavior of ShouldRunDistTest could be overridden by an individual builder by setting a the shouldRunDistTest field to a custom policy implementation. A pitfall of this approach is that it requires each custom policy to reimplement the default cmd/dist test policy as well. If that isn't done when attempting to make a small adjustment to the default policy, the result is that the default policy is ignored. The fasterTrybots policy, as the name suggests, exists to skip some tests in trybot mode. Due to the problem above, it was causing some builders to run more tests in trybot mode than they would if it weren't applied. For example, the freebsd-amd64-12_0 builder should have been skipping the "api" test in trybot mode, but wasn't. This refactor fixes that by modifying the approach for how individual builders can customize the cmd/dist test behavior. Now, a function distTestAdjust is called with the first parameter specifying the default policy decision. That function can adjust the decision as it sees fit, but otherwise it preserves its initial value in situations where no change is intended to be made. For golang/go#38279. Change-Id: I220248f51abfc8e77195ae0c9134a17f7cac9bc2 Reviewed-on: https://go-review.googlesource.com/c/build/+/227777 Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-04-09 19:47:03 +03:00
distTestAdjust: fasterTrybots,
tryBot: defaultTrySet(),
dashboard: update Debian version for 386 builders This change updates various linux-386 builders after the release of bullseye: - linux-386 has been updated to bullseye. - linux-386-jessie has had the notes updated. - linux-386-buster has been added. Updates golang/go#47721 Change-Id: Iab0048fcad4d51839013074a51ad37b424d3a7af Reviewed-on: https://go-review.googlesource.com/c/build/+/343632 Trust: Carlos Amedee <carlos@golang.org> Run-TryBot: Carlos Amedee <carlos@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2021-08-19 00:45:09 +03:00
Notes: "Debian stable (currently Debian bullseye).",
env, buildlet, dashboard: support outbound firewalls for Linux Updates golang/go#30612 Change-Id: Ib13a286d0944a7f4a13b9e93a01533693052858b Reviewed-on: https://go-review.googlesource.com/c/build/+/165637 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-03-06 05:06:17 +03:00
env: []string{
"GOARCH=386",
"GOHOSTARCH=386",
cmd/coordinator: follow module proxy redirects And because this uses new-in-Go1.13 http.Header.Clone, restrict all the files to Go 1.13+ and bump the Dockerfile Go version. Fixes golang/go#33928 Change-Id: I147cc4465e393d2c34e59c70edf65f3901a5e132 Reviewed-on: https://go-review.googlesource.com/c/build/+/192341 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-08-29 21:28:01 +03:00
"GO_DISABLE_OUTBOUND_NETWORK=1",
env, buildlet, dashboard: support outbound firewalls for Linux Updates golang/go#30612 Change-Id: Ib13a286d0944a7f4a13b9e93a01533693052858b Reviewed-on: https://go-review.googlesource.com/c/build/+/165637 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-03-06 05:06:17 +03:00
},
all: split builder config into builder & host configs Our builders are named of the form "GOOS-GOARCH" or "GOOS-GOARCH-suffix". Over time we've grown many builders. This CL doesn't change that. Builders continue to be named and operate as before. Previously the build configuration file (dashboard/builders.go) made each builder type ("linux-amd64-race", etc) define how to create a host running a buildlet of that type, even though many builders had identical host configs. For example, these builders all share the same host type (a Kubernetes container): linux-amd64 linux-amd64-race linux-386 linux-386-387 And these are the same host type (a GCE VM): windows-amd64-gce windows-amd64-race windows-386-gce This CL creates a new concept of a "hostType" which defines how the buildlet is created (Kube, GCE, Reverse, and how), and then each builder itself references a host type. Users never see the hostType. (except perhaps in gomote list output) But they at least never need to care about them. Reverse buildlets now can only be one hostType at a time, which simplifies things. We were no longer using multiple roles per machine once moving to VMs for OS X. gomote continues to operate as it did previously but its underlying protocol changed and clients will need to be updated. As a new feature, gomote now has a new flag to let you reuse a buildlet host connection for different builder rules if they share the same underlying host type. But users can ignore that. This CL is a long-standing TODO (previously attempted and aborted) and will make many things easier and faster, including the linux-arm cross-compilation effort, and keeping pre-warmed buildlets of VM types ready to go. Updates golang/go#17104 Change-Id: Iad8387f48680424a8441e878a2f4762bf79ea4d2 Reviewed-on: https://go-review.googlesource.com/29551 Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2016-09-22 00:27:37 +03:00
numTestHelpers: 1,
numTryTestHelpers: 3,
build: convert most Linux builders to be VM-based We still use Dockerfiles to describe them, but then a new tool (docker2boot) converts them into VM images suitable for booting on GCE, running the buildlet like all the other operating systems. Things are easier if everything acts the same way. Note that since we're no longer moving around Docker images, the image size and the layer accumulation cruft no longer matters. We can now have Dockerfile lines like "RUN rm -rf /usr/share/doc" and it actually results in a smaller VM image, since we just "docker export" the files out of it to create the VM image. This doesn't yet convert the clang, sid, or nacl builders. The coordinator still runs those under Docker directly. A future change will convert those to VM images as well. Change-Id: Iedb136ae3daf888c955eb843bdcc9a638d08f5e9 Reviewed-on: https://go-review.googlesource.com/3341 Reviewed-by: Andrew Gerrand <adg@golang.org>
2015-01-28 01:22:21 +03:00
})
dashboard: add linux-386-softfloat builder It's based on the linux-386-387 builder, but with GO386=softfloat and Debian 9 (stretch) instead of Debian 8 (jessie). Test it on a subset of golang.org/x repos (golang.org/x/crypto), since that should provide sufficient test coverage. For golang/go#41840. For golang/go#41848. Change-Id: Ida95fce198056a05c359ac5ab816c582bdd5930f Reviewed-on: https://go-review.googlesource.com/c/build/+/260201 Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Trust: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-10-07 18:24:42 +03:00
addBuilder(BuildConfig{
Name: "linux-386-softfloat",
Notes: "GO386=softfloat",
buildsRepo: func(repo, branch, goBranch string) bool {
// GO386=softfloat is added in Go 1.16 (golang.org/issue/41848).
return atLeastGo1(goBranch, 16) && (repo == "go" || repo == "crypto")
},
GoDeps: []string{
"04b8a9fea57e37589d82410281f22ebde0027808", // CL 260017, "all: implement GO386=softfloat".
},
HostType: "host-linux-stretch",
env: []string{"GOARCH=386", "GOHOSTARCH=386", "GO386=softfloat"},
})
build: convert most Linux builders to be VM-based We still use Dockerfiles to describe them, but then a new tool (docker2boot) converts them into VM images suitable for booting on GCE, running the buildlet like all the other operating systems. Things are easier if everything acts the same way. Note that since we're no longer moving around Docker images, the image size and the layer accumulation cruft no longer matters. We can now have Dockerfile lines like "RUN rm -rf /usr/share/doc" and it actually results in a smaller VM image, since we just "docker export" the files out of it to create the VM image. This doesn't yet convert the clang, sid, or nacl builders. The coordinator still runs those under Docker directly. A future change will convert those to VM images as well. Change-Id: Iedb136ae3daf888c955eb843bdcc9a638d08f5e9 Reviewed-on: https://go-review.googlesource.com/3341 Reviewed-by: Andrew Gerrand <adg@golang.org>
2015-01-28 01:22:21 +03:00
addBuilder(BuildConfig{
dashboard: disable more builders Make exp off by default, as it barely passes anywhere. Lock in the places where it does pass. Also disable x/net master combined with Go 1.11 on FreeBSD 32-bit. It doesn't work on our builders and it'll never be fixed (we're not going to make changes to Go 1.11) and there's resistance to skipping those tests (my CL 170777), so I'm just turning off the builder. That's not ideal, but it's better than people getting used to red on the dashboard. Fixes golang/go#31221 Change-Id: Id0228e4b0ee73715a0c2da84f68dcfc989a48aaf Reviewed-on: https://go-review.googlesource.com/c/build/+/170782 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-04-05 08:26:05 +03:00
Name: "linux-amd64",
dashboard: update Debian version for amd64 builders This change updates various linux-amd64 builders after the release of bullseye: - linux-amd64 has been updated to bullseye. - linux-amd64-nocgo has been updated to bullseye. - linux-amd64-noopt has been updated to bullseye. - linux-amd64-ssacheck has been updated to bullseye. - linux-amd64-racecompile has been updated to bullseye. - linux-amd64-race has been updated to bullseye. - linux-amd64-jessie has had the notes updated. - misc-compile has been updated to bullseye. Fixes golang/go#47721 Change-Id: I29c5d76da6ad6b28647bd423a8ef7dc13b919ca2 Reviewed-on: https://go-review.googlesource.com/c/build/+/343633 Trust: Carlos Amedee <carlos@golang.org> Run-TryBot: Carlos Amedee <carlos@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2021-08-19 20:27:43 +03:00
HostType: "host-linux-bullseye",
dashboard: disable more builders Make exp off by default, as it barely passes anywhere. Lock in the places where it does pass. Also disable x/net master combined with Go 1.11 on FreeBSD 32-bit. It doesn't work on our builders and it'll never be fixed (we're not going to make changes to Go 1.11) and there's resistance to skipping those tests (my CL 170777), so I'm just turning off the builder. That's not ideal, but it's better than people getting used to red on the dashboard. Fixes golang/go#31221 Change-Id: Id0228e4b0ee73715a0c2da84f68dcfc989a48aaf Reviewed-on: https://go-review.googlesource.com/c/build/+/170782 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-04-05 08:26:05 +03:00
tryBot: defaultTrySet(),
dashboard: don't build x/build on so many builders It was kinda overkill. Change-Id: Ied7fc42aa59a1ed18cc1da20e51c8ed386d84629 Reviewed-on: https://go-review.googlesource.com/c/build/+/210378 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-12-07 08:14:28 +03:00
buildsRepo: defaultPlusExpBuild,
env, buildlet, dashboard: support outbound firewalls for Linux Updates golang/go#30612 Change-Id: Ib13a286d0944a7f4a13b9e93a01533693052858b Reviewed-on: https://go-review.googlesource.com/c/build/+/165637 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-03-06 05:06:17 +03:00
env: []string{
cmd/coordinator: follow module proxy redirects And because this uses new-in-Go1.13 http.Header.Clone, restrict all the files to Go 1.13+ and bump the Dockerfile Go version. Fixes golang/go#33928 Change-Id: I147cc4465e393d2c34e59c70edf65f3901a5e132 Reviewed-on: https://go-review.googlesource.com/c/build/+/192341 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-08-29 21:28:01 +03:00
"GO_DISABLE_OUTBOUND_NETWORK=1",
env, buildlet, dashboard: support outbound firewalls for Linux Updates golang/go#30612 Change-Id: Ib13a286d0944a7f4a13b9e93a01533693052858b Reviewed-on: https://go-review.googlesource.com/c/build/+/165637 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-03-06 05:06:17 +03:00
},
all: use Container-Optimized VMs instead of Kubernetes for buildlet containers Fixes golang/go#25108 Change-Id: I084669b52b699700ed26a7fdd890d9205a8b9dc9 Reviewed-on: https://go-review.googlesource.com/111267 Reviewed-by: Andrew Bonventre <andybons@golang.org>
2018-05-04 05:42:17 +03:00
numTestHelpers: 1,
numTryTestHelpers: 4,
all: run dev coordinator locally, allow debug in prod * flag allows debug (trigger a build) in prod mode * use default zone (us-central1-f) in staging * compute.CloudPlatformScope satisfies compute and storage scope requirements * new buildlet for linux-amd64 on Kubernetes Updates golang/go#12546 Change-Id: I1a0cbc0e0c0552b7ac56943c646378194508d48f Reviewed-on: https://go-review.googlesource.com/17102 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2015-11-20 21:00:06 +03:00
})
dashboard, buildlet: add a disabled builder with nested virt, for testing This adds a linux-amd64 COS builder that should be just like our existing linux-amd64 COS builder except that it's using a forked image that has the VMX license bit enabled for nested virtualization. (GCE appears to be using the license mechanism as some sort of opt-in mechanism for features that aren't yet GA; might go away?) Once this is in, it won't do any new builds as regular+trybot builders are disabled. But it means I can then use gomote + debugnewvm to work on preparing the other four image types. Updates golang/go#15581 (solaris) Updates golang/go#23060 (dragonfly) Updates golang/go#30262 (riscv) Updates golang/go#30267 (fuchsia) Updates golang/go#23824 (android) Change-Id: Ic55f17eea17908dba7f58618d8cd162a2ed9b015 Reviewed-on: https://go-review.googlesource.com/c/162959 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-02-16 01:05:42 +03:00
addBuilder(BuildConfig{
all: add a macOS 10.14 (Mojave) builder Fixes golang/go#27806 Change-Id: I576f563acb2c50cd0456cd4e6a1271b9aa59c9df Reviewed-on: https://go-review.googlesource.com/c/build/+/169498 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-03-27 02:04:31 +03:00
Name: "linux-amd64-vmx",
HostType: "host-linux-stretch-vmx",
buildsRepo: disabledBuilder,
dashboard, buildlet: add a disabled builder with nested virt, for testing This adds a linux-amd64 COS builder that should be just like our existing linux-amd64 COS builder except that it's using a forked image that has the VMX license bit enabled for nested virtualization. (GCE appears to be using the license mechanism as some sort of opt-in mechanism for features that aren't yet GA; might go away?) Once this is in, it won't do any new builds as regular+trybot builders are disabled. But it means I can then use gomote + debugnewvm to work on preparing the other four image types. Updates golang/go#15581 (solaris) Updates golang/go#23060 (dragonfly) Updates golang/go#30262 (riscv) Updates golang/go#30267 (fuchsia) Updates golang/go#23824 (android) Change-Id: Ic55f17eea17908dba7f58618d8cd162a2ed9b015 Reviewed-on: https://go-review.googlesource.com/c/162959 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-02-16 01:05:42 +03:00
})
dashboard: remove alpine builder It's doing no good being red all the time, especially as nobody is working on it. Updates golang/go#22689 Updates golang/go#19938 Change-Id: Icb947878d85e920f24ea458eb0f319844ca5bd60 Reviewed-on: https://go-review.googlesource.com/78575 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2017-11-18 06:31:35 +03:00
const testAlpine = false // Issue 22689 (hide all red builders), Issue 19938 (get Alpine passing)
if testAlpine {
addBuilder(BuildConfig{
Name: "linux-amd64-alpine",
HostType: "host-linux-x86-alpine",
})
}
dashboard: include at most 3 ports per misc-compile TryBot Between Go 1.15, 1.16 and tip, the following 29 ports don't have a real TryBot and instead rely on misc-compile TryBots for their pre-submit coverage: • aix/ppc64 • darwin/amd64 • darwin/arm64 • dragonfly/amd64 • freebsd/386 • freebsd/arm • freebsd/arm64 • illumos/amd64 • linux/mips • linux/mips64 • linux/mipsle • linux/mips64le • linux/ppc64 • linux/ppc64le • linux/riscv64 • linux/s390x • netbsd/386 • netbsd/amd64 • netbsd/arm • netbsd/arm64 • openbsd/386 • openbsd/arm • openbsd/arm64 • openbsd/mips64 • plan9/386 • plan9/amd64 • plan9/arm • solaris/amd64 • windows/arm The previous approach for misc-compile target selection was to break them up primarily by GOOS value. However, as new architectures were added over time, some misc-compile TryBots got to a point where they were testing upwards of 5 ports (for example, misc-compile-openbsd was testing 386, amd64, arm, arm64, and mips64 architectures). Since each port is tested sequentially, allocating too many to one misc-compile TryBot can cause it to become the bottleneck of an entire TryBot run, exceeding the 10 minute completion time goal. Arrange it so misc-compile TryBot target selection is done explicitly in x/build, and pick 3 as max number of targets per TryBot for now. Based on recent timing observations, that should strike a decent balance between resource use (spinning up a builder) vs chance of a misc-compile TryBot becoming a bottleneck. It will also give us an opportunity to compare timing of 1, 2 and 3 targets per misc-compile in the future. (When we start tracking timing for TryBot completion time holistically, we'll be in a better position to refine this strategy further.) Making misc-compile target selection explicit in x/build also enables removing unnecessary duplicate misc-compile coverage from ports that already have a real TryBot (for example, openbsd/amd64 was previously tested via both the openbsd-amd64-68 TryBot and misc-compile-openbsd). This shouldn't be needed, so it's no longer done. For golang/go#17104. Fixes golang/go#32632. Change-Id: Iac918377b91af3e48780b38ffdf3153e213eeba2 Reviewed-on: https://go-review.googlesource.com/c/build/+/313210 Trust: Dmitri Shuralyov <dmitshur@golang.org> Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Heschi Kreinick <heschi@google.com> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-04-23 23:56:37 +03:00
// addMiscCompileGo1 adds a misc-compile TryBot that
// runs buildall.bash on the specified target(s), up to 3 max.
// The targets are matched against the "go tool dist list" name,
// but with hyphens instead of forward slashes ("linux-amd64", etc).
dashboard: promote misc-compile-darwinarm64 builder to a TryBot Testing shows that misc-compile-darwinarm64 is working as expected on recent commits. Promote it to a normal misc-compile TryBot now, with the exception that it needs to run on Go 1.16 and newer only. Fixes golang/go#42341. Change-Id: Ib4de19a2ff904fb9de056db5076ac63690eb934c Reviewed-on: https://go-review.googlesource.com/c/build/+/267123 Trust: Dmitri Shuralyov <dmitshur@golang.org> Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-11-03 06:25:51 +03:00
// If min is non-zero, it specifies the minimum Go 1.x version.
dashboard: include at most 3 ports per misc-compile TryBot Between Go 1.15, 1.16 and tip, the following 29 ports don't have a real TryBot and instead rely on misc-compile TryBots for their pre-submit coverage: • aix/ppc64 • darwin/amd64 • darwin/arm64 • dragonfly/amd64 • freebsd/386 • freebsd/arm • freebsd/arm64 • illumos/amd64 • linux/mips • linux/mips64 • linux/mipsle • linux/mips64le • linux/ppc64 • linux/ppc64le • linux/riscv64 • linux/s390x • netbsd/386 • netbsd/amd64 • netbsd/arm • netbsd/arm64 • openbsd/386 • openbsd/arm • openbsd/arm64 • openbsd/mips64 • plan9/386 • plan9/amd64 • plan9/arm • solaris/amd64 • windows/arm The previous approach for misc-compile target selection was to break them up primarily by GOOS value. However, as new architectures were added over time, some misc-compile TryBots got to a point where they were testing upwards of 5 ports (for example, misc-compile-openbsd was testing 386, amd64, arm, arm64, and mips64 architectures). Since each port is tested sequentially, allocating too many to one misc-compile TryBot can cause it to become the bottleneck of an entire TryBot run, exceeding the 10 minute completion time goal. Arrange it so misc-compile TryBot target selection is done explicitly in x/build, and pick 3 as max number of targets per TryBot for now. Based on recent timing observations, that should strike a decent balance between resource use (spinning up a builder) vs chance of a misc-compile TryBot becoming a bottleneck. It will also give us an opportunity to compare timing of 1, 2 and 3 targets per misc-compile in the future. (When we start tracking timing for TryBot completion time holistically, we'll be in a better position to refine this strategy further.) Making misc-compile target selection explicit in x/build also enables removing unnecessary duplicate misc-compile coverage from ports that already have a real TryBot (for example, openbsd/amd64 was previously tested via both the openbsd-amd64-68 TryBot and misc-compile-openbsd). This shouldn't be needed, so it's no longer done. For golang/go#17104. Fixes golang/go#32632. Change-Id: Iac918377b91af3e48780b38ffdf3153e213eeba2 Reviewed-on: https://go-review.googlesource.com/c/build/+/313210 Trust: Dmitri Shuralyov <dmitshur@golang.org> Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Heschi Kreinick <heschi@google.com> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-04-23 23:56:37 +03:00
addMiscCompileGo1 := func(min int, suffix string, targets ...string) {
if len(targets) > 3 {
// This limit will do until we have better visibility
// into holistic TryBot completion times via metrics.
panic("at most 3 targets may be specified to avoid making TryBots slow; see issues 32632 and 17104")
}
dashboard: promote misc-compile-darwinarm64 builder to a TryBot Testing shows that misc-compile-darwinarm64 is working as expected on recent commits. Promote it to a normal misc-compile TryBot now, with the exception that it needs to run on Go 1.16 and newer only. Fixes golang/go#42341. Change-Id: Ib4de19a2ff904fb9de056db5076ac63690eb934c Reviewed-on: https://go-review.googlesource.com/c/build/+/267123 Trust: Dmitri Shuralyov <dmitshur@golang.org> Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-11-03 06:25:51 +03:00
var v types.MajorMinor
var alsoNote string
if min != 0 {
v = types.MajorMinor{1, min}
alsoNote = fmt.Sprintf(" Applies to Go 1.%d and newer.", min)
}
cmd/coordinator: shard the misc-compile builder over 4 builders Now that it's on Kubernetes, it's fast and cheap to shard it wider. Change-Id: I2858f4c417ef26661575a348fbfc47f4c25d81a2 Reviewed-on: https://go-review.googlesource.com/22795 Reviewed-by: Andrew Gerrand <adg@golang.org>
2016-05-05 03:42:49 +03:00
addBuilder(BuildConfig{
dashboard: enable outbound firewall on a few more Linux-based builders Updates golang/go#30612 Change-Id: Id472e753043976b8671c15491b82bb57e3804d95 Reviewed-on: https://go-review.googlesource.com/c/build/+/165638 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-03-06 08:20:12 +03:00
Name: "misc-compile" + suffix,
HostType: "host-linux-jessie",
tryBot: defaultTrySet(),
env: []string{
cmd/coordinator: follow module proxy redirects And because this uses new-in-Go1.13 http.Header.Clone, restrict all the files to Go 1.13+ and bump the Dockerfile Go version. Fixes golang/go#33928 Change-Id: I147cc4465e393d2c34e59c70edf65f3901a5e132 Reviewed-on: https://go-review.googlesource.com/c/build/+/192341 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-08-29 21:28:01 +03:00
"GO_DISABLE_OUTBOUND_NETWORK=1",
dashboard: enable outbound firewall on a few more Linux-based builders Updates golang/go#30612 Change-Id: Id472e753043976b8671c15491b82bb57e3804d95 Reviewed-on: https://go-review.googlesource.com/c/build/+/165638 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-03-06 08:20:12 +03:00
},
dashboard: promote misc-compile-darwinarm64 builder to a TryBot Testing shows that misc-compile-darwinarm64 is working as expected on recent commits. Promote it to a normal misc-compile TryBot now, with the exception that it needs to run on Go 1.16 and newer only. Fixes golang/go#42341. Change-Id: Ib4de19a2ff904fb9de056db5076ac63690eb934c Reviewed-on: https://go-review.googlesource.com/c/build/+/267123 Trust: Dmitri Shuralyov <dmitshur@golang.org> Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-11-03 06:25:51 +03:00
tryOnly: true,
MinimumGoVersion: v,
CompileOnly: true,
dashboard: include at most 3 ports per misc-compile TryBot Between Go 1.15, 1.16 and tip, the following 29 ports don't have a real TryBot and instead rely on misc-compile TryBots for their pre-submit coverage: • aix/ppc64 • darwin/amd64 • darwin/arm64 • dragonfly/amd64 • freebsd/386 • freebsd/arm • freebsd/arm64 • illumos/amd64 • linux/mips • linux/mips64 • linux/mipsle • linux/mips64le • linux/ppc64 • linux/ppc64le • linux/riscv64 • linux/s390x • netbsd/386 • netbsd/amd64 • netbsd/arm • netbsd/arm64 • openbsd/386 • openbsd/arm • openbsd/arm64 • openbsd/mips64 • plan9/386 • plan9/amd64 • plan9/arm • solaris/amd64 • windows/arm The previous approach for misc-compile target selection was to break them up primarily by GOOS value. However, as new architectures were added over time, some misc-compile TryBots got to a point where they were testing upwards of 5 ports (for example, misc-compile-openbsd was testing 386, amd64, arm, arm64, and mips64 architectures). Since each port is tested sequentially, allocating too many to one misc-compile TryBot can cause it to become the bottleneck of an entire TryBot run, exceeding the 10 minute completion time goal. Arrange it so misc-compile TryBot target selection is done explicitly in x/build, and pick 3 as max number of targets per TryBot for now. Based on recent timing observations, that should strike a decent balance between resource use (spinning up a builder) vs chance of a misc-compile TryBot becoming a bottleneck. It will also give us an opportunity to compare timing of 1, 2 and 3 targets per misc-compile in the future. (When we start tracking timing for TryBot completion time holistically, we'll be in a better position to refine this strategy further.) Making misc-compile target selection explicit in x/build also enables removing unnecessary duplicate misc-compile coverage from ports that already have a real TryBot (for example, openbsd/amd64 was previously tested via both the openbsd-amd64-68 TryBot and misc-compile-openbsd). This shouldn't be needed, so it's no longer done. For golang/go#17104. Fixes golang/go#32632. Change-Id: Iac918377b91af3e48780b38ffdf3153e213eeba2 Reviewed-on: https://go-review.googlesource.com/c/build/+/313210 Trust: Dmitri Shuralyov <dmitshur@golang.org> Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Heschi Kreinick <heschi@google.com> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-04-23 23:56:37 +03:00
Notes: "Runs buildall.bash to cross-compile & vet std+cmd packages for " + strings.Join(targets, " & ") + ", but doesn't run any tests." + alsoNote,
cmd/coordinator: shard the misc-compile builder over 4 builders Now that it's on Kubernetes, it's fast and cheap to shard it wider. Change-Id: I2858f4c417ef26661575a348fbfc47f4c25d81a2 Reviewed-on: https://go-review.googlesource.com/22795 Reviewed-by: Andrew Gerrand <adg@golang.org>
2016-05-05 03:42:49 +03:00
allScriptArgs: []string{
// Filtering pattern to buildall.bash:
dashboard: include at most 3 ports per misc-compile TryBot Between Go 1.15, 1.16 and tip, the following 29 ports don't have a real TryBot and instead rely on misc-compile TryBots for their pre-submit coverage: • aix/ppc64 • darwin/amd64 • darwin/arm64 • dragonfly/amd64 • freebsd/386 • freebsd/arm • freebsd/arm64 • illumos/amd64 • linux/mips • linux/mips64 • linux/mipsle • linux/mips64le • linux/ppc64 • linux/ppc64le • linux/riscv64 • linux/s390x • netbsd/386 • netbsd/amd64 • netbsd/arm • netbsd/arm64 • openbsd/386 • openbsd/arm • openbsd/arm64 • openbsd/mips64 • plan9/386 • plan9/amd64 • plan9/arm • solaris/amd64 • windows/arm The previous approach for misc-compile target selection was to break them up primarily by GOOS value. However, as new architectures were added over time, some misc-compile TryBots got to a point where they were testing upwards of 5 ports (for example, misc-compile-openbsd was testing 386, amd64, arm, arm64, and mips64 architectures). Since each port is tested sequentially, allocating too many to one misc-compile TryBot can cause it to become the bottleneck of an entire TryBot run, exceeding the 10 minute completion time goal. Arrange it so misc-compile TryBot target selection is done explicitly in x/build, and pick 3 as max number of targets per TryBot for now. Based on recent timing observations, that should strike a decent balance between resource use (spinning up a builder) vs chance of a misc-compile TryBot becoming a bottleneck. It will also give us an opportunity to compare timing of 1, 2 and 3 targets per misc-compile in the future. (When we start tracking timing for TryBot completion time holistically, we'll be in a better position to refine this strategy further.) Making misc-compile target selection explicit in x/build also enables removing unnecessary duplicate misc-compile coverage from ports that already have a real TryBot (for example, openbsd/amd64 was previously tested via both the openbsd-amd64-68 TryBot and misc-compile-openbsd). This shouldn't be needed, so it's no longer done. For golang/go#17104. Fixes golang/go#32632. Change-Id: Iac918377b91af3e48780b38ffdf3153e213eeba2 Reviewed-on: https://go-review.googlesource.com/c/build/+/313210 Trust: Dmitri Shuralyov <dmitshur@golang.org> Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Heschi Kreinick <heschi@google.com> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-04-23 23:56:37 +03:00
"^(" + strings.Join(targets, "|") + ")$",
cmd/coordinator: shard the misc-compile builder over 4 builders Now that it's on Kubernetes, it's fast and cheap to shard it wider. Change-Id: I2858f4c417ef26661575a348fbfc47f4c25d81a2 Reviewed-on: https://go-review.googlesource.com/22795 Reviewed-by: Andrew Gerrand <adg@golang.org>
2016-05-05 03:42:49 +03:00
},
})
}
dashboard: promote misc-compile-darwinarm64 builder to a TryBot Testing shows that misc-compile-darwinarm64 is working as expected on recent commits. Promote it to a normal misc-compile TryBot now, with the exception that it needs to run on Go 1.16 and newer only. Fixes golang/go#42341. Change-Id: Ib4de19a2ff904fb9de056db5076ac63690eb934c Reviewed-on: https://go-review.googlesource.com/c/build/+/267123 Trust: Dmitri Shuralyov <dmitshur@golang.org> Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-11-03 06:25:51 +03:00
// addMiscCompile adds a misc-compile TryBot
// for all supported Go versions.
dashboard: include at most 3 ports per misc-compile TryBot Between Go 1.15, 1.16 and tip, the following 29 ports don't have a real TryBot and instead rely on misc-compile TryBots for their pre-submit coverage: • aix/ppc64 • darwin/amd64 • darwin/arm64 • dragonfly/amd64 • freebsd/386 • freebsd/arm • freebsd/arm64 • illumos/amd64 • linux/mips • linux/mips64 • linux/mipsle • linux/mips64le • linux/ppc64 • linux/ppc64le • linux/riscv64 • linux/s390x • netbsd/386 • netbsd/amd64 • netbsd/arm • netbsd/arm64 • openbsd/386 • openbsd/arm • openbsd/arm64 • openbsd/mips64 • plan9/386 • plan9/amd64 • plan9/arm • solaris/amd64 • windows/arm The previous approach for misc-compile target selection was to break them up primarily by GOOS value. However, as new architectures were added over time, some misc-compile TryBots got to a point where they were testing upwards of 5 ports (for example, misc-compile-openbsd was testing 386, amd64, arm, arm64, and mips64 architectures). Since each port is tested sequentially, allocating too many to one misc-compile TryBot can cause it to become the bottleneck of an entire TryBot run, exceeding the 10 minute completion time goal. Arrange it so misc-compile TryBot target selection is done explicitly in x/build, and pick 3 as max number of targets per TryBot for now. Based on recent timing observations, that should strike a decent balance between resource use (spinning up a builder) vs chance of a misc-compile TryBot becoming a bottleneck. It will also give us an opportunity to compare timing of 1, 2 and 3 targets per misc-compile in the future. (When we start tracking timing for TryBot completion time holistically, we'll be in a better position to refine this strategy further.) Making misc-compile target selection explicit in x/build also enables removing unnecessary duplicate misc-compile coverage from ports that already have a real TryBot (for example, openbsd/amd64 was previously tested via both the openbsd-amd64-68 TryBot and misc-compile-openbsd). This shouldn't be needed, so it's no longer done. For golang/go#17104. Fixes golang/go#32632. Change-Id: Iac918377b91af3e48780b38ffdf3153e213eeba2 Reviewed-on: https://go-review.googlesource.com/c/build/+/313210 Trust: Dmitri Shuralyov <dmitshur@golang.org> Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Heschi Kreinick <heschi@google.com> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-04-23 23:56:37 +03:00
addMiscCompile := func(suffix string, targets ...string) { addMiscCompileGo1(0, suffix, targets...) }
// Arrange so that no more than 3 ports are tested sequentially in each misc-compile
// TryBot to avoid any individual misc-compile TryBot from becoming a bottleneck for
// overall TryBot completion time (currently 10 minutes; see golang.org/issue/17104).
//
// The TestTryBotsCompileAllPorts test is used to detect any gaps in TryBot coverage
// when new ports are added, and the misc-compile pairs below can be re-arranged.
//
// (In the past, we used flexible regexp patterns that matched all architectures
// for a given GOOS value. However, over time as new architectures were added,
// some misc-compile TryBot could become much slower than others.)
//
// See golang.org/issue/32632.
dashboard: add compile-only TryBot coverage for windows/arm64 port The windows/arm64 port has been completed in golang.org/issue/36439 recently. Add pre-submit compile-only coverage via a misc-compile TryBot for the new port. It can be upgraded to a normal TryBot that also executes tests after a windows/arm64 builder becomes available. Tested locally by running 'buildall.bash windows-arm64' on a recent Go tip commit (CL 313849), which passed. For golang/go#36439. Updates golang/go#42604. Change-Id: Ie4988a876aa18392a583eb407c6c3621eb30a4f8 Reviewed-on: https://go-review.googlesource.com/c/build/+/314430 Trust: Dmitri Shuralyov <dmitshur@golang.org> Trust: Alexander Rakoczy <alex@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org> Reviewed-by: Heschi Kreinick <heschi@google.com> Reviewed-by: Alexander Rakoczy <alex@golang.org>
2021-04-27 22:33:27 +03:00
addMiscCompile("-mac-win", "darwin-amd64", "windows-arm", "windows-arm64")
dashboard: include at most 3 ports per misc-compile TryBot Between Go 1.15, 1.16 and tip, the following 29 ports don't have a real TryBot and instead rely on misc-compile TryBots for their pre-submit coverage: • aix/ppc64 • darwin/amd64 • darwin/arm64 • dragonfly/amd64 • freebsd/386 • freebsd/arm • freebsd/arm64 • illumos/amd64 • linux/mips • linux/mips64 • linux/mipsle • linux/mips64le • linux/ppc64 • linux/ppc64le • linux/riscv64 • linux/s390x • netbsd/386 • netbsd/amd64 • netbsd/arm • netbsd/arm64 • openbsd/386 • openbsd/arm • openbsd/arm64 • openbsd/mips64 • plan9/386 • plan9/amd64 • plan9/arm • solaris/amd64 • windows/arm The previous approach for misc-compile target selection was to break them up primarily by GOOS value. However, as new architectures were added over time, some misc-compile TryBots got to a point where they were testing upwards of 5 ports (for example, misc-compile-openbsd was testing 386, amd64, arm, arm64, and mips64 architectures). Since each port is tested sequentially, allocating too many to one misc-compile TryBot can cause it to become the bottleneck of an entire TryBot run, exceeding the 10 minute completion time goal. Arrange it so misc-compile TryBot target selection is done explicitly in x/build, and pick 3 as max number of targets per TryBot for now. Based on recent timing observations, that should strike a decent balance between resource use (spinning up a builder) vs chance of a misc-compile TryBot becoming a bottleneck. It will also give us an opportunity to compare timing of 1, 2 and 3 targets per misc-compile in the future. (When we start tracking timing for TryBot completion time holistically, we'll be in a better position to refine this strategy further.) Making misc-compile target selection explicit in x/build also enables removing unnecessary duplicate misc-compile coverage from ports that already have a real TryBot (for example, openbsd/amd64 was previously tested via both the openbsd-amd64-68 TryBot and misc-compile-openbsd). This shouldn't be needed, so it's no longer done. For golang/go#17104. Fixes golang/go#32632. Change-Id: Iac918377b91af3e48780b38ffdf3153e213eeba2 Reviewed-on: https://go-review.googlesource.com/c/build/+/313210 Trust: Dmitri Shuralyov <dmitshur@golang.org> Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Heschi Kreinick <heschi@google.com> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-04-23 23:56:37 +03:00
addMiscCompileGo1(16, "-darwinarm64", "darwin-arm64") // darwin/arm64 (for Go 1.16 and newer) only.
addMiscCompile("-mips", "linux-mips", "linux-mips64")
addMiscCompile("-mipsle", "linux-mipsle", "linux-mips64le")
addMiscCompile("-ppc", "linux-ppc64", "linux-ppc64le", "aix-ppc64")
addMiscCompile("-freebsd", "freebsd-386", "freebsd-arm", "freebsd-arm64")
addMiscCompile("-netbsd", "netbsd-386", "netbsd-amd64")
addMiscCompile("-netbsd-arm", "netbsd-arm", "netbsd-arm64")
addMiscCompile("-openbsd", "openbsd-386", "openbsd-mips64")
addMiscCompile("-openbsd-arm", "openbsd-arm", "openbsd-arm64")
addMiscCompile("-plan9", "plan9-386", "plan9-amd64", "plan9-arm")
addMiscCompile("-other-1", "solaris-amd64", "illumos-amd64", "dragonfly-amd64")
addMiscCompile("-other-2", "linux-riscv64", "linux-s390x", "linux-arm-arm5") // 'linux-arm-arm5' is linux/arm with GOARM=5.
dashboard: remove misc-compile-ios builder Since sending CL 257617, I've learned that iOS was never a supported target for misc-compile. It was always filtered out unconditionally in GOROOT/src/buildall.bash, and the comment suggesting that we test iOS targets was inaccurate. Remove the naive misc-compile-ios builder since it's not expected to start working soon, and update misc-builder architecture filtering code and comments to match the current reality better. After this change, it will be viable to remove mobile target filtering from GOROOT/src/buildall.bash. If that's done, then x/build/dashboard will become a centralized place for all misc-builder target selection. Ignore darwin-arm in TestTryBotsCompileAllPorts so the test doesn't fail when executed with Go 1.14. Not having that builder is intended. For golang/go#41610. Updates golang/go#25963. Change-Id: Ic6f5dc95edb67f8e6ebc0a77979e55eb0cc325cb Reviewed-on: https://go-review.googlesource.com/c/build/+/258057 Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Trust: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-09-28 18:04:35 +03:00
// TODO: Issue 25963, get the misc-compile trybots for Android/iOS.
// Then consider subrepos too, so "mobile" can at least be included
// as a misc-compile for ^android- and ^ios-.
cmd/coordinator: shard the misc-compile builder over 4 builders Now that it's on Kubernetes, it's fast and cheap to shard it wider. Change-Id: I2858f4c417ef26661575a348fbfc47f4c25d81a2 Reviewed-on: https://go-review.googlesource.com/22795 Reviewed-by: Andrew Gerrand <adg@golang.org>
2016-05-05 03:42:49 +03:00
build: convert most Linux builders to be VM-based We still use Dockerfiles to describe them, but then a new tool (docker2boot) converts them into VM images suitable for booting on GCE, running the buildlet like all the other operating systems. Things are easier if everything acts the same way. Note that since we're no longer moving around Docker images, the image size and the layer accumulation cruft no longer matters. We can now have Dockerfile lines like "RUN rm -rf /usr/share/doc" and it actually results in a smaller VM image, since we just "docker export" the files out of it to create the VM image. This doesn't yet convert the clang, sid, or nacl builders. The coordinator still runs those under Docker directly. A future change will convert those to VM images as well. Change-Id: Iedb136ae3daf888c955eb843bdcc9a638d08f5e9 Reviewed-on: https://go-review.googlesource.com/3341 Reviewed-by: Andrew Gerrand <adg@golang.org>
2015-01-28 01:22:21 +03:00
addBuilder(BuildConfig{
cmd/coordinator: delete useScheduler knob & old pre-scheduler hacks, code, TODOs Updates golang/go#19178 Change-Id: Id4d8b016c41f57bfaba5ee4ab046285607047493 Reviewed-on: https://go-review.googlesource.com/c/build/+/207178 Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-11-14 18:43:33 +03:00
Name: "linux-amd64-nocgo",
dashboard: update Debian version for amd64 builders This change updates various linux-amd64 builders after the release of bullseye: - linux-amd64 has been updated to bullseye. - linux-amd64-nocgo has been updated to bullseye. - linux-amd64-noopt has been updated to bullseye. - linux-amd64-ssacheck has been updated to bullseye. - linux-amd64-racecompile has been updated to bullseye. - linux-amd64-race has been updated to bullseye. - linux-amd64-jessie has had the notes updated. - misc-compile has been updated to bullseye. Fixes golang/go#47721 Change-Id: I29c5d76da6ad6b28647bd423a8ef7dc13b919ca2 Reviewed-on: https://go-review.googlesource.com/c/build/+/343633 Trust: Carlos Amedee <carlos@golang.org> Run-TryBot: Carlos Amedee <carlos@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2021-08-19 20:27:43 +03:00
HostType: "host-linux-bullseye",
cmd/coordinator: delete useScheduler knob & old pre-scheduler hacks, code, TODOs Updates golang/go#19178 Change-Id: Id4d8b016c41f57bfaba5ee4ab046285607047493 Reviewed-on: https://go-review.googlesource.com/c/build/+/207178 Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-11-14 18:43:33 +03:00
Notes: "cgo disabled",
dashboard: add some "exp" trybots back, clean up some config Fixes golang/go#30781 Change-Id: I2672ca2949837c3cfe8fa6bd8011d1b9b43cca03 Reviewed-on: https://go-review.googlesource.com/c/build/+/167640 Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-03-14 23:35:32 +03:00
buildsRepo: func(repo, branch, goBranch string) bool {
switch repo {
case "perf":
// Requires sqlite, which requires cgo.
return false
case "mobile":
return false
dashboard: stop testing x/build on even more builders Follow up to CL 210378 after looking at: https://build.golang.org/?repo=golang.org%2fx%2fbuild Updates golang/go#30640 Change-Id: I81505a9ca2985ffa6e75922665b14ef4de2af860 Reviewed-on: https://go-review.googlesource.com/c/build/+/211100 Reviewed-by: Bryan C. Mills <bcmills@google.com> Run-TryBot: Bryan C. Mills <bcmills@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-12-12 20:43:59 +03:00
case "build":
return false
dashboard: add some "exp" trybots back, clean up some config Fixes golang/go#30781 Change-Id: I2672ca2949837c3cfe8fa6bd8011d1b9b43cca03 Reviewed-on: https://go-review.googlesource.com/c/build/+/167640 Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-03-14 23:35:32 +03:00
}
return true
},
build: convert most Linux builders to be VM-based We still use Dockerfiles to describe them, but then a new tool (docker2boot) converts them into VM images suitable for booting on GCE, running the buildlet like all the other operating systems. Things are easier if everything acts the same way. Note that since we're no longer moving around Docker images, the image size and the layer accumulation cruft no longer matters. We can now have Dockerfile lines like "RUN rm -rf /usr/share/doc" and it actually results in a smaller VM image, since we just "docker export" the files out of it to create the VM image. This doesn't yet convert the clang, sid, or nacl builders. The coordinator still runs those under Docker directly. A future change will convert those to VM images as well. Change-Id: Iedb136ae3daf888c955eb843bdcc9a638d08f5e9 Reviewed-on: https://go-review.googlesource.com/3341 Reviewed-by: Andrew Gerrand <adg@golang.org>
2015-01-28 01:22:21 +03:00
env: []string{
"CGO_ENABLED=0",
cmd/coordinator: follow module proxy redirects And because this uses new-in-Go1.13 http.Header.Clone, restrict all the files to Go 1.13+ and bump the Dockerfile Go version. Fixes golang/go#33928 Change-Id: I147cc4465e393d2c34e59c70edf65f3901a5e132 Reviewed-on: https://go-review.googlesource.com/c/build/+/192341 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-08-29 21:28:01 +03:00
"GO_DISABLE_OUTBOUND_NETWORK=1",
build: convert most Linux builders to be VM-based We still use Dockerfiles to describe them, but then a new tool (docker2boot) converts them into VM images suitable for booting on GCE, running the buildlet like all the other operating systems. Things are easier if everything acts the same way. Note that since we're no longer moving around Docker images, the image size and the layer accumulation cruft no longer matters. We can now have Dockerfile lines like "RUN rm -rf /usr/share/doc" and it actually results in a smaller VM image, since we just "docker export" the files out of it to create the VM image. This doesn't yet convert the clang, sid, or nacl builders. The coordinator still runs those under Docker directly. A future change will convert those to VM images as well. Change-Id: Iedb136ae3daf888c955eb843bdcc9a638d08f5e9 Reviewed-on: https://go-review.googlesource.com/3341 Reviewed-by: Andrew Gerrand <adg@golang.org>
2015-01-28 01:22:21 +03:00
// This USER=root was required for Docker-based builds but probably isn't required
// in the VM anymore, since the buildlet probably already has this in its environment.
// (It was required because without cgo, it couldn't find the username)
"USER=root",
},
})
addBuilder(BuildConfig{
dashboard: update some build policy Mostly some regressions (building too much) from the massive refactoring in CL 166218, but also some minor changes. See test updates. Change-Id: Ie30f51b425367f18a49764ac971e262131959bc8 Reviewed-on: https://go-review.googlesource.com/c/build/+/166557 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-03-11 07:20:46 +03:00
Name: "linux-amd64-noopt",
Notes: "optimizations and inlining disabled",
dashboard: update Debian version for amd64 builders This change updates various linux-amd64 builders after the release of bullseye: - linux-amd64 has been updated to bullseye. - linux-amd64-nocgo has been updated to bullseye. - linux-amd64-noopt has been updated to bullseye. - linux-amd64-ssacheck has been updated to bullseye. - linux-amd64-racecompile has been updated to bullseye. - linux-amd64-race has been updated to bullseye. - linux-amd64-jessie has had the notes updated. - misc-compile has been updated to bullseye. Fixes golang/go#47721 Change-Id: I29c5d76da6ad6b28647bd423a8ef7dc13b919ca2 Reviewed-on: https://go-review.googlesource.com/c/build/+/343633 Trust: Carlos Amedee <carlos@golang.org> Run-TryBot: Carlos Amedee <carlos@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2021-08-19 20:27:43 +03:00
HostType: "host-linux-bullseye",
dashboard: update some build policy Mostly some regressions (building too much) from the massive refactoring in CL 166218, but also some minor changes. See test updates. Change-Id: Ie30f51b425367f18a49764ac971e262131959bc8 Reviewed-on: https://go-review.googlesource.com/c/build/+/166557 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-03-11 07:20:46 +03:00
buildsRepo: onlyGo,
dashboard: enable outbound firewall on a few more Linux-based builders Updates golang/go#30612 Change-Id: Id472e753043976b8671c15491b82bb57e3804d95 Reviewed-on: https://go-review.googlesource.com/c/build/+/165638 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-03-06 08:20:12 +03:00
env: []string{
cmd/coordinator: follow module proxy redirects And because this uses new-in-Go1.13 http.Header.Clone, restrict all the files to Go 1.13+ and bump the Dockerfile Go version. Fixes golang/go#33928 Change-Id: I147cc4465e393d2c34e59c70edf65f3901a5e132 Reviewed-on: https://go-review.googlesource.com/c/build/+/192341 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-08-29 21:28:01 +03:00
"GO_DISABLE_OUTBOUND_NETWORK=1",
dashboard: enable outbound firewall on a few more Linux-based builders Updates golang/go#30612 Change-Id: Id472e753043976b8671c15491b82bb57e3804d95 Reviewed-on: https://go-review.googlesource.com/c/build/+/165638 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-03-06 08:20:12 +03:00
"GO_GCFLAGS=-N -l",
},
build: convert most Linux builders to be VM-based We still use Dockerfiles to describe them, but then a new tool (docker2boot) converts them into VM images suitable for booting on GCE, running the buildlet like all the other operating systems. Things are easier if everything acts the same way. Note that since we're no longer moving around Docker images, the image size and the layer accumulation cruft no longer matters. We can now have Dockerfile lines like "RUN rm -rf /usr/share/doc" and it actually results in a smaller VM image, since we just "docker export" the files out of it to create the VM image. This doesn't yet convert the clang, sid, or nacl builders. The coordinator still runs those under Docker directly. A future change will convert those to VM images as well. Change-Id: Iedb136ae3daf888c955eb843bdcc9a638d08f5e9 Reviewed-on: https://go-review.googlesource.com/3341 Reviewed-by: Andrew Gerrand <adg@golang.org>
2015-01-28 01:22:21 +03:00
})
dashboard: add SSA check builder Now that SSA checks are off by default, let's run a builder with the checks on just to make sure they keep passing. Change-Id: I7edd2a2865681a135ccf755be7a14f8645c0a8aa Reviewed-on: https://go-review.googlesource.com/20882 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2016-03-19 01:30:38 +03:00
addBuilder(BuildConfig{
build: move Linux and nacl builders to Kubernetes, add ssacheck trybot And shard the nacl builders. Fixes golang/go#15242 Fixes golang/go#12546 Change-Id: I3311678dbb64a76026f75f5691cffac0f25d976b Reviewed-on: https://go-review.googlesource.com/22780 Reviewed-by: Andrew Gerrand <adg@golang.org> Reviewed-by: Ross Light <light@google.com>
2016-05-04 20:47:12 +03:00
Name: "linux-amd64-ssacheck",
dashboard: update Debian version for amd64 builders This change updates various linux-amd64 builders after the release of bullseye: - linux-amd64 has been updated to bullseye. - linux-amd64-nocgo has been updated to bullseye. - linux-amd64-noopt has been updated to bullseye. - linux-amd64-ssacheck has been updated to bullseye. - linux-amd64-racecompile has been updated to bullseye. - linux-amd64-race has been updated to bullseye. - linux-amd64-jessie has had the notes updated. - misc-compile has been updated to bullseye. Fixes golang/go#47721 Change-Id: I29c5d76da6ad6b28647bd423a8ef7dc13b919ca2 Reviewed-on: https://go-review.googlesource.com/c/build/+/343633 Trust: Carlos Amedee <carlos@golang.org> Run-TryBot: Carlos Amedee <carlos@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2021-08-19 20:27:43 +03:00
HostType: "host-linux-bullseye",
dashboard: update some build policy Mostly some regressions (building too much) from the massive refactoring in CL 166218, but also some minor changes. See test updates. Change-Id: Ie30f51b425367f18a49764ac971e262131959bc8 Reviewed-on: https://go-review.googlesource.com/c/build/+/166557 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-03-11 07:20:46 +03:00
buildsRepo: onlyGo,
dashboard, cmd/coordinator: move more config out of coordinator into dashboard And with the increased flexibility, enable trybots for more repos for the "sys" project. Details: https://github.com/golang/go/issues/24893#issuecomment-433517519 Fixes golang/go#24893 Change-Id: I89ce6f219be986201332f8fad11c774bdd6246da Reviewed-on: https://go-review.googlesource.com/c/145157 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2018-10-26 22:21:58 +03:00
tryBot: nil, // TODO: add a func to conditionally run this trybot if compiler dirs are touched
build: move Linux and nacl builders to Kubernetes, add ssacheck trybot And shard the nacl builders. Fixes golang/go#15242 Fixes golang/go#12546 Change-Id: I3311678dbb64a76026f75f5691cffac0f25d976b Reviewed-on: https://go-review.googlesource.com/22780 Reviewed-by: Andrew Gerrand <adg@golang.org> Reviewed-by: Ross Light <light@google.com>
2016-05-04 20:47:12 +03:00
CompileOnly: true,
Notes: "SSA internal checks enabled",
dashboard: enable outbound firewall on a few more Linux-based builders Updates golang/go#30612 Change-Id: Id472e753043976b8671c15491b82bb57e3804d95 Reviewed-on: https://go-review.googlesource.com/c/build/+/165638 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-03-06 08:20:12 +03:00
env: []string{
cmd/coordinator: follow module proxy redirects And because this uses new-in-Go1.13 http.Header.Clone, restrict all the files to Go 1.13+ and bump the Dockerfile Go version. Fixes golang/go#33928 Change-Id: I147cc4465e393d2c34e59c70edf65f3901a5e132 Reviewed-on: https://go-review.googlesource.com/c/build/+/192341 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-08-29 21:28:01 +03:00
"GO_DISABLE_OUTBOUND_NETWORK=1",
dashboard: enable outbound firewall on a few more Linux-based builders Updates golang/go#30612 Change-Id: Id472e753043976b8671c15491b82bb57e3804d95 Reviewed-on: https://go-review.googlesource.com/c/build/+/165638 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-03-06 08:20:12 +03:00
"GO_GCFLAGS=-d=ssa/check/on,dclstack",
},
cmd/coordinator: move git ancestor checking before buildlet creation, generalize This also adds GoDeps to the builders config file and adds a dependency for the ssacheck builder, which currently fails on older release branches with: > Failed on linux-amd64-ssacheck: > https://storage.googleapis.com/go-build-log/1255edaf/linux-amd64-ssacheck_18f7e015.log > > ... > compile: unknown debug key -d dclstack Change-Id: I4ad0a298705feed56f467f3c8d9488f8be2e4af5 Reviewed-on: https://go-review.googlesource.com/43774 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2017-05-20 00:23:14 +03:00
GoDeps: []string{
"f65abf6ddc8d1f3d403a9195fd74eaffa022b07f", // adds dclstack
},
dashboard: add SSA check builder Now that SSA checks are off by default, let's run a builder with the checks on just to make sure they keep passing. Change-Id: I7edd2a2865681a135ccf755be7a14f8645c0a8aa Reviewed-on: https://go-review.googlesource.com/20882 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2016-03-19 01:30:38 +03:00
})
dashboard: add linux-amd64-staticlockranking builder This is a new linux/amd64 GCE builder for the purpose of testing the GOEXPERIMENT=staticlockranking configuration which is landing to master branch soon. It runs only on the main Go repository master branch and release branches 1.15 and newer. No golang.org/x repos. Not included in trybots by default, but can be requested via SlowBots by saying TRY=linux-amd64-staticlockranking. Also expand documentation of BuildConfig.buildsRepo field to make it more clear how to use it. A common mistake for custom buildsRepo functions that are added is to create a custom policy with a minor adjustment and forgetting to preserve the rest of the default build repo policy. Fixes golang/go#37937 Change-Id: Ic56f82e9ede30cb6fe8f6cc7d0282d6a0a12c1b8 Reviewed-on: https://go-review.googlesource.com/c/build/+/224078 Reviewed-by: Robert Findley <rfindley@google.com> Reviewed-by: Alexander Rakoczy <alex@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org> Run-TryBot: Alexander Rakoczy <alex@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2020-03-19 17:34:11 +03:00
addBuilder(BuildConfig{
dashboard: unset known issue for staticlockranking builder CL 207619 has been submitted to master, fixing golang.org/issue/38029. The linux-amd64-staticlockranking builder has been passing since then. Unset its known issue so that if there are regressions in the future, they can be investigated. For golang/go#38029. For golang/go#37937. Change-Id: I0de8e4ac122effee56aac73c741fab41512ff0c2 Reviewed-on: https://go-review.googlesource.com/c/build/+/227544 Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Dan Scales <danscales@google.com>
2020-04-08 19:13:38 +03:00
Name: "linux-amd64-staticlockranking",
HostType: "host-linux-stretch",
Notes: "builder with GOEXPERIMENT=staticlockranking, see golang.org/issue/37937",
dashboard: add linux-amd64-staticlockranking builder This is a new linux/amd64 GCE builder for the purpose of testing the GOEXPERIMENT=staticlockranking configuration which is landing to master branch soon. It runs only on the main Go repository master branch and release branches 1.15 and newer. No golang.org/x repos. Not included in trybots by default, but can be requested via SlowBots by saying TRY=linux-amd64-staticlockranking. Also expand documentation of BuildConfig.buildsRepo field to make it more clear how to use it. A common mistake for custom buildsRepo functions that are added is to create a custom policy with a minor adjustment and forgetting to preserve the rest of the default build repo policy. Fixes golang/go#37937 Change-Id: Ic56f82e9ede30cb6fe8f6cc7d0282d6a0a12c1b8 Reviewed-on: https://go-review.googlesource.com/c/build/+/224078 Reviewed-by: Robert Findley <rfindley@google.com> Reviewed-by: Alexander Rakoczy <alex@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org> Run-TryBot: Alexander Rakoczy <alex@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2020-03-19 17:34:11 +03:00
buildsRepo: func(repo, branch, goBranch string) bool {
return repo == "go" && atLeastGo1(goBranch, 15)
},
env: []string{
"GO_DISABLE_OUTBOUND_NETWORK=1",
"GOEXPERIMENT=staticlockranking",
},
GoDeps: []string{
"02057906f7272a4787b8a0b5b7cafff8ad3024f0", // A master commit from 2020/03/19, just before CL 222925 and CL 207619 have landed.
},
})
dashboard: add linux-amd64-unified builder This adds a trybot+builder for testing cmd/compile's unified IR construction experiment on the dev.typeparams branch. Fixes golang/go#46786. Change-Id: Ibf3d5ab7a727c52cf13486798e02077faa08e2b5 Reviewed-on: https://go-review.googlesource.com/c/build/+/328891 Run-TryBot: Matthew Dempsky <mdempsky@google.com> TryBot-Result: Go Bot <gobot@golang.org> Trust: Matthew Dempsky <mdempsky@google.com> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-06-17 09:08:22 +03:00
addBuilder(BuildConfig{
Name: "linux-amd64-unified",
HostType: "host-linux-buster",
Notes: "builder with GOEXPERIMENT=unified, see golang.org/issue/46786",
tryBot: func(repo, branch, goBranch string) bool {
dashboard: enable linux-amd64-unified on master dev.typeparams has been merged to master, so continue testing that GOEXPERIMENT=unified works there too. Updates golang/go#46786. Change-Id: Iea048ef081f4c40a5b1c516a7c0d68178fd0b196 Reviewed-on: https://go-review.googlesource.com/c/build/+/341931 Trust: Matthew Dempsky <mdempsky@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2021-08-13 01:11:41 +03:00
return (repo == "go" || repo == "tools") && (goBranch == "master" || goBranch == "dev.typeparams")
dashboard: add linux-amd64-unified builder This adds a trybot+builder for testing cmd/compile's unified IR construction experiment on the dev.typeparams branch. Fixes golang/go#46786. Change-Id: Ibf3d5ab7a727c52cf13486798e02077faa08e2b5 Reviewed-on: https://go-review.googlesource.com/c/build/+/328891 Run-TryBot: Matthew Dempsky <mdempsky@google.com> TryBot-Result: Go Bot <gobot@golang.org> Trust: Matthew Dempsky <mdempsky@google.com> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-06-17 09:08:22 +03:00
},
buildsRepo: func(repo, branch, goBranch string) bool {
dashboard: enable linux-amd64-unified on master dev.typeparams has been merged to master, so continue testing that GOEXPERIMENT=unified works there too. Updates golang/go#46786. Change-Id: Iea048ef081f4c40a5b1c516a7c0d68178fd0b196 Reviewed-on: https://go-review.googlesource.com/c/build/+/341931 Trust: Matthew Dempsky <mdempsky@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2021-08-13 01:11:41 +03:00
return (repo == "go" || repo == "tools") && (goBranch == "master" || goBranch == "dev.typeparams")
dashboard: add linux-amd64-unified builder This adds a trybot+builder for testing cmd/compile's unified IR construction experiment on the dev.typeparams branch. Fixes golang/go#46786. Change-Id: Ibf3d5ab7a727c52cf13486798e02077faa08e2b5 Reviewed-on: https://go-review.googlesource.com/c/build/+/328891 Run-TryBot: Matthew Dempsky <mdempsky@google.com> TryBot-Result: Go Bot <gobot@golang.org> Trust: Matthew Dempsky <mdempsky@google.com> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-06-17 09:08:22 +03:00
},
env: []string{
"GO_DISABLE_OUTBOUND_NETWORK=1",
"GOEXPERIMENT=unified",
},
GoDeps: []string{
"804ecc2581caf33ae347d6a1ce67436d1f74e93b", // CL 328215, which added GOEXPERIMENT=unified on dev.typeparams
},
numTestHelpers: 1,
numTryTestHelpers: 4,
})
dashboard, cmd/coordinator: add -racecompile builder Change-Id: Idc5756f82081de6b5dfd17036f2bfc9cf11c5659 Reviewed-on: https://go-review.googlesource.com/41414 Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
2017-04-22 02:41:12 +03:00
addBuilder(BuildConfig{
Name: "linux-amd64-racecompile",
dashboard: update Debian version for amd64 builders This change updates various linux-amd64 builders after the release of bullseye: - linux-amd64 has been updated to bullseye. - linux-amd64-nocgo has been updated to bullseye. - linux-amd64-noopt has been updated to bullseye. - linux-amd64-ssacheck has been updated to bullseye. - linux-amd64-racecompile has been updated to bullseye. - linux-amd64-race has been updated to bullseye. - linux-amd64-jessie has had the notes updated. - misc-compile has been updated to bullseye. Fixes golang/go#47721 Change-Id: I29c5d76da6ad6b28647bd423a8ef7dc13b919ca2 Reviewed-on: https://go-review.googlesource.com/c/build/+/343633 Trust: Carlos Amedee <carlos@golang.org> Run-TryBot: Carlos Amedee <carlos@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2021-08-19 20:27:43 +03:00
HostType: "host-linux-bullseye",
dashboard, cmd/coordinator: move more config out of coordinator into dashboard And with the increased flexibility, enable trybots for more repos for the "sys" project. Details: https://github.com/golang/go/issues/24893#issuecomment-433517519 Fixes golang/go#24893 Change-Id: I89ce6f219be986201332f8fad11c774bdd6246da Reviewed-on: https://go-review.googlesource.com/c/145157 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2018-10-26 22:21:58 +03:00
tryBot: nil, // TODO: add a func to conditionally run this trybot if compiler dirs are touched
dashboard, cmd/coordinator: add -racecompile builder Change-Id: Idc5756f82081de6b5dfd17036f2bfc9cf11c5659 Reviewed-on: https://go-review.googlesource.com/41414 Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
2017-04-22 02:41:12 +03:00
CompileOnly: true,
cmd/coordinator: make linux-amd64-racecompile rebuild std/cmd with -c=128 Updates golang/go#19962 Change-Id: I8b938a39b33d4e6b9b8dc5bc0802da95e889ee46 Reviewed-on: https://go-review.googlesource.com/41613 Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
2017-04-25 00:14:05 +03:00
SkipSnapshot: true,
StopAfterMake: true,
dashboard: use race-enabled linker in racecompile builder When the compiler's backend became concurrent, we introduced the racecompile builder to run the compiler with the race detector enabled to catch races. In the new linker work the linker will become more concurrent as well. It is probably a good idea to run the linker with the race detector as well. Change-Id: Ib50f030cba0509730714b74462f5a8942fe2fd2d Reviewed-on: https://go-review.googlesource.com/c/build/+/226369 Run-TryBot: Cherry Zhang <cherryyz@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2020-03-30 18:20:59 +03:00
InstallRacePackages: []string{"cmd/compile", "cmd/link"},
Notes: "race-enabled cmd/compile and cmd/link",
dashboard: enable outbound firewall on a few more Linux-based builders Updates golang/go#30612 Change-Id: Id472e753043976b8671c15491b82bb57e3804d95 Reviewed-on: https://go-review.googlesource.com/c/build/+/165638 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-03-06 08:20:12 +03:00
env: []string{
cmd/coordinator: follow module proxy redirects And because this uses new-in-Go1.13 http.Header.Clone, restrict all the files to Go 1.13+ and bump the Dockerfile Go version. Fixes golang/go#33928 Change-Id: I147cc4465e393d2c34e59c70edf65f3901a5e132 Reviewed-on: https://go-review.googlesource.com/c/build/+/192341 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-08-29 21:28:01 +03:00
"GO_DISABLE_OUTBOUND_NETWORK=1",
dashboard: enable outbound firewall on a few more Linux-based builders Updates golang/go#30612 Change-Id: Id472e753043976b8671c15491b82bb57e3804d95 Reviewed-on: https://go-review.googlesource.com/c/build/+/165638 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-03-06 08:20:12 +03:00
},
cmd/coordinator: move git ancestor checking before buildlet creation, generalize This also adds GoDeps to the builders config file and adds a dependency for the ssacheck builder, which currently fails on older release branches with: > Failed on linux-amd64-ssacheck: > https://storage.googleapis.com/go-build-log/1255edaf/linux-amd64-ssacheck_18f7e015.log > > ... > compile: unknown debug key -d dclstack Change-Id: I4ad0a298705feed56f467f3c8d9488f8be2e4af5 Reviewed-on: https://go-review.googlesource.com/43774 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2017-05-20 00:23:14 +03:00
GoDeps: []string{
"22f1b56dab29d397d2bdbdd603d85e60fb678089", // adds cmd/compile -c; Issue 20222
},
dashboard, cmd/coordinator: add -racecompile builder Change-Id: Idc5756f82081de6b5dfd17036f2bfc9cf11c5659 Reviewed-on: https://go-review.googlesource.com/41414 Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
2017-04-22 02:41:12 +03:00
})
build: convert most Linux builders to be VM-based We still use Dockerfiles to describe them, but then a new tool (docker2boot) converts them into VM images suitable for booting on GCE, running the buildlet like all the other operating systems. Things are easier if everything acts the same way. Note that since we're no longer moving around Docker images, the image size and the layer accumulation cruft no longer matters. We can now have Dockerfile lines like "RUN rm -rf /usr/share/doc" and it actually results in a smaller VM image, since we just "docker export" the files out of it to create the VM image. This doesn't yet convert the clang, sid, or nacl builders. The coordinator still runs those under Docker directly. A future change will convert those to VM images as well. Change-Id: Iedb136ae3daf888c955eb843bdcc9a638d08f5e9 Reviewed-on: https://go-review.googlesource.com/3341 Reviewed-by: Andrew Gerrand <adg@golang.org>
2015-01-28 01:22:21 +03:00
addBuilder(BuildConfig{
cmd/coordinator, dashboard: remove some trybots, shard others wider I'm aiming to have trybot runs finish in under 5 minutes. This CL removes openbsd-386-gce58 and freebsd-386-gce101 from the trybot set. openbsd-386-gce58 is the slowest builder. It has an average speed of 722 seconds (and 95 percentile of 923 seconds) over the past week, and that's sharded over 4 machines. Too slow. It's not worth the resources to keep it as a trybot. It hasn't caught any interesting bugs. This builder will still run, but not as a pre-submit trybot. freebsd-386-gce101 is not slow, but we're removing it to shift its resources to shard other builders wider. The coordinator now supports varying the build sharding width based on whether a build is for a trybot or not. This CL defines separate numbers for each, sharding builds wider as needed for some trybots. freebsd-amd64-gce101 goes from 4 to 5 machines in try runs, and down to 3 when not in try runs. linux-amd64-race gets one more machine during try runs, and one fewer in regular runs. linux-arm goes from 7 machines always, to 3 or 8, depending on whether it's a try run. openbsd-amd64-58 goes from 4 to 3 or 6. windows-amd64-gce goes from 4 to 2 or 6. windows-amd64-race goes from 4 to 2 or 6. darwin-amd64-10_11 goes from 3 to 3 or 4. I'll see how these do over the next few days and readjust as needed. Also in this CL: fix the constants for the expected duration of make.bash, which impact when we schedule the creation of test sharding helper buildlets. We were creating them too early before, wasting resources. Change-Id: I38a9b24841e196f1eb668de058c49af8c1d1c64f Reviewed-on: https://go-review.googlesource.com/29116 Reviewed-by: Quentin Smith <quentin@golang.org>
2016-09-14 01:45:48 +03:00
Name: "linux-amd64-race",
dashboard: update Debian version for amd64 builders This change updates various linux-amd64 builders after the release of bullseye: - linux-amd64 has been updated to bullseye. - linux-amd64-nocgo has been updated to bullseye. - linux-amd64-noopt has been updated to bullseye. - linux-amd64-ssacheck has been updated to bullseye. - linux-amd64-racecompile has been updated to bullseye. - linux-amd64-race has been updated to bullseye. - linux-amd64-jessie has had the notes updated. - misc-compile has been updated to bullseye. Fixes golang/go#47721 Change-Id: I29c5d76da6ad6b28647bd423a8ef7dc13b919ca2 Reviewed-on: https://go-review.googlesource.com/c/build/+/343633 Trust: Carlos Amedee <carlos@golang.org> Run-TryBot: Carlos Amedee <carlos@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2021-08-19 20:27:43 +03:00
HostType: "host-linux-bullseye",
dashboard, cmd/coordinator: move more config out of coordinator into dashboard And with the increased flexibility, enable trybots for more repos for the "sys" project. Details: https://github.com/golang/go/issues/24893#issuecomment-433517519 Fixes golang/go#24893 Change-Id: I89ce6f219be986201332f8fad11c774bdd6246da Reviewed-on: https://go-review.googlesource.com/c/145157 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2018-10-26 22:21:58 +03:00
tryBot: defaultTrySet(),
dashboard: don't build x/build on so many builders It was kinda overkill. Change-Id: Ied7fc42aa59a1ed18cc1da20e51c8ed386d84629 Reviewed-on: https://go-review.googlesource.com/c/build/+/210378 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-12-07 08:14:28 +03:00
buildsRepo: defaultPlusExpBuild,
dashboard: refactor mechanism for adjusting cmd/dist test policy Previously, the behavior of ShouldRunDistTest could be overridden by an individual builder by setting a the shouldRunDistTest field to a custom policy implementation. A pitfall of this approach is that it requires each custom policy to reimplement the default cmd/dist test policy as well. If that isn't done when attempting to make a small adjustment to the default policy, the result is that the default policy is ignored. The fasterTrybots policy, as the name suggests, exists to skip some tests in trybot mode. Due to the problem above, it was causing some builders to run more tests in trybot mode than they would if it weren't applied. For example, the freebsd-amd64-12_0 builder should have been skipping the "api" test in trybot mode, but wasn't. This refactor fixes that by modifying the approach for how individual builders can customize the cmd/dist test behavior. Now, a function distTestAdjust is called with the first parameter specifying the default policy decision. That function can adjust the decision as it sees fit, but otherwise it preserves its initial value in situations where no change is intended to be made. For golang/go#38279. Change-Id: I220248f51abfc8e77195ae0c9134a17f7cac9bc2 Reviewed-on: https://go-review.googlesource.com/c/build/+/227777 Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-04-09 19:47:03 +03:00
distTestAdjust: fasterTrybots,
all: use Container-Optimized VMs instead of Kubernetes for buildlet containers Fixes golang/go#25108 Change-Id: I084669b52b699700ed26a7fdd890d9205a8b9dc9 Reviewed-on: https://go-review.googlesource.com/111267 Reviewed-by: Andrew Bonventre <andybons@golang.org>
2018-05-04 05:42:17 +03:00
numTestHelpers: 1,
cmd/coordinator, dashboard: remove some trybots, shard others wider I'm aiming to have trybot runs finish in under 5 minutes. This CL removes openbsd-386-gce58 and freebsd-386-gce101 from the trybot set. openbsd-386-gce58 is the slowest builder. It has an average speed of 722 seconds (and 95 percentile of 923 seconds) over the past week, and that's sharded over 4 machines. Too slow. It's not worth the resources to keep it as a trybot. It hasn't caught any interesting bugs. This builder will still run, but not as a pre-submit trybot. freebsd-386-gce101 is not slow, but we're removing it to shift its resources to shard other builders wider. The coordinator now supports varying the build sharding width based on whether a build is for a trybot or not. This CL defines separate numbers for each, sharding builds wider as needed for some trybots. freebsd-amd64-gce101 goes from 4 to 5 machines in try runs, and down to 3 when not in try runs. linux-amd64-race gets one more machine during try runs, and one fewer in regular runs. linux-arm goes from 7 machines always, to 3 or 8, depending on whether it's a try run. openbsd-amd64-58 goes from 4 to 3 or 6. windows-amd64-gce goes from 4 to 2 or 6. windows-amd64-race goes from 4 to 2 or 6. darwin-amd64-10_11 goes from 3 to 3 or 4. I'll see how these do over the next few days and readjust as needed. Also in this CL: fix the constants for the expected duration of make.bash, which impact when we schedule the creation of test sharding helper buildlets. We were creating them too early before, wasting resources. Change-Id: I38a9b24841e196f1eb668de058c49af8c1d1c64f Reviewed-on: https://go-review.googlesource.com/29116 Reviewed-by: Quentin Smith <quentin@golang.org>
2016-09-14 01:45:48 +03:00
numTryTestHelpers: 5,
dashboard: enable outbound firewall on a few more Linux-based builders Updates golang/go#30612 Change-Id: Id472e753043976b8671c15491b82bb57e3804d95 Reviewed-on: https://go-review.googlesource.com/c/build/+/165638 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-03-06 08:20:12 +03:00
env: []string{
cmd/coordinator: follow module proxy redirects And because this uses new-in-Go1.13 http.Header.Clone, restrict all the files to Go 1.13+ and bump the Dockerfile Go version. Fixes golang/go#33928 Change-Id: I147cc4465e393d2c34e59c70edf65f3901a5e132 Reviewed-on: https://go-review.googlesource.com/c/build/+/192341 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-08-29 21:28:01 +03:00
"GO_DISABLE_OUTBOUND_NETWORK=1",
dashboard: enable outbound firewall on a few more Linux-based builders Updates golang/go#30612 Change-Id: Id472e753043976b8671c15491b82bb57e3804d95 Reviewed-on: https://go-review.googlesource.com/c/build/+/165638 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-03-06 08:20:12 +03:00
},
all: add env support, freebsd-386, freebsd-amd64-race, re-enable plan9 Update golang/go#9491 Update golang/go#8639 Change-Id: I1f43c751777f10a6d5870ca9bbb8c2a3430189bf Reviewed-on: https://go-review.googlesource.com/3170 Reviewed-by: Andrew Gerrand <adg@golang.org>
2015-01-22 02:15:48 +03:00
})
env: convert linux-x86-clang to be VM-based Change-Id: I583d751652dec9176169b4d4705f3813506017e3 Reviewed-on: https://go-review.googlesource.com/4790 Reviewed-by: Minux Ma <minux@golang.org>
2015-02-13 06:45:02 +03:00
addBuilder(BuildConfig{
cmd/coordinator: delete useScheduler knob & old pre-scheduler hacks, code, TODOs Updates golang/go#19178 Change-Id: Id4d8b016c41f57bfaba5ee4ab046285607047493 Reviewed-on: https://go-review.googlesource.com/c/build/+/207178 Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-11-14 18:43:33 +03:00
Name: "linux-386-clang",
HostType: "host-linux-clang",
Notes: "Debian jessie + clang 3.9 instead of gcc",
env: []string{"CC=/usr/bin/clang", "GOHOSTARCH=386"},
env: convert linux-x86-clang to be VM-based Change-Id: I583d751652dec9176169b4d4705f3813506017e3 Reviewed-on: https://go-review.googlesource.com/4790 Reviewed-by: Minux Ma <minux@golang.org>
2015-02-13 06:45:02 +03:00
})
addBuilder(BuildConfig{
cmd/coordinator: delete useScheduler knob & old pre-scheduler hacks, code, TODOs Updates golang/go#19178 Change-Id: Id4d8b016c41f57bfaba5ee4ab046285607047493 Reviewed-on: https://go-review.googlesource.com/c/build/+/207178 Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-11-14 18:43:33 +03:00
Name: "linux-amd64-clang",
HostType: "host-linux-clang",
Notes: "Debian jessie + clang 3.9 instead of gcc",
env: []string{"CC=/usr/bin/clang"},
env: convert linux-x86-clang to be VM-based Change-Id: I583d751652dec9176169b4d4705f3813506017e3 Reviewed-on: https://go-review.googlesource.com/4790 Reviewed-by: Minux Ma <minux@golang.org>
2015-02-13 06:45:02 +03:00
})
env: update linux-x86-sid to be a VM Fixes #9873 Change-Id: I08c501ef52ebde4cdbc48fe45a5342488000c433 Reviewed-on: https://go-review.googlesource.com/4874 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2015-02-14 06:01:32 +03:00
addBuilder(BuildConfig{
cmd/coordinator: delete useScheduler knob & old pre-scheduler hacks, code, TODOs Updates golang/go#19178 Change-Id: Id4d8b016c41f57bfaba5ee4ab046285607047493 Reviewed-on: https://go-review.googlesource.com/c/build/+/207178 Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-11-14 18:43:33 +03:00
Name: "linux-386-sid",
HostType: "host-linux-sid",
Notes: "Debian sid (unstable)",
env: []string{"GOHOSTARCH=386"},
env: update linux-x86-sid to be a VM Fixes #9873 Change-Id: I08c501ef52ebde4cdbc48fe45a5342488000c433 Reviewed-on: https://go-review.googlesource.com/4874 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2015-02-14 06:01:32 +03:00
})
addBuilder(BuildConfig{
cmd/coordinator: delete useScheduler knob & old pre-scheduler hacks, code, TODOs Updates golang/go#19178 Change-Id: Id4d8b016c41f57bfaba5ee4ab046285607047493 Reviewed-on: https://go-review.googlesource.com/c/build/+/207178 Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-11-14 18:43:33 +03:00
Name: "linux-amd64-sid",
HostType: "host-linux-sid",
Notes: "Debian sid (unstable)",
env: update linux-x86-sid to be a VM Fixes #9873 Change-Id: I08c501ef52ebde4cdbc48fe45a5342488000c433 Reviewed-on: https://go-review.googlesource.com/4874 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2015-02-14 06:01:32 +03:00
})
env/linux-x86-fedora: add Fedora builder Not using a Fedora kernel yet, but better than nothing. Updates golang/go#26796 Change-Id: I228442bf6606696407a8d3fbd098df59df7353e1 Reviewed-on: https://go-review.googlesource.com/c/build/+/175999 Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-05-08 19:25:38 +03:00
addBuilder(BuildConfig{
cmd/coordinator: delete useScheduler knob & old pre-scheduler hacks, code, TODOs Updates golang/go#19178 Change-Id: Id4d8b016c41f57bfaba5ee4ab046285607047493 Reviewed-on: https://go-review.googlesource.com/c/build/+/207178 Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-11-14 18:43:33 +03:00
Name: "linux-amd64-fedora",
HostType: "host-linux-fedora",
Notes: "Fedora",
env/linux-x86-fedora: add Fedora builder Not using a Fedora kernel yet, but better than nothing. Updates golang/go#26796 Change-Id: I228442bf6606696407a8d3fbd098df59df7353e1 Reviewed-on: https://go-review.googlesource.com/c/build/+/175999 Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-05-08 19:25:38 +03:00
})
dashboard: add a GOOS=linux builder & trybot for Android Updates golang/go#30482 Change-Id: I198dd40dfdb86b20e681c373141754680351e82f Reviewed-on: https://go-review.googlesource.com/c/164740 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Elias Naur <mail@eliasnaur.com>
2019-03-01 19:54:25 +03:00
addBuilder(BuildConfig{
Name: "linux-amd64-androidemu",
HostType: "host-android-amd64-emu",
env: []string{
"GOARCH=amd64",
"GOOS=linux",
"CGO_ENABLED=1",
cmd/coordinator: follow module proxy redirects And because this uses new-in-Go1.13 http.Header.Clone, restrict all the files to Go 1.13+ and bump the Dockerfile Go version. Fixes golang/go#33928 Change-Id: I147cc4465e393d2c34e59c70edf65f3901a5e132 Reviewed-on: https://go-review.googlesource.com/c/build/+/192341 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-08-29 21:28:01 +03:00
"GO_DISABLE_OUTBOUND_NETWORK=1",
dashboard: add a GOOS=linux builder & trybot for Android Updates golang/go#30482 Change-Id: I198dd40dfdb86b20e681c373141754680351e82f Reviewed-on: https://go-review.googlesource.com/c/164740 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Elias Naur <mail@eliasnaur.com>
2019-03-01 19:54:25 +03:00
},
dashboard: unify post-submit & trybot policy in builders.go, not coordinator This removes a bunch of TODOs about scattered policy and moves it all into builders.go. While here, * add a bunch of tests * unexport some things * rename some things * document some things * adjust FreeBSD policy as function of branch (per Go 1.12 being last to support FreeBSD 10.x, and to unblock CL 165801) * set GO_BUILDER_SET_GOPROXY=coordinator for reverse buildlets, which is necessary to remove the oauth2 & build special cases in the config * change Elias Naur's mobile builder to how I think he wants it (he was fighting the old system) * add $HOME on the Solaris smartos builder, which was missing & causing tests to fail lately * make the (currently failing) longtest builder have GOPROXY set * remove an allocation in version.ParseReleaseBranch Fixes golang/go#9603 Updates golang/go#14594 Change-Id: I50a23ad7cdf478c95b14bee9b3931ba361baacfa Reviewed-on: https://go-review.googlesource.com/c/build/+/166218 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-03-07 20:44:41 +03:00
tryBot: func(repo, branch, goBranch string) bool {
dashboard: add a GOOS=linux builder & trybot for Android Updates golang/go#30482 Change-Id: I198dd40dfdb86b20e681c373141754680351e82f Reviewed-on: https://go-review.googlesource.com/c/164740 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Elias Naur <mail@eliasnaur.com>
2019-03-01 19:54:25 +03:00
// Only for mobile repo for now, not "go":
dashboard: unify post-submit & trybot policy in builders.go, not coordinator This removes a bunch of TODOs about scattered policy and moves it all into builders.go. While here, * add a bunch of tests * unexport some things * rename some things * document some things * adjust FreeBSD policy as function of branch (per Go 1.12 being last to support FreeBSD 10.x, and to unblock CL 165801) * set GO_BUILDER_SET_GOPROXY=coordinator for reverse buildlets, which is necessary to remove the oauth2 & build special cases in the config * change Elias Naur's mobile builder to how I think he wants it (he was fighting the old system) * add $HOME on the Solaris smartos builder, which was missing & causing tests to fail lately * make the (currently failing) longtest builder have GOPROXY set * remove an allocation in version.ParseReleaseBranch Fixes golang/go#9603 Updates golang/go#14594 Change-Id: I50a23ad7cdf478c95b14bee9b3931ba361baacfa Reviewed-on: https://go-review.googlesource.com/c/build/+/166218 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-03-07 20:44:41 +03:00
return repo == "mobile" && branch == "master" && goBranch == "master"
dashboard: add a GOOS=linux builder & trybot for Android Updates golang/go#30482 Change-Id: I198dd40dfdb86b20e681c373141754680351e82f Reviewed-on: https://go-review.googlesource.com/c/164740 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Elias Naur <mail@eliasnaur.com>
2019-03-01 19:54:25 +03:00
},
dashboard: unify post-submit & trybot policy in builders.go, not coordinator This removes a bunch of TODOs about scattered policy and moves it all into builders.go. While here, * add a bunch of tests * unexport some things * rename some things * document some things * adjust FreeBSD policy as function of branch (per Go 1.12 being last to support FreeBSD 10.x, and to unblock CL 165801) * set GO_BUILDER_SET_GOPROXY=coordinator for reverse buildlets, which is necessary to remove the oauth2 & build special cases in the config * change Elias Naur's mobile builder to how I think he wants it (he was fighting the old system) * add $HOME on the Solaris smartos builder, which was missing & causing tests to fail lately * make the (currently failing) longtest builder have GOPROXY set * remove an allocation in version.ParseReleaseBranch Fixes golang/go#9603 Updates golang/go#14594 Change-Id: I50a23ad7cdf478c95b14bee9b3931ba361baacfa Reviewed-on: https://go-review.googlesource.com/c/build/+/166218 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-03-07 20:44:41 +03:00
buildsRepo: func(repo, branch, goBranch string) bool {
return repo == "mobile" && branch == "master" && goBranch == "master"
dashboard: add a GOOS=linux builder & trybot for Android Updates golang/go#30482 Change-Id: I198dd40dfdb86b20e681c373141754680351e82f Reviewed-on: https://go-review.googlesource.com/c/164740 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Elias Naur <mail@eliasnaur.com>
2019-03-01 19:54:25 +03:00
},
Notes: "Runs GOOS=linux but with the Android emulator attached, for running x/mobile host tests.",
})
dashboard, env: add Debian Stretch builder, rename Jessie builder Updates golang/go#25386 Change-Id: I65e3dda63e5ea86fa94480e3305556b73afe121a Reviewed-on: https://go-review.googlesource.com/113059 Reviewed-by: Bryan C. Mills <bcmills@google.com>
2018-05-14 20:22:45 +03:00
addBuilder(BuildConfig{
cmd/coordinator: delete useScheduler knob & old pre-scheduler hacks, code, TODOs Updates golang/go#19178 Change-Id: Id4d8b016c41f57bfaba5ee4ab046285607047493 Reviewed-on: https://go-review.googlesource.com/c/build/+/207178 Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-11-14 18:43:33 +03:00
Name: "linux-amd64-jessie",
HostType: "host-linux-jessie",
dashboard: update Debian version for amd64 builders This change updates various linux-amd64 builders after the release of bullseye: - linux-amd64 has been updated to bullseye. - linux-amd64-nocgo has been updated to bullseye. - linux-amd64-noopt has been updated to bullseye. - linux-amd64-ssacheck has been updated to bullseye. - linux-amd64-racecompile has been updated to bullseye. - linux-amd64-race has been updated to bullseye. - linux-amd64-jessie has had the notes updated. - misc-compile has been updated to bullseye. Fixes golang/go#47721 Change-Id: I29c5d76da6ad6b28647bd423a8ef7dc13b919ca2 Reviewed-on: https://go-review.googlesource.com/c/build/+/343633 Trust: Carlos Amedee <carlos@golang.org> Run-TryBot: Carlos Amedee <carlos@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2021-08-19 20:27:43 +03:00
Notes: "Debian Jessie.",
dashboard: enable outbound firewall on a few more Linux-based builders Updates golang/go#30612 Change-Id: Id472e753043976b8671c15491b82bb57e3804d95 Reviewed-on: https://go-review.googlesource.com/c/build/+/165638 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-03-06 08:20:12 +03:00
env: []string{
cmd/coordinator: follow module proxy redirects And because this uses new-in-Go1.13 http.Header.Clone, restrict all the files to Go 1.13+ and bump the Dockerfile Go version. Fixes golang/go#33928 Change-Id: I147cc4465e393d2c34e59c70edf65f3901a5e132 Reviewed-on: https://go-review.googlesource.com/c/build/+/192341 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-08-29 21:28:01 +03:00
"GO_DISABLE_OUTBOUND_NETWORK=1",
dashboard: enable outbound firewall on a few more Linux-based builders Updates golang/go#30612 Change-Id: Id472e753043976b8671c15491b82bb57e3804d95 Reviewed-on: https://go-review.googlesource.com/c/build/+/165638 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-03-06 08:20:12 +03:00
},
dashboard, env: add Debian Stretch builder, rename Jessie builder Updates golang/go#25386 Change-Id: I65e3dda63e5ea86fa94480e3305556b73afe121a Reviewed-on: https://go-review.googlesource.com/113059 Reviewed-by: Bryan C. Mills <bcmills@google.com>
2018-05-14 20:22:45 +03:00
})
cmd/release, dashboard: implement builder plan for Go 1.16 See https://golang.org/issues/40561#issuecomment-731482962 for the plan description and rationale. Add new builder definitions that are needed for the plan. Don't rely on generic builder name like "linux-amd64" to avoid a repeat of golang.org/issue/31293; use a specific builder name instead. Remove support and test cases for Go 1.13, it's unsupported per release policy. For golang/go#40561. Change-Id: I070744e15be9f1932d649a27ee7e4599e467553f Reviewed-on: https://go-review.googlesource.com/c/build/+/276034 Trust: Dmitri Shuralyov <dmitshur@golang.org> Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
2020-12-08 06:31:50 +03:00
addBuilder(BuildConfig{
Name: "linux-amd64-stretch",
HostType: "host-linux-stretch",
Notes: "Debian Stretch. Same as the normal 'linux-amd64' builder at this time, but with -stretch suffix. Used for release builds.",
buildsRepo: disabledBuilder, // Disabled because the "linux-amd64" builder does identical work.
env: []string{
"GO_DISABLE_OUTBOUND_NETWORK=1",
},
})
env/linux-x86-bullseye, dashboard: add Debian Bullseye builder This change adds the host-linux-bullseye host as well as the configuration for the host. The linux-amd64-bullseye builder has been added in this change as well. Fixes golang/go#47720 Change-Id: I607f937133a0db3ed6a841f53dc7fef5e310a5a4 Reviewed-on: https://go-review.googlesource.com/c/build/+/342630 Trust: Carlos Amedee <carlos@golang.org> Run-TryBot: Carlos Amedee <carlos@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org>
2021-08-16 22:35:39 +03:00
addBuilder(BuildConfig{
Name: "linux-amd64-bullseye",
HostType: "host-linux-bullseye",
Notes: "Debian Bullseye.",
env: []string{
"GO_DISABLE_OUTBOUND_NETWORK=1",
},
})
env/linux-x86-buster: add Debian Buster builder This change adds a Debian Buster builder. Fixes golang/go#42410 Change-Id: If11ebe6df3bad88aac40b0a6f841edc5fb3a5b4a Reviewed-on: https://go-review.googlesource.com/c/build/+/267937 Trust: Carlos Amedee <carlos@golang.org> Run-TryBot: Carlos Amedee <carlos@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2020-11-02 23:11:10 +03:00
addBuilder(BuildConfig{
Name: "linux-amd64-buster",
HostType: "host-linux-buster",
Notes: "Debian Buster.",
env: []string{
"GO_DISABLE_OUTBOUND_NETWORK=1",
},
})
cmd/release, dashboard: implement builder plan for Go 1.16 See https://golang.org/issues/40561#issuecomment-731482962 for the plan description and rationale. Add new builder definitions that are needed for the plan. Don't rely on generic builder name like "linux-amd64" to avoid a repeat of golang.org/issue/31293; use a specific builder name instead. Remove support and test cases for Go 1.13, it's unsupported per release policy. For golang/go#40561. Change-Id: I070744e15be9f1932d649a27ee7e4599e467553f Reviewed-on: https://go-review.googlesource.com/c/build/+/276034 Trust: Dmitri Shuralyov <dmitshur@golang.org> Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
2020-12-08 06:31:50 +03:00
addBuilder(BuildConfig{
dashboard: update Debian version for 386 builders This change updates various linux-386 builders after the release of bullseye: - linux-386 has been updated to bullseye. - linux-386-jessie has had the notes updated. - linux-386-buster has been added. Updates golang/go#47721 Change-Id: Iab0048fcad4d51839013074a51ad37b424d3a7af Reviewed-on: https://go-review.googlesource.com/c/build/+/343632 Trust: Carlos Amedee <carlos@golang.org> Run-TryBot: Carlos Amedee <carlos@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2021-08-19 00:45:09 +03:00
Name: "linux-386-jessie",
HostType: "host-linux-jessie",
Notes: "Debian Jessie, 32-bit builder.",
cmd/release, dashboard: implement builder plan for Go 1.16 See https://golang.org/issues/40561#issuecomment-731482962 for the plan description and rationale. Add new builder definitions that are needed for the plan. Don't rely on generic builder name like "linux-amd64" to avoid a repeat of golang.org/issue/31293; use a specific builder name instead. Remove support and test cases for Go 1.13, it's unsupported per release policy. For golang/go#40561. Change-Id: I070744e15be9f1932d649a27ee7e4599e467553f Reviewed-on: https://go-review.googlesource.com/c/build/+/276034 Trust: Dmitri Shuralyov <dmitshur@golang.org> Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
2020-12-08 06:31:50 +03:00
env: []string{
"GOARCH=386",
"GOHOSTARCH=386",
"GO_DISABLE_OUTBOUND_NETWORK=1",
},
})
addBuilder(BuildConfig{
Name: "linux-386-stretch",
HostType: "host-linux-stretch",
Notes: "Debian Stretch, 32-bit builder.",
env: []string{
"GOARCH=386",
"GOHOSTARCH=386",
"GO_DISABLE_OUTBOUND_NETWORK=1",
},
})
dashboard: update Debian version for 386 builders This change updates various linux-386 builders after the release of bullseye: - linux-386 has been updated to bullseye. - linux-386-jessie has had the notes updated. - linux-386-buster has been added. Updates golang/go#47721 Change-Id: Iab0048fcad4d51839013074a51ad37b424d3a7af Reviewed-on: https://go-review.googlesource.com/c/build/+/343632 Trust: Carlos Amedee <carlos@golang.org> Run-TryBot: Carlos Amedee <carlos@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2021-08-19 00:45:09 +03:00
addBuilder(BuildConfig{
Name: "linux-386-buster",
HostType: "host-linux-buster",
Notes: "Debian Buster, 32-bit builder.",
env: []string{
"GOARCH=386",
"GOHOSTARCH=386",
"GO_DISABLE_OUTBOUND_NETWORK=1",
},
})
dashboard: add "longtest" builder to run all tests, without -short mode Fixes golang/go#12508 Change-Id: I62e1e296f10c7622c1967a5530c4b0e07c136dfd Reviewed-on: https://go-review.googlesource.com/113436 Reviewed-by: Bryan C. Mills <bcmills@google.com>
2018-05-16 22:50:39 +03:00
addBuilder(BuildConfig{
cmd/coordinator: delete useScheduler knob & old pre-scheduler hacks, code, TODOs Updates golang/go#19178 Change-Id: Id4d8b016c41f57bfaba5ee4ab046285607047493 Reviewed-on: https://go-review.googlesource.com/c/build/+/207178 Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-11-14 18:43:33 +03:00
Name: "linux-amd64-longtest",
HostType: "host-linux-stretch-morecpu",
Notes: "Debian Stretch with go test -short=false",
dashboard: include longtest builders for trybots on release branches The goal of this test policy change is to decrease the chance of a Go repository release branch from getting into a state where some longtest post-submit builders are failing, as this will interfere with the ability to issue releases after golang/go#29252 is fixed. There are not that many CLs sent to release branches, so the extra latency caused by slower longtest builders being a part of trybots should not be very disruptive. For golang/go#37827. For golang/go#29252. Change-Id: I819b120bb7b763df4acf4583375cdb840793d323 Reviewed-on: https://go-review.googlesource.com/c/build/+/235338 Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
2020-05-27 01:19:05 +03:00
tryBot: func(repo, branch, goBranch string) bool {
onReleaseBranch := strings.HasPrefix(branch, "release-branch.")
return repo == "go" && onReleaseBranch // See issue 37827.
},
dashboard, cmd/coordinator: various fixes * add support for longtest builders for subrepos. * fix race builders for subrepos (they weren't passing the -race flag). * adjust the policy for the js-wasm builders to build fewer subrepos where it'll never work or isn't worth it. * fix the android emu builders which disappeared because an empty string was being passed to buildsRepoAtAll. In some places in the code an empty string for goBranch for the "go" repo meant the same as branch, but I forgot that in the new code, so an old caller was confusing the new config hooks. Rather than make all policy funcs be aware of both ways, the new code in this CL now maps an empty string to the same as the repo's branch when the repo == "go". Adds a test too. * fix some outdated comments. Change-Id: Icf3fb85e5542a4d314443b59d02517b306ef46b7 Reviewed-on: https://go-review.googlesource.com/c/build/+/166897 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-03-11 21:49:56 +03:00
buildsRepo: func(repo, branch, goBranch string) bool {
dashboard: document and clarify defaultBuildsRepoPolicy The defaultBuildsRepoPolicy function was very important in that any changes to it would affect many builders, but it was not documented well. This refactor attempts to improve the situation. It turned out better to also rename defaultBuildsRepoPolicy to buildRepoByDefault and limit its scope to report whether a given repo should be built by default. The branch and goBranch parameters were never used, and thus misleading. If they're needed in the future, we can easily re-add them. The name defaultBuildsRepoPolicy made it sound as if it was the entire "builds repo" policy, bun reality most of the policy is implemented in the BuildConfig.buildsRepoAtAll method and is non-configurable. Only a small part of the policy was in defaultBuildsRepoPolicy. For golang/go#39156. Change-Id: Ie7c6a68bd624056362dc7d964bd8d640f665b35a Reviewed-on: https://go-review.googlesource.com/c/build/+/240687 Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-07-01 21:45:10 +03:00
// Test all repos, ignoring buildRepoByDefault.
// For golang.org/x repos, don't test non-latest versions.
dashboard, cmd/coordinator: various fixes * add support for longtest builders for subrepos. * fix race builders for subrepos (they weren't passing the -race flag). * adjust the policy for the js-wasm builders to build fewer subrepos where it'll never work or isn't worth it. * fix the android emu builders which disappeared because an empty string was being passed to buildsRepoAtAll. In some places in the code an empty string for goBranch for the "go" repo meant the same as branch, but I forgot that in the new code, so an old caller was confusing the new config hooks. Rather than make all policy funcs be aware of both ways, the new code in this CL now maps an empty string to the same as the repo's branch when the repo == "go". Adds a test too. * fix some outdated comments. Change-Id: Icf3fb85e5542a4d314443b59d02517b306ef46b7 Reviewed-on: https://go-review.googlesource.com/c/build/+/166897 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-03-11 21:49:56 +03:00
return repo == "go" || (branch == "master" && goBranch == "master")
},
dashboard: unify post-submit & trybot policy in builders.go, not coordinator This removes a bunch of TODOs about scattered policy and moves it all into builders.go. While here, * add a bunch of tests * unexport some things * rename some things * document some things * adjust FreeBSD policy as function of branch (per Go 1.12 being last to support FreeBSD 10.x, and to unblock CL 165801) * set GO_BUILDER_SET_GOPROXY=coordinator for reverse buildlets, which is necessary to remove the oauth2 & build special cases in the config * change Elias Naur's mobile builder to how I think he wants it (he was fighting the old system) * add $HOME on the Solaris smartos builder, which was missing & causing tests to fail lately * make the (currently failing) longtest builder have GOPROXY set * remove an allocation in version.ParseReleaseBranch Fixes golang/go#9603 Updates golang/go#14594 Change-Id: I50a23ad7cdf478c95b14bee9b3931ba361baacfa Reviewed-on: https://go-review.googlesource.com/c/build/+/166218 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-03-07 20:44:41 +03:00
needsGoProxy: true, // for cmd/go module tests
dashboard: increase TIMEOUT_SCALE to 2 for linux-amd64-longtest runtime tests take about 180s in long mode, too close to the timeout limit (180 seconds), and this is causing sporadic timeout failures like https://build.golang.org/log/cda568a39f396e3fee64778a475528be452632ed Bump up the builder's timeout by 2x. Fixes golang/go#25629 Change-Id: I77e0f86f0b236cb953457ebdcbfc5763c2c6d54b Reviewed-on: https://go-review.googlesource.com/115016 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-05-29 22:03:23 +03:00
env: []string{
dashboard: increase timeout scale for longtest builders, add more CPU Updates golang/go#25886 Change-Id: I5168e291ab77cbd3843bdc39e319a68dfa65aedd Reviewed-on: https://go-review.googlesource.com/c/build/+/167638 Reviewed-by: Russ Cox <rsc@golang.org>
2019-03-14 19:06:57 +03:00
"GO_TEST_TIMEOUT_SCALE=5", // give them lots of time
dashboard: increase TIMEOUT_SCALE to 2 for linux-amd64-longtest runtime tests take about 180s in long mode, too close to the timeout limit (180 seconds), and this is causing sporadic timeout failures like https://build.golang.org/log/cda568a39f396e3fee64778a475528be452632ed Bump up the builder's timeout by 2x. Fixes golang/go#25629 Change-Id: I77e0f86f0b236cb953457ebdcbfc5763c2c6d54b Reviewed-on: https://go-review.googlesource.com/115016 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-05-29 22:03:23 +03:00
},
dashboard: pick 4 TryBot helpers for -longtest SlowBots Based on the data we have so far, 4 helpers seems like a good number to pick. It provides a balance between resource use and diminishing returns. We may need to adjust it as future work for issue 42661, but a first step will be to reduce overhead in accessing historical build timing data. Fixes golang/go#37439. Updates golang/go#42661. Change-Id: I1bfddf85f2dba8b42dc7b8f8749bbf45b614e93d Reviewed-on: https://go-review.googlesource.com/c/build/+/279513 Trust: Dmitri Shuralyov <dmitshur@golang.org> Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-12-21 22:42:00 +03:00
numTryTestHelpers: 4, // Target time is < 15 min for golang.org/issue/42661.
dashboard: add "longtest" builder to run all tests, without -short mode Fixes golang/go#12508 Change-Id: I62e1e296f10c7622c1967a5530c4b0e07c136dfd Reviewed-on: https://go-review.googlesource.com/113436 Reviewed-by: Bryan C. Mills <bcmills@google.com>
2018-05-16 22:50:39 +03:00
})
dashboard: add windows and 32-bit longtest builders Fixes golang/go#29641 Fixes golang/go#26529 Change-Id: I5ab3aa443276ab36b866d7fabc1d78a82b34cb64 Reviewed-on: https://go-review.googlesource.com/c/build/+/202479 Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-10-21 19:58:47 +03:00
addBuilder(BuildConfig{
cmd/coordinator: delete useScheduler knob & old pre-scheduler hacks, code, TODOs Updates golang/go#19178 Change-Id: Id4d8b016c41f57bfaba5ee4ab046285607047493 Reviewed-on: https://go-review.googlesource.com/c/build/+/207178 Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-11-14 18:43:33 +03:00
Name: "linux-386-longtest",
HostType: "host-linux-stretch-morecpu",
Notes: "Debian Stretch with go test -short=false; to get 32-bit coverage",
dashboard: include longtest builders for trybots on release branches The goal of this test policy change is to decrease the chance of a Go repository release branch from getting into a state where some longtest post-submit builders are failing, as this will interfere with the ability to issue releases after golang/go#29252 is fixed. There are not that many CLs sent to release branches, so the extra latency caused by slower longtest builders being a part of trybots should not be very disruptive. For golang/go#37827. For golang/go#29252. Change-Id: I819b120bb7b763df4acf4583375cdb840793d323 Reviewed-on: https://go-review.googlesource.com/c/build/+/235338 Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
2020-05-27 01:19:05 +03:00
tryBot: func(repo, branch, goBranch string) bool {
onReleaseBranch := strings.HasPrefix(branch, "release-branch.")
return repo == "go" && onReleaseBranch // See issue 37827.
},
dashboard: add windows and 32-bit longtest builders Fixes golang/go#29641 Fixes golang/go#26529 Change-Id: I5ab3aa443276ab36b866d7fabc1d78a82b34cb64 Reviewed-on: https://go-review.googlesource.com/c/build/+/202479 Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-10-21 19:58:47 +03:00
buildsRepo: func(repo, branch, goBranch string) bool {
dashboard: document and clarify defaultBuildsRepoPolicy The defaultBuildsRepoPolicy function was very important in that any changes to it would affect many builders, but it was not documented well. This refactor attempts to improve the situation. It turned out better to also rename defaultBuildsRepoPolicy to buildRepoByDefault and limit its scope to report whether a given repo should be built by default. The branch and goBranch parameters were never used, and thus misleading. If they're needed in the future, we can easily re-add them. The name defaultBuildsRepoPolicy made it sound as if it was the entire "builds repo" policy, bun reality most of the policy is implemented in the BuildConfig.buildsRepoAtAll method and is non-configurable. Only a small part of the policy was in defaultBuildsRepoPolicy. For golang/go#39156. Change-Id: Ie7c6a68bd624056362dc7d964bd8d640f665b35a Reviewed-on: https://go-review.googlesource.com/c/build/+/240687 Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-07-01 21:45:10 +03:00
b := buildRepoByDefault(repo)
if repo != "go" && !(branch == "master" && goBranch == "master") {
// For golang.org/x repos, don't test non-latest versions.
b = false
dashboard: stop testing x/build on even more builders Follow up to CL 210378 after looking at: https://build.golang.org/?repo=golang.org%2fx%2fbuild Updates golang/go#30640 Change-Id: I81505a9ca2985ffa6e75922665b14ef4de2af860 Reviewed-on: https://go-review.googlesource.com/c/build/+/211100 Reviewed-by: Bryan C. Mills <bcmills@google.com> Run-TryBot: Bryan C. Mills <bcmills@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-12-12 20:43:59 +03:00
}
dashboard: document and clarify defaultBuildsRepoPolicy The defaultBuildsRepoPolicy function was very important in that any changes to it would affect many builders, but it was not documented well. This refactor attempts to improve the situation. It turned out better to also rename defaultBuildsRepoPolicy to buildRepoByDefault and limit its scope to report whether a given repo should be built by default. The branch and goBranch parameters were never used, and thus misleading. If they're needed in the future, we can easily re-add them. The name defaultBuildsRepoPolicy made it sound as if it was the entire "builds repo" policy, bun reality most of the policy is implemented in the BuildConfig.buildsRepoAtAll method and is non-configurable. Only a small part of the policy was in defaultBuildsRepoPolicy. For golang/go#39156. Change-Id: Ie7c6a68bd624056362dc7d964bd8d640f665b35a Reviewed-on: https://go-review.googlesource.com/c/build/+/240687 Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-07-01 21:45:10 +03:00
return b
dashboard: add windows and 32-bit longtest builders Fixes golang/go#29641 Fixes golang/go#26529 Change-Id: I5ab3aa443276ab36b866d7fabc1d78a82b34cb64 Reviewed-on: https://go-review.googlesource.com/c/build/+/202479 Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-10-21 19:58:47 +03:00
},
needsGoProxy: true, // for cmd/go module tests
env: []string{
dashboard: set GO{,HOST}ARCH in linux-386-longtest builder The linux-386-longtest builder was added in CL 202479 with these environment variables unset, which caused the builder to run long tests for the linux/amd64 pair. Setting these to "386" makes the builder run long tests for linux/386 instead, which increases 32-bit test coverage as originally intended. For golang/go#29641. Change-Id: I9fbebc9e37e5f27f21d840d86006192daeaee8c5 Reviewed-on: https://go-review.googlesource.com/c/build/+/234520 Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-05-19 02:34:15 +03:00
"GOARCH=386",
"GOHOSTARCH=386",
dashboard: add windows and 32-bit longtest builders Fixes golang/go#29641 Fixes golang/go#26529 Change-Id: I5ab3aa443276ab36b866d7fabc1d78a82b34cb64 Reviewed-on: https://go-review.googlesource.com/c/build/+/202479 Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-10-21 19:58:47 +03:00
"GO_TEST_TIMEOUT_SCALE=5", // give them lots of time
},
dashboard: pick 4 TryBot helpers for -longtest SlowBots Based on the data we have so far, 4 helpers seems like a good number to pick. It provides a balance between resource use and diminishing returns. We may need to adjust it as future work for issue 42661, but a first step will be to reduce overhead in accessing historical build timing data. Fixes golang/go#37439. Updates golang/go#42661. Change-Id: I1bfddf85f2dba8b42dc7b8f8749bbf45b614e93d Reviewed-on: https://go-review.googlesource.com/c/build/+/279513 Trust: Dmitri Shuralyov <dmitshur@golang.org> Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-12-21 22:42:00 +03:00
numTryTestHelpers: 4, // Target time is < 15 min for golang.org/issue/42661.
dashboard: add windows and 32-bit longtest builders Fixes golang/go#29641 Fixes golang/go#26529 Change-Id: I5ab3aa443276ab36b866d7fabc1d78a82b34cb64 Reviewed-on: https://go-review.googlesource.com/c/build/+/202479 Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-10-21 19:58:47 +03:00
})
env/js-wasm, dashboard: add start of a js-wasm builder Updates golang/go#18892 Change-Id: I4b16afe8f0ce74f6ecd022f8bca14ada94ab308f Reviewed-on: https://go-review.googlesource.com/112736 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-05-11 08:54:32 +03:00
addBuilder(BuildConfig{
dashboard: speed up js/wasm trybots We don't need to run all the tests in a trybot. Run just enough to get good coverage, without going over 5 minutes. Any regressions elsewhere will be caught by the build.golang.org (slower) runs. Change-Id: I32f1fc17681bfb509844d2bd35b05c950806d283 Reviewed-on: https://go-review.googlesource.com/121938 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-06-28 23:30:35 +03:00
Name: "js-wasm",
HostType: "host-js-wasm",
dashboard, cmd/coordinator: move more config out of coordinator into dashboard And with the increased flexibility, enable trybots for more repos for the "sys" project. Details: https://github.com/golang/go/issues/24893#issuecomment-433517519 Fixes golang/go#24893 Change-Id: I89ce6f219be986201332f8fad11c774bdd6246da Reviewed-on: https://go-review.googlesource.com/c/145157 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2018-10-26 22:21:58 +03:00
tryBot: explicitTrySet("go"),
dashboard: update some build policy Mostly some regressions (building too much) from the massive refactoring in CL 166218, but also some minor changes. See test updates. Change-Id: Ie30f51b425367f18a49764ac971e262131959bc8 Reviewed-on: https://go-review.googlesource.com/c/build/+/166557 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-03-11 07:20:46 +03:00
buildsRepo: func(repo, branch, goBranch string) bool {
dashboard, cmd/coordinator: various fixes * add support for longtest builders for subrepos. * fix race builders for subrepos (they weren't passing the -race flag). * adjust the policy for the js-wasm builders to build fewer subrepos where it'll never work or isn't worth it. * fix the android emu builders which disappeared because an empty string was being passed to buildsRepoAtAll. In some places in the code an empty string for goBranch for the "go" repo meant the same as branch, but I forgot that in the new code, so an old caller was confusing the new config hooks. Rather than make all policy funcs be aware of both ways, the new code in this CL now maps an empty string to the same as the repo's branch when the repo == "go". Adds a test too. * fix some outdated comments. Change-Id: Icf3fb85e5542a4d314443b59d02517b306ef46b7 Reviewed-on: https://go-review.googlesource.com/c/build/+/166897 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-03-11 21:49:56 +03:00
switch repo {
case "go":
return true
dashboard: stop testing x/build on even more builders Follow up to CL 210378 after looking at: https://build.golang.org/?repo=golang.org%2fx%2fbuild Updates golang/go#30640 Change-Id: I81505a9ca2985ffa6e75922665b14ef4de2af860 Reviewed-on: https://go-review.googlesource.com/c/build/+/211100 Reviewed-by: Bryan C. Mills <bcmills@google.com> Run-TryBot: Bryan C. Mills <bcmills@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-12-12 20:43:59 +03:00
case "build", "mobile", "exp", "benchmarks", "debug", "perf", "talks", "tools", "tour", "website":
dashboard, cmd/coordinator: various fixes * add support for longtest builders for subrepos. * fix race builders for subrepos (they weren't passing the -race flag). * adjust the policy for the js-wasm builders to build fewer subrepos where it'll never work or isn't worth it. * fix the android emu builders which disappeared because an empty string was being passed to buildsRepoAtAll. In some places in the code an empty string for goBranch for the "go" repo meant the same as branch, but I forgot that in the new code, so an old caller was confusing the new config hooks. Rather than make all policy funcs be aware of both ways, the new code in this CL now maps an empty string to the same as the repo's branch when the repo == "go". Adds a test too. * fix some outdated comments. Change-Id: Icf3fb85e5542a4d314443b59d02517b306ef46b7 Reviewed-on: https://go-review.googlesource.com/c/build/+/166897 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-03-11 21:49:56 +03:00
return false
default:
return branch == "master" && goBranch == "master"
}
dashboard: update some build policy Mostly some regressions (building too much) from the massive refactoring in CL 166218, but also some minor changes. See test updates. Change-Id: Ie30f51b425367f18a49764ac971e262131959bc8 Reviewed-on: https://go-review.googlesource.com/c/build/+/166557 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-03-11 07:20:46 +03:00
},
cmd/coordinator, dashboard: don't skip cmd/dist tests for SlowBots When a user explicitly requests a SlowBot builder via the TRY= syntax, it is a signal that they're willing to wait longer for test results. It's more useful to not skip any tests and give the user confidence that the SlowBot run will catch all issues that would otherwise be caught only by post-submit builders after the CL is submitted. This is done by considering a TryBot that isn't an explicit SlowBot to be a "normal TryBot", and the ShouldRunDistTest policy function is provided a parameter specifying whether it's a normal TryBot (non-SlowBot) run. Tests are only skipped during normal TryBot runs, not SlowBot runs. Fixes golang/go#38279. For golang/go#34501. Change-Id: Ia11fd51c71f9de1089b6b6f4c027014893f224b9 Reviewed-on: https://go-review.googlesource.com/c/build/+/227779 Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-04-09 23:24:30 +03:00
distTestAdjust: func(run bool, distTest string, isNormalTry bool) bool {
if isNormalTry {
dashboard: speed up js/wasm trybots We don't need to run all the tests in a trybot. Run just enough to get good coverage, without going over 5 minutes. Any regressions elsewhere will be caught by the build.golang.org (slower) runs. Change-Id: I32f1fc17681bfb509844d2bd35b05c950806d283 Reviewed-on: https://go-review.googlesource.com/121938 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-06-28 23:30:35 +03:00
if strings.Contains(distTest, "/internal/") ||
strings.Contains(distTest, "vendor/golang.org/x/arch") {
return false
}
switch distTest {
dashboard: disable reboot test on js-wasm builder The reboot test verifies that the modified version of Go can be used for bootstrapping purposes. The js-wasm builder is configured with GOHOSTOS=linux and GOHOSTARCH=amd64 so that it builds packages and commands for host, linux/amd64, which is used during bootstrapping in the reboot test. Since this doesn't provide additional coverage beyond what we get from linux-amd64's execution of the reboot test, disable this test on the js-wasm builder. Also remove some accidental duplicate code. For golang/go#35170. Change-Id: Ic770d04f6f5adee7cb836b1a8440dc818fdc5e5b Reviewed-on: https://go-review.googlesource.com/c/build/+/347295 Trust: Dmitri Shuralyov <dmitshur@golang.org> Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com> Reviewed-by: Alexander Rakoczy <alex@golang.org>
2021-09-01 00:45:43 +03:00
case "cmd/go", "nolibgcc:crypto/x509", "reboot":
dashboard: speed up js/wasm trybots We don't need to run all the tests in a trybot. Run just enough to get good coverage, without going over 5 minutes. Any regressions elsewhere will be caught by the build.golang.org (slower) runs. Change-Id: I32f1fc17681bfb509844d2bd35b05c950806d283 Reviewed-on: https://go-review.googlesource.com/121938 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-06-28 23:30:35 +03:00
return false
}
}
dashboard: refactor mechanism for adjusting cmd/dist test policy Previously, the behavior of ShouldRunDistTest could be overridden by an individual builder by setting a the shouldRunDistTest field to a custom policy implementation. A pitfall of this approach is that it requires each custom policy to reimplement the default cmd/dist test policy as well. If that isn't done when attempting to make a small adjustment to the default policy, the result is that the default policy is ignored. The fasterTrybots policy, as the name suggests, exists to skip some tests in trybot mode. Due to the problem above, it was causing some builders to run more tests in trybot mode than they would if it weren't applied. For example, the freebsd-amd64-12_0 builder should have been skipping the "api" test in trybot mode, but wasn't. This refactor fixes that by modifying the approach for how individual builders can customize the cmd/dist test behavior. Now, a function distTestAdjust is called with the first parameter specifying the default policy decision. That function can adjust the decision as it sees fit, but otherwise it preserves its initial value in situations where no change is intended to be made. For golang/go#38279. Change-Id: I220248f51abfc8e77195ae0c9134a17f7cac9bc2 Reviewed-on: https://go-review.googlesource.com/c/build/+/227777 Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-04-09 19:47:03 +03:00
return run
dashboard: speed up js/wasm trybots We don't need to run all the tests in a trybot. Run just enough to get good coverage, without going over 5 minutes. Any regressions elsewhere will be caught by the build.golang.org (slower) runs. Change-Id: I32f1fc17681bfb509844d2bd35b05c950806d283 Reviewed-on: https://go-review.googlesource.com/121938 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-06-28 23:30:35 +03:00
},
dashboard: re-enable js/wasm test dir tests They were removed to make the build faster but they've let a few bugs slip by now. Re-enable. The 10 test shards adds 812 seconds of test time. Over 5 machines, that's 162.4 seconds each. Add another test shard to bring it down more. js-wasm trybots are currently at ~350 seconds, so we have some budget to get a bit slower and still be under 5 minutes. Updates golang/go#34768 Change-Id: Ic40c226d26e873227510e644756b0cc969151186 Reviewed-on: https://go-review.googlesource.com/c/build/+/200477 Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-10-10 22:09:40 +03:00
numTryTestHelpers: 5,
env/js-wasm, dashboard: add start of a js-wasm builder Updates golang/go#18892 Change-Id: I4b16afe8f0ce74f6ecd022f8bca14ada94ab308f Reviewed-on: https://go-review.googlesource.com/112736 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-05-11 08:54:32 +03:00
env: []string{
"GOOS=js", "GOARCH=wasm", "GOHOSTOS=linux", "GOHOSTARCH=amd64",
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/workdir/go/misc/wasm",
cmd/coordinator: follow module proxy redirects And because this uses new-in-Go1.13 http.Header.Clone, restrict all the files to Go 1.13+ and bump the Dockerfile Go version. Fixes golang/go#33928 Change-Id: I147cc4465e393d2c34e59c70edf65f3901a5e132 Reviewed-on: https://go-review.googlesource.com/c/build/+/192341 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-08-29 21:28:01 +03:00
"GO_DISABLE_OUTBOUND_NETWORK=1",
env/js-wasm, dashboard: add start of a js-wasm builder Updates golang/go#18892 Change-Id: I4b16afe8f0ce74f6ecd022f8bca14ada94ab308f Reviewed-on: https://go-review.googlesource.com/112736 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-05-11 08:54:32 +03:00
},
})
dashboard: add OpenBSD 6.8 builders This change builds on top of CL 268297, using the updated env script to build OpenBSD 6.8 images, and adding new builders that use them. This has been tested with cmd/debugnewvm on a recent Go tip commit, as well as Go 1.15.4 and 1.14.11 commits. It passed, although there were also some failures seen. Once the builder is added, we'll know the test results for a wider set of commits, and can decide how to proceed from there. Minimize the configuration to a bare minimum. Specifically, remove MinCPUPlatform since setting it "Intel Skylake" was found not to have a noticeable effect on the speed of the build compared to the zero value: # With MinCPUPlatform = "Intel Skylake": $ time debugnewvm -run-build=openbsd-amd64-68 -rev=go1.15.4 [...] real 17m8.308s # With a zero MinCPUPlatform value: $ time debugnewvm -run-build=openbsd-amd64-68 -rev=go1.15.4 [...] real 16m28.518s For golang/go#35712. Updates golang/go#42426. Change-Id: I517a85738a034a3a4074488ccecd4408a91a4a07 Reviewed-on: https://go-review.googlesource.com/c/build/+/269577 Trust: Dmitri Shuralyov <dmitshur@golang.org> Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
2020-11-12 20:27:32 +03:00
addBuilder(BuildConfig{
dashboard: shard openbsd-amd64-68 TryBot also The goal in CL 279512 was to reduce numTryTestHelpers from 5 to 4, but it unintentionally set the field only in the 386 builder, not the amd64 one. This caused the amd64 OpenBSD TryBot to get very slow (20-25 minutes), making it very much a bottleneck for overall TryBot completion time. Also stop explicitly setting some fields in openbsd-amd64-62 builder that have no effect by now, to simplify configuration. For golang/go#17104. Updates golang/go#35712. Change-Id: I22823e4848cab65f11bde2a1cc70527929d0792d Reviewed-on: https://go-review.googlesource.com/c/build/+/279723 Trust: Dmitri Shuralyov <dmitshur@golang.org> Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-12-23 21:33:19 +03:00
Name: "openbsd-amd64-68",
HostType: "host-openbsd-amd64-68",
tryBot: defaultTrySet(),
distTestAdjust: noTestDirAndNoReboot,
numTryTestHelpers: 4,
dashboard: add OpenBSD 6.8 builders This change builds on top of CL 268297, using the updated env script to build OpenBSD 6.8 images, and adding new builders that use them. This has been tested with cmd/debugnewvm on a recent Go tip commit, as well as Go 1.15.4 and 1.14.11 commits. It passed, although there were also some failures seen. Once the builder is added, we'll know the test results for a wider set of commits, and can decide how to proceed from there. Minimize the configuration to a bare minimum. Specifically, remove MinCPUPlatform since setting it "Intel Skylake" was found not to have a noticeable effect on the speed of the build compared to the zero value: # With MinCPUPlatform = "Intel Skylake": $ time debugnewvm -run-build=openbsd-amd64-68 -rev=go1.15.4 [...] real 17m8.308s # With a zero MinCPUPlatform value: $ time debugnewvm -run-build=openbsd-amd64-68 -rev=go1.15.4 [...] real 16m28.518s For golang/go#35712. Updates golang/go#42426. Change-Id: I517a85738a034a3a4074488ccecd4408a91a4a07 Reviewed-on: https://go-review.googlesource.com/c/build/+/269577 Trust: Dmitri Shuralyov <dmitshur@golang.org> Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
2020-11-12 20:27:32 +03:00
})
addBuilder(BuildConfig{
dashboard: skip x/review on slow 32-bit builders x/review is fairly disk-intensive (it shells out to 'git'), and also mostly for use by Go contributors (a smaller subset of Go users in general). It frequently times out on two particular 32-bit builders. I don't think we particularly need the coverage from those two builders for this repo, so rather than trying to speed up the test or tweak the scale factors, let's just skip it. Fixes golang/go#49529 Fixes golang/go#49530 Change-Id: I8da1216ebdf9287ae78d52bb03187b65eb33c9ee Reviewed-on: https://go-review.googlesource.com/c/build/+/363495 Trust: Bryan C. Mills <bcmills@google.com> Run-TryBot: Bryan C. Mills <bcmills@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-11-11 23:45:01 +03:00
Name: "openbsd-386-68",
HostType: "host-openbsd-386-68",
tryBot: explicitTrySet("sys"),
buildsRepo: func(repo, branch, goBranch string) bool {
if repo == "review" {
// https://golang.org/issue/49529: git seems to be too slow on this
// platform.
env,dashboard: add OpenBSD 7.0 all.bash takes a long time on these builders. I'll keep an eye on them to see if it causes trouble. For golang/go#48977. Change-Id: I0239a93c004157855968dab397e68abe38f5f439 Reviewed-on: https://go-review.googlesource.com/c/build/+/364094 Trust: Heschi Kreinick <heschi@google.com> Run-TryBot: Heschi Kreinick <heschi@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-11-15 23:02:19 +03:00
return false
}
return buildRepoByDefault(repo)
},
distTestAdjust: noTestDirAndNoReboot,
numTryTestHelpers: 4,
})
addBuilder(BuildConfig{
buildlet,dashboard: tweaks to Darwin, OpenBSD, and Linux builders - Add more Monterey builders and remove the known issue: it's stable. Also use it for race and nocgo. Update slowbot aliases to point to it. - Don't test 1.16 on OpenBSD 7.0, it segfaults. - Pin the Linux (ContainerOS) builders to an older version to avoid the pprof kernel bug. For golang/go#49065, golang/go#49149, and golang/go#48977. Change-Id: Ibec2fa735183ec65e5066c7c752ac356b7360550 Reviewed-on: https://go-review.googlesource.com/c/build/+/365777 Trust: Heschi Kreinick <heschi@google.com> Run-TryBot: Heschi Kreinick <heschi@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org>
2021-11-19 22:46:52 +03:00
Name: "openbsd-amd64-70",
HostType: "host-openbsd-amd64-70",
tryBot: defaultTrySet(),
distTestAdjust: noTestDirAndNoReboot,
buildsRepo: func(repo, branch, goBranch string) bool {
// https://github.com/golang/go/issues/48977#issuecomment-971763553:
// 1.16 seems to be incompatible with 7.0.
return atLeastGo1(goBranch, 17) && buildRepoByDefault(repo)
},
env,dashboard: add OpenBSD 7.0 all.bash takes a long time on these builders. I'll keep an eye on them to see if it causes trouble. For golang/go#48977. Change-Id: I0239a93c004157855968dab397e68abe38f5f439 Reviewed-on: https://go-review.googlesource.com/c/build/+/364094 Trust: Heschi Kreinick <heschi@google.com> Run-TryBot: Heschi Kreinick <heschi@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-11-15 23:02:19 +03:00
numTryTestHelpers: 4,
})
dashboard: add n1 hosts for OpenBSD and NetBSD We recently migrated in CL 354757 from n1 to e2 hosts around the same time issues started appearing with these builders. n1 hosts are notably different in that they are older generation CPUs with hyperthreading disabled. For golang/go#49209 Change-Id: I8b6822fbd4910602f61d504b34580282c97c0ca7 Reviewed-on: https://go-review.googlesource.com/c/build/+/365776 Trust: Alexander Rakoczy <alex@golang.org> Run-TryBot: Alexander Rakoczy <alex@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2021-11-19 22:22:11 +03:00
addBuilder(BuildConfig{
buildlet,dashboard: tweaks to Darwin, OpenBSD, and Linux builders - Add more Monterey builders and remove the known issue: it's stable. Also use it for race and nocgo. Update slowbot aliases to point to it. - Don't test 1.16 on OpenBSD 7.0, it segfaults. - Pin the Linux (ContainerOS) builders to an older version to avoid the pprof kernel bug. For golang/go#49065, golang/go#49149, and golang/go#48977. Change-Id: Ibec2fa735183ec65e5066c7c752ac356b7360550 Reviewed-on: https://go-review.googlesource.com/c/build/+/365777 Trust: Heschi Kreinick <heschi@google.com> Run-TryBot: Heschi Kreinick <heschi@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org>
2021-11-19 22:46:52 +03:00
Name: "openbsd-amd64-70-n1",
HostType: "host-openbsd-amd64-70-n1",
tryBot: defaultTrySet(),
distTestAdjust: noTestDirAndNoReboot,
buildsRepo: func(repo, branch, goBranch string) bool {
// https://github.com/golang/go/issues/48977#issuecomment-971763553:
// 1.16 seems to be incompatible with 7.0.
return atLeastGo1(goBranch, 17) && buildRepoByDefault(repo)
},
dashboard: add n1 hosts for OpenBSD and NetBSD We recently migrated in CL 354757 from n1 to e2 hosts around the same time issues started appearing with these builders. n1 hosts are notably different in that they are older generation CPUs with hyperthreading disabled. For golang/go#49209 Change-Id: I8b6822fbd4910602f61d504b34580282c97c0ca7 Reviewed-on: https://go-review.googlesource.com/c/build/+/365776 Trust: Alexander Rakoczy <alex@golang.org> Run-TryBot: Alexander Rakoczy <alex@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2021-11-19 22:22:11 +03:00
numTryTestHelpers: 4,
})
env,dashboard: add OpenBSD 7.0 all.bash takes a long time on these builders. I'll keep an eye on them to see if it causes trouble. For golang/go#48977. Change-Id: I0239a93c004157855968dab397e68abe38f5f439 Reviewed-on: https://go-review.googlesource.com/c/build/+/364094 Trust: Heschi Kreinick <heschi@google.com> Run-TryBot: Heschi Kreinick <heschi@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-11-15 23:02:19 +03:00
addBuilder(BuildConfig{
Name: "openbsd-386-70",
HostType: "host-openbsd-386-70",
tryBot: explicitTrySet("sys"),
buildsRepo: func(repo, branch, goBranch string) bool {
if repo == "review" {
// https://golang.org/issue/49529: git seems to be too slow on this
// platform.
dashboard: skip x/review on slow 32-bit builders x/review is fairly disk-intensive (it shells out to 'git'), and also mostly for use by Go contributors (a smaller subset of Go users in general). It frequently times out on two particular 32-bit builders. I don't think we particularly need the coverage from those two builders for this repo, so rather than trying to speed up the test or tweak the scale factors, let's just skip it. Fixes golang/go#49529 Fixes golang/go#49530 Change-Id: I8da1216ebdf9287ae78d52bb03187b65eb33c9ee Reviewed-on: https://go-review.googlesource.com/c/build/+/363495 Trust: Bryan C. Mills <bcmills@google.com> Run-TryBot: Bryan C. Mills <bcmills@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-11-11 23:45:01 +03:00
return false
}
buildlet,dashboard: tweaks to Darwin, OpenBSD, and Linux builders - Add more Monterey builders and remove the known issue: it's stable. Also use it for race and nocgo. Update slowbot aliases to point to it. - Don't test 1.16 on OpenBSD 7.0, it segfaults. - Pin the Linux (ContainerOS) builders to an older version to avoid the pprof kernel bug. For golang/go#49065, golang/go#49149, and golang/go#48977. Change-Id: Ibec2fa735183ec65e5066c7c752ac356b7360550 Reviewed-on: https://go-review.googlesource.com/c/build/+/365777 Trust: Heschi Kreinick <heschi@google.com> Run-TryBot: Heschi Kreinick <heschi@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org>
2021-11-19 22:46:52 +03:00
// https://github.com/golang/go/issues/48977#issuecomment-971763553:
// 1.16 seems to be incompatible with 7.0.
return atLeastGo1(goBranch, 17) && buildRepoByDefault(repo)
dashboard: skip x/review on slow 32-bit builders x/review is fairly disk-intensive (it shells out to 'git'), and also mostly for use by Go contributors (a smaller subset of Go users in general). It frequently times out on two particular 32-bit builders. I don't think we particularly need the coverage from those two builders for this repo, so rather than trying to speed up the test or tweak the scale factors, let's just skip it. Fixes golang/go#49529 Fixes golang/go#49530 Change-Id: I8da1216ebdf9287ae78d52bb03187b65eb33c9ee Reviewed-on: https://go-review.googlesource.com/c/build/+/363495 Trust: Bryan C. Mills <bcmills@google.com> Run-TryBot: Bryan C. Mills <bcmills@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-11-11 23:45:01 +03:00
},
dashboard: promote OpenBSD 6.8 builders to be default The known issue with OpenBSD 6.8 builders appears to be resolved via CL 278732 and CL 279134. Promote them to the primary OpenBSD post-submit builders and TryBots. Having test coverage from OpenBSD 6.8 and 6.4 builders gives us us more confidence that Go works on supported OpenBSD versions (which are 6.8 and 6.7 at this time, per past policy decision in https://golang.org/issue/15227#issuecomment-293319876). Reduce numTryTestHelpers from 5 to 4 based on some data from golang.org/issue/37439 showing that going from 3 to 5 helpers doesn't make a significant difference. We can adjust further if we learn that OpenBSD TryBots become the bottleneck. Fixes golang/go#35712. For golang/go#42542. For golang/go#42064. Updates golang/go#42426. Change-Id: Id2fa4be7c3161f89752c1428146846fe06ca63db Reviewed-on: https://go-review.googlesource.com/c/build/+/279512 Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org> Trust: Dmitri Shuralyov <dmitshur@golang.org>
2020-12-21 22:12:25 +03:00
distTestAdjust: noTestDirAndNoReboot,
numTryTestHelpers: 4,
dashboard: add OpenBSD 6.8 builders This change builds on top of CL 268297, using the updated env script to build OpenBSD 6.8 images, and adding new builders that use them. This has been tested with cmd/debugnewvm on a recent Go tip commit, as well as Go 1.15.4 and 1.14.11 commits. It passed, although there were also some failures seen. Once the builder is added, we'll know the test results for a wider set of commits, and can decide how to proceed from there. Minimize the configuration to a bare minimum. Specifically, remove MinCPUPlatform since setting it "Intel Skylake" was found not to have a noticeable effect on the speed of the build compared to the zero value: # With MinCPUPlatform = "Intel Skylake": $ time debugnewvm -run-build=openbsd-amd64-68 -rev=go1.15.4 [...] real 17m8.308s # With a zero MinCPUPlatform value: $ time debugnewvm -run-build=openbsd-amd64-68 -rev=go1.15.4 [...] real 16m28.518s For golang/go#35712. Updates golang/go#42426. Change-Id: I517a85738a034a3a4074488ccecd4408a91a4a07 Reviewed-on: https://go-review.googlesource.com/c/build/+/269577 Trust: Dmitri Shuralyov <dmitshur@golang.org> Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
2020-11-12 20:27:32 +03:00
})
dashboard: add n1 hosts for OpenBSD and NetBSD We recently migrated in CL 354757 from n1 to e2 hosts around the same time issues started appearing with these builders. n1 hosts are notably different in that they are older generation CPUs with hyperthreading disabled. For golang/go#49209 Change-Id: I8b6822fbd4910602f61d504b34580282c97c0ca7 Reviewed-on: https://go-review.googlesource.com/c/build/+/365776 Trust: Alexander Rakoczy <alex@golang.org> Run-TryBot: Alexander Rakoczy <alex@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2021-11-19 22:22:11 +03:00
addBuilder(BuildConfig{
Name: "openbsd-386-70-n1",
HostType: "host-openbsd-386-70-n1",
tryBot: explicitTrySet("sys"),
buildsRepo: func(repo, branch, goBranch string) bool {
if repo == "review" {
// https://golang.org/issue/49529: git seems to be too slow on this
// platform.
return false
}
buildlet,dashboard: tweaks to Darwin, OpenBSD, and Linux builders - Add more Monterey builders and remove the known issue: it's stable. Also use it for race and nocgo. Update slowbot aliases to point to it. - Don't test 1.16 on OpenBSD 7.0, it segfaults. - Pin the Linux (ContainerOS) builders to an older version to avoid the pprof kernel bug. For golang/go#49065, golang/go#49149, and golang/go#48977. Change-Id: Ibec2fa735183ec65e5066c7c752ac356b7360550 Reviewed-on: https://go-review.googlesource.com/c/build/+/365777 Trust: Heschi Kreinick <heschi@google.com> Run-TryBot: Heschi Kreinick <heschi@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org>
2021-11-19 22:46:52 +03:00
// https://github.com/golang/go/issues/48977#issuecomment-971763553:
// 1.16 seems to be incompatible with 7.0.
return atLeastGo1(goBranch, 17) && buildRepoByDefault(repo)
dashboard: add n1 hosts for OpenBSD and NetBSD We recently migrated in CL 354757 from n1 to e2 hosts around the same time issues started appearing with these builders. n1 hosts are notably different in that they are older generation CPUs with hyperthreading disabled. For golang/go#49209 Change-Id: I8b6822fbd4910602f61d504b34580282c97c0ca7 Reviewed-on: https://go-review.googlesource.com/c/build/+/365776 Trust: Alexander Rakoczy <alex@golang.org> Run-TryBot: Alexander Rakoczy <alex@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2021-11-19 22:22:11 +03:00
},
distTestAdjust: noTestDirAndNoReboot,
numTryTestHelpers: 4,
})
dashboard: add openbsd-arm-jsing builder This will allow the openbsd-arm builder to be migrated to a buildlet. Change-Id: If38788daf2d1e7f237ebdcce384a098e08bffb0b Reviewed-on: https://go-review.googlesource.com/c/build/+/192598 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-08-30 20:09:05 +03:00
addBuilder(BuildConfig{
dashboard: reduce the amount of work for openbsd/arm Given the performance of this hardware, reduce the amount of work that it is trying to do. Updates golang/go#25498 Change-Id: I943ab2b01bd10c5217c097464d3904b0f4de2d0a Reviewed-on: https://go-review.googlesource.com/c/build/+/228597 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2020-04-16 18:47:53 +03:00
Name: "openbsd-arm-jsing",
HostType: "host-openbsd-arm-joelsing",
SkipSnapshot: true,
buildsRepo: func(repo, branch, goBranch string) bool {
switch repo {
dashboard: add openbsd-arm64-jsing builder (reverse) Update golang/go#31656 Fixes golang/go#34035 Change-Id: I7e3d20be148ad5111efeb78702352da14274938e Reviewed-on: https://go-review.googlesource.com/c/build/+/250117 Trust: Joel Sing <joel@sing.id.au> Run-TryBot: Joel Sing <joel@sing.id.au> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2020-08-20 21:42:01 +03:00
case "go", "net", "sys":
return branch == "master" && goBranch == "master"
default:
return false
}
},
distTestAdjust: noTestDirAndNoReboot,
tryBot: nil,
env: []string{
// The machine is slow.
"GO_TEST_TIMEOUT_SCALE=5",
},
})
addBuilder(BuildConfig{
Name: "openbsd-arm64-jsing",
HostType: "host-openbsd-arm64-joelsing",
SkipSnapshot: true,
buildsRepo: func(repo, branch, goBranch string) bool {
switch repo {
dashboard: add openbsd-mips64-jsing builder (reverse) Update golang/go#40995 Change-Id: Ie733438ab1823d65109e684a6a5e3098cdfdb252 Reviewed-on: https://go-review.googlesource.com/c/build/+/250118 Trust: Joel Sing <joel@sing.id.au> Run-TryBot: Joel Sing <joel@sing.id.au> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2020-08-23 19:02:40 +03:00
case "go", "net", "sys":
return branch == "master" && goBranch == "master"
default:
return false
}
},
distTestAdjust: noTestDirAndNoReboot,
tryBot: nil,
env: []string{
// The machine is slow.
"GO_TEST_TIMEOUT_SCALE=5",
},
})
addBuilder(BuildConfig{
Name: "openbsd-mips64-jsing",
HostType: "host-openbsd-mips64-joelsing",
SkipSnapshot: true,
buildsRepo: func(repo, branch, goBranch string) bool {
switch repo {
dashboard: reduce the amount of work for openbsd/arm Given the performance of this hardware, reduce the amount of work that it is trying to do. Updates golang/go#25498 Change-Id: I943ab2b01bd10c5217c097464d3904b0f4de2d0a Reviewed-on: https://go-review.googlesource.com/c/build/+/228597 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2020-04-16 18:47:53 +03:00
case "go", "net", "sys":
return branch == "master" && goBranch == "master"
default:
return false
}
},
dashboard: refactor mechanism for adjusting cmd/dist test policy Previously, the behavior of ShouldRunDistTest could be overridden by an individual builder by setting a the shouldRunDistTest field to a custom policy implementation. A pitfall of this approach is that it requires each custom policy to reimplement the default cmd/dist test policy as well. If that isn't done when attempting to make a small adjustment to the default policy, the result is that the default policy is ignored. The fasterTrybots policy, as the name suggests, exists to skip some tests in trybot mode. Due to the problem above, it was causing some builders to run more tests in trybot mode than they would if it weren't applied. For example, the freebsd-amd64-12_0 builder should have been skipping the "api" test in trybot mode, but wasn't. This refactor fixes that by modifying the approach for how individual builders can customize the cmd/dist test behavior. Now, a function distTestAdjust is called with the first parameter specifying the default policy decision. That function can adjust the decision as it sees fit, but otherwise it preserves its initial value in situations where no change is intended to be made. For golang/go#38279. Change-Id: I220248f51abfc8e77195ae0c9134a17f7cac9bc2 Reviewed-on: https://go-review.googlesource.com/c/build/+/227777 Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-04-09 19:47:03 +03:00
distTestAdjust: noTestDirAndNoReboot,
tryBot: nil,
dashboard: add openbsd-arm-jsing builder This will allow the openbsd-arm builder to be migrated to a buildlet. Change-Id: If38788daf2d1e7f237ebdcce384a098e08bffb0b Reviewed-on: https://go-review.googlesource.com/c/build/+/192598 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-08-30 20:09:05 +03:00
env: []string{
// The machine is slow.
"GO_TEST_TIMEOUT_SCALE=5",
},
})
dashboard, env/netbsd-{amd64,386}: add NetBSD 9.0 builders Update NetBSD amd64 and 386 builders to the latest NetBSD 9.0 release, using the latest versions of anita (2.3) and pkgsrc packages (2019Q4). Also increase the image disk size from 4 GB to 16 GB. The motivation is to help tests that require more disk space and large dependencies. Additional adjustments to the image building script will need to be made to make use of increased disk space for the /tmp volume, as it is currently being mounted in RAM with size equal to 25% of memory. Remove the manual invocation of the mozilla-rootcerts install command. It no longer works as is in NetBSD 9.0. The mozilla-rootcerts-openssl package is added instead, which successfully performs that same task. Hopefully this will be a more robust approach for the future. Continue to use an explicit URL for the NetBSD release download, and remove unused find_latest_release. The plan is to roll out these builders first (without trybots), gain more confidence, enable trybots, and then remove NetBSD 8.0 builders. For golang/go#37244. For golang/go#30839. For golang/go#31858. For golang/go#31726. For golang/go#38283. Change-Id: Id41278eaebe64cfbcf7b1ab9bfd64e550ee0bbf1 Reviewed-on: https://go-review.googlesource.com/c/build/+/226977 Reviewed-by: Benny Siegert <bsiegert@gmail.com> Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-04-02 15:55:30 +03:00
addBuilder(BuildConfig{
dashboard: refactor mechanism for adjusting cmd/dist test policy Previously, the behavior of ShouldRunDistTest could be overridden by an individual builder by setting a the shouldRunDistTest field to a custom policy implementation. A pitfall of this approach is that it requires each custom policy to reimplement the default cmd/dist test policy as well. If that isn't done when attempting to make a small adjustment to the default policy, the result is that the default policy is ignored. The fasterTrybots policy, as the name suggests, exists to skip some tests in trybot mode. Due to the problem above, it was causing some builders to run more tests in trybot mode than they would if it weren't applied. For example, the freebsd-amd64-12_0 builder should have been skipping the "api" test in trybot mode, but wasn't. This refactor fixes that by modifying the approach for how individual builders can customize the cmd/dist test behavior. Now, a function distTestAdjust is called with the first parameter specifying the default policy decision. That function can adjust the decision as it sees fit, but otherwise it preserves its initial value in situations where no change is intended to be made. For golang/go#38279. Change-Id: I220248f51abfc8e77195ae0c9134a17f7cac9bc2 Reviewed-on: https://go-review.googlesource.com/c/build/+/227777 Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-04-09 19:47:03 +03:00
Name: "netbsd-amd64-9_0",
HostType: "host-netbsd-amd64-9_0",
distTestAdjust: noTestDirAndNoReboot,
tryBot: explicitTrySet("sys"),
dashboard, env/netbsd-{amd64,386}: add NetBSD 9.0 builders Update NetBSD amd64 and 386 builders to the latest NetBSD 9.0 release, using the latest versions of anita (2.3) and pkgsrc packages (2019Q4). Also increase the image disk size from 4 GB to 16 GB. The motivation is to help tests that require more disk space and large dependencies. Additional adjustments to the image building script will need to be made to make use of increased disk space for the /tmp volume, as it is currently being mounted in RAM with size equal to 25% of memory. Remove the manual invocation of the mozilla-rootcerts install command. It no longer works as is in NetBSD 9.0. The mozilla-rootcerts-openssl package is added instead, which successfully performs that same task. Hopefully this will be a more robust approach for the future. Continue to use an explicit URL for the NetBSD release download, and remove unused find_latest_release. The plan is to roll out these builders first (without trybots), gain more confidence, enable trybots, and then remove NetBSD 8.0 builders. For golang/go#37244. For golang/go#30839. For golang/go#31858. For golang/go#31726. For golang/go#38283. Change-Id: Id41278eaebe64cfbcf7b1ab9bfd64e550ee0bbf1 Reviewed-on: https://go-review.googlesource.com/c/build/+/226977 Reviewed-by: Benny Siegert <bsiegert@gmail.com> Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-04-02 15:55:30 +03:00
})
dashboard: add n1 hosts for OpenBSD and NetBSD We recently migrated in CL 354757 from n1 to e2 hosts around the same time issues started appearing with these builders. n1 hosts are notably different in that they are older generation CPUs with hyperthreading disabled. For golang/go#49209 Change-Id: I8b6822fbd4910602f61d504b34580282c97c0ca7 Reviewed-on: https://go-review.googlesource.com/c/build/+/365776 Trust: Alexander Rakoczy <alex@golang.org> Run-TryBot: Alexander Rakoczy <alex@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2021-11-19 22:22:11 +03:00
addBuilder(BuildConfig{
Name: "netbsd-amd64-9_0-n1",
HostType: "host-netbsd-amd64-9_0-n1",
distTestAdjust: noTestDirAndNoReboot,
tryBot: explicitTrySet("sys"),
})
dashboard, env/netbsd-{amd64,386}: add NetBSD 9.0 builders Update NetBSD amd64 and 386 builders to the latest NetBSD 9.0 release, using the latest versions of anita (2.3) and pkgsrc packages (2019Q4). Also increase the image disk size from 4 GB to 16 GB. The motivation is to help tests that require more disk space and large dependencies. Additional adjustments to the image building script will need to be made to make use of increased disk space for the /tmp volume, as it is currently being mounted in RAM with size equal to 25% of memory. Remove the manual invocation of the mozilla-rootcerts install command. It no longer works as is in NetBSD 9.0. The mozilla-rootcerts-openssl package is added instead, which successfully performs that same task. Hopefully this will be a more robust approach for the future. Continue to use an explicit URL for the NetBSD release download, and remove unused find_latest_release. The plan is to roll out these builders first (without trybots), gain more confidence, enable trybots, and then remove NetBSD 8.0 builders. For golang/go#37244. For golang/go#30839. For golang/go#31858. For golang/go#31726. For golang/go#38283. Change-Id: Id41278eaebe64cfbcf7b1ab9bfd64e550ee0bbf1 Reviewed-on: https://go-review.googlesource.com/c/build/+/226977 Reviewed-by: Benny Siegert <bsiegert@gmail.com> Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-04-02 15:55:30 +03:00
addBuilder(BuildConfig{
dashboard: refactor mechanism for adjusting cmd/dist test policy Previously, the behavior of ShouldRunDistTest could be overridden by an individual builder by setting a the shouldRunDistTest field to a custom policy implementation. A pitfall of this approach is that it requires each custom policy to reimplement the default cmd/dist test policy as well. If that isn't done when attempting to make a small adjustment to the default policy, the result is that the default policy is ignored. The fasterTrybots policy, as the name suggests, exists to skip some tests in trybot mode. Due to the problem above, it was causing some builders to run more tests in trybot mode than they would if it weren't applied. For example, the freebsd-amd64-12_0 builder should have been skipping the "api" test in trybot mode, but wasn't. This refactor fixes that by modifying the approach for how individual builders can customize the cmd/dist test behavior. Now, a function distTestAdjust is called with the first parameter specifying the default policy decision. That function can adjust the decision as it sees fit, but otherwise it preserves its initial value in situations where no change is intended to be made. For golang/go#38279. Change-Id: I220248f51abfc8e77195ae0c9134a17f7cac9bc2 Reviewed-on: https://go-review.googlesource.com/c/build/+/227777 Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-04-09 19:47:03 +03:00
Name: "netbsd-386-9_0",
HostType: "host-netbsd-386-9_0",
distTestAdjust: noTestDirAndNoReboot,
dashboard, env/netbsd-{amd64,386}: add NetBSD 9.0 builders Update NetBSD amd64 and 386 builders to the latest NetBSD 9.0 release, using the latest versions of anita (2.3) and pkgsrc packages (2019Q4). Also increase the image disk size from 4 GB to 16 GB. The motivation is to help tests that require more disk space and large dependencies. Additional adjustments to the image building script will need to be made to make use of increased disk space for the /tmp volume, as it is currently being mounted in RAM with size equal to 25% of memory. Remove the manual invocation of the mozilla-rootcerts install command. It no longer works as is in NetBSD 9.0. The mozilla-rootcerts-openssl package is added instead, which successfully performs that same task. Hopefully this will be a more robust approach for the future. Continue to use an explicit URL for the NetBSD release download, and remove unused find_latest_release. The plan is to roll out these builders first (without trybots), gain more confidence, enable trybots, and then remove NetBSD 8.0 builders. For golang/go#37244. For golang/go#30839. For golang/go#31858. For golang/go#31726. For golang/go#38283. Change-Id: Id41278eaebe64cfbcf7b1ab9bfd64e550ee0bbf1 Reviewed-on: https://go-review.googlesource.com/c/build/+/226977 Reviewed-by: Benny Siegert <bsiegert@gmail.com> Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-04-02 15:55:30 +03:00
})
dashboard: add n1 hosts for OpenBSD and NetBSD We recently migrated in CL 354757 from n1 to e2 hosts around the same time issues started appearing with these builders. n1 hosts are notably different in that they are older generation CPUs with hyperthreading disabled. For golang/go#49209 Change-Id: I8b6822fbd4910602f61d504b34580282c97c0ca7 Reviewed-on: https://go-review.googlesource.com/c/build/+/365776 Trust: Alexander Rakoczy <alex@golang.org> Run-TryBot: Alexander Rakoczy <alex@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2021-11-19 22:22:11 +03:00
addBuilder(BuildConfig{
Name: "netbsd-386-9_0-n1",
HostType: "host-netbsd-386-9_0-n1",
distTestAdjust: noTestDirAndNoReboot,
})
dashboard: add netbsd-arm-bsiegert This seems to be the last builder still using the old builder binary and not buildlet. It needs an entry in here to switch to buildlet and stage0. Change-Id: If2fdd2298ab88d56b6850061050caf27ca7c8983 Reviewed-on: https://go-review.googlesource.com/c/build/+/168061 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-03-19 16:28:04 +03:00
addBuilder(BuildConfig{
dashboard: skip x/review on slow 32-bit builders x/review is fairly disk-intensive (it shells out to 'git'), and also mostly for use by Go contributors (a smaller subset of Go users in general). It frequently times out on two particular 32-bit builders. I don't think we particularly need the coverage from those two builders for this repo, so rather than trying to speed up the test or tweak the scale factors, let's just skip it. Fixes golang/go#49529 Fixes golang/go#49530 Change-Id: I8da1216ebdf9287ae78d52bb03187b65eb33c9ee Reviewed-on: https://go-review.googlesource.com/c/build/+/363495 Trust: Bryan C. Mills <bcmills@google.com> Run-TryBot: Bryan C. Mills <bcmills@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-11-11 23:45:01 +03:00
Name: "netbsd-arm-bsiegert",
HostType: "host-netbsd-arm-bsiegert",
buildsRepo: func(repo, branch, goBranch string) bool {
if repo == "review" {
// https://golang.org/issue/49530: This test seems to be too slow even
// with a long scale factor.
return false
}
return buildRepoByDefault(repo)
},
dashboard: refactor mechanism for adjusting cmd/dist test policy Previously, the behavior of ShouldRunDistTest could be overridden by an individual builder by setting a the shouldRunDistTest field to a custom policy implementation. A pitfall of this approach is that it requires each custom policy to reimplement the default cmd/dist test policy as well. If that isn't done when attempting to make a small adjustment to the default policy, the result is that the default policy is ignored. The fasterTrybots policy, as the name suggests, exists to skip some tests in trybot mode. Due to the problem above, it was causing some builders to run more tests in trybot mode than they would if it weren't applied. For example, the freebsd-amd64-12_0 builder should have been skipping the "api" test in trybot mode, but wasn't. This refactor fixes that by modifying the approach for how individual builders can customize the cmd/dist test behavior. Now, a function distTestAdjust is called with the first parameter specifying the default policy decision. That function can adjust the decision as it sees fit, but otherwise it preserves its initial value in situations where no change is intended to be made. For golang/go#38279. Change-Id: I220248f51abfc8e77195ae0c9134a17f7cac9bc2 Reviewed-on: https://go-review.googlesource.com/c/build/+/227777 Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-04-09 19:47:03 +03:00
distTestAdjust: noTestDirAndNoReboot,
tryBot: nil,
dashboard: add netbsd-arm-bsiegert This seems to be the last builder still using the old builder binary and not buildlet. It needs an entry in here to switch to buildlet and stage0. Change-Id: If2fdd2298ab88d56b6850061050caf27ca7c8983 Reviewed-on: https://go-review.googlesource.com/c/build/+/168061 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-03-19 16:28:04 +03:00
env: []string{
// The machine is slow.
"GO_TEST_TIMEOUT_SCALE=10",
},
})
dashboard: add netbsd-arm64-bsiegert builder (reverse) Update golang/go#34036 Change-Id: Ic92093e764fa163d97a2f125a4534581adf217da Reviewed-on: https://go-review.googlesource.com/c/build/+/249137 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2020-08-18 21:42:30 +03:00
addBuilder(BuildConfig{
dashboard: only build tip on netbsd-arm64-bsiegert builder The build fails on 1.15 and earlier. Updates golang/go#30824 Change-Id: I1f62d37cc05dde988cbb7e5124bd97916b38bcb3 Reviewed-on: https://go-review.googlesource.com/c/build/+/257938 Trust: Tobias Klauser <tobias.klauser@gmail.com> Trust: Benny Siegert <bsiegert@gmail.com> Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Benny Siegert <bsiegert@gmail.com>
2020-09-28 11:28:32 +03:00
Name: "netbsd-arm64-bsiegert",
HostType: "host-netbsd-arm64-bsiegert",
buildsRepo: func(repo, branch, goBranch string) bool {
return atLeastGo1(goBranch, 16) && buildRepoByDefault(repo)
},
dashboard: add netbsd-arm64-bsiegert builder (reverse) Update golang/go#34036 Change-Id: Ic92093e764fa163d97a2f125a4534581adf217da Reviewed-on: https://go-review.googlesource.com/c/build/+/249137 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2020-08-18 21:42:30 +03:00
distTestAdjust: noTestDirAndNoReboot,
tryBot: nil,
env: []string{
// The machine is slow.
"GO_TEST_TIMEOUT_SCALE=10",
},
})
dashboard: move builder configuration out of coordinator into a package The new package will be used by the buildlet command line tool in the future. Also use the metadata package in the coordinator for the master build key, since it was already depending on that package. Might as well use it. Change-Id: I6ad7593787ebba364090951ccbaecb4aa38e1dd6 Reviewed-on: https://go-review.googlesource.com/2920 Reviewed-by: Andrew Gerrand <adg@golang.org>
2015-01-15 23:46:22 +03:00
addBuilder(BuildConfig{
all: split builder config into builder & host configs Our builders are named of the form "GOOS-GOARCH" or "GOOS-GOARCH-suffix". Over time we've grown many builders. This CL doesn't change that. Builders continue to be named and operate as before. Previously the build configuration file (dashboard/builders.go) made each builder type ("linux-amd64-race", etc) define how to create a host running a buildlet of that type, even though many builders had identical host configs. For example, these builders all share the same host type (a Kubernetes container): linux-amd64 linux-amd64-race linux-386 linux-386-387 And these are the same host type (a GCE VM): windows-amd64-gce windows-amd64-race windows-386-gce This CL creates a new concept of a "hostType" which defines how the buildlet is created (Kube, GCE, Reverse, and how), and then each builder itself references a host type. Users never see the hostType. (except perhaps in gomote list output) But they at least never need to care about them. Reverse buildlets now can only be one hostType at a time, which simplifies things. We were no longer using multiple roles per machine once moving to VMs for OS X. gomote continues to operate as it did previously but its underlying protocol changed and clients will need to be updated. As a new feature, gomote now has a new flag to let you reuse a buildlet host connection for different builder rules if they share the same underlying host type. But users can ignore that. This CL is a long-standing TODO (previously attempted and aborted) and will make many things easier and faster, including the linux-arm cross-compilation effort, and keeping pre-warmed buildlets of VM types ready to go. Updates golang/go#17104 Change-Id: Iad8387f48680424a8441e878a2f4762bf79ea4d2 Reviewed-on: https://go-review.googlesource.com/29551 Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2016-09-22 00:27:37 +03:00
Name: "plan9-386",
HostType: "host-plan9-386-gce",
dashboard: only test plan9 at master The plan9 port doesn't need to be tested against older Go releases. Its userbase runs at master. Change-Id: I710d98560b234c7555c7be526aa2efdb6473c439 Reviewed-on: https://go-review.googlesource.com/c/build/+/170519 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-04-03 08:03:37 +03:00
numTestHelpers: 1,
dashboard: disable plan9-386 builder It hasn't passed in months and now spins, wasting resources. Updates golang/go#31261 Updates golang/go#29801 Change-Id: Idcf13ae915bad4febb156c5c5d49f07f76cf9d49 Reviewed-on: https://go-review.googlesource.com/c/build/+/172797 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-04-18 22:33:19 +03:00
tryOnly: true, // disable it for now; Issue 31261, Issue 29801
cmd/coordinator, dashboard: don't skip cmd/dist tests for SlowBots When a user explicitly requests a SlowBot builder via the TRY= syntax, it is a signal that they're willing to wait longer for test results. It's more useful to not skip any tests and give the user confidence that the SlowBot run will catch all issues that would otherwise be caught only by post-submit builders after the CL is submitted. This is done by considering a TryBot that isn't an explicit SlowBot to be a "normal TryBot", and the ShouldRunDistTest policy function is provided a parameter specifying whether it's a normal TryBot (non-SlowBot) run. Tests are only skipped during normal TryBot runs, not SlowBot runs. Fixes golang/go#38279. For golang/go#34501. Change-Id: Ia11fd51c71f9de1089b6b6f4c027014893f224b9 Reviewed-on: https://go-review.googlesource.com/c/build/+/227779 Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-04-09 23:24:30 +03:00
distTestAdjust: func(run bool, distTest string, isNormalTry bool) bool {
dashboard: refactor mechanism for adjusting cmd/dist test policy Previously, the behavior of ShouldRunDistTest could be overridden by an individual builder by setting a the shouldRunDistTest field to a custom policy implementation. A pitfall of this approach is that it requires each custom policy to reimplement the default cmd/dist test policy as well. If that isn't done when attempting to make a small adjustment to the default policy, the result is that the default policy is ignored. The fasterTrybots policy, as the name suggests, exists to skip some tests in trybot mode. Due to the problem above, it was causing some builders to run more tests in trybot mode than they would if it weren't applied. For example, the freebsd-amd64-12_0 builder should have been skipping the "api" test in trybot mode, but wasn't. This refactor fixes that by modifying the approach for how individual builders can customize the cmd/dist test behavior. Now, a function distTestAdjust is called with the first parameter specifying the default policy decision. That function can adjust the decision as it sees fit, but otherwise it preserves its initial value in situations where no change is intended to be made. For golang/go#38279. Change-Id: I220248f51abfc8e77195ae0c9134a17f7cac9bc2 Reviewed-on: https://go-review.googlesource.com/c/build/+/227777 Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-04-09 19:47:03 +03:00
switch distTest {
env/plan9: revert SMP changes, stop running cmd/go tests Revert "env: enable SMP in Plan 9 image" This reverts commit 6478d3e7bdff4c8d810ba44f970ca4b0bfc92405. Revert "dashboard: bump plan9 GCE image" This reverts commit ce208996bf954a87d68789bb6a9000256f016c77. Updates golang/go#29822 Change-Id: I47e75d135e092ee9d5a80c66e91c1048c67cd5f6 Reviewed-on: https://go-review.googlesource.com/c/158542 Reviewed-by: Bryan C. Mills <bcmills@google.com> Reviewed-by: David du Colombier <0intro@gmail.com>
2019-01-19 01:19:58 +03:00
case "api",
"go_test:cmd/go": // takes over 20 minutes without working SMP
return false
}
dashboard: refactor mechanism for adjusting cmd/dist test policy Previously, the behavior of ShouldRunDistTest could be overridden by an individual builder by setting a the shouldRunDistTest field to a custom policy implementation. A pitfall of this approach is that it requires each custom policy to reimplement the default cmd/dist test policy as well. If that isn't done when attempting to make a small adjustment to the default policy, the result is that the default policy is ignored. The fasterTrybots policy, as the name suggests, exists to skip some tests in trybot mode. Due to the problem above, it was causing some builders to run more tests in trybot mode than they would if it weren't applied. For example, the freebsd-amd64-12_0 builder should have been skipping the "api" test in trybot mode, but wasn't. This refactor fixes that by modifying the approach for how individual builders can customize the cmd/dist test behavior. Now, a function distTestAdjust is called with the first parameter specifying the default policy decision. That function can adjust the decision as it sees fit, but otherwise it preserves its initial value in situations where no change is intended to be made. For golang/go#38279. Change-Id: I220248f51abfc8e77195ae0c9134a17f7cac9bc2 Reviewed-on: https://go-review.googlesource.com/c/build/+/227777 Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-04-09 19:47:03 +03:00
return run
env/plan9: revert SMP changes, stop running cmd/go tests Revert "env: enable SMP in Plan 9 image" This reverts commit 6478d3e7bdff4c8d810ba44f970ca4b0bfc92405. Revert "dashboard: bump plan9 GCE image" This reverts commit ce208996bf954a87d68789bb6a9000256f016c77. Updates golang/go#29822 Change-Id: I47e75d135e092ee9d5a80c66e91c1048c67cd5f6 Reviewed-on: https://go-review.googlesource.com/c/158542 Reviewed-by: Bryan C. Mills <bcmills@google.com> Reviewed-by: David du Colombier <0intro@gmail.com>
2019-01-19 01:19:58 +03:00
},
dashboard: omit the website repo on plan9 The plan9 builders have relatively slow filesystems, and the website repo is relatively filesystem-intensive — many of its tests walk and process large trees of website content. Instead of trying to precisely tune the x/website tests to pass on the plan9 builders, let's skip this repo on them. For golang/go#49338. Change-Id: Ieaf50fb6955a6b39012a23ff72e61c1c31e4ce7c Reviewed-on: https://go-review.googlesource.com/c/build/+/362975 Trust: Bryan C. Mills <bcmills@google.com> Run-TryBot: Bryan C. Mills <bcmills@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2021-11-10 19:28:50 +03:00
buildsRepo: plan9Default,
dashboard: move builder configuration out of coordinator into a package The new package will be used by the buildlet command line tool in the future. Also use the metadata package in the coordinator for the master build key, since it was already depending on that package. Might as well use it. Change-Id: I6ad7593787ebba364090951ccbaecb4aa38e1dd6 Reviewed-on: https://go-review.googlesource.com/2920 Reviewed-by: Andrew Gerrand <adg@golang.org>
2015-01-15 23:46:22 +03:00
})
dashboard: add configs for new windows 2008, 2012, 2016 builders See also: https://golang.org/cl/41142 Updates golang/go#17513 Change-Id: Ie743ae4604e65892e28423ed0af008450b647197 Reviewed-on: https://go-review.googlesource.com/41393 Reviewed-by: Jeff Johnson <jrjohnson@google.com> Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
2017-04-21 22:35:28 +03:00
addBuilder(BuildConfig{
dashboard: refactor mechanism for adjusting cmd/dist test policy Previously, the behavior of ShouldRunDistTest could be overridden by an individual builder by setting a the shouldRunDistTest field to a custom policy implementation. A pitfall of this approach is that it requires each custom policy to reimplement the default cmd/dist test policy as well. If that isn't done when attempting to make a small adjustment to the default policy, the result is that the default policy is ignored. The fasterTrybots policy, as the name suggests, exists to skip some tests in trybot mode. Due to the problem above, it was causing some builders to run more tests in trybot mode than they would if it weren't applied. For example, the freebsd-amd64-12_0 builder should have been skipping the "api" test in trybot mode, but wasn't. This refactor fixes that by modifying the approach for how individual builders can customize the cmd/dist test behavior. Now, a function distTestAdjust is called with the first parameter specifying the default policy decision. That function can adjust the decision as it sees fit, but otherwise it preserves its initial value in situations where no change is intended to be made. For golang/go#38279. Change-Id: I220248f51abfc8e77195ae0c9134a17f7cac9bc2 Reviewed-on: https://go-review.googlesource.com/c/build/+/227777 Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-04-09 19:47:03 +03:00
Name: "windows-amd64-2008",
HostType: "host-windows-amd64-2008",
distTestAdjust: noTestDirAndNoReboot,
dashboard: do not sunset windows 2008 CL 363474 added a 2012 builder, which is good, but it also took a major step in sunsetting Windows 7 support by removing all 2008 builders from trybots and for Go 1.18+. This will surely bitrot Windows 7 support and force us to remove support for it, which isn't okay. Rather than line up dominoes to force our hand on policy, such huge policy changes with sweeping ramifications should be properly decided. So, this commit keeps the 2008 builders in the running, alongside the 2012 ones. Updates golang/go#47845. Updates golang/go#49490. Change-Id: I574f51ea22f638d2f9b09fcd00fdfc3097631771 Reviewed-on: https://go-review.googlesource.com/c/build/+/366656 Trust: Jason A. Donenfeld <Jason@zx2c4.com> Trust: Carlos Amedee <carlos@golang.org> Run-TryBot: Jason A. Donenfeld <Jason@zx2c4.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-11-23 23:55:31 +03:00
buildsRepo: onlyGo,
dashboard: increase TIMEOUT_SCALE to 2 for windows-amd64 cmd/go takes ~188 seconds on windows-amd64, which is over the default 180 second limit. Bump up the timout rate. Change-Id: Ib0e5289ab6c4d6127d18375e3a72453818b984a8 Reviewed-on: https://go-review.googlesource.com/110415 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-05-01 00:16:33 +03:00
env: []string{
"GOARCH=amd64",
"GOHOSTARCH=amd64",
// cmd/go takes ~188 seconds on windows-amd64
// now, which is over the 180 second default
// dist test timeout. So, bump this builder
// up:
"GO_TEST_TIMEOUT_SCALE=2",
},
dashboard: add configs for new windows 2008, 2012, 2016 builders See also: https://golang.org/cl/41142 Updates golang/go#17513 Change-Id: Ie743ae4604e65892e28423ed0af008450b647197 Reviewed-on: https://go-review.googlesource.com/41393 Reviewed-by: Jeff Johnson <jrjohnson@google.com> Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
2017-04-21 22:35:28 +03:00
})
cmd/buildlet, dashboard: support for v2 windows images, 386 builder - images now include 32bit gcc in c:\godeps\gcc32 - 64bit gcc has moved to c:\godeps\gcc64 - update host definitions for new image versions - add windows-386-2008 host Updates golang/go#17513 Change-Id: Ib3f5d4b2b1702c05089cf633529ebf38db270a43 Reviewed-on: https://go-review.googlesource.com/42014 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2017-04-27 23:12:08 +03:00
addBuilder(BuildConfig{
dashboard: do not sunset windows 2008 CL 363474 added a 2012 builder, which is good, but it also took a major step in sunsetting Windows 7 support by removing all 2008 builders from trybots and for Go 1.18+. This will surely bitrot Windows 7 support and force us to remove support for it, which isn't okay. Rather than line up dominoes to force our hand on policy, such huge policy changes with sweeping ramifications should be properly decided. So, this commit keeps the 2008 builders in the running, alongside the 2012 ones. Updates golang/go#47845. Updates golang/go#49490. Change-Id: I574f51ea22f638d2f9b09fcd00fdfc3097631771 Reviewed-on: https://go-review.googlesource.com/c/build/+/366656 Trust: Jason A. Donenfeld <Jason@zx2c4.com> Trust: Carlos Amedee <carlos@golang.org> Run-TryBot: Jason A. Donenfeld <Jason@zx2c4.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-11-23 23:55:31 +03:00
Name: "windows-386-2008",
HostType: "host-windows-amd64-2008",
buildsRepo: defaultPlusExpBuild,
distTestAdjust: fasterTrybots,
env: []string{"GOARCH=386", "GOHOSTARCH=386"},
tryBot: defaultTrySet(),
dashboard: add windows-386-2012 builder This change adds a Windows 386 builder with version 2012 of Windows server. It replaces Windows 2008 as the version of Windows used for trybot runs. It also ensures Go 1.17 is the latest version of go being tested on Windows 2008 builders. Windows 2008 has reached its end of life and will be phased out. Updates golang/go#47845 Fixes golang/go#49490 Change-Id: I2307c68a74dfb41eb6a1a24177cc58835989d56b Reviewed-on: https://go-review.googlesource.com/c/build/+/363474 Trust: Carlos Amedee <carlos@golang.org> Run-TryBot: Carlos Amedee <carlos@golang.org> Reviewed-by: Heschi Kreinick <heschi@google.com> Reviewed-by: Alexander Rakoczy <alex@golang.org> TryBot-Result: Go Bot <gobot@golang.org>
2021-11-11 23:05:21 +03:00
numTryTestHelpers: 4,
})
addBuilder(BuildConfig{
dashboard: do not sunset windows 2008 CL 363474 added a 2012 builder, which is good, but it also took a major step in sunsetting Windows 7 support by removing all 2008 builders from trybots and for Go 1.18+. This will surely bitrot Windows 7 support and force us to remove support for it, which isn't okay. Rather than line up dominoes to force our hand on policy, such huge policy changes with sweeping ramifications should be properly decided. So, this commit keeps the 2008 builders in the running, alongside the 2012 ones. Updates golang/go#47845. Updates golang/go#49490. Change-Id: I574f51ea22f638d2f9b09fcd00fdfc3097631771 Reviewed-on: https://go-review.googlesource.com/c/build/+/366656 Trust: Jason A. Donenfeld <Jason@zx2c4.com> Trust: Carlos Amedee <carlos@golang.org> Run-TryBot: Jason A. Donenfeld <Jason@zx2c4.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-11-23 23:55:31 +03:00
Name: "windows-386-2012",
HostType: "host-windows-amd64-2012",
distTestAdjust: fasterTrybots,
buildsRepo: onlyGo,
env: []string{"GOARCH=386", "GOHOSTARCH=386"},
tryBot: defaultTrySet(),
cmd/coordinator, dashboard: add mac race builder, skip redundant trybot tests This adds a darwin-amd64-race builder. Of the four platforms that support the race detector, this was the only one missing a builder. To make up for the extra work, skip some redundant tests that the Mac (and other) builds do. The test directory is really slow, and doesn't have platform-specific code. We have amd64 & 386 test coverage for that directory via the Linux builders, which is fine. No need to do it on some of the heavier VM builders, or for trybots. Fixes golang/go#17674 Change-Id: I888119b3fdd28884b87a59d78746ab1fef6960c7 Reviewed-on: https://go-review.googlesource.com/82395 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2017-12-07 03:40:40 +03:00
numTryTestHelpers: 4,
cmd/buildlet, dashboard: support for v2 windows images, 386 builder - images now include 32bit gcc in c:\godeps\gcc32 - 64bit gcc has moved to c:\godeps\gcc64 - update host definitions for new image versions - add windows-386-2008 host Updates golang/go#17513 Change-Id: Ib3f5d4b2b1702c05089cf633529ebf38db270a43 Reviewed-on: https://go-review.googlesource.com/42014 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2017-04-27 23:12:08 +03:00
})
dashboard: add configs for new windows 2008, 2012, 2016 builders See also: https://golang.org/cl/41142 Updates golang/go#17513 Change-Id: Ie743ae4604e65892e28423ed0af008450b647197 Reviewed-on: https://go-review.googlesource.com/41393 Reviewed-by: Jeff Johnson <jrjohnson@google.com> Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
2017-04-21 22:35:28 +03:00
addBuilder(BuildConfig{
dashboard: refactor mechanism for adjusting cmd/dist test policy Previously, the behavior of ShouldRunDistTest could be overridden by an individual builder by setting a the shouldRunDistTest field to a custom policy implementation. A pitfall of this approach is that it requires each custom policy to reimplement the default cmd/dist test policy as well. If that isn't done when attempting to make a small adjustment to the default policy, the result is that the default policy is ignored. The fasterTrybots policy, as the name suggests, exists to skip some tests in trybot mode. Due to the problem above, it was causing some builders to run more tests in trybot mode than they would if it weren't applied. For example, the freebsd-amd64-12_0 builder should have been skipping the "api" test in trybot mode, but wasn't. This refactor fixes that by modifying the approach for how individual builders can customize the cmd/dist test behavior. Now, a function distTestAdjust is called with the first parameter specifying the default policy decision. That function can adjust the decision as it sees fit, but otherwise it preserves its initial value in situations where no change is intended to be made. For golang/go#38279. Change-Id: I220248f51abfc8e77195ae0c9134a17f7cac9bc2 Reviewed-on: https://go-review.googlesource.com/c/build/+/227777 Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-04-09 19:47:03 +03:00
Name: "windows-amd64-2012",
HostType: "host-windows-amd64-2012",
distTestAdjust: noTestDirAndNoReboot,
buildsRepo: onlyGo,
dashboard: increase TIMEOUT_SCALE to 2 for windows-amd64 cmd/go takes ~188 seconds on windows-amd64, which is over the default 180 second limit. Bump up the timout rate. Change-Id: Ib0e5289ab6c4d6127d18375e3a72453818b984a8 Reviewed-on: https://go-review.googlesource.com/110415 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-05-01 00:16:33 +03:00
env: []string{
"GOARCH=amd64",
"GOHOSTARCH=amd64",
// cmd/go takes ~188 seconds on windows-amd64
// now, which is over the 180 second default
// dist test timeout. So, bump this builder
// up:
"GO_TEST_TIMEOUT_SCALE=2",
},
dashboard: add configs for new windows 2008, 2012, 2016 builders See also: https://golang.org/cl/41142 Updates golang/go#17513 Change-Id: Ie743ae4604e65892e28423ed0af008450b647197 Reviewed-on: https://go-review.googlesource.com/41393 Reviewed-by: Jeff Johnson <jrjohnson@google.com> Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
2017-04-21 22:35:28 +03:00
})
build: use buildlet on Windows Also adds "gomote rmall" command. Fixes golang/go#8640 Fixes golang/go#9799 Change-Id: I41b62b16df5e855e1367a9cc1c370bb1f928c242 Reviewed-on: https://go-review.googlesource.com/4132 Reviewed-by: Andrew Gerrand <adg@golang.org> Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
2015-02-07 04:32:15 +03:00
addBuilder(BuildConfig{
dashboard: refactor mechanism for adjusting cmd/dist test policy Previously, the behavior of ShouldRunDistTest could be overridden by an individual builder by setting a the shouldRunDistTest field to a custom policy implementation. A pitfall of this approach is that it requires each custom policy to reimplement the default cmd/dist test policy as well. If that isn't done when attempting to make a small adjustment to the default policy, the result is that the default policy is ignored. The fasterTrybots policy, as the name suggests, exists to skip some tests in trybot mode. Due to the problem above, it was causing some builders to run more tests in trybot mode than they would if it weren't applied. For example, the freebsd-amd64-12_0 builder should have been skipping the "api" test in trybot mode, but wasn't. This refactor fixes that by modifying the approach for how individual builders can customize the cmd/dist test behavior. Now, a function distTestAdjust is called with the first parameter specifying the default policy decision. That function can adjust the decision as it sees fit, but otherwise it preserves its initial value in situations where no change is intended to be made. For golang/go#38279. Change-Id: I220248f51abfc8e77195ae0c9134a17f7cac9bc2 Reviewed-on: https://go-review.googlesource.com/c/build/+/227777 Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-04-09 19:47:03 +03:00
Name: "windows-amd64-2016",
HostType: "host-windows-amd64-2016",
buildsRepo: defaultPlusExpBuild,
distTestAdjust: fasterTrybots,
dashboard: increase TIMEOUT_SCALE to 2 for windows-amd64 cmd/go takes ~188 seconds on windows-amd64, which is over the default 180 second limit. Bump up the timout rate. Change-Id: Ib0e5289ab6c4d6127d18375e3a72453818b984a8 Reviewed-on: https://go-review.googlesource.com/110415 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-05-01 00:16:33 +03:00
env: []string{
"GOARCH=amd64",
"GOHOSTARCH=amd64",
// cmd/go takes ~188 seconds on windows-amd64
// now, which is over the 180 second default
// dist test timeout. So, bump this builder
// up:
"GO_TEST_TIMEOUT_SCALE=2",
},
dashboard, cmd/coordinator: move more config out of coordinator into dashboard And with the increased flexibility, enable trybots for more repos for the "sys" project. Details: https://github.com/golang/go/issues/24893#issuecomment-433517519 Fixes golang/go#24893 Change-Id: I89ce6f219be986201332f8fad11c774bdd6246da Reviewed-on: https://go-review.googlesource.com/c/145157 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2018-10-26 22:21:58 +03:00
tryBot: defaultTrySet(),
dashboard: update Windows builders, remove old manual Windows builder And clean up some other builders. Change-Id: I1f71836da7008e58d3e76e2cc3170e96cd57ddf6 Reviewed-on: https://go-review.googlesource.com/51432 Reviewed-by: Jeff Johnson <jrjohnson@google.com> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2017-07-27 07:52:12 +03:00
numTryTestHelpers: 5,
build: use buildlet on Windows Also adds "gomote rmall" command. Fixes golang/go#8640 Fixes golang/go#9799 Change-Id: I41b62b16df5e855e1367a9cc1c370bb1f928c242 Reviewed-on: https://go-review.googlesource.com/4132 Reviewed-by: Andrew Gerrand <adg@golang.org> Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
2015-02-07 04:32:15 +03:00
})
dashboard: add windows and 32-bit longtest builders Fixes golang/go#29641 Fixes golang/go#26529 Change-Id: I5ab3aa443276ab36b866d7fabc1d78a82b34cb64 Reviewed-on: https://go-review.googlesource.com/c/build/+/202479 Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-10-21 19:58:47 +03:00
addBuilder(BuildConfig{
cmd/coordinator: delete useScheduler knob & old pre-scheduler hacks, code, TODOs Updates golang/go#19178 Change-Id: Id4d8b016c41f57bfaba5ee4ab046285607047493 Reviewed-on: https://go-review.googlesource.com/c/build/+/207178 Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-11-14 18:43:33 +03:00
Name: "windows-amd64-longtest",
dashboard: upsize machine type for windows-amd64-{longtest,race} builders Start using n1-highcpu-8 machine type instead of n1-highcpu-4 for windows-amd64-longtest and windows-amd64-race builders. Continue to use the existing n1-highcpu-4 machine type for all other windows/386 and windows/amd64 builders for now. The primary motivation of this change is to increase amount of RAM available on the longtest builder from 3.6 GB to 7.2 GB, because it is consistenly failing due to being out of memory. Also fix minor typos, and increase consistency in comments and notes. Updates golang/go#33951 Change-Id: Iac5da67fa584059842850a9c8deda98c242741f2 Reviewed-on: https://go-review.googlesource.com/c/build/+/208500 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org>
2019-11-23 02:01:25 +03:00
HostType: "host-windows-amd64-2016-big",
cmd/coordinator: delete useScheduler knob & old pre-scheduler hacks, code, TODOs Updates golang/go#19178 Change-Id: Id4d8b016c41f57bfaba5ee4ab046285607047493 Reviewed-on: https://go-review.googlesource.com/c/build/+/207178 Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-11-14 18:43:33 +03:00
Notes: "Windows Server 2016 with go test -short=false",
dashboard: include longtest builders for trybots on release branches The goal of this test policy change is to decrease the chance of a Go repository release branch from getting into a state where some longtest post-submit builders are failing, as this will interfere with the ability to issue releases after golang/go#29252 is fixed. There are not that many CLs sent to release branches, so the extra latency caused by slower longtest builders being a part of trybots should not be very disruptive. For golang/go#37827. For golang/go#29252. Change-Id: I819b120bb7b763df4acf4583375cdb840793d323 Reviewed-on: https://go-review.googlesource.com/c/build/+/235338 Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
2020-05-27 01:19:05 +03:00
tryBot: func(repo, branch, goBranch string) bool {
onReleaseBranch := strings.HasPrefix(branch, "release-branch.")
return repo == "go" && onReleaseBranch // See issue 37827.
},
dashboard: add windows and 32-bit longtest builders Fixes golang/go#29641 Fixes golang/go#26529 Change-Id: I5ab3aa443276ab36b866d7fabc1d78a82b34cb64 Reviewed-on: https://go-review.googlesource.com/c/build/+/202479 Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-10-21 19:58:47 +03:00
buildsRepo: func(repo, branch, goBranch string) bool {
dashboard: document and clarify defaultBuildsRepoPolicy The defaultBuildsRepoPolicy function was very important in that any changes to it would affect many builders, but it was not documented well. This refactor attempts to improve the situation. It turned out better to also rename defaultBuildsRepoPolicy to buildRepoByDefault and limit its scope to report whether a given repo should be built by default. The branch and goBranch parameters were never used, and thus misleading. If they're needed in the future, we can easily re-add them. The name defaultBuildsRepoPolicy made it sound as if it was the entire "builds repo" policy, bun reality most of the policy is implemented in the BuildConfig.buildsRepoAtAll method and is non-configurable. Only a small part of the policy was in defaultBuildsRepoPolicy. For golang/go#39156. Change-Id: Ie7c6a68bd624056362dc7d964bd8d640f665b35a Reviewed-on: https://go-review.googlesource.com/c/build/+/240687 Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-07-01 21:45:10 +03:00
b := defaultPlusExpBuild(repo, branch, goBranch)
if repo != "go" && !(branch == "master" && goBranch == "master") {
// For golang.org/x repos, don't test non-latest versions.
b = false
dashboard: adjust the windows-amd64-longtest policy It shouldn't build more repos than windows-amd64-2016. Fixes golang/go#35185 Change-Id: I310e2ccc521d88692fcf904fe53b4bae94f3f0f0 Reviewed-on: https://go-review.googlesource.com/c/build/+/203885 Reviewed-by: Bryan C. Mills <bcmills@google.com> Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-10-28 23:16:36 +03:00
}
dashboard: document and clarify defaultBuildsRepoPolicy The defaultBuildsRepoPolicy function was very important in that any changes to it would affect many builders, but it was not documented well. This refactor attempts to improve the situation. It turned out better to also rename defaultBuildsRepoPolicy to buildRepoByDefault and limit its scope to report whether a given repo should be built by default. The branch and goBranch parameters were never used, and thus misleading. If they're needed in the future, we can easily re-add them. The name defaultBuildsRepoPolicy made it sound as if it was the entire "builds repo" policy, bun reality most of the policy is implemented in the BuildConfig.buildsRepoAtAll method and is non-configurable. Only a small part of the policy was in defaultBuildsRepoPolicy. For golang/go#39156. Change-Id: Ie7c6a68bd624056362dc7d964bd8d640f665b35a Reviewed-on: https://go-review.googlesource.com/c/build/+/240687 Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-07-01 21:45:10 +03:00
return b
dashboard: add windows and 32-bit longtest builders Fixes golang/go#29641 Fixes golang/go#26529 Change-Id: I5ab3aa443276ab36b866d7fabc1d78a82b34cb64 Reviewed-on: https://go-review.googlesource.com/c/build/+/202479 Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-10-21 19:58:47 +03:00
},
needsGoProxy: true, // for cmd/go module tests
env: []string{
"GO_TEST_TIMEOUT_SCALE=5", // give them lots of time
},
dashboard: pick 4 TryBot helpers for -longtest SlowBots Based on the data we have so far, 4 helpers seems like a good number to pick. It provides a balance between resource use and diminishing returns. We may need to adjust it as future work for issue 42661, but a first step will be to reduce overhead in accessing historical build timing data. Fixes golang/go#37439. Updates golang/go#42661. Change-Id: I1bfddf85f2dba8b42dc7b8f8749bbf45b614e93d Reviewed-on: https://go-review.googlesource.com/c/build/+/279513 Trust: Dmitri Shuralyov <dmitshur@golang.org> Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-12-21 22:42:00 +03:00
numTryTestHelpers: 4, // Target time is < 15 min for golang.org/issue/42661.
dashboard: add windows and 32-bit longtest builders Fixes golang/go#29641 Fixes golang/go#26529 Change-Id: I5ab3aa443276ab36b866d7fabc1d78a82b34cb64 Reviewed-on: https://go-review.googlesource.com/c/build/+/202479 Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-10-21 19:58:47 +03:00
})
build: use buildlet on Windows Also adds "gomote rmall" command. Fixes golang/go#8640 Fixes golang/go#9799 Change-Id: I41b62b16df5e855e1367a9cc1c370bb1f928c242 Reviewed-on: https://go-review.googlesource.com/4132 Reviewed-by: Andrew Gerrand <adg@golang.org> Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
2015-02-07 04:32:15 +03:00
addBuilder(BuildConfig{
dashboard, cmd/coordinator: various fixes * add support for longtest builders for subrepos. * fix race builders for subrepos (they weren't passing the -race flag). * adjust the policy for the js-wasm builders to build fewer subrepos where it'll never work or isn't worth it. * fix the android emu builders which disappeared because an empty string was being passed to buildsRepoAtAll. In some places in the code an empty string for goBranch for the "go" repo meant the same as branch, but I forgot that in the new code, so an old caller was confusing the new config hooks. Rather than make all policy funcs be aware of both ways, the new code in this CL now maps an empty string to the same as the repo's branch when the repo == "go". Adds a test too. * fix some outdated comments. Change-Id: Icf3fb85e5542a4d314443b59d02517b306ef46b7 Reviewed-on: https://go-review.googlesource.com/c/build/+/166897 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-03-11 21:49:56 +03:00
Name: "windows-amd64-race",
dashboard: update windows-amd64-race builder to Windows Server 2016 image The purpose of the -race builder is to check for bugs on the Go side, so use the newer available Windows version with the expectation that there should be fewer unfixed bugs left in it. Take this easy step to eliminate a variable for issue golang/go#39196. Also delete host-windows-amd64-2008-big, since it's no longer used. For golang/go#39196. Change-Id: I9095a73ab21a151e5972219bccd5a16b249765b4 Reviewed-on: https://go-review.googlesource.com/c/build/+/235419 Reviewed-by: Carlos Amedee <carlos@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-05-27 18:58:11 +03:00
HostType: "host-windows-amd64-2016-big",
dashboard, cmd/coordinator: various fixes * add support for longtest builders for subrepos. * fix race builders for subrepos (they weren't passing the -race flag). * adjust the policy for the js-wasm builders to build fewer subrepos where it'll never work or isn't worth it. * fix the android emu builders which disappeared because an empty string was being passed to buildsRepoAtAll. In some places in the code an empty string for goBranch for the "go" repo meant the same as branch, but I forgot that in the new code, so an old caller was confusing the new config hooks. Rather than make all policy funcs be aware of both ways, the new code in this CL now maps an empty string to the same as the repo's branch when the repo == "go". Adds a test too. * fix some outdated comments. Change-Id: Icf3fb85e5542a4d314443b59d02517b306ef46b7 Reviewed-on: https://go-review.googlesource.com/c/build/+/166897 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-03-11 21:49:56 +03:00
Notes: "Only runs -race tests (./race.bat)",
dashboard: increase TIMEOUT_SCALE to 2 for windows-amd64 cmd/go takes ~188 seconds on windows-amd64, which is over the default 180 second limit. Bump up the timout rate. Change-Id: Ib0e5289ab6c4d6127d18375e3a72453818b984a8 Reviewed-on: https://go-review.googlesource.com/110415 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-05-01 00:16:33 +03:00
env: []string{
"GOARCH=amd64",
"GOHOSTARCH=amd64",
// cmd/go takes ~188 seconds on windows-amd64
// now, which is over the 180 second default
// dist test timeout. So, bump this builder
// up:
"GO_TEST_TIMEOUT_SCALE=2"},
build: use buildlet on Windows Also adds "gomote rmall" command. Fixes golang/go#8640 Fixes golang/go#9799 Change-Id: I41b62b16df5e855e1367a9cc1c370bb1f928c242 Reviewed-on: https://go-review.googlesource.com/4132 Reviewed-by: Andrew Gerrand <adg@golang.org> Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
2015-02-07 04:32:15 +03:00
})
dashboard: add new Windows ARM builder This partially reverts CL 229559, while adjusting it to use a new builder. The builder will be used now for windows/arm and later for windows/arm64. Updates golang/go#42604. Updates golang/go#38607. Updates golang/go#36439. Change-Id: I1ab661a6585e64de8196955c077715b32dc732b8 Reviewed-on: https://go-review.googlesource.com/c/build/+/272106 Run-TryBot: Jason A. Donenfeld <Jason@zx2c4.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Trust: Alex Brainman <alex.brainman@gmail.com> Trust: Jason A. Donenfeld <Jason@zx2c4.com>
2020-11-20 20:35:33 +03:00
addBuilder(BuildConfig{
dashboard: remove issue for windows/arm builder We originally left the issue attached to the builder, before we knew that it worked. But it appears to be working well, with no shortage of work to do from farmer. Fixes golang/go#38607. Change-Id: I0e654c825a92b64132efbc409b768e5e83967802 Reviewed-on: https://go-review.googlesource.com/c/build/+/273786 Trust: Jason A. Donenfeld <Jason@zx2c4.com> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2020-11-28 17:02:05 +03:00
Name: "windows-arm-zx2c4",
HostType: "host-windows-arm64-zx2c4",
dashboard: add new Windows ARM builder This partially reverts CL 229559, while adjusting it to use a new builder. The builder will be used now for windows/arm and later for windows/arm64. Updates golang/go#42604. Updates golang/go#38607. Updates golang/go#36439. Change-Id: I1ab661a6585e64de8196955c077715b32dc732b8 Reviewed-on: https://go-review.googlesource.com/c/build/+/272106 Run-TryBot: Jason A. Donenfeld <Jason@zx2c4.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Trust: Alex Brainman <alex.brainman@gmail.com> Trust: Jason A. Donenfeld <Jason@zx2c4.com>
2020-11-20 20:35:33 +03:00
env: []string{
"GOARM=7",
dashboard: expand time scale for windows-arm builder This builder has very slow I/O and could benefit from some extra time on a few tests. At the moment, the tests for x/review/git-codereview are failing with a time out, for example. Updates golang/go#42925. Change-Id: I532f4ddd3b9e7fd0a66ea9631b3fa7ce363e05a4 Reviewed-on: https://go-review.googlesource.com/c/build/+/274572 Trust: Jason A. Donenfeld <Jason@zx2c4.com> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2020-12-01 22:06:23 +03:00
"GO_TEST_TIMEOUT_SCALE=3"},
dashboard: add new Windows ARM builder This partially reverts CL 229559, while adjusting it to use a new builder. The builder will be used now for windows/arm and later for windows/arm64. Updates golang/go#42604. Updates golang/go#38607. Updates golang/go#36439. Change-Id: I1ab661a6585e64de8196955c077715b32dc732b8 Reviewed-on: https://go-review.googlesource.com/c/build/+/272106 Run-TryBot: Jason A. Donenfeld <Jason@zx2c4.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Trust: Alex Brainman <alex.brainman@gmail.com> Trust: Jason A. Donenfeld <Jason@zx2c4.com>
2020-11-20 20:35:33 +03:00
})
dashboard: add windows-arm64-10 reverse buildlet This adds a reverse buildlet for Windows ARM64 running under qemu on M1 Mac Minis. The VM currently boots, but more changes are needed to stage0 and buildlet in order for the builder to run. For golang/go#42604. Change-Id: Ie86e596c0f166d2ca8c075c0c971fcd1e34e940f Reviewed-on: https://go-review.googlesource.com/c/build/+/328311 Trust: Alexander Rakoczy <alex@golang.org> Run-TryBot: Alexander Rakoczy <alex@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Heschi Kreinick <heschi@google.com> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-06-15 23:42:51 +03:00
addBuilder(BuildConfig{
Name: "windows-arm64-10",
HostType: "host-windows-arm64-mini",
numTryTestHelpers: 1,
buildsRepo: func(repo, branch, goBranch string) bool {
return atLeastGo1(goBranch, 17) && buildRepoByDefault(repo)
},
dashboard: add windows-arm-10 builder This adds a windows-arm builder using host-windows-arm64-mini hosts. The new BuildConfig has a KnownIssue until we can verify that the builders are building correctly. For golang/go#47019 Change-Id: I0505d4dc367dbb687290facbddc718eb13567025 Reviewed-on: https://go-review.googlesource.com/c/build/+/344990 Trust: Alexander Rakoczy <alex@golang.org> Run-TryBot: Alexander Rakoczy <alex@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-08-25 22:50:51 +03:00
env: []string{
"GOARCH=arm64",
},
})
cmd/coordinator, dashboard: add mac race builder, skip redundant trybot tests This adds a darwin-amd64-race builder. Of the four platforms that support the race detector, this was the only one missing a builder. To make up for the extra work, skip some redundant tests that the Mac (and other) builds do. The test directory is really slow, and doesn't have platform-specific code. We have amd64 & 386 test coverage for that directory via the Linux builders, which is fine. No need to do it on some of the heavier VM builders, or for trybots. Fixes golang/go#17674 Change-Id: I888119b3fdd28884b87a59d78746ab1fef6960c7 Reviewed-on: https://go-review.googlesource.com/82395 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2017-12-07 03:40:40 +03:00
addBuilder(BuildConfig{
dashboard: refactor mechanism for adjusting cmd/dist test policy Previously, the behavior of ShouldRunDistTest could be overridden by an individual builder by setting a the shouldRunDistTest field to a custom policy implementation. A pitfall of this approach is that it requires each custom policy to reimplement the default cmd/dist test policy as well. If that isn't done when attempting to make a small adjustment to the default policy, the result is that the default policy is ignored. The fasterTrybots policy, as the name suggests, exists to skip some tests in trybot mode. Due to the problem above, it was causing some builders to run more tests in trybot mode than they would if it weren't applied. For example, the freebsd-amd64-12_0 builder should have been skipping the "api" test in trybot mode, but wasn't. This refactor fixes that by modifying the approach for how individual builders can customize the cmd/dist test behavior. Now, a function distTestAdjust is called with the first parameter specifying the default policy decision. That function can adjust the decision as it sees fit, but otherwise it preserves its initial value in situations where no change is intended to be made. For golang/go#38279. Change-Id: I220248f51abfc8e77195ae0c9134a17f7cac9bc2 Reviewed-on: https://go-review.googlesource.com/c/build/+/227777 Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-04-09 19:47:03 +03:00
Name: "darwin-amd64-10_12",
HostType: "host-darwin-10_12",
distTestAdjust: macTestPolicy,
dashboard: limit macOS 10.12 builders to at most go1.16 macOS will not be supported after go1.16. This change limits the macOS 10.12 builders to build previous to go1.17. Updates golang/go#44303 For golang/go#23011 Change-Id: Ib6b4e7939ffb9032b255574d3f0b4fffaad97803 Reviewed-on: https://go-review.googlesource.com/c/build/+/312692 Trust: Carlos Amedee <carlos@golang.org> Run-TryBot: Carlos Amedee <carlos@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2021-04-22 20:51:16 +03:00
buildsRepo: func(repo, branch, goBranch string) bool {
// macOS 10.12 not supported after Go 1.16
return atMostGo1(goBranch, 16) && buildRepoByDefault(repo)
},
dashboard: restore darwin-386 buidler Fixes golang/go#17009 Change-Id: Ia9008bcb7b47b7671c158c1a00afc4e765805e4e Reviewed-on: https://go-review.googlesource.com/41617 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2017-04-25 02:03:42 +03:00
})
all: add a macOS 10.14 (Mojave) builder Fixes golang/go#27806 Change-Id: I576f563acb2c50cd0456cd4e6a1271b9aa59c9df Reviewed-on: https://go-review.googlesource.com/c/build/+/169498 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-03-27 02:04:31 +03:00
addBuilder(BuildConfig{
dashboard: refactor mechanism for adjusting cmd/dist test policy Previously, the behavior of ShouldRunDistTest could be overridden by an individual builder by setting a the shouldRunDistTest field to a custom policy implementation. A pitfall of this approach is that it requires each custom policy to reimplement the default cmd/dist test policy as well. If that isn't done when attempting to make a small adjustment to the default policy, the result is that the default policy is ignored. The fasterTrybots policy, as the name suggests, exists to skip some tests in trybot mode. Due to the problem above, it was causing some builders to run more tests in trybot mode than they would if it weren't applied. For example, the freebsd-amd64-12_0 builder should have been skipping the "api" test in trybot mode, but wasn't. This refactor fixes that by modifying the approach for how individual builders can customize the cmd/dist test behavior. Now, a function distTestAdjust is called with the first parameter specifying the default policy decision. That function can adjust the decision as it sees fit, but otherwise it preserves its initial value in situations where no change is intended to be made. For golang/go#38279. Change-Id: I220248f51abfc8e77195ae0c9134a17f7cac9bc2 Reviewed-on: https://go-review.googlesource.com/c/build/+/227777 Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-04-09 19:47:03 +03:00
Name: "darwin-amd64-10_14",
HostType: "host-darwin-10_14",
distTestAdjust: macTestPolicy,
buildsRepo: defaultPlusExp,
all: add a macOS 10.14 (Mojave) builder Fixes golang/go#27806 Change-Id: I576f563acb2c50cd0456cd4e6a1271b9aa59c9df Reviewed-on: https://go-review.googlesource.com/c/build/+/169498 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-03-27 02:04:31 +03:00
})
dashboard, cmd/makemac: add macOS 10.15 Catalina builder Added the macOS 10.15 Catalina builder to the dashboard. Catalina builders should use the Mojave guestid in vSphere 6.7 update 3 because the Catalina guestid is not available in this version of vSphere. A Catalina builder will run on this version of vSphere with the Mojave guestid. Modified the expected number of vm instances running on Macstadium due to the addition of macOS 10.15 builders. Updates golang/go#34748 Change-Id: If14f6e088afc05dd1783460f21907e9345e3c338 Reviewed-on: https://go-review.googlesource.com/c/build/+/207937 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-11-19 17:55:01 +03:00
addBuilder(BuildConfig{
dashboard: refactor mechanism for adjusting cmd/dist test policy Previously, the behavior of ShouldRunDistTest could be overridden by an individual builder by setting a the shouldRunDistTest field to a custom policy implementation. A pitfall of this approach is that it requires each custom policy to reimplement the default cmd/dist test policy as well. If that isn't done when attempting to make a small adjustment to the default policy, the result is that the default policy is ignored. The fasterTrybots policy, as the name suggests, exists to skip some tests in trybot mode. Due to the problem above, it was causing some builders to run more tests in trybot mode than they would if it weren't applied. For example, the freebsd-amd64-12_0 builder should have been skipping the "api" test in trybot mode, but wasn't. This refactor fixes that by modifying the approach for how individual builders can customize the cmd/dist test behavior. Now, a function distTestAdjust is called with the first parameter specifying the default policy decision. That function can adjust the decision as it sees fit, but otherwise it preserves its initial value in situations where no change is intended to be made. For golang/go#38279. Change-Id: I220248f51abfc8e77195ae0c9134a17f7cac9bc2 Reviewed-on: https://go-review.googlesource.com/c/build/+/227777 Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-04-09 19:47:03 +03:00
Name: "darwin-amd64-10_15",
HostType: "host-darwin-10_15",
distTestAdjust: macTestPolicy,
buildsRepo: defaultPlusExpBuild,
dashboard, cmd/makemac: add macOS 10.15 Catalina builder Added the macOS 10.15 Catalina builder to the dashboard. Catalina builders should use the Mojave guestid in vSphere 6.7 update 3 because the Catalina guestid is not available in this version of vSphere. A Catalina builder will run on this version of vSphere with the Mojave guestid. Modified the expected number of vm instances running on Macstadium due to the addition of macOS 10.15 builders. Updates golang/go#34748 Change-Id: If14f6e088afc05dd1783460f21907e9345e3c338 Reviewed-on: https://go-review.googlesource.com/c/build/+/207937 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-11-19 17:55:01 +03:00
})
dashboard: add macOS 11.0 Big Sur builder This change adds macOS 11.0 Big Sur builder to the dashboard. It will not spawn an instance on the older MacStadium cluster which we are migrating away from. It will be marked with a known issue until until we have migrated away from the old cluster and makemac has been updated to process the new macOS versioning convention. Updates golang/go#40762 Updates golang/go#40757 Updates golang/go#40761 Change-Id: I24ce615a2c0c55dd75beed4f821119d4058ad8af Reviewed-on: https://go-review.googlesource.com/c/build/+/288192 Trust: Carlos Amedee <carlos@golang.org> Run-TryBot: Carlos Amedee <carlos@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2021-01-29 23:47:13 +03:00
addBuilder(BuildConfig{
Name: "darwin-amd64-11_0",
HostType: "host-darwin-amd64-11_0",
distTestAdjust: macTestPolicy,
buildsRepo: defaultPlusExpBuild,
})
dashboard: add darwin-amd64-nocgo config, remove nacl-386 trybot Also remove dead nacl-arm. It hasn't run in ages. And update netbsd comment about why 386 doesn't run. And correct its VM image name. Updates golang/go#31705 Updates golang/go#31726 Change-Id: I9de4605f34a052d0a84684fca098388d75602a82 Reviewed-on: https://go-review.googlesource.com/c/build/+/174637 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-05-01 01:37:31 +03:00
addBuilder(BuildConfig{
all: add macOS 12 AMD64 builders I'm not 100% sure they'll work, so just one for now. I've updated the documentation according to my very shaky understanding. For golang/go#49149. Change-Id: Ic8f3b7fe96a1a6c18c0940a4aa2d08ec9dc2138b Reviewed-on: https://go-review.googlesource.com/c/build/+/365194 Trust: Heschi Kreinick <heschi@google.com> Run-TryBot: Heschi Kreinick <heschi@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-11-17 01:09:45 +03:00
Name: "darwin-amd64-12_0",
HostType: "host-darwin-amd64-12_0",
distTestAdjust: macTestPolicy,
buildsRepo: defaultPlusExpBuild,
})
addBuilder(BuildConfig{
dashboard: refactor mechanism for adjusting cmd/dist test policy Previously, the behavior of ShouldRunDistTest could be overridden by an individual builder by setting a the shouldRunDistTest field to a custom policy implementation. A pitfall of this approach is that it requires each custom policy to reimplement the default cmd/dist test policy as well. If that isn't done when attempting to make a small adjustment to the default policy, the result is that the default policy is ignored. The fasterTrybots policy, as the name suggests, exists to skip some tests in trybot mode. Due to the problem above, it was causing some builders to run more tests in trybot mode than they would if it weren't applied. For example, the freebsd-amd64-12_0 builder should have been skipping the "api" test in trybot mode, but wasn't. This refactor fixes that by modifying the approach for how individual builders can customize the cmd/dist test behavior. Now, a function distTestAdjust is called with the first parameter specifying the default policy decision. That function can adjust the decision as it sees fit, but otherwise it preserves its initial value in situations where no change is intended to be made. For golang/go#38279. Change-Id: I220248f51abfc8e77195ae0c9134a17f7cac9bc2 Reviewed-on: https://go-review.googlesource.com/c/build/+/227777 Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-04-09 19:47:03 +03:00
Name: "darwin-amd64-nocgo",
buildlet,dashboard: tweaks to Darwin, OpenBSD, and Linux builders - Add more Monterey builders and remove the known issue: it's stable. Also use it for race and nocgo. Update slowbot aliases to point to it. - Don't test 1.16 on OpenBSD 7.0, it segfaults. - Pin the Linux (ContainerOS) builders to an older version to avoid the pprof kernel bug. For golang/go#49065, golang/go#49149, and golang/go#48977. Change-Id: Ibec2fa735183ec65e5066c7c752ac356b7360550 Reviewed-on: https://go-review.googlesource.com/c/build/+/365777 Trust: Heschi Kreinick <heschi@google.com> Run-TryBot: Heschi Kreinick <heschi@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org>
2021-11-19 22:46:52 +03:00
HostType: "host-darwin-amd64-12_0",
dashboard: refactor mechanism for adjusting cmd/dist test policy Previously, the behavior of ShouldRunDistTest could be overridden by an individual builder by setting a the shouldRunDistTest field to a custom policy implementation. A pitfall of this approach is that it requires each custom policy to reimplement the default cmd/dist test policy as well. If that isn't done when attempting to make a small adjustment to the default policy, the result is that the default policy is ignored. The fasterTrybots policy, as the name suggests, exists to skip some tests in trybot mode. Due to the problem above, it was causing some builders to run more tests in trybot mode than they would if it weren't applied. For example, the freebsd-amd64-12_0 builder should have been skipping the "api" test in trybot mode, but wasn't. This refactor fixes that by modifying the approach for how individual builders can customize the cmd/dist test behavior. Now, a function distTestAdjust is called with the first parameter specifying the default policy decision. That function can adjust the decision as it sees fit, but otherwise it preserves its initial value in situations where no change is intended to be made. For golang/go#38279. Change-Id: I220248f51abfc8e77195ae0c9134a17f7cac9bc2 Reviewed-on: https://go-review.googlesource.com/c/build/+/227777 Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-04-09 19:47:03 +03:00
distTestAdjust: noTestDirAndNoReboot,
env: []string{"CGO_ENABLED=0"},
dashboard: add darwin-amd64-nocgo config, remove nacl-386 trybot Also remove dead nacl-arm. It hasn't run in ages. And update netbsd comment about why 386 doesn't run. And correct its VM image name. Updates golang/go#31705 Updates golang/go#31726 Change-Id: I9de4605f34a052d0a84684fca098388d75602a82 Reviewed-on: https://go-review.googlesource.com/c/build/+/174637 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-05-01 01:37:31 +03:00
})
dashboard: add new darwin-arm64-11_0-toothrot builder These builders are run by the Go team. They are non-hermetic. For golang/go#39782 Change-Id: Ie402faf9a92ff6c667c8da6fc0dac8528331df48 Reviewed-on: https://go-review.googlesource.com/c/build/+/278432 Trust: Alexander Rakoczy <alex@golang.org> Run-TryBot: Alexander Rakoczy <alex@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
2020-12-15 20:58:50 +03:00
addBuilder(BuildConfig{
Name: "darwin-arm64-11_0-toothrot",
HostType: "host-darwin-arm64-11_0-toothrot",
distTestAdjust: macTestPolicy,
dashboard: add darwin-arm64-12_0 builder Upgrades our arm64-11_0 builder. For golang/go#49149 Change-Id: I517cc5a999cea0cbc96ee2a6d3901840e2473b6a Reviewed-on: https://go-review.googlesource.com/c/build/+/365736 Trust: Alexander Rakoczy <alex@golang.org> Run-TryBot: Alexander Rakoczy <alex@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2021-11-18 19:31:07 +03:00
buildsRepo: defaultPlusExpBuild,
})
addBuilder(BuildConfig{
Name: "darwin-arm64-12_0-toothrot",
HostType: "host-darwin-arm64-12_0-toothrot",
KnownIssue: 49149,
distTestAdjust: macTestPolicy,
buildsRepo: defaultPlusExpBuild,
dashboard: add new darwin-arm64-11_0-toothrot builder These builders are run by the Go team. They are non-hermetic. For golang/go#39782 Change-Id: Ie402faf9a92ff6c667c8da6fc0dac8528331df48 Reviewed-on: https://go-review.googlesource.com/c/build/+/278432 Trust: Alexander Rakoczy <alex@golang.org> Run-TryBot: Alexander Rakoczy <alex@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
2020-12-15 20:58:50 +03:00
})
build: add macoS Sierra builder Change-Id: I10a7a9e44a158fdce5287a9b0b065b35d11fb502 Reviewed-on: https://go-review.googlesource.com/35470 Reviewed-by: Chris Broadfoot <cbro@golang.org>
2017-01-19 01:45:12 +03:00
addBuilder(BuildConfig{
dashboard: refactor mechanism for adjusting cmd/dist test policy Previously, the behavior of ShouldRunDistTest could be overridden by an individual builder by setting a the shouldRunDistTest field to a custom policy implementation. A pitfall of this approach is that it requires each custom policy to reimplement the default cmd/dist test policy as well. If that isn't done when attempting to make a small adjustment to the default policy, the result is that the default policy is ignored. The fasterTrybots policy, as the name suggests, exists to skip some tests in trybot mode. Due to the problem above, it was causing some builders to run more tests in trybot mode than they would if it weren't applied. For example, the freebsd-amd64-12_0 builder should have been skipping the "api" test in trybot mode, but wasn't. This refactor fixes that by modifying the approach for how individual builders can customize the cmd/dist test behavior. Now, a function distTestAdjust is called with the first parameter specifying the default policy decision. That function can adjust the decision as it sees fit, but otherwise it preserves its initial value in situations where no change is intended to be made. For golang/go#38279. Change-Id: I220248f51abfc8e77195ae0c9134a17f7cac9bc2 Reviewed-on: https://go-review.googlesource.com/c/build/+/227777 Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-04-09 19:47:03 +03:00
Name: "darwin-amd64-race",
buildlet,dashboard: tweaks to Darwin, OpenBSD, and Linux builders - Add more Monterey builders and remove the known issue: it's stable. Also use it for race and nocgo. Update slowbot aliases to point to it. - Don't test 1.16 on OpenBSD 7.0, it segfaults. - Pin the Linux (ContainerOS) builders to an older version to avoid the pprof kernel bug. For golang/go#49065, golang/go#49149, and golang/go#48977. Change-Id: Ibec2fa735183ec65e5066c7c752ac356b7360550 Reviewed-on: https://go-review.googlesource.com/c/build/+/365777 Trust: Heschi Kreinick <heschi@google.com> Run-TryBot: Heschi Kreinick <heschi@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org>
2021-11-19 22:46:52 +03:00
HostType: "host-darwin-amd64-12_0",
dashboard: refactor mechanism for adjusting cmd/dist test policy Previously, the behavior of ShouldRunDistTest could be overridden by an individual builder by setting a the shouldRunDistTest field to a custom policy implementation. A pitfall of this approach is that it requires each custom policy to reimplement the default cmd/dist test policy as well. If that isn't done when attempting to make a small adjustment to the default policy, the result is that the default policy is ignored. The fasterTrybots policy, as the name suggests, exists to skip some tests in trybot mode. Due to the problem above, it was causing some builders to run more tests in trybot mode than they would if it weren't applied. For example, the freebsd-amd64-12_0 builder should have been skipping the "api" test in trybot mode, but wasn't. This refactor fixes that by modifying the approach for how individual builders can customize the cmd/dist test behavior. Now, a function distTestAdjust is called with the first parameter specifying the default policy decision. That function can adjust the decision as it sees fit, but otherwise it preserves its initial value in situations where no change is intended to be made. For golang/go#38279. Change-Id: I220248f51abfc8e77195ae0c9134a17f7cac9bc2 Reviewed-on: https://go-review.googlesource.com/c/build/+/227777 Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-04-09 19:47:03 +03:00
distTestAdjust: macTestPolicy,
buildsRepo: onlyGo,
build: add macoS Sierra builder Change-Id: I10a7a9e44a158fdce5287a9b0b065b35d11fb502 Reviewed-on: https://go-review.googlesource.com/35470 Reviewed-by: Chris Broadfoot <cbro@golang.org>
2017-01-19 01:45:12 +03:00
})
dashboard: add Zenly's virtual iOS builders Fixes golang/go#31722 Change-Id: Ia7f0d3f2e1a25000c42a10ff58d7e91d384b1219 Reviewed-on: https://go-review.googlesource.com/c/build/+/174797 Run-TryBot: Elias Naur <mail@eliasnaur.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-05-01 21:33:26 +03:00
addBuilder(BuildConfig{
dashboard,env: move iOS builders to GOOS=ios Updates golang/go#41610 Change-Id: Ie8cb3294da5566e09fc06c01e679b8b773a439cf Reviewed-on: https://go-review.googlesource.com/c/build/+/259358 Run-TryBot: Elias Naur <mail@eliasnaur.com> TryBot-Result: Go Bot <gobot@golang.org> Trust: Elias Naur <mail@eliasnaur.com> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2020-10-03 12:37:07 +03:00
Name: "ios-arm64-corellium",
HostType: "host-ios-arm64-corellium-ios",
dashboard, cmd/coordinator: support multiple builder owners Prefer displaying GitHub usernames for convenience of pinging in GitHub issues, since all current builder owners have them. Keep it possible to use an email or something else if no GitHub account exists. Update the owner of the Corellium builders to be individual GitHub account, so that it can be pinged in an issue. The GitHub organization is already mentioned in the builder notes. Also switch to html/template (instead of text/template) for rendering the builders page. Fixes golang/go#49596. Change-Id: I89a96eb8f7a3057bc1f3aaee16abc9c776c45ee7 Reviewed-on: https://go-review.googlesource.com/c/build/+/363983 Trust: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-11-16 20:42:29 +03:00
Notes: "Virtual iPhone SE running on Corellium; owned by zenly (github.com/znly)",
dashboard: add Zenly's virtual iOS builders Fixes golang/go#31722 Change-Id: Ia7f0d3f2e1a25000c42a10ff58d7e91d384b1219 Reviewed-on: https://go-review.googlesource.com/c/build/+/174797 Run-TryBot: Elias Naur <mail@eliasnaur.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-05-01 21:33:26 +03:00
buildsRepo: func(repo, branch, goBranch string) bool {
dashboard: add Zenly's virtual Android builders Change-Id: Id0be99f343a819c19b0004dc4160175c9bb0ace9 Reviewed-on: https://go-review.googlesource.com/c/build/+/174798 Run-TryBot: Elias Naur <mail@eliasnaur.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-05-01 23:28:43 +03:00
return repo == "go" && branch == "master" && goBranch == "master"
},
dashboard: remove known issue for iOS and Android builders The old known issue has been resolved: the builders have been restarted and are back. That uncovered what appears to be a recent regression, reported as golang/go#49616. Also add a new builder owner based on conversation at https://groups.google.com/g/golang-dev/c/oiuIE7qrWp0. Updates golang/go#48772. Updates golang/go#49048. Updates golang/go#49616. Change-Id: I7a6a89b7fb088373a70bd3496ad2091ec7a3d79f Reviewed-on: https://go-review.googlesource.com/c/build/+/363985 Trust: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org>
2021-11-16 20:42:29 +03:00
KnownIssue: 49616,
dashboard: add Zenly's virtual Android builders Change-Id: Id0be99f343a819c19b0004dc4160175c9bb0ace9 Reviewed-on: https://go-review.googlesource.com/c/build/+/174798 Run-TryBot: Elias Naur <mail@eliasnaur.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-05-01 23:28:43 +03:00
})
addBuilder(BuildConfig{
Name: "android-arm64-corellium",
HostType: "host-android-arm64-corellium-android",
dashboard, cmd/coordinator: support multiple builder owners Prefer displaying GitHub usernames for convenience of pinging in GitHub issues, since all current builder owners have them. Keep it possible to use an email or something else if no GitHub account exists. Update the owner of the Corellium builders to be individual GitHub account, so that it can be pinged in an issue. The GitHub organization is already mentioned in the builder notes. Also switch to html/template (instead of text/template) for rendering the builders page. Fixes golang/go#49596. Change-Id: I89a96eb8f7a3057bc1f3aaee16abc9c776c45ee7 Reviewed-on: https://go-review.googlesource.com/c/build/+/363983 Trust: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-11-16 20:42:29 +03:00
Notes: "Virtual Android running on Corellium; owned by zenly (github.com/znly)",
dashboard: add Zenly's virtual Android builders Change-Id: Id0be99f343a819c19b0004dc4160175c9bb0ace9 Reviewed-on: https://go-review.googlesource.com/c/build/+/174798 Run-TryBot: Elias Naur <mail@eliasnaur.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-05-01 23:28:43 +03:00
buildsRepo: func(repo, branch, goBranch string) bool {
dashboard: add Zenly's virtual iOS builders Fixes golang/go#31722 Change-Id: Ia7f0d3f2e1a25000c42a10ff58d7e91d384b1219 Reviewed-on: https://go-review.googlesource.com/c/build/+/174797 Run-TryBot: Elias Naur <mail@eliasnaur.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-05-01 21:33:26 +03:00
return repo == "go" && branch == "master" && goBranch == "master"
},
})
dashboard: add android/arm builder on Corellium Updates golang/go#31722 Change-Id: I07d9f85d47b6cda5eadcfc4f93adff42316f0ff5 Reviewed-on: https://go-review.googlesource.com/c/build/+/174857 TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-05-02 01:54:54 +03:00
addBuilder(BuildConfig{
Name: "android-arm-corellium",
HostType: "host-android-arm64-corellium-android",
dashboard, cmd/coordinator: support multiple builder owners Prefer displaying GitHub usernames for convenience of pinging in GitHub issues, since all current builder owners have them. Keep it possible to use an email or something else if no GitHub account exists. Update the owner of the Corellium builders to be individual GitHub account, so that it can be pinged in an issue. The GitHub organization is already mentioned in the builder notes. Also switch to html/template (instead of text/template) for rendering the builders page. Fixes golang/go#49596. Change-Id: I89a96eb8f7a3057bc1f3aaee16abc9c776c45ee7 Reviewed-on: https://go-review.googlesource.com/c/build/+/363983 Trust: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-11-16 20:42:29 +03:00
Notes: "Virtual Android running on Corellium; owned by zenly (github.com/znly)",
dashboard: add android/arm builder on Corellium Updates golang/go#31722 Change-Id: I07d9f85d47b6cda5eadcfc4f93adff42316f0ff5 Reviewed-on: https://go-review.googlesource.com/c/build/+/174857 TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-05-02 01:54:54 +03:00
buildsRepo: func(repo, branch, goBranch string) bool {
return repo == "go" && branch == "master" && goBranch == "master"
},
env: []string{
"CGO_ENABLED=1",
"GOARCH=arm",
},
})
env/android-amd64-emu: add new Android emulator container + VM image Now that we can do nested virtualization on GCE, that means we can run the Android emulator (which requires KVM) on GCE and at least get fast trybots and such for android-amd64. Updates golang/go#23824 Change-Id: I0da38c7fa0f15492230a31291d2921ba72f2151d Reviewed-on: https://go-review.googlesource.com/c/163738 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-02-25 23:35:29 +03:00
addBuilder(BuildConfig{
Name: "android-386-emu",
HostType: "host-android-amd64-emu", // same amd64 host is used for 386 builder
Notes: "Android emulator on GCE",
dashboard: unify post-submit & trybot policy in builders.go, not coordinator This removes a bunch of TODOs about scattered policy and moves it all into builders.go. While here, * add a bunch of tests * unexport some things * rename some things * document some things * adjust FreeBSD policy as function of branch (per Go 1.12 being last to support FreeBSD 10.x, and to unblock CL 165801) * set GO_BUILDER_SET_GOPROXY=coordinator for reverse buildlets, which is necessary to remove the oauth2 & build special cases in the config * change Elias Naur's mobile builder to how I think he wants it (he was fighting the old system) * add $HOME on the Solaris smartos builder, which was missing & causing tests to fail lately * make the (currently failing) longtest builder have GOPROXY set * remove an allocation in version.ParseReleaseBranch Fixes golang/go#9603 Updates golang/go#14594 Change-Id: I50a23ad7cdf478c95b14bee9b3931ba361baacfa Reviewed-on: https://go-review.googlesource.com/c/build/+/166218 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-03-07 20:44:41 +03:00
buildsRepo: func(repo, branch, goBranch string) bool {
dashboard: add a GOOS=linux builder & trybot for Android Updates golang/go#30482 Change-Id: I198dd40dfdb86b20e681c373141754680351e82f Reviewed-on: https://go-review.googlesource.com/c/164740 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Elias Naur <mail@eliasnaur.com>
2019-03-01 19:54:25 +03:00
switch repo {
dashboard: stop testing x/build on even more builders Follow up to CL 210378 after looking at: https://build.golang.org/?repo=golang.org%2fx%2fbuild Updates golang/go#30640 Change-Id: I81505a9ca2985ffa6e75922665b14ef4de2af860 Reviewed-on: https://go-review.googlesource.com/c/build/+/211100 Reviewed-by: Bryan C. Mills <bcmills@google.com> Run-TryBot: Bryan C. Mills <bcmills@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-12-12 20:43:59 +03:00
case "build", "blog", "talks", "review", "tour", "website":
dashboard: add a GOOS=linux builder & trybot for Android Updates golang/go#30482 Change-Id: I198dd40dfdb86b20e681c373141754680351e82f Reviewed-on: https://go-review.googlesource.com/c/164740 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Elias Naur <mail@eliasnaur.com>
2019-03-01 19:54:25 +03:00
return false
}
dashboard: unify post-submit & trybot policy in builders.go, not coordinator This removes a bunch of TODOs about scattered policy and moves it all into builders.go. While here, * add a bunch of tests * unexport some things * rename some things * document some things * adjust FreeBSD policy as function of branch (per Go 1.12 being last to support FreeBSD 10.x, and to unblock CL 165801) * set GO_BUILDER_SET_GOPROXY=coordinator for reverse buildlets, which is necessary to remove the oauth2 & build special cases in the config * change Elias Naur's mobile builder to how I think he wants it (he was fighting the old system) * add $HOME on the Solaris smartos builder, which was missing & causing tests to fail lately * make the (currently failing) longtest builder have GOPROXY set * remove an allocation in version.ParseReleaseBranch Fixes golang/go#9603 Updates golang/go#14594 Change-Id: I50a23ad7cdf478c95b14bee9b3931ba361baacfa Reviewed-on: https://go-review.googlesource.com/c/build/+/166218 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-03-07 20:44:41 +03:00
return atLeastGo1(branch, 13) && atLeastGo1(goBranch, 13)
dashboard: add a GOOS=linux builder & trybot for Android Updates golang/go#30482 Change-Id: I198dd40dfdb86b20e681c373141754680351e82f Reviewed-on: https://go-review.googlesource.com/c/164740 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Elias Naur <mail@eliasnaur.com>
2019-03-01 19:54:25 +03:00
},
env/android-amd64-emu: add new Android emulator container + VM image Now that we can do nested virtualization on GCE, that means we can run the Android emulator (which requires KVM) on GCE and at least get fast trybots and such for android-amd64. Updates golang/go#23824 Change-Id: I0da38c7fa0f15492230a31291d2921ba72f2151d Reviewed-on: https://go-review.googlesource.com/c/163738 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-02-25 23:35:29 +03:00
env: []string{
"GOARCH=386",
"GOOS=android",
"GOHOSTARCH=amd64",
"GOHOSTOS=linux",
"CGO_ENABLED=1",
},
})
addBuilder(BuildConfig{
dashboard: add Android trybots Updates golang/go#23824 Change-Id: I6386bc267c35856858b3e46d0998fb60e3406e02 Reviewed-on: https://go-review.googlesource.com/c/build/+/172798 Reviewed-by: Elias Naur <mail@eliasnaur.com> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-04-18 22:45:32 +03:00
Name: "android-amd64-emu",
HostType: "host-android-amd64-emu",
Notes: "Android emulator on GCE",
numTryTestHelpers: 3,
dashboard: unify post-submit & trybot policy in builders.go, not coordinator This removes a bunch of TODOs about scattered policy and moves it all into builders.go. While here, * add a bunch of tests * unexport some things * rename some things * document some things * adjust FreeBSD policy as function of branch (per Go 1.12 being last to support FreeBSD 10.x, and to unblock CL 165801) * set GO_BUILDER_SET_GOPROXY=coordinator for reverse buildlets, which is necessary to remove the oauth2 & build special cases in the config * change Elias Naur's mobile builder to how I think he wants it (he was fighting the old system) * add $HOME on the Solaris smartos builder, which was missing & causing tests to fail lately * make the (currently failing) longtest builder have GOPROXY set * remove an allocation in version.ParseReleaseBranch Fixes golang/go#9603 Updates golang/go#14594 Change-Id: I50a23ad7cdf478c95b14bee9b3931ba361baacfa Reviewed-on: https://go-review.googlesource.com/c/build/+/166218 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-03-07 20:44:41 +03:00
tryBot: func(repo, branch, goBranch string) bool {
dashboard: add android subrepo trybots too Missing from CL 172798 Updates golang/go#23824 Change-Id: I8f1b09ce8f52b555162b344ceddfbfe88e19ad4e Reviewed-on: https://go-review.googlesource.com/c/build/+/174079 Reviewed-by: Elias Naur <mail@eliasnaur.com> Run-TryBot: Elias Naur <mail@eliasnaur.com> TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-04-26 17:04:00 +03:00
switch repo {
case "go", "mobile", "sys", "net", "tools", "crypto", "sync", "text", "time":
return atLeastGo1(branch, 13) && atLeastGo1(goBranch, 13)
}
return false
cmd/coordinator: test x/mobile on android emu builders; and exp on some Updates golang/go#23824 Change-Id: Iee70bcbfef6a271fdd26694d848a703a200dc0c0 Reviewed-on: https://go-review.googlesource.com/c/164478 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
2019-02-28 20:32:45 +03:00
},
dashboard: unify post-submit & trybot policy in builders.go, not coordinator This removes a bunch of TODOs about scattered policy and moves it all into builders.go. While here, * add a bunch of tests * unexport some things * rename some things * document some things * adjust FreeBSD policy as function of branch (per Go 1.12 being last to support FreeBSD 10.x, and to unblock CL 165801) * set GO_BUILDER_SET_GOPROXY=coordinator for reverse buildlets, which is necessary to remove the oauth2 & build special cases in the config * change Elias Naur's mobile builder to how I think he wants it (he was fighting the old system) * add $HOME on the Solaris smartos builder, which was missing & causing tests to fail lately * make the (currently failing) longtest builder have GOPROXY set * remove an allocation in version.ParseReleaseBranch Fixes golang/go#9603 Updates golang/go#14594 Change-Id: I50a23ad7cdf478c95b14bee9b3931ba361baacfa Reviewed-on: https://go-review.googlesource.com/c/build/+/166218 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-03-07 20:44:41 +03:00
buildsRepo: func(repo, branch, goBranch string) bool {
dashboard: add a GOOS=linux builder & trybot for Android Updates golang/go#30482 Change-Id: I198dd40dfdb86b20e681c373141754680351e82f Reviewed-on: https://go-review.googlesource.com/c/164740 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Elias Naur <mail@eliasnaur.com>
2019-03-01 19:54:25 +03:00
switch repo {
dashboard: stop testing x/build on even more builders Follow up to CL 210378 after looking at: https://build.golang.org/?repo=golang.org%2fx%2fbuild Updates golang/go#30640 Change-Id: I81505a9ca2985ffa6e75922665b14ef4de2af860 Reviewed-on: https://go-review.googlesource.com/c/build/+/211100 Reviewed-by: Bryan C. Mills <bcmills@google.com> Run-TryBot: Bryan C. Mills <bcmills@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-12-12 20:43:59 +03:00
case "build", "blog", "talks", "review", "tour", "website":
dashboard: add a GOOS=linux builder & trybot for Android Updates golang/go#30482 Change-Id: I198dd40dfdb86b20e681c373141754680351e82f Reviewed-on: https://go-review.googlesource.com/c/164740 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Elias Naur <mail@eliasnaur.com>
2019-03-01 19:54:25 +03:00
return false
}
dashboard: unify post-submit & trybot policy in builders.go, not coordinator This removes a bunch of TODOs about scattered policy and moves it all into builders.go. While here, * add a bunch of tests * unexport some things * rename some things * document some things * adjust FreeBSD policy as function of branch (per Go 1.12 being last to support FreeBSD 10.x, and to unblock CL 165801) * set GO_BUILDER_SET_GOPROXY=coordinator for reverse buildlets, which is necessary to remove the oauth2 & build special cases in the config * change Elias Naur's mobile builder to how I think he wants it (he was fighting the old system) * add $HOME on the Solaris smartos builder, which was missing & causing tests to fail lately * make the (currently failing) longtest builder have GOPROXY set * remove an allocation in version.ParseReleaseBranch Fixes golang/go#9603 Updates golang/go#14594 Change-Id: I50a23ad7cdf478c95b14bee9b3931ba361baacfa Reviewed-on: https://go-review.googlesource.com/c/build/+/166218 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-03-07 20:44:41 +03:00
return atLeastGo1(branch, 13) && atLeastGo1(goBranch, 13)
dashboard: add a GOOS=linux builder & trybot for Android Updates golang/go#30482 Change-Id: I198dd40dfdb86b20e681c373141754680351e82f Reviewed-on: https://go-review.googlesource.com/c/164740 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Elias Naur <mail@eliasnaur.com>
2019-03-01 19:54:25 +03:00
},
env/android-amd64-emu: add new Android emulator container + VM image Now that we can do nested virtualization on GCE, that means we can run the Android emulator (which requires KVM) on GCE and at least get fast trybots and such for android-amd64. Updates golang/go#23824 Change-Id: I0da38c7fa0f15492230a31291d2921ba72f2151d Reviewed-on: https://go-review.googlesource.com/c/163738 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-02-25 23:35:29 +03:00
env: []string{
"GOARCH=amd64",
"GOOS=android",
"GOHOSTARCH=amd64",
"GOHOSTOS=linux",
"CGO_ENABLED=1",
},
})
dashboard: add interim illumos builder While the work to make illumos a first class GCE guest is completed, use this interim zone provided by an illumos community member to run illumos builds. Updates golang/go#15581 Change-Id: I1784847e5407894d01ce0aadf489b38d7e5c1924 Reviewed-on: https://go-review.googlesource.com/c/build/+/201597 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-10-17 11:48:13 +03:00
addBuilder(BuildConfig{
Name: "illumos-amd64",
HostType: "host-illumos-amd64-jclulow",
MinimumGoVersion: types.MajorMinor{1, 13},
})
x/build: add Solaris amd64 release builder to dashboard Fixes golang/go#20949 Change-Id: I4f85ee76783a1240a2ebd4a2a1f167ab849fcebe Reviewed-on: https://go-review.googlesource.com/47811 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2017-07-07 23:41:20 +03:00
addBuilder(BuildConfig{
Name: "solaris-amd64-oraclerel",
HostType: "host-solaris-oracle-amd64-oraclerel",
Notes: "Oracle Solaris release version",
})
all: add ppc64 (big endian) builders Change-Id: I0fddda33179555a50b6f03dcff13338544b742c3 Reviewed-on: https://go-review.googlesource.com/30599 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2016-10-06 23:30:58 +03:00
addBuilder(BuildConfig{
dashboard: refactor mechanism for adjusting cmd/dist test policy Previously, the behavior of ShouldRunDistTest could be overridden by an individual builder by setting a the shouldRunDistTest field to a custom policy implementation. A pitfall of this approach is that it requires each custom policy to reimplement the default cmd/dist test policy as well. If that isn't done when attempting to make a small adjustment to the default policy, the result is that the default policy is ignored. The fasterTrybots policy, as the name suggests, exists to skip some tests in trybot mode. Due to the problem above, it was causing some builders to run more tests in trybot mode than they would if it weren't applied. For example, the freebsd-amd64-12_0 builder should have been skipping the "api" test in trybot mode, but wasn't. This refactor fixes that by modifying the approach for how individual builders can customize the cmd/dist test behavior. Now, a function distTestAdjust is called with the first parameter specifying the default policy decision. That function can adjust the decision as it sees fit, but otherwise it preserves its initial value in situations where no change is intended to be made. For golang/go#38279. Change-Id: I220248f51abfc8e77195ae0c9134a17f7cac9bc2 Reviewed-on: https://go-review.googlesource.com/c/build/+/227777 Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-04-09 19:47:03 +03:00
Name: "linux-ppc64-buildlet",
HostType: "host-linux-ppc64-osu",
FlakyNet: true,
distTestAdjust: ppc64DistTestPolicy,
dashboard: increase GO_TEST_TIMEOUT_SCALE for ppc64 builders This sets the GO_TEST_TIMEOUT_SCALE=2 for ppc64 builders. The runtime tests have been timing out intermittently on ppc64le power8; it has happened on ppc64 and ppc64le power9 but very rarely. Updates golang/go#44422 Change-Id: I663f3f211a368a59e38fbff9ce43c925c6c7a209 Reviewed-on: https://go-review.googlesource.com/c/build/+/300870 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Trust: Lynn Boger <laboger@linux.vnet.ibm.com>
2021-03-11 22:00:33 +03:00
env: []string{"GO_TEST_TIMEOUT_SCALE=2"}, // see golang.org/issues/44422
all: add ppc64 (big endian) builders Change-Id: I0fddda33179555a50b6f03dcff13338544b742c3 Reviewed-on: https://go-review.googlesource.com/30599 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2016-10-06 23:30:58 +03:00
})
dashboard: add solaris, ppc64le, s390x reverse buildlets Change-Id: I5061ae1da7884ed82786e8a042e966ccbe423e89 Reviewed-on: https://go-review.googlesource.com/20135 Reviewed-by: Andrew Gerrand <adg@golang.org>
2016-03-01 07:49:24 +03:00
addBuilder(BuildConfig{
dashboard: refactor mechanism for adjusting cmd/dist test policy Previously, the behavior of ShouldRunDistTest could be overridden by an individual builder by setting a the shouldRunDistTest field to a custom policy implementation. A pitfall of this approach is that it requires each custom policy to reimplement the default cmd/dist test policy as well. If that isn't done when attempting to make a small adjustment to the default policy, the result is that the default policy is ignored. The fasterTrybots policy, as the name suggests, exists to skip some tests in trybot mode. Due to the problem above, it was causing some builders to run more tests in trybot mode than they would if it weren't applied. For example, the freebsd-amd64-12_0 builder should have been skipping the "api" test in trybot mode, but wasn't. This refactor fixes that by modifying the approach for how individual builders can customize the cmd/dist test behavior. Now, a function distTestAdjust is called with the first parameter specifying the default policy decision. That function can adjust the decision as it sees fit, but otherwise it preserves its initial value in situations where no change is intended to be made. For golang/go#38279. Change-Id: I220248f51abfc8e77195ae0c9134a17f7cac9bc2 Reviewed-on: https://go-review.googlesource.com/c/build/+/227777 Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-04-09 19:47:03 +03:00
Name: "linux-ppc64le-buildlet",
HostType: "host-linux-ppc64le-osu",
FlakyNet: true,
distTestAdjust: ppc64DistTestPolicy,
dashboard: increase GO_TEST_TIMEOUT_SCALE for ppc64 builders This sets the GO_TEST_TIMEOUT_SCALE=2 for ppc64 builders. The runtime tests have been timing out intermittently on ppc64le power8; it has happened on ppc64 and ppc64le power9 but very rarely. Updates golang/go#44422 Change-Id: I663f3f211a368a59e38fbff9ce43c925c6c7a209 Reviewed-on: https://go-review.googlesource.com/c/build/+/300870 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Trust: Lynn Boger <laboger@linux.vnet.ibm.com>
2021-03-11 22:00:33 +03:00
env: []string{"GO_TEST_TIMEOUT_SCALE=2"}, // see golang.org/issues/44422
dashboard: add arm64 buildlet, update ppc64 buildlet definition Change-Id: I25c6de0f97c3338f9d32f4125b2c7c68c3f3c13c Reviewed-on: https://go-review.googlesource.com/21656 Reviewed-by: David Crawshaw <crawshaw@golang.org>
2016-04-07 22:27:15 +03:00
})
dashboard: add linux-ppc64le POWER9 builder This change adds an extra linux-ppc64le builder running on POWER9. Fixes golang/go#30138 Change-Id: I23bdfc116bb89c6d96be330c2c87827c47705b6b Reviewed-on: https://go-review.googlesource.com/c/163757 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-02-25 23:05:12 +03:00
addBuilder(BuildConfig{
dashboard: refactor mechanism for adjusting cmd/dist test policy Previously, the behavior of ShouldRunDistTest could be overridden by an individual builder by setting a the shouldRunDistTest field to a custom policy implementation. A pitfall of this approach is that it requires each custom policy to reimplement the default cmd/dist test policy as well. If that isn't done when attempting to make a small adjustment to the default policy, the result is that the default policy is ignored. The fasterTrybots policy, as the name suggests, exists to skip some tests in trybot mode. Due to the problem above, it was causing some builders to run more tests in trybot mode than they would if it weren't applied. For example, the freebsd-amd64-12_0 builder should have been skipping the "api" test in trybot mode, but wasn't. This refactor fixes that by modifying the approach for how individual builders can customize the cmd/dist test behavior. Now, a function distTestAdjust is called with the first parameter specifying the default policy decision. That function can adjust the decision as it sees fit, but otherwise it preserves its initial value in situations where no change is intended to be made. For golang/go#38279. Change-Id: I220248f51abfc8e77195ae0c9134a17f7cac9bc2 Reviewed-on: https://go-review.googlesource.com/c/build/+/227777 Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-04-09 19:47:03 +03:00
Name: "linux-ppc64le-power9osu",
HostType: "host-linux-ppc64le-power9-osu",
FlakyNet: true,
distTestAdjust: ppc64DistTestPolicy,
dashboard: increase GO_TEST_TIMEOUT_SCALE for ppc64 builders This sets the GO_TEST_TIMEOUT_SCALE=2 for ppc64 builders. The runtime tests have been timing out intermittently on ppc64le power8; it has happened on ppc64 and ppc64le power9 but very rarely. Updates golang/go#44422 Change-Id: I663f3f211a368a59e38fbff9ce43c925c6c7a209 Reviewed-on: https://go-review.googlesource.com/c/build/+/300870 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Trust: Lynn Boger <laboger@linux.vnet.ibm.com>
2021-03-11 22:00:33 +03:00
env: []string{"GO_TEST_TIMEOUT_SCALE=2"}, // see golang.org/issues/44422
dashboard: add linux-ppc64le POWER9 builder This change adds an extra linux-ppc64le builder running on POWER9. Fixes golang/go#30138 Change-Id: I23bdfc116bb89c6d96be330c2c87827c47705b6b Reviewed-on: https://go-review.googlesource.com/c/163757 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-02-25 23:05:12 +03:00
})
build: update NetBSD from 7.0 to 7.1, disable builds, add arm64 packet builder Change-Id: I73b8d6e6ccaeca3b932238448ddcceead34105b4 Reviewed-on: https://go-review.googlesource.com/39851 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2017-04-07 01:16:28 +03:00
addBuilder(BuildConfig{
Name: "linux-arm64-packet",
HostType: "host-linux-arm64-packet",
all: remove config for arm64 linaro builders They've been decomissioned for some months now. Our arm64 builders are on packet.net these days. Change-Id: Iaede189157e0298812b2971cdd63599f6096d73d Reviewed-on: https://go-review.googlesource.com/c/build/+/165781 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-03-07 08:16:49 +03:00
FlakyNet: true, // maybe not flaky, but here conservatively
build: update NetBSD from 7.0 to 7.1, disable builds, add arm64 packet builder Change-Id: I73b8d6e6ccaeca3b932238448ddcceead34105b4 Reviewed-on: https://go-review.googlesource.com/39851 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2017-04-07 01:16:28 +03:00
})
dashboard: add host-linux-arm64-aws builder Add the host-linux-arm64-aws builder. The HostConfig IsVM method has been modified to support the concept that an EC2 VM can be created which runs the buildlet binary in a container. This configuration would have both a VM image and a container image set in the host configuration. For golang/go#36841 Change-Id: I092b48b1dda24d24bcd0b10dff0b2126f5f43017 Reviewed-on: https://go-review.googlesource.com/c/build/+/249118 Run-TryBot: Carlos Amedee <carlos@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2020-08-18 20:32:05 +03:00
addBuilder(BuildConfig{
dashboard: add linux-arm64-aws to set of trybots This change adds linux-arm64-aws to the set of trybots. It also removes linux-arm and linux-arm64 from the list of builds attempted in the misc-compile-linuxarm builder. Updates golang/go#36170 Fixes golang/go#45065 Change-Id: If25fed08e35b2a91c9c9dbbf31701ff5464dc913 Reviewed-on: https://go-review.googlesource.com/c/build/+/303232 Trust: Carlos Amedee <carlos@golang.org> Run-TryBot: Carlos Amedee <carlos@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2021-03-19 21:53:06 +03:00
Name: "linux-arm64-aws",
HostType: "host-linux-arm64-aws",
tryBot: defaultTrySet(),
numTryTestHelpers: 1,
dashboard: add host-linux-arm64-aws builder Add the host-linux-arm64-aws builder. The HostConfig IsVM method has been modified to support the concept that an EC2 VM can be created which runs the buildlet binary in a container. This configuration would have both a VM image and a container image set in the host configuration. For golang/go#36841 Change-Id: I092b48b1dda24d24bcd0b10dff0b2126f5f43017 Reviewed-on: https://go-review.googlesource.com/c/build/+/249118 Run-TryBot: Carlos Amedee <carlos@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2020-08-18 20:32:05 +03:00
})
env/linux-arm/aws, dashboard: add linux-arm-aws builder This change adds the linux-arm-aws builder. This builder will be started on a EC2 instance which will spawn a single instance of the container. After the builder work is done, the instance will be destroyed. Updates golang/go#41867 Change-Id: I09d911c2f99d1dff8fe02feb458457e07e7d0cdd Reviewed-on: https://go-review.googlesource.com/c/build/+/260797 Trust: Carlos Amedee <carlos@golang.org> Run-TryBot: Carlos Amedee <carlos@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-10-08 23:49:48 +03:00
addBuilder(BuildConfig{
dashboard: add linux-arm-aws to trybots This change adds linux-arm-aws to the set of trybots. The related changes include: - Renaming the linux-arm builder to linux-arm-scaleway. - Adding slowbot aliases which use linux-arm-aws as the default builder for linux-arm builds. Fixes golang/go#35628 Fixes golang/go#40872 Updates golang/go#45064 Change-Id: Ib49f50e03fc4f766015fdd78080b804c2784a398 Reviewed-on: https://go-review.googlesource.com/c/build/+/303230 Trust: Carlos Amedee <carlos@golang.org> Run-TryBot: Carlos Amedee <carlos@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2021-03-19 20:54:17 +03:00
Name: "linux-arm-aws",
HostType: "host-linux-arm-aws",
tryBot: defaultTrySet(),
numTryTestHelpers: 1,
dashboard: set GO{,HOST}ARCH, GOARM on linux-arm-aws builder This change modifies the environmental variables for the linux-arm-aws builder. It standardizes the environment that the linux/arm is tested and released on. Fixes golang/go#47554 Change-Id: Ie96ad93be20434636f29ab9c778fbed12fdb8d82 Reviewed-on: https://go-review.googlesource.com/c/build/+/340112 Trust: Carlos Amedee <carlos@golang.org> Run-TryBot: Carlos Amedee <carlos@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2021-08-05 18:02:29 +03:00
env: []string{
"GOARCH=arm",
"GOARM=6",
"GOHOSTARCH=arm",
},
env/linux-arm/aws, dashboard: add linux-arm-aws builder This change adds the linux-arm-aws builder. This builder will be started on a EC2 instance which will spawn a single instance of the container. After the builder work is done, the instance will be destroyed. Updates golang/go#41867 Change-Id: I09d911c2f99d1dff8fe02feb458457e07e7d0cdd Reviewed-on: https://go-review.googlesource.com/c/build/+/260797 Trust: Carlos Amedee <carlos@golang.org> Run-TryBot: Carlos Amedee <carlos@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-10-08 23:49:48 +03:00
})
dashboard: add linux-loong64-3a5000 builder Updates golang/go#46229 Change-Id: Ib48e1e64d96a07d5c5282d78935124faf75c89d7 Reviewed-on: https://go-review.googlesource.com/c/build/+/347309 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org> Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Trust: Cherry Mui <cherryyz@google.com>
2021-09-02 10:51:28 +03:00
addBuilder(BuildConfig{
HostType: "host-linux-loong64-3a5000",
Name: "linux-loong64-3a5000",
SkipSnapshot: true,
distTestAdjust: loong64DistTestPolicy,
buildsRepo: loong64BuildsRepoPolicy,
env: []string{
"GOARCH=loong64",
"GOHOSTARCH=loong64",
},
KnownIssue: 46229,
})
dashboard: add linux-mipsle-mengzhuo builder Updates golang/go#31217 Change-Id: I7bda4f3ddf7dc76bdd974726747621f69a7acfa0 Reviewed-on: https://go-review.googlesource.com/c/build/+/191577 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-09-10 02:21:11 +03:00
addBuilder(BuildConfig{
dashboard: refactor mechanism for adjusting cmd/dist test policy Previously, the behavior of ShouldRunDistTest could be overridden by an individual builder by setting a the shouldRunDistTest field to a custom policy implementation. A pitfall of this approach is that it requires each custom policy to reimplement the default cmd/dist test policy as well. If that isn't done when attempting to make a small adjustment to the default policy, the result is that the default policy is ignored. The fasterTrybots policy, as the name suggests, exists to skip some tests in trybot mode. Due to the problem above, it was causing some builders to run more tests in trybot mode than they would if it weren't applied. For example, the freebsd-amd64-12_0 builder should have been skipping the "api" test in trybot mode, but wasn't. This refactor fixes that by modifying the approach for how individual builders can customize the cmd/dist test behavior. Now, a function distTestAdjust is called with the first parameter specifying the default policy decision. That function can adjust the decision as it sees fit, but otherwise it preserves its initial value in situations where no change is intended to be made. For golang/go#38279. Change-Id: I220248f51abfc8e77195ae0c9134a17f7cac9bc2 Reviewed-on: https://go-review.googlesource.com/c/build/+/227777 Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-04-09 19:47:03 +03:00
FlakyNet: true,
HostType: "host-linux-mipsle-mengzhuo",
Name: "linux-mips64le-mengzhuo",
SkipSnapshot: true,
distTestAdjust: mipsDistTestPolicy,
buildsRepo: mipsBuildsRepoPolicy,
dashboard: add MIPS builders at RT-RK Updates golang/go#31217 Change-Id: I3c947fa1dcd0f1d938ffa4dfe565a35b2ccc3533 Reviewed-on: https://go-review.googlesource.com/c/build/+/204043 Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-10-29 23:35:07 +03:00
})
addBuilder(BuildConfig{
dashboard: refactor mechanism for adjusting cmd/dist test policy Previously, the behavior of ShouldRunDistTest could be overridden by an individual builder by setting a the shouldRunDistTest field to a custom policy implementation. A pitfall of this approach is that it requires each custom policy to reimplement the default cmd/dist test policy as well. If that isn't done when attempting to make a small adjustment to the default policy, the result is that the default policy is ignored. The fasterTrybots policy, as the name suggests, exists to skip some tests in trybot mode. Due to the problem above, it was causing some builders to run more tests in trybot mode than they would if it weren't applied. For example, the freebsd-amd64-12_0 builder should have been skipping the "api" test in trybot mode, but wasn't. This refactor fixes that by modifying the approach for how individual builders can customize the cmd/dist test behavior. Now, a function distTestAdjust is called with the first parameter specifying the default policy decision. That function can adjust the decision as it sees fit, but otherwise it preserves its initial value in situations where no change is intended to be made. For golang/go#38279. Change-Id: I220248f51abfc8e77195ae0c9134a17f7cac9bc2 Reviewed-on: https://go-review.googlesource.com/c/build/+/227777 Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-04-09 19:47:03 +03:00
FlakyNet: true,
HostType: "host-linux-mips64le-rtrk",
Name: "linux-mips64le-rtrk",
SkipSnapshot: true,
distTestAdjust: mipsDistTestPolicy,
buildsRepo: mipsBuildsRepoPolicy,
dashboard: add MIPS builders at RT-RK Updates golang/go#31217 Change-Id: I3c947fa1dcd0f1d938ffa4dfe565a35b2ccc3533 Reviewed-on: https://go-review.googlesource.com/c/build/+/204043 Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-10-29 23:35:07 +03:00
env: []string{
"GOARCH=mips64le",
"GOHOSTARCH=mips64le",
},
})
addBuilder(BuildConfig{
dashboard: refactor mechanism for adjusting cmd/dist test policy Previously, the behavior of ShouldRunDistTest could be overridden by an individual builder by setting a the shouldRunDistTest field to a custom policy implementation. A pitfall of this approach is that it requires each custom policy to reimplement the default cmd/dist test policy as well. If that isn't done when attempting to make a small adjustment to the default policy, the result is that the default policy is ignored. The fasterTrybots policy, as the name suggests, exists to skip some tests in trybot mode. Due to the problem above, it was causing some builders to run more tests in trybot mode than they would if it weren't applied. For example, the freebsd-amd64-12_0 builder should have been skipping the "api" test in trybot mode, but wasn't. This refactor fixes that by modifying the approach for how individual builders can customize the cmd/dist test behavior. Now, a function distTestAdjust is called with the first parameter specifying the default policy decision. That function can adjust the decision as it sees fit, but otherwise it preserves its initial value in situations where no change is intended to be made. For golang/go#38279. Change-Id: I220248f51abfc8e77195ae0c9134a17f7cac9bc2 Reviewed-on: https://go-review.googlesource.com/c/build/+/227777 Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-04-09 19:47:03 +03:00
FlakyNet: true,
HostType: "host-linux-mips64le-rtrk",
Name: "linux-mipsle-rtrk",
SkipSnapshot: true,
distTestAdjust: mipsDistTestPolicy,
buildsRepo: mipsBuildsRepoPolicy,
dashboard: add MIPS builders at RT-RK Updates golang/go#31217 Change-Id: I3c947fa1dcd0f1d938ffa4dfe565a35b2ccc3533 Reviewed-on: https://go-review.googlesource.com/c/build/+/204043 Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-10-29 23:35:07 +03:00
env: []string{
"GOARCH=mipsle",
"GOHOSTARCH=mipsle",
},
})
addBuilder(BuildConfig{
dashboard: refactor mechanism for adjusting cmd/dist test policy Previously, the behavior of ShouldRunDistTest could be overridden by an individual builder by setting a the shouldRunDistTest field to a custom policy implementation. A pitfall of this approach is that it requires each custom policy to reimplement the default cmd/dist test policy as well. If that isn't done when attempting to make a small adjustment to the default policy, the result is that the default policy is ignored. The fasterTrybots policy, as the name suggests, exists to skip some tests in trybot mode. Due to the problem above, it was causing some builders to run more tests in trybot mode than they would if it weren't applied. For example, the freebsd-amd64-12_0 builder should have been skipping the "api" test in trybot mode, but wasn't. This refactor fixes that by modifying the approach for how individual builders can customize the cmd/dist test behavior. Now, a function distTestAdjust is called with the first parameter specifying the default policy decision. That function can adjust the decision as it sees fit, but otherwise it preserves its initial value in situations where no change is intended to be made. For golang/go#38279. Change-Id: I220248f51abfc8e77195ae0c9134a17f7cac9bc2 Reviewed-on: https://go-review.googlesource.com/c/build/+/227777 Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-04-09 19:47:03 +03:00
FlakyNet: true,
HostType: "host-linux-mips64-rtrk",
Name: "linux-mips64-rtrk",
SkipSnapshot: true,
distTestAdjust: mipsDistTestPolicy,
buildsRepo: mipsBuildsRepoPolicy,
dashboard: add MIPS builders at RT-RK Updates golang/go#31217 Change-Id: I3c947fa1dcd0f1d938ffa4dfe565a35b2ccc3533 Reviewed-on: https://go-review.googlesource.com/c/build/+/204043 Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-10-29 23:35:07 +03:00
env: []string{
"GOARCH=mips64",
"GOHOSTARCH=mips64",
},
})
addBuilder(BuildConfig{
dashboard: refactor mechanism for adjusting cmd/dist test policy Previously, the behavior of ShouldRunDistTest could be overridden by an individual builder by setting a the shouldRunDistTest field to a custom policy implementation. A pitfall of this approach is that it requires each custom policy to reimplement the default cmd/dist test policy as well. If that isn't done when attempting to make a small adjustment to the default policy, the result is that the default policy is ignored. The fasterTrybots policy, as the name suggests, exists to skip some tests in trybot mode. Due to the problem above, it was causing some builders to run more tests in trybot mode than they would if it weren't applied. For example, the freebsd-amd64-12_0 builder should have been skipping the "api" test in trybot mode, but wasn't. This refactor fixes that by modifying the approach for how individual builders can customize the cmd/dist test behavior. Now, a function distTestAdjust is called with the first parameter specifying the default policy decision. That function can adjust the decision as it sees fit, but otherwise it preserves its initial value in situations where no change is intended to be made. For golang/go#38279. Change-Id: I220248f51abfc8e77195ae0c9134a17f7cac9bc2 Reviewed-on: https://go-review.googlesource.com/c/build/+/227777 Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-04-09 19:47:03 +03:00
FlakyNet: true,
HostType: "host-linux-mips64-rtrk",
Name: "linux-mips-rtrk",
SkipSnapshot: true,
distTestAdjust: mipsDistTestPolicy,
buildsRepo: mipsBuildsRepoPolicy,
dashboard: add MIPS builders at RT-RK Updates golang/go#31217 Change-Id: I3c947fa1dcd0f1d938ffa4dfe565a35b2ccc3533 Reviewed-on: https://go-review.googlesource.com/c/build/+/204043 Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-10-29 23:35:07 +03:00
env: []string{
"GOARCH=mips",
"GOHOSTARCH=mips",
dashboard: add linux-mipsle-mengzhuo builder Updates golang/go#31217 Change-Id: I7bda4f3ddf7dc76bdd974726747621f69a7acfa0 Reviewed-on: https://go-review.googlesource.com/c/build/+/191577 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-09-10 02:21:11 +03:00
},
dashboard: add linux-riscv64-jsing builder (reverse) Also use this as the riscv64 slowbot, since linux-riscv64-unleashed has been MIA for quite some time. For golang/go#27532 Change-Id: I01309a66b96d4c35dbc9d1e08f55f5357b0acba0 Reviewed-on: https://go-review.googlesource.com/c/build/+/257917 Trust: Joel Sing <joel@sing.id.au> Run-TryBot: Joel Sing <joel@sing.id.au> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2020-09-27 21:25:41 +03:00
})
addBuilder(BuildConfig{
dashboard: disable api/reboot test for unmatched It takes >15m to complete api and reboot tests for Hifive unmatched which we don't need. Change-Id: Ie4535264d2e8ef4496c54fb5f412d5ec773f538a Reviewed-on: https://go-review.googlesource.com/c/build/+/346709 Trust: Meng Zhuo <mzh@golangcn.org> Run-TryBot: Meng Zhuo <mzh@golangcn.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Go Bot <gobot@golang.org>
2021-09-01 10:08:03 +03:00
HostType: "host-linux-riscv64-joelsing",
Name: "linux-riscv64-jsing",
SkipSnapshot: true,
env: []string{"GO_TEST_TIMEOUT_SCALE=4"},
distTestAdjust: riscvDistTestPolicy,
dashboard: add linux-riscv64-jsing builder (reverse) Also use this as the riscv64 slowbot, since linux-riscv64-unleashed has been MIA for quite some time. For golang/go#27532 Change-Id: I01309a66b96d4c35dbc9d1e08f55f5357b0acba0 Reviewed-on: https://go-review.googlesource.com/c/build/+/257917 Trust: Joel Sing <joel@sing.id.au> Run-TryBot: Joel Sing <joel@sing.id.au> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2020-09-27 21:25:41 +03:00
buildsRepo: func(repo, branch, goBranch string) bool {
switch repo {
case "go", "net", "sys":
return branch == "master" && goBranch == "master"
default:
return false
}
},
dashboard: add linux-mipsle-mengzhuo builder Updates golang/go#31217 Change-Id: I7bda4f3ddf7dc76bdd974726747621f69a7acfa0 Reviewed-on: https://go-review.googlesource.com/c/build/+/191577 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-09-10 02:21:11 +03:00
})
dashboard, cmd/coordinator: add linux-riscv64 builder, fix bug from CL 205608 Updates golang/go#27532 Updates golang/go#30262 Change-Id: If1593ab6e5653a9dce625f25a16bf8a082796592 Reviewed-on: https://go-review.googlesource.com/c/build/+/206639 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-11-12 09:19:21 +03:00
addBuilder(BuildConfig{
dashboard: disable api/reboot test for unmatched It takes >15m to complete api and reboot tests for Hifive unmatched which we don't need. Change-Id: Ie4535264d2e8ef4496c54fb5f412d5ec773f538a Reviewed-on: https://go-review.googlesource.com/c/build/+/346709 Trust: Meng Zhuo <mzh@golangcn.org> Run-TryBot: Meng Zhuo <mzh@golangcn.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Go Bot <gobot@golang.org>
2021-09-01 10:08:03 +03:00
HostType: "host-linux-riscv64-unleashed",
Name: "linux-riscv64-unleashed",
SkipSnapshot: true,
env: []string{"GO_TEST_TIMEOUT_SCALE=4"},
distTestAdjust: riscvDistTestPolicy,
dashboard, cmd/coordinator: add linux-riscv64 builder, fix bug from CL 205608 Updates golang/go#27532 Updates golang/go#30262 Change-Id: If1593ab6e5653a9dce625f25a16bf8a082796592 Reviewed-on: https://go-review.googlesource.com/c/build/+/206639 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-11-12 09:19:21 +03:00
buildsRepo: func(repo, branch, goBranch string) bool {
switch repo {
case "go", "net", "sys":
return branch == "master" && goBranch == "master"
default:
return false
}
},
})
dashboard: add linux-riscv64-unmatched This adds a reverse buildlet for linux riscv64 running under Hifive Unmatched. For golang/go#46804. Change-Id: If5ed8b47205cecebcb90a4d1b03c463e1eb466c7 Reviewed-on: https://go-review.googlesource.com/c/build/+/329489 Trust: Meng Zhuo <mzh@golangcn.org> Run-TryBot: Meng Zhuo <mzh@golangcn.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2021-08-27 07:26:14 +03:00
addBuilder(BuildConfig{
dashboard: disable api/reboot test for unmatched It takes >15m to complete api and reboot tests for Hifive unmatched which we don't need. Change-Id: Ie4535264d2e8ef4496c54fb5f412d5ec773f538a Reviewed-on: https://go-review.googlesource.com/c/build/+/346709 Trust: Meng Zhuo <mzh@golangcn.org> Run-TryBot: Meng Zhuo <mzh@golangcn.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Go Bot <gobot@golang.org>
2021-09-01 10:08:03 +03:00
HostType: "host-linux-riscv64-unmatched",
Name: "linux-riscv64-unmatched",
env: []string{"GO_TEST_TIMEOUT_SCALE=4"},
FlakyNet: true,
buildsRepo: onlyMasterDefault,
distTestAdjust: riscvDistTestPolicy,
dashboard: add linux-riscv64-unmatched This adds a reverse buildlet for linux riscv64 running under Hifive Unmatched. For golang/go#46804. Change-Id: If5ed8b47205cecebcb90a4d1b03c463e1eb466c7 Reviewed-on: https://go-review.googlesource.com/c/build/+/329489 Trust: Meng Zhuo <mzh@golangcn.org> Run-TryBot: Meng Zhuo <mzh@golangcn.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2021-08-27 07:26:14 +03:00
})
dashboard: add solaris, ppc64le, s390x reverse buildlets Change-Id: I5061ae1da7884ed82786e8a042e966ccbe423e89 Reviewed-on: https://go-review.googlesource.com/20135 Reviewed-by: Andrew Gerrand <adg@golang.org>
2016-03-01 07:49:24 +03:00
addBuilder(BuildConfig{
Name: "linux-s390x-ibm",
all: split builder config into builder & host configs Our builders are named of the form "GOOS-GOARCH" or "GOOS-GOARCH-suffix". Over time we've grown many builders. This CL doesn't change that. Builders continue to be named and operate as before. Previously the build configuration file (dashboard/builders.go) made each builder type ("linux-amd64-race", etc) define how to create a host running a buildlet of that type, even though many builders had identical host configs. For example, these builders all share the same host type (a Kubernetes container): linux-amd64 linux-amd64-race linux-386 linux-386-387 And these are the same host type (a GCE VM): windows-amd64-gce windows-amd64-race windows-386-gce This CL creates a new concept of a "hostType" which defines how the buildlet is created (Kube, GCE, Reverse, and how), and then each builder itself references a host type. Users never see the hostType. (except perhaps in gomote list output) But they at least never need to care about them. Reverse buildlets now can only be one hostType at a time, which simplifies things. We were no longer using multiple roles per machine once moving to VMs for OS X. gomote continues to operate as it did previously but its underlying protocol changed and clients will need to be updated. As a new feature, gomote now has a new flag to let you reuse a buildlet host connection for different builder rules if they share the same underlying host type. But users can ignore that. This CL is a long-standing TODO (previously attempted and aborted) and will make many things easier and faster, including the linux-arm cross-compilation effort, and keeping pre-warmed buildlets of VM types ready to go. Updates golang/go#17104 Change-Id: Iad8387f48680424a8441e878a2f4762bf79ea4d2 Reviewed-on: https://go-review.googlesource.com/29551 Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2016-09-22 00:27:37 +03:00
HostType: "host-linux-s390x",
cmd/coordinator, dashboard: remove some trybots, shard others wider I'm aiming to have trybot runs finish in under 5 minutes. This CL removes openbsd-386-gce58 and freebsd-386-gce101 from the trybot set. openbsd-386-gce58 is the slowest builder. It has an average speed of 722 seconds (and 95 percentile of 923 seconds) over the past week, and that's sharded over 4 machines. Too slow. It's not worth the resources to keep it as a trybot. It hasn't caught any interesting bugs. This builder will still run, but not as a pre-submit trybot. freebsd-386-gce101 is not slow, but we're removing it to shift its resources to shard other builders wider. The coordinator now supports varying the build sharding width based on whether a build is for a trybot or not. This CL defines separate numbers for each, sharding builds wider as needed for some trybots. freebsd-amd64-gce101 goes from 4 to 5 machines in try runs, and down to 3 when not in try runs. linux-amd64-race gets one more machine during try runs, and one fewer in regular runs. linux-arm goes from 7 machines always, to 3 or 8, depending on whether it's a try run. openbsd-amd64-58 goes from 4 to 3 or 6. windows-amd64-gce goes from 4 to 2 or 6. windows-amd64-race goes from 4 to 2 or 6. darwin-amd64-10_11 goes from 3 to 3 or 4. I'll see how these do over the next few days and readjust as needed. Also in this CL: fix the constants for the expected duration of make.bash, which impact when we schedule the creation of test sharding helper buildlets. We were creating them too early before, wasting resources. Change-Id: I38a9b24841e196f1eb668de058c49af8c1d1c64f Reviewed-on: https://go-review.googlesource.com/29116 Reviewed-by: Quentin Smith <quentin@golang.org>
2016-09-14 01:45:48 +03:00
numTestHelpers: 0,
dashboard: add solaris, ppc64le, s390x reverse buildlets Change-Id: I5061ae1da7884ed82786e8a042e966ccbe423e89 Reviewed-on: https://go-review.googlesource.com/20135 Reviewed-by: Andrew Gerrand <adg@golang.org>
2016-03-01 07:49:24 +03:00
})
dashboard: add s390x cross-compile-only builder and release config Also modifies cmd/release to wire it up. Updates golang/go#16443 Change-Id: Ie80a224957f6f662d0aec0fd083e2a57e30436c4 Reviewed-on: https://go-review.googlesource.com/28172 Reviewed-by: Jessica Frazelle <me@jessfraz.com> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2016-08-30 23:32:33 +03:00
addBuilder(BuildConfig{
all: split builder config into builder & host configs Our builders are named of the form "GOOS-GOARCH" or "GOOS-GOARCH-suffix". Over time we've grown many builders. This CL doesn't change that. Builders continue to be named and operate as before. Previously the build configuration file (dashboard/builders.go) made each builder type ("linux-amd64-race", etc) define how to create a host running a buildlet of that type, even though many builders had identical host configs. For example, these builders all share the same host type (a Kubernetes container): linux-amd64 linux-amd64-race linux-386 linux-386-387 And these are the same host type (a GCE VM): windows-amd64-gce windows-amd64-race windows-386-gce This CL creates a new concept of a "hostType" which defines how the buildlet is created (Kube, GCE, Reverse, and how), and then each builder itself references a host type. Users never see the hostType. (except perhaps in gomote list output) But they at least never need to care about them. Reverse buildlets now can only be one hostType at a time, which simplifies things. We were no longer using multiple roles per machine once moving to VMs for OS X. gomote continues to operate as it did previously but its underlying protocol changed and clients will need to be updated. As a new feature, gomote now has a new flag to let you reuse a buildlet host connection for different builder rules if they share the same underlying host type. But users can ignore that. This CL is a long-standing TODO (previously attempted and aborted) and will make many things easier and faster, including the linux-arm cross-compilation effort, and keeping pre-warmed buildlets of VM types ready to go. Updates golang/go#17104 Change-Id: Iad8387f48680424a8441e878a2f4762bf79ea4d2 Reviewed-on: https://go-review.googlesource.com/29551 Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2016-09-22 00:27:37 +03:00
Name: "linux-s390x-crosscompile",
dashboard: remove misleading -kube suffixes from host types They used to run on Kubernetes (and still could) but they're just a container and we run them on COS nowadays. Change-Id: I21a6db90efc5071997c6d6dcae1a42a7c28afa3a Reviewed-on: https://go-review.googlesource.com/c/build/+/209957 Reviewed-by: Alexander Rakoczy <alex@golang.org> Run-TryBot: Alexander Rakoczy <alex@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-12-05 00:27:28 +03:00
HostType: "host-s390x-cross",
all: split builder config into builder & host configs Our builders are named of the form "GOOS-GOARCH" or "GOOS-GOARCH-suffix". Over time we've grown many builders. This CL doesn't change that. Builders continue to be named and operate as before. Previously the build configuration file (dashboard/builders.go) made each builder type ("linux-amd64-race", etc) define how to create a host running a buildlet of that type, even though many builders had identical host configs. For example, these builders all share the same host type (a Kubernetes container): linux-amd64 linux-amd64-race linux-386 linux-386-387 And these are the same host type (a GCE VM): windows-amd64-gce windows-amd64-race windows-386-gce This CL creates a new concept of a "hostType" which defines how the buildlet is created (Kube, GCE, Reverse, and how), and then each builder itself references a host type. Users never see the hostType. (except perhaps in gomote list output) But they at least never need to care about them. Reverse buildlets now can only be one hostType at a time, which simplifies things. We were no longer using multiple roles per machine once moving to VMs for OS X. gomote continues to operate as it did previously but its underlying protocol changed and clients will need to be updated. As a new feature, gomote now has a new flag to let you reuse a buildlet host connection for different builder rules if they share the same underlying host type. But users can ignore that. This CL is a long-standing TODO (previously attempted and aborted) and will make many things easier and faster, including the linux-arm cross-compilation effort, and keeping pre-warmed buildlets of VM types ready to go. Updates golang/go#17104 Change-Id: Iad8387f48680424a8441e878a2f4762bf79ea4d2 Reviewed-on: https://go-review.googlesource.com/29551 Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2016-09-22 00:27:37 +03:00
Notes: "s390x cross-compile builder for releases; doesn't run tests",
CompileOnly: true,
dashboard: unify post-submit & trybot policy in builders.go, not coordinator This removes a bunch of TODOs about scattered policy and moves it all into builders.go. While here, * add a bunch of tests * unexport some things * rename some things * document some things * adjust FreeBSD policy as function of branch (per Go 1.12 being last to support FreeBSD 10.x, and to unblock CL 165801) * set GO_BUILDER_SET_GOPROXY=coordinator for reverse buildlets, which is necessary to remove the oauth2 & build special cases in the config * change Elias Naur's mobile builder to how I think he wants it (he was fighting the old system) * add $HOME on the Solaris smartos builder, which was missing & causing tests to fail lately * make the (currently failing) longtest builder have GOPROXY set * remove an allocation in version.ParseReleaseBranch Fixes golang/go#9603 Updates golang/go#14594 Change-Id: I50a23ad7cdf478c95b14bee9b3931ba361baacfa Reviewed-on: https://go-review.googlesource.com/c/build/+/166218 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-03-07 20:44:41 +03:00
tryOnly: true, // but not in trybot set for now
dashboard: add s390x cross-compile-only builder and release config Also modifies cmd/release to wire it up. Updates golang/go#16443 Change-Id: Ie80a224957f6f662d0aec0fd083e2a57e30436c4 Reviewed-on: https://go-review.googlesource.com/28172 Reviewed-by: Jessica Frazelle <me@jessfraz.com> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2016-08-30 23:32:33 +03:00
env: []string{
"CGO_ENABLED=1",
"GOARCH=s390x",
"GOHOSTARCH=amd64",
"CC_FOR_TARGET=s390x-linux-gnu-gcc",
},
})
cmd/coordinator, cmd/buildlet, cmd/gomote: add SSH support This adds an SSH server to farmer.golang.org on port 2222 that proxies SSH connections to users' gomote-created buildlet instances. For example: $ gomote create openbsd-amd64-60 user-bradfitz-openbsd-amd64-60-1 $ gomote ssh user-bradfitz-openbsd-amd64-60-1 Warning: Permanently added '[localhost]:33351' (ECDSA) to the list of known hosts. OpenBSD 6.0 (GENERIC.MP) golang/go#2319: Tue Jul 26 13:00:43 MDT 2016 Welcome to OpenBSD: The proactively secure Unix-like operating system. Please use the sendbug(1) utility to report bugs in the system. Before reporting a bug, please try to reproduce it with the latest version of the code. With bug reports, please try to ensure that enough information to reproduce the problem is enclosed, and if a known fix for it exists, include that as well. $ As before, if the coordinator process is restarted (or crashes, is evicted, etc), all gomote instances die. Not yet supported: * scp (help wanted) * not all host types are configured. most are. some will need slight config tweaks to the Docker image (e.g. adding openssh-server) Supports currently: * linux-amd64 (host type shared by 386, nacl) * linux-arm * linux-arm64 * darwin * freebsd * openbsd * plan9-386 * windows Implementation details: * the ssh server process listens on port 2222 in the coordinator (farmer.golang.org), which is behind a GKE TCP load balancer. * the ssh server library is github.com/gliderlabs/ssh * authentication is done via Github users' public keys. It's assumed that gomote user == github user. But there's a mapping in the code for known exceptions. * we can't give out access to this too widely. too many things are accessible from within the host environment if you look in the right places. Details omitted. But the Go team and other trusted gomote users can use this. * the buildlet binary has a new /connect-ssh handler that acts like a CONNECT request but instead of taking an explicit host:port, just says "give me your machine's SSH connection". The buildlet can also start sshd if needed for the environment. The /connect-ssh handler also installs the coordinator's public key. * a new buildlet client library method "ConnectSSH" hits the /connect-ssh handler and returns a net.Conn. * the coordinator's ssh.Handler is just running the OpenSSH ssh client. * because the OpenSSH ssh child process can't connect to a net.Conn, an emphemeral localhost port is created on the coordinator to proxy between the ssh client and the net.Conn returned by ConnectSSH. * The /connect-ssh handler requires http.Hijacker, which requires fully compliant net.Conn implementations as of Go 1.8. So I needed to flesh out revdial too, testing it with the golang.org/x/net/nettest package. * plan9 doesn't have an ssh server, so we use 0intro's new conterm program (drawterm without GUI support) to connect to plan9 from the coordinator ssh proxy instead of using the OpenSSH ssh client binary. * windows doesn't have an ssh server, so we enable the telnet service and the coordinator ssh proxy uses telnet instead on the backend on the private network. (There is a Windows ssh server but only in new versions.) Happy debugging over ssh! Fixes golang/go#19956 Change-Id: I80a62064c5f85af1f195f980c862ba29af4015f0 Reviewed-on: https://go-review.googlesource.com/50750 Reviewed-by: Herbie Ong <herbie@google.com> Reviewed-by: Jessie Frazelle <me@jessfraz.com> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2017-07-22 22:15:56 +03:00
addBuilder(BuildConfig{
Name: "linux-amd64-localdev",
HostType: "host-linux-amd64-localdev",
Notes: "for localhost development only",
dashboard: unify post-submit & trybot policy in builders.go, not coordinator This removes a bunch of TODOs about scattered policy and moves it all into builders.go. While here, * add a bunch of tests * unexport some things * rename some things * document some things * adjust FreeBSD policy as function of branch (per Go 1.12 being last to support FreeBSD 10.x, and to unblock CL 165801) * set GO_BUILDER_SET_GOPROXY=coordinator for reverse buildlets, which is necessary to remove the oauth2 & build special cases in the config * change Elias Naur's mobile builder to how I think he wants it (he was fighting the old system) * add $HOME on the Solaris smartos builder, which was missing & causing tests to fail lately * make the (currently failing) longtest builder have GOPROXY set * remove an allocation in version.ParseReleaseBranch Fixes golang/go#9603 Updates golang/go#14594 Change-Id: I50a23ad7cdf478c95b14bee9b3931ba361baacfa Reviewed-on: https://go-review.googlesource.com/c/build/+/166218 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-03-07 20:44:41 +03:00
tryOnly: true,
cmd/coordinator, cmd/buildlet, cmd/gomote: add SSH support This adds an SSH server to farmer.golang.org on port 2222 that proxies SSH connections to users' gomote-created buildlet instances. For example: $ gomote create openbsd-amd64-60 user-bradfitz-openbsd-amd64-60-1 $ gomote ssh user-bradfitz-openbsd-amd64-60-1 Warning: Permanently added '[localhost]:33351' (ECDSA) to the list of known hosts. OpenBSD 6.0 (GENERIC.MP) golang/go#2319: Tue Jul 26 13:00:43 MDT 2016 Welcome to OpenBSD: The proactively secure Unix-like operating system. Please use the sendbug(1) utility to report bugs in the system. Before reporting a bug, please try to reproduce it with the latest version of the code. With bug reports, please try to ensure that enough information to reproduce the problem is enclosed, and if a known fix for it exists, include that as well. $ As before, if the coordinator process is restarted (or crashes, is evicted, etc), all gomote instances die. Not yet supported: * scp (help wanted) * not all host types are configured. most are. some will need slight config tweaks to the Docker image (e.g. adding openssh-server) Supports currently: * linux-amd64 (host type shared by 386, nacl) * linux-arm * linux-arm64 * darwin * freebsd * openbsd * plan9-386 * windows Implementation details: * the ssh server process listens on port 2222 in the coordinator (farmer.golang.org), which is behind a GKE TCP load balancer. * the ssh server library is github.com/gliderlabs/ssh * authentication is done via Github users' public keys. It's assumed that gomote user == github user. But there's a mapping in the code for known exceptions. * we can't give out access to this too widely. too many things are accessible from within the host environment if you look in the right places. Details omitted. But the Go team and other trusted gomote users can use this. * the buildlet binary has a new /connect-ssh handler that acts like a CONNECT request but instead of taking an explicit host:port, just says "give me your machine's SSH connection". The buildlet can also start sshd if needed for the environment. The /connect-ssh handler also installs the coordinator's public key. * a new buildlet client library method "ConnectSSH" hits the /connect-ssh handler and returns a net.Conn. * the coordinator's ssh.Handler is just running the OpenSSH ssh client. * because the OpenSSH ssh child process can't connect to a net.Conn, an emphemeral localhost port is created on the coordinator to proxy between the ssh client and the net.Conn returned by ConnectSSH. * The /connect-ssh handler requires http.Hijacker, which requires fully compliant net.Conn implementations as of Go 1.8. So I needed to flesh out revdial too, testing it with the golang.org/x/net/nettest package. * plan9 doesn't have an ssh server, so we use 0intro's new conterm program (drawterm without GUI support) to connect to plan9 from the coordinator ssh proxy instead of using the OpenSSH ssh client binary. * windows doesn't have an ssh server, so we enable the telnet service and the coordinator ssh proxy uses telnet instead on the backend on the private network. (There is a Windows ssh server but only in new versions.) Happy debugging over ssh! Fixes golang/go#19956 Change-Id: I80a62064c5f85af1f195f980c862ba29af4015f0 Reviewed-on: https://go-review.googlesource.com/50750 Reviewed-by: Herbie Ong <herbie@google.com> Reviewed-by: Jessie Frazelle <me@jessfraz.com> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2017-07-22 22:15:56 +03:00
})
dashboard: update dragonfly-amd64 and freebsd-arm to be buildlet-based Updates golang/go#21191 Change-Id: I636204e9e4cbe77e34b2bbad65312fb6f2ace2dc Reviewed-on: https://go-review.googlesource.com/53490 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2017-08-06 01:56:18 +03:00
addBuilder(BuildConfig{
dashboard: refactor mechanism for adjusting cmd/dist test policy Previously, the behavior of ShouldRunDistTest could be overridden by an individual builder by setting a the shouldRunDistTest field to a custom policy implementation. A pitfall of this approach is that it requires each custom policy to reimplement the default cmd/dist test policy as well. If that isn't done when attempting to make a small adjustment to the default policy, the result is that the default policy is ignored. The fasterTrybots policy, as the name suggests, exists to skip some tests in trybot mode. Due to the problem above, it was causing some builders to run more tests in trybot mode than they would if it weren't applied. For example, the freebsd-amd64-12_0 builder should have been skipping the "api" test in trybot mode, but wasn't. This refactor fixes that by modifying the approach for how individual builders can customize the cmd/dist test behavior. Now, a function distTestAdjust is called with the first parameter specifying the default policy decision. That function can adjust the decision as it sees fit, but otherwise it preserves its initial value in situations where no change is intended to be made. For golang/go#38279. Change-Id: I220248f51abfc8e77195ae0c9134a17f7cac9bc2 Reviewed-on: https://go-review.googlesource.com/c/build/+/227777 Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-04-09 19:47:03 +03:00
Name: "dragonfly-amd64",
HostType: "host-dragonfly-amd64-master",
Notes: "DragonFly BSD master, run by DragonFly team",
distTestAdjust: noTestDirAndNoReboot,
dashboard: increase GO_TEST_TIMEOUT_SCALE for dragonfly builders This sets the GO_TEST_TIMEOUT_SCALE=2 for dragonfly builders in order to try to solve intermittent test failures. For golang/go#45215. For golang/go#34034. Fixes golang/go#45216. Change-Id: I81dae533c137ff1903bf6b1a52cf9752e8921b7c GitHub-Last-Rev: 9b4fb8017baebab4126a9d318f14f707eea79f9a GitHub-Pull-Request: golang/build#35 Reviewed-on: https://go-review.googlesource.com/c/build/+/315329 Run-TryBot: Carlos Amedee <carlos@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Trust: Carlos Amedee <carlos@golang.org>
2021-04-30 16:35:02 +03:00
env: []string{"GO_TEST_TIMEOUT_SCALE=2"}, // see golang.org/issue/45216
dashboard: refactor mechanism for adjusting cmd/dist test policy Previously, the behavior of ShouldRunDistTest could be overridden by an individual builder by setting a the shouldRunDistTest field to a custom policy implementation. A pitfall of this approach is that it requires each custom policy to reimplement the default cmd/dist test policy as well. If that isn't done when attempting to make a small adjustment to the default policy, the result is that the default policy is ignored. The fasterTrybots policy, as the name suggests, exists to skip some tests in trybot mode. Due to the problem above, it was causing some builders to run more tests in trybot mode than they would if it weren't applied. For example, the freebsd-amd64-12_0 builder should have been skipping the "api" test in trybot mode, but wasn't. This refactor fixes that by modifying the approach for how individual builders can customize the cmd/dist test behavior. Now, a function distTestAdjust is called with the first parameter specifying the default policy decision. That function can adjust the decision as it sees fit, but otherwise it preserves its initial value in situations where no change is intended to be made. For golang/go#38279. Change-Id: I220248f51abfc8e77195ae0c9134a17f7cac9bc2 Reviewed-on: https://go-review.googlesource.com/c/build/+/227777 Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-04-09 19:47:03 +03:00
SkipSnapshot: true,
dashboard: disable failing repos on misconfigured builders Updates golang/go#32836 Updates golang/go#31567 Updates golang/go#11811 Change-Id: I5443b61cf7732abf906ce2e93eca5408579a55c8 Reviewed-on: https://go-review.googlesource.com/c/build/+/192327 Run-TryBot: Bryan C. Mills <bcmills@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org>
2019-08-29 19:37:31 +03:00
buildsRepo: func(repo, branch, goBranch string) bool {
dashboard: document and clarify defaultBuildsRepoPolicy The defaultBuildsRepoPolicy function was very important in that any changes to it would affect many builders, but it was not documented well. This refactor attempts to improve the situation. It turned out better to also rename defaultBuildsRepoPolicy to buildRepoByDefault and limit its scope to report whether a given repo should be built by default. The branch and goBranch parameters were never used, and thus misleading. If they're needed in the future, we can easily re-add them. The name defaultBuildsRepoPolicy made it sound as if it was the entire "builds repo" policy, bun reality most of the policy is implemented in the BuildConfig.buildsRepoAtAll method and is non-configurable. Only a small part of the policy was in defaultBuildsRepoPolicy. For golang/go#39156. Change-Id: Ie7c6a68bd624056362dc7d964bd8d640f665b35a Reviewed-on: https://go-review.googlesource.com/c/build/+/240687 Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-07-01 21:45:10 +03:00
return atLeastGo1(goBranch, 14) && buildRepoByDefault(repo)
dashboard: disable failing repos on misconfigured builders Updates golang/go#32836 Updates golang/go#31567 Updates golang/go#11811 Change-Id: I5443b61cf7732abf906ce2e93eca5408579a55c8 Reviewed-on: https://go-review.googlesource.com/c/build/+/192327 Run-TryBot: Bryan C. Mills <bcmills@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org>
2019-08-29 19:37:31 +03:00
},
dashboard: update dragonfly-amd64 and freebsd-arm to be buildlet-based Updates golang/go#21191 Change-Id: I636204e9e4cbe77e34b2bbad65312fb6f2ace2dc Reviewed-on: https://go-review.googlesource.com/53490 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2017-08-06 01:56:18 +03:00
})
addBuilder(BuildConfig{
dashboard: refactor mechanism for adjusting cmd/dist test policy Previously, the behavior of ShouldRunDistTest could be overridden by an individual builder by setting a the shouldRunDistTest field to a custom policy implementation. A pitfall of this approach is that it requires each custom policy to reimplement the default cmd/dist test policy as well. If that isn't done when attempting to make a small adjustment to the default policy, the result is that the default policy is ignored. The fasterTrybots policy, as the name suggests, exists to skip some tests in trybot mode. Due to the problem above, it was causing some builders to run more tests in trybot mode than they would if it weren't applied. For example, the freebsd-amd64-12_0 builder should have been skipping the "api" test in trybot mode, but wasn't. This refactor fixes that by modifying the approach for how individual builders can customize the cmd/dist test behavior. Now, a function distTestAdjust is called with the first parameter specifying the default policy decision. That function can adjust the decision as it sees fit, but otherwise it preserves its initial value in situations where no change is intended to be made. For golang/go#38279. Change-Id: I220248f51abfc8e77195ae0c9134a17f7cac9bc2 Reviewed-on: https://go-review.googlesource.com/c/build/+/227777 Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-04-09 19:47:03 +03:00
Name: "freebsd-arm-paulzhol",
HostType: "host-freebsd-arm-paulzhol",
distTestAdjust: noTestDirAndNoReboot,
SkipSnapshot: true,
dashboard: unify post-submit & trybot policy in builders.go, not coordinator This removes a bunch of TODOs about scattered policy and moves it all into builders.go. While here, * add a bunch of tests * unexport some things * rename some things * document some things * adjust FreeBSD policy as function of branch (per Go 1.12 being last to support FreeBSD 10.x, and to unblock CL 165801) * set GO_BUILDER_SET_GOPROXY=coordinator for reverse buildlets, which is necessary to remove the oauth2 & build special cases in the config * change Elias Naur's mobile builder to how I think he wants it (he was fighting the old system) * add $HOME on the Solaris smartos builder, which was missing & causing tests to fail lately * make the (currently failing) longtest builder have GOPROXY set * remove an allocation in version.ParseReleaseBranch Fixes golang/go#9603 Updates golang/go#14594 Change-Id: I50a23ad7cdf478c95b14bee9b3931ba361baacfa Reviewed-on: https://go-review.googlesource.com/c/build/+/166218 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-03-07 20:44:41 +03:00
buildsRepo: func(repo, branch, goBranch string) bool {
// This was a fragile little machine with limited memory.
// Only run a few of the core subrepos for now while
// we figure out what's killing it.
switch repo {
case "go", "sys", "net":
return true
}
return false
},
dashboard: update dragonfly-amd64 and freebsd-arm to be buildlet-based Updates golang/go#21191 Change-Id: I636204e9e4cbe77e34b2bbad65312fb6f2ace2dc Reviewed-on: https://go-review.googlesource.com/53490 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2017-08-06 01:56:18 +03:00
env: []string{
"GOARM=7",
"CGO_ENABLED=1",
},
})
dashboard: add FreeBSD arm64 builder Updates golang/go#24715 Change-Id: I8eb3f6372d69718b6f453e5415cdf16aa00ccae6 Reviewed-on: https://go-review.googlesource.com/c/build/+/198597 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-10-03 15:56:19 +03:00
addBuilder(BuildConfig{
Name: "freebsd-arm64-dmgk",
HostType: "host-freebsd-arm64-dmgk",
dashboard: only build tip on freebsd-arm64-dmgk builder Updates golang/go#24715 Change-Id: I9f1aeac0d3b5f625e0f68b6bad17445a41a87b22 Reviewed-on: https://go-review.googlesource.com/c/build/+/203420 Run-TryBot: Bryan C. Mills <bcmills@google.com> Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-10-25 22:21:38 +03:00
buildsRepo: func(repo, branch, goBranch string) bool {
dashboard: document and clarify defaultBuildsRepoPolicy The defaultBuildsRepoPolicy function was very important in that any changes to it would affect many builders, but it was not documented well. This refactor attempts to improve the situation. It turned out better to also rename defaultBuildsRepoPolicy to buildRepoByDefault and limit its scope to report whether a given repo should be built by default. The branch and goBranch parameters were never used, and thus misleading. If they're needed in the future, we can easily re-add them. The name defaultBuildsRepoPolicy made it sound as if it was the entire "builds repo" policy, bun reality most of the policy is implemented in the BuildConfig.buildsRepoAtAll method and is non-configurable. Only a small part of the policy was in defaultBuildsRepoPolicy. For golang/go#39156. Change-Id: Ie7c6a68bd624056362dc7d964bd8d640f665b35a Reviewed-on: https://go-review.googlesource.com/c/build/+/240687 Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-07-01 21:45:10 +03:00
return atLeastGo1(goBranch, 14) && buildRepoByDefault(repo)
dashboard: only build tip on freebsd-arm64-dmgk builder Updates golang/go#24715 Change-Id: I9f1aeac0d3b5f625e0f68b6bad17445a41a87b22 Reviewed-on: https://go-review.googlesource.com/c/build/+/203420 Run-TryBot: Bryan C. Mills <bcmills@google.com> Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-10-25 22:21:38 +03:00
},
dashboard: add FreeBSD arm64 builder Updates golang/go#24715 Change-Id: I8eb3f6372d69718b6f453e5415cdf16aa00ccae6 Reviewed-on: https://go-review.googlesource.com/c/build/+/198597 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-10-03 15:56:19 +03:00
})
dashboard: convert plan9-arm, plan9-amd64-9front, nacl-arm to buildlets Updates golang/go#21191 Change-Id: I577dad51d490d7e281ccde9fa7d82bb1bee95bd9 Reviewed-on: https://go-review.googlesource.com/53417 Reviewed-by: David du Colombier <0intro@gmail.com>
2017-08-06 18:57:06 +03:00
addBuilder(BuildConfig{
dashboard: refactor mechanism for adjusting cmd/dist test policy Previously, the behavior of ShouldRunDistTest could be overridden by an individual builder by setting a the shouldRunDistTest field to a custom policy implementation. A pitfall of this approach is that it requires each custom policy to reimplement the default cmd/dist test policy as well. If that isn't done when attempting to make a small adjustment to the default policy, the result is that the default policy is ignored. The fasterTrybots policy, as the name suggests, exists to skip some tests in trybot mode. Due to the problem above, it was causing some builders to run more tests in trybot mode than they would if it weren't applied. For example, the freebsd-amd64-12_0 builder should have been skipping the "api" test in trybot mode, but wasn't. This refactor fixes that by modifying the approach for how individual builders can customize the cmd/dist test behavior. Now, a function distTestAdjust is called with the first parameter specifying the default policy decision. That function can adjust the decision as it sees fit, but otherwise it preserves its initial value in situations where no change is intended to be made. For golang/go#38279. Change-Id: I220248f51abfc8e77195ae0c9134a17f7cac9bc2 Reviewed-on: https://go-review.googlesource.com/c/build/+/227777 Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-04-09 19:47:03 +03:00
Name: "plan9-arm",
HostType: "host-plan9-arm-0intro",
distTestAdjust: noTestDirAndNoReboot,
dashboard: omit the website repo on plan9 The plan9 builders have relatively slow filesystems, and the website repo is relatively filesystem-intensive — many of its tests walk and process large trees of website content. Instead of trying to precisely tune the x/website tests to pass on the plan9 builders, let's skip this repo on them. For golang/go#49338. Change-Id: Ieaf50fb6955a6b39012a23ff72e61c1c31e4ce7c Reviewed-on: https://go-review.googlesource.com/c/build/+/362975 Trust: Bryan C. Mills <bcmills@google.com> Run-TryBot: Bryan C. Mills <bcmills@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2021-11-10 19:28:50 +03:00
buildsRepo: plan9Default,
dashboard: convert plan9-arm, plan9-amd64-9front, nacl-arm to buildlets Updates golang/go#21191 Change-Id: I577dad51d490d7e281ccde9fa7d82bb1bee95bd9 Reviewed-on: https://go-review.googlesource.com/53417 Reviewed-by: David du Colombier <0intro@gmail.com>
2017-08-06 18:57:06 +03:00
})
addBuilder(BuildConfig{
dashboard: rename plan9-amd64-9front builder to plan9-386-0intro The builder name was misleading, because plan9-amd64-9front hasn't been running 9front anymore since 2016, but is running the 9k kernel from Bell Labs. Change-Id: I6929583b784d076a86b818649c64b821a538ed1c Reviewed-on: https://go-review.googlesource.com/c/build/+/307409 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Trust: Carlos Amedee <carlos@golang.org> Trust: David du Colombier <0intro@gmail.com> Run-TryBot: Carlos Amedee <carlos@golang.org> TryBot-Result: Go Bot <gobot@golang.org>
2021-04-05 21:42:05 +03:00
Name: "plan9-amd64-0intro",
dashboard: use snapshots on Plan 9 builders This change enables snapshots on Plan 9 builders, so running the tests for each of the golang.org/x repositories will not require to rebuild Go every time. Change-Id: I305b713a48c155ebba2c03f39633d17a6fd71e6f Reviewed-on: https://go-review.googlesource.com/c/build/+/208359 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-11-22 22:19:11 +03:00
HostType: "host-plan9-amd64-0intro",
cmd/coordinator, dashboard: don't skip cmd/dist tests for SlowBots When a user explicitly requests a SlowBot builder via the TRY= syntax, it is a signal that they're willing to wait longer for test results. It's more useful to not skip any tests and give the user confidence that the SlowBot run will catch all issues that would otherwise be caught only by post-submit builders after the CL is submitted. This is done by considering a TryBot that isn't an explicit SlowBot to be a "normal TryBot", and the ShouldRunDistTest policy function is provided a parameter specifying whether it's a normal TryBot (non-SlowBot) run. Tests are only skipped during normal TryBot runs, not SlowBot runs. Fixes golang/go#38279. For golang/go#34501. Change-Id: Ia11fd51c71f9de1089b6b6f4c027014893f224b9 Reviewed-on: https://go-review.googlesource.com/c/build/+/227779 Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-04-09 23:24:30 +03:00
distTestAdjust: func(run bool, distTest string, isNormalTry bool) bool {
run = noTestDirAndNoReboot(run, distTest, isNormalTry)
dashboard: refactor mechanism for adjusting cmd/dist test policy Previously, the behavior of ShouldRunDistTest could be overridden by an individual builder by setting a the shouldRunDistTest field to a custom policy implementation. A pitfall of this approach is that it requires each custom policy to reimplement the default cmd/dist test policy as well. If that isn't done when attempting to make a small adjustment to the default policy, the result is that the default policy is ignored. The fasterTrybots policy, as the name suggests, exists to skip some tests in trybot mode. Due to the problem above, it was causing some builders to run more tests in trybot mode than they would if it weren't applied. For example, the freebsd-amd64-12_0 builder should have been skipping the "api" test in trybot mode, but wasn't. This refactor fixes that by modifying the approach for how individual builders can customize the cmd/dist test behavior. Now, a function distTestAdjust is called with the first parameter specifying the default policy decision. That function can adjust the decision as it sees fit, but otherwise it preserves its initial value in situations where no change is intended to be made. For golang/go#38279. Change-Id: I220248f51abfc8e77195ae0c9134a17f7cac9bc2 Reviewed-on: https://go-review.googlesource.com/c/build/+/227777 Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-04-09 19:47:03 +03:00
switch distTest {
dashboard: disable cmd/go tests on plan9-amd64-9front The cmd/go tests are timing out on the plan9/amd64 builder because they take more than 20m to complete. This change disables the cmd/go tests on the plan9/amd64 builder, like it was done on the plan9/386 builder. Updates golang/go#31910. Change-Id: I157b472da8498ddce0f641cccb79bc50ae65ca3b Reviewed-on: https://go-review.googlesource.com/c/build/+/175898 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-05-08 20:42:00 +03:00
case "api",
"go_test:cmd/go": // takes over 20 minutes without working SMP
return false
}
dashboard: refactor mechanism for adjusting cmd/dist test policy Previously, the behavior of ShouldRunDistTest could be overridden by an individual builder by setting a the shouldRunDistTest field to a custom policy implementation. A pitfall of this approach is that it requires each custom policy to reimplement the default cmd/dist test policy as well. If that isn't done when attempting to make a small adjustment to the default policy, the result is that the default policy is ignored. The fasterTrybots policy, as the name suggests, exists to skip some tests in trybot mode. Due to the problem above, it was causing some builders to run more tests in trybot mode than they would if it weren't applied. For example, the freebsd-amd64-12_0 builder should have been skipping the "api" test in trybot mode, but wasn't. This refactor fixes that by modifying the approach for how individual builders can customize the cmd/dist test behavior. Now, a function distTestAdjust is called with the first parameter specifying the default policy decision. That function can adjust the decision as it sees fit, but otherwise it preserves its initial value in situations where no change is intended to be made. For golang/go#38279. Change-Id: I220248f51abfc8e77195ae0c9134a17f7cac9bc2 Reviewed-on: https://go-review.googlesource.com/c/build/+/227777 Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-04-09 19:47:03 +03:00
return run
dashboard: disable cmd/go tests on plan9-amd64-9front The cmd/go tests are timing out on the plan9/amd64 builder because they take more than 20m to complete. This change disables the cmd/go tests on the plan9/amd64 builder, like it was done on the plan9/386 builder. Updates golang/go#31910. Change-Id: I157b472da8498ddce0f641cccb79bc50ae65ca3b Reviewed-on: https://go-review.googlesource.com/c/build/+/175898 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-05-08 20:42:00 +03:00
},
dashboard: omit the website repo on plan9 The plan9 builders have relatively slow filesystems, and the website repo is relatively filesystem-intensive — many of its tests walk and process large trees of website content. Instead of trying to precisely tune the x/website tests to pass on the plan9 builders, let's skip this repo on them. For golang/go#49338. Change-Id: Ieaf50fb6955a6b39012a23ff72e61c1c31e4ce7c Reviewed-on: https://go-review.googlesource.com/c/build/+/362975 Trust: Bryan C. Mills <bcmills@google.com> Run-TryBot: Bryan C. Mills <bcmills@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2021-11-10 19:28:50 +03:00
buildsRepo: plan9Default,
dashboard: convert plan9-arm, plan9-amd64-9front, nacl-arm to buildlets Updates golang/go#21191 Change-Id: I577dad51d490d7e281ccde9fa7d82bb1bee95bd9 Reviewed-on: https://go-review.googlesource.com/53417 Reviewed-by: David du Colombier <0intro@gmail.com>
2017-08-06 18:57:06 +03:00
})
dashboard: add plan9-386-0intro builder This change adds the plan9-386-0intro builder configuration. Updates golang/go#29801. Change-Id: I5f64f4592a499eb1691498805fd0dfa5082c33d5 Reviewed-on: https://go-review.googlesource.com/c/build/+/175901 Run-TryBot: David du Colombier <0intro@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-05-09 19:52:40 +03:00
addBuilder(BuildConfig{
dashboard: use snapshots on Plan 9 builders This change enables snapshots on Plan 9 builders, so running the tests for each of the golang.org/x repositories will not require to rebuild Go every time. Change-Id: I305b713a48c155ebba2c03f39633d17a6fd71e6f Reviewed-on: https://go-review.googlesource.com/c/build/+/208359 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-11-22 22:19:11 +03:00
Name: "plan9-386-0intro",
HostType: "host-plan9-386-0intro",
cmd/coordinator, dashboard: don't skip cmd/dist tests for SlowBots When a user explicitly requests a SlowBot builder via the TRY= syntax, it is a signal that they're willing to wait longer for test results. It's more useful to not skip any tests and give the user confidence that the SlowBot run will catch all issues that would otherwise be caught only by post-submit builders after the CL is submitted. This is done by considering a TryBot that isn't an explicit SlowBot to be a "normal TryBot", and the ShouldRunDistTest policy function is provided a parameter specifying whether it's a normal TryBot (non-SlowBot) run. Tests are only skipped during normal TryBot runs, not SlowBot runs. Fixes golang/go#38279. For golang/go#34501. Change-Id: Ia11fd51c71f9de1089b6b6f4c027014893f224b9 Reviewed-on: https://go-review.googlesource.com/c/build/+/227779 Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-04-09 23:24:30 +03:00
distTestAdjust: func(run bool, distTest string, isNormalTry bool) bool {
run = noTestDirAndNoReboot(run, distTest, isNormalTry)
dashboard: refactor mechanism for adjusting cmd/dist test policy Previously, the behavior of ShouldRunDistTest could be overridden by an individual builder by setting a the shouldRunDistTest field to a custom policy implementation. A pitfall of this approach is that it requires each custom policy to reimplement the default cmd/dist test policy as well. If that isn't done when attempting to make a small adjustment to the default policy, the result is that the default policy is ignored. The fasterTrybots policy, as the name suggests, exists to skip some tests in trybot mode. Due to the problem above, it was causing some builders to run more tests in trybot mode than they would if it weren't applied. For example, the freebsd-amd64-12_0 builder should have been skipping the "api" test in trybot mode, but wasn't. This refactor fixes that by modifying the approach for how individual builders can customize the cmd/dist test behavior. Now, a function distTestAdjust is called with the first parameter specifying the default policy decision. That function can adjust the decision as it sees fit, but otherwise it preserves its initial value in situations where no change is intended to be made. For golang/go#38279. Change-Id: I220248f51abfc8e77195ae0c9134a17f7cac9bc2 Reviewed-on: https://go-review.googlesource.com/c/build/+/227777 Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-04-09 19:47:03 +03:00
switch distTest {
dashboard: add plan9-386-0intro builder This change adds the plan9-386-0intro builder configuration. Updates golang/go#29801. Change-Id: I5f64f4592a499eb1691498805fd0dfa5082c33d5 Reviewed-on: https://go-review.googlesource.com/c/build/+/175901 Run-TryBot: David du Colombier <0intro@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-05-09 19:52:40 +03:00
case "api",
"go_test:cmd/go": // takes over 20 minutes without working SMP
return false
}
dashboard: refactor mechanism for adjusting cmd/dist test policy Previously, the behavior of ShouldRunDistTest could be overridden by an individual builder by setting a the shouldRunDistTest field to a custom policy implementation. A pitfall of this approach is that it requires each custom policy to reimplement the default cmd/dist test policy as well. If that isn't done when attempting to make a small adjustment to the default policy, the result is that the default policy is ignored. The fasterTrybots policy, as the name suggests, exists to skip some tests in trybot mode. Due to the problem above, it was causing some builders to run more tests in trybot mode than they would if it weren't applied. For example, the freebsd-amd64-12_0 builder should have been skipping the "api" test in trybot mode, but wasn't. This refactor fixes that by modifying the approach for how individual builders can customize the cmd/dist test behavior. Now, a function distTestAdjust is called with the first parameter specifying the default policy decision. That function can adjust the decision as it sees fit, but otherwise it preserves its initial value in situations where no change is intended to be made. For golang/go#38279. Change-Id: I220248f51abfc8e77195ae0c9134a17f7cac9bc2 Reviewed-on: https://go-review.googlesource.com/c/build/+/227777 Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-04-09 19:47:03 +03:00
return run
dashboard: add plan9-386-0intro builder This change adds the plan9-386-0intro builder configuration. Updates golang/go#29801. Change-Id: I5f64f4592a499eb1691498805fd0dfa5082c33d5 Reviewed-on: https://go-review.googlesource.com/c/build/+/175901 Run-TryBot: David du Colombier <0intro@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-05-09 19:52:40 +03:00
},
dashboard: omit the website repo on plan9 The plan9 builders have relatively slow filesystems, and the website repo is relatively filesystem-intensive — many of its tests walk and process large trees of website content. Instead of trying to precisely tune the x/website tests to pass on the plan9 builders, let's skip this repo on them. For golang/go#49338. Change-Id: Ieaf50fb6955a6b39012a23ff72e61c1c31e4ce7c Reviewed-on: https://go-review.googlesource.com/c/build/+/362975 Trust: Bryan C. Mills <bcmills@google.com> Run-TryBot: Bryan C. Mills <bcmills@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2021-11-10 19:28:50 +03:00
buildsRepo: plan9Default,
dashboard: add plan9-386-0intro builder This change adds the plan9-386-0intro builder configuration. Updates golang/go#29801. Change-Id: I5f64f4592a499eb1691498805fd0dfa5082c33d5 Reviewed-on: https://go-review.googlesource.com/c/build/+/175901 Run-TryBot: David du Colombier <0intro@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-05-09 19:52:40 +03:00
})
dashboard: add aix-ppc64 builder This is a starting point for adding an aix-ppc64 reverse builder. There can be further adjustments made to it as needed. Updates golang/go#27160. Change-Id: I8a61adbe21f1a08265df9a206d94bb04e880a508 Reviewed-on: https://go-review.googlesource.com/131335 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-08-24 21:41:05 +03:00
addBuilder(BuildConfig{
dashboard: require at least Go 1.12 for aix builder Fixes golang/go#30799 Change-Id: I0b3b53b715e2dc00f10548f74443224190003818 Reviewed-on: https://go-review.googlesource.com/c/build/+/167357 Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-03-13 11:29:15 +03:00
Name: "aix-ppc64",
HostType: "host-aix-ppc64-osuosl",
MinimumGoVersion: types.MajorMinor{1, 12},
dashboard: add /opt/freeware/bin to PATH on aix builder Updates golang/go#29065 Fixes golang/go#29078 Change-Id: Ifa9355c9dc988a460b6198913431647ec2c5e6ac Reviewed-on: https://go-review.googlesource.com/c/152457 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-12-04 17:46:08 +03:00
env: []string{
"PATH=/opt/freeware/bin:/usr/bin:/etc:/usr/sbin:/usr/ucb:/usr/bin/X11:/sbin:/usr/java7_64/jre/bin:/usr/java7_64/bin",
},
dashboard: don't build x/net master against Go 1.12 on aix Per request in https://golang.org/issue/31564#issuecomment-484786144 Change-Id: I52e496083b84aa021e7d7cdcda19c96980e1518f Reviewed-on: https://go-review.googlesource.com/c/build/+/172878 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-04-19 17:37:11 +03:00
buildsRepo: func(repo, branch, goBranch string) bool {
switch repo {
dashboard: skip vulndb on aix-ppc64 vulndb currently uses a dependency which does not build cleanly on this os/arch combo. Until that issue is resolved, skip (this code is not intended to be run on a large variety of systems, so it should be fine for the foreseeable future.) Fixes golang/go#49218 Change-Id: I174daf963dbb4493141330bae55398e9f97f0fc1 Reviewed-on: https://go-review.googlesource.com/c/build/+/360414 Trust: Roland Shoemaker <roland@golang.org> Trust: Bryan C. Mills <bcmills@google.com> Run-TryBot: Roland Shoemaker <roland@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2021-11-01 20:57:34 +03:00
case "vulndb", "vuln":
// vulndb currently uses a dependency which does not build cleanly
// on aix-ppc64. Until that issue is resolved, skip vulndb on
// this builder.
// (https://golang.org/issue/49218).
return false
dashboard: don't build x/net master against Go 1.12 on aix Per request in https://golang.org/issue/31564#issuecomment-484786144 Change-Id: I52e496083b84aa021e7d7cdcda19c96980e1518f Reviewed-on: https://go-review.googlesource.com/c/build/+/172878 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-04-19 17:37:11 +03:00
}
dashboard, cmd/release: remove what's unused as of Go 1.16 release We can drop the 32-bit and 10.11 macOS builders, they're unused. Remove FreeBSD 11.1 builder, it's unused. Update freebsd-amd64-race to use the latest available FreeBSD builder. Simplify build policy and comments, removing anything that mentioned Go version 1.14 or older, since they're not relevant anymore. Drop Go 1.14 support from minSupportedMacOSVersion, and add support for the future Go 1.17 release, which will increase minimum macOS version requirement to macOS 10.13. Fixes golang/go#40562. Change-Id: I7beeec0952a516afe99a5154b22c7f25ceeb3da1 Reviewed-on: https://go-review.googlesource.com/c/build/+/293771 Trust: Dmitri Shuralyov <dmitshur@golang.org> Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org>
2021-02-19 20:17:28 +03:00
return buildRepoByDefault(repo)
dashboard: don't build x/net master against Go 1.12 on aix Per request in https://golang.org/issue/31564#issuecomment-484786144 Change-Id: I52e496083b84aa021e7d7cdcda19c96980e1518f Reviewed-on: https://go-review.googlesource.com/c/build/+/172878 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-04-19 17:37:11 +03:00
},
dashboard: add aix-ppc64 builder This is a starting point for adding an aix-ppc64 reverse builder. There can be further adjustments made to it as needed. Updates golang/go#27160. Change-Id: I8a61adbe21f1a08265df9a206d94bb04e880a508 Reviewed-on: https://go-review.googlesource.com/131335 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-08-24 21:41:05 +03:00
})
dashboard: add new linux-amd64-wsl builder Updates golang/go#17365 Change-Id: I90b9c1c8f82ae9164deacea36ff5dafd008eef03 Reviewed-on: https://go-review.googlesource.com/c/build/+/272266 Trust: Meng Zhuo <mzh@golangcn.org> Run-TryBot: Meng Zhuo <mzh@golangcn.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
2020-11-24 19:44:38 +03:00
addBuilder(BuildConfig{
Name: "linux-amd64-wsl",
HostType: "host-linux-amd64-wsl",
Notes: "Windows 10 WSL2 Ubuntu",
FlakyNet: true,
SkipSnapshot: true, // The builder has a slow uplink bandwidth.
})
dashboard: add linux-amd64-perf host and builder Add a new builder to run the x/benchmarks performance tests on linux-amd64. For now, this runs on a GCE C2 instance type, as these instances have well-defined, consistent CPUs and other server architecture components. In basic noise testing, even standard VMs of this type appear to be fairly low noise. As we gain experience with actual monitoring, we may change this to a sole-tenant VM type or even a dedicated machine if necessary. For golang/go#49207 Change-Id: I17eaeeb5349af925249940bebd5b860a2579e6df Reviewed-on: https://go-review.googlesource.com/c/build/+/354311 Trust: Michael Pratt <mpratt@google.com> Run-TryBot: Michael Pratt <mpratt@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org>
2021-10-01 21:59:33 +03:00
addBuilder(BuildConfig{
dashboard: SkipSnapshot for linux-amd64-perf Since this builder doesn't build the go repo, it will be waiting forever for a snapshot. Instead, just build Go for each run. For golang/go#49207 Change-Id: I34a73b507278db402c478b4f5956633996772aae Reviewed-on: https://go-review.googlesource.com/c/build/+/361754 Trust: Alexander Rakoczy <alex@golang.org> Run-TryBot: Alexander Rakoczy <alex@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2021-11-05 22:01:27 +03:00
Name: "linux-amd64-perf",
HostType: "host-linux-amd64-perf",
Notes: "Performance testing for linux-amd64",
dashboard: add linux-amd64-perf host and builder Add a new builder to run the x/benchmarks performance tests on linux-amd64. For now, this runs on a GCE C2 instance type, as these instances have well-defined, consistent CPUs and other server architecture components. In basic noise testing, even standard VMs of this type appear to be fairly low noise. As we gain experience with actual monitoring, we may change this to a sole-tenant VM type or even a dedicated machine if necessary. For golang/go#49207 Change-Id: I17eaeeb5349af925249940bebd5b860a2579e6df Reviewed-on: https://go-review.googlesource.com/c/build/+/354311 Trust: Michael Pratt <mpratt@google.com> Run-TryBot: Michael Pratt <mpratt@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org>
2021-10-01 21:59:33 +03:00
buildsRepo: func(repo, branch, goBranch string) bool {
return repo == "benchmarks"
},
dashboard: SkipSnapshot for linux-amd64-perf Since this builder doesn't build the go repo, it will be waiting forever for a snapshot. Instead, just build Go for each run. For golang/go#49207 Change-Id: I34a73b507278db402c478b4f5956633996772aae Reviewed-on: https://go-review.googlesource.com/c/build/+/361754 Trust: Alexander Rakoczy <alex@golang.org> Run-TryBot: Alexander Rakoczy <alex@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2021-11-05 22:01:27 +03:00
RunBench: true,
SkipSnapshot: true,
dashboard: add linux-amd64-perf host and builder Add a new builder to run the x/benchmarks performance tests on linux-amd64. For now, this runs on a GCE C2 instance type, as these instances have well-defined, consistent CPUs and other server architecture components. In basic noise testing, even standard VMs of this type appear to be fairly low noise. As we gain experience with actual monitoring, we may change this to a sole-tenant VM type or even a dedicated machine if necessary. For golang/go#49207 Change-Id: I17eaeeb5349af925249940bebd5b860a2579e6df Reviewed-on: https://go-review.googlesource.com/c/build/+/354311 Trust: Michael Pratt <mpratt@google.com> Run-TryBot: Michael Pratt <mpratt@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org>
2021-10-01 21:59:33 +03:00
})
all: split builder config into builder & host configs Our builders are named of the form "GOOS-GOARCH" or "GOOS-GOARCH-suffix". Over time we've grown many builders. This CL doesn't change that. Builders continue to be named and operate as before. Previously the build configuration file (dashboard/builders.go) made each builder type ("linux-amd64-race", etc) define how to create a host running a buildlet of that type, even though many builders had identical host configs. For example, these builders all share the same host type (a Kubernetes container): linux-amd64 linux-amd64-race linux-386 linux-386-387 And these are the same host type (a GCE VM): windows-amd64-gce windows-amd64-race windows-386-gce This CL creates a new concept of a "hostType" which defines how the buildlet is created (Kube, GCE, Reverse, and how), and then each builder itself references a host type. Users never see the hostType. (except perhaps in gomote list output) But they at least never need to care about them. Reverse buildlets now can only be one hostType at a time, which simplifies things. We were no longer using multiple roles per machine once moving to VMs for OS X. gomote continues to operate as it did previously but its underlying protocol changed and clients will need to be updated. As a new feature, gomote now has a new flag to let you reuse a buildlet host connection for different builder rules if they share the same underlying host type. But users can ignore that. This CL is a long-standing TODO (previously attempted and aborted) and will make many things easier and faster, including the linux-arm cross-compilation effort, and keeping pre-warmed buildlets of VM types ready to go. Updates golang/go#17104 Change-Id: Iad8387f48680424a8441e878a2f4762bf79ea4d2 Reviewed-on: https://go-review.googlesource.com/29551 Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2016-09-22 00:27:37 +03:00
}
dashboard: add s390x cross-compile-only builder and release config Also modifies cmd/release to wire it up. Updates golang/go#16443 Change-Id: Ie80a224957f6f662d0aec0fd083e2a57e30436c4 Reviewed-on: https://go-review.googlesource.com/28172 Reviewed-by: Jessica Frazelle <me@jessfraz.com> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2016-08-30 23:32:33 +03:00
dashboard, cmd/coordinator: support multiple builder owners Prefer displaying GitHub usernames for convenience of pinging in GitHub issues, since all current builder owners have them. Keep it possible to use an email or something else if no GitHub account exists. Update the owner of the Corellium builders to be individual GitHub account, so that it can be pinged in an issue. The GitHub organization is already mentioned in the builder notes. Also switch to html/template (instead of text/template) for rendering the builders page. Fixes golang/go#49596. Change-Id: I89a96eb8f7a3057bc1f3aaee16abc9c776c45ee7 Reviewed-on: https://go-review.googlesource.com/c/build/+/363983 Trust: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
2021-11-16 20:42:29 +03:00
// addBuilder adds c to the Builders map after doing some checks.
dashboard: move builder configuration out of coordinator into a package The new package will be used by the buildlet command line tool in the future. Also use the metadata package in the coordinator for the master build key, since it was already depending on that package. Might as well use it. Change-Id: I6ad7593787ebba364090951ccbaecb4aa38e1dd6 Reviewed-on: https://go-review.googlesource.com/2920 Reviewed-by: Andrew Gerrand <adg@golang.org>
2015-01-15 23:46:22 +03:00
func addBuilder(c BuildConfig) {
if c.Name == "" {
panic("empty name")
}
all: split builder config into builder & host configs Our builders are named of the form "GOOS-GOARCH" or "GOOS-GOARCH-suffix". Over time we've grown many builders. This CL doesn't change that. Builders continue to be named and operate as before. Previously the build configuration file (dashboard/builders.go) made each builder type ("linux-amd64-race", etc) define how to create a host running a buildlet of that type, even though many builders had identical host configs. For example, these builders all share the same host type (a Kubernetes container): linux-amd64 linux-amd64-race linux-386 linux-386-387 And these are the same host type (a GCE VM): windows-amd64-gce windows-amd64-race windows-386-gce This CL creates a new concept of a "hostType" which defines how the buildlet is created (Kube, GCE, Reverse, and how), and then each builder itself references a host type. Users never see the hostType. (except perhaps in gomote list output) But they at least never need to care about them. Reverse buildlets now can only be one hostType at a time, which simplifies things. We were no longer using multiple roles per machine once moving to VMs for OS X. gomote continues to operate as it did previously but its underlying protocol changed and clients will need to be updated. As a new feature, gomote now has a new flag to let you reuse a buildlet host connection for different builder rules if they share the same underlying host type. But users can ignore that. This CL is a long-standing TODO (previously attempted and aborted) and will make many things easier and faster, including the linux-arm cross-compilation effort, and keeping pre-warmed buildlets of VM types ready to go. Updates golang/go#17104 Change-Id: Iad8387f48680424a8441e878a2f4762bf79ea4d2 Reviewed-on: https://go-review.googlesource.com/29551 Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2016-09-22 00:27:37 +03:00
if c.HostType == "" {
panic(fmt.Sprintf("missing HostType for builder %q", c.Name))
}
dashboard: move builder configuration out of coordinator into a package The new package will be used by the buildlet command line tool in the future. Also use the metadata package in the coordinator for the master build key, since it was already depending on that package. Might as well use it. Change-Id: I6ad7593787ebba364090951ccbaecb4aa38e1dd6 Reviewed-on: https://go-review.googlesource.com/2920 Reviewed-by: Andrew Gerrand <adg@golang.org>
2015-01-15 23:46:22 +03:00
if _, dup := Builders[c.Name]; dup {
dashboard: convert plan9-arm, plan9-amd64-9front, nacl-arm to buildlets Updates golang/go#21191 Change-Id: I577dad51d490d7e281ccde9fa7d82bb1bee95bd9 Reviewed-on: https://go-review.googlesource.com/53417 Reviewed-by: David du Colombier <0intro@gmail.com>
2017-08-06 18:57:06 +03:00
panic("dup name " + c.Name)
dashboard: move builder configuration out of coordinator into a package The new package will be used by the buildlet command line tool in the future. Also use the metadata package in the coordinator for the master build key, since it was already depending on that package. Might as well use it. Change-Id: I6ad7593787ebba364090951ccbaecb4aa38e1dd6 Reviewed-on: https://go-review.googlesource.com/2920 Reviewed-by: Andrew Gerrand <adg@golang.org>
2015-01-15 23:46:22 +03:00
}
all: split builder config into builder & host configs Our builders are named of the form "GOOS-GOARCH" or "GOOS-GOARCH-suffix". Over time we've grown many builders. This CL doesn't change that. Builders continue to be named and operate as before. Previously the build configuration file (dashboard/builders.go) made each builder type ("linux-amd64-race", etc) define how to create a host running a buildlet of that type, even though many builders had identical host configs. For example, these builders all share the same host type (a Kubernetes container): linux-amd64 linux-amd64-race linux-386 linux-386-387 And these are the same host type (a GCE VM): windows-amd64-gce windows-amd64-race windows-386-gce This CL creates a new concept of a "hostType" which defines how the buildlet is created (Kube, GCE, Reverse, and how), and then each builder itself references a host type. Users never see the hostType. (except perhaps in gomote list output) But they at least never need to care about them. Reverse buildlets now can only be one hostType at a time, which simplifies things. We were no longer using multiple roles per machine once moving to VMs for OS X. gomote continues to operate as it did previously but its underlying protocol changed and clients will need to be updated. As a new feature, gomote now has a new flag to let you reuse a buildlet host connection for different builder rules if they share the same underlying host type. But users can ignore that. This CL is a long-standing TODO (previously attempted and aborted) and will make many things easier and faster, including the linux-arm cross-compilation effort, and keeping pre-warmed buildlets of VM types ready to go. Updates golang/go#17104 Change-Id: Iad8387f48680424a8441e878a2f4762bf79ea4d2 Reviewed-on: https://go-review.googlesource.com/29551 Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2016-09-22 00:27:37 +03:00
if _, ok := Hosts[c.HostType]; !ok {
panic(fmt.Sprintf("undefined HostType %q for builder %q", c.HostType, c.Name))
cmd/coordinator: start of a Kubernetes buildlet pool Updates golang/go#12546 Change-Id: Ia70baeb1df441509f70c77e7b94e7eb9fddccbf3 Reviewed-on: https://go-review.googlesource.com/14392 Reviewed-by: Evan Brown <evanbrown@google.com>
2015-09-09 01:18:47 +03:00
}
dashboard, cmd/coordinator: clean up snapshot code, add snapshot opt-out The snapshot code had an old workaround that's no longer relevant. Remove it. Also, add a BuildConfig.SkipSnapshot bool, and use it for the slow mips builders. Fixes golang/go#19953 Change-Id: I114bb0a524184eaaae5be4715ce63f6adc519c2e Reviewed-on: https://go-review.googlesource.com/40505 Reviewed-by: Sarah Adams <shadams@google.com>
2017-04-13 07:34:42 +03:00
if c.SkipSnapshot && (c.numTestHelpers > 0 || c.numTryTestHelpers > 0) {
panic(fmt.Sprintf("config %q's SkipSnapshot is not compatible with sharded test helpers", c.Name))
}
all: misc cleanup, docs, actually add Alpine builder -- move policy of which builders are trybots out of coordinator and into dashboard/builders.go. -- move some GCE-specific code from coordinator.go to gce.go. -- rename an old "watcher" reference to "gitmirror" -- add some docs -- actually add the Alpine builder, missing from https://golang.org/cl/33890 Fixes golang/go#17891 Change-Id: Ia63671ca09aec322ed57b3663e0ac5042cdc56f2 Reviewed-on: https://go-review.googlesource.com/40395 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2017-04-12 03:35:37 +03:00
types := 0
all: rename Kube to Container and IsGCE to IsVM Once containers run on COS instead of Kubernetes, one name (Kube*) is wrong and the other (GCE) is ambiguous. So rename them now to be more specific. No behavior changes. Just renaming in this step, to reduce size of next CL. Updates golang/go#25108 Change-Id: Ib09eb682ef74acbbf6ed50b46074f834ef5e0c0b Reviewed-on: https://go-review.googlesource.com/111639 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-05-05 19:36:05 +03:00
for _, fn := range []func() bool{c.IsReverse, c.IsContainer, c.IsVM} {
all: misc cleanup, docs, actually add Alpine builder -- move policy of which builders are trybots out of coordinator and into dashboard/builders.go. -- move some GCE-specific code from coordinator.go to gce.go. -- rename an old "watcher" reference to "gitmirror" -- add some docs -- actually add the Alpine builder, missing from https://golang.org/cl/33890 Fixes golang/go#17891 Change-Id: Ia63671ca09aec322ed57b3663e0ac5042cdc56f2 Reviewed-on: https://go-review.googlesource.com/40395 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2017-04-12 03:35:37 +03:00
if fn() {
types++
}
}
if types != 1 {
all: use Container-Optimized VMs instead of Kubernetes for buildlet containers Fixes golang/go#25108 Change-Id: I084669b52b699700ed26a7fdd890d9205a8b9dc9 Reviewed-on: https://go-review.googlesource.com/111267 Reviewed-by: Andrew Bonventre <andybons@golang.org>
2018-05-04 05:42:17 +03:00
panic(fmt.Sprintf("build config %q host type inconsistent (must be Reverse, Image, or VM)", c.Name))
all: misc cleanup, docs, actually add Alpine builder -- move policy of which builders are trybots out of coordinator and into dashboard/builders.go. -- move some GCE-specific code from coordinator.go to gce.go. -- rename an old "watcher" reference to "gitmirror" -- add some docs -- actually add the Alpine builder, missing from https://golang.org/cl/33890 Fixes golang/go#17891 Change-Id: Ia63671ca09aec322ed57b3663e0ac5042cdc56f2 Reviewed-on: https://go-review.googlesource.com/40395 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2017-04-12 03:35:37 +03:00
}
all: change dashboard.BuildConfig to be a pointer type, like HostConfig BuildConfig grew way too large and has too many slices & such to be a value type. Make it a pointer type, which matches HostConfig. Change-Id: Ie625bece9d6d8c1ec6cff26e77416bd9b1f256d8 Reviewed-on: https://go-review.googlesource.com/c/145077 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2018-10-26 20:52:25 +03:00
Builders[c.Name] = &c
dashboard: move builder configuration out of coordinator into a package The new package will be used by the buildlet command line tool in the future. Also use the metadata package in the coordinator for the master build key, since it was already depending on that package. Might as well use it. Change-Id: I6ad7593787ebba364090951ccbaecb4aa38e1dd6 Reviewed-on: https://go-review.googlesource.com/2920 Reviewed-by: Andrew Gerrand <adg@golang.org>
2015-01-15 23:46:22 +03:00
}
all: misc cleanup, docs, actually add Alpine builder -- move policy of which builders are trybots out of coordinator and into dashboard/builders.go. -- move some GCE-specific code from coordinator.go to gce.go. -- rename an old "watcher" reference to "gitmirror" -- add some docs -- actually add the Alpine builder, missing from https://golang.org/cl/33890 Fixes golang/go#17891 Change-Id: Ia63671ca09aec322ed57b3663e0ac5042cdc56f2 Reviewed-on: https://go-review.googlesource.com/40395 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2017-04-12 03:35:37 +03:00
dashboard: promote misc-compile-darwinarm64 builder to a TryBot Testing shows that misc-compile-darwinarm64 is working as expected on recent commits. Promote it to a normal misc-compile TryBot now, with the exception that it needs to run on Go 1.16 and newer only. Fixes golang/go#42341. Change-Id: Ib4de19a2ff904fb9de056db5076ac63690eb934c Reviewed-on: https://go-review.googlesource.com/c/build/+/267123 Trust: Dmitri Shuralyov <dmitshur@golang.org> Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-11-03 06:25:51 +03:00
// tryNewMiscCompile is an intermediate step towards adding a real addMiscCompile TryBot.
// It adds a post-submit-only builder with KnownIssue, GoDeps set to the provided values,
// and runs on a limited set of branches to get test results without potential disruption
// for contributors. It can be modified as needed when onboarding a misc-compile builder.
func tryNewMiscCompile(suffix, rx string, knownIssue int, goDeps []string) {
if knownIssue == 0 {
panic("tryNewMiscCompile: knownIssue parameter must be non-zero")
}
addBuilder(BuildConfig{
Name: "misc-compile" + suffix,
dashboard: update Debian version for amd64 builders This change updates various linux-amd64 builders after the release of bullseye: - linux-amd64 has been updated to bullseye. - linux-amd64-nocgo has been updated to bullseye. - linux-amd64-noopt has been updated to bullseye. - linux-amd64-ssacheck has been updated to bullseye. - linux-amd64-racecompile has been updated to bullseye. - linux-amd64-race has been updated to bullseye. - linux-amd64-jessie has had the notes updated. - misc-compile has been updated to bullseye. Fixes golang/go#47721 Change-Id: I29c5d76da6ad6b28647bd423a8ef7dc13b919ca2 Reviewed-on: https://go-review.googlesource.com/c/build/+/343633 Trust: Carlos Amedee <carlos@golang.org> Run-TryBot: Carlos Amedee <carlos@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2021-08-19 20:27:43 +03:00
HostType: "host-linux-bullseye",
dashboard: promote misc-compile-darwinarm64 builder to a TryBot Testing shows that misc-compile-darwinarm64 is working as expected on recent commits. Promote it to a normal misc-compile TryBot now, with the exception that it needs to run on Go 1.16 and newer only. Fixes golang/go#42341. Change-Id: Ib4de19a2ff904fb9de056db5076ac63690eb934c Reviewed-on: https://go-review.googlesource.com/c/build/+/267123 Trust: Dmitri Shuralyov <dmitshur@golang.org> Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-11-03 06:25:51 +03:00
buildsRepo: func(repo, branch, goBranch string) bool { return repo == "go" && branch == "master" },
KnownIssue: knownIssue,
GoDeps: goDeps,
env: []string{"GO_DISABLE_OUTBOUND_NETWORK=1"},
CompileOnly: true,
Notes: fmt.Sprintf("Tries buildall.bash to cross-compile & vet std+cmd packages for "+rx+", but doesn't run any tests. See golang.org/issue/%d.", knownIssue),
allScriptArgs: []string{
// Filtering pattern to buildall.bash:
rx,
},
})
}
dashboard: refactor mechanism for adjusting cmd/dist test policy Previously, the behavior of ShouldRunDistTest could be overridden by an individual builder by setting a the shouldRunDistTest field to a custom policy implementation. A pitfall of this approach is that it requires each custom policy to reimplement the default cmd/dist test policy as well. If that isn't done when attempting to make a small adjustment to the default policy, the result is that the default policy is ignored. The fasterTrybots policy, as the name suggests, exists to skip some tests in trybot mode. Due to the problem above, it was causing some builders to run more tests in trybot mode than they would if it weren't applied. For example, the freebsd-amd64-12_0 builder should have been skipping the "api" test in trybot mode, but wasn't. This refactor fixes that by modifying the approach for how individual builders can customize the cmd/dist test behavior. Now, a function distTestAdjust is called with the first parameter specifying the default policy decision. That function can adjust the decision as it sees fit, but otherwise it preserves its initial value in situations where no change is intended to be made. For golang/go#38279. Change-Id: I220248f51abfc8e77195ae0c9134a17f7cac9bc2 Reviewed-on: https://go-review.googlesource.com/c/build/+/227777 Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-04-09 19:47:03 +03:00
// fasterTrybots is a distTestAdjust policy function.
// It skips (returns false) the test/ directory and reboot tests for trybots.
cmd/coordinator, dashboard: don't skip cmd/dist tests for SlowBots When a user explicitly requests a SlowBot builder via the TRY= syntax, it is a signal that they're willing to wait longer for test results. It's more useful to not skip any tests and give the user confidence that the SlowBot run will catch all issues that would otherwise be caught only by post-submit builders after the CL is submitted. This is done by considering a TryBot that isn't an explicit SlowBot to be a "normal TryBot", and the ShouldRunDistTest policy function is provided a parameter specifying whether it's a normal TryBot (non-SlowBot) run. Tests are only skipped during normal TryBot runs, not SlowBot runs. Fixes golang/go#38279. For golang/go#34501. Change-Id: Ia11fd51c71f9de1089b6b6f4c027014893f224b9 Reviewed-on: https://go-review.googlesource.com/c/build/+/227779 Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-04-09 23:24:30 +03:00
func fasterTrybots(run bool, distTest string, isNormalTry bool) bool {
if isNormalTry {
dashboard: refactor mechanism for adjusting cmd/dist test policy Previously, the behavior of ShouldRunDistTest could be overridden by an individual builder by setting a the shouldRunDistTest field to a custom policy implementation. A pitfall of this approach is that it requires each custom policy to reimplement the default cmd/dist test policy as well. If that isn't done when attempting to make a small adjustment to the default policy, the result is that the default policy is ignored. The fasterTrybots policy, as the name suggests, exists to skip some tests in trybot mode. Due to the problem above, it was causing some builders to run more tests in trybot mode than they would if it weren't applied. For example, the freebsd-amd64-12_0 builder should have been skipping the "api" test in trybot mode, but wasn't. This refactor fixes that by modifying the approach for how individual builders can customize the cmd/dist test behavior. Now, a function distTestAdjust is called with the first parameter specifying the default policy decision. That function can adjust the decision as it sees fit, but otherwise it preserves its initial value in situations where no change is intended to be made. For golang/go#38279. Change-Id: I220248f51abfc8e77195ae0c9134a17f7cac9bc2 Reviewed-on: https://go-review.googlesource.com/c/build/+/227777 Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-04-09 19:47:03 +03:00
if strings.HasPrefix(distTest, "test:") || distTest == "reboot" {
builders: skip the reboot test in more places, especially on slower builders It's not important to run the new "reboot" bootstrapping test on all builders; a few is fine. Skip it on plan9-arm, etc, and on trybots that aren't super widely sharded. Change-Id: I35d5408eb5e034f42df5976f2b9b7f7d478d580a Reviewed-on: https://go-review.googlesource.com/c/build/+/167998 Reviewed-by: David du Colombier <0intro@gmail.com>
2019-03-18 01:28:15 +03:00
return false // skip test
}
cmd/coordinator, dashboard: add mac race builder, skip redundant trybot tests This adds a darwin-amd64-race builder. Of the four platforms that support the race detector, this was the only one missing a builder. To make up for the extra work, skip some redundant tests that the Mac (and other) builds do. The test directory is really slow, and doesn't have platform-specific code. We have amd64 & 386 test coverage for that directory via the Linux builders, which is fine. No need to do it on some of the heavier VM builders, or for trybots. Fixes golang/go#17674 Change-Id: I888119b3fdd28884b87a59d78746ab1fef6960c7 Reviewed-on: https://go-review.googlesource.com/82395 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2017-12-07 03:40:40 +03:00
}
dashboard: refactor mechanism for adjusting cmd/dist test policy Previously, the behavior of ShouldRunDistTest could be overridden by an individual builder by setting a the shouldRunDistTest field to a custom policy implementation. A pitfall of this approach is that it requires each custom policy to reimplement the default cmd/dist test policy as well. If that isn't done when attempting to make a small adjustment to the default policy, the result is that the default policy is ignored. The fasterTrybots policy, as the name suggests, exists to skip some tests in trybot mode. Due to the problem above, it was causing some builders to run more tests in trybot mode than they would if it weren't applied. For example, the freebsd-amd64-12_0 builder should have been skipping the "api" test in trybot mode, but wasn't. This refactor fixes that by modifying the approach for how individual builders can customize the cmd/dist test behavior. Now, a function distTestAdjust is called with the first parameter specifying the default policy decision. That function can adjust the decision as it sees fit, but otherwise it preserves its initial value in situations where no change is intended to be made. For golang/go#38279. Change-Id: I220248f51abfc8e77195ae0c9134a17f7cac9bc2 Reviewed-on: https://go-review.googlesource.com/c/build/+/227777 Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-04-09 19:47:03 +03:00
return run
cmd/coordinator, dashboard: add mac race builder, skip redundant trybot tests This adds a darwin-amd64-race builder. Of the four platforms that support the race detector, this was the only one missing a builder. To make up for the extra work, skip some redundant tests that the Mac (and other) builds do. The test directory is really slow, and doesn't have platform-specific code. We have amd64 & 386 test coverage for that directory via the Linux builders, which is fine. No need to do it on some of the heavier VM builders, or for trybots. Fixes golang/go#17674 Change-Id: I888119b3fdd28884b87a59d78746ab1fef6960c7 Reviewed-on: https://go-review.googlesource.com/82395 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2017-12-07 03:40:40 +03:00
}
dashboard: refactor mechanism for adjusting cmd/dist test policy Previously, the behavior of ShouldRunDistTest could be overridden by an individual builder by setting a the shouldRunDistTest field to a custom policy implementation. A pitfall of this approach is that it requires each custom policy to reimplement the default cmd/dist test policy as well. If that isn't done when attempting to make a small adjustment to the default policy, the result is that the default policy is ignored. The fasterTrybots policy, as the name suggests, exists to skip some tests in trybot mode. Due to the problem above, it was causing some builders to run more tests in trybot mode than they would if it weren't applied. For example, the freebsd-amd64-12_0 builder should have been skipping the "api" test in trybot mode, but wasn't. This refactor fixes that by modifying the approach for how individual builders can customize the cmd/dist test behavior. Now, a function distTestAdjust is called with the first parameter specifying the default policy decision. That function can adjust the decision as it sees fit, but otherwise it preserves its initial value in situations where no change is intended to be made. For golang/go#38279. Change-Id: I220248f51abfc8e77195ae0c9134a17f7cac9bc2 Reviewed-on: https://go-review.googlesource.com/c/build/+/227777 Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-04-09 19:47:03 +03:00
// noTestDirAndNoReboot is a distTestAdjust policy function.
// It skips (returns false) the test/ directory and reboot tests for all builds.
cmd/coordinator, dashboard: don't skip cmd/dist tests for SlowBots When a user explicitly requests a SlowBot builder via the TRY= syntax, it is a signal that they're willing to wait longer for test results. It's more useful to not skip any tests and give the user confidence that the SlowBot run will catch all issues that would otherwise be caught only by post-submit builders after the CL is submitted. This is done by considering a TryBot that isn't an explicit SlowBot to be a "normal TryBot", and the ShouldRunDistTest policy function is provided a parameter specifying whether it's a normal TryBot (non-SlowBot) run. Tests are only skipped during normal TryBot runs, not SlowBot runs. Fixes golang/go#38279. For golang/go#34501. Change-Id: Ia11fd51c71f9de1089b6b6f4c027014893f224b9 Reviewed-on: https://go-review.googlesource.com/c/build/+/227779 Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-04-09 23:24:30 +03:00
func noTestDirAndNoReboot(run bool, distTest string, isNormalTry bool) bool {
builders: skip the reboot test in more places, especially on slower builders It's not important to run the new "reboot" bootstrapping test on all builders; a few is fine. Skip it on plan9-arm, etc, and on trybots that aren't super widely sharded. Change-Id: I35d5408eb5e034f42df5976f2b9b7f7d478d580a Reviewed-on: https://go-review.googlesource.com/c/build/+/167998 Reviewed-by: David du Colombier <0intro@gmail.com>
2019-03-18 01:28:15 +03:00
if strings.HasPrefix(distTest, "test:") || distTest == "reboot" {
cmd/coordinator, dashboard: add mac race builder, skip redundant trybot tests This adds a darwin-amd64-race builder. Of the four platforms that support the race detector, this was the only one missing a builder. To make up for the extra work, skip some redundant tests that the Mac (and other) builds do. The test directory is really slow, and doesn't have platform-specific code. We have amd64 & 386 test coverage for that directory via the Linux builders, which is fine. No need to do it on some of the heavier VM builders, or for trybots. Fixes golang/go#17674 Change-Id: I888119b3fdd28884b87a59d78746ab1fef6960c7 Reviewed-on: https://go-review.googlesource.com/82395 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2017-12-07 03:40:40 +03:00
return false // skip test
}
dashboard: refactor mechanism for adjusting cmd/dist test policy Previously, the behavior of ShouldRunDistTest could be overridden by an individual builder by setting a the shouldRunDistTest field to a custom policy implementation. A pitfall of this approach is that it requires each custom policy to reimplement the default cmd/dist test policy as well. If that isn't done when attempting to make a small adjustment to the default policy, the result is that the default policy is ignored. The fasterTrybots policy, as the name suggests, exists to skip some tests in trybot mode. Due to the problem above, it was causing some builders to run more tests in trybot mode than they would if it weren't applied. For example, the freebsd-amd64-12_0 builder should have been skipping the "api" test in trybot mode, but wasn't. This refactor fixes that by modifying the approach for how individual builders can customize the cmd/dist test behavior. Now, a function distTestAdjust is called with the first parameter specifying the default policy decision. That function can adjust the decision as it sees fit, but otherwise it preserves its initial value in situations where no change is intended to be made. For golang/go#38279. Change-Id: I220248f51abfc8e77195ae0c9134a17f7cac9bc2 Reviewed-on: https://go-review.googlesource.com/c/build/+/227777 Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-04-09 19:47:03 +03:00
return run
cmd/coordinator, dashboard: add mac race builder, skip redundant trybot tests This adds a darwin-amd64-race builder. Of the four platforms that support the race detector, this was the only one missing a builder. To make up for the extra work, skip some redundant tests that the Mac (and other) builds do. The test directory is really slow, and doesn't have platform-specific code. We have amd64 & 386 test coverage for that directory via the Linux builders, which is fine. No need to do it on some of the heavier VM builders, or for trybots. Fixes golang/go#17674 Change-Id: I888119b3fdd28884b87a59d78746ab1fef6960c7 Reviewed-on: https://go-review.googlesource.com/82395 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2017-12-07 03:40:40 +03:00
}
dashboard, cmd/coordinator: move more config out of coordinator into dashboard And with the increased flexibility, enable trybots for more repos for the "sys" project. Details: https://github.com/golang/go/issues/24893#issuecomment-433517519 Fixes golang/go#24893 Change-Id: I89ce6f219be986201332f8fad11c774bdd6246da Reviewed-on: https://go-review.googlesource.com/c/145157 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2018-10-26 22:21:58 +03:00
dashboard: refactor mechanism for adjusting cmd/dist test policy Previously, the behavior of ShouldRunDistTest could be overridden by an individual builder by setting a the shouldRunDistTest field to a custom policy implementation. A pitfall of this approach is that it requires each custom policy to reimplement the default cmd/dist test policy as well. If that isn't done when attempting to make a small adjustment to the default policy, the result is that the default policy is ignored. The fasterTrybots policy, as the name suggests, exists to skip some tests in trybot mode. Due to the problem above, it was causing some builders to run more tests in trybot mode than they would if it weren't applied. For example, the freebsd-amd64-12_0 builder should have been skipping the "api" test in trybot mode, but wasn't. This refactor fixes that by modifying the approach for how individual builders can customize the cmd/dist test behavior. Now, a function distTestAdjust is called with the first parameter specifying the default policy decision. That function can adjust the decision as it sees fit, but otherwise it preserves its initial value in situations where no change is intended to be made. For golang/go#38279. Change-Id: I220248f51abfc8e77195ae0c9134a17f7cac9bc2 Reviewed-on: https://go-review.googlesource.com/c/build/+/227777 Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-04-09 19:47:03 +03:00
// ppc64DistTestPolicy is a distTestAdjust policy function
dashboard: skip reboot test on ppc64x builders Updates golang/go#35233 Change-Id: Icfa19da5d0dd3c47ffa94d11f069b8836c86cdcc Reviewed-on: https://go-review.googlesource.com/c/build/+/204038 Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-10-29 18:02:19 +03:00
// that's shared by linux-ppc64le, -ppc64le-power9osu, and -ppc64.
cmd/coordinator, dashboard: don't skip cmd/dist tests for SlowBots When a user explicitly requests a SlowBot builder via the TRY= syntax, it is a signal that they're willing to wait longer for test results. It's more useful to not skip any tests and give the user confidence that the SlowBot run will catch all issues that would otherwise be caught only by post-submit builders after the CL is submitted. This is done by considering a TryBot that isn't an explicit SlowBot to be a "normal TryBot", and the ShouldRunDistTest policy function is provided a parameter specifying whether it's a normal TryBot (non-SlowBot) run. Tests are only skipped during normal TryBot runs, not SlowBot runs. Fixes golang/go#38279. For golang/go#34501. Change-Id: Ia11fd51c71f9de1089b6b6f4c027014893f224b9 Reviewed-on: https://go-review.googlesource.com/c/build/+/227779 Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-04-09 23:24:30 +03:00
func ppc64DistTestPolicy(run bool, distTest string, isNormalTry bool) bool {
dashboard: skip reboot test on ppc64x builders Updates golang/go#35233 Change-Id: Icfa19da5d0dd3c47ffa94d11f069b8836c86cdcc Reviewed-on: https://go-review.googlesource.com/c/build/+/204038 Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-10-29 18:02:19 +03:00
if distTest == "reboot" {
// Skip test. It seems to use a lot of memory?
// See https://golang.org/issue/35233.
return false
}
dashboard: refactor mechanism for adjusting cmd/dist test policy Previously, the behavior of ShouldRunDistTest could be overridden by an individual builder by setting a the shouldRunDistTest field to a custom policy implementation. A pitfall of this approach is that it requires each custom policy to reimplement the default cmd/dist test policy as well. If that isn't done when attempting to make a small adjustment to the default policy, the result is that the default policy is ignored. The fasterTrybots policy, as the name suggests, exists to skip some tests in trybot mode. Due to the problem above, it was causing some builders to run more tests in trybot mode than they would if it weren't applied. For example, the freebsd-amd64-12_0 builder should have been skipping the "api" test in trybot mode, but wasn't. This refactor fixes that by modifying the approach for how individual builders can customize the cmd/dist test behavior. Now, a function distTestAdjust is called with the first parameter specifying the default policy decision. That function can adjust the decision as it sees fit, but otherwise it preserves its initial value in situations where no change is intended to be made. For golang/go#38279. Change-Id: I220248f51abfc8e77195ae0c9134a17f7cac9bc2 Reviewed-on: https://go-review.googlesource.com/c/build/+/227777 Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-04-09 19:47:03 +03:00
return run
dashboard: skip reboot test on ppc64x builders Updates golang/go#35233 Change-Id: Icfa19da5d0dd3c47ffa94d11f069b8836c86cdcc Reviewed-on: https://go-review.googlesource.com/c/build/+/204038 Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-10-29 18:02:19 +03:00
}
dashboard: refactor mechanism for adjusting cmd/dist test policy Previously, the behavior of ShouldRunDistTest could be overridden by an individual builder by setting a the shouldRunDistTest field to a custom policy implementation. A pitfall of this approach is that it requires each custom policy to reimplement the default cmd/dist test policy as well. If that isn't done when attempting to make a small adjustment to the default policy, the result is that the default policy is ignored. The fasterTrybots policy, as the name suggests, exists to skip some tests in trybot mode. Due to the problem above, it was causing some builders to run more tests in trybot mode than they would if it weren't applied. For example, the freebsd-amd64-12_0 builder should have been skipping the "api" test in trybot mode, but wasn't. This refactor fixes that by modifying the approach for how individual builders can customize the cmd/dist test behavior. Now, a function distTestAdjust is called with the first parameter specifying the default policy decision. That function can adjust the decision as it sees fit, but otherwise it preserves its initial value in situations where no change is intended to be made. For golang/go#38279. Change-Id: I220248f51abfc8e77195ae0c9134a17f7cac9bc2 Reviewed-on: https://go-review.googlesource.com/c/build/+/227777 Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-04-09 19:47:03 +03:00
// mipsDistTestPolicy is a distTestAdjust policy function
dashboard: add MIPS builders at RT-RK Updates golang/go#31217 Change-Id: I3c947fa1dcd0f1d938ffa4dfe565a35b2ccc3533 Reviewed-on: https://go-review.googlesource.com/c/build/+/204043 Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-10-29 23:35:07 +03:00
// that's shared by the slow mips builders.
cmd/coordinator, dashboard: don't skip cmd/dist tests for SlowBots When a user explicitly requests a SlowBot builder via the TRY= syntax, it is a signal that they're willing to wait longer for test results. It's more useful to not skip any tests and give the user confidence that the SlowBot run will catch all issues that would otherwise be caught only by post-submit builders after the CL is submitted. This is done by considering a TryBot that isn't an explicit SlowBot to be a "normal TryBot", and the ShouldRunDistTest policy function is provided a parameter specifying whether it's a normal TryBot (non-SlowBot) run. Tests are only skipped during normal TryBot runs, not SlowBot runs. Fixes golang/go#38279. For golang/go#34501. Change-Id: Ia11fd51c71f9de1089b6b6f4c027014893f224b9 Reviewed-on: https://go-review.googlesource.com/c/build/+/227779 Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-04-09 23:24:30 +03:00
func mipsDistTestPolicy(run bool, distTest string, isNormalTry bool) bool {
dashboard: add MIPS builders at RT-RK Updates golang/go#31217 Change-Id: I3c947fa1dcd0f1d938ffa4dfe565a35b2ccc3533 Reviewed-on: https://go-review.googlesource.com/c/build/+/204043 Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-10-29 23:35:07 +03:00
switch distTest {
case "api", "reboot":
return false
}
dashboard: refactor mechanism for adjusting cmd/dist test policy Previously, the behavior of ShouldRunDistTest could be overridden by an individual builder by setting a the shouldRunDistTest field to a custom policy implementation. A pitfall of this approach is that it requires each custom policy to reimplement the default cmd/dist test policy as well. If that isn't done when attempting to make a small adjustment to the default policy, the result is that the default policy is ignored. The fasterTrybots policy, as the name suggests, exists to skip some tests in trybot mode. Due to the problem above, it was causing some builders to run more tests in trybot mode than they would if it weren't applied. For example, the freebsd-amd64-12_0 builder should have been skipping the "api" test in trybot mode, but wasn't. This refactor fixes that by modifying the approach for how individual builders can customize the cmd/dist test behavior. Now, a function distTestAdjust is called with the first parameter specifying the default policy decision. That function can adjust the decision as it sees fit, but otherwise it preserves its initial value in situations where no change is intended to be made. For golang/go#38279. Change-Id: I220248f51abfc8e77195ae0c9134a17f7cac9bc2 Reviewed-on: https://go-review.googlesource.com/c/build/+/227777 Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-04-09 19:47:03 +03:00
return run
dashboard: add MIPS builders at RT-RK Updates golang/go#31217 Change-Id: I3c947fa1dcd0f1d938ffa4dfe565a35b2ccc3533 Reviewed-on: https://go-review.googlesource.com/c/build/+/204043 Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-10-29 23:35:07 +03:00
}
// mipsBuildsRepoPolicy is a buildsRepo policy function
// that's shared by the slow mips builders.
func mipsBuildsRepoPolicy(repo, branch, goBranch string) bool {
switch repo {
case "go", "net", "sys":
return branch == "master" && goBranch == "master"
default:
return false
}
}
dashboard: disable api/reboot test for unmatched It takes >15m to complete api and reboot tests for Hifive unmatched which we don't need. Change-Id: Ie4535264d2e8ef4496c54fb5f412d5ec773f538a Reviewed-on: https://go-review.googlesource.com/c/build/+/346709 Trust: Meng Zhuo <mzh@golangcn.org> Run-TryBot: Meng Zhuo <mzh@golangcn.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Go Bot <gobot@golang.org>
2021-09-01 10:08:03 +03:00
// riscvDistTestPolicy is same as mipsDistTestPolicy for now.
var riscvDistTestPolicy = mipsDistTestPolicy
dashboard: add linux-loong64-3a5000 builder Updates golang/go#46229 Change-Id: Ib48e1e64d96a07d5c5282d78935124faf75c89d7 Reviewed-on: https://go-review.googlesource.com/c/build/+/347309 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org> Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Trust: Cherry Mui <cherryyz@google.com>
2021-09-02 10:51:28 +03:00
// loong64DistTestPolicy is a distTestAdjust policy function
func loong64DistTestPolicy(run bool, distTest string, isNormalTry bool) bool {
switch distTest {
case "api", "reboot":
return false
}
return run
}
// loong64BuildsRepoPolicy is a buildsRepo policy function
func loong64BuildsRepoPolicy(repo, branch, goBranch string) bool {
switch repo {
case "go", "net", "sys":
return branch == "master" && goBranch == "master"
default:
return false
}
}
dashboard, cmd/coordinator: move more config out of coordinator into dashboard And with the increased flexibility, enable trybots for more repos for the "sys" project. Details: https://github.com/golang/go/issues/24893#issuecomment-433517519 Fixes golang/go#24893 Change-Id: I89ce6f219be986201332f8fad11c774bdd6246da Reviewed-on: https://go-review.googlesource.com/c/145157 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2018-10-26 22:21:58 +03:00
// TryBuildersForProject returns the builders that should run as part of
// a TryBot set for the given project.
// The project argument is of the form "go", "net", "sys", etc.
dashboard: unify post-submit & trybot policy in builders.go, not coordinator This removes a bunch of TODOs about scattered policy and moves it all into builders.go. While here, * add a bunch of tests * unexport some things * rename some things * document some things * adjust FreeBSD policy as function of branch (per Go 1.12 being last to support FreeBSD 10.x, and to unblock CL 165801) * set GO_BUILDER_SET_GOPROXY=coordinator for reverse buildlets, which is necessary to remove the oauth2 & build special cases in the config * change Elias Naur's mobile builder to how I think he wants it (he was fighting the old system) * add $HOME on the Solaris smartos builder, which was missing & causing tests to fail lately * make the (currently failing) longtest builder have GOPROXY set * remove an allocation in version.ParseReleaseBranch Fixes golang/go#9603 Updates golang/go#14594 Change-Id: I50a23ad7cdf478c95b14bee9b3931ba361baacfa Reviewed-on: https://go-review.googlesource.com/c/build/+/166218 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-03-07 20:44:41 +03:00
// The branch is the branch of that project ("master", "release-branch.go1.12", etc)
// The goBranch is the branch of Go to use. If proj == "go", then branch == goBranch.
func TryBuildersForProject(proj, branch, goBranch string) []*BuildConfig {
dashboard, cmd/coordinator: move more config out of coordinator into dashboard And with the increased flexibility, enable trybots for more repos for the "sys" project. Details: https://github.com/golang/go/issues/24893#issuecomment-433517519 Fixes golang/go#24893 Change-Id: I89ce6f219be986201332f8fad11c774bdd6246da Reviewed-on: https://go-review.googlesource.com/c/145157 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2018-10-26 22:21:58 +03:00
var confs []*BuildConfig
for _, conf := range Builders {
dashboard: unify post-submit & trybot policy in builders.go, not coordinator This removes a bunch of TODOs about scattered policy and moves it all into builders.go. While here, * add a bunch of tests * unexport some things * rename some things * document some things * adjust FreeBSD policy as function of branch (per Go 1.12 being last to support FreeBSD 10.x, and to unblock CL 165801) * set GO_BUILDER_SET_GOPROXY=coordinator for reverse buildlets, which is necessary to remove the oauth2 & build special cases in the config * change Elias Naur's mobile builder to how I think he wants it (he was fighting the old system) * add $HOME on the Solaris smartos builder, which was missing & causing tests to fail lately * make the (currently failing) longtest builder have GOPROXY set * remove an allocation in version.ParseReleaseBranch Fixes golang/go#9603 Updates golang/go#14594 Change-Id: I50a23ad7cdf478c95b14bee9b3931ba361baacfa Reviewed-on: https://go-review.googlesource.com/c/build/+/166218 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-03-07 20:44:41 +03:00
if conf.BuildsRepoTryBot(proj, branch, goBranch) {
dashboard, cmd/coordinator: move more config out of coordinator into dashboard And with the increased flexibility, enable trybots for more repos for the "sys" project. Details: https://github.com/golang/go/issues/24893#issuecomment-433517519 Fixes golang/go#24893 Change-Id: I89ce6f219be986201332f8fad11c774bdd6246da Reviewed-on: https://go-review.googlesource.com/c/145157 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2018-10-26 22:21:58 +03:00
confs = append(confs, conf)
}
}
sort.Slice(confs, func(i, j int) bool {
return confs[i].Name < confs[j].Name
})
return confs
}
dashboard: unify post-submit & trybot policy in builders.go, not coordinator This removes a bunch of TODOs about scattered policy and moves it all into builders.go. While here, * add a bunch of tests * unexport some things * rename some things * document some things * adjust FreeBSD policy as function of branch (per Go 1.12 being last to support FreeBSD 10.x, and to unblock CL 165801) * set GO_BUILDER_SET_GOPROXY=coordinator for reverse buildlets, which is necessary to remove the oauth2 & build special cases in the config * change Elias Naur's mobile builder to how I think he wants it (he was fighting the old system) * add $HOME on the Solaris smartos builder, which was missing & causing tests to fail lately * make the (currently failing) longtest builder have GOPROXY set * remove an allocation in version.ParseReleaseBranch Fixes golang/go#9603 Updates golang/go#14594 Change-Id: I50a23ad7cdf478c95b14bee9b3931ba361baacfa Reviewed-on: https://go-review.googlesource.com/c/build/+/166218 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-03-07 20:44:41 +03:00
dashboard: treat dev branches as current If a dev branch is active, it will be current. If it is not active, it is probably not current, but we don't build on the branch so it doesn't matter anyway. Specifically, the dev.link branch is currently under active development, and it would be good to have a large set of builders run on the branch to catch problems. (E.g., AIX build was broken but not noticed.) Fixes golang/go#37953. Change-Id: Idd7e3edaaff1b4d2cee17da552a96df2b00716e9 Reviewed-on: https://go-review.googlesource.com/c/build/+/223381 Run-TryBot: Cherry Zhang <cherryyz@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2020-03-14 00:09:21 +03:00
// atLeastGo1 reports whether branch is "release-branch.go1.N" where N >= min.
// It assumes "master" and "dev.*" branches are already greater than min, and
// always includes them.
dashboard: unify post-submit & trybot policy in builders.go, not coordinator This removes a bunch of TODOs about scattered policy and moves it all into builders.go. While here, * add a bunch of tests * unexport some things * rename some things * document some things * adjust FreeBSD policy as function of branch (per Go 1.12 being last to support FreeBSD 10.x, and to unblock CL 165801) * set GO_BUILDER_SET_GOPROXY=coordinator for reverse buildlets, which is necessary to remove the oauth2 & build special cases in the config * change Elias Naur's mobile builder to how I think he wants it (he was fighting the old system) * add $HOME on the Solaris smartos builder, which was missing & causing tests to fail lately * make the (currently failing) longtest builder have GOPROXY set * remove an allocation in version.ParseReleaseBranch Fixes golang/go#9603 Updates golang/go#14594 Change-Id: I50a23ad7cdf478c95b14bee9b3931ba361baacfa Reviewed-on: https://go-review.googlesource.com/c/build/+/166218 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-03-07 20:44:41 +03:00
func atLeastGo1(branch string, min int) bool {
if branch == "master" {
return true
}
dashboard: treat dev branches as current If a dev branch is active, it will be current. If it is not active, it is probably not current, but we don't build on the branch so it doesn't matter anyway. Specifically, the dev.link branch is currently under active development, and it would be good to have a large set of builders run on the branch to catch problems. (E.g., AIX build was broken but not noticed.) Fixes golang/go#37953. Change-Id: Idd7e3edaaff1b4d2cee17da552a96df2b00716e9 Reviewed-on: https://go-review.googlesource.com/c/build/+/223381 Run-TryBot: Cherry Zhang <cherryyz@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2020-03-14 00:09:21 +03:00
if strings.HasPrefix(branch, "dev.") {
// Treat dev branches current.
// If a dev branch is active, it will be current.
// If it is not active, it doesn't matter anyway.
// TODO: dev.boringcrypto.go1.N branches may be the
// exception. Currently we only build boringcrypto
// on linux/amd64 and windows/386, which support all
// versions of Go, so it doesn't actually matter.
return true
}
dashboard: unify post-submit & trybot policy in builders.go, not coordinator This removes a bunch of TODOs about scattered policy and moves it all into builders.go. While here, * add a bunch of tests * unexport some things * rename some things * document some things * adjust FreeBSD policy as function of branch (per Go 1.12 being last to support FreeBSD 10.x, and to unblock CL 165801) * set GO_BUILDER_SET_GOPROXY=coordinator for reverse buildlets, which is necessary to remove the oauth2 & build special cases in the config * change Elias Naur's mobile builder to how I think he wants it (he was fighting the old system) * add $HOME on the Solaris smartos builder, which was missing & causing tests to fail lately * make the (currently failing) longtest builder have GOPROXY set * remove an allocation in version.ParseReleaseBranch Fixes golang/go#9603 Updates golang/go#14594 Change-Id: I50a23ad7cdf478c95b14bee9b3931ba361baacfa Reviewed-on: https://go-review.googlesource.com/c/build/+/166218 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-03-07 20:44:41 +03:00
major, minor, ok := version.ParseReleaseBranch(branch)
return ok && major == 1 && minor >= min
}
dashboard: disable darwin-amd64-10_11 builder on Go 1.15+ Go 1.15 will require macOS 10.12 and newer¹, so keep the OS X 10.11 builder enabled only on release branches for 1.14 and older. ¹ https://golang.org/doc/go1.14#darwin Fixes golang/go#37425 Change-Id: I89ab0eb56729fe16fb3a524c8f7b9bb25751d708 Reviewed-on: https://go-review.googlesource.com/c/build/+/222797 Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
2020-03-10 20:26:20 +03:00
// atMostGo1 reports whether branch is "release-branch.go1.N" where N <= max.
// It assumes "master" branch is already greater than max, and doesn't include it.
func atMostGo1(branch string, max int) bool {
major, minor, ok := version.ParseReleaseBranch(branch)
return ok && major == 1 && minor <= max
}
dashboard: unify post-submit & trybot policy in builders.go, not coordinator This removes a bunch of TODOs about scattered policy and moves it all into builders.go. While here, * add a bunch of tests * unexport some things * rename some things * document some things * adjust FreeBSD policy as function of branch (per Go 1.12 being last to support FreeBSD 10.x, and to unblock CL 165801) * set GO_BUILDER_SET_GOPROXY=coordinator for reverse buildlets, which is necessary to remove the oauth2 & build special cases in the config * change Elias Naur's mobile builder to how I think he wants it (he was fighting the old system) * add $HOME on the Solaris smartos builder, which was missing & causing tests to fail lately * make the (currently failing) longtest builder have GOPROXY set * remove an allocation in version.ParseReleaseBranch Fixes golang/go#9603 Updates golang/go#14594 Change-Id: I50a23ad7cdf478c95b14bee9b3931ba361baacfa Reviewed-on: https://go-review.googlesource.com/c/build/+/166218 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-03-07 20:44:41 +03:00
// onlyGo is a common buildsRepo policy value that only builds the main "go" repo.
func onlyGo(repo, branch, goBranch string) bool { return repo == "go" }
all: add a macOS 10.14 (Mojave) builder Fixes golang/go#27806 Change-Id: I576f563acb2c50cd0456cd4e6a1271b9aa59c9df Reviewed-on: https://go-review.googlesource.com/c/build/+/169498 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-03-27 02:04:31 +03:00
dashboard: don't build x/exp and x/mobile on the plan9 builders Updates golang/go#11811 Change-Id: I776652a3b980772f34d65d8c245a143814aadb6a Reviewed-on: https://go-review.googlesource.com/c/build/+/209338 Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-11-28 13:33:00 +03:00
// onlyMasterDefault is a common buildsRepo policy value that only builds
// default repos on the master branch.
func onlyMasterDefault(repo, branch, goBranch string) bool {
dashboard: document and clarify defaultBuildsRepoPolicy The defaultBuildsRepoPolicy function was very important in that any changes to it would affect many builders, but it was not documented well. This refactor attempts to improve the situation. It turned out better to also rename defaultBuildsRepoPolicy to buildRepoByDefault and limit its scope to report whether a given repo should be built by default. The branch and goBranch parameters were never used, and thus misleading. If they're needed in the future, we can easily re-add them. The name defaultBuildsRepoPolicy made it sound as if it was the entire "builds repo" policy, bun reality most of the policy is implemented in the BuildConfig.buildsRepoAtAll method and is non-configurable. Only a small part of the policy was in defaultBuildsRepoPolicy. For golang/go#39156. Change-Id: Ie7c6a68bd624056362dc7d964bd8d640f665b35a Reviewed-on: https://go-review.googlesource.com/c/build/+/240687 Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-07-01 21:45:10 +03:00
return branch == "master" && goBranch == "master" && buildRepoByDefault(repo)
dashboard, cmd/coordinator: remove Joyent builders Joyent.com is shutting down their public cloud, so we no longer have our GOOS=solaris or GOOS=illumos builders there. Maybe somebody will find a new place to run them. Or maybe the ports will be abandoned. We'll see. Updates golang/go#15581 Change-Id: I0590227ce61b6b298b6aa4554e5e3bc9e4c464b5 Reviewed-on: https://go-review.googlesource.com/c/build/+/200219 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-10-10 07:17:45 +03:00
}
dashboard: only test plan9 at master The plan9 port doesn't need to be tested against older Go releases. Its userbase runs at master. Change-Id: I710d98560b234c7555c7be526aa2efdb6473c439 Reviewed-on: https://go-review.googlesource.com/c/build/+/170519 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-04-03 08:03:37 +03:00
dashboard: omit the website repo on plan9 The plan9 builders have relatively slow filesystems, and the website repo is relatively filesystem-intensive — many of its tests walk and process large trees of website content. Instead of trying to precisely tune the x/website tests to pass on the plan9 builders, let's skip this repo on them. For golang/go#49338. Change-Id: Ieaf50fb6955a6b39012a23ff72e61c1c31e4ce7c Reviewed-on: https://go-review.googlesource.com/c/build/+/362975 Trust: Bryan C. Mills <bcmills@google.com> Run-TryBot: Bryan C. Mills <bcmills@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2021-11-10 19:28:50 +03:00
// plan9Default is like onlyMasterDefault, but also omits repos that are
// both filesystem-intensive and unlikely to be relevant to plan9 users.
func plan9Default(repo, branch, goBranch string) bool {
switch repo {
case "website":
// The x/website tests read and check the website code snippets,
// which require many filesystem walk and read operations.
return false
default:
return onlyMasterDefault(repo, branch, goBranch)
}
}
all: add a macOS 10.14 (Mojave) builder Fixes golang/go#27806 Change-Id: I576f563acb2c50cd0456cd4e6a1271b9aa59c9df Reviewed-on: https://go-review.googlesource.com/c/build/+/169498 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-03-27 02:04:31 +03:00
// disabledBuilder is a buildsRepo policy function that always return false.
func disabledBuilder(repo, branch, goBranch string) bool { return false }
dashboard: skip some slow tests on Macs We have limited Mac resources. It's not worth wasting time testing portable things on them. Change-Id: Iaaa1d02582da7f8435217fbb32dcb7235c560a41 Reviewed-on: https://go-review.googlesource.com/c/build/+/169681 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-03-28 02:03:50 +03:00
// macTestPolicy is the test policy for Macs.
//
// We have limited Mac resources. It's not worth wasting time testing
// portable things on them. That is, if there's a slow test that will
// still fail slowly on another builder where we have more resources
// (like linux-amd64), then there's no point testing it redundantly on
// the Macs.
cmd/coordinator, dashboard: don't skip cmd/dist tests for SlowBots When a user explicitly requests a SlowBot builder via the TRY= syntax, it is a signal that they're willing to wait longer for test results. It's more useful to not skip any tests and give the user confidence that the SlowBot run will catch all issues that would otherwise be caught only by post-submit builders after the CL is submitted. This is done by considering a TryBot that isn't an explicit SlowBot to be a "normal TryBot", and the ShouldRunDistTest policy function is provided a parameter specifying whether it's a normal TryBot (non-SlowBot) run. Tests are only skipped during normal TryBot runs, not SlowBot runs. Fixes golang/go#38279. For golang/go#34501. Change-Id: Ia11fd51c71f9de1089b6b6f4c027014893f224b9 Reviewed-on: https://go-review.googlesource.com/c/build/+/227779 Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-04-09 23:24:30 +03:00
func macTestPolicy(run bool, distTest string, isNormalTry bool) bool {
dashboard: skip some slow tests on Macs We have limited Mac resources. It's not worth wasting time testing portable things on them. Change-Id: Iaaa1d02582da7f8435217fbb32dcb7235c560a41 Reviewed-on: https://go-review.googlesource.com/c/build/+/169681 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-03-28 02:03:50 +03:00
if strings.HasPrefix(distTest, "test:") {
return false
}
switch distTest {
case "reboot", "api", "doc_progs",
"wiki", "bench_go1", "codewalk":
return false
}
cmd/coordinator, dashboard: don't skip cmd/dist tests for SlowBots When a user explicitly requests a SlowBot builder via the TRY= syntax, it is a signal that they're willing to wait longer for test results. It's more useful to not skip any tests and give the user confidence that the SlowBot run will catch all issues that would otherwise be caught only by post-submit builders after the CL is submitted. This is done by considering a TryBot that isn't an explicit SlowBot to be a "normal TryBot", and the ShouldRunDistTest policy function is provided a parameter specifying whether it's a normal TryBot (non-SlowBot) run. Tests are only skipped during normal TryBot runs, not SlowBot runs. Fixes golang/go#38279. For golang/go#34501. Change-Id: Ia11fd51c71f9de1089b6b6f4c027014893f224b9 Reviewed-on: https://go-review.googlesource.com/c/build/+/227779 Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-04-09 23:24:30 +03:00
if isNormalTry {
dashboard: skip some slow tests on Macs We have limited Mac resources. It's not worth wasting time testing portable things on them. Change-Id: Iaaa1d02582da7f8435217fbb32dcb7235c560a41 Reviewed-on: https://go-review.googlesource.com/c/build/+/169681 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-03-28 02:03:50 +03:00
switch distTest {
case "runtime:cpu124", "race", "moved_goroot":
return false
}
// TODO: more. Look at bigquery results once we have more data.
}
dashboard: refactor mechanism for adjusting cmd/dist test policy Previously, the behavior of ShouldRunDistTest could be overridden by an individual builder by setting a the shouldRunDistTest field to a custom policy implementation. A pitfall of this approach is that it requires each custom policy to reimplement the default cmd/dist test policy as well. If that isn't done when attempting to make a small adjustment to the default policy, the result is that the default policy is ignored. The fasterTrybots policy, as the name suggests, exists to skip some tests in trybot mode. Due to the problem above, it was causing some builders to run more tests in trybot mode than they would if it weren't applied. For example, the freebsd-amd64-12_0 builder should have been skipping the "api" test in trybot mode, but wasn't. This refactor fixes that by modifying the approach for how individual builders can customize the cmd/dist test behavior. Now, a function distTestAdjust is called with the first parameter specifying the default policy decision. That function can adjust the decision as it sees fit, but otherwise it preserves its initial value in situations where no change is intended to be made. For golang/go#38279. Change-Id: I220248f51abfc8e77195ae0c9134a17f7cac9bc2 Reviewed-on: https://go-review.googlesource.com/c/build/+/227777 Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-04-09 19:47:03 +03:00
return run
dashboard: skip some slow tests on Macs We have limited Mac resources. It's not worth wasting time testing portable things on them. Change-Id: Iaaa1d02582da7f8435217fbb32dcb7235c560a41 Reviewed-on: https://go-review.googlesource.com/c/build/+/169681 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-03-28 02:03:50 +03:00
}