CL 191378 removed some +build appengine tags from
./analysis/appengine, so those packages now require
GO111MODULE=on in order to resolve the dependency
on google.golang.org/appengine.
Fixesgolang/go#33952
Change-Id: I20c9ba3f42d488265726fda9b1c790e48a1433b3
Reviewed-on: https://go-review.googlesource.com/c/build/+/192333
Run-TryBot: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Alexander Rakoczy <alex@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Issue golang.org/issue/30612 implemented a restriction on outbound
network access to help catch unintended uses of internet in tests.
We're currently relying on GOPROXY=https://proxy.golang.org for running
tests in module mode in Go 1.13+, but proxy.golang.org can serve some
module zip files by 302 redirecting to other URLs. Those URLs may not
be accessible due to this builder network restriction.
Disable the outbound network restriction as the first step to return
builders to a working state. The next step will be implement a long
term solution, then we can re-enable the network restriction again.
Updates golang/go#33928
Updates golang/go#30612
Change-Id: I1a87b8d107df8806753117fc60a3e795e1e1b182
Reviewed-on: https://go-review.googlesource.com/c/build/+/192317
Reviewed-by: Alexander Rakoczy <alex@golang.org>
They don't have filesystem access implemented, nor can they compile
Go code (as currently done in golangorg tests).
x/website is sufficiently tested on many other architectures, and
it's more of an internal repository without plans to support mobile.
Fixesgolang/go#33935
Change-Id: I6e3d06a72aeb865b9532fcf2dc699d8f180c0c2e
Reviewed-on: https://go-review.googlesource.com/c/build/+/192318
Reviewed-by: Bryan C. Mills <bcmills@google.com>
This app is now running as a service called "build" instead of
on a named version in the default service. Update the app.yaml and
deployment instructions to account for this.
Change-Id: I5d68128e89bad6782982380b2012300a492ddb34
Reviewed-on: https://go-review.googlesource.com/c/build/+/191975
Reviewed-by: Alexander Rakoczy <alex@golang.org>
This reverts commit CL 190902.
Reason for revert: it breaks deploys due to filesystem changes in go111
Change-Id: Ic357050da2707be1c53d79af6f0062ff4feaffbb
Reviewed-on: https://go-review.googlesource.com/c/build/+/191974
Reviewed-by: Alexander Rakoczy <alex@golang.org>
release-branch.go1.12 has failures for darwin-386-10_14, which are fixed
in Go 1.13. darwin-386 is not a first-class port and on a deprecated
architecture for macOS, and can be skipped on older branches.
Fixesgolang/go#33756
Change-Id: I5f2e95fe9515978a85b986787d8126655f849629
Reviewed-on: https://go-review.googlesource.com/c/build/+/191878
Run-TryBot: Alexander Rakoczy <alex@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
We don't want any files in the release staging directory from a
previous releasebot run to have a chance to influence a future run.
Start using a temporary directory inside <work>/release-staging to
prevent that from happening.
I considered cleaning the <work>/release-staging directory, but relying
on new temporary directories instead of os.RemoveAll seems safer. Users
can clean their go-releasebot-work directory themselves if they wish.
Change-Id: I2ca38267559aa356992faf7cbec9441c102aba45
Reviewed-on: https://go-review.googlesource.com/c/build/+/191166
Reviewed-by: Alexander Rakoczy <alex@golang.org>
If the build releases step fails, stop there, instead of proceeding
or printing misleading next steps. Those next steps apply only after
the build releases step succeeds.
Also add a sentence at the end of the GitHub comment posted on the
release issue when there was a problem, so it's easier to see.
Fixesgolang/go#33764
Updates golang/go#33760
Change-Id: Ice9ef155cd64173a2f48f96efc809634628d9165
Reviewed-on: https://go-review.googlesource.com/c/build/+/191162
Reviewed-by: Alexander Rakoczy <alex@golang.org>
Using a directory inside the system temporary directory for staging
releases can be problematic, because os.Rename may fail to move the
file if /tmp is mounted on a filesystem that is different from where
the releasebot work directory is.
Use $HOME/go-releasebot-work/goX.Y.Z/release-staging as the staging
directory instead.
Change-Id: I74f5f6ace1c0df154a1de989e825f67e808d3c34
Reviewed-on: https://go-review.googlesource.com/c/build/+/191159
Reviewed-by: Andrew Bonventre <andybons@golang.org>
Reviewed-by: Alexander Rakoczy <alex@golang.org>
Binary releases need to build Go and include binaries such as bin/go,
bin/gofmt, and others. Previously, this was accomplished by running
all.bash script for some GOOS/GOARCH pairs, and make.bash for others
where it wasn't viable to run tests as part of the release process.
This change makes the release process more consistent by always
packaging the release archive file after running make.bash. We still
run all.bash in situations where it was previously run, but we do so
after the release file has already been created. This avoids the
risk of any changes to GOROOT that may occur as part of all.bash
(including changing file permissions to be read-only) being included
in the final release file.
Add a step to check that files in the buildlet's $WORKDIR/go and
$WORKDIR/go/bin directories have expected permissions before
creating the release file.
Fixesgolang/go#33537
Updates golang/go#30316
Change-Id: I7d40716dba656a8aca711377f2995df4880166c5
Reviewed-on: https://go-review.googlesource.com/c/build/+/189537
Reviewed-by: Andrew Bonventre <andybons@golang.org>
In 2016, there were problems with git mirroring being slow and flaky,
as described in golang/go#16388. In order to resolve that problem,
CL 25110 added a custom mirroring implementation that effectively
split up the work done by a single git push -f --mirror dest command
into smaller batches of work. It was noted in the commit message and
updated documentation of CL 127315 that it may no longer be needed in
newer versions of git, and that the problem seemed to affect the HTTPS
transport, not so much the SSH transport.
By now, the following conditions are different:
• It's 2019.
• We're using a newer version of git.
• We're using the SSH transport (which may or may not be better than
the HTTPS transport).
• We are mirroring only the refs/heads/* and refs/tags/* namespaces,
not refs/changes/* which contains an order of magnitude more refs.
As a result, it should be possible to revert to a simple mirroring
implementation that uses a single git push -f --mirror dest command.
This removes complexity in our code, leveraging the git binary to do
more for us (at the expense of relinquishing tighter control and
ability to do things differently if we start encountering problems).
It resolves the issue of deleted refs not being deleted from mirrors
(golang/go#23099) without having to add even more complexity.
Debian 10 (buster) is the current stable release of Debian.
Start using it, since it comes with a newer version of git
(git version 2.20.1, compared to git version 2.11.0 in Debian 9),
which we want in hopes of git being better capable of pushing
a large number of refs without problems.
If we run into problems again, we might have to add some of the
complexity back, but hopefully that won't happen. Remove the old
and unused code since it's easy to get it back from VCS history.
Fixesgolang/go#23099Fixesgolang/go#29560
Updates golang/go#16388
Change-Id: Id63419a5e4df5a87d7c9dc4be4e7e38a4849b3df
Reviewed-on: https://go-review.googlesource.com/c/build/+/190897
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Alexander Rakoczy <alex@golang.org>
Use the --prune flag when running git fetch origin commands
to delete remote-tracking refs in the local repositories.
This prevents branches and tags that have been deleted on
the origin server from reappearing in the GitHub mirrors.
It will take more work to get refs automatically deleted from
the mirror repositories, because the current Repo.push uses
a custom implementation rather than git push -f --mirror dest,
and it doesn't take the need to delete refs into account yet.
See https://git-scm.com/docs/git-fetch#Documentation/git-fetch.txt---prune
for information on the --prune flag. The --prune-tags flag isn't
needed because the local repositories are cloned with --mirror.
Updates golang/go#23099
Change-Id: I762b2d1365287d5dac40236310713c3df15d59d8
Reviewed-on: https://go-review.googlesource.com/c/build/+/190879
Reviewed-by: Alexander Rakoczy <alex@golang.org>
Reviewed-by: Andrew Bonventre <andybons@golang.org>
There is no public release status issue in security release mode,
because we want to avoid the risk of leaking sensitive test failures.
Don't write the release status issue title to stdout, because that
makes it seem like there might be a public issue, which can be
worrisome to see when doing a security release.
Change-Id: Ice9b3fe1067a8d68cf8d22cd944a3b92a3abb606
Reviewed-on: https://go-review.googlesource.com/c/build/+/190801
Reviewed-by: Alexander Rakoczy <alex@golang.org>
Since we can't add another GitHub issue template specifically for gopls,
have Gopherbot add a comment requesting more information. For now, the
comment just links to the Troubleshooting wiki page, but we can make it
more sophisticated if necessary.
Change-Id: Iec7e5f8347caa6132416df305bbd7f2c1db45eef
Reviewed-on: https://go-review.googlesource.com/c/build/+/190401
Run-TryBot: Rebecca Stambler <rstambler@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Previously, the release binary dynamically looked for the releaselet.go
source code. It first checked in the current working directory, and
then used the go/build package to find the location of source code of
golang.org/x/build/cmd/release package on disk.
The release binary is not generally executed by hand, rather it is used
internally by releasebot command. That makes it hard to know in advance
where the current working directory for release will be (that directory
is $HOME/go-releasebot-work/<release> that releasebot creates itself).
Further, with the proliferation of module mode, it's no longer viable
to be able to find the location of a package by its import path via
go/build.Import. As a result, there's now a higher risk of cmd/release
not finding, or finding a wrong version of releaselet.go.
Change the release binary to instead statically embed the releaselet.go
source code as a constant string, in a static.go file that is generated
via a go:generate directive.
Add a test to ensure the embedded copy of releaselet.go doesn't get out
of sync.
The embedding approach was loosely based on approach taken in package
golang.org/x/tools/godoc/static, but it was simplified for the smaller
needs of cmd/release (i.e., embedding a single text file). We rely on
fmt's %q verb to do the escaping, which has the risk of changing between
Go versions, but it's unlikely to happen often to warrant worrying yet.
Fixesgolang/go#33443
Change-Id: Ie7a9481c33a7d9668d696d3827e5681b07b37094
Reviewed-on: https://go-review.googlesource.com/c/build/+/189817
Reviewed-by: Filippo Valsorda <filippo@golang.org>
I don't have corp laptop with me all the times
but still want to hack on some Go code and send people Gerrit CLs.
You can verify the email belongs to the same github account
of mine: https://github.com/x1ddos.
Change-Id: Idfe43d22b0183f27f5803a1ab3cc90cb6c859dac
Reviewed-on: https://go-review.googlesource.com/c/build/+/189399
Reviewed-by: Andrew Bonventre <andybons@golang.org>
MacStadium is migrating off the current datastore. As part of the
transition, we copied our base images to the new storage, adding _nfs as
the suffix. The new images were snapshotted to allow for copy on write.
New base image names were suffixed with _nfs to accommodate the
migration. The new storage device was named GGLGLN-A-001-STV1 by
MacStadium.
Updated vmware-notes.txt to include manual instructions for a test run
of this latest change. The VM started and successfully connected to the
coordinator.
Updates golang/go#33155
Change-Id: I69976ebc6faa4f95a0aefd4d2e4c8d6a5bcf8a0f
Reviewed-on: https://go-review.googlesource.com/c/build/+/188219
Run-TryBot: Alexander Rakoczy <alex@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
This change adds health monitoring for the tip.golang.org website
to farmer.golang.org#health, so we can know when there's an issue
with it without having to manually check it separately from every
other item already monitored at farmer.golang.org#health.
This is possible now that golang/go#32949 is resolved and
the _tipstatus page reports currently ongoing errors only.
Tested by running coordinator locally in dev mode.
Fixesgolang/go#15266
Updates golang/go#32949
Change-Id: I3680a916e4b45336df837c72ac7bc555d37f3a9b
Reviewed-on: https://go-review.googlesource.com/c/build/+/185981
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Alexander Rakoczy <alex@golang.org>
If DATASTORE_EMULATOR_HOST is set, datastore.NewClient will use
it to connect to a locally-running datastore instance. Clear it
before creating the client and set a 30-second timeout on the
context so that the process doesn't appear to hang indefinitely
when the env var is set.
Change-Id: I605ee699a55f4537c174a5667c92044936f10d74
Reviewed-on: https://go-review.googlesource.com/c/build/+/185142
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
We want the error to represent an ongoing problem. It's less useful
to constantly show the last error; we have logs for that.
This will help make the _tipstatus output more readable for humans,
and more friendly to being monitored by farmer.golang.org#health.
Updates golang/go#15266Fixesgolang/go#32949
Change-Id: I20e5f180209c54b31e81e29bd45af0e2d205d3cc
Reviewed-on: https://go-review.googlesource.com/c/build/+/185139
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
Check that the required repository HEAD revisions are present before
attempting to clone said repositories.
Use '--' to separate paths and make git invocations less ambiguous.
Updates golang/go#32949
Change-Id: Ie0c771c38a047d674fea5a74318ed396fd03c7ce
Reviewed-on: https://go-review.googlesource.com/c/build/+/185138
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
NetBSD 8.1 is a bugfix release on the stable (8) branch and has
a number of recommended updates and fixes.
Updates golang/go#31858
Change-Id: Id0f4a2180af1f84486aa6bc1c4406c5e8d5a0cb3
Reviewed-on: https://go-review.googlesource.com/c/build/+/185217
Reviewed-by: Andrew Bonventre <andybons@golang.org>
Using "https://go.googlesource.com" without "-review" as the Gerrit API
URL no longer works. Use "https://go-review.googlesource.com" instead.
This is the canonical Gerrit API URL for the Go project, and it's
already being used in many other places in x/build.
Return and handle errors from gerritMetaMap. Make it so gitmirror treats
an error from gerritMetaMap on startup as fatal, otherwise it would have
started in an incorrect state. While running, skip transient errors from
gerritMetaMap as before, but also log them when they happen.
Remove unneeded trailing slashes from the Gerrit API URLs in packages
maintner/maintnerd and maintner/maintnerd/maintapi for consistency.
This is a no-op as the gerrit client library used to trim them anyway.
Fixesgolang/go#32931
Change-Id: Icc20b798946d6317ee89533691f92b1f4d1564f0
Reviewed-on: https://go-review.googlesource.com/c/build/+/184922
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Alexander Rakoczy <alex@golang.org>
Rearrange the maintner API documentation, and reflect the guarantee of
at least two Go releases on success.
Updates golang/go#32606
Change-Id: I90d5017f280254cc6482e747ffc1534f8a51bc20
Reviewed-on: https://go-review.googlesource.com/c/build/+/182981
Run-TryBot: Alexander Rakoczy <alex@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
This is a followup to CL 180925 to prevent gopherbot from fighting
people (too much) if they decide to override gopherbot's behavior
and re-add the NeedsDecision label. It's done by adding a check for
whether gopherbot has already taken the action and avoid repeating
it if so.
Since this action is removing a label, we can't just check for any
"labeled" event, as those are likely to happen for other reasons.
So add a more precise check for whether gopherbot has previously
removed the "NeedsDecision" label from the target issue.
Updates golang/go#31788
Updates golang/go#21312
Change-Id: Iaf4dd69a5bfd637694995ee60869f94362110a7d
Reviewed-on: https://go-review.googlesource.com/c/build/+/183624
Reviewed-by: Andrew Bonventre <andybons@golang.org>
Currently, the subheading2 cells in the subrepo tables are shifted by
one to the left due to the additional cell for the Go commit. There are
4 fixed <th> cells while there are 5 <td> cells (subrepo commit, Go
commit, user, time and description).
Make the first <th> have a colspan of 2 in case of the subrepo table so
the subheading2 cells align correctly.
Change-Id: I1ca7157cecf6530cff5883154f367684b0695081
Reviewed-on: https://go-review.googlesource.com/c/build/+/183217
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
This is a followup to CL 181217 to address a minor code review comment.
Check the slice length before access, to prevent a possible panic on
unexpected input.
Also improve error string to not end with punctuation, per style guide
entry at golang.org/s/style#error-strings.
Change-Id: I220f6744396740b0e7f92705367be56c83ee23da
Reviewed-on: https://go-review.googlesource.com/c/build/+/182337
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Alexander Rakoczy <alex@golang.org>
Reviewed-by: Andrew Bonventre <andybons@golang.org>
This is a followup to CL 179877 that adds missing arguments to
a fmt.Sprintf call.
Also change the if condition to ensure the regexp had exactly 3
matches as expected. That makes it more clear there won't be
a panic when accessing matches.
Change-Id: Ibf57d87565c8297cd68dd9a63b4dc709bce613de
Reviewed-on: https://go-review.googlesource.com/c/build/+/182017
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
This adds information on warnings & errors to makemac's JSON status
handler that is then parsed by the coordinator's health checking code,
which already polls this JSON endpoint.
Updates golang/go#32449
Updates golang/go#15760
Change-Id: I69bea7b07c184d1f62a358bc317376aa97018230
Reviewed-on: https://go-review.googlesource.com/c/build/+/181217
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Support for GOOS=illumos was added in CL 174457 which will be part of Go 1.13
Change-Id: I11c5b646c6f8147185526c58b6357d8a0e607187
Reviewed-on: https://go-review.googlesource.com/c/build/+/180157
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
For years I've been rebuilding retrybuilds with a locally-edited
flakyPhrases list whenever I wanted to do one-off retries. This
finally makes it a first-class mode.
Change-Id: Ibc3101ecf3af3cdc33b9f3d8f46e65eea1161213
Reviewed-on: https://go-review.googlesource.com/c/build/+/179619
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
I flubbed my git in CL 179419. This was supposed to be included.
Also updates some logging after watching it spam during testing.
Updates golang/go#21305
Change-Id: Icbf32a064d8725c46ffddd422938bd8ff208c829
Reviewed-on: https://go-review.googlesource.com/c/build/+/179617
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
And delete an old, unused field from the dashboard package.
Change-Id: Ib3472c30d6a436c80d91d63d4136d8524c2bd093
Reviewed-on: https://go-review.googlesource.com/c/build/+/179420
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Bryan C. Mills
Dmitri Shuralyov
Andrew Bonventre
Alex Vaghin
Alexander Rakoczy
Rebecca Stambler
Katie Hockman
Benny Siegert
kawakami
Tai
Tobias Klauser
Brad Fitzpatrick