This adds scripts to create a new builder host image that acts like
Container-Optimized OS (has docker, runs konlet on startup) but with a
Debian 9 kernel + userspace that permits KVM for nested
virtualization.
Updates golang/go#15581 (solaris)
Updates golang/go#23060 (dragonfly)
Updates golang/go#30262 (riscv)
Updates golang/go#30267 (fuchsia)
Updates golang/go#23824 (android)
Change-Id: Ib1d3a250556703856083c222be2a70c4e8d91884
Reviewed-on: https://go-review.googlesource.com/c/163301
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
As part of golang/go#29206, the golang.org website is being moved to
from x/tools to x/website repository. This change updates the tip
command to start deploying it from its new location.
The new website is using Go modules for reproducible builds. This
simplifies the code, because we no longer need to manually fetch
any dependent repositories (such as x/net).
Update instances of "godoc" (the binary that was previously used to
host the website) to "golangorg", the new binary name in x/website.
Fixesgolang/go#30232
Updates golang/go#29206
Change-Id: I428eddf95a7ed6a43a25138ffeb919e79f42f9d2
Reviewed-on: https://go-review.googlesource.com/c/162908
Reviewed-by: Andrew Bonventre <andybons@golang.org>
Change https://dev.golang.org/reviews page to be more helpful
at displaying CLs that are in need of a general code review.
Don't hide CLs containing DO NOT SUBMIT. Per Interim Code Review
and Issue Tracker Conventions¹:
DO NOT SUBMIT means that while the CL is okay to review,
the author wants Gerrit to make sure the CL is not submitted
in its current form.
That means we should not be excluding CLs containing DO NOT SUBMIT
from the reviews page.
Remove a log line from GerritMeta.LabelVotes method. It's not serving
a helpful purpose at this time, but produces a large amount of output.
¹ https://groups.google.com/d/msg/golang-dev/YU56eKYiXJg/K1CkXo7Iy0gJ
Change-Id: Ibe40f756e6a0dab0a13c75fa53998e2d56d287e3
Reviewed-on: https://go-review.googlesource.com/c/163159
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
We want to be able to test cmd/tip changes on staging before pushing
to production. This change adds configuration files to be used for
the staging environment.
Mirror the production environment, but use a staging-specific load
balancer IP, domain, and autocert GCS bucket (golang-tip-dev-autocert).
Also rename the existing production configuration files to mirror the
pattern in other directories. This is especially important now that
there are both production and staging configuration files.
Updates golang/go#30232
Change-Id: I11c8cf643654d2c5bfa4add50f9ac8ca6771606b
Reviewed-on: https://go-review.googlesource.com/c/162914
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This adds a linux-amd64 COS builder that should be just like our
existing linux-amd64 COS builder except that it's using a forked image
that has the VMX license bit enabled for nested virtualization. (GCE
appears to be using the license mechanism as some sort of opt-in
mechanism for features that aren't yet GA; might go away?)
Once this is in, it won't do any new builds as regular+trybot builders
are disabled. But it means I can then use gomote + debugnewvm to work
on preparing the other four image types.
Updates golang/go#15581 (solaris)
Updates golang/go#23060 (dragonfly)
Updates golang/go#30262 (riscv)
Updates golang/go#30267 (fuchsia)
Updates golang/go#23824 (android)
Change-Id: Ic55f17eea17908dba7f58618d8cd162a2ed9b015
Reviewed-on: https://go-review.googlesource.com/c/162959
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
maintner represents Gerrit users via its underlying low-level
git author representation, such as:
Foo Bar <GerritUserID@GerritServerID>
The server ID represents the Gerrit instance and doesn't change.
The user ID uniquely identifies a Gerrit user on the Gerrit instance.
However, Gerrit is not consistent about the name it uses. Sometimes
it's the actual name, but other times it's "Gerrit User <NumericID>".
For example, both of these forms come up:
Dmitri Shuralyov <6005@62eb7196-b449-3ce5-99f1-c037f21e1705>
Gerrit User 6005 <6005@62eb7196-b449-3ce5-99f1-c037f21e1705>
Fix the author comparison logic in unwaitCLs task by comparing only
the git email of Gerrit users.
Fixesgolang/go#30172
Change-Id: Ib193de844ecc6212723344765fc920bc08d906a4
Reviewed-on: https://go-review.googlesource.com/c/161977
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
The purpose of function onLatestCL is to check that maintner data
isn't behind Gerrit's data (as determined by talking to the Gerrit
REST API directly). However, there are some meta commits that the
Gerrit REST API doesn't return a corresponding message to, so
sometimes maintner data is ahead of "latest Gerrit message".
Since ahead is not _behind_, make onLatestCL report positively in
such situations by checking all cl.Metas, not just the latest one.
Do so in reverse order as an optimization (if there's a match, it's
most likely near the end of cl.Metas slice).
An example of such a meta commit is one that updates an uploaded
patch set and sets the Reviewer field:
$ git show fc20e3671f7b1ff889d5f947b11d7fda0b608751
commit fc20e3671f7b1ff889d5f947b11d7fda0b608751 (meta/35/111335/meta)
Author: Gerrit User 14805 <14805@62eb7196-b449-3ce5-99f1-c037f21e1705>
Date: Mon Feb 11 17:49:09 2019 +0000
Update patch set 6
Patch-set: 6
Reviewer: Gerrit User 14805 <14805@62eb7196-b449-3ce5-99f1-c037f21e1705>
It has no corresponding message with same ID in the output from a Gerrit
REST API call at
https://go-review.googlesource.com/changes/111335/detail?o=MESSAGES,
nor does it show up visually in the Gerrit UI at
https://go-review.googlesource.com/c/crypto/+/111335.
Before this change (with fix in CL 161977):
2019/02/11 15:02:33 Reloaded data from log *maintner.netMutSource.
2019/02/11 15:02:34 https://golang.org/cl/111335 -- remove wait-author; reply from Andreas Auernhammer <...@mail.de>
2019/02/11 15:02:35 onLatestCL: maintner metadata for CL 111335 is behind; skipping action for now.
After this change:
2019/02/11 15:06:46 Reloaded data from log *maintner.netMutSource.
2019/02/11 15:06:46 https://golang.org/cl/111335 -- remove wait-author; reply from Andreas Auernhammer <...@mail.de>
2019/02/11 15:06:46 [dry run] would remove hashtag 'wait-author' from CL 111335
Updates golang/go#30172
Change-Id: I1575ccffbf8c3edb337f55bf1bf2c96a4a04bbdd
Reviewed-on: https://go-review.googlesource.com/c/161901
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
The githubRepoPoller type has a field c, which is a shortcut for
gr.github.c. Other githubRepoPoller methods use it, so start using
it in syncIssues as well.
This makes the code more consistent and easier to read.
Also remove a spurious period in GitHubRepo.newMutationFromIssue
documentation.
Change-Id: Iecebd2ba6c69e3479e8d627fea691fcf9415aa51
Reviewed-on: https://go-review.googlesource.com/c/160841
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
A GitHub issue can become 404. Attempting to lock it will produce a
404 response from the GitHub API. Don't treat it as a fatal error
when it happens.
Add a check for the NotExist field. This will help after golang/go#30184
is resolved.
Updates golang/go#30182
Change-Id: Ia04c59879909b1de00bd681606bfa331fe642cd4
Reviewed-on: https://go-review.googlesource.com/c/161906
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Rookie mistake: I forgot to git commit before mailing out.
Change-Id: I7b76686d620c889cea4ebc5396104432e9213cde
Reviewed-on: https://go-review.googlesource.com/c/161965
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
The buildlet's /removeall handler previously restored any deletes of
important directories (the $WORKDIR, $GOCACHE, and $TMP) immediately
after the deletes, before the /removeall handler returned to the
client.
Unfortunately, some clients actually do want to delete $GOCACHE and
$TMP: in CL 159257 (submitted too early), cmd/release wants to delete
those before tarring/zipping up the rest of $WORKDIR to ship a
release.
This CL changes the remove handler to just remove stuff without fixing
mistakes. Instead, missing directories are created lazily later if
needed.
Also, minor debug logging change to cmd/release that I hadn't yet
uploaded to CL 159257 before it was submitted.
Fixesgolang/go#29906
Change-Id: Ifb5d3c16e2d83e7ac9e7f8e353d1b7b866f7d338
Reviewed-on: https://go-review.googlesource.com/c/159320
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
This is a followup to CL 161222, where I forgot to handle the case
where a commit message has a multi-line subject and no body.
Updates golang/go#30101
Change-Id: I01222e8cf783bc3b0631a332cf379717aa85cebc
Reviewed-on: https://go-review.googlesource.com/c/161797
Reviewed-by: Andrew Bonventre <andybons@golang.org>
In git, the text up to the first blank line in a commit message
is treated as the commit subject.
Most subjects are short and fit in one line (hopefully under
50 characters). In some cases, they end up being split into
multiple lines.
Report the entire subject rather than just the first line,
and join multiple lines with spaces.
Reference: https://git-scm.com/docs/git-commit#_discussion.
Fixesgolang/go#30101
Change-Id: I1ea8212202fce0033e5ac8075352d5d7e0ae823c
Reviewed-on: https://go-review.googlesource.com/c/161222
Reviewed-by: Andrew Bonventre <andybons@golang.org>
In CL 107296, the maintner API had undergone a change such that all
public endpoints would only return GerritCLs that are considered
complete. Complete CLs are defined to have certain properties that
make working with them more friendly. One of those properties is that
a complete GerritCL always has a non-nil Commit field.
Methods of GerritCL type are expected to operate on complete CLs
that were acquired via one of the public means (e.g., ForeachOpenCL,
ForeachCLUnsorted, CL), so it's a precondition that the CL must be
complete. (If the user makes a GerritCL type from scratch, they are
expected to follow the documented requirement that GerritCL.Commit
must be non-nil.)
The Footer, ChangeID methods were already relying on the fact that
Commit is non-nil. This change updates the GerritCL.Subject method
to also make use of that property, which makes the cl.Commit nil
check not necessary.
This leads to simpler code and higher confidence that our internal
invariants are maintained.
Change-Id: I5867fd248274b8dd94a0ca98a91a773d48b3bdec
Reviewed-on: https://go-review.googlesource.com/c/161221
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Follow-up from breakage introduced in CL 160818
I keep forgetting that we have trybots here again.
Change-Id: I64bb5dc1ee033425e92bf9b2f31acb6346bbb9be
Reviewed-on: https://go-review.googlesource.com/c/161617
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Baokun Lee <nototon@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
The phrase "CL https://golang.org/issue/NNNN" does not make sense.
It was a typo in CL 39352 that implemented this functionality.
Also remove a stray trailing backslash introduced in CL 51116.
Change-Id: Ic1c9d1095c075cb6d9063a3d9e881df6b70414f0
Reviewed-on: https://go-review.googlesource.com/c/161458
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
This change largely reuses the logic to parse CL subjects used
on gochanges.org. There is no support for expanding curly brackets
like "image/{png,jpeg}", but this wasn't implemented here previously
either.
There is room for performance improvement by performing less
allocations, but performance is not a bottleneck for current
needs, so I am currently prioritizing adding support for edge
cases (unless performance proves to be a blocker).
For brevity, use x/subrepo instead of golang.org/x/subrepo form.
Omit displaying the path prefix in the title when displaying CLs
grouped under their full import path, to reduce stutter.
Fixesgolang/go#30096
Change-Id: I957661724d1a26af4dfebea1eee20e6a7595c7b9
Reviewed-on: https://go-review.googlesource.com/c/161219
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
The updatecontrib command came to be in CL 157238 by reducing the scope
of a previous updateac command. The updateac command was modified to
fit the needs of updatecontrib. The modification was aiming to be
minimal in nature, it focused on removing CLA checking code and removing
the functionality that updated the AUTHORS file.
Most of the output that the new updatecontrib command produced was only
relevant to the original updateac program, and wasn't very helpful for
the purposes of updatecontrib.
At that time, I did not have a complete understanding of the output,
so I chose to make minimal modifications that I understood. By now,
I've gained that understanding and was able to overhaul the output of
updatecontrib to make it useful, as well as refactor the code to make
it simpler and more readable. There is no change in behavior.
The output is now separated into 3 categories:
• actions - list of actions taken during the run
• warnings - list of situations where a valid-looking name couldn't be
determined, causing the git commit author to not be added
to the CONTRIBUTORS file
• errors - list of unexpected errors (e.g., a network error talking
to GitHub API)
Warnings are a normal outcome, since there are many git commit
authors that don't have a name set in any of the sources that
updatecontrib currently checks.
Errors are not expected, and are no longer treated the same way as
harmless warnings, which made output harder to read.
Factor out code that queries GitHub API into a new github.go file.
Add a new nameFix entry to use a normalized name capitalization.
Change-Id: I433ebc5afd8986709b5f323358efc93e3995b2b8
Reviewed-on: https://go-review.googlesource.com/c/160277
Reviewed-by: Andrew Bonventre <andybons@golang.org>
The build system knows how to use modules now, so no need to hide it.
Change-Id: I9c7a8258d45aa4afc78e562b371a54e2c45a98ee
Reviewed-on: https://go-review.googlesource.com/c/160818
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
We do not have any remaining old-style builders that use cmd/builder.
They've all either migrated to use the new cmd/buildlet and coordinator,
or have stopped running.
As a result, cmd/builder is no longer needed and can now be deleted.
Fixesgolang/go#21191
Change-Id: Ie51f7fa3a43a7c4799b614d65a9409e970c201d9
Reviewed-on: https://go-review.googlesource.com/c/161118
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
So people can see linux-amd64-nocgo vs linux-amd64-noopt, etc.
Currently it only works on mouseover on desktop, and few people know that.
Add a new row to make it explicit.
While at it:
* remove some dead code
* make this testable locally, detecting dev_appserver mode and populating some
fake data
* remove the s/amd64/x86/ in the subheadings. that's just confusion and we don't
do that to other longer fields like arm64 or ppc64le, etc
Change-Id: Id3de9406f383c93202be2e298f07d5c4fef8ae85
Reviewed-on: https://go-review.googlesource.com/c/160657
Reviewed-by: Andrew Bonventre <andybons@golang.org>
This reverts CL 159699 (commit 269e03abe7)
Reason for revert: I now get failures:
bradfitz@gdev:~/src/golang.org/x/build/cmd/coordinator$ ./version.sh
./version.sh: 12: [: missing ]
a377ef91745ef6c4602d363321b0ee7e4a8be5a1-bradfitz-2019-01-30T19:45:32Z
Change-Id: I9e28ac41df327c29628f2b7ac3e0dd20cd31a921
Reviewed-on: https://go-review.googlesource.com/c/160480
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
We previously hard-coded a Linux-only static map of 2015 data in for
the critical path scheduling of cmd/dist tests over N sharded
buildlets. That worked well only for Linux and only in 2015.
Instead, query BigQuery to find out what the recent timing data looks
like for all builders.
I'd started to work on this back in CL 30716 (Oct 2016) but apparently
never finished. Yay me. But skip the writing-to-CSV step. BigQuery is
much faster than I remember (maybe it got faster?), so just query it
directly. The query takes about 2 seconds, and we only do it every
hour (which is still overkill; daily is probably fine).
Change-Id: I498fc09dfaf24fb1f11b2c0ab4b952b2f15f9c32
Reviewed-on: https://go-review.googlesource.com/c/160037
Reviewed-by: Andrew Bonventre <andybons@golang.org>
This change introduces two cloud functions: wikiwebhook and
sendwikidiff. The former handles GitHub Wiki change events
sent over HTTP and enqueues them on a pubsub topic for the
latter to pick up. sendwikidiff then checks out the wiki
repo and sends an email with the diff of the change to
golang-wikichanges@.
The reason it is split into two functions is due to GitHub’s
timeout limit on webhook handlers (ten seconds). In testing,
a cold boot of a function that does everything described
above would sometimes hit that limit.
Updates golang/go#27313
Change-Id: I1974e1434c7003482df724d6ea3b537e22231c36
Reviewed-on: https://go-review.googlesource.com/c/158642
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Document that https.ListenAndServe always returns a non-nil error,
and use that property to stop gerritbot if the HTTPS server fails
to run.
Remove an unneccessary check for ln != nil in https.ListenAndServe.
net.Listen returns a valid listener when err == nil, and the
err != nil case is already handled above.
Change-Id: Ibbadd551aa36b5376bf062011f12e3e3787bd6b3
Reviewed-on: https://go-review.googlesource.com/c/159698
Reviewed-by: Andrew Bonventre <andybons@golang.org>
There was a race condition when the opt.AutocertCacheBucket field was
specified. The handler = http.HandlerFunc(redirectToHTTPS) line could
run before (or after) the value of handler was used in the following
line that did serveAutocertTLS(handler, opt.AutocertCacheBucket).
As a result, it was possible for the HTTPS server to end up using the
redirectToHTTPS handler, instead of the intended original handler
parameter passed to the ListenAndServe function.
Change-Id: I95967eed1176f236a87131d0c054ded680886591
Reviewed-on: https://go-review.googlesource.com/c/159697
Reviewed-by: Andrew Bonventre <andybons@golang.org>
The new BuildConfig.MinimumGoVersion field specifies the minimum
Go version the builder is allowed to use. It's useful when some
of the builders are too new, and do not support all of the supported
Go releases (e.g., openbsd-amd64-64 and freebsd-amd64-12_0 currently
require Go 1.11 and don't work on Go 1.10).
It only needs to be set when a builder doesn't support all supported
Go releases, since we don't typically test unsupported Go releases.
To allow cmd/coordinator to use this field and filter out work it
receives from maintner/maintnerd's GoFindTryWork RPC call,
we add a GoVersion slice to apipb.GerritTryWorkItem,
and populate it in maintapi.apiService.GoFindTryWork method.
For trybots on the Go repo, the GoVersion field is determined from
the branch name. For "release-branch.goX.Y" branches, it parses out
the major-minor Go version from the branch name. For master and
other branches, it assumes the latest Go release.
For trybots on subrepos, we already have the Go version information
available, so use it directly.
Afterwards, all that's left is to modify newTrySet in cmd/coordinator
to make use of BuildConfig.MinimumGoVersion and work.GoVersion to skip
builders that are too new for the Go version that needs to be tested.
Fixesgolang/go#29265
Change-Id: I50b01830647e33e37e9eb8b89e0f2518812fa44f
Reviewed-on: https://go-review.googlesource.com/c/155463
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
x/tools/cmd/godoc uses at least one Go package from x/net as of
CL 157197.
Don't add it to signature because we don't want the signature to
change whenever new commits to x/net are pushed, causing tip.golang.org
to be redeployed. This is because x/net is not considered a critical
component of the website at this time, and that's not expected to
change soon.
When the website begins using modules, it will specify the x/net
version precisely and this decision will no longer matter.
Fixesgolang/go#29874
Change-Id: I1fa76bb81f8d2ffc2314375e2dfe4898c3af58de
Reviewed-on: https://go-review.googlesource.com/c/158937
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
The cmd/go tests are currently timing out
after 20m on the plan9/386 builder.
This change enables SMP on the plan9/386
builder so the cmd/go tests could complete
in 10m approximately.
Fixesgolang/go#29822.
Change-Id: I4fb92340ea7f5f4cb933aa1d4e29c310a6dc6673
Reviewed-on: https://go-review.googlesource.com/c/158597
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This new Plan 9 image includes a fix to the
ramfs file server so that the number of open
file descriptors is correctly initialized
when creating a file.
Fixesgolang/go#29033.
Change-Id: I24c536e2d8fc5bf5b2239ecdc01a5ac8fcbe9bdb
Reviewed-on: https://go-review.googlesource.com/c/152037
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
I built and tested the v7 image from CL 152037 per the instructions in
env/plan9-386/README.
Change-Id: Ia4dcb1ac8429ab3e54e8ed21719d3ad894529927
Reviewed-on: https://go-review.googlesource.com/c/152238
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Brad Fitzpatrick
Dmitri Shuralyov
Bryan C. Mills
Katie Hockman
Andrew Bonventre
saitarunreddy
Quinten Yearsley
Baokun Lee
David du Colombier