Граф коммитов

59 Коммитов

Автор SHA1 Сообщение Дата
Bryan C. Mills 24ce8bdc49 maintner: return an explicit error from (*GerritProject).GitCommit
This will hopefully help in diagnosing an occasional panic I observe
in cmd/fetchlogs. It also brings the API more in line with
https://go.dev/wiki/CodeReviewComments#in-band-errors.

Fixes golang/go#50474.

Change-Id: Idc644ec8ef7a22332f9cea8fb15c67eac5676e7f
Reviewed-on: https://go-review.googlesource.com/c/build/+/407394
Run-TryBot: Bryan Mills <bcmills@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Bryan Mills <bcmills@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
2022-05-24 15:47:23 +00:00
Russ Cox 993f2c6f22 all: gofmt
Gofmt to update doc comments to the new formatting.

For golang/go#51082.

Change-Id: Ie063c2fd890641b551c0f304fee5fa19121f1b38
Reviewed-on: https://go-review.googlesource.com/c/build/+/399555
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Russ Cox <rsc@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
2022-04-12 02:01:23 +00:00
Bryan C. Mills ecfecf7bea all: consistently set PWD to match Dir for subprocesses
Also consistently deduplicate Env entries, mainly to reduce confusion
when reading logs.

This change is probably larger than what is strictly needed to fix the
issue, but it seemed simpler than trying to figure out which of the
many calls to exec.Command in the module are actually relevant.
(It also provides some useful case studies for possible additions to
the Go standard library.)

For golang/go#33598

Change-Id: Ia2bce4549e4a71b56fb497e3df093f79fbcf7f29
Reviewed-on: https://go-review.googlesource.com/c/build/+/353549
Trust: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2021-10-08 15:15:56 +00:00
Brad Fitzpatrick 0ea156d652 maintner: record, note deleted Gerrit refs
When run in --config=devgo mode (which forks the main mutation log and
runs it locally, adding new mutations as needed), I now see:

2019/11/27 13:48:41 gerrit go.googlesource.com/go: gerrit ref "refs/heads/dev.link" = "27c0aeee3fc7ed35f9b2eac4725b9147902537da"
2019/11/27 13:48:41 gerrit go.googlesource.com/go: gerrit ref "refs/heads/release-branch.go1.12-security" now deleted
2019/11/27 13:48:41 gerrit go.googlesource.com/go: gerrit ref "refs/heads/release-branch.go1.11-security" now deleted
2019/11/27 13:48:41 gerrit go.googlesource.com/go: gerrit ref "refs/tags/andybons/goroot.mailed" now deleted
2019/11/27 13:48:41 gerrit go.googlesource.com/go: gerrit ref "refs/heads/release-branch.go1.10-security" now deleted
2019/11/27 13:48:41 gerrit go.googlesource.com/go: gerrit ref "refs/tags/1.10beta1.mailed" now deleted
2019/11/27 13:48:41 gerrit go.googlesource.com/go: gerrit ref "refs/tags/andybons/blog.mailed" now deleted
2019/11/27 13:48:41 gerrit go.googlesource.com/go: gerrit ref "refs/tags/andybons/cachedir.mailed" now deleted
2019/11/27 13:48:41 gerrit go.googlesource.com/go: gerrit ref "refs/heads/release-branch.go1.13-security" now deleted
2019/11/27 13:48:41 gerrit go.googlesource.com/go: gerrit ref "refs/heads/tmp.shadams" now deleted

Change-Id: I0da673a3de7064f50406de9b369e2f1b9f246730
Reviewed-on: https://go-review.googlesource.com/c/build/+/209117
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-12-03 19:17:26 +00:00
Bryan C. Mills 2923e3dac3 maintner: add tracing on 'git fetch origin'
Also increase its timeout and terminate it with a friendlier signal
and add some logging of how long git operations took.

Updates golang/go#35293
Updates golang/go#35124

Change-Id: I1ed466d872a11f60751953ef5274be96cea0294b
Reviewed-on: https://go-review.googlesource.com/c/build/+/205860
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
2019-11-07 22:58:31 +00:00
Dmitri Shuralyov 9e83d85870 devapp: show CLs containing DO NOT SUBMIT
Change https://dev.golang.org/reviews page to be more helpful
at displaying CLs that are in need of a general code review.

Don't hide CLs containing DO NOT SUBMIT. Per Interim Code Review
and Issue Tracker Conventions¹:

	DO NOT SUBMIT means that while the CL is okay to review,
	the author wants Gerrit to make sure the CL is not submitted
	in its current form.

That means we should not be excluding CLs containing DO NOT SUBMIT
from the reviews page.

Remove a log line from GerritMeta.LabelVotes method. It's not serving
a helpful purpose at this time, but produces a large amount of output.

¹ https://groups.google.com/d/msg/golang-dev/YU56eKYiXJg/K1CkXo7Iy0gJ

Change-Id: Ibe40f756e6a0dab0a13c75fa53998e2d56d287e3
Reviewed-on: https://go-review.googlesource.com/c/163159
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-02-21 02:47:21 +00:00
Dmitri Shuralyov 357a85b84a maintner: handle multi-line subject edge-case
This is a followup to CL 161222, where I forgot to handle the case
where a commit message has a multi-line subject and no body.

Updates golang/go#30101

Change-Id: I01222e8cf783bc3b0631a332cf379717aa85cebc
Reviewed-on: https://go-review.googlesource.com/c/161797
Reviewed-by: Andrew Bonventre <andybons@golang.org>
2019-02-11 18:03:06 +00:00
Dmitri Shuralyov 77d92a99a5 maintner: support multi-line commit message subjects
In git, the text up to the first blank line in a commit message
is treated as the commit subject.

Most subjects are short and fit in one line (hopefully under
50 characters). In some cases, they end up being split into
multiple lines.

Report the entire subject rather than just the first line,
and join multiple lines with spaces.

Reference: https://git-scm.com/docs/git-commit#_discussion.

Fixes golang/go#30101

Change-Id: I1ea8212202fce0033e5ac8075352d5d7e0ae823c
Reviewed-on: https://go-review.googlesource.com/c/161222
Reviewed-by: Andrew Bonventre <andybons@golang.org>
2019-02-08 21:37:38 +00:00
Dmitri Shuralyov 9fbc418bb1 maintner: remove nil check in GerritCL.Subject method
In CL 107296, the maintner API had undergone a change such that all
public endpoints would only return GerritCLs that are considered
complete. Complete CLs are defined to have certain properties that
make working with them more friendly. One of those properties is that
a complete GerritCL always has a non-nil Commit field.

Methods of GerritCL type are expected to operate on complete CLs
that were acquired via one of the public means (e.g., ForeachOpenCL,
ForeachCLUnsorted, CL), so it's a precondition that the CL must be
complete. (If the user makes a GerritCL type from scratch, they are
expected to follow the documented requirement that GerritCL.Commit
must be non-nil.)

The Footer, ChangeID methods were already relying on the fact that
Commit is non-nil. This change updates the GerritCL.Subject method
to also make use of that property, which makes the cl.Commit nil
check not necessary.

This leads to simpler code and higher confidence that our internal
invariants are maintained.

Change-Id: I5867fd248274b8dd94a0ca98a91a773d48b3bdec
Reviewed-on: https://go-review.googlesource.com/c/161221
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-02-08 21:33:55 +00:00
Brad Fitzpatrick 228e5d2805 maintner: fix GerritMeta.Hashtags to look at earlier meta parents for answer
The Gerrit meta commit graph is a linear history. The most recent meta
with a "Hashtags: " footer line has the complete set. We just have to
go back and look for it.

Fixes golang/go#28318
Updates golang/go#28510 (fixes after gopherbot re-deployed)
Updates golang/go#28320 (fixes after gopherbot re-deployed)

Change-Id: I43705075800ae3d353c1c8f60ab7685883ea5602
Reviewed-on: https://go-review.googlesource.com/c/152779
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2018-12-07 20:50:57 +00:00
Andrew Bonventre 8cc7007a72 cmd/gerritbot: post proper display name of message author to GitHub
Gerrit's NoteDB git-based metadata format changed recently to display
"Gerrit User 1234" as the display name for meta commit authors. Use
the Gerrit REST API to retrieve the proper display name and cache the
result to prevent superfluous API calls.

Also updates the maintner doc for the Author property of GerritMessage
to reflect NoteDB's format.

Fixes golang/go#28663

Change-Id: I549474ad139e48c736d715414e82b6db83a9fdf3
Reviewed-on: https://go-review.googlesource.com/c/148560
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-11-08 22:52:26 +00:00
Andrew Bonventre 5df336cd7f maintner, devapp: display proper owner names instead of Gerrit User NNN
This change deletes the OwnerName method on GerritCL, since Gerrit's
format for its NoteDB backing store changed to display "Gerrit User NNN"
instead of the proper display name in the meta commit.

Update golang/go#28663

Change-Id: Ifa64ca2b2694b17e888451582b9c63f3f37280a9
Reviewed-on: https://go-review.googlesource.com/c/148557
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-11-08 19:05:21 +00:00
Dmitri Shuralyov 0eda5f7a6a maintner: skip incomplete CLs in ForeachOpenCL
CL 107296 has created a GerritCL.complete method and started
using it to enforce non-nil Meta and Commit fields in GerritCL
that are returned to the client.

It was used in ForeachCLUnsorted to skip incomplete CLs when
iterating over the gp.cls map, but due to an oversight, it
was not added to ForeachOpenCL. As a result, it was possible
for incomplete CLs to be iterated over via ForeachOpenCL.
This change fixes that by adding the missing complete check.

Update TestGerritMetaNonNil test to check ForeachOpenCL too.

Add missing period at the end of a sentence in ForeachCLUnsorted
documentation.

Fixes golang/go#27536.

Change-Id: I453d0f2b2b2793ed1b8a55bae1c2254906d74792
Reviewed-on: https://go-review.googlesource.com/135677
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-09-18 15:15:53 +00:00
Andrew Bonventre 952c4101bf cmd/gerritbot: autoassign reviewers on changes
Look for changes that haven’t had any human aside from the author
on them and assign reviewers/CCs based on entries in golang.org/s/owners.
If no owners can be found, the tag 'no-owners' is added to the
change to prevent gopherbot from making unneeded API calls.

Also updates gopherbot's Dockerfile to use 1.11 and update deps.

Change-Id: I2650a10dd324532d86bc902be419c5f29ae980db
Reviewed-on: https://go-review.googlesource.com/121018
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-09-03 22:18:01 +00:00
Dmitri Shuralyov e7a2a012f3 maintner: deduplicate issue references in parseGithubRefs
When parsing commit messages for references to GitHub issues,
parseGithubRefs used to add entries to GerritCL.GitHubIssueRefs
slice whenever there was a reference, even if that reference has
already appeared before.

The GitHubIssueRef type only contains minimal information about
a reference: the repository and the issue number. As a result,
there is very little to gain from tracking multiple references
separately. Most users of the API would likely prefer a list
of unique issue references than the raw list. So, change it to
return unique issue references only, and document that.

If in the future there is a need to track issue references
at a more granular level, where it matters whether there was
one or multiple references, the API can be revisited then.

Fixes golang/go#26113.

Change-Id: Ib51372736bcf49e3eae4cf111d0258733c742e61
Reviewed-on: https://go-review.googlesource.com/128119
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-08-07 00:39:16 +00:00
Dmitri Shuralyov dcbfa19f24 maintner: propagate and use internal invariants in finishProcessingCL
This change simplifies and cleans up finishProcessingCL. It documents
a precondition for finishProcessingCL that cl.Meta must be non-nil.

finishProcessingCL is only ever called on CLs that have been noted as
dirty via noteDirtyCL, and CLs are always given a non-nil Meta before
noteDirtyCL is called on them.

foreachCommitParent sounded like it called f on all parents of commit,
not including commit itself. In reality, it called f on commit as well.
Rename it to foreachCommit and improve its documentation to make this
more visible.

As a result, it becomes more clear that cl.Metas will contain at least
one element (e.g., at least mostRecentMetaCommit). Then, cl.Metas[0]
can be used to find the CL creation time.

Also change foreachCommit to return an error if a parent commit isn't
found. Otherwise, we'd be cutting the history walk short and the caller
wouldn't know. This shouldn't happen, but if it does, it'll be logged
(by finishProcessingCL).

Define a CL to be complete if its Meta and Commit fields are non-nil,
and the Metas slice contains at least 1 element. Use this to simplify
OwnerName and OwnerID methods and remove the no longer needed
firstMetaCommit method.

Finally, use the simplified finishProcessingCL to set cl.Created to
the actual creation time of the CL (rather than "last updated" time).
It's easy now, because we can deduce that len(cl.Metas) >= 1 is
guaranteed to be true, and therefore cl.Metas[0] will not panic.
There's no need to add guards or think about what to do if cl.Metas
is empty.

Document GerritCL.Created field, since it's exported.

Background

This change was started with the goal of fixing golang/go#24744.
It was hard to fix that bug with a short diff, because there were
too few guarantees inside finishProcessingCL that could be relied on.
I wanted to improve that as part of my fix.

We already established some guarantees for the user-facing CLs (i.e.,
that their Commit, Meta fields are non-nil and len(cl.Metas) >= 1).
I started propagating and using that property in a few more places
in internal code.

It allowed simplifications such as:

	 // OwnerName returns the name of the CL’s owner or an empty string on error.
	 func (cl *GerritCL) OwnerName() string {
	-	m := cl.firstMetaCommit()
	-	if m == nil {
	-		return ""
	-	}
	-	return m.Author.Name()
	+	if !cl.complete() {
	+		return ""
	+	}
	+	return cl.Metas[0].Commit.Author.Name()
	 }
	-
	-func (cl *GerritCL) firstMetaCommit() *GitCommit {
	-	m := cl.Meta
	-	if m == nil { // TODO: Can this actually happen, besides in one of the contrived tests? Remove?
	-		return nil
	-	}
	-	c := m.Commit
	-	for c != nil && len(c.Parents) > 0 {
	-		c = c.Parents[0] // Meta commits don’t have more than one parent.
	-	}
	-	return c
	-}

	 // complete reports whether cl is complete.
	 // A CL is considered complete if its Meta and Commit fields are non-nil,
	 // and the Metas slice contains at least 1 element.
	 func (cl *GerritCL) complete() bool {
	 	return cl.Meta != nil &&
	 		len(cl.Metas) >= 1 &&
	 		cl.Commit != nil
	 }

Also, previously, it was hard to be sure that the
gc, ok := ... line wouldn't panic:

	// called with Corpus.mu Locked
	func (gp *GerritProject) finishProcessingCL(cl *GerritCL) {

		// What happens if cl.Meta is nil?
		// Should there be a if cl.Meta == nil guard here?
		// What should it do if cl.Meta is indeed nil?
		gc, ok := c.gitCommit[cl.Meta.Commit.Hash]
		if !ok {
			log.Printf("WARNING: GerritProject(%q).finishProcessingCL failed to find CL %v hash %s",
				gp.ServerSlashProject(), cl.Number, cl.Meta.Commit.Hash)
			return
		}

By documenting that cl.Meta must be non-nil as a precondition of
finishProcessingCL, it became possible to know it won't panic:

	// finishProcessingCL fixes up invariants before the cl can be returned back to the user.
	// cl.Meta must be non-nil.
	//
	// called with Corpus.mu Locked
	func (gp *GerritProject) finishProcessingCL(cl *GerritCL) {

		// cl.Meta can't be nil due to precondition.
		gc, ok := c.gitCommit[cl.Meta.Commit.Hash]
		if !ok {
			log.Printf("WARNING: GerritProject(%q).finishProcessingCL failed to find CL %v hash %s",
				gp.ServerSlashProject(), cl.Number, cl.Meta.Commit.Hash)
			return
		}

(It would be a bug to call finishProcessingCL with a cl where
cl.Meta is nil, because it'd be violating the precondition.)

I think stating preconditions and propagating more internal invariants
is a good general direction, because it allows the code to be simpler
and easier to verify as being correct. Otherwise, one needs to keep
checking if fields are non-nil everywhere before using them and think
what to do if they are nil, and that's unpleasant.

Fixes golang/go#24744.

Change-Id: I07e362d52e30089a9ba03c30b04ad19b2c385722
Reviewed-on: https://go-review.googlesource.com/111877
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-08-06 16:50:14 +00:00
Brad Fitzpatrick dd35151f24 maintner: track hashtag edits on GerritMeta, make HashtagEdits method faster
Updates golang/go#24836

Change-Id: I75cae7de574af7525964bdf420328d3e553a044c
Reviewed-on: https://go-review.googlesource.com/107305
Reviewed-by: Andrew Bonventre <andybons@golang.org>
2018-04-17 02:28:26 +00:00
Brad Fitzpatrick 0b261d0443 maintner: add hashtag mutation accessors on GerritMeta
Updates golang/go#24836

Change-Id: Id529f12dba54366d49feb112662473ca214ba2f6
Reviewed-on: https://go-review.googlesource.com/107297
Reviewed-by: Chris Broadfoot <cbro@golang.org>
2018-04-16 21:12:23 +00:00
Brad Fitzpatrick ce9aca4b8e maintner: change type/name of GerritCL meta fields, ensure Meta always non-nil
They were of type GitCommit, but we want to hang the useful methods
off the existing GerritMeta wrapper.

This was the plan earlier (when the GerritMeta type was added), but
that earlier CL didn't go all the way and convert callers.

This will likely break some maintner-using code, but there aren't many
callers.

Also, document & enforce that Meta & Commit are always non-nil when
enumerating CLs and when looking up CLs. Previously, two deleted CLs
didn't have this property, as the Gerrit server was returning a ref
for a version of the commit, but no meta ref. Filter those out.

Fixes golang/go#22060
Fixes golang/go#21984

Change-Id: I67727977c7a25e750f51a6ff7f3e00aaf960ef9f
Reviewed-on: https://go-review.googlesource.com/107296
Reviewed-by: Dmitri Shuralyov <dmitri@shuralyov.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-04-15 21:06:32 +00:00
Dmitri Shuralyov 398adf893f maintner: add GerritCL.CommitAtVersion method
Previous versions were available via GerritProject.remote, but that
symbol is not exported and can't be used by callers of this package.

Update test for go-github API change in google/go-github#816, so the
tests can pass locally. Since go-github isn't vendored, it's understood
that the latest upstream version is targeted.

Change-Id: Ib4b78abacb19d73f95df2215b4a248a24376ae10
Reviewed-on: https://go-review.googlesource.com/95136
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-02-19 17:27:22 +00:00
Andrew Bonventre 7dd1e7da00 maintner,cmd/gerritbot: post to GitHub when new Gerrit comments are made
Adds an Owner() method to GerritCL that returns the GitPerson author of
the Gerrit change’s first commit, which is how Gerrit assigns the owner.

Updates golang/go#18517

Change-Id: I27cf185d5c68db44732170d583f1851491f2d8a3
Reviewed-on: https://go-review.googlesource.com/87157
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-01-11 14:54:48 +00:00
Brad Fitzpatrick fd5ece6d24 maintner: account for CLs changing branches
Fixes golang/go#23007

Change-Id: I1e791d1a96033e77edf077909c2a20afaea511c3
Reviewed-on: https://go-review.googlesource.com/83635
Reviewed-by: Andrew Bonventre <andybons@golang.org>
2017-12-13 02:53:10 +00:00
Brad Fitzpatrick 26626971b6 maintner: misc fixes and tests
As far as I can tell, the data structure inconsistency check that
panic-failed earlier has been fixed by the last few CLs (the
"finishProcessing" ones). The new tests double check.

Also, fix up Footer to not allocate by changing the requirement of its
argument.

And fix a failing test that was crashing due to its *Gerrit not being
initialized in a non-Gerrit test.

Fixes golang/go#22753 (already fixed, but now tested)

Change-Id: If09cfad6f69d8b3c97410f3e4c3a2e2281c666a5
Reviewed-on: https://go-review.googlesource.com/78155
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2017-11-18 03:29:02 +00:00
Brad Fitzpatrick 04dbfb5668 maintner: fix label counting, which affected trybot work-to-do caching
A recent CL (golang.org/cl/77690) broke how labelChanges was
incremented. It needs to be renamed and tested, but this gets trybots
working again.

Fixes golang/go#22763

Change-Id: I068fc66bbde283291191f41acdc436876232e88b
Reviewed-on: https://go-review.googlesource.com/78536
Reviewed-by: Andrew Bonventre <andybons@golang.org>
2017-11-17 20:56:21 +00:00
Andrew Bonventre ee43607332 maintner,cmd/gerritbot: minimal processing for changes
+ Adds a Footer method to maintner.GerritCL that returns the value
  of a git footer with the given name.
+ Iterates through whitelisted GitHub repos and Gerrit projects
  looking for changes to update. Handles the case where no update
  is required.

Update: golang/go#18517

Change-Id: Id133df6956b7213752059a0b0e84dd76ee8ea620
Reviewed-on: https://go-review.googlesource.com/76374
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2017-11-17 20:40:19 +00:00
Brad Fitzpatrick ee6321b540 maintner: handle Gerrit ref meta changes occurring before dependent git commits
A previous bug in maintner caused Gerrit ref meta updates occuring in
the mutation log before the git meta hash was added to the log. This
CL handles such entries. We add CLs to a dirty set when we see a ref
update and wait until the end of the mutations to parse the fields out
of the meta commits, at which time the git commits will have appeared
in the log.

Also, don't enumerate over private CLs in ForeachOpenCL, which I
thought was the same bug but turned out to be different.

Fixes golang/go#22653

Change-Id: I79120f06b13c5e74c7761d04494215bf0da2e53d
Reviewed-on: https://go-review.googlesource.com/77690
Reviewed-by: Andrew Bonventre <andybons@golang.org>
2017-11-15 21:03:45 +00:00
Andrew Bonventre 2f835b60ec x/build/maintner: add GerritCL.OwnerName() function
To easily get the name of the original author of a change.

Change-Id: Id006b6520fd72657252cbbf934aaf8fd10202fbf
Reviewed-on: https://go-review.googlesource.com/65073
Reviewed-by: Francesc Campoy Flores <campoy@golang.org>
2017-09-22 18:52:20 +00:00
Andrew Bonventre f7c15b66cf maintner: decrease poll interval from 15 to 5 min
When Gerrit emails are not sent, maintner polls Gerrit every
15 minutes. We can safely reduce this to every 5 min given the
relatively small number of bots using maintner and the current
quota of 0.2 qps per day that Gerrit permits for unauthenticated
requests.

Update golang/go#21192

Change-Id: I233bc50998927daedb34d81436e182579d2b4031
Reviewed-on: https://go-review.googlesource.com/57710
Reviewed-by: Sarah Adams <shadams@google.com>
2017-08-21 21:01:17 +00:00
Brad Fitzpatrick 8a70402ebb maintner: add unused code for parsing Gerrit NoteDB info, sum label votes
Change-Id: I7208c1ed6cda42fb5ad2d13374bd34304c87644c
Reviewed-on: https://go-review.googlesource.com/51912
Reviewed-by: Kevin Burke <kev@inburke.com>
2017-08-07 21:21:00 +00:00
Brad Fitzpatrick fff6e1803a maintner/maintnerd: use Gerrit to calculate TryBot/Result votes
In https://golang.org/cl/51590 I tried to calculate the Run-TryBot and
TryBot-Result vote sums using the Gerrit NoteDB mirror we have in
maintner, but after two days of fighting it now, I've concluded it's
tricky. While I work on the tricky bits in parallel, this CL partially
reverts CL 51590, which had enough other cleanup and progress that it
wasn't worth reverting in its entirety.

Instead, move the Gerrit querying to maintnerd (instead of coordinator).
This isn't bad because maintnerd already queries Gerrit.

And coordinator will still be polling every second (introduced in
51590) instead of the old 60 seconds, but this CL now adds caching in
the RPC handler.

Change-Id: I80a519d9026a0981e3abf43d54a32b4684bda0e5
Reviewed-on: https://go-review.googlesource.com/51970
Reviewed-by: Kevin Burke <kev@inburke.com>
2017-07-30 16:09:47 +00:00
Brad Fitzpatrick 7d8c8e4252 maintner/maintnerd, cmd/coordinator: use maintnerd to find TryBot work
Instead of polling Gerrit every 60 seconds, poll maintnerd every
second. This should improve TryBot-requested to TryBot-running latency
by 30 seconds on average.

(Ideally it'd long poll for changes and not even have ~500ms delay,
but this is an improvement.)

Also, in the process this does most of the work for golang/go#17626.
And this cleans up the repo head tracking complication in the coordinator
and just asks maintner for it instead.

Running benchmarks in the coordinator has been disabled since
2017-06-23 but this CL disables it further, removing some code it'd
need to work. But considering that benchmarking would need code
repairs to get working again anyway, I consider this acceptable in the
short term. I left TODO notes.

Updates golang/go#17626
Updates golang/go#19871

Change-Id: Idab43f65a9cca861bffb37748b036a5c9baee864
Reviewed-on: https://go-review.googlesource.com/51590
Reviewed-by: Andrew Bonventre <andybons@golang.org>
2017-07-28 22:26:36 +00:00
Andrew Bonventre e3b7b1d23d devapp: remove godash code in favor of maintner
Adds two new functions to maintner:
+ (*GerritCL).Subject() returns the first line of the latest
  commit message.
+ (*GitHubRepo).ForeachMilestone calls a passed function for each
  milestone in the repo.

Change-Id: I08c50ea53b781064c8ab2ddaff2177f51ebb091d
Reviewed-on: https://go-review.googlesource.com/50652
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2017-07-22 21:35:17 +00:00
Kevin Burke f72df8f025 cmd/gopherbot: remove sequential order assumption
Previously we assumed that Gerrit issued CL numbers in sequential
order: If it issued CL N, any new CL would be in the range [n+1, ∞).
However, observation shows this not to be true; I submitted a number
of CL's just now and got this ordering: 49910, 49911, 49891, 49852.

This explains why there are ten people who have yet to be
congratulated for submitting their first CL: Gerrit issued a newer
CL number and we ignored any CL's older than that. I didn't catch
this in testing because I wasn't running in daemon mode locally, so
mostRecentCL was always 0.

Change-Id: I822ca12a093bd6c186701a4d49b47b2671bcee8a
Reviewed-on: https://go-review.googlesource.com/49853
Reviewed-by: Jessie Frazelle <me@jessfraz.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2017-07-20 20:02:30 +00:00
Kevin Burke 9306cbc43a cmd/gopherbot: Congratulate users when they add their first CL
It's nice to encourage people who submit their first CL, and I figured
we could provide them useful info - the things Josh usually emails to
new contributors.

Parse Author information from Gerrit commit messages, so we can check
whether Gopherbot sent a given message (instead of another user).

Also fix a TODO involving the ordering of messages appended to
a GerritCL.

Change-Id: Ifde950ac08d76a6b4c50c356697068c6d6beb0d7
Reviewed-on: https://go-review.googlesource.com/46390
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2017-07-15 15:12:22 +00:00
Andrew Bonventre 6990c3464a maintner, devapp: initial integration of maintner
+ Add UpdateWithLocker method to maintner.Corpus to allow an
  external locker to be passed that will be held during mutations.
+ Add OwnerID to maintner.GerritCL that returns the Gerrit user ID
  of a change using its meta commits.
+ Add maintner corpus to server.
+ Add /imfeelinglucky and /imfeelinghelpful endpoints that redirect
  to a random HelpWanted GitHub issue.
+ Add initial scaffold for GopherCon dashboard.

Change-Id: I2567b084f012cad5a9b4dee07e92be05088ac9d8
Reviewed-on: https://go-review.googlesource.com/47830
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2017-07-07 20:23:05 +00:00
Kevin Burke 8f4eb5d280 maintner: parse Gerrit reply bodies
The contents of a Gerrit reply are stored in the body of the commit
message for a meta commit, but we currently ignore them. Parse the
message text when we walk the tree of meta commits, and return it when
we return a CL.

The API returns these in very raw fashion - what you see in a reply
in the UI is what you get in the git commit. We don't do any parsing
either. We could add helpers on the GerritMessage object to try to
return you better formatted information.

To see what the API returns for these messages, visit this URL:
https://go-review.googlesource.com/changes/42180?o=MESSAGES. The
messages attached to a specific line and file are stored a different
way and returned via a different API. It's trickier to get those so
I'll add them in a separate commit.

Change-Id: Ibd56fb828c225b57650157d08b27034ece75848a
Reviewed-on: https://go-review.googlesource.com/42452
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2017-05-25 01:45:18 +00:00
Brad Fitzpatrick ae78535d33 maintner: repair unindexed Gerrit commits
We previously tried to have the invariant that a Gerrit ref mutation
never was recorded in the log until all its reachable commits were
already recorded in the log.

But due to unknown bug(s) in the past, that wasn't the case.

This adds a fix-up step to the beginning of sync. The new maintnerd
--config=devgo config type has made testing this both possible and
easy: devgo clones the live config and then starts running locally
from that point. Then the existing --sync-and-quit flag now also adds
a Check step at the end. Everything checks fine now, so I think the
invariant violation has since been fixed. At least if it recurs, we'll
catch it now.

Change-Id: Ia6cfd4ff8d793904472f4f07d6eef9cb4eb3afb8
Reviewed-on: https://go-review.googlesource.com/44090
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2017-05-24 20:35:24 +00:00
Brad Fitzpatrick e40272de26 gitmirror: subscribe to maintner to reduce push-to-github latency
Change-Id: I6c3098712e1655d8129c693dd8035b8e00e63650
Reviewed-on: https://go-review.googlesource.com/43624
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2017-05-19 17:53:18 +00:00
Brad Fitzpatrick 9e63ba0fc5 maintner: record all gerrit refs, not just change refs
Change-Id: I820b970f1e2ebdb365feb90b1e06650862cfcf66
Reviewed-on: https://go-review.googlesource.com/43616
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2017-05-18 19:17:27 +00:00
Brad Fitzpatrick b77708e47f maintner: add GitCommit.HasAncestor method
maintnerd will have an API to query this in a subsequent CL.

Updates golang/go#20222

Change-Id: I3cb0ce2897eea782f627c2e27c4d02ef0eb6c66b
Reviewed-on: https://go-review.googlesource.com/43554
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2017-05-16 21:25:14 +00:00
Brad Fitzpatrick 891b12dcbd cmd/gopherstats, internal/gophers: start of tool to collect community stats
Total mess, but a functional mess, and a starting point for the future.

Change-Id: Ifcc72660f30a0ee2692668b4ea53acd1c4015777
Reviewed-on: https://go-review.googlesource.com/41077
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2017-05-03 17:36:15 +00:00
Kevin Burke c176fc1dc0 maintner: add GerritProject.CL(int32)
This matches GitHubRepo.Issue(n) on the GitHub side.

Also document some public API's that lacked documentation.

Change-Id: I88d9ef924bfec9048c5741fb7f6098e283ac3290
Reviewed-on: https://go-review.googlesource.com/42451
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2017-05-02 20:30:18 +00:00
Kevin Burke d802da8c4f maintner/gostats: fix build
This started failing when we removed
corpus.QueryFrequentlyModifiedFiles. Instead replace that function
with a similar one that uses the Foreach functions.

Change-Id: I243975b7638f8e74c8364804bc1f1c752367cb8b
Reviewed-on: https://go-review.googlesource.com/42183
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2017-05-02 20:30:10 +00:00
Kevin Burke 58ea07d1da maintner: fix status parsing
Previously we checked for Status: flags that were indented by
4 spaces. I think this is from me running "git log" which indents
4 spaces in the UI. On disk, the messages are not stored with leading
spaces.

Verified this is correct by fetching the corpus with godata.Get,
finding a CL in the Corpus and verifying it had the correct status
set.

Change-Id: I7439c2e92f244a6f780e5f84429dd116b6d83090
Reviewed-on: https://go-review.googlesource.com/42450
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2017-05-02 14:39:25 +00:00
Brad Fitzpatrick 1a1ef8e92a maintner: support for updating corpus in-place from its mutation source
And update godata package docs.

Change-Id: I781e50b20dfa1494fa7d65400cff8a2637ecf9e0
Reviewed-on: https://go-review.googlesource.com/42174
Reviewed-by: Kevin Burke <kev@inburke.com>
2017-04-30 21:30:16 +00:00
Brad Fitzpatrick 04f8c525c7 maintner: flesh out godata package, add caching network mutation source
Also, the maintner.Corpus constructor is now gone. An API is added for
maintnerd to become the leader and specify the cache dir.

Updates golang/go#19866

Change-Id: Ia726aa00ca1337b6c130cfee040ff9a1f935d0c2
Reviewed-on: https://go-review.googlesource.com/42148
Reviewed-by: Kevin Burke <kev@inburke.com>
2017-04-29 20:11:35 +00:00
Brad Fitzpatrick 8160f0cf8e maintner: add support for storing mutations on Google Cloud Storage
And Kubernetes config.

Updates golang/go#19866

Change-Id: I67b9c93c9c931c19d5c4275a55753ee146e13f21
Reviewed-on: https://go-review.googlesource.com/41824
Reviewed-by: Kevin Burke <kev@inburke.com>
2017-04-29 04:19:24 +00:00
Brad Fitzpatrick 1eecef36c8 cmd/gopherbot: mention gerrit CLs on Github (cl2issue port)
This adds indexing of Github mentions to maintner, then adds gopherbot
functionality to say "CL https://golang.org/cl/NNNN mentions this issue."
on Github when a Gerrit CL references it.

Also the start of the cherry-pick milestone bot which needs the same
Github issue reference tracking. But that part's only barely started
and still disabled.

Fixes golang/go#18818
Updates golang/go#19776

Change-Id: Ie5f7f6845317a7c39cc36d7397c7539bf99c3f92
Reviewed-on: https://go-review.googlesource.com/39352
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2017-04-18 05:39:29 +00:00
Brad Fitzpatrick da737d3d09 maintnerd: subscribe to pubsubhelper
Change-Id: Ia76b53c6448bc92850d4a65c15247547b132860c
Reviewed-on: https://go-review.googlesource.com/39635
Reviewed-by: Kevin Burke <kev@inburke.com>
2017-04-06 15:19:04 +00:00
Kevin Burke 9673075136 all: fix "go vet" errors
The one remaining error reported by "go vet" is the un-canceled
context in cmd/coordinator/remote.go, which isn't an error.

Change-Id: Ie8d9269fc77203cf82d0a724dd33e4871a8113a7
Reviewed-on: https://go-review.googlesource.com/39350
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2017-04-03 19:14:46 +00:00