And add a dummy query for fun.
Change-Id: Icc3f846691b83433f4b47d68b0a6c102be275ae9
Reviewed-on: https://go-review.googlesource.com/37994
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
No functional changes, just moving Github related code to its own
file.
Change-Id: I9600518142202c5519984564743a1996615d693a
Reviewed-on: https://go-review.googlesource.com/37992
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Currently the x/build test suite fails on Travis CI, since Travis
runs on GKE but doesn't have any of the correct metadata set up. We
should skip these tests in that environment like we do when we are not
running on GKE.
Change-Id: Ib99ad1e9f33ce9f05aaf690653ab1bf23718ce2b
Reviewed-on: https://go-review.googlesource.com/37942
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This is just the start of git commit metadata importing, but without
any of the git-specific bits. This is just the proto changes and the
changes to wire up the guts with maintner. The guts will come in a
following CL.
Change-Id: I321590dc30ba5c62346911526fed647fc0d0b3ad
Reviewed-on: https://go-review.googlesource.com/37938
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Due to the design of the github paging API it's possible to process
messages twice in the same paging run, which can lead to early
termination since the second run of the message will have the same
"UpdatedAt" as the first time we saw it. Keep track of messages we've
seen before in this paging run and ignore them the second/third times
we see them.
Also skip subdirectories of the `--data-dir` argument; we don't want
to search any other directories for data files than the one provided.
Change-Id: Ib4b867cc50568f7bfc5a3d3a0158e284d2038808
Reviewed-on: https://go-review.googlesource.com/37794
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This should comprise most of the fields that are needed by the
existing godash/devapp app. Will wire up the Milestone code in
another commit.
Change-Id: I83c399afebce5865dd03c00e295be5a19e1e36b2
Reviewed-on: https://go-review.googlesource.com/37753
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Eventually we will add other mutation sources but for the moment
let's catch up from the same proto directory we are writing changes
to.
Change-Id: I6efa3c0fb6f49cf4240d8a365f558bbb1c52602e
Reviewed-on: https://go-review.googlesource.com/37733
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
If the context returns, immediately return from the function instead of
continuing to sleep. Also, remove deprecated oauth2.NoContext.
Change-Id: If92c18d9d4dc1a801be8df5ad2e78ae809b9f274
Reviewed-on: https://go-review.googlesource.com/37650
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
For now, open and close the file each time we write a protobuf. We can improve
the performance in the future as necessary.
Change-Id: Idcc75df5aa59a604c0c0aeeff6ea0e0ff0f2db8f
Reviewed-on: https://go-review.googlesource.com/37626
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
When we get an issue from Github, add the issue and associated
details to the Corpus. Check whether the `Updated` timestamp on an
issue is newer than the one we have in the Corpus; if the timestamp
is the same, we can stop processing since we are sorting by the
updated date and no older issues will have new data.
If the mutation changes the Corpus, add it to an array of Mutations
that we will eventually flush to disk.
Adds a helper function to convert a github.Issue into a
maintpb.Mutation - this way we can reuse the logic for adding data to
the Corpus. Adds some more tests around data processing.
Change-Id: I9475e293941784179420ca7b638917943be1f29e
Reviewed-on: https://go-review.googlesource.com/37490
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
It was using some CPU.
Kinda useless to save a virtual screen.
Change-Id: If2ca5a39818e190b44f87299837a305cfe966c3c
Reviewed-on: https://go-review.googlesource.com/37493
Reviewed-by: Kevin Burke <kev@inburke.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
These never work and are a waste of resources and unnecessary scary
red on the page.
Change-Id: I2d8f8908096dee01618129a9082e98f3ffd92bb7
Reviewed-on: https://go-review.googlesource.com/37467
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Go policy is to support the past two released Go version, which is
currently Go 1.7 and Go 1.8. Most of the Go repos are broken for Go 1.6 now
because everybody is using subtests and contexts.
Previously:
https://golang.org/cl/31767 removed Go 1.5.
https://golang.org/cl/20843 removed Go 1.4.
Change-Id: I64b4af609a8be79a0f25a855430b73ba4d442ae0
Reviewed-on: https://go-review.googlesource.com/37466
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
macOS Sierra's network stack gets wedged after a few runs of all.bash.
Instead, halt the machine after a build is complete. cmd/makemac will make
a new one for the next build.
Change-Id: Icb1b21c045376cff9958a0d3b4cfaf784c9948d0
Reviewed-on: https://go-review.googlesource.com/37465
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Poll the local VMWare status and the farmer.golang.org status and compare
the two, deleting or creating VMs as appropriate.
Change-Id: Ifaee94b0b3ce062f0865bd9c4fb80937a560f650
Reviewed-on: https://go-review.googlesource.com/37464
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Will be used for dynamic creation/destruction of Mac VMs in subsequent CL.
Updates golang/go#9495 (Mac virtualization)
Updates golang/go#15760 (monitoring)
Change-Id: I48b17589b258d5d742bad5a3ddae18de98778149
Reviewed-on: https://go-review.googlesource.com/37457
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Now you can actually pass repos to maintnerd and tell a Corpus to
track/store them. Adds a hook to "catch up" a given repo, though it's
unimplemented at the moment, so all requests hit the API. Also only
Github is implemented.
Change-Id: Ib074c02437d6530fc8d57dd433486c05cd75ee64
Reviewed-on: https://go-review.googlesource.com/37299
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
The latest master for go-github (google/go-github@23d6cb9c) adds
Context parameters as the first argument to every function call.
Update code that calls go-github to have contexts, where appropriate,
and adds timeouts to some long-running commands that didn't
previously have them.
Fixesgolang/go#19208.
Change-Id: I25203de5d10ada1dcd3a97eb5434a85bb328ce7e
Reviewed-on: https://go-review.googlesource.com/37293
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
(been in production for past few days)
Updates golang/go#19166
Change-Id: Iee301aac5aa87c6644ac1253d7c0bd6c522ff066
Reviewed-on: https://go-review.googlesource.com/37380
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Probably not doing this right yet, but it's done.
Change-Id: Ic38b02c6cb0d42eb7e661c67f860254ff9f2fc76
Reviewed-on: https://go-review.googlesource.com/37296
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
If new commits are submitted to the remote repository while gitmirror
is running, gitmirror won't be able to find them when a user asks for
that commit to be exported.
If a request asks for an unknown commit, attempt to fetch it
on-demand from the remote and serve from there.
I checked that this worked by starting a local gitmirror, verifying
the commit was not present in the local repo, submitting a changeset,
asking for that commit, and verifying that gitmirror fetched the
remote commit to serve it.
Adds a very basic test for the archive functionality that won't go
over the network or hit the disk.
Fixesgolang/go#19166.
Change-Id: I1ad017b9c8cd258849585c091122c889ea656017
Reviewed-on: https://go-review.googlesource.com/37225
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
WIP; not functional yet, but submitting so
Kevin can continue if he wants.
Change-Id: I1dab38f9845ebcbd69ec2a6f07e98793d3ed7cfb
Reviewed-on: https://go-review.googlesource.com/37053
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This adds more utility to the Gerrit client and lets us reuse the
Gerrit API logic, instead of inventing it on an adhoc basis inside
the gitmirror code.
I added a test that this deserializes the correct data from the
Gerrit API for go.googlesource.com (and then removed it).
Change-Id: I449594b0257ed15d9b5aa7aa1eda7fb371fad6c1
Reviewed-on: https://go-review.googlesource.com/37252
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Also add closing HTML tags to the /debug/watcher page.
Change-Id: I487874802a2a79ae442ca996f00ef2f04ef6dc16
Reviewed-on: https://go-review.googlesource.com/37221
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
The talks repo got too big.
Also clean up the gitmirror fetching a code while there. It no longer
runs as a subprocess, so we don't need to account for its start-up
time.
Fixesgolang/go#19162
Change-Id: I0f80b95360d079989254c26ea0406dab9633f0c1
Reviewed-on: https://go-review.googlesource.com/37217
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
In the last iteration, Brad got an inconsistent (i.e. non-working) set
of packages because (a) the CDN uses different mirrors, some of which
are more stale than others and (b) it was using bleeding-edge packages.
Switch to a hardcoded mirror (the main one) and stable (2016Q4)
packages.
Change-Id: I45e5ec8596b522c6284ff4b8f08edd1a29f5597b
Reviewed-on: https://go-review.googlesource.com/37190
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
They were moved around a few months ago and I didn't
update the notes.
Change-Id: I435864acbb967a5c9f0eb6a0f969fe711cd720c9
Reviewed-on: https://go-review.googlesource.com/37143
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
golang/go placeholder had to be changed to 19117 given the previous
placeholder cannot be reopened.
golang/oauth2 is using the correct PR number now.
Change-Id: I9ee4801d6b4abc340fa1c9a74e3c0a6649f537f0
Reviewed-on: https://go-review.googlesource.com/37108
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Brad Fitzpatrick
Kevin Burke
Yuval Pavel Zholkover
Benny Siegert
Chris Broadfoot
Jaana Burcu Dogan