Add a RequestInfo.State field to record the current state of a request
for display.
Add a function internal.RequestState to for updating the state,
and use it for long-running requests.
Display the state on the worker main page.
Change-Id: I1a04036982aba23cf27776fac75248972a1336f3
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/568515
Reviewed-by: Michael Matloob <matloob@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
kokoro-CI: kokoro <noreply+kokoro@google.com>
Reimplement some reflection-based collection functions to
use generics.
In some cases we still need reflection, but at least we
can provide a type-safe wrapper with generics.
Change-Id: Id95949a7a22ee687166ecdfc1191150d79568889
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/389657
Trust: Jonathan Amsterdam <jba@google.com>
Run-TryBot: Jonathan Amsterdam <jba@google.com>
TryBot-Result: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jamal Carvalho <jamal@golang.org>
When the latest version of a module is inserted, delete
symbol_search_documents rows for symbols not in that version of the
package, so that stale rows are removed.
For golang/go#44142
Change-Id: I9a5c45eff4713cb765fd15b57530bf139385a3a8
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/349892
Trust: Julie Qiu <julie@golang.org>
Run-TryBot: Julie Qiu <julie@golang.org>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
This experiment is live in all environments and can be removed.
For golang/go#44142
Change-Id: Ie653e833b8fc04d5a538fab41137ca77a0989125
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/341451
Trust: Julie Qiu <julie@golang.org>
Run-TryBot: Julie Qiu <julie@golang.org>
TryBot-Result: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
Data is now inserted into documentation_symbols if the
insert-symbol-search-documents experiment is on, as opposed to if
skip-insert-symbols is off.
This table is needed for the symbol search design, but it's unnecessary
to insert into that table unless we are also inserting into symbol
search documents.
The experiment is added to various symbol history tests, since it uses
data from that table as a check against data in the symbol_history
table.
For golang/go#44142
Change-Id: I24f1689a9456e555274bf1c6725e917a8e9a8d45
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/335266
Trust: Julie Qiu <julie@golang.org>
Run-TryBot: Julie Qiu <julie@golang.org>
TryBot-Result: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jamal Carvalho <jamal@golang.org>
We don't re-insert rows to symbol_names in insertSymbols, since there's
no reason to do so, but tsv_name_tokens is populated by an ON INSERT OR
UPDATE trigger.
In order to make the trigger activate, set name=name when a symbol_name
is not inserted.
For golang/go#44142
Change-Id: I5eac5075727fd2db80c1a0ff7d66fa4df44c5c54
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/335394
Trust: Julie Qiu <julie@golang.org>
Run-TryBot: Julie Qiu <julie@golang.org>
TryBot-Result: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jamal Carvalho <jamal@golang.org>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
This code is only used in tests.
For golang/go#44142
Change-Id: Ice0479b439b64c3dd3c2556401cd1dc533d75924
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/330131
Trust: Julie Qiu <julie@golang.org>
Run-TryBot: Julie Qiu <julie@golang.org>
TryBot-Result: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
upsertSearchDocumentSymbols is changed based on the new schema.
For golang/go#44142
Change-Id: I1857b04cceeb9b702e4c54fc3a8d39efa811a3ce
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/329493
Trust: Julie Qiu <julie@golang.org>
Run-TryBot: Julie Qiu <julie@golang.org>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
Logic for insertSymbols is moved out of insertUnits to saveModule. This
will allows us to run that code after the check for isLatest, which is
necessary to insert symbols where the latest unit is not a
release version.
For golang/go#44142
Change-Id: Id917b6b74020957549ad1c6594f21d3f8d27e464
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/329349
Trust: Julie Qiu <julie@golang.org>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
When skip-insert-symbols is on, skip only the documentation_symbols
table, not the others.
Change-Id: Ia1f6bf975f02a2cc5aad15cd220b8a6c54585e97
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/324790
Trust: Jonathan Amsterdam <jba@google.com>
Run-TryBot: Jonathan Amsterdam <jba@google.com>
Reviewed-by: Julie Qiu <julie@golang.org>
Data is now upserted into documentation_symbols, to fix a bug where the
package symbol ID in the database did not match that of the one that was
going to be inserted.
This was due to changes made to the package_symbols table in CL 315309
and CL 315310.
Change-Id: I1d26132a55339d1b747d8c2fb5dfa1cb0b153558
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/320689
Trust: Julie Qiu <julie@golang.org>
Run-TryBot: Julie Qiu <julie@golang.org>
TryBot-Result: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
Inserting a package symbol with the same name but different parent name
than an existing row in the database previously resulted in a 500 which
is fixed.
Change-Id: Ia9c4f25a8bdf2aca87f0d736595778ba2baa62ad
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/316909
Trust: Julie Qiu <julie@golang.org>
Run-TryBot: Julie Qiu <julie@golang.org>
TryBot-Result: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
The children of a symbol will always have the same build context and no
other children. As a result, Symbol.Children are changed to the
SymbolMeta type.
For golang/go#37102
Change-Id: I89c361f86eff7179c18572e57a5dd0788fea4b1d
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/315432
Trust: Julie Qiu <julie@golang.org>
Run-TryBot: Julie Qiu <julie@golang.org>
TryBot-Result: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
CompareAPIVersions is added, which compares the since version data for
symbols with the data for apiVersions. This replaces CompareStdLib.
For golang/go#37102
Change-Id: I60815b4154a2c25b592127b4207bd6c5482f21c7
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/314536
Trust: Julie Qiu <julie@golang.org>
Run-TryBot: Julie Qiu <julie@golang.org>
TryBot-Result: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
ParsePackageAPIInfo is updated to handle apidata for both stdlib and
third-party packages.
It also handles information for constants and variables.
For golang/go#37102
Change-Id: Ia2a6a374cecb843700b9187e77ac7c395c23392b
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/314535
Trust: Julie Qiu <julie@golang.org>
Run-TryBot: Julie Qiu <julie@golang.org>
TryBot-Result: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
GetSymbolHistooryWithPackageSymbols and GetSymbolHistoryFromTable are
exported for use in test scripts.
For golang/go#37102
Change-Id: I39f0dfd075ae89b69d08561be40ecc6de043f158
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/314509
Trust: Julie Qiu <julie@golang.org>
Run-TryBot: Julie Qiu <julie@golang.org>
TryBot-Result: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
upsertSymbolHistory is added, which inserts data into the symbol_history
table.
For golang/go#37102
Change-Id: Ia288e47d82ab9868db4bce196574ec2806216cce
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/313596
Trust: Julie Qiu <julie@golang.org>
Run-TryBot: Julie Qiu <julie@golang.org>
TryBot-Result: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
postgres.GetPackageSymbol and symbol.IntroducedHistory are factored into
a single GetSymbolHistory function, in preparation for optimizing this
logic.
For golang/go#37102
Change-Id: I04f726ae6f9731ee2e099b47e414aeaef90d5779
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/312049
Trust: Julie Qiu <julie@golang.org>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
We want to rename new_documentation to documentation, and drop the
current documentation table.
INSERTs into new_documentation are added as a first step.
Change-Id: I2f4038890436946b2c140c108faa4740e16abe4e
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/310375
Trust: Julie Qiu <julie@golang.org>
Run-TryBot: Julie Qiu <julie@golang.org>
TryBot-Result: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
A packageSymbol for a given package is now identified by the name,
synopsis and section.
For golang/go#37102
Change-Id: I30b730194e10d52d28518d744c24f5e6f1829f7a
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/301051
Trust: Julie Qiu <julie@golang.org>
Run-TryBot: Julie Qiu <julie@golang.org>
TryBot-Result: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
Code related to inserting into the symbol_history table is deleted. That
table is no longer used and will be dropped in a future CL.
Change-Id: I4c7e5646a529770a03b2946916255ccaa9630dfd
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/300209
Trust: Julie Qiu <julie@golang.org>
Run-TryBot: Julie Qiu <julie@golang.org>
TryBot-Result: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jamal Carvalho <jamal@golang.org>
IntroducedHistory is added, which returns the first version when a
symbol name is added to the package API.
For golang/go#37102
Change-Id: Ia7c60d2a0258aa20d4e6c3ea71446c75d7784623
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/298072
Trust: Julie Qiu <julie@golang.org>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
A bug is fixed in getUnitSymbols where buildToSymbols was never
populated.
Various bugs in the tests are also fixed
Change-Id: If1dba8ea983e9a5a959adbdd94970a64af85b7f5
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/296450
Trust: Julie Qiu <julie@golang.org>
Run-TryBot: Julie Qiu <julie@golang.org>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
Data is now written to the documentation_symbols table. An experiment
flag "insert-symbols" is added for writing data to symbol_names,
package_symbols and documentation_symbols.
For golang/go#37102
Change-Id: Id95c754740a7dd5f836bf0766815b274dba6790b
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/296189
Trust: Julie Qiu <julie@golang.org>
Run-TryBot: Julie Qiu <julie@golang.org>
TryBot-Result: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
insertDocs now inserts new_goos and new_goarch. It returns a map of
pathToDocIDToDoc.
For golang/go#37102
Change-Id: Ifdeead539709427c445c5a106689c1208617fa65
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/296190
Trust: Julie Qiu <julie@golang.org>
Run-TryBot: Julie Qiu <julie@golang.org>
TryBot-Result: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
To avoid unnecessary write attempts when inserting symbols information,
run a SELECT query first to get existing symbol names and package
symbols, and only insert rows that we know do not exist.
For golang/go#37102
Change-Id: I6aaf96a34702c3736be6bf1e9386d5139404fafb
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/295270
Reviewed-by: Jonathan Amsterdam <jba@google.com>
Trust: Julie Qiu <julie@golang.org>
Instead of using a custom string as the key in getSymbolHistory, just
use internal.BuildContext.
For golang/go#37102
Change-Id: I87feabd36a6b33fc9a34bd5cf3ab0012bdbbf012
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/294689
Trust: Julie Qiu <julie@golang.org>
Run-TryBot: Julie Qiu <julie@golang.org>
TryBot-Result: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
insertSymbols is updated to match the new schema for symbols.
For golang/go#37102
Change-Id: Icd29fffe5d2d9a2ddf0201af7add9fb45fb9a30e
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/293954
Trust: Julie Qiu <julie@golang.org>
Run-TryBot: Julie Qiu <julie@golang.org>
TryBot-Result: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
Table symbols was renamed to symbol_names. References in the code are
also changed.
For golang/go#37102
Change-Id: I09e03be81ce3d23ba484b438a10d978e75cbb63a
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/294009
Trust: Julie Qiu <julie@golang.org>
Run-TryBot: Julie Qiu <julie@golang.org>
TryBot-Result: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
A script is added at cmd/internal/testing/stdlibsymbol/main.go, which
compares the stdlib API data in
https://go.googlesource.com/go/+/refs/heads/master/api with that of the
symbol_history table.
This script is used to check that data on pkgsite is accurate during
development.
Change-Id: Ib453dc8d63ee45cc3c8eb40365f9baf5636adec3
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/293429
Trust: Julie Qiu <julie@golang.org>
Run-TryBot: Julie Qiu <julie@golang.org>
TryBot-Result: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
The insert-symbol-history experiment is moved so that symbol names are
no longer inserted. This is done so that we can rename symbols to
symbol_names in a future CL.
TestInsertSymbolNamesAndHistory is skipped until the migrations are
merged.
Change-Id: Idfffaab82cca2365f01b61d36c3967523b0647be
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/293949
Trust: Julie Qiu <julie@golang.org>
Run-TryBot: Julie Qiu <julie@golang.org>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
TryBot-Result: kokoro <noreply+kokoro@google.com>
Data for a package's symbol history is inserted into the symbol_history
table.
This change is added behind a feature flag as additional tests will be
added.
This change does not take into account cases where the GOOS/GOARCH
for a package changes from "all/all" to all of the supported build
context combinations at a later version. This edge case will be
addressed in a later CL.
For golang/go#37102
Change-Id: Ia3abf1cf63159cb5b1b2b926a305ab606926a4ec
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/292511
Trust: Julie Qiu <julie@golang.org>
Run-TryBot: Julie Qiu <julie@golang.org>
TryBot-Result: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
shouldUpdateSymbolHistory is added, which reports whether the row for
the given symbolName should be updated.
For golang/go#37102
Change-Id: I2e0330ad1aac7957917c0f0e8f90eeb64acca0f6
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/291349
Trust: Julie Qiu <julie@golang.org>
Run-TryBot: Julie Qiu <julie@golang.org>
TryBot-Result: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
Insert symbol names into the symbols table when package documentation is
inserted.
For golang/go#37102
Change-Id: I82cc0425ec9cdadc2c5449038fedfd6920e1adc0
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/291089
Trust: Julie Qiu <julie@golang.org>
Run-TryBot: Julie Qiu <julie@golang.org>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
Jonathan Amsterdam
Hana (Hyang-Ah) Kim
Julie Qiu