go
/
pkgsite
зеркало из https://github.com/golang/pkgsite.git
1
0
Форкнуть 0
Код Задачи Пакеты Проекты Релизы Вики Активность
pkgsite/internal/source/source_test.go

803 строки
26 KiB
Go
Исходник Обычный вид История

internal/source: a package for references to source Add internal/source, which will know how to generate URLs that refer to the source code or other files inside modules. This package will be used to link to module and package source code, and also for relative links in README files. For better testing, introduce the httpreplay package. With -record, it makes HTTP requests to the actual servers and records the responses. With no flags, it replays the recordings. Fixes b/141771930. Updates b/140431331. Change-Id: I72dcd5535b8be7c6b27dd89245a048d23fd14160 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/561217 CI-Result: Cloud Build <devtools-proctor-result-processor@system.gserviceaccount.com> Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-09-27 23:26:07 +03:00
// Copyright 2019 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package source
import (
internal/source: handle meta tags Use the go command's meta-tag mechanism to find source code for vanity module paths. If a module path doesn't match one of our built-in patterns, make a GET request to the path and parse the go-import and go-source meta tags on the page to learn where the source lives and how to link to it. This process is slightly different than godoc.org's (as implemented in go.googlesource.com/gddo): - godoc.org uses package import paths; we use module paths instead. That can result in different behavior when a module doesn't correspond to a package and no tags are served at the module path URL, even though its package URLs do serve tags. The discovery site will not show source links for those packages. This is a genuine breaking behavior change, but we're going to make it because serving tags at the module level is the right thing to do, and we can easily fix the handful of modules where this is a problem. - godoc.org needs to fetch the code, so it needs information from the tags like the vcs, and it needs to keep the go-import and go-source tags separate (the first being for fetching, the second for linking). Since the discovery site already has the code from the zip file, we just need go-source information, using go-imports as a fallback. - The go-source tag contents don't provide a way to insert a version, so they're not usable as-is for linking to versioned code. For now, we ignore the URL templates in the tag and just try to match the repo. So a vanity URL that redirects to GitHub will work fine, since we know how to link to files in GitHub; but one that redirects to some random code-hosting site will not have source links on the discovery site, even though it might on godoc.org. Fixes b/141771975. Change-Id: Ie07bff3a77350cdaa7e45af0c7e5bce0a922e945 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/562176 Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-09-28 22:43:15 +03:00
"context"
internal/source,postgres: save source info to the DB - Add a SourceInfo field to VersionInfo. - Add functions to convert a source.Info to and from JSON. - Modify postgres statements to read and write from the DB. Keep the VersionInfo.RepositoryURL field around for now to avoid messing with the frontend. Updates b/132219298, b/138752200. Change-Id: I6ebb3af3afa14fda92c2804ab2d0294742a3bf41 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/575156 CI-Result: Cloud Build <devtools-proctor-result-processor@system.gserviceaccount.com> Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-10-15 13:00:58 +03:00
"encoding/json"
internal/source: a package for references to source Add internal/source, which will know how to generate URLs that refer to the source code or other files inside modules. This package will be used to link to module and package source code, and also for relative links in README files. For better testing, introduce the httpreplay package. With -record, it makes HTTP requests to the actual servers and records the responses. With no flags, it replays the recordings. Fixes b/141771930. Updates b/140431331. Change-Id: I72dcd5535b8be7c6b27dd89245a048d23fd14160 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/561217 CI-Result: Cloud Build <devtools-proctor-result-processor@system.gserviceaccount.com> Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-09-27 23:26:07 +03:00
"flag"
"fmt"
internal/source: handle meta tags Use the go command's meta-tag mechanism to find source code for vanity module paths. If a module path doesn't match one of our built-in patterns, make a GET request to the path and parse the go-import and go-source meta tags on the page to learn where the source lives and how to link to it. This process is slightly different than godoc.org's (as implemented in go.googlesource.com/gddo): - godoc.org uses package import paths; we use module paths instead. That can result in different behavior when a module doesn't correspond to a package and no tags are served at the module path URL, even though its package URLs do serve tags. The discovery site will not show source links for those packages. This is a genuine breaking behavior change, but we're going to make it because serving tags at the module level is the right thing to do, and we can easily fix the handful of modules where this is a problem. - godoc.org needs to fetch the code, so it needs information from the tags like the vcs, and it needs to keep the go-import and go-source tags separate (the first being for fetching, the second for linking). Since the discovery site already has the code from the zip file, we just need go-source information, using go-imports as a fallback. - The go-source tag contents don't provide a way to insert a version, so they're not usable as-is for linking to versioned code. For now, we ignore the URL templates in the tag and just try to match the repo. So a vanity URL that redirects to GitHub will work fine, since we know how to link to files in GitHub; but one that redirects to some random code-hosting site will not have source links on the discovery site, even though it might on godoc.org. Fixes b/141771975. Change-Id: Ie07bff3a77350cdaa7e45af0c7e5bce0a922e945 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/562176 Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-09-28 22:43:15 +03:00
"io/ioutil"
internal/source: a package for references to source Add internal/source, which will know how to generate URLs that refer to the source code or other files inside modules. This package will be used to link to module and package source code, and also for relative links in README files. For better testing, introduce the httpreplay package. With -record, it makes HTTP requests to the actual servers and records the responses. With no flags, it replays the recordings. Fixes b/141771930. Updates b/140431331. Change-Id: I72dcd5535b8be7c6b27dd89245a048d23fd14160 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/561217 CI-Result: Cloud Build <devtools-proctor-result-processor@system.gserviceaccount.com> Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-09-27 23:26:07 +03:00
"net/http"
"os"
"path/filepath"
internal/source: handle meta tags Use the go command's meta-tag mechanism to find source code for vanity module paths. If a module path doesn't match one of our built-in patterns, make a GET request to the path and parse the go-import and go-source meta tags on the page to learn where the source lives and how to link to it. This process is slightly different than godoc.org's (as implemented in go.googlesource.com/gddo): - godoc.org uses package import paths; we use module paths instead. That can result in different behavior when a module doesn't correspond to a package and no tags are served at the module path URL, even though its package URLs do serve tags. The discovery site will not show source links for those packages. This is a genuine breaking behavior change, but we're going to make it because serving tags at the module level is the right thing to do, and we can easily fix the handful of modules where this is a problem. - godoc.org needs to fetch the code, so it needs information from the tags like the vcs, and it needs to keep the go-import and go-source tags separate (the first being for fetching, the second for linking). Since the discovery site already has the code from the zip file, we just need go-source information, using go-imports as a fallback. - The go-source tag contents don't provide a way to insert a version, so they're not usable as-is for linking to versioned code. For now, we ignore the URL templates in the tag and just try to match the repo. So a vanity URL that redirects to GitHub will work fine, since we know how to link to files in GitHub; but one that redirects to some random code-hosting site will not have source links on the discovery site, even though it might on godoc.org. Fixes b/141771975. Change-Id: Ie07bff3a77350cdaa7e45af0c7e5bce0a922e945 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/562176 Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-09-28 22:43:15 +03:00
"strings"
internal/source: a package for references to source Add internal/source, which will know how to generate URLs that refer to the source code or other files inside modules. This package will be used to link to module and package source code, and also for relative links in README files. For better testing, introduce the httpreplay package. With -record, it makes HTTP requests to the actual servers and records the responses. With no flags, it replays the recordings. Fixes b/141771930. Updates b/140431331. Change-Id: I72dcd5535b8be7c6b27dd89245a048d23fd14160 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/561217 CI-Result: Cloud Build <devtools-proctor-result-processor@system.gserviceaccount.com> Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-09-27 23:26:07 +03:00
"testing"
internal/source: add Client source.Client is added, which replaces the used of http.DefaultClient when fetching source code. This also allows us to set a custom timeout for tests vs when running the worker. Change-Id: I5b7b0fd32fa7a2cf836b951af1ad471751fade00 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/709198 Reviewed-by: Jonathan Amsterdam <jba@google.com>
2020-04-03 20:15:18 +03:00
"time"
internal/source: a package for references to source Add internal/source, which will know how to generate URLs that refer to the source code or other files inside modules. This package will be used to link to module and package source code, and also for relative links in README files. For better testing, introduce the httpreplay package. With -record, it makes HTTP requests to the actual servers and records the responses. With no flags, it replays the recordings. Fixes b/141771930. Updates b/140431331. Change-Id: I72dcd5535b8be7c6b27dd89245a048d23fd14160 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/561217 CI-Result: Cloud Build <devtools-proctor-result-processor@system.gserviceaccount.com> Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-09-27 23:26:07 +03:00
internal/source: handle meta tags Use the go command's meta-tag mechanism to find source code for vanity module paths. If a module path doesn't match one of our built-in patterns, make a GET request to the path and parse the go-import and go-source meta tags on the page to learn where the source lives and how to link to it. This process is slightly different than godoc.org's (as implemented in go.googlesource.com/gddo): - godoc.org uses package import paths; we use module paths instead. That can result in different behavior when a module doesn't correspond to a package and no tags are served at the module path URL, even though its package URLs do serve tags. The discovery site will not show source links for those packages. This is a genuine breaking behavior change, but we're going to make it because serving tags at the module level is the right thing to do, and we can easily fix the handful of modules where this is a problem. - godoc.org needs to fetch the code, so it needs information from the tags like the vcs, and it needs to keep the go-import and go-source tags separate (the first being for fetching, the second for linking). Since the discovery site already has the code from the zip file, we just need go-source information, using go-imports as a fallback. - The go-source tag contents don't provide a way to insert a version, so they're not usable as-is for linking to versioned code. For now, we ignore the URL templates in the tag and just try to match the repo. So a vanity URL that redirects to GitHub will work fine, since we know how to link to files in GitHub; but one that redirects to some random code-hosting site will not have source links on the discovery site, even though it might on godoc.org. Fixes b/141771975. Change-Id: Ie07bff3a77350cdaa7e45af0c7e5bce0a922e945 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/562176 Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-09-28 22:43:15 +03:00
"github.com/google/go-cmp/cmp"
internal/source: a package for references to source Add internal/source, which will know how to generate URLs that refer to the source code or other files inside modules. This package will be used to link to module and package source code, and also for relative links in README files. For better testing, introduce the httpreplay package. With -record, it makes HTTP requests to the actual servers and records the responses. With no flags, it replays the recordings. Fixes b/141771930. Updates b/140431331. Change-Id: I72dcd5535b8be7c6b27dd89245a048d23fd14160 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/561217 CI-Result: Cloud Build <devtools-proctor-result-processor@system.gserviceaccount.com> Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-09-27 23:26:07 +03:00
"github.com/google/go-replayers/httpreplay"
)
internal/source: add Client source.Client is added, which replaces the used of http.DefaultClient when fetching source code. This also allows us to set a custom timeout for tests vs when running the worker. Change-Id: I5b7b0fd32fa7a2cf836b951af1ad471751fade00 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/709198 Reviewed-by: Jonathan Amsterdam <jba@google.com>
2020-04-03 20:15:18 +03:00
var (
internal/proxy: deprecate TestVersion in favor of TestModule proxy.TestVersion is deprecated in favor of proxy.TestModule. TestModule represents a module version used to generate testdata. Change-Id: I718045917a352af218ccebb45fa0d2875a5b9610 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/709704 Reviewed-by: Jonathan Amsterdam <jba@google.com>
2020-04-04 19:21:29 +03:00
testTimeout = 2 * time.Second
internal/source: add Client source.Client is added, which replaces the used of http.DefaultClient when fetching source code. This also allows us to set a custom timeout for tests vs when running the worker. Change-Id: I5b7b0fd32fa7a2cf836b951af1ad471751fade00 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/709198 Reviewed-by: Jonathan Amsterdam <jba@google.com>
2020-04-03 20:15:18 +03:00
record = flag.Bool("record", false, "record interactions with other systems, for replay")
)
internal/source: a package for references to source Add internal/source, which will know how to generate URLs that refer to the source code or other files inside modules. This package will be used to link to module and package source code, and also for relative links in README files. For better testing, introduce the httpreplay package. With -record, it makes HTTP requests to the actual servers and records the responses. With no flags, it replays the recordings. Fixes b/141771930. Updates b/140431331. Change-Id: I72dcd5535b8be7c6b27dd89245a048d23fd14160 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/561217 CI-Result: Cloud Build <devtools-proctor-result-processor@system.gserviceaccount.com> Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-09-27 23:26:07 +03:00
func TestModuleInfo(t *testing.T) {
client, done := newReplayClient(t, *record)
defer done()
internal/source: handle raw links to the std module specially The module directory we use to create links to the standard module's source is "src". That is only partly true, however: the .go files are there, but the README is at the repo root. This matters when constructing raw links from the README. We add a special check for this case in Info.RawURL. Note that this is a bit fragile and hacky: if we ever wanted a raw link to a source file, for example, the URL would be wrong. But the only alternative I can see now is to make everything more complicated just for this one case, which isn't worth it. Fixes b/143208537. Change-Id: Ide47b3739eaaf9f6b7e9b4631811ca3695353bb3 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/584337 CI-Result: Cloud Build <devtools-proctor-result-processor@system.gserviceaccount.com> Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-10-26 01:39:20 +03:00
check := func(t *testing.T, msg, got, want string) {
if got != want {
t.Fatalf("%s:\ngot %s\nwant %s", msg, got, want)
}
res, err := client.Head(got)
if err != nil {
t.Fatalf("%s: %v", got, err)
}
defer res.Body.Close()
if res.StatusCode != 200 {
t.Fatalf("%s: %v", got, res.Status)
}
}
internal/source: a package for references to source Add internal/source, which will know how to generate URLs that refer to the source code or other files inside modules. This package will be used to link to module and package source code, and also for relative links in README files. For better testing, introduce the httpreplay package. With -record, it makes HTTP requests to the actual servers and records the responses. With no flags, it replays the recordings. Fixes b/141771930. Updates b/140431331. Change-Id: I72dcd5535b8be7c6b27dd89245a048d23fd14160 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/561217 CI-Result: Cloud Build <devtools-proctor-result-processor@system.gserviceaccount.com> Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-09-27 23:26:07 +03:00
for _, test := range []struct {
internal/source: support raw content templates Add support for a URL template that can produce URLs for raw file content. To be used to generate relative links in READMEs. Change-Id: I6f6e1662825b96ddccd9dbcc4e48fbc92e7b0d82 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/572879 CI-Result: Cloud Build <devtools-proctor-result-processor@system.gserviceaccount.com> Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-10-15 02:40:13 +03:00
desc string
modulePath, version, file string
wantRepo, wantModule, wantFile, wantLine, wantRaw string
internal/source: a package for references to source Add internal/source, which will know how to generate URLs that refer to the source code or other files inside modules. This package will be used to link to module and package source code, and also for relative links in README files. For better testing, introduce the httpreplay package. With -record, it makes HTTP requests to the actual servers and records the responses. With no flags, it replays the recordings. Fixes b/141771930. Updates b/140431331. Change-Id: I72dcd5535b8be7c6b27dd89245a048d23fd14160 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/561217 CI-Result: Cloud Build <devtools-proctor-result-processor@system.gserviceaccount.com> Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-09-27 23:26:07 +03:00
}{
{
"standard library",
"std", "v1.12.0", "bytes/buffer.go",
"https://github.com/golang/go",
"https://github.com/golang/go/tree/go1.12/src",
"https://github.com/golang/go/blob/go1.12/src/bytes/buffer.go",
"https://github.com/golang/go/blob/go1.12/src/bytes/buffer.go#L1",
internal/source: handle raw links to the std module specially The module directory we use to create links to the standard module's source is "src". That is only partly true, however: the .go files are there, but the README is at the repo root. This matters when constructing raw links from the README. We add a special check for this case in Info.RawURL. Note that this is a bit fragile and hacky: if we ever wanted a raw link to a source file, for example, the URL would be wrong. But the only alternative I can see now is to make everything more complicated just for this one case, which isn't worth it. Fixes b/143208537. Change-Id: Ide47b3739eaaf9f6b7e9b4631811ca3695353bb3 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/584337 CI-Result: Cloud Build <devtools-proctor-result-processor@system.gserviceaccount.com> Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-10-26 01:39:20 +03:00
// The raw URLs for the standard library are relative to the repo root, not
// the module directory.
"",
internal/source: a package for references to source Add internal/source, which will know how to generate URLs that refer to the source code or other files inside modules. This package will be used to link to module and package source code, and also for relative links in README files. For better testing, introduce the httpreplay package. With -record, it makes HTTP requests to the actual servers and records the responses. With no flags, it replays the recordings. Fixes b/141771930. Updates b/140431331. Change-Id: I72dcd5535b8be7c6b27dd89245a048d23fd14160 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/561217 CI-Result: Cloud Build <devtools-proctor-result-processor@system.gserviceaccount.com> Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-09-27 23:26:07 +03:00
},
{
"old standard library",
"std", "v1.3.0", "bytes/buffer.go",
"https://github.com/golang/go",
"https://github.com/golang/go/tree/go1.3/src/pkg",
"https://github.com/golang/go/blob/go1.3/src/pkg/bytes/buffer.go",
"https://github.com/golang/go/blob/go1.3/src/pkg/bytes/buffer.go#L1",
internal/source: handle raw links to the std module specially The module directory we use to create links to the standard module's source is "src". That is only partly true, however: the .go files are there, but the README is at the repo root. This matters when constructing raw links from the README. We add a special check for this case in Info.RawURL. Note that this is a bit fragile and hacky: if we ever wanted a raw link to a source file, for example, the URL would be wrong. But the only alternative I can see now is to make everything more complicated just for this one case, which isn't worth it. Fixes b/143208537. Change-Id: Ide47b3739eaaf9f6b7e9b4631811ca3695353bb3 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/584337 CI-Result: Cloud Build <devtools-proctor-result-processor@system.gserviceaccount.com> Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-10-26 01:39:20 +03:00
// The raw URLs for the standard library are relative to the repo root, not
// the module directory.
"",
internal/source: a package for references to source Add internal/source, which will know how to generate URLs that refer to the source code or other files inside modules. This package will be used to link to module and package source code, and also for relative links in README files. For better testing, introduce the httpreplay package. With -record, it makes HTTP requests to the actual servers and records the responses. With no flags, it replays the recordings. Fixes b/141771930. Updates b/140431331. Change-Id: I72dcd5535b8be7c6b27dd89245a048d23fd14160 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/561217 CI-Result: Cloud Build <devtools-proctor-result-processor@system.gserviceaccount.com> Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-09-27 23:26:07 +03:00
},
{
"github module at repo root",
"github.com/pkg/errors", "v0.8.1", "errors.go",
"https://github.com/pkg/errors",
"https://github.com/pkg/errors/tree/v0.8.1",
"https://github.com/pkg/errors/blob/v0.8.1/errors.go",
"https://github.com/pkg/errors/blob/v0.8.1/errors.go#L1",
internal/source: use better github raw template Replace the previous github template for raw data (raw.githubusercontent...) with a new one that is more uniform (the same as the blob template, with "raw" instead of "blob"). The main advantage is that we can drop the "repoPath" parameter to the template, which was necessary only to support the unusual GitHub URLs. Now that GitHub and GitLab templates are identical, we can drop the GitLab ones. We preserve "gitlab" in the map from kinds to templates so we can continue to unmarshal DB modules.source_info columns that use "gitlab". Change-Id: Id3d45929f0eb2c040109449d4208279cca44f079 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/243217 Reviewed-by: Julie Qiu <julie@golang.org>
2020-07-17 01:21:49 +03:00
"https://github.com/pkg/errors/raw/v0.8.1/errors.go",
internal/source: a package for references to source Add internal/source, which will know how to generate URLs that refer to the source code or other files inside modules. This package will be used to link to module and package source code, and also for relative links in README files. For better testing, introduce the httpreplay package. With -record, it makes HTTP requests to the actual servers and records the responses. With no flags, it replays the recordings. Fixes b/141771930. Updates b/140431331. Change-Id: I72dcd5535b8be7c6b27dd89245a048d23fd14160 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/561217 CI-Result: Cloud Build <devtools-proctor-result-processor@system.gserviceaccount.com> Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-09-27 23:26:07 +03:00
},
{
"github module not at repo root",
"github.com/hashicorp/consul/sdk", "v0.2.0", "freeport/freeport.go",
"https://github.com/hashicorp/consul",
"https://github.com/hashicorp/consul/tree/sdk/v0.2.0/sdk",
"https://github.com/hashicorp/consul/blob/sdk/v0.2.0/sdk/freeport/freeport.go",
"https://github.com/hashicorp/consul/blob/sdk/v0.2.0/sdk/freeport/freeport.go#L1",
internal/source: use better github raw template Replace the previous github template for raw data (raw.githubusercontent...) with a new one that is more uniform (the same as the blob template, with "raw" instead of "blob"). The main advantage is that we can drop the "repoPath" parameter to the template, which was necessary only to support the unusual GitHub URLs. Now that GitHub and GitLab templates are identical, we can drop the GitLab ones. We preserve "gitlab" in the map from kinds to templates so we can continue to unmarshal DB modules.source_info columns that use "gitlab". Change-Id: Id3d45929f0eb2c040109449d4208279cca44f079 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/243217 Reviewed-by: Julie Qiu <julie@golang.org>
2020-07-17 01:21:49 +03:00
"https://github.com/hashicorp/consul/raw/sdk/v0.2.0/sdk/freeport/freeport.go",
internal/source: a package for references to source Add internal/source, which will know how to generate URLs that refer to the source code or other files inside modules. This package will be used to link to module and package source code, and also for relative links in README files. For better testing, introduce the httpreplay package. With -record, it makes HTTP requests to the actual servers and records the responses. With no flags, it replays the recordings. Fixes b/141771930. Updates b/140431331. Change-Id: I72dcd5535b8be7c6b27dd89245a048d23fd14160 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/561217 CI-Result: Cloud Build <devtools-proctor-result-processor@system.gserviceaccount.com> Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-09-27 23:26:07 +03:00
},
{
"bitbucket",
"bitbucket.org/plazzaro/kami", "v1.2.1", "defaults.go",
"https://bitbucket.org/plazzaro/kami",
"https://bitbucket.org/plazzaro/kami/src/v1.2.1",
"https://bitbucket.org/plazzaro/kami/src/v1.2.1/defaults.go",
"https://bitbucket.org/plazzaro/kami/src/v1.2.1/defaults.go#lines-1",
internal/source: support raw content templates Add support for a URL template that can produce URLs for raw file content. To be used to generate relative links in READMEs. Change-Id: I6f6e1662825b96ddccd9dbcc4e48fbc92e7b0d82 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/572879 CI-Result: Cloud Build <devtools-proctor-result-processor@system.gserviceaccount.com> Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-10-15 02:40:13 +03:00
"https://bitbucket.org/plazzaro/kami/raw/v1.2.1/defaults.go",
internal/source: a package for references to source Add internal/source, which will know how to generate URLs that refer to the source code or other files inside modules. This package will be used to link to module and package source code, and also for relative links in README files. For better testing, introduce the httpreplay package. With -record, it makes HTTP requests to the actual servers and records the responses. With no flags, it replays the recordings. Fixes b/141771930. Updates b/140431331. Change-Id: I72dcd5535b8be7c6b27dd89245a048d23fd14160 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/561217 CI-Result: Cloud Build <devtools-proctor-result-processor@system.gserviceaccount.com> Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-09-27 23:26:07 +03:00
},
{
"incompatible",
"github.com/airbrake/gobrake", "v3.5.1+incompatible", "gobrake.go",
"https://github.com/airbrake/gobrake",
"https://github.com/airbrake/gobrake/tree/v3.5.1",
"https://github.com/airbrake/gobrake/blob/v3.5.1/gobrake.go",
"https://github.com/airbrake/gobrake/blob/v3.5.1/gobrake.go#L1",
internal/source: use better github raw template Replace the previous github template for raw data (raw.githubusercontent...) with a new one that is more uniform (the same as the blob template, with "raw" instead of "blob"). The main advantage is that we can drop the "repoPath" parameter to the template, which was necessary only to support the unusual GitHub URLs. Now that GitHub and GitLab templates are identical, we can drop the GitLab ones. We preserve "gitlab" in the map from kinds to templates so we can continue to unmarshal DB modules.source_info columns that use "gitlab". Change-Id: Id3d45929f0eb2c040109449d4208279cca44f079 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/243217 Reviewed-by: Julie Qiu <julie@golang.org>
2020-07-17 01:21:49 +03:00
"https://github.com/airbrake/gobrake/raw/v3.5.1/gobrake.go",
internal/source: a package for references to source Add internal/source, which will know how to generate URLs that refer to the source code or other files inside modules. This package will be used to link to module and package source code, and also for relative links in README files. For better testing, introduce the httpreplay package. With -record, it makes HTTP requests to the actual servers and records the responses. With no flags, it replays the recordings. Fixes b/141771930. Updates b/140431331. Change-Id: I72dcd5535b8be7c6b27dd89245a048d23fd14160 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/561217 CI-Result: Cloud Build <devtools-proctor-result-processor@system.gserviceaccount.com> Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-09-27 23:26:07 +03:00
},
{
"x/tools",
"golang.org/x/tools", "v0.0.0-20190927191325-030b2cf1153e", "README.md",
"https://github.com/golang/tools",
"https://github.com/golang/tools/tree/030b2cf1153e",
"https://github.com/golang/tools/blob/030b2cf1153e/README.md",
"https://github.com/golang/tools/blob/030b2cf1153e/README.md#L1",
internal/source: use better github raw template Replace the previous github template for raw data (raw.githubusercontent...) with a new one that is more uniform (the same as the blob template, with "raw" instead of "blob"). The main advantage is that we can drop the "repoPath" parameter to the template, which was necessary only to support the unusual GitHub URLs. Now that GitHub and GitLab templates are identical, we can drop the GitLab ones. We preserve "gitlab" in the map from kinds to templates so we can continue to unmarshal DB modules.source_info columns that use "gitlab". Change-Id: Id3d45929f0eb2c040109449d4208279cca44f079 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/243217 Reviewed-by: Julie Qiu <julie@golang.org>
2020-07-17 01:21:49 +03:00
"https://github.com/golang/tools/raw/030b2cf1153e/README.md",
internal/source: a package for references to source Add internal/source, which will know how to generate URLs that refer to the source code or other files inside modules. This package will be used to link to module and package source code, and also for relative links in README files. For better testing, introduce the httpreplay package. With -record, it makes HTTP requests to the actual servers and records the responses. With no flags, it replays the recordings. Fixes b/141771930. Updates b/140431331. Change-Id: I72dcd5535b8be7c6b27dd89245a048d23fd14160 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/561217 CI-Result: Cloud Build <devtools-proctor-result-processor@system.gserviceaccount.com> Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-09-27 23:26:07 +03:00
},
{
"x/tools/gopls",
internal/source: update tests Running with `go test -record` revealed some 404s (pages that used to exist but no longer do). I replaced them with similar pages. Change-Id: I6903229e8221923e22338bdae3f112308aecd19e Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/756267 CI-Result: Cloud Build <devtools-proctor-result-processor@system.gserviceaccount.com> Reviewed-by: Julie Qiu <julieqiu@google.com>
2020-05-28 20:16:18 +03:00
"golang.org/x/tools/gopls", "v0.4.0", "main.go",
internal/source: a package for references to source Add internal/source, which will know how to generate URLs that refer to the source code or other files inside modules. This package will be used to link to module and package source code, and also for relative links in README files. For better testing, introduce the httpreplay package. With -record, it makes HTTP requests to the actual servers and records the responses. With no flags, it replays the recordings. Fixes b/141771930. Updates b/140431331. Change-Id: I72dcd5535b8be7c6b27dd89245a048d23fd14160 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/561217 CI-Result: Cloud Build <devtools-proctor-result-processor@system.gserviceaccount.com> Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-09-27 23:26:07 +03:00
"https://github.com/golang/tools",
internal/source: update tests Running with `go test -record` revealed some 404s (pages that used to exist but no longer do). I replaced them with similar pages. Change-Id: I6903229e8221923e22338bdae3f112308aecd19e Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/756267 CI-Result: Cloud Build <devtools-proctor-result-processor@system.gserviceaccount.com> Reviewed-by: Julie Qiu <julieqiu@google.com>
2020-05-28 20:16:18 +03:00
"https://github.com/golang/tools/tree/gopls/v0.4.0/gopls",
"https://github.com/golang/tools/blob/gopls/v0.4.0/gopls/main.go",
"https://github.com/golang/tools/blob/gopls/v0.4.0/gopls/main.go#L1",
internal/source: use better github raw template Replace the previous github template for raw data (raw.githubusercontent...) with a new one that is more uniform (the same as the blob template, with "raw" instead of "blob"). The main advantage is that we can drop the "repoPath" parameter to the template, which was necessary only to support the unusual GitHub URLs. Now that GitHub and GitLab templates are identical, we can drop the GitLab ones. We preserve "gitlab" in the map from kinds to templates so we can continue to unmarshal DB modules.source_info columns that use "gitlab". Change-Id: Id3d45929f0eb2c040109449d4208279cca44f079 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/243217 Reviewed-by: Julie Qiu <julie@golang.org>
2020-07-17 01:21:49 +03:00
"https://github.com/golang/tools/raw/gopls/v0.4.0/gopls/main.go",
internal/source: a package for references to source Add internal/source, which will know how to generate URLs that refer to the source code or other files inside modules. This package will be used to link to module and package source code, and also for relative links in README files. For better testing, introduce the httpreplay package. With -record, it makes HTTP requests to the actual servers and records the responses. With no flags, it replays the recordings. Fixes b/141771930. Updates b/140431331. Change-Id: I72dcd5535b8be7c6b27dd89245a048d23fd14160 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/561217 CI-Result: Cloud Build <devtools-proctor-result-processor@system.gserviceaccount.com> Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-09-27 23:26:07 +03:00
},
{
"googlesource.com",
"go.googlesource.com/image.git", "v0.0.0-20190910094157-69e4b8554b2a", "math/fixed/fixed.go",
"https://go.googlesource.com/image",
"https://go.googlesource.com/image/+/69e4b8554b2a",
"https://go.googlesource.com/image/+/69e4b8554b2a/math/fixed/fixed.go",
"https://go.googlesource.com/image/+/69e4b8554b2a/math/fixed/fixed.go#1",
internal/source: support raw content templates Add support for a URL template that can produce URLs for raw file content. To be used to generate relative links in READMEs. Change-Id: I6f6e1662825b96ddccd9dbcc4e48fbc92e7b0d82 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/572879 CI-Result: Cloud Build <devtools-proctor-result-processor@system.gserviceaccount.com> Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-10-15 02:40:13 +03:00
"",
internal/source: a package for references to source Add internal/source, which will know how to generate URLs that refer to the source code or other files inside modules. This package will be used to link to module and package source code, and also for relative links in README files. For better testing, introduce the httpreplay package. With -record, it makes HTTP requests to the actual servers and records the responses. With no flags, it replays the recordings. Fixes b/141771930. Updates b/140431331. Change-Id: I72dcd5535b8be7c6b27dd89245a048d23fd14160 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/561217 CI-Result: Cloud Build <devtools-proctor-result-processor@system.gserviceaccount.com> Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-09-27 23:26:07 +03:00
},
internal/source: support git.apache.org git.apache.org repos are hosted on GitHub. The final ".git" in the module path throws off the meta-tag matcher, so handle it specially. Change-Id: I57a8da4d4a000f0acd72945b6efa4439579e6518 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/572877 CI-Result: Cloud Build <devtools-proctor-result-processor@system.gserviceaccount.com> Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-10-15 00:18:52 +03:00
{
"git.apache.org",
"git.apache.org/thrift.git", "v0.12.0", "lib/go/thrift/client.go",
"https://github.com/apache/thrift",
"https://github.com/apache/thrift/tree/v0.12.0",
"https://github.com/apache/thrift/blob/v0.12.0/lib/go/thrift/client.go",
"https://github.com/apache/thrift/blob/v0.12.0/lib/go/thrift/client.go#L1",
internal/source: use better github raw template Replace the previous github template for raw data (raw.githubusercontent...) with a new one that is more uniform (the same as the blob template, with "raw" instead of "blob"). The main advantage is that we can drop the "repoPath" parameter to the template, which was necessary only to support the unusual GitHub URLs. Now that GitHub and GitLab templates are identical, we can drop the GitLab ones. We preserve "gitlab" in the map from kinds to templates so we can continue to unmarshal DB modules.source_info columns that use "gitlab". Change-Id: Id3d45929f0eb2c040109449d4208279cca44f079 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/243217 Reviewed-by: Julie Qiu <julie@golang.org>
2020-07-17 01:21:49 +03:00
"https://github.com/apache/thrift/raw/v0.12.0/lib/go/thrift/client.go",
internal/source: support git.apache.org git.apache.org repos are hosted on GitHub. The final ".git" in the module path throws off the meta-tag matcher, so handle it specially. Change-Id: I57a8da4d4a000f0acd72945b6efa4439579e6518 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/572877 CI-Result: Cloud Build <devtools-proctor-result-processor@system.gserviceaccount.com> Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-10-15 00:18:52 +03:00
},
internal/source: handle meta tags Use the go command's meta-tag mechanism to find source code for vanity module paths. If a module path doesn't match one of our built-in patterns, make a GET request to the path and parse the go-import and go-source meta tags on the page to learn where the source lives and how to link to it. This process is slightly different than godoc.org's (as implemented in go.googlesource.com/gddo): - godoc.org uses package import paths; we use module paths instead. That can result in different behavior when a module doesn't correspond to a package and no tags are served at the module path URL, even though its package URLs do serve tags. The discovery site will not show source links for those packages. This is a genuine breaking behavior change, but we're going to make it because serving tags at the module level is the right thing to do, and we can easily fix the handful of modules where this is a problem. - godoc.org needs to fetch the code, so it needs information from the tags like the vcs, and it needs to keep the go-import and go-source tags separate (the first being for fetching, the second for linking). Since the discovery site already has the code from the zip file, we just need go-source information, using go-imports as a fallback. - The go-source tag contents don't provide a way to insert a version, so they're not usable as-is for linking to versioned code. For now, we ignore the URL templates in the tag and just try to match the repo. So a vanity URL that redirects to GitHub will work fine, since we know how to link to files in GitHub; but one that redirects to some random code-hosting site will not have source links on the discovery site, even though it might on godoc.org. Fixes b/141771975. Change-Id: Ie07bff3a77350cdaa7e45af0c7e5bce0a922e945 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/562176 Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-09-28 22:43:15 +03:00
{
"vanity for github",
"cloud.google.com/go/spanner", "v1.0.0", "doc.go",
"https://github.com/googleapis/google-cloud-go",
"https://github.com/googleapis/google-cloud-go/tree/spanner/v1.0.0/spanner",
"https://github.com/googleapis/google-cloud-go/blob/spanner/v1.0.0/spanner/doc.go",
"https://github.com/googleapis/google-cloud-go/blob/spanner/v1.0.0/spanner/doc.go#L1",
internal/source: use better github raw template Replace the previous github template for raw data (raw.githubusercontent...) with a new one that is more uniform (the same as the blob template, with "raw" instead of "blob"). The main advantage is that we can drop the "repoPath" parameter to the template, which was necessary only to support the unusual GitHub URLs. Now that GitHub and GitLab templates are identical, we can drop the GitLab ones. We preserve "gitlab" in the map from kinds to templates so we can continue to unmarshal DB modules.source_info columns that use "gitlab". Change-Id: Id3d45929f0eb2c040109449d4208279cca44f079 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/243217 Reviewed-by: Julie Qiu <julie@golang.org>
2020-07-17 01:21:49 +03:00
"https://github.com/googleapis/google-cloud-go/raw/spanner/v1.0.0/spanner/doc.go",
internal/source: handle meta tags Use the go command's meta-tag mechanism to find source code for vanity module paths. If a module path doesn't match one of our built-in patterns, make a GET request to the path and parse the go-import and go-source meta tags on the page to learn where the source lives and how to link to it. This process is slightly different than godoc.org's (as implemented in go.googlesource.com/gddo): - godoc.org uses package import paths; we use module paths instead. That can result in different behavior when a module doesn't correspond to a package and no tags are served at the module path URL, even though its package URLs do serve tags. The discovery site will not show source links for those packages. This is a genuine breaking behavior change, but we're going to make it because serving tags at the module level is the right thing to do, and we can easily fix the handful of modules where this is a problem. - godoc.org needs to fetch the code, so it needs information from the tags like the vcs, and it needs to keep the go-import and go-source tags separate (the first being for fetching, the second for linking). Since the discovery site already has the code from the zip file, we just need go-source information, using go-imports as a fallback. - The go-source tag contents don't provide a way to insert a version, so they're not usable as-is for linking to versioned code. For now, we ignore the URL templates in the tag and just try to match the repo. So a vanity URL that redirects to GitHub will work fine, since we know how to link to files in GitHub; but one that redirects to some random code-hosting site will not have source links on the discovery site, even though it might on godoc.org. Fixes b/141771975. Change-Id: Ie07bff3a77350cdaa7e45af0c7e5bce0a922e945 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/562176 Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-09-28 22:43:15 +03:00
},
{
"vanity for bitbucket",
"badc0de.net/pkg/glagolitic", "v0.0.0-20180930175637-92f736eb02d6", "doc.go",
"https://bitbucket.org/ivucica/go-glagolitic",
"https://bitbucket.org/ivucica/go-glagolitic/src/92f736eb02d6",
"https://bitbucket.org/ivucica/go-glagolitic/src/92f736eb02d6/doc.go",
"https://bitbucket.org/ivucica/go-glagolitic/src/92f736eb02d6/doc.go#lines-1",
internal/source: support raw content templates Add support for a URL template that can produce URLs for raw file content. To be used to generate relative links in READMEs. Change-Id: I6f6e1662825b96ddccd9dbcc4e48fbc92e7b0d82 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/572879 CI-Result: Cloud Build <devtools-proctor-result-processor@system.gserviceaccount.com> Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-10-15 02:40:13 +03:00
"https://bitbucket.org/ivucica/go-glagolitic/raw/92f736eb02d6/doc.go",
internal/source: handle meta tags Use the go command's meta-tag mechanism to find source code for vanity module paths. If a module path doesn't match one of our built-in patterns, make a GET request to the path and parse the go-import and go-source meta tags on the page to learn where the source lives and how to link to it. This process is slightly different than godoc.org's (as implemented in go.googlesource.com/gddo): - godoc.org uses package import paths; we use module paths instead. That can result in different behavior when a module doesn't correspond to a package and no tags are served at the module path URL, even though its package URLs do serve tags. The discovery site will not show source links for those packages. This is a genuine breaking behavior change, but we're going to make it because serving tags at the module level is the right thing to do, and we can easily fix the handful of modules where this is a problem. - godoc.org needs to fetch the code, so it needs information from the tags like the vcs, and it needs to keep the go-import and go-source tags separate (the first being for fetching, the second for linking). Since the discovery site already has the code from the zip file, we just need go-source information, using go-imports as a fallback. - The go-source tag contents don't provide a way to insert a version, so they're not usable as-is for linking to versioned code. For now, we ignore the URL templates in the tag and just try to match the repo. So a vanity URL that redirects to GitHub will work fine, since we know how to link to files in GitHub; but one that redirects to some random code-hosting site will not have source links on the discovery site, even though it might on godoc.org. Fixes b/141771975. Change-Id: Ie07bff3a77350cdaa7e45af0c7e5bce0a922e945 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/562176 Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-09-28 22:43:15 +03:00
},
{
"vanity for googlesource.com",
"cuelang.org/go", "v0.0.9", "cuego/doc.go",
"https://cue.googlesource.com/cue",
"https://cue.googlesource.com/cue/+/v0.0.9",
"https://cue.googlesource.com/cue/+/v0.0.9/cuego/doc.go",
"https://cue.googlesource.com/cue/+/v0.0.9/cuego/doc.go#1",
internal/source: support raw content templates Add support for a URL template that can produce URLs for raw file content. To be used to generate relative links in READMEs. Change-Id: I6f6e1662825b96ddccd9dbcc4e48fbc92e7b0d82 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/572879 CI-Result: Cloud Build <devtools-proctor-result-processor@system.gserviceaccount.com> Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-10-15 02:40:13 +03:00
"",
internal/source: handle meta tags Use the go command's meta-tag mechanism to find source code for vanity module paths. If a module path doesn't match one of our built-in patterns, make a GET request to the path and parse the go-import and go-source meta tags on the page to learn where the source lives and how to link to it. This process is slightly different than godoc.org's (as implemented in go.googlesource.com/gddo): - godoc.org uses package import paths; we use module paths instead. That can result in different behavior when a module doesn't correspond to a package and no tags are served at the module path URL, even though its package URLs do serve tags. The discovery site will not show source links for those packages. This is a genuine breaking behavior change, but we're going to make it because serving tags at the module level is the right thing to do, and we can easily fix the handful of modules where this is a problem. - godoc.org needs to fetch the code, so it needs information from the tags like the vcs, and it needs to keep the go-import and go-source tags separate (the first being for fetching, the second for linking). Since the discovery site already has the code from the zip file, we just need go-source information, using go-imports as a fallback. - The go-source tag contents don't provide a way to insert a version, so they're not usable as-is for linking to versioned code. For now, we ignore the URL templates in the tag and just try to match the repo. So a vanity URL that redirects to GitHub will work fine, since we know how to link to files in GitHub; but one that redirects to some random code-hosting site will not have source links on the discovery site, even though it might on godoc.org. Fixes b/141771975. Change-Id: Ie07bff3a77350cdaa7e45af0c7e5bce0a922e945 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/562176 Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-09-28 22:43:15 +03:00
},
internal/source: support gitlab module paths gitlab.com serves source information just like github. Add it to the list of known patterns. Change-Id: I0b102d2c56c03a4ec8904f089f09ef6793db9c7f Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/562177 Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-09-30 18:14:34 +03:00
{
internal/source: support gitee.com and gitlab.* Add two more patterns to our hard-coded list: - gitee.com is a GitHub look-alike, and links behave the same. - We'll assume any host of the form "gitlab.*" (e.g. "gitlab.ghn.vn") behaves like gitlab.com, which itself behaves like git. Change-Id: I4e4110699086ecd303ec4a4e5a882fb66f36c3b9 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/570673 CI-Result: Cloud Build <devtools-proctor-result-processor@system.gserviceaccount.com> Reviewed-by: Robert Findley <rfindley@google.com>
2019-10-12 19:44:45 +03:00
"gitlab.com",
internal/source: support gitlab module paths gitlab.com serves source information just like github. Add it to the list of known patterns. Change-Id: I0b102d2c56c03a4ec8904f089f09ef6793db9c7f Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/562177 Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-09-30 18:14:34 +03:00
"gitlab.com/akita/akita", "v1.4.1", "event.go",
"https://gitlab.com/akita/akita",
"https://gitlab.com/akita/akita/tree/v1.4.1",
"https://gitlab.com/akita/akita/blob/v1.4.1/event.go",
"https://gitlab.com/akita/akita/blob/v1.4.1/event.go#L1",
internal/source: support raw content templates Add support for a URL template that can produce URLs for raw file content. To be used to generate relative links in READMEs. Change-Id: I6f6e1662825b96ddccd9dbcc4e48fbc92e7b0d82 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/572879 CI-Result: Cloud Build <devtools-proctor-result-processor@system.gserviceaccount.com> Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-10-15 02:40:13 +03:00
"https://gitlab.com/akita/akita/raw/v1.4.1/event.go",
internal/source: support gitlab module paths gitlab.com serves source information just like github. Add it to the list of known patterns. Change-Id: I0b102d2c56c03a4ec8904f089f09ef6793db9c7f Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/562177 Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-09-30 18:14:34 +03:00
},
internal/source: support gitee.com and gitlab.* Add two more patterns to our hard-coded list: - gitee.com is a GitHub look-alike, and links behave the same. - We'll assume any host of the form "gitlab.*" (e.g. "gitlab.ghn.vn") behaves like gitlab.com, which itself behaves like git. Change-Id: I4e4110699086ecd303ec4a4e5a882fb66f36c3b9 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/570673 CI-Result: Cloud Build <devtools-proctor-result-processor@system.gserviceaccount.com> Reviewed-by: Robert Findley <rfindley@google.com>
2019-10-12 19:44:45 +03:00
{
"other gitlab",
"gitlab.66xue.com/daihao/logkit", "v0.1.18", "color.go",
"https://gitlab.66xue.com/daihao/logkit",
"https://gitlab.66xue.com/daihao/logkit/tree/v0.1.18",
"https://gitlab.66xue.com/daihao/logkit/blob/v0.1.18/color.go",
"https://gitlab.66xue.com/daihao/logkit/blob/v0.1.18/color.go#L1",
internal/source: support raw content templates Add support for a URL template that can produce URLs for raw file content. To be used to generate relative links in READMEs. Change-Id: I6f6e1662825b96ddccd9dbcc4e48fbc92e7b0d82 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/572879 CI-Result: Cloud Build <devtools-proctor-result-processor@system.gserviceaccount.com> Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-10-15 02:40:13 +03:00
"https://gitlab.66xue.com/daihao/logkit/raw/v0.1.18/color.go",
internal/source: support gitee.com and gitlab.* Add two more patterns to our hard-coded list: - gitee.com is a GitHub look-alike, and links behave the same. - We'll assume any host of the form "gitlab.*" (e.g. "gitlab.ghn.vn") behaves like gitlab.com, which itself behaves like git. Change-Id: I4e4110699086ecd303ec4a4e5a882fb66f36c3b9 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/570673 CI-Result: Cloud Build <devtools-proctor-result-processor@system.gserviceaccount.com> Reviewed-by: Robert Findley <rfindley@google.com>
2019-10-12 19:44:45 +03:00
},
{
"gitee.com",
internal/source: update tests Running with `go test -record` revealed some 404s (pages that used to exist but no longer do). I replaced them with similar pages. Change-Id: I6903229e8221923e22338bdae3f112308aecd19e Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/756267 CI-Result: Cloud Build <devtools-proctor-result-processor@system.gserviceaccount.com> Reviewed-by: Julie Qiu <julieqiu@google.com>
2020-05-28 20:16:18 +03:00
"gitee.com/Billcoding/gotypes", "v0.1.0", "type.go",
internal/source: support gitee.com and gitlab.* Add two more patterns to our hard-coded list: - gitee.com is a GitHub look-alike, and links behave the same. - We'll assume any host of the form "gitlab.*" (e.g. "gitlab.ghn.vn") behaves like gitlab.com, which itself behaves like git. Change-Id: I4e4110699086ecd303ec4a4e5a882fb66f36c3b9 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/570673 CI-Result: Cloud Build <devtools-proctor-result-processor@system.gserviceaccount.com> Reviewed-by: Robert Findley <rfindley@google.com>
2019-10-12 19:44:45 +03:00
internal/source: update tests Running with `go test -record` revealed some 404s (pages that used to exist but no longer do). I replaced them with similar pages. Change-Id: I6903229e8221923e22338bdae3f112308aecd19e Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/756267 CI-Result: Cloud Build <devtools-proctor-result-processor@system.gserviceaccount.com> Reviewed-by: Julie Qiu <julieqiu@google.com>
2020-05-28 20:16:18 +03:00
"https://gitee.com/Billcoding/gotypes",
"https://gitee.com/Billcoding/gotypes/tree/v0.1.0",
"https://gitee.com/Billcoding/gotypes/blob/v0.1.0/type.go",
"https://gitee.com/Billcoding/gotypes/blob/v0.1.0/type.go#L1",
"https://gitee.com/Billcoding/gotypes/raw/v0.1.0/type.go",
internal/source: support gitee.com and gitlab.* Add two more patterns to our hard-coded list: - gitee.com is a GitHub look-alike, and links behave the same. - We'll assume any host of the form "gitlab.*" (e.g. "gitlab.ghn.vn") behaves like gitlab.com, which itself behaves like git. Change-Id: I4e4110699086ecd303ec4a4e5a882fb66f36c3b9 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/570673 CI-Result: Cloud Build <devtools-proctor-result-processor@system.gserviceaccount.com> Reviewed-by: Robert Findley <rfindley@google.com>
2019-10-12 19:44:45 +03:00
},
internal/source: add sourcehut templates Add source templates for the sourcehut git hosting site (git.sr.ht). Fixes golang/go#38986. Change-Id: I581389affe3c777828189d8aa35e3910ffe1d130 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/245039 Reviewed-by: Julie Qiu <julie@golang.org>
2020-07-28 01:17:24 +03:00
{
"sourcehut",
"gioui.org", "v0.0.0-20200726090130-3b95e2918359", "op/op.go",
"https://git.sr.ht/~eliasnaur/gio",
"https://git.sr.ht/~eliasnaur/gio/tree/3b95e2918359",
"https://git.sr.ht/~eliasnaur/gio/tree/3b95e2918359/op/op.go",
"https://git.sr.ht/~eliasnaur/gio/tree/3b95e2918359/op/op.go#L1",
"https://git.sr.ht/~eliasnaur/gio/blob/3b95e2918359/op/op.go",
},
{
"sourcehut nested",
"gioui.org/app", "v0.0.0-20200726090130-3b95e2918359", "app.go",
"https://git.sr.ht/~eliasnaur/gio",
"https://git.sr.ht/~eliasnaur/gio/tree/3b95e2918359/app",
"https://git.sr.ht/~eliasnaur/gio/tree/3b95e2918359/app/app.go",
"https://git.sr.ht/~eliasnaur/gio/tree/3b95e2918359/app/app.go#L1",
"https://git.sr.ht/~eliasnaur/gio/blob/3b95e2918359/app/app.go",
},
internal/source: handle versions v2 and higher - Remove "vN" when constructing the module directory. - Distinguish subdirectory and branch cases by looking for a go.mod file. Fixes b/141769404. Change-Id: I20328ace5257c2102c0248e6f92bdce2debe94e5 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/562701 Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-09-30 20:36:59 +03:00
{
"v2 as a branch",
"github.com/jrick/wsrpc/v2", "v2.1.1", "rpc.go",
"https://github.com/jrick/wsrpc",
"https://github.com/jrick/wsrpc/tree/v2.1.1",
"https://github.com/jrick/wsrpc/blob/v2.1.1/rpc.go",
"https://github.com/jrick/wsrpc/blob/v2.1.1/rpc.go#L1",
internal/source: use better github raw template Replace the previous github template for raw data (raw.githubusercontent...) with a new one that is more uniform (the same as the blob template, with "raw" instead of "blob"). The main advantage is that we can drop the "repoPath" parameter to the template, which was necessary only to support the unusual GitHub URLs. Now that GitHub and GitLab templates are identical, we can drop the GitLab ones. We preserve "gitlab" in the map from kinds to templates so we can continue to unmarshal DB modules.source_info columns that use "gitlab". Change-Id: Id3d45929f0eb2c040109449d4208279cca44f079 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/243217 Reviewed-by: Julie Qiu <julie@golang.org>
2020-07-17 01:21:49 +03:00
"https://github.com/jrick/wsrpc/raw/v2.1.1/rpc.go",
internal/source: handle versions v2 and higher - Remove "vN" when constructing the module directory. - Distinguish subdirectory and branch cases by looking for a go.mod file. Fixes b/141769404. Change-Id: I20328ace5257c2102c0248e6f92bdce2debe94e5 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/562701 Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-09-30 20:36:59 +03:00
},
{
"v2 as subdirectory",
"gitlab.com/akita/akita/v2", "v2.0.0-rc.2", "event.go",
"https://gitlab.com/akita/akita",
"https://gitlab.com/akita/akita/tree/v2.0.0-rc.2/v2",
"https://gitlab.com/akita/akita/blob/v2.0.0-rc.2/v2/event.go",
"https://gitlab.com/akita/akita/blob/v2.0.0-rc.2/v2/event.go#L1",
internal/source: support raw content templates Add support for a URL template that can produce URLs for raw file content. To be used to generate relative links in READMEs. Change-Id: I6f6e1662825b96ddccd9dbcc4e48fbc92e7b0d82 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/572879 CI-Result: Cloud Build <devtools-proctor-result-processor@system.gserviceaccount.com> Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-10-15 02:40:13 +03:00
"https://gitlab.com/akita/akita/raw/v2.0.0-rc.2/v2/event.go",
internal/source: handle versions v2 and higher - Remove "vN" when constructing the module directory. - Distinguish subdirectory and branch cases by looking for a go.mod file. Fixes b/141769404. Change-Id: I20328ace5257c2102c0248e6f92bdce2debe94e5 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/562701 Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-09-30 20:36:59 +03:00
},
internal/source: match go-source templates against known patterns It's possible that a vanity import path's meta tags mention a known repo pattern only in the URL templates. This is what gopkg.in tags do. For example: <meta name="go-import" content="gopkg.in/yaml.v2 git https://gopkg.in/yaml.v2"> <meta name="go-source" content="gopkg.in/yaml.v2 _ https://github.com/go-yaml/yaml/tree/v2.2.3{/dir} https://github.com/go-yaml/yaml/blob/v2.2.3{/dir}/{file}#L{line}"> In this case, we can match the templates in the tag to our list of known patterns, and use the version-aware templates we get from that match instead of the ones in the tag. Here, for example, we can note that the source lives on GitHub and use the pattern that has "{commit}" instead of the hardcoded "v2.2.3", making it possible to refer to other versions of the module. Change-Id: Ia998f47ec52996eb7f71377624c638bedfe1994f Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/562704 Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-10-02 14:35:31 +03:00
{
"gopkg.in, one element",
"gopkg.in/yaml.v2", "v2.2.2", "yaml.go",
"https://github.com/go-yaml/yaml",
"https://github.com/go-yaml/yaml/tree/v2.2.2",
"https://github.com/go-yaml/yaml/blob/v2.2.2/yaml.go",
"https://github.com/go-yaml/yaml/blob/v2.2.2/yaml.go#L1",
internal/source: use better github raw template Replace the previous github template for raw data (raw.githubusercontent...) with a new one that is more uniform (the same as the blob template, with "raw" instead of "blob"). The main advantage is that we can drop the "repoPath" parameter to the template, which was necessary only to support the unusual GitHub URLs. Now that GitHub and GitLab templates are identical, we can drop the GitLab ones. We preserve "gitlab" in the map from kinds to templates so we can continue to unmarshal DB modules.source_info columns that use "gitlab". Change-Id: Id3d45929f0eb2c040109449d4208279cca44f079 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/243217 Reviewed-by: Julie Qiu <julie@golang.org>
2020-07-17 01:21:49 +03:00
"https://github.com/go-yaml/yaml/raw/v2.2.2/yaml.go",
internal/source: match go-source templates against known patterns It's possible that a vanity import path's meta tags mention a known repo pattern only in the URL templates. This is what gopkg.in tags do. For example: <meta name="go-import" content="gopkg.in/yaml.v2 git https://gopkg.in/yaml.v2"> <meta name="go-source" content="gopkg.in/yaml.v2 _ https://github.com/go-yaml/yaml/tree/v2.2.3{/dir} https://github.com/go-yaml/yaml/blob/v2.2.3{/dir}/{file}#L{line}"> In this case, we can match the templates in the tag to our list of known patterns, and use the version-aware templates we get from that match instead of the ones in the tag. Here, for example, we can note that the source lives on GitHub and use the pattern that has "{commit}" instead of the hardcoded "v2.2.3", making it possible to refer to other versions of the module. Change-Id: Ia998f47ec52996eb7f71377624c638bedfe1994f Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/562704 Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-10-02 14:35:31 +03:00
},
{
"gopkg.in, two elements",
"gopkg.in/boltdb/bolt.v1", "v1.3.0", "doc.go",
"https://github.com/boltdb/bolt",
"https://github.com/boltdb/bolt/tree/v1.3.0",
"https://github.com/boltdb/bolt/blob/v1.3.0/doc.go",
"https://github.com/boltdb/bolt/blob/v1.3.0/doc.go#L1",
internal/source: use better github raw template Replace the previous github template for raw data (raw.githubusercontent...) with a new one that is more uniform (the same as the blob template, with "raw" instead of "blob"). The main advantage is that we can drop the "repoPath" parameter to the template, which was necessary only to support the unusual GitHub URLs. Now that GitHub and GitLab templates are identical, we can drop the GitLab ones. We preserve "gitlab" in the map from kinds to templates so we can continue to unmarshal DB modules.source_info columns that use "gitlab". Change-Id: Id3d45929f0eb2c040109449d4208279cca44f079 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/243217 Reviewed-by: Julie Qiu <julie@golang.org>
2020-07-17 01:21:49 +03:00
"https://github.com/boltdb/bolt/raw/v1.3.0/doc.go",
internal/source: match go-source templates against known patterns It's possible that a vanity import path's meta tags mention a known repo pattern only in the URL templates. This is what gopkg.in tags do. For example: <meta name="go-import" content="gopkg.in/yaml.v2 git https://gopkg.in/yaml.v2"> <meta name="go-source" content="gopkg.in/yaml.v2 _ https://github.com/go-yaml/yaml/tree/v2.2.3{/dir} https://github.com/go-yaml/yaml/blob/v2.2.3{/dir}/{file}#L{line}"> In this case, we can match the templates in the tag to our list of known patterns, and use the version-aware templates we get from that match instead of the ones in the tag. Here, for example, we can note that the source lives on GitHub and use the pattern that has "{commit}" instead of the hardcoded "v2.2.3", making it possible to refer to other versions of the module. Change-Id: Ia998f47ec52996eb7f71377624c638bedfe1994f Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/562704 Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-10-02 14:35:31 +03:00
},
internal/source: read body even on error Some sites (gonum.org in particular) return an error status (like 404) to a ?go=get=1 URL, but also serve a body with meta tags. Ignore the 404 for the http request. Fixes b/144596157. Change-Id: Ib59177b6847890975768de8fe4b623e188828f1f Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/622164 CI-Result: Cloud Build <devtools-proctor-result-processor@system.gserviceaccount.com> Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-12-18 02:25:11 +03:00
{
"gonum.org",
"gonum.org/v1/gonum", "v0.6.1", "doc.go",
"https://github.com/gonum/gonum",
"https://github.com/gonum/gonum/tree/v0.6.1",
"https://github.com/gonum/gonum/blob/v0.6.1/doc.go",
"https://github.com/gonum/gonum/blob/v0.6.1/doc.go#L1",
internal/source: use better github raw template Replace the previous github template for raw data (raw.githubusercontent...) with a new one that is more uniform (the same as the blob template, with "raw" instead of "blob"). The main advantage is that we can drop the "repoPath" parameter to the template, which was necessary only to support the unusual GitHub URLs. Now that GitHub and GitLab templates are identical, we can drop the GitLab ones. We preserve "gitlab" in the map from kinds to templates so we can continue to unmarshal DB modules.source_info columns that use "gitlab". Change-Id: Id3d45929f0eb2c040109449d4208279cca44f079 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/243217 Reviewed-by: Julie Qiu <julie@golang.org>
2020-07-17 01:21:49 +03:00
"https://github.com/gonum/gonum/raw/v0.6.1/doc.go",
internal/source: read body even on error Some sites (gonum.org in particular) return an error status (like 404) to a ?go=get=1 URL, but also serve a body with meta tags. Ignore the 404 for the http request. Fixes b/144596157. Change-Id: Ib59177b6847890975768de8fe4b623e188828f1f Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/622164 CI-Result: Cloud Build <devtools-proctor-result-processor@system.gserviceaccount.com> Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-12-18 02:25:11 +03:00
},
internal/source: a package for references to source Add internal/source, which will know how to generate URLs that refer to the source code or other files inside modules. This package will be used to link to module and package source code, and also for relative links in README files. For better testing, introduce the httpreplay package. With -record, it makes HTTP requests to the actual servers and records the responses. With no flags, it replays the recordings. Fixes b/141771930. Updates b/140431331. Change-Id: I72dcd5535b8be7c6b27dd89245a048d23fd14160 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/561217 CI-Result: Cloud Build <devtools-proctor-result-processor@system.gserviceaccount.com> Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-09-27 23:26:07 +03:00
} {
t.Run(test.desc, func(t *testing.T) {
internal/source: really use replay client for test We were creating a reply client, but not passing it in, so TestModuleInfo was actually hitting the internet. Fixes b/158850091. Change-Id: I4e981299a538750cb519c441e05805b29c7925a8 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/769682 Reviewed-by: Julie Qiu <julieqiu@google.com>
2020-06-13 00:48:21 +03:00
info, err := ModuleInfo(context.Background(), &Client{client}, test.modulePath, test.version)
internal/source: a package for references to source Add internal/source, which will know how to generate URLs that refer to the source code or other files inside modules. This package will be used to link to module and package source code, and also for relative links in README files. For better testing, introduce the httpreplay package. With -record, it makes HTTP requests to the actual servers and records the responses. With no flags, it replays the recordings. Fixes b/141771930. Updates b/140431331. Change-Id: I72dcd5535b8be7c6b27dd89245a048d23fd14160 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/561217 CI-Result: Cloud Build <devtools-proctor-result-processor@system.gserviceaccount.com> Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-09-27 23:26:07 +03:00
if err != nil {
t.Fatal(err)
}
internal/source: handle raw links to the std module specially The module directory we use to create links to the standard module's source is "src". That is only partly true, however: the .go files are there, but the README is at the repo root. This matters when constructing raw links from the README. We add a special check for this case in Info.RawURL. Note that this is a bit fragile and hacky: if we ever wanted a raw link to a source file, for example, the URL would be wrong. But the only alternative I can see now is to make everything more complicated just for this one case, which isn't worth it. Fixes b/143208537. Change-Id: Ide47b3739eaaf9f6b7e9b4631811ca3695353bb3 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/584337 CI-Result: Cloud Build <devtools-proctor-result-processor@system.gserviceaccount.com> Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-10-26 01:39:20 +03:00
check(t, "repo", info.repoURL, test.wantRepo)
check(t, "module", info.ModuleURL(), test.wantModule)
check(t, "file", info.FileURL(test.file), test.wantFile)
check(t, "line", info.LineURL(test.file, 1), test.wantLine)
internal/source: support raw content templates Add support for a URL template that can produce URLs for raw file content. To be used to generate relative links in READMEs. Change-Id: I6f6e1662825b96ddccd9dbcc4e48fbc92e7b0d82 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/572879 CI-Result: Cloud Build <devtools-proctor-result-processor@system.gserviceaccount.com> Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-10-15 02:40:13 +03:00
if test.wantRaw != "" {
internal/source: handle raw links to the std module specially The module directory we use to create links to the standard module's source is "src". That is only partly true, however: the .go files are there, but the README is at the repo root. This matters when constructing raw links from the README. We add a special check for this case in Info.RawURL. Note that this is a bit fragile and hacky: if we ever wanted a raw link to a source file, for example, the URL would be wrong. But the only alternative I can see now is to make everything more complicated just for this one case, which isn't worth it. Fixes b/143208537. Change-Id: Ide47b3739eaaf9f6b7e9b4631811ca3695353bb3 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/584337 CI-Result: Cloud Build <devtools-proctor-result-processor@system.gserviceaccount.com> Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-10-26 01:39:20 +03:00
check(t, "raw", info.RawURL(test.file), test.wantRaw)
internal/source: support raw content templates Add support for a URL template that can produce URLs for raw file content. To be used to generate relative links in READMEs. Change-Id: I6f6e1662825b96ddccd9dbcc4e48fbc92e7b0d82 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/572879 CI-Result: Cloud Build <devtools-proctor-result-processor@system.gserviceaccount.com> Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-10-15 02:40:13 +03:00
}
internal/source: a package for references to source Add internal/source, which will know how to generate URLs that refer to the source code or other files inside modules. This package will be used to link to module and package source code, and also for relative links in README files. For better testing, introduce the httpreplay package. With -record, it makes HTTP requests to the actual servers and records the responses. With no flags, it replays the recordings. Fixes b/141771930. Updates b/140431331. Change-Id: I72dcd5535b8be7c6b27dd89245a048d23fd14160 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/561217 CI-Result: Cloud Build <devtools-proctor-result-processor@system.gserviceaccount.com> Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-09-27 23:26:07 +03:00
})
}
internal/source: handle raw links to the std module specially The module directory we use to create links to the standard module's source is "src". That is only partly true, however: the .go files are there, but the README is at the repo root. This matters when constructing raw links from the README. We add a special check for this case in Info.RawURL. Note that this is a bit fragile and hacky: if we ever wanted a raw link to a source file, for example, the URL would be wrong. But the only alternative I can see now is to make everything more complicated just for this one case, which isn't worth it. Fixes b/143208537. Change-Id: Ide47b3739eaaf9f6b7e9b4631811ca3695353bb3 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/584337 CI-Result: Cloud Build <devtools-proctor-result-processor@system.gserviceaccount.com> Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-10-26 01:39:20 +03:00
t.Run("stdlib-raw", func(t *testing.T) {
// Test raw URLs from the standard library, which are a special case.
internal/source: really use replay client for test We were creating a reply client, but not passing it in, so TestModuleInfo was actually hitting the internet. Fixes b/158850091. Change-Id: I4e981299a538750cb519c441e05805b29c7925a8 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/769682 Reviewed-by: Julie Qiu <julieqiu@google.com>
2020-06-13 00:48:21 +03:00
info, err := ModuleInfo(context.Background(), &Client{client}, "std", "v1.13.3")
internal/source: handle raw links to the std module specially The module directory we use to create links to the standard module's source is "src". That is only partly true, however: the .go files are there, but the README is at the repo root. This matters when constructing raw links from the README. We add a special check for this case in Info.RawURL. Note that this is a bit fragile and hacky: if we ever wanted a raw link to a source file, for example, the URL would be wrong. But the only alternative I can see now is to make everything more complicated just for this one case, which isn't worth it. Fixes b/143208537. Change-Id: Ide47b3739eaaf9f6b7e9b4631811ca3695353bb3 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/584337 CI-Result: Cloud Build <devtools-proctor-result-processor@system.gserviceaccount.com> Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-10-26 01:39:20 +03:00
if err != nil {
t.Fatal(err)
}
const (
file = "doc/gopher/fiveyears.jpg"
internal/source: use better github raw template Replace the previous github template for raw data (raw.githubusercontent...) with a new one that is more uniform (the same as the blob template, with "raw" instead of "blob"). The main advantage is that we can drop the "repoPath" parameter to the template, which was necessary only to support the unusual GitHub URLs. Now that GitHub and GitLab templates are identical, we can drop the GitLab ones. We preserve "gitlab" in the map from kinds to templates so we can continue to unmarshal DB modules.source_info columns that use "gitlab". Change-Id: Id3d45929f0eb2c040109449d4208279cca44f079 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/243217 Reviewed-by: Julie Qiu <julie@golang.org>
2020-07-17 01:21:49 +03:00
want = "https://github.com/golang/go/raw/go1.13.3/doc/gopher/fiveyears.jpg"
internal/source: handle raw links to the std module specially The module directory we use to create links to the standard module's source is "src". That is only partly true, however: the .go files are there, but the README is at the repo root. This matters when constructing raw links from the README. We add a special check for this case in Info.RawURL. Note that this is a bit fragile and hacky: if we ever wanted a raw link to a source file, for example, the URL would be wrong. But the only alternative I can see now is to make everything more complicated just for this one case, which isn't worth it. Fixes b/143208537. Change-Id: Ide47b3739eaaf9f6b7e9b4631811ca3695353bb3 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/584337 CI-Result: Cloud Build <devtools-proctor-result-processor@system.gserviceaccount.com> Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-10-26 01:39:20 +03:00
)
check(t, "raw", info.RawURL(file), want)
})
internal/source: a package for references to source Add internal/source, which will know how to generate URLs that refer to the source code or other files inside modules. This package will be used to link to module and package source code, and also for relative links in README files. For better testing, introduce the httpreplay package. With -record, it makes HTTP requests to the actual servers and records the responses. With no flags, it replays the recordings. Fixes b/141771930. Updates b/140431331. Change-Id: I72dcd5535b8be7c6b27dd89245a048d23fd14160 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/561217 CI-Result: Cloud Build <devtools-proctor-result-processor@system.gserviceaccount.com> Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-09-27 23:26:07 +03:00
}
internal/source: handle meta tags Use the go command's meta-tag mechanism to find source code for vanity module paths. If a module path doesn't match one of our built-in patterns, make a GET request to the path and parse the go-import and go-source meta tags on the page to learn where the source lives and how to link to it. This process is slightly different than godoc.org's (as implemented in go.googlesource.com/gddo): - godoc.org uses package import paths; we use module paths instead. That can result in different behavior when a module doesn't correspond to a package and no tags are served at the module path URL, even though its package URLs do serve tags. The discovery site will not show source links for those packages. This is a genuine breaking behavior change, but we're going to make it because serving tags at the module level is the right thing to do, and we can easily fix the handful of modules where this is a problem. - godoc.org needs to fetch the code, so it needs information from the tags like the vcs, and it needs to keep the go-import and go-source tags separate (the first being for fetching, the second for linking). Since the discovery site already has the code from the zip file, we just need go-source information, using go-imports as a fallback. - The go-source tag contents don't provide a way to insert a version, so they're not usable as-is for linking to versioned code. For now, we ignore the URL templates in the tag and just try to match the repo. So a vanity URL that redirects to GitHub will work fine, since we know how to link to files in GitHub; but one that redirects to some random code-hosting site will not have source links on the discovery site, even though it might on godoc.org. Fixes b/141771975. Change-Id: Ie07bff3a77350cdaa7e45af0c7e5bce0a922e945 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/562176 Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-09-28 22:43:15 +03:00
func newReplayClient(t *testing.T, record bool) (*http.Client, func()) {
replayFilePath := filepath.Join("testdata", t.Name()+".replay")
if record {
httpreplay.DebugHeaders()
t.Logf("Recording into %s", replayFilePath)
if err := os.MkdirAll(filepath.Dir(replayFilePath), 0755); err != nil {
t.Fatal(err)
}
rec, err := httpreplay.NewRecorder(replayFilePath, nil)
if err != nil {
t.Fatal(err)
}
return rec.Client(), func() {
if err := rec.Close(); err != nil {
t.Fatal(err)
}
}
} else {
rep, err := httpreplay.NewReplayer(replayFilePath)
if err != nil {
t.Fatal(err)
}
return rep.Client(), func() { _ = rep.Close() }
}
}
internal/source: a package for references to source Add internal/source, which will know how to generate URLs that refer to the source code or other files inside modules. This package will be used to link to module and package source code, and also for relative links in README files. For better testing, introduce the httpreplay package. With -record, it makes HTTP requests to the actual servers and records the responses. With no flags, it replays the recordings. Fixes b/141771930. Updates b/140431331. Change-Id: I72dcd5535b8be7c6b27dd89245a048d23fd14160 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/561217 CI-Result: Cloud Build <devtools-proctor-result-processor@system.gserviceaccount.com> Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-09-27 23:26:07 +03:00
func TestMatchStatic(t *testing.T) {
for _, test := range []struct {
internal/source: handle versions v2 and higher - Remove "vN" when constructing the module directory. - Distinguish subdirectory and branch cases by looking for a go.mod file. Fixes b/141769404. Change-Id: I20328ace5257c2102c0248e6f92bdce2debe94e5 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/562701 Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-09-30 20:36:59 +03:00
in string
wantRepo, wantSuffix string
internal/source: a package for references to source Add internal/source, which will know how to generate URLs that refer to the source code or other files inside modules. This package will be used to link to module and package source code, and also for relative links in README files. For better testing, introduce the httpreplay package. With -record, it makes HTTP requests to the actual servers and records the responses. With no flags, it replays the recordings. Fixes b/141771930. Updates b/140431331. Change-Id: I72dcd5535b8be7c6b27dd89245a048d23fd14160 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/561217 CI-Result: Cloud Build <devtools-proctor-result-processor@system.gserviceaccount.com> Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-09-27 23:26:07 +03:00
}{
{"github.com/a/b", "github.com/a/b", ""},
{"bitbucket.org/a/b", "bitbucket.org/a/b", ""},
{"github.com/a/b/c/d", "github.com/a/b", "c/d"},
{"bitbucket.org/a/b/c/d", "bitbucket.org/a/b", "c/d"},
{"foo.googlesource.com/a/b/c", "foo.googlesource.com/a/b/c", ""},
{"foo.googlesource.com/a/b/c.git", "foo.googlesource.com/a/b/c", ""},
{"foo.googlesource.com/a/b/c.git/d", "foo.googlesource.com/a/b/c", "d"},
internal/source: handle versions v2 and higher - Remove "vN" when constructing the module directory. - Distinguish subdirectory and branch cases by looking for a go.mod file. Fixes b/141769404. Change-Id: I20328ace5257c2102c0248e6f92bdce2debe94e5 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/562701 Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-09-30 20:36:59 +03:00
{"git.com/repo.git", "git.com/repo", ""},
{"git.com/repo.git/dir", "git.com/repo", "dir"},
internal/source: a package for references to source Add internal/source, which will know how to generate URLs that refer to the source code or other files inside modules. This package will be used to link to module and package source code, and also for relative links in README files. For better testing, introduce the httpreplay package. With -record, it makes HTTP requests to the actual servers and records the responses. With no flags, it replays the recordings. Fixes b/141771930. Updates b/140431331. Change-Id: I72dcd5535b8be7c6b27dd89245a048d23fd14160 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/561217 CI-Result: Cloud Build <devtools-proctor-result-processor@system.gserviceaccount.com> Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-09-27 23:26:07 +03:00
{"mercurial.com/repo.hg", "mercurial.com/repo", ""},
{"mercurial.com/repo.hg/dir", "mercurial.com/repo", "dir"},
} {
t.Run(test.in, func(t *testing.T) {
internal/source: handle versions v2 and higher - Remove "vN" when constructing the module directory. - Distinguish subdirectory and branch cases by looking for a go.mod file. Fixes b/141769404. Change-Id: I20328ace5257c2102c0248e6f92bdce2debe94e5 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/562701 Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-09-30 20:36:59 +03:00
gotRepo, gotSuffix, _, err := matchStatic(test.in)
internal/source: a package for references to source Add internal/source, which will know how to generate URLs that refer to the source code or other files inside modules. This package will be used to link to module and package source code, and also for relative links in README files. For better testing, introduce the httpreplay package. With -record, it makes HTTP requests to the actual servers and records the responses. With no flags, it replays the recordings. Fixes b/141771930. Updates b/140431331. Change-Id: I72dcd5535b8be7c6b27dd89245a048d23fd14160 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/561217 CI-Result: Cloud Build <devtools-proctor-result-processor@system.gserviceaccount.com> Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-09-27 23:26:07 +03:00
if err != nil {
t.Fatal(err)
}
internal/source: handle versions v2 and higher - Remove "vN" when constructing the module directory. - Distinguish subdirectory and branch cases by looking for a go.mod file. Fixes b/141769404. Change-Id: I20328ace5257c2102c0248e6f92bdce2debe94e5 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/562701 Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-09-30 20:36:59 +03:00
if gotRepo != test.wantRepo || gotSuffix != test.wantSuffix {
t.Errorf("got %q, %q; want %q, %q", gotRepo, gotSuffix, test.wantRepo, test.wantSuffix)
internal/source: a package for references to source Add internal/source, which will know how to generate URLs that refer to the source code or other files inside modules. This package will be used to link to module and package source code, and also for relative links in README files. For better testing, introduce the httpreplay package. With -record, it makes HTTP requests to the actual servers and records the responses. With no flags, it replays the recordings. Fixes b/141771930. Updates b/140431331. Change-Id: I72dcd5535b8be7c6b27dd89245a048d23fd14160 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/561217 CI-Result: Cloud Build <devtools-proctor-result-processor@system.gserviceaccount.com> Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-09-27 23:26:07 +03:00
}
})
}
}
internal/source: handle meta tags Use the go command's meta-tag mechanism to find source code for vanity module paths. If a module path doesn't match one of our built-in patterns, make a GET request to the path and parse the go-import and go-source meta tags on the page to learn where the source lives and how to link to it. This process is slightly different than godoc.org's (as implemented in go.googlesource.com/gddo): - godoc.org uses package import paths; we use module paths instead. That can result in different behavior when a module doesn't correspond to a package and no tags are served at the module path URL, even though its package URLs do serve tags. The discovery site will not show source links for those packages. This is a genuine breaking behavior change, but we're going to make it because serving tags at the module level is the right thing to do, and we can easily fix the handful of modules where this is a problem. - godoc.org needs to fetch the code, so it needs information from the tags like the vcs, and it needs to keep the go-import and go-source tags separate (the first being for fetching, the second for linking). Since the discovery site already has the code from the zip file, we just need go-source information, using go-imports as a fallback. - The go-source tag contents don't provide a way to insert a version, so they're not usable as-is for linking to versioned code. For now, we ignore the URL templates in the tag and just try to match the repo. So a vanity URL that redirects to GitHub will work fine, since we know how to link to files in GitHub; but one that redirects to some random code-hosting site will not have source links on the discovery site, even though it might on godoc.org. Fixes b/141771975. Change-Id: Ie07bff3a77350cdaa7e45af0c7e5bce0a922e945 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/562176 Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-09-28 22:43:15 +03:00
// This test adapted from gddo/gosrc/gosrc_test.go:TestGetDynamic.
internal/{source,frontend}: escape HTML Escape any strings that may come from outside the program and will be rendered as HTML. Also, fix some typos. Also, put args to a cmp.Diff call in the right order. Change-Id: I1fc6013e200326af8719b14c28a607fc7936e20a Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/239478 Reviewed-by: Julie Qiu <julie@golang.org>
2020-06-18 03:22:39 +03:00
func TestModuleInfoDynamic(t *testing.T) {
internal/source: handle meta tags Use the go command's meta-tag mechanism to find source code for vanity module paths. If a module path doesn't match one of our built-in patterns, make a GET request to the path and parse the go-import and go-source meta tags on the page to learn where the source lives and how to link to it. This process is slightly different than godoc.org's (as implemented in go.googlesource.com/gddo): - godoc.org uses package import paths; we use module paths instead. That can result in different behavior when a module doesn't correspond to a package and no tags are served at the module path URL, even though its package URLs do serve tags. The discovery site will not show source links for those packages. This is a genuine breaking behavior change, but we're going to make it because serving tags at the module level is the right thing to do, and we can easily fix the handful of modules where this is a problem. - godoc.org needs to fetch the code, so it needs information from the tags like the vcs, and it needs to keep the go-import and go-source tags separate (the first being for fetching, the second for linking). Since the discovery site already has the code from the zip file, we just need go-source information, using go-imports as a fallback. - The go-source tag contents don't provide a way to insert a version, so they're not usable as-is for linking to versioned code. For now, we ignore the URL templates in the tag and just try to match the repo. So a vanity URL that redirects to GitHub will work fine, since we know how to link to files in GitHub; but one that redirects to some random code-hosting site will not have source links on the discovery site, even though it might on godoc.org. Fixes b/141771975. Change-Id: Ie07bff3a77350cdaa7e45af0c7e5bce0a922e945 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/562176 Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-09-28 22:43:15 +03:00
// For this test, fake the HTTP requests so we can cover cases that may not appear in the wild.
internal/source: add Client source.Client is added, which replaces the used of http.DefaultClient when fetching source code. This also allows us to set a custom timeout for tests vs when running the worker. Change-Id: I5b7b0fd32fa7a2cf836b951af1ad471751fade00 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/709198 Reviewed-by: Jonathan Amsterdam <jba@google.com>
2020-04-03 20:15:18 +03:00
client := &Client{
httpClient: &http.Client{
Transport: testTransport(testWeb),
Timeout: testTimeout,
},
}
internal/source: handle meta tags Use the go command's meta-tag mechanism to find source code for vanity module paths. If a module path doesn't match one of our built-in patterns, make a GET request to the path and parse the go-import and go-source meta tags on the page to learn where the source lives and how to link to it. This process is slightly different than godoc.org's (as implemented in go.googlesource.com/gddo): - godoc.org uses package import paths; we use module paths instead. That can result in different behavior when a module doesn't correspond to a package and no tags are served at the module path URL, even though its package URLs do serve tags. The discovery site will not show source links for those packages. This is a genuine breaking behavior change, but we're going to make it because serving tags at the module level is the right thing to do, and we can easily fix the handful of modules where this is a problem. - godoc.org needs to fetch the code, so it needs information from the tags like the vcs, and it needs to keep the go-import and go-source tags separate (the first being for fetching, the second for linking). Since the discovery site already has the code from the zip file, we just need go-source information, using go-imports as a fallback. - The go-source tag contents don't provide a way to insert a version, so they're not usable as-is for linking to versioned code. For now, we ignore the URL templates in the tag and just try to match the repo. So a vanity URL that redirects to GitHub will work fine, since we know how to link to files in GitHub; but one that redirects to some random code-hosting site will not have source links on the discovery site, even though it might on godoc.org. Fixes b/141771975. Change-Id: Ie07bff3a77350cdaa7e45af0c7e5bce0a922e945 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/562176 Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-09-28 22:43:15 +03:00
// The version doesn't figure into the interesting work and we test versions to commits
// elsewhere, so use the same version throughout.
const version = "v1.2.3"
for _, test := range []struct {
modulePath string
want *Info // if nil, then want error
}{
{
"alice.org/pkg",
&Info{
internal/source,postgres: save source info to the DB - Add a SourceInfo field to VersionInfo. - Add functions to convert a source.Info to and from JSON. - Modify postgres statements to read and write from the DB. Keep the VersionInfo.RepositoryURL field around for now to avoid messing with the frontend. Updates b/132219298, b/138752200. Change-Id: I6ebb3af3afa14fda92c2804ab2d0294742a3bf41 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/575156 CI-Result: Cloud Build <devtools-proctor-result-processor@system.gserviceaccount.com> Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-10-15 13:00:58 +03:00
repoURL: "https://github.com/alice/pkg",
internal/source: handle meta tags Use the go command's meta-tag mechanism to find source code for vanity module paths. If a module path doesn't match one of our built-in patterns, make a GET request to the path and parse the go-import and go-source meta tags on the page to learn where the source lives and how to link to it. This process is slightly different than godoc.org's (as implemented in go.googlesource.com/gddo): - godoc.org uses package import paths; we use module paths instead. That can result in different behavior when a module doesn't correspond to a package and no tags are served at the module path URL, even though its package URLs do serve tags. The discovery site will not show source links for those packages. This is a genuine breaking behavior change, but we're going to make it because serving tags at the module level is the right thing to do, and we can easily fix the handful of modules where this is a problem. - godoc.org needs to fetch the code, so it needs information from the tags like the vcs, and it needs to keep the go-import and go-source tags separate (the first being for fetching, the second for linking). Since the discovery site already has the code from the zip file, we just need go-source information, using go-imports as a fallback. - The go-source tag contents don't provide a way to insert a version, so they're not usable as-is for linking to versioned code. For now, we ignore the URL templates in the tag and just try to match the repo. So a vanity URL that redirects to GitHub will work fine, since we know how to link to files in GitHub; but one that redirects to some random code-hosting site will not have source links on the discovery site, even though it might on godoc.org. Fixes b/141771975. Change-Id: Ie07bff3a77350cdaa7e45af0c7e5bce0a922e945 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/562176 Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-09-28 22:43:15 +03:00
moduleDir: "",
commit: "v1.2.3",
templates: githubURLTemplates,
},
},
{
"alice.org/pkg/sub",
&Info{
internal/source,postgres: save source info to the DB - Add a SourceInfo field to VersionInfo. - Add functions to convert a source.Info to and from JSON. - Modify postgres statements to read and write from the DB. Keep the VersionInfo.RepositoryURL field around for now to avoid messing with the frontend. Updates b/132219298, b/138752200. Change-Id: I6ebb3af3afa14fda92c2804ab2d0294742a3bf41 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/575156 CI-Result: Cloud Build <devtools-proctor-result-processor@system.gserviceaccount.com> Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-10-15 13:00:58 +03:00
repoURL: "https://github.com/alice/pkg",
internal/source: handle meta tags Use the go command's meta-tag mechanism to find source code for vanity module paths. If a module path doesn't match one of our built-in patterns, make a GET request to the path and parse the go-import and go-source meta tags on the page to learn where the source lives and how to link to it. This process is slightly different than godoc.org's (as implemented in go.googlesource.com/gddo): - godoc.org uses package import paths; we use module paths instead. That can result in different behavior when a module doesn't correspond to a package and no tags are served at the module path URL, even though its package URLs do serve tags. The discovery site will not show source links for those packages. This is a genuine breaking behavior change, but we're going to make it because serving tags at the module level is the right thing to do, and we can easily fix the handful of modules where this is a problem. - godoc.org needs to fetch the code, so it needs information from the tags like the vcs, and it needs to keep the go-import and go-source tags separate (the first being for fetching, the second for linking). Since the discovery site already has the code from the zip file, we just need go-source information, using go-imports as a fallback. - The go-source tag contents don't provide a way to insert a version, so they're not usable as-is for linking to versioned code. For now, we ignore the URL templates in the tag and just try to match the repo. So a vanity URL that redirects to GitHub will work fine, since we know how to link to files in GitHub; but one that redirects to some random code-hosting site will not have source links on the discovery site, even though it might on godoc.org. Fixes b/141771975. Change-Id: Ie07bff3a77350cdaa7e45af0c7e5bce0a922e945 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/562176 Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-09-28 22:43:15 +03:00
moduleDir: "sub",
commit: "sub/v1.2.3",
templates: githubURLTemplates,
},
},
{
"alice.org/pkg/http",
&Info{
internal/source,postgres: save source info to the DB - Add a SourceInfo field to VersionInfo. - Add functions to convert a source.Info to and from JSON. - Modify postgres statements to read and write from the DB. Keep the VersionInfo.RepositoryURL field around for now to avoid messing with the frontend. Updates b/132219298, b/138752200. Change-Id: I6ebb3af3afa14fda92c2804ab2d0294742a3bf41 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/575156 CI-Result: Cloud Build <devtools-proctor-result-processor@system.gserviceaccount.com> Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-10-15 13:00:58 +03:00
repoURL: "https://github.com/alice/pkg",
internal/source: handle meta tags Use the go command's meta-tag mechanism to find source code for vanity module paths. If a module path doesn't match one of our built-in patterns, make a GET request to the path and parse the go-import and go-source meta tags on the page to learn where the source lives and how to link to it. This process is slightly different than godoc.org's (as implemented in go.googlesource.com/gddo): - godoc.org uses package import paths; we use module paths instead. That can result in different behavior when a module doesn't correspond to a package and no tags are served at the module path URL, even though its package URLs do serve tags. The discovery site will not show source links for those packages. This is a genuine breaking behavior change, but we're going to make it because serving tags at the module level is the right thing to do, and we can easily fix the handful of modules where this is a problem. - godoc.org needs to fetch the code, so it needs information from the tags like the vcs, and it needs to keep the go-import and go-source tags separate (the first being for fetching, the second for linking). Since the discovery site already has the code from the zip file, we just need go-source information, using go-imports as a fallback. - The go-source tag contents don't provide a way to insert a version, so they're not usable as-is for linking to versioned code. For now, we ignore the URL templates in the tag and just try to match the repo. So a vanity URL that redirects to GitHub will work fine, since we know how to link to files in GitHub; but one that redirects to some random code-hosting site will not have source links on the discovery site, even though it might on godoc.org. Fixes b/141771975. Change-Id: Ie07bff3a77350cdaa7e45af0c7e5bce0a922e945 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/562176 Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-09-28 22:43:15 +03:00
moduleDir: "http",
commit: "http/v1.2.3",
templates: githubURLTemplates,
},
},
{
"alice.org/pkg/source",
// Has a go-source tag, but we can't use the templates.
&Info{
internal/source,postgres: save source info to the DB - Add a SourceInfo field to VersionInfo. - Add functions to convert a source.Info to and from JSON. - Modify postgres statements to read and write from the DB. Keep the VersionInfo.RepositoryURL field around for now to avoid messing with the frontend. Updates b/132219298, b/138752200. Change-Id: I6ebb3af3afa14fda92c2804ab2d0294742a3bf41 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/575156 CI-Result: Cloud Build <devtools-proctor-result-processor@system.gserviceaccount.com> Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-10-15 13:00:58 +03:00
repoURL: "http://alice.org/pkg",
internal/source: handle meta tags Use the go command's meta-tag mechanism to find source code for vanity module paths. If a module path doesn't match one of our built-in patterns, make a GET request to the path and parse the go-import and go-source meta tags on the page to learn where the source lives and how to link to it. This process is slightly different than godoc.org's (as implemented in go.googlesource.com/gddo): - godoc.org uses package import paths; we use module paths instead. That can result in different behavior when a module doesn't correspond to a package and no tags are served at the module path URL, even though its package URLs do serve tags. The discovery site will not show source links for those packages. This is a genuine breaking behavior change, but we're going to make it because serving tags at the module level is the right thing to do, and we can easily fix the handful of modules where this is a problem. - godoc.org needs to fetch the code, so it needs information from the tags like the vcs, and it needs to keep the go-import and go-source tags separate (the first being for fetching, the second for linking). Since the discovery site already has the code from the zip file, we just need go-source information, using go-imports as a fallback. - The go-source tag contents don't provide a way to insert a version, so they're not usable as-is for linking to versioned code. For now, we ignore the URL templates in the tag and just try to match the repo. So a vanity URL that redirects to GitHub will work fine, since we know how to link to files in GitHub; but one that redirects to some random code-hosting site will not have source links on the discovery site, even though it might on godoc.org. Fixes b/141771975. Change-Id: Ie07bff3a77350cdaa7e45af0c7e5bce0a922e945 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/562176 Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-09-28 22:43:15 +03:00
moduleDir: "source",
commit: "source/v1.2.3",
// empty templates
},
},
{
"alice.org/pkg/ignore",
// Stop at the first go-source.
&Info{
internal/source,postgres: save source info to the DB - Add a SourceInfo field to VersionInfo. - Add functions to convert a source.Info to and from JSON. - Modify postgres statements to read and write from the DB. Keep the VersionInfo.RepositoryURL field around for now to avoid messing with the frontend. Updates b/132219298, b/138752200. Change-Id: I6ebb3af3afa14fda92c2804ab2d0294742a3bf41 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/575156 CI-Result: Cloud Build <devtools-proctor-result-processor@system.gserviceaccount.com> Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-10-15 13:00:58 +03:00
repoURL: "http://alice.org/pkg",
internal/source: handle meta tags Use the go command's meta-tag mechanism to find source code for vanity module paths. If a module path doesn't match one of our built-in patterns, make a GET request to the path and parse the go-import and go-source meta tags on the page to learn where the source lives and how to link to it. This process is slightly different than godoc.org's (as implemented in go.googlesource.com/gddo): - godoc.org uses package import paths; we use module paths instead. That can result in different behavior when a module doesn't correspond to a package and no tags are served at the module path URL, even though its package URLs do serve tags. The discovery site will not show source links for those packages. This is a genuine breaking behavior change, but we're going to make it because serving tags at the module level is the right thing to do, and we can easily fix the handful of modules where this is a problem. - godoc.org needs to fetch the code, so it needs information from the tags like the vcs, and it needs to keep the go-import and go-source tags separate (the first being for fetching, the second for linking). Since the discovery site already has the code from the zip file, we just need go-source information, using go-imports as a fallback. - The go-source tag contents don't provide a way to insert a version, so they're not usable as-is for linking to versioned code. For now, we ignore the URL templates in the tag and just try to match the repo. So a vanity URL that redirects to GitHub will work fine, since we know how to link to files in GitHub; but one that redirects to some random code-hosting site will not have source links on the discovery site, even though it might on godoc.org. Fixes b/141771975. Change-Id: Ie07bff3a77350cdaa7e45af0c7e5bce0a922e945 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/562176 Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-09-28 22:43:15 +03:00
moduleDir: "ignore",
commit: "ignore/v1.2.3",
// empty templates
},
},
{"alice.org/pkg/multiple", nil},
{"alice.org/pkg/notfound", nil},
{
"bob.com/pkg",
&Info{
// The go-import tag's repo root ends in ".git", but according to the spec
// there should not be a .vcs suffix, so we include the ".git" in the repo URL.
internal/source,postgres: save source info to the DB - Add a SourceInfo field to VersionInfo. - Add functions to convert a source.Info to and from JSON. - Modify postgres statements to read and write from the DB. Keep the VersionInfo.RepositoryURL field around for now to avoid messing with the frontend. Updates b/132219298, b/138752200. Change-Id: I6ebb3af3afa14fda92c2804ab2d0294742a3bf41 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/575156 CI-Result: Cloud Build <devtools-proctor-result-processor@system.gserviceaccount.com> Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-10-15 13:00:58 +03:00
repoURL: "https://vcs.net/bob/pkg.git",
internal/source: handle meta tags Use the go command's meta-tag mechanism to find source code for vanity module paths. If a module path doesn't match one of our built-in patterns, make a GET request to the path and parse the go-import and go-source meta tags on the page to learn where the source lives and how to link to it. This process is slightly different than godoc.org's (as implemented in go.googlesource.com/gddo): - godoc.org uses package import paths; we use module paths instead. That can result in different behavior when a module doesn't correspond to a package and no tags are served at the module path URL, even though its package URLs do serve tags. The discovery site will not show source links for those packages. This is a genuine breaking behavior change, but we're going to make it because serving tags at the module level is the right thing to do, and we can easily fix the handful of modules where this is a problem. - godoc.org needs to fetch the code, so it needs information from the tags like the vcs, and it needs to keep the go-import and go-source tags separate (the first being for fetching, the second for linking). Since the discovery site already has the code from the zip file, we just need go-source information, using go-imports as a fallback. - The go-source tag contents don't provide a way to insert a version, so they're not usable as-is for linking to versioned code. For now, we ignore the URL templates in the tag and just try to match the repo. So a vanity URL that redirects to GitHub will work fine, since we know how to link to files in GitHub; but one that redirects to some random code-hosting site will not have source links on the discovery site, even though it might on godoc.org. Fixes b/141771975. Change-Id: Ie07bff3a77350cdaa7e45af0c7e5bce0a922e945 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/562176 Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-09-28 22:43:15 +03:00
moduleDir: "",
commit: "v1.2.3",
// empty templates
},
},
{
"bob.com/pkg/sub",
&Info{
internal/source,postgres: save source info to the DB - Add a SourceInfo field to VersionInfo. - Add functions to convert a source.Info to and from JSON. - Modify postgres statements to read and write from the DB. Keep the VersionInfo.RepositoryURL field around for now to avoid messing with the frontend. Updates b/132219298, b/138752200. Change-Id: I6ebb3af3afa14fda92c2804ab2d0294742a3bf41 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/575156 CI-Result: Cloud Build <devtools-proctor-result-processor@system.gserviceaccount.com> Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-10-15 13:00:58 +03:00
repoURL: "https://vcs.net/bob/pkg.git",
internal/source: handle meta tags Use the go command's meta-tag mechanism to find source code for vanity module paths. If a module path doesn't match one of our built-in patterns, make a GET request to the path and parse the go-import and go-source meta tags on the page to learn where the source lives and how to link to it. This process is slightly different than godoc.org's (as implemented in go.googlesource.com/gddo): - godoc.org uses package import paths; we use module paths instead. That can result in different behavior when a module doesn't correspond to a package and no tags are served at the module path URL, even though its package URLs do serve tags. The discovery site will not show source links for those packages. This is a genuine breaking behavior change, but we're going to make it because serving tags at the module level is the right thing to do, and we can easily fix the handful of modules where this is a problem. - godoc.org needs to fetch the code, so it needs information from the tags like the vcs, and it needs to keep the go-import and go-source tags separate (the first being for fetching, the second for linking). Since the discovery site already has the code from the zip file, we just need go-source information, using go-imports as a fallback. - The go-source tag contents don't provide a way to insert a version, so they're not usable as-is for linking to versioned code. For now, we ignore the URL templates in the tag and just try to match the repo. So a vanity URL that redirects to GitHub will work fine, since we know how to link to files in GitHub; but one that redirects to some random code-hosting site will not have source links on the discovery site, even though it might on godoc.org. Fixes b/141771975. Change-Id: Ie07bff3a77350cdaa7e45af0c7e5bce0a922e945 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/562176 Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-09-28 22:43:15 +03:00
moduleDir: "sub",
commit: "sub/v1.2.3",
// empty templates
},
},
{
"azul3d.org/examples/abs",
// The go-source tag has a template that is handled incorrectly by godoc; but we
// ignore the templates.
&Info{
internal/source,postgres: save source info to the DB - Add a SourceInfo field to VersionInfo. - Add functions to convert a source.Info to and from JSON. - Modify postgres statements to read and write from the DB. Keep the VersionInfo.RepositoryURL field around for now to avoid messing with the frontend. Updates b/132219298, b/138752200. Change-Id: I6ebb3af3afa14fda92c2804ab2d0294742a3bf41 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/575156 CI-Result: Cloud Build <devtools-proctor-result-processor@system.gserviceaccount.com> Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-10-15 13:00:58 +03:00
repoURL: "https://github.com/azul3d/examples",
internal/source: handle meta tags Use the go command's meta-tag mechanism to find source code for vanity module paths. If a module path doesn't match one of our built-in patterns, make a GET request to the path and parse the go-import and go-source meta tags on the page to learn where the source lives and how to link to it. This process is slightly different than godoc.org's (as implemented in go.googlesource.com/gddo): - godoc.org uses package import paths; we use module paths instead. That can result in different behavior when a module doesn't correspond to a package and no tags are served at the module path URL, even though its package URLs do serve tags. The discovery site will not show source links for those packages. This is a genuine breaking behavior change, but we're going to make it because serving tags at the module level is the right thing to do, and we can easily fix the handful of modules where this is a problem. - godoc.org needs to fetch the code, so it needs information from the tags like the vcs, and it needs to keep the go-import and go-source tags separate (the first being for fetching, the second for linking). Since the discovery site already has the code from the zip file, we just need go-source information, using go-imports as a fallback. - The go-source tag contents don't provide a way to insert a version, so they're not usable as-is for linking to versioned code. For now, we ignore the URL templates in the tag and just try to match the repo. So a vanity URL that redirects to GitHub will work fine, since we know how to link to files in GitHub; but one that redirects to some random code-hosting site will not have source links on the discovery site, even though it might on godoc.org. Fixes b/141771975. Change-Id: Ie07bff3a77350cdaa7e45af0c7e5bce0a922e945 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/562176 Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-09-28 22:43:15 +03:00
moduleDir: "abs",
commit: "abs/v1.2.3",
templates: githubURLTemplates,
},
},
{
"myitcv.io/blah2",
// Ignore the "mod" vcs type.
&Info{
internal/source,postgres: save source info to the DB - Add a SourceInfo field to VersionInfo. - Add functions to convert a source.Info to and from JSON. - Modify postgres statements to read and write from the DB. Keep the VersionInfo.RepositoryURL field around for now to avoid messing with the frontend. Updates b/132219298, b/138752200. Change-Id: I6ebb3af3afa14fda92c2804ab2d0294742a3bf41 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/575156 CI-Result: Cloud Build <devtools-proctor-result-processor@system.gserviceaccount.com> Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-10-15 13:00:58 +03:00
repoURL: "https://github.com/myitcv/x",
internal/source: handle meta tags Use the go command's meta-tag mechanism to find source code for vanity module paths. If a module path doesn't match one of our built-in patterns, make a GET request to the path and parse the go-import and go-source meta tags on the page to learn where the source lives and how to link to it. This process is slightly different than godoc.org's (as implemented in go.googlesource.com/gddo): - godoc.org uses package import paths; we use module paths instead. That can result in different behavior when a module doesn't correspond to a package and no tags are served at the module path URL, even though its package URLs do serve tags. The discovery site will not show source links for those packages. This is a genuine breaking behavior change, but we're going to make it because serving tags at the module level is the right thing to do, and we can easily fix the handful of modules where this is a problem. - godoc.org needs to fetch the code, so it needs information from the tags like the vcs, and it needs to keep the go-import and go-source tags separate (the first being for fetching, the second for linking). Since the discovery site already has the code from the zip file, we just need go-source information, using go-imports as a fallback. - The go-source tag contents don't provide a way to insert a version, so they're not usable as-is for linking to versioned code. For now, we ignore the URL templates in the tag and just try to match the repo. So a vanity URL that redirects to GitHub will work fine, since we know how to link to files in GitHub; but one that redirects to some random code-hosting site will not have source links on the discovery site, even though it might on godoc.org. Fixes b/141771975. Change-Id: Ie07bff3a77350cdaa7e45af0c7e5bce0a922e945 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/562176 Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-09-28 22:43:15 +03:00
moduleDir: "",
commit: "v1.2.3",
templates: githubURLTemplates,
},
},
{
"alice.org/pkg/default",
&Info{
internal/source,postgres: save source info to the DB - Add a SourceInfo field to VersionInfo. - Add functions to convert a source.Info to and from JSON. - Modify postgres statements to read and write from the DB. Keep the VersionInfo.RepositoryURL field around for now to avoid messing with the frontend. Updates b/132219298, b/138752200. Change-Id: I6ebb3af3afa14fda92c2804ab2d0294742a3bf41 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/575156 CI-Result: Cloud Build <devtools-proctor-result-processor@system.gserviceaccount.com> Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-10-15 13:00:58 +03:00
repoURL: "https://github.com/alice/pkg",
internal/source: handle meta tags Use the go command's meta-tag mechanism to find source code for vanity module paths. If a module path doesn't match one of our built-in patterns, make a GET request to the path and parse the go-import and go-source meta tags on the page to learn where the source lives and how to link to it. This process is slightly different than godoc.org's (as implemented in go.googlesource.com/gddo): - godoc.org uses package import paths; we use module paths instead. That can result in different behavior when a module doesn't correspond to a package and no tags are served at the module path URL, even though its package URLs do serve tags. The discovery site will not show source links for those packages. This is a genuine breaking behavior change, but we're going to make it because serving tags at the module level is the right thing to do, and we can easily fix the handful of modules where this is a problem. - godoc.org needs to fetch the code, so it needs information from the tags like the vcs, and it needs to keep the go-import and go-source tags separate (the first being for fetching, the second for linking). Since the discovery site already has the code from the zip file, we just need go-source information, using go-imports as a fallback. - The go-source tag contents don't provide a way to insert a version, so they're not usable as-is for linking to versioned code. For now, we ignore the URL templates in the tag and just try to match the repo. So a vanity URL that redirects to GitHub will work fine, since we know how to link to files in GitHub; but one that redirects to some random code-hosting site will not have source links on the discovery site, even though it might on godoc.org. Fixes b/141771975. Change-Id: Ie07bff3a77350cdaa7e45af0c7e5bce0a922e945 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/562176 Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-09-28 22:43:15 +03:00
moduleDir: "default",
commit: "default/v1.2.3",
templates: githubURLTemplates,
},
},
internal/{source,frontend}: escape HTML Escape any strings that may come from outside the program and will be rendered as HTML. Also, fix some typos. Also, put args to a cmp.Diff call in the right order. Change-Id: I1fc6013e200326af8719b14c28a607fc7936e20a Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/239478 Reviewed-by: Julie Qiu <julie@golang.org>
2020-06-18 03:22:39 +03:00
{
internal/source: remove HTML escaping Now that the entire server uses safehtml, it is no longer necessary to do anything special with the user-provided values in this package. Every use of the Info.XXXURL methods is part of rendering a safe template, so all the escaping will happen then. Change-Id: I5e64b08d4c7b8e7fd56cf1a0bed49691f2a8ee82 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/242917 Reviewed-by: Julie Qiu <julie@golang.org>
2020-07-16 04:39:34 +03:00
// Bad repo URLs. These are not escaped here, but they are whenever we render a template.
internal/{source,frontend}: escape HTML Escape any strings that may come from outside the program and will be rendered as HTML. Also, fix some typos. Also, put args to a cmp.Diff call in the right order. Change-Id: I1fc6013e200326af8719b14c28a607fc7936e20a Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/239478 Reviewed-by: Julie Qiu <julie@golang.org>
2020-06-18 03:22:39 +03:00
"bob.com/bad/github",
&Info{
internal/source: remove HTML escaping Now that the entire server uses safehtml, it is no longer necessary to do anything special with the user-provided values in this package. Every use of the Info.XXXURL methods is part of rendering a safe template, so all the escaping will happen then. Change-Id: I5e64b08d4c7b8e7fd56cf1a0bed49691f2a8ee82 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/242917 Reviewed-by: Julie Qiu <julie@golang.org>
2020-07-16 04:39:34 +03:00
repoURL: `https://github.com/bob/bad/">$`,
internal/{source,frontend}: escape HTML Escape any strings that may come from outside the program and will be rendered as HTML. Also, fix some typos. Also, put args to a cmp.Diff call in the right order. Change-Id: I1fc6013e200326af8719b14c28a607fc7936e20a Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/239478 Reviewed-by: Julie Qiu <julie@golang.org>
2020-06-18 03:22:39 +03:00
moduleDir: "",
commit: "v1.2.3",
templates: githubURLTemplates,
},
},
{
"bob.com/bad/apache",
&Info{
internal/source: remove HTML escaping Now that the entire server uses safehtml, it is no longer necessary to do anything special with the user-provided values in this package. Every use of the Info.XXXURL methods is part of rendering a safe template, so all the escaping will happen then. Change-Id: I5e64b08d4c7b8e7fd56cf1a0bed49691f2a8ee82 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/242917 Reviewed-by: Julie Qiu <julie@golang.org>
2020-07-16 04:39:34 +03:00
repoURL: "https://git.apache.org/>$",
internal/{source,frontend}: escape HTML Escape any strings that may come from outside the program and will be rendered as HTML. Also, fix some typos. Also, put args to a cmp.Diff call in the right order. Change-Id: I1fc6013e200326af8719b14c28a607fc7936e20a Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/239478 Reviewed-by: Julie Qiu <julie@golang.org>
2020-06-18 03:22:39 +03:00
moduleDir: "",
commit: "v1.2.3",
templates: githubURLTemplates,
},
},
internal/source: handle meta tags Use the go command's meta-tag mechanism to find source code for vanity module paths. If a module path doesn't match one of our built-in patterns, make a GET request to the path and parse the go-import and go-source meta tags on the page to learn where the source lives and how to link to it. This process is slightly different than godoc.org's (as implemented in go.googlesource.com/gddo): - godoc.org uses package import paths; we use module paths instead. That can result in different behavior when a module doesn't correspond to a package and no tags are served at the module path URL, even though its package URLs do serve tags. The discovery site will not show source links for those packages. This is a genuine breaking behavior change, but we're going to make it because serving tags at the module level is the right thing to do, and we can easily fix the handful of modules where this is a problem. - godoc.org needs to fetch the code, so it needs information from the tags like the vcs, and it needs to keep the go-import and go-source tags separate (the first being for fetching, the second for linking). Since the discovery site already has the code from the zip file, we just need go-source information, using go-imports as a fallback. - The go-source tag contents don't provide a way to insert a version, so they're not usable as-is for linking to versioned code. For now, we ignore the URL templates in the tag and just try to match the repo. So a vanity URL that redirects to GitHub will work fine, since we know how to link to files in GitHub; but one that redirects to some random code-hosting site will not have source links on the discovery site, even though it might on godoc.org. Fixes b/141771975. Change-Id: Ie07bff3a77350cdaa7e45af0c7e5bce0a922e945 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/562176 Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-09-28 22:43:15 +03:00
} {
t.Run(test.modulePath, func(t *testing.T) {
got, err := moduleInfoDynamic(context.Background(), client, test.modulePath, version)
if err != nil {
if test.want == nil {
return
}
internal/source: a package for references to source Add internal/source, which will know how to generate URLs that refer to the source code or other files inside modules. This package will be used to link to module and package source code, and also for relative links in README files. For better testing, introduce the httpreplay package. With -record, it makes HTTP requests to the actual servers and records the responses. With no flags, it replays the recordings. Fixes b/141771930. Updates b/140431331. Change-Id: I72dcd5535b8be7c6b27dd89245a048d23fd14160 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/561217 CI-Result: Cloud Build <devtools-proctor-result-processor@system.gserviceaccount.com> Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-09-27 23:26:07 +03:00
t.Fatal(err)
}
internal/{source,frontend}: escape HTML Escape any strings that may come from outside the program and will be rendered as HTML. Also, fix some typos. Also, put args to a cmp.Diff call in the right order. Change-Id: I1fc6013e200326af8719b14c28a607fc7936e20a Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/239478 Reviewed-by: Julie Qiu <julie@golang.org>
2020-06-18 03:22:39 +03:00
if diff := cmp.Diff(test.want, got, cmp.AllowUnexported(Info{}, urlTemplates{})); diff != "" {
internal/source: handle meta tags Use the go command's meta-tag mechanism to find source code for vanity module paths. If a module path doesn't match one of our built-in patterns, make a GET request to the path and parse the go-import and go-source meta tags on the page to learn where the source lives and how to link to it. This process is slightly different than godoc.org's (as implemented in go.googlesource.com/gddo): - godoc.org uses package import paths; we use module paths instead. That can result in different behavior when a module doesn't correspond to a package and no tags are served at the module path URL, even though its package URLs do serve tags. The discovery site will not show source links for those packages. This is a genuine breaking behavior change, but we're going to make it because serving tags at the module level is the right thing to do, and we can easily fix the handful of modules where this is a problem. - godoc.org needs to fetch the code, so it needs information from the tags like the vcs, and it needs to keep the go-import and go-source tags separate (the first being for fetching, the second for linking). Since the discovery site already has the code from the zip file, we just need go-source information, using go-imports as a fallback. - The go-source tag contents don't provide a way to insert a version, so they're not usable as-is for linking to versioned code. For now, we ignore the URL templates in the tag and just try to match the repo. So a vanity URL that redirects to GitHub will work fine, since we know how to link to files in GitHub; but one that redirects to some random code-hosting site will not have source links on the discovery site, even though it might on godoc.org. Fixes b/141771975. Change-Id: Ie07bff3a77350cdaa7e45af0c7e5bce0a922e945 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/562176 Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-09-28 22:43:15 +03:00
t.Errorf("mismatch (-want +got):\n%s", diff)
}
})
}
}
internal/source: handle versions v2 and higher - Remove "vN" when constructing the module directory. - Distinguish subdirectory and branch cases by looking for a go.mod file. Fixes b/141769404. Change-Id: I20328ace5257c2102c0248e6f92bdce2debe94e5 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/562701 Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-09-30 20:36:59 +03:00
func TestRemoveVersionSuffix(t *testing.T) {
for _, test := range []struct {
in string
want string
}{
{"", ""},
{"v1", "v1"},
{"v2", ""},
{"v17", ""},
{"foo/bar", "foo/bar"},
{"foo/bar/v1", "foo/bar/v1"},
{"foo/bar.v2", "foo/bar.v2"},
{"foo/bar/v2", "foo/bar"},
{"foo/bar/v17", "foo/bar"},
} {
got := removeVersionSuffix(test.in)
if got != test.want {
t.Errorf("%q: got %q, want %q", test.in, got, test.want)
}
internal/source: handle meta tags Use the go command's meta-tag mechanism to find source code for vanity module paths. If a module path doesn't match one of our built-in patterns, make a GET request to the path and parse the go-import and go-source meta tags on the page to learn where the source lives and how to link to it. This process is slightly different than godoc.org's (as implemented in go.googlesource.com/gddo): - godoc.org uses package import paths; we use module paths instead. That can result in different behavior when a module doesn't correspond to a package and no tags are served at the module path URL, even though its package URLs do serve tags. The discovery site will not show source links for those packages. This is a genuine breaking behavior change, but we're going to make it because serving tags at the module level is the right thing to do, and we can easily fix the handful of modules where this is a problem. - godoc.org needs to fetch the code, so it needs information from the tags like the vcs, and it needs to keep the go-import and go-source tags separate (the first being for fetching, the second for linking). Since the discovery site already has the code from the zip file, we just need go-source information, using go-imports as a fallback. - The go-source tag contents don't provide a way to insert a version, so they're not usable as-is for linking to versioned code. For now, we ignore the URL templates in the tag and just try to match the repo. So a vanity URL that redirects to GitHub will work fine, since we know how to link to files in GitHub; but one that redirects to some random code-hosting site will not have source links on the discovery site, even though it might on godoc.org. Fixes b/141771975. Change-Id: Ie07bff3a77350cdaa7e45af0c7e5bce0a922e945 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/562176 Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-09-28 22:43:15 +03:00
}
internal/source: handle versions v2 and higher - Remove "vN" when constructing the module directory. - Distinguish subdirectory and branch cases by looking for a go.mod file. Fixes b/141769404. Change-Id: I20328ace5257c2102c0248e6f92bdce2debe94e5 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/562701 Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-09-30 20:36:59 +03:00
}
func TestAdjustVersionedModuleDirectory(t *testing.T) {
ctx := context.Background()
internal/source: add Client source.Client is added, which replaces the used of http.DefaultClient when fetching source code. This also allows us to set a custom timeout for tests vs when running the worker. Change-Id: I5b7b0fd32fa7a2cf836b951af1ad471751fade00 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/709198 Reviewed-by: Jonathan Amsterdam <jba@google.com>
2020-04-03 20:15:18 +03:00
client := NewClient(testTimeout)
client.httpClient.Transport = testTransport(map[string]string{
internal/source: handle versions v2 and higher - Remove "vN" when constructing the module directory. - Distinguish subdirectory and branch cases by looking for a go.mod file. Fixes b/141769404. Change-Id: I20328ace5257c2102c0248e6f92bdce2debe94e5 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/562701 Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-09-30 20:36:59 +03:00
// Repo "branch" follows the "major branch" convention: versions 2 and higher
// live in the same directory as versions 0 and 1, but on a different branch (or tag).
"http://x.com/branch/v1.0.0/go.mod": "", // v1 module at the root
"http://x.com/branch/v2.0.0/go.mod": "", // v2 module at the root
"http://x.com/branch/dir/v1.0.0/dir/go.mod": "", // v1 module in a subdirectory
"http://x.com/branch/dir/v2.0.0/dir/go.mod": "", // v2 module in a subdirectory
// Repo "sub" follows the "major subdirectory" convention: versions 2 and higher
// live in a "vN" subdirectory.
"http://x.com/sub/v1.0.0/go.mod": "", // v1 module at the root
"http://x.com/sub/v2.0.0/v2/go.mod": "", // v2 module at root/v2.
"http://x.com/sub/dir/v1.0.0/dir/go.mod": "", // v1 module in a subdirectory
"http://x.com/sub/dir/v2.0.0/dir/v2/go.mod": "", // v2 module in subdirectory/v2
internal/source: add Client source.Client is added, which replaces the used of http.DefaultClient when fetching source code. This also allows us to set a custom timeout for tests vs when running the worker. Change-Id: I5b7b0fd32fa7a2cf836b951af1ad471751fade00 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/709198 Reviewed-by: Jonathan Amsterdam <jba@google.com>
2020-04-03 20:15:18 +03:00
})
internal/source: handle versions v2 and higher - Remove "vN" when constructing the module directory. - Distinguish subdirectory and branch cases by looking for a go.mod file. Fixes b/141769404. Change-Id: I20328ace5257c2102c0248e6f92bdce2debe94e5 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/562701 Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-09-30 20:36:59 +03:00
for _, test := range []struct {
repo, moduleDir, commit string
want string
}{
{
// module path is x.com/branch
"branch", "", "v1.0.0",
"",
},
{
// module path is x.com/branch/v2; remove the "v2" to get the module dir
"branch", "v2", "v2.0.0",
"",
},
{
// module path is x.com/branch/dir
"branch", "dir", "dir/v1.0.0",
"dir",
},
{
// module path is x.com/branch/dir/v2; remove the v2 to get the module dir
"branch", "dir/v2", "dir/v2.0.0",
"dir",
},
{
// module path is x.com/sub
"sub", "", "v1.0.0",
"",
},
{
// module path is x.com/sub/v2; do not remove the v2
"sub", "v2", "v2.0.0",
"v2",
},
{
// module path is x.com/sub/dir
"sub", "dir", "dir/v1.0.0",
"dir",
},
{
// module path is x.com/sub/dir/v2; do not remove the v2
"sub", "dir/v2", "dir/v2.0.0",
"dir/v2",
},
} {
t.Run(test.repo+","+test.moduleDir+","+test.commit, func(t *testing.T) {
info := &Info{
internal/source,postgres: save source info to the DB - Add a SourceInfo field to VersionInfo. - Add functions to convert a source.Info to and from JSON. - Modify postgres statements to read and write from the DB. Keep the VersionInfo.RepositoryURL field around for now to avoid messing with the frontend. Updates b/132219298, b/138752200. Change-Id: I6ebb3af3afa14fda92c2804ab2d0294742a3bf41 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/575156 CI-Result: Cloud Build <devtools-proctor-result-processor@system.gserviceaccount.com> Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-10-15 13:00:58 +03:00
repoURL: "http://x.com/" + test.repo,
internal/source: handle versions v2 and higher - Remove "vN" when constructing the module directory. - Distinguish subdirectory and branch cases by looking for a go.mod file. Fixes b/141769404. Change-Id: I20328ace5257c2102c0248e6f92bdce2debe94e5 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/562701 Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-09-30 20:36:59 +03:00
moduleDir: test.moduleDir,
commit: test.commit,
internal/source,postgres: save source info to the DB - Add a SourceInfo field to VersionInfo. - Add functions to convert a source.Info to and from JSON. - Modify postgres statements to read and write from the DB. Keep the VersionInfo.RepositoryURL field around for now to avoid messing with the frontend. Updates b/132219298, b/138752200. Change-Id: I6ebb3af3afa14fda92c2804ab2d0294742a3bf41 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/575156 CI-Result: Cloud Build <devtools-proctor-result-processor@system.gserviceaccount.com> Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-10-15 13:00:58 +03:00
templates: urlTemplates{File: "{repo}/{commit}/{file}"},
internal/source: handle versions v2 and higher - Remove "vN" when constructing the module directory. - Distinguish subdirectory and branch cases by looking for a go.mod file. Fixes b/141769404. Change-Id: I20328ace5257c2102c0248e6f92bdce2debe94e5 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/562701 Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-09-30 20:36:59 +03:00
}
adjustVersionedModuleDirectory(ctx, client, info)
got := info.moduleDir
if got != test.want {
t.Errorf("got %q, want %q", got, test.want)
}
})
internal/source: a package for references to source Add internal/source, which will know how to generate URLs that refer to the source code or other files inside modules. This package will be used to link to module and package source code, and also for relative links in README files. For better testing, introduce the httpreplay package. With -record, it makes HTTP requests to the actual servers and records the responses. With no flags, it replays the recordings. Fixes b/141771930. Updates b/140431331. Change-Id: I72dcd5535b8be7c6b27dd89245a048d23fd14160 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/561217 CI-Result: Cloud Build <devtools-proctor-result-processor@system.gserviceaccount.com> Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-09-27 23:26:07 +03:00
}
}
func TestCommitFromVersion(t *testing.T) {
for _, test := range []struct {
version, dir string
want string
}{
{
"v1.2.3", "",
"v1.2.3",
},
{
"v1.2.3", "foo",
"foo/v1.2.3",
},
internal/source: handle versions v2 and higher - Remove "vN" when constructing the module directory. - Distinguish subdirectory and branch cases by looking for a go.mod file. Fixes b/141769404. Change-Id: I20328ace5257c2102c0248e6f92bdce2debe94e5 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/562701 Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-09-30 20:36:59 +03:00
{
"v1.2.3", "foo/v1",
"foo/v1/v1.2.3", // don't remove "/vN" if N = 1
},
{
"v1.2.3", "v1", // ditto
"v1/v1.2.3",
},
{
"v3.1.0", "foo/v3",
"foo/v3.1.0", // do remove "/v2" and higher
},
{
"v3.1.0", "v3",
"v3.1.0", // ditto
},
internal/source: a package for references to source Add internal/source, which will know how to generate URLs that refer to the source code or other files inside modules. This package will be used to link to module and package source code, and also for relative links in README files. For better testing, introduce the httpreplay package. With -record, it makes HTTP requests to the actual servers and records the responses. With no flags, it replays the recordings. Fixes b/141771930. Updates b/140431331. Change-Id: I72dcd5535b8be7c6b27dd89245a048d23fd14160 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/561217 CI-Result: Cloud Build <devtools-proctor-result-processor@system.gserviceaccount.com> Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-09-27 23:26:07 +03:00
{
"v6.1.1-0.20190615154606-3a9541ec9974", "",
"3a9541ec9974",
},
{
"v6.1.1-0.20190615154606-3a9541ec9974", "foo",
"3a9541ec9974",
},
} {
t.Run(fmt.Sprintf("%s,%s", test.version, test.dir), func(t *testing.T) {
if got := commitFromVersion(test.version, test.dir); got != test.want {
t.Errorf("got %s, want %s", got, test.want)
}
// Adding "+incompatible" shouldn't make a difference.
if got := commitFromVersion(test.version+"+incompatible", test.dir); got != test.want {
t.Errorf("+incompatible: got %s, want %s", got, test.want)
}
})
}
}
internal/source: handle meta tags Use the go command's meta-tag mechanism to find source code for vanity module paths. If a module path doesn't match one of our built-in patterns, make a GET request to the path and parse the go-import and go-source meta tags on the page to learn where the source lives and how to link to it. This process is slightly different than godoc.org's (as implemented in go.googlesource.com/gddo): - godoc.org uses package import paths; we use module paths instead. That can result in different behavior when a module doesn't correspond to a package and no tags are served at the module path URL, even though its package URLs do serve tags. The discovery site will not show source links for those packages. This is a genuine breaking behavior change, but we're going to make it because serving tags at the module level is the right thing to do, and we can easily fix the handful of modules where this is a problem. - godoc.org needs to fetch the code, so it needs information from the tags like the vcs, and it needs to keep the go-import and go-source tags separate (the first being for fetching, the second for linking). Since the discovery site already has the code from the zip file, we just need go-source information, using go-imports as a fallback. - The go-source tag contents don't provide a way to insert a version, so they're not usable as-is for linking to versioned code. For now, we ignore the URL templates in the tag and just try to match the repo. So a vanity URL that redirects to GitHub will work fine, since we know how to link to files in GitHub; but one that redirects to some random code-hosting site will not have source links on the discovery site, even though it might on godoc.org. Fixes b/141771975. Change-Id: Ie07bff3a77350cdaa7e45af0c7e5bce0a922e945 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/562176 Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-09-28 22:43:15 +03:00
internal/source: handle versions v2 and higher - Remove "vN" when constructing the module directory. - Distinguish subdirectory and branch cases by looking for a go.mod file. Fixes b/141769404. Change-Id: I20328ace5257c2102c0248e6f92bdce2debe94e5 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/562701 Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-09-30 20:36:59 +03:00
type testTransport map[string]string
func (t testTransport) RoundTrip(req *http.Request) (*http.Response, error) {
statusCode := http.StatusOK
req.URL.RawQuery = ""
body, ok := t[req.URL.String()]
if !ok {
statusCode = http.StatusNotFound
}
resp := &http.Response{
StatusCode: statusCode,
Body: ioutil.NopCloser(strings.NewReader(body)),
}
return resp, nil
}
internal/source: handle meta tags Use the go command's meta-tag mechanism to find source code for vanity module paths. If a module path doesn't match one of our built-in patterns, make a GET request to the path and parse the go-import and go-source meta tags on the page to learn where the source lives and how to link to it. This process is slightly different than godoc.org's (as implemented in go.googlesource.com/gddo): - godoc.org uses package import paths; we use module paths instead. That can result in different behavior when a module doesn't correspond to a package and no tags are served at the module path URL, even though its package URLs do serve tags. The discovery site will not show source links for those packages. This is a genuine breaking behavior change, but we're going to make it because serving tags at the module level is the right thing to do, and we can easily fix the handful of modules where this is a problem. - godoc.org needs to fetch the code, so it needs information from the tags like the vcs, and it needs to keep the go-import and go-source tags separate (the first being for fetching, the second for linking). Since the discovery site already has the code from the zip file, we just need go-source information, using go-imports as a fallback. - The go-source tag contents don't provide a way to insert a version, so they're not usable as-is for linking to versioned code. For now, we ignore the URL templates in the tag and just try to match the repo. So a vanity URL that redirects to GitHub will work fine, since we know how to link to files in GitHub; but one that redirects to some random code-hosting site will not have source links on the discovery site, even though it might on godoc.org. Fixes b/141771975. Change-Id: Ie07bff3a77350cdaa7e45af0c7e5bce0a922e945 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/562176 Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-09-28 22:43:15 +03:00
var testWeb = map[string]string{
// Package at root of a GitHub repo.
"https://alice.org/pkg": `<head> <meta name="go-import" content="alice.org/pkg git https://github.com/alice/pkg"></head>`,
// Package in sub-directory.
"https://alice.org/pkg/sub": `<head> <meta name="go-import" content="alice.org/pkg git https://github.com/alice/pkg"><body>`,
// Fallback to http.
"http://alice.org/pkg/http": `<head> <meta name="go-import" content="alice.org/pkg git https://github.com/alice/pkg">`,
// Meta tag in sub-directory does not match meta tag at root.
"https://alice.org/pkg/mismatch": `<head> <meta name="go-import" content="alice.org/pkg hg https://github.com/alice/pkg">`,
// More than one matching meta tag.
"http://alice.org/pkg/multiple": `<head> ` +
`<meta name="go-import" content="alice.org/pkg git https://github.com/alice/pkg">` +
`<meta name="go-import" content="alice.org/pkg git https://github.com/alice/pkg">`,
// Package with go-source meta tag.
"https://alice.org/pkg/source": `<head>` +
`<meta name="go-import" content="alice.org/pkg git https://github.com/alice/pkg">` +
`<meta name="go-source" content="alice.org/pkg http://alice.org/pkg http://alice.org/pkg{/dir} http://alice.org/pkg{/dir}?f={file}#Line{line}">`,
"https://alice.org/pkg/ignore": `<head>` +
`<title>Hello</title>` +
// Unknown meta name
`<meta name="go-junk" content="alice.org/pkg http://alice.org/pkg http://alice.org/pkg{/dir} http://alice.org/pkg{/dir}?f={file}#Line{line}">` +
// go-source before go-meta
`<meta name="go-source" content="alice.org/pkg http://alice.org/pkg http://alice.org/pkg{/dir} http://alice.org/pkg{/dir}?f={file}#Line{line}">` +
// go-import tag for the package
`<meta name="go-import" content="alice.org/pkg git https://github.com/alice/pkg">` +
// go-import with wrong number of fields
`<meta name="go-import" content="alice.org/pkg https://github.com/alice/pkg">` +
// go-import with no fields
`<meta name="go-import" content="">` +
// go-source with wrong number of fields
`<meta name="go-source" content="alice.org/pkg blah">` +
// meta tag for a different package
`<meta name="go-import" content="alice.org/other git https://github.com/alice/other">` +
// meta tag for a different package
`<meta name="go-import" content="alice.org/other git https://github.com/alice/other">` +
`</head>` +
// go-import outside of head
`<meta name="go-import" content="alice.org/pkg git https://github.com/alice/pkg">`,
// go-source repo defaults to go-import
"http://alice.org/pkg/default": `<head>
<meta name="go-import" content="alice.org/pkg git https://github.com/alice/pkg">
<meta name="go-source" content="alice.org/pkg _ foo bar">
</head>`,
// Package at root of a Git repo.
"https://bob.com/pkg": `<head> <meta name="go-import" content="bob.com/pkg git https://vcs.net/bob/pkg.git">`,
// Package at in sub-directory of a Git repo.
"https://bob.com/pkg/sub": `<head> <meta name="go-import" content="bob.com/pkg git https://vcs.net/bob/pkg.git">`,
internal/{source,frontend}: escape HTML Escape any strings that may come from outside the program and will be rendered as HTML. Also, fix some typos. Also, put args to a cmp.Diff call in the right order. Change-Id: I1fc6013e200326af8719b14c28a607fc7936e20a Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/239478 Reviewed-by: Julie Qiu <julie@golang.org>
2020-06-18 03:22:39 +03:00
"https://bob.com/bad/github": `
<head><meta name="go-import" content="bob.com/bad/github git https://github.com/bob/bad/&quot;&gt;$">`,
"https://bob.com/bad/apache": `
<head><meta name="go-import" content="bob.com/bad/apache git https://git.apache.org/&gt;$">`,
internal/source: handle meta tags Use the go command's meta-tag mechanism to find source code for vanity module paths. If a module path doesn't match one of our built-in patterns, make a GET request to the path and parse the go-import and go-source meta tags on the page to learn where the source lives and how to link to it. This process is slightly different than godoc.org's (as implemented in go.googlesource.com/gddo): - godoc.org uses package import paths; we use module paths instead. That can result in different behavior when a module doesn't correspond to a package and no tags are served at the module path URL, even though its package URLs do serve tags. The discovery site will not show source links for those packages. This is a genuine breaking behavior change, but we're going to make it because serving tags at the module level is the right thing to do, and we can easily fix the handful of modules where this is a problem. - godoc.org needs to fetch the code, so it needs information from the tags like the vcs, and it needs to keep the go-import and go-source tags separate (the first being for fetching, the second for linking). Since the discovery site already has the code from the zip file, we just need go-source information, using go-imports as a fallback. - The go-source tag contents don't provide a way to insert a version, so they're not usable as-is for linking to versioned code. For now, we ignore the URL templates in the tag and just try to match the repo. So a vanity URL that redirects to GitHub will work fine, since we know how to link to files in GitHub; but one that redirects to some random code-hosting site will not have source links on the discovery site, even though it might on godoc.org. Fixes b/141771975. Change-Id: Ie07bff3a77350cdaa7e45af0c7e5bce0a922e945 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/562176 Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-09-28 22:43:15 +03:00
// Package with go-source meta tag, where {file} appears on the right of '#' in the file field URL template.
"https://azul3d.org/examples/abs": `<!DOCTYPE html><html><head>` +
`<meta http-equiv="Content-Type" content="text/html; charset=utf-8"/>` +
`<meta name="go-import" content="azul3d.org/examples git https://github.com/azul3d/examples">` +
`<meta name="go-source" content="azul3d.org/examples https://github.com/azul3d/examples https://gotools.org/azul3d.org/examples{/dir} https://gotools.org/azul3d.org/examples{/dir}#{file}-L{line}">` +
`<meta http-equiv="refresh" content="0; url=https://godoc.org/azul3d.org/examples/abs">` +
`</head>`,
// Multiple go-import meta tags; one of which is a vgo-special mod vcs type
"http://myitcv.io/blah2": `<!DOCTYPE html><html><head>` +
`<meta http-equiv="Content-Type" content="text/html; charset=utf-8"/>` +
`<meta name="go-import" content="myitcv.io/blah2 git https://github.com/myitcv/x">` +
`<meta name="go-import" content="myitcv.io/blah2 mod https://raw.githubusercontent.com/myitcv/pubx/master">` +
`</head>`,
}
internal/source,postgres: save source info to the DB - Add a SourceInfo field to VersionInfo. - Add functions to convert a source.Info to and from JSON. - Modify postgres statements to read and write from the DB. Keep the VersionInfo.RepositoryURL field around for now to avoid messing with the frontend. Updates b/132219298, b/138752200. Change-Id: I6ebb3af3afa14fda92c2804ab2d0294742a3bf41 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/575156 CI-Result: Cloud Build <devtools-proctor-result-processor@system.gserviceaccount.com> Reviewed-by: Julie Qiu <julieqiu@google.com>
2019-10-15 13:00:58 +03:00
func TestJSON(t *testing.T) {
for _, test := range []struct {
in *Info
want string
}{
{
nil,
`null`,
},
{
&Info{repoURL: "r", moduleDir: "m", commit: "c"},
`{"RepoURL":"r","ModuleDir":"m","Commit":"c"}`,
},
{
&Info{repoURL: "r", moduleDir: "m", commit: "c", templates: githubURLTemplates},
`{"RepoURL":"r","ModuleDir":"m","Commit":"c","Kind":"github"}`,
},
{
&Info{repoURL: "r", moduleDir: "m", commit: "c", templates: urlTemplates{File: "f"}},
`{"RepoURL":"r","ModuleDir":"m","Commit":"c","Templates":{"Directory":"","File":"f","Line":"","Raw":""}}`,
},
} {
bytes, err := json.Marshal(&test.in)
if err != nil {
t.Fatal(err)
}
got := string(bytes)
if got != test.want {
t.Errorf("%#v:\ngot %s\nwant %s", test.in, got, test.want)
continue
}
var out Info
if err := json.Unmarshal(bytes, &out); err != nil {
t.Fatal(err)
}
var want Info
if test.in != nil {
want = *test.in
}
if out != want {
t.Errorf("got %#v\nwant %#v", out, want)
}
}
}