go
/
dep
зеркало из https://github.com/golang/dep.git
1
0
Форкнуть 0
Код Задачи Пакеты Проекты Релизы Вики Активность

Merge branch 'master' into new-ensure

This commit is contained in:
sam boyer 2017-07-31 11:42:17 -04:00
Родитель f38e02bbef f6f8423c3e
Коммит b461c3f829
78 изменённых файлов: 3129 добавлений и 1101 удалений
Показать статистику Скачать Patch файл Скачать Diff файл Показать все файлы Свернуть все файлы

27
.github/CODEOWNERS поставляемый Normal file
Просмотреть файл

@ -0,0 +1,27 @@
# general
* @sdboyer
# init
/cmd/dep/init* @carolynvs
/cmd/dep/gopath_scanner* @carolynvs
/cmd/dep/root_analyzer* @carolynvs
/cmd/dep/*_importer* @carolynvs
/cmd/dep/testdata/glide @carolynvs
/cmd/dep/testdata/godep @carolynvs
/cmd/dep/testdata/harness_tests/init @carolynvs
/cmd/dep/testdata/init** @carolynvs
/analyzer* @carolynvs
/testdata/analyzer @carolynvs
/internal/feedback @carolynvs
# ensure
/cmd/dep/ensure* @ibrasho
/cmd/dep/testdata/harness_tests/ensure** @ibrasho
# status
/cmd/dep/status* @darkowlzz
/cmd/dep/testdata/harness_tests/status** @darkowlzz
/cmd/dep/graphviz* @darkowlzz
# gps caching
/internal/gps/source_cache* @jmank88

2
.github/ISSUE_TEMPLATE.md поставляемый
Просмотреть файл

@ -1,7 +1,7 @@
<!--
Thanks for filing an issue! If this is a question or feature request, just delete
everthing here and write out the request, providing as much context as you can.
everything here and write out the request, providing as much context as you can.
-->

4
.github/PULL_REQUEST_TEMPLATE.md поставляемый
Просмотреть файл

@ -1,4 +1,4 @@
<--
<!--
Work-in-progress PRs are welcome as a way to get early feedback - just prefix
the title with [WIP].
-->
@ -11,7 +11,7 @@ the title with [WIP].
### Which issue(s) does this PR fix?
<--
<!--
fixes #
fixes #

2
CONTRIBUTING.md
Просмотреть файл

@ -8,7 +8,7 @@ Keep an eye on the [Roadmap](https://github.com/golang/dep/wiki/Roadmap) for a s
## Filing issues
Please check the existing issues and [FAQ](FAQ.md) to see if your feedback has already been reported.
Please check the existing issues and [FAQ](docs/FAQ.md) to see if your feedback has already been reported.
When [filing an issue](https://github.com/golang/dep/issues/new), make sure to answer these five questions:

8
Gopkg.lock сгенерированный
Просмотреть файл

@ -25,6 +25,12 @@
packages = ["."]
revision = "cd8b52f8269e0feb286dfeef29f8fe4d5b397e0b"
[[projects]]
branch = "master"
name = "github.com/nightlyone/lockfile"
packages = ["."]
revision = "e83dc5e7bba095e8d32fb2124714bf41f2a30cb5"
[[projects]]
name = "github.com/pelletier/go-buffruneio"
packages = ["."]
@ -52,6 +58,6 @@
[solve-meta]
analyzer-name = "dep"
analyzer-version = 1
inputs-digest = "1096dfb111cbe243aa24ea824ea3e1db7bb178c01d5565107c6d9290d225d722"
inputs-digest = "48b2f69d59b2f321b14ee11547f4ac94468ac91bb99aad48337d42b75b791975"
solver-name = "gps-cdcl"
solver-version = 1

2
MAINTAINERS.md
Просмотреть файл

@ -12,6 +12,6 @@ General maintainers:
* source manager: (vacant)
* root deduction: (vacant)
* source/vcs interaction: (vacant)
* caching: (vacant)
* caching: Jordan Krage (@jmank88)
* pkgtree: (vacant)
* versions and constraints: (vacant)

189
README.md
Просмотреть файл

@ -8,15 +8,16 @@ Dep is a prototype dependency management tool. It requires Go 1.7 or newer to co
## Current status
**Alpha**.
Functionality is known to be broken, missing or incomplete. Changes are planned
to the CLI commands soon. *It would be unwise to write scripts atop `dep` before then.*
The repository is open to solicit feedback and contributions from the community.
Please see below for feedback and contribution guidelines.
`dep` is safe for production use. That means two things:
`Gopkg.toml` and `Gopkg.lock` have reached a stable structure, and it is safe to
commit them in your projects. We plan to add more to these files, but we
guarantee these changes will be backwards-compatible.
* Any valid metadata file (`Gopkg.toml` and `Gopkg.lock`) will be readable and considered valid by any future version of `dep`.
* Generally speaking, it has comparable or fewer bugs than other tools out there.
That said, keep in mind the following:
* `dep` is still changing rapidly. If you need stability (e.g. for CI), it's best to rely on a released version, not tip.
* [Some changes](https://github.com/golang/dep/pull/489) are pending to the CLI interface. Scripting on dep before they land is unwise.
* `dep`'s exported API interface will continue to change in unpredictable, backwards-incompatible ways until we tag a v1.0.0 release.
## Context
@ -26,9 +27,9 @@ guarantee these changes will be backwards-compatible.
- [User Stories](https://docs.google.com/document/d/1wT8e8wBHMrSRHY4UF_60GCgyWGqvYye4THvaDARPySs/edit)
- [Features](https://docs.google.com/document/d/1JNP6DgSK-c6KqveIhQk-n_HAw3hsZkL-okoleM43NgA/edit)
- [Design Space](https://docs.google.com/document/d/1TpQlQYovCoX9FkpgsoxzdvZplghudHAiQOame30A-v8/edit)
- [Frequently Asked Questions](FAQ.md)
- [Frequently Asked Questions](docs/FAQ.md)
## Usage
## Setup
Get the tool via
@ -36,20 +37,173 @@ Get the tool via
$ go get -u github.com/golang/dep/cmd/dep
```
Typical usage on a new repo might be
To start managing dependencies using dep, run the following from your project root directory:
```sh
$ dep init
```
This does the following:
1. Look for [existing dependency management files](docs/FAQ.md#what-external-tools-are-supported) to convert
1. Check if your dependencies use dep
1. Identify your dependencies
1. Back up your existing `vendor/` directory (if you have one) to
`_vendor-TIMESTAMP/`
1. Pick the highest compatible version for each dependency
1. Generate [`Gopkg.toml`](docs/Gopkg.toml.md) ("manifest") and `Gopkg.lock` files
1. Install the dependencies in `vendor/`
## Usage
There is one main subcommand you will use: `dep ensure`. `ensure` first makes
sure `Gopkg.lock` is consistent with your `import`s and `Gopkg.toml`. If any
changes are detected, it then populates `vendor/` with exactly what's described
in `Gopkg.lock`.
`dep ensure` is safe to run early and often. See the help text for more detailed
usage instructions.
```sh
$ dep help ensure
```
### Installing dependencies
(if your `vendor/` directory isn't [checked in with your code](docs/FAQ.md#should-i-commit-my-vendor-directory))
<!-- may change with https://github.com/golang/dep/pull/489 -->
```sh
$ dep ensure
```
If a dependency already exists in your `vendor/` folder, dep will ensure it
matches the constraints from the manifest. If the dependency is missing from
`vendor/`, the latest version allowed by your manifest will be installed.
### Adding a dependency
1. `import` the package in your `*.go` source code file(s).
1. Run the following command to update your `Gopkg.lock` and populate `vendor/` with the new dependency.
```sh
$ dep ensure
```
### Changing dependencies
If you want to:
* Change the allowed `version`/`branch`/`revision`
* Switch to using a fork
for one or more dependencies, do the following:
1. Modify your `Gopkg.toml`.
1. Run
```sh
$ dep ensure
```
### Checking the status of dependencies
Run `dep status` to see the current status of all your dependencies.
```sh
$ dep status
PROJECT CONSTRAINT VERSION REVISION LATEST
github.com/Masterminds/semver branch 2.x branch 2.x 139cc09 c2e7f6c
github.com/Masterminds/vcs ^1.11.0 v1.11.1 3084677 3084677
github.com/armon/go-radix * branch master 4239b77 4239b77
```
On top of that, if you have added new imports to your project or modified the manifest file without running `dep ensure` again, `dep status` will tell you there is a mismatch between the lock file and the current status of the project.
```sh
$ dep status
Lock inputs-digest mismatch due to the following packages missing from the lock:
PROJECT MISSING PACKAGES
github.com/Masterminds/goutils [github.com/Masterminds/goutils]
This happens when a new import is added. Run `dep ensure` to install the missing packages.
```
As `dep status` suggests, run `dep ensure` to update your lockfile. Then run `dep status` again, and the lock mismatch should go away.
### Updating dependencies
(to the latest version allowed by the manifest)
```sh
$ dep ensure -update
```
To update a dependency to a new version, you might run
### Removing dependencies
```sh
$ dep ensure github.com/pkg/errors@^0.8.0
```
1. Remove the `import`s and all usage from your code.
1. Run
See the help text for more detailed usage instructions.
```sh
$ dep ensure
```
1. Remove from `Gopkg.toml`, if it was in there.
### Testing changes to a dependency
Making changes in your `vendor/` directory directly is not recommended, as dep
will overwrite any changes. Instead:
1. Delete the dependency from the `vendor/` directory.
```sh
rm -rf vendor/<dependency>
```
1. Add that dependency to your `GOPATH`, if it isn't already.
```sh
$ go get <dependency>
```
1. Modify the dependency in `$GOPATH/src/<dependency>`.
1. Test, build, etc.
Don't run `dep ensure` until you're done. `dep ensure` will reinstall the
dependency into `vendor/` based on your manifest, as if you were installing from
scratch.
This solution works for short-term use, but for something long-term, take a look
at [virtualgo](https://github.com/GetStream/vg).
To test out code that has been pushed as a new version, or to a branch or fork,
see [changing dependencies](#changing-dependencies).
## Semantic Versioning
`dep ensure` a uses an external [semver library](https://github.com/Masterminds/semver) to interpret the version constraints you specify in the manifest. The comparison operators are:
* `=`: equal
* `!=`: not equal
* `>`: greater than
* `<`: less than
* `>=`: greater than or equal to
* `<=`: less than or equal to
* `-`: literal range. Eg: 1.2 - 1.4.5 is equivalent to >= 1.2, <= 1.4.5
* `~`: minor range. Eg: ~1.2.3 is equivalent to >= 1.2.3, < 1.3.0
* `^`: major range. Eg: ^1.2.3 is equivalent to >= 1.2.3, < 2.0.0
* `[xX*]`: wildcard. Eg: 1.2.x is equivalent to >= 1.2.0, < 1.3.0
You might, for example, include a constraint in your manifest that specifies `version = "=2.0.0"` to pin a dependency to version 2.0.0, or constrain to minor releases with: `version = "2.*"`. Refer to the [semver library](https://github.com/Masterminds/semver) documentation for more info.
**Note**: When you specify a version *without an operator*, `dep` automatically uses the `^` operator by default. `dep ensure` will interpret the given version as the min-boundry of a range, for example:
* `1.2.3` becomes the range `>=1.2.3, <2.0.0`
* `0.2.3` becomes the range `>=0.2.3, <0.3.0`
* `0.0.3` becomes the range `>=0.0.3, <0.1.0`
## Feedback
@ -57,7 +211,7 @@ Feedback is greatly appreciated.
At this stage, the maintainers are most interested in feedback centered on the user experience (UX) of the tool.
Do you have workflows that the tool supports well, or doesn't support at all?
Do any of the commands have surprising effects, output, or results?
Please check the existing issues and [FAQ](FAQ.md) to see if your feedback has already been reported.
Please check the existing issues and [FAQ](docs/FAQ.md) to see if your feedback has already been reported.
If not, please file an issue, describing what you did or wanted to do, what you expected to happen, and what actually happened.
## Contributing
@ -67,4 +221,3 @@ The maintainers actively manage the issues list, and try to highlight issues sui
The project follows the typical GitHub pull request model.
See [CONTRIBUTING.md](CONTRIBUTING.md) for more details.
Before starting any work, please either comment on an existing issue, or file a new one.

3
analyzer.go
Просмотреть файл

@ -40,8 +40,7 @@ func (a Analyzer) DeriveManifestAndLock(path string, n gps.ProjectRoot) (gps.Man
if err != nil {
return nil, nil, err
}
// TODO: No need to return lock til we decide about preferred versions, see
// https://github.com/sdboyer/gps/wiki/gps-for-Implementors#preferred-versions.
return m, nil, nil
}

4
appveyor.yml
Просмотреть файл

@ -8,10 +8,6 @@ clone_folder: c:\gopath\src\github.com\golang\dep
environment:
GOPATH: c:\gopath
DEPTESTBYPASS501: 1
matrix:
- environment:
GOVERSION: 1.7.5
- environment:
GOVERSION: 1.8
init:

104
cmd/dep/ensure.go
Просмотреть файл

@ -6,13 +6,11 @@ package main
import (
"bytes"
"encoding/hex"
"flag"
"go/build"
"os"
"path/filepath"
"sort"
"strconv"
"strings"
"github.com/golang/dep"
@ -633,119 +631,43 @@ func (s *stringSlice) Set(value string) error {
*s = append(*s, value)
return nil
}
func getProjectConstraint(arg string, sm gps.SourceManager) (gps.ProjectConstraint, string, error) {
emptyPC := gps.ProjectConstraint{
Constraint: gps.Any(), // default to any; avoids panics later
}
path, source, versionStr := parseSpecArg(arg)
pr, err := sm.DeduceProjectRoot(path)
if err != nil {
return emptyPC, "", errors.Wrapf(err, "could not infer project root from dependency path: %s", path) // this should go through to the user
}
pi := gps.ProjectIdentifier{ProjectRoot: pr, Source: source}
var c gps.Constraint
if versionStr != "" {
c, err = deduceConstraint(versionStr, pi, sm)
if err != nil {
return emptyPC, "", err
}
} else {
c = gps.Any()
}
return gps.ProjectConstraint{Ident: pi, Constraint: c}, path, nil
}
// parseSpecArg takes a spec, as provided to dep ensure on the CLI, and splits
// it into its constitutent path, source, and constraint parts.
func parseSpecArg(arg string) (path, source, constraint string) {
path = arg
// try to split on '@'
// When there is no `@`, use any version
versionStr := "*"
atIndex := strings.Index(arg, "@")
if atIndex > 0 {
parts := strings.SplitN(arg, "@", 2)
path = parts[0]
constraint = parts[1]
arg = parts[0]
versionStr = parts[1]
}
// TODO: if we decide to keep equals.....
// split on colon if there is a network location
var source string
colonIndex := strings.Index(arg, ":")
if colonIndex > 0 {
parts := strings.SplitN(arg, ":", 2)
path = parts[0]
arg = parts[0]
source = parts[1]
}
return path, source, constraint
}
// deduceConstraint tries to puzzle out what kind of version is given in a string -
// semver, a revision, or as a fallback, a plain tag
func deduceConstraint(s string, pi gps.ProjectIdentifier, sm gps.SourceManager) (gps.Constraint, error) {
if s == "" {
// Find the default branch
versions, err := sm.ListVersions(pi)
pr, err := sm.DeduceProjectRoot(arg)
if err != nil {
return nil, errors.Wrapf(err, "list versions for %s(%s)", pi.ProjectRoot, pi.Source) // means repo does not exist
return emptyPC, "", errors.Wrapf(err, "could not infer project root from dependency path: %s", arg) // this should go through to the user
}
gps.SortPairedForUpgrade(versions)
for _, v := range versions {
if v.Type() == gps.IsBranch {
return v.Unpair(), nil
}
}
}
// always semver if we can
c, err := gps.NewSemverConstraintIC(s)
if err == nil {
return c, nil
}
slen := len(s)
if slen == 40 {
if _, err = hex.DecodeString(s); err == nil {
// Whether or not it's intended to be a SHA1 digest, this is a
// valid byte sequence for that, so go with Revision. This
// covers git and hg
return gps.Revision(s), nil
}
}
// Next, try for bzr, which has a three-component GUID separated by
// dashes. There should be two, but the email part could contain
// internal dashes
if strings.Count(s, "-") >= 2 {
// Work from the back to avoid potential confusion from the email
i3 := strings.LastIndex(s, "-")
// Skip if - is last char, otherwise this would panic on bounds err
if slen == i3+1 {
return gps.NewVersion(s), nil
}
i2 := strings.LastIndex(s[:i3], "-")
if _, err = strconv.ParseUint(s[i2+1:i3], 10, 64); err == nil {
// Getting this far means it'd pretty much be nuts if it's not a
// bzr rev, so don't bother parsing the email.
return gps.Revision(s), nil
}
}
// call out to network and get the package's versions
versions, err := sm.ListVersions(pi)
pi := gps.ProjectIdentifier{ProjectRoot: pr, Source: source}
c, err := sm.InferConstraint(versionStr, pi)
if err != nil {
return nil, errors.Wrapf(err, "list versions for %s(%s)", pi.ProjectRoot, pi.Source) // means repo does not exist
return emptyPC, "", err
}
for _, version := range versions {
if s == version.String() {
return version.Unpair(), nil
}
}
return nil, errors.Errorf("%s is not a valid version for the package %s(%s)", s, pi.ProjectRoot, pi.Source)
return gps.ProjectConstraint{Ident: pi, Constraint: c}, arg, nil
}
func checkErrors(m map[string]pkgtree.PackageOrErr) error {

20
cmd/dep/glide_importer.go
Просмотреть файл

@ -178,11 +178,11 @@ func (g *glideImporter) convert(pr gps.ProjectRoot) (*dep.Manifest, *dep.Lock, e
lock = &dep.Lock{}
for _, pkg := range g.lock.Imports {
lp := g.buildLockedProject(pkg)
lp := g.buildLockedProject(pkg, manifest)
lock.P = append(lock.P, lp)
}
for _, pkg := range g.lock.TestImports {
lp := g.buildLockedProject(pkg)
lp := g.buildLockedProject(pkg, manifest)
lock.P = append(lock.P, lp)
}
}
@ -206,7 +206,10 @@ func (g *glideImporter) buildProjectConstraint(pkg glidePackage) (pc gps.Project
}
pc.Ident = gps.ProjectIdentifier{ProjectRoot: gps.ProjectRoot(pkg.Name), Source: pkg.Repository}
pc.Constraint, err = deduceConstraint(pkg.Reference, pc.Ident, g.sm)
pc.Constraint, err = g.sm.InferConstraint(pkg.Reference, pc.Ident)
if err != nil {
return
}
f := fb.NewConstraintFeedback(pc, fb.DepTypeImported)
f.LogFeedback(g.logger)
@ -214,19 +217,18 @@ func (g *glideImporter) buildProjectConstraint(pkg glidePackage) (pc gps.Project
return
}
func (g *glideImporter) buildLockedProject(pkg glideLockedPackage) gps.LockedProject {
func (g *glideImporter) buildLockedProject(pkg glideLockedPackage, manifest *dep.Manifest) gps.LockedProject {
pi := gps.ProjectIdentifier{
ProjectRoot: gps.ProjectRoot(pkg.Name),
Source: pkg.Repository,
}
revision := gps.Revision(pkg.Reference)
pp := manifest.Constraints[pi.ProjectRoot]
version, err := lookupVersionForRevision(revision, pi, g.sm)
version, err := lookupVersionForLockedProject(pi, pp.Constraint, revision, g.sm)
if err != nil {
// Warn about the problem, it is not enough to warrant failing
warn := errors.Wrapf(err, "Unable to lookup the version represented by %s in %s(%s). Falling back to locking the revision only.", revision, pi.ProjectRoot, pi.Source)
g.logger.Printf(warn.Error())
version = revision
// Only warn about the problem, it is not enough to warrant failing
g.logger.Println(err.Error())
}
lp := gps.NewLockedProject(pi, version, nil)

8
cmd/dep/glide_importer_test.go
Просмотреть файл

@ -68,7 +68,7 @@ func TestGlideConfig_Import(t *testing.T) {
t.Fatal("Expected the lock to be generated")
}
goldenFile := "glide/expected_import_output.txt"
goldenFile := "glide/golden.txt"
got := verboseOutput.String()
want := h.GetTestFileString(goldenFile)
if want != got {
@ -91,9 +91,9 @@ func TestGlideConfig_Import_MissingLockFile(t *testing.T) {
h.Must(err)
defer sm.Release()
h.TempDir(filepath.Join("src", "glidetest"))
h.TempCopy(filepath.Join("glidetest", glideYamlName), "glide/glide.yaml")
projectRoot := h.Path("glidetest")
h.TempDir(filepath.Join("src", testGlideProjectRoot))
h.TempCopy(filepath.Join(testGlideProjectRoot, glideYamlName), "glide/glide.yaml")
projectRoot := h.Path(testGlideProjectRoot)
g := newGlideImporter(ctx.Err, true, sm)
if !g.HasDepMetadata(projectRoot) {

34
cmd/dep/godep_importer.go
Просмотреть файл

@ -125,18 +125,18 @@ func (g *godepImporter) convert(pr gps.ProjectRoot) (*dep.Manifest, *dep.Lock, e
ProjectRoot: gps.ProjectRoot(pkg.ImportPath),
}
revision := gps.Revision(pkg.Rev)
version, err := lookupVersionForRevision(revision, pi, g.sm)
if err != nil {
warn := errors.Wrapf(err, "Unable to lookup the version represented by %s in %s. Falling back to locking the revision only.", pkg.Rev, pi.ProjectRoot)
g.logger.Printf(warn.Error())
version = revision
}
version, err := lookupVersionForLockedProject(pi, nil, revision, g.sm)
if err != nil {
// Only warn about the problem, it is not enough to warrant failing
g.logger.Println(err.Error())
} else {
pp := getProjectPropertiesFromVersion(version)
if pp.Constraint != nil {
pkg.Comment = pp.Constraint.String()
}
}
}
if pkg.Comment != "" {
// If there's a comment, use it to create project constraint
@ -147,7 +147,7 @@ func (g *godepImporter) convert(pr gps.ProjectRoot) (*dep.Manifest, *dep.Lock, e
manifest.Constraints[pc.Ident.ProjectRoot] = gps.ProjectProperties{Constraint: pc.Constraint}
}
lp := g.buildLockedProject(pkg)
lp := g.buildLockedProject(pkg, manifest)
lock.P = append(lock.P, lp)
}
@ -158,7 +158,10 @@ func (g *godepImporter) convert(pr gps.ProjectRoot) (*dep.Manifest, *dep.Lock, e
// create a project constraint
func (g *godepImporter) buildProjectConstraint(pkg godepPackage) (pc gps.ProjectConstraint, err error) {
pc.Ident = gps.ProjectIdentifier{ProjectRoot: gps.ProjectRoot(pkg.ImportPath)}
pc.Constraint, err = deduceConstraint(pkg.Comment, pc.Ident, g.sm)
pc.Constraint, err = g.sm.InferConstraint(pkg.Comment, pc.Ident)
if err != nil {
return
}
f := fb.NewConstraintFeedback(pc, fb.DepTypeImported)
f.LogFeedback(g.logger)
@ -167,16 +170,15 @@ func (g *godepImporter) buildProjectConstraint(pkg godepPackage) (pc gps.Project
}
// buildLockedProject uses the package Rev and Comment to create lock project
func (g *godepImporter) buildLockedProject(pkg godepPackage) gps.LockedProject {
func (g *godepImporter) buildLockedProject(pkg godepPackage, manifest *dep.Manifest) gps.LockedProject {
pi := gps.ProjectIdentifier{ProjectRoot: gps.ProjectRoot(pkg.ImportPath)}
revision := gps.Revision(pkg.Rev)
pp := manifest.Constraints[pi.ProjectRoot]
var version gps.Version
if pkg.Comment != "" {
ver := gps.NewVersion(pkg.Comment)
version = ver.Pair(gps.Revision(pkg.Rev))
} else {
version = gps.Revision(pkg.Rev)
version, err := lookupVersionForLockedProject(pi, pp.Constraint, revision, g.sm)
if err != nil {
// Only warn about the problem, it is not enough to warrant failing
g.logger.Println(err.Error())
}
lp := gps.NewLockedProject(pi, version, nil)

45
cmd/dep/godep_importer_test.go
Просмотреть файл

@ -161,6 +161,47 @@ func TestGodepConfig_ConvertProject(t *testing.T) {
}
}
func TestGodepConfig_ConvertProject_WithSemverSuffix(t *testing.T) {
h := test.NewHelper(t)
defer h.Cleanup()
ctx := newTestContext(h)
sm, err := ctx.SourceManager()
h.Must(err)
defer sm.Release()
g := newGodepImporter(discardLogger, true, sm)
g.json = godepJSON{
Imports: []godepPackage{
{
ImportPath: "github.com/sdboyer/deptest",
Rev: "ff2948a2ac8f538c4ecd55962e919d1e13e74baf",
Comment: "v1.12.0-12-g2fd980e",
},
},
}
manifest, lock, err := g.convert("")
if err != nil {
t.Fatal(err)
}
d, ok := manifest.Constraints["github.com/sdboyer/deptest"]
if !ok {
t.Fatal("Expected the manifest to have a dependency for 'github.com/sdboyer/deptest' but got none")
}
v := d.Constraint.String()
if v != ">=1.12.0, <=12.0.0-g2fd980e" {
t.Fatalf("Expected manifest constraint to be >=1.12.0, <=12.0.0-g2fd980e, got %s", v)
}
p := lock.P[0]
if p.Ident().ProjectRoot != "github.com/sdboyer/deptest" {
t.Fatalf("Expected the lock to have a project for 'github.com/sdboyer/deptest' but got '%s'", p.Ident().ProjectRoot)
}
}
func TestGodepConfig_ConvertProject_EmptyComment(t *testing.T) {
h := test.NewHelper(t)
defer h.Cleanup()
@ -214,8 +255,8 @@ func TestGodepConfig_ConvertProject_EmptyComment(t *testing.T) {
}
ver := lpv.String()
if ver != "^1.0.0" {
t.Fatalf("Expected locked version to be '^1.0.0', got %s", ver)
if ver != "v1.0.0" {
t.Fatalf("Expected locked version to be 'v1.0.0', got %s", ver)
}
}

15
cmd/dep/gopath_scanner.go
Просмотреть файл

@ -219,7 +219,12 @@ func (g *gopathScanner) scanGopathForDependencies() (projectData, error) {
go syncDep(pr, g.sm)
dependencies[pr] = []string{ip}
v, err := g.ctx.VersionInWorkspace(pr)
abs, err := g.ctx.AbsForImport(string(pr))
if err != nil {
notondisk[pr] = true
continue
}
v, err := gps.VCSVersion(abs)
if err != nil {
notondisk[pr] = true
continue
@ -283,7 +288,13 @@ func (g *gopathScanner) scanGopathForDependencies() (projectData, error) {
// was found in the initial pass on direct imports. We know it's
// the former if there's no entry for it in the ondisk map.
if _, in := ondisk[pr]; !in {
v, err := g.ctx.VersionInWorkspace(pr)
abs, err := g.ctx.AbsForImport(string(pr))
if err != nil {
colors[pkg] = black
notondisk[pr] = true
return nil
}
v, err := gps.VCSVersion(abs)
if err != nil {
// Even if we know it's on disk, errors are still
// possible when trying to deduce version. If we

25
cmd/dep/init.go
Просмотреть файл

@ -26,7 +26,7 @@ specified, use the current directory.
When configuration for another dependency management tool is detected, it is
imported into the initial manifest and lock. Use the -skip-tools flag to
disable this behavior. The following external tools are supported: glide.
disable this behavior. The following external tools are supported: glide, godep.
Any dependencies that are not constrained by external configuration use the
GOPATH analysis below.
@ -116,16 +116,12 @@ func (cmd *initCommand) Run(ctx *dep.Ctx, args []string) error {
return errors.Errorf("invalid state: manifest %q does not exist, but lock %q does", mf, lf)
}
ip, err := ctx.SplitAbsoluteProjectRoot(root)
ip, err := ctx.ImportForAbs(root)
if err != nil {
return errors.Wrap(err, "determineProjectRoot")
return errors.Wrap(err, "root project import")
}
p.ImportRoot = gps.ProjectRoot(ip)
pkgT, directDeps, err := getDirectDependencies(p)
if err != nil {
return err
}
sm, err := ctx.SourceManager()
if err != nil {
return errors.Wrap(err, "getSourceManager")
@ -133,6 +129,11 @@ func (cmd *initCommand) Run(ctx *dep.Ctx, args []string) error {
sm.UseDefaultSignalHandling()
defer sm.Release()
pkgT, directDeps, err := getDirectDependencies(sm, p)
if err != nil {
return err
}
// Initialize with imported data, then fill in the gaps using the GOPATH
rootAnalyzer := newRootAnalyzer(cmd.skipTools, ctx, directDeps, sm)
p.Manifest, p.Lock, err = rootAnalyzer.InitializeRootManifestAndLock(root, p.ImportRoot)
@ -207,7 +208,7 @@ func (cmd *initCommand) Run(ctx *dep.Ctx, args []string) error {
return nil
}
func getDirectDependencies(p *dep.Project) (pkgtree.PackageTree, map[string]bool, error) {
func getDirectDependencies(sm gps.SourceManager, p *dep.Project) (pkgtree.PackageTree, map[string]bool, error) {
pkgT, err := pkgtree.ListPackages(p.ResolvedAbsRoot, string(p.ImportRoot))
if err != nil {
return pkgtree.PackageTree{}, nil, errors.Wrap(err, "gps.ListPackages")
@ -215,8 +216,12 @@ func getDirectDependencies(p *dep.Project) (pkgtree.PackageTree, map[string]bool
directDeps := map[string]bool{}
rm, _ := pkgT.ToReachMap(true, true, false, nil)
for _, pr := range rm.FlattenFn(paths.IsStandardImportPath) {
directDeps[pr] = true
for _, ip := range rm.FlattenFn(paths.IsStandardImportPath) {
pr, err := sm.DeduceProjectRoot(ip)
if err != nil {
return pkgtree.PackageTree{}, nil, err
}
directDeps[string(pr)] = true
}
return pkgT, directDeps, nil

41
cmd/dep/init_test.go Normal file
Просмотреть файл

@ -0,0 +1,41 @@
// Copyright 2016 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package main
import (
"testing"
"path/filepath"
"github.com/golang/dep"
"github.com/golang/dep/internal/gps"
"github.com/golang/dep/internal/test"
)
func TestGetDirectDependencies_ConsolidatesRootProjects(t *testing.T) {
h := test.NewHelper(t)
defer h.Cleanup()
ctx := newTestContext(h)
sm, err := ctx.SourceManager()
h.Must(err)
defer sm.Release()
testprj := "directdepstest"
testdir := filepath.Join("src", testprj)
h.TempDir(testdir)
h.TempCopy(filepath.Join(testdir, "main.go"), "init/directdeps/main.go")
testpath := h.Path(testdir)
prj := &dep.Project{AbsRoot: testpath, ResolvedAbsRoot: testpath, ImportRoot: gps.ProjectRoot(testprj)}
_, dd, err := getDirectDependencies(sm, prj)
h.Must(err)
wantpr := "github.com/carolynvs/deptest-subpkg"
if _, has := dd[wantpr]; !has {
t.Fatalf("Expected direct dependencies to contain %s, got %v", wantpr, dd)
}
}

130
cmd/dep/prune.go
Просмотреть файл

@ -7,9 +7,15 @@ package main
import (
"bytes"
"flag"
"io/ioutil"
"log"
"os"
"path/filepath"
"sort"
"strings"
"github.com/golang/dep"
"github.com/golang/dep/internal/fs"
"github.com/golang/dep/internal/gps"
"github.com/golang/dep/internal/gps/pkgtree"
"github.com/pkg/errors"
@ -80,5 +86,127 @@ func (cmd *pruneCommand) Run(ctx *dep.Ctx, args []string) error {
if ctx.Verbose {
pruneLogger = ctx.Err
}
return dep.PruneProject(p, sm, pruneLogger)
return pruneProject(p, sm, pruneLogger)
}
// pruneProject removes unused packages from a project.
func pruneProject(p *dep.Project, sm gps.SourceManager, logger *log.Logger) error {
td, err := ioutil.TempDir(os.TempDir(), "dep")
if err != nil {
return errors.Wrap(err, "error while creating temp dir for writing manifest/lock/vendor")
}
defer os.RemoveAll(td)
if err := gps.WriteDepTree(td, p.Lock, sm, true); err != nil {
return err
}
var toKeep []string
for _, project := range p.Lock.Projects() {
projectRoot := string(project.Ident().ProjectRoot)
for _, pkg := range project.Packages() {
toKeep = append(toKeep, filepath.Join(projectRoot, pkg))
}
}
toDelete, err := calculatePrune(td, toKeep, logger)
if err != nil {
return err
}
if logger != nil {
if len(toDelete) > 0 {
logger.Println("Calculated the following directories to prune:")
for _, d := range toDelete {
logger.Printf(" %s\n", d)
}
} else {
logger.Println("No directories found to prune")
}
}
if err := deleteDirs(toDelete); err != nil {
return err
}
vpath := filepath.Join(p.AbsRoot, "vendor")
vendorbak := vpath + ".orig"
var failerr error
if _, err := os.Stat(vpath); err == nil {
// Move out the old vendor dir. just do it into an adjacent dir, to
// try to mitigate the possibility of a pointless cross-filesystem
// move with a temp directory.
if _, err := os.Stat(vendorbak); err == nil {
// If the adjacent dir already exists, bite the bullet and move
// to a proper tempdir.
vendorbak = filepath.Join(td, "vendor.orig")
}
failerr = fs.RenameWithFallback(vpath, vendorbak)
if failerr != nil {
goto fail
}
}
// Move in the new one.
failerr = fs.RenameWithFallback(td, vpath)
if failerr != nil {
goto fail
}
os.RemoveAll(vendorbak)
return nil
fail:
fs.RenameWithFallback(vendorbak, vpath)
return failerr
}
func calculatePrune(vendorDir string, keep []string, logger *log.Logger) ([]string, error) {
if logger != nil {
logger.Println("Calculating prune. Checking the following packages:")
}
sort.Strings(keep)
toDelete := []string{}
err := filepath.Walk(vendorDir, func(path string, info os.FileInfo, err error) error {
if _, err := os.Lstat(path); err != nil {
return nil
}
if !info.IsDir() {
return nil
}
if path == vendorDir {
return nil
}
name := strings.TrimPrefix(path, vendorDir+string(filepath.Separator))
if logger != nil {
logger.Printf(" %s", name)
}
i := sort.Search(len(keep), func(i int) bool {
return name <= keep[i]
})
if i >= len(keep) || !strings.HasPrefix(keep[i], name) {
toDelete = append(toDelete, path)
}
return nil
})
return toDelete, err
}
func deleteDirs(toDelete []string) error {
// sort by length so we delete sub dirs first
sort.Sort(byLen(toDelete))
for _, path := range toDelete {
if err := os.RemoveAll(path); err != nil {
return err
}
}
return nil
}
type byLen []string
func (a byLen) Len() int { return len(a) }
func (a byLen) Swap(i, j int) { a[i], a[j] = a[j], a[i] }
func (a byLen) Less(i, j int) bool { return len(a[i]) > len(a[j]) }

46
cmd/dep/prune_test.go Normal file
Просмотреть файл

@ -0,0 +1,46 @@
// Copyright 2017 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package main
import (
"path/filepath"
"reflect"
"sort"
"testing"
"github.com/golang/dep/internal/test"
)
func TestCalculatePrune(t *testing.T) {
h := test.NewHelper(t)
defer h.Cleanup()
vendorDir := "vendor"
h.TempDir(vendorDir)
h.TempDir(filepath.Join(vendorDir, "github.com/keep/pkg/sub"))
h.TempDir(filepath.Join(vendorDir, "github.com/prune/pkg/sub"))
toKeep := []string{
filepath.FromSlash("github.com/keep/pkg"),
filepath.FromSlash("github.com/keep/pkg/sub"),
}
got, err := calculatePrune(h.Path(vendorDir), toKeep, nil)
if err != nil {
t.Fatal(err)
}
sort.Sort(byLen(got))
want := []string{
h.Path(filepath.Join(vendorDir, "github.com/prune/pkg/sub")),
h.Path(filepath.Join(vendorDir, "github.com/prune/pkg")),
h.Path(filepath.Join(vendorDir, "github.com/prune")),
}
if !reflect.DeepEqual(want, got) {
t.Fatalf("calculated prune paths are not as expected.\n(WNT) %s\n(GOT) %s", want, got)
}
}

26
cmd/dep/root_analyzer.go
Просмотреть файл

@ -173,19 +173,41 @@ func (a *rootAnalyzer) Info() gps.ProjectAnalyzerInfo {
}
}
func lookupVersionForRevision(rev gps.Revision, pi gps.ProjectIdentifier, sm gps.SourceManager) (gps.Version, error) {
// lookupVersionForLockedProject figures out the appropriate version for a locked
// project based on the locked revision and the constraint from the manifest.
// First try matching the revision to a version, then try the constraint from the
// manifest, then finally the revision.
func lookupVersionForLockedProject(pi gps.ProjectIdentifier, c gps.Constraint, rev gps.Revision, sm gps.SourceManager) (gps.Version, error) {
// Find the version that goes with this revision, if any
versions, err := sm.ListVersions(pi)
if err != nil {
return nil, errors.Wrapf(err, "Unable to list versions for %s(%s)", pi.ProjectRoot, pi.Source)
return rev, errors.Wrapf(err, "Unable to lookup the version represented by %s in %s(%s). Falling back to locking the revision only.", rev, pi.ProjectRoot, pi.Source)
}
gps.SortPairedForUpgrade(versions) // Sort versions in asc order
for _, v := range versions {
if v.Revision() == rev {
// If the constraint is semver, make sure the version is acceptable.
// This prevents us from suggesting an incompatible version, which
// helps narrow the field when there are multiple matching versions.
if c != nil {
_, err := gps.NewSemverConstraint(c.String())
if err == nil && !c.Matches(v) {
continue
}
}
return v, nil
}
}
// Use the version from the manifest as long as it wasn't a range
switch tv := c.(type) {
case gps.PairedVersion:
return tv.Unpair().Pair(rev), nil
case gps.UnpairedVersion:
return tv.Pair(rev), nil
}
// Give up and lock only to a revision
return rev, nil
}

94
cmd/dep/root_analyzer_test.go
Просмотреть файл

@ -4,7 +4,12 @@
package main
import "testing"
import (
"testing"
"github.com/golang/dep/internal/gps"
"github.com/golang/dep/internal/test"
)
func TestRootAnalyzer_Info(t *testing.T) {
testCases := map[bool]string{
@ -19,3 +24,90 @@ func TestRootAnalyzer_Info(t *testing.T) {
}
}
}
func TestLookupVersionForLockedProject_MatchRevisionToTag(t *testing.T) {
h := test.NewHelper(t)
defer h.Cleanup()
ctx := newTestContext(h)
sm, err := ctx.SourceManager()
h.Must(err)
defer sm.Release()
pi := gps.ProjectIdentifier{ProjectRoot: gps.ProjectRoot("github.com/sdboyer/deptest")}
rev := gps.Revision("ff2948a2ac8f538c4ecd55962e919d1e13e74baf")
v, err := lookupVersionForLockedProject(pi, nil, rev, sm)
h.Must(err)
wantV := "v1.0.0"
gotV := v.String()
if gotV != wantV {
t.Fatalf("Expected the locked version to be the tag paired with the manifest's pinned revision: wanted '%s', got '%s'", wantV, gotV)
}
}
func TestLookupVersionForLockedProject_MatchRevisionToMultipleTags(t *testing.T) {
h := test.NewHelper(t)
defer h.Cleanup()
ctx := newTestContext(h)
sm, err := ctx.SourceManager()
h.Must(err)
defer sm.Release()
pi := gps.ProjectIdentifier{ProjectRoot: gps.ProjectRoot("github.com/sdboyer/deptest")}
// Both 0.8.0 and 1.0.0 use the same rev, force dep to pick the lower version
c, _ := gps.NewSemverConstraint("<1.0.0")
rev := gps.Revision("ff2948a2ac8f538c4ecd55962e919d1e13e74baf")
v, err := lookupVersionForLockedProject(pi, c, rev, sm)
h.Must(err)
wantV := "v0.8.0"
gotV := v.String()
if gotV != wantV {
t.Fatalf("Expected the locked version to satisfy the manifest's semver constraint: wanted '%s', got '%s'", wantV, gotV)
}
}
func TestLookupVersionForLockedProject_FallbackToConstraint(t *testing.T) {
h := test.NewHelper(t)
defer h.Cleanup()
ctx := newTestContext(h)
sm, err := ctx.SourceManager()
h.Must(err)
defer sm.Release()
pi := gps.ProjectIdentifier{ProjectRoot: gps.ProjectRoot("github.com/sdboyer/deptest")}
c := gps.NewBranch("master")
rev := gps.Revision("c575196502940c07bf89fd6d95e83b999162e051")
v, err := lookupVersionForLockedProject(pi, c, rev, sm)
h.Must(err)
wantV := c.String()
gotV := v.String()
if gotV != wantV {
t.Fatalf("Expected the locked version to be defaulted from the manifest's branch constraint: wanted '%s', got '%s'", wantV, gotV)
}
}
func TestLookupVersionForLockedProject_FallbackToRevision(t *testing.T) {
h := test.NewHelper(t)
defer h.Cleanup()
ctx := newTestContext(h)
sm, err := ctx.SourceManager()
h.Must(err)
defer sm.Release()
pi := gps.ProjectIdentifier{ProjectRoot: gps.ProjectRoot("github.com/sdboyer/deptest")}
rev := gps.Revision("c575196502940c07bf89fd6d95e83b999162e051")
v, err := lookupVersionForLockedProject(pi, nil, rev, sm)
h.Must(err)
wantV := rev.String()
gotV := v.String()
if gotV != wantV {
t.Fatalf("Expected the locked version to be the manifest's pinned revision: wanted '%s', got '%s'", wantV, gotV)
}
}

6
cmd/dep/status.go
Просмотреть файл

@ -174,7 +174,11 @@ func (out *dotOutput) BasicFooter() {
}
func (out *dotOutput) BasicLine(bs *BasicStatus) {
out.g.createNode(bs.ProjectRoot, bs.Version.String(), bs.Children)
version := formatVersion(bs.Revision)
if bs.Version != nil {
version = formatVersion(bs.Version)
}
out.g.createNode(bs.ProjectRoot, version, bs.Children)
}
func (out *dotOutput) MissingHeader() {}

39
cmd/dep/status_test.go
Просмотреть файл

@ -5,8 +5,12 @@
package main
import (
"bytes"
"testing"
"strings"
"github.com/golang/dep"
"github.com/golang/dep/internal/gps"
)
@ -26,3 +30,38 @@ func TestStatusFormatVersion(t *testing.T) {
}
}
}
func TestBasicLine(t *testing.T) {
project := dep.Project{}
var tests = []struct {
status BasicStatus
expected string
}{
{BasicStatus{
Version: nil,
Revision: gps.Revision("flooboofoobooo"),
}, `[label="\nflooboo"];`},
{BasicStatus{
Version: gps.NewVersion("1.0.0"),
Revision: gps.Revision("flooboofoobooo"),
}, `[label="\n1.0.0"];`},
}
for _, test := range tests {
var buf bytes.Buffer
out := &dotOutput{
p: &project,
w: &buf,
}
out.BasicHeader()
out.BasicLine(&test.status)
out.BasicFooter()
if ok := strings.Contains(buf.String(), test.expected); !ok {
t.Fatalf("Did not find expected node label: \n\t(GOT) %v \n\t(WNT) %v", buf.String(), test.status)
}
}
}

3
cmd/dep/testdata/glide/expected_import_output.txt → cmd/dep/testdata/glide/golden.txt поставляемый
Просмотреть файл

@ -2,6 +2,7 @@ Detected glide configuration files...
Converting from glide.yaml and glide.lock...
Using master as initial constraint for imported dep github.com/sdboyer/deptest
Using ^2.0.0 as initial constraint for imported dep github.com/sdboyer/deptestdos
Using master as initial constraint for imported dep github.com/golang/lint
Using * as initial constraint for imported dep github.com/golang/lint
Trying v0.8.1 (3f4c3be) as initial lock for imported dep github.com/sdboyer/deptest
Trying v2.0.0 (5c60720) as initial lock for imported dep github.com/sdboyer/deptestdos
Trying * (cb00e56) as initial lock for imported dep github.com/golang/lint

2
cmd/dep/testdata/godep/expected_import_output.txt поставляемый
Просмотреть файл

@ -1,6 +1,6 @@
Detected godep configuration files...
Converting from Godeps.json ...
Using ^0.8.1 as initial constraint for imported dep github.com/sdboyer/deptest
Trying ^0.8.1 (3f4c3be) as initial lock for imported dep github.com/sdboyer/deptest
Trying v0.8.1 (3f4c3be) as initial lock for imported dep github.com/sdboyer/deptest
Using ^2.0.0 as initial constraint for imported dep github.com/sdboyer/deptestdos
Trying v2.0.0 (5c60720) as initial lock for imported dep github.com/sdboyer/deptestdos

1
cmd/dep/testdata/harness_tests/ensure/empty/case1/final/Gopkg.toml поставляемый
Просмотреть файл

@ -1,4 +1,3 @@
[[constraint]]
name = "github.com/sdboyer/deptest"
version = "1.0.0"

2
cmd/dep/testdata/harness_tests/init/godep/case1/final/Gopkg.lock сгенерированный поставляемый
Просмотреть файл

@ -5,7 +5,7 @@
name = "github.com/sdboyer/deptest"
packages = ["."]
revision = "3f4c3bea144e112a69bbe5d8d01c1b09a544253f"
version = "master"
version = "v0.8.1"
[[projects]]
name = "github.com/sdboyer/deptestdos"

9
cmd/dep/testdata/init/directdeps/main.go поставляемый Normal file
Просмотреть файл

@ -0,0 +1,9 @@
// Copyright 2016 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package main
import _ "github.com/carolynvs/deptest-subpkg/subby"
func main() {}

116
context.go
Просмотреть файл

@ -9,9 +9,7 @@ import (
"os"
"path/filepath"
"runtime"
"strings"
"github.com/Masterminds/vcs"
"github.com/golang/dep/internal/fs"
"github.com/golang/dep/internal/gps"
"github.com/pkg/errors"
@ -43,18 +41,8 @@ type Ctx struct {
Verbose bool // Enables more verbose logging.
}
// SetPaths sets the WorkingDir and GOPATHSs fields.
//
// ctx := &dep.Ctx{
// Out: log.New(os.Stdout, "", 0),
// Err: log.New(os.Stderr, "", 0),
// }
//
// err := ctx.SetPaths(workingDir, filepath.SplitList(os.Getenv("GOPATH"))
// if err != nil {
// // Empty GOPATH
// }
//
// SetPaths sets the WorkingDir and GOPATHs fields. If GOPATHs is empty, then
// the GOPATH environment variable (or the default GOPATH) is used instead.
func (c *Ctx) SetPaths(wd string, GOPATHs ...string) error {
if wd == "" {
return errors.New("cannot set Ctx.WorkingDir to an empty path")
@ -62,24 +50,16 @@ func (c *Ctx) SetPaths(wd string, GOPATHs ...string) error {
c.WorkingDir = wd
if len(GOPATHs) == 0 {
GOPATHs = getGOPATHs(os.Environ())
}
for _, gp := range GOPATHs {
c.GOPATHs = append(c.GOPATHs, filepath.ToSlash(gp))
}
return nil
}
// getGOPATH returns the GOPATHs from the passed environment variables.
// If GOPATH is not defined, fallback to defaultGOPATH().
func getGOPATHs(env []string) []string {
GOPATH := os.Getenv("GOPATH")
if GOPATH == "" {
GOPATH = defaultGOPATH()
}
GOPATHs = filepath.SplitList(GOPATH)
}
return filepath.SplitList(GOPATH)
c.GOPATHs = append(c.GOPATHs, GOPATHs...)
return nil
}
// defaultGOPATH gets the default GOPATH that was added in 1.8
@ -131,9 +111,9 @@ func (c *Ctx) LoadProject() (*Project, error) {
return nil, err
}
ip, err := c.SplitAbsoluteProjectRoot(p.AbsRoot)
ip, err := c.ImportForAbs(p.AbsRoot)
if err != nil {
return nil, errors.Wrap(err, "split absolute project root")
return nil, errors.Wrap(err, "root project import")
}
p.ImportRoot = gps.ProjectRoot(ip)
@ -230,21 +210,16 @@ func (c *Ctx) DetectProjectGOPATH(p *Project) (string, error) {
// detectGOPATH detects the GOPATH for a given path from ctx.GOPATHs.
func (c *Ctx) detectGOPATH(path string) (string, error) {
for _, gp := range c.GOPATHs {
if fs.HasFilepathPrefix(filepath.FromSlash(path), gp) {
if fs.HasFilepathPrefix(path, gp) {
return gp, nil
}
}
return "", errors.Errorf("%s is not within a known GOPATH", path)
}
// SplitAbsoluteProjectRoot takes an absolute path and compares it against the detected
// GOPATH to determine what portion of the input path should be treated as an
// import path - as a project root.
func (c *Ctx) SplitAbsoluteProjectRoot(path string) (string, error) {
if c.GOPATH == "" {
return "", errors.Errorf("no GOPATH detected in this context")
}
// ImportForAbs returns the import path for an absolute project path by trimming the
// `$GOPATH/src/` prefix. Returns an error for paths equal to, or without this prefix.
func (c *Ctx) ImportForAbs(path string) (string, error) {
srcprefix := filepath.Join(c.GOPATH, "src") + string(filepath.Separator)
if fs.HasFilepathPrefix(path, srcprefix) {
if len(path) <= len(srcprefix) {
@ -256,13 +231,13 @@ func (c *Ctx) SplitAbsoluteProjectRoot(path string) (string, error) {
return filepath.ToSlash(path[len(srcprefix):]), nil
}
return "", errors.Errorf("%s not in any GOPATH", path)
return "", errors.Errorf("%s not in GOPATH", path)
}
// absoluteProjectRoot determines the absolute path to the project root
// AbsForImport returns the absolute path for the project root
// including the $GOPATH. This will not work with stdlib packages and the
// package directory needs to exist.
func (c *Ctx) absoluteProjectRoot(path string) (string, error) {
func (c *Ctx) AbsForImport(path string) (string, error) {
posspath := filepath.Join(c.GOPATH, "src", path)
dirOK, err := fs.IsDir(posspath)
if err != nil {
@ -273,62 +248,3 @@ func (c *Ctx) absoluteProjectRoot(path string) (string, error) {
}
return posspath, nil
}
func (c *Ctx) VersionInWorkspace(root gps.ProjectRoot) (gps.Version, error) {
pr, err := c.absoluteProjectRoot(string(root))
if err != nil {
return nil, errors.Wrapf(err, "determine project root for %s", root)
}
repo, err := vcs.NewRepo("", pr)
if err != nil {
return nil, errors.Wrapf(err, "creating new repo for root: %s", pr)
}
ver, err := repo.Current()
if err != nil {
return nil, errors.Wrapf(err, "finding current branch/version for root: %s", pr)
}
rev, err := repo.Version()
if err != nil {
return nil, errors.Wrapf(err, "getting repo version for root: %s", pr)
}
// First look through tags.
tags, err := repo.Tags()
if err != nil {
return nil, errors.Wrapf(err, "getting repo tags for root: %s", pr)
}
// Try to match the current version to a tag.
if contains(tags, ver) {
// Assume semver if it starts with a v.
if strings.HasPrefix(ver, "v") {
return gps.NewVersion(ver).Pair(gps.Revision(rev)), nil
}
return nil, errors.Errorf("version for root %s does not start with a v: %q", pr, ver)
}
// Look for the current branch.
branches, err := repo.Branches()
if err != nil {
return nil, errors.Wrapf(err, "getting repo branch for root: %s")
}
// Try to match the current version to a branch.
if contains(branches, ver) {
return gps.NewBranch(ver).Pair(gps.Revision(rev)), nil
}
return gps.Revision(rev), nil
}
// contains checks if a array of strings contains a value
func contains(a []string, b string) bool {
for _, v := range a {
if b == v {
return true
}
}
return false
}

73
context_test.go
Просмотреть файл

@ -14,7 +14,6 @@ import (
"testing"
"unicode"
"github.com/golang/dep/internal/gps"
"github.com/golang/dep/internal/test"
)
@ -22,7 +21,7 @@ var (
discardLogger = log.New(ioutil.Discard, "", 0)
)
func TestSplitAbsoluteProjectRoot(t *testing.T) {
func TestCtx_ProjectImport(t *testing.T) {
h := test.NewHelper(t)
defer h.Cleanup()
@ -38,7 +37,8 @@ func TestSplitAbsoluteProjectRoot(t *testing.T) {
for _, want := range importPaths {
fullpath := filepath.Join(depCtx.GOPATH, "src", want)
got, err := depCtx.SplitAbsoluteProjectRoot(fullpath)
h.TempDir(filepath.Join("src", want))
got, err := depCtx.ImportForAbs(fullpath)
if err != nil {
t.Fatal(err)
}
@ -48,13 +48,13 @@ func TestSplitAbsoluteProjectRoot(t *testing.T) {
}
// test where it should return an error when directly within $GOPATH/src
got, err := depCtx.SplitAbsoluteProjectRoot(filepath.Join(depCtx.GOPATH, "src"))
got, err := depCtx.ImportForAbs(filepath.Join(depCtx.GOPATH, "src"))
if err == nil || !strings.Contains(err.Error(), "GOPATH/src") {
t.Fatalf("should have gotten an error for use directly in GOPATH/src, but got %s", got)
}
// test where it should return an error
got, err = depCtx.SplitAbsoluteProjectRoot("tra/la/la/la")
got, err = depCtx.ImportForAbs("tra/la/la/la")
if err == nil {
t.Fatalf("should have gotten an error but did not for tra/la/la/la: %s", got)
}
@ -80,7 +80,7 @@ func TestAbsoluteProjectRoot(t *testing.T) {
}
for i, ok := range importPaths {
got, err := depCtx.absoluteProjectRoot(i)
got, err := depCtx.AbsForImport(i)
if ok {
h.Must(err)
want := h.Path(filepath.Join("src", i))
@ -97,57 +97,12 @@ func TestAbsoluteProjectRoot(t *testing.T) {
// test that a file fails
h.TempFile("src/thing/thing.go", "hello world")
_, err := depCtx.absoluteProjectRoot("thing/thing.go")
_, err := depCtx.AbsForImport("thing/thing.go")
if err == nil {
t.Fatal("error should not be nil for a file found")
}
}
func TestVersionInWorkspace(t *testing.T) {
test.NeedsExternalNetwork(t)
test.NeedsGit(t)
h := test.NewHelper(t)
defer h.Cleanup()
h.TempDir("src")
h.Setenv("GOPATH", h.Path("."))
depCtx := &Ctx{GOPATH: h.Path(".")}
importPaths := map[string]struct {
rev gps.Version
checkout bool
}{
"github.com/pkg/errors": {
rev: gps.NewVersion("v0.8.0").Pair("645ef00459ed84a119197bfb8d8205042c6df63d"), // semver
checkout: true,
},
"github.com/Sirupsen/logrus": {
rev: gps.Revision("42b84f9ec624953ecbf81a94feccb3f5935c5edf"), // random sha
checkout: true,
},
"github.com/rsc/go-get-default-branch": {
rev: gps.NewBranch("another-branch").Pair("8e6902fdd0361e8fa30226b350e62973e3625ed5"),
},
}
// checkout the specified revisions
for ip, info := range importPaths {
h.RunGo("get", ip)
repoDir := h.Path("src/" + ip)
if info.checkout {
h.RunGit(repoDir, "checkout", info.rev.String())
}
got, err := depCtx.VersionInWorkspace(gps.ProjectRoot(ip))
h.Must(err)
if got != info.rev {
t.Fatalf("expected %q, got %q", got.String(), info.rev.String())
}
}
}
func TestLoadProject(t *testing.T) {
h := test.NewHelper(t)
defer h.Cleanup()
@ -345,7 +300,8 @@ func TestCaseInsentitiveGOPATH(t *testing.T) {
ip := "github.com/pkg/errors"
fullpath := filepath.Join(depCtx.GOPATH, "src", ip)
pr, err := depCtx.SplitAbsoluteProjectRoot(fullpath)
h.TempDir(filepath.Join("src", ip))
pr, err := depCtx.ImportForAbs(fullpath)
if err != nil {
t.Fatal(err)
}
@ -366,13 +322,14 @@ func TestDetectProjectGOPATH(t *testing.T) {
}
h.TempDir("go/src/real/path")
h.TempDir("go/src/sym")
// Another directory used as a GOPATH
h.TempDir("go-two/src/real/path")
h.TempDir("go-two/src/sym")
h.TempDir("sym") // Directory for symlinks
h.TempDir(filepath.Join(".", "sym/symlink")) // Directory for symlinks
h.TempDir(filepath.Join("go", "src", "sym", "path"))
h.TempDir(filepath.Join("go", "src", " real", "path"))
h.TempDir(filepath.Join("go-two", "src", "real", "path"))
testcases := []struct {
name string
@ -463,6 +420,10 @@ func TestDetectGOPATH(t *testing.T) {
th.Path("gotwo"),
}}
th.TempDir(filepath.Join("code", "src", "github.com", "username", "package"))
th.TempDir(filepath.Join("go", "src", "github.com", "username", "package"))
th.TempDir(filepath.Join("gotwo", "src", "github.com", "username", "package"))
testcases := []struct {
GOPATH string
path string

230
FAQ.md → docs/FAQ.md
Просмотреть файл

@ -6,35 +6,38 @@ _The first rule of FAQ is don't bikeshed the FAQ, leave that for
Please contribute to the FAQ! Found an explanation in an issue or pull request helpful?
Summarize the question and quote the reply, linking back to the original comment.
## Concepts
* [Does `dep` replace `go get`?](#does-dep-replace-go-get)
* [Why is it `dep ensure` instead of `dep install`?](#why-is-it-dep-ensure-instead-of-dep-install)
* [What is the difference between Gopkg.toml (the "manifest") and Gopkg.lock (the "lock")?](#what-is-the-difference-between-gopkgtoml-the-manifest-and-gopkglock-the-lock)
* [When should I use `constraint`, `override` `required`, or `ignored` in the Gopkg.toml?](#when-should-i-use-constraint-override-required-or-ignored-in-gopkgtoml)
* [What is a direct or transitive dependency?](#what-is-a-direct-or-transitive-dependency)
* [How does `dep` decide what version of a dependency to use?](#how-does-dep-decide-what-version-of-a-dependency-to-use)
* [Why is `dep` ignoring a version constraint in the manifest?](#why-is-dep-ignoring-a-version-constraint-in-the-manifest)
## Configuration
* [What is the difference between Gopkg.toml (the "manifest") and Gopkg.lock (the "lock")?](#what-is-the-difference-between-gopkgtoml-the-manifest-and-gopkglock-the-lock)
* [How do I constrain a transitive dependency's version?](#how-do-i-constrain-a-transitive-dependencys-version)
* [Why did `dep` use a different revision for package X instead of the revision in the lock file?](#why-did-dep-use-a-different-revision-for-package-x-instead-of-the-revision-in-the-lock-file)
* [Should I commit my vendor directory?](#should-i-commit-my-vendor-directory)
* [`dep` deleted my files in the vendor directory!](#dep-deleted-my-files-in-the-vendor-directory)
* [Can I put the manifest and lock in the vendor directory?](#can-i-put-the-manifest-and-lock-in-the-vendor-directory)
* [How can I test changes to a dependency?](#how-can-i-test-changes-to-a-dependency)
* [How do I get `dep` to authenticate to a `git` repo?](#how-do-i-get-dep-to-authenticate-to-a-git-repo)
## Behavior
* [How does `dep` decide what version of a dependency to use?](#how-does-dep-decide-what-version-of-a-dependency-to-use)
* [What external tools are supported?](#what-external-tools-are-supported)
* [Why is `dep` ignoring a version constraint in the manifest?](#why-is-dep-ignoring-a-version-constraint-in-the-manifest)
* [Why did `dep` use a different revision for package X instead of the revision in the lock file?](#why-did-dep-use-a-different-revision-for-package-x-instead-of-the-revision-in-the-lock-file)
* [Why is `dep` slow?](#why-is-dep-slow)
* [How does `dep` handle symbolic links?](#how-does-dep-handle-symbolic-links)
## Best Practices
* [Should I commit my vendor directory?](#should-i-commit-my-vendor-directory)
* [How do I roll releases that `dep` will be able to use?](#how-do-i-roll-releases-that-dep-will-be-able-to-use)
* [What semver version should I use?](#what-semver-version-should-i-use)
* [Is it OK to make backwards-incompatible changes now?](#is-it-ok-to-make-backwards-incompatible-changes-now)
* [My dependers don't use `dep` yet. What should I do?](#my-dependers-dont-use-dep-yet-what-should-i-do)
## Does `dep` replace `go get`?
___
No, `dep` is an experiment and is still in its infancy. Depending on how this
experiment goes, it may be considered for inclusion in the go project in some form
or another in the future but that is not guaranteed.
## Concepts
### Does `dep` replace `go get`?
No. `dep` and `go get` serve mostly different purposes.
Here are some suggestions for when you could use `dep` or `go get`:
> I would say that dep doesn't replace go get, but they both can do similar things. Here's how I use them:
@ -48,7 +51,7 @@ Here are some suggestions for when you could use `dep` or `go get`:
> is for people consuming Go code, and dep-family commands are for people developing it.
-[@sdboyer in #376](https://github.com/golang/dep/issues/376#issuecomment-294045873)
## Why is it `dep ensure` instead of `dep install`?
### Why is it `dep ensure` instead of `dep install`?
> Yeah, we went round and round on names. [A lot](https://gist.github.com/jessfraz/315db91b272441f510e81e449f675a8b).
>
@ -57,25 +60,99 @@ Here are some suggestions for when you could use `dep` or `go get`:
> We opted for this approach because we came to the conclusion that allowing the tool to perform partial work/exit in intermediate states ended up creating a tool that had more commands, had far more possible valid exit and input states, and was generally full of footguns. In this approach, the user has most of the same ultimate control, but exercises it differently (by modifying the code/manifest and re-running dep ensure).
-[@sdboyer in #371](https://github.com/golang/dep/issues/371#issuecomment-293246832)
### What is a direct or transitive dependency?
* Direct dependencies are dependencies that are imported directly by your project: they appear in at least one import statement from your project.
* Transitive dependencies are the dependencies of your dependencies. Necessary to compile but are not directly used by your code.
## What is the difference between `Gopkg.toml` (the "manifest") and `Gopkg.lock` (the "lock")?
## Configuration
### What is the difference between `Gopkg.toml` (the "manifest") and `Gopkg.lock` (the "lock")?
> The manifest describes user intent, and the lock describes computed outputs. There's flexibility in manifests that isn't present in locks..., as the "branch": "master" constraint will match whatever revision master HAPPENS to be at right now, whereas the lock is nailed down to a specific revision.
>
> This flexibility is important because it allows us to provide easy commands (e.g. `dep ensure -update`) that can manage an update process for you, within the constraints you specify, AND because it allows your project, when imported by someone else, to collaboratively specify the constraints for your own dependencies.
-[@sdboyer in #281](https://github.com/golang/dep/issues/281#issuecomment-284118314)
## When should I use `constraint`, `override`, `required`, or `ignored` in `Gopkg.toml`?
## How do I constrain a transitive dependency's version?
First, if you're wondering about this because you're trying to keep the version
of the transitive dependency from changing, then you're working against `dep`'s
design. The lock file, `Gopkg.lock`, will keep the selected version of the
transitive dependency stable, unless you explicitly request an upgrade or it's
impossible to find a solution without changing that version.
* Use `constraint` to constrain a [direct dependency](#what-is-a-direct-or-transitive-dependency) to a specific branch, version range, revision, or specify an alternate source such as a fork.
* Use `override` to constrain a [transitive dependency](#what-is-a-direct-or-transitive-dependency). See [How do I constrain a transitive dependency's version?](#how-do-i-constrain-a-transitive-dependencys-version) for more details on how overrides differ from constraints. Overrides should be used cautiously, sparingly, and temporarily.
* Use `required` to explicitly add a dependency that is not imported directly or transitively, for example a development package used for code generation.
* Use `ignored` to ignore a package and any of that package's unique dependencies.
If that isn't your use case and you still need to constrain a transitive
dependency, you have a couple of options:
## What is a direct or transitive dependency?
* Direct dependencies are dependencies that are imported directly by your project: they appear in at least one import statement from your project.
* Transitive dependencies are the dependencies of your dependencies. Necessary to compile but are not directly used by your code.
## How does `dep` decide what version of a dependency to use?
1. Make the transitive dependency a direct one, either with a dummy import or an entry in the `required` list in `Gopkg.toml`.
2. Use an override.
Overrides are a sledgehammer, and should only be used as a last resort. While
constraints and overrides are declared in the same way in `Gopkg.toml`, they
behave differently:
* Constraints:
1. Can be declared by any project's manifest, yours or a dependency
2. Apply only to direct dependencies of the project declaring the constraint
3. Must not conflict with the `constraint` entries declared in any other project's manifest
* Overrides:
1. Are only utilized from the current/your project's manifest
2. Apply globally, to direct and transitive dependencies
3. Supersede constraints declared in all manifests, yours or a dependency's
Overrides are also discussed with some visuals in [the gps docs](https://github.com/sdboyer/gps/wiki/gps-for-Implementors#overrides).
## Can I put the manifest and lock in the vendor directory?
No.
> Placing these files inside `vendor/` would concretely bind us to `vendor/` in the long term.
> We prefer to treat the `vendor/` as an implementation detail.
-[@sdboyer on go package management list](https://groups.google.com/d/msg/go-package-management/et1qFUjrkP4/LQFCHP4WBQAJ)
## How do I get dep to authenticate to a git repo?
`dep` currently uses the `git` command under the hood, so configuring the credentials
for each repository you wish to authenticate to will allow `dep` to use an
authenticated repository.
First, configure `git` to use the credentials option for the specific repository.
For example, if you use gitlab, and you wish to access `https://gitlab.example.com/example/package.git`,
then you would want to use the following configuration:
```
$ git config --global credential.https://gitlab.example.com.example yourusername
```
In the example the hostname `gitlab.example.com.username` string seems incorrect, but
it's actually the hostname plus the name of the repo you are accessing which is `username`.
The trailing 'yourusername' is the username you would use for the actual authentication.
You also need to configure `git` with the authentication provider you wish to use. You can get
a list of providers, with the command:
```
$ git help -a | grep credential-
credential-cache remote-fd
credential-cache--daemon remote-ftp
credential-osxkeychain remote-ftps
credential-store remote-http
```
You would then choose an appropriate provider. For example, to use the osxkeychain, you
would use the following:
```
git config --global credential.helper osxkeychain
```
If you need to do this for a CI system, then you may want to use the "store" provider.
Please see the documentation on how to configure that: https://git-scm.com/docs/git-credential-store
After configuring `git`, you may need to use `git` manually once to have it store the
credentials. Once you've checked out the repo manually, it will then use the stored
credentials. This at least appears to be the behavior for the osxkeychain provider.
## Behavior
### How does `dep` decide what version of a dependency to use?
The full algorithm is complex, but the most important thing to understand is
that `dep` tries versions in a [certain
@ -112,57 +189,26 @@ trying to figure out why `dep` is doing what it does, understanding that its
basic action is to attempt versions in this order should help you to reason
about what's going on.
## What external tools are supported?
During `dep init` configuration from other dependency managers is detected
and imported, unless `-skip-tools` is specified.
The following tools are supported: `glide` and `godep`.
See [#186](https://github.com/golang/dep/issues/186#issuecomment-306363441) for
how to add support for another tool.
## Why is `dep` ignoring a version constraint in the manifest?
Only your project's directly imported dependencies are affected by a `constraint` entry
in the manifest. Transitive dependencies are unaffected.
Use an `override` entry for transitive dependencies.
By default, when you specify a version without an operator, such as `~` or `=`,
`dep` automatically adds a caret operator, `^`. The caret operator pins the
left-most non-zero digit in the version. For example:
```
^1.2.3 means 1.2.3 <= X < 2.0.0
^0.2.3 means 0.2.3 <= X < 0.3.0
^0.0.3 means 0.0.3 <= X < 0.0.4
```
To pin a version of direct dependency in manifest, prefix the version with `=`.
For example:
```
[[constraint]]
[[override]]
name = "github.com/pkg/errors"
version = "=0.8.0"
```
## How do I constrain a transitive dependency's version?
First, if you're wondering about this because you're trying to keep the version
of the transitive dependency from changing, then you're working against `dep`'s
design. The lock file, `Gopkg.lock`, will keep the selected version of the
transitive dependency stable, unless you explicitly request an upgrade or it's
impossible to find a solution without changing that version.
If that isn't your use case and you still need to constrain a transitive
dependency, you have a couple of options:
1. Make the transitive dependency a direct one, either with a dummy import or an entry in the `required` list in `Gopkg.toml`.
2. Use an override.
Overrides are a sledgehammer, and should only be used as a last resort. While
constraints and overrides are declared in the same way in `Gopkg.toml`, they
behave differently:
* Constraints:
1. Can be declared by any project's manifest, yours or a dependency
2. Apply only to direct dependencies of the project declaring the constraint
3. Must not conflict with the `constraint` entries declared in any other project's manifest
* Overrides:
1. Are only utilized from the current/your project's manifest
2. Apply globally, to direct and transitive dependencies
3. Supersede constraints declared in all manifests, yours or a dependency's
Overrides are also discussed with some visuals in [the gps docs](https://github.com/sdboyer/gps/wiki/gps-for-Implementors#overrides).
## Why did `dep` use a different revision for package X instead of the revision in the lock file?
Sometimes the revision specified in the lock file is no longer valid. There are a few
ways this can occur:
@ -183,43 +229,6 @@ Unable to update checked out version: fatal: reference is not a tree: 4dfc6a8a7e
> Under most circumstances, if those arguments don't change, then the lock remains fine and correct. You've hit one one of the few cases where that guarantee doesn't apply. The fact that you ran dep ensure and it DID a solve is a product of some arguments changing; that solving failed because this particular commit had become stale is a separate problem.
-[@sdboyer in #405](https://github.com/golang/dep/issues/405#issuecomment-295998489)
## Should I commit my vendor directory?
It's up to you:
**Pros**
- it's the only way to get truly reproducible builds, as it guards against upstream renames and deletes
- you don't need an extra `dep ensure` step (to fetch dependencies) on fresh clones to build your repo
**Cons**
- your repo will be bigger, potentially a lot bigger
- PR diffs are more annoying
## `dep` deleted my files in the vendor directory!
If you just ran `dep init`, there should be a copy of your original vendor directory named `_vendor-TIMESTAMP` in your project root. The other commands do not make a backup before modifying the vendor directory.
> dep assumes complete control of vendor/, and may indeed blow things away if it feels like it.
-[@peterbourgon in #206](https://github.com/golang/dep/issues/206#issuecomment-277139419)
## Can I put the manifest and lock in the vendor directory?
No.
> Placing these files inside `vendor/` would concretely bind us to `vendor/` in the long term.
> We prefer to treat the `vendor/` as an implementation detail.
-[@sdboyer on go package management list](https://groups.google.com/d/msg/go-package-management/et1qFUjrkP4/LQFCHP4WBQAJ)
## How can I test changes to a dependency?
> I would recommend against ever working in your vendor directory since dep will overwrite any changes. It’s too easy to lose work that way.
-[@carolynvs in #706](https://github.com/golang/dep/issues/706#issuecomment-305807261)
If you have a fork, add a `[[constraint]]` entry for the project in `Gopkg.toml` and set `source` to the fork source. This will ensure that `dep` will fetch the project from the fork instead of the original source.
Otherwise, if you want to test changes locally, you can delete the package from `vendor/` and make changes directly in `GOPATH/src/*package*` so that your changes are picked up by the go tool chain.
## Why is `dep` slow?
There are two things that really slow `dep` down. One is unavoidable; for the other, we have a plan.
@ -269,6 +278,21 @@ When `dep` is invoked with a project root that is a symlink, it will be resolved
This is the only symbolic link support that `dep` really intends to provide. In keeping with the general practices of the `go` tool, `dep` tends to either ignore symlinks (when walking) or copy the symlink itself, depending on the filesystem operation being performed.
## Best Practices
### Should I commit my vendor directory?
It's up to you:
**Pros**
- it's the only way to get truly reproducible builds, as it guards against upstream renames and deletes
- you don't need an extra `dep ensure` step (to fetch dependencies) on fresh clones to build your repo
**Cons**
- your repo will be bigger, potentially a lot bigger
- PR diffs are more annoying
## How do I roll releases that `dep` will be able to use?
In short: make sure you've committed your `Gopkg.toml` and `Gopkg.lock`, then

168
docs/Gopkg.toml.md Normal file
Просмотреть файл

@ -0,0 +1,168 @@
# Gopkg.toml
## `required`
`required` lists a set of packages (not projects) that must be included in
Gopkg.lock. This list is merged with the set of packages imported by the current
project.
```toml
required = ["github.com/user/thing/cmd/thing"]
```
**Use this for:** linters, generators, and other development tools that
* Are needed by your project
* Aren't `import`ed by your project, [directly or transitively](FAQ.md#what-is-a-direct-or-transitive-dependency)
* You don't want put in your `GOPATH`, and/or you want to lock the version
Please note that this only pulls in the sources of these dependencies. It does not install or compile them. So, if you need the tool to be installed you should still run the following (manually or from a `Makefile`) after each `dep ensure`:
```bash
cd vendor/pkg/to/install
go install .
```
This only works reliably if this is the only project to install these executables. This is not enough if you want to be able to run a different version of the same executable depending on the project you're working. In that case you have to use a different `GOBIN` for each project, by doing something like this before running the above commands:
```bash
export GOBIN=$PWD/bin
export PATH=$GOBIN:$PATH
```
You might also try [virtualgo](https://github.com/GetStream/vg), which installs dependencies in the `required` list automatically in a project specific `GOBIN`.
## `ignored`
`ignored` lists a set of packages (not projects) that are ignored when dep statically analyzes source code. Ignored packages can be in this project, or in a dependency.
```toml
ignored = ["github.com/user/project/badpkg"]
```
**Use this for:** preventing a package and any of that package's unique
dependencies from being installed.
## `metadata`
`metadata` can exist at the root as well as under `constraint` and `override` declarations.
`metadata` declarations are ignored by dep and are meant for usage by other independent systems.
A `metadata` declaration at the root defines metadata about the project itself. While a `metadata` declaration under a `constraint` or an `override` defines metadata about that `constraint` or `override`.
```toml
[metadata]
key1 = "value that convey data to other systems"
system1-data = "value that is used by a system"
system2-data = "value that is used by another system"
```
## `constraint`
A `constraint` provides rules for how a [direct dependency](FAQ.md#what-is-a-direct-or-transitive-dependency) may be incorporated into the
dependency graph.
They are respected by dep whether coming from the Gopkg.toml of the current project or a dependency.
```toml
[[constraint]]
# Required: the root import path of the project being constrained.
name = "github.com/user/project"
# Recommended: the version constraint to enforce for the project.
# Only one of "branch", "version" or "revision" can be specified.
version = "1.0.0"
branch = "master"
revision = "abc123"
# Optional: an alternate location (URL or import path) for the project's source.
source = "https://github.com/myfork/package.git"
# Optional: metadata about the constraint or override that could be used by other independent systems
[metadata]
key1 = "value that convey data to other systems"
system1-data = "value that is used by a system"
system2-data = "value that is used by another system"
```
**Use this for:** having a [direct dependency](FAQ.md#what-is-a-direct-or-transitive-dependency)
use a specific branch, version range, revision, or alternate source (such as a
fork).
## `override`
An `override` has the same structure as a `constraint` declaration, but supersede all `constraint` declarations from all projects. Only `override` declarations from the current project's are applied.
```toml
[[override]]
# Required: the root import path of the project being constrained.
name = "github.com/user/project"
# Optional: specifying a version constraint override will cause all other constraints on this project to be ignored; only the overridden constraint needs to be satisfied. Again, only one of "branch", "version" or "revision" can be specified.
version = "1.0.0"
branch = "master"
revision = "abc123"
# Optional: specifying an alternate source location as an override will enforce that the alternate location is used for that project, regardless of what source location any dependent projects specify.
source = "https://github.com/myfork/package.git"
# Optional: metadata about the constraint or override that could be used by other independent systems
[metadata]
key1 = "value that convey data to other systems"
system1-data = "value that is used by a system"
system2-data = "value that is used by another system"
```
**Use this for:** all the same things as a [`constraint`](#constraint), but for
[transitive dependencies](FAQ.md#what-is-a-direct-or-transitive-dependency).
See [How do I constrain a transitive dependency's version?](FAQ.md#how-do-i-constrain-a-transitive-dependencys-version)
for more details on how overrides differ from `constraint`s. _Overrides should
be used cautiously, sparingly, and temporarily._
## `version`
`version` is a property of `constraint`s and `override`s. It is used to specify
version constraint of a specific dependency.
Internally, dep uses [Masterminds/semver](https://github.com/Masterminds/semver)
to work with semver versioning.
`~` and `=` operators can be used with the versions. When a version is specified
without any operator, `dep` automatically adds a caret operator, `^`. The caret
operator pins the left-most non-zero digit in the version. For example:
```
^1.2.3 means 1.2.3 <= X < 2.0.0
^0.2.3 means 0.2.3 <= X < 0.3.0
^0.0.3 means 0.0.3 <= X < 0.1.0
```
To pin a version of direct dependency in manifest, prefix the version with `=`.
For example:
```toml
[[constraint]]
name = "github.com/pkg/errors"
version = "=0.8.0"
```
[Why is dep ignoring a version constraint in the manifest?](FAQ.md#why-is-dep-ignoring-a-version-constraint-in-the-manifest)
# Example
Here's an example of a sample Gopkg.toml with most of the elements
```toml
required = ["github.com/user/thing/cmd/thing"]
ignored = ["github.com/user/project/pkgX", "bitbucket.org/user/project/pkgA/pkgY"]
[metadata]
codename = "foo"
[[constraint]]
name = "github.com/user/project"
version = "1.0.0"
[metadata]
property1 = "value1"
property2 = 10
[[constraint]]
name = "github.com/user/project2"
branch = "dev"
source = "github.com/myfork/project2"
[[override]]
name = "github.com/x/y"
version = "2.4.0"
[metadata]
propertyX = "valueX"
```

5
internal/feedback/feedback.go
Просмотреть файл

@ -73,7 +73,7 @@ func (cf ConstraintFeedback) LogFeedback(logger *log.Logger) {
if cf.Constraint != "" {
logger.Printf(" %v", GetUsingFeedback(cf.Constraint, cf.ConstraintType, cf.DependencyType, cf.ProjectPath))
}
if cf.LockedVersion != "" && cf.Revision != "" {
if cf.Revision != "" {
logger.Printf(" %v", GetLockingFeedback(cf.LockedVersion, cf.Revision, cf.DependencyType, cf.ProjectPath))
}
}
@ -104,6 +104,9 @@ func GetLockingFeedback(version, revision, depType, projectPath string) string {
}
if depType == DepTypeImported {
if version == "" {
version = "*"
}
return fmt.Sprintf("Trying %s (%s) as initial lock for %s %s", version, revision, depType, projectPath)
}
return fmt.Sprintf("Locking in %s (%s) for %s %s", version, revision, depType, projectPath)

8
internal/feedback/feedback_test.go
Просмотреть файл

@ -30,6 +30,10 @@ func TestFeedback_Constraint(t *testing.T) {
feedback: NewConstraintFeedback(gps.ProjectConstraint{Constraint: ver, Ident: pi}, DepTypeImported),
want: "Using ^1.0.0 as initial constraint for imported dep github.com/foo/bar",
},
{
feedback: NewConstraintFeedback(gps.ProjectConstraint{Constraint: gps.Any(), Ident: pi}, DepTypeImported),
want: "Using * as initial constraint for imported dep github.com/foo/bar",
},
{
feedback: NewConstraintFeedback(gps.ProjectConstraint{Constraint: rev, Ident: pi}, DepTypeDirect),
want: "Using 1b8edb3 as hint for direct dep github.com/foo/bar",
@ -68,6 +72,10 @@ func TestFeedback_LockedProject(t *testing.T) {
feedback: NewLockedProjectFeedback(gps.NewLockedProject(pi, v, nil), DepTypeImported),
want: "Trying v1.1.4 (bc29b4f) as initial lock for imported dep github.com/foo/bar",
},
{
feedback: NewLockedProjectFeedback(gps.NewLockedProject(pi, gps.NewVersion("").Pair("bc29b4f"), nil), DepTypeImported),
want: "Trying * (bc29b4f) as initial lock for imported dep github.com/foo/bar",
},
{
feedback: NewLockedProjectFeedback(gps.NewLockedProject(pi, b, nil), DepTypeTransitive),
want: "Locking in master (436f39d) for transitive dep github.com/foo/bar",

7
internal/fs/fs.go
Просмотреть файл

@ -334,9 +334,6 @@ func copySymlink(src, dst string) error {
func IsDir(name string) (bool, error) {
// TODO: lstat?
fi, err := os.Stat(name)
if os.IsNotExist(err) {
return false, nil
}
if err != nil {
return false, err
}
@ -349,8 +346,10 @@ func IsDir(name string) (bool, error) {
// IsNonEmptyDir determines if the path given is a non-empty directory or not.
func IsNonEmptyDir(name string) (bool, error) {
isDir, err := IsDir(name)
if !isDir || err != nil {
if err != nil && !os.IsNotExist(err) {
return false, err
} else if !isDir {
return false, nil
}
// Get file descriptor

73
internal/fs/fs_test.go
Просмотреть файл

@ -112,7 +112,7 @@ func TestHasFilepathPrefix_Files(t *testing.T) {
}
existingFile := filepath.Join(dir, "exists")
if _, err := os.Create(existingFile); err != nil {
if err = os.MkdirAll(existingFile, 0755); err != nil {
t.Fatal(err)
}
@ -123,8 +123,8 @@ func TestHasFilepathPrefix_Files(t *testing.T) {
prefix string
want bool
}{
{existingFile, filepath.Join(dir2), false},
{nonExistingFile, filepath.Join(dir2), true},
{existingFile, filepath.Join(dir2), true},
{nonExistingFile, filepath.Join(dir2), false},
}
for _, c := range cases {
@ -733,7 +733,7 @@ func TestIsDir(t *testing.T) {
}{
wd: {true, false},
filepath.Join(wd, "testdata"): {true, false},
filepath.Join(wd, "main.go"): {false, false},
filepath.Join(wd, "main.go"): {false, true},
filepath.Join(wd, "this_file_does_not_exist.thing"): {false, true},
dn: {false, true},
}
@ -748,14 +748,9 @@ func TestIsDir(t *testing.T) {
for f, want := range tests {
got, err := IsDir(f)
if err != nil {
if want.exists != got {
t.Fatalf("expected %t for %s, got %t", want.exists, f, got)
}
if !want.err {
if err != nil && !want.err {
t.Fatalf("expected no error, got %v", err)
}
}
if got != want.exists {
t.Fatalf("expected %t for %s, got %t", want.exists, f, got)
@ -763,7 +758,7 @@ func TestIsDir(t *testing.T) {
}
}
func TestIsEmpty(t *testing.T) {
func TestIsNonEmptyDir(t *testing.T) {
wd, err := os.Getwd()
if err != nil {
t.Fatal(err)
@ -773,33 +768,47 @@ func TestIsEmpty(t *testing.T) {
defer h.Cleanup()
h.TempDir("empty")
tests := map[string]string{
wd: "true",
"testdata": "true",
filepath.Join(wd, "fs.go"): "err",
filepath.Join(wd, "this_file_does_not_exist.thing"): "false",
h.Path("empty"): "false",
testCases := []struct {
path string
empty bool
err bool
}{
{wd, true, false},
{"testdata", true, false},
{filepath.Join(wd, "fs.go"), false, true},
{filepath.Join(wd, "this_file_does_not_exist.thing"), false, false},
{h.Path("empty"), false, false},
}
for f, want := range tests {
empty, err := IsNonEmptyDir(f)
if want == "err" {
if err == nil {
t.Fatalf("Wanted an error for %v, but it was nil", f)
}
if empty {
t.Fatalf("Wanted false with error for %v, but got true", f)
}
} else if err != nil {
t.Fatalf("Wanted no error for %v, got %v", f, err)
// This test case doesn't work on Microsoft Windows because of the
// differences in how file permissions are implemented.
if runtime.GOOS != "windows" {
var inaccessibleDir string
cleanup := setupInaccessibleDir(t, func(dir string) error {
inaccessibleDir = filepath.Join(dir, "empty")
return os.Mkdir(inaccessibleDir, 0777)
})
defer cleanup()
testCases = append(testCases, struct {
path string
empty bool
err bool
}{inaccessibleDir, false, true})
}
if want == "true" && !empty {
t.Fatalf("Wanted true for %v, but got false", f)
for _, want := range testCases {
got, err := IsNonEmptyDir(want.path)
if want.err && err == nil {
if got {
t.Fatalf("wanted false with error for %v, but got true", want.path)
}
t.Fatalf("wanted an error for %v, but it was nil", want.path)
}
if want == "false" && empty {
t.Fatalf("Wanted false for %v, but got true", f)
if got != want.empty {
t.Fatalf("wanted %t for %v, but got %t", want.empty, want.path, got)
}
}
}

2
internal/gps/bridge.go
Просмотреть файл

@ -201,8 +201,6 @@ func (b *bridge) breakLock() {
for _, lp := range b.s.rd.rl.Projects() {
if _, is := b.s.sel.selected(lp.pi); !is {
// TODO(sdboyer) use this as an opportunity to detect
// inconsistencies between upstream and the lock (e.g., moved tags)?
pi, v := lp.pi, lp.Version()
go func() {
// Sync first

130
internal/gps/cmd.go
Просмотреть файл

@ -10,6 +10,7 @@ import (
"fmt"
"os/exec"
"sync"
"sync/atomic"
"time"
"github.com/Masterminds/vcs"
@ -25,6 +26,18 @@ type monitoredCmd struct {
stderr *activityBuffer
}
// noProgressError indicates that the monitored process was terminated due to
// exceeding exceeding the progress timeout.
type noProgressError struct {
timeout time.Duration
}
// killCmdError indicates that an error occurred while sending a kill signal to
// the monitored process.
type killCmdError struct {
err error
}
func newMonitoredCmd(cmd *exec.Cmd, timeout time.Duration) *monitoredCmd {
stdout, stderr := newActivityBuffer(), newActivityBuffer()
cmd.Stdout, cmd.Stderr = stdout, stderr
@ -37,46 +50,73 @@ func newMonitoredCmd(cmd *exec.Cmd, timeout time.Duration) *monitoredCmd {
}
// run will wait for the command to finish and return the error, if any. If the
// command does not show any activity for more than the specified timeout the
// process will be killed.
// command does not show any progress, as indicated by writing to stdout or
// stderr, for more than the specified timeout, the process will be killed.
func (c *monitoredCmd) run(ctx context.Context) error {
// Check for cancellation before even starting
if ctx.Err() != nil {
return ctx.Err()
}
ticker := time.NewTicker(c.timeout)
done := make(chan error, 1)
defer ticker.Stop()
err := c.cmd.Start()
if err != nil {
return err
}
ticker := time.NewTicker(c.timeout)
defer ticker.Stop()
// Atomic marker to track proc exit state. Guards against bad channel
// select receive order, where a tick or context cancellation could come
// in at the same time as process completion, but one of the former are
// picked first; in such a case, cmd.Process could(?) be nil by the time we
// call signal methods on it.
var isDone *int32 = new(int32)
done := make(chan error, 1)
go func() {
// Wait() can only be called once, so this must act as the completion
// indicator for both normal *and* signal-induced termination.
done <- c.cmd.Wait()
atomic.CompareAndSwapInt32(isDone, 0, 1)
}()
var killerr error
selloop:
for {
select {
case <-ticker.C:
if c.hasTimedOut() {
if err := c.cmd.Process.Kill(); err != nil {
return &killCmdError{err}
}
return &timeoutError{c.timeout}
}
case <-ctx.Done():
if err := c.cmd.Process.Kill(); err != nil {
return &killCmdError{err}
}
return ctx.Err()
case err := <-done:
return err
case <-ticker.C:
if !atomic.CompareAndSwapInt32(isDone, 1, 1) && c.hasTimedOut() {
if err := killProcess(c.cmd, isDone); err != nil {
killerr = &killCmdError{err}
} else {
killerr = &noProgressError{c.timeout}
}
break selloop
}
case <-ctx.Done():
if !atomic.CompareAndSwapInt32(isDone, 1, 1) {
if err := killProcess(c.cmd, isDone); err != nil {
killerr = &killCmdError{err}
} else {
killerr = ctx.Err()
}
break selloop
}
}
}
// This is only reachable on the signal-induced termination path, so block
// until a message comes through the channel indicating that the command has
// exited.
//
// TODO(sdboyer) if the signaling process errored (resulting in a
// killCmdError stored in killerr), is it possible that this receive could
// block forever on some kind of hung process?
<-done
return killerr
}
func (c *monitoredCmd) hasTimedOut() bool {
@ -87,10 +127,11 @@ func (c *monitoredCmd) hasTimedOut() bool {
func (c *monitoredCmd) combinedOutput(ctx context.Context) ([]byte, error) {
if err := c.run(ctx); err != nil {
return c.stderr.buf.Bytes(), err
return c.stderr.Bytes(), err
}
return c.stdout.buf.Bytes(), nil
// FIXME(sdboyer) this is not actually combined output
return c.stdout.Bytes(), nil
}
// activityBuffer is a buffer that keeps track of the last time a Write
@ -109,39 +150,58 @@ func newActivityBuffer() *activityBuffer {
func (b *activityBuffer) Write(p []byte) (int, error) {
b.Lock()
b.lastActivityStamp = time.Now()
defer b.Unlock()
b.lastActivityStamp = time.Now()
return b.buf.Write(p)
}
func (b *activityBuffer) String() string {
b.Lock()
defer b.Unlock()
return b.buf.String()
}
func (b *activityBuffer) Bytes() []byte {
b.Lock()
defer b.Unlock()
return b.buf.Bytes()
}
func (b *activityBuffer) lastActivity() time.Time {
b.Lock()
defer b.Unlock()
return b.lastActivityStamp
}
type timeoutError struct {
timeout time.Duration
}
func (e timeoutError) Error() string {
func (e noProgressError) Error() string {
return fmt.Sprintf("command killed after %s of no activity", e.timeout)
}
type killCmdError struct {
err error
}
func (e killCmdError) Error() string {
return fmt.Sprintf("error killing command: %s", e.err)
}
func runFromCwd(ctx context.Context, cmd string, args ...string) ([]byte, error) {
c := newMonitoredCmd(exec.Command(cmd, args...), 2*time.Minute)
func runFromCwd(ctx context.Context, timeout time.Duration, cmd string, args ...string) ([]byte, error) {
c := newMonitoredCmd(exec.Command(cmd, args...), timeout)
return c.combinedOutput(ctx)
}
func runFromRepoDir(ctx context.Context, repo vcs.Repo, cmd string, args ...string) ([]byte, error) {
c := newMonitoredCmd(repo.CmdFromDir(cmd, args...), 2*time.Minute)
func runFromRepoDir(ctx context.Context, repo vcs.Repo, timeout time.Duration, cmd string, args ...string) ([]byte, error) {
c := newMonitoredCmd(repo.CmdFromDir(cmd, args...), timeout)
return c.combinedOutput(ctx)
}
const (
// expensiveCmdTimeout is meant to be used in a command that is expensive
// in terms of computation and we know it will take long or one that uses
// the network, such as clones, updates, ....
expensiveCmdTimeout = 2 * time.Minute
// defaultCmdTimeout is just an umbrella value for all other commands that
// should not take much.
defaultCmdTimeout = 10 * time.Second
)

59
internal/gps/cmd_test.go
Просмотреть файл

@ -16,7 +16,7 @@ import (
func mkTestCmd(iterations int) *monitoredCmd {
return newMonitoredCmd(
exec.Command("./echosleep", "-n", fmt.Sprint(iterations)),
500*time.Millisecond,
490*time.Millisecond,
)
}
@ -32,49 +32,60 @@ func TestMonitoredCmd(t *testing.T) {
}
defer os.Remove("./echosleep")
cmd := mkTestCmd(2)
err = cmd.run(context.Background())
if err != nil {
t.Errorf("Expected command not to fail: %s", err)
tests := []struct {
name string
iterations int
output string
err bool
timeout bool
}{
{"success", 2, "foo\nfoo\n", false, false},
{"timeout", 5, "foo\nfoo\nfoo\nfoo\n", true, true},
}
expectedOutput := "foo\nfoo\n"
if cmd.stdout.buf.String() != expectedOutput {
t.Errorf("Unexpected output:\n\t(GOT): %s\n\t(WNT): %s", cmd.stdout.buf.String(), expectedOutput)
for _, want := range tests {
t.Run(want.name, func(t *testing.T) {
cmd := mkTestCmd(want.iterations)
err := cmd.run(context.Background())
if !want.err && err != nil {
t.Errorf("Eexpected command not to fail, got error: %s", err)
} else if want.err && err == nil {
t.Error("expected command to fail")
}
cmd2 := mkTestCmd(10)
err = cmd2.run(context.Background())
if err == nil {
t.Error("Expected command to fail")
got := cmd.stdout.String()
if want.output != got {
t.Errorf("unexpected output:\n\t(GOT):\n%s\n\t(WNT):\n%s", got, want.output)
}
_, ok := err.(*timeoutError)
if want.timeout {
_, ok := err.(*noProgressError)
if !ok {
t.Errorf("Expected a timeout error, but got: %s", err)
}
expectedOutput = "foo\nfoo\nfoo\nfoo\n"
if cmd2.stdout.buf.String() != expectedOutput {
t.Errorf("Unexpected output:\n\t(GOT): %s\n\t(WNT): %s", cmd2.stdout.buf.String(), expectedOutput)
}
})
}
t.Run("cancel", func(t *testing.T) {
ctx, cancel := context.WithCancel(context.Background())
sync1, errchan := make(chan struct{}), make(chan error)
cmd3 := mkTestCmd(2)
sync, errchan := make(chan struct{}), make(chan error)
cmd := mkTestCmd(2)
go func() {
close(sync1)
errchan <- cmd3.run(ctx)
close(sync)
errchan <- cmd.run(ctx)
}()
// Make sure goroutine is at least started before we cancel the context.
<-sync1
<-sync
// Give it a bit to get the process started.
<-time.After(5 * time.Millisecond)
cancel()
err = <-errchan
err := <-errchan
if err != context.Canceled {
t.Errorf("should have gotten canceled error, got %s", err)
t.Errorf("expected a canceled error, got %s", err)
}
})
}

56
internal/gps/cmd_unix.go Normal file
Просмотреть файл

@ -0,0 +1,56 @@
// Copyright 2017 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build !windows
package gps
import (
"os"
"os/exec"
"sync/atomic"
"time"
)
// killProcess manages the termination of subprocesses in a way that tries to be
// gentle (via os.Interrupt), but resorts to Kill if needed.
//
//
// TODO(sdboyer) should the return differentiate between whether gentle methods
// succeeded vs. falling back to a hard kill?
func killProcess(cmd *exec.Cmd, isDone *int32) error {
if err := cmd.Process.Signal(os.Interrupt); err != nil {
// If an error comes back from attempting to signal, proceed immediately
// to hard kill.
return cmd.Process.Kill()
}
// If the process doesn't exit immediately, check every 50ms, up to 3s,
// after which send a hard kill.
//
// Cannot rely on cmd.ProcessState.Exited() here, as that is not set
// correctly when the process exits due to a signal. See
// https://github.com/golang/go/issues/19798 . Also cannot rely on it
// because cmd.ProcessState will be nil before the process exits, and
// checking if nil create a data race.
if !atomic.CompareAndSwapInt32(isDone, 1, 1) {
to := time.NewTimer(3 * time.Second)
tick := time.NewTicker(50 * time.Millisecond)
defer to.Stop()
defer tick.Stop()
// Loop until the ProcessState shows up, indicating the proc has exited,
// or the timer expires and
for !atomic.CompareAndSwapInt32(isDone, 1, 1) {
select {
case <-to.C:
return cmd.Process.Kill()
case <-tick.C:
}
}
}
return nil
}

14
internal/gps/cmd_windows.go Normal file
Просмотреть файл

@ -0,0 +1,14 @@
// Copyright 2017 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build windows
package gps
import "os/exec"
func killProcess(cmd *exec.Cmd, isDone *int32) error {
// TODO it'd be great if this could be more sophisticated...
return cmd.Process.Kill()
}

41
internal/gps/constraint_test.go
Просмотреть файл

@ -7,6 +7,8 @@ package gps
import (
"fmt"
"testing"
"github.com/pkg/errors"
)
// gu - helper func for stringifying what we assume is a VersionPair (otherwise
@ -93,9 +95,6 @@ func TestBranchConstraintOps(t *testing.T) {
}
// Now add same rev to different branches
// TODO(sdboyer) this might not actually be a good idea, when you consider the
// semantics of floating versions...matching on an underlying rev might be
// nice in the short term, but it's probably shit most of the time
v5 := v2.Pair(Revision("snuffleupagus")).(versionPair)
if !v5.Matches(v3) {
t.Errorf("%s should match %s", gu(v5), gu(v3))
@ -927,3 +926,39 @@ func TestTypedConstraintString(t *testing.T) {
}
}
}
func TestConstraintsIdentical(t *testing.T) {
for _, test := range []struct {
a, b Constraint
eq bool
}{
{Any(), Any(), true},
{none, noneConstraint{}, true},
{NewVersion("test"), NewVersion("test"), true},
{NewVersion("test"), NewVersion("test2"), false},
{NewBranch("test"), NewBranch("test"), true},
{NewBranch("test"), newDefaultBranch("test"), false},
{newDefaultBranch("test"), newDefaultBranch("test"), true},
{Revision("test"), Revision("test"), true},
{Revision("test"), Revision("test2"), false},
{testSemverConstraint(t, "v2.10.7"), testSemverConstraint(t, "v2.10.7"), true},
{versionTypeUnion{NewVersion("test"), NewBranch("branch")},
versionTypeUnion{NewBranch("branch"), NewVersion("test")}, true},
} {
if test.eq != test.a.identical(test.b) {
want := "identical"
if !test.eq {
want = "not " + want
}
t.Errorf("expected %s:\n\t(a) %#v\n\t(b) %#v", want, test.a, test.b)
}
}
}
func testSemverConstraint(t *testing.T, body string) Constraint {
c, err := NewSemverConstraint(body)
if err != nil {
t.Fatal(errors.Wrapf(err, "failed to create semver constraint: %s", body))
}
return c
}

56
internal/gps/constraints.go
Просмотреть файл

@ -52,6 +52,14 @@ type Constraint interface {
// It also forces Constraint to be a private/sealed interface, which is a
// design goal of the system.
typedString() string
// identical returns true if the constraints are identical.
//
// Identical Constraints behave identically for all methods defined by the
// interface. A Constraint is always identical to itself.
//
// Constraints serialized for caching are de-serialized into identical instances.
identical(Constraint) bool
}
// NewSemverConstraint attempts to construct a semver Constraint object from the
@ -172,6 +180,14 @@ func (c semverConstraint) Intersect(c2 Constraint) Constraint {
return none
}
func (c semverConstraint) identical(c2 Constraint) bool {
sc2, ok := c2.(semverConstraint)
if !ok {
return false
}
return c.c.String() == sc2.c.String()
}
// IsAny indicates if the provided constraint is the wildcard "Any" constraint.
func IsAny(c Constraint) bool {
_, ok := c.(anyConstraint)
@ -211,6 +227,10 @@ func (anyConstraint) Intersect(c Constraint) Constraint {
return c
}
func (anyConstraint) identical(c Constraint) bool {
return IsAny(c)
}
// noneConstraint is the empty set - it matches no versions. It mirrors the
// behavior of the semver package's none type.
type noneConstraint struct{}
@ -239,6 +259,11 @@ func (noneConstraint) Intersect(Constraint) Constraint {
return none
}
func (noneConstraint) identical(c Constraint) bool {
_, ok := c.(noneConstraint)
return ok
}
// A ProjectConstraint combines a ProjectIdentifier with a Constraint. It
// indicates that, if packages contained in the ProjectIdentifier enter the
// depgraph, they must do so at a version that is allowed by the Constraint.
@ -310,37 +335,6 @@ func (m ProjectConstraints) asSortedSlice() []ProjectConstraint {
return pcs
}
// merge pulls in all the constraints from other ProjectConstraints map(s),
// merging them with the receiver into a new ProjectConstraints map.
//
// If duplicate ProjectRoots are encountered, the constraints are intersected
// together and the latter's NetworkName, if non-empty, is taken.
func (m ProjectConstraints) merge(other ...ProjectConstraints) (out ProjectConstraints) {
plen := len(m)
for _, pcm := range other {
plen += len(pcm)
}
out = make(ProjectConstraints, plen)
for pr, pp := range m {
out[pr] = pp
}
for _, pcm := range other {
for pr, pp := range pcm {
if rpp, exists := out[pr]; exists {
pp.Constraint = pp.Constraint.Intersect(rpp.Constraint)
if pp.Source == "" {
pp.Source = rpp.Source
}
}
out[pr] = pp
}
}
return
}
// overrideAll treats the receiver ProjectConstraints map as a set of override
// instructions, and applies overridden values to the ProjectConstraints.
//

4
internal/gps/lock.go
Просмотреть файл

@ -226,7 +226,7 @@ func prepLock(l Lock) safeLock {
}
// SortLockedProjects sorts a slice of LockedProject in alphabetical order by
// ProjectRoot.
// ProjectIdentifier.
func SortLockedProjects(lps []LockedProject) {
sort.Stable(lpsorter(lps))
}
@ -242,5 +242,5 @@ func (lps lpsorter) Len() int {
}
func (lps lpsorter) Less(i, j int) bool {
return lps[i].pi.ProjectRoot < lps[j].pi.ProjectRoot
return lps[i].Ident().less(lps[j].Ident())
}

4
internal/gps/manager_test.go
Просмотреть файл

@ -713,10 +713,6 @@ func TestSignalHandling(t *testing.T) {
t.Error("Releasing flag did not get set")
}
lpath := filepath.Join(sm.cachedir, "sm.lock")
if _, err := os.Stat(lpath); err == nil {
t.Fatal("Expected error on statting what should be an absent lock file")
}
clean()
// Test again, this time with a running call

40
internal/gps/manifest.go
Просмотреть файл

@ -5,8 +5,7 @@
package gps
// Manifest represents manifest-type data for a project at a particular version.
// That means dependency constraints, both for normal dependencies and for
// tests. The constraints expressed in a manifest determine the set of versions that
// The constraints expressed in a manifest determine the set of versions that
// are acceptable to try for a given project.
//
// Expressing a constraint in a manifest does not guarantee that a particular
@ -20,12 +19,6 @@ package gps
type Manifest interface {
// Returns a list of project-level constraints.
DependencyConstraints() ProjectConstraints
// Returns a list of constraints applicable to test imports.
//
// These are applied only when tests are incorporated. Typically, that
// will only be for root manifests.
TestDependencyConstraints() ProjectConstraints
}
// RootManifest extends Manifest to add special controls over solving that are
@ -70,7 +63,7 @@ type RootManifest interface {
// the fly for projects with no manifest metadata, or metadata through a foreign
// tool's idioms.
type SimpleManifest struct {
Deps, TestDeps ProjectConstraints
Deps ProjectConstraints
}
var _ Manifest = SimpleManifest{}
@ -80,26 +73,16 @@ func (m SimpleManifest) DependencyConstraints() ProjectConstraints {
return m.Deps
}
// TestDependencyConstraints returns the project's test dependencies.
func (m SimpleManifest) TestDependencyConstraints() ProjectConstraints {
return m.TestDeps
}
// simpleRootManifest exists so that we have a safe value to swap into solver
// params when a nil Manifest is provided.
//
// Also, for tests.
type simpleRootManifest struct {
c, tc, ovr ProjectConstraints
c, ovr ProjectConstraints
ig, req map[string]bool
}
func (m simpleRootManifest) DependencyConstraints() ProjectConstraints {
return m.c
}
func (m simpleRootManifest) TestDependencyConstraints() ProjectConstraints {
return m.tc
}
func (m simpleRootManifest) Overrides() ProjectConstraints {
return m.ovr
}
@ -112,7 +95,6 @@ func (m simpleRootManifest) RequiredPackages() map[string]bool {
func (m simpleRootManifest) dup() simpleRootManifest {
m2 := simpleRootManifest{
c: make(ProjectConstraints, len(m.c)),
tc: make(ProjectConstraints, len(m.tc)),
ovr: make(ProjectConstraints, len(m.ovr)),
ig: make(map[string]bool, len(m.ig)),
req: make(map[string]bool, len(m.req)),
@ -121,9 +103,6 @@ func (m simpleRootManifest) dup() simpleRootManifest {
for k, v := range m.c {
m2.c[k] = v
}
for k, v := range m.tc {
m2.tc[k] = v
}
for k, v := range m.ovr {
m2.ovr[k] = v
}
@ -149,11 +128,9 @@ func prepManifest(m Manifest) SimpleManifest {
}
deps := m.DependencyConstraints()
ddeps := m.TestDependencyConstraints()
rm := SimpleManifest{
Deps: make(ProjectConstraints, len(deps)),
TestDeps: make(ProjectConstraints, len(ddeps)),
}
for k, d := range deps {
@ -171,16 +148,5 @@ func prepManifest(m Manifest) SimpleManifest {
rm.Deps[k] = d
}
for k, d := range ddeps {
if d.Constraint == nil {
if d.Source == "" {
continue
}
d.Constraint = anyConstraint{}
}
rm.TestDeps[k] = d
}
return rm
}

13
internal/gps/manifest_test.go
Просмотреть файл

@ -15,28 +15,15 @@ func TestPrepManifest(t *testing.T) {
Source: "whatever",
},
},
TestDeps: ProjectConstraints{
ProjectRoot("baz"): ProjectProperties{},
ProjectRoot("qux"): ProjectProperties{
Source: "whatever",
},
},
}
prepped := prepManifest(m)
d := prepped.DependencyConstraints()
td := prepped.TestDependencyConstraints()
if len(d) != 1 {
t.Error("prepManifest did not eliminate empty ProjectProperties from deps map")
}
if len(td) != 1 {
t.Error("prepManifest did not eliminate empty ProjectProperties from test deps map")
}
if d[ProjectRoot("bar")].Constraint != any {
t.Error("prepManifest did not normalize nil constraint to anyConstraint in deps map")
}
if td[ProjectRoot("qux")].Constraint != any {
t.Error("prepManifest did not normalize nil constraint to anyConstraint in test deps map")
}
}

1
internal/gps/result.go
Просмотреть файл

@ -73,7 +73,6 @@ func WriteDepTree(basedir string, l Lock, sm SourceManager, sv bool) error {
if sv {
filepath.Walk(to, stripVendor)
}
// TODO(sdboyer) dump version metadata file
}
return nil

5
internal/gps/rootdata.go
Просмотреть файл

@ -80,8 +80,7 @@ func (rd rootdata) externalImportList(stdLibFn func(string) bool) []string {
}
func (rd rootdata) getApplicableConstraints(stdLibFn func(string) bool) []workingConstraint {
// Merge the normal and test constraints together
pc := rd.rm.DependencyConstraints().merge(rd.rm.TestDependencyConstraints())
pc := rd.rm.DependencyConstraints()
// Ensure that overrides which aren't in the combined pc map already make it
// in. Doing so makes input hashes equal in more useful cases.
@ -139,7 +138,7 @@ func (rd rootdata) getApplicableConstraints(stdLibFn func(string) bool) []workin
}
func (rd rootdata) combineConstraints() []workingConstraint {
return rd.ovr.overrideAll(rd.rm.DependencyConstraints().merge(rd.rm.TestDependencyConstraints()))
return rd.ovr.overrideAll(rd.rm.DependencyConstraints())
}
// needVersionListFor indicates whether we need a version list for a given

74
internal/gps/solve_basic_test.go
Просмотреть файл

@ -194,7 +194,6 @@ type depspec struct {
n ProjectRoot
v Version
deps []ProjectConstraint
devdeps []ProjectConstraint
pkgs []tpkg
}
@ -205,9 +204,6 @@ type depspec struct {
// described - see the docs on mkAtom for details. subsequent strings are
// interpreted as dep constraints of that dep at that version. See the docs on
// mkPDep for details.
//
// If a string other than the first includes a "(dev) " prefix, it will be
// treated as a test-only dependency.
func mkDepspec(pi string, deps ...string) depspec {
pa := mkAtom(pi)
if string(pa.id.ProjectRoot) != pa.id.Source && pa.id.Source != "" {
@ -220,15 +216,7 @@ func mkDepspec(pi string, deps ...string) depspec {
}
for _, dep := range deps {
var sl *[]ProjectConstraint
if strings.HasPrefix(dep, "(dev) ") {
dep = strings.TrimPrefix(dep, "(dev) ")
sl = &ds.devdeps
} else {
sl = &ds.deps
}
*sl = append(*sl, mkPCstrnt(dep))
ds.deps = append(ds.deps, mkPCstrnt(dep))
}
return ds
@ -356,13 +344,6 @@ func computeBasicReachMap(ds []depspec) reachMap {
for _, dep := range d.deps {
lm[n] = append(lm[n], string(dep.Ident.ProjectRoot))
}
// first is root
if k == 0 {
for _, dep := range d.devdeps {
lm[n] = append(lm[n], string(dep.Ident.ProjectRoot))
}
}
}
return rm
@ -439,19 +420,15 @@ func (f basicFixture) solution() map[ProjectIdentifier]LockedProject {
func (f basicFixture) rootmanifest() RootManifest {
return simpleRootManifest{
c: pcSliceToMap(f.ds[0].deps),
tc: pcSliceToMap(f.ds[0].devdeps),
ovr: f.ovr,
}
}
func (f basicFixture) rootTree() pkgtree.PackageTree {
var imp, timp []string
var imp []string
for _, dep := range f.ds[0].deps {
imp = append(imp, string(dep.Ident.ProjectRoot))
}
for _, dep := range f.ds[0].devdeps {
timp = append(timp, string(dep.Ident.ProjectRoot))
}
n := string(f.ds[0].n)
pt := pkgtree.PackageTree{
@ -462,7 +439,6 @@ func (f basicFixture) rootTree() pkgtree.PackageTree {
ImportPath: n,
Name: n,
Imports: imp,
TestImports: timp,
},
},
},
@ -917,38 +893,6 @@ var basicFixtures = map[string]basicFixture{
"foo ptaggerino oldrev",
),
},
"includes root package's dev dependencies": {
ds: []depspec{
mkDepspec("root 1.0.0", "(dev) foo 1.0.0", "(dev) bar 1.0.0"),
mkDepspec("foo 1.0.0"),
mkDepspec("bar 1.0.0"),
},
r: mksolution(
"foo 1.0.0",
"bar 1.0.0",
),
},
"includes dev dependency's transitive dependencies": {
ds: []depspec{
mkDepspec("root 1.0.0", "(dev) foo 1.0.0"),
mkDepspec("foo 1.0.0", "bar 1.0.0"),
mkDepspec("bar 1.0.0"),
},
r: mksolution(
"foo 1.0.0",
"bar 1.0.0",
),
},
"ignores transitive dependency's dev dependencies": {
ds: []depspec{
mkDepspec("root 1.0.0", "(dev) foo 1.0.0"),
mkDepspec("foo 1.0.0", "(dev) bar 1.0.0"),
mkDepspec("bar 1.0.0"),
},
r: mksolution(
"foo 1.0.0",
),
},
"no version that matches requirement": {
ds: []depspec{
mkDepspec("root 0.0.0", "foo ^1.0.0"),
@ -1550,6 +1494,15 @@ func (sm *depspecSourceManager) ignore() map[string]bool {
return sm.ig
}
// InferConstraint tries to puzzle out what kind of version is given in a string -
// semver, a revision, or as a fallback, a plain tag. This current implementation
// is a panic because there's no current circumstance under which the depspecSourceManager
// is useful outside of the gps solving tests, and it shouldn't be used anywhere else without a conscious and intentional
// expansion of its semantics.
func (sm *depspecSourceManager) InferConstraint(s string, pi ProjectIdentifier) (Constraint, error) {
panic("depsecSourceManager is only for gps solving tests")
}
type depspecBridge struct {
*bridge
}
@ -1613,11 +1566,6 @@ func (ds depspec) DependencyConstraints() ProjectConstraints {
return pcSliceToMap(ds.deps)
}
// impl Spec interface
func (ds depspec) TestDependencyConstraints() ProjectConstraints {
return pcSliceToMap(ds.devdeps)
}
type fixLock []LockedProject
func (fixLock) SolverVersion() string {

1
internal/gps/solve_bimodal_test.go
Просмотреть файл

@ -1087,7 +1087,6 @@ func (f bimodalFixture) solution() map[ProjectIdentifier]LockedProject {
func (f bimodalFixture) rootmanifest() RootManifest {
m := simpleRootManifest{
c: pcSliceToMap(f.ds[0].deps),
tc: pcSliceToMap(f.ds[0].devdeps),
ovr: f.ovr,
ig: make(map[string]bool),
req: make(map[string]bool),

28
internal/gps/solve_test.go
Просмотреть файл

@ -6,7 +6,6 @@ package gps
import (
"bytes"
"flag"
"fmt"
"io/ioutil"
"log"
@ -21,13 +20,6 @@ import (
"github.com/golang/dep/internal/gps/pkgtree"
)
var fixtorun string
// TODO(sdboyer) regression test ensuring that locks with only revs for projects don't cause errors
func init() {
flag.StringVar(&fixtorun, "gps.fix", "", "A single fixture to run in TestBasicSolves or TestBimodalSolves")
}
// overrideMkBridge overrides the base bridge with the depspecBridge that skips
// verifyRootDir calls
func overrideMkBridge(s *solver, sm SourceManager, down bool) sourceBridge {
@ -74,13 +66,8 @@ func fixSolve(params SolveParameters, sm SourceManager, t *testing.T) (Solution,
//
// Or, just the one named in the fix arg.
func TestBasicSolves(t *testing.T) {
if fixtorun != "" {
if fix, exists := basicFixtures[fixtorun]; exists {
solveBasicsAndCheck(fix, t)
}
} else {
// sort them by their keys so we get stable output
var names []string
names := make([]string, 0, len(basicFixtures))
for n := range basicFixtures {
names = append(names, n)
}
@ -88,11 +75,10 @@ func TestBasicSolves(t *testing.T) {
sort.Strings(names)
for _, n := range names {
t.Run(n, func(t *testing.T) {
//t.Parallel() // until trace output is fixed in parallel
t.Parallel()
solveBasicsAndCheck(basicFixtures[n], t)
})
}
}
}
func solveBasicsAndCheck(fix basicFixture, t *testing.T) (res Solution, err error) {
@ -122,13 +108,8 @@ func solveBasicsAndCheck(fix basicFixture, t *testing.T) (res Solution, err erro
//
// Or, just the one named in the fix arg.
func TestBimodalSolves(t *testing.T) {
if fixtorun != "" {
if fix, exists := bimodalFixtures[fixtorun]; exists {
solveBimodalAndCheck(fix, t)
}
} else {
// sort them by their keys so we get stable output
var names []string
names := make([]string, 0, len(bimodalFixtures))
for n := range bimodalFixtures {
names = append(names, n)
}
@ -136,11 +117,10 @@ func TestBimodalSolves(t *testing.T) {
sort.Strings(names)
for _, n := range names {
t.Run(n, func(t *testing.T) {
//t.Parallel() // until trace output is fixed in parallel
t.Parallel()
solveBimodalAndCheck(bimodalFixtures[n], t)
})
}
}
}
func solveBimodalAndCheck(fix bimodalFixture, t *testing.T) (res Solution, err error) {

36
internal/gps/source_cache.go
Просмотреть файл

@ -221,3 +221,39 @@ func (c *singleSourceCacheMemory) toUnpaired(v Version) (UnpairedVersion, bool)
panic(fmt.Sprintf("unknown version type %T", v))
}
}
// cachedManifest implements RootManifest and is populated from cached data.
type cachedManifest struct {
constraints, overrides ProjectConstraints
ignored, required map[string]bool
}
func (m *cachedManifest) DependencyConstraints() ProjectConstraints {
return m.constraints
}
func (m *cachedManifest) Overrides() ProjectConstraints {
return m.overrides
}
func (m *cachedManifest) IgnoredPackages() map[string]bool {
return m.ignored
}
func (m *cachedManifest) RequiredPackages() map[string]bool {
return m.required
}
// cachedManifest implements Lock and is populated from cached data.
type cachedLock struct {
inputHash []byte
projects []LockedProject
}
func (l *cachedLock) InputHash() []byte {
return l.inputHash
}
func (l *cachedLock) Projects() []LockedProject {
return l.projects
}

418
internal/gps/source_cache_test.go Normal file
Просмотреть файл

@ -0,0 +1,418 @@
// Copyright 2017 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package gps
import (
"sort"
"testing"
"github.com/golang/dep/internal/gps/pkgtree"
"github.com/pkg/errors"
)
var testAnalyzerInfo = ProjectAnalyzerInfo{
Name: "test-analyzer",
Version: 1,
}
func TestSingleSourceCache(t *testing.T) {
const root = "example.com/test"
t.Run("info", func(t *testing.T) {
const rev Revision = "revision"
c := newMemoryCache()
var m Manifest = &cachedManifest{
constraints: ProjectConstraints{
ProjectRoot("foo"): ProjectProperties{},
ProjectRoot("bar"): ProjectProperties{
Source: "whatever",
Constraint: testSemverConstraint(t, "> 1.3"),
},
},
overrides: ProjectConstraints{
ProjectRoot("b"): ProjectProperties{
Constraint: NewVersion("2.0.0"),
},
},
ignored: map[string]bool{
"a": true,
"b": true,
},
required: map[string]bool{
"c": true,
"d": true,
},
}
var l Lock = &cachedLock{
inputHash: []byte("test_hash"),
projects: []LockedProject{
NewLockedProject(mkPI("github.com/sdboyer/gps"), NewVersion("v0.10.0"), []string{"gps"}),
NewLockedProject(mkPI("github.com/sdboyer/gps"), NewVersion("v0.10.0"), nil),
NewLockedProject(mkPI("github.com/sdboyer/gps"), NewVersion("v0.10.0"), []string{"gps", "flugle"}),
NewLockedProject(mkPI("foo"), NewVersion("nada"), []string{"foo"}),
NewLockedProject(mkPI("github.com/sdboyer/gps"), NewVersion("v0.10.0"), []string{"flugle", "gps"}),
},
}
c.setManifestAndLock(rev, testAnalyzerInfo, m, l)
gotM, gotL, ok := c.getManifestAndLock(rev, testAnalyzerInfo)
if !ok {
t.Error("no manifest and lock found for revision")
}
compareManifests(t, m, gotM)
if dl := DiffLocks(l, gotL); dl != nil {
t.Errorf("lock differences:\n\t %#v", dl)
}
m = &cachedManifest{
constraints: ProjectConstraints{
ProjectRoot("foo"): ProjectProperties{
Source: "whatever",
},
},
overrides: ProjectConstraints{
ProjectRoot("bar"): ProjectProperties{
Constraint: NewVersion("2.0.0"),
},
},
ignored: map[string]bool{
"c": true,
"d": true,
},
required: map[string]bool{
"a": true,
"b": true,
},
}
l = &cachedLock{
inputHash: []byte("different_test_hash"),
projects: []LockedProject{
NewLockedProject(mkPI("github.com/sdboyer/gps"), NewVersion("v0.10.0").Pair("278a227dfc3d595a33a77ff3f841fd8ca1bc8cd0"), []string{"gps"}),
NewLockedProject(mkPI("github.com/sdboyer/gps"), NewVersion("v0.11.0"), []string{"gps"}),
NewLockedProject(mkPI("github.com/sdboyer/gps"), Revision("278a227dfc3d595a33a77ff3f841fd8ca1bc8cd0"), []string{"gps"}),
},
}
c.setManifestAndLock(rev, testAnalyzerInfo, m, l)
gotM, gotL, ok = c.getManifestAndLock(rev, testAnalyzerInfo)
if !ok {
t.Error("no manifest and lock found for revision")
}
compareManifests(t, m, gotM)
if dl := DiffLocks(l, gotL); dl != nil {
t.Errorf("lock differences:\n\t %#v", dl)
}
})
t.Run("pkgTree", func(t *testing.T) {
c := newMemoryCache()
const rev Revision = "rev_adsfjkl"
if got, ok := c.getPackageTree(rev); ok {
t.Fatalf("unexpected result before setting package tree: %v", got)
}
pt := pkgtree.PackageTree{
ImportRoot: root,
Packages: map[string]pkgtree.PackageOrErr{
"simple": {
P: pkgtree.Package{
ImportPath: "simple",
CommentPath: "comment",
Name: "simple",
Imports: []string{
"github.com/golang/dep/internal/gps",
"sort",
},
},
},
"m1p": {
P: pkgtree.Package{
ImportPath: "m1p",
CommentPath: "",
Name: "m1p",
Imports: []string{
"github.com/golang/dep/internal/gps",
"os",
"sort",
},
},
},
},
}
c.setPackageTree(rev, pt)
got, ok := c.getPackageTree(rev)
if !ok {
t.Errorf("no package tree found:\n\t(WNT): %#v", pt)
}
comparePackageTree(t, pt, got)
pt = pkgtree.PackageTree{
ImportRoot: root,
Packages: map[string]pkgtree.PackageOrErr{
"test": {
Err: errors.New("error"),
},
},
}
c.setPackageTree(rev, pt)
got, ok = c.getPackageTree(rev)
if !ok {
t.Errorf("no package tree found:\n\t(WNT): %#v", pt)
}
comparePackageTree(t, pt, got)
})
t.Run("versions", func(t *testing.T) {
c := newMemoryCache()
const rev1, rev2 = "rev1", "rev2"
const br, ver = "branch_name", "2.10"
versions := []PairedVersion{
NewBranch(br).Pair(rev1),
NewVersion(ver).Pair(rev2),
}
SortPairedForDowngrade(versions)
c.storeVersionMap(versions, true)
t.Run("getAllVersions", func(t *testing.T) {
got := c.getAllVersions()
if len(got) != len(versions) {
t.Errorf("unexpected versions:\n\t(GOT): %#v\n\t(WNT): %#v", got, versions)
} else {
SortPairedForDowngrade(got)
for i := range versions {
if !versions[i].identical(got[i]) {
t.Errorf("unexpected versions:\n\t(GOT): %#v\n\t(WNT): %#v", got, versions)
break
}
}
}
})
revToUV := map[Revision]UnpairedVersion{
rev1: NewBranch(br),
rev2: NewVersion(ver),
}
t.Run("getVersionsFor", func(t *testing.T) {
for rev, want := range revToUV {
rev, want := rev, want
t.Run(string(rev), func(t *testing.T) {
uvs, ok := c.getVersionsFor(rev)
if !ok {
t.Errorf("no version found:\n\t(WNT) %#v", want)
} else if len(uvs) != 1 {
t.Errorf("expected one result but got %d", len(uvs))
} else {
uv := uvs[0]
if uv.Type() != want.Type() {
t.Errorf("expected version type %d but got %d", want.Type(), uv.Type())
}
if uv.String() != want.String() {
t.Errorf("expected version %q but got %q", want.String(), uv.String())
}
}
})
}
})
t.Run("getRevisionFor", func(t *testing.T) {
for want, uv := range revToUV {
want, uv := want, uv
t.Run(uv.String(), func(t *testing.T) {
rev, ok := c.getRevisionFor(uv)
if !ok {
t.Errorf("expected revision %q but got none", want)
} else if rev != want {
t.Errorf("expected revision %q but got %q", want, rev)
}
})
}
})
t.Run("toRevision", func(t *testing.T) {
for want, uv := range revToUV {
want, uv := want, uv
t.Run(uv.String(), func(t *testing.T) {
rev, ok := c.toRevision(uv)
if !ok {
t.Errorf("expected revision %q but got none", want)
} else if rev != want {
t.Errorf("expected revision %q but got %q", want, rev)
}
})
}
})
t.Run("toUnpaired", func(t *testing.T) {
for rev, want := range revToUV {
rev, want := rev, want
t.Run(want.String(), func(t *testing.T) {
uv, ok := c.toUnpaired(rev)
if !ok {
t.Errorf("no UnpairedVersion found:\n\t(WNT): %#v", uv)
} else if !uv.identical(want) {
t.Errorf("unexpected UnpairedVersion:\n\t(GOT): %#v\n\t(WNT): %#v", uv, want)
}
})
}
})
})
}
// compareManifests compares two manifests and reports differences as test errors.
func compareManifests(t *testing.T, want, got Manifest) {
{
want, got := want.DependencyConstraints(), got.DependencyConstraints()
if !projectConstraintsEqual(want, got) {
t.Errorf("unexpected constraints:\n\t(GOT): %#v\n\t(WNT): %#v", got, want)
}
}
wantRM, wantOK := want.(RootManifest)
gotRM, gotOK := got.(RootManifest)
if wantOK && !gotOK {
t.Errorf("expected RootManifest:\n\t(GOT): %#v", got)
return
}
if gotOK && !wantOK {
t.Errorf("didn't expected RootManifest:\n\t(GOT): %#v", got)
return
}
{
want, got := wantRM.IgnoredPackages(), gotRM.IgnoredPackages()
if !mapStringBoolEqual(want, got) {
t.Errorf("unexpected ignored packages:\n\t(GOT): %#v\n\t(WNT): %#v", got, want)
}
}
{
want, got := wantRM.Overrides(), gotRM.Overrides()
if !projectConstraintsEqual(want, got) {
t.Errorf("unexpected overrides:\n\t(GOT): %#v\n\t(WNT): %#v", got, want)
}
}
{
want, got := wantRM.RequiredPackages(), gotRM.RequiredPackages()
if !mapStringBoolEqual(want, got) {
t.Errorf("unexpected required packages:\n\t(GOT): %#v\n\t(WNT): %#v", got, want)
}
}
}
// comparePackageTree compares two pkgtree.PackageTree and reports differences as test errors.
func comparePackageTree(t *testing.T, want, got pkgtree.PackageTree) {
if got.ImportRoot != want.ImportRoot {
t.Errorf("expected package tree root %q but got %q", want.ImportRoot, got.ImportRoot)
}
{
want, got := want.Packages, got.Packages
if len(want) != len(got) {
t.Errorf("unexpected packages:\n\t(GOT): %#v\n\t(WNT): %#v", got, want)
} else {
for k, v := range want {
if v2, ok := got[k]; !ok {
t.Errorf("key %q: expected %v but got none", k, v)
} else if !packageOrErrEqual(v, v2) {
t.Errorf("key %q: expected %v but got %v", k, v, v2)
}
}
}
}
}
func projectConstraintsEqual(want, got ProjectConstraints) bool {
loop, check := want, got
if len(got) > len(want) {
loop, check = got, want
}
for pr, pp := range loop {
pp2, ok := check[pr]
if !ok {
return false
}
if pp.Source != pp2.Source {
return false
}
if pp.Constraint == nil || pp2.Constraint == nil {
if pp.Constraint != nil || pp2.Constraint != nil {
return false
}
} else if !pp.Constraint.identical(pp2.Constraint) {
return false
}
}
return true
}
func mapStringBoolEqual(exp, got map[string]bool) bool {
loop, check := exp, got
if len(got) > len(exp) {
loop, check = got, exp
}
for k, v := range loop {
v2, ok := check[k]
if !ok || v != v2 {
return false
}
}
return true
}
func safeError(err error) string {
if err == nil {
return ""
}
return err.Error()
}
// packageOrErrEqual return true if the pkgtree.PackageOrErrs are equal. Error equality is
// string based. Imports and TestImports are treated as sets, and will be sorted.
func packageOrErrEqual(a, b pkgtree.PackageOrErr) bool {
if safeError(a.Err) != safeError(b.Err) {
return false
}
if a.P.Name != b.P.Name {
return false
}
if a.P.ImportPath != b.P.ImportPath {
return false
}
if a.P.CommentPath != b.P.CommentPath {
return false
}
if len(a.P.Imports) != len(b.P.Imports) {
return false
}
sort.Strings(a.P.Imports)
sort.Strings(b.P.Imports)
for i := range a.P.Imports {
if a.P.Imports[i] != b.P.Imports[i] {
return false
}
}
if len(a.P.TestImports) != len(b.P.TestImports) {
return false
}
sort.Strings(a.P.TestImports)
sort.Strings(b.P.TestImports)
for i := range a.P.TestImports {
if a.P.TestImports[i] != b.P.TestImports[i] {
return false
}
}
return true
}

139
internal/gps/source_manager.go
Просмотреть файл

@ -6,17 +6,21 @@ package gps
import (
"context"
"encoding/hex"
"fmt"
"os"
"os/signal"
"path/filepath"
"runtime"
"strconv"
"strings"
"sync"
"sync/atomic"
"time"
"github.com/golang/dep/internal/gps/pkgtree"
"github.com/nightlyone/lockfile"
"github.com/pkg/errors"
"github.com/sdboyer/constext"
)
@ -72,6 +76,10 @@ type SourceManager interface {
// no longer safe to call methods against it; all method calls will
// immediately result in errors.
Release()
// InferConstraint tries to puzzle out what kind of version is given in a string -
// semver, a revision, or as a fallback, a plain tag
InferConstraint(s string, pi ProjectIdentifier) (Constraint, error)
}
// A ProjectAnalyzer is responsible for analyzing a given path for Manifest and
@ -108,7 +116,7 @@ func (p ProjectAnalyzerInfo) String() string {
// tools; control via dependency injection is intended to be sufficient.
type SourceMgr struct {
cachedir string // path to root of cache dir
lf *os.File // handle for the sm lock file on disk
lf *lockfile.Lockfile // handle for the sm lock file on disk
suprvsr *supervisor // subsystem that supervises running calls/io
cancelAll context.CancelFunc // cancel func to kill all running work
deduceCoord *deductionCoordinator // subsystem that manages import path deduction
@ -146,30 +154,65 @@ func NewSourceManager(cachedir string) (*SourceMgr, error) {
return nil, err
}
glpath := filepath.Join(cachedir, "sm.lock")
_, err = os.Stat(glpath)
if err == nil {
return nil, CouldNotCreateLockError{
Path: glpath,
Err: fmt.Errorf("cache lock file %s exists - another process crashed or is still running?", glpath),
}
}
// Fix for #820
//
// Consult https://godoc.org/github.com/nightlyone/lockfile for the lockfile
// behaviour. It's magic. It deals with stale processes, and if there is
// a process keeping the lock busy, it will pass back a temporary error that
// we can spin on.
fi, err := os.OpenFile(glpath, os.O_CREATE|os.O_EXCL, 0600) // is 0600 sane for this purpose?
glpath := filepath.Join(cachedir, "sm.lock")
lockfile, err := lockfile.New(glpath)
if err != nil {
return nil, CouldNotCreateLockError{
Path: glpath,
Err: fmt.Errorf("err on attempting to create global cache lock: %s", err),
Err: fmt.Errorf("unable to create lock %s: %s", glpath, err.Error()),
}
}
process, err := lockfile.GetOwner()
if err == nil {
// If we didn't get an error, then the lockfile exists already. We should
// check to see if it's us already:
if process.Pid == os.Getpid() {
return nil, CouldNotCreateLockError{
Path: glpath,
Err: fmt.Errorf("lockfile %s already locked by this process", glpath),
}
}
// There is a lockfile, but it's owned by someone else. We'll try to lock
// it anyway.
}
// If it's a TemporaryError, we retry every second. Otherwise, we fail
// permanently.
//
// TODO: After some time, we should emit some kind of warning that we're waiting
// for the lockfile to be released. #534 should be address before we will do that.
err = lockfile.TryLock()
for err != nil {
if _, ok := err.(interface {
Temporary() bool
}); ok {
time.Sleep(time.Second * 1)
} else {
return nil, CouldNotCreateLockError{
Path: glpath,
Err: fmt.Errorf("unable to lock %s: %s", glpath, err.Error()),
}
}
err = lockfile.TryLock()
}
ctx, cf := context.WithCancel(context.TODO())
superv := newSupervisor(ctx)
deducer := newDeductionCoordinator(superv)
sm := &SourceMgr{
cachedir: cachedir,
lf: fi,
lf: &lockfile,
suprvsr: superv,
cancelAll: cf,
deduceCoord: deducer,
@ -307,7 +350,7 @@ func (sm *SourceMgr) doRelease() {
sm.suprvsr.wait()
// Close the file handle for the lock file and remove it from disk
sm.lf.Close()
sm.lf.Unlock()
os.Remove(filepath.Join(sm.cachedir, "sm.lock"))
// Close the qch, if non-nil, so the signal handlers run out. This will
@ -455,6 +498,76 @@ func (sm *SourceMgr) DeduceProjectRoot(ip string) (ProjectRoot, error) {
return ProjectRoot(pd.root), err
}
// InferConstraint tries to puzzle out what kind of version is given in a
// string. Preference is given first for revisions, then branches, then semver
// constraints, and then plain tags.
func (sm *SourceMgr) InferConstraint(s string, pi ProjectIdentifier) (Constraint, error) {
if s == "" {
return Any(), nil
}
slen := len(s)
if slen == 40 {
if _, err := hex.DecodeString(s); err == nil {
// Whether or not it's intended to be a SHA1 digest, this is a
// valid byte sequence for that, so go with Revision. This
// covers git and hg
return Revision(s), nil
}
}
// Next, try for bzr, which has a three-component GUID separated by
// dashes. There should be two, but the email part could contain
// internal dashes
if strings.Contains(s, "@") && strings.Count(s, "-") >= 2 {
// Work from the back to avoid potential confusion from the email
i3 := strings.LastIndex(s, "-")
// Skip if - is last char, otherwise this would panic on bounds err
if slen == i3+1 {
return NewVersion(s), nil
}
i2 := strings.LastIndex(s[:i3], "-")
if _, err := strconv.ParseUint(s[i2+1:i3], 10, 64); err == nil {
// Getting this far means it'd pretty much be nuts if it's not a
// bzr rev, so don't bother parsing the email.
return Revision(s), nil
}
}
// Lookup the string in the repository
var version PairedVersion
versions, err := sm.ListVersions(pi)
if err != nil {
return nil, errors.Wrapf(err, "list versions for %s(%s)", pi.ProjectRoot, pi.Source) // means repo does not exist
}
SortPairedForUpgrade(versions)
for _, v := range versions {
if s == v.String() {
version = v
break
}
}
// Branch
if version != nil && version.Type() == IsBranch {
return version.Unpair(), nil
}
// Semver Constraint
c, err := NewSemverConstraintIC(s)
if c != nil && err == nil {
return c, nil
}
// Tag
if version != nil {
return version.Unpair(), nil
}
return nil, errors.Errorf("%s is not a valid version for the package %s(%s)", s, pi.ProjectRoot, pi.Source)
}
type timeCount struct {
count int
start time.Time

47
cmd/dep/ensure_test.go → internal/gps/source_manager_test.go
Просмотреть файл

@ -2,43 +2,50 @@
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package main
package gps
import (
"reflect"
"testing"
"github.com/golang/dep/internal/gps"
"github.com/golang/dep/internal/test"
)
func TestDeduceConstraint(t *testing.T) {
func TestSourceManager_InferConstraint(t *testing.T) {
t.Parallel()
h := test.NewHelper(t)
cacheDir := "gps-repocache"
h.TempDir(cacheDir)
sm, err := gps.NewSourceManager(h.Path(cacheDir))
sm, err := NewSourceManager(h.Path(cacheDir))
h.Must(err)
sv, err := gps.NewSemverConstraintIC("v0.8.1")
sv, err := NewSemverConstraintIC("v0.8.1")
if err != nil {
t.Fatal(err)
}
constraints := map[string]gps.Constraint{
"v0.8.1": sv,
"master": gps.NewBranch("master"),
"5b3352dc16517996fb951394bcbbe913a2a616e3": gps.Revision("5b3352dc16517996fb951394bcbbe913a2a616e3"),
// valid bzr rev
"jess@linux.com-20161116211307-wiuilyamo9ian0m7": gps.Revision("jess@linux.com-20161116211307-wiuilyamo9ian0m7"),
// invalid bzr rev
"go4@golang.org-sadfasdf-": gps.NewVersion("go4@golang.org-sadfasdf-"),
svs, err := NewSemverConstraintIC("v0.12.0-12-de4dcafe0")
if err != nil {
t.Fatal(err)
}
pi := gps.ProjectIdentifier{ProjectRoot: "github.com/sdboyer/deptest"}
constraints := map[string]Constraint{
"": Any(),
"v0.8.1": sv,
"v2": NewBranch("v2"),
"v0.12.0-12-de4dcafe0": svs,
"master": NewBranch("master"),
"5b3352dc16517996fb951394bcbbe913a2a616e3": Revision("5b3352dc16517996fb951394bcbbe913a2a616e3"),
// valid bzr rev
"jess@linux.com-20161116211307-wiuilyamo9ian0m7": Revision("jess@linux.com-20161116211307-wiuilyamo9ian0m7"),
// invalid bzr rev
"go4@golang.org-sadfasdf-": NewVersion("go4@golang.org-sadfasdf-"),
}
pi := ProjectIdentifier{ProjectRoot: "github.com/carolynvs/deptest"}
for str, want := range constraints {
got, err := deduceConstraint(str, pi, sm)
got, err := sm.InferConstraint(str, pi)
h.Must(err)
wantT := reflect.TypeOf(want)
@ -52,12 +59,12 @@ func TestDeduceConstraint(t *testing.T) {
}
}
func TestDeduceConstraint_InvalidInput(t *testing.T) {
func TestSourceManager_InferConstraint_InvalidInput(t *testing.T) {
h := test.NewHelper(t)
cacheDir := "gps-repocache"
h.TempDir(cacheDir)
sm, err := gps.NewSourceManager(h.Path(cacheDir))
sm, err := NewSourceManager(h.Path(cacheDir))
h.Must(err)
constraints := []string{
@ -66,9 +73,9 @@ func TestDeduceConstraint_InvalidInput(t *testing.T) {
"20120425195858-psty8c35ve2oej8t",
}
pi := gps.ProjectIdentifier{ProjectRoot: "github.com/sdboyer/deptest"}
pi := ProjectIdentifier{ProjectRoot: "github.com/sdboyer/deptest"}
for _, str := range constraints {
_, err := deduceConstraint(str, pi, sm)
_, err := sm.InferConstraint(str, pi)
if err == nil {
t.Errorf("expected %s to produce an error", str)
}

34
internal/gps/vcs_repo.go
Просмотреть файл

@ -80,7 +80,7 @@ func newVcsLocalErrorOr(msg string, err error, out string) error {
}
func (r *gitRepo) get(ctx context.Context) error {
out, err := runFromCwd(ctx, "git", "clone", "--recursive", r.Remote(), r.LocalPath())
out, err := runFromCwd(ctx, expensiveCmdTimeout, "git", "clone", "--recursive", "-v", "--progress", r.Remote(), r.LocalPath())
if err != nil {
return newVcsRemoteErrorOr("unable to get repository", err, string(out))
}
@ -90,7 +90,7 @@ func (r *gitRepo) get(ctx context.Context) error {
func (r *gitRepo) fetch(ctx context.Context) error {
// Perform a fetch to make sure everything is up to date.
out, err := runFromRepoDir(ctx, r, "git", "fetch", "--tags", "--prune", r.RemoteLocation)
out, err := runFromRepoDir(ctx, r, expensiveCmdTimeout, "git", "fetch", "--tags", "--prune", r.RemoteLocation)
if err != nil {
return newVcsRemoteErrorOr("unable to update repository", err, string(out))
}
@ -98,7 +98,7 @@ func (r *gitRepo) fetch(ctx context.Context) error {
}
func (r *gitRepo) updateVersion(ctx context.Context, v string) error {
out, err := runFromRepoDir(ctx, r, "git", "checkout", v)
out, err := runFromRepoDir(ctx, r, defaultCmdTimeout, "git", "checkout", v)
if err != nil {
return newVcsLocalErrorOr("Unable to update checked out version", err, string(out))
}
@ -110,20 +110,20 @@ func (r *gitRepo) updateVersion(ctx context.Context, v string) error {
// submodules. Or nested submodules. What a great idea, submodules.
func (r *gitRepo) defendAgainstSubmodules(ctx context.Context) error {
// First, update them to whatever they should be, if there should happen to be any.
out, err := runFromRepoDir(ctx, r, "git", "submodule", "update", "--init", "--recursive")
out, err := runFromRepoDir(ctx, r, expensiveCmdTimeout, "git", "submodule", "update", "--init", "--recursive")
if err != nil {
return newVcsLocalErrorOr("unexpected error while defensively updating submodules", err, string(out))
}
// Now, do a special extra-aggressive clean in case changing versions caused
// one or more submodules to go away.
out, err = runFromRepoDir(ctx, r, "git", "clean", "-x", "-d", "-f", "-f")
out, err = runFromRepoDir(ctx, r, defaultCmdTimeout, "git", "clean", "-x", "-d", "-f", "-f")
if err != nil {
return newVcsLocalErrorOr("unexpected error while defensively cleaning up after possible derelict submodule directories", err, string(out))
}
// Then, repeat just in case there are any nested submodules that went away.
out, err = runFromRepoDir(ctx, r, "git", "submodule", "foreach", "--recursive", "git", "clean", "-x", "-d", "-f", "-f")
out, err = runFromRepoDir(ctx, r, defaultCmdTimeout, "git", "submodule", "foreach", "--recursive", "git", "clean", "-x", "-d", "-f", "-f")
if err != nil {
return newVcsLocalErrorOr("unexpected error while defensively cleaning up after possible derelict nested submodule directories", err, string(out))
}
@ -144,7 +144,7 @@ func (r *bzrRepo) get(ctx context.Context) error {
}
}
out, err := runFromCwd(ctx, "bzr", "branch", r.Remote(), r.LocalPath())
out, err := runFromCwd(ctx, expensiveCmdTimeout, "bzr", "branch", r.Remote(), r.LocalPath())
if err != nil {
return newVcsRemoteErrorOr("unable to get repository", err, string(out))
}
@ -153,7 +153,7 @@ func (r *bzrRepo) get(ctx context.Context) error {
}
func (r *bzrRepo) fetch(ctx context.Context) error {
out, err := runFromRepoDir(ctx, r, "bzr", "pull")
out, err := runFromRepoDir(ctx, r, expensiveCmdTimeout, "bzr", "pull")
if err != nil {
return newVcsRemoteErrorOr("unable to update repository", err, string(out))
}
@ -161,7 +161,7 @@ func (r *bzrRepo) fetch(ctx context.Context) error {
}
func (r *bzrRepo) updateVersion(ctx context.Context, version string) error {
out, err := runFromRepoDir(ctx, r, "bzr", "update", "-r", version)
out, err := runFromRepoDir(ctx, r, expensiveCmdTimeout, "bzr", "update", "-r", version)
if err != nil {
return newVcsLocalErrorOr("unable to update checked out version", err, string(out))
}
@ -173,7 +173,7 @@ type hgRepo struct {
}
func (r *hgRepo) get(ctx context.Context) error {
out, err := runFromCwd(ctx, "hg", "clone", r.Remote(), r.LocalPath())
out, err := runFromCwd(ctx, expensiveCmdTimeout, "hg", "clone", r.Remote(), r.LocalPath())
if err != nil {
return newVcsRemoteErrorOr("unable to get repository", err, string(out))
}
@ -182,7 +182,7 @@ func (r *hgRepo) get(ctx context.Context) error {
}
func (r *hgRepo) fetch(ctx context.Context) error {
out, err := runFromRepoDir(ctx, r, "hg", "pull")
out, err := runFromRepoDir(ctx, r, expensiveCmdTimeout, "hg", "pull")
if err != nil {
return newVcsRemoteErrorOr("unable to fetch latest changes", err, string(out))
}
@ -190,7 +190,7 @@ func (r *hgRepo) fetch(ctx context.Context) error {
}
func (r *hgRepo) updateVersion(ctx context.Context, version string) error {
out, err := runFromRepoDir(ctx, r, "hg", "update", version)
out, err := runFromRepoDir(ctx, r, expensiveCmdTimeout, "hg", "update", version)
if err != nil {
return newVcsRemoteErrorOr("unable to update checked out version", err, string(out))
}
@ -210,7 +210,7 @@ func (r *svnRepo) get(ctx context.Context) error {
remote = "file:///" + remote
}
out, err := runFromCwd(ctx, "svn", "checkout", remote, r.LocalPath())
out, err := runFromCwd(ctx, expensiveCmdTimeout, "svn", "checkout", remote, r.LocalPath())
if err != nil {
return newVcsRemoteErrorOr("unable to get repository", err, string(out))
}
@ -219,7 +219,7 @@ func (r *svnRepo) get(ctx context.Context) error {
}
func (r *svnRepo) update(ctx context.Context) error {
out, err := runFromRepoDir(ctx, r, "svn", "update")
out, err := runFromRepoDir(ctx, r, expensiveCmdTimeout, "svn", "update")
if err != nil {
return newVcsRemoteErrorOr("unable to update repository", err, string(out))
}
@ -228,7 +228,7 @@ func (r *svnRepo) update(ctx context.Context) error {
}
func (r *svnRepo) updateVersion(ctx context.Context, version string) error {
out, err := runFromRepoDir(ctx, r, "svn", "update", "-r", version)
out, err := runFromRepoDir(ctx, r, expensiveCmdTimeout, "svn", "update", "-r", version)
if err != nil {
return newVcsRemoteErrorOr("unable to update checked out version", err, string(out))
}
@ -250,7 +250,7 @@ func (r *svnRepo) CommitInfo(id string) (*vcs.CommitInfo, error) {
Commit commit `xml:"entry>commit"`
}
out, err := runFromRepoDir(ctx, r, "svn", "info", "-r", id, "--xml")
out, err := runFromRepoDir(ctx, r, defaultCmdTimeout, "svn", "info", "-r", id, "--xml")
if err != nil {
return nil, newVcsLocalErrorOr("unable to retrieve commit information", err, string(out))
}
@ -267,7 +267,7 @@ func (r *svnRepo) CommitInfo(id string) (*vcs.CommitInfo, error) {
}
}
out, err := runFromRepoDir(ctx, r, "svn", "log", "-r", id, "--xml")
out, err := runFromRepoDir(ctx, r, defaultCmdTimeout, "svn", "log", "-r", id, "--xml")
if err != nil {
return nil, newVcsRemoteErrorOr("unable to retrieve commit information", err, string(out))
}

43
internal/gps/vcs_source.go
Просмотреть файл

@ -107,9 +107,6 @@ func (bs *baseVCSSource) exportRevisionTo(ctx context.Context, r Revision, to st
return unwrapVcsErr(err)
}
// TODO(sdboyer) this is a simplistic approach and relying on the tools
// themselves might make it faster, but git's the overwhelming case (and has
// its own method) so fine for now
return fs.CopyDir(bs.repo.LocalPath(), to)
}
@ -136,7 +133,7 @@ func (s *gitSource) exportRevisionTo(ctx context.Context, rev Revision, to strin
// could have an err here...but it's hard to imagine how?
defer fs.RenameWithFallback(bak, idx)
out, err := runFromRepoDir(ctx, r, "git", "read-tree", rev.String())
out, err := runFromRepoDir(ctx, r, defaultCmdTimeout, "git", "read-tree", rev.String())
if err != nil {
return fmt.Errorf("%s: %s", out, err)
}
@ -152,7 +149,7 @@ func (s *gitSource) exportRevisionTo(ctx context.Context, rev Revision, to strin
// though we have a bunch of housekeeping to do to set up, then tear
// down, the sparse checkout controls, as well as restore the original
// index and HEAD.
out, err = runFromRepoDir(ctx, r, "git", "checkout-index", "-a", "--prefix="+to)
out, err = runFromRepoDir(ctx, r, defaultCmdTimeout, "git", "checkout-index", "-a", "--prefix="+to)
if err != nil {
return fmt.Errorf("%s: %s", out, err)
}
@ -353,6 +350,18 @@ type bzrSource struct {
baseVCSSource
}
func (s *bzrSource) exportRevisionTo(ctx context.Context, rev Revision, to string) error {
if err := s.baseVCSSource.exportRevisionTo(ctx, rev, to); err != nil {
return err
}
if err := os.RemoveAll(filepath.Join(to, ".bzr")); err != nil {
return err
}
return nil
}
func (s *bzrSource) listVersions(ctx context.Context) ([]PairedVersion, error) {
r := s.repo
@ -365,7 +374,7 @@ func (s *bzrSource) listVersions(ctx context.Context) ([]PairedVersion, error) {
}
// Now, list all the tags
out, err := runFromRepoDir(ctx, r, "bzr", "tags", "--show-ids", "-v")
out, err := runFromRepoDir(ctx, r, defaultCmdTimeout, "bzr", "tags", "--show-ids", "-v")
if err != nil {
return nil, fmt.Errorf("%s: %s", err, string(out))
}
@ -373,7 +382,7 @@ func (s *bzrSource) listVersions(ctx context.Context) ([]PairedVersion, error) {
all := bytes.Split(bytes.TrimSpace(out), []byte("\n"))
var branchrev []byte
branchrev, err = runFromRepoDir(ctx, r, "bzr", "version-info", "--custom", "--template={revision_id}", "--revision=branch:.")
branchrev, err = runFromRepoDir(ctx, r, defaultCmdTimeout, "bzr", "version-info", "--custom", "--template={revision_id}", "--revision=branch:.")
br := string(branchrev)
if err != nil {
return nil, fmt.Errorf("%s: %s", err, br)
@ -403,6 +412,20 @@ type hgSource struct {
baseVCSSource
}
func (s *hgSource) exportRevisionTo(ctx context.Context, rev Revision, to string) error {
// TODO: use hg instead of the generic approach in
// baseVCSSource.exportRevisionTo to make it faster.
if err := s.baseVCSSource.exportRevisionTo(ctx, rev, to); err != nil {
return err
}
if err := os.RemoveAll(filepath.Join(to, ".hg")); err != nil {
return err
}
return nil
}
func (s *hgSource) listVersions(ctx context.Context) ([]PairedVersion, error) {
var vlist []PairedVersion
@ -416,7 +439,7 @@ func (s *hgSource) listVersions(ctx context.Context) ([]PairedVersion, error) {
}
// Now, list all the tags
out, err := runFromRepoDir(ctx, r, "hg", "tags", "--debug", "--verbose")
out, err := runFromRepoDir(ctx, r, defaultCmdTimeout, "hg", "tags", "--debug", "--verbose")
if err != nil {
return nil, fmt.Errorf("%s: %s", err, string(out))
}
@ -450,7 +473,7 @@ func (s *hgSource) listVersions(ctx context.Context) ([]PairedVersion, error) {
// bookmarks next, because the presence of the magic @ bookmark has to
// determine how we handle the branches
var magicAt bool
out, err = runFromRepoDir(ctx, r, "hg", "bookmarks", "--debug")
out, err = runFromRepoDir(ctx, r, defaultCmdTimeout, "hg", "bookmarks", "--debug")
if err != nil {
// better nothing than partial and misleading
return nil, fmt.Errorf("%s: %s", err, string(out))
@ -483,7 +506,7 @@ func (s *hgSource) listVersions(ctx context.Context) ([]PairedVersion, error) {
}
}
out, err = runFromRepoDir(ctx, r, "hg", "branches", "-c", "--debug")
out, err = runFromRepoDir(ctx, r, defaultCmdTimeout, "hg", "branches", "-c", "--debug")
if err != nil {
// better nothing than partial and misleading
return nil, fmt.Errorf("%s: %s", err, string(out))

116
internal/gps/vcs_source_test.go
Просмотреть файл

@ -8,11 +8,15 @@ import (
"context"
"io/ioutil"
"net/url"
"os"
"os/exec"
"path/filepath"
"reflect"
"strings"
"sync"
"testing"
"github.com/golang/dep/internal/test"
)
// Parent test that executes all the slow vcs interaction tests in parallel.
@ -321,7 +325,7 @@ func testBzrSourceInteractions(t *testing.T) {
err = isrc.initLocal(ctx)
if err != nil {
t.Fatalf("Error on cloning git repo: %s", err)
t.Fatalf("Error on cloning bzr repo: %s", err)
}
src, ok := isrc.(*bzrSource)
@ -433,7 +437,7 @@ func testHgSourceInteractions(t *testing.T) {
err = isrc.initLocal(ctx)
if err != nil {
t.Fatalf("Error on cloning git repo: %s", err)
t.Fatalf("Error on cloning hg repo: %s", err)
}
src, ok := isrc.(*hgSource)
@ -520,6 +524,114 @@ func testHgSourceInteractions(t *testing.T) {
<-donech
}
func Test_bzrSource_exportRevisionTo_removeVcsFiles(t *testing.T) {
t.Parallel()
// This test is slow, so skip it on -short
if testing.Short() {
t.Skip("Skipping hg source version fetching test in short mode")
}
requiresBins(t, "bzr")
h := test.NewHelper(t)
defer h.Cleanup()
h.TempDir("smcache")
cpath := h.Path("smcache")
repoPath := filepath.Join(h.Path("."), "repo")
rev := Revision("matt@mattfarina.com-20150731135137-pbphasfppmygpl68")
n := "launchpad.net/govcstestbzrrepo"
un := "https://" + n
u, err := url.Parse(un)
if err != nil {
t.Errorf("URL was bad, lolwut? errtext: %s", err)
return
}
mb := maybeBzrSource{u}
ctx := context.Background()
superv := newSupervisor(ctx)
isrc, _, err := mb.try(ctx, cpath, newMemoryCache(), superv)
if err != nil {
t.Fatalf("unexpected error while setting up hgSource for test repo: %s", err)
}
err = isrc.initLocal(ctx)
if err != nil {
t.Fatalf("Error on cloning bzr repo: %s", err)
}
src, ok := isrc.(*bzrSource)
if !ok {
t.Fatalf("expected a bzrSource, got a %T", isrc)
}
if err := src.exportRevisionTo(ctx, rev, repoPath); err != nil {
t.Fatalf("unexpected error: %v", err)
}
_, err = os.Stat(filepath.Join(repoPath, ".bzr"))
if err == nil {
t.Fatal("expected .bzr/ to not exists")
} else if !os.IsNotExist(err) {
t.Fatalf("unexpected error: %v", err)
}
}
func Test_hgSource_exportRevisionTo_removeVcsFiles(t *testing.T) {
t.Parallel()
// This test is slow, so skip it on -short
if testing.Short() {
t.Skip("Skipping hg source version fetching test in short mode")
}
requiresBins(t, "hg")
h := test.NewHelper(t)
defer h.Cleanup()
h.TempDir("smcache")
cpath := h.Path("smcache")
repoPath := filepath.Join(h.Path("."), "repo")
rev := Revision("6f55e1f03d91f8a7cce35d1968eb60a2352e4d59")
n := "bitbucket.org/golang-dep/dep-test"
un := "https://" + n
u, err := url.Parse(un)
if err != nil {
t.Errorf("URL was bad, lolwut? errtext: %s", err)
return
}
mb := maybeHgSource{u}
ctx := context.Background()
superv := newSupervisor(ctx)
isrc, _, err := mb.try(ctx, cpath, newMemoryCache(), superv)
if err != nil {
t.Fatalf("unexpected error while setting up hgSource for test repo: %s", err)
}
err = isrc.initLocal(ctx)
if err != nil {
t.Fatalf("Error on cloning hg repo: %s", err)
}
src, ok := isrc.(*hgSource)
if !ok {
t.Fatalf("expected a hgSource, got a %T", isrc)
}
if err := src.exportRevisionTo(ctx, rev, repoPath); err != nil {
t.Fatalf("unexpected error: %v", err)
}
_, err = os.Stat(filepath.Join(repoPath, ".hg"))
if err == nil {
t.Fatal("expected .hg/ to not exists")
} else if !os.IsNotExist(err) {
t.Fatalf("unexpected error: %v", err)
}
}
// Fail a test if the specified binaries aren't installed.
func requiresBins(t *testing.T, bins ...string) {
for _, b := range bins {

67
internal/gps/vcs_version.go Normal file
Просмотреть файл

@ -0,0 +1,67 @@
// Copyright 2017 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package gps
import (
"strings"
"github.com/Masterminds/vcs"
"github.com/pkg/errors"
)
// VCSVersion returns the current project version for an absolute path.
func VCSVersion(path string) (Version, error) {
repo, err := vcs.NewRepo("", path)
if err != nil {
return nil, errors.Wrapf(err, "creating new repo for root: %s", path)
}
ver, err := repo.Current()
if err != nil {
return nil, errors.Wrapf(err, "finding current branch/version for root: %s", path)
}
rev, err := repo.Version()
if err != nil {
return nil, errors.Wrapf(err, "getting repo version for root: %s", path)
}
// First look through tags.
tags, err := repo.Tags()
if err != nil {
return nil, errors.Wrapf(err, "getting repo tags for root: %s", path)
}
// Try to match the current version to a tag.
if contains(tags, ver) {
// Assume semver if it starts with a v.
if strings.HasPrefix(ver, "v") {
return NewVersion(ver).Pair(Revision(rev)), nil
}
return nil, errors.Errorf("version for root %s does not start with a v: %q", path, ver)
}
// Look for the current branch.
branches, err := repo.Branches()
if err != nil {
return nil, errors.Wrapf(err, "getting repo branch for root: %s")
}
// Try to match the current version to a branch.
if contains(branches, ver) {
return NewBranch(ver).Pair(Revision(rev)), nil
}
return Revision(rev), nil
}
// contains checks if a array of strings contains a value
func contains(a []string, b string) bool {
for _, v := range a {
if b == v {
return true
}
}
return false
}

57
internal/gps/vcs_version_test.go Normal file
Просмотреть файл

@ -0,0 +1,57 @@
// Copyright 2017 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package gps
import (
"path/filepath"
"testing"
"github.com/golang/dep/internal/test"
)
func TestVCSVersion(t *testing.T) {
test.NeedsExternalNetwork(t)
test.NeedsGit(t)
h := test.NewHelper(t)
defer h.Cleanup()
h.TempDir("src")
gopath := h.Path(".")
h.Setenv("GOPATH", gopath)
importPaths := map[string]struct {
rev Version
checkout bool
}{
"github.com/pkg/errors": {
rev: NewVersion("v0.8.0").Pair("645ef00459ed84a119197bfb8d8205042c6df63d"), // semver
checkout: true,
},
"github.com/sirupsen/logrus": {
rev: Revision("42b84f9ec624953ecbf81a94feccb3f5935c5edf"), // random sha
checkout: true,
},
"github.com/rsc/go-get-default-branch": {
rev: NewBranch("another-branch").Pair("8e6902fdd0361e8fa30226b350e62973e3625ed5"),
},
}
// checkout the specified revisions
for ip, info := range importPaths {
h.RunGo("get", ip)
repoDir := h.Path("src/" + ip)
if info.checkout {
h.RunGit(repoDir, "checkout", info.rev.String())
}
abs := filepath.FromSlash(filepath.Join(gopath, "src", ip))
got, err := VCSVersion(abs)
h.Must(err)
if got != info.rev {
t.Fatalf("expected %q, got %q", got.String(), info.rev.String())
}
}
}

43
internal/gps/version.go
Просмотреть файл

@ -190,6 +190,14 @@ func (r Revision) Intersect(c Constraint) Constraint {
return none
}
func (r Revision) identical(c Constraint) bool {
r2, ok := c.(Revision)
if !ok {
return false
}
return r == r2
}
type branchVersion struct {
name string
isDefault bool
@ -274,6 +282,14 @@ func (v branchVersion) Pair(r Revision) PairedVersion {
}
}
func (v branchVersion) identical(c Constraint) bool {
v2, ok := c.(branchVersion)
if !ok {
return false
}
return v == v2
}
type plainVersion string
func (v plainVersion) String() string {
@ -355,6 +371,14 @@ func (v plainVersion) Pair(r Revision) PairedVersion {
}
}
func (v plainVersion) identical(c Constraint) bool {
v2, ok := c.(plainVersion)
if !ok {
return false
}
return v == v2
}
type semVersion struct {
sv semver.Version
}
@ -446,6 +470,14 @@ func (v semVersion) Pair(r Revision) PairedVersion {
}
}
func (v semVersion) identical(c Constraint) bool {
v2, ok := c.(semVersion)
if !ok {
return false
}
return v == v2
}
type versionPair struct {
v UnpairedVersion
r Revision
@ -547,6 +579,17 @@ func (v versionPair) Intersect(c2 Constraint) Constraint {
return none
}
func (v versionPair) identical(c Constraint) bool {
v2, ok := c.(versionPair)
if !ok {
return false
}
if v.r != v2.r {
return false
}
return v.v.identical(v2.v)
}
// compareVersionType is a sort func helper that makes a coarse-grained sorting
// decision based on version type.
//

25
internal/gps/version_unifier.go
Просмотреть файл

@ -269,3 +269,28 @@ func (vtu versionTypeUnion) Intersect(c Constraint) Constraint {
return none
}
func (vtu versionTypeUnion) identical(c Constraint) bool {
vtu2, ok := c.(versionTypeUnion)
if !ok {
return false
}
if len(vtu) != len(vtu2) {
return false
}
used := make([]bool, len(vtu))
outter:
for _, v := range vtu {
for i, v2 := range vtu2 {
if used[i] {
continue
}
if v.identical(v2) {
used[i] = true
continue outter
}
}
return false
}
return true
}

27
internal/test/integration_testproj.go
Просмотреть файл

@ -50,16 +50,7 @@ func NewTestProject(t *testing.T, initPath, wd string, externalProc bool, run Ru
new.TempDir(ProjectRoot, "vendor")
new.CopyTree(initPath)
// Note that the Travis darwin platform, directories with certain roots such
// as /var are actually links to a dirtree under /private. Without the patch
// below the wd, and therefore the GOPATH, is recorded as "/var/..." but the
// actual process runs in "/private/var/..." and dies due to not being in the
// GOPATH because the roots don't line up.
if externalProc && runtime.GOOS == "darwin" && needsPrivateLeader(new.tempdir) {
new.Setenv("GOPATH", filepath.Join("/private", new.tempdir))
} else {
new.Setenv("GOPATH", new.tempdir)
}
return new
}
@ -283,6 +274,12 @@ func (p *IntegrationTestProject) makeRootTempDir() {
var err error
p.tempdir, err = ioutil.TempDir("", "gotest")
p.Must(err)
// Fix for OSX where the tempdir is a symlink:
if runtime.GOOS == "darwin" {
p.tempdir, err = filepath.EvalSymlinks(p.tempdir)
p.Must(err)
}
}
}
@ -298,15 +295,3 @@ func (p *IntegrationTestProject) Must(err error) {
p.t.Fatalf("%+v", err)
}
}
// Checks for filepath beginnings that result in the "/private" leader
// on Mac platforms
func needsPrivateLeader(path string) bool {
var roots = []string{"/var", "/tmp", "/etc"}
for _, root := range roots {
if strings.HasPrefix(path, root) {
return true
}
}
return false
}

6
manifest.go
Просмотреть файл

@ -316,12 +316,6 @@ func (m *Manifest) DependencyConstraints() gps.ProjectConstraints {
return m.Constraints
}
// TestDependencyConstraints remains unimplemented by returning nil for now.
func (m *Manifest) TestDependencyConstraints() gps.ProjectConstraints {
// TODO decide whether we're going to incorporate this or not
return nil
}
// Overrides returns a list of project-level override constraints.
func (m *Manifest) Overrides() gps.ProjectConstraints {
return m.Ovr

2
project.go
Просмотреть файл

@ -87,7 +87,7 @@ func (p *Project) MakeParams() gps.SolveParameters {
func BackupVendor(vpath, suffix string) (string, error) {
// Check if there's a non-empty vendor directory
vendorExists, err := fs.IsNonEmptyDir(vpath)
if err != nil {
if err != nil && !os.IsNotExist(err) {
return "", err
}
if vendorExists {

71
testdata/txn_writer/expected_manifest.toml поставляемый
Просмотреть файл

@ -1,69 +1,24 @@
## Gopkg.toml example (these lines may be deleted)
## "metadata" defines metadata about the project that could be used by other independent
## systems. The metadata defined here will be ignored by dep.
# [metadata]
# key1 = "value that convey data to other systems"
# system1-data = "value that is used by a system"
# system2-data = "value that is used by another system"
## "required" lists a set of packages (not projects) that must be included in
## Gopkg.lock. This list is merged with the set of packages imported by the current
## project. Use it when your project needs a package it doesn't explicitly import -
## including "main" packages.
# Gopkg.toml example
#
# Refer to https://github.com/golang/dep/blob/master/docs/Gopkg.toml.md
# for detailed Gopkg.toml documentation.
#
# required = ["github.com/user/thing/cmd/thing"]
## "ignored" lists a set of packages (not projects) that are ignored when
## dep statically analyzes source code. Ignored packages can be in this project,
## or in a dependency.
# ignored = ["github.com/user/project/badpkg"]
## Constraints are rules for how directly imported projects
## may be incorporated into the depgraph. They are respected by
## dep whether coming from the Gopkg.toml of the current project or a dependency.
# ignored = ["github.com/user/project/pkgX", "bitbucket.org/user/project/pkgA/pkgY"]
#
# [[constraint]]
## Required: the root import path of the project being constrained.
# name = "github.com/user/project"
#
## Recommended: the version constraint to enforce for the project.
## Only one of "branch", "version" or "revision" can be specified.
# version = "1.0.0"
# branch = "master"
# revision = "abc123"
#
## Optional: an alternate location (URL or import path) for the project's source.
# source = "https://github.com/myfork/package.git"
# [[constraint]]
# name = "github.com/user/project2"
# branch = "dev"
# source = "github.com/myfork/project2"
#
## "metadata" defines metadata about the dependency or override that could be used
## by other independent systems. The metadata defined here will be ignored by dep.
# [metadata]
# key1 = "value that convey data to other systems"
# system1-data = "value that is used by a system"
# system2-data = "value that is used by another system"
## Overrides have the same structure as [[constraint]], but supersede all
## [[constraint]] declarations from all projects. Only [[override]] from
## the current project's are applied.
##
## Overrides are a sledgehammer. Use them only as a last resort.
# [[override]]
## Required: the root import path of the project being constrained.
# name = "github.com/user/project"
#
## Optional: specifying a version constraint override will cause all other
## constraints on this project to be ignored; only the overridden constraint
## need be satisfied.
## Again, only one of "branch", "version" or "revision" can be specified.
# version = "1.0.0"
# branch = "master"
# revision = "abc123"
#
## Optional: specifying an alternate source location as an override will
## enforce that the alternate location is used for that project, regardless of
## what source location any dependent projects specify.
# source = "https://github.com/myfork/package.git"
# name = "github.com/x/y"
# version = "2.4.0"
[[constraint]]

200
txn_writer.go
Просмотреть файл

@ -11,8 +11,6 @@ import (
"log"
"os"
"path/filepath"
"sort"
"strings"
"github.com/golang/dep/internal/fs"
"github.com/golang/dep/internal/gps"
@ -24,71 +22,26 @@ import (
// if no dependencies are found in the project
// during `dep init`
var exampleTOML = []byte(`
## Gopkg.toml example (these lines may be deleted)
## "metadata" defines metadata about the project that could be used by other independent
## systems. The metadata defined here will be ignored by dep.
# [metadata]
# key1 = "value that convey data to other systems"
# system1-data = "value that is used by a system"
# system2-data = "value that is used by another system"
## "required" lists a set of packages (not projects) that must be included in
## Gopkg.lock. This list is merged with the set of packages imported by the current
## project. Use it when your project needs a package it doesn't explicitly import -
## including "main" packages.
# Gopkg.toml example
#
# Refer to https://github.com/golang/dep/blob/master/docs/Gopkg.toml.md
# for detailed Gopkg.toml documentation.
#
# required = ["github.com/user/thing/cmd/thing"]
## "ignored" lists a set of packages (not projects) that are ignored when
## dep statically analyzes source code. Ignored packages can be in this project,
## or in a dependency.
# ignored = ["github.com/user/project/badpkg"]
## Constraints are rules for how directly imported projects
## may be incorporated into the depgraph. They are respected by
## dep whether coming from the Gopkg.toml of the current project or a dependency.
# ignored = ["github.com/user/project/pkgX", "bitbucket.org/user/project/pkgA/pkgY"]
#
# [[constraint]]
## Required: the root import path of the project being constrained.
# name = "github.com/user/project"
#
## Recommended: the version constraint to enforce for the project.
## Only one of "branch", "version" or "revision" can be specified.
# version = "1.0.0"
# branch = "master"
# revision = "abc123"
#
## Optional: an alternate location (URL or import path) for the project's source.
# source = "https://github.com/myfork/package.git"
# [[constraint]]
# name = "github.com/user/project2"
# branch = "dev"
# source = "github.com/myfork/project2"
#
## "metadata" defines metadata about the dependency or override that could be used
## by other independent systems. The metadata defined here will be ignored by dep.
# [metadata]
# key1 = "value that convey data to other systems"
# system1-data = "value that is used by a system"
# system2-data = "value that is used by another system"
## Overrides have the same structure as [[constraint]], but supersede all
## [[constraint]] declarations from all projects. Only [[override]] from
## the current project's are applied.
##
## Overrides are a sledgehammer. Use them only as a last resort.
# [[override]]
## Required: the root import path of the project being constrained.
# name = "github.com/user/project"
#
## Optional: specifying a version constraint override will cause all other
## constraints on this project to be ignored; only the overridden constraint
## need be satisfied.
## Again, only one of "branch", "version" or "revision" can be specified.
# version = "1.0.0"
# branch = "master"
# revision = "abc123"
#
## Optional: specifying an alternate source location as an override will
## enforce that the alternate location is used for that project, regardless of
## what source location any dependent projects specify.
# source = "https://github.com/myfork/package.git"
# name = "github.com/x/y"
# version = "2.4.0"
`)
@ -111,7 +64,8 @@ type SafeWriter struct {
writeLock bool
}
// NewSafeWriter sets up a SafeWriter to write a set of config yaml, lock and vendor tree.
// NewSafeWriter sets up a SafeWriter to write a set of manifest, lock, and
// vendor tree.
//
// - If manifest is provided, it will be written to the standard manifest file
// name beneath root.
@ -286,7 +240,7 @@ func (sw SafeWriter) validate(root string, sm gps.SourceManager) error {
return errors.New("root path must be non-empty")
}
if is, err := fs.IsDir(root); !is {
if err != nil {
if err != nil && !os.IsNotExist(err) {
return err
}
return errors.Errorf("root path %q does not exist", root)
@ -508,131 +462,9 @@ func (sw *SafeWriter) PrintPreparedActions(output *log.Logger) error {
return nil
}
// PruneProject removes unused packages from a project.
func PruneProject(p *Project, sm gps.SourceManager, logger *log.Logger) error {
td, err := ioutil.TempDir(os.TempDir(), "dep")
if err != nil {
return errors.Wrap(err, "error while creating temp dir for writing manifest/lock/vendor")
}
defer os.RemoveAll(td)
if err := gps.WriteDepTree(td, p.Lock, sm, true); err != nil {
return err
}
var toKeep []string
for _, project := range p.Lock.Projects() {
projectRoot := string(project.Ident().ProjectRoot)
for _, pkg := range project.Packages() {
toKeep = append(toKeep, filepath.Join(projectRoot, pkg))
}
}
toDelete, err := calculatePrune(td, toKeep, logger)
if err != nil {
return err
}
if logger != nil {
if len(toDelete) > 0 {
logger.Println("Calculated the following directories to prune:")
for _, d := range toDelete {
logger.Printf(" %s\n", d)
}
} else {
logger.Println("No directories found to prune")
}
}
if err := deleteDirs(toDelete); err != nil {
return err
}
vpath := filepath.Join(p.AbsRoot, "vendor")
vendorbak := vpath + ".orig"
var failerr error
if _, err := os.Stat(vpath); err == nil {
// Move out the old vendor dir. just do it into an adjacent dir, to
// try to mitigate the possibility of a pointless cross-filesystem
// move with a temp directory.
if _, err := os.Stat(vendorbak); err == nil {
// If the adjacent dir already exists, bite the bullet and move
// to a proper tempdir.
vendorbak = filepath.Join(td, "vendor.orig")
}
failerr = fs.RenameWithFallback(vpath, vendorbak)
if failerr != nil {
goto fail
}
}
// Move in the new one.
failerr = fs.RenameWithFallback(td, vpath)
if failerr != nil {
goto fail
}
os.RemoveAll(vendorbak)
return nil
fail:
fs.RenameWithFallback(vendorbak, vpath)
return failerr
}
func calculatePrune(vendorDir string, keep []string, logger *log.Logger) ([]string, error) {
if logger != nil {
logger.Println("Calculating prune. Checking the following packages:")
}
sort.Strings(keep)
toDelete := []string{}
err := filepath.Walk(vendorDir, func(path string, info os.FileInfo, err error) error {
if _, err := os.Lstat(path); err != nil {
return nil
}
if !info.IsDir() {
return nil
}
if path == vendorDir {
return nil
}
name := strings.TrimPrefix(path, vendorDir+"/")
if logger != nil {
logger.Printf(" %s", name)
}
i := sort.Search(len(keep), func(i int) bool {
return name <= keep[i]
})
if i >= len(keep) || !strings.HasPrefix(keep[i], name) {
toDelete = append(toDelete, path)
}
return nil
})
return toDelete, err
}
func deleteDirs(toDelete []string) error {
// sort by length so we delete sub dirs first
sort.Sort(byLen(toDelete))
for _, path := range toDelete {
if err := os.RemoveAll(path); err != nil {
return err
}
}
return nil
}
// hasDotGit checks if a given path has .git file or directory in it.
func hasDotGit(path string) bool {
gitfilepath := filepath.Join(path, ".git")
_, err := os.Stat(gitfilepath)
return err == nil
}
type byLen []string
func (a byLen) Len() int { return len(a) }
func (a byLen) Swap(i, j int) { a[i], a[j] = a[j], a[i] }
func (a byLen) Less(i, j int) bool { return len(a[i]) > len(a[j]) }

27
vendor/github.com/nightlyone/lockfile/.gitignore сгенерированный поставляемый Normal file
Просмотреть файл

@ -0,0 +1,27 @@
# Compiled Object files, Static and Dynamic libs (Shared Objects)
*.o
*.a
*.so
# Folders
_obj
_test
# popular temporaries
.err
.out
.diff
# Architecture specific extensions/prefixes
*.[568vq]
[568vq].out
*.cgo1.go
*.cgo2.c
_cgo_defun.c
_cgo_gotypes.go
_cgo_export.*
_testmain.go
*.exe

3
vendor/github.com/nightlyone/lockfile/.gitmodules сгенерированный поставляемый Normal file
Просмотреть файл

@ -0,0 +1,3 @@
[submodule "git-hooks"]
path = git-hooks
url = https://github.com/nightlyone/git-hooks

14
vendor/github.com/nightlyone/lockfile/.travis.yml сгенерированный поставляемый Normal file
Просмотреть файл

@ -0,0 +1,14 @@
language: go
go:
- 1.4.3
- 1.6.2
- tip
# Only test commits to production branch and all pull requests
branches:
only:
- master
matrix:
allow_failures:
- go: tip

19
vendor/github.com/nightlyone/lockfile/LICENSE сгенерированный поставляемый Normal file
Просмотреть файл

@ -0,0 +1,19 @@
Copyright (c) 2012 Ingo Oeser
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.

52
vendor/github.com/nightlyone/lockfile/README.md сгенерированный поставляемый Normal file
Просмотреть файл

@ -0,0 +1,52 @@
lockfile
=========
Handle locking via pid files.
[![Build Status Unix][1]][2]
[![Build status Windows][3]][4]
[1]: https://secure.travis-ci.org/nightlyone/lockfile.png
[2]: https://travis-ci.org/nightlyone/lockfile
[3]: https://ci.appveyor.com/api/projects/status/7mojkmauj81uvp8u/branch/master?svg=true
[4]: https://ci.appveyor.com/project/nightlyone/lockfile/branch/master
install
-------
Install [Go 1][5], either [from source][6] or [with a prepackaged binary][7].
For Windows suport, Go 1.4 or newer is required.
Then run
go get github.com/nightlyone/lockfile
[5]: http://golang.org
[6]: http://golang.org/doc/install/source
[7]: http://golang.org/doc/install
LICENSE
-------
MIT
documentation
-------------
[package documentation at godoc.org](http://godoc.org/github.com/nightlyone/lockfile)
install
-------------------
go get github.com/nightlyone/lockfile
contributing
============
Contributions are welcome. Please open an issue or send me a pull request for a dedicated branch.
Make sure the git commit hooks show it works.
git commit hooks
-----------------------
enable commit hooks via
cd .git ; rm -rf hooks; ln -s ../git-hooks hooks ; cd ..

12
vendor/github.com/nightlyone/lockfile/appveyor.yml сгенерированный поставляемый Normal file
Просмотреть файл

@ -0,0 +1,12 @@
clone_folder: c:\gopath\src\github.com\nightlyone\lockfile
environment:
GOPATH: c:\gopath
install:
- go version
- go env
- go get -v -t ./...
build_script:
- go test -v ./...

201
vendor/github.com/nightlyone/lockfile/lockfile.go сгенерированный поставляемый Normal file
Просмотреть файл

@ -0,0 +1,201 @@
// Package lockfile handles pid file based locking.
// While a sync.Mutex helps against concurrency issues within a single process,
// this package is designed to help against concurrency issues between cooperating processes
// or serializing multiple invocations of the same process. You can also combine sync.Mutex
// with Lockfile in order to serialize an action between different goroutines in a single program
// and also multiple invocations of this program.
package lockfile
import (
"errors"
"fmt"
"io"
"io/ioutil"
"os"
"path/filepath"
)
// Lockfile is a pid file which can be locked
type Lockfile string
// TemporaryError is a type of error where a retry after a random amount of sleep should help to mitigate it.
type TemporaryError string
func (t TemporaryError) Error() string { return string(t) }
// Temporary returns always true.
// It exists, so you can detect it via
// if te, ok := err.(interface{ Temporary() bool }); ok {
// fmt.Println("I am a temporay error situation, so wait and retry")
// }
func (t TemporaryError) Temporary() bool { return true }
// Various errors returned by this package
var (
ErrBusy = TemporaryError("Locked by other process") // If you get this, retry after a short sleep might help
ErrNotExist = TemporaryError("Lockfile created, but doesn't exist") // If you get this, retry after a short sleep might help
ErrNeedAbsPath = errors.New("Lockfiles must be given as absolute path names")
ErrInvalidPid = errors.New("Lockfile contains invalid pid for system")
ErrDeadOwner = errors.New("Lockfile contains pid of process not existent on this system anymore")
ErrRogueDeletion = errors.New("Lockfile owned by me has been removed unexpectedly")
)
// New describes a new filename located at the given absolute path.
func New(path string) (Lockfile, error) {
if !filepath.IsAbs(path) {
return Lockfile(""), ErrNeedAbsPath
}
return Lockfile(path), nil
}
// GetOwner returns who owns the lockfile.
func (l Lockfile) GetOwner() (*os.Process, error) {
name := string(l)
// Ok, see, if we have a stale lockfile here
content, err := ioutil.ReadFile(name)
if err != nil {
return nil, err
}
// try hard for pids. If no pid, the lockfile is junk anyway and we delete it.
pid, err := scanPidLine(content)
if err != nil {
return nil, err
}
running, err := isRunning(pid)
if err != nil {
return nil, err
}
if running {
proc, err := os.FindProcess(pid)
if err != nil {
return nil, err
}
return proc, nil
}
return nil, ErrDeadOwner
}
// TryLock tries to own the lock.
// It Returns nil, if successful and and error describing the reason, it didn't work out.
// Please note, that existing lockfiles containing pids of dead processes
// and lockfiles containing no pid at all are simply deleted.
func (l Lockfile) TryLock() error {
name := string(l)
// This has been checked by New already. If we trigger here,
// the caller didn't use New and re-implemented it's functionality badly.
// So panic, that he might find this easily during testing.
if !filepath.IsAbs(name) {
panic(ErrNeedAbsPath)
}
tmplock, err := ioutil.TempFile(filepath.Dir(name), filepath.Base(name)+".")
if err != nil {
return err
}
cleanup := func() {
_ = tmplock.Close()
_ = os.Remove(tmplock.Name())
}
defer cleanup()
if err := writePidLine(tmplock, os.Getpid()); err != nil {
return err
}
// return value intentionally ignored, as ignoring it is part of the algorithm
_ = os.Link(tmplock.Name(), name)
fiTmp, err := os.Lstat(tmplock.Name())
if err != nil {
return err
}
fiLock, err := os.Lstat(name)
if err != nil {
// tell user that a retry would be a good idea
if os.IsNotExist(err) {
return ErrNotExist
}
return err
}
// Success
if os.SameFile(fiTmp, fiLock) {
return nil
}
proc, err := l.GetOwner()
switch err {
default:
// Other errors -> defensively fail and let caller handle this
return err
case nil:
if proc.Pid != os.Getpid() {
return ErrBusy
}
case ErrDeadOwner, ErrInvalidPid:
// cases we can fix below
}
// clean stale/invalid lockfile
err = os.Remove(name)
if err != nil {
// If it doesn't exist, then it doesn't matter who removed it.
if !os.IsNotExist(err) {
return err
}
}
// now that the stale lockfile is gone, let's recurse
return l.TryLock()
}
// Unlock a lock again, if we owned it. Returns any error that happend during release of lock.
func (l Lockfile) Unlock() error {
proc, err := l.GetOwner()
switch err {
case ErrInvalidPid, ErrDeadOwner:
return ErrRogueDeletion
case nil:
if proc.Pid == os.Getpid() {
// we really own it, so let's remove it.
return os.Remove(string(l))
}
// Not owned by me, so don't delete it.
return ErrRogueDeletion
default:
// This is an application error or system error.
// So give a better error for logging here.
if os.IsNotExist(err) {
return ErrRogueDeletion
}
// Other errors -> defensively fail and let caller handle this
return err
}
}
func writePidLine(w io.Writer, pid int) error {
_, err := io.WriteString(w, fmt.Sprintf("%d\n", pid))
return err
}
func scanPidLine(content []byte) (int, error) {
if len(content) == 0 {
return 0, ErrInvalidPid
}
var pid int
if _, err := fmt.Sscanln(string(content), &pid); err != nil {
return 0, ErrInvalidPid
}
if pid <= 0 {
return 0, ErrInvalidPid
}
return pid, nil
}

308
vendor/github.com/nightlyone/lockfile/lockfile_test.go сгенерированный поставляемый Normal file
Просмотреть файл

@ -0,0 +1,308 @@
package lockfile
import (
"fmt"
"io/ioutil"
"math/rand"
"os"
"path/filepath"
"strconv"
"testing"
)
func ExampleLockfile() {
lock, err := New(filepath.Join(os.TempDir(), "lock.me.now.lck"))
if err != nil {
fmt.Printf("Cannot init lock. reason: %v", err)
panic(err) // handle properly please!
}
err = lock.TryLock()
// Error handling is essential, as we only try to get the lock.
if err != nil {
fmt.Printf("Cannot lock %q, reason: %v", lock, err)
panic(err) // handle properly please!
}
defer lock.Unlock()
fmt.Println("Do stuff under lock")
// Output: Do stuff under lock
}
func TestBasicLockUnlock(t *testing.T) {
path, err := filepath.Abs("test_lockfile.pid")
if err != nil {
panic(err)
}
lf, err := New(path)
if err != nil {
t.Fail()
fmt.Println("Error making lockfile: ", err)
return
}
err = lf.TryLock()
if err != nil {
t.Fail()
fmt.Println("Error locking lockfile: ", err)
return
}
err = lf.Unlock()
if err != nil {
t.Fail()
fmt.Println("Error unlocking lockfile: ", err)
return
}
}
func GetDeadPID() int {
// I have no idea how windows handles large PIDs, or if they even exist.
// So limit it to be less or equal to 4096 to be safe.
const maxPid = 4095
// limited iteration, so we finish one day
seen := map[int]bool{}
for len(seen) < maxPid {
pid := rand.Intn(maxPid + 1) // see https://godoc.org/math/rand#Intn why
if seen[pid] {
continue
}
seen[pid] = true
running, err := isRunning(pid)
if err != nil {
fmt.Println("Error checking PID: ", err)
continue
}
if !running {
return pid
}
}
panic(fmt.Sprintf("all pids lower %d are used, cannot test this", maxPid))
}
func TestBusy(t *testing.T) {
path, err := filepath.Abs("test_lockfile.pid")
if err != nil {
t.Fatal(err)
return
}
pid := os.Getppid()
if err := ioutil.WriteFile(path, []byte(strconv.Itoa(pid)+"\n"), 0666); err != nil {
t.Fatal(err)
return
}
defer os.Remove(path)
lf, err := New(path)
if err != nil {
t.Fatal(err)
return
}
got := lf.TryLock()
if got != ErrBusy {
t.Fatalf("expected error %q, got %v", ErrBusy, got)
return
}
}
func TestRogueDeletion(t *testing.T) {
path, err := filepath.Abs("test_lockfile.pid")
if err != nil {
t.Fatal(err)
return
}
lf, err := New(path)
if err != nil {
t.Fatal(err)
return
}
err = lf.TryLock()
if err != nil {
t.Fatal(err)
return
}
err = os.Remove(path)
if err != nil {
t.Fatal(err)
return
}
got := lf.Unlock()
if got != ErrRogueDeletion {
t.Fatalf("unexpected error: %v", got)
return
}
}
func TestRogueDeletionDeadPid(t *testing.T) {
path, err := filepath.Abs("test_lockfile.pid")
if err != nil {
t.Fatal(err)
return
}
lf, err := New(path)
if err != nil {
t.Fatal(err)
return
}
err = lf.TryLock()
if err != nil {
t.Fatal(err)
return
}
pid := GetDeadPID()
if err := ioutil.WriteFile(path, []byte(strconv.Itoa(pid)+"\n"), 0666); err != nil {
t.Fatal(err)
return
}
defer os.Remove(path)
err = lf.Unlock()
if err != ErrRogueDeletion {
t.Fatalf("unexpected error: %v", err)
return
}
if _, err := os.Stat(path); os.IsNotExist(err) {
t.Fatal("lockfile should not be deleted by us, if we didn't create it")
} else {
if err != nil {
t.Fatalf("unexpected error %v", err)
}
}
}
func TestRemovesStaleLockOnDeadOwner(t *testing.T) {
path, err := filepath.Abs("test_lockfile.pid")
if err != nil {
t.Fatal(err)
return
}
lf, err := New(path)
if err != nil {
t.Fatal(err)
return
}
pid := GetDeadPID()
if err := ioutil.WriteFile(path, []byte(strconv.Itoa(pid)+"\n"), 0666); err != nil {
t.Fatal(err)
return
}
err = lf.TryLock()
if err != nil {
t.Fatal(err)
return
}
if err := lf.Unlock(); err != nil {
t.Fatal(err)
return
}
}
func TestInvalidPidLeadToReplacedLockfileAndSuccess(t *testing.T) {
path, err := filepath.Abs("test_lockfile.pid")
if err != nil {
t.Fatal(err)
return
}
if err := ioutil.WriteFile(path, []byte("\n"), 0666); err != nil {
t.Fatal(err)
return
}
defer os.Remove(path)
lf, err := New(path)
if err != nil {
t.Fatal(err)
return
}
if err := lf.TryLock(); err != nil {
t.Fatalf("unexpected error: %v", err)
return
}
// now check if file exists and contains the correct content
got, err := ioutil.ReadFile(path)
if err != nil {
t.Fatalf("unexpected error %v", err)
return
}
want := fmt.Sprintf("%d\n", os.Getpid())
if string(got) != want {
t.Fatalf("got %q, want %q", got, want)
}
}
func TestScanPidLine(t *testing.T) {
tests := [...]struct {
input []byte
pid int
xfail error
}{
{
xfail: ErrInvalidPid,
},
{
input: []byte(""),
xfail: ErrInvalidPid,
},
{
input: []byte("\n"),
xfail: ErrInvalidPid,
},
{
input: []byte("-1\n"),
xfail: ErrInvalidPid,
},
{
input: []byte("0\n"),
xfail: ErrInvalidPid,
},
{
input: []byte("a\n"),
xfail: ErrInvalidPid,
},
{
input: []byte("1\n"),
pid: 1,
},
}
// test positive cases first
for step, tc := range tests {
if tc.xfail != nil {
continue
}
want := tc.pid
got, err := scanPidLine(tc.input)
if err != nil {
t.Fatalf("%d: unexpected error %v", step, err)
}
if got != want {
t.Errorf("%d: expected pid %d, got %d", step, want, got)
}
}
// test negative cases now
for step, tc := range tests {
if tc.xfail == nil {
continue
}
want := tc.xfail
_, got := scanPidLine(tc.input)
if got != want {
t.Errorf("%d: expected error %v, got %v", step, want, got)
}
}
}

20
vendor/github.com/nightlyone/lockfile/lockfile_unix.go сгенерированный поставляемый Normal file
Просмотреть файл

@ -0,0 +1,20 @@
// +build darwin dragonfly freebsd linux nacl netbsd openbsd solaris
package lockfile
import (
"os"
"syscall"
)
func isRunning(pid int) (bool, error) {
proc, err := os.FindProcess(pid)
if err != nil {
return false, err
}
if err := proc.Signal(syscall.Signal(0)); err != nil {
return false, nil
}
return true, nil
}

30
vendor/github.com/nightlyone/lockfile/lockfile_windows.go сгенерированный поставляемый Normal file
Просмотреть файл

@ -0,0 +1,30 @@
package lockfile
import (
"syscall"
)
//For some reason these consts don't exist in syscall.
const (
error_invalid_parameter = 87
code_still_active = 259
)
func isRunning(pid int) (bool, error) {
procHnd, err := syscall.OpenProcess(syscall.PROCESS_QUERY_INFORMATION, true, uint32(pid))
if err != nil {
if scerr, ok := err.(syscall.Errno); ok {
if uintptr(scerr) == error_invalid_parameter {
return false, nil
}
}
}
var code uint32
err = syscall.GetExitCodeProcess(procHnd, &code)
if err != nil {
return false, err
}
return code == code_still_active, nil
}