Граф коммитов

473 Коммитов

Автор SHA1 Сообщение Дата
sam boyer c20530499e gps: Fix up odds and ends around IgnoreRulesets
The most crucial piece is fixing the hasher to make its prefix checks
without the wildcard suffix for the purpose of determining whether the
ignore affects the root project; otherwise, wildcards on the root of the
current project (why would you do this?) would be erroneously included
in hash inputs.
2017-10-13 10:11:56 -04:00
sam boyer b0face73a0 gps: Convert to use IgnoredRuleset 2017-10-13 09:11:24 -04:00
sam boyer 4eb59de7fe gps: Remove CreateIgnorePrefixTree 2017-10-13 04:18:54 -04:00
sam boyer 5be0c0000c gps: Fully refactor pkgtree to use IgnoredRuleset 2017-10-13 04:16:12 -04:00
sam boyer cb51cd0949 gps: Introduce IgnoredRuleset
This provides a single type for handling both literal and wildcard
ignore patterns, allowing us to tighten up a few interfaces.
2017-10-13 02:05:08 -04:00
sam boyer 8de86c40c3 gps: Strings for calltypes 2017-10-12 22:41:27 -04:00
sam boyer 6bbd19d908 gps: Force subprocesses into new pgroup on unix
This puts us more squarely in control of subprocess signaling.
2017-10-12 22:40:53 -04:00
sam boyer bea24e2592 gps: Break ctx chain so that Kill isn't autofired 2017-10-12 22:40:47 -04:00
Jordan Krage 83ed215b10 gps: vcs repo error fix 2017-10-11 13:39:39 -05:00
Sunny ec7ae32621 gps: minute cleanup 2017-10-07 19:47:58 +05:30
Sunny 0cd7862701 gps: check wildcard ignored and required conflict 2017-10-07 19:47:58 +05:30
Sunny 2139b31f96 gps: no ineffectual wildcard igs in hash inputs 2017-10-07 19:47:58 +05:30
Sunny 9c4acd4f40 gps: wildcard ignore skip ineffectual ignores 2017-10-07 19:47:58 +05:30
Sunny fbfbad5235 test(gps): add test for rootdata.isIgnored() 2017-10-07 19:47:58 +05:30
Sunny f271bcc915 gps: skip global ignores (*) and add tests
Adds test for createIgnorePrefixTree() which verifies the global
ignore skip.

Also, unifies createIgnorePrefixTree() of solver and pkgtree, moving it
to pkgtree package.
2017-10-07 19:47:58 +05:30
Sunny 23c9329084 gps: create trie only when wildcard is found
* Create trie only when wildcard is found
* Rename 'recursive' to 'wildcard' ignore
2017-10-07 19:47:58 +05:30
Sunny 415108cc07 fix(gps): add recursive ignore support 2017-10-07 19:47:58 +05:30
sam boyer 667d662063 Merge pull request #1017 from rtfb/respect-import-comments-issue-902
[WIP] Respect canonical import comments
2017-10-07 07:48:33 -04:00
Vytautas Šaltenis 611e64ba45 Fix lint 2017-10-07 09:40:01 +03:00
sam boyer b7e513cac2 gps: move digest fixtures into proper testdata dir 2017-10-06 21:14:20 -04:00
Sunny 7fba0bb5a7 Merge pull request #1235 from jmank88/external_proc
integration tests: remove unused parameter
2017-10-07 00:23:01 +05:30
Vytautas Šaltenis 47e42c7305 Address feedback: naming, wording and comments 2017-10-06 20:50:13 +03:00
Steve Kim 82b5519797 Merge the latest strip codes fromt the master 2017-10-04 10:52:23 -07:00
Steve Kim 345fa0e2f5 Merge branch 'master' into skip_broken_vendor_symlink 2017-10-04 10:33:34 -07:00
sam boyer 742e949925 Merge pull request #1233 from glasser/glasser/case-mismatch-error
internal/gps: fix case mismatch error with multiple dependers
2017-10-04 07:37:53 -04:00
Jordan Krage 3642e27deb integration tests: remove unused parameter 2017-10-03 06:57:24 -05:00
sam boyer 453ac4642a gps: Rename result*.go to solution*.go 2017-10-02 19:36:27 -04:00
sam boyer 04877a0c99 gps: s/Lock.InputsHash()/Lock.InputsDigest()/
This was a bad naming in the first place, and the confusing symmetry
with Solver.HashInputs() just made it worse.
2017-10-02 19:36:27 -04:00
David Glasser 681e6f5a3d internal/gps: fix case mismatch error with multiple dependers
Previously, errors like the following could occur:

  v0.4.2: Could not introduce github.com/sirupsen/logrus due to a case-only variation: it depends on "github.com/Sirupsen/logrus", but "github.com/evalphobia/logrus_sentry@v0.4.2" was already established as the case variant for that project root by the following other dependers:
  (root)
  github.com/docker/docker@8510adf8c856d6f0871650216a0e1d7e6ece46ee

This fix matches the correct behavior when there is only one depender earlier in
the function.
2017-10-02 12:29:57 -07:00
sam boyer ac1a1624a6 Merge pull request #1215 from jmank88/mem-cache-opt
gps: source cache: memory cache optimization
2017-09-30 07:54:31 -04:00
Ayan George 3fd5bb377b Disable file locking when DEPNOLOCK set (#1206)
* Disable file locking when DEPNOLOCK set

* Add DisableLocking bool members to Ctx and gps.SourceManagerConfig structs.
  This effectively communicates DEPNOLOCK from the shell, to Ctx, to
  SourceManager.

  The member is named DisableLocking to make its zero-value useful.

* Add locker interface which implements TryLock(), Unlock(), and GetOwner()
  which lockfile.Lockfile alredy adheres to.  This interface replaces the new
  type for the lf member of the SourceMgr struct.

* Add a FalseLocker type which adheres to the Locker interface which does
  nothing.

* Conditionally set the lf member of SourceMgr to either an instance of
  lockfile.Lockfile or FalseLocker depending on the value of
  SourceManagerConfig.DisableLocking.

Signed-off-by: Ayan George <ayan@ayan.net>

* Revert stray edit.

Signed-off-by: Ayan George <ayan@ayan.net>

* Improve comment for DisableLocking

Signed-off-by: Ayan George <ayan@ayan.net>

* Fix comment type-os

* Fix comment type-os

Signed-off-by: Ayan George <ayan@ayan.net>

* Fix yet more type-os.

Signed-off-by: Ayan George <ayan@ayan.net>
2017-09-29 17:20:59 -04:00
michael-go 6fb987b90d gvt (and gb-vendor) importer (#1149) 2017-09-28 10:11:37 -05:00
Jordan Krage c1da7b85eb pkgtree: add os.IsPermission check for walkFn err arg 2017-09-25 21:43:16 -05:00
sam boyer 914a0bf26f Merge pull request #1212 from jmank88/git-ls-versions
gps: vcs: dedupe git version list
2017-09-25 21:14:11 -04:00
Jordan Krage 5a97f68a49 gps: lint - just return error 2017-09-24 07:45:33 -05:00
Jordan Krage 31ede573e7 gps: source cache: add ok bool to singleSourceCache.getAllVersions() in order to avoid depending on len(0) vs nil distinction 2017-09-23 10:18:32 -05:00
Jordan Krage ca3d62d6aa gps: source cache: optimize and fix mem cache getAllVersions() via slice store-and-copy and proper locking 2017-09-23 10:10:07 -05:00
Jordan Krage 57bcc65f28 gps: vcs: dedupe git version list 2017-09-23 08:22:48 -05:00
Jordan Krage 59eca1b52f Merge pull request #1127 from jmank88/source_cache_proto
gps: source cache: protobuf integration
2017-09-22 08:45:42 -05:00
sam boyer c564e7898a Merge pull request #1114 from sudo-suhas/check-cfg-filename-case
check Gopkg filenames case on case insensitive systems
2017-09-22 09:20:19 -04:00
Ibrahim AshShohail 9b6892cfce
internal/gps: export PruneProject and fix minor nits
Signed-off-by: Ibrahim AshShohail <ibra.sho@gmail.com>
2017-09-22 09:47:40 +03:00
Ibrahim AshShohail 98dd01efa9
Add AUTHORS and CONTRIBUTERS to legal files list
Signed-off-by: Ibrahim AshShohail <ibra.sho@gmail.com>
2017-09-22 09:18:26 +03:00
Ibrahim AshShohail feac5a4220
internal/gps: add pruneProject func
Signed-off-by: Ibrahim AshShohail <ibra.sho@gmail.com>
2017-09-22 09:18:26 +03:00
Ibrahim AshShohail b14d646c4b
internal/gps: update stripVendor functions
Signed-off-by: Ibrahim AshShohail <ibra.sho@gmail.com>
2017-09-22 09:18:26 +03:00
Ibrahim AshShohail 1315bd80e8
internal/gps: Add prune functions to gps
Signed-off-by: Ibrahim AshShohail <ibra.sho@gmail.com>
2017-09-22 09:18:26 +03:00
Tamir Duberstein be90d47432
gps: avoid race condition in variable assignment
Instead, we use the existing `waitDone` channel to wait for the
command to return and then clean up the timer unconditionally. This
may hold the goroutine open for longer than strictly necessary, but
that seems harmless.

Closes #1194.
2017-09-21 20:29:27 -04:00
Jordan Krage d67284a3f3 expand cache keys; note future optimizations 2017-09-21 07:17:24 -05:00
Seungyoung "Steve" Kim fce851e91e Skip broken vendor symlink rather than returning an error. 2017-09-20 17:41:44 -07:00
Suhas Karanth d451fa03fc rename func, improve comments
- `project.go`
  - Rename method {checkCfgFilenames => checkGopkgFilenames}
  - Improve funciton comment as suggested by @sdboyer
  - Fix ambigious comment explaining rationale behind early return.
- Add comment explaining why we do not use `fs.IsCaseSensitiveFilesystem` for
  skipping following tests:
  - context_test.go#TestLoadProjectGopkgFilenames
  - project_test.go#TestCheckGopkgFilenames
  - fs_test.go#TestReadActualFilenames
2017-09-20 11:36:04 +05:30
Suhas Karanth 28f4f359fe refactor requested changes, minor improvements
- `project`
  - `checkCfgFilenames`
    - Improve function and code comments
    - Use boolean value indicating whether value was found in actual filenames.
      If manifest file was not found, return `errProjectNotFound`. Use boolean
      to determine if lock file was not found instead of length check.
  - `TestCheckCfgFilenames` - Add code comments for test cases explaining the
    expected behavior
- `fs`
  - `ReadActualFilenames`
    - Use cleaner check(`<=` ➡ `==`) to see if `names` parameter for
      `ReadActualFilenames` actually has any values.
    - Use `Readdirnames` instead of `Readdir`. This is expected to perform
      better in most cases.
  - `TestReadActualFilenames` - Add code comments for test cases explaining the
    expected behavior
- general
  - Make line length for code comments consistent(90), add periods where
    appropriate.
  - String formatting, use %q instead of '%s'
2017-09-20 11:36:04 +05:30
Suhas Karanth afaf870131 check cfg filename case on case insensitive systems
- `fs`
  - Export `IsCaseSensitiveFilesystem`. Add and run test on windows, linux and
    macOS.
  - Add function `ReadActualFilenames` to read actual file names of given
    string slice. Add tests to be run on windows and macOS.
- `project`
  - Add function `checkCfgFilenames` to check the filenames for manifest
    and lock have the expected case. Use `fs#IsCaseSensitiveFilesystem`
    for an early return as the check is costly. Add test to be run on windows
    and macOS.
- `context`
  - Call `project.go#checkCfgFilenames` after resolving project root. Add test
    for invalid manifest file name to be run on windows and macOS.
2017-09-20 11:35:54 +05:30
Sunny 687134cd04 docs(gps): add docs in gps example 2017-09-19 19:53:34 +05:30
sam boyer 118a565b49 Merge pull request #1166 from michael-go/source-manager-expose-deduced-sources
add SourceURLsForPath() to SourceManager ineterface
2017-09-19 09:17:04 -04:00
Michael Gokhman 348b8694c1 improve doc for SourceURLsForPath()
and an error msg in `maybeSources.try()`
2017-09-19 10:48:43 +03:00
Tamir Duberstein 7d388fa747
gps: use commandContext 2017-09-18 19:47:51 -04:00
Tamir Duberstein a374f1cebd
gps: DRY 2017-09-18 19:47:43 -04:00
Tamir Duberstein 2b5ebb1ced
gps: include output in error 2017-09-18 19:46:29 -04:00
Sunny c4b0dcf63e docs(gps): DeduceRootProject -> DeduceProjectRoot 2017-09-18 17:46:35 +05:30
sam boyer 166626d8a8 Merge pull request #1138 from jmank88/multi_cache
gps: source cache: adding multiCache
2017-09-17 23:05:22 -04:00
sam boyer 92657ba181 Merge pull request #1165 from jmank88/discard_logger
testing: de-globalize discard loggers
2017-09-17 22:39:31 -04:00
sam boyer 5aa4ffe3c2 Merge pull request #1079 from sdboyer/warn-case-mismatch
Add satisfiability check for case variants
2017-09-17 22:27:17 -04:00
sam boyer cbf031832d gps: don't fold inputs to root deduction
But, still preserve the rule that we record the canonical folded URL in
memory, so that we can have non-canonical inputs come in first and still
converge with subsequent canonical, or other-case-variant forms later.
2017-09-17 22:16:34 -04:00
sam boyer 101c2d9664 gps: Add helpful debugging info to test logging
Keeping track of what maps to what in the sourceGateway setup can be
really tricky with all the combinations; in the event of failures in
this test, this will show the mapping tables, which helps a lot with
understanding the actual final state.
2017-09-17 22:11:51 -04:00
Jordan Krage 8008324d65 gps: source cache: protobuf integration 2017-09-15 10:50:17 -05:00
Jordan Krage d61fa38258 testing: de-globalize discard loggers 2017-09-14 14:57:02 -05:00
Carolyn Van Slyck d62440d304 Merge pull request #1145 from carolynvs/importers-package
cmd/dep: Move importers under internal/importers
2017-09-14 14:31:29 -05:00
Michael Gokhman cacb8ff9c2 add SourceURLsForPath to SourceManager ineterface
to expose the deduced possible source URLs for a given import path.
This information is useful for importing configuration from other tools
that support vendoring forks
2017-09-13 18:32:14 +03:00
Tamir Duberstein 9e660334de
gps: remove arbitrary command timeouts
Use better context plumbing while I'm here.
2017-09-12 23:46:46 -04:00
Carolyn Van Slyck 02bf38cf28
importers: add missing license header 2017-09-12 20:51:00 -05:00
Tamir Duberstein 01209b5e7d
gps: use golang.org/x/sync/errgroup
...instead of custom code that does the same thing. Return
informative errors rather than logging directly (into the abyss).

Also remove sourceFailure{,s} while I'm here, which are unnecessary.

Also `dep prune` real quick.
2017-09-12 14:13:59 -04:00
Carolyn Van Slyck 8de823b7c4
internal/importers: fix filename stutter 2017-09-11 22:02:27 -05:00
Carolyn Van Slyck 4eac4ce179
cmd/dep: Move importers under internal/importers 2017-09-11 22:02:27 -05:00
Tamir Duberstein 3a25b0b519
Add golint
/Users/tamird/src/go/src/github.com/golang/dep/context.go:86:1: exported method Ctx.SourceManager should have comment or be unexported
/Users/tamird/src/go/src/github.com/golang/dep/internal/gps/cmd.go:75:13: should omit type *int32 from declaration of var isDone; it will be inferred from the right-hand side
/Users/tamird/src/go/src/github.com/golang/dep/internal/gps/version.go:122:1: exported method Revision.ImpliedCaretString should have comment or be unexported
/Users/tamird/src/go/src/github.com/golang/dep/internal/test/test.go:27:2: exported var ExeSuffix should have comment or be unexported
/Users/tamird/src/go/src/github.com/golang/dep/internal/test/test.go:29:15: should omit type *bool from declaration of var PrintLogs; it will be inferred from the right-hand side
/Users/tamird/src/go/src/github.com/golang/dep/internal/test/test.go:30:15: should omit type *bool from declaration of var UpdateGolden; it will be inferred from the right-hand side
/Users/tamird/src/go/src/github.com/golang/dep/internal/test/test.go:194:1: comment on exported method Helper.Run should be of the form "Run ..."
/Users/tamird/src/go/src/github.com/golang/dep/internal/test/test.go:449:1: comment on exported method Helper.GetFile should be of the form "GetFile ..."
/Users/tamird/src/go/src/github.com/golang/dep/internal/test/test.go:611:1: exported method Helper.GetCommit should have comment or be unexported
2017-09-11 19:22:23 -04:00
Tamir Duberstein d9c8bd6dd1
Add honnef.co/go/tools/cmd/unused
cmd/dep/ensure.go:138:2: field overrides is unused (U1000)
cmd/dep/ensure.go:702:6: type stringSlice is unused (U1000)
internal/gps/bridge.go:60:2: field crp is unused (U1000)
internal/gps/pkgtree/pkgtree_test.go:36:3: field Internal is unused (U1000)
internal/gps/pkgtree/pkgtree_test.go:36:13: field External is unused (U1000)
internal/gps/selection.go:76:21: func (*selection).setDependenciesOn is unused (U1000)
internal/gps/selection.go:99:21: func (*selection).getSelectedPackagesIn is unused (U1000)
internal/gps/solve_basic_test.go:1356:33: func (*depspecSourceManager).ExternalReach is unused (U1000)
internal/gps/solve_basic_test.go:1571:16: func fixLock.SolverVersion is unused (U1000)
internal/gps/solve_basic_test.go:1588:18: func dummyLock.SolverVersion is unused (U1000)
internal/gps/solve_failures.go:20:2: const warning is unused (U1000)
internal/gps/solve_failures.go:21:2: const mustResolve is unused (U1000)
internal/gps/solve_failures.go:22:2: const cannotResolve is unused (U1000)
internal/gps/source_manager.go:217:4: func Temporary is unused (U1000)
internal/gps/source_manager.go:712:2: const ctCheckoutVersion is unused (U1000)
internal/gps/typed_radix.go:34:23: func (*deducerTrie).Delete is unused (U1000)
internal/gps/vcs_repo.go:201:6: type svnRepo is unused (U1000)
internal/gps/vcs_repo_test.go:122:6: func testSvnRepo is unused (U1000)
internal/gps/vcs_source.go:573:6: type repo is unused (U1000)
internal/gps/version_queue_test.go:30:23: func (*fakeBridge).ListVersions is unused (U1000)
internal/gps/version_queue_test.go:45:27: func (*fakeFailBridge).ListVersions is unused (U1000)
internal/test/integration/testproj.go:34:2: field h is unused (U1000)
2017-09-11 19:22:23 -04:00
sam boyer aa8e0765ed Merge pull request #1154 from tamird/remove-unnecessary-once
gps: simplify shutdown cleanup
2017-09-11 19:19:46 -04:00
Tamir Duberstein 45da465d4c
gps: simplify shutdown cleanup
The previous code here was doing some odd things with CAS, none of
which was necessary. There's no functional change here, it's just
simpler.
2017-09-11 15:39:09 -04:00
Tamir Duberstein a0b325d96e
gps: avoid a goroutine using time.AfterFunc 2017-09-11 15:35:35 -04:00
Tamir Duberstein e5685bfbd9
Build all packages before running linters
`go vet`, in particular, is known for relying on the package cache
for code analysis.

Fix addional `go vet` warnings revealed by this.
2017-09-11 15:32:26 -04:00
Tamir Duberstein 0d49338683
gps: remove useless loop 2017-09-11 15:24:56 -04:00
Tamir Duberstein 43326804e4
gps: use atomic.LoadInt32 instead of CAS 2017-09-11 13:34:37 -04:00
sam boyer 9e038e69dc Ensure URLs are folded as well
Also add an up-front check for case variant map lookup misses, and
update the map accordingly.
2017-09-10 23:22:07 -04:00
Tamir Duberstein 02eed27ce1
gps: remove unused context.WithCancel
Remove unnecessary getLifetimeContext method while I'm here.
2017-09-10 20:45:22 -04:00
sam boyer a665a6e85a Accommodate case folding in older mgr test 2017-09-10 19:31:52 -04:00
sam boyer bddd3f4bba Treat ProjectRoot as case-insensitive in srcCoord 2017-09-10 19:31:52 -04:00
sam boyer 70d2f5b695 Tests for combined source & case variance 2017-09-10 19:29:25 -04:00
sam boyer 56ce3a4513 Add test for import path case-sense handling in SM 2017-09-10 19:29:25 -04:00
sam boyer 3d369c50a9 Add the wrongCaseFailure, and scads more fixtures 2017-09-10 19:29:25 -04:00
sam boyer 6592101eeb Case insensitivity magic for roots in fixtures
This effectively makes them case-insensitive, case-preserving.
2017-09-10 19:29:25 -04:00
sam boyer 8670e66f5a Add wrongCaseFailure
May or may not end up using this right away, but it'll be in place for
when we have the slightly stronger failure case of a project being
addressed with an incorrect case, as indicated by the project's way
of referencing its own packages.
2017-09-10 19:29:25 -04:00
sam boyer 538f5d33f0 Add a bunch more test cases 2017-09-10 19:29:25 -04:00
sam boyer 65c8f22216 Fixes off-by-one error; rename method 2017-09-10 19:29:25 -04:00
sam boyer 5c96c1022c Use case folding to normalize lookup map key
This lifts the exact folding algorithm and use pattern followed by the
toolchain up into gps. Not only does it solve the strings.ToLower()
inadequacy, but it means we're using the exact same logic the
go compiler does to decide this same question.
2017-09-10 19:29:25 -04:00
sam boyer e64c82e377 Add satisfiability check for case variants 2017-09-10 19:29:25 -04:00
sam boyer 5a05adf659 Merge pull request #1132 from sdboyer/gopkgin-separation
Properly separate sources for different gopkg.in versions & github
2017-09-10 12:48:38 -04:00
Tamir Duberstein cec5ca1112
gps: avoid hiding errors
It was previously possible for the following to occur:
```
$ dep ensure
ensure Solve(): remote repository at https://code.googlesource.com/gocloud does not exist, or is inaccessible
```

This patch exposes the underlying error:
```
chdir /Users/tamird/src/go/pkg/dep/sources/https---code.googlesource.com-gocloud: no such file or directory
```

Which is easily fixed:
```
$ rm -r /Users/tamird/src/go/pkg/dep
```
2017-09-10 11:25:57 -04:00
Vytautas Saltenis 1853dc3bb1 Fix NonCanonicalImportRoot test 2017-09-10 11:51:37 +03:00
sam boyer 6890c50baa Merge pull request #1142 from jmank88/vcs_err_fix
gps: fix unwrapVcsErr to handle nil causes
2017-09-09 14:19:04 -04:00
Vytautas Saltenis 90d2a5e92d Record NonCanonicalImportRoot error instead of immediate return 2017-09-09 20:56:17 +03:00
Sunny 9db233ab47 Merge pull request #1053 from Minnozz/fix-1051
Don't assume every git repository has a HEAD
2017-09-09 21:25:52 +05:30
Jordan Krage 74c081892b gps: add unwrapVcsErr nil cause test 2017-09-09 07:20:51 -05:00