* Add experimental NuGet detector for framework and dev dependencies
This PR does 3 things.
1. Adds `TargetFramework` to NuGet package references. This can be useful when querying component data to understand if components are used in a place where a vulnerability applies.
2. Adds _framework package_ handling. The .NET SDK will do [conflict resolution](https://github.com/dotnet/sdk/tree/main/src/Tasks/Common/ConflictResolution) and drop assets from packages that overlap with the framework. NuGet is planning to do the same https://github.com/NuGet/Home/issues/7344 but until then, it's beneficial to have component detection duplicate some of this logic. When a package is identified as overlapping with the framework we'll treat it as a Development Dependency so that it might be auto-dismissed.
- .NETFramework projects do not get this - .NETFramework does not participate in conflict resolution by default. Even when enabled framework assemblies can be bypassed using bindingRedirects, or avoiding references to them. Due this fragility it's not safe to apply framework package rules to .NETFramework.
- packages.config usage is also excluded since it precludes SDK conflict resolution (and is also only used on .NETFramework projects).
3. Recognizes `ExcludeAssets="Runtime"` usage as a Development Dependencies, also any packages which don't contribute to "runtime" will be developement dependencies.
I reused _Development Dependency_ rather than plumbing a new concept.
I only mapped data for the `Microsoft.NETCore.App` - the default shared framework. We could consider doing the same for `Microsoft.ASPNETCore.App` and `Microsoft.WindowsDesktop.App` but we'd need to plumb the reference information out of the assets file - currently that's not read and I'm not aware of a supported NuGet API for reading it (though it is present under `project/frameworks/<framework>/frameworkReferences/<name>`
.NET Core 1.x has no data since it was packages itself. I have a fallback for future frameworks to read the data from the targeting packs.
* Address feedback
* add feature that removes some python files that are created during pip install report dry run
* move to more central file detector
* add tests and fix bugs
* remove extra dir
* fix dotnet 8 styling
* semaphore to only run a single cleanup process at a time for a given detector
* add test
* add test
* refactor to abstract file and directory operations out to allow for unit tests with a mocked file system
* break out the cleanup changes to its own abstract class
* pr feedback
* rename vars and fix tests
* torevert: quick console log for test
* revert log and add file to source control
* os agnostic test file paths
* update snapshot verify, and bump report version
* add python to verification pipeline
* adding back setup file
* Use MSTest meta package
This enables running MSTest Analyzers on test code to help avoid common test problems.
* Fix code coverage
* Fixes
Upgrade to latest and fix unnecessary type param
* Fix tests
* PR feedback
* Fix CC
* fix bug where pipreport used index urls in requirements.txt
* update tests
* docs
* add --no-input to pip install, so we do not hang waiting for user input
* pr feedback: performance and cleanup
* bump version
* add support for python -m pip
* update pip command service to accept python exe
* swap so we use pip as default
* fixing remote build
* fix tests
* add unit tests
* ignore pregenerated pipreports that don't cover the correct set of dependencies
* add validation to the pre-generated pipreport to prevent underdetection for overridden reports
* dispose of telemetry object
* move re-used code to a common utility method
* revert experiment graduation, bump threads, and enable fast deps
* put reqs back
* add ability for pip to detect pregenerated reports with a specific naming scheme
* better directory handling
* improve logging
* add fallback logic to pipreport for cases where we shouldn't reach out to remote feed, and should parse source instead
* add the manual detection as fallback if pip report fails
* add option to skip or fallback to a source code scan
* add docs and fix tests
* remove fallback
* add fallback back, and env var to allow for skipping fallback
* merged pnpm6 into pnpm detector
* bump version and make the factory method private
* name of type
* improved logging
* added telemetry record for pnpm, and other minor updates to methods for conciseness
* standardize the invalid version telemetry object
* removed invalid version file
* Update RustCli Parsing to process pkgId, and allow manual override to fallback with DisableRustCliScan
* add tests
* Update detector version
* Update cli detector to use manifest packages instead of manually parsing
* Support development dependencies for the Gradle detector
Lack of development dependency detection for Gradle is a problem for
Android teams, especially in the context of Component Governance
alerts. Unfortunately Gradle doesn't provide enough information to
definitively identify dev dependencies in all cases, so manual
configuration is required. This change adds dev dependency
classification through two mechanisms
1. `buildscript-gradle.lockfile` and `settings-gradle.lockfile`
contain only build-system dependencies, so always classify these as
development dependencies.
2. Processing based on two new environment variables:
`GRADLE_PROD_CONFIGURATIONS_REGEX` and
`GRADLE_DEV_CONFIGURATIONS_REGEX`. Gradle lockfiles indicate which
Gradle configuration(s) each dependency is required by.
`GRADLE_PROD_CONFIGURATIONS_REGEX` allows specifying
production configurations explicitly. All other configurations are
considered development. Alternately, dev configurations may be
specified in `GRADLE_DEV_CONFIGURATIONS_REGEX` and all others are
considered production.
* Changes based on meeting prior to the holidays
* fluent assertions
* Visual studio recommendations
* More fluent assertsions
* Fix test to be cross-platform
* Fix the cross-platform test fix
* Fix code coverage by removing dead code check
* Address code review comments
Greg Villicana
Eric StJohn
dependabot[bot]
Amitla Vannikumar
Paul Dorsch
Coby Allred
cobya
stan-sz
Fernando Rojo
Juan Carlos Fiorenzano
Rushabh
James Oakley