This commit is contained in:
Jamie Magee 2021-12-07 09:21:22 -08:00 коммит произвёл GitHub
Родитель 8a6d475d53
Коммит 7537eed5e6
Не найден ключ, соответствующий данной подписи
Идентификатор ключа GPG: 4AEE18F83AFDEB23
8 изменённых файлов: 94 добавлений и 88 удалений

12
docs/detectors/README.md Normal file
Просмотреть файл

@ -0,0 +1,12 @@
# Detectors
- CocoaPods
- [Go](go.md)
- [Gradle](gradle.md)
- [Linux](linux.md)
- [Maven](maven.md)
- NPM
- NuGet
- [Pip](pip.md)
- Ruby
- Rust

34
docs/detectors/go.md Normal file
Просмотреть файл

@ -0,0 +1,34 @@
# Go Detection
## Requirements
Go detection depends on the following to successfully run:
- Go v1.11+.
## Detection strategy
Go detection is performed by parsing output from executing `go mod graph`.
Full dependency graph generation is supported if Go v1.11+ is present on the build agent.
If no Go v1.11+ is present, a fallback detection strategy is performed, dependent on:
- One or more `go.mod` or `go.sum` files.
For the fallback strategy:
Go detection is performed by parsing any `go.mod` or `go.sum` found under the scan directory.
Only root dependency information is generated instead of full graph.
I.e. tags the top level component or explicit dependency a given transitive dependency was brought by.
Given a dependency tree A -> B -> C, C's root dependency is A.
## Known limitations
Dev dependency tagging is not supported.
Go detection will fallback if no Go v1.11+ is present.
If executing `go mod graph` takes too long (currently if it takes more than 10 seconds), go detection will fall back.
This can happen if modules are not restored before the scan.
Due to the nature of `go.sum` containing references for all dependencies, including historical, no-longer-needed dependencies; the fallback strategy can result in over detection.
Executing `go mod tidy` before detection via fallback is encouraged.

Просмотреть файл

@ -1,16 +1,20 @@
# Gradle Detection
## Requirements
Gradle detection depends on the following to successfully run:
- Gradle 7 or prior using [Single File lock](https://docs.gradle.org/6.8.1/userguide/dependency_locking.html#single_lock_file_per_project).
- One or more <em>*.lockfile</em> files.
- Gradle 7 or prior using [Single File lock](https://docs.gradle.org/6.8.1/userguide/dependency_locking.html#single_lock_file_per_project)
- One or more `.lockfile` files
## Detection strategy
Gradle detection is performed by parsing any <em>*.lockfile</em> found under the scan directory.
Gradle detection is performed by parsing any `*.lockfile` found under the scan directory.
## Known limitations
Gradle detection will not work if lock files are not being used.
*Dev dependency* tagging is not supported.
Dev dependency tagging is not supported.
Full dependency graph generation is not supported.
Full dependency graph generation is not supported.

16
docs/detectors/linux.md Normal file
Просмотреть файл

@ -0,0 +1,16 @@
# Linux Detection
## Requirements
Linux detection depends on the following:
- [Docker](https://www.docker.com/)
## Detection strategy
Linux package detection is performed by running [Syft](https://github.com/anchore/syft) and parsing the output.
The output contains the package name, version, and the layer of the container in which it was found.
## Known limitations
- Windows container scanning is not supported

Просмотреть файл

@ -1,17 +1,21 @@
# Maven Detection
## Requirements
Maven detection depends on the following to successfully run:
- Maven CLI as part of your PATH. mvn should be runnable from a given command line.
- Maven Dependency Plugin (installed with Maven).
- One or more *pom.xml* files.
- One or more `pom.xml` files.
## Detection strategy
Maven detection is performed by running *mvn dependency:tree -f {pom.xml}* for each pom file and parsing down the results.
Components tagged as a *test* dependency are marked as *development dependencies*.
Maven detection is performed by running `mvn dependency:tree -f {pom.xml}` for each pom file and parsing down the results.
Components tagged as a test dependency are marked as development dependencies.
Full dependency graph generation is supported.
## Known limitations
Maven detection will not run if *mvn* is unavailable.
Maven detection will not run if `mvn` is unavailable.

Просмотреть файл

@ -1,13 +1,16 @@
# Pip Detection
## Requirements
Pip detection depends on the following to successfully run:
- Python 2 or Python 3.
- Internet connection.
- One or more <em>setup.py</em> or *requirements.txt* files.
- Python 2 or Python 3
- Internet connection
- One or more `setup.py` or `requirements.txt` files
## Detection strategy
Pip detection is performed by running the following code snippet on every <em>setup.py</em>:
Pip detection is performed by running the following code snippet on every *setup.py*:
```python
import distutils.core;
@ -15,19 +18,20 @@ Pip detection is performed by running the following code snippet on every <em>se
print(setup.install_requires);
```
The code above allows Pip detection to detect any runtime dependendies.
The code above allows Pip detection to detect any runtime dependencies.
*requirements.txt* files are parsed; a Git component is created for every *git+* url.
`requirements.txt` files are parsed; a Git component is created for every `git+` url.
For every top level component, Pip detection makes http calls to Pip in order to determine latest version available, as well as to resolve the dependency tree by parsing the *METADATA* file on a given release's *bdist_wheel* or *bdist_egg*.
For every top level component, Pip detection makes http calls to Pip in order to determine latest version available, as well as to resolve the dependency tree by parsing the `METADATA` file on a given release's `bdist_wheel` or `bdist_egg`.
Full dependency graph generation is supported.
## Known limitations
*Dev dependency* tagging is not supported.
Pip detection will not run if *python* is unavailable.
Dev dependency tagging is not supported.
If no *bdist_wheel* or *bdist_egg* are available for a given component, dependencies will not be fetched.
Pip detection will not run if `python` is unavailable.
If no internet connection or a component cannot be found in Pypi, said component and its dependencies will be skipped.
If no `bdist_wheel` or `bdist_egg` are available for a given component, dependencies will not be fetched.
If no internet connection or a component cannot be found in Pypi, said component and its dependencies will be skipped.

Просмотреть файл

@ -1,27 +0,0 @@
# Go Detection
## Requirements
Go detection depends on the following to successfully run:
- Go v1.11+.
## Detection strategy
Go detection is performed by parsing output from executing *go mod graph*.
Full dependency graph generation is supported if Go v1.11+ is present on the build agent.
If no Go v1.11+ is present, a fallback detection strategy is performed, dependent on:
- One or more *go.mod* or *go.sum* files.
For the fallback strategy:
Go detection is performed by parsing any *go.mod* or *go.sum* found under the scan directory.
Only root dependency information is generated instead of full graph. Ie. tags the top level component or explicit dependency a given transitive dependency was brought by. Given a dependency tree A -> B -> C, C's root dependency is A.
## Known limitations
*Dev dependency* tagging is not supported.
Go detection will fallback if no Go v1.11+ is present. If executing `go mod graph` takes too long (currently if it takes more than 10 seconds), go detection will fall back. This can happen if modules are not restored before the scan.
Due to the nature of *go.sum* containing references for all dependencies, including historical, no-longer-needed dependencies; the fallback strategy can result in over detection. Executing *go mod tidy* before detection via fallback is encouraged.

Просмотреть файл

@ -1,41 +0,0 @@
# Developing on the Linux Scanner
The Linux scanner uses a custom container image built from [this repository](https://github.com/Microsoft/LinuxScanning) we created to execute [tern](https://github.com/tern-tools/tern) on a target image.
As of 6/26/2020, tern does not officially support scanning images in Windows so by extension, our Linux scanner does not either.
There are 2 options for working on the Linux scanner:
* Set up and work in a virtual Linux environment
* Work with a limited set of images in Windows
## The Linux way (Recommended)
**NOTE**: Docker + WSL2 is not a shortcut for this :( The bind mount is still not support even if it's created in WSL2. (See the Windows way to udnerstand why it's necessary)
1. [Using Hyper-V Quick Create](https://blogs.windows.com/windowsdeveloper/2018/09/17/run-ubuntu-virtual-machines-made-even-easier-with-hyper-v-quick-create/), create a Ubuntu virtual machine. Make sure you give it at least 30gb of storage, docker and the images use a lot of space.
2. Clone Component Detection in your VM
3. Install docker
4. Start developing!
## The Windows way
There are _some_ things the tern scanner can do on Windows.
Limitations around scanning in Windows:
* The image being scanned is a base OS image (eg: Ubuntu, Fedora, CentOS, etc.). That means no node, no python, definitely no custom Dockerfile images.
* Tern cannot attempt to create a bind mount to a folder
How to do it:
1. Install docker
2. Comment out the [OS platform check](https://github.com/microsoft/componentdetection-bcde/blob/d5743af1bda5a8f3b4eb96c08517b05513bf29d2/src/Detectors/linux/LinuxContainerDetector.cs#L42-L48)
3. Retarget the [scanner image](https://github.com/microsoft/componentdetection-bcde/blob/d5743af1bda5a8f3b4eb96c08517b05513bf29d2/src/Detectors/linux/LinuxScanner.cs#L22) to point to `tevoinea/ternwrapper:windows-latest`
* 3.1 **Alternatively**:
* Clone [LinuxScanning](https://github.com/microsoft/linuxscanning)
* Change the [default args](https://github.com/microsoft/LinuxScanning/blob/main/index.js#L6) to the set that doesn't contain the `-b` argument. `-b` means bind which we don't want on Windows
* Rebuild the image locally
* Follow step 3 again but point to your local image
4. Remember to only scan base OS images (scan will fail otherwise)
5. Remember to undo step 2 and 3 before PR