This commit is contained in:
Jon Ruskin 2021-07-10 20:44:23 -07:00
Родитель c40004dac3
Коммит 09bd9f86b6
11 изменённых файлов: 420 добавлений и 173 удалений

Просмотреть файл

@ -2,87 +2,12 @@
A configuration file specifies the details of enumerating and operating on license metadata for apps.
Configuration can be specified in either YML or JSON formats. Examples below are given in YML.
Configuration can be specified in either YML or JSON formats, with examples given in YML. The example
below describes common configuration values and their purposes. See [configuration options documentation](./configuration)
for in depth information.
## Configuration Paths
Additionally, some dependency sources have their own specific configuration options. See the [source documentation](./sources) for details.
`licensed` requires a path to enumerate dependencies at (`source_path`) and a path to store cached metadata (`cache_path`).
To determine these paths across multiple environments where absolute paths will differ, a known root path is needed to evaluate relative paths against.
In using a root, relative source and cache paths can be specified in the configuration file.
When using a configuration file, the root property can be set as either a path that can be expanded from the configuration file directory using `File.expand_path`, or the value `true` to use the configuration file directory as the root.
When creating a `Licensed::Dependency` manually with a `root` property, the property must be an absolute path - no path expansion will occur.
If a root path is not specified, it will default to using the following, in order of precedence
1. the root of the local git repository, if run inside a git repository
2. the current directory
### Source paths
A source path is the directory in which licensed should run to enumerate dependencies. This is often dependent on the project type, for example the bundler source should be run from the directory containing a `Gemfile` or `gems.rb` while the go source should be run from the directory containing an entrypoint function.
#### Using glob patterns
The `source_path` property can use one or more glob patterns to share configuration properties across multiple application entrypoints.
For example, there is a common pattern in Go projects to include multiple executable entrypoints under folders in `cmd`. Using a glob pattern allows users to avoid manually configuring and maintaining multiple licensed application `source_path`s. Using a glob pattern will also ensure that any new entrypoints matching the pattern are automatically picked up by licensed commands as they are added.
```yml
sources:
go: true
# treat all directories under `cmd` as separate apps
source_path: cmd/*
```
In order to better filter the results from glob patterns, the `source_path` property also accepts an array of inclusion and exclusion glob patterns similar to gitignore files. Inclusion patterns will add matching directory paths to resulting set of source paths, while exclusion patterns will remove matching directory paths.
```yml
source_path:
- "projects/*" # include by default all directories under "projects"
- "!projects/*Test" # exclude all projects ending in "Test"
```
Glob patterns are syntactic sugar for, and provide the same functionality as, manually specifying multiple `source_path` values. See the instructions on [specifying multiple apps](./#specifying-multiple-apps) below for additional considerations when using multiple apps.
## Restricting sources
The `sources` configuration property specifies which sources `licensed` will use to enumerate dependencies.
By default, `licensed` will generally try to enumerate dependencies from all sources. As a result,
the configuration property should be used to explicitly disable sources rather than to enable a particular source.
Be aware that this configuration is separate from an individual sources `#enabled?` method, which determines
whether the source is valid for the current project. Even if a source is enabled in the configuration
it may still determine that it can't enumerate dependencies for a project.
```yml
sources:
bower: true
bundler: false
```
`licensed` determines which sources will try to enumerate dependencies based on the following rules:
1. If no sources are configured, all sources are enabled
2. If no sources are set to true, any unconfigured sources are enabled
```yml
sources:
bower: false
# all other sources are enabled by default since there are no sources set to true
```
3. If any sources are set to true, any unconfigured sources are disabled
```yml
sources:
bower: true
# all other sources are disabled by default because a source was set to true
```
## Applications
What is an "app"? In the context of `licensed`, an app is a combination of a source path and a cache path.
Configuration can be set up for single or multiple applications in the same repo. There are a number of settings available for each app:
```yml
# If not set, defaults to the directory name of `source_path`
name: 'My application'
@ -141,100 +66,11 @@ reviewed:
bower:
- classlist # public domain
- octicons
```
### Specifying a single app
To specify a single app, either include a single app with `source_path` in the `apps` configuration, or remove the `apps` setting entirely.
If the configuration does not contain an `apps` value, the root configuration will be used as an app definition. In this scenario, the `source_path` is not a required value and will default to the directory that `licensed` was executed from.
If the configuration contains an `apps` value with a single app configuration, `source_path` must be specified. Additionally, the applications inherited `cache_path` value will contain the application name. See [Inherited cache_path values](#inherited_cache_path_values)
### Specifying multiple apps
The configuration file can specify multiple source paths to enumerate metadata, each with their own configuration.
Nearly all configuration settings can be inherited from root configuration to app configuration. Only `source_path` is required to define an app.
Here are some examples:
#### Inheriting configuration
```yml
sources:
go: true
bundler: false
ignored:
bundler:
- some-internal-gem
reviewed:
bundler:
- bcrypt-ruby
cache_path: 'path/to/cache'
# A single configuration file can be used to enumerate dependencies for multiple
# projects. Each configuration is referred to as an "application" and must include
# a source path, at a minimum
apps:
- source_path: 'path/to/app1'
- source_path: 'path/to/app2'
sources:
bundler: true
go: false
- source_path: path/to/application1
- source_path: path/to/application2
```
In this example, two apps have been declared. The first app, with `source_path` `path/to/app1`, inherits all configuration settings from the root configuration. The second app, with `source_path` `path/to/app2`, overrides the `sources` configuration and inherits all other settings.
#### Default app names
An app will not inherit a name set from the root configuration. If not provided, the `name` value will default to the directory name from `source_path`.
```yml
apps:
- source_path: 'path/to/app1'
- source_path: 'path/to/app2'
```
In this example, the apps have names of `app1` and `app2`, respectively.
#### Inherited cache_path values
When an app inherits a `cache_path` from the root configuration, it will automatically append it's name to the end of the path to separate it's metadata from other apps. To force multiple apps to use the same path to cached metadata, explicitly set the `cache_path` value for each app.
```yml
cache_path: 'path/to/cache'
apps:
- source_path: 'path/to/app1'
name: 'app1'
- source_path: 'path/to/app2'
name: 'app2'
- source_path: 'path/to/app3'
name: 'app3'
cache_path: 'path/to/app3/cache'
```
In this example `app1` and `app2` have `cache_path` values of `path/to/cache/app1` and `path/to/cache/app2`, respectively. `app3` has an explicit path set to `path/to/app3/cache`
```yml
apps:
- source_path: 'path/to/app1'
```
In this example, the root configuration will contain a default cache path of `.licenses`. `app1` will inherit this value and append it's name, resulting in a cache path of `.licenses/app1`.
### Sharing caches between apps
Dependency caches can be shared between apps by setting the same cache path on each app.
```yaml
apps:
- source_path: "path/to/app1"
cache_path: ".licenses/apps"
- source_path: "path/to/app2"
cache_path: ".licenses/apps"
```
When using a source path with a glob pattern, the apps created from the glob pattern can share a dependency by setting an explicit cache path and setting `shared_cache` to true.
```yaml
source_path: "path/to/apps/*"
cache_path: ".licenses/apps"
shared_cache: true
```
## Source specific configuration
See the [source documentation](./sources) for details on any source specific configuration.

Просмотреть файл

@ -0,0 +1,11 @@
# Configuration options
1. [Application source path](./application_source.md)
1. [Dependency metadata cache](./metadata_cache.md)
1. [Configuring multiple applications / monorepo support](./configuring_multiple_apps.md)
1. [Configuration root](./configuration_root.md)
1. [Application name](./application_name.md)
1. [Dependency source enumerators](./dependency_source_enumerators.md)
1. [Allowed licenses](./allowed_licenses.md)
1. [Ignoring dependencies](./ignoring_dependencies.md)
1. [Reviewing dependencies](./reviewing_dependencies.md)

Просмотреть файл

@ -0,0 +1,17 @@
# Allowed licenses
**Key**: allowed
**Default Value**: none
The list of allowed licenses is used with the [status command](../commands.md#status) to detail which licenses are allowable for use in the current project and do not need further review. If a dependency uses a license that is not included in the allowed list, and the dependency is not on the ignored or reviewed dependency lists, it will be flagged and the status command will fail.
This configuration value accepts an array of lower-cased [open source license SPDX identifiers](https://spdx.org/licenses/).
```yml
# accepts lowercase SPDX license identifiers
allowed:
- mit
- bsd-2-clause
- bsd-3-clause
- isc
```

Просмотреть файл

@ -0,0 +1,63 @@
# Application name
**Key**: name
**Default value**: The directory name of the application's source path
The name of the application is primarily used for organizational and display purposes. Application names are not included with
dependency metadata information included in cached files.
```yml
source_path: path/to/application1
name: application1
```
## Dynamically generated application names
### Source path directory name
When not specified, an application's name will be the directory name from the application's source path.
```yml
# if not explicitly set the name will be inferred from the source path
source_path: path/to/application1
# name: application1
# or, use the `directory_name` name generator to explicitly set this behavior
source_path: path/to/application1
name:
generator: directory_name
# name: application1
```
### Relative path from configuration root
Application names can be created from the path from the configuration root to the application source path.
This can be useful when specifying multiple applications using a glob pattern in a directory structure where directory names
are not unique.
As an example, given the following directory structure and configuration YML, the resulting application names
would be `linux.installer`, `windows.installer` and `mac.installer`. The optional arguments `separator` and `depth` are used
to better control the resulting application name.
```text
path
|_to
|_linux
|_installer
|_windows
|_installer
|_mac
|_installer
```
```yml
source_path: '**/installer'
name:
generator: relative_path
# separator controls what character separates path parts in the application name
# default: "-"
separator: '.'
# depth controls how many of the path parts are used in the application name
# default: 0 / all path parts
depth: 2
```

Просмотреть файл

@ -0,0 +1,64 @@
# Application source path
**Key**: source_path
**Default value**:
- if the `apps` key is not present, then the current working directory where `licensed` was executed
- if the `apps` key is present, then `nil`
The source path is the directory in which licensed should run to enumerate dependencies. This is often dependent
on the project type, for example the bundler source should be run from the directory containing a `Gemfile` or `gems.rb`
while the go source should be run from the directory containing an entrypoint function.
The source path is required to run `licensed`. A default value is available only when the configuration file specifies a single application.
When multiple applications are configured, each application must specify a source path.
Paths can be given as absolute or relative paths, and can use special path identifiers. If a relative path is given, it will be based on the application's root path.
```yml
# when apps is not set, a source path does not need to be specified. it will default to the users current directory
sources:
bundler: true
# ------
# or a path can be given as either an absolute or relative path
sources:
bundler: true
source_path: path/to/application1
# ------
# when apps is set, each application must specify a source_path
sources:
bundler: true
apps:
- source_path: relative/path/to/application1
- source_path: /absolute/path/to/application2
- source_path: ~/path/from/home/to/application3
```
## Expanding source paths with glob patterns
The `source_path` property can use one or more glob patterns to share configuration properties across multiple application entrypoints.
For example, there is a common pattern in Go projects to include multiple executable entrypoints under folders in `cmd`. Using a glob pattern allows users to avoid manually configuring and maintaining multiple licensed application `source_path`s. Using a glob pattern will also ensure that any new entrypoints matching the pattern are automatically picked up by licensed commands as they are added.
```yml
sources:
go: true
# treat all directories under `cmd` as separate apps
source_path: cmd/*
```
In order to better filter the results from glob patterns, the `source_path` property also accepts an array of
inclusion and exclusion glob patterns similar to gitignore files. Inclusion patterns will add matching directory
paths to resulting set of source paths, while exclusion patterns will remove matching directory paths.
```yml
source_path:
- "projects/*" # include by default all directories under "projects"
- "!projects/*Test" # exclude all projects ending in "Test"
```
Glob patterns are syntactic sugar for, and provide the same functionality as, manually specifying multiple `source_path` values.
See the instructions on [specifying multiple apps](../configuration.md#specifying-multiple-apps) below for additional considerations when using multiple apps.

Просмотреть файл

@ -0,0 +1,27 @@
# Configuration root
**Key**: root
**Default value**:
1. the root of the local git repository, if run inside a git repository
1. the directory that `licensed` is run from
An application's root path is used as the base for any relative configuration paths in the application.
From a configuration file, the root value can be specified as one of the following. Path string values can contain special path characters.
- a relative path from the configuration file location
- an absolute path
- `true` to use the configuration file's directory as the root
When creating a `Licensed::AppConfiguration` manually with a `root` property, the property must be an absolute path - no path expansion will occur.
```yml
root: path/from/configuration
# or
root: /absolute/path/to/root
# or
root: ~/path/from/home/to/root
# or
root: true
```

Просмотреть файл

@ -0,0 +1,58 @@
# Configuring multiple application definitions
**Key**: apps
**Required**: false
The configuration file can specify multiple source paths to enumerate metadata, each with their own configuration by using the `apps` key.
Each source path and any additional configuration make up an "application". Root configuration settings are inherited into each application,
allowing applications to share a common configuration and reducing the overall size of the configuration file.
When the apps key is not given, the root configuration is treated as a single application.
```yml
apps:
# application definition for "go-application"
- source_path: path/to/go-application
sources:
go: true
allowed:
- mit
# application definition for "ruby-application"
- source_path: path/to/ruby-application
sources:
bundler: true
allowed:
- bsd-3-clause
```
## Inheriting configuration
Applications inherit all root configuration settings. Inherited settings will be overridden by any configuration set directly on the application definition.
In this example, two apps have been declared. The first app, with `source_path: path/to/application1`, inherits all configuration settings from the root configuration. The second app, with `source_path: path/to/application2`, overrides the `sources` configuration and inherits all other settings.
```yml
sources:
go: true
bundler: false
ignored:
bundler:
- some-internal-gem
reviewed:
bundler:
- bcrypt-ruby
cache_path: 'path/to/cache'
apps:
# inherits all settings from the root configuration
- source_path: 'path/to/application1'
# inherits all settings except for "sources" from the root configuration
- source_path: 'path/to/application2'
sources:
bundler: true
go: false
```

Просмотреть файл

@ -0,0 +1,28 @@
# Specifying dependency sources to use in an application
**Key**: sources
**Required**: false
**Default value**: All sources enabled
The sources configuration property specifies which sources `licensed` will use to enumerate dependencies.
By default, `licensed` will try to enumerate dependencies from all sources. As a result,
the configuration property should be used to explicitly disable sources rather than to enable a particular source.
This configuration value does not guarantee that a source will enumerate dependencies. Each
configured source's `enabled?` method must return true for licensed to pull dependency information.
`licensed` determines which sources will try to enumerate dependencies based on the following rules:
1. If no sources are configured, all sources are enabled
2. If no sources are set to true, any unconfigured sources are enabled
3. If any sources are set to true, any unconfigured sources are disabled
```yml
# all other sources are enabled by default since there are no sources set to true
sources:
bower: false
# all other sources are disabled by default because a source was set to true
sources:
bower: true
```

Просмотреть файл

@ -0,0 +1,19 @@
# Ignoring dependencies
**Key**: ignored
**Default value**: none
This configuration property is used to fully ignore a dependency during all `licensed` commands. Any dependency on this list will not
be enumerated, or have its metadata cached or checked for compliance. This is intended for dependencies that do not require attribution
or compliance checking - internal or 1st party dependencies, or dependencies that do not ship with the product such as test frameworks.
The ignored dependency list is organized based on the dependency source type - `bundler`, `go`, etc. Add a dependency's metadata identifier to the appropriate source type sub-property to cause `licensed` to no longer take action on the dependency. Glob patterns can be used to identify multiple internal dependencies without having to manage a large list.
```yml
ignored:
bundler:
- my-internal-gem
- my-first-party-gem
go:
- github.com/me/my-repo/**/*
```

Просмотреть файл

@ -0,0 +1,106 @@
# Dependency metadata cache
**Key**: cache_path
**Default value**: .licenses
The cache path is the root directory where `licensed` will write cached metadata files for each dependency.
By default, files will be written to a folder under the cache path in a structure like:
```text
<cache path>
|_<source name>
|_<dependency identifier>.dep.yml
```
Cache paths can be given as an absolute or relative path and can contain special path characters
```yml
cache_path: relative/path/to/cache
# or
cache_path: /absolute/path/to/cache
# or
cache_path: ~/path/from/home/to/cache
```
## Configuring caches for multiple applications
When multiple applications are specified in a configuration file caches can be shared, inherited and explicitly configured.
Unless otherwise specified, preference is given based on the locality and intention of the configuration options.
1. explicitly configured cache paths are preferred to inherited or shared cache paths
2. shared cache paths are preferred to inherited cache paths
3. cache paths that are not otherwise set by an application will be inherited from the root configuration
### Explicit cache usage for multiple applications
Individual applications in a multi-application configuration can explicitly set cache paths.
```yml
apps:
- source_path: path/to/application1
cache_path: path/to/application1/.licenses
- source_path: path/to/application2
cache_path: path/to/application2/.licenses
```
### Sharing a single cache for multiple applications
Sharing a cache across multiple applications is possible by setting `cache_path: <path>` and `shared_cache: true` at the root level.
Individual applications can opt out of sharing a cache by explicitly setting a cache path.
```yml
shared_cache: true
cache_path: .cache/shared
apps:
# application1 and application2 will share a cache at .cache/shared
- source_path: path/to/application1
- source_path: path/to/application2
# application3 will use a separate cache at .cache/application3
- source_path: path/to/application3
cache_path: .cache/application3
```
This is equivalent to explicitly configuring the same cache for many applications
```yaml
apps:
- source_path: "path/to/application1"
cache_path: ".cache/shared"
- source_path: "path/to/application2"
cache_path: ".cache/shared"
```
Using the `shared_cache` key is primarily useful when specifying `source_path` as a glob pattern.
```yml
shared_cache: true
cache_path: .cache/shared
source_path: path/to/*
```
### Inheriting cache usage from the root configuration
When not otherwise specified, applications will inherit a cache path from the root configuration.
If the root configuration does not explicitly set a cache path value, the default cache path value is used.
Inherited cache paths are treated as the root location for each application's metadata cache. Each application
will store metadata in a named subdirectory of the root location to avoid file path clashes between
applications.
```yml
# optional. if not specified, the default value `.licenses` will be used
cache_path: .cache
apps:
- source_path: path/to/application1
- source_path: path/to/application2
```
```text
.cache
|_application1
|_<source type>
|_<dependency identifier>.dep.yml
|_application2
|_<source type>
|_<dependency identifier>.dep.yml
```

Просмотреть файл

@ -0,0 +1,18 @@
# Reviewing dependencies
**Key**: reviewed
**Default value**: none
Sometimes your projects will use a dependency with an OSS license that you don't want to globally allow but can use with individual review.
The list of reviewed dependencies is meant to cover this scenario and will prevent the status command from raising an error for
a dependency with a license not on the allowed list.
The reviewed dependency list is organized based on the dependency source type - `bundler`, `go`, etc. Add a dependency's metadata identifier to the appropriate source type sub-property to cause `licensed` to ignore license compliance failures. Glob patterns can be used to identify multiple internal dependencies without having to manage a large list.
_NOTE: marking a dependency as reviewed will not prevent licensed from raising an error on missing license information._
```yml
reviewed:
bundler:
- gem-using-unallowed-license
```