From ad573502049a3a3ee2619ac2e9d1d503c6764c28 Mon Sep 17 00:00:00 2001 From: Jon Ruskin Date: Sat, 7 Aug 2021 15:02:16 -0700 Subject: [PATCH] update commands documentation --- README.md | 7 +- docs/commands.md | 130 ------------------------- docs/commands/README.md | 51 ++++++++++ docs/commands/cache.md | 37 +++++++ docs/commands/env.md | 12 +++ docs/commands/list.md | 23 +++++ docs/commands/migrate.md | 10 ++ docs/commands/notices.md | 12 +++ docs/commands/status.md | 36 +++++++ docs/commands/version.md | 3 + docs/configuration/allowed_licenses.md | 2 +- 11 files changed, 187 insertions(+), 136 deletions(-) delete mode 100644 docs/commands.md create mode 100644 docs/commands/README.md create mode 100644 docs/commands/cache.md create mode 100644 docs/commands/env.md create mode 100644 docs/commands/list.md create mode 100644 docs/commands/migrate.md create mode 100644 docs/commands/notices.md create mode 100644 docs/commands/status.md create mode 100644 docs/commands/version.md diff --git a/README.md b/README.md index 8a19722..3632f12 100644 --- a/README.md +++ b/README.md @@ -80,8 +80,9 @@ For system wide usage, install licensed to a location on `$PATH`, e.g. `/usr/loc - `licensed notices`: Write a `NOTICE` file for each application configuration. - `licensed version`: Show current installed version of Licensed. Aliases: `-v|--version` - `licensed env`: Output environment information from the licensed configuration. +- `licensed migrate`: Migrate licensed configuration and cached files from a previous major version. -See the [commands documentation](./docs/commands.md) for additional documentation, or run `licensed -h` to see all of the current available commands. +See the [commands documentation](./docs/commands) for additional documentation, or run `licensed -h` to see all of the current available commands. ### Automation @@ -166,10 +167,6 @@ end See the [documentation on adding new sources](./docs/adding_a_new_source.md) for more information. -#### Adding Commands - -See the [documentation on commands](./docs/commands.md) for information about adding a new CLI command. - ## Contributing Bug reports and pull requests are welcome on GitHub at https://github.com/github/licensed. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [Contributor Covenant](http://contributor-covenant.org/) code of conduct. See [CONTRIBUTING](CONTRIBUTING.md) for more details. diff --git a/docs/commands.md b/docs/commands.md deleted file mode 100644 index 22a034e..0000000 --- a/docs/commands.md +++ /dev/null @@ -1,130 +0,0 @@ -# Commands - -Run `licensed -h` to see help content for running licensed commands. - -## `list` - -Running the list command finds the dependencies for all sources in all configured applications. No additional actions are taken on each dependency. - -An optional `--sources` (`-s`) flag can be given to limit which dependency sources are run. This is a filter over sources that are enabled via the licensed configuration file and cannot be used to run licensed with a disabled source. - -An optional `--licenses` (`-l`) flag can be given to include the detected license in the reported output. - -### Reported Data - -The following data is reported for each dependency when the YAML or JSON report formats are used - -- name: the licensed recognized name for the dependency including the app and source name - - e.g. the full name for the `thor` bundler dependency used by this tool is `licensed.bundler.thor` -- version: the version of the enumerated dependency -- license: (optional) the dependency's SPDX license identifier - -## `cache` - -The cache command finds all dependencies and ensures that each dependency has an up-to-date cached record. - -An optional `--sources` flag can be given to limit which dependency sources are run. This is a filter over sources that are enabled via the licensed configuration file and cannot be used to run licensed with a disabled source. - -Dependency records will be saved if: -1. The `force` option is set -2. No cached record is found -3. The cached record's version is different than the current dependency's version - - If the cached record's license text contents matches the current dependency's license text then the `license` metadata from the cached record is retained for the new saved record. - -After the cache command is run, any cached records that don't match up to a current application dependency will be deleted. - -### Reported Data - -The following data is reported for each dependency when the YAML or JSON report formats are used - -- name: the licensed recognized name for the dependency including the app and source name - - e.g. the full name for the `thor` bundler dependency used by this tool is `licensed.bundler.thor` -- cached: true when the dependency's cached metadata file was updated, false otherwise -- version: the version of the enumerated dependency -- license: the dependency's SPDX license identifier -- filename: the full path on disk to the dependency's cached metadata file, if available -- warnings: any warning messages encountered while enumerating and caching dependency metadata, if available - -## `status` - -The status command finds all dependencies and checks whether each dependency has a valid cached record. - -An optional `--sources` flag can be given to limit which dependency sources are run. This is a filter over sources that are enabled via the licensed configuration file and cannot be used to run licensed with a disabled source. - -A dependency will fail the status checks if: -1. No cached record is found -2. The cached record's version is different than the current dependency's version -3. The cached record's `licenses` data is empty -4. The cached record's `license` metadata doesn't match an `allowed` license from the dependency's application configuration. - - If `license: other` is specified and all of the `licenses` entries match an `allowed` license a failure will not be logged -5. The cached record is flagged for re-review. - - This occurs when the record's license text has changed since the record was reviewed. - -### Reported Data - -The following data is reported for each dependency when the YAML or JSON report formats are used - -- name: the licensed recognized name for the dependency including the app and source name - - e.g. the full name for the `thor` bundler dependency used by this tool is `licensed.bundler.thor` -- allowed: true if the dependency has passed all checks, false otherwise -- version: the version of the enumerated dependency -- license: the dependency's SPDX license identifier -- filename: the full path on disk to the dependency's cached metadata file, if available -- errors: any error messages from failed status checks, if available - -## `notices` - -Outputs license and notice text for all dependencies in each app into a `NOTICE` file in the app's `cache_path`. If an app uses a shared cache path, the file name will contain the app name as well, e.g. `NOTICE.my_app`. - -An optional `--sources` flag can be given to limit which dependency sources are run. This is a filter over sources that are enabled via the licensed configuration file and cannot be used to run licensed with a disabled source. - -The `NOTICE` file contents are retrieved from cached records, with the assumption that cached records have already been reviewed in a compliance workflow. - -## `env` - -Prints the runtime environment used by licensed after loading a configuration file. By default the output is in YAML format, but can be output in JSON using the `--json` flag. - -The output will not be equivalent to configuration input. For example, all paths will be - -## `version` - -Displays the current licensed version. - -# Adding a new command - -## Implement new `Command` class - -Licensed commands inherit and override the [`Licensed::Sources::Command`](../lib/licensed/commands/command.rb) class. - -#### Required method overrides -1. `Licensed::Commands::Command#evaluate_dependency` - - Runs a command execution on an application dependency. - -The `evaluate_dependency` method should contain the specific command logic. This method has access to the application configuration, dependency source enumerator and dependency currently being evaluated as well as a reporting hash to contain information about the command execution. - -#### Optional method overrides - -The following methods break apart the different levels of command execution. Each method wraps lower levels of command execution in a corresponding reporter method. - -1. `Licensed::Commands::Command#run` - - Runs `run_app` for each application configuration found. Wraps the execution of all applications in `Reporter#report_run`. -2. `Licensed::Commands::Command#run_app` - - Runs `run_source` for each dependency source enumerator enabled for the application configuration. Wraps the execution of all sources in `Reporter#report_app`. -3. `Licensed::Commands::Command#run_source` - - Runs `run_dependency` for each dependency found in the source. Wraps the execution of all dependencies in `Reporter#report_source`. -4. `Licensed::Commands::Command#run_dependency` - - Runs `evaluate_dependency` for the dependency. Wraps the execution of all dependencies in `Reporter#report_dependency`. - -As an example, `Licensed::Commands::Command#run_app` calls `Reporter#report_app` to wrap every call to `Licensed::Commands::Command#run_source`. - -##### Specifying additional report data - -The `run` methods can be overridden and pass a block to `super` to provide additional reporting data or functionality. - -```ruby -def run_app(app) - super do |report| - report["my_app_data"] = true - end -end -``` diff --git a/docs/commands/README.md b/docs/commands/README.md new file mode 100644 index 0000000..dc54cc7 --- /dev/null +++ b/docs/commands/README.md @@ -0,0 +1,51 @@ +# Commands + +Run `licensed -h` to see help content for running licensed commands. + +- [cache](cache.md) +- [env](env.md) +- [list](list.md) +- [migrate](migrate.md) +- [notices](notices.md) +- [status](status.md) +- [version](verison.md) + +## Adding a new command + +### Implement new `Command` class + +Licensed commands inherit and override the [`Licensed::Sources::Command`](../lib/licensed/commands/command.rb) class. + +### Required method overrides + +1. `Licensed::Commands::Command#evaluate_dependency` + - Runs a command execution on an application dependency. + +The `evaluate_dependency` method should contain the specific command logic. This method has access to the application configuration, dependency source enumerator and dependency currently being evaluated as well as a reporting hash to contain information about the command execution. + +### Optional method overrides + +The following methods break apart the different levels of command execution. Each method wraps lower levels of command execution in a corresponding reporter method. + +1. `Licensed::Commands::Command#run` + - Runs `run_app` for each application configuration found. Wraps the execution of all applications in `Reporter#report_run`. +2. `Licensed::Commands::Command#run_app` + - Runs `run_source` for each dependency source enumerator enabled for the application configuration. Wraps the execution of all sources in `Reporter#report_app`. +3. `Licensed::Commands::Command#run_source` + - Runs `run_dependency` for each dependency found in the source. Wraps the execution of all dependencies in `Reporter#report_source`. +4. `Licensed::Commands::Command#run_dependency` + - Runs `evaluate_dependency` for the dependency. Wraps the execution of all dependencies in `Reporter#report_dependency`. + +As an example, `Licensed::Commands::Command#run_app` calls `Reporter#report_app` to wrap every call to `Licensed::Commands::Command#run_source`. + +### Specifying additional report data + +The `run` methods can be overridden and pass a block to `super` to provide additional reporting data or functionality. + +```ruby +def run_app(app) + super do |report| + report["my_app_data"] = true + end +end +``` diff --git a/docs/commands/cache.md b/docs/commands/cache.md new file mode 100644 index 0000000..8dd0799 --- /dev/null +++ b/docs/commands/cache.md @@ -0,0 +1,37 @@ +# `licensed cache` + +The cache command finds all dependencies and ensures that each dependency has an up-to-date cached record. + +An optional `--sources` flag can be given to limit which dependency sources are run. This is a filter over sources that are enabled via the licensed configuration file and cannot be used to run licensed with a disabled source. + +Dependency records will be saved if: + +1. The `force` option is set +2. No cached record is found +3. The cached record's version is different than the current dependency's version + - If the cached record's license text contents matches the current dependency's license text then the `license` metadata from the cached record is retained for the new saved record. + +After the cache command is run, any cached records that don't match up to a current application dependency will be deleted. + +## Options + +- `--config`/`-c`: the path to the licensed configuration file + - default value: `./.licensed.yml` +- `--sources`/`-s`: runtime filter on which dependency sources are run. Sources must also be enabled in the licensed configuration file. + - default value: not set, all configured sources +- `--format`/`-f`: the output format + - default value: `yaml` +- `--force`: if set, forces all dependency metadata files to be recached + - default value: not set + +## Reported Data + +The following data is reported for each dependency when the YAML or JSON report formats are used + +- name: the licensed recognized name for the dependency including the app and source name + - e.g. the full name for the `thor` bundler dependency used by this tool is `licensed.bundler.thor` +- cached: true when the dependency's cached metadata file was updated, false otherwise +- version: the version of the enumerated dependency +- license: the dependency's SPDX license identifier +- filename: the full path on disk to the dependency's cached metadata file, if available +- warnings: any warning messages encountered while enumerating and caching dependency metadata, if available diff --git a/docs/commands/env.md b/docs/commands/env.md new file mode 100644 index 0000000..e803f93 --- /dev/null +++ b/docs/commands/env.md @@ -0,0 +1,12 @@ +# `licensed env` + +Prints the runtime environment used by licensed after loading a configuration file. This can be different from the configuration file inputs, for example all paths will be given as absolute file paths and glob paths may be expanded. + +## Options + +- `--config`/`-c`: the path to the licensed configuration file + - required: true + - default value: `./.licensed.yml` +- `--format`/`-f`: the output format + - required: false + - default value: `yaml` diff --git a/docs/commands/list.md b/docs/commands/list.md new file mode 100644 index 0000000..e0e6605 --- /dev/null +++ b/docs/commands/list.md @@ -0,0 +1,23 @@ +# `licensed list` + +The list command finds and prints the dependencies for all sources in all configured applications. No additional actions are taken on dependencies. + +## Options + +- `--config`/`-c`: the path to the licensed configuration file + - default value: `./.licensed.yml` +- `--sources`/`-s`: runtime filter on which dependency sources are run. Sources must also be enabled in the licensed configuration file. + - default value: not set, all configured sources +- `--format`/`-f`: the output format + - default value: `yaml` +- `--licenses`/`-l`: if set, includes each dependency's detected license in the output + - default value: not set + +### Reported Data + +The following data is reported for each dependency when the YAML or JSON report formats are used + +- name: the licensed recognized name for the dependency including the app and source name + - e.g. the full name for the `thor` bundler dependency used by this tool is `licensed.bundler.thor` +- version: the version of the enumerated dependency +- license: (optional) the dependency's SPDX license identifier diff --git a/docs/commands/migrate.md b/docs/commands/migrate.md new file mode 100644 index 0000000..a4ff7a5 --- /dev/null +++ b/docs/commands/migrate.md @@ -0,0 +1,10 @@ +# `licensed migrate` + +Migrates the licensed configuration and cached metadata files from a previous version to the most recent version. This is not required for all major version updates. See [migrations documentation](../migrations) for details on the migrations needed for each major version. + +## Options + +- `--config`/`-c`: the path to the licensed configuration file + - default value: `./.licensed.yml` +- `--from`/`-f`: the licensed version to migrate from + - required diff --git a/docs/commands/notices.md b/docs/commands/notices.md new file mode 100644 index 0000000..97b4970 --- /dev/null +++ b/docs/commands/notices.md @@ -0,0 +1,12 @@ +# `licensed notices` + +Outputs license and notice text for all dependencies in each app into a `NOTICE` file in the app's `cache_path`. If an app uses a shared cache path, the file name will contain the app name as well, e.g. `NOTICE.my_app`. + +`NOTICE` file contents are retrieved from cached records, with the assumption that cached records have already been reviewed in a compliance workflow. + +## Options + +- `--config`/`-c`: the path to the licensed configuration file + - default value: `./.licensed.yml` +- `--sources`/`-s`: runtime filter on which dependency sources are run. Sources must also be enabled in the licensed configuration file. + - default value: not set, all configured sources diff --git a/docs/commands/status.md b/docs/commands/status.md new file mode 100644 index 0000000..87de4c5 --- /dev/null +++ b/docs/commands/status.md @@ -0,0 +1,36 @@ +# `licensed status` + +The status command finds all dependencies and checks whether each dependency has a valid cached record. + +A dependency will fail the status checks if: + +1. No cached record is found +2. The cached record's version is different than the current dependency's version +3. The cached record's `licenses` data is empty +4. The cached record's `license` metadata doesn't match an `allowed` license from the dependency's application configuration. + - If `license: other` is specified and all of the `licenses` entries match an `allowed` license a failure will not be logged +5. The cached record is flagged for re-review. + - This occurs when the record's license text has changed since the record was reviewed. + +## Options + +- `--config`/`-c`: the path to the licensed configuration file + - default value: `./.licensed.yml` +- `--sources`/`-s`: runtime filter on which dependency sources are run. Sources must also be enabled in the licensed configuration file. + - default value: not set, all configured sources +- `--format`/`-f`: the output format + - default value: `yaml` +- `--force`: if set, forces all dependency metadata files to be recached + - default value: not set + +## Reported Data + +The following data is reported for each dependency when the YAML or JSON report formats are used + +- name: the licensed recognized name for the dependency including the app and source name + - e.g. the full name for the `thor` bundler dependency used by this tool is `licensed.bundler.thor` +- allowed: true if the dependency has passed all checks, false otherwise +- version: the version of the enumerated dependency +- license: the dependency's SPDX license identifier +- filename: the full path on disk to the dependency's cached metadata file, if available +- errors: any error messages from failed status checks, if available diff --git a/docs/commands/version.md b/docs/commands/version.md new file mode 100644 index 0000000..63e373c --- /dev/null +++ b/docs/commands/version.md @@ -0,0 +1,3 @@ +# `licensed version` + +Displays the current licensed version. Can also be run with `licensed -v` and `licensed --version`. diff --git a/docs/configuration/allowed_licenses.md b/docs/configuration/allowed_licenses.md index f2f554b..5415a5e 100644 --- a/docs/configuration/allowed_licenses.md +++ b/docs/configuration/allowed_licenses.md @@ -3,7 +3,7 @@ **Key**: allowed **Default Value**: none -The list of allowed licenses is used with the [status command](../commands.md#status) to detail which licenses are allowable for use in the current project and do not need further review. If a dependency uses a license that is not included in the allowed list, and the dependency is not on the ignored or reviewed dependency lists, it will be flagged and the status command will fail. +The list of allowed licenses is used with the [status command](../commands/status.md) to detail which licenses are allowable for use in the current project and do not need further review. If a dependency uses a license that is not included in the allowed list, and the dependency is not on the ignored or reviewed dependency lists, it will be flagged and the status command will fail. This configuration value accepts an array of lower-cased [open source license SPDX identifiers](https://spdx.org/licenses/).