Update CI docs with Assert-PSRule (#377)

This commit is contained in:
Bernie White 2020-01-01 14:17:15 +10:00 коммит произвёл GitHub
Родитель bb3fa0bebd
Коммит a1245c640b
Не найден ключ, соответствующий данной подписи
Идентификатор ключа GPG: 4AEE18F83AFDEB23
6 изменённых файлов: 291 добавлений и 89 удалений

Просмотреть файл

@ -1,4 +1,4 @@
# Packaging rules in modules
# Packaging rules in a module
PSRule supports distribution of rules within modules.
Using a module, rules can be published and installed using standard PowerShell cmdlets.

Просмотреть файл

@ -0,0 +1,24 @@
#
# Azure DevOps pipeline
#
steps:
# Install dependencies
- powershell: ./pipeline-deps.ps1
displayName: 'Install dependencies'
# Validate templates
- powershell: ./validate-files.ps1
displayName: 'Validate files'
# Publish pipeline results
- task: PublishTestResults@2
displayName: 'Publish PSRule results'
inputs:
testRunTitle: 'PSRule'
testRunner: NUnit
testResultsFiles: 'reports/rule-report.xml'
mergeTestResults: true
publishRunAttachments: true
condition: succeededOrFailed()

Просмотреть файл

@ -2,13 +2,13 @@
# Licensed under the MIT License.
# Synopsis: Check file includes copyright header
Rule 'file.Header' -If { $TargetObject.Extension -in '.ps1', '.psm1', '.psd1', '.yaml', '.yml' } {
Rule 'File.Header' -If { $TargetObject.Extension -in '.ps1', '.psm1', '.psd1' } {
$fileContent = Get-Content -Path $TargetObject.FullName -Raw;
$fileContent -match '^(\# Copyright \(c\) Microsoft Corporation.(\r|\n|\r\n)\# Licensed under the MIT License\.)';
}
# Synopsis: File encoding should be UTF-8
Rule 'file.Encoding' -If { $TargetObject.Extension -in '.ps1', '.psm1', '.psd1', '.yaml', '.yml' } {
Rule 'File.Encoding' -If { $TargetObject.Extension -in '.ps1', '.psm1', '.psd1' } {
try {
$reader = New-Object -TypeName System.IO.StreamReader -ArgumentList @($TargetObject.FullName, [System.Text.Encoding]::UTF8, $True);
$Null = $reader.Peek();

Просмотреть файл

@ -0,0 +1,11 @@
# Copyright (c) Microsoft Corporation.
# Licensed under the MIT License.
# Install dependencies for connecting to PowerShell Gallery
if ($Null -eq (Get-PackageProvider -Name NuGet -ErrorAction SilentlyContinue)) {
Install-PackageProvider -Name NuGet -Force -Scope CurrentUser;
}
if ($Null -eq (Get-InstalledModule -Name PowerShellGet -MinimumVersion '2.2.1' -ErrorAction SilentlyContinue)) {
Install-Module PowerShellGet -MinimumVersion '2.2.1' -Scope CurrentUser -Force -AllowClobber;
}

Просмотреть файл

@ -0,0 +1,17 @@
# Copyright (c) Microsoft Corporation.
# Licensed under the MIT License.
# Install PSRule module
if ($Null -eq (Get-InstalledModule -Name PSRule -MinimumVersion '0.12.0' -ErrorAction SilentlyContinue)) {
Install-Module -Name PSRule -Scope CurrentUser -MinimumVersion '0.12.0' -Force;
}
# Validate files
$assertParams = @{
Path = './.ps-rule/'
Style = 'AzurePipelines'
OutputFormat = 'NUnit3'
OutputPath = 'reports/rule-report.xml'
}
$items = Get-ChildItem -Recurse -Path .\src\,.\tests\ -Include *.ps1,*.psd1,*.psm1,*.yaml;
$items | Assert-PSRule $assertParams -ErrorAction Stop;

Просмотреть файл

@ -1,143 +1,172 @@
# Validation pipeline example
# Using within continuous integration
This example covers how PSRule can be used within a DevOps pipeline to validate files, templates and objects.
PSRule supports several features that make it easy to a continuous integration (CI) pipeline.
When added to a pipeline, PSRule can validate files, template and objects dynamically.
This scenario covers the following:
- Installing PSRule within a continuous integration (CI) pipeline
- Failing the pipeline based on validation results
- Installing within a CI pipeline
- Validating objects
- Formatting output
- Failing the pipeline
- Generating NUnit output
- Additional options
## Installing PSRule within a CI pipeline
## Installing within a CI pipeline
Typically PSRule is not pre-installed on CI worker nodes, so within a CI pipeline the PSRule PowerShell module needs to be installed prior to calling PSRule cmdlets such as `Invoke-PSRule`.
If your CI pipeline runs on a persistent virtual machine that you control consider pre-installing PSRule.
Typically, PSRule is not pre-installed on CI worker nodes and must be installed.
If your CI pipeline runs on a persistent virtual machine that you control, consider pre-installing PSRule.
The following examples focus on installing PSRule dynamically during execution of the pipeline.
Which is suitable for cloud based CI worker nodes.
Which is suitable for cloud-based CI worker nodes.
To install PSRule within a CI pipeline execute the `Install-Module` PowerShell cmdlet.
In the example below:
- When installing modules on Windows, by default modules will be installed into _Program Files_, which requires administrator permissions. Depending on your environment, the CI worker process may not have administrative permissions. Instead we can install PSRule for the current context running the CI pipeline by using the `-Scope CurrentUser` parameter.
- By default this cmdlet will install the module from the PowerShell Gallery which is not trusted by default. Since a CI pipeline is not interactive the `-Force` switch is used to suppress a prompt to install modules from PowerShell Gallery.
- When installing modules on Windows, modules will be installed into _Program Files_ by default, which requires administrator permissions.
Depending on your environment, the CI worker process may not have administrative permissions.
Instead we can install PSRule for the current context running the CI pipeline by using the `-Scope CurrentUser` parameter.
- By default, this cmdlet will install the module from the PowerShell Gallery which is not trusted by default.
Since a CI pipeline is not interactive, use the `-Force` switch to suppress the confirmation prompt.
```powershell
$Null = Install-Module -Name PSRule -Scope CurrentUser -Force;
Install-Module -Name PSRule -Scope CurrentUser -Force;
```
In some cases installing NuGet may be required before the module can be installed.
In some cases, installing NuGet and PowerShellGet may be required to connect to the PowerShell Gallery.
The NuGet package provider can be installed using the `Install-PackageProvider` PowerShell cmdlet.
```powershell
$Null = Install-PackageProvider -Name NuGet -Scope CurrentUser -Force;
Install-PackageProvider -Name NuGet -Scope CurrentUser -Force;
Install-Module PowerShellGet -MinimumVersion '2.2.1' -Scope CurrentUser -Force -AllowClobber;
```
The example below includes both steps together with checks:
```powershell
if ($Null -eq (Get-PackageProvider -Name NuGet -ErrorAction Ignore)) {
$Null = Install-PackageProvider -Name NuGet -Scope CurrentUser -Force;
if ($Null -eq (Get-PackageProvider -Name NuGet -ErrorAction SilentlyContinue)) {
Install-PackageProvider -Name NuGet -Scope CurrentUser -Force;
}
if ($Null -eq (Get-InstalledModule -Name PSRule -MinimumVersion '0.9.0' -ErrorAction Ignore)) {
$Null = Install-Module -Name PSRule -Scope CurrentUser -MinimumVersion '0.9.0' -Force;
if ($Null -eq (Get-InstalledModule -Name PowerShellGet -MinimumVersion '2.2.1' -ErrorAction Ignore)) {
Install-Module PowerShellGet -MinimumVersion '2.2.1' -Scope CurrentUser -Force -AllowClobber;
}
```
```powershell
if ($Null -eq (Get-InstalledModule -Name PSRule -MinimumVersion '0.12.0' -ErrorAction SilentlyContinue)) {
Install-Module -Name PSRule -Scope CurrentUser -MinimumVersion '0.12.0' -Force;
}
```
See the [change log](https://github.com/Microsoft/PSRule/blob/master/CHANGELOG.md) for the latest version.
### Using Invoke-Build
## Validating objects
`Invoke-Build` is a build automation cmdlet that can be installed from the PowerShell Gallery by installing the _InvokeBuild_ module.
Within Invoke-Build, each build process is broken into tasks.
To validate objects use `Invoke-PSRule`, `Assert-PSRule` or `Test-PSRuleTarget`.
In a CI pipeline, `Assert-PSRule` is recommended.
`Assert-PSRule` outputs preformatted results ideal for use within a CI pipeline.
The following example shows an example of installing _PSRule_ using _InvokeBuild_ tasks.
For rules within the same source control repository, put rules in the `.ps-rule` directory.
A directory `.ps-rule` in the repository root, is used by convention.
In the following example, objects are validated against rules from the `./.ps-rule/` directory:
```powershell
# Synopsis: Install NuGet
task InstallNuGet {
if ($Null -eq (Get-PackageProvider -Name NuGet -ErrorAction Ignore)) {
$Null = Install-PackageProvider -Name NuGet -Scope CurrentUser -Force;
}
}
# Synopsis: Install PSRule
task InstallPSRule InstallNuGet, {
if ($Null -eq (Get-InstalledModule -Name PSRule -MinimumVersion '0.9.0' -ErrorAction Ignore)) {
$Null = Install-Module -Name PSRule -Scope CurrentUser -MinimumVersion '0.9.0' -Force;
}
}
$items | Assert-PSRule -Path './.ps-rule/'
```
## Fail the pipeline
Example output:
When using PSRule within a continuous integration pipeline, typically we need to catch errors and failures and stop the pipeline if any occur.
```text
-> ObjectFromFile.psd1 : System.IO.FileInfo
When using `Invoke-PSRule` an easy way to catch an failure or error conditions is to use the `-Outcome Fail,Error` parameter.
By using this parameter only errors or failures are returned to the pipeline.
A simple `$Null` test can then throw a terminating error to stop the pipeline.
[PASS] File.Header
[PASS] File.Encoding
[WARN] Target object 'ObjectFromFile.yaml' has not been processed because no matching rules were found.
[WARN] Target object 'ObjectFromNestedFile.yaml' has not been processed because no matching rules were found.
[WARN] Target object 'Baseline.Rule.yaml' has not been processed because no matching rules were found.
```powershell
$result = Invoke-PSRule -Outcome Fail,Error;
if ($Null -ne $result) {
throw 'PSRule validation failed.'
}
-> FromFile.Rule.ps1 : System.IO.FileInfo
[FAIL] File.Header
[PASS] File.Encoding
```
Extending on this further, PSRule has additional options that we can use to log passing/ failing validation rules to informational streams.
By using the `Logging.RuleFail` option shown in the next example an error will be created for each failure so that meaningful information is logged to the CI pipeline.
In the next example, objects from file are validated against pre-defined rules from a module:
```powershell
$option = New-PSRuleOption -LoggingRuleFail Error;
$result = $inputObjects | Invoke-PSRule -Option $option -Outcome Fail,Error;
if ($Null -ne $result) {
throw 'PSRule validation failed.'
}
Assert-PSRule -InputPath .\resources-*.json -Module PSRule.Rules.Azure;
```
### Calling from Pester
## Formatting output
If you are looking at integrating PSRule into a CI pipeline, there is a good chance that you are already using Pester.
Pester is a unit test framework for PowerShell that can be installed from the PowerShell Gallery.
When executing a CI pipeline, feedback on any validation failures is important.
The `Assert-PSRule` cmdlet provides easy to read formatted output instead of PowerShell objects.
PSRule can complement Pester unit tests with dynamic validation rules.
By using `-If` or `-Type` pre-conditions rules can dynamically provide validation for a range of use cases.
Additionally, `Assert-PSRule` supports styling formatted output for Azure Pipelines and GitHub Actions.
Use the `-Style AzurePipelines` or `-Style GitHubActions` parameter to style output.
In our example we are going to validate the script files themselves:
- Have a copyright file header
- Are encoded as UTF-8
Within a Pester test script include the following example:
For example:
```powershell
Describe 'Project files' {
Context 'Script files' {
It 'Use content rules' {
$option = New-PSRuleOption -LoggingRuleFail Error;
$inputObjects = Get-ChildItem -Path *.ps1 -Recurse;
$inputObjects | Invoke-PSRule -Option $option -Outcome Fail,Error | Should -BeNullOrEmpty;
}
}
$items | Assert-PSRule -Path './.ps-rule/' -Style AzurePipelines;
```
## Failing the pipeline
When using PSRule within a CI pipeline, a failed rule should stop the pipeline.
When using `Assert-PSRule` if any rules fail, an error will be generated.
```text
Assert-PSRule : One or more rules reported failure.
At line:1 char:10
+ $items | Assert-PSRule -Path ./.ps-rule/
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidData: (:) [Assert-PSRule], FailPipelineException
+ FullyQualifiedErrorId : PSRule.Fail,Assert-PSRule
```
A single PowerShell error is typically enough to stop a CI pipeline.
If you are using a different configuration additionally `-ErrorAction Stop` can be used.
For example:
```powershell
$items | Assert-PSRule -Path './.ps-rule/' -ErrorAction Stop;
```
Using `-ErrorAction Stop` will stop the current script and return an exit code of 1.
To continue running the current script but return an exit code, use:
```powershell
try {
$items | Assert-PSRule -Path './.ps-rule/' -ErrorAction Stop;
}
catch {
$Host.SetShouldExit(1);
}
```
## Generating NUnit output
NUnit is a popular unit test framework for .NET. NUnit generates a test report format that is widely interpreted by CI systems.
While PSRule does not use NUnit, it can output the same test report format allowing integration with any system that supports the NUnit3 for publishing test results.
NUnit is a popular unit test framework for .NET.
NUnit generates a test report format that is widely interpreted by CI systems.
While PSRule does not use NUnit directly, it support outputting validation results in the NUnit3 format.
Using a common format allows integration with any system that supports the NUnit3 for publishing test results.
To generate an NUnit report use the `-OutputFormat NUnit3` parameter.
To generate an NUnit report:
- Use the `-OutputFormat NUnit3` parameter.
- Use the `-OutputPath` parameter to specify the path of the report file to write.
```powershell
$option = New-PSRuleOption -LoggingRuleFail Error;
$inputObjects = Get-ChildItem -Path *.ps1 -Recurse;
$inputObjects | Invoke-PSRule -Option $option -OutputFormat NUnit3 | Set-Content -Path reports/rule.report.xml;
$items | Assert-PSRule -Path './.ps-rule/' -OutputFormat NUnit3 -OutputPath reports/rule-report.xml;
```
The output path will be created if it does not exist.
### Publishing NUnit report with Azure DevOps
With Azure DevOps, an NUnit report can be published using [Publish Test Results task][publish-test-results].
@ -151,25 +180,146 @@ An example YAML snippet is included below:
inputs:
testRunTitle: 'PSRule'
testRunner: NUnit
testResultsFiles: 'reports/rule.report.xml'
testResultsFiles: 'reports/rule-report.xml'
mergeTestResults: true
publishRunAttachments: true
condition: succeededOrFailed()
```
## Examples
## Complete example
For our example we ran:
Putting each of these steps together.
### Install dependencies
```powershell
$option = New-PSRuleOption -LoggingRuleFail Error;
$inputObjects = Get-ChildItem -Path src/PSRule -Include *.ps1,*.psm1,*.psd1 -Recurse;
$inputObjects | Invoke-PSRule -Path docs/scenarios/validation-pipeline -Option $option -OutputFormat NUnit3 | Set-Content -Path reports/rule.report.xml;
# Install dependencies for connecting to PowerShell Gallery
if ($Null -eq (Get-PackageProvider -Name NuGet -ErrorAction SilentlyContinue)) {
Install-PackageProvider -Name NuGet -Force -Scope CurrentUser;
}
if ($Null -eq (Get-InstalledModule -Name PowerShellGet -MinimumVersion '2.2.1' -ErrorAction SilentlyContinue)) {
Install-Module PowerShellGet -MinimumVersion '2.2.1' -Scope CurrentUser -Force -AllowClobber;
}
```
### Validate files
```powershell
# Install PSRule module
if ($Null -eq (Get-InstalledModule -Name PSRule -MinimumVersion '0.12.0' -ErrorAction SilentlyContinue)) {
Install-Module -Name PSRule -Scope CurrentUser -MinimumVersion '0.12.0' -Force;
}
# Validate files
$assertParams = @{
Path = './.ps-rule/'
Style = 'AzurePipelines'
OutputFormat = 'NUnit3'
OutputPath = 'reports/rule-report.xml'
}
$items = Get-ChildItem -Recurse -Path .\src\,.\tests\ -Include *.ps1,*.psd1,*.psm1,*.yaml;
$items | Assert-PSRule $assertParams -ErrorAction Stop;
```
### Azure DevOps Pipeline
```yaml
steps:
# Install dependencies
- powershell: ./pipeline-deps.ps1
displayName: 'Install dependencies'
# Validate templates
- powershell: ./validate-files.ps1
displayName: 'Validate files'
# Publish pipeline results
- task: PublishTestResults@2
displayName: 'Publish PSRule results'
inputs:
testRunTitle: 'PSRule'
testRunner: NUnit
testResultsFiles: 'reports/rule-report.xml'
mergeTestResults: true
publishRunAttachments: true
condition: succeededOrFailed()
```
## Additional options
### Using Invoke-Build
`Invoke-Build` is a build automation cmdlet that can be installed from the PowerShell Gallery by installing the _InvokeBuild_ module.
Within Invoke-Build, each build process is broken into tasks.
The following example shows an example of using PSRule with _InvokeBuild_ tasks.
```powershell
# Synopsis: Install PSRule
task PSRule {
if ($Null -eq (Get-InstalledModule -Name PSRule -MinimumVersion '0.12.0' -ErrorAction SilentlyContinue)) {
Install-Module -Name PSRule -Scope CurrentUser -MinimumVersion '0.12.0' -Force;
}
}
# Synopsis: Validate files
task ValidateFiles PSRule, {
$assertParams = @{
Path = './.ps-rule/'
Style = 'AzurePipelines'
OutputFormat = 'NUnit3'
OutputPath = 'reports/rule-report.xml'
}
$items = Get-ChildItem -Recurse -Path .\src\,.\tests\ -Include *.ps1,*.psd1,*.psm1,*.yaml;
$items | Assert-PSRule $assertParams -ErrorAction Stop;
}
# Synopsis: Run all build tasks
task Build ValidateFiles
```
```powershell
Invoke-Build Build;
```
### Calling from Pester
Pester is a unit test framework for PowerShell that can be installed from the PowerShell Gallery.
Typically, Pester unit tests are built for a particular pipeline.
PSRule can complement Pester unit tests by providing dynamic and sharable rules that are easy to reuse.
By using `-If` or `-Type` pre-conditions, rules can dynamically provide validation for a range of use cases.
When calling PSRule from Pester use `Invoke-PSRule` instead of `Assert-PSRule`.
`Invoke-PSRule` returns validation result objects that can be tested by Pester `Should` conditions.
Additionally, the `Logging.RuleFail` option can be included to generate an error message for each failing rule.
For example:
```powershell
Describe 'Azure' {
Context 'Resource templates' {
It 'Use content rules' {
$invokeParams = @{
Path = './.ps-rule/'
OutputFormat = 'NUnit3'
OutputPath = 'reports/rule-report.xml'
}
$items = Get-ChildItem -Recurse -Path .\src\,.\tests\ -Include *.ps1,*.psd1,*.psm1,*.yaml;
Invoke-PSRule @invokeParams -Outcome Fail,Error | Should -BeNullOrEmpty;
}
}
}
```
## More information
- [file.Rule.ps1] - Example rules for validating script files.
- [pipeline-deps.ps1](pipeline-deps.ps1) - Example script installing pipeline dependencies.
- [file.Rule.ps1](file.Rule.ps1) - Example rules for validating script files.
- [validate-files.ps1](validate-files.ps1) - Example script for running files validation.
- [azure-pipelines.yaml](azure-pipelines.yaml) - An example Azure DevOps Pipeline.
[file.Rule.ps1]: file.Rule.ps1
[publish-test-results]: https://docs.microsoft.com/en-us/azure/devops/pipelines/tasks/test/publish-test-results