diff --git a/Azure-Pipelines.md b/Azure-Pipelines.md index 0ef29fe..528e41a 100644 --- a/Azure-Pipelines.md +++ b/Azure-Pipelines.md @@ -101,6 +101,12 @@ if ($ARM_CLIENT_SECRET) { --query 'id' } +$ConfigVariableGroupId = az pipelines variable-group create ` + --name 'azops' ` + --variables ` + "AZOPS_MODULE_VERSION=" "AZOPS_CUSTOM_SORT_ORDER=false" ` + --query 'id' + # Add a secret to the variable group just created using id from above if using service principal if ($ARM_CLIENT_SECRET) { az pipelines variable-group variable create ` @@ -157,11 +163,9 @@ $Body = @{ $Uri = "`"https://dev.azure.com/$Organization/_apis/accesscontrolentries/${AzureReposSecurityNamespaceId}?api-version=6.0`"" az rest --method post --uri $Uri --body $Body --resource $AzureDevOpsGlobalAppId -o json -# Add pipeline permissions for all three pipelines to the credentials Variable Group +# Add pipeline permissions for all three pipelines to the credentials Variable Groups $AzureDevOpsGlobalAppId = '499b84ac-1321-427f-aa17-267ca6975798' $Pipelines = az pipelines list --query "[? contains(name,'AzOps')].{id:id,name:name}" | ConvertFrom-Json -$VariableGroup = az pipelines variable-group list --query "[?name=='credentials'].{id:id,name:name}" | ConvertFrom-Json -$Uri = "`"https://dev.azure.com/$Organization/$ProjectName/_apis/pipelines/pipelinepermissions/variablegroup/$($VariableGroup.id)?api-version=6.1-preview.1`"" $Body = @( @{ resource = @{} @@ -175,7 +179,11 @@ $Body = @( ) } ) | ConvertTo-Json -Depth 5 -Compress | ConvertTo-Json # Convert to json twice to properly escape characters for Python interpreter -az rest --method patch --uri $Uri --body $Body --resource $AzureDevOpsGlobalAppId -o json +foreach($groupName in 'credentials','azops') { + $VariableGroup = az pipelines variable-group list --query "[?name=='$groupName'].{id:id,name:name}" | ConvertFrom-Json + $Uri = "`"https://dev.azure.com/$Organization/$ProjectName/_apis/pipelines/pipelinepermissions/variablegroup/$($VariableGroup.id)?api-version=6.1-preview.1`"" + az rest --method patch --uri $Uri --body $Body --resource $AzureDevOpsGlobalAppId -o json +} ``` - Your new Project is now ready. Skip down to [Configuration, clean up and triggering the pipelines](#configuration-clean-up-and-triggering-the-pipelines) to get started. diff --git a/Frequently-Asked-Questions.md b/Frequently-Asked-Questions.md index d758822..307c67c 100644 --- a/Frequently-Asked-Questions.md +++ b/Frequently-Asked-Questions.md @@ -131,3 +131,10 @@ Yes, ensure the following setting combinations are applied (replace `rgname1`, ` "Core.SubscriptionsToIncludeResourceGroups": ["SubscriptionId1","SubscriptionId2"] ``` + +### **I want to deploy a set of templates in a specific order** + +Can AzOps settings be configured to enable this? + +Yes, ensure that the variable `AZOPS_CUSTOM_SORT_ORDER` is set to `true` and create a file named `.order` in the same folder as your template files. +Template files listed in the order file will be deployed in the order specified in the file and before any other templates. diff --git a/Settings.md b/Settings.md index 7d18ce2..730166a 100644 --- a/Settings.md +++ b/Settings.md @@ -1,4 +1,10 @@ -# AzOps Settings +## In this guide +- [AzOps Settings](#azops-settings) +- [Workflow / Pipeline Settings](#workflow--pipeline-settings) + - [Workflow Settings in GitHub](#workflow-settings-in-github) + - [Pipeline Settings in Azure DevOps](#pipeline-settings-in-azure-devops) + +## AzOps Settings The following configuration values can be modified within the `settings.json` file to change the default behavior of AzOps. @@ -27,5 +33,27 @@ The following configuration values can be modified within the `settings.json` fi | 21 | State | Folder to store AzOpsState artefact, defaults to `root` | `"Core.State: "/root"` | | 22 | SubscriptionsToIncludeResourceGroups | Filter which Subscriptions should include Resource Groups in pull | `"Core.SubscriptionsToIncludeResourceGroups": ["*"]` | | 23 | TemplateParameterFileSuffix | Default template file suffix. *Not recommended to change* | `"Core.TemplateParameterFileSuffix": ".json"` | -| 24 | ThrottleLimit | Throttle limit used in Foreach-Object -Parallel for resource/subscription discovery | `"Core.ThrottleLimit": 10` | +| 24 | ThrottleLimit | Default template file suffix. *Not recommended to change* | `"Core.ThrottleLimit": 10` | | 25 | WhatifExcludedChangeTypes | Exclude specific change types from WhatIf operations | `"Core.WhatifExcludedChangeTypes": ["NoChange","Ignore"]` | + +## Workflow / Pipeline Settings + +The following settings can be modified as variables inside GitHub or Azure DevOps and will affect how the workflow or pipeline is run. + +* **AZOPS_MODULE_VERSION** + Set this to the version of the AzOps module you want to use. If the variable is missing, the latest version will be used. + Typically used to pin the version of the module to a specific version and update to new versions at a controlled manner. + Make sure to have a process to continously update this variable if it is set. + +* **AZOPS_CUSTOM_SORT_ORDER** + Set this variable to `true` to enable custom sort ordering. When enabled, create a file named `.order` in a folder where you want to control the deployment order of templates. + Any file that is listed by name in `.order` will will be deployed before other files and in the order that they are listed. + +### Workflow Settings in GitHub + +Settings in GitHub are configured as Secrets by navigating to Settings -> Secrets -> Actions as described in [GitHub Secrets](https://docs.github.com/en/actions/security-guides/encrypted-secrets#creating-encrypted-secrets-for-a-repository). + +### Pipeline Settings in Azure DevOps + +Settings in Azure DevOps are configured as variable groups, as described in [Azure DevOps Variable Groups](https://docs.microsoft.com/azure/devops/pipelines/library/variable-groups). + diff --git a/Steps.md b/Steps.md index 3ffc837..8df0d6c 100644 --- a/Steps.md +++ b/Steps.md @@ -11,36 +11,64 @@ * **Checkout** This stage checkouts out the repository source code from the Source Control Platform to the CI/CD runner. -* **Dependencies** - Download and install AzOps (*Pre-release*), Az.Accounts, Az.Billing, Az.Resources and PSFramework modules. +* **Get Latest AzOps version** *(from sharedSteps template/action)* + *Condition*: Only runs if variable `AZOPS_MODULE_VERSION` IS NOT set + Get the latest AzOps version from PowerShell Gallery and set the variable `AZOPS_MODULE_VERSION` to the version number. -* **Connect** - Authenticate the PowerShell session on the runner via Service Principal. +* **Cache AzOps module** *(from sharedSteps template/action)* + *Condition*: Only runs if variable `AZOPS_MODULE_VERSION` IS set + Search cache for the AzOps module with the version number set in the variable `AZOPS_MODULE_VERSION` and restore that. -* **Diff** +* **Dependencies** *(from sharedSteps template/action)* + *Condition*: Only runs if AzOps module was not restored from cache + Download and install AzOps and its dependencies (Az.Accounts, Az.Billing, Az.Resources and PSFramework) from PowerShell gallery. + +* **Connect** *(from sharedSteps template/action)* + Authenticate the PowerShell session on the runner via Service Principal or Managed Identity. + +* **Diff** *(from validate-deploy template/action)* Validate if there have been any changes within Azure Resource Manager and the Git representation of the hierarchy. -* **Validate** - Validate the new template changes to Azure Resource Manager +* **Custom Sorting** *(from validate-deploy template/action)* + *Condition*: Only runs if variable `AZOPS_CUSTOM_SORT_ORDER` is `true` + Import all files from **Diff** step and check for a `.order` file in the same directory. + Rearrange files in the order specified in the `.order` file to be deployed before other files in the same directory. + +* **Validate** *(from validate-deploy template/action)* + Push the new template changes to Azure Resource Manager * **Results** - Post the results from the What-If API into the Pull Request + Post the results from the What-If API into the Pull Request ### Push * **Checkout** This stage checkouts out the repository source code from the Source Control Platform to the CI/CD runner. -* **Dependencies** - Download and install AzOps (*Pre-release*), Az.Accounts, Az.Billing, Az.Resources and PSFramework modules. +* **Get Latest AzOps version** *(from sharedSteps template/action)* + *Condition*: Only runs if variable `AZOPS_MODULE_VERSION` IS NOT set + Get the latest AzOps version from PowerShell Gallery and set the variable `AZOPS_MODULE_VERSION` to the version number. -* **Connect** - Authenticate the PowerShell session on the runner via Service Principal. +* **Cache AzOps module** *(from sharedSteps template/action)* + *Condition*: Only runs if variable `AZOPS_MODULE_VERSION` IS set + Search cache for the AzOps module with the version number set in the variable `AZOPS_MODULE_VERSION` and restore that. -* **Diff** +* **Dependencies** *(from sharedSteps template/action)* + *Condition*: Only runs if AzOps module was not restored from cache + Download and install AzOps and its dependencies (Az.Accounts, Az.Billing, Az.Resources and PSFramework) from PowerShell gallery. + +* **Connect** *(from sharedSteps template/action)* + Authenticate the PowerShell session on the runner via Service Principal or Managed Identity. + +* **Diff** *(from validate-deploy template/action)* Validate if there have been any changes within Azure Resource Manager and the Git representation of the hierarchy. -* **Deploy** +* **Custom Sorting** *(from validate-deploy template/action)* + *Condition*: Only runs if variable `AZOPS_CUSTOM_SORT_ORDER` is `true` + Import all files from **Diff** step and check for a `.order` file in the same directory. + Rearrange files in the order specified in the `.order` file to be deployed before other files in the same directory. + +* **Deploy** *(from validate-deploy template/action)* Push the new template changes to Azure Resource Manager ### Pull @@ -48,18 +76,27 @@ * **Checkout** This stage checkouts out the repository source code from the Source Control Platform to the CI/CD runner. +* **Get Latest AzOps version** *(from sharedSteps template/action)* + *Condition*: Only runs if variable `AZOPS_MODULE_VERSION` IS NOT set + Get the latest AzOps version from PowerShell Gallery and set the variable `AZOPS_MODULE_VERSION` to the version number. + +* **Cache AzOps module** *(from sharedSteps template/action)* + *Condition*: Only runs if variable `AZOPS_MODULE_VERSION` IS set + Search cache for the AzOps module with the version number set in the variable `AZOPS_MODULE_VERSION` and restore that. + +* **Dependencies** *(from sharedSteps template/action)* + *Condition*: Only runs if AzOps module was not restored from cache + Download and install AzOps and its dependencies (Az.Accounts, Az.Billing, Az.Resources and PSFramework) from PowerShell gallery. + +* **Connect** *(from sharedSteps template/action)* + Authenticate the PowerShell session on the runner via Service Principal or Managed Identity. + * **Configure** - Setup the local git command line tools to allow to commit and push changes back to the repository. + Setup the local git command-line tools to allow to commit and push changes back to the repository. * **Checkout** Switch branches from the main branch to a newly created automated branch. -* **Dependencies** - Download and install AzOps, Az.Accounts, Az.Billing, Az.Resources and PSFramework modules. - -* **Connect** - Authenticate the PowerShell session on the runner via Service Principal. - * **Initialize** Generate a local file system structure of the Azure Resource Manager hierarchy within the runner.