Merged PR 186: automating secrets scope creation using databricks CLI

automating secrets scope creation


Former-commit-id: 87b7aee0a2d1a8ed6918f37f2a2a4bf89853de1b

infrastructure-as-code/scripts/DatabricksSecrets.ps1

@@ -39,9 +39,13 @@ if(! $assigment){
     New-AzRoleAssignment -ObjectId $principal.Id -Scope $lake.Id -RoleDefinitionName "Storage Blob Data Contributor" 
 }
 
-Write-Host "Finished! Please configure databricks with the following values:" -ForegroundColor Blue
-Write-Host "`t Scope:`t`t dataops" -ForegroundColor Cyan
-Write-Host "`t DNS Name:`t $($kv.VaultUri)" -ForegroundColor Cyan
-Write-Host "`t Resource ID:`t $($kv.ResourceId)" -ForegroundColor Cyan
+Write-Host "Creating the Key Vault secret scope on Databricks..." -ForegroundColor Green
+$accessToken = Get-AzAccessToken -ResourceUrl 2ff814a6-3304-4ab8-85cb-cd0e6f879c1d
+$env:DATABRICKS_TOKEN = $accessToken.Token
+$env:DATABRICKS_HOST = "https://$($dbw.Url)"
+$scopesList = databricks secrets list-scopes --output json | ConvertFrom-Json
+if (! $scopesList.scopes.name -contains "dataops") {
+    databricks secrets create-scope --scope 'dataops' --scope-backend-type AZURE_KEYVAULT --resource-id $kv.ResourceId --dns-name $kv.VaultUri
+}
 
-Start-Process "https://$($dbw.Url)#secrets/createScope"
+Write-Host "Finished!" -ForegroundColor Blue

quickstart/docs/0-prerequisites.md

@@ -27,7 +27,7 @@
 
 ## PowerShell
 
-1. The lab requires **PowerShel 7.1** with [PowerShell Core](https://docs.microsoft.com/en-us/powershell/module/microsoft.powershell.core/?view=powershell-7.1) module, which can be installed either on Windows or Linux.
+1. The lab requires **PowerShell 7.1** with [PowerShell Core](https://docs.microsoft.com/en-us/powershell/module/microsoft.powershell.core/?view=powershell-7.1) module, which can be installed either on Windows or Linux.
 
   - If you have a preference to run PowerShell on Windows, follow the [Installing PowerShell on Windows](https://docs.microsoft.com/en-us/powershell/scripting/install/installing-powershell-core-on-windows?view=powershell-7.1) instructions.
   - Otherwise, follow the [Installing PowerShell on Linux](https://docs.microsoft.com/en-us/powershell/scripting/install/installing-powershell-core-on-linux?view=powershell-7.1) instructions.
@@ -36,4 +36,12 @@
 
 2. Install the [Azure Az PowerShell module](https://docs.microsoft.com/en-us/powershell/azure/install-az-ps?view=azps-6.2.0).
 
+## Azure CLI
 
+1. Install the [Azure CLI](https://docs.microsoft.com/en-us/cli/azure/install-azure-cli).
+
+2. Install the [Azure DevOps CLI](https://docs.microsoft.com/en-us/azure/devops/cli/?view=azure-devops).
+
+## Other Tools
+
+1. Install the [Databricks CLI](https://docs.microsoft.com/en-us/azure/databricks/dev-tools/cli/#install-the-cli).

quickstart/docs/4-create-databricks-secrets-scope.md

@@ -1,13 +1,22 @@
 # Create Databricks secrets scope
 
-After completing the [Preparing your Azure DevOps project](./3-azdo-setup.md) step, make sure the Infrastructure as Code CD pipeline `dataops-iac-cd` is executed successfully.
+## Infrastructure Pipeline
 
-Then, run the PowerShell script located at `infrastructure-as-code/scripts` to create the Databricks secrets scope for each environment:
+After completing the [Preparing your Azure DevOps project](./3-azdo-setup.md) step, make sure the Infrastructure as Code CD pipeline `dataops-iac-cd` is executed successfully when triggered from the `develop` branch.
+
+Right after, open a PR from `develop` to `qa` to promote the code changes to the QA environment. Please wait again for the creation of the QA infrastructure.
+Repeat the process one last time, opening a PR from `qa` to `main` to promote the code changes to the PROD environment. Please wait again for the creation of the PROD infrastructure.
+
+## Databricks Secrets Scope
+
+Then, run the PowerShell script located at `infrastructure-as-code/scripts` to create the Databricks secrets scope for **each environment**:
 
 ```
+$clientSecret = ConvertTo-SecureString -AsPlainText
+
 ./DatabricksSecrets.ps1 `
   -ClientID "<client_id>" `
-  -ClientSecret "<client_secret>" `
+  -ClientSecret $clientSecret `
   -DataResourceGroup "<data_resource_group_name>" `
   -ComputeResourceGroup "<compute_resource_group_name>" `
   -KeyVaultName "<kv_name>" `