enhancements and documents for IaC
This commit is contained in:
Dennis Eikelenboom
Родитель
5577f74007
Коммит
035f22ab51
|
|
@ -1,8 +1,36 @@
|
|||
# Azure DevOps Configuration
|
||||
# Infrastructure as Code
|
||||
|
||||
To get started:
|
||||
This folder contains examples for how to bootstrap your machine learning workflow.
|
||||
Azure Resource Manager (ARM) templates & Azure ML CLI commands can easily be used to bootstrap and provision workspaces for your data scientists prior to enabling them to begin data preparation & model training.
|
||||
|
||||
* Create a new service connection in Azure DevOps
|
||||
* Choose the Azure Resources Manager connection type
|
||||
* In deploy-infra.yml replace <your-serviceconnection> with your created service connection name.
|
||||
* Adapt globally unique resource names in the ARM templates e.g. storage accounts.
|
||||
* **[ARM-Templates](arm-templates)** contains infrastructure-as-code templates and parameter files for two sample environments (dev + test). The use of ARM templates gives you the most flexibility in customizing your Azure resources.
|
||||
* **[Scripts](scripts)** contains Azure CLI scripts for resource deployment. The use of CLI commands for deployment provides the most lean way to deploy resources to Azure.
|
||||
* **[Build-and-Release](build-and-release)** contains pipeline definitions for Azure DevOps to automate infrastructure roll out. Included moreover is a PowerShell script that can be used for test deployments of the infrastructure resources.
|
||||
|
||||
## Automated roll out of infrastructure
|
||||
|
||||
In this section you will learn about how you could use [Azure Pipelines](https://azure.microsoft.com/en-us/services/devops/pipelines/) for the automated deployment of infrastructure. This way of working enables you to incrementally deploy changes to your resources, stage the changes over different environments, and build confidence as your system growths more complex.
|
||||
|
||||
### Getting started
|
||||
|
||||
Complete the below steps to set up your pipeline for infrastructure roll out.
|
||||
|
||||
* Navigate to [Azure DevOps](http://dev.azure.com/) and create a new organization and project. You can also re-use an existing organization and/or project.
|
||||
* Create a new [service connection](https://docs.microsoft.com/en-us/azure/devops/pipelines/library/service-endpoints?view=azure-devops&tabs=yaml) in Azure DevOps of the Azure Resources Manager connection type. Azure DevOps will authenticate using this connection to make deployments to your Azure Subscription.
|
||||
* In [deploy-infra.yml](build-and-release/deploy-infra.yml) replace `<your-service-connection-name>` by the name of the service connection that you created in the previous step.
|
||||
* Some Azure resources require you to use globally unique names across Azure. This holds for example for storage account resources. Adapt resource names in the ARM parameter files to a name that is globally unique. Note that you should update the parameter files for the ML workspace and ML compute resources as well once you update the names of the underlying resources.
|
||||
* Make a test deployment using the provided powershell script `deploy-infra.ps1`.
|
||||
* Set up a new pipeline in Azure DevOps with the option to re-use an existing template. Point to the pipeline definition [deploy-infra.yml](build-and-release/deploy-infra.yml) in your repository.
|
||||
* Run your pipeline from Azure DevOps. On completion, you should see a result like the below.
|
||||
|
||||
|
||||
### Best practices on customizing the templates for your environment and team
|
||||
|
||||
* Many teams already have existing resources in their Azure tenant for e.g. Keyvault and Application Insights. These resources can be re-used by Azure Machine Learning. Simply point to these resources in the [Machine Learning Workspace template](arm-templates/mlworkspace/template.json). For ease of modification, we have provided separate templates for each of the resources in this repository.
|
||||
* In most situations data already resides on existing storage in Azure. The [Azure CLI ML Extension](https://docs.microsoft.com/en-us/azure/machine-learning/reference-azure-machine-learning-cli) allows for a lean way to add storage as a [Datastore](https://docs.microsoft.com/en-us/azure/machine-learning/concept-data) in Azure Machine Learning. The [Azure CLI task](https://docs.microsoft.com/en-us/azure/devops/pipelines/tasks/deploy/azure-cli?view=azure-devops) in Azure DevOps can help you to automate the datastore attachment process as part of the infrastructure roll out.
|
||||
* Many teams choose to deploy multiple environments to work with, for example DEV, INT and PROD. In this way infrastructure can be rolled out in a phased way and with more confidence as your system becomes more complex.
|
||||
* As one rolls out additional infrastructural resources, it becomes valuable to stage changes across the different environments. You could consider to run a set of integration or component tests before rolling out to PRD.
|
||||
* It is a sound practice to protect the roll out of changes to PRD from originating from branches other than master. [Conditions](https://docs.microsoft.com/en-us/azure/devops/pipelines/process/conditions?view=azure-devops&tabs=yaml) in Azure pipelines can you help to set controls like these.
|
||||
* One could specify a security group of users that require to give their [approval](https://docs.microsoft.com/en-us/azure/devops/pipelines/process/approvals?view=azure-devops&tabs=check-pass#approvals) to make roll outs to specific environments.
|
||||
* It is important to note that in the MLOps way of working, we make a separation of concerns between the roll out of infrastructure and the roll out of ML artifacts. Hence the two types are rolled out at different moments and with different automation pipelines.
|
||||
* Multiple additional security controls (virtual network rules, role-based access control and custom identities) can be applied on the Azure resources that are found in this repository. Controls can be added directly from the ARM templates. Consult the [documentation](https://docs.microsoft.com/en-us/azure/templates/) on Azure Resource Manager to find the possible modifications that can be made to each Azure Resource. As an example on modifications for the template for Azure ML compute, one can find a [template](arm-templates/mlcompute/template-vnet.json) in this repository that adds a SSH user and virtual network controls to the managed compute virtual machines.
|
||||
|
|
@ -3,13 +3,10 @@
|
|||
"contentVersion": "1.0.0.0",
|
||||
"parameters": {
|
||||
"appInsightsName": {
|
||||
"value": "amlquickstartinsights"
|
||||
"value": "mlops-ain-dev"
|
||||
},
|
||||
"regionId": {
|
||||
"value": "westeurope"
|
||||
},
|
||||
"environment": {
|
||||
"value": "dev"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,12 @@
|
|||
{
|
||||
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentParameters.json#",
|
||||
"contentVersion": "1.0.0.0",
|
||||
"parameters": {
|
||||
"appInsightsName": {
|
||||
"value": "mlops-ain-test"
|
||||
},
|
||||
"regionId": {
|
||||
"value": "westeurope"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -7,16 +7,13 @@
|
|||
},
|
||||
"regionId": {
|
||||
"type": "string"
|
||||
},
|
||||
"environment": {
|
||||
"type": "string"
|
||||
}
|
||||
},
|
||||
"resources": [
|
||||
{
|
||||
"type": "Microsoft.Insights/components",
|
||||
"location": "[parameters('regionId')]",
|
||||
"name": "[concat(parameters('appInsightsName'), parameters('environment'))]",
|
||||
"name": "[parameters('appInsightsName')]",
|
||||
"apiVersion": "2015-05-01",
|
||||
"kind": "web",
|
||||
"properties": {
|
||||
|
|
|
|||
|
|
@ -3,13 +3,10 @@
|
|||
"contentVersion": "1.0.0.0",
|
||||
"parameters": {
|
||||
"crname": {
|
||||
"value": "amlquickstartacr"
|
||||
"value": "mlopscrdev"
|
||||
},
|
||||
"location": {
|
||||
"value": "westeurope"
|
||||
},
|
||||
"environment": {
|
||||
"value": "dev"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,12 @@
|
|||
{
|
||||
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentParameters.json#",
|
||||
"contentVersion": "1.0.0.0",
|
||||
"parameters": {
|
||||
"crname": {
|
||||
"value": "mlopscrtest"
|
||||
},
|
||||
"location": {
|
||||
"value": "westeurope"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -7,9 +7,6 @@
|
|||
},
|
||||
"location": {
|
||||
"type": "string"
|
||||
},
|
||||
"environment": {
|
||||
"type": "string"
|
||||
}
|
||||
},
|
||||
"variables": {},
|
||||
|
|
@ -20,7 +17,7 @@
|
|||
"name": "Basic",
|
||||
"tier": "Basic"
|
||||
},
|
||||
"name": "[concat(parameters('crname'), parameters('environment'))]",
|
||||
"name": "[parameters('crname')]",
|
||||
"apiVersion": "2017-10-01",
|
||||
"location": "[parameters('location')]",
|
||||
"tags": {},
|
||||
|
|
|
|||
|
|
@ -3,16 +3,13 @@
|
|||
"contentVersion": "1.0.0.0",
|
||||
"parameters": {
|
||||
"kvname": {
|
||||
"value": "amlquickstartkv"
|
||||
"value": "mlops-kv-dev"
|
||||
},
|
||||
"location": {
|
||||
"value": "westeurope"
|
||||
},
|
||||
"createMode": {
|
||||
"value": "default"
|
||||
},
|
||||
"environment": {
|
||||
"value": "dev"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,15 @@
|
|||
{
|
||||
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentParameters.json#",
|
||||
"contentVersion": "1.0.0.0",
|
||||
"parameters": {
|
||||
"kvname": {
|
||||
"value": "mlops-kv-test"
|
||||
},
|
||||
"location": {
|
||||
"value": "westeurope"
|
||||
},
|
||||
"createMode": {
|
||||
"value": "default"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -10,9 +10,6 @@
|
|||
},
|
||||
"createMode": {
|
||||
"type": "string"
|
||||
},
|
||||
"environment": {
|
||||
"type": "string"
|
||||
}
|
||||
},
|
||||
"variables": {
|
||||
|
|
@ -20,7 +17,7 @@
|
|||
"resources": [
|
||||
{
|
||||
"type": "Microsoft.KeyVault/vaults",
|
||||
"name": "[concat(parameters('kvname'), parameters('environment'))]",
|
||||
"name": "[parameters('kvname')]",
|
||||
"apiVersion": "2018-02-14",
|
||||
"location": "[parameters('location')]",
|
||||
"tags": {},
|
||||
|
|
|
|||
|
|
@ -0,0 +1,33 @@
|
|||
{
|
||||
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentParameters.json#",
|
||||
"contentVersion": "1.0.0.0",
|
||||
"parameters": {
|
||||
"workspaceName": {
|
||||
"value": "mlops-mls-dev"
|
||||
},
|
||||
"clusterName": {
|
||||
"value": "cpu-compute"
|
||||
},
|
||||
"vmSize": {
|
||||
"value": "STANDARD_D3_V2"
|
||||
},
|
||||
"minNodeCount": {
|
||||
"value": 0
|
||||
},
|
||||
"maxNodeCount": {
|
||||
"value": 3
|
||||
},
|
||||
"scaleDownTime": {
|
||||
"value": "PT15M"
|
||||
},
|
||||
"subnetId": {
|
||||
"value": "/subscriptions/xxxx/resourceGroups/yyyy/providers/Microsoft.Network/virtualNetworks/zzzz/subnets/ssss"
|
||||
},
|
||||
"adminUserName": {
|
||||
"value": "<Fill-here>"
|
||||
},
|
||||
"adminUserPassword": {
|
||||
"value": "<Fill-here>"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,33 @@
|
|||
{
|
||||
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentParameters.json#",
|
||||
"contentVersion": "1.0.0.0",
|
||||
"parameters": {
|
||||
"workspaceName": {
|
||||
"value": "mlops-mls-test"
|
||||
},
|
||||
"clusterName": {
|
||||
"value": "cpu-compute"
|
||||
},
|
||||
"vmSize": {
|
||||
"value": "STANDARD_D3_V2"
|
||||
},
|
||||
"minNodeCount": {
|
||||
"value": 0
|
||||
},
|
||||
"maxNodeCount": {
|
||||
"value": 3
|
||||
},
|
||||
"scaleDownTime": {
|
||||
"value": "PT15M"
|
||||
},
|
||||
"subnetId": {
|
||||
"value": "/subscriptions/xxxx/resourceGroups/yyyy/providers/Microsoft.Network/virtualNetworks/zzzz/subnets/ssss"
|
||||
},
|
||||
"adminUserName": {
|
||||
"value": "<Fill-here>"
|
||||
},
|
||||
"adminUserPassword": {
|
||||
"value": "<Fill-here>"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -3,25 +3,22 @@
|
|||
"contentVersion": "1.0.0.0",
|
||||
"parameters": {
|
||||
"workspaceName": {
|
||||
"value": "amlquickstartworkspace"
|
||||
"value": "mlops-mls-dev"
|
||||
},
|
||||
"clusterName": {
|
||||
"value": "mycompute"
|
||||
"value": "cpu-compute"
|
||||
},
|
||||
"vmSize": {
|
||||
"value": "Standard_D3_v2"
|
||||
"value": "STANDARD_D3_V2"
|
||||
},
|
||||
"minNodeCount": {
|
||||
"value": 0
|
||||
},
|
||||
"maxNodeCount": {
|
||||
"value": 25
|
||||
"value": 3
|
||||
},
|
||||
"scaleDownTime": {
|
||||
"value": "PT10M"
|
||||
},
|
||||
"environment": {
|
||||
"value": "dev"
|
||||
"value": "PT15M"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,24 @@
|
|||
{
|
||||
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentParameters.json#",
|
||||
"contentVersion": "1.0.0.0",
|
||||
"parameters": {
|
||||
"workspaceName": {
|
||||
"value": "mlops-mls-test"
|
||||
},
|
||||
"clusterName": {
|
||||
"value": "cpu-compute"
|
||||
},
|
||||
"vmSize": {
|
||||
"value": "STANDARD_D3_V2"
|
||||
},
|
||||
"minNodeCount": {
|
||||
"value": 0
|
||||
},
|
||||
"maxNodeCount": {
|
||||
"value": 3
|
||||
},
|
||||
"scaleDownTime": {
|
||||
"value": "PT15M"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,66 @@
|
|||
{
|
||||
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
|
||||
"contentVersion": "1.0.0.0",
|
||||
"parameters": {
|
||||
"workspaceName": {
|
||||
"type": "string"
|
||||
},
|
||||
"clusterName": {
|
||||
"type": "string"
|
||||
},
|
||||
"vmSize": {
|
||||
"type": "string"
|
||||
},
|
||||
"minNodeCount": {
|
||||
"type": "int"
|
||||
},
|
||||
"maxNodeCount": {
|
||||
"type": "int"
|
||||
},
|
||||
"scaleDownTime": {
|
||||
"type": "string"
|
||||
},
|
||||
"subnetId": {
|
||||
"type": "string"
|
||||
},
|
||||
"adminUserName": {
|
||||
"type": "string"
|
||||
},
|
||||
"adminUserPassword": {
|
||||
"type": "string"
|
||||
}
|
||||
},
|
||||
"variables": {},
|
||||
"resources": [
|
||||
{
|
||||
"type": "Microsoft.MachineLearningServices/workspaces/computes",
|
||||
"name": "[concat(parameters('workspaceName'), '/', parameters('clusterName'))]",
|
||||
"apiVersion": "2018-11-19",
|
||||
"location" : "[resourceGroup().location]",
|
||||
"properties": {
|
||||
"computeType": "AmlCompute",
|
||||
"computeLocation" : "[resourceGroup().location]",
|
||||
"properties":
|
||||
{
|
||||
"scaleSettings":
|
||||
{
|
||||
"minNodeCount" : "[parameters('minNodeCount')]",
|
||||
"maxNodeCount" : "[parameters('maxNodeCount')]",
|
||||
"nodeIdleTimeBeforeScaleDown": "[parameters('scaleDownTime')]"
|
||||
},
|
||||
"vmPriority": "Dedicated",
|
||||
"vmSize" : "[parameters('vmSize')]",
|
||||
"userAccountCredentials" :
|
||||
{
|
||||
"adminUserName" : "[parameters('adminUserName')]",
|
||||
"adminUserPassword" : "[parameters('adminUserPassword')]"
|
||||
},
|
||||
"subnet" :
|
||||
{
|
||||
"id" : "[parameters('subnetId')]"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
|
|
@ -19,16 +19,13 @@
|
|||
},
|
||||
"scaleDownTime": {
|
||||
"type": "string"
|
||||
},
|
||||
"environment": {
|
||||
"type": "string"
|
||||
}
|
||||
},
|
||||
"variables": {},
|
||||
"resources": [
|
||||
{
|
||||
"type": "Microsoft.MachineLearningServices/workspaces/computes",
|
||||
"name": "[concat(parameters('workspaceName'), parameters('environment'), '/', parameters('clusterName'))]",
|
||||
"name": "[concat(parameters('workspaceName'), '/', parameters('clusterName'))]",
|
||||
"apiVersion": "2018-11-19",
|
||||
"location" : "[resourceGroup().location]",
|
||||
"properties": {
|
||||
|
|
@ -43,7 +40,7 @@
|
|||
"nodeIdleTimeBeforeScaleDown": "[parameters('scaleDownTime')]"
|
||||
},
|
||||
"vmPriority": "Dedicated",
|
||||
"vmSize" : "[parameters('vmSize')]"
|
||||
"vmSize" : "[parameters('vmSize')]"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -3,22 +3,19 @@
|
|||
"contentVersion": "1.0.0.0",
|
||||
"parameters": {
|
||||
"workspaceName": {
|
||||
"value": "amlquickstartworkspace"
|
||||
"value": "mlops-mls-dev"
|
||||
},
|
||||
"keyVaultName": {
|
||||
"value": "amlquickstartkv"
|
||||
"value": "mlops-kv-dev"
|
||||
},
|
||||
"applicationInsightsName": {
|
||||
"value": "amlquickstartinsights"
|
||||
"value": "mlops-ain-dev"
|
||||
},
|
||||
"containerRegistryName": {
|
||||
"value": "amlquickstartacr"
|
||||
"value": "mlopscrdev"
|
||||
},
|
||||
"storageAccountName": {
|
||||
"value": "amlquickstartstorage"
|
||||
},
|
||||
"environment": {
|
||||
"value": "dev"
|
||||
"value": "mlopssadev"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,21 @@
|
|||
{
|
||||
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentParameters.json#",
|
||||
"contentVersion": "1.0.0.0",
|
||||
"parameters": {
|
||||
"workspaceName": {
|
||||
"value": "mlops-mls-test"
|
||||
},
|
||||
"keyVaultName": {
|
||||
"value": "mlops-kv-test"
|
||||
},
|
||||
"applicationInsightsName": {
|
||||
"value": "mlops-ain-test"
|
||||
},
|
||||
"containerRegistryName": {
|
||||
"value": "mlopscrtest"
|
||||
},
|
||||
"storageAccountName": {
|
||||
"value": "mlopssatest"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -16,14 +16,11 @@
|
|||
},
|
||||
"storageAccountName": {
|
||||
"type": "string"
|
||||
},
|
||||
"environment": {
|
||||
"type": "string"
|
||||
}
|
||||
},
|
||||
"resources": [
|
||||
{
|
||||
"name": "[concat(parameters('workspaceName'), parameters('environment'))]",
|
||||
"name": "[parameters('workspaceName')]",
|
||||
"type": "Microsoft.MachineLearningServices/workspaces",
|
||||
"apiVersion": "2018-11-19",
|
||||
"location": "[resourceGroup().location]",
|
||||
|
|
@ -31,10 +28,10 @@
|
|||
"type": "systemAssigned"
|
||||
},
|
||||
"properties": {
|
||||
"keyVault": "[resourceId('Microsoft.KeyVault/vaults', concat(parameters('keyVaultName'), parameters('environment')))]",
|
||||
"applicationInsights": "[resourceId('Microsoft.Insights/components', concat(parameters('applicationInsightsName'), parameters('environment')))]",
|
||||
"containerRegistry": "[resourceId('Microsoft.ContainerRegistry/registries', concat(parameters('containerRegistryName'), parameters('environment')))]",
|
||||
"storageAccount": "[resourceId('Microsoft.Storage/storageAccounts', concat(parameters('storageAccountName'), parameters('environment')))]"
|
||||
"keyVault": "[resourceId('Microsoft.KeyVault/vaults', parameters('keyVaultName'))]",
|
||||
"applicationInsights": "[resourceId('Microsoft.Insights/components', parameters('applicationInsightsName'))]",
|
||||
"containerRegistry": "[resourceId('Microsoft.ContainerRegistry/registries', parameters('containerRegistryName'))]",
|
||||
"storageAccount": "[resourceId('Microsoft.Storage/storageAccounts', parameters('storageAccountName'))]"
|
||||
}
|
||||
}
|
||||
]
|
||||
|
|
|
|||
|
|
@ -3,10 +3,7 @@
|
|||
"contentVersion": "1.0.0.0",
|
||||
"parameters": {
|
||||
"saname": {
|
||||
"value": "amlquickstartstorage"
|
||||
},
|
||||
"environment": {
|
||||
"value": "dev"
|
||||
"value": "mlopssadev"
|
||||
},
|
||||
"location": {
|
||||
"value": "westeurope"
|
||||
|
|
@ -0,0 +1,21 @@
|
|||
{
|
||||
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentParameters.json#",
|
||||
"contentVersion": "1.0.0.0",
|
||||
"parameters": {
|
||||
"saname": {
|
||||
"value": "mlopssatest"
|
||||
},
|
||||
"location": {
|
||||
"value": "westeurope"
|
||||
},
|
||||
"accountType": {
|
||||
"value": "Standard_RAGRS"
|
||||
},
|
||||
"kind": {
|
||||
"value": "StorageV2"
|
||||
},
|
||||
"accessTier": {
|
||||
"value": "Hot"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -5,9 +5,6 @@
|
|||
"saname": {
|
||||
"type": "string"
|
||||
},
|
||||
"environment": {
|
||||
"type": "string"
|
||||
},
|
||||
"location": {
|
||||
"type": "string"
|
||||
},
|
||||
|
|
@ -29,7 +26,7 @@
|
|||
"name": "[parameters('accountType')]"
|
||||
},
|
||||
"kind": "[parameters('kind')]",
|
||||
"name": "[concat(parameters('saname'), parameters('environment'))]",
|
||||
"name": "[parameters('saname')]",
|
||||
"apiVersion": "2018-07-01",
|
||||
"location": "[parameters('location')]",
|
||||
"properties": {
|
||||
|
|
|
|||
|
|
@ -0,0 +1,40 @@
|
|||
# Deployment script for machine learning resources
|
||||
# Run locally to debug changes in the resource configuration
|
||||
# Use `deploy-infrastructure.yml` for automation of deployments.
|
||||
|
||||
# Prompt users for resource group and location
|
||||
$resourceGroupName = Read-Host -Prompt "Provide a resource group name"
|
||||
$location = Read-Host -Prompt "Provide a DC location"
|
||||
|
||||
# Create a Resource Group
|
||||
New-AzResourceGroup -Name $resourceGroupName -Location $location
|
||||
|
||||
# Deploy Storage Account
|
||||
New-AzResourceGroupDeployment -ResourceGroupName $resourceGroupName `
|
||||
-TemplateFile $PSScriptRoot/../arm-templates/storage/template.json `
|
||||
-TemplateParameterFile $PSScriptRoot/../arm-templates/storage/parameters.dev.json
|
||||
|
||||
# Deploy Container Registry
|
||||
New-AzResourceGroupDeployment -ResourceGroupName $resourceGroupName `
|
||||
-TemplateFile $PSScriptRoot/../arm-templates/containerregistry/template.json `
|
||||
-TemplateParameterFile $PSScriptRoot/../arm-templates/containerregistry/parameters.dev.json
|
||||
|
||||
# Deploy Application Insights
|
||||
New-AzResourceGroupDeployment -ResourceGroupName $resourceGroupName `
|
||||
-TemplateFile $PSScriptRoot/../arm-templates/appinsights/template.json `
|
||||
-TemplateParameterFile $PSScriptRoot/../arm-templates/appinsights/parameters.dev.json
|
||||
|
||||
# Deploy Key Vault
|
||||
New-AzResourceGroupDeployment -ResourceGroupName $resourceGroupName `
|
||||
-TemplateFile $PSScriptRoot/../arm-templates/keyvault/template.json `
|
||||
-TemplateParameterFile $PSScriptRoot/../arm-templates/keyvault/parameters.dev.json
|
||||
|
||||
# Deploy Workspace
|
||||
New-AzResourceGroupDeployment -ResourceGroupName $resourceGroupName `
|
||||
-TemplateFile $PSScriptRoot/../arm-templates/mlworkspace/template.json `
|
||||
-TemplateParameterFile $PSScriptRoot/../arm-templates/mlworkspace/parameters.dev.json
|
||||
|
||||
# Deploy Compute
|
||||
New-AzResourceGroupDeployment -ResourceGroupName $resourceGroupName `
|
||||
-TemplateFile $PSScriptRoot/../arm-templates/mlcompute/template.json `
|
||||
-TemplateParameterFile $PSScriptRoot/../arm-templates/mlcompute/parameters.dev.json
|
||||
|
|
@ -2,7 +2,6 @@
|
|||
parameters:
|
||||
- name: environment
|
||||
type: string
|
||||
default: dev
|
||||
- name: serviceConnection
|
||||
type: string
|
||||
|
||||
|
|
@ -14,7 +13,7 @@ jobs:
|
|||
environment: ${{ parameters.environment }}
|
||||
variables:
|
||||
- name: resourceGroupName
|
||||
value: amlquickstart-${{ parameters.environment }}
|
||||
value: mlopsexample-${{ parameters.environment }}
|
||||
- name: resourceGroupLocation
|
||||
value: westeurope
|
||||
strategy:
|
||||
|
|
@ -27,15 +26,25 @@ jobs:
|
|||
displayName: 'List dirs'
|
||||
|
||||
- task: AzureResourceGroupDeployment@2
|
||||
displayName: 'Deploy Storage'
|
||||
displayName: 'Deploy Storage Account for AML'
|
||||
inputs:
|
||||
azureSubscription: ${{ parameters.serviceConnection }}
|
||||
resourceGroupName: $(resourceGroupName)
|
||||
location: $(resourceGroupLocation)
|
||||
csmFile: '$(Pipeline.Workspace)/infratemplates/storage/template.json'
|
||||
csmParametersFile: '$(Pipeline.Workspace)/infratemplates/storage/parameters.json'
|
||||
overrideParameters: |
|
||||
-environment "${{ parameters.environment }}"
|
||||
csmParametersFile: '$(Pipeline.Workspace)/infratemplates/storage/parameters.${{ parameters.environment }}.json'
|
||||
|
||||
# Optional - Add a second storage account to host data for machine learning
|
||||
# - task: AzureResourceGroupDeployment@2
|
||||
# displayName: 'Deploy Storage Account for Data'
|
||||
# inputs:
|
||||
# azureSubscription: ${{ parameters.serviceConnection }}
|
||||
# resourceGroupName: $(resourceGroupName)
|
||||
# location: $(resourceGroupLocation)
|
||||
# csmFile: '$(Pipeline.Workspace)/infratemplates/storage/template.json'
|
||||
# csmParametersFile: '$(Pipeline.Workspace)/infratemplates/storage/parameters.${{ parameters.environment }}.json'
|
||||
# overrideParameters: |
|
||||
# -name "mlopssadata${{ parameters.environment }}"
|
||||
|
||||
- task: AzureResourceGroupDeployment@2
|
||||
displayName: 'Deploy Container Registry'
|
||||
|
|
@ -44,9 +53,7 @@ jobs:
|
|||
resourceGroupName: $(resourceGroupName)
|
||||
location: $(resourceGroupLocation)
|
||||
csmFile: '$(Pipeline.Workspace)/infratemplates/containerregistry/template.json'
|
||||
csmParametersFile: '$(Pipeline.Workspace)/infratemplates/containerregistry/parameters.json'
|
||||
overrideParameters: |
|
||||
-environment "${{ parameters.environment }}"
|
||||
csmParametersFile: '$(Pipeline.Workspace)/infratemplates/containerregistry/parameters.${{ parameters.environment }}.json'
|
||||
|
||||
- task: AzureResourceGroupDeployment@2
|
||||
displayName: 'Deploy Application Insights'
|
||||
|
|
@ -55,9 +62,7 @@ jobs:
|
|||
resourceGroupName: $(resourceGroupName)
|
||||
location: $(resourceGroupLocation)
|
||||
csmFile: '$(Pipeline.Workspace)/infratemplates/appinsights/template.json'
|
||||
csmParametersFile: '$(Pipeline.Workspace)/infratemplates/appinsights/parameters.json'
|
||||
overrideParameters: |
|
||||
-environment "${{ parameters.environment }}"
|
||||
csmParametersFile: '$(Pipeline.Workspace)/infratemplates/appinsights/parameters.${{ parameters.environment }}.json'
|
||||
|
||||
- task: AzureResourceGroupDeployment@2
|
||||
displayName: 'Deploy Key Vault'
|
||||
|
|
@ -66,9 +71,7 @@ jobs:
|
|||
resourceGroupName: $(resourceGroupName)
|
||||
location: $(resourceGroupLocation)
|
||||
csmFile: '$(Pipeline.Workspace)/infratemplates/keyvault/template.json'
|
||||
csmParametersFile: '$(Pipeline.Workspace)/infratemplates/keyvault/parameters.json'
|
||||
overrideParameters: |
|
||||
-environment "${{ parameters.environment }}"
|
||||
csmParametersFile: '$(Pipeline.Workspace)/infratemplates/keyvault/parameters.${{ parameters.environment }}.json'
|
||||
|
||||
- task: AzureResourceGroupDeployment@2
|
||||
displayName: 'Deploy ML Workspace'
|
||||
|
|
@ -77,9 +80,20 @@ jobs:
|
|||
resourceGroupName: $(resourceGroupName)
|
||||
location: $(resourceGroupLocation)
|
||||
csmFile: '$(Pipeline.Workspace)/infratemplates/mlworkspace/template.json'
|
||||
csmParametersFile: '$(Pipeline.Workspace)/infratemplates/mlworkspace/parameters.json'
|
||||
overrideParameters: |
|
||||
-environment "${{ parameters.environment }}"
|
||||
csmParametersFile: '$(Pipeline.Workspace)/infratemplates/mlworkspace/parameters.${{ parameters.environment }}.json'
|
||||
|
||||
# Optional - Add a second ML workspace using the same underlying infrastructure
|
||||
# - task: AzureResourceGroupDeployment@2
|
||||
# displayName: 'Deploy ML Workspace'
|
||||
# inputs:
|
||||
# azureSubscription: ${{ parameters.serviceConnection }}
|
||||
# resourceGroupName: $(resourceGroupName)
|
||||
# location: $(resourceGroupLocation)
|
||||
# csmFile: '$(Pipeline.Workspace)/infratemplates/mlworkspace/template.json'
|
||||
# csmParametersFile: '$(Pipeline.Workspace)/infratemplates/mlworkspace/parameters.${{ parameters.environment }}.json'
|
||||
# csmParametersFile: '$(Pipeline.Workspace)/infratemplates/storage/parameters.${{ parameters.environment }}.json'
|
||||
# overrideParameters: |
|
||||
# -name "mlops-mls2-${{ parameters.environment }}"
|
||||
|
||||
- task: AzureResourceGroupDeployment@2
|
||||
displayName: 'Deploy ML Compute'
|
||||
|
|
@ -88,6 +102,4 @@ jobs:
|
|||
resourceGroupName: $(resourceGroupName)
|
||||
location: $(resourceGroupLocation)
|
||||
csmFile: '$(Pipeline.Workspace)/infratemplates/mlcompute/template.json'
|
||||
csmParametersFile: '$(Pipeline.Workspace)/infratemplates/mlcompute/parameters.json'
|
||||
overrideParameters: |
|
||||
-environment "${{ parameters.environment }}"
|
||||
csmParametersFile: '$(Pipeline.Workspace)/infratemplates/mlcompute/parameters.${{ parameters.environment }}.json'
|
||||
|
|
@ -32,7 +32,7 @@ stages:
|
|||
- template: deploy-infra.template.yml
|
||||
parameters:
|
||||
environment: dev
|
||||
serviceConnection: <your-service-connection>
|
||||
serviceConnection: <your-service-connection-name>
|
||||
|
||||
- stage: TEST
|
||||
# only make deployments to TEST originating from the master branch
|
||||
|
|
@ -42,5 +42,5 @@ stages:
|
|||
- template: deploy-infra.template.yml
|
||||
parameters:
|
||||
environment: test
|
||||
serviceConnection: <your-service-connection>
|
||||
serviceConnection: <your-service-connection-name>
|
||||
|
||||
|
|
@ -1,40 +0,0 @@
|
|||
# Deployment script for machine learning resources
|
||||
# Run locally to debug changes in the resource configuration
|
||||
# Use `deploy-infrastructure.yml` for deployment automation.
|
||||
|
||||
# Prompt users for resource group and location
|
||||
$resourceGroupName = Read-Host -Prompt "Provide a resource group name"
|
||||
$location = Read-Host -Prompt "Provide a DC location"
|
||||
|
||||
# Create a Resource Group
|
||||
New-AzResourceGroup -Name $resourceGroupName -Location $location
|
||||
|
||||
# Deploy Storage Account
|
||||
New-AzResourceGroupDeployment -ResourceGroupName $resourceGroupName `
|
||||
-TemplateFile arm-templates/storage/template.json `
|
||||
-TemplateParameterFile arm-templates/storage/parameters.json
|
||||
|
||||
# Deploy Container Registry
|
||||
New-AzResourceGroupDeployment -ResourceGroupName $resourceGroupName `
|
||||
-TemplateFile arm-templates/containerregistry/template.json `
|
||||
-TemplateParameterFile arm-templates/containerregistry/parameters.json
|
||||
|
||||
# Deploy Application Insights
|
||||
New-AzResourceGroupDeployment -ResourceGroupName $resourceGroupName `
|
||||
-TemplateFile arm-templates/appinsights/template.json `
|
||||
-TemplateParameterFile arm-templates/appinsights/parameters.json
|
||||
|
||||
# Deploy Key Vault
|
||||
New-AzResourceGroupDeployment -ResourceGroupName $resourceGroupName `
|
||||
-TemplateFile arm-templates/keyvault/template.json `
|
||||
-TemplateParameterFile arm-templates/keyvault/parameters.json
|
||||
|
||||
# Deploy Workspace
|
||||
New-AzResourceGroupDeployment -ResourceGroupName $resourceGroupName `
|
||||
-TemplateFile arm-templates/mlworkspace/template.json `
|
||||
-TemplateParameterFile arm-templates/mlworkspace/parameters.json
|
||||
|
||||
# Deploy Compute
|
||||
New-AzResourceGroupDeployment -ResourceGroupName $resourceGroupName `
|
||||
-TemplateFile arm-templates/mlcompute/template.json `
|
||||
-TemplateParameterFile arm-templates/mlcompute/parameters.json
|
||||
Двоичный файл не отображается.
|
После Ширина: | Высота: | Размер: 12 KiB |
Разница между файлами не показана из-за своего большого размера
Загрузить разницу
|
|
@ -0,0 +1 @@
|
|||
az ml computetarget create aks -n myaks
|
||||
|
|
@ -0,0 +1 @@
|
|||
az ml computetarget create amlcompute -n cpu --min-nodes 1 --max-nodes 1 -s STANDARD_D3_V2
|
||||
|
|
@ -0,0 +1,2 @@
|
|||
az group create -n myresourcegroup -l westus2
|
||||
az ml workspace create -w myworkspace -g myresourcegroup
|
||||
Загрузка…
Ссылка в новой задаче