azure-sdk-for-python/eng/common/TestResources

Live Test Resource Management

Running and recording live tests often requires first creating some resources in Azure. Service directories that include a test-resources.json or test-resources.bicep file require running New-TestResources.ps1 to create these resources and output environment variables you must set.

The following scripts can be used both in on your desktop for developer scenarios as well as on hosted agents for continuous integration testing.

• New-TestResources.ps1 - Creates new test resources for a given service.
• Remove-TestResources.ps1 - Deletes previously created resources.

Prerequisites

1. Install PowerShell version 7.0 or newer.
2. Install the Azure PowerShell.

On the Desktop

To set up your Azure account to run live tests, you'll need to log into Azure, and create the resources defined in your test-resources.json or test-resources.bicep template as shown in the following example using Azure Key Vault. The script will create a service principal automatically, or you may create a service principal that can be reused subsequently.

Note that -Subscription is an optional parameter but recommended if your account is a member of multiple subscriptions. If you didn't specify it when logging in, you should select your desired subscription using Select-AzSubscription. The default can be saved using Set-AzDefault for future sessions.

Connect-AzAccount -Subscription 'YOUR SUBSCRIPTION ID'
eng\common\TestResources\New-TestResources.ps1 keyvault

The OutFile switch will be set by default if you are running this for a .NET project on Windows. This will save test environment settings into a test-resources.json.env file next to test-resources.json or a test-resources.bicep.env file next to test-resources.bicep. The file is protected via DPAPI. The environment file would be scoped to the current repository directory and avoids the need to set environment variables or restart your IDE to recognize them.

Along with some log messages, this will output environment variables based on your current shell like in the following example:

${env:KEYVAULT_TENANT_ID} = '<<secret>>'
${env:KEYVAULT_CLIENT_ID} = '<<secret>>'
${env:KEYVAULT_CLIENT_SECRET} = '<<secret>>'
${env:KEYVAULT_SUBSCRIPTION_ID} = 'YOUR SUBSCRIPTION ID'
${env:KEYVAULT_RESOURCE_GROUP} = 'rg-myusername'
${env:KEYVAULT_LOCATION} = 'westus'
${env:KEYVAULT_SKU} = 'premium'
${env:AZURE_KEYVAULT_URL} = '<<url>>'

For security reasons we do not set these environment variables automatically for either the current process or persistently for future sessions. You must do that yourself based on your current platform and shell.

If your current shell was detected properly, you should be able to copy and paste the output directly in your terminal and add to your profile script. For example, in PowerShell on Windows you can copy the output above and paste it back into the terminal to set those environment variables for the current process. To persist these variables for future terminal sessions or for applications started outside the terminal, you could copy and paste the following commands:

setx KEYVAULT_TENANT_ID ${env:KEYVAULT_TENANT_ID}
setx KEYVAULT_CLIENT_ID ${env:KEYVAULT_CLIENT_ID}
setx KEYVAULT_CLIENT_SECRET ${env:KEYVAULT_CLIENT_SECRET}
setx KEYVAULT_SUBSCRIPTION_ID ${env:KEYVAULT_SUBSCRIPTION_ID}
setx KEYVAULT_RESOURCE_GROUP ${env:KEYVAULT_RESOURCE_GROUP}
setx KEYVAULT_LOCATION ${env:KEYVAULT_LOCATION}
setx KEYVAULT_SKU ${env:KEYVAULT_SKU}
setx AZURE_KEYVAULT_URL ${env:AZURE_KEYVAULT_URL}

Pre- and Post- Scripts

Sometimes creating test resources requires either some work to be done prior to or after the main test-resources.json script is executed. For these scenarios a test-resources-pre.ps1 or test-resources-post.ps1, respectively, can be created in the same folder as the test-resources.json file.

For example, it may be necessary to create artifacts prior to provisioning the actual resource, such as a certificate. Typically the created artifact will need to be passed to test-resources.json to be used in the ARM template or as output (or both).

Below is an example of how $templateFileParameters can be used to pass data from the pre- script to test-resources.json.

Snippet from test-resources-pre.ps1

Import-Module -Name ./eng/common/scripts/X509Certificate2
$cert = New-X509Certificate2 -SubjectName 'E=opensource@microsoft.com, CN=Azure SDK, OU=Azure SDK, O=Microsoft, L=Frisco, S=TX, C=US' -ValidDays 3652
# Create new entries in $templateFileParameters
$templateFileParameters['ConfidentialLedgerPrincipalPEM'] = Format-X509Certificate2 -Certificate $cert
$templateFileParameters['ConfidentialLedgerPrincipalPEMPK'] = Format-X509Certificate2 -Type Pkcs8 -Certificate $cert

Snippet from the corresponding test-resources.json.

Note that the values present in $templateFileParameters will map to parameters of the same name.

{
  "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
  "contentVersion": "1.0.0.0",
  "parameters": {
    "_comment": "Other required parameters would go here... (this is not part of the actual test-resources.json)",
    "ConfidentialLedgerPrincipalPEM": {
      "type": "string",
      "metadata": {
        "description": "The certificate to configure as a certBasedSecurityPrincipal."
      }
    },
    "ConfidentialLedgerPrincipalPEMPK": {
      "type": "string",
      "metadata": {
        "description": "The certificate to configure as a certBasedSecurityPrincipal."
      }
    }
  },
}

Cleaning up Resources

By default, resource groups are tagged with a DeleteAfter value and date according to the default or specified value for the -DeleteAfterHours switch. You can use this tag in scheduled jobs to remove older resources based on that date.

If you are not ready for the resources to be deleted, you can update the resource group by running Update-TestResources.ps1:

Update-TestResources.ps1 keyvault

This will extend the expiration time by the default value (e.g. 48 hours) from now.

Alternatively, after running or recording live tests, if you do not plan on further testing you can immediately remove the test resources you created above by running Remove-TestResources.ps1:

Remove-TestResources.ps1 keyvault -Force

If you persisted environment variables, you should also remove those as well.

Passing Additional Arguments

Some test-resources.json templates utilize the AdditionalParameters parameter to control additional resource configuration options. For example:

New-TestResources.ps1 keyvault -AdditionalParameters @{enableHsm = $true}

In CI

Test pipelines should include deploy-test-resources.yml and remove-test-resources.yml like in the following examples:

- template: /eng/common/TestResources/deploy-test-resources.yml
  parameters:
    ServiceDirectory: '${{ parameters.ServiceDirectory }}'

# Run tests

- template: /eng/common/TestResources/remove-test-resources.yml

Be sure to link the Secrets for Resource Provisioner variable group into the test pipeline for these scripts to work.

Documentation

To regenerate documentation for scripts within this directory, you can install platyPS and run it like in the following example:

Install-Module platyPS -Scope CurrentUser -Force
New-MarkdownHelp -Command .\New-TestResources.ps1 -OutputFolder . -Force

After the markdown files are generated, please make sure all "http" URIs use "https".

PowerShell markdown documentation created with platyPS.