bedrock/test

Cluster Validation Testing in Bedrock

Summary

This section describes how to build integration and validation tests for your bedrock cluster environments using the Go testing packages and the terratest modules.

Terratest is a Go library that makes it easier to write automated tests for your infrastructure code. It provides a variety of helper functions and patterns for common infrastructure testing tasks.

In addition, the bedrock test suite allows for better collaboration with embedding into CI/CD tools such as Travis or Azure DevOps Pipelines.

Included is an option to set up your environment using docker container to support bedrock deployments.

Prerequisites

Please install bedrock required tools in addition to the following:

• Golang 1.11 or later
• Dep Optional, but required for now in order for VSCode intellisense and linting to work. See issue 2317.
• An Azure subscription
• A service principal with owner role status
• An azure storage account for tracking terraform remote backend state.
• git

Test Setup Option: Local Machine

In this example we are using the azure-simple for a template integration test.

Setup for linux users

1. Install Go using:

   sudo snap install --classic go

2. Run go get -m github.com/microsoft/bedrock/test and navigate to the bedrock test repository in the /go/src/github.com/microsoft/bedrock/test directory in the $GOPATH

3. Change all instances of the module source in the main.tf file pointing to github.com/Microsoft/bedrock/cluster to be your local development path ../...

   i.e. source = "github.com/Microsoft/bedrock/cluster/azure/aks-gitops" becomes source = "../../azure/aks-gitops"

   Otherwise the tests will go against the code which is in the master branch in the GitHub repository.

4. Provide values for the environment variables and export for authenticating Terraform to provision resources within your subscription.

   export ARM_CLIENT_ID="${clientid}"
   export ARM_CLIENT_SECRET="${clientsecret}"
   export ARM_SUBSCRIPTION_ID="${subscriptionid}"
   export ARM_TENANT_ID="${tenantid}"
   export DATACENTER_LOCATION="${location}"
   export ARM_BACKEND_STORAGE_NAME="${storageaccount}"
   export ARM_BACKEND_STORAGE_KEY="${storagekey}"
   export ARM_BACKEND_STORAGE_CONTAINER="${storagecontainer}"
   export ssh_key=$(readlink -f id_rsa)
   export public_key=$(cat id_rsa.pub)
   

5. Run go test -v -timeout 99999s to execute all tests in your test suite. Or run go test -v -run <You_Test_Name> -timeout 999999s to run a targeted test.

6. (Optional) Install Deps

   Install Dep with: curl https://raw.githubusercontent.com/golang/dep/master/install.sh | sh

   Then install the dependencies to the traditional path via running this from the test directory: dep ensure

Test Setup Option: Docker

The benefit with running the test suite through docker is that developers don't need to worry about setting up their local environment.

Base Image Setup

Our test harness uses a base docker image to pre-package dependencies like Terraform, Go, Azure CLI, Terratest vendor packages, etc.

• Optional Step - Bedrock uses the public msftcse/bedrock-test-base base image by default. We also provide a utility script to generate a new base image.
• Rebuilding a new base image is as simple as running

./test-harness/build-base-image.sh -g "<go_version>" -t "<terraform_version>"

Note: Bedrock currently expects terraform version 0.11.x or newer.

Script Arguments

• -g | --go_version: Golang version specification. This argument drives the version of the golang stretch base image. Defaults to 1.11.
• -t | --tf_version: Terraform version specification. This argument drives which terraform version release this image will use.. Defaults to 0.11.13

Keep in mind that the terraform version should align with the version from the provider module

• The base image will be tagged as:

msftcse/bedrock-test-base:g${GO_VERSION}t${TERRAFORM_VERSION}

Local Run Script

Run the test runner by calling the below script from the project's root directory. This is one of two options.

./test/local-run.sh

Script Arguments

• -t | --template_name_override: The template folder to include for the test harness run(i.e. -t "azure-simple"). When set, the git log will be ignored. Defaults to the git log.
• -b | --docker_base_image_name: The base image to use for the test harness continer. Defaults to msftcse/bedrock-test-base:g${GO_VERSION}t${TF_VERSION}.

Test Setup CI/CD

Linting can be done for code style on terraform and golang files prior to commits using a git hook. Validate your modified changes are formatted prior to commits. A script can also verify that trailing whitespaces are removed. Bedrock confirms that these changes have been made as a step in our CI pipeline. This is done as a step in azure-pipelines.yml.

For test setup using a continuous integration pipeline, refer to the azure-pipelines.yml for how to configure your agent, setup script for installing prerequisites and executing scripts. Be sure to add the environment path your test is leveraging to the CI yml. For example paths/include: section in the azure-pipelines.yml.