Terratest is a Go library that makes it easier to write automated tests for your infrastructure code. It provides a variety of helper functions and patterns for common infrastructure testing tasks.
In addition, the cobalt test suite allows for better collaboration with embedding into CI/CD tools such as Travis or Azure DevOps Pipelines.
This test harness runs automated tests for only the deployment templates that have changed by comparing the changes in your git log versus upstream master.
This module includes a library that simplifies writing unit and integration [Note: integration test support is *pending*] tests against templates. It aims to extract out the most painful pieces of this process and provide common-sense implementations that can be shared across any template. Care is taken to provide hooks for more in-depth testing if it is needed by the template maintainer.
- Run `terraform init`, `terraform workspace select`, `terraform plan` and parse the plan output into a [Terraform Plan](https://github.com/hashicorp/terraform/blob/master/terraform/plan.go)
- Validate that running the test would only create and not update/delete resources. (Note: This should always be true, otherwise the test is not running in isolation. Not running the test in isolation can be very dangerous and may cause resources to be deleted)
- Validate that the resource <--> attribute <--> attribute value mappings match those supplied via the `ExpectedResourceAttributeValues` parameter. This only asserts that the supplied mappings exist and match the terraform plan. If there are more resources or attributes, the test will not fail.
- Validate that the correct number of resources are created
Also note that the harness provides a hook that allows a list of user-defined functions that accept a handle to the GoTest and Terraform Plan objects. Users can supply custom test logic via this hook by supplying a non-nil `PlanAssertions` argument to `infratests.UnitTestFixture`. This feature is not used in the example below.
The below example shows how easy it is to write an integration test that automatically coordinates the following:
- Run `terraform init`, `terraform workspace select`, `terraform apply` and parse the template outputs into a Go struct
- Validate that the terraform outputs are correct by asserting that the correct number exist and that any user-supplied key-value mappings are reflected in that output.
- Pass terraform output to user-defined test functions for use-case specific tests. In this case, we simply validate that the application endpoint responds as expected
The benefit with running the test harness through docker is that developers don't need to worry about setting up their local environment. We strongly recommend running `local-run.sh` before submitting a PR as our devops pipeline runs the dockerized version of the test harness.
- An azure storage account for tracking terraform remote backend state. You can use our backend state setup [template](../infra/templates/backend-state-setup/README.md) to provision the storage resources.
Our test harness uses a base docker image to pre-package dependencies like Terraform, Go, Azure CLI, Terratest vendor packages, etc.
- **Optional Step** - Cobalt uses the public [msftcse/cobalt-test-base](https://hub.docker.com/r/msftcse/cobalt-test-base) base image by default. We also provide a utility script to generate a new base image.
- Rebuilding a new base image is as simple as running
-`-g` | `--go_version`: Golang version specification. This argument drives the version of the `golang` stretch base image. **Defaults** to `1.11`.
-`-t` | `--tf_version`: Terraform version specification. This argument drives which terraform version release this image will use.. **Defaults** to `0.11.13`
-`-t` | `--template_name_override`: The template folder to include for the test harness run(i.e. -t "azure-simple-hw"). When set, the git log will be ignored. **Defaults** to the git log.
-`-b` | `--docker_base_image_name`: The base image to use for the test harness continer. **Defaults** to `msftcse/cobalt-test-base:g${GO_VERSION}t${TF_VERSION}`.
### Option 2: Manual Setup
The benefit with setting up the test harness manually is that runtimes are quicker as we're not rebuilding the test harness image on each run.
The clear downside here is that you'll need to set up all cobalt base software packages and responsible for managing version dependency upgrades over time. Our central base image in docker hub is supported by CSE as well as version dependency upgrades.
The other downside is that you'll need to install this project within your `GOPATH` and pull down all `dep` vendor dependency packages.
#### Prerequisites
- An Azure subscription
- A [service principal](https://docs.microsoft.com/en-us/azure/active-directory/develop/howto-create-service-principal-portal)
- An azure storage account for tracking terraform remote backend state. You can use our backend state setup [template](../infra/templates/backend-state-setup/README.md) to provision the storage resources.
-`-t` | `--template_name_override`: The template folder to include for the test harness run(i.e. -t "azure-simple-hw"). When set, the git log will be ignored. **Defaults** to the git log.
-`-c` | `--tf_state_container`: The storage container name responsible for tracking remote state for terraform deployments. **Defaults** to `cobaltfstate-remote-state-container`
-`-a` | `--tf_state_storage_acct`: The storage account name responsible for tracking remote state for terraform deployments. **Defaults** to `cobaltfstate`.