Legacy tests

This document describes the legacy tests located in the following three folders:

azure-mgmt/tests
azure-servicebus/tests
azure-servicemanagement-legacy/tests

and how to port them to use azure-devtools.

If you haven't read Contributing to the tests, please do so before reading this document. It describes some basic concepts necessary for understanding all SDK tests.

If you're trying to write new tests, disregard this document! Read Contributing to the tests instead.

Legacy test structure

The legacy test folders are all designed the same way:

A testsettings_local.json : Recording configuration file
A mgmt_settings_fake.py : Azure mock configuration file
A mgmt_settings_real.py : Azure real configuration file (not in the repository)
A <libtype>_testcase.py : Common code shared by several tests
Several test_<file>.py : Test files themselves

The testsettings_local.json file is analogous to testsettings_local.cfg: a configuration file which you use to designate in what recording mode the tests should be run.

For legacy tests, this file should (as indicated by the extension) be in JSON format. It should look like the following:

{
    "mode" : "Playback"
}

Valid string values for the "mode" setting are:

Playback : Use the recordings on disk to test (i.e. offline mock mode). Use configuration from mgmt_settings_fake.py.
Record : Connect directly to azure and record HTTP queries/answers. Use configuration from mgmt_settings_real.py.
Live : Connect directly to azure, but don't record HTTP interactions. Use configuration from mgmt_settings_real.py.

The recordings are saved in the recordings subfolder.

The mgmt_settings_real.py and mgmt_settings_fake.py files are used like the ones in azure-sdk-testutils/devtools_testutils and should be set up the same way.

Legacy tests use the @record decorator to indicate that HTTP requests and responses should be recorded to or replayed from a file. Use the decorator like this:

@record
def test_usage(self):
    usages = list(self.compute_client.usage.list(self.region))
    self.assertGreater(len(usages), 0)

Porting legacy tests to use azure-devtools

This section describes porting the legacy tests to use the azure-devtools framework.

Legacy tests and azure-devtools-based tests differ in the following ways:

Tests using azure-devtools are located in the tests dir alongside the modules they test, e.g. azure-mgmt-storage/tests, rather than being lumped together in a single directory.
Tests using azure-devtools do not use the @record decorator. They will have recording functionality by virtue of being subclasses of azure-devtools's ReplayableTest class.
Newer tests use azure-devtools's "preparers" to create auxiliary resources like resource groups and storage rather than doing this kind of work in the setUp method.
testsettings_local.json is replaced by testsettings_local.cfg. Its single live-mode entry must be either true or false: true corresponds to testsettings_local.json's "Record" mode, whereas false tries to run tests in "Playback" mode but falls back to live mode to produce new recordings if the required ones aren't available.

Locations of already-ported tests

There are azure-devtools-compatible tests in at least the following locations:

azure-mgmt-storage
azure-mgmt-media
azure-mgmt-containerregistry

You can use them as a reference to see what azure-devtools-based tests look like. The next section contains more detailed information about what changes may be necessary.

Porting a test

For an example of the changes necessary to port a legacy test to use azure-devtools, see this diff of the test_mgmt_containerregistry.py file. What follows is a list of the notable changes therein.

Add new imports: from devtools_testutils, import the base class AzureMgmtTestCase as well as the preparers for any necessary auxiliary resources needed for the test, such as ResourceGroupPreparer.
(Maybe) Add a fake resource for use in playback mode: when running in live mode, preparers will give you a model object such as StorageAccount representing an Azure resource. In recording mode, you can inject a fake object, with any attributes necessary for the test, to the preparer so that the tests can still run. Some tests may not need this.
Remove auxiliary resource creation from setUp(). It is now handled by azure-devtools's preparers.
Add decorators and arguments for preparers to test methods. Here is an example of a preparer decorator on a method of a test case illustrating most of its features:
```
@ResourceGroupPreparer(parameter_name='rsrc_group',
                       playback_fake_resource=FAKE_RESOURCE_GROUP)
def test_something(self, rsrc_group):
    self.assertEqual(rsrc_group.name, 'test_group')
```
The keyword arguments to this decorator have the following effects:
- parameter_name provides the name of the parameter via which the created resource will be made available to the decorated function. Here, the created resource group will be passed into test_something as the rsrc_group parameter.
- playback_fake_resource allows you to pass in a mock object that will represent the resource in playback mode, as described in item 2 above. Here we pass in a constant named FAKE_RESOURCE_GROUP which has been created to have a name attribute so that the test can run on it.
Update attribute references for auxiliary resources. Since those resources are passed in as parameters by the preparers instead of assigned to attributes of self in setUp(), any references to them need to be changed to reflect that.