Содержание

Unit testing with Polly - with examples

[1] I want to unit-test what my code does, independent of Polly policies I apply

Use DI to separate policy creation from policy usage

Using a mock to verify that the system-under-test uses the policy

Use PolicyRegistry with DI, to work with policy collections

Stubbing Polly out of tests with PolicyRegistry
Using mock policies in tests, with PolicyRegistry

Use a policy factory, if you generate policies dynamically

[2] I want to test that I've configured Polly policies to achieve my desired resilience strategy

[2a] Test policies with integration tests through the consuming code
[2b] Test policies with unit-tests, independent of consuming code

Recommendation: Focus on testing your own logic rather than Polly

[3] I want to unit-test how my code reacts to results or faults returned by the execution through Polly
[4] I want to unit-test that the Polly policies I apply actually do what they say on the tin

Questions?

Этот файл содержит невидимые символы Юникода, которые могут быть отображены не так, как показано ниже. Если это намеренно, можете спокойно проигнорировать это предупреждение. Используйте кнопку Экранировать, чтобы показать скрытые символы.

Этот файл содержит неоднозначные символы Юникода, которые могут быть перепутаны с другими в текущей локали. Если это намеренно, можете спокойно проигнорировать это предупреждение. Используйте кнопку Экранировать, чтобы подсветить эти символы.

Unit testing with Polly - with examples

ℹ️ This documentation describes the previous Polly v7 API. If you are using the new v8 API, please refer to pollydocs.org.

How to approach unit-testing code wrapped in Polly policies depends what you are aiming to test.

This page provides a longer version of our main unit-testing page, adding code examples.

[1] I want to unit-test what my code does, independent of Polly policies I apply

TL:DR; Polly's NoOpPolicy allows you to stub out Polly, to test your code as if Polly were not in the mix. Use DI to provide policies to consuming classes; tests can then stub out Polly by injecting NoOpPolicy in place of real policies.

A common need is to test the logic of your system-under-test as if Polly were not part of the mix.

Perhaps you have code modules for which you already had unit tests, including success and failure cases. You then retro-fit Polly for resilience. How does having the Polly policy in play affect your existing unit tests? Do all the tests need adjusting? How do I test what my code does without Polly 'interfering'?

Equally, when you include Polly from the outset of a project, it can still make sense to test concerns separately - have one set of tests which assert on behaviour without Polly; and other tests which test the additional resilience, separately.

Use DI to separate policy creation from policy usage

A simple strategy uses dependency injection, so that creating the policy is outside the system-under-test:

public class Repository<T> : IRepository<T> {
    public Repository(IAsyncPolicy policy, /* other constructor params */)
    {
        this.resiliencePolicy = policy;
    }

    public async Task<T> GetById(string id)
    {
        return await resiliencePolicy.ExecuteAsync(() => /* code for getting from the underlying system */);
    }

    // ...
}

For tests where you want to remove Polly from the mix, you can then inject NoOpPolicy rather than the policies you use in production.

var repoToTestWithNoPolly = new Repository<Foo>(Policy.NoOpAsync(), ...);

When code is executed through NoOpPolicy, it does nothing but execute the code as if Polly was not involved. Polly is effectively stubbed out of the test.

The example is a repository class, but the pattern can be used anywhere you use Polly: when guarding calls to third-party APIs; to cloud components through cloud-provider SDKs; etc.

Using a mock to verify that the system-under-test uses the policy

You can also construct tests using mocking tools like Moq or NSubstitute to inject eg a Mock<IAsyncPolicy>:

var mockPolicy = new Mock<IAsyncPolicy>();
var repoToTestWithMockPolicy = new Repository<Foo>(mockPolicy.Object, ...);

This allows you to write tests confirming that the system-under-test does use the passed-in policy:

var resultNotOfInterest = repoToTestWithMockPolicy.GetById("someId");
mockPolicy.Verify(p => p.ExecuteAsync(It.IsAny<Func<Task<Foo>>>()));

Use PolicyRegistry with DI, to work with policy collections

The above example was intentionally the simplest possible to demonstrate providing policies by DI, and using NoOpPolicy or mocks to stub out Polly. However, injecting IAsyncPolicy policy directly is too simplistic for many scenarios.

If IAsyncPolicy policy will be resolved from a DI container, this can imply you only have one registration of IAsyncPolicy for the whole application. (This would be the case for example with .NET Core's in-built DI.) For real-world applications using Polly in multiple areas, this is unrealistic - you typically need different resilience strategies for multiple external systems.

PolicyRegistry is designed to help you manage a collection of policies in your application, while still playing nicely with DI.

To consume policies by DI with PolicyRegistry, make a PolicyRegistry instance the item injected into the consuming classes. From that policy registry instance, retrieve the relevant policy/ies to use:

public class Repository<T> : IRepository<T> {
    public Repository(IReadOnlyPolicyRegistry<string> policyRegistry, /* other constructor params */)
    {
        this.resiliencePolicy = policyRegistry.Get<IAsyncPolicy>("MyRepositoryPolicy"); // As an example - of course you can use consts rather than magic strings.
    }

    public async Task<T> GetById(string id)
    {
        return await resiliencePolicy.ExecuteAsync(() => /* code for getting from the underlying system */);
    }
}

At the point of use, we only want to read policies from the registry, so we can inject IReadOnlyPolicyRegistry<string>.

Stubbing Polly out of tests with PolicyRegistry

With this pattern, tests can stub Polly out of the picture by configuring the policy registry to return NoOpPolicy policies:

var registryReturningNoOpPolicy = new PolicyRegistry {
    { "MyRepositoryPolicy", Policy.NoOpAsync(); }
};
var repoToTestWithNoPolly = new Repository<T>(registryReturningNoOpPolicy, ...);

As an alternative route to achieving the same end as the above snippet, you could use a mocking tool like Moq and inject a Mock<IReadOnlyPolicyRegistry<string>> configured to return Policy.NoOpAsync() for the policy to use.

Using mock policies in tests, with PolicyRegistry

Finally, with the PolicyRegistry case, you can also configure a registry to return mock policies:

var mockPolicy = new Mock<IAsyncPolicy>();
var registryReturningMockPolicy = new PolicyRegistry {
    { "MyRepositoryPolicy", mockPolicy.Object; }
};
var repoWithMockPolicy = new Repository<T>(registryReturningMockPolicy, ...);

Use a policy factory, if you generate policies dynamically

Some applications have a requirement to update policies dynamically at run-time, for example to respond to changes in external configuration. One approach to this is to have changes in configuration push updates to a PolicyRegistry. If you use that approach, testing patterns for the code consuming policies would follow the strategies with PolicyRegistry above.

You may alternatively have code which generates policies dynamically each time they are needed, at run-time. If taking this approach, we recommend extracting the policy-generation logic as a policy factory:

interface IPolicyFactory {
    IAsyncPolicy CreatePolicy(); // (example overload - you may have a wider range)
}

with consuming classes taking the IPolicyFactory by DI:

public class Repository<T> : IRepository<T> {
    public Repository(IPolicyFactory policyFactory, /* other constructor params */)
    {
        this.resiliencePolicy = policyFactory.CreatePolicy();
    }

    // ...
}

With this pattern, all the strategies previously described for PolicyRegistry can be applied in tests:

Tests can create a NoOpPolicyFactory : IPolicyFactory or Mock<IPolicyFactory> whose CreatePolicy() method always returns a NoOpPolicy - allowing you to stub Polly out of tests.
Tests can create a policy factory configured to return instances of Mock<IAsyncPolicy>, allowing you to pass mock policies into the system-under-test.

[2] I want to test that I've configured Polly policies to achieve my desired resilience strategy

TL;DR: Configure a mock of the underlying system to return faults the policies should handle. Can be useful as a specification for, and regression check on, the faults you intend to handle.

With policy creation separated from policy consumption as described above, you can:

[2a] test policies with integration tests through the consuming code; or
[2b] test policies with unit-tests, independent of the consuming code.

[2a] Test policies with integration tests through the consuming code

Let's expand our earlier respository example, imagining we have some underlying database, IUnderlyingDatabase db:

public class Repository<T> : IRepository<T> {
    private IAsyncPolicy resiliencePolicy;
    private IUnderlyingDatabase db;

    public Repository(IReadOnlyPolicyRegistry<string> policyRegistry, IUnderlyingDatabase db)
    {
        this.resiliencePolicy = policyRegistry.Get<IAsyncPolicy>("MyRepositoryPolicy");
        this.db = db;
    }

    // (simple example - add CancellationToken support where available, in real code)
    public async Task<T> GetById(string id);
    {
        return await resiliencePolicy.ExecuteAsync(() => db.GetByIdAsync(id));
    }
}

To focus on structuring test code, we'll take a simple example, that the resilience policy just retries once for TimeoutException:

public PolicyRegistry<string> ConfigureResiliencePolicies()
{
    return new PolicyRegistry {
        { "MyRepositoryPolicy", Policy.Handle<TimeoutException>().Retry(1); },
        // ... and other policies for elsewhere round the app
    };
}

In our integration test, we then use a mock of the underlying system (IUnderlyingDb) to throw particular faults or a certain fault sequence, and check that our policy handles them:

public async Task When_db_get_throws_single_TimeoutException_Then_policy_handles_exception()
{
    // Arrange - registry
    var registry = ConfigureResiliencePolicies(); // example kept simple; realistically, this method probably lives on a configuration or startup class in your app

    // Arrange - mock database to throw one TimeoutException
    Mock<IUnderlyingDb> dbMock = new Mock<IUnderlyingDb>();
    int invocations = 0;
    Foo someFoo = new Foo();
    dbMock.Setup(db => db.GetById<Foo>(It.IsAny<string>()))
        .Returns(invocations++ == 0 ? throw new TimeoutException() : someFoo);

    // Arrange - repository
    var repo = new Repository(registry, dbMock.Object, ...);

    // Act
    var returned = await repo.GetById("someId");

    // Assert.
    Assert.AreSame(someFoo, returned);
}

This style of test can be useful if you want tests that the policies you have configured do handle particular faults in a certain way. The tests can also have regression value: if somebody changes the resilience configuration, the test fails.

[2b] Test policies with unit-tests, independent of consuming code

With policy creation separated from policy consumption (as described in section [1] above), you can alternatively unit-test how your policies respond to faults, independent of the consuming system.

public async Task MyRepositoryPolicy_handles_single_TimeoutException()
{
    // Arrange - registry
    var registry = ConfigureResiliencePolicies();
    var policy = registry.Get<IAsyncPolicy>("MyRepositoryPolicy");

    // Act
    int invocations = 0;
    var result = await policy
        .ExecuteAsync<bool>(invocations++ == 0 ? throw new TimeoutException() : Task.FromResult(true));

    // Assert.
    Assert.IsTrue(result);
}

Recommendation: Focus on testing your own logic rather than Polly

Note: With either [2a] or [2b], it is easy to stray towards recreating Polly's own test suite (see point [4] below]). For example, tests such as "If I configure 1 retry it does 1; if I configure 2 retries it does 2", or "if I attach an onRetryAsync delegate to the policy, it does get invoked" would both duplicate tests Polly already covers. More useful are tests which verify the logic specific to your app ("if a retry is invoked, this gets logged to my logger").

[3] I want to unit-test how my code reacts to results or faults returned by the execution through Polly

TL;DR Mock your policies to return or throw particular outcomes, to test how your code responds.

You may want to test how your code reacts to results or faults returned by an execution through Polly. For instance, you may want to test how your code reacts if, despite resilience strategies, the execution eventually fails.

Polly policies all fulfil execution interfaces (ISyncPolicy, ISyncPolicy<TResult>, IAsyncPolicy and IAsyncPolicy<TResult>). These interfaces describe the .Execute/Async() overloads available on policies.

You can configure these methods on a mock policy, to return or throw faults you want to simulate. Let's expand the Repository<T> example:

public class Repository<T> : IRepository<T> {
    /* elided - private variables to store params */

    public Repository(IReadOnlyPolicyRegistry<string> policyRegistry,
        IUnderlyingDatabase db,
        ILogger<IRepository<T>> logger)
    {
        /* elided - assign params to private variables */
    }

    public async Task<T> GetById(string id);
    {
        try {
            return await resiliencePolicy.ExecuteAsync(() => db.GetByIdAsync(id));
        }
        catch (Exception ex)
        {
            logger.logError(ex, /* etc */);
            throw;
        }
    }
}

We can then write a test:

public async Task When_execution_eventually_fails_Then_logs()
{
    // Arrange
    var toThrow = new BrokenCircuitException();
    var mockPolicy = new Mock<IAsyncPolicy>();
    mockPolicy.Setup(p => p.ExecuteAsync(It.IsAny<Func<Task<Foo>>>())
        .Throws(toThrow);
    var registryReturningMockPolicy = new PolicyRegistry {
        { "MyRepositoryPolicy", mockPolicy.Object; }
    };

    var mockLogger = new Mock<ILogger<IRepository<Foo>>>();

    var repo = new Repository<Foo>(registryReturningMockPolicy, new Mock<IUnderlyingDb>().Object, mockLogger);

    // Act
    await Assert.ThrowsAsync<BrokenCircuitException>(() => repo.GetById("someId"));

    // Assert
    mockLogger.Verify(l => l.LogError(toThrow, /* etc */));
}

If using the policy.ExecuteAndCapture/Async(...) methods, the return types PolicyResult and PolicyResult<TResult> have public factory methods, allowing you to mock .ExecuteAndCapture(...) overloads to return the PolicyResult of your choice.

[4] I want to unit-test that the Polly policies I apply actually do what they say on the tin

TL:DR; Bear in mind the Polly codebase already tests this for you extensively.

An understandable desire when introducing Polly to a project is to want to check the Polly policy does what it says on the tin. If I configure Policy.Handle<FooException>().Retry(3), it would be nice to check it really works, right?

Writing unit-tests to verify that Polly works can be a very valuable way to explore and understand what Polly does. But, to allow you to concentrate on delivering your business value rather than reinventing Polly's test wheel, keep in mind that the Polly codebase tests its own operation extensively. (As at Polly v6.0, the Polly codebase has around 1700 tests per target framework.)

You can also explore and run the Polly-samples to see policies working individually, and in combination.

If you write your own integration tests around policies in your project, note the possibility to manipulate Polly's abstracted SystemClock. Where a test would usually incur a delay (for example, waiting the time for a circuit-breaker to transition from open to half-open state), manipulating the abstracted clock can avoid real-time delays. For insight into how to do this, pull down the codebase and check out how Polly's own unit tests manipulate the clock.

Questions?

Thoughts/questions about unit-testing? Post an issue on the issues board.