In some complex combination of test cases, which needs new_env and early
deployment phase, they may be put into the same environments, but
returned later. The extra environments will be deleted in the period,
so the test results cannot be run. With this change, the not deployed
environments won't be deleted until the whole runner completed.