[tests] Ensure that the emulator is closed (#309)

A "funny thing" happened while testing [PR #308][0]: the PR builder:

1. Launched the emulator (yay!)
2. Deployed and ran the unit tests (yay!)
3. The unit test execution failed (...eh?)
4. The `<RunInstrumentationTests/>` task failed (...)
5. The `make run-nunit-tests` target exited, and the emulator was
    never exited (!).

Which means on the [*following* PR build][2], a new emulator wasn't
launched. Instead, the emulator instance from build 296 was used.

On the one hand, this is kinda interesting, in that the 297 build
didn't need to create a new emulator and wait for it to boot.

On the other hand, this makes for less reproducible builds: if
"something" is installed on the emulator instance which interferes
with test execution, it'll be ~impossible to diagnose/debug it.

What we (think we?) want is a reproducible, *consistent*, test
environment. The best way to do this is by using *new* emulator
instances on every test execution.

To consistently do *that*, update the `<RunInstrumentationTests/>`
task execution to set `ContinueOnError="True"`, so that errors are
ignored from the `RunUnitTestApks` target. This will allow execution
to continue to the `ReleaseAndroidTarget` target, shutting down the
emulator instance.

[0]: https://github.com/xamarin/xamarin-android/pull/308
[1]: https://jenkins.mono-project.com/job/xamarin-android-pr-builder/296/consoleText
[2]: https://jenkins.mono-project.com/job/xamarin-android-pr-builder/297/consoleText
This commit is contained in:
Jonathan Pryor 2016-11-22 12:52:28 -05:00 коммит произвёл Dean Ellis
Родитель a5946eb552
Коммит 6358a64327
1 изменённых файлов: 1 добавлений и 0 удалений

Просмотреть файл

@ -87,6 +87,7 @@
<Target Name="RunUnitTestApks"
Condition=" '@(UnitTestApk)' != '' ">
<RunInstrumentationTests
ContinueOnError="True"
AdbTarget="$(_AdbTarget)"
AdbOptions="$(AdbOptions)"
Component="%(UnitTestApk.Package)/%(UnitTestApk.InstrumentationType)"