This PR adds support to the use of *.ignore files for iOS/tvOS/WatchOS. The files are very similar to those that are use in xtro. We have:
* common-{TestProjectName}.ignore: For those tests that fail in all platforms.
* tvOS-{TestProjectName}.ignore: For tvOS specific failures.
* watchOS-{TestProjectName}.ignore: For watchOS specific failures.
* iOS-{TestProjectName}.ignore: for iOS specific projects.
Two test projects that were ignored previously have been added to show that the code does as advertised:
* SystemDataXunit: xUnit test project example. Fixes https://github.com/xamarin/maccore/issues/1131
* SystemServiceModelWebTests: NUnit test project example. Fixes https://github.com/xamarin/maccore/issues/1137
The files take the name of the failing tests, although we do support ignoring by class/namespace and other combinations, that would make the *.ignore files format more complicated and it is not worth it.
First step to get to the point in which we can filter the execution of
the bcl tests via filters. This commit adds the filtering support to the
runners, which later will be used by xharness to pass those tests
filtered.
The end goal is to not skip a complete assembly but just those failing
tests.
This change adds support to execute the tests provided by mono as assemblies.This includes:
1. App generation to run the bcl tests on modern and full.
2. Needed workaround to delay the compilation of the assemblies until we have them from the SDK.
3. All failing tests are ignored.
Mono know provides a set of testing assemblies that contain the tests to be ran in each of the supported platforms. This commit adds the following:
1. Two different runners that can be used to execute NUnit and xUnit tests.
2. The necessary glue to get xharness to run the new tests and report the results.
3. A new test that will ensure that if mono adds new tests, xharness will report failures if the tests are not run or not ignored.
Manuel de la Pena