As with other tests, this have been added because they were present when
we built the test assemblies locally, but they are not present in the
Mono SDK.
Fixes https://github.com/xamarin/maccore/issues/1132
As with other tests, this assembly was added when we built the tests in
the iOS makefile, but they are not present in the Mono SDK.
Fixes https://github.com/xamarin/maccore/issues/1140
This PR adds support to the use of *.ignore files for iOS/tvOS/WatchOS. The files are very similar to those that are use in xtro. We have:
* common-{TestProjectName}.ignore: For those tests that fail in all platforms.
* tvOS-{TestProjectName}.ignore: For tvOS specific failures.
* watchOS-{TestProjectName}.ignore: For watchOS specific failures.
* iOS-{TestProjectName}.ignore: for iOS specific projects.
Two test projects that were ignored previously have been added to show that the code does as advertised:
* SystemDataXunit: xUnit test project example. Fixes https://github.com/xamarin/maccore/issues/1131
* SystemServiceModelWebTests: NUnit test project example. Fixes https://github.com/xamarin/maccore/issues/1137
The files take the name of the failing tests, although we do support ignoring by class/namespace and other combinations, that would make the *.ignore files format more complicated and it is not worth it.
We now download the ios Mono sdk which means that there is no need for
us to build the test assemblies on iOS, WatchOS and tvOS BUT we need to
do so for the Mac tests.
The change includes:
1. Use fo the downloaded test assemblies.
2. Removal of the dependency of building them in the tests.
3. Move back to using reflection for the project generation.
4. Remove the cached project details added in the workaround.
This change adds support to execute the tests provided by mono as assemblies.This includes:
1. App generation to run the bcl tests on modern and full.
2. Needed workaround to delay the compilation of the assemblies until we have them from the SDK.
3. All failing tests are ignored.
* [Xharness] Add a workaround to not build the xunit tests when not needed.
This is a workaround that does not use reflection to build the bcl test
projects. We need this until we have complete support of using mono as
an SDK.
Only build the BCL tests when we need them, because we can't build them when
we're running the device tests (because we don't have a built source tree in
that case).
Since the test assemblies are required when generating the BCL projects, this
means we also delay creating the BCL projects until we need them.
Mono know provides a set of testing assemblies that contain the tests to be ran in each of the supported platforms. This commit adds the following:
1. Two different runners that can be used to execute NUnit and xUnit tests.
2. The necessary glue to get xharness to run the new tests and report the results.
3. A new test that will ensure that if mono adds new tests, xharness will report failures if the tests are not run or not ignored.
Manuel de la Pena
Rolf Bjarne Kvinge