This updates the project generation. We cannot yet fully remove the submodule because:
* We are missing the xunit dlls which should be added in the SDK.
* We have not yet remove all the old style tests. Would make the PR huge, better to deal with it in a diff PR.
* The xunit CoreLib tests have issues loading all the tests, needs some extra work and again, the PR is already large.
Fixes: xamarin/maccore#1199Fixes: xamarin/maccore#1204Fixes: xamarin/maccore#1209Fixes: xamarin/maccore#1510
This commit improves the state of the BCL testing in the following ways:
1. Improve the device tets running. Less apps, faster results.
2. WatchOS apps are left as they were to ensure that we do not have deplouyment/run issues.
We now support the ignore files per assembly name to simplify the
tracking of the ignored tests. All
Rework BCL test importation to generate projects that won't compile instead of
giving up completely in case of generation failure.
This means that xharness can still be used for non-BCL tests even if the
generation of the BCL tests fail.
We now download the ios Mono sdk which means that there is no need for
us to build the test assemblies on iOS, WatchOS and tvOS BUT we need to
do so for the Mac tests.
The change includes:
1. Use fo the downloaded test assemblies.
2. Removal of the dependency of building them in the tests.
3. Move back to using reflection for the project generation.
4. Remove the cached project details added in the workaround.
This change adds support to execute the tests provided by mono as assemblies.This includes:
1. App generation to run the bcl tests on modern and full.
2. Needed workaround to delay the compilation of the assemblies until we have them from the SDK.
3. All failing tests are ignored.
* [Xharness] Add a workaround to not build the xunit tests when not needed.
This is a workaround that does not use reflection to build the bcl test
projects. We need this until we have complete support of using mono as
an SDK.
Mono know provides a set of testing assemblies that contain the tests to be ran in each of the supported platforms. This commit adds the following:
1. Two different runners that can be used to execute NUnit and xUnit tests.
2. The necessary glue to get xharness to run the new tests and report the results.
3. A new test that will ensure that if mono adds new tests, xharness will report failures if the tests are not run or not ignored.
Manuel de la Pena
Alexander Köplinger
Rolf Bjarne Kvinge