Mono know provides a set of testing assemblies that contain the tests to be ran in each of the supported platforms. This commit adds the following:
1. Two different runners that can be used to execute NUnit and xUnit tests.
2. The necessary glue to get xharness to run the new tests and report the results.
3. A new test that will ensure that if mono adds new tests, xharness will report failures if the tests are not run or not ignored.
Improve test reporting using xml by adding custom information printed during
the test run into the xml, and then extract this information when reporting
test results in xharness (which is much easier than re-constructing the
information from the xml, which caused information to get lost since not
everything piece of interesting is in the xml).
https://bugzilla.xamarin.com/show_bug.cgi?id=51661
* Bump to Xcode 9 beta 2
* [CoreImage] Stub out new CIImage filters in Xcode 9 beta 2.
* [monotouch-test] Update tests according to Xcode 9 beta 2.
* [xharness] Ask watch simulator tests to write output directly to a file.
It seems the watch simulator in Xcode 9 beta 2 has completely lost network
access (everything times out), so instead ask the tests to store results
directly in the resulting file.
This works for now since the watch simulator does not enforce the sandbox.
* [xharness] Rewrite the logic to write unit output to a file so that it works with Jenkins.
For Jenkins we ask the unit tests to produce XML, and this needs some special
treatment.
* [xharness] Protect against exceptions in the listener thread, so that it doesn't take xharness down.
* [xharness] Protect against a potential NullReferenceException, and fix one case where it would occur.
* [xharness] A few logging fixes.
* [Intents] Fix API & tests according to beta 2 changes.
* Add a server mode, which launches a web server (and a web page) that can be
used to interactively run tests and view their results.
* Add support for running test assemblies in a today extension (generating a
new set of projects, similar to how we generate tvOS/watchOS projects based
on the iOS project, we now generate a today extension project in addition to
the tvOS and watchOS projects).
* Load all the different tests (and show them in the html report, although
they show up as 'ignored'), even for disabled/ignored tests. This makes
disabled/ignored tests more visible, and also makes it possible to actually
run them using the embedded web server.
* Add support for running tests on device. Tests will be executed on multiple
devices simulatenously (any connected devices will be used).
* [xharness] Don't crash if we can't find a simulator.
* [xharness] Create a device pair if none applicable is found.
* [xharness] Use an enum instead of string values for the target.
* [xharness] Unify the simulator selection code between Jenkins and Wrench.
* [Jenkins] Make test to write output as an xml file so that it can be parsed by the jenkins bot.
* Point to the correct Touvh.Unit repo.
* Use the available property to determine if we are being ran in Jenkins.
* Log where are test results stored.
* Add @MonkeyWrench: prefix.
* Ensure that we do set the build env in jenkins/run-tests.sh
* Do not mix Wrench with Jenkins. The reports in jenkins can be Xml, in Wrench we prefer the old style.
* Ensure that the main node of the unit tests does contain the target, that will improve the tests results reporting.
* Revert "Fix binding project LinkWithAttributes generation to prevent rebuilds" (#1018)
* Added xslt to be used to keep the old Test Reports until we move to only Jenkins reports.
* Add an extra log for the xslt transformation.
* Point to the correcto dir in jenkins.
* Deal with the xslt once we have finished rather than in a batch.
* Remove noise.
* Readd case removed in rebase.
* Fix indentation.
* Skip lock keychain.
Split out the code to prepare the simulator from the AppRunner class,
which is now just handling the logic required for each test run.
This way it's easier to handle simulator preparation for multiple
test runs with the same simulator.
Also revamp logging to avoid printing directly to the console, but
instead use the logging classes that permit redirecting logging
to a file. This makes the html report show better logging.
Manuel de la Pena
Vincent Dondain
Rolf Bjarne Kvinge
Chris Hamons
Sebastien Pouliot
Vincent Dondain
Timothy Risi
Manuel de la Pena