Граф коммитов

37 Коммитов

Автор SHA1 Сообщение Дата
Rolf Bjarne Kvinge 15b25359c6 [xharness] Determine additional tests to run based on the modified files in a pull request. (#1166)
Example log output when bumping mono:

> Found 1 modified file(s) in the pull request #1161.
>     external/mono
> Enabled 'mtouch' tests because the modified file 'external/mono' matches prefix 'external/mono'
> Enabled 'mmp' tests because the modified file 'external/mono' matches prefix 'external/mono'
> Enabled 'bcl' tests because the modified file 'external/mono' matches prefix 'external/mono'
> Found 1 label(s) in the pull request #1161: cla-already-signed

Or when changing the static registrar:

> Found 1 modified file(s) in the pull request #1164.
>     tools/common/StaticRegistrar.cs
> Enabled 'mtouch' tests because the modified file 'tools/common/StaticRegistrar.cs' matches prefix 'tools/common'
> Enabled 'mmp' tests because the modified file 'tools/common/StaticRegistrar.cs' matches prefix 'tools/common'
> Found 2 label(s) in the pull request #1164: cla-already-signed, run-mmp-tests
> Enabled 'mmp' tests because the label 'run-mmp-tests' is set.
2016-11-14 10:43:35 +01:00
Rolf Bjarne Kvinge 503061f6ab [tests] Update to NUnit 3 in mtouch tests. (#1119) 2016-11-07 12:22:37 +01:00
Rolf Bjarne Kvinge 3351e2a633 Add a 'repositorypath' to our NuGet.Config and update paths everywhere accordingly. (#1110)
This makes us only put packages in one directory (saves disk space and time),
and it also makes project files in multiple solutions work properly
(mtouch.csproj is in tests/tests.sln and tests/mtouch/mtouch.sln).
2016-11-04 09:13:33 +01:00
Rolf Bjarne Kvinge 4c8867c1b2 [xharness] Give the mtouch tests 120 minutes to finish. (#1095)
Give the mtouch tests 120 minutes to finish, since that's what wrench
currently gives them.
2016-11-03 11:53:49 +01:00
Rolf Bjarne Kvinge f1420510da [xharness] Add support to configure which tests to run on Jenkins by using labels on pull requests. (#1082) 2016-11-02 16:46:32 +01:00
Rolf Bjarne Kvinge 5415c90092 [xharness] Add support for running the mtouch tests on Jenkins. (#1077) 2016-11-01 14:43:37 +01:00
Rolf Bjarne Kvinge e598db9621 [xharness] Add Jenkins support for watchOS Series 2 simulators. (#885) 2016-09-23 09:06:05 -04:00
Rolf Bjarne Kvinge 9000e48034 [tests] Remove Classic test projects, and make the Unified test projects the master projects. (#858) 2016-09-21 22:55:10 +02:00
Rolf Bjarne Kvinge 395a497281 [xharness] Fail properly if we fail to load simulators. (#520) 2016-08-02 18:24:34 +02:00
Rolf Bjarne Kvinge 74312ec77c [xharness] Collect and report crash reports for Xamarin.Mac tests. (#316) 2016-06-29 09:21:03 -07:00
Rolf Bjarne Kvinge 610ef84d37 [xharness] Fix result propagation. (#321) 2016-06-29 07:50:02 -07:00
Rolf Bjarne Kvinge 1080164506 [jenkins] Fail if test run fails. (#245)
* [jenkins] Fail if test run fails.

* [xharness] Fix success check.
2016-06-29 03:23:26 -07:00
Rolf Bjarne Kvinge 0636832673 [xharness] Run watchOS tests on jenkins. (#313) 2016-06-29 01:48:53 -07:00
Rolf Bjarne Kvinge dff594bb5a [xharness] Add the XI MSBuild tests to the jenkins test run. 2016-06-29 00:06:35 +02:00
Rolf Bjarne Kvinge d91258fb32 [xharness] Simplify non-grouped tasks in the html report a little bit. 2016-06-29 00:06:31 +02:00
Rolf Bjarne Kvinge 0d2d6e36b1 [xharness] Don't prepare simulator for tests that didn't build. (#297) 2016-06-28 07:03:40 -07:00
Rolf Bjarne Kvinge 613fb2118c [jenkins] Skip mmptest to see if it'll speed up the test run on jenkins. (#248) 2016-06-28 13:05:00 +02:00
Rolf Bjarne Kvinge 85d94face2 [xharness] Improve logging.
Make sure as much as possible is written to the logs, and not stdout.

Also copy an extra 1024 bytes when fetching the system log (and continue
capturing a while after stopped, since there may be unflushed system log
data in memory).
2016-06-28 13:04:52 +02:00
Rolf Bjarne Kvinge c355cf91be [xharness] Refactor how we do simulator configuration.
Split out the code to prepare the simulator from the AppRunner class,
which is now just handling the logic required for each test run.

This way it's easier to handle simulator preparation for multiple
test runs with the same simulator.

Also revamp logging to avoid printing directly to the console, but
instead use the logging classes that permit redirecting logging
to a file. This makes the html report show better logging.
2016-06-28 13:04:47 +02:00
Rolf Bjarne Kvinge 61485522d4 [xharness] Always setup/cleanup tvOS test runs.
Something strange is going on on the bots where only the first tvOS
test run actually launches the test app successfully.
2016-06-28 13:04:41 +02:00
Rolf Bjarne Kvinge 1e6be7f547 [xharness] Add a few more variables to control what to include in a test run. 2016-06-28 13:02:21 +02:00
Rolf Bjarne Kvinge c98cc1dc19 [xharness] Ignore ignored tests when deciding which color to use for a group of tests. 2016-06-28 13:02:16 +02:00
Rolf Bjarne Kvinge fa33176ba8 [xharness] Cleanup must be done by the last test, not the first test. 2016-06-28 13:02:09 +02:00
Rolf Bjarne Kvinge 93f0b92eac [xharness] iPhone 4s doesn't exist anymore, use iPhone 5 instead. 2016-06-28 13:02:03 +02:00
Rolf Bjarne Kvinge c09c74dba5 [xharness] Improve html report a bit when some tests are ignored. 2016-06-28 12:59:54 +02:00
Rolf Bjarne Kvinge ab7bd56e63 [xharness] Add a few variables to control if to include iOS/Mac tests in jenkins test mode. 2016-06-28 12:59:50 +02:00
Rolf Bjarne Kvinge a7b20c1a43 [xharness] Fix emptyness check. 2016-06-28 12:59:42 +02:00
Rolf Bjarne Kvinge 64bc391669 [xharness] Fix NRE when running from the command line. (#207)
* [xharness] Log simulator loading and handle any exceptions.

* [xharness] Add missing tests.

* [xharness] Throw a helpful exception instead of NRE when a simulator can't be found.

* [xharness] Fix null reference exception.
2016-06-15 18:57:47 -07:00
Rolf Bjarne Kvinge 15a5af3342 [xharness] Exclude bcl tests by default. 2016-06-15 10:51:14 -07:00
Rolf Bjarne Kvinge 126de6836a [xharness] Use full path to mdtool. 2016-06-15 10:51:14 -07:00
Rolf Bjarne Kvinge 0910cacf51 [xharness] Fix processing of ignored tasks. 2016-06-15 10:51:14 -07:00
Rolf Bjarne Kvinge dcc605ed22 [xharness] Build XM/Classic projects using mdtool. 2016-06-15 10:51:14 -07:00
Rolf Bjarne Kvinge af4a93a181 [xharness] Pass the right environment variables when invoking xbuild. 2016-06-15 10:51:14 -07:00
Rolf Bjarne Kvinge ae1bf7fb5f [xharness] Add support for ignoring tests.
And use it to ignore framework-test (never run on wrench) and
watchOS tests (currently ignored on wrench).
2016-06-15 10:51:14 -07:00
Rolf Bjarne Kvinge d9bf0e1728 [xharness] Add support for building and running mac tests on jenkins. 2016-06-15 10:51:14 -07:00
Rolf Bjarne Kvinge 0c1f235800 [xharness] Fix simulator configuration to do all configuration before first launch. 2016-06-15 10:51:14 -07:00
Rolf Bjarne Kvinge d135612f59 [tests] Implement support in xharness for running the simulator tests grouped by simulator, and write out an html report. 2016-06-15 10:51:14 -07:00