Граф коммитов

157 Коммитов

Автор SHA1 Сообщение Дата
Rolf Bjarne Kvinge 43e6f82803 [xharness] Look for mlaunch next to mtouch. (#1243) 2016-11-25 12:37:02 +01:00
Rolf Bjarne Kvinge 7535eae204 [xharness] Fix mlaunch download if extraction dir already exists. (#1241) 2016-11-25 11:27:08 +01:00
Rolf Bjarne Kvinge 77d4769f54 [xharness] Make sure DownloadMlaunch isn't run multiple times simultaneously. (#1236) 2016-11-24 15:12:51 +01:00
Rolf Bjarne Kvinge db00a33848 [xharness] Add support for [skip|run]-all-tests labels. (#1208) 2016-11-18 07:11:14 +01:00
Rolf Bjarne Kvinge 66f82d1f94 [jenkins] Add support for enabling device builds using labels. (#1184)
* [jenkins] Add support for enabling device builds using labels.

* [xharness] Give the iOS MSBuild tests 30 minutes to finish.

* [mtouch tests] Give the BuildTestProject 10 minutes to compile each test case.

Wrench bots build the dontlink test in ~3m40, but that's apparently not enough
for the Jenkins bots (slower bots?), which time out the test after 5 minutes.

So double the timeout to 10 minutes, which will hopefully give the Jenkins
bots enough time to run the test to completion.
2016-11-17 10:22:26 +01:00
Rolf Bjarne Kvinge 89757e6f60 [xharness] Capture entire log files on wrench. (#1206) 2016-11-17 08:07:09 +01:00
Rolf Bjarne Kvinge c83fa8a9bb [xharness] Fix target selection for simulators on wrench. (#1204) 2016-11-16 19:06:26 +01:00
Rolf Bjarne Kvinge 15b1204874 [xharness] Unify simulator selection code between Jenkins and Wrench and automatically create device pairs if none applicable is found. (#1202)
* [xharness] Don't crash if we can't find a simulator.

* [xharness] Create a device pair if none applicable is found.

* [xharness] Use an enum instead of string values for the target.

* [xharness] Unify the simulator selection code between Jenkins and Wrench.
2016-11-16 15:23:11 +01:00
Rolf Bjarne Kvinge a2570c6b84 [xharness] Improve cleaning by removing saved simulator state and the CoreSimulatorService. (#1203)
Also do this at the very start, to make sure no stale CoreSimulatorService is
around at build time. This will hopefully fix bug #[46097][1].

[1]: https://bugzilla.xamarin.com/show_bug.cgi?id=46097
2016-11-16 15:21:20 +01:00
Rolf Bjarne Kvinge d646d522f3 [xharness] When finding watchos device types, only find the ones that are in available device pairs. (#1198) 2016-11-15 20:20:36 +01:00
Manuel de la Pena aa38ab68ac [Jenkins] Add nuint xml logs (#883)
* [Jenkins] Make test to write output as an xml file so that it can be parsed by the jenkins bot.

* Point to the correct Touvh.Unit repo.

* Use the available property to determine if we are being ran in Jenkins.

* Log where are test results stored.

* Add @MonkeyWrench: prefix.

* Ensure that we do set the build env in jenkins/run-tests.sh

* Do not mix Wrench with Jenkins. The reports in jenkins can be Xml, in Wrench we prefer the old style.

* Ensure that the main node of the unit tests does contain the target, that will improve the tests results reporting.

* Revert "Fix binding project LinkWithAttributes generation to prevent rebuilds" (#1018)

* Added xslt to be used to keep the old Test Reports until we move to only Jenkins reports.

* Add an extra log for the xslt transformation.

* Point to the correcto dir in jenkins.

* Deal with the xslt once we have finished rather than in a batch.

* Remove noise.

* Readd case removed in rebase.

* Fix indentation.

* Skip lock keychain.
2016-11-15 19:04:37 +01:00
Rolf Bjarne Kvinge 816fca0fa1 [xharness] Show better log names. 2016-11-14 14:47:23 +01:00
Rolf Bjarne Kvinge 15b25359c6 [xharness] Determine additional tests to run based on the modified files in a pull request. (#1166)
Example log output when bumping mono:

> Found 1 modified file(s) in the pull request #1161.
>     external/mono
> Enabled 'mtouch' tests because the modified file 'external/mono' matches prefix 'external/mono'
> Enabled 'mmp' tests because the modified file 'external/mono' matches prefix 'external/mono'
> Enabled 'bcl' tests because the modified file 'external/mono' matches prefix 'external/mono'
> Found 1 label(s) in the pull request #1161: cla-already-signed

Or when changing the static registrar:

> Found 1 modified file(s) in the pull request #1164.
>     tools/common/StaticRegistrar.cs
> Enabled 'mtouch' tests because the modified file 'tools/common/StaticRegistrar.cs' matches prefix 'tools/common'
> Enabled 'mmp' tests because the modified file 'tools/common/StaticRegistrar.cs' matches prefix 'tools/common'
> Found 2 label(s) in the pull request #1164: cla-already-signed, run-mmp-tests
> Enabled 'mmp' tests because the label 'run-mmp-tests' is set.
2016-11-14 10:43:35 +01:00
Rolf Bjarne Kvinge 7bf290278c Bump maccore to get mlaunch changes. (#1143) 2016-11-09 16:14:59 +01:00
Rolf Bjarne Kvinge 503061f6ab [tests] Update to NUnit 3 in mtouch tests. (#1119) 2016-11-07 12:22:37 +01:00
Rolf Bjarne Kvinge 5ef045d764 [xharness] Capture stdout/stderr from simulator apps. (#1116)
And if we have a tty, just show the output directly there. Otherwise capture
it, and store it in log files.
2016-11-04 12:13:20 +01:00
Rolf Bjarne Kvinge 3351e2a633 Add a 'repositorypath' to our NuGet.Config and update paths everywhere accordingly. (#1110)
This makes us only put packages in one directory (saves disk space and time),
and it also makes project files in multiple solutions work properly
(mtouch.csproj is in tests/tests.sln and tests/mtouch/mtouch.sln).
2016-11-04 09:13:33 +01:00
Rolf Bjarne Kvinge 4c8867c1b2 [xharness] Give the mtouch tests 120 minutes to finish. (#1095)
Give the mtouch tests 120 minutes to finish, since that's what wrench
currently gives them.
2016-11-03 11:53:49 +01:00
Rolf Bjarne Kvinge f1420510da [xharness] Add support to configure which tests to run on Jenkins by using labels on pull requests. (#1082) 2016-11-02 16:46:32 +01:00
Rolf Bjarne Kvinge b13bbe0dfb [xharness] Detect and report launch failures better. (#1079) 2016-11-02 15:43:55 +01:00
Rolf Bjarne Kvinge dddd243dfd [xharness] Update to remove OpenTK-1.0 instead of OpenTK references after XI/Classic removal. (#1084) 2016-11-01 13:13:52 -04:00
Rolf Bjarne Kvinge 5415c90092 [xharness] Add support for running the mtouch tests on Jenkins. (#1077) 2016-11-01 14:43:37 +01:00
Sebastien Pouliot 8c93e7758d Merge branch 'xcode8.1' 2016-10-26 15:56:34 -04:00
Sebastien Pouliot ecf90178fa [tests] Update mlaunch executable to work with Xcode 8.1 GM 2016-10-24 18:28:18 -04:00
Rolf Bjarne Kvinge 1b84d7bbd9 [xharness] Add support for specifying 32-bit and 64-bit mode when running on device.
Also add support for running both 32-bit and 64-bit tests on device
with one target.
2016-10-11 19:52:35 +02:00
Rolf Bjarne Kvinge d940b592c8 [xharness] Fetch watch crash reports when running watch apps. (#973) 2016-10-11 19:30:11 +02:00
Sebastien Pouliot 0eff36c466 Bump Xcode 8.1 to beta 2 (#945)
* Bump Xcode 8.1 to beta 2
* Bump maccore to get simulator support fix
* [xharness] Get updated mlaunch binary. (#3)
2016-10-05 13:28:36 -04:00
Rolf Bjarne Kvinge 876698ce25 [xharness] Define FEATURE_NO_BSD_SOCKETS for watchOS test projects. (#909) 2016-09-28 11:26:36 -04:00
Sebastien Pouliot e4ce16e989 Update branch name (#882)
* Update branch name

* Update SDK version for iOS and watchOS (but not tvOS)

* [xharness] Add support for watchOS Series 2 simulators. (#812)

* [xharness] Add Jenkins support for watchOS Series 2 simulators. (#885)
2016-09-23 11:50:02 -04:00
Rolf Bjarne Kvinge e598db9621 [xharness] Add Jenkins support for watchOS Series 2 simulators. (#885) 2016-09-23 09:06:05 -04:00
Rolf Bjarne Kvinge 9000e48034 [tests] Remove Classic test projects, and make the Unified test projects the master projects. (#858) 2016-09-21 22:55:10 +02:00
Rolf Bjarne Kvinge 5408c056d0 [xharness] Remove Classic logic from makefile generation. (#823) 2016-09-13 10:04:24 -04:00
Rolf Bjarne Kvinge 16e36ce163 [xharness] Add support for watchOS Series 2 simulators. (#812) 2016-09-12 10:14:55 -04:00
Sebastien Pouliot 38148c718a Merge branch 'xcode8' 2016-09-09 15:29:37 -04:00
Rolf Bjarne Kvinge cbfd83be92 [xharness] Make sure a project reference's Name matches the file name of the reference. Fixes #43181. (#709)
Apparently XS can't find referenced project (to build them) otherwise.

It works fine when building using xbuild though.

https://bugzilla.xamarin.com/show_bug.cgi?id=43181
2016-08-30 13:32:32 -04:00
Rolf Bjarne Kvinge d9f8461cc2 [xharness] Make sure a project reference's Name matches the file name of the reference. Fixes #43181. (#709)
Apparently XS can't find referenced project (to build them) otherwise.

It works fine when building using xbuild though.

https://bugzilla.xamarin.com/show_bug.cgi?id=43181
2016-08-30 13:31:40 -04:00
Rolf Bjarne Kvinge a617bee08f [xharness] Crashing is not OK. (#680) 2016-08-26 09:12:44 -04:00
Sebastien Pouliot 144d96999b Merge branch 'cycle8' into xcode8 2016-08-25 14:08:23 -04:00
Rolf Bjarne Kvinge 1d4019e966 [xharness] Don't run into an infinite loop if failing to run sqlite3. (#666) 2016-08-25 11:08:40 -04:00
Rolf Bjarne Kvinge 451781a98f [xharness] Don't run classic tests on wrench. (#661) 2016-08-25 13:11:51 +02:00
Sebastien Pouliot 9fe13aaa58 Bump for Xcode8 beta 6 (#610)
* Bump for Xcode8 beta 6

* Bump maccore (to bump maciostools) and update mlaunch download location to run tests on the simulator
2016-08-15 20:19:51 -04:00
Sebastien Pouliot ff77f85a4c Merge branch 'cycle8' into xcode8 2016-08-08 15:29:36 -04:00
Rolf Bjarne Kvinge 812051ee18 [xharness] Don't generate makefile targets for non-executable projects. (#567) 2016-08-06 10:58:03 -04:00
Rolf Bjarne Kvinge 56fc26b26f [xharness] Split the mscorlib/System test run for watchOS into multiple executions. (#564)
* [tests] Implement filtering based on test name to make it possible to run a subset of tests manually.

* [xharness] Add support for passing environment variables to test apps.

* [xharness] Split the mscorlib/System test run for watchOS into multiple executions.

When running on a watch, the complete set of mscorlib and System tests use too
much memory, so they end up crashing the app.

So instead don't run all the tests in the same test run, but
instead split them up.

This splits the System tests in 5 different sets, based on the first
character of the test class name.

The mscorlib tests are split in 26 different sets, one for each
character (A-Z).
2016-08-05 22:28:13 +02:00
Rolf Bjarne Kvinge d22c7adcda [tests] Split mscorlib into two test assemblies. Fixes #41746. (#562)
On watchOS we can't include all the tests for device in a single
assembly, because the native object code becomes too big.

So instead split the tests into two assemblies. Luckily this
is fairly easy for test projects, since they have few dependencies
between source files (there are some however, so source files
are grouped per directory so that files in the same directory do
not end up in different test projects). There is also some hard-coded
logic.

https://bugzilla.xamarin.com/show_bug.cgi?id=41746
2016-08-05 21:04:46 +02:00
Rolf Bjarne Kvinge 9f3d438169 [xharness] Attach a native debugger when running watchOS tests on device. (#560)
Attaching a native debugger prevents the watch from backgrounding
the app, and makes the test run able to complete without having
to babysit it.
2016-08-05 21:03:56 +02:00
Rolf Bjarne Kvinge 4f3af5eefb [xharness] Simplify BCL project generation by merging iOS and tvOS logic. (#558) 2016-08-05 19:45:59 +02:00
Rolf Bjarne Kvinge 916838c7e3 [tests] Split mscorlib into two test assemblies. Fixes #41746. (#544)
On watchOS we can't include all the tests for device in a single
assembly, because the native object code becomes too big.

So instead split the tests into two assemblies. Luckily this
is fairly easy for test projects, since they have few dependencies
between source files (there are some however, so source files
are grouped per directory so that files in the same directory do
not end up in different test projects). There is also some hard-coded
logic.

https://bugzilla.xamarin.com/show_bug.cgi?id=41746
2016-08-04 15:47:05 +02:00
Sebastien Pouliot c99bca8695 Merge pull request #518 from spouliot/beta4-bump
Bump Xcode to beta 4 to run tests against the newest toolchain and SDKs
2016-08-02 13:59:24 -04:00
Rolf Bjarne Kvinge 395a497281 [xharness] Fail properly if we fail to load simulators. (#520) 2016-08-02 18:24:34 +02:00