40137dcfb6
[xharness] Don't use variables (isSimulator) before they've been initialized. ( #433 )
...
Fixes an issue where we would try to fetch crash logs from device when
running in the simulator.
2016-07-19 14:16:02 -04:00
ea9cfe533c
[xharness] Don't use variables (isSimulator) before they've been initialized. ( #397 )
...
Fixes an issue where we would try to fetch crash logs from device when
running in the simulator.
2016-07-14 15:31:55 -04:00
a429d2981c
[xharness] Add support for re-reading console output.
...
Fixes a confusing exception/error message when a test run times out.
2016-07-14 19:23:02 +02:00
2be5478669
[xharness] Mark timed out runs as timed out.
2016-07-14 19:23:02 +02:00
cf27c2e7a8
Merge branch 'cycle8' into xcode8
2016-07-13 16:43:00 -04:00
8af1e47588
[tests] Change location to download mlaunch, old one is not available anymore ( #352 )
2016-07-06 19:40:27 -04:00
88bccb268d
[tests] Change location to download mlaunch, old one is not available anymore ( #352 )
2016-07-06 17:04:08 -04:00
cf2d43e85c
[xharness] Create a file if it doesn't exist when creating a writer for a log file. ( #332 )
2016-06-30 02:39:33 -07:00
74312ec77c
[xharness] Collect and report crash reports for Xamarin.Mac tests. ( #316 )
2016-06-29 09:21:03 -07:00
610ef84d37
[xharness] Fix result propagation. ( #321 )
2016-06-29 07:50:02 -07:00
1080164506
[jenkins] Fail if test run fails. ( #245 )
...
* [jenkins] Fail if test run fails.
* [xharness] Fix success check.
2016-06-29 03:23:26 -07:00
0636832673
[xharness] Run watchOS tests on jenkins. ( #313 )
2016-06-29 01:48:53 -07:00
7e773080a9
Merge remote-tracking branch 'origin/master' into xcode8
2016-06-29 10:23:03 +02:00
dff594bb5a
[xharness] Add the XI MSBuild tests to the jenkins test run.
2016-06-29 00:06:35 +02:00
d91258fb32
[xharness] Simplify non-grouped tasks in the html report a little bit.
2016-06-29 00:06:31 +02:00
38af4434e0
Merge pull request #256 from rolfbjarne/jenkins-ios-msbuild
...
[xharness] Add the XI MSBuild tests to the jenkins test run.
2016-06-28 15:05:25 -07:00
fb607bef03
[xharness] Rename variable to fix compiler warning. ( #308 )
...
Fixes this warning:
SimpleHttpListener.cs(12,8): warning CS0108: `xharness.SimpleHttpListener.connected' hides inherited member `xharness.SimpleListener.connected'. Use the new keyword if hiding was intended
2016-06-28 17:13:36 -04:00
cc8a1bd07b
[xharness] Fix boolean logic for skipping simulator setup. ( #305 )
2016-06-28 13:34:29 -04:00
d32b7746c5
[xharness] Protect against duplicated connected/finished notifications. ( #301 )
2016-06-28 08:20:07 -07:00
63c6ae06dd
[xharness] Add the XI MSBuild tests to the jenkins test run.
2016-06-28 16:05:48 +02:00
c4591f44d2
[xharness] Simplify non-grouped tasks in the html report a little bit.
2016-06-28 16:05:27 +02:00
3bf7b92d88
[xharness] Give permissions to watchkit apps as well. ( #296 )
...
* [xharness] Only execute sqlite3 once.
* [xharness] Give permissions to watchkit apps as well.
2016-06-28 07:04:10 -07:00
0d2d6e36b1
[xharness] Don't prepare simulator for tests that didn't build. ( #297 )
2016-06-28 07:03:40 -07:00
613fb2118c
[jenkins] Skip mmptest to see if it'll speed up the test run on jenkins. ( #248 )
2016-06-28 13:05:00 +02:00
455c741820
[xharness] Try to bump the launch timeout to 3 minutes.
2016-06-28 13:04:56 +02:00
85d94face2
[xharness] Improve logging.
...
Make sure as much as possible is written to the logs, and not stdout.
Also copy an extra 1024 bytes when fetching the system log (and continue
capturing a while after stopped, since there may be unflushed system log
data in memory).
2016-06-28 13:04:52 +02:00
c355cf91be
[xharness] Refactor how we do simulator configuration.
...
Split out the code to prepare the simulator from the AppRunner class,
which is now just handling the logic required for each test run.
This way it's easier to handle simulator preparation for multiple
test runs with the same simulator.
Also revamp logging to avoid printing directly to the console, but
instead use the logging classes that permit redirecting logging
to a file. This makes the html report show better logging.
2016-06-28 13:04:47 +02:00
61485522d4
[xharness] Always setup/cleanup tvOS test runs.
...
Something strange is going on on the bots where only the first tvOS
test run actually launches the test app successfully.
2016-06-28 13:04:41 +02:00
a59a762e84
[xharness] Download mlaunch if we don't have a local copy.
2016-06-28 13:04:10 +02:00
1e6be7f547
[xharness] Add a few more variables to control what to include in a test run.
2016-06-28 13:02:21 +02:00
c98cc1dc19
[xharness] Ignore ignored tests when deciding which color to use for a group of tests.
2016-06-28 13:02:16 +02:00
1a48c36efd
[xharness] When capturing the system log for simulator runs, don't capture contents of the log from before our test run starts.
2016-06-28 13:02:12 +02:00
fa33176ba8
[xharness] Cleanup must be done by the last test, not the first test.
2016-06-28 13:02:09 +02:00
93f0b92eac
[xharness] iPhone 4s doesn't exist anymore, use iPhone 5 instead.
2016-06-28 13:02:03 +02:00
c09c74dba5
[xharness] Improve html report a bit when some tests are ignored.
2016-06-28 12:59:54 +02:00
ab7bd56e63
[xharness] Add a few variables to control if to include iOS/Mac tests in jenkins test mode.
2016-06-28 12:59:50 +02:00
a7b20c1a43
[xharness] Fix emptyness check.
2016-06-28 12:59:42 +02:00
3a825fc5e0
[xharness] Improve watching objects in the debugger.
2016-06-28 12:59:36 +02:00
ddc3a56e68
[tests][harness] Add kTCCServiceMediaLibrary to avoid privacy popup ( #230 )
...
Running some API without permission can crash some tests.
2016-06-28 12:59:25 +02:00
1a1958f52d
Merge remote-tracking branch 'origin/master' into xcode8
2016-06-28 12:44:08 +02:00
4a1f7fa778
Merge branch 'master' into xcode8
2016-06-27 16:30:29 -04:00
833ea3fc8b
[xharness] Fix name of referenced projects.
2016-06-27 13:07:37 +02:00
278550795e
[xharness] Fix simulator validation check to not fail when it shouldn't. ( #265 )
2016-06-24 01:03:36 -07:00
d62295f0bd
[jenkins] Skip mmptest to see if it'll speed up the test run on jenkins. ( #248 )
2016-06-22 07:12:51 -07:00
fdccb019a5
[xharness] Try to bump the launch timeout to 3 minutes.
2016-06-22 10:42:22 +02:00
fb37a21ef9
[xharness] Improve logging.
...
Make sure as much as possible is written to the logs, and not stdout.
Also copy an extra 1024 bytes when fetching the system log (and continue
capturing a while after stopped, since there may be unflushed system log
data in memory).
2016-06-22 10:42:22 +02:00
dfd0fbe81a
[xharness] Refactor how we do simulator configuration.
...
Split out the code to prepare the simulator from the AppRunner class,
which is now just handling the logic required for each test run.
This way it's easier to handle simulator preparation for multiple
test runs with the same simulator.
Also revamp logging to avoid printing directly to the console, but
instead use the logging classes that permit redirecting logging
to a file. This makes the html report show better logging.
2016-06-22 10:42:22 +02:00
755a0622bd
[xharness] Always setup/cleanup tvOS test runs.
...
Something strange is going on on the bots where only the first tvOS
test run actually launches the test app successfully.
2016-06-22 10:42:22 +02:00
6b5a394244
[xharness] Add a few more variables to control what to include in a test run.
2016-06-22 10:42:22 +02:00
0a39ec940f
[xharness] Ignore ignored tests when deciding which color to use for a group of tests.
2016-06-22 10:42:22 +02:00
1e1bbec8a3
[xharness] When capturing the system log for simulator runs, don't capture contents of the log from before our test run starts.
2016-06-22 10:42:22 +02:00
bd993fbe0d
[xharness] Cleanup must be done by the last test, not the first test.
2016-06-22 10:42:22 +02:00
cf060f4611
[xharness] iPhone 4s doesn't exist anymore, use iPhone 5 instead.
2016-06-22 10:42:22 +02:00
d6f27c2e65
[xharness] Improve html report a bit when some tests are ignored.
2016-06-22 10:42:22 +02:00
ade84d6397
[xharness] Add a few variables to control if to include iOS/Mac tests in jenkins test mode.
2016-06-22 10:42:22 +02:00
64bf75c533
[xharness] Fix emptyness check.
2016-06-22 10:42:22 +02:00
b2431898b0
[xharness] Improve watching objects in the debugger.
2016-06-22 10:42:22 +02:00
5a9c9003fa
[xharness] Use Xcode 8 when debugging.
2016-06-22 10:42:22 +02:00
d9fb8dc937
[tests][harness] Add kTCCServiceMediaLibrary to avoid privacy popup ( #230 )
...
Running some API without permission can crash some tests.
2016-06-18 06:36:59 -07:00
b8ceeeb044
[xharness] Fix logdirectory path to use correct directory separator. ( #212 )
2016-06-16 00:03:04 -07:00
70747b3317
[xharness] Fix logdirectory path to use correct directory separator. ( #210 )
2016-06-15 22:05:12 -07:00
64bc391669
[xharness] Fix NRE when running from the command line. ( #207 )
...
* [xharness] Log simulator loading and handle any exceptions.
* [xharness] Add missing tests.
* [xharness] Throw a helpful exception instead of NRE when a simulator can't be found.
* [xharness] Fix null reference exception.
2016-06-15 18:57:47 -07:00
8a164bc8d0
[xharness] Download mlaunch if we don't have a local copy.
2016-06-15 17:00:09 -07:00
f17241e6e3
[xharness] Fix null reference exception.
2016-06-15 16:52:45 -07:00
df326a1d59
[xharness] Throw a helpful exception instead of NRE when a simulator can't be found.
2016-06-15 16:52:24 -07:00
ed1cbe003e
[xharness] Add missing tests.
2016-06-15 16:51:42 -07:00
73b3a9b53e
[xharness] Log simulator loading and handle any exceptions.
2016-06-15 13:32:09 -07:00
15a5af3342
[xharness] Exclude bcl tests by default.
2016-06-15 10:51:14 -07:00
a5f5a76cc0
[xharness] Try and fix some strange file sharing exceptions.
2016-06-15 10:51:14 -07:00
126de6836a
[xharness] Use full path to mdtool.
2016-06-15 10:51:14 -07:00
0910cacf51
[xharness] Fix processing of ignored tasks.
2016-06-15 10:51:14 -07:00
dcc605ed22
[xharness] Build XM/Classic projects using mdtool.
2016-06-15 10:51:14 -07:00
af4a93a181
[xharness] Pass the right environment variables when invoking xbuild.
2016-06-15 10:51:14 -07:00
ae1bf7fb5f
[xharness] Add support for ignoring tests.
...
And use it to ignore framework-test (never run on wrench) and
watchOS tests (currently ignored on wrench).
2016-06-15 10:51:14 -07:00
d9bf0e1728
[xharness] Add support for building and running mac tests on jenkins.
2016-06-15 10:51:14 -07:00
9a20b42bd6
[xharness] No need to prepare the simulator more than once.
2016-06-15 10:51:14 -07:00
0c1f235800
[xharness] Fix simulator configuration to do all configuration before first launch.
2016-06-15 10:51:14 -07:00
2d3001cd25
[xharness] Don't fail a machine has never seen a crash report.
2016-06-15 10:51:14 -07:00
d135612f59
[tests] Implement support in xharness for running the simulator tests grouped by simulator, and write out an html report.
2016-06-15 10:51:14 -07:00
adde200556
[xharness] Exclude bcl tests by default.
2016-06-15 09:57:24 -07:00
fdf2bd5d7b
[xharness] Try and fix some strange file sharing exceptions.
2016-06-15 09:57:24 -07:00
4bad2c1485
[xharness] Use full path to mdtool.
2016-06-15 09:57:24 -07:00
266606528b
[xharness] Fix processing of ignored tasks.
2016-06-15 09:57:24 -07:00
eec56fefc6
[xharness] Build XM/Classic projects using mdtool.
2016-06-15 09:57:24 -07:00
f6948f8183
[xharness] Pass the right environment variables when invoking xbuild.
2016-06-15 09:57:24 -07:00
39a0873403
[xharness] Add support for ignoring tests.
...
And use it to ignore framework-test (never run on wrench) and
watchOS tests (currently ignored on wrench).
2016-06-15 09:57:24 -07:00
ad229cef71
[xharness] Add support for building and running mac tests on jenkins.
2016-06-15 09:57:24 -07:00
969f2d7ca1
[xharness] No need to prepare the simulator more than once.
2016-06-15 09:57:24 -07:00
3d5455b54c
[xharness] Fix simulator configuration to do all configuration before first launch.
2016-06-15 09:57:24 -07:00
646bd59c85
[xharness] Don't fail a machine has never seen a crash report.
2016-06-15 09:57:24 -07:00
d087ba2d5a
[tests] Implement support in xharness for running the simulator tests grouped by simulator, and write out an html report.
2016-06-15 09:57:24 -07:00
8af0326105
[xharness] Clean targets are named 'Clean', not 'clean'. ( #162 )
...
At least according to mdtool.
Fixes this:
[...]
Target 'clean not supported
make[4]: *** [clean-mac-classic-introspection] Error 1
2016-06-09 14:23:58 -04:00
e3cc7c31bc
[xharness] Write relative paths in the makefiles. ( #160 )
...
Fixes the packaged mac tests when they're executed on a different
machine with a different path.
2016-06-09 17:36:32 +02:00
9c01fe6c9d
[tests] Use the right sources for dontlink, linksdk and linkall tests. ( #152 )
2016-06-08 23:39:47 +02:00
e450dae136
[xharness] Add support for the introspection tests. ( #151 )
...
* [xharness] Add support for the introspection tests.
* [tests] Add support for mac introspection tests to xharness.
2016-06-08 19:45:31 +02:00
aaae585d52
[xharness] Xcode 7.3 ships symbolicatecrash in a new place, so look there as well. ( #134 )
2016-06-06 15:32:09 +02:00
4a714f3c43
Make logic not depend on the checkout directory being named 'xamarin-macios'. ( #129 )
2016-06-06 12:18:52 +02:00
715c2fa5d4
[xharness] Add separate solution to make it load faster in the IDE. ( #124 )
2016-06-02 12:17:52 +02:00
7fad8dfb40
[tests] Build the bitcode configuration on arm64 only, armv7 doesn't seem to link:
...
Undefined symbols for architecture armv7:
"___gxx_personality_v0", referenced from:
2016-05-27 21:54:49 -04:00
c8b35f9b6a
[xharness] Fix mlaunch lookup.
2016-05-26 15:15:08 +02:00
ecfdea9508
[tests] Import
2016-05-26 15:06:52 +02:00
Rolf Bjarne Kvinge
Sebastien Pouliot
Zoltan Varga