* [XM] Seperate parsing from compilation in AOT code
- Parsing command line options was too entangled with actually compiling
- Refactor test code to centralize Compile, reducing some complexity of tests
- Groundwork for hybrid vs non-hybrid AOT work
* Add hybrid/standard AOT support
* Add xammac_tests to makefile/jenkins
* Add flag to force AOT
* Disable MonoTouchFixtures.Contacts.ContactStoreTest.GetUnifiedContacts on XM due to prompts on Jenkins
* [xharness] Show test xml files in the html report.
* [xharness] Force links to xml files to show up in a new top-level window.
Jenkins shows the html report in an iframe, and links will load in that
iframe. This means that the browser renders the contents of the link as if it
was html, and rendering xml as html results in a blank page.
So set the target attribute on links to xml files, so that those xml files
open as top-level pages, and then the browser will rendering the xml like xml.
Don't create multiple writer streams for the same underlying file, instead
store the writer stream on the log instance, and return it whenever needed.
This fixes an issue where there would be multiple writer streams, which could
cause exceptions when creating more writer streams (access defined errors
because the file is already open).
* [tests] Use the target directory from the loaded configuration.
* [xharness] Find the root directory based on xharness.exe's location (unless specified).
* [tests] Add makefile target to generate test config using the system XI.
* [xharness] Don't copy a ExecutionResult before executing.
The source might be an ignored test, even though the cloned task isn't, so if
we copy the ExecutionResult we might unexpectedly ignore the cloned task.
* [xharness] Provide better failure message for failing mac tests.
Unexpected exceptions are now caught in a more central location
(TestTask.RunInternalAsync), which means it's not necessary to catch
unexpected exceptions in other places (so those catch handlers have been
removed).
Additionally we now set the failure message when such an exception occurs, in
addition to writing the full exception details to a custom log file.
We're shipping mlaunch now, which means we have other file system locations
where it might be.
Also don't look in Xamarin Studio anymore, it will always be an old version,
instead check the system's Xamarin.iOS version.
* Add a server mode, which launches a web server (and a web page) that can be
used to interactively run tests and view their results.
* Add support for running test assemblies in a today extension (generating a
new set of projects, similar to how we generate tvOS/watchOS projects based
on the iOS project, we now generate a today extension project in addition to
the tvOS and watchOS projects).
* Load all the different tests (and show them in the html report, although
they show up as 'ignored'), even for disabled/ignored tests. This makes
disabled/ignored tests more visible, and also makes it possible to actually
run them using the embedded web server.
* Add support for running tests on device. Tests will be executed on multiple
devices simulatenously (any connected devices will be used).
* [jenkins] Make jenkins non-parellel to see if it fixes#46097.
* [jenkins] Limit serialization to watchos builds.
* [jenkins] Only allow Mac builds to be paralellized.
* [xharness] Add Jenkins support for mac binding project tests.
* [jenkins] Rework how mac binding projects are tested.
Facts:
* The wrench-mac-binding-project target will clean itself when done: [1]
* The mac-msbuild tests depend on mac-binding-project build artifacts (but
will build those artifacts automatically if they don't exist when the mac-
msbuild tests are built).
* Jenkins runs tests in parallel.
Result:
* The mac-msbuild tests may fail if the mac-binding-project tests are run
concurrently and delete its build artifacts.
Fix:
* Don't call the wrench-mac-binding-project target (which will clean itself),
instead just run the tests (without cleaning).
[1]: d99212a712/tests/Makefile (L280-L282)
* [tests] Fix framework-test to actually work.
* [xharness] Properly replace 'ios' with corresponding platform for paths to our test frameworks as well.
* [framework-test] Fix watchOS build.
* Bump maccore to get new mlaunch.
A new mlaunch that:
* Should have fewer random failures when launching watchOS apps.
* Supports launching extensions on device.
* Supports uninstalling apps from devices.
* [jenkins] Automatically detect mono bumps and enable device build.
And do this before fetching labels, so that we can skip fetching labels if we
know we're already enabling the device build.
* [tests] Bumping LLVM merits enabling device build and running mtouch + BCL tests.
e.g. running twice
> make run-ios-sim32-introspection
results in
Unhandled Exception:
System.AggregateException: One or more errors occurred. ---> System.IO.IOException: /Users/poupou/git/master/xamarin-macios/tests/logs/exec-ios-sim32-introspection/iPhone 5.log already exists
at System.IO.File.Copy (System.String sourceFileName, System.String destFileName, System.Boolean overwrite) [0x001bd] in /private/tmp/source-mono-4.8.0/bockbuild-mono-4.8.0-branch/profiles/mono-mac-xamarin/build-root/mono-x86/mcs/class/corlib/System.IO/File.cs:109
at System.IO.File.Copy (System.String sourceFileName, System.String destFileName) [0x00000] in /private/tmp/source-mono-4.8.0/bockbuild-mono-4.8.0-branch/profiles/mono-mac-xamarin/build-root/mono-x86/mcs/class/corlib/System.IO/File.cs:69
at xharness.CaptureLog.StopCapture () [0x00019] in /Users/poupou/git/master/xamarin-macios/tests/xharness/Log.cs:252
Rolf Bjarne Kvinge
Chris Hamons
Sebastien Pouliot