Rework how the two different xammac configurations are done, so that we don't
generate identical makefile targets for both variations (we'll generate a
single makefile target, and the configuration is selected using an environment
variable).
Fixes these makefile warnings:
Makefile-mac.inc:303: warning: overriding commands for target `build-mac-unified-xammac_tests'
Makefile-mac.inc:287: warning: ignoring old commands for target `build-mac-unified-xammac_tests'
Makefile-mac.inc:307: warning: overriding commands for target `clean-mac-unified-xammac_tests'
Makefile-mac.inc:291: warning: ignoring old commands for target `clean-mac-unified-xammac_tests'
Makefile-mac.inc:310: warning: overriding commands for target `exec-mac-unified-xammac_tests'
Makefile-mac.inc:294: warning: ignoring old commands for target `exec-mac-unified-xammac_tests'
Makefile-mac.inc:313: warning: overriding commands for target `run-mac-unified-xammac_tests'
Makefile-mac.inc:297: warning: ignoring old commands for target `run-mac-unified-xammac_tests'
https://bugzilla.xamarin.com/show_bug.cgi?id=56492
This affects introspection-ios/introspection-mac (which are now grouped in an
`introspection` test), and dontlink-mac (which now shows up in the `dont link`
tests).
* [ObjCRuntime] Remove XM limitation. Fixes#55857.
I can't remember why I added this limitation, and git history provides now
clues.
Everything compiles without the limitation, and the bug is fixed, so I can't
see a reason not to fix it.
https://bugzilla.xamarin.com/show_bug.cgi?id=55857
* [tests] Run xammac tests both in Debug and Release configurations.
* [tests] Convert mmp tests to a standard NUnit test library.
* [xharness] Add support for restoring nugets before building projects.
And restore nugets for tests-mac.sln before building mac test projects.
* [tests] Don't create test packages by default.
Don't create test packages by default, instead add a new target to create test
packages. This new target is called on wrench, which means the packages will
still be created when needed, but they won't be built locally in every build
(and if a packaged test fails to build, it won't fail the entire build).
* [tests] Use a project reference instead of assembly reference for GuiUnit.exe
Use a project reference instead of assembly reference for GuiUnit.exe, so that
the GuiUnit reference is automatically built if necessary.
This also makes it required to build a sln for Classic (since mdtool can't
find referenced projects from a csproj).
Make the top-level test name for equivalent tests between iOS and Mac have the
same name, so that these tests are grouped together.
For instance: we used to have a System-Mac tests and System tests (the latter
being iOS tests), now there are just System tests, and nested inside are both
the Mac and iOS System tests.
* [XM] Seperate parsing from compilation in AOT code
- Parsing command line options was too entangled with actually compiling
- Refactor test code to centralize Compile, reducing some complexity of tests
- Groundwork for hybrid vs non-hybrid AOT work
* Add hybrid/standard AOT support
* Add xammac_tests to makefile/jenkins
* Add flag to force AOT
* Disable MonoTouchFixtures.Contacts.ContactStoreTest.GetUnifiedContacts on XM due to prompts on Jenkins
* [tests] Use the target directory from the loaded configuration.
* [xharness] Find the root directory based on xharness.exe's location (unless specified).
* [tests] Add makefile target to generate test config using the system XI.
We're shipping mlaunch now, which means we have other file system locations
where it might be.
Also don't look in Xamarin Studio anymore, it will always be an old version,
instead check the system's Xamarin.iOS version.
* Add a server mode, which launches a web server (and a web page) that can be
used to interactively run tests and view their results.
* Add support for running test assemblies in a today extension (generating a
new set of projects, similar to how we generate tvOS/watchOS projects based
on the iOS project, we now generate a today extension project in addition to
the tvOS and watchOS projects).
* Load all the different tests (and show them in the html report, although
they show up as 'ignored'), even for disabled/ignored tests. This makes
disabled/ignored tests more visible, and also makes it possible to actually
run them using the embedded web server.
* Add support for running tests on device. Tests will be executed on multiple
devices simulatenously (any connected devices will be used).
* Bump maccore to get new mlaunch.
A new mlaunch that:
* Should have fewer random failures when launching watchOS apps.
* Supports launching extensions on device.
* Supports uninstalling apps from devices.
* [XM] Add 32-bit variants of apitest and introspection and fix bindings
- Previous to this we were only running 64-bit tests, which caused us
to make mistakes
* Fix cherry-pick to build
* Correctly skip jenkins XM45 when asked
* Fix logic
* Run 32-bit tests on Jenkins
* Fix Jenkins grouping by respecting -32 suffix
* More fixing of jenkins. This time make enum for 32-bit types
* [xharness] Don't crash if we can't find a simulator.
* [xharness] Create a device pair if none applicable is found.
* [xharness] Use an enum instead of string values for the target.
* [xharness] Unify the simulator selection code between Jenkins and Wrench.
* [Jenkins] Make test to write output as an xml file so that it can be parsed by the jenkins bot.
* Point to the correct Touvh.Unit repo.
* Use the available property to determine if we are being ran in Jenkins.
* Log where are test results stored.
* Add @MonkeyWrench: prefix.
* Ensure that we do set the build env in jenkins/run-tests.sh
* Do not mix Wrench with Jenkins. The reports in jenkins can be Xml, in Wrench we prefer the old style.
* Ensure that the main node of the unit tests does contain the target, that will improve the tests results reporting.
* Revert "Fix binding project LinkWithAttributes generation to prevent rebuilds" (#1018)
* Added xslt to be used to keep the old Test Reports until we move to only Jenkins reports.
* Add an extra log for the xslt transformation.
* Point to the correcto dir in jenkins.
* Deal with the xslt once we have finished rather than in a batch.
* Remove noise.
* Readd case removed in rebase.
* Fix indentation.
* Skip lock keychain.
* [tests] Implement filtering based on test name to make it possible to run a subset of tests manually.
* [xharness] Add support for passing environment variables to test apps.
* [xharness] Split the mscorlib/System test run for watchOS into multiple executions.
When running on a watch, the complete set of mscorlib and System tests use too
much memory, so they end up crashing the app.
So instead don't run all the tests in the same test run, but
instead split them up.
This splits the System tests in 5 different sets, based on the first
character of the test class name.
The mscorlib tests are split in 26 different sets, one for each
character (A-Z).
On watchOS we can't include all the tests for device in a single
assembly, because the native object code becomes too big.
So instead split the tests into two assemblies. Luckily this
is fairly easy for test projects, since they have few dependencies
between source files (there are some however, so source files
are grouped per directory so that files in the same directory do
not end up in different test projects). There is also some hard-coded
logic.
https://bugzilla.xamarin.com/show_bug.cgi?id=41746
On watchOS we can't include all the tests for device in a single
assembly, because the native object code becomes too big.
So instead split the tests into two assemblies. Luckily this
is fairly easy for test projects, since they have few dependencies
between source files (there are some however, so source files
are grouped per directory so that files in the same directory do
not end up in different test projects). There is also some hard-coded
logic.
https://bugzilla.xamarin.com/show_bug.cgi?id=41746
Split out the code to prepare the simulator from the AppRunner class,
which is now just handling the logic required for each test run.
This way it's easier to handle simulator preparation for multiple
test runs with the same simulator.
Also revamp logging to avoid printing directly to the console, but
instead use the logging classes that permit redirecting logging
to a file. This makes the html report show better logging.
Timothy Risi
Rolf Bjarne Kvinge
Chris Hamons
Manuel de la Pena
Sebastien Pouliot
Sebastien Pouliot