At this point the BCLTestProjectGenerator class just represents a configuration class that knows about the project that are particular to xamarin-macios. All the logic that creates the projects has been moved to the XamariniOSTemplate which creates the project using the information passed by the config class.
At this stage, we have the ability to generate test projects by simply passing the info classes, which can be built either by a class (like the generator one) or a command line. We have fully decoupled the project generation from Xharness.
Tests have been added or moved to the correct location.
The name of the tests is picked up in a more inner node than
expected. Fwd the test name to make sure that noe all
crashes/timeouts/launches have the same name.
Add extra information when we have a timeout. We want to know the
name and the bot used to ensure that it is easier to identify patterns.
Timeout tests now have:
test title as: App Timeout {AppName} {Variation} on bot {device_name}
message: AppName} {Variation} Test run timed out after {timeout.TotalMinutes} minute(s) on bot {device_name}.
Build issues now have:
test title: App Build {projectTask.TestName} {projectTask.Variation}
That way we know the exact app that failed to build and the exact app that timeout and the device used to run it.
Move the project templates to be a resource in the dll. That way we can
move all the code outside of xharness and used it outside. This is not a
fill refactor but a first step to decouple the bcl test generation from
the xamarin-macios project.
XHarness should not see a difference, everythign works and simply makes
the dlls larger due to the new resources.
The code is going to take into account that the mono team will want to
use and unmanaged template with no dependencies on Xamarin.iOS and
Xamarin.Mac.
Co-Authored-By: Rolf Bjarne Kvinge <rolf@xamarin.com>
Co-authored-by: Rolf Bjarne Kvinge <rolf@xamarin.com>
Move the project templates to be a resource in the dll. That way we can
move all the code outside of xharness and used it outside. This is not a
fill refactor but a first step to decouple the bcl test generation from
the xamarin-macios project.
XHarness should not see a difference, everythign works and simply makes
the dlls larger due to the new resources.
The code is going to take into account that the mono team will want to
use and unmanaged template with no dependencies on Xamarin.iOS and
Xamarin.Mac.
Add tests for the SimDevice and Simulators classes that ensure that the
process is correctly called and the xml with the simulators parsed and
returns the correct number of sims.
Update the generation of the xml to include a more descriptive title
than "AppCrash". After this commit the name of the testcase and the test
follow this pattern:
- "App Crash {ApplicationName} {Variation}"
This will make it easier to filter since the an example failure will be:
"App Crash Monotouch (Debug)"
Note, the '()' in the example are part of the variation and that is
build by xharness and passed to the runner.
Fixes: https://github.com/xamarin/xamarin-macios/issues/8077
Moved the implementations under Hardare. Add the following to simplify
the testing of the AppRunner:
1. Add interfaces for the listing/loading of devices and simulators.
2. Add a special class for mlaunch args. That way we are sure that
arguments are correctly passed and we do not have typos.
3. Added tests for the new classes and for the device and devices
classes.
4. Added tests for the management of the TCC db.
Tests for the SimDevice and Simulators will come in a following PR. The
only reason for it is that I realize that the commit was getting to
large.
Co-authored-by: Rolf Bjarne Kvinge <rolf@xamarin.com>
Co-authored-by: Přemek Vysoký <premek.vysoky@microsoft.com>
Moved the implementations under Hardware. Add the following to simplify
the testing of the AppRunner:
1. Add interfaces for the listing/loading of devices and simulators.
2. Add a special class for mlaunch args. That way we are sure that
arguments are correctly passed and we do not have typos.
3. Added tests for the new classes and for the device and devices
classes.
4. Added tests for the management of the TCC db.
Tests for the SimDevice and Simulators will come in a following PR. The
only reason for it is that I realize that the commit was getting to
large.
Co-Authored-By: Rolf Bjarne Kvinge <rolf@xamarin.com>
Co-Authored-By: Přemek Vysoký <premek.vysoky@microsoft.com>
Move all the extension methods to a class. After this refactor, we will
be able to DI the manager in the other classes and assert that the
processes are called with the correct parameters without the need of
launching them.
Also added tests for the manager. We create a dummy console app that
will be executed by the tests. The console app has a number of
parameters that will be used to ensure that the new process behaves as
we want:
- Use the passed exit code.
- Create child proecesses if needed.
- Sleep to force a timeout.
- Writer messages to stdout and stderr.
Our tests call the dummy app and ensures that the results match the
behaviour expected by the dummy app.
Co-authored-by: Manuel de la Pena <mandel@microsoft.com>
Co-authored-by: Premek Vysoky <prvysoky@microsoft.com>
Co-authored-by: Rolf Bjarne Kvinge <rolf@xamarin.com>
* [Harness] Refactor process management to be testable.
Move all the extension methods to a class. After this refactor, we will
be able to DI the manager in the other classes and assert that the
processes are called with the correct parameters without the need of
launching them.
Also added tests for the manager. We create a dummy console app that
will be executed by the tests. The console app has a number of
parameters that will be used to ensure that the new process behaves as
we want:
- Use the passed exit code.
- Create child proecesses if needed.
- Sleep to force a timeout.
- Writer messages to stdout and stderr.
Our tests call the dummy app and ensures that the results match the
behaviour expected by the dummy app.
* Apply suggestions from code review
Co-Authored-By: Přemek Vysoký <premek.vysoky@microsoft.com>
* Move out utils into a separate namespace
* Move Cache to the test project
* Fix namespaces after merge
* Remove unneeded code
* Move Target files
* Sort csproj
* Refactor Targets
* Rename Cache to TempDirectory
* Fix using
* Move ProjectFileExtensions
* Remove dead code
* Move Extensions
* Add empty StringUtilsTests
* Add StringUtils tests
* Revert refactorings
* Update tests/xharness/Utilities/StringUtils.cs
Co-Authored-By: Rolf Bjarne Kvinge <rolf@xamarin.com>
* Update tests/xharness/Utilities/StringUtils.cs
Co-Authored-By: Rolf Bjarne Kvinge <rolf@xamarin.com>
* Add better formatarguments test
* Update tests/xharness/Utilities/StringUtils.cs
Co-Authored-By: Rolf Bjarne Kvinge <rolf@xamarin.com>
Co-authored-by: Manuel de la Pena <mandel@microsoft.com>
Co-authored-by: Premek Vysoky <prvysoky@microsoft.com>
Co-authored-by: Rolf Bjarne Kvinge <rolf@xamarin.com>
This also meant adding support for running .NET tests in xharness. Some
refactoring was done to extract common code to shared members, in order to
avoid duplicating a lot of code.
Move all the extension methods to a class. After this refactor, we will
be able to DI the manager in the other classes and assert that the
processes are called with the correct parameters without the need of
launching them.
Also added tests for the manager. We create a dummy console app that
will be executed by the tests. The console app has a number of
parameters that will be used to ensure that the new process behaves as
we want:
- Use the passed exit code.
- Create child proecesses if needed.
- Sleep to force a timeout.
- Writer messages to stdout and stderr.
Our tests call the dummy app and ensures that the results match the
behaviour expected by the dummy app.
Co-authored-by: Přemek Vysoký <premek.vysoky@microsoft.com>
Provide tests that will ensure that the Tcp listener and File listener
will work correctly. Tests ensure that both listeners receive the data
and write it in the ILog
The tcp test is an interesting one because it redirects the call of the
method to a diff callback that will write the data in a stream to check
the final result.
Move to use interfaces, that will let us later add tests that will
verify that all the correct logging is performed. As an example, added a
test for XmlResultParser that ensures that the failures are correctly
generated. The test uses Moq to pass the different paths to be used and
later be able to verify the wirtten xml.
Co-authored-by: Přemek Vysoký <premek.vysoky@microsoft.com>
Move to use interfaces, that will let us later add tests that will
verify that all the correct logging is performed. As an example, added a
test for XmlResultParser that ensures that the failures are correctly
generated. The test uses Moq to pass the different paths to be used and
later be able to verify the wirtten xml.
Some crashes are reported as tcp connection issues because they happen
before the app had the chance to write anything in the log. In that
case, we checked for the presence of the file, and if not present we
decided it was a tcp issue when it is not the case.
In this commit, we check for the tcp erorr message in the main_log so
that we are certain that the issue was with the connection and not
anyother.
Testingis simple, ran tests without the phone being part of the same
network and test with a branch that has a crash. For example the one in
https://github.com/xamarin/xamarin-macios/pull/8009 hash 83240612e8
We support different outputs, lets add the avility for the caller to
decide which one to use. We default to NUnit V3 due to or dependency to
it in VSTS.
We do not longer compile the bcl tests, but we consume them as part or
mono. The old BCL targets and classes are not needed. As you can see,
the lists were empty.
Add tests to make sure that we can be safe when we make changes to that
part of the code.
Some methods are not tested but due to dependencies. Will work un
separating things a little to keep moving fwd.
We initially added a cmd to do the bcl test import generation that has
never been used.
We are moving the bcl test importer code to xharness and creating a
NUnit test project to run the tests that we already added. Unit tests
pass.
It will be nice to be able to have a smaller set of tests to be ran by
xharness. In this commit we add a set of labels to be able to divide the
test in the following groups:
* Xamarin - all tests BUT monotouch.
* Monotouch - just monotouch tests.
* Old BCL - The BCL tests that are based on NUnit
* New BCL - The BCL tests that are based on xUnit.
* mscorlib - Just the test that exercise mscorlib (and the different
groups, mscorlib 1, mscorlib 2 etc..)
This will allow to parallelize the execution of the full test suit in
different agents in VSTS.
Xml should be:
```
<failure>
<message>Foo</message>
<stack-trace>Bar</stack-trace>
</failure>
```
But we generate:
```
<failure>
<message>Foo
<stack-trace>Bar</stack-trace>
</message>
</failure>
```
Makes the parsing of the failures impossible.
In order to simplify the monitoring job add the device name to the
following failures:
* Installation
* Launch
* Tcp Connection
All the above are most of the time due to a misconfigured device. The
device name is useful information for the monitoring person to be able
to reach IT and address the issue.
The parsing code set the crashed variable, but it was not set as an
output variable, that meant that the value was not used. Further in the
code we use the variable to decide if we had a crash or not. Most of the
time, there is no probel since if we have a real crash, we will get the
crash reports and to the right thing, but in the case of tcp connection
issues we do not have them, and therefore the crash xml result is not
created.
Bonus: the local logs variable was hidding a variable in another scope,
fixed that.
xharness does not only allow to run tests in CI, but also helps to run
specific project. That feature is used for the mtouch tests. In that
scenario, we do not have a build task and therefore we will not have the
build logs.
fixes: https://github.com/xamarin/maccore/issues/2154
If we are in VSTS and we want to have correct reportings we need to
generate a test failure when the applicaiton times out. If not, we have
a missmatch between the results from xharness (we have failures) and
VSTS (success).
On devices that cannot reach the host via TCP we do not have a log, this
means that in the if statement needs to have a case for it.
The main problem is that when the device cannot connect to the host, we
do not get a log OR a crash reason from the crash logs. It makes sense
not to have a crash reason, because the app did not crash. In these
sitations, we have to create a xml crash report (since we really do not
know if we can parse the file) that will tell vsts that there was an
issue. Adding the main log will let the monitoring person see the
results of the test run.
Because the PublishTest task is taking a regular expression, it is
importing the test results more than one, which gives wrong stats. Add a
prefix to better filter those files we are interested in (thos with
attachments).
As a bonus, refactored the xml failure code for less copy pasting and to
have a single place where we had to add the prefix.
VSTS does not provide a good way to report an app installation issue,
but we can fake a failure in the test when the installation does not
happen.
In the case of the installation failure a xml test result is generated
that will expose all the required information and will attach all the
needed logs (install logs). An example of the generated result can be
seen here: https://gist.github.com/mandel-macaque/2274bcd8785eebd636b98142e228afa9
As per the documentation of the VSTS test uploader (https://docs.microsoft.com/en-us/azure/devops/pipelines/tasks/test/publish-test-results?view=azure-devops&tabs=yaml#attachments-support)
the tests support the addition of attachments. We want to be able to
access the logs of the different tests runs, this is achieved the
following way:
1. Move TouchUnit to use NUnit V3 which allows to add attachments.
2. Once the tests are ran, add the attachments to the following nodes:
1. The very first test-suite. This will allow to have the logs for
succesul tests.
2. Add logs to failing tests. Reduces the number of clicks to be done
to access to the logs when a test case fails.
3. Modify the assembly name of the test-suit to match the name of the
application. This ensures two things.
1. We have a consistent name for the file column in VSTS, that can be
used to see recurrent failing tests.
2. The name is more readable, since if not, it will contain the UUID
of the device.
Logs are not added to succesful tests because it will have the following
problems:
* Larger data storage usage.
* Longer upload time. The addtion of the logs per tests (succesful or
failed) was tested and resulted in an upload time LONGER than 6 hours
for all TouchUnit, NUnit blc tests and xUnit bcl tests.
In order for this to be useful, the task in the pipeline SHOULD NOT
merge test runs. We should have a test run PER application so that we do
not mix the logs.
Co-Authored-By: Rolf Bjarne Kvinge <rolf@xamarin.com>
One of the reason why we lost the TestReport was because the xml added
from the new code did not use the same name for the log as the expected
one (https://github.com/xamarin/xamarin-macios/pull/7835/files#diff-03773861bcef485bf343300f31b60b0eR374).
If we are going to be using the description as a way to decide what is
going to be done with it, lets use static vars so that we do not have
logs added with the wrong name.
Add code to write the TestReport when we have xunit results. Also
refactored code for the NUnit format to not use LINQ, it will use less
memory and in the future we can move to async (not now since it might
raise other problems).
fixes: https://github.com/xamarin/xamarin-macios/issues/7826
Co-authored-by: Manuel de la Pena <mandel@microsoft.com>
As per the documentation of the VSTS test uploader (https://docs.microsoft.com/en-us/azure/devops/pipelines/tasks/test/publish-test-results?view=azure-devops&tabs=yaml#attachments-support)
the tests support the addition of attachments. We want to be able to
access the logs of the different tests runs, this is achieved the
following way:
1. Move TouchUnit to use NUnit V3 which allows to add attachments.
2. Once the tests are ran, add the attachments to the following nodes:
1. The very first test-suite. This will allow to have the logs for
succesul tests.
2. Add logs to failing tests. Reduces the number of clicks to be done
to access to the logs when a test case fails.
3. Modify the assembly name of the test-suit to match the name of the
application. This ensures two things.
1. We have a consistent name for the file column in VSTS, that can be
used to see recurrent failing tests.
2. The name is more readable, since if not, it will contain the UUID
of the device.
Logs are not added to succesful tests because it will have the following
problems:
* Larger data storage usage.
* Longer upload time. The addtion of the logs per tests (succesful or
failed) was tested and resulted in an upload time LONGER than 6 hours
for all TouchUnit, NUnit blc tests and xUnit bcl tests.
In order for this to be useful, the task in the pipeline SHOULD NOT
merge test runs. We should have a test run PER application so that we do
not mix the logs.
Co-Authored-By: Rolf Bjarne Kvinge <rolf@xamarin.com>
One of the reason why we lost the TestReport was because the xml added
from the new code did not use the same name for the log as the expected
one (https://github.com/xamarin/xamarin-macios/pull/7835/files#diff-03773861bcef485bf343300f31b60b0eR374).
If we are going to be using the description as a way to decide what is
going to be done with it, lets use static vars so that we do not have
logs added with the wrong name.
Add code to write the TestReport when we have xunit results. Also
refactored code for the NUnit format to not use LINQ, it will use less
memory and in the future we can move to async (not now since it might
raise other problems).
fixes: https://github.com/xamarin/xamarin-macios/issues/7826
Only one to start... it's been discussed before but we generally
found other ways to do them. Let's continue to pick the best place
but we now have more options :)
We ping the tcp listener to know that we have a tcp connection, that is
written in the xml logs, which means that parsing will not work. Ignore
the ping, parse xml, and make sure that the xml that will be consume by
vsts is valid.
This PR also fixes https://github.com/xamarin/maccore/issues/827 which does not longer happen.
Looks like we have issues with the internet sharing in the VSTS bots,
this means that now that we always try to parse the XML on CI, we get an
exception, catch it and do not show the results.
The workaround simply tries to read the xml, if possible, we will parse
it, else deal with the text only log. The fix will show the results, but
it is a workaround for a configuration issue in the CI.
Unify the harness properties to just look at InCI and remove all the
other ones. There is no real need to have differences between jenkins
and VSTS and Wrench is gone.
We used to test only on Jenkins, and if the build version had jenkins on
it, rather than doing so, just check if we are in the CI by looking if
BUILD_REVISION is present in the env.
Use the xml parsing helper methods to decide if the xml can be parsed
and if we will be able to generate the report. That way we avoid an
exception that makes the CI noise.
Co-authored-by: Manuel de la Pena <mandel@microsoft.com>
Use the xml parsing helper methods to decide if the xml can be parsed
and if we will be able to generate the report. That way we avoid an
exception that makes the CI noise.
Use the xml parsing helper methods to decide if the xml can be parsed
and if we will be able to generate the report. That way we avoid an
exception that makes the CI noise.
We ping the tcp listener to know that we have a tcp connection, that is
written in the xml logs, which means that parsing will not work. Ignore
the ping, parse xml, and make sure that the xml that will be consume by
vsts is valid.
This PR also fixes https://github.com/xamarin/maccore/issues/827 which does not longer happen.
Looks like we have issues with the internet sharing in the VSTS bots,
this means that now that we always try to parse the XML on CI, we get an
exception, catch it and do not show the results.
The workaround simply tries to read the xml, if possible, we will parse
it, else deal with the text only log. The fix will show the results, but
it is a workaround for a configuration issue in the CI.
Unify the harness properties to just look at InCI and remove all the
other ones. There is no real need to have differences between jenkins
and VSTS and Wrench is gone.
We used to test only on Jenkins, and if the build version had jenkins on
it, rather than doing so, just check if we are in the CI by looking if
BUILD_REVISION is present in the env.
We ping the tcp listener to know that we have a tcp connection, that is
written in the xml logs, which means that parsing will not work. Ignore
the ping, parse xml, and make sure that the xml that will be consume by
vsts is valid.
This PR also fixes https://github.com/xamarin/maccore/issues/827 which does not longer happen.
* Acquire permission for the microphone (the microphone permission dialog is
not a blocking dialog, which is why things have worked so far).
* Set the last_modified field in the TCC database to now instead of 1970.
Apparently some permissions time out (kTCCServiceMediaLibrary), which means
that the last_modified field is important to get right.
Looks like we have issues with the internet sharing in the VSTS bots,
this means that now that we always try to parse the XML on CI, we get an
exception, catch it and do not show the results.
The workaround simply tries to read the xml, if possible, we will parse
it, else deal with the text only log. The fix will show the results, but
it is a workaround for a configuration issue in the CI.
Unify the harness properties to just look at InCI and remove all the
other ones. There is no real need to have differences between jenkins
and VSTS and Wrench is gone.
We used to test only on Jenkins, and if the build version had jenkins on
it, rather than doing so, just check if we are in the CI by looking if
BUILD_REVISION is present in the env.
* Bump Xamarin.MacDev.
New commits in xamarin/Xamarin.MacDev:
* xamarin/Xamarin.MacDev@210c664 Adds net451 to Xamarin.MacDev.csproj
* xamarin/Xamarin.MacDev@64db365 [winios] Changes provisioning profiles default path
* xamarin/Xamarin.MacDev@d34430a Switch to short-form projects and build for both net461 and netstandard2.0. (#68)
Diff: 0f578f51e6..210c664e56
* [msbuild] Update to latest Mono.Cecil.
The older version doesn't support netstandard2.0.
No code changes were required.
* [msbuild] Remove unused usings.
* [msbuild] Make ILMerge work when building for netstandard2.0.
Also unify/deduplicate the ILMerge logic between Xamarin.iOS and Xamarin.Mac.
* [msbuild] Build for netstandard2.0 in addition to net461.
* [msbuild] Use custom project configurations to support running the tests for both netstandard2.0 and net461.
Use custom project configurations to support running the tests for when the
tasks assembly is built for netstandard2.0 and net461.
* [tests] Make command-line based 'make test-ios-tasks' run tests for both netstandard2.0 and net461.
* [xharness] Add test configuration to run iOS MSBuild tests using either netstandard2.0 or net461.
* [msbuild] Make the netstandard2.0-buils task assemblies the default.
* [msbuild] ILRepack lib assemblies, not ref assemblies.
Ask MSBuild to copy lib assemblies to the output folder when building for
netstandard2.0, this way we can easily find the actual implementation
libraries to pass to ILRepack.
* [msbuild] Merge System.Text.Encodings.Web.dll as well.
* [xharness] Fix build of MSBuild tests for iOS.
* [xharness] Fix two compiler warnings.
Fixes these warnings:
Jenkins.cs(538,37): warning CS1998: This async method lacks 'await' operators and will run synchronously. Consider using the 'await' operator to await non-blocking API calls, or 'await Task.Run(...)' to do CPU-bound work on a background thread.
Jenkins.cs(877,14): warning CS1998: This async method lacks 'await' operators and will run synchronously. Consider using the 'await' operator to await non-blocking API calls, or 'await Task.Run(...)' to do CPU-bound work on a background thread.
* [xharness] Simplify async code a bit.
* [msbuild] Convert to short-form csproj.
* [msbuild] Make asserts more useful.
* [msbuild] Make tests ignore the actual location of the test assembly.
* [msbuild] Short-style projects default to deterministic builds, which is not compatible with wildcard versions.
* [msbuild] Adjust test.
* Update .gitignore.
* Bump NUnit.ConsoleRunner version.
* [msbuild] Fix indentation.
* [msbuild] Simplify csproj.
The 32bits **debug** binaries are now too big for Apple's native linker
to process, which gives us (non useful) build errors on the bots.
This will still run the release builds configuration of the tests since
they are smaller and still within the limits of the tooling.
Co-authored-by: Sebastien Pouliot <sebastien.pouliot@gmail.com>
The 32bits **debug** binaries are now too big for Apple's native linker
to process, which gives us (non useful) build errors on the bots.
This will still run the release builds configuration of the tests since
they are smaller and still within the limits of the tooling.
Bump mono to get the new splited test dlls and add them to be ran in
xharness. Special logic is used for mscorlib so we make sure that all
the 'parts' of the test dll do have the same configurations.
Co-authored-by: Manuel de la Pena <mandel@microsoft.com>
Co-authored-by: Waleed Chaudhry <54864665+wachaudh@users.noreply.github.com>
Bump mono to get the new splited test dlls and add them to be ran in
xharness. Special logic is used for mscorlib so we make sure that all
the 'parts' of the test dll do have the same configurations.
Co-Authored-By: Waleed Chaudhry <54864665+wachaudh@users.noreply.github.com>
**Problem**
32 bit tests could only be executed with 64 bit tests. E.g: `run-ios-32-tests` didn't work on its own, only `run-ios-tests,run-ios-32-tests` worked
**Solution**
- Change the logic so that 32 bit tests are only looking for `IncludeiOS32` and 64 bit tests only `IncludeiOS64`
- To keep `run-ios-tests` and `skip-ios-tests` working, hack `SetEnabled` so `IncludeiOS` also sets `IncludeiOS32` and `IncludeiOS64`.
Otherwise the way the code is currently organised we'd have to use `skip-ios-32-tests` and `skip-ios-64-tests`.
Co-authored-by: Vincent Dondain <vidondai@microsoft.com>
After this commit we will have more application but they will be the
appropiate size so that they can be built with the linker for iOS 32b.
It is important to note that the following apps WILL CONTINUE to fail
since the dlls need to be splitted in mono:
* mscorlib tests
* Mono BCL tests group 5 - Which is monotouch_System.Core_xunit-test.dll
and is too large.
After this commit we will have more application but they will be the
appropiate size so that they can be built with the linker for iOS 32b.
It is important to note that the following apps WILL CONTINUE to fail
since the dlls need to be splitted in mono:
* mscorlib tests
* Mono BCL tests group 5 - Which is monotouch_System.Core_xunit-test.dll
and is too large.
After this commit we will have more application but they will be the
appropiate size so that they can be built with the linker for iOS 32b.
It is important to note that the following apps WILL CONTINUE to fail
since the dlls need to be splitted in mono:
* mscorlib tests
* Mono BCL tests group 5 - Which is monotouch_System.Core_xunit-test.dll
and is too large.
Updating the msbuild tasks to use netstandard2.0 requires us to bump NUnit to 3+.
This requires:
* A few code changes due to breaking API changes in NUnit.
* Changes in xharness and a makefile to cope with the new location for the
NUnit console runner (I added a helper script to make things slightly
easier).
**Problem**
32 bit tests could only be executed with 64 bit tests. E.g: `run-ios-32-tests` didn't work on its own, only `run-ios-tests,run-ios-32-tests` worked
**Solution**
- Change the logic so that 32 bit tests are only looking for `IncludeiOS32` and 64 bit tests only `IncludeiOS64`
- To keep `run-ios-tests` and `skip-ios-tests` working, hack `SetEnabled` so `IncludeiOS` also sets `IncludeiOS32` and `IncludeiOS64`.
Otherwise the way the code is currently organised we'd have to use `skip-ios-32-tests` and `skip-ios-64-tests`.
The latest SDK version and the latest OS version does not necessarily have to
match (for instance the iOS 13.2 SDK can support both iOS 13.2 and iOS 13.3),
so keep track of them separately.
Also use the latest OS version to determine which simulator to run, instead of
the latest SDK version (Xcode 11.3 ships with the iOS 13.2 SDK but only has an
iOS 13.3 simulator, not an iOS 13.2 simulator).
Fixes https://github.com/xamarin/maccore/issues/2066.
* Bump mono to a hash with archives and use them.
New commits in mono/mono:
* mono/mono@6af4ae7635 [2019-06][ci] Add Xcode 11.2beta2 for XI/XM Mono SDK builds
Diff: 476d72b9e3..6af4ae7635
* Bump mono to get min iOS version fix.
New commits in mono/mono:
* mono/mono@3775d5ac0a [sdks] Bump min iOS version to 7.0.
Diff: 6af4ae7635..3775d5ac0a
* [xharness] Bump mtouch tests timeout to 3h, we have a couple of new PR bots which are old and slow.
The new PR bots are late 2012 mac minis, so quite slow.
* Implement a different escaping/quoting algorithm for arguments to System.Diagnostics.Process.
mono changed how quotes should be escaped when passed to
System.Diagnostic.Process, so we need to change accordingly.
The main difference is that single quotes don't have to be escaped anymore.
This solves problems like this:
System.ComponentModel.Win32Exception : ApplicationName='nuget', CommandLine='restore '/Users/vsts/agent/2.158.0/work/1/s/tests/sampletester/bin/Debug/repositories/ios-samples/WorkingWithTables/Part 3 - Customizing a Table\'s appearance/3 - CellCustomTable/CellCustomTable.sln' -Verbosity detailed -SolutionDir '/Users/vsts/agent/2.158.0/work/1/s/tests/sampletester/bin/Debug/repositories/ios-samples/WorkingWithTables/Part 3 - Customizing a Table\'s appearance/3 - CellCustomTable'', CurrentDirectory='/Users/vsts/agent/2.158.0/work/1/s/tests/sampletester/bin/Debug/repositories', Native error= Cannot find the specified file
at System.Diagnostics.Process.StartWithCreateProcess (System.Diagnostics.ProcessStartInfo startInfo) [0x0029f] in /Users/builder/jenkins/workspace/build-package-osx-mono/2019-08/external/bockbuild/builds/mono-x64/mcs/class/System/System.Diagnostics/Process.cs:778
ref: https://github.com/mono/mono/pull/15047
* Rework process arguments to pass arrays/lists around instead of quoted strings.
And then only convert to a string at the very end when we create the Process
instance.
In the future there will be a ProcessStartInfo.ArgumentList property we can
use to give the original array/list of arguments directly to the BCL so that
we can avoid quoting at all. These changes gets us almost all the way there
already (except that the ArgumentList property isn't available quite yet).
We also have to bump to target framework version v4.7.2 from v4.5 in several
places because of 'Array.Empty<T> ()' which is now used in more places.
* Parse linker flags from LinkWith attributes.
* [sampletester] Bump to v4.7.2 for Array.Empty<T> ().
* Fix typo.
* Rename GetVerbosity -> AddVerbosity.
* Remove unnecessary string interpolation.
* Remove unused variable.
* [mtouch] Simplify code a bit.
* Use implicitly typed arrays.
This has a couple of advantages:
* It makes it easier to add a catalyst version of these libraries (because it
becomes cumbersome to build for catalyst when the build rules assumes we're
building for both simulator and device).
* It makes it easier to create an xcframework of our libraries, because the
contents in an xcframework is split like this.
* macOS 10.15 starts putting up permission dialogs we can't automatically
dismiss anymore, so start honoring the 'IncludeSystemPermissionTests' option
for macOS tests.
* Improve the 'IncludeSystemPermissionTests' option to have three states: if
set (either true or false), that takes precedence, but if not set, we now
don't run any tests that require permission dialogs on macOS or on device if
we're running in CI. Tests executed locally will still put up dialogs, both
on macOS and on device.
* This needed a few changes to the html report, since the
'IncludeSystemPermissionTests' is exposed in the UI and the code didn't
handle the three different states.
* Update a few tests to check for permission to the contacts.
Fixes https://github.com/xamarin/maccore/issues/1856.
We have tests whose behavior changes when executed on CI, and those tests use
the BUILD_REVISION variable to detect where they're being executed. For this
to work we need to propagate the BUILD_REVISION variable to the test
executable.
Hopefully fixes https://github.com/xamarin/maccore/issues/649 now.
Trying to find a simulator will mark the test as a failure if the simulator
couldn't be found, and we don't want that to happen to ignored tests.
This should fix an issue where xharness seems to try to run the 32-bit
simulator tests when asked to run only device tests.
We have tests whose behavior changes when executed on CI, and those tests use
the BUILD_REVISION variable to detect where they're being executed. For this
to work we need to propagate the BUILD_REVISION variable to the test
executable.
Hopefully fixes https://github.com/xamarin/maccore/issues/649 now.
This boils down to the makefile-generation code having the information it
needs (and that information being correct).
This fixes running of tests on other macOS bots (older/newer), because now we
can build the test package again.
Currently we execute most of the same logic both during the configure phase
and when running tests, and the Harness.Mac value is only set in the configure
phase.
While it doesn't matter right now, this makes sure there aren't any future
surprises in this area, since otherwise we could end up with different
behavior between the configure phase and when running tests.
Harness.AutoConfigureMac now loads all the mac test projects both when
configuring and running tests, the only difference is that the test projects
that must be generated are only generated when configuring. This means that
the Harness.MacTestProject list contains the exact same test projects both
when configuring and when running tests.
This made it possible to remove logic to clone (mac) test projects the Jenkins
class (since Harness.MacTestProjects contains all the test projects already).
Consolidate logic to generate (mac) test projects:
* First we generate BCL and mono-native projects from their templates.
* Then we generate Full/System variations of any project that needs it.
This way we can remove logic to generate Full/System variations from the logic
to generate BCL/mono-native projects, which means less duplicated (and less
confusing) code.
To this purpose, significant changes were required:
* MacTestProject.TargetFrameworkFlavor has been modified to contain a bit mask
of the variations to generate.
* MacMonoNativeInfo has been significantly simplified, and some of the
generated code has been moved to the actual template instead.
* Some project generation (in MacTarget) to make things work as expected.
Fixes https://github.com/xamarin/xamarin-macios/issues/6322.
* Share code with GetInfoPListInclude to find the same Info.plist nodes, so
that FixInfoPListNode also finds existing nodes whose names isn't
"Info.plist".
* Add support for specifying the new Info.plist name.
Rename TestPlatform fields to be more in line with the rest of the code, and
drop the Unified prefix, since everything is Unified now.
* Unified -> Modern
* UnifiedXM45 -> Full
* UnifiedSystem -> System
* Bump VSMac to 8.1.0.2742 to fix msbuild issues (#6279)
* Bump VSMac to 8.1.0.2742 to fix msbuild issues
This is required to get the support for the msbuild `ToolsVersion`
change from `15.0` to `Current`.
* [tests][msbuild] Fix Binding resources test with updated msbuild
Test failure with updated msbuild and vsmac 8.1:
```
Xamarin.iOS.Tasks.NativeReferencesNoEmbedding("iPhone").ShouldNotUnnecessarilyRebuildBindingProject(True)
Binding project build did not create package?
Expected: True
But was: False
at Xamarin.iOS.Tasks.NativeReferencesNoEmbedding.ShouldNotUnnecessarilyRebuildBindingProject (System.Boolean framework) [0x000a0] in <74b8f7d8a53e40109916d305bb4d7403>:0
at (wrapper managed-to-native) System.Reflection.RuntimeMethodInfo.InternalInvoke(System.Reflection.RuntimeMethodInfo,object,object[],System.Exception&)
at System.Reflection.RuntimeMethodInfo.Invoke (System.Object obj, System.Reflection.BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo cul
ture) [0x0006a] in <0519fa732e8845b6a809ce9180f541db>:0
```
The test builds the project multiple times. Before the 3rd build, the project
file's timestamp is updated and expects that the binding package will be
rebuilt. But it is not, because the target `_CreateBindingResourcePackage`
doesn't depend on that project file. So, add that to the target inputs.
* [nuget] Use xibuild to run nuget
Fix errors seen during `nuget restore` for tests:
```
Users/builder/jenkins/workspace/xamarin-macios-pr-builder/tests/xammac_tests/xammac_tests.csproj(213,3): error MSB4024: The imported project file "/Library/Frameworks/Mono.framework/External/xbuild/Xamarin/Mac/Xamarin.Mac.CSharp.targets" could not be loaded. Could not find file "/Library/Frameworks/Mono.framework/External/xbuild/Xamarin/Mac/Xamarin.Mac.CSharp.targets"
```
* [xibuild] Fix incorrect mscorlib.dll being used (#6068)
* [xibuild] Fix incorrect mscorlib.dll being used
The `GuiUnit_NET_4_5` project, when built with `xibuild` uses the wrong `mscorlib.dll`.
From https://github.com/xamarin/xamarin-macios/issues/5760#issuecomment-472457202 :
```
- mscorlib.dll is being used from mono/4.5 and the other system assemblies are from mono/4.5-api
- GuiNet* project is built with xibuild
What is happening here is:
xibuild sets[1] `SetToolsetProperty ("TargetFrameworkRootPath", FrameworksDirectory + Path.DirectorySeparatorChar);`
which points to `/Library/Frameworks/Mono.framework/Versions/Current/lib/mono/xbuild-frameworks`.
This causes $(FrameworkPathOverride) to be set[2] to `/Library/Frameworks/Mono.framework/Versions/Current/lib/mono/xbuild-frameworks/.NETFramework/v4.5`,
but that doesn't have a mscorlib.dll, so it gets reset[3] to /Library/Frameworks/Mono.framework/Versions/5.22.0/lib/mono/4.5/.
If we don't set TargetFrameworkRoothPath, then we get `FrameworkPathOverride = /Library/Frameworks/Mono.framework/Versions/5.22.0/lib/mono/4.5-api`,
causing `_ExplicitReference=/Library/Frameworks/Mono.framework/Versions/5.22.0/lib/mono/4.5-api/mscorlib.dll`(correct one) to be used.
```
Fixes https://github.com/xamarin/xamarin-macios/issues/5760
1. https://github.com/xamarin/xamarin-macios/blob/master/tools/xibuild/Main.cs#L209
2. https://github.com/mono/msbuild/blob/xplat-master/src/Tasks/Microsoft.Common.CurrentVersion.targets#L79
3. https://github.com/mono/msbuild/blob/xplat-master/src/Tasks/Microsoft.Common.CurrentVersion.targets#L84
* Revert "Workaround https://github.com/xamarin/xamarin-macios/issues/5760 in generator csproj"
This reverts commit 9bd927bb7f.
The previous commit for xibuild removes the need for this.
* [xibuild] Handle "incorrectly" cased msbuild property names (#6202)
msbuild property names are case insensitive. While generating the custom
app.config, in `SetToolsetProperty(..)` we try to update the property if
it already exists. But the name lookup was case sensitive, thus causing
the lookup to fail, resulting in two entries for the same property name
differing only in case. Eg. `MSBuildSDKsPath` vs `MSBuildSdksPath`.
Fixed to ignore case.
Fixes https://github.com/mono/mono/issues/14765 .
* Bump VSMac to 8.1.0.2742 to fix msbuild issues
This is required to get the support for the msbuild `ToolsVersion`
change from `15.0` to `Current`.
* [tests][msbuild] Fix Binding resources test with updated msbuild
Test failure with updated msbuild and vsmac 8.1:
```
Xamarin.iOS.Tasks.NativeReferencesNoEmbedding("iPhone").ShouldNotUnnecessarilyRebuildBindingProject(True)
Binding project build did not create package?
Expected: True
But was: False
at Xamarin.iOS.Tasks.NativeReferencesNoEmbedding.ShouldNotUnnecessarilyRebuildBindingProject (System.Boolean framework) [0x000a0] in <74b8f7d8a53e40109916d305bb4d7403>:0
at (wrapper managed-to-native) System.Reflection.RuntimeMethodInfo.InternalInvoke(System.Reflection.RuntimeMethodInfo,object,object[],System.Exception&)
at System.Reflection.RuntimeMethodInfo.Invoke (System.Object obj, System.Reflection.BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo cul
ture) [0x0006a] in <0519fa732e8845b6a809ce9180f541db>:0
```
The test builds the project multiple times. Before the 3rd build, the project
file's timestamp is updated and expects that the binding package will be
rebuilt. But it is not, because the target `_CreateBindingResourcePackage`
doesn't depend on that project file. So, add that to the target inputs.
* [nuget] Use xibuild to run nuget
Fix errors seen during `nuget restore` for tests:
```
Users/builder/jenkins/workspace/xamarin-macios-pr-builder/tests/xammac_tests/xammac_tests.csproj(213,3): error MSB4024: The imported project file "/Library/Frameworks/Mono.framework/External/xbuild/Xamarin/Mac/Xamarin.Mac.CSharp.targets" could not be loaded. Could not find file "/Library/Frameworks/Mono.framework/External/xbuild/Xamarin/Mac/Xamarin.Mac.CSharp.targets"
```
Usually it's when working on beta Xcodes we make mistakes that only show up
when running on older simulators (and devices):
* Missing/wrong availability attributes.
* Tests for new API that don't check the OS if that API is available.
So automatically enable the tests on older simulators for PR builds when using
a beta Xcode.
* [tests] Remove 32-bit Xamarin.Mac tests (both Classic and Unified).
We're removing support for 32-bit Xamarin.Mac apps (#6300), which means we
don't need to run the tests anymore.
This part of the implementation for #6300, I'm starting with the tests because
some of our XM/Classic tests are failing and making the bots unnecessarily red
(since we'll remove XM/Classic support anyway). Also CI will be faster if we
don't run these tests.
* [mmptest] Fix build.
* [tests] Fix build for mono-native-mac.csproj.
* [xharness] Show links to previous test runs in html report in server mode. (#6031)
* [xharness] Change url for server mode.
* [xharness] Show links to previous test runs in html report in server mode.
This makes it much easier to see what failed in a previous test run.
* [XHarness] Show when we have a watch HE0038 error. (#6060)
To simplify the life of the monitorer, if we get a crash with a HE0038
we will propagate the result to the html and will provide a link to the
issue so that it is easy to report it.
* [xharness] Look for HE0038 anywhere in a log file. (#6175)
Most log files now have timestamps, which means that looking for 'HE00380' at
the start of each line won't work anymore.
Fixes an issue where the HE0038 issue isn't detected properly.
This also makes it so that we only parse the log file once (when the test
finishes), instead of parsing every log named 'Run log' every time we produce
the html report.
The change is a pre step to fix https://github.com/xamarin/xamarin-macios/issues/6162
We are removing the not longer child directory bcl-test/BCLTests since
there is not longer any reason for it to exist. The changes make sure
that the tests work as expected in the new directory.
Next step, move all the test apps to their own directory.
Most log files now have timestamps, which means that looking for 'HE00380' at
the start of each line won't work anymore.
Fixes an issue where the HE0038 issue isn't detected properly.
* [Metal] Sprinkle [return: Release] on all 'new*' selectors. Fixes#5941.
Also add tests for all the API I could figure out how to use.
Fixes https://github.com/xamarin/xamarin-macios/issues/5941.
* [tests] MTLFunctionConstantValues didn't have a default ctor until Xcode 9.
* [tests] Use a higher offset when calling MTLBuffer.CreateTexture to try to comply with the requirements for the API.
Hopefully fixes this assertion:
> 07:42:06.7701360 validateStrideTextureParameters:1512: failed assertion `Linear texture: bytesPerRow (64) must be aligned to 256 bytes'
which doesn't happen on my machine.
* Fix whitespace.
* Simplify nested usings.
* Fix availability correctly.
* [XHarness] Remove the old style mscorlib tests.
Remove the old style test and replace it with the xunit equivalent which
has more tests and is provided by the mono package.
* Only skip the mscorlib tests on watchOS devices with 32b. Run them anywhere else.
The tests in the mini project are the same ones found in
monotouch_Mono.Runtime.Tests_test.dll which is already ran via the
autogenerated bcl tests.
Mini tests therefore are redundant and require the mono sources which
will be removed.
* [xharness] Timestamp all the things.
I find myself wanting timestamps if I don't have them much more than wanting
them gone when I do.
Worst case scenario: timestamps can still be disabled for specific logs if
such a desire ever becomes overwhelming.
* [xharness] Don't timestamp xml files.
* [xharness] Refactor a bit to use named types for a few unnamed types with numerous fields.
Anonymous types becomes quite unwieldy the more fields they have.
* [xharness] Remove unnecessary field assignments.
* [xharness] Don't use a dash in the bundle identifer for watchOS projects.
It causes problems with the mscorlib test project, which can't be launched properly.
I'm not sure what's the underlying cause, but here are some of the symptoms:
* The watch app actually shows up fine on the device, but:
* mlaunch isn't notified about the new process, so it thinks the app didn't
launch.
* The new process doesn't receive any environment variables we try to give it,
which for instance means that it won't auto-start the tests upon launch.
* If we ask mlaunch to attach with lldb, mlaunch will ask watchOS to launch
the process in a suspended state while lldb attaches. Yet the watch app
shows up on the device as if not asked to be suspended upon launch.
It seems that the dash (I assume, because I haven't investigated this very
deeply, I just happened to find a solution that worked) makes watchOS launch
the app as if tapped, instead of launched from an IDE.
The strangest part is that this only happens with the mscorlib test project,
not any of the other test projects we run on the watch, and they all have
dashes in their bundle identifiers... yet replacing the dash with another
character (underscore, letter, removing it altogether) all made things work as
expected.
* [monotouch-test] Adjust expected value for watchOS bundle id.
The watchOS bundle ID changed in fc5067ee67, and
the test failure wasn't caught properly.
Trying to find a simulator will mark the test as a failure if the simulator
couldn't be found, and we don't want that to happen to ignored tests.
This should fix an issue where xharness seems to try to run the 32-bit
simulator tests when asked to run only device tests.
If a report already exists, it's probably because a previous attempt failed
for some reason. This is no reason to not try again, overwriting the previous
attempt.
It causes problems with the mscorlib test project, which can't be launched properly.
I'm not sure what's the underlying cause, but here are some of the symptoms:
* The watch app actually shows up fine on the device, but:
* mlaunch isn't notified about the new process, so it thinks the app didn't
launch.
* The new process doesn't receive any environment variables we try to give it,
which for instance means that it won't auto-start the tests upon launch.
* If we ask mlaunch to attach with lldb, mlaunch will ask watchOS to launch
the process in a suspended state while lldb attaches. Yet the watch app
shows up on the device as if not asked to be suspended upon launch.
It seems that the dash (I assume, because I haven't investigated this very
deeply, I just happened to find a solution that worked) makes watchOS launch
the app as if tapped, instead of launched from an IDE.
The strangest part is that this only happens with the mscorlib test project,
not any of the other test projects we run on the watch, and they all have
dashes in their bundle identifiers... yet replacing the dash with another
character (underscore, letter, removing it altogether) all made things work as
expected.
The entire Xml parsing code has been changed to make the following
improvements:
1. Simplify the logic.
2. Ensure that the tests are not added more than once in the human text
log.
3. Do not use the APIs that load the entire tests into memory. XmlReader
is used to read the file and parse the results.
Fixes: https://github.com/xamarin/xamarin-macios/issues/6072
Fixes: https://github.com/xamarin/maccore/issues/1632
Allow to add extra mtouch arguments to the bcl test applications to configure them. This will allow to pass required specific settings that some tests have, for example, for the linker.
This updates the project generation. We cannot yet fully remove the submodule because:
* We are missing the xunit dlls which should be added in the SDK.
* We have not yet remove all the old style tests. Would make the PR huge, better to deal with it in a diff PR.
* The xunit CoreLib tests have issues loading all the tests, needs some extra work and again, the PR is already large.
Fixes: xamarin/maccore#1199Fixes: xamarin/maccore#1204Fixes: xamarin/maccore#1209Fixes: xamarin/maccore#1510
To simplify the life of the monitorer, if we get a crash with a HE0038
we will propagate the result to the html and will provide a link to the
issue so that it is easy to report it.
The change allows to state the tests that have to be ran. ATM with these
changes, the vsts pipeline must add the following to the env vars:
* tvOS device pipelines: Must add run-tvos-tests to the labels.
* iOS device pipelines: Must add run-ios-tests to the labels.
This will ensure that only the tests for the devices are ran and if the
tests pass we get a green build with no unexpected skips.
We had issues in the code that adds a type found in an assembly to
ensure that it was not removed by the linker. This resulted in some
assemblies having 0 tests.
Added the needed ignore for the corlib tests and system ones.
* [xharness] Change url for server mode.
* [xharness] Show links to previous test runs in html report in server mode.
This makes it much easier to see what failed in a previous test run.
This makes it possible to run tests that are marked as 'BuildOnly' to see if
they've been fixed or not.
There's already a 'build' button if only a build is desired.
We automatically create simulators when needed, but it won't work if the
simulator runtime isn't installed. So handle the case where a test might not
have a simulator to execute in correctly.
Add support for running tests with the earliest possible simulator, and use it for introspection tests.
* Add support for running tests with the earliest possible simulator
(currently iOS 8.1, tvOS 9.1 and watchOS 2.0).
* Make the introspection tests run with the earliest possible simulator.
* Fix several binding issues (mostly missing availability attributes), and a
couple of issues in the introspection tests themselves.
Reference: https://github.com/xamarin/xamarin-macios/issues/3668.
The change allows to state the tests that have to be ran. ATM with these
changes, the vsts pipeline must add the following to the env vars:
* tvOS device pipelines: Must add run-tvos-tests to the labels.
* iOS device pipelines: Must add run-ios-tests to the labels.
This will ensure that only the tests for the devices are ran and if the
tests pass we get a green build with no unexpected skips.
We automatically create simulators when needed, but it won't work if the
simulator runtime isn't installed. So handle the case where a test might not
have a simulator to execute in correctly.
* [linker] Mark protocol interfaces when using the dynamic registrar.
Fixes this monotouch-test failure when using the dynamic registrar and the
linker at the same time:
[FAIL] RegistrarTest.TestProtocolRegistration : UIApplicationDelegate/17669
Expected: True
But was: False
* [tests] Adjust test after linker change.
All Xamarin.iOS apps will now link with QuickLook when using the dynamic
registrar, because NSUrl implements a QuickLook protocol:
fcac64ad6e/src/foundation.cs (L5445).
Adjust LinkAll_Frameworks accordingly, and add a new test that verifies that
the old behavior (not linking with QuickLook when linking all assemblies) is
still correct.
* Share much more marshalling code between the dynamic and static registrar
(by refactoring code into separate functions used by both).
* Fix an issue with multiple out/ref parameters in the dynamic registrar.
* Throw an error instead of silently ignoring out/ref parameter types we don't
completely understand in the dynamic registrar.
* Fix returning Class/SEL out/ref parameters in the static registrar.
* Fix returning NSString out/ref parameters in the dynamic registrar.
* Add/fix support for out/ref arrays everywhere.
* Fix support for the various supported out/ref parameter types in the
generator.
* Add lots of tests.
Fixes https://github.com/xamarin/xamarin-macios/issues/5171.
* [tests] Add sample tester.
Add a unit project that looks for iOS/macOS/tvOS sample projects in several
repositories, and builds them all.
* [tests][sampletester] Remove known issue which has now been fixed.
* [tests] Only run sample tests on CI in Azure Devops.
* Remove the possibility of automatically running the sample tests with
xharness (so the sample tests won't run on PR bots or internal bots when the
'run-all-tests' label is added). It's still possible to run the sample tests
manually from the xharness web UI.
* Automatically trigger the sample test run in Azure Devops if the
'run-sample-tests' label is applied to a PR (and that PR is executed on
internal Jenkins).
* Fix typo.
* Fix path.
* Verbose output to track down scheduling failure.
* Bump maccore to get improved debug spew.
Diff: f527c9c526..f89d74b165
* [tests][sampletester] Fix build for TodoWCF.
https://github.com/xamarin/xamarin-macios/pull/4884 introduced the logic of only building certain `RunTestTask`.
This was meant to disable iOS Extensions as part of a fix to https://github.com/xamarin/maccore/issues/1008.
However this didn't quite work and iOS extensions were still running (and failing).
The reason being that `BuildOnly` was set to a `RunDeviceTask` that's added to a list which is then given to `CreateTestVariations` which creates new instances of `RunDeviceTask`.
We now propagate `BuildOnly` to the new variation instance.
This commit improves the state of the BCL testing in the following ways:
1. Improve the device tets running. Less apps, faster results.
2. WatchOS apps are left as they were to ensure that we do not have deplouyment/run issues.
We now support the ignore files per assembly name to simplify the
tracking of the ignored tests. All
* [XHarness] Ensure we do a nuget restore on bcl imported tests. Fixes#5383
We need to ensure that a nuget restore is done, otherwise we might have
build issues with missing libs.
Fixes https://github.com/xamarin/xamarin-macios/issues/5383
Most of the tests are comming from the downloaded SDK, but some of them
have to reman because they are running/passing more tests. This
differences are due to how the tests that come from mono are written
(missing resources etc..)
Please take a look at
https://devdiv.visualstudio.com/DevDiv/_workitems/edit/794210/ for the
list of bcl tests and test number differences.
It's harder to forget setting the define if it's already set by default.
Fixes the mini tests on watchOS device:
[FAIL] JitTests.Exceptions : System.Reflection.TargetInvocationException : Exception has been thrown by the target of an invocation.
----> System.ExecutionEngineException : Attempting to JIT compile method 'ExceptionTests:test_1_basic_filter_catch ()' while running in aot-only mode. See https://docs.microsoft.com/xamarin/ios/internals/limitations for more information.
since BITCODE wasn't defined everywhere it needed to be.
* Instead of calculating a timeout based on the app size and an estimated
transfer speed, keep transferring as long as mlaunch outputs something to
stdout. The actual transfer speed can vary a lot, in particular if
installing to multiple nearby watches simultaneously.
* Improve progress text in the html report during installation to include more
statistics for the impatient.
By default xharness will clean the project after a successful test run (which
can be required if running many device tests without enormous amounts of disk
space).
However, sometimes this can be annoying, in particular if trying to re-run a
particular test manually.
So add an option in the UI to make cleaning optional.
The "Gizmo" and "Companion" are child elements, not attributes on the SimDevicePair.
Also replaced the custom Distinct() implementation with a comparer which can be used with standard LINQ.
The "Gizmo" and "Companion" are child elements, not attributes on the SimDevicePair.
Also replaced the custom Distinct() implementation with a comparer which can be used with standard LINQ.
Previously the queue at the right would move to the top (above the test list)
if a node with a lot of text in the test list was expanded. This is annoying
(since things would move out of view unexpectedly), so change the css so that
the queue on the right stays on the right no matter what.
This also required reordering the left and right divs in the html, which is
most of the diff.
This fixes an issue where we'd consume a thread pool thread until the launch
timeout if the app launched, but the test run never started (it crashed at
launch for instance).
Rework BCL test importation to generate projects that won't compile instead of
giving up completely in case of generation failure.
This means that xharness can still be used for non-BCL tests even if the
generation of the BCL tests fail.
* [tests] Use latest version of NUnit for test projects to get support for parallelized tests.
* [xharness] Automatically find the correct nunit-console executable for NUnit tests.
Jenkins has a limitation where you can't mark a step a failure, it has to
*fail* to be reported as such.
This means that running multiple tests, and reporting a failure if any of
those tests fail is not possible.
We run into this with the normal test run + the docs tests; where we currently
don't show a red result in the UI if any of those fail.
So incorporate the test-docs step into xharness, so that we only have one
thing to in the Jenkinsfile, which makes it possible for us to fail the test
run step properly.
This also required a few upgrades to xharness to get more info for pull
requests, since the logic to enable the docs tests is a bit more complicated
than anything else we have (if the current branch (or the target branch for a
PR) is 'master' AND xamarin mode is enabled).
* [tests] Add test to ensure LLVM succeeds on watchOS for every assembly we ship.
References:
* https://github.com/mono/mono/issues/12131: LLVM failed for 'UIEdgeInsets.Equals': opcode r4_cneq
* https://github.com/mono/mono/issues/12130: [watchOS] MT3001: Could not AOT the assembly mscorlib.dll
* https://github.com/xamarin/xamarin-macios/issues/4763: [RFC] Improve handling of filter clauses with LLVM/Bitcode
* https://bugzilla.xamarin.com/show_bug.cgi?id=58209: LLVM/bitcode can't handle filter clauses
* LLVM failure in mscorlib.dll is now fixed.
* Revert "LLVM failure in mscorlib.dll is now fixed."
This reverts commit ace3c930a9.
* [tests] Run mtouch tests inprocess.
Fixes a remoting exception:
System.Runtime.Remoting.RemotingException: Cannot create channel sink to connect to URL fff8ce5c_3219_4dd0_941f_500dd49e02a5/94f23f2_4.rem. An appropriate channel has probably not been registered.
Server stack trace:
at System.Runtime.Remoting.RemotingServices.GetClientChannelSinkChain (System.String url, System.Object channelData, System.String& objectUri) [0x00019] in <96207d0baa204f48a53ad6be05f5ecba>:0
at System.Runtime.Remoting.RemotingServices.GetOrCreateClientIdentity (System.Runtime.Remoting.ObjRef objRef, System.Type proxyType, System.Object& clientProxy) [0x0001d] in <96207d0baa204f48a53ad6be05f5ecba>:0
at System.Runtime.Remoting.RemotingServices.GetRemoteObject (System.Runtime.Remoting.ObjRef objRef, System.Type proxyType) [0x00000] in <96207d0baa204f48a53ad6be05f5ecba>:0
at System.Runtime.Remoting.RemotingServices.GetProxyForRemoteObject (System.Runtime.Remoting.ObjRef objref, System.Type classToProxy) [0x0001b] in <96207d0baa204f48a53ad6be05f5ecba>:0
at System.Runtime.Remoting.RemotingServices.Unmarshal (System.Runtime.Remoting.ObjRef objectRef, System.Boolean fRefine) [0x0007a] in <96207d0baa204f48a53ad6be05f5ecba>:0
at System.Runtime.Remoting.RemotingServices.Unmarshal (System.Runtime.Remoting.ObjRef objectRef) [0x00000] in <96207d0baa204f48a53ad6be05f5ecba>:0
at System.Runtime.Remoting.ObjRef.GetRealObject (System.Runtime.Serialization.StreamingContext context) [0x0000f] in <96207d0baa204f48a53ad6be05f5ecba>:0
at System.Runtime.Serialization.ObjectManager.ResolveObjectReference (System.Runtime.Serialization.ObjectHolder holder) [0x00010] in <96207d0baa204f48a53ad6be05f5ecba>:0
at System.Runtime.Serialization.ObjectManager.DoFixups () [0x0007f] in <96207d0baa204f48a53ad6be05f5ecba>:0
at System.Runtime.Serialization.Formatters.Binary.ObjectReader.Deserialize (System.Runtime.Remoting.Messaging.HeaderHandler handler, System.Runtime.Serialization.Formatters.Binary.__BinaryParser serParser, System.Boolean fCheck, System.Boolean isCrossAppDomain, System.Runtime.Remoting.Messaging.IMethodCallMessage methodCallMessage) [0x00077] in <96207d0baa204f48a53ad6be05f5ecba>:0
at System.Runtime.Serialization.Formatters.Binary.BinaryFormatter.Deserialize (System.IO.Stream serializationStream, System.Runtime.Remoting.Messaging.HeaderHandler handler, System.Boolean fCheck, System.Boolean isCrossAppDomain, System.Runtime.Remoting.Messaging.IMethodCallMessage methodCallMessage) [0x000a2] in <96207d0baa204f48a53ad6be05f5ecba>:0
at System.Runtime.Serialization.Formatters.Binary.BinaryFormatter.Deserialize (System.IO.Stream serializationStream, System.Runtime.Remoting.Messaging.HeaderHandler handler, System.Boolean fCheck, System.Runtime.Remoting.Messaging.IMethodCallMessage methodCallMessage) [0x00000] in <96207d0baa204f48a53ad6be05f5ecba>:0
at System.Runtime.Serialization.Formatters.Binary.BinaryFormatter.DeserializeMethodResponse (System.IO.Stream serializationStream, System.Runtime.Remoting.Messaging.HeaderHandler handler, System.Runtime.Remoting.Messaging.IMethodCallMessage methodCallMessage) [0x00000] in <96207d0baa204f48a53ad6be05f5ecba>:0
at System.Runtime.Remoting.Channels.BinaryClientFormatterSink.SyncProcessMessage (System.Runtime.Remoting.Messaging.IMessage msg) [0x00083] in <dcf627861d9e44ce818b79d685496eeb>:0
Exception rethrown at [0]:
at (wrapper managed-to-native) System.Object.__icall_wrapper_mono_remoting_wrapper(intptr,intptr)
at (wrapper remoting-invoke) NUnit.Engine.Agents.RemoteTestAgent.Load()
at NUnit.Engine.Runners.ProcessRunner.LoadPackage () [0x00040] in <28c9cb3cd2594f14ad436553a9b77f5d>:0
at NUnit.Engine.Runners.AbstractTestRunner.Load () [0x00000] in <28c9cb3cd2594f14ad436553a9b77f5d>:0
at NUnit.Engine.Runners.MasterTestRunner.LoadPackage () [0x0006b] in <28c9cb3cd2594f14ad436553a9b77f5d>:0
at NUnit.Engine.Runners.MasterTestRunner..ctor (NUnit.Engine.IServiceLocator services, NUnit.Engine.TestPackage package) [0x0007a] in <28c9cb3cd2594f14ad436553a9b77f5d>:0
at NUnit.Engine.TestEngine.GetRunner (NUnit.Engine.TestPackage package) [0x0001e] in <28c9cb3cd2594f14ad436553a9b77f5d>:0
at NUnit.ConsoleRunner.ConsoleRunner.RunTests (NUnit.Engine.TestPackage package, NUnit.Engine.TestFilter filter) [0x00095] in <93782c327b3c4ab5aae551528dfe1cae>:0
at NUnit.ConsoleRunner.ConsoleRunner.Execute () [0x000b6] in <93782c327b3c4ab5aae551528dfe1cae>:0
at NUnit.ConsoleRunner.Program.Main (System.String[] args) [0x001b5] in <93782c327b3c4ab5aae551528dfe1cae>:0
* [xharness] Run mtouch tests in-process.
* [tests] Don't run the LLVM watchOS tests if we didn't build for device.
* [tests] Update NoLLVMFailuresInWatchOS known failures according to latest mono bump.
We now download the ios Mono sdk which means that there is no need for
us to build the test assemblies on iOS, WatchOS and tvOS BUT we need to
do so for the Mac tests.
The change includes:
1. Use fo the downloaded test assemblies.
2. Removal of the dependency of building them in the tests.
3. Move back to using reflection for the project generation.
4. Remove the cached project details added in the workaround.
- Existing binding projects embed the native libraries within the assembly as managed resource
- This does not scale well and has performance implications
- This PR creates a new property, NoBindingEmbedding which when true processes the building and consumption of binding projects differently.
- Existing binding projects are not affected, they will continue as is
- I've written a full XM test suite and ported a subset to iOS. Since iOS only supports checked in projects, and I didn't want to make the existing situation worse by adding more, I only wrote tests that could use the existing test projects.
-When we complete some form of msbuild testing reform, we'll revisit these tests.
- Remove two files in MyiOSFrameworkBinding that are not used (we use copies elsewhere)
- Remove unnecessary sleep and fix broken touch command
- Output failing test log to console instead of test output
- VSfM does not handle thousands of lines of test failure message well
- Add ability to generate binding projects with LinkWith
This change adds support to execute the tests provided by mono as assemblies.This includes:
1. App generation to run the bcl tests on modern and full.
2. Needed workaround to delay the compilation of the assemblies until we have them from the SDK.
3. All failing tests are ignored.
* [mtouch] do not strip assemblies when interpreter is used
* [xharness] add release mixed-mode configuration
* [xharness] enable debug mixed-mode config for mscorlib
* [mtouch] add link to mono issue
Apparently Process.HasExited may throw exceptions, so handle those gracefully
(by ignoring them completely):
Unhandled Exception:
System.AggregateException: One or more errors occurred. (No process is associated with this object.) ---> System.InvalidOperationException: No process is associated with this object.
at System.Diagnostics.Process.EnsureState (System.Diagnostics.Process+State state) [0x00018] in <d012d9eca6d14975a47488863de2f4c6>:0
at System.Diagnostics.Process.get_HasExited () [0x0000b] in <d012d9eca6d14975a47488863de2f4c6>:0
at (wrapper remoting-invoke-with-check) System.Diagnostics.Process.get_HasExited()
at xharness.Process_Extensions+<>c__DisplayClass2_0.<RunAsync>b__2 () [0x00001] in <285dbdcf9e034cd496a3fef953fac640>:0
at System.Threading.CancellationToken.ActionToActionObjShunt (System.Object obj) [0x00000] in <96207d0baa204f48a53ad6be05f5ecba>:0
at System.Threading.CancellationCallbackInfo.ExecutionContextCallback (System.Object obj) [0x00007] in <96207d0baa204f48a53ad6be05f5ecba>:0
at System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state, System.Boolean preserveSyncCtx) [0x00071] in <96207d0baa204f48a53ad6be05f5ecba>:0
at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state, System.Boolean preserveSyncCtx) [0x00000] in <96207d0baa204f48a53ad6be05f5ecba>:0
at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state) [0x0002b] in <96207d0baa204f48a53ad6be05f5ecba>:0
at System.Threading.CancellationCallbackInfo.ExecuteCallback () [0x00024] in <96207d0baa204f48a53ad6be05f5ecba>:0
at System.Threading.CancellationTokenSource.CancellationCallbackCoreWork (System.Threading.CancellationCallbackCoreWorkArguments args) [0x00042] in <96207d0baa204f48a53ad6be05f5ecba>:0
at System.Threading.CancellationTokenSource.ExecuteCallbackHandlers (System.Boolean throwOnFirstException) [0x000c0] in <96207d0baa204f48a53ad6be05f5ecba>:0
--- End of inner exception stack trace ---
at System.Threading.CancellationTokenSource.ExecuteCallbackHandlers (System.Boolean throwOnFirstException) [0x00132] in <96207d0baa204f48a53ad6be05f5ecba>:0
at System.Threading.CancellationTokenSource.NotifyCancellation (System.Boolean throwOnFirstException) [0x0005f] in <96207d0baa204f48a53ad6be05f5ecba>:0
at System.Threading.CancellationTokenSource.Cancel (System.Boolean throwOnFirstException) [0x00006] in <96207d0baa204f48a53ad6be05f5ecba>:0
at System.Threading.CancellationTokenSource.Cancel () [0x00000] in <96207d0baa204f48a53ad6be05f5ecba>:0
at xharness.AppRunner+<>c__DisplayClass73_0.<RunAsync>b__0 (System.Object v) [0x00029] in <285dbdcf9e034cd496a3fef953fac640>:0
at System.Threading.QueueUserWorkItemCallback.WaitCallback_Context (System.Object state) [0x00007] in <96207d0baa204f48a53ad6be05f5ecba>:0
at System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state, System.Boolean preserveSyncCtx) [0x00071] in <96207d0baa204f48a53ad6be05f5ecba>:0
at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state, System.Boolean preserveSyncCtx) [0x00000] in <96207d0baa204f48a53ad6be05f5ecba>:0
at System.Threading.QueueUserWorkItemCallback.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem () [0x00021] in <96207d0baa204f48a53ad6be05f5ecba>:0
at System.Threading.ThreadPoolWorkQueue.Dispatch () [0x00074] in <96207d0baa204f48a53ad6be05f5ecba>:0
at System.Threading._ThreadPoolWaitCallback.PerformWaitCallback () [0x00000] in <96207d0baa204f48a53ad6be05f5ecba>:0
---> (Inner Exception #0) System.InvalidOperationException: No process is associated with this object.
at System.Diagnostics.Process.EnsureState (System.Diagnostics.Process+State state) [0x00018] in <d012d9eca6d14975a47488863de2f4c6>:0
at System.Diagnostics.Process.get_HasExited () [0x0000b] in <d012d9eca6d14975a47488863de2f4c6>:0
at (wrapper remoting-invoke-with-check) System.Diagnostics.Process.get_HasExited()
at xharness.Process_Extensions+<>c__DisplayClass2_0.<RunAsync>b__2 () [0x00001] in <285dbdcf9e034cd496a3fef953fac640>:0
at System.Threading.CancellationToken.ActionToActionObjShunt (System.Object obj) [0x00000] in <96207d0baa204f48a53ad6be05f5ecba>:0
at System.Threading.CancellationCallbackInfo.ExecutionContextCallback (System.Object obj) [0x00007] in <96207d0baa204f48a53ad6be05f5ecba>:0
at System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state, System.Boolean preserveSyncCtx) [0x00071] in <96207d0baa204f48a53ad6be05f5ecba>:0
at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state, System.Boolean preserveSyncCtx) [0x00000] in <96207d0baa204f48a53ad6be05f5ecba>:0
at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state) [0x0002b] in <96207d0baa204f48a53ad6be05f5ecba>:0
at System.Threading.CancellationCallbackInfo.ExecuteCallback () [0x00024] in <96207d0baa204f48a53ad6be05f5ecba>:0
at System.Threading.CancellationTokenSource.CancellationCallbackCoreWork (System.Threading.CancellationCallbackCoreWorkArguments args) [0x00042] in <96207d0baa204f48a53ad6be05f5ecba>:0
at System.Threading.CancellationTokenSource.ExecuteCallbackHandlers (System.Boolean throwOnFirstException) [0x000c0] in <96207d0baa204f48a53ad6be05f5ecba>:0 <---
Recursively printing diagnostics can leave us with _many_ lldb processes, each
pretty much hanging, and the system overloaded, which, in the worst case
scenario, would require a hard reboot.
Most commonly the "only" thing that happens is an OutOfMemoryException when
the OS eventually tells xharness NO to launching new processes.
Fixes https://github.com/xamarin/maccore/issues/1163.
That way the report is still available even if xharness dies.
Also improve report generation to write directly to a temporary file on disk,
instead of writing to memory and then writing to disk (improves memory usage).
* [Xharness] Add a workaround to not build the xunit tests when not needed.
This is a workaround that does not use reflection to build the bcl test
projects. We need this until we have complete support of using mono as
an SDK.
* [Harness] Reduce the noise in the make files.
The Makefile.cs code generates a lot of noise becuase it tries to build
a make rule for the new BCL tests, these rules do not work and can be
ignored.
Fixes issue: https://github.com/xamarin/maccore/issues/1156
* Use a full path to xibuild.
Use a full path to xibuild everywhere, since it's easier than making sure PATH
is correct every time we want to invoke xibuild.
Also remove the xbuild-in-place script, it's not used anymore.
* Fix xibuild path lookup.
* [xammac_tests] Remove unneeded csproj changes.
Only build the BCL tests when we need them, because we can't build them when
we're running the device tests (because we don't have a built source tree in
that case).
Since the test assemblies are required when generating the BCL projects, this
means we also delay creating the BCL projects until we need them.
* xibuild: New wrapper tool to run msbuild or managed executables
MSBuild supports fallback paths for projects imported using
`$(MSBuildExtensionsPath)`, but these must be specified explicitly in
the app.config of the main executable. There was a PR to allow use of
properties for this in the app.config, but that was not accepted
upstream.
This is required for being able to:
1. build projects with msbuild against the in-tree XI/XM build output
2. and to run nunit tests against the same.
For this we introduce a new tool, `xibuild`, based on XA's `xabuild`.
This supports the fallback paths to be specified via the environment variable
`MSBuildExtensionsPathFallbackPathsOverride`[1].
It essentially operates in 3 modes:
1. `xibuild -c /path/to/foo.exe`
Generates /path/to/foo.exe.config with the fallback paths inserted into that.
2. `xibuild -- /v:diag /path/to/project.csproj`
Runs msbuild with the arguments after `--` with a custom app.config based on
`MSBuild.dll.config`, with the fallback paths correctly inserted.
This is in a temporary file and the original config file is not touched.
3. `xibuild -t -- /path/to/managed_tool.exe args`
Generates `/path/to/managed_tool.exe.config` based on `MSBuild.dll.config` with
the fallback paths inserted, and runs `managed_tool.exe` with the arguments.
The default is to overwrite the config file.
But there is also a switch to merge it with an existing config file.
--
1. Value of the environment variable $MSBuildExtensionsPathFallbackPathsOverride
is prepended to any existing list of search paths in `MSBuild.dll.config`, IOW,
it takes precedence. So, the order of lookup becomes:
- Value of the property `$(MSBuildExtensionsPath)`
- Value of the environment variable `$MSBuildExtensionsPathFallbackPathsOverride`
- /Library/Frameworks/Mono.framework/External/xbuild on macOS
* Integrate use of `xibuild` with the tests
Update all uses of `msbuild` and invocations of tools like nunit that
might depend on using the in-tree builds to use `xibuild`.
* xibuild: Move help descriptions to OptionSet itself.
Mono know provides a set of testing assemblies that contain the tests to be ran in each of the supported platforms. This commit adds the following:
1. Two different runners that can be used to execute NUnit and xUnit tests.
2. The necessary glue to get xharness to run the new tests and report the results.
3. A new test that will ensure that if mono adds new tests, xharness will report failures if the tests are not run or not ignored.
Make sure we don't run 'nuget restore' for test-mac.sln multiple times in
parallel by creating a separate make target that runs 'nuget restore', and
making every build-mac-* target depend on that target.
Fixes https://github.com/xamarin/maccore/issues/1122.
* [tests] Point MSBuild to the right Xamarin.Mac location when building packaged Xamarin.Mac tests. Fixes maccore#1115.
Fixes https://github.com/xamarin/maccore/issues/1115.
* [xharness] Ensure all makefile targets set the proper environment variables.
While trying to debug an msbuild unit tests it was not possible to
successfully copy/paste the command from the execution log and
run it in a shell.
It's not that hard to run xharness and figure it out - but the
same can happen on bots (which could be harder).
So that little change prints out the host and xharness changes
to the environment variables to make copy/pasters life even
lazier :)
Unce upon a time we used a separate mono submodule for watchOS support, to make
development of watchOS support easier (we referenced mono/master, to avoid
backporting fixes for watchOS support through various release branches in
mono).
This only worked until our watchOS support became stable, since then we had to
start using a stable version of mono for watchOS support.
This means that our build support for using a separate mono clone for watchOS
support is no longer needed; and in any case it's broken because of build
changes done later.
* Bump to mono:2018-06
* Bump mono
* Updates compression to work with the public span
* Bump mono
* Fixes pointer check logic in Deflater
* Bump mono
* Fixes pointer check logic in Deflater
* Bump mono
* Bump Mono
* [runtime] always use `mono_jit_set_aot_mode` (#4491)
`mono_jit_set_aot_only` is deprecated and accidentally broke with
https://github.com/mono/mono/pull/7887
This should fix device tests with `mono-2018-06`
* Testing with Zoltan's patch
* Include libmono-system-native on Xamarin.Mac
* Bump Mono
Commit list for mono/mono:
* mono/mono@7bcda192a0 Bump llvm to release_60/fc854b8ec5873d294b80afa3e6cf6a88c5c48886. (#9786). (#9804)
* mono/mono@23e95ec7ad Apply F# portable pdb debug fix for pinvokes & bump (#9797)
* mono/mono@295f6d32af [2018-06] [MacOS] On Mac, use the copyfile API to copy files (#9696)
Diff: 7d5f4b6136...7bcda192a0
* Revert 4bacab3d5c, it doesn't fix the ios aot problems.
* Bump mono
* [tests] Adjust the MT0137 test for mcs change in behavior.
Starting with mono 5.16 mcs will now add assembly references when the assembly
is only used in attributes (this was already the case for csc in both 5.14 and
5.16, so it seems to be a compatibility change).
Adjust the MT0137 test accordingly.
* [msbuild] Fix parsing of json parser errors to handle trailing periods in the error message.
Fixes this test:
1) Test Failure : Xamarin.iOS.Tasks.Bug60536.TestACToolTaskCatchesJsonException
ColumnNumber
Expected: 2
But was: 0
* Bump mono
* [builds] Install the old llvm binaries into the LLVM36 directory and make the 32 bit builds use that.
* Bump mono
* Bump mono
* [jenkins] Don't give VSTS a fake branch. (#4667)
Something in VSTS changed, and now fake branch names don't work anymore.
So instead use real branch names (and for pull requests I've created a
'pull-request' branch we can use).
* Assembly.LoadFile accepts only absolute path
* [linker] Add new Facade (System.Threading.Tasks.Extensions).
Fixes these MTouch test failures:
1. Xamarin.Linker.SdkTest.iOS_Unified : Facades
Expected:
But was: < "System.Threading.Tasks.Extensions" >
2. Xamarin.Linker.SdkTest.tvOS : Facades
Expected:
But was: < "System.Threading.Tasks.Extensions" >
3. Xamarin.Linker.SdkTest.watchOS : Facades
Expected:
But was: < "System.Threading.Tasks.Extensions" >
* [mono-sdks] Necessary changes to unify the LLVM provisioning for both iOS and Android. (#4732)
* Bump Mono
* [mtouch] add mixed-mode support (#4751)
* [mtouch] add --interp-mixed option
When enabling this option, mtouch will AOT compile `mscorlib.dll`. At
runtime that means every method that wasn't AOT'd will be executed by
the runtime interpreter.
* [mtouch] Add support to --interpreter to list the assemblies to (not) interpret.
* [msbuild] Simplify interpreter code to use a single variable.
* Fix whitespace.
* [mtouch] Move mtouch-specific code to mtouch-specific file.
* [msbuild] An empty string is a valid value for 'Interpreter', so make it a non-required property.
* [mtouch] Add sanity check for aot-compiling interpreted assemblies.
* Bump Mono
* [linker] Updates SDKs facades list
* Bump mono
* [msbuild] Adds facades which might override default nuget version to framework list
The collision resolver task reads them from here https://github.com/dotnet/sdk/blob/master/src/Tasks/Common/ConflictResolution/FrameworkListReader.cs
* Bump to a VSfM version that can build XM Classic projects.
Přemek Vysoký
Manuel de la Pena
Rolf Bjarne Kvinge
Alex Soto
monojenkins
Sebastien Pouliot
Pramit Mallick
Vincent Dondain
Ankit Jain
Sebastien Pouliot
Alexander Köplinger
Bernhard Urban
Bernhard Urban
Marek Safar
Chris Hamons
Ludovic Henry
Martin Baulig
Jo Shields