* [XHarness] Ensure we do a nuget restore on bcl imported tests. Fixes#5383
We need to ensure that a nuget restore is done, otherwise we might have
build issues with missing libs.
Fixes https://github.com/xamarin/xamarin-macios/issues/5383
The previous implementation was broken, because:
* We called GetIsARM64CallingConvention in Runtime's static constructor.
* Runtime.Arch hadn't been set yet in the static constructor, which means it
was Arch.DEVICE (0).
* This meant GetIsARM64CallingConvention would return true on iOS/tvOS x86-64
simulator.
So delay the initialization of the IsARM64CallingConvention field until we've
been initialized.
In some cases, httpbin might return errors or other results. In that
case, we cannot assert that the auth headers are not present. We set the
test to inconclusive to ensure that we do not fail due to the external
page.
Fixes https://github.com/xamarin/maccore/issues/1453
After commit 2619d8f5ca the System.Core
tests does not longer exists. Which makes the mtouch test that depends
on it, fail too. This changes the test to use a present test.
* Fixes https://devdiv.visualstudio.com/DevDiv/_workitems/edit/801867
System.Memory.dll not being copied anywhere when Roslyn package is referenced
The issue was that the XM FrameworkList was shared for the 2 profiles and there are differences.
The Full profile for instance doesn't have `System.Memory` and that causes some issues with Nuget package references.
* Updated tests to report issues against the XM Full and Mobile Framework lists
* Now using 2 separate XM FrameworkList files (updated makefile) since the list of assemblies is different and that's expected by the mono team.
Instead download the assemblies (part of existing bundle.zip) of the
current release and generate the XML locally (no storage in github)
before doing the diff against the newer, just built, assemblies.
reference: https://github.com/xamarin/xamarin-macios/issues/4891
Most of the tests are comming from the downloaded SDK, but some of them
have to reman because they are running/passing more tests. This
differences are due to how the tests that come from mono are written
(missing resources etc..)
Please take a look at
https://devdiv.visualstudio.com/DevDiv/_workitems/edit/794210/ for the
list of bcl tests and test number differences.
* We only need to do a runtime check on watchOS.
* On watchOS assume we're on a ARM64-based CPU as long as we're not on a
ARMv7k CPU.
Fixes an issue where we failed to detect a 64-bit iOS CPU as ARM64-based (in
particular iPhone X's ARM64v8).
The Mono AOT compiler maintains a set of signatures of compiled methods.
Those signatures are necessary to emit wrappers to enable the transition
from interpreter->AOT code. Thus, they must be collected for each
assembly.
* Improve the behavior when we fail to download the .dvtdownloadableindex file
to report a "Not Found" if we get a "Forbidden" (403) response, since that's
what it really is.
* Improve the behavior when we fail to download the .dvtdownloadableindex file
to check if the requested simulators are already installed instead of always
failing. This makes it possible to manually install the required simulators
if needed (Apple might not have published simulators for the Xcode version
we're using), and this would allow any checks to pass if the required
simulators are already installed.
* Improve the code to modify the PackageInfo XML to not use simple string
replacement (it's too fragile), use proper XML parsing instead.
* Fix "time left" math, I have no idea what my previous math was doing.
Fixes#5639
The results reported in the xharness run do not add up. Total tests
should include the number of skipped tests. This change ensures that the
tests results correctly add up.
Example nunit fixed tests: [NUnit] Mono SystemIOCompressionTests (total should
be 24, passed 4, ignored 20).
Example xunit fixed tests: [xUnit] Mono SystemDataXunit (total should be
1061, passed 1054, ignored 7).
Fixes https://github.com/xamarin/xamarin-macios/issues/5639
Clang will automatically compile .bc files into object code if needed, but
it's done serially. If we do the compilation ourselves, it'll be parallelized.
This makes the dontlink tests build in half the time when building for watchOS
/ Release (it drops from ~15 minutes to ~7 minutes).
It's harder to forget setting the define if it's already set by default.
Fixes the mini tests on watchOS device:
[FAIL] JitTests.Exceptions : System.Reflection.TargetInvocationException : Exception has been thrown by the target of an invocation.
----> System.ExecutionEngineException : Attempting to JIT compile method 'ExceptionTests:test_1_basic_filter_catch ()' while running in aot-only mode. See https://docs.microsoft.com/xamarin/ios/internals/limitations for more information.
since BITCODE wasn't defined everywhere it needed to be.
The AOT compiler uses the 'outfile' as the base for a temporary .bc file. If
it's not set, the path to the assembly is used as the base instead. This does
not work if we compile the same assembly to multiple architectures (for
instance armv7k+arm64_32), because the temporary file will clash between those
AOT compilations.
This is not a problem for iOS (armv7+armv7s) because we don't compile to
bitcode (.bc files) on iOS.
* [mtouch] Don't use the native linker to create fat executables.
Don't use the native linker to create fat executables, instead link each
architecture separately, and then manually lipo everything together at the
end. This requires a few changes since we need to keep track of the linker
flags per architecture.
The problem is that bitcode files (.bc) do not correspond with a particular
architecture, so the linker can't distinguish between .bc files for armv7k and
.bc files for arm64_32. So if we pass all together to the linker, the linker
will add all .bc files to both architectures, thus duplicating everything (and
the linking fails with duplicate symbols errors).
* [mtouch] Fix building symlinked simulator executables.
* [mtouch] Fix several assumptions about each Target only producing a single executable.
The current code did not consider that overrides could be swept later,
if unused/unmarked by the linker, so we were missing opportunities to
make methods as final.
* Instead of calculating a timeout based on the app size and an estimated
transfer speed, keep transferring as long as mlaunch outputs something to
stdout. The actual transfer speed can vary a lot, in particular if
installing to multiple nearby watches simultaneously.
* Improve progress text in the html report during installation to include more
statistics for the impatient.
Also change output to use the full path to files as nodes, instead of just the
filename, and instead use a label to set what's shown to just the filename.
This makes the graph correct when we have multiple files with the same name,
but different paths.
By default xharness will clean the project after a successful test run (which
can be required if running many device tests without enormous amounts of disk
space).
However, sometimes this can be annoying, in particular if trying to re-run a
particular test manually.
So add an option in the UI to make cleaning optional.
Alexander Köplinger
Manuel de la Pena
Rolf Bjarne Kvinge
Vincent Dondain
Sebastien Pouliot
Chris Hamons
Bernhard Urban