* [XHarness] Remove the old style mscorlib tests.
Remove the old style test and replace it with the xunit equivalent which
has more tests and is provided by the mono package.
* Only skip the mscorlib tests on watchOS devices with 32b. Run them anywhere else.
This fixes an issue with the api comparison since the api comparison fails if
it detects unexpected modified files: https://github.com/xamarin/xamarin-macios/pull/6133#issuecomment-496283224.
Putting the temporary files in APIDIFF_DIR makes sure the api comparison
doesn't see those files as unexpectedly modified.
* [xharness] Refactor a bit to use named types for a few unnamed types with numerous fields.
Anonymous types becomes quite unwieldy the more fields they have.
* [xharness] Remove unnecessary field assignments.
Mono did remove all the tests in a number of assemblies on
ios/tvos/watchos. With a recent change, we report test runs with no
tests as a failure (correctly since it will bring up issues with the
runners).
In this case, the tree assemblies have to be ignored because they trully
do not have tests and the runners are doing the right thing.
Fixes: https://github.com/xamarin/maccore/issues/1652
We can't assume that the cached `lipo` output is part of the `.app`
since the directory for the later could be different.
E.g. we build two .csproj in out `mmptest` inside the same directory
and they share the same `obj` directory (used for caching) but have
different output directories
```
./obj/Release/mmp-cache/libmono-native.dylib
./bin/Release/UnifiedExample.app/Contents/MonoBundle/libmono-native.dylib
./bin/Release/XM45Example.app/Contents/MonoBundle/libmono-native.dylib
```
Without an update building the XM45 project would not include the
**required** `libmono-native.dylib` library and would crash when
executed.
So far this only applies to `QTKit`...
XM will now, by default, avoid natively link with QTKit unless it's
instructed to so explicitly using `--link-prohibited-frameworks`
ref: https://github.com/xamarin/xamarin-macios/issues/6039
* [apidiff] Add rule to get mono-api-info.exe and mono-api-html.exe.
This fixes an issue when bumping mono: when bumping mono, the already
downloaded mono archive isn't applicable (because we've changed to the
previous commit, which has the previous mono hash, whose archive hasn't been
downloaded).
So add a rule to get mono-api-info.exe and mono-api-html.exe, by downloading
the current mono archive.
* [apidiff] Change the name of the unzip stamp and download dir to contain the hash.
This way the logic doesn't get confused when the hash changes (or there's an
old unzip stamp in the directory), and things are downloaded again as
expected.
* [apidiff] Make make not delete temporary files.
Things end up confused if temporary files have been removed by make, but our
stamp file that the temporary files are still there is present.
* [apidiff] No need to make everything depend on the bundled zip.
If everything that needs the bundled zip already depends on it.
* [apidiff] Restore original hash before calculating api diff.
This makes it less annoying when the api diff calculation changes, because
with the previous behavior they were impossible to test in a PR, since any
changes wouldn't take effect until after the PR was merged.
Allow to add extra mtouch arguments to the bcl test applications to configure them. This will allow to pass required specific settings that some tests have, for example, for the linker.
This updates the project generation. We cannot yet fully remove the submodule because:
* We are missing the xunit dlls which should be added in the SDK.
* We have not yet remove all the old style tests. Would make the PR huge, better to deal with it in a diff PR.
* The xunit CoreLib tests have issues loading all the tests, needs some extra work and again, the PR is already large.
Fixes: xamarin/maccore#1199Fixes: xamarin/maccore#1204Fixes: xamarin/maccore#1209Fixes: xamarin/maccore#1510
This allows the optimization to be disabled in cases where one, or
many, a custom attribute(s) are required by the application at runtime.
While not ideal disabling this single step is much better than disabling
linking for the whole application.
A better approach is described in https://github.com/xamarin/xamarin-macios/issues/6048
but this configuration optimization makes sense independently of it.
Fix https://github.com/xamarin/xamarin-macios/issues/3655
```
Tuning.cs(300,8): warning CS0108: 'LoadOptionalReferencesStep.ProcessReferences(AssemblyDefinition)' hides inherited member 'LoadReferencesStep.ProcessReferences(AssemblyDefinition)'. Use the new keyword if hiding was intended. [/Users/poupou/git/master/xamarin-macios/tools/mmp/mmp.csproj]
```
it's not virtual and it's not being called back annyway.
This allows the optimization to be disabled in cases where one, or
many, a custom attribute(s) are required by the application at runtime.
While not ideal disabling this single step is much better than disabling
linking for the whole application.
A better approach is described in https://github.com/xamarin/xamarin-macios/issues/6048
but this configuration optimization makes sense independently of it.
Fix https://github.com/xamarin/xamarin-macios/issues/3655
Use for producing API diff for release notes without waiting for a PR,
bots and/or approvals...
Also useful to produce API diff between any versions, not just between
the current revision and a baseline (last stable).
We had issues in the code that adds a type found in an assembly to
ensure that it was not removed by the linker. This resulted in some
assemblies having 0 tests.
Added the needed ignore for the corlib tests and system ones.
An mlaunch fix in master required a code change in xamarin-macios; when
backporting this mlaunch fix to d16-2 the corresponding xamarin-macios fix was
not, causing the build to break.
Fixes this:
[...]
Build FAILED.
"/Users/builder/jenkins/workspace/xamarin-macios/maccore/tools/mlaunch/Xamarin.Hosting/Xamarin.Hosting.sln" (default target) (1) ->
"/Users/builder/jenkins/workspace/xamarin-macios/maccore/tools/mlaunch/Xamarin.Hosting/Xamarin.Hosting.csproj" (default target) (2) ->
(CoreCompile target) ->
/Users/builder/jenkins/workspace/xamarin-macios/xamarin-macios/tools/common/MachO.cs(10,15): error CS0234: The type or namespace name 'Bundler' does not exist in the namespace 'Xamarin' (are you missing an assembly reference?) [/Users/builder/jenkins/workspace/xamarin-macios/maccore/tools/mlaunch/Xamarin.Hosting/Xamarin.Hosting.csproj]
0 Warning(s)
1 Error(s)
Time Elapsed 00:00:03.63
make[3]: *** [Xamarin.Hosting/Xamarin.Launcher/bin/Debug/mlaunch.app] Error 1
The arm64_32 slice for watchOS apps will always use the 'unified' mode, while
the armv7k can be both 'unified' and 'compat' depending on the deployment
target, so we need to keep track of this per Target.
This PR does not change anything related to arm64_32, that will come in a
later PR.
The arm64_32 slice for watchOS apps will always use the 'unified' mode, while
the armv7k can be both 'unified' and 'compat' depending on the deployment
target, so we need to keep track of this per Target.
This PR does not change anything related to arm64_32, that will come in a
later PR.
* [mtouch/mmp] Split out the RunCommand[Async] methods to a separate file so that the generator can reuse more easily.
* [generator] Show proper errors when failing to compile something.
* Fix grammar
* [linker] Mark protocol interfaces when using the dynamic registrar.
Fixes this monotouch-test failure when using the dynamic registrar and the
linker at the same time:
[FAIL] RegistrarTest.TestProtocolRegistration : UIApplicationDelegate/17669
Expected: True
But was: False
* [tests] Adjust test after linker change.
All Xamarin.iOS apps will now link with QuickLook when using the dynamic
registrar, because NSUrl implements a QuickLook protocol:
fcac64ad6e/src/foundation.cs (L5445).
Adjust LinkAll_Frameworks accordingly, and add a new test that verifies that
the old behavior (not linking with QuickLook when linking all assemblies) is
still correct.
Before we unzip, we remove the target directory. This is a bad idea if the
target directory is also used for other things: in particular it breaks
parallel make if some other target tries to write to the temporary directory.
Instead unzip downloaded files into a subdirectory exclusively used by those
unzipped files, which means we can remove at will before unzipping.
We often (e.g. previews, service releases) update the API diff during
a release cycle. The current code generated a new GUID every time, which
is not what correct since it's the same document.
This uses an MD5 digest of the filename as the source of the GUID so
it will remain constant once created (and updated).
We often (e.g. previews, service releases) update the API diff during
a release cycle. The current code generated a new GUID every time, which
is not what correct since it's the same document.
This uses an MD5 digest of the filename as the source of the GUID so
it will remain constant once created (and updated).
* Share much more marshalling code between the dynamic and static registrar
(by refactoring code into separate functions used by both).
* Fix an issue with multiple out/ref parameters in the dynamic registrar.
* Throw an error instead of silently ignoring out/ref parameter types we don't
completely understand in the dynamic registrar.
* Fix returning Class/SEL out/ref parameters in the static registrar.
* Fix returning NSString out/ref parameters in the dynamic registrar.
* Add/fix support for out/ref arrays everywhere.
* Fix support for the various supported out/ref parameter types in the
generator.
* Add lots of tests.
Fixes https://github.com/xamarin/xamarin-macios/issues/5171.
* [builds] Add support for using cached downloads of the mono archives from ~/Library/Caches.
* [apidiff] Add support for using cached downloads of apidiff bundle from ~/Library/Caches.
* [builds] Move the facade check targets down in the Makefile.
Move the facade check targets below the declaration of their prerequisite
variables (*_BCL_TARGETS), since otherwise the prerequisite variable will be
empty when the facade check targets are read by make, they end up with no
prerequisites at all, and the targets fail.
* [builds] Make the tools build use mono's packaged logic instead of our own.
Make the 'tools64' build use mono's packaged build logic instead of our own.
This is the first step to consuming the BCL from the mono archive.
Also completely refactor the 'tools64' build by removing everything we don't
need and renaming it to 'bcl' (since that's more representative of what it
does).
* [apidiff] Unzip downloaded files into a temporary subdirectory.
Before we unzip, we remove the target directory. This is a bad idea if the
target directory is also used for other things: in particular it breaks
parallel make if some other target tries to write to the temporary directory.
Instead unzip downloaded files into a subdirectory exclusively used by those
unzipped files, which means we can remove at will before unzipping.
* [apidiff] Use mono-api-[info|html].exe from the mono archive.
Some code now need to be initialized a few lines earlier... otherwise we
end up with an error like:
```
/Library/Frameworks/Xamarin.Mac.framework/Versions/Current/bin/mmp --version
error MM0000: Unexpected error - Please file a bug report at https://github.com/xamarin/xamarin-macios/issues/new
System.InvalidOperationException: Nullable object must have a value.
at System.Nullable`1[T].get_Value () [0x0000d] in /Users/builder/jenkins/workspace/build-package-osx-mono/2019-02/external/bockbuild/builds/mono-x64/external/corefx/src/Common/src/CoreLib/System/Nullable.cs:48
at Xamarin.Bundler.Driver.get_TargetFramework () [0x00001] in /Users/poupou/git/xamarin/xamarin-macios/tools/common/Driver.cs:198
at Xamarin.Bundler.Application.get_Platform () [0x00001] in /Users/poupou/git/xamarin/xamarin-macios/tools/common/Application.cs:62
at Xamarin.Bundler.RuntimeOptions.ParseHttpMessageHandler (Xamarin.Bundler.Application app, System.String value) [0x0002f] in /Users/poupou/git/xamarin/xamarin-macios/src/ObjCRuntime/RuntimeOptions.cs:43
at Xamarin.Bundler.RuntimeOptions.Create (Xamarin.Bundler.Application app, System.String http_message_handler, System.String tls_provider) [0x00007] in /Users/poupou/git/xamarin/xamarin-macios/src/ObjCRuntime/RuntimeOptions.cs:34
at Xamarin.Bundler.Driver.Main2 (System.String[] args) [0x00a78] in /Users/poupou/git/xamarin/xamarin-macios/tools/mmp/driver.cs:377
at Xamarin.Bundler.Driver.Main (System.String[] args) [0x00015] in /Users/poupou/git/xamarin/xamarin-macios/tools/mmp/driver.cs:211
```
* [runtime] Add an inner exception parameter to Runtime.CreateProductException.
This allows us to simplify code by using inner (and outer) exceptions as
a means to provide information instead of passing extra information
around in order to create decent exceptions.
One example is how we pass the selector and method name to the method
that converts from a native id to a managed NSObject instance: passing
this information is not necessary anymore if we can use two exceptions,
one for the failure to convert from an id to a NSObject instance,
wrapped in a second that tells which method/selector call ran into this
conversion problem.
* [runtime] Throw better exceptions when the dynamic registrar can't marshal something.
* [runtime] Throw a better exception when something goes wrong when trying to marshal a return value.
* [runtime] Use inner exceptions to convey failure information instead of trying to create a single exception with all we know.
* Fix merge problem.
* [registrar] Make a copy of the static registrar for Xamarin.Mac/Classic.
Make a copy of the static registrar for Xamarin.Mac/Classic, one which is
compatible with the binary artifacts we ship for Xamarin.Mac/Classic
(libxammac, XamMac.dll).
This way the static registrar can evolve in the future, without breaking
Xamarin.Mac/Classic.
* [runtime] Make a copy of the runtime headers too.
* Update comment.
* [mmp] Delay creating a Target instance until we know if we're a Classic or Unified app.
Otherwise the wrong static registrar might be created.
* [tests] Add sample tester.
Add a unit project that looks for iOS/macOS/tvOS sample projects in several
repositories, and builds them all.
* [tests][sampletester] Remove known issue which has now been fixed.
* [tests] Only run sample tests on CI in Azure Devops.
* Remove the possibility of automatically running the sample tests with
xharness (so the sample tests won't run on PR bots or internal bots when the
'run-all-tests' label is added). It's still possible to run the sample tests
manually from the xharness web UI.
* Automatically trigger the sample test run in Azure Devops if the
'run-sample-tests' label is applied to a PR (and that PR is executed on
internal Jenkins).
* Fix typo.
* Fix path.
* Verbose output to track down scheduling failure.
* Bump maccore to get improved debug spew.
Diff: f527c9c526..f89d74b165
* [tests][sampletester] Fix build for TodoWCF.
This fixes an issue with the api comparison since the api comparison fails if
it detects unexpected modified files. Putting the temporary files in
APIDIFF_DIR makes sure the api comparison doesn't see those files as
unexpectedly modified.
Fixes https://github.com/xamarin/maccore/issues/1522.
Fixes this NRE:
error MT0000: Unexpected error - Please file a bug report at https://github.com/xamarin/xamarin-macios/issues/new
System.ArgumentNullException: Value cannot be null.
Parameter name: collection
at System.Collections.Generic.List`1[T].InsertRange (System.Int32 index, System.Collections.Generic.IEnumerable`1[T] collection) [0x00003] in <a104f9cbbafd4348bcc580acb0a3f8a8>:0
at System.Collections.Generic.List`1[T].AddRange (System.Collections.Generic.IEnumerable`1[T] collection) [0x00000] in <a104f9cbbafd4348bcc580acb0a3f8a8>:0
at Xamarin.Bundler.Application.BuildApp () [0x00040] in /work/maccore/master/xamarin-macios/tools/mtouch/Application.cs:1541
at Xamarin.Bundler.Application.BuildNative () [0x0001f] in /work/maccore/master/xamarin-macios/tools/mtouch/Application.cs:903
at Xamarin.Bundler.Application+<>c.<BuildAll>b__148_2 (Xamarin.Bundler.Application v) [0x00000] in /work/maccore/master/xamarin-macios/tools/mtouch/Application.cs:840
at System.Collections.Generic.List`1[T].ForEach (System.Action`1[T] action) [0x0001e] in <a104f9cbbafd4348bcc580acb0a3f8a8>:0
at Xamarin.Bundler.Application.BuildAll () [0x00076] in /work/maccore/master/xamarin-macios/tools/mtouch/Application.cs:840
at Xamarin.Bundler.Driver.Main2 (System.String[] args) [0x004a1] in /work/maccore/master/xamarin-macios/tools/mtouch/mtouch.cs:1369
at Xamarin.Bundler.Driver.Main (System.String[] args) [0x00015] in /work/maccore/master/xamarin-macios/tools/mtouch/mtouch.cs:877
- A long while ago, this hack was added to mmp:
// Shutup the warning until we decide on bug: 36478
if (shortendName.ToLowerInvariant () == "intl" && IsUnifiedFullXamMacFramework)
- This breaks use cases were we explicitly ask for that file, so let's
fix that for now until we can fix the root cause for real
It's needed at runtime since it changes exception handling behavior.
The "link sdk" Linker_RuntimeWrappedException() test relies on it,
but this never showed up there since "link sdk" doesn't link the main assembly.
Existing binding binaries won't have the `[Preserve]` attribute on
the `Handler` field and, with the new optimization, would not work
properly.
This tweak make sure that older, already linker-safe, bindings will
remain this way (safe) in this (and future) versions of both iOS and
macOS SDK.
According to Rolf it's fine to always add since the native linker will
figure out if it's really needed and so customers don't need to do
anything when using -all_load.
Mono started using System.IO.File from CoreFX and it has a different
behavior for File.Copy() when the target is a directory:
It just puts the file into the directory instead of raising an exception.
In order to fix the MT1015 test we just check ourselves whether the target
is a directory.
Up to this commit the test dlls used for the bcl tests were from iOS. Since
the Mono SDK provides the dlls for both missing platforms (TvOS and
WatchOS) we can now use the correct path for the dlls.
There is a small trick to minimize the project generation, since there
is a simple stirng.Replace, the logic now checks the platform under
test and does:
* TvOS - Goes from monotouch_TEST_NAME to monotouch_tv_TEST_NAME
* WatchOS - Gores from monotouch_TEST_NAME to monotouch_watch_TEST_NAME
This commit improves the state of the BCL testing in the following ways:
1. Improve the device tets running. Less apps, faster results.
2. WatchOS apps are left as they were to ensure that we do not have deplouyment/run issues.
We now support the ignore files per assembly name to simplify the
tracking of the ignored tests. All
```
/Users/poupou/git/master/xamarin-macios/tools/linker/MonoTouch.Tuner/RemoveBitcodeIncompatibleCodeStep.cs(14,7): warning CS0105: The using directive for 'Xamarin.Linker' appeared previously in this namespace [/Users/poupou/git/master/xamarin-macios/tools/mtouch/mtouch.csproj]
```
This also centralize other interpreter checks and options in the same
location (making it easier to read / update).
* Warn and switch the REPL if the interpreter is enabled on simulator
Why ? It's confusing to build the same code using different options for
simulator and devices. This is what happens if you try to use features
like `dynamic` or `System.Reflection.Emit`.
So instead of an error, we warn that the interpreter is not supported
and switch to the existing REPL mode.
The JIT remains the only option for the simulator but it allows testing
features without a device.
* Fail early if the interpreter is used on 32bits [1]
The current interpreter only works on 64 bits (so ARM64). However the
error won't be reported, back to the developer, until deployment time.
This temporary [1] fix spot the condition very early and report an error
```
error MT0099 : Internal error : The interpreter is currently only available for 64 bits.
```
instead of the current one at deploy time
```
IncorrectArchitecture: Failed to find matching arch for 32-bit Mach-O input file /private/var/installd/Library/Caches/com.apple.mobile.installd.staging/temp.tNKDlx/extracted/X.app/X
error MT1006: Could not install the application 'X.app' on the device 'Mercure': AMDeviceSecureInstallApplicationBundle returned: 0xe8000087 (kAMDIncorrectArchitectureError).
Application could not be uploaded to the device.
```
[1] https://github.com/mono/mono/issues/9871
* [tests] Fix/renumbered MT0138
The test was using simulator + interpreter which is not _really_
possible, we use REPL in that case - so we're now checking if
assemblies were specified with `--interpreter` to cover both cases.
Also 0138 was already used by `mmp` and the warning was **not**
registered or documented in the errors documents. To avoid
confusion it has been renumbered to 0142 and documented.
* [Foundation] Add an NSArray.FromNSObjects overload that can take an array of INativeObjects.
* [runtime] Use mono_array_setref instead of mono_array_set.
Otherwise the GC won't know about the assignment, and the assigned value can
be freed if it's no longer referenced anywhere else.
Some optimizations are not safe to apply when dynamically loading code
at runtime. This is not possible when ahead-of-time (AOT) compilation
is used, so those options were always enabled in the past. The
interpreter is changing the situation so those optimizations are now
disabled, when the interpreter is enabled.
The disabled optimizations are:
* 'remove-dynamic-registrar' when using the interpreter
This avoid the following exception when unknown (at build time) code
tries to register (at runtime)
```
Unhandled Exception:
System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> ObjCRuntime.RuntimeException: Can't register the class SubclassDemo.CustomView when the dynamic registrar has been linked away.
```
* 'inline-isdirectbinding' when using the interpreter
This avoid crashing (at runtime) when types are subclasses by the
interpreter (code unknown to `mtouch` at build time).
* 'register-protocols' when the interpreter is enabled
Dynamically loaded code can include protocols. When the optimization is
enabled then they won't be registered and won't work as expected.
Note: It is possible to re-enable the optimization(s), either one or all,
if the interpreter is not used to dynamically load code at runtime.
In both cases we use a different binaries for a few assemblies. They
include support for SRE and a few other things that are not normally
usable on iOS. That's totally fine and not something that can be fixed
(unless you stop using the feature). So this PR simply ignore that
specific case so we don't warn about things that can't be changed (and
are not a problem)
E.g.
```
Warning MT0109: The assembly 'mscorlib.dll' was loaded from a different path than the provided path (provided path: /Users/poupou/git/intr/xamarin-macios/_ios-build/Library/Frameworks/Xamarin.iOS.framework/Versions/Current/lib/mono/Xamarin.iOS/mscorlib.dll, actual path: /Users/poupou/git/intr/xamarin-macios/_ios-build/Library/Frameworks/Xamarin.iOS.framework/Versions/git/lib/mono/Xamarin.iOS/repl/mscorlib.dll). (MT0109) (XXX)
Warning MT0109: The assembly 'System.Core.dll' was loaded from a different path than the provided path (provided path: /Users/poupou/git/intr/xamarin-macios/_ios-build/Library/Frameworks/Xamarin.iOS.framework/Versions/Current/lib/mono/Xamarin.iOS/System.Core.dll, actual path: /Users/poupou/git/intr/xamarin-macios/_ios-build/Library/Frameworks/Xamarin.iOS.framework/Versions/git/lib/mono/Xamarin.iOS/repl/System.Core.dll). (MT0109) (XXX)
Warning MT0109: The assembly 'System.dll' was loaded from a different path than the provided path (provided path: /Users/poupou/git/intr/xamarin-macios/_ios-build/Library/Frameworks/Xamarin.iOS.framework/Versions/Current/lib/mono/Xamarin.iOS/System.dll, actual path: /Users/poupou/git/intr/xamarin-macios/_ios-build/Library/Frameworks/Xamarin.iOS.framework/Versions/git/lib/mono/Xamarin.iOS/repl/System.dll). (MT0109) (XXX)
Warning MT0109: The assembly 'System.Xml.dll' was loaded from a different path than the provided path (provided path: /Users/poupou/git/intr/xamarin-macios/_ios-build/Library/Frameworks/Xamarin.iOS.framework/Versions/Current/lib/mono/Xamarin.iOS/System.Xml.dll, actual path: /Users/poupou/git/intr/xamarin-macios/_ios-build/Library/Frameworks/Xamarin.iOS.framework/Versions/git/lib/mono/Xamarin.iOS/repl/System.Xml.dll). (MT0109) (XXX)
```
E.g. `MetalPerformanceShaders` is unusable in 10.3.1 on the simulator so
trying to build with the static registrar cause simlauncher to be rebuilt
with a **strong** (and not _weak_) reference to the framework. This cause
a crash then the application start
```
Dyld Error Message:
Library not loaded: /System/Library/Frameworks/MetalPerformanceShaders.framework/MetalPerformanceShaders
Referenced from: /Users/USER/Library/Developer/CoreSimulator/Devices/95679F95-CCE7-4733-8376-35E91E97039C/data/Containers/Bundle/Application/0F665B2D-ADAC-4CA0-BD04-33E968B3F268/FailerApp.app/FailerApp
Reason: no suitable image found. Did find:
/System/Library/Frameworks/MetalPerformanceShaders.framework/MetalPerformanceShaders: mach-o, but not built for iOS simulator
```
ref: https://github.com/xamarin/xamarin-macios/issues/5753
document it and adjust the optimization tests.
This allows the optimization to be:
* disabled (if ever needed) on fully AOT'ed applications
* re-enabled for the interpreter (at your own risk)
Fix comparison so the XM profiles do not report _inexistent_ changes for `_Attribute`, `_EventInfo` and `_Exception`. That noise makes it harder to review the diffs (for us) and is incorrect in our published API diffs (for release notes).
Also
* Delete .unzip.stamp on clean
* Skip classic XM diff (binaries won't change)