Now it's possible to reference a compressed xcframework like this:
```xml
<NativeReference Include="path/to/myframework.xcframework.zip" />
```
This makes it easier to work with xcframeworks on Windows, because
xcframeworks can contain symlinks, and by zipping them up the whole symlink
problem is avoided.
Turns out there's already a NuGet that adds a .xcframework.zip file as a
NativeReference (Microsoft.ML.OnnxRuntime), so this makes that NuGet work.
Fixes https://github.com/xamarin/xamarin-macios/issues/21294.
Fixes https://github.com/xamarin/xamarin-macios/issues/21450.
This test doesn't work when building on iossimulator-arm64 or
tvossimulator-arm64, because we're trying to use a fat framework which doesn't
have that architecture (it contains the device arm64 architecture instead).
So just disable this test on iOS and tvOS - the solution is using an
xcframework, and we already have a different test for that.
---------
Co-authored-by: Manuel de la Pena <mandel@microsoft.com>
Add support for the LinkWithSwiftSystemLibraries metadata to specify whether a native library is a Swift library, in which case we'll automatically set the `LinkWithSwiftSystemLibraries` MSBuild property to `true`.
Also add a test.
Once upon a time we needed to special case a higher min OS version that the
min OS version we supported for certain compiler/linker arguments, because we
used features not supported in the min OS version we supported.
That time has passed; in all cases our min OS version is now higher than the
special-cased min OS versions passed to native compilers/linkers, so we can
just use the actual min OS version we support.
Change all null checking expressions to use 'is null' and 'is not null'
instead of '== null' and '!= null'.
This was mostly done with sed, so code can probably be improved in many
other ways with manual inspection, but that will come over time.
Also add code to the autoformat script to automatically fix these issues in the future.
This pull request updates the following dependencies
## Coherency Updates
The following updates ensure that dependencies with a *CoherentParentDependency*
attribute were produced in a build used as input to the parent dependency's build.
See [Dependency Description Format](https://github.com/dotnet/arcade/blob/master/Documentation/DependencyDescriptionFormat.md#dependency-description-overview)
- **Coherency Updates**:
- **Microsoft.NET.ILLink.Tasks**: from 7.0.100-1.22564.1 to 8.0.100-1.23055.2 (parent: Microsoft.Dotnet.Sdk.Internal)
- **Microsoft.AspNetCore.App.Ref**: from 8.0.0-alpha.1.22558.5 to 8.0.0-alpha.1.23058.7 (parent: Microsoft.Dotnet.Sdk.Internal)
- **Microsoft.NETCore.App.Ref**: from 8.0.0-alpha.1.22559.2 to 8.0.0-alpha.1.23058.2 (parent: Microsoft.Dotnet.Sdk.Internal)
- **Microsoft.NET.Workload.Emscripten.net7.Manifest-8.0.100**: from 8.0.0-alpha.1.22554.1 to 8.0.0-alpha.1.22620.1 (parent: Microsoft.NETCore.App.Ref)
## From https://github.com/dotnet/installer
- **Subscription**: 9a2944cb-7dee-4bf2-a65c-08dabd10ae64
- **Build**: 20230113.2
- **Date Produced**: January 13, 2023 12:09:55 PM UTC
- **Commit**: e0284b3f3c72d8f21e4825ed1ac723d2e829d0a5
- **Branch**: refs/heads/main
- **Updates**:
- **Microsoft.Dotnet.Sdk.Internal**: [from 8.0.100-alpha.1.22605.6 to 8.0.100-alpha.1.23063.2][191]
- **Microsoft.NET.ILLink.Tasks**: [from 7.0.100-1.22564.1 to 8.0.100-1.23055.2][192]
- **Microsoft.AspNetCore.App.Ref**: [from 8.0.0-alpha.1.22558.5 to 8.0.0-alpha.1.23058.7][193]
- **Microsoft.NETCore.App.Ref**: [from 8.0.0-alpha.1.22559.2 to 8.0.0-alpha.1.23058.2][194]
- **Microsoft.NET.Workload.Emscripten.net7.Manifest-8.0.100**: [from 8.0.0-alpha.1.22554.1 to 8.0.0-alpha.1.22620.1][195]
[191]: cbc7313...e0284b3
[192]: 13b8d6d...4b3f78c
[193]: 1bee0af...cefc6cc
[194]: dd7fdb7...5da4a9e
[195]: b6656f5...66b9845
Co-authored-by: Rolf Bjarne Kvinge <rolf@xamarin.com>
This turned out a bit complicated, because we're lipo'ing together both the simulator
and device slices, and that doesn't work anymore when ARM64 is both a simulator and
a device architecture. The solution is a custom lipo logic that doesn't include the
simulator version of the ARM64 architecture in the final binary.
* [tests] Create a libtest.xcframework and libtest2.xcframework
* [tests] Make bindings-test and bindings-test2 use an xcframework instead of plain static library
* [msbuild] Add support for xcframeworks with static libraries in them.
* List the frameworks libtest needs.
* [tests] Update .NET unit tests according to test project changes.
* [tests] Add new test to verify that packing an old-style binding project doesn't work.
This makes it easier to iterate over all the *_SDK_VERSION variables in
template code, because they're all named using the standard platform names we
use elsewhere.
.NET/MSBuild don't handle symlinks properly [1], which means that we can't ask
.NET to copy frameworks to the app bundle, since frameworks may contain
symlinks.
In our case, the symptom was that instead of copying symlinks, the file the
symlink pointed to was copied instead, and then codesign complained about
invalid bundle format when we tried to sign the framework.
We fix this by having our own target (_CopyFrameworksToBundle) to copy
frameworks to the app bundle (instead of adding all the files in the
frameworks to the ResolvedFileToPublish item group), and then using 'ditto' to
copy the frameworks.
In order to create a test case for this, I also made the macOS and Mac
Catalyst versions of the XTest framework use symlinks:
* Create a proper XTest framework bundle hierarchy for macOS and Mac Catalyst
by using the typical symlink structure (actual files in the Versions/A
subdirectory, and then symlinks pointing into that directory).
* Create a separate Info.plist for each platform for XTest.framework, since
using an otherwise correct framework makes tooling (such as codesign)
complain if the Info.plist isn't correct too.
This made our existing tests show the bug.
Finally I had to fix signing frameworks where the executable is a symlink.
We were first resolving symlinks for the input - say we had an
Example.framework/Example symlink to Example.framework/Versions/A/Example -
and then checking the parent directory if it's a framework. The parent
directory of 'Example.framework/Versions/A/Example' is 'A', which did not meet
our framewrok condition (if it ends with '.framework').
The fix is to adjust the logic to resolve symlinks after checking if the input
is a framework or not.
[1]: https://github.com/dotnet/msbuild/issues/6821
Fixes https://github.com/xamarin/xamarin-macios/issues/12369.
When checking whether a type is a user type or not, we might have to do a
dynamic check if the dynamic registrar is available. Otherwise we may run into
a situation where the static registrar ran during the app build, but then the
app loaded additional assemblies at runtime, and those assemblies contained
user types that were registered using the dynamic registrar, so the static
registrar doesn't know about those custom types (in other words: we need to
check at runtime).
* Pass any native references to the LinkNativeCode task so that they're linked
with the main executable.
* Copy frameworks and dynamic libraries to the app bundle (also add support for
MSBuild metadata - the CopyToAppBundle property - to avoid copying a framework
/ dynamic library to the app bundle).
* Add tests.
Fixes https://github.com/xamarin/xamarin-macios/issues/11061.
* [runtime] Build our runtime for Mac Catalyst/ARM64 for .NET.
* [ObjCRuntime] There's no need for the StartWWAN implementation on Mac Catalyst.
This also fixes a build error:
error MT5214: Native linking failed, undefined symbol: _xamarin_start_wwan. This symbol was referenced by the managed member ObjCRuntime.Runtime.xamarin_start_wwan.
* Only exclude xamarin_start_wwan in the .NET version of Mac Catalyst.
* [tests] Update to not run the StartWWAN test on Mac Catalyst.
* Update conditional logic.
* Fix build with newer make versions.
* [tests] Build test-libraries for Mac Catalyst.
* [msbuild] Add support for Mac Catalyst binding projects.
* [mtouch] Allow frameworks for Mac Catalyst apps.
* [mtouch] Put frameworks in the expected location for Mac Catalyst apps.
* [msbuild] Create the Resources directory before trying to put files in it.
This is done early so we can resolve the inner framework, inside the
xcframework, and let the existing framework support do most of the
work.
The resolving code has unit tests. Custom projects for "NoEmbedding"
exists for all supported platforms and executed by xharness.
A sample `xcframework` with tests projects is also available
[here](https://github.com/spouliot/xcframework).
The xcframework test case is based on Rolf's earlier/partial implementation.
https://github.com/rolfbjarne/xamarin-macios/commit/xcframework
Things to note:
Do not rename a framework (like XTest) to use it in an xcframework
(like XCTest). That will fail at codesign but won't give anything
useful. You might think signing the framework (instead of the inner
binary) would solve it. It does, as it codesign, but then the app
crash at startup. At some point you realize some symbols are still
using XTest (not XCTest) and then you can delete several other weird
workarounds (like for `ld`) because all of it was cause by this
never identified rename.
dSYM support (and tests) to be done in a separate PR.
* [watchOS] Add x86_64 simulator support
* Build runtime/registrar x86_64 slices
* Produce a 64 bit version of Xamarin.WatchOS.dll
* Allow building x86_64 for watch simulators in mtouch
* Let xharness know about x86_64
* [tests] Add x86_64 arch to test-libraries
* Make dotnet package aware of x64
* [ObjCRuntime] Fix computing if we're calling a stret function or not in a 64-bit watchOS simulator.
* [xharness] Re-enable some watchOS tests.
Co-authored-by: Rolf Bjarne Kvinge <rolf@xamarin.com>
In make 4.3 the pattern `.libs/watchos/%.arm64_32.dylib` in mk/rules.mk takes
precedence over the pattern `.libs/watchos/libtest.%.dylib` in
tests/test-libraries/Makefile when the pattern replacements are 'libtest' and
'arm64_32' respectively, because the former is shorter.
This seems to be a bug (in make 3.81) that has been fixed, because that's the
documented behavior.
Rewrite the logic to not use a pattern target in test/test-libraries/Makefile.
Fix building the test libraries so that the native compiler gets passed the
right arguments (which makes us build the tvOS and watchOS versions of these
libraries with bitcode).
This fixes a build problem with the framework-test test for tvOS and watchOS.
Also remove an outdated comment.
This has a couple of advantages:
* It makes it easier to add a catalyst version of these libraries (because it
becomes cumbersome to build for catalyst when the build rules assumes we're
building for both simulator and device).
* It makes it easier to create an xcframework of our libraries, because the
contents in an xcframework is split like this.
* [linker] Optimize calls to BlockLiteral.SetupBlock to inject the block signature.
Optimize calls to BlockLiteral.SetupBlock[Unsafe] to calculate the block
signature at build time, and inject it into the call site.
This makes block invocations 10-15x faster (I've added tests that asserts at
least an 8x increase).
It's also required in order to be able to remove the dynamic registrar code in
the future (since calculating the block signature at runtime requires the
dynamic registrar).
* [mtouch/mmp] Add support for reporting errors/warnings that point to the code line causing the error/warning.
Add support for reporting errors/warnings that point to the code line causing
the error/warning by adding ErrorHelper overloads that take the exact
instruction to report (previously we defaulted to the first line/instruction
in a method).
* [tests] Add support for asserting filename/linenumber in warning messages.
* Make all methods that manually create BlockLiterals optimizable.
* [tests] Create a BaseOptimizeGeneratedCodeTest test that's included in both XI's and XM's link all test.
* [tests] Add link all test (for both XI and XM) to test the BlockLiteral.SetupBlock optimization.
* [tests] Add mtouch/mmp tests for the BlockLiteral.SetupBlock optimization.
* [tests][linker] Make the base test class abstract, so tests in the base class aren't executed twice.
* [tests][linker] Don't execute linkall-only tests in linksdk.
The optimization tests only apply when the test assembly is linked, and that
only happens in linkall, so exclude those tests in linksdk.
* [tests][mmptest] Update test according to mmp changes.
Fixes these test failures:
1) Failed : Xamarin.MMP.Tests.MMPTests.MM0132("inline-runtime-arch")
The warning 'MM0132: Unknown optimization: 'inline-runtime-arch'. Valid optimizations are: remove-uithread-checks, dead-code-elimination, inline-isdirectbinding, inline-intptr-size.' was not found in the output:
Message #1 did not match:
actual: 'Unknown optimization: 'inline-runtime-arch'. Valid optimizations are: remove-uithread-checks, dead-code-elimination, inline-isdirectbinding, inline-intptr-size, blockliteral-setupblock.'
expected: 'Unknown optimization: 'inline-runtime-arch'. Valid optimizations are: remove-uithread-checks, dead-code-elimination, inline-isdirectbinding, inline-intptr-size.'
Message #2 did not match:
actual: 'Unknown optimization: 'inline-runtime-arch'. Valid optimizations are: remove-uithread-checks, dead-code-elimination, inline-isdirectbinding, inline-intptr-size, blockliteral-setupblock.'
expected: 'Unknown optimization: 'inline-runtime-arch'. Valid optimizations are: remove-uithread-checks, dead-code-elimination, inline-isdirectbinding, inline-intptr-size.'
2) Failed : Xamarin.MMP.Tests.MMPTests.MM0132("foo")
The warning 'MM0132: Unknown optimization: 'foo'. Valid optimizations are: remove-uithread-checks, dead-code-elimination, inline-isdirectbinding, inline-intptr-size.' was not found in the output:
Message #1 did not match:
actual: 'Unknown optimization: 'foo'. Valid optimizations are: remove-uithread-checks, dead-code-elimination, inline-isdirectbinding, inline-intptr-size, blockliteral-setupblock.'
expected: 'Unknown optimization: 'foo'. Valid optimizations are: remove-uithread-checks, dead-code-elimination, inline-isdirectbinding, inline-intptr-size.'
Message #2 did not match:
actual: 'Unknown optimization: 'foo'. Valid optimizations are: remove-uithread-checks, dead-code-elimination, inline-isdirectbinding, inline-intptr-size, blockliteral-setupblock.'
expected: 'Unknown optimization: 'foo'. Valid optimizations are: remove-uithread-checks, dead-code-elimination, inline-isdirectbinding, inline-intptr-size.'
* [tests][linker] Fix typo.
Fixes this test failure:
1) SetupBlock_CustomDelegate (Linker.Shared.BaseOptimizeGeneratedCodeTest.SetupBlock_CustomDelegate)
Counter
Expected: 1
But was: 2
* [registrar] Minor adjustment to error message to match previous (and better) behavior.
Fixes this test failure:
1) Failed : Xamarin.Registrar.GenericType_WithInvalidParameterTypes
The error 'MT4136: The registrar cannot marshal the parameter type 'System.Collections.Generic.List`1<U>' of the parameter 'arg' in the method 'Open`1.Bar(System.Collections.Generic.List`1<U>)'' was not found in the output:
Message #1 did not match:
actual: 'The registrar cannot marshal the parameter type 'System.Collections.Generic.List`1<Foundation.NSObject>' of the parameter 'arg' in the method 'Open`1.Bar(System.Collections.Generic.List`1<U>)''
expected: 'The registrar cannot marshal the parameter type 'System.Collections.Generic.List`1<U>' of the parameter 'arg' in the method 'Open`1.Bar(System.Collections.Generic.List`1<U>)''
* [docs] mmp shows MM errors/warnings.
* [docs] Improve according to reviews.
* [tests] Fix merge failure causing test duplication.
* [tests] Build the native test library for macOS and create a binding project for it.
Also add the new binding project to the xammac and link all XM test projects,
which allows us to stop excluding tests that require the native library and
the corresponding bindings.
* [tests] Include more tests in xammac_tests.
* [tests] Correctly ignore the ObjC exception tests in release mode.
monotouch-test has a wildcard to automatically include new test files, but
this should not include generated files, because:
* The generated files are generated when needed, which means we can't rely on
the wildcard to trigger their generation, because the wildcard won't find
them before they exist, and as such msbuild won't detect that they're
needed.
* This means the generated files must be listed separately, but in that case
they shouldn't be found by the wildcard too, because that leads to:
/Library/Frameworks/Mono.framework/Versions/5.4.0/lib/mono/msbuild/15.0/bin/Roslyn/Microsoft.CSharp.Core.targets(84,5): error MSB3105: The item "ObjCRuntime/TrampolineTest.generated.cs" was specified more than once in the "Sources" parameter. Duplicate items are not supported by the "Sources" parameter.
So move the generated files to a different directory, so that the wildcard
doesn't find them.
The native linker treats object files (.o) and static libraries (.a files,
which are archives of .o files) differently.
The native linker will always include object files into the executable:
$ echo "void xxx () {}" > foo.m
$ clang -c foo.m -o foo.o -arch x86_64
$ ld foo.o -dylib -o foo.dylib -macosx_version_min 10.12 -arch x86_64
$ nm foo.dylib
0000000000000fe0 T _xxx
However, if the object file is inside a static library:
$ echo "void xxx () {}" > foo.m
$ clang -c foo.m -o foo.o -arch x86_64
$ ar cru foo.a foo.o
$ ld foo.a -dylib -o foo.dylib -macosx_version_min 10.12 -arch x86_64
$ nm foo.dylib
<no output>
This means that our testing library (libtest.a) which is a fat library of
_object files_, do not show the problems reported in bug #51548.
So:
a) I've fixed the creation of libtest.a to be a fat library of _static
libraries_. This causes the `FastDev_LinkWithTest` test to fail exactly
like in bug #51548.
b) I've made mtouch pass `-u <native symbol>` to the native linker, for every
native symbol referenced in a managed assembly, when creating a dylib.
Amazingly this seems to work fine even with symbols to Objective-C classes
(`_OBJC_CLASS_$_<class name>`).
c) This also required adding support for collecting the Objective-C names of
all managed types registered with Objective-C to the linker. The
information is already available in the static registrar, but that would
require us to make sure the static registrar is executed before compiling
dylibs, which means those two tasks won't be able to run in parallel (also
there's no guarantee we'll even run the static registrar).
https://bugzilla.xamarin.com/show_bug.cgi?id=51548
Generate trampoline and registrar tests that tests if a return type requires objc_msgSend or objc_msgSend_stret.
Now it's much easier to test new return types (a single line of code), which
avoids a _lot_ of copy-pasting, and makes sure all the different variations
are tested properly.
These new tests found several bugs, which are fixed in subsequent commits.
Rolf Bjarne Kvinge
dotnet-maestro[bot]
Sebastien Pouliot
Alex Soto