TL&DR: This PR
1. Removes the creation of the `.dSYM` based on `Debug Information` [1]
2. Adds dSYM support to XM msbuild (now shared with XI implementation)
3. Archive the `.dSYM` directories (plural) properly, e.g.
```
msbuild -p:Configuration=Release -p:ArchiveOnBuild=true
```
Why ? The long story...
Historically `.dSYM` for Xamarin.Mac have not been very useful, largely
because (most of) the code is JITed so not much is known before runtime.
So they were simply not generated during the builds...
However AOT options were added to Xamarin.Mac, making them potentially
more useful. Also symbols from `libmono` and other native libraries /
frameworks can prove useful when diagnosing application crashes.
Unsurprisingly developers looking to get symbols eventually found _a way_
[1] to get a `.dSYM` for their applications - but it was not quite
correct because:
* setting the debug information option meant that `mmp` would be supplied with `-debug`. This disables several optimizations that are, by default, enabled for release builds. IOW generating symbols should have no effect on the executing code (but it had);
* it was produced when compiling the native launcher, so the symbols coverage was incomplete. How much depends if mono was statically or dynamically linked. However this would not cover any AOTed code nor bundled libraries or user frameworks.
* the .dSYM was produced inside the `x.app/Contents/MacOS/`, side-by-side with the native executable, which makes it part of the **signed** `.app` and also part of the created (and signed) `.pkg`. This had a large impact on the application's, disk and download, size(s). Manually (re)moving the `.dSYM` means re-signing the app and re-creating (and signing) the `.pkg` is not a good solution.
[1] https://forums.xamarin.com/discussion/139705/how-to-symbolicate-a-xam-mac-crash-log
Additional fixes
* Use `Directory.Move` instead of running the `mv` command
While the result is identical there is a cost to spawn several `mv`
processes. Doing it in parallel (might have) helped but that setup
also comes at a cost.
`Directory.Move` the four `.dylib.dSYM` of an app takes 1 ms, while
the existing code took 17 ms to do the same.
* Fix building mmptest since the DeleteDebugSymbolCommand constant is not present (nor used) anymore
These two tasks did essentially the same thing, so we can just merge them. I
kept the "EmbedProvisionProfile" name, because that sounded like the most
applicable to all platforms.
instead _indirectly_ thru a binding project.
It got broken when I fixed the msbuild tests (to avoid re-building
needlessly the project).
Test case: https://github.com/spouliot/xcframework
Also don't resolve when building binding projects - since it's not useful (as we already package the whole .xcframework)
* Install the Mac Catalyst versions of the mono libraries and BCL.
* The BCL is the same as the one for Xamarin.iOS, which means it has to be post-processed a bit to work with a Xamarin.MacCatalyst.dll
* Build our runtime for Mac Catalyst.
* Build a Xamarin.MacCatalyst.dll with the Mac Catalyst API (it compiles, but I haven't looked at the API surface at all). This PR assumes we're going to have a new TargetFrameworkIdentifier for Mac Catalyst, but a final decision has not been made (see https://github.com/dotnet/runtime/issues/44882), so this may change.
* Build a Xamarin.iOS.dll that contains type forwarders to Mac Catalyst for all the types that exist in both Mac Catalyst and Xamarin.iOS.
* Add support to xharness for running introspection on Mac Catalyst (there are a lot of failures because the API surface is wrong)
* Add support to our msbuild tasks and mtouch for building Mac Catalyst apps. This basically comes down to adding a new case in numerous places to either do things the iOS way or the macOS way, depending on each case.
* Add a __MACCATALYST__ define (which is in addition to the __IOS__ define).
The build logic will put reference assemblies in
'ResolvedCompileFileDefinitions' instead of '_ReferencesFromNuGetPackages'
when using the new-style packageref format, so we need to handle
'ResolvedCompileFileDefinitions' the same way we handle
'_ReferencesFromNuGetPackages'.
I was unfortunately not able to create a unit test for this scenario, because
xibuild sets MSBUILD_EXE_PATH to make msbuild look in our local build files,
and then the restore fails with:
> Issue9266.csproj : error MSB4236: The SDK 'Msbuild.Sdk.Extras/2.1.2' specified could not be found.
If I don't use "Msbuild.Sdk.Extras" in the test, then it doesn't hit the
broken path this PR is fixing. If I change xibuild to not set
MSBUILD_EXE_PATH, we're not testing our local bits anymore, but the system
Xamarin.Mac.
Fixes https://github.com/xamarin/xamarin-macios/issues/9266.
* Add a stamp file to the Mac version of _CompileToNative. This is used to
force a rebuild of the container app when an app extension changes (and was
already implemented for iOS: 8b376efa4b)
* Rename the iOS stamp file to 'bundler.stamp' instead of 'mtouch.stamp' to
ease code sharing.
* Rename the 'MmpReferencePath' and 'MTouchReferencePath' properties to
'_BundlerReferencePath' to ease code sharing.
* Merge the _GetCompileToNativeInputs targets. The only difference between iOS
and Mac (after the above changes) is that the Mac version now includes
'@(ReferencePath)' in '_CompileToNativeInput'. This should be fine as far as
I know.
This is done early so we can resolve the inner framework, inside the
xcframework, and let the existing framework support do most of the
work.
The resolving code has unit tests. Custom projects for "NoEmbedding"
exists for all supported platforms and executed by xharness.
A sample `xcframework` with tests projects is also available
[here](https://github.com/spouliot/xcframework).
The xcframework test case is based on Rolf's earlier/partial implementation.
https://github.com/rolfbjarne/xamarin-macios/commit/xcframework
Things to note:
Do not rename a framework (like XTest) to use it in an xcframework
(like XCTest). That will fail at codesign but won't give anything
useful. You might think signing the framework (instead of the inner
binary) would solve it. It does, as it codesign, but then the app
crash at startup. At some point you realize some symbols are still
using XTest (not XCTest) and then you can delete several other weird
workarounds (like for `ld`) because all of it was cause by this
never identified rename.
dSYM support (and tests) to be done in a separate PR.
* Share the following targets:
* _ResolveAppExtensionReferences
* _SplitAppExtensionReferencesByExistent
* _AssignAppExtensionConfiguration
* _SeparateAppExtensionReferences
* _CopyAppExtensionsToBundle
The first four were pretty much identical between iOS and Mac already, and
needed very few changes.
The latter, _CopyAppExtensionsToBundle, required a little bit of tweaking to
the targets to make sure it works for both iOS and Mac:
* Removed the conditions on the Ditto task to check if we have app extensions
- this is unnecessary because Inputs/Outputs on the target itself (which
weren't there when the Ditto task conditions were written).
* Added a '_PlaceAppExtensions' target that calculates where in the
container's app bundle extensions should be placed (the '_AppExtensionRoot'
property), and the name of the containing folder (the 'ContainerName'
metadata).
* Add an AOTCompile task that runs the AOT compiler for a given set of input assemblies.
This task will eventually be deprecated by an equivalent task provided by Mono,
but this works for now.
* Add an _AOTCompile target that takes care of running the AOT compiler and compile
the result into object files.
* [msbuild] Make the default value for IsMacEnabled property dependent upon the OS.
Also make it overridable by only setting it if it's not already set.
* Use $([MSBuild]::IsOSPlatform('windows')) instead of $(OS)
The $(OS) value isn't standardized, so there's no sane check we can do against
it.
* Ship mlaunch in the iOS, tvOS and watchOS NuGets. It should probably go into
a separate NuGet (to avoid shipping the same mlaunch executable in three different
packages), but that can be done at a later stage.
* Add a GetMlaunchArguments task that computes the mlaunch arguments to install
or launch an app in either the simulator or on device.
* Implement the MSBuild logic to make the Run target (provided by .NET) launch
mlaunch (for iOS, tvOS and watchOS) or the built app (for macOS). This is done
by setting the RunCommand and RunArguments properties (which the Run target uses)
to the correct values.
Ideally I'd would make 'dotnet run' work too, but that runs into a different problem which
I haven't figured out yet:
A fatal error was encountered. The library 'libhostpolicy.dylib' required to execute the application was not found in '/Users/rolf/work/maccore/onedotnet/xamarin-macios/tests/dotnet/MySingleView/bin/Debug/net5.0-ios/ios-x64/'.
Failed to run as a self-contained app.
- The application was run as a self-contained app because '/Users/rolf/work/maccore/onedotnet/xamarin-macios/tests/dotnet/MySingleView/bin/Debug/net5.0-ios/ios-x64/MySingleView.runtimeconfig.json' did not specify a framework.
- If this should be a framework-dependent app, specify the appropriate framework in '/Users/rolf/work/maccore/onedotnet/xamarin-macios/tests/dotnet/MySingleView/bin/Debug/net5.0-ios/ios-x64/MySingleView.runtimeconfig.json'.
That's for a different pull request though.
Ref: https://github.com/xamarin/net6-samples/issues/35.
Xamarin.Mac didn't have support for CoreML models before, so this effectively adds
support for CoreML models to Xamarin.Mac.
Also add a test to make sure it actually works.
* This fixes a bug in the Xamarin.Mac version, where the _ForgeMetal's Outputs
value didn't match the actual output path (this was fixed in the iOS version).
* Also update the definition of the _ForgedMetal itemgroup to use a nested ItemGroup
instead of the CreateItem task.
The Xamarin.Mac version of _CreatePkgInfo had a 'Ditto' call that seemed like
a copy-paste error (the target below has the exac same call to 'Ditto'), and
there's no reason to call 'Ditto' when creating the PkgInfo file, so I just
removed it.
The 'Ditto' call was added here: ca028ea150 (diff-ccded350e40e767d69171655acb86b01R586-R606)
The Xamarin.iOS version is more updated, so that's the version now for both
Xamarin.iOS and Xamarin.Mac:
* The iOS version contains Inputs/Outputs (important for incremental builds).
* It doesn't use FileWrites (see e97d69b25c why
this was removed for iOS)
Additionally define the _SceneKitCache variable for macOS as well, previously
it was only defined for iOS (I'm adding more variables than just
_SceneKitCache to macOS, most of the others will be used in in macOS in
upcoming pull requests, and this way turned out to be simplest).
And also add a new Xamarin.Mac test that exercises the CompileSceneKitAssets targets.
Merge the CodesignNativeLibraries task into the Codesign task, since they're almost
identical (so this turns out to be a significant code reduction).
* Add a StampPath property to Codesign task, which, if set, specifies if and where
a stamp file should be created for each signed file/directory. This replaces the
IntermediateOutputPath property from the CodesignNativeLibraries task.
* Copy logic to detect if a file/directory needs to be resigned from the CodesignNativeLibraries
task to the Codesign task, using any stamp files.
* Use the more descriptive error messages E0004/E0005 from the CodesignNativeLibraries
task.
* The CodesignNativeLibraries task recursed into directories to find *.dylibs and
*.metallib files to sign; replace this with MSBuild item group globbing.
The Xamarin.iOS version seemed more updated, so that's the version now for
both Xamarin.iOS and Xamarin.Mac.
A few differences in the Xamarin.iOS versions are:
* The _CoreCompileInterfaceDefinitions target contains Inputs/Outputs
(important for incremental builds).
* EnableOnDemandResources is set to $(EnableOnDemandResources), not hardcoded
to 'false' (there is no real difference right now, because
$(EnableOnDemandResources) defaults to 'false' in Xamarin.Mac).
* It doesn't use FileWrites (see e97d69b25c why
this was removed for iOS)
Unfortunately due to when things happen in the .NET build logic, we need to
define the DebuggerSupport property (which determines whether the app should
include debugging support or not) before importing the .NET build files. Since
we want to use the _BundlerDebug property (a.k.a. MtouchDebug/MmpDebug) to
determine if the app should include debugging support, we must figure out the
value of the _BundlerDebug property before we can define the DebuggerSupport
property. This turned out complicated, because we're currently defining
_BundlerDebug in our old-style MSBuild logic, which is imported after we
import the .NET build logic.
The end result is that we can either shuffle around a lot of MSBuild code, or
copy a few lines to set the _BundlerDebug property. Neither option makes me
very happy, but copying a few lines of code seemed the better option, so
that's what I did.
Fixes these linkall test failures in Release mode:
LinkAll.Attributes.AttributeTest
[FAIL] DebugAssemblyAttributes : DebuggableAttribute
Expected: False
But was: True
at LinkAll.Attributes.AttributeTest.DebugAssemblyAttributes()
[FAIL] DebugConstructorAttributes : No debug attribute in release mode
Expected: 0
But was: 2
at LinkAll.Attributes.AttributeTest.DebugConstructorAttributes()
[FAIL] DebugPropertyAttributes : DebuggerBrowsable
Expected: False
But was: True
at LinkAll.Attributes.AttributeTest.DebugPropertyAttributes()
[FAIL] DebugTypeAttributes : no debug attribute in release mode
Expected: 0
But was: 5
at LinkAll.Attributes.AttributeTest.DebugTypeAttributes()
[FAIL] DebuggerTypeProxy_24203 : proxy
Expected: null
But was: <System.Collections.Generic.IDictionaryDebugView`2[K,V]>
at LinkAll.Attributes.AttributeTest.DebuggerTypeProxy_24203()
Fixes this warning:
warning MSB9004: ManifestResourceWithNoCulture item type is deprecated. Emit EmbeddedResource items instead, with metadata WithCulture='false', Type='Resx', and optional LogicalName.
There is a slight difference with regards to the warning message: the Type
metadata is set to 'Non-Resx' instead of 'Resx' (because these resources
aren't resx files).
Fixes https://github.com/xamarin/xamarin-macios/issues/3876.
- Fixes https://github.com/xamarin/xamarin-macios/issues/9458
- Defining the variable over multiple lines changes output to include incorrect newline
- Auto tests did not catch, as they don't sign on bots due to infrastructure issues.
- Was refactored _after_ manual tests and not retested
Co-authored-by: Chris Hamons <chris.hamons@xamarin.com>
* Use ItemGroup inside a target, since using CreateItem is deprecated.
* Use target batching to make sure the _CompressNativeFrameworkResources
target is called once per framework.
* Also add a comment for my future self describing the whole process, because
I'll forget this by next week.
The end result is that we will now only re-compress a framework if any file in
that particular framework changed (as opposed to re-compressing all frameworks
in a binding project when any of them changed).
There were 2 differences that had to be reconciled:
* The Xamarin.Mac version took 'IntermediateOutputPath' as the intermediate
output path, while the Xamarin.iOS version used
'DeviceSpecificIntermediateOutputPath'.
'DeviceSpecificIntermediateOutputPath' is defined to be
'IntermediateOutputPath' for Xamarin.Mac, so make the shared version use
'DeviceSpecificIntermediateOutputPath', since that works in both cases.
* The Xamarin.Mac version was writing the output of PackLibraryResources to
the FileWrites item group. This was removed on purpose from the Xamarin.iOS
version in e97d69b25c, so it seemed best to
leave it out in the shared version as well.
Also re-use the _EmbeddedResourcePrefix variable to determine the resource prefix.
These only apply to Xamarin.iOS right now, since they're Windows-specific (and
we don't support Xamarin.Mac binding projects on Windows), but if we ever do
end up supporting binding projects for Xamarin.Mac on Windows, we'll need
these targets there, so just make them shared code.
Rolf Bjarne Kvinge
Sebastien Pouliot
Wojciech Kulik
monojenkins