This was mostly a clean merge, with a few minor differences:
* We no longer compute whether we're running in the simulator or not when building for Mac Catalyst.
* The task now supports building remotely for macOS (due to code sharing).
Will be useful if we ever support building macOS apps remotely.
* We now call AppleSdkSettings.Init () on macOS. No idea why we weren't
before, but it seems logical for macOS to behave like our other platforms.
There shouldn't be any other functional differences.
Apple removed the -z / --minimize option in Xocde 13.3, so now if you use it
you get a warning: "ignoring unknown option: -z".
So just don't pass -z when using Xcode >= 13.3
Ref: https://github.com/dotnet/runtime/issues/66770
Ref: 5d07dc8977
Only sign the prebuilt hotrestart app in CI, to avoid making it required for
developers to configure code signing.
Also fix DetectSigningIdentity to not require a code signing certificate for
device builds when RequireCodeSigning is false.
Co-authored-by: Alexander Köplinger <alex.koeplinger@outlook.com>
* Unify the CompileSceneKitAsset task implementation between iOS and Mac.
There were no real differences, so might as well use the same code
everywhere.
* Use existing facilities for process launching.
* Parallelize compiling.
Make our local .NET the default .NET (in the root's global.json), and then if
a directory wants to use the system .NET, then that directory would have to
opt-in (using its own global.json).
This way we don't have to copy global.json/NuGet.config files around to run
tests with the correct .NET setup.
With remote builds, a dedicated dotnet location is being used so the right versioning can be used and managed from VS in Windows. This dedicated dotnet location requires a custom .home location so the donet and nuget caches doesn't conflict with the global installation.
We need to consider and use this custom .home location when running dotnet commands remotely so the right caches are used
This should fix Bug: https://devdiv.visualstudio.com/DevDiv/_workitems/edit/1543495
The compiled entitlements should be placed in the intermediate Hot Restart app bundle so those can be picked up by the HotRestart Codesign task. Prior to this change, entitlements set in the project wouldn't be included in the app, making things like Keychain Access fail, even though it was configured.
Fixes https://developercommunity.visualstudio.com/t/Unable-to-use-MSAL-with-locally-connecte/1573064
The current directory at launch is the root directory of the app bundle. This
means that any files written to the current directory when an app is executed,
will be placed there. This becomes a problem when the app is rebuilt (and
resigned), because a valid macOS app bundle doesn't have any files in the root
directory of the app bundle, so signing fails.
We have logic to automatically crash crash reports from the app bundle, but it
turns out this is a more common problem with other types of files (and
folders), so improve the logic a bit:
* Add support for setting a property to automatically clean up everything from
an app bundle we don't think should be there (which is anything not in a
Contents/ subdirectory).
* Use the same property to add support for disabling any cleaning (we already
clean mono's crash reports by default).
* Improve detection of unwanted files to include directories inside the app
bundle, not only files.
Ref: https://github.com/dotnet/maui/issues/7353
Give the full path to the symbols list file in the extension project to the
main project, so that strip can find it.
When building a solution remotely from Windows, we can't compute a relative
path between projects, because they're laid out differently on disk. This
means that we have to use full paths when passing paths between projects (such
as the path to the symbol list file).
Fixes https://github.com/xamarin/xamarin-macios/issues/15046.
Improve error reporting when an external tool fails to print some of stderr (up to 1024 characters).
Before:
error : clang exited with code 1
After:
error : clang exited with code 1:
error : [...]/xamarin-macios/tests/dotnet/MyInterpretedApp/iOS/obj/Debug/net6.0-ios/iossimulator-x64/linker-cache/main.x86_64.mm:58:25: error: use of undeclared identifier 'MONO_AOT_MODE_INTERP_ONLY'; did you mean 'MONO_AOT_MODE_INTERP'?
error : mono_jit_set_aot_mode (MONO_AOT_MODE_INTERP_ONLY);
error : ^~~~~~~~~~~~~~~~~~~~~~~~~
error : MONO_AOT_MODE_INTERP
error : [...]/xamarin-macios/builds/downloads/dotnet-sdk-6.0.301-rtm.22254.17-osx-x64/packs/Microsoft.iOS.Runtime.iossimulator-x64/15.4.16-ci.x64-interpreter-only/runtimes/iossimulator-x64/native/xamarin/mono-runtime.h:452:2: note: 'MONO_AOT_MODE_INTERP' declared here
error : MONO_AOT_MODE_INTERP,
error : ^
* This is a potential mitigation for slower transition to native code when
exception marshalling is enabled (#14812).
* A minor modification was required in the linker, to make sure any modified
assemblies are saved.
Fixes https://github.com/xamarin/xamarin-macios/issues/4940.
Pick up --aot arguments in MtouchExtraArgs and pass them to the AOT compiler
when building a .NET project. This makes it possible to work around #14887 by
manually increasing the number of trampolines.
Ref: https://github.com/xamarin/xamarin-macios/issues/14887
When building a binding project, we need to execute bgen (and csc) on the mac. Figuring
out where these files are on the Mac is rather complicated from a remotely executed
task, so instead we execute a sub-build that computes these properties.
In legacy Xamarin this was accomplished by building the 'Xamarin.iOS.ObjCBinding.Common.props'
file using msbuild, and invoking a custom target that prints the property we're looking
for (the 'targetGetPropertyValue_*' targets).
For multiple reasons this approach doesn't work in .NET anymore (in particular it
seems that the 'Xamarin.iOS.ObjCBinding.Common.After.targets' file with the custom
'targetGetPropertyValue_*' targets is nowhere to be found, but logic has also moved
around in the .targets/.props files which makes just building the 'Xamarin.iOS.ObjCBinding.Common.props'
not work correctly since the properties we need wouldn't be set).
So I'm adding a new task that does a sub-build, using either msbuild or dotnet as
appropriate, to compute the properties we need. Instead of building the 'Xamarin.iOS.ObjCBinding.Common.props'
file, the task creates an actual binding project (an empty one), and executes the
new '_WriteRemoteGeneratorProperties' target in this binding project.
An additional advantage in this new task is that it will only execute one sub-build
where all the properties are computed (the previous approach executed one sub-msbuild
per property).
In order to keep code as similar as possible between legacy Xamarin and .NET, the
new task is being used for legacy Xamarin as well (and the old approach deleted).
This fixes building binding projects on Windows in .NET.
The output directory might or might not be where the app bundle is: by default
it is, but if someone sets the PkgPackageDir variable to provide an alternate
directory for the pkg, then that won't be where we'll find the app bundle.
The good news is that we already have a property that tells us where the app
bundle is (the 'AppBundleDir' property), so just use that instead.
Also:
* Make sure that the output directory exists before we try to write to it.
* Only pass full paths to productbuild, which for some reason doesn't seem to
like relative paths.
Fixes https://github.com/xamarin/xamarin-macios/issues/14751.
It includes a change revert because of threading freeze issues.
An issue has been detected in Xamarin.Messaging 1.5.26 and 1.6.1 that makes the code unstable and sensitive to thread freeze issues, so we are reverting back the changes until we can stabilize it.
This delays the fix of a bug in MSBuild command line builds for iOS remote builds.
This also meant:
* Using 'latest' as the C# language version for all msbuild/ project files.
* Enabling warnaserror for nullability warnings.
* Fix any nullability warnings in the CompileAppManifest files.
* Fix a nullability warning in the Ditto task.
* Fix any '== null' or '!= null' to use 'is null' and 'is not null'.
Use 'Microsoft.<platform>' as the product name instead of 'Xamarin.[iOS|Mac]' for .NET builds. That makes error messages like this:
> [...]\Microsoft.iOS.Sdk\15.4.200-ci.windows2.62\tools\msbuild\iOS\Xamarin.Shared.targets(1676,3): Could not find Xamarin.iOS in /usr/local/share/dotnet/packs/Microsoft.iOS.Sdk/15.4.200-ci.windows2.62/.
look a bit better in .NET:
> [...]\Microsoft.iOS.Sdk\15.4.200-ci.windows2.62\tools\msbuild\iOS\Xamarin.Shared.targets(1676,3): Could not find Microsoft.iOS in /usr/local/share/dotnet/packs/Microsoft.iOS.Sdk/15.4.200-ci.windows2.62/.
* Move logic to validate the UIDeviceFamily value to shared code.
* Remove logic related to watchOS 1 apps, it's been dead for a while.
* Change logic to not overwriting any existing UIDeviceFamily values in the customer's
Info.plist.
This makes the code more platform-agnostic and easier to work with across all platforms
(such as adding new validations).
Fixes this warning from the Codesign task:
C:\Users\rolf\...\Microsoft.iOS.Sdk\15.4.200-...\tools\msbuild\iOS\Xamarin.Shared.targets(2045,3): Cannot create 'C:\Users\rolf\source\iOSApp4\bin\Debug\net6.0-ios\ios-arm64\device-builds\iphone14.2-15.3.1\iOSApp4.app\Frameworks\ArcGIS-arm64.framework' because a file or directory with the same name already exists.
C:\Users\rolf\...\Microsoft.iOS.Sdk\15.4.200-...\tools\msbuild\iOS\Xamarin.Shared.targets(2045,3): Cannot create 'C:\Users\rolf\source\iOSApp4\bin\Debug\net6.0-ios\ios-arm64\device-builds\iphone14.2-15.3.1\iOSApp4.app\Frameworks\Runtimecore.framework' because a file or directory with the same name already exists.
which occurs when the Codesign task asks XVS to create output files for files from
inside ditto'ed directories, and if XVS created output files for those directories
in the Ditto task, then XVS would be trying to create files inside these output files
as if they were directories. That doesn't work (thus the warning).
I've fixed this by:
* Removing the 'ShouldCreateOutputFile' implementation. The ShouldCreateOutputFile
method is called on Windows, and we can't determine from Windows whether the destination
is a directory or a file.
* Remove the [Output] attribute for the Destination property, this way XVS doesn't
automatically try to create an output file for whatever the destination is.
* Add another CopiedFiles output property, which contains all the copied files
(and only files), so that XVS mirrors this with output files on Windows.
Fixes part of https://dev.azure.com/devdiv/DevDiv/_workitems/edit/1505990/.
When XVS creates output/stamp files on Windows, the paths must be relative,
because otherwise XVS will append the full macOS path to the current Windows
directory and get garbage.
XVS also expects only files as output, so don't return any directories.
Fixes part of https://dev.azure.com/devdiv/DevDiv/_workitems/edit/1505990/.
* Fix resolving paths to required test files (test files can be found relative to the root path of the repository, not relative to where Xamarin.Mac is installed)
* Don't try to sign symlinks - we can end up trying to sign the target of the symlink twice simultaneously.
* Fix finding libxammac.dylib and Xamarin.Mac.dll when testing a system installation (when MAC_DESTDIR or TESTS_USE_SYSTEM are set).
* Remove a few .NET tests we don't need anymore.
Fixes https://github.com/xamarin/maccore/issues/2560.
Co-authored-by: Manuel de la Pena <mandel@microsoft.com>
Sometimes we want to copy the entire input directory from Windows to the Mac
when executing the Ditto task remotely, and sometimes we don't.
In particular we do not want to copy the input directory when the directory on
Windows is an incomplete mirror of what's on the Mac - one scenario being when
copying the app bundle to prepare for IPA creation. The .app directory on
Windows is not complete - all the files are there (maybe? not quite sure, but
that's beside the point here), but some may be empty, because when we only
care about the timestamp for a file, we'll create an empty file on Windows to
mirror the actual file on Mac. Copying this incomplete directory to the Mac,
overwriting the correct files there, will break things badly.
However, sometimes we're not mirroring a directory on Windows, but instead we
have directories as actual build input (for instances frameworks from NuGets),
and in that case we want to copy everything to the Mac.
So this PR adds a parameter to the Ditto task to optionally copy the directory
from Windows for remote builds, and we enable this behavior when we want it -
specifically when copying frameworks.
Fixes https://devdiv.visualstudio.com/DevDiv/_workitems/edit/1506009 while not
regressing https://dev.azure.com/devdiv/DevDiv/_workitems/edit/1492635.
Ref: https://devdiv.visualstudio.com/DevDiv/_workitems/edit/1506009
Ref: https://dev.azure.com/devdiv/DevDiv/_workitems/edit/1492635
Ref: https://github.com/xamarin/xamarin-macios/pull/14375
Updated to 1.5.8, including fixes for a hang in build cancellation during SayHello and also an improvement in the port forwarding logic when establishing the SSH connection
Change dSYM generation and native stripping to occur immediately before code signing,
in a newly minted post processing target.
Challenges:
* Both calling 'strip' and 'codesign' on an executable modifies that executable,
which means that we must make sure to not call 'dsymutil' on the same binary at
a later point unless it's been rebuilt.
* Thus we must make sure to update 'dsymutil's stamp file whenever we call 'strip'
and/or 'codesign' on an executable.
* Just like for code signing, we must store the libraries (either static or dynamic)
we post process in extension/watch/rid-specific projects, so that these libraries
can be loaded in containing projects and processed there.
* In universal .NET builds, debug symbols are created for the universal app bundle,
not for each rid-specific version of the app bundle. So I had to add logic to create
the native symbol lists (MtouchSymbolsList) for each rid-specific build, but then
collect them and merge those lists for the universal app bundle.
The existing SymbolStrip call we did right after linking the native executable has
been removed, because we have to do that after creating the dSYM (which the GenerateDebugSymbols
target does).
Also add tests.
Fixes https://github.com/xamarin/xamarin-macios/issues/14067.
Also establish when we should do post processing: only in the outermost build. This
is a slight change from previous behavior, where we'd run strip/dsymutil separately
for app extensions and watch apps.
* Save all the NativeReference metadata in binding resource packages.
* Copy all the NativeReference metadata to new items when resolving native references.
This makes it possible to set custom metadata on NativeReferences, and have that
metadata show up when it's needed, which might not be in the same project (for instance
if the native reference is in a binding project, we might want the custom metadata
when we load the native references from the binding project's resource package -
another case is when app extensions have native references, we might want any custom
metadata in the main executable project to know how to handle certain types of native
references).
Also sort the metadata we write to binding resource packages, so that the output
is stable. This required updating the corresponding tests.
The main theme here is that code signing will be done in the outermost
executable project, not in any app extension projects or watch projects, nor
during the RID-specific build of a .NET universal app. This makes codesigning
easier to reason about and other affected logic (such as strip/dsymutil)
easier to handle, in particular for .NET universal apps. Another benefit is
that the differences between the iOS and macOS code bases have been
eliminated.
The first step is to collect all the information we need from the targets
files. Every app bundle (be it app extension, watch app or main app) will add
its own output app bundle (.app/.appex) to the _CodesignBundle item group.
Then every app bundle will load this informarion from referenced app bundles,
and finally store this information on disk (in the 'codesign-bundle.items'
file). This means that in the end the main app bundle will have a list of all
contained app bundles in the app (recursively), in the _CodesignBundle item
group.
Separately we keep a list of other items that need signing, in the
_CodesignItems item group, and we do the same store/load logic for every
contained/contained app bundle (in the 'codesign.items' file, so a the end the
main app bundle will have a list of all the _CodesignItems for all contained
app bundles (recursively).
The previous steps occur in the _CollectCodesigningData and
_StoreCodesigningData targets.
The next step is to use the new ComputeCodesignItems task to compute
everything we need to know for code signing. This task takes over the
responsibility for listing all the *.dylib and *.metallib files, and the
*.framework directories in the app bundles, that need signing (which was
previously done in the targets file). This logic is significantly easier to
write, debug and test in C# than MSBuild.
In addition the ComputeCodesignItems also figures out a stamp file path we use
to determine if something needs (re-)signing. Previously .framework
directories did not have a stamp location, so they'd always end up resigned in
a rebuild, while now we'll automatically skip signing *.framework directories
unless something changed in them.
I've also tried to comment everything thorougly, for the next poor soul having
to deal with any bugs.
Behavioral differences:
* We were always signing *.dylib files for macOS. We're now doing the same
thing for all platforms.
* We're now always signing *.framework directories for all platforms (like we
do for *.dylib files), since frameworks are pretty much like dylibs anyways.
I've verified that this works both by running the submission tests and running
and launching a sample project on device from Windows.
The ComputeCodesignItems does not touch any files, and all the input files
should already exist on the mac, so there's no need to copy files back and
forth.
Mac Catalyst and macOS projects support an 'EnableCodeSigning' property to determine
whether an app is signed or not. In order to bring parity on mobile platforms, add
support for this property for iOS, tvOS and watchOS projects as well - if not set,
we'll still have the old behavior of signing device builds and not signing simulator builds.
Co-authored-by: Manuel de la Pena <mandel@microsoft.com>
The CodesignEntitlements and CodesignResourceRules properties can be relative paths,
and they might be coming from a referenced project. This means that if they're relative
paths, we must resolve them to a full path using the project that defined them (which
is specified using the 'SourceProjectPath' metadata).
The main theme here is that code signing will be done in the outermost executable
project, not in any app extension projects or watch projects, nor during the RID-specific
build of a .NET universal app. This makes codesigning easier to reason about and
other affected logic (such as strip/dsymutil) easier to handle, in particular for
.NET universal apps. Another benefit is that the differences between the iOS and
macOS code bases have been eliminated.
The first step is to collect all the information we need from the targets files.
Every app bundle (be it app extension, watch app or main app) will add its own output
app bundle (.app/.appex) to the _CodesignBundle item group. Then every app bundle
will load this informarion from referenced app bundles, and finally store this information
on disk (in the 'codesign-bundle.items' file). This means that in the end the main
app bundle will have a list of all contained app bundles in the app (recursively),
in the _CodesignBundle item group.
Separately we keep a list of other items that need signing, in the _CodesignItems
item group, and we do the same store/load logic for every contained/contained app
bundle (in the 'codesign.items' file, so a the end the main app bundle will have
a list of all the _CodesignItems for all contained app bundles (recursively).
The previous steps occur in the _CollectCodesigningData and _StoreCodesigningData
targets.
The next step is to use the new ComputeCodesignItems task to compute everything we
need to know for code signing. This task takes over the responsibility for listing
all the *.dylib and *.metallib files, and the *.framework directories in the app
bundles, that need signing (which was previously done in the targets file). This
logic is significantly easier to write, debug and test in C# than MSBuild.
In addition the ComputeCodesignItems also figures out a stamp file path we use to
determine if something needs (re-)signing. Previously .framework directories did
not have a stamp location, so they'd always end up resigned in a rebuild, while now
we'll automatically skip signing *.framework directories unless something changed
in them.
I've also tried to comment everything thorougly, for the next poor soul having to
deal with any bugs, as well has adding a comprehensive test for the new task.
Behavioral differences:
* We were always signing *.dylib files for macOS. We're now doing the same thing
for all platforms.
* We're now always signing *.framework directories for all platforms (like we do
for *.dylib files), since frameworks are pretty much like dylibs anyways.
I'm removing the old logic first, because the new logic is so different that a diff
just complicates understanding what's happening (there's not much value in comparing
textually what's changed when it's pretty much a complete rewrite).
* [DittoTask] If Source is a folder, add all its content as ITaskItem
* [DittoTask] Undoing undo...
* Accidentally hit Cmd+Z and removed an using statement
Co-authored-by: Manuel de la Pena <mandel@microsoft.com>
Co-authored-by: Alex Soto <alex@alexsoto.me>
For reasons I don't quite understand, ditto might fail when executed by XMA:
Target Name=_CopyDirectoriesToBundle Project=C:\Users\rolf\source\iOSApp4\iOSApp4.csproj
Building target "_CopyDirectoriesToBundle" completely.
Output file "bin\Debug\net6.0-ios\iossimulator-x64\publish\..\device-builds\iphone11.6-14.8.1\iOSApp4.app\\Frameworks\\ArcGIS-arm64.framework/ArcGIS-arm64" does not exist.
Output file "bin\Debug\net6.0-ios\iossimulator-x64\publish\..\device-builds\iphone11.6-14.8.1\iOSApp4.app\\Frameworks\\Runtimecore.framework/Runtimecore" does not exist.
Ditto
Assembly = C:\Users\rolf\source\maui\bin\dotnet\packs\Microsoft.iOS.Sdk\15.2.303-ci.ditto-windows.56\tools\msbuild\iOS\..\iOS\Xamarin.iOS.Tasks.dll
Parameters
Destination = bin\Debug\net6.0-ios\iossimulator-x64\publish\..\device-builds\iphone11.6-14.8.1\iOSApp4.app\\Frameworks\\ArcGIS-arm64.framework
TouchDestinationFiles = True
SessionId = <SessionId>
Source = C:\Users\rolf\.nuget\packages\esri.arcgisruntime.runtimes.ios\100.13.0\framework\ios-arm64\native\ArcGIS-arm64.framework\
Ditto: <timestamp> - Started
Ditto: <timestamp> - Initializing
[xma]: Trying to get a Build Connection for Session '<SessionId>': Xamarin.Messaging.Build.Client.BuildConnection.<SessionId>, Lifetime: Build
Ditto: <timestamp> - Initialized
Ditto: <timestamp> - There's no available inputs to copy to the Mac
Ditto: <timestamp> - Serializing intputs
Ditto: <timestamp> - Executing
[xma]: Starting remote task execution for 'iOSApp4': Xamarin.MacDev.Tasks.Ditto
[xma]: Sending Request Xamarin.Messaging.Build.Contracts.ExecuteTaskMessage to topic xvs/build/execute-task/iOSApp4/15f3833002fDitto
[xma]: Received Response of Xamarin.Messaging.Build.Contracts.ExecuteTaskMessage to topic build<SessionId>4396rolf/+/xvs/build/execute-task/iOSApp4/15f3833002fDitto
Ditto: <timestamp> - Logging messages
/usr/bin/ditto C:/Users/rolf/.nuget/packages/esri.arcgisruntime.runtimes.ios/100.13.0/framework/ios-arm64/native/ArcGIS-arm64.framework/ bin/Debug/net6.0-ios/iossimulator-x64/publish/../device-builds/iphone11.6-14.8.1/iOSApp4.app//Frameworks//ArcGIS-arm64.framework
ditto: bin/Debug/net6.0-ios/iossimulator-x64/publish/../device-builds/iphone11.6-14.8.1/iOSApp4.app//Frameworks//ArcGIS-arm64.framework: File exists
Errors
C:\Users\rolf\source\maui\bin\dotnet\packs\Microsoft.iOS.Sdk\15.2.303-ci.ditto-windows.56\targets\Xamarin.Shared.Sdk.targets(668,3): error MSB6006: "ditto" exited with code 1. [C:\Users\rolf\source\iOSApp4\iOSApp4.csproj]
Ditto: <timestamp> - Finished
This doesn't happen when building on macOS, nor if I copy the offending ditto
command and execute it manually on macOS.
Since I don't know why the problem occurs in the first place, I don't know why
passing full paths to 'ditto' works either. It shouldn't cause problems
elsewhere though.
Ref: https://github.com/xamarin/xamarin-macios/issues/13665
* Enable nullability and fix code accordingly.
* Augment it to be able to take multiple files to run dsymutil on at the same time.
* Execute using xcrun (ref: #3931)
* Pass the full path to the executable file to dsymutil, to make command lines
easier to copy-paste.
* Enable nullability and fix code accordingly.
* Augment it to be able to take multiple files to strip at the same time.
* Strip in parallel.
* Execute using xcrun (ref: #3931)
* Pass the full path to the executable file to strip, to make command lines
easier to copy-paste.
* Remove test that is now outdated. We have other tests that run strip
anyways, so this shouldn't be a problem.
This fixes an issue where we'd do logic with Windows-style paths on macOS, and that's
never the right thing to do.
For the LinkNativeCode task, this would manifest as this error when building from windows:
> ld: file too small (length=0) file 'obj/Debug/net6.0-ios/iossimulator-x64/nativelibraries/libSystem.Native.dylib' for architecture x86_64
because the 'ShouldCopyToBuildServer' method would return incorrect results.
For the Codesign task, it would manifest as an exception trying to create a
directory with an empty string (because the directory name of a windows-style
path is an empty string on macOS).
Since this exception was quite useless (just getting the exception message
didn't tell me much about what caused the exception, because it had no stack
trace information), I've also improved error reporting in both of these tasks.
Make the CollectBundleResourcesDependsOn property public, so that custom
targets can inject themselves into the build early enough to add additional
BundleResource or Content items (by adding their custom target's name to the
CollectBundleResourcesDependsOn property).
Fixes https://github.com/xamarin/xamarin-macios/issues/11984.
These properties aren't used.
This also allows us to remove the CompiledArchitectures output property from
the MTouch task, because it's not longer used anymore either.
Also change the key for our Info.plist entry with the version number in .NET, and document the change.
We now use "com.microsoft.<platform in lower case>" instead of "com.xamarin.ios" (for all platforms).
Fixes https://github.com/xamarin/xamarin-macios/issues/14108.
Co-authored-by: TJ Lambert <50846373+tj-devel709@users.noreply.github.com>
* Remove ObjCRuntime.nfloat (in favor of System.Runtime.InteropServices.NFloat).
* Automatically add a reference to the System.Runtime.InteropServices.Internal
package, so that developers get the new NFloat API (with operators) we've
added post .NET 6 (but don't do this for .NET 7).
* Automatically add a global using alias for
System.Runtime.InteropServices.NFloat -> nfloat. This is not behind the
usual `ImplicitUsings` condition our other implicit usings are, because
they're off by default for existing projects, and the main target for the
global using alias for nfloat is upgraded projects.
* Automatically generate a global using alias (like above) in the generator
for all code the generator compiles.
* Update xtro entries to reference System.Runtime.InteropServices.NFloat
instead of ObjCRuntime.nfloat.
* Add a workaround for a hopefully temporary issue with .NET/CoreCLR where the
wrong runtime pack is selected otherwise (without the new NFloat API, so
nothing works at runtime).
Ref: https://github.com/xamarin/xamarin-macios/issues/13087
This way it's easier to copy-paste the path to the these files from terminal output
and open/run it (with a relative/partial path you'll need to know the current directory,
which is just an annoying thing to figure out sometimes).
Rename our product assemblies to:
* Microsoft.iOS.dll
* Microsoft.tvOS.dll
* Microsoft.macOS.dll
* Microsoft.MacCatalyst.dll
This makes it easy to distinguish between legacy Xamarin and .NET whenever the
product assembly is mentioned, and I've also chosen the platform part of the
name to match how the platforms are named elsewhere (this also makes it
possible to simplify our build logic, since we can remove a lot of special
casing).
Fixes https://github.com/xamarin/xamarin-macios/issues/13748.
Add support for the PublishFolderType metadata on Content and BundleResource
items, which makes it possible to change the location in the app bundle for
these items (this was possible to do before with the Link metadata), but most
importantly it also makes it possible to choose to *not* bundle these items in
the app bundle (which was not possible before with the Link metadata, nor any
other means).
At first I thought setting CopyToPublishDirectory /
CopyToOutputDirectory=Never would accomplish that, but it turns out we don't
honor those, and since we already have this behavior of ignoring
CopyToPublishDirectory / CopyToOutputDirectory in legacy Xamarin, I didn't
want to change it in .NET.
So I added support for honoring the PublishFolderType metadata instead, which
is new and we already support for other item groups. This is accomplished by
adding all Content and BundleResource items with PublishFolderType set to the
ResolvedFileToPublish item group (where we already handle any
PublishFolderType values), and then we ignore such Content and BundleResource
items in our CollectBundleResources task.
Also update the documentation and add tests.
Hopefully fixes random build failures like this:
> xamarin-macios/msbuild/Versions.ios.g.cs(3,1): error CS1022: Type or namespace definition, or end-of-file expected
Not embedding third-party libraries in the binding assembly is the future, and
let's try to enable it by default starting with .NET.
Fixes https://github.com/xamarin/xamarin-macios/issues/12530.
Co-authored-by: Manuel de la Pena <mandel@microsoft.com>
* Changes covered by this PR:
- Removed unused references
- Removed Xamarin.PreBuilt.iOS.app.zip since it's now built and included automatically as part of the CI build
- Added missing signing and versioning
- Replaced DotNetZip with System.IO.Compression.ZipFile: now that .net provides its own zip implementation we no longer require DotNetZip reference for unzipping purposes
- Fixes the Tasks missing CodePages reference: we no longer need to replace the CodePages reference assembly by its implementation, since this project already specifies the `win` RID. The replacement was failing because `RuntimeTargetsCopyLocalItems` is empty now that the RID is set, so we ended up removing the reference.
Co-authored-by: Emanuel Fernandez Dell'Oca <ema@xamarin.com>
We must set `ResolveAssemblyConflicts=true` before loading
Microsoft.Common.targets (which is loaded by Microsoft.CSharp.targets),
because otherwise we won'd do any conflict resolution at all, since the
variable isn't 'true' when it needs to be.
Also add test.
Fixes https://github.com/xamarin/xamarin-macios/issues/11691.
* Add support for specifying metadata on items that are to be codesigned to
override any general codesign setting.
* Make the Codesign task able to sign files and directories that may depend on
eachother.
Implement support for ordering signing so that directories containing files
that also must be signed are signed after those files.
This is implemented by:
1. Normalize all input (resolve symlinks, create full path, etc.)
2. Sort by path length (longest to shortest paths). This way we're certain
that if we find a directory, we'll know that we won't find any files
later in the list from inside that directory.
3. Group into an ordered list of buckets, where each bucket contains files
and directories that don't depend on eachother (i.e. they can all be
signed in parallel).
This makes it possible to call Codesign once, listing both the app bundle
itself, and all the individual files or directories inside that need
signing, and the Codesign task will sign the items in an order that ensures
parent directories are always signed after any files or directories inside.
* Finally rework code signing to sign everything with a single call to the
Codesign task in the _CodesignAppBundle target for the executable project,
instead of having multiple calls to the Codesign task from multiple targets
(and projects). This makes it easier to reason about what's being signed,
and it also makes it easier to add files to the signing process.
* Also make the Codesign task able to figure out if something needs to be
signed, and change the _CodesignAppBundle target to not keep track of
inputs/outputs, because it becomes quite complex (for directories, it needs
to keep track of all the files inside that directory, and also if there's
anything in the directory that's also being signed).
The targets were somewhat different, where the Mac version seemed much less
evolved than the iOS version, so this meant mostly update the iOS version to
be Mac-compatible as well (and use it for macOS).
The FilterStaticFrameworks task takes a list of frameworks to process, but the
files actually needed by the task are inside those frameworks, which means we
have to tell XVS that those have to be copied to the Mac for the build as well.
Fixes https://github.com/xamarin/xamarin-macios/issues/13804.
In .NET, all files that should be published (put into the final .app bundle) are put into the @(ResolvedFileToPublish) item group, and at the end of the build process, .NET will publish all the files in that item group. Which files are in this item group, and how they're put in there, is out of our control (it's just how the build process works in .NET), so we'll have to cope.
Additionally, publishing an app for Apple platforms is different than publishing other .NET apps, because we can't just put all the files in the a directory like .NET usually does, we have a fairly strict directory structure we need to follow, and it also differs between platforms (the structure is different between macOS and iOS for instance).
This means that for every file in the `ResolvedFileToPublish` item group, we have to figure out:
* Should it be put into the app bundle in the first place?
* If so, in which subdirectory (if any)?
This PR implements these changes. The exact details are explained in a document in the PR, but the general logic is:
* We make an educated guess for some types of files we know about (assemblies, unmanaged libraries, images, etc).
* We provide a way to set metadata on each item specifying which type of item it is (assembly, unmanaged library, image, etc), and we'll treat the item as such. This method can also be used to override the guess we made (for files that shouldn't be published for instance).
* We warn if we run into files we're not educated enough to be able to guess about, and for which there's no custom metadata set.
Fixes https://github.com/xamarin/xamarin-macios/issues/12572.
Touch all files when copying an app extension into the app.
This fixes an issue where we'd in some cases run into this scenario:
* Build once (successfully), and sign the app (and app extension), including all dylibs.
* We build again, copy the (unsigned) app extension into the app again, but
not sign dylibs because the timestamp went back into the past.
This fixes the following problem:
* App with framework is built and signed.
* App is rebuilt, and the framework is copied in again.
* This time, the framework's executable's timestamp will be earlier than the
timestamp when it was last signed, and as such it won't be signed again.
Fix this by touching all the copied files when copying a directory to the app bundle.
The symlink might point to a file that has been updated, and if that file
needs more processing and the logic checks the symlink to check if the file is
up-to-date, then we need to reflect that in the symlink (for instance if a
framework binary is updated (which is a symlink on some platforms), it needs
to be resigned, even if the symlink itself didn't change).
We only want the file name of the app manifest path to define the outputs because the app manifest will be always located at the app bundle root level.
For MAUI, the app manifest source file could be located on a sub-folder (like \Platforms\iOS\Info.plist) so if we take the full value of the property we will end up looking the manifest in the wrong place inside the app bundle, causing the inputs/outputs checks to fail and forcing many tasks to always run (like code sign), also causing unwanted behaviors like breaking incremental builds and incremental deployments
Co-authored-by: Alex Soto <alex@alexsoto.me>
That's what the _FrameworkNativeReference item group contains: framework binaries
instead of the framework directory itself. This makes us process frameworks in binding
packages correctly.
Previously we'd only support passing in a fake reference to a file inside the *.xcframework.
This makes some other code simpler later on.
Best viewed by ignoring whitespace, since this is mostly whitespace changes.
named 'Info.plist', and assume that's the app manifest.
That doesn't quite work when we end up with multiple 'Info.plist' entries in any
of those item groups (one example being a framework as a BundleResource - all frameworks
have an Info.plist, and there's no good way to distinguish what the developer's intention
was).
So:
1. Implement a 'AppManifestDetectionEnabled' property to disable automatic app manifest
detection.
2. Add a public 'AppBundleManifest' property that specifies the app manifest
(this is just a renamed version of our previously private '_AppManifest' property).
This makes it possible for app developers to:
* Disable automatic app manifest detection.
* Still have an app manifest by specifying it manually.
* Disable automatic app manifest detection, but also not specify an app manifest
manually (so no custom app manifest at all).
Also:
* Rename '_AppBundleManifest' to '_AppBundleManifestPath' to make it less confusing
with the new 'AppBundleManifest' property.
Symlinks to directories are treated the same as other symlinks (as files), not
as directories. This way we don't end up re-creating a directory hierarchy
when we only have to create a symlink.
There can't be any files in the root directory of the app bundle for macOS and
Mac Catalyst, otherwise code signing will fail. The problem is that Mono will
create a crash report in the current directory if the process crashes, and the
current directory is the root directory of the app bundle, which means that if
running an app crashes, the next build will likely fail because of the crash
report.
We had logic to detect this and remove any crash reports, but our crash report
detection pattern wasn't good enough and let some files through. This PR
updates that pattern, and also improves the code to report warnings for any
other files in the app bundle's root directory.
The GetNuGetShortFolderName task only exists in .NET builds, which means we
have to find another way to compute the path to resources in nugets for legacy
builds. Do it the simple way by hardcoding the possible values.
Fixes https://github.com/xamarin/xamarin-macios/issues/13469.
* [tests] Create a libtest.xcframework and libtest2.xcframework
* [tests] Make bindings-test and bindings-test2 use an xcframework instead of plain static library
* [msbuild] Add support for xcframeworks with static libraries in them.
* List the frameworks libtest needs.
* [tests] Update .NET unit tests according to test project changes.
* [tests] Add new test to verify that packing an old-style binding project doesn't work.
It's not needed, and may in some cases do the wrong thing: for instance, this
logic will automatically reference any *.dll files in the project directory,
which breaks the build if those *.dll files aren't actually assemblies.
The '_AppExtensionRoot' contains the correct parent directory of the 'PlugIns'
directory for all platforms, so use that instead of appending 'PlugIns' to
'_AppBundlePath' - which is incorrect on macOS and Mac Catalyst, because the
'PlugIns' parent directory is '$(_AppBundlePath)/Contents' on those platforms.
Fixes https://github.com/xamarin/xamarin-macios/issues/13415.
This fixes an issue where we'd create the stamp file even if 'NoBindingEmbedding' wasn't set.
Also remove SkipBindingResourcePackage property, it doesn't show up anywhere
else in our code base, nor in the history, nor anywhere relevant in Google.
Build the Xamarin.PreBuilt.iOS app bundle instead of using a prebuilt bundle.
This makes sure that we're always using the latest BCL.
Some accurate build massaging was needed, because:
* To build the prebuilt app we need the iOS workload installed (into our build-local
.NET installation).
* The iOS workload contains the Microsoft.iOS.Windows.Sdk pack.
* The Microsoft.iOS.Windows.Sdk pack contains the prebuilt app.
Thus we had a circular reference. Fortunately, the Microsoft.iOS.Windows.Sdk pack
is only required on Windows, which means we can break this circular reference by:
* Mark Microsoft.iOS.Windows.Sdk pack as only to be installed on Windows (unfortunately
it seems we have to list the exact runtime identifiers for the platforms where
to install the pack, so we can't do something like "win-*", but new variations
of the "win-*" runtime identifier shouldn't show up all that often).
* Build the prebuilt app on macOS.
This way we don't need the Microsoft.iOS.Windows.Sdk pack when installing the iOS
workload locally.
The .NET build order is now:
* Build general sdk, runtime and ref packs for .NET.
* Build the prebuilt app.
* Build the Microsoft.iOS.Windows.Sdk pack.
Fixes https://github.com/xamarin/xamarin-macios/issues/12945.
* Submodule MonoTouch.Dialog.
Submodule MonoTouch.Dialog, so that we can easily build it using .NET. This
submodule will become redundant when/if we publish a .NET version of
MonoTouch.Dialog, but until that happens we need it at least for our own test
suites.
This also means we have to copy our NuGet.config and global.json files to the
MonoTouch.Dialog project directory so that we point msbuild to use our local
build.
New commits in spouliot/Touch.Unit:
* spouliot/Touch.Unit@cbda703 [Touch.Client] Use MonoTouch.Dialog from a submodule. (#109)
Diff: 3345db2f4e..cbda703583
* Use relative path for submodule.
And fix indentation and set the branch name.
* Don't use 'RootTestsDirectory' when it might not be defined yet.
* [tests] Our test projects don't need to reference MonoTouch.Dialog directly.
The projects get the MonoTouch.Dialog reference indirectly through the
Touch.Client project reference.
* [tests] Only validate unique errors in the .NET unit tests.
* [tests] No need to reference System.Json anymore, that's handled directly in the MonoTouch.Dialog project.
* [tests] Reference nunit.framework.targets so we get a workaround for an NUnit issue everywhere.
* [msbuild] Only try to create a package if we're able to create an app bundle.
This fixes an issue where a library project would try (and fail) to create a
package when 'CreatePackage=true' (which could be set for the executable
project, but inherited by the library project since the executable project
depends on it).
* [tests] Adjust PackTest.BindingXcFrameworksProject to not set the AssemblyName property.
MSBuild ends up being very confused when the project we're trying to build
depends on other projects, because AssemblyName is set for all the projects
being build, and MSBuild complains about ambiguous projects:
> error: Ambiguous project name 'bindings-xcframework-test'
* [net6] Fix ILStrip'ed apps to actually work again
- In a late minute change to the ILStrip PR (https://github.com/xamarin/xamarin-macios/pull/12563) a change to support XVS support broke execution of Apps that were stripped
- Applications were broken because none of the stripped assemblies were actually copied into the bundle
- However, the tests still passed, because all assemblies that were there had no IL (zero assemblies total)
Now why did this happen?
- The stripped assemblies were changed to return via an msbuild Output Element
- Output Element can return an Property or ItemGroup, depending if you use the PropertyName or ItemName attributes
- Unfortunately I used PropertyName, when I expected an ItemGroup. So I silently had a property created instead.
- Thus zero items were added to the list of files to copy into the bundle
- Which was undetected as the test did not confirm files were copied in, and manual tests were not run so late into the PR (3 weeks after PR was opened)
How was it fixed?
- Correctly using ItemName on Output created a valid item group to reference
- However, that still failed with an absurdly confusing error:
PATH/Microsoft.NET.Publish.targets(277,5): error MSB3024: Could not copy the file FILE to the destination file PATH, because the destination is a folder instead of a file. To copy the source file into a folder, consider using the DestinationFolder parameter instead of DestinationFiles.
- After a splunking through netcore targets, I found the metadata on these assemblies references really matters. Without it, they are not processed correctly at all.
- Thus, I updated ILStripBase to clone the existing metadata when changing the original assembly reference to the stripped path
- Finally, I corrected the test to assert that required files are copied in. I also manually ran our device test.
Co-authored-by: Rolf Bjarne Kvinge <rolf@xamarin.com>
Add support for 'dotnet pack', by:
1. Add a workaround for the fact that as soon as a project has a
'NativeReference' item, .NET's MSBuild logic wants to include a
'Native.$(AssemblyName).manifest' file in the NuGet. This obviously breaks,
because we don't create such a file, so we work around it by removing the
file in question from the corresponding item groups.
2. Add any binding resource packages to the NuGet.
3. Add tests.
Fixes https://github.com/xamarin/xamarin-macios/issues/12631.
Fixes these warnings, which we don't care about:
CSC : warning CS8002: Referenced assembly 'Xamarin.MacDev.Tasks.Core, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null' does not have a strong name. [(...)/xamarin-macios/msbuild/Xamarin.iOS.Tasks/Xamarin.iOS.Tasks.csproj]
CSC : warning CS8002: Referenced assembly 'Xamarin.iOS.Tasks.Core, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null' does not have a strong name. [(...)/xamarin-macios/msbuild/Xamarin.iOS.Tasks/Xamarin.iOS.Tasks.csproj]
CSC : warning CS8002: Referenced assembly 'Xamarin.MacDev.Tasks, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null' does not have a strong name. [(...)/xamarin-macios/msbuild/Xamarin.iOS.Tasks/Xamarin.iOS.Tasks.csproj]
CSC : warning CS8002: Referenced assembly 'Xamarin.Localization.MSBuild, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null' does not have a strong name. [(...)/xamarin-macios/msbuild/Xamarin.iOS.Tasks/Xamarin.iOS.Tasks.csproj]
CSC : warning CS8002: Referenced assembly 'Xamarin.MacDev.Tasks.Core, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null' does not have a strong name. [(...)/xamarin-macios/msbuild/Xamarin.Mac.Tasks/Xamarin.Mac.Tasks.csproj]
CSC : warning CS8002: Referenced assembly 'Xamarin.Mac.Tasks.Core, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null' does not have a strong name. [(...)/xamarin-macios/msbuild/Xamarin.Mac.Tasks/Xamarin.Mac.Tasks.csproj]
Augment the CreateBindingResourcePackage to support creating a zipped binding
resource package (which is just a zipped version of the binding resource
package). This can either be manually chosen by the new 'Compressed' property,
or automatically detected (create a zipped version when there's a symlink in
the binding resource package).
The default is to not create a zipped version in legacy Xamarin, and
automatically detect for .NET.
The problem this is trying to solve is when creating a NuGet package - NuGet
doesn't handle symlinks correctly and it's not possible to create a NuGet with
symlinks. Instead we need to create a zipfile with all the binding resources.
The default has been chosen so that we automatically create a zip file when
it's required for .NET, while still maintaining old behavior with legacy
Xamarin.
We don't need to compile project-level assets for every RuntimeIdentifier in
multi-rid builds, we can instead compile them just once in the outer build.
There is also a correctness issue here: we can't compile assets more than once
and expect to get the exact same compiled result every time (in particular
actool seems to be adding random bytes in to the compiled output), and this
creates a problem when trying to merge the different runtime-specific compiled
output into a universal binary.
We accomplish this by:
* Processing these assets in the outer build, before we execute the
rid-specific inner builds.
* Store the paths to the assets we've processed in a file.
* In the inner builds, we read that file, and remove any matches from the
corresponding item group.
* Make sure to copy the compiled assets to the app bundle at the end of the
outer build.
These are the assets we currently handle this way:
* BundleResource
* ImageAsset
* InterfaceDefinition
* SceneKitAsset
* Collada
* TextureAtlas
* CoreMLModel
Also:
* Add a new test case (AppWithResource) that contains all these different
types of assets.
* Add support for the ScnTool task on Mac Catalyst (which the new test case
revealed was missing).
Fixes https://github.com/xamarin/xamarin-macios/issues/12410.
Fixes this problem:
error MSB4018: The "FindItemWithLogicalName" task failed unexpectedly.
error MSB4018: System.IO.FileLoadException: Could not load file or assembly 'Xamarin.Localization.MSBuild, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null' or one of its dependencies. A strongly-named assembly is required. (Exception from HRESULT: 0x80131044)
error MSB4018: File name: 'Xamarin.Localization.MSBuild, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null'
error MSB4018: at Xamarin.MacDev.Tasks.FindItemWithLogicalNameTaskBase.Execute()
error MSB4018: at Microsoft.Build.BackEnd.TaskExecutionHost.Microsoft.Build.BackEnd.ITaskExecutionHost.Execute()
error MSB4018: at Microsoft.Build.BackEnd.TaskBuilder.<ExecuteInstantiatedTask>d__26.MoveNext()
We need to strongname our MSBuild assemblies, so that different versions
can be loaded side-by-side (one example being having both a legacy and a
.NET project in the same solution).
This required setting a version for Xamarin.iOS.Tasks.dll and
Xamarin.Mac.Tasks.dll, otherwise strong-naming won't work properly (all
versions of an assembly would have the same identity).
Also sign the corresponding test assemblies, since they poke into the
internals of the task assemblies.
Fixes https://github.com/xamarin/xamarin-macios/issues/9835.
Rolf Bjarne Kvinge
Mauro Agnoletti
Emanuel Fernandez Dell'Oca
VS MobileTools Engineering Service 2
Manuel de la Pena
Israel Soto
Mauro Agnoletti
TJ Lambert
Chris Hamons