* Changes covered by this PR:
- Removed unused references
- Removed Xamarin.PreBuilt.iOS.app.zip since it's now built and included automatically as part of the CI build
- Added missing signing and versioning
- Replaced DotNetZip with System.IO.Compression.ZipFile: now that .net provides its own zip implementation we no longer require DotNetZip reference for unzipping purposes
- Fixes the Tasks missing CodePages reference: we no longer need to replace the CodePages reference assembly by its implementation, since this project already specifies the `win` RID. The replacement was failing because `RuntimeTargetsCopyLocalItems` is empty now that the RID is set, so we ended up removing the reference.
Co-authored-by: Emanuel Fernandez Dell'Oca <ema@xamarin.com>
We must set `ResolveAssemblyConflicts=true` before loading
Microsoft.Common.targets (which is loaded by Microsoft.CSharp.targets),
because otherwise we won'd do any conflict resolution at all, since the
variable isn't 'true' when it needs to be.
Also add test.
Fixes https://github.com/xamarin/xamarin-macios/issues/11691.
* Add support for specifying metadata on items that are to be codesigned to
override any general codesign setting.
* Make the Codesign task able to sign files and directories that may depend on
eachother.
Implement support for ordering signing so that directories containing files
that also must be signed are signed after those files.
This is implemented by:
1. Normalize all input (resolve symlinks, create full path, etc.)
2. Sort by path length (longest to shortest paths). This way we're certain
that if we find a directory, we'll know that we won't find any files
later in the list from inside that directory.
3. Group into an ordered list of buckets, where each bucket contains files
and directories that don't depend on eachother (i.e. they can all be
signed in parallel).
This makes it possible to call Codesign once, listing both the app bundle
itself, and all the individual files or directories inside that need
signing, and the Codesign task will sign the items in an order that ensures
parent directories are always signed after any files or directories inside.
* Finally rework code signing to sign everything with a single call to the
Codesign task in the _CodesignAppBundle target for the executable project,
instead of having multiple calls to the Codesign task from multiple targets
(and projects). This makes it easier to reason about what's being signed,
and it also makes it easier to add files to the signing process.
* Also make the Codesign task able to figure out if something needs to be
signed, and change the _CodesignAppBundle target to not keep track of
inputs/outputs, because it becomes quite complex (for directories, it needs
to keep track of all the files inside that directory, and also if there's
anything in the directory that's also being signed).
The targets were somewhat different, where the Mac version seemed much less
evolved than the iOS version, so this meant mostly update the iOS version to
be Mac-compatible as well (and use it for macOS).
The FilterStaticFrameworks task takes a list of frameworks to process, but the
files actually needed by the task are inside those frameworks, which means we
have to tell XVS that those have to be copied to the Mac for the build as well.
Fixes https://github.com/xamarin/xamarin-macios/issues/13804.
In .NET, all files that should be published (put into the final .app bundle) are put into the @(ResolvedFileToPublish) item group, and at the end of the build process, .NET will publish all the files in that item group. Which files are in this item group, and how they're put in there, is out of our control (it's just how the build process works in .NET), so we'll have to cope.
Additionally, publishing an app for Apple platforms is different than publishing other .NET apps, because we can't just put all the files in the a directory like .NET usually does, we have a fairly strict directory structure we need to follow, and it also differs between platforms (the structure is different between macOS and iOS for instance).
This means that for every file in the `ResolvedFileToPublish` item group, we have to figure out:
* Should it be put into the app bundle in the first place?
* If so, in which subdirectory (if any)?
This PR implements these changes. The exact details are explained in a document in the PR, but the general logic is:
* We make an educated guess for some types of files we know about (assemblies, unmanaged libraries, images, etc).
* We provide a way to set metadata on each item specifying which type of item it is (assembly, unmanaged library, image, etc), and we'll treat the item as such. This method can also be used to override the guess we made (for files that shouldn't be published for instance).
* We warn if we run into files we're not educated enough to be able to guess about, and for which there's no custom metadata set.
Fixes https://github.com/xamarin/xamarin-macios/issues/12572.
Touch all files when copying an app extension into the app.
This fixes an issue where we'd in some cases run into this scenario:
* Build once (successfully), and sign the app (and app extension), including all dylibs.
* We build again, copy the (unsigned) app extension into the app again, but
not sign dylibs because the timestamp went back into the past.
This fixes the following problem:
* App with framework is built and signed.
* App is rebuilt, and the framework is copied in again.
* This time, the framework's executable's timestamp will be earlier than the
timestamp when it was last signed, and as such it won't be signed again.
Fix this by touching all the copied files when copying a directory to the app bundle.
The symlink might point to a file that has been updated, and if that file
needs more processing and the logic checks the symlink to check if the file is
up-to-date, then we need to reflect that in the symlink (for instance if a
framework binary is updated (which is a symlink on some platforms), it needs
to be resigned, even if the symlink itself didn't change).
We only want the file name of the app manifest path to define the outputs because the app manifest will be always located at the app bundle root level.
For MAUI, the app manifest source file could be located on a sub-folder (like \Platforms\iOS\Info.plist) so if we take the full value of the property we will end up looking the manifest in the wrong place inside the app bundle, causing the inputs/outputs checks to fail and forcing many tasks to always run (like code sign), also causing unwanted behaviors like breaking incremental builds and incremental deployments
Co-authored-by: Alex Soto <alex@alexsoto.me>
That's what the _FrameworkNativeReference item group contains: framework binaries
instead of the framework directory itself. This makes us process frameworks in binding
packages correctly.
Previously we'd only support passing in a fake reference to a file inside the *.xcframework.
This makes some other code simpler later on.
Best viewed by ignoring whitespace, since this is mostly whitespace changes.
named 'Info.plist', and assume that's the app manifest.
That doesn't quite work when we end up with multiple 'Info.plist' entries in any
of those item groups (one example being a framework as a BundleResource - all frameworks
have an Info.plist, and there's no good way to distinguish what the developer's intention
was).
So:
1. Implement a 'AppManifestDetectionEnabled' property to disable automatic app manifest
detection.
2. Add a public 'AppBundleManifest' property that specifies the app manifest
(this is just a renamed version of our previously private '_AppManifest' property).
This makes it possible for app developers to:
* Disable automatic app manifest detection.
* Still have an app manifest by specifying it manually.
* Disable automatic app manifest detection, but also not specify an app manifest
manually (so no custom app manifest at all).
Also:
* Rename '_AppBundleManifest' to '_AppBundleManifestPath' to make it less confusing
with the new 'AppBundleManifest' property.
Symlinks to directories are treated the same as other symlinks (as files), not
as directories. This way we don't end up re-creating a directory hierarchy
when we only have to create a symlink.
There can't be any files in the root directory of the app bundle for macOS and
Mac Catalyst, otherwise code signing will fail. The problem is that Mono will
create a crash report in the current directory if the process crashes, and the
current directory is the root directory of the app bundle, which means that if
running an app crashes, the next build will likely fail because of the crash
report.
We had logic to detect this and remove any crash reports, but our crash report
detection pattern wasn't good enough and let some files through. This PR
updates that pattern, and also improves the code to report warnings for any
other files in the app bundle's root directory.
The GetNuGetShortFolderName task only exists in .NET builds, which means we
have to find another way to compute the path to resources in nugets for legacy
builds. Do it the simple way by hardcoding the possible values.
Fixes https://github.com/xamarin/xamarin-macios/issues/13469.
* [tests] Create a libtest.xcframework and libtest2.xcframework
* [tests] Make bindings-test and bindings-test2 use an xcframework instead of plain static library
* [msbuild] Add support for xcframeworks with static libraries in them.
* List the frameworks libtest needs.
* [tests] Update .NET unit tests according to test project changes.
* [tests] Add new test to verify that packing an old-style binding project doesn't work.
It's not needed, and may in some cases do the wrong thing: for instance, this
logic will automatically reference any *.dll files in the project directory,
which breaks the build if those *.dll files aren't actually assemblies.
The '_AppExtensionRoot' contains the correct parent directory of the 'PlugIns'
directory for all platforms, so use that instead of appending 'PlugIns' to
'_AppBundlePath' - which is incorrect on macOS and Mac Catalyst, because the
'PlugIns' parent directory is '$(_AppBundlePath)/Contents' on those platforms.
Fixes https://github.com/xamarin/xamarin-macios/issues/13415.
This fixes an issue where we'd create the stamp file even if 'NoBindingEmbedding' wasn't set.
Also remove SkipBindingResourcePackage property, it doesn't show up anywhere
else in our code base, nor in the history, nor anywhere relevant in Google.
Build the Xamarin.PreBuilt.iOS app bundle instead of using a prebuilt bundle.
This makes sure that we're always using the latest BCL.
Some accurate build massaging was needed, because:
* To build the prebuilt app we need the iOS workload installed (into our build-local
.NET installation).
* The iOS workload contains the Microsoft.iOS.Windows.Sdk pack.
* The Microsoft.iOS.Windows.Sdk pack contains the prebuilt app.
Thus we had a circular reference. Fortunately, the Microsoft.iOS.Windows.Sdk pack
is only required on Windows, which means we can break this circular reference by:
* Mark Microsoft.iOS.Windows.Sdk pack as only to be installed on Windows (unfortunately
it seems we have to list the exact runtime identifiers for the platforms where
to install the pack, so we can't do something like "win-*", but new variations
of the "win-*" runtime identifier shouldn't show up all that often).
* Build the prebuilt app on macOS.
This way we don't need the Microsoft.iOS.Windows.Sdk pack when installing the iOS
workload locally.
The .NET build order is now:
* Build general sdk, runtime and ref packs for .NET.
* Build the prebuilt app.
* Build the Microsoft.iOS.Windows.Sdk pack.
Fixes https://github.com/xamarin/xamarin-macios/issues/12945.
* Submodule MonoTouch.Dialog.
Submodule MonoTouch.Dialog, so that we can easily build it using .NET. This
submodule will become redundant when/if we publish a .NET version of
MonoTouch.Dialog, but until that happens we need it at least for our own test
suites.
This also means we have to copy our NuGet.config and global.json files to the
MonoTouch.Dialog project directory so that we point msbuild to use our local
build.
New commits in spouliot/Touch.Unit:
* spouliot/Touch.Unit@cbda703 [Touch.Client] Use MonoTouch.Dialog from a submodule. (#109)
Diff: 3345db2f4e..cbda703583
* Use relative path for submodule.
And fix indentation and set the branch name.
* Don't use 'RootTestsDirectory' when it might not be defined yet.
* [tests] Our test projects don't need to reference MonoTouch.Dialog directly.
The projects get the MonoTouch.Dialog reference indirectly through the
Touch.Client project reference.
* [tests] Only validate unique errors in the .NET unit tests.
* [tests] No need to reference System.Json anymore, that's handled directly in the MonoTouch.Dialog project.
* [tests] Reference nunit.framework.targets so we get a workaround for an NUnit issue everywhere.
* [msbuild] Only try to create a package if we're able to create an app bundle.
This fixes an issue where a library project would try (and fail) to create a
package when 'CreatePackage=true' (which could be set for the executable
project, but inherited by the library project since the executable project
depends on it).
* [tests] Adjust PackTest.BindingXcFrameworksProject to not set the AssemblyName property.
MSBuild ends up being very confused when the project we're trying to build
depends on other projects, because AssemblyName is set for all the projects
being build, and MSBuild complains about ambiguous projects:
> error: Ambiguous project name 'bindings-xcframework-test'
* [net6] Fix ILStrip'ed apps to actually work again
- In a late minute change to the ILStrip PR (https://github.com/xamarin/xamarin-macios/pull/12563) a change to support XVS support broke execution of Apps that were stripped
- Applications were broken because none of the stripped assemblies were actually copied into the bundle
- However, the tests still passed, because all assemblies that were there had no IL (zero assemblies total)
Now why did this happen?
- The stripped assemblies were changed to return via an msbuild Output Element
- Output Element can return an Property or ItemGroup, depending if you use the PropertyName or ItemName attributes
- Unfortunately I used PropertyName, when I expected an ItemGroup. So I silently had a property created instead.
- Thus zero items were added to the list of files to copy into the bundle
- Which was undetected as the test did not confirm files were copied in, and manual tests were not run so late into the PR (3 weeks after PR was opened)
How was it fixed?
- Correctly using ItemName on Output created a valid item group to reference
- However, that still failed with an absurdly confusing error:
PATH/Microsoft.NET.Publish.targets(277,5): error MSB3024: Could not copy the file FILE to the destination file PATH, because the destination is a folder instead of a file. To copy the source file into a folder, consider using the DestinationFolder parameter instead of DestinationFiles.
- After a splunking through netcore targets, I found the metadata on these assemblies references really matters. Without it, they are not processed correctly at all.
- Thus, I updated ILStripBase to clone the existing metadata when changing the original assembly reference to the stripped path
- Finally, I corrected the test to assert that required files are copied in. I also manually ran our device test.
Co-authored-by: Rolf Bjarne Kvinge <rolf@xamarin.com>
Add support for 'dotnet pack', by:
1. Add a workaround for the fact that as soon as a project has a
'NativeReference' item, .NET's MSBuild logic wants to include a
'Native.$(AssemblyName).manifest' file in the NuGet. This obviously breaks,
because we don't create such a file, so we work around it by removing the
file in question from the corresponding item groups.
2. Add any binding resource packages to the NuGet.
3. Add tests.
Fixes https://github.com/xamarin/xamarin-macios/issues/12631.
Fixes these warnings, which we don't care about:
CSC : warning CS8002: Referenced assembly 'Xamarin.MacDev.Tasks.Core, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null' does not have a strong name. [(...)/xamarin-macios/msbuild/Xamarin.iOS.Tasks/Xamarin.iOS.Tasks.csproj]
CSC : warning CS8002: Referenced assembly 'Xamarin.iOS.Tasks.Core, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null' does not have a strong name. [(...)/xamarin-macios/msbuild/Xamarin.iOS.Tasks/Xamarin.iOS.Tasks.csproj]
CSC : warning CS8002: Referenced assembly 'Xamarin.MacDev.Tasks, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null' does not have a strong name. [(...)/xamarin-macios/msbuild/Xamarin.iOS.Tasks/Xamarin.iOS.Tasks.csproj]
CSC : warning CS8002: Referenced assembly 'Xamarin.Localization.MSBuild, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null' does not have a strong name. [(...)/xamarin-macios/msbuild/Xamarin.iOS.Tasks/Xamarin.iOS.Tasks.csproj]
CSC : warning CS8002: Referenced assembly 'Xamarin.MacDev.Tasks.Core, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null' does not have a strong name. [(...)/xamarin-macios/msbuild/Xamarin.Mac.Tasks/Xamarin.Mac.Tasks.csproj]
CSC : warning CS8002: Referenced assembly 'Xamarin.Mac.Tasks.Core, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null' does not have a strong name. [(...)/xamarin-macios/msbuild/Xamarin.Mac.Tasks/Xamarin.Mac.Tasks.csproj]
Augment the CreateBindingResourcePackage to support creating a zipped binding
resource package (which is just a zipped version of the binding resource
package). This can either be manually chosen by the new 'Compressed' property,
or automatically detected (create a zipped version when there's a symlink in
the binding resource package).
The default is to not create a zipped version in legacy Xamarin, and
automatically detect for .NET.
The problem this is trying to solve is when creating a NuGet package - NuGet
doesn't handle symlinks correctly and it's not possible to create a NuGet with
symlinks. Instead we need to create a zipfile with all the binding resources.
The default has been chosen so that we automatically create a zip file when
it's required for .NET, while still maintaining old behavior with legacy
Xamarin.
We don't need to compile project-level assets for every RuntimeIdentifier in
multi-rid builds, we can instead compile them just once in the outer build.
There is also a correctness issue here: we can't compile assets more than once
and expect to get the exact same compiled result every time (in particular
actool seems to be adding random bytes in to the compiled output), and this
creates a problem when trying to merge the different runtime-specific compiled
output into a universal binary.
We accomplish this by:
* Processing these assets in the outer build, before we execute the
rid-specific inner builds.
* Store the paths to the assets we've processed in a file.
* In the inner builds, we read that file, and remove any matches from the
corresponding item group.
* Make sure to copy the compiled assets to the app bundle at the end of the
outer build.
These are the assets we currently handle this way:
* BundleResource
* ImageAsset
* InterfaceDefinition
* SceneKitAsset
* Collada
* TextureAtlas
* CoreMLModel
Also:
* Add a new test case (AppWithResource) that contains all these different
types of assets.
* Add support for the ScnTool task on Mac Catalyst (which the new test case
revealed was missing).
Fixes https://github.com/xamarin/xamarin-macios/issues/12410.
Fixes this problem:
error MSB4018: The "FindItemWithLogicalName" task failed unexpectedly.
error MSB4018: System.IO.FileLoadException: Could not load file or assembly 'Xamarin.Localization.MSBuild, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null' or one of its dependencies. A strongly-named assembly is required. (Exception from HRESULT: 0x80131044)
error MSB4018: File name: 'Xamarin.Localization.MSBuild, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null'
error MSB4018: at Xamarin.MacDev.Tasks.FindItemWithLogicalNameTaskBase.Execute()
error MSB4018: at Microsoft.Build.BackEnd.TaskExecutionHost.Microsoft.Build.BackEnd.ITaskExecutionHost.Execute()
error MSB4018: at Microsoft.Build.BackEnd.TaskBuilder.<ExecuteInstantiatedTask>d__26.MoveNext()
We need to strongname our MSBuild assemblies, so that different versions
can be loaded side-by-side (one example being having both a legacy and a
.NET project in the same solution).
This required setting a version for Xamarin.iOS.Tasks.dll and
Xamarin.Mac.Tasks.dll, otherwise strong-naming won't work properly (all
versions of an assembly would have the same identity).
Also sign the corresponding test assemblies, since they poke into the
internals of the task assemblies.
Fixes https://github.com/xamarin/xamarin-macios/issues/9835.
New commits in xamarin/Xamarin.MacDev:
* xamarin/Xamarin.MacDev@9e6e29f [Xamarin.MacDev] Return valid iOS/macOS versions when converting betweeen iOS and macOS versions for Mac Catalyst.
Diff: 41d91e0de0..9e6e29f2a4
Fixes this warning:
> Xamarin.Shared.targets(992,3): warning MSB6002: The command-line for the "BTouch" task is too long. Command-lines longer than 32000 characters are likely to fail. Try reducing the length of the command-line by breaking down the call to "BTouch" into multiple calls with fewer parameters per call.
* Automatically include *.ttf, *.ttc and *.otf in .NET projects as BundleResource
items (if these files are found within the Resources/ subdirectory).
* Add support for a 'RegisterFont' metadata on BundleResource items, where if set
to 'true', we'll register the font file in the Info.plist as required by the target
platform.
* Add tests.
Fixes https://github.com/xamarin/xamarin-macios/issues/12536.
Fix parsing extra bundler arguments where a space separates the name and the
value of the argument, like this: '--xml file.xml' (as opposed to
'--xml:file.xml' or '--xml=file.xml').
Fixes https://devdiv.visualstudio.com/DevDiv/_workitems/edit/1385946.
Rolf Bjarne Kvinge
Mauro Agnoletti
VS MobileTools Engineering Service 2
Israel Soto
Mauro Agnoletti
TJ Lambert
Chris Hamons