This is because `EnablePreviewFeatures=true` doesn't quite work, since it
requires the building .NET and the target .NET to be on the same version.
We might want to build with .NET 9, but the Xcode branch is targeting .NET 8,
so it doesn't work.
This behavior is explained here:
https://github.com/dotnet/designs/blob/main/accepted/2021/preview-features/preview-features.md#meaning-of-property-in-multi-targeted-projects
The best solution seems to switch to using the Experimental attribute instead,
which was designed for our scenario (and explicitly to fix the problem we're
running into): bba3216250/accepted/2023/preview-apis/preview-apis.md
This also meant we had to augment `-nowarn` for bgen to:
* Pass any nowarn values to the compiler when bgen compiles stuff.
* Pass `$(NoWarn)` (the MSBuild property) to bgen when building a binding project.
---------
Co-authored-by: Alex Soto <alex@soto.dev>
Add support for signing with a placeholder key ("-") from the developer,
and then use this to sign the prebuilt HotRestart app this way.
The app will have to be signed anyway on the end user machine, so this
shouldn't have any real effect (I've tested it locally and Hot Restart
still works).
This simplifies our build (both locally and on bots), because we won't
need a certificate around to do the actual signing.
Ignore a few warnings by default, when reporting about types that we
couldn't register because they're deprecated/removed.
Also add a way to re-enable these warnings.
Fixes https://github.com/xamarin/xamarin-macios/issues/20670.
There's no need to support `RuntimeIdentifiers` (plural) for Hot Restart
(because we don't have any scenarios where multiple runtime identifiers
applies to iOS; a single runtime identifier can always be used).
Adding support would make our code base more complex, so just avoid it by
showing an early error if someone tries (which is likely to be accidental
anyways).
This way we show an actionable error message for a scenario customers will
probably be confused about (because the build would fail in rather
inexplicable ways) if they run into it.
Partial fix for https://github.com/xamarin/xamarin-macios/issues/19262.
This reverts commit f8552e9294.
This change is only supposed to be released with .NET 9, and we might
release new .NET 8 updates from main. Thus we need to make sure these
changes are only in the net9.0 branch (they already are).
* Set the Architectures array, which Xcode does. This requires passing RuntimeIdentifier(s)
to the task, so do that.
* Set SchemeName, which Xcode does.
This is a partial fix for https://github.com/xamarin/xamarin-macios/issues/20714.
---------
Co-authored-by: Alex Soto <alex@alexsoto.me>
This is part of the effort to migrate the Pair to Mac agents .NET.
As the Xamarin.iOS.Tasks.Windows project targets netstandard2.0, I'm
removing the Build agent reference, and modifying the Makefile to take
it from it's output directory. Note: the agent zip file is generated in
the intermediate output directory.
---------
Co-authored-by: Alex Soto <alex@alexsoto.me>
Co-authored-by: Mauro Agnoletti <mauro.agnoletti@gmail.com>
Co-authored-by: Rolf Bjarne Kvinge <rolf@xamarin.com>
Change how we compute DTPlatformName so that it's 'macosx' for Mac Catalyst.
The PlatformUtils.GetTargetPlatform returns SdkPlatform for all platforms
except Mac Catalyst, where it returns the same as for macOS (i.e. 'macosx').
It also returns a lowercased value, so we don't need to do that either.
This is a partial fix for https://github.com/xamarin/xamarin-macios/issues/20714.
Version the Xamarin.Localization.MSBuild assembly correctly (instead of
hardcoding a 1.0.0 version - implicitly so because no version was set), so
that we can load multiple versions of the assembly into the same process.
Fixes https://github.com/xamarin/xamarin-macios/issues/20062.
iOS (and presumably tvOS) app bundles can't contain a subdirectory named "Resources".
Apple says:
> Note: An iOS app bundle cannot include a custom folder named “Resources.”
Ref: https://developer.apple.com/library/archive/documentation/CoreFoundation/Conceptual/CFBundles/BundleTypes/BundleTypes.html#//apple_ref/doc/uid/10000123i-CH101-SW1
Unfortunately Apple's toolchain won't show a helpful error message, the
eventual failure is codesign saying something like this:
bin/Release/net8.0-ios/ios-arm64/SomeApp.app: code object is not signed at all
In subcomponent: bin/Release/net8.0-ios/ios-arm64/SomeApp.app/System.Security.Cryptography.aotdata.arm64
Which is confusing, to say the least.
After debugging this myself a few times, and seeing customers running into the
same issue periodically, it's time to make the error more actionable.
I've added code to our Codesign task to detect the "Resources" subdirectory,
and show a better error message. There's also a way to disable the validation
(by setting `CodesignDisallowResourcesSubdirectoryInAppBundle=false`), just in
case we end up being overeager with the validation.
Fixes https://github.com/xamarin/xamarin-macios/issues/20135.
---------
Co-authored-by: Manuel de la Pena <mandel@microsoft.com>
Add support for the LinkWithSwiftSystemLibraries metadata to specify whether a native library is a Swift library, in which case we'll automatically set the `LinkWithSwiftSystemLibraries` MSBuild property to `true`.
Also add a test.
A `*.framework/` native reference and a `*.framework` reference are the same
thing, so treat them the same. We accomplish this by removing any trailing
path separators (both windows style and non-windows-style for consistency)
before doing any processing.
Fixes https://github.com/xamarin/xamarin-macios/issues/15430.
The code incorrectly assumed that the contents of a compressed
xcframework would
always be a framework, but this is incorrect (it can be a dynamic
library (.dylib),
static library (.a) or a framework (.framework).
Fix this by:
* Change variable and parameter names to clearly indicate that the
output from
an xcframework is a native library.
* Unify code to handle metadata for native libraries from xcframeworks,
so that
the code isn't duplicated (or even worse, different) between compressed
and non-compressed
xcframeworks.
The latter point fixes a problem where we'd compute the incorrect path
to the static
library inside a compressed xcframework.
PR #20546 contains a test case that triggers this.
Co-authored-by: Manuel de la Pena <mandel@microsoft.com>
This looks like an incorrect merge conflict resolution: the property
_UnpackLibraryResourcesDependsOn was declared twice, the second time
overwriting the first.
Pick the first declaration, since it's the newest and more complete one
(70a5d0d09a).
If the project in question needs a custom NuGet.config, this NuGet.config file
will be in a base directory of where the .NET executable is located. I'm not
certain what the current directory is when executing `dotnet` in this task, so
make it explicit in case it's a different path.
This in particular might affect us for remote tests using .NET previews:
/Users/builder/Library/Caches/Xamarin/mtbs/builds/bindingsframeworktest/8b36a8b30efab69178e7774426ff34b9aded7e20a789eb18f485be2f81a2dccd/obj/Debug/net9.0-ios/ComputeRemoteGeneratorProperties/ComputeRemoteGeneratorProperties.csproj : error NU1102: Unable to find package Microsoft.NET.ILLink.Tasks with version (>= 9.0.0-preview.4.24223.11)
/Users/builder/Library/Caches/Xamarin/mtbs/builds/bindingsframeworktest/8b36a8b30efab69178e7774426ff34b9aded7e20a789eb18f485be2f81a2dccd/obj/Debug/net9.0-ios/ComputeRemoteGeneratorProperties/ComputeRemoteGeneratorProperties.csproj : error NU1102: - Found 21 version(s) in nuget.org [ Nearest version: 9.0.0-preview.3.24172.9 ]
Failed to restore /Users/builder/Library/Caches/Xamarin/mtbs/builds/bindingsframeworktest/8b36a8b30efab69178e7774426ff34b9aded7e20a789eb18f485be2f81a2dccd/obj/Debug/net9.0-ios/ComputeRemoteGeneratorProperties/ComputeRemoteGeneratorProperties.csproj (in 424 ms).
* Unpack into a per-assembly directory. This way identically named resources from
different assemblies won't overwrite eachother.
* Remove logic to detect overwriting of existing bundle resources, because it's broken.
In particular this piece of code is missing a single character (there should be a `!` in the condition):
```cs
if (string.IsNullOrEmpty (logicalName))
ignore.Add (logicalName);
```
This means we're only ignoring bundle resources without LogicalName, which shouldn't
happen. So just remove the code, since it doesn't do anything useful. A proper fix
for resource collision is in progress and will come in a separate pull request.
* Fix task to work for incremental builds. Previously we'd detect that any resources
had been extracted and not re-extract, but we wouldn't add those extracted resources
to the `_BundleResourceWithLogicalName` item group. The end result would be that
on incremental builds, any bundled resources wouldn't be picked up at all. Typically
this wouldn't be a problem (because the resources would be in the resulting app
bundle), but doing anything out of the ordinary could cause problems. The fix is
to write the unpacked items + their metadata to a file, and load those when we
detect that no re-extraction is necessary (this file also does double duty as a
stamp file in the code).
* Augment `WriteItemsToFile` to make its `Write` method accessible from different classes.
* Improve logging a bit.
By default we limit the error message to 1024 characters in case the native
linker fails. The problem is that the native linker may show a lot of warnings
in the beginning, so if that happened, the error output would show only a list
of warnings and nothing else, which is rather confusing.
So change the output slightly when reporting an error in the native linker:
* Try to filter out warnings.
* Show a max of 25 lines of errors (as opposed to 1024 characters, which tended to be 3-4 lines).
So now instead of this error:
[...]/targets/Xamarin.Shared.Sdk.targets(1623,3): clang++ exited with code 1:
ld: warning: object file ([...]/packs/Microsoft.NETCore.App.Runtime.Mono.maccatalyst-arm64/8.0.3/runtimes/maccatalyst-arm64/native/libSystem.Globalization.Native.a(pal_calendarData.c.o)) was built for newer Mac Catalyst version (14.2) than being linked (14.0)
ld: warning: object file ([...]/packs/Microsoft.NETCore.App.Runtime.Mono.maccatalyst-arm64/8.0.3/runtimes/maccatalyst-arm64/native/libSystem.IO.Compression.Native.a(pal_zlib.c.o)) was built for newer Mac Catalyst version (14.2) than being linked (14.0)
ld: warning: object file ([...]/packs/Microsoft.NETCore.App.Runtime.Mono.maccatalyst-arm64/8.0.3/runtimes/maccatalyst-arm64/native/libSystem.IO.Compression.Native.a(constants.c.o)) was built for newer Mac Catalyst versio
we get:
[...]/targets/Xamarin.Shared.Sdk.targets(1623,3): clang++ exited with code 1:
Undefined symbols for architecture arm64:
"__swift_FORCE_LOAD_$_swiftCompatibility56", referenced from:
__swift_FORCE_LOAD_$_swiftCompatibility56_$_main in libSwiftTest.a(libSwiftTest.arm64.o)
(maybe you meant: __swift_FORCE_LOAD_$_swiftCompatibility56_$_main)
"__swift_FORCE_LOAD_$_swiftCompatibilityConcurrency", referenced from:
__swift_FORCE_LOAD_$_swiftCompatibilityConcurrency_$_main in libSwiftTest.a(libSwiftTest.arm64.o)
(maybe you meant: __swift_FORCE_LOAD_$_swiftCompatibilityConcurrency_$_main)
ld: symbol(s) not found for architecture arm64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
which makes much more sense for an error.
The full linker output is still available in the build log if needed.
Stop using the term 'Xamarin' in our error messages.
🙈🙊🙉
Note: this may not be complete, since we compute error messages in numerous
places, and those aren't fixed here (if there are any that still says
'Xamarin' in any way).
---------
Co-authored-by: Michael Cummings (MSFT) <mcumming@microsoft.com>
We can determine whether we're building for the simulator or not as soon as we know
the RuntimeIdentifier, so do that, and set _SdkIsSimulator accordingly.
In legacy Xamarin we had to execute a target in order to compute _SdkIsSimulator
(_DetectSdkLocations), but with this change we won't have to in .NET anymore, which
makes it possible to simplify some logic.
The first thing we do here is to also move the computation of the trimmer configuration
to right after computing _SdkIsSimulator (because we only need to know three things
to compute the default trimmer mode: the target platform + whether we're building
for the simulator or not + the configuration, and those are all known at this point
now).
This will make it possible to stop suppressing trimmer warnings depending on the
trimming mode in effect (we have to suppress (or decide not to) the warnings in the
initial MSBuild evaluation phase, before any targets are executed, because the properties
in question are read during the evaluation phase in .NET). Note that this PR is not
changing any trimmer warnings, that will only happen in .NET 9.
I've seen multiple cases of the following error:
> A bundle identifier is required. Either add an 'ApplicationId'
property in the project file, or add a 'CFBundleIdentifier' entry in the
project's Info.plist file.
In all cases this has been been because the build fails to find
Info.plist,
when it's actually there.
One source of this problem was found and fixed with #20374, but that's
not the
problem in other scenarios.
So I'm adding a bit more diagnostic output to the
FindItemWithLogicalName
task, to try to track down why the task fails to find `Info.plist`.
This fixes an issue where we'd compute a LogicalName property that was outside
of the app bundle, resulting in broken resources in the app bundle (in
particular the app icon would be missing).
So instead of something like this:
LogicalName: ../../../../../../../../../tmp/testapp/obj/Debug/net8.0-ios/iossimulator-arm64/actool/bundle/Assets.car
we'll now get:
LogicalName: Assets.car
and the icons will be correctly embedded in the app bundle.
Fixes https://github.com/xamarin/xamarin-macios/issues/20330.
Error messages aren't meant to be an exercise in mental capacity, so make
things easier for developers to figure out which versions are valid and which
are not by sorting them.
So instead of this:
Valid Mac Catalyst versions are: 15.0, 13.3.1, 16.2, 14.3, 15.5, 13.1, 16.0, 17.2, 14.1, 15.3, 16.5, 14.6, 13.4, 17.0, 16.3, 15.6, 13.2, 14.4, 16.1, 14.2, 15.4, 16.6, 13.5, 14.7, 17.1, 15.2, 14.0, 16.4, 13.3, 14.5
We now get:
Valid Mac Catalyst versions are: 13.1, 13.2, 13.3, 13.3.1, 13.4, 13.5, 14.0, 14.1, 14.2, 14.3, 14.4, 14.5, 14.6, 14.7, 15.0, 15.2, 15.3, 15.4, 15.5, 15.6, 16.0, 16.1, 16.2, 16.3, 16.4, 16.5, 16.6, 17.0, 17.1, 17.2
* Enable generation of the documentation file (.xml) by the C# compiler when building projects (by passing the `DocumentationFlag` argument to the Csc task).
* Also disable the CSC1591 warning, which complains about having public APIs without xml documentation; it just turns out to be annoying rather than helpful, since most APIs won't be documented.
* Add support in the generator for reading the .xml file next to the compiled api definition assembly, and then looking for documentation there when generating binding code.
* When enabled, enable generation of the documentation file if the generator is compiling the api definitions.
* Make it an opt-out, in case the new code causes problems.
* Add tests.
Fixes https://github.com/xamarin/xamarin-macios/issues/17397.
---------
Co-authored-by: Haritha Mohan <harithamohan@microsoft.com>
This deduplicates some code, and also ensures we specifiy a max limit to the
level of parallelism. This fixes an issue in the AOTCompile and ScnTool tasks,
where we'd have no limit, launching as many subprocesses as tasks there are to
execute. In particular for the AOTCompile task, this can put a rather heavy
burden on build machines, slowing everything down to a crawl.
Fixes https://github.com/xamarin/xamarin-macios/issues/20210.
We currently have an issue with codesigning duplicated files in parallel
(#20193). The proper fix is somewhat complex, so this implements a simple
workaround for customers, where they can just disable parallelism if they run
into this problem (and since it's simple, it's easier to backport too).
Additionally, it's not a bad idea to be able to configure the level of parallelism.
Ref: #20193.
If we copy a file back to Windows during the remote build, we don't want to
also create the output file, because then we'll just replace the file we
copied with a zero-length version.
* Enumerate files and directories instead of fetching them all in one go. This
is more efficient when there are a lot of files and directories to process,
but most importantly it'll also make enumeration cancellable.
* Stop enumerating if cancellation was requested.
* Add a few logging statements.
This might at least help to get better diagnostic information if for some
reason this task takes too long to execute. Ref: #20183.
The extracted files show up in the build later on, and if only the 0-length
mirror file exists on Windows, it'll be copied as such to the Mac, producing
incorrect build results.
This particular condition shouldn't happen, but occasional build errors in the
wild (with exception stack traces) seem to indicate otherwise, so be a bit
defensive here.
At the moment even with issues the build will be successful so that we
have green builds while we identify and fix several issues.
---------
Co-authored-by: Alex Soto <alex@alexsoto.me>
Co-authored-by: Rolf Bjarne Kvinge <rolf@xamarin.com>
After this the Xamarin.iOS.Tasks and Xamarin.Mac.Tasks assemblies are empty,
so the next step will be to remove them completely (this will happen in a
different pull request).
---------
Co-authored-by: Alex Soto <alex@alexsoto.me>
#### Background
While cyclic assembly references are unusual they are supported by the runtime and encountered in practice. In our project we use a version of Xamarin.Forms retargeted for a modern .NET (as a stopgap solution before an update to full MAUI stack). Xamarin.Forms historically come with such an assembly cycle: `Xamarin.Forms.Core.dll` -> `Xamarin.Forms.Platform.dll` -> `Xamarin.Forms.Platform.iOS.dll` -> `Xamarin.Forms.Core.dll`. This is produced by first compiling `Xamarin.Forms.Core.dll` against a dummy `Xamarin.Forms.Platform.dll` (`netstandard2.0` assembly). Then the Platform assemblies are built, and finally forwarder versions of `Xamarin.Forms.Platform.dll` are created for each platform. This is all packaged into NuGets in such a way that each TFM gets the same `Xamarin.Forms.Core.dll` but a different set of `Xamarin.Forms.Platform.dll` and `Xamarin.Forms.Platform.<platform>.dll` assemblies.
#### Problem
With current workloads each incremental rebuild fails with:
```
/usr/local/share/dotnet/packs/Microsoft.iOS.Sdk/16.4.7125/targets/Xamarin.Shared.Sdk.targets(1047,3): error : Encountered an assembly reference cycle related to the assembly obj/Debug/net7.0-ios/iossimulator-arm64/linked/Xamarin.Forms.Core.dll
```
#### Solution
The PR updates the algorithm in `AOTCompile` to detect cycles using [Tarjan's algorithm](https://en.wikipedia.org/wiki/Tarjan%27s_strongly_connected_components_algorithm) for finding [strongly connected components](https://en.wikipedia.org/wiki/Strongly_connected_component) in a graph. When a cycle is detect any assembly in the cycle is considered up-to-date only if all the assemblies in the cycle are up-to-date.
With this change the project does incremental rebuilds correctly. I verified in the build output that the assembly cycle is considered up-to-date, and that any changes to the application assemblies still return `false` from the up-to-date check and get rebuilt.
---------
Co-authored-by: Filip Navara <filip.navara@gmail.com>
This avoids the Mono.Cecil dependency, and also MetadataLoadContext is both
faster and better maintained.
A temporary downside is that all the reference assemblies will have to be
copied to the remote Mac for remote builds, but this will be fixed in a future
change where we'll execute this task mostly locally on Windows, and we'll
only copy the unpacked resources to the Mac.
* Enable nullability and fix the resulting issues.
* Convert to XamarinTask (instead of XaxmarinToolTask): this allows us to run multiple
'scntool' invocations in parallel.
* Use 'xcrun' to call 'scntool' instead of computing the path [1].
* Fix bug in the Cancel method: it shouldn't call base.Execute.
* Change the targets logic to match the pattern of other resource-related targets.
* This makes it easier to understand the code, since understanding one resource-related target works for the other ones too.
* Not using the CollectBundleResources task means computing LogicalName in the ScnTool task directly.
[1]: https://github.com/xamarin/xamarin-macios/issues/11172#issuecomment-816324118
* Enable nullability and fix any issues.
* Improve error reporting by:
* Not bailing out when the first error is encountered, keep processing in
case we can be produce and find more problems to report.
* Show a better error message by including the path of the problematic
file in the error message.
Due to indentation changes, this PR migth be easier to review by [hiding
whitespace changes](https://github.com/xamarin/xamarin-macios/pull/19998/files?w=1).
Rolf Bjarne Kvinge
Alex Soto
Emanuel Fernandez Dell'Oca
Manuel de la Pena
Filip Navara