Backport of https://github.com/xamarin/xamarin-macios/pull/20952
--------
## Description
Previous fix https://github.com/xamarin/xamarin-macios/pull/20945 did
not take into account that when we target non arm64 apple mobile
platforms we are using `MONO_AOT_MODE_INTERP_ONLY` which cannot use
dedup optimization as it discards AOT images during runtime.
## Changes
- Enable dedup only when targeting ARM64
- Fix tests to cover builds for both ARM64 and X64 builds
Finally, the change was tested against MAUI iossimulator-x64 template
app
---------
Co-authored-by: Ivan Povazan <ivan.povazan@gmail.com>
Co-authored-by: Ivan Povazan <55002338+ivanpovazan@users.noreply.github.com>
This post-build pipeline should only run against net8.0 servicing
branches that don't have the commit (2abbaf900b) that moved maestro
publishing into the regular build pipeline.
## Description
This is a follow-up PR to:
https://github.com/xamarin/xamarin-macios/pull/20936
We should not enable dedup when targeting `maccatalyst-x64` because in
case when the project file specifies
`MtouchInterpreter=all,-System.Private.CoreLib`, the build will run the
full AOT compiler with interpreter enabled.
In this setup the runtime is configured to run in interp only mode:
97a91cc4e3/tools/common/Target.cs (L812-L813)
which means that during runtime, AOT images will be ignored - aot
runtime will load them but mark them as unusuable since they are
compiled with `full` compiler switch and the code falls back to
interpreter (ref:
efebf202a4/src/mono/mono/mini/aot-runtime.c (L2131-L2148)
)
This is problematic for the `aot-instances` container image, which has a
special handling at the runtime and does not accept it to be marked as
unusable (we might want to revisit this in the future):
efebf202a4/src/mono/mono/mini/aot-runtime.c (L2527-L2529)
## Changes
This PR disables dedup optimization when targeting `maccatalyst-x64` and
updates the required tests to match the behavior.
---
Backports:
- [x] https://github.com/xamarin/xamarin-macios/pull/20946
- [x] Original reenabling of dedup for .NET 9 already includes these
changes: https://github.com/xamarin/xamarin-macios/pull/20941
---------
Co-authored-by: GitHub Actions Autoformatter <github-actions-autoformatter@xamarin.com>
## Description
As part of the fix for: https://github.com/dotnet/runtime/issues/99248
we disabled dedup optimization in partial/hybrid AOT mode (when both
interpreter and AOT compiler are enabled). This change got backported to
.NET 8 and with the latest servicing release regressed build times and
app sizes significantly as reported in:
https://github.com/xamarin/xamarin-macios/issues/20848
However, it turns out that disabling dedup optimization is not required
to fix https://github.com/dotnet/runtime/issues/99248 but instead we
should correct the Xamarin SDK integration with this optimization which
this PR is doing. The following section describes the initial problem in
more details.
## Overview of AOT modes and dedup optimization
When the repro project from
https://github.com/dotnet/runtime/issues/99248 is built with dedup
enabled in hybrid AOT+interpreter mode, the app crashes with:
```
024-07-23 14:32:37.524110+0200 IvansApp[12711:20244208] debug: AOT NOT FOUND: (wrapper other) object:gsharedvt_out_sig (intptr).
2024-07-23 14:32:37.524120+0200 IvansApp[12711:20244208] error: * Assertion at /Users/ivan/repos/runtime-mono-iOS/src/mono/mono/mini/interp/interp.c:2667, condition `is_ok (error)' not met, function:init_jit_call_info, Attempting to JIT compile method '(wrapper other) void object:gsharedvt_out_sig (intptr)' while running in aot-only mode. See https://learn.microsoft.com/xamarin/ios/internals/limitations for more information.
```
To track down why these wrappers which are used to transition from
interpreter to AOT code, are not generated we need to understand when
they are compiled in different AOT modes with and without dedup
optimization enabled:
- In full AOT setup - all assemblies AOT compiled
- `gsharedvt_out_sig` methods are never generated
- In hybrid AOT + interpreter setup - all assemblies AOT compiled:
`MtouchInterpreter=-all`
- Dedup OFF:
- `gsharedvt_out_sig` methods are generated in AOT images of every
assembly (to enable interpreter calling into each specific assembly -
here wrappers with same signatures are duplicated)
- Dedup ON:
- `gsharedvt_out_sig` methods are generated only in `aot-instances` AOT
image
- during AOT compilation of individual assemblies generation of
`gsharedvt_out_sig` is skipped
- during AOT compilation of `aot-instances` assembly we collect all
`gsharedvt_out_sig` variants from the full program scope and generate
code for them in `aot-instances` AOT image
- In hybrid AOT + interpreter setup - all assemblies interpreted except
a given assembly: `MtouchInterpreter=all,-MyAssembly`
- Dedup OFF:
- `gsharedvt_out_sig` methods are generated in AOT image of `MyAssembly`
(to enable interpreter calling into it)
- Dedup ON: <- $${\color{red} ISSUE }$$
- `gsharedvt_out_sig` methods *should* be generated only in
`aot-instances` AOT image, but the `aot-instances` image is missing
- explanation:
- what happens is that generation of `gsharedvt_out_sig` is skipped
during AOT compilation of `MyAssembly` (as expected).
- But, the build does not mark `aot-instances` assembly as the one that
should be AOT compiled.
- The reason for this is that we have a global `_IsDedupEnabled` flag,
but when custom linker step analysis `aot-instances.dll` it does not see
it as an assembly which should not be interpreted.
- To explain that better: we mark *all* assemblies as to be interpreted
(via: `MtouchInterpreter=all`), but exclude only `MyAssembly` (via:
`MtouchInterpreter=all,-MyAssembly`).
- So when custom linker step processes `aot-instaces.dll` it treats it
as an assembly to be interpreted, so it does not mark it for AOT
compilation.
- This further results with `aot-instances` AOT image missing, and all
the methods which we skipped during AOT compilation never get generated.
## The fix
To fix this and address regressions reported in:
https://github.com/xamarin/xamarin-macios/issues/20848 we are reenabling
dedup optimization whenever AOT compilation is requested and fixing the
issue where the custom linker step for generating AOT parameters always
treates the dedup assembly as the one to be AOTed.
Once approved this should be backported to .NET 8 as servicing releases
are also affected with it.
---------
Co-authored-by: GitHub Actions Autoformatter <github-actions-autoformatter@xamarin.com>
fixes: https://github.com/dotnet/maui/issues/23577
## Cause:
`MtouchInterpreter` is set as
d1ec7a793f/msbuild/Xamarin.Shared/Xamarin.Shared.props (L308)
`DynamicCodeSupport` is set by
d1ec7a793f/dotnet/targets/Xamarin.Shared.Sdk.targets (L146)
`Xamarin.Shared.Sdk.targets` is imported *before* `Xamarin.Shared.props`
as shown below (courtesy of @ivanpovazan)
<img
src="https://github.com/user-attachments/assets/c97b7f01-2372-4f9d-bedf-83060eed1c50">
so unless the value of `MtouchInterpreter` is set in the project, it's
value will be empty when the `DynamicCodeSupport` property is evaluated.
## Resolution:
To minimize the impact of this change, until it can be investigated more
fully, the value of `MtouchInterpreter` is evaluated as
d1ec7a793f/msbuild/Xamarin.Shared/Xamarin.Shared.props (L308)
So adding `( '$(MtouchInterpreter)' == '' And '$(UseInterpreter)' ==
'false' )` to the definition of `DynamicCodeSupport` to match the
`MtouchInterpreter` definition.
A further fix might be to either:
1. Reorder the imports so that the props are included before the
targets, although the ramifications of that change could be significant
2. Move the definition of `DynamicCodeSupport` to
`Xamarin.Shared.props`. This at first glance seems to be less
significant than 1) but would need review and testing.
---------
Co-authored-by: Ivan Povazan <55002338+ivanpovazan@users.noreply.github.com>
I have cleaned the yaml files while I was trying to debug the issue with
the provisioning of the simulators on Xcode 16. We should back port this
change there.
Add support for signing with a placeholder key ("-") from the developer,
and then use this to sign the prebuilt HotRestart app this way.
The app will have to be signed anyway on the end user machine, so this
shouldn't have any real effect (I've tested it locally and Hot Restart
still works).
This simplifies our build (both locally and on bots), because we won't
need a certificate around to do the actual signing.
* Remove code to test NWPath.EnumerateInterfaces, because this method is already tested elsewhere.
* Assume that if the test fails to find any gateways, it might be because the
current machine doesn't have any (which happens on one of my machines), and
in that case ignore the test.
Move the architecture-specific vargs implementation of UIAppearance.GetAppareance into a more generic way of calling objc_msgSend with variadic arguments.
This prepares the way for more APIs with variadic arguments (which is coming in Xcode 16).
When building universal apps, each inner build must add the runtime identifier to the output path, otherwise the inner builds may conflict with eachother, overwriting eachother's files.
That's bad.
So we explicitly set `AppendRuntimeIdentifierToOutputPath` to `true` when building inner builds.
* xtro: Fix how we build the u2todo project to actually build the correct project.
* Don't import eng/Versions.props in several test projects, it's already imported in a Directory.Build.props further up the directory hierarchy.
Split installing new and old simulators, so that we can choose to only
install the old simulators if we want so.
And then only install the old simulators when we're doing beta builds,
because that's the only time we need them.
Ignore a few warnings by default, when reporting about types that we
couldn't register because they're deprecated/removed.
Also add a way to re-enable these warnings.
Fixes https://github.com/xamarin/xamarin-macios/issues/20670.
Add a simple makefile to the src/bgen directory that only builds and
tests bgen.
This is very useful when working on bgen to make sure your changes are
at building and working.
Convert all projects except xtro-sharpie.csproj to .NET projects.
xtro-sharpie.csproj can't be converted yet, because it depends on
Objective-Sharpie, which hasn't been converted yet.
Apple says the bug on their side causing a runtime crash has been fixed since
macOS 12, so unignore the code and add a version check for macOS 12.
Fixes https://github.com/xamarin/maccore/issues/2345.
The C# build server makes trouble for us, because:
* We're using parallel make, and parallel make will start a jobserver, managed
by file descriptors, where these file descriptors must be closed in all
subprocesses for make to realize it's done.
* 'dotnet build' might have started a build server
* The build server does not close any file descriptors it may have inherited
when daemonizing itself.
* Thus the build server (which will still be alive after we're done building)
might have a file descriptor open which make is waiting for.
* The proper fix is to fix the build server to close its file descriptors.
* The intermediate working is to shut down the build server instead. An
alternative solution would be to pass /p:UseSharedCompilation=false to
'dotnet pack' to disable the usage of the build server.
* Note that build server will exit automatically after 10 minutes of idle
time, so the hang is only a 10 minute delay at works.
For the siminstaller, which builds using the system .NET, the simplest
solution is to just not use the build server.
This fixes a problem where the build would hang for 10 minutes after running
the system dependency check (which builds and runs siminstaller).
This pull request updates the following dependencies
## From https://github.com/dotnet/installer
- **Subscription**: 80cb9ffd-f92f-4fc8-9f8b-08dbca46abfb
- **Build**: 20240624.1
- **Date Produced**: June 24, 2024 9:52:55 PM UTC
- **Commit**: 03065cafae0f89b376fa983773e909e341db96c0
- **Branch**: refs/heads/release/8.0.1xx
- **Updates**:
- **Microsoft.Dotnet.Sdk.Internal**: [from 8.0.107-servicing.24317.4 to 8.0.107-servicing.24324.1][14]
[14]: de7be3dce6...03065cafae
---------
Co-authored-by: Manuel de la Pena <mandel@microsoft.com>
When finding an enumerator for the given code:
```cs
var collection = new NSSet<NSNumber> ();
foreach (var item in collection) {
// ...
}
```
the C# compiler will first look for any `GetEnumerator` methods. The non-generic `NSSet` class defines a `IEnumerator<NSObject> GetEnumerator<NSObject> ()` method, which, since the generic `NSSet<T>` class doesn't define such a method, is selected.
The end result is that the type of the foreach element is `NSObject`
(`GetEnumerator`'s return type') - which is somewhat unexpected:
```cs
var collection = new NSSet<NSNumber> ();
foreach (var item in collection) {
Console.WriteLine (item.LongValue); // error CS1061: 'NSObject' does not contain a definition for 'LongValue'
}
```
The fix is to define a `IEnumerator<T> GetEnumerator<T> ()` method in the
generic `NSSet<T>` class, which the C# will find and choose over the base
class' method. Then the type of the foreach element is the correct type, and
the following code works:
```cs
var collection = new NSSet<NSNumber> ();
foreach (var item in collection) {
Console.WriteLine (item.LongValue); // it works!
}
```
Do this for all our generic collection classes.
Also document these methods + all the other public `GetEnumerator` methods.
This way it can easily be overridden when building from the command line (to provide a different url if the normal url doesn't work for some temporary reason).
There's no need to support `RuntimeIdentifiers` (plural) for Hot Restart
(because we don't have any scenarios where multiple runtime identifiers
applies to iOS; a single runtime identifier can always be used).
Adding support would make our code base more complex, so just avoid it by
showing an early error if someone tries (which is likely to be accidental
anyways).
This way we show an actionable error message for a scenario customers will
probably be confused about (because the build would fail in rather
inexplicable ways) if they run into it.
Partial fix for https://github.com/xamarin/xamarin-macios/issues/19262.
This reverts commit f8552e9294.
This change is only supposed to be released with .NET 9, and we might
release new .NET 8 updates from main. Thus we need to make sure these
changes are only in the net9.0 branch (they already are).
We fixed the credscan issue in two diff ways:
1. When the job allows it, we checkout all repos using our own checkout template.
2. When the jib does not allow it, we create an empty json file. In the future we can add any needed exception.
We also needed to fix the signature because the VS code moved to net core which changed the extension of their build.exe to build.dll.
Fix BuildBindingsTest expectations to expect the resources in either a sidecar or a zipped sidecar.
Fixes this test failure:
Xamarin.Tests.DotNetProjectTest.BuildBindingsTest(TVOS): Bundle existence
Expected: file or directory exists
But was: "/Users/builder/azdo/_work/1/s/xamarin-macios/tests/bindings-test/dotnet/tvOS/bin/Debug/net8.0-tvos/bindings-test.resources.zip"
Matt Sylvia
Alex Soto
Peter Collins
Ivan Povazan
dotnet-maestro[bot]
Michael Cummings (MSFT)
Manuel de la Pena
Rolf Bjarne Kvinge
Alex Soto
Mike Bond