We're going to change the pack names to support multi-targeting, so ahead
of the pack name change I'm changing the existing logic to use a variable
for the pack name in most places (this will make the rename much easier and
simpler).
These changes should have no effect by themselves.
Added default entitlements for MacCatalyst templates.
For Debug, the com.apple.security.get-task-allow entitlement that allows for using developer tools when developing MAUI Blazor apps.
For release, com.apple.security.app-sandbox is required to publish MacCatalyst apps to the Mac App Store.
Also added unit test to check for entitlements when project is created.
Fixes#18344
---------
Co-authored-by: GitHub Actions Autoformatter <github-actions-autoformatter@xamarin.com>
Co-authored-by: Rolf Bjarne Kvinge <rolf@xamarin.com>
* Don't strip resource assemblies, there's no code in them to strip anyways.
* Use the relative path inside the app bundle when computing the intermediate
location for stripped assemblies, so that if we were to find two identically
named assemblies in different directories, they're handled correctly (by
putting them in different intermediate locations, instead of overwriting
eachother).
Fixes https://github.com/xamarin/xamarin-macios/issues/17262.
The .mobile.props file is a file created and written by the mobile VS
extension to store property values that needs to be read early enough in
the build chain, as in design time builds, and that can't be set by CPS
because of a limitation in the project system. See more information
here: https://github.com/xamarin/XamarinVS/pull/13606
Initially it was named .user.env file and then was renamed in another PR
as part of a feedback from the project system team. See more information
here: https://github.com/xamarin/XamarinVS/pull/13628
Because this file was saved in the intermediate output path, it was
meant to be imported automatically by MSBuild, however we recently
detected that this was not happening reliably. Because of this, some
things like C# Hot Reload for iOS stopped working because Roslyn was
reading incorrect values from the Design Time Builds.
For that reason and to avoid relying on the project system, I'm
importing this file explicitly (and removing old .user.env import), so
the values in the file are always available and the dependent properties
are calculated correctly and available for all the consumers (including
Roslyn).
This should fix the following bugs:
https://devdiv.visualstudio.com/DevDiv/_workitems/edit/1822041https://devdiv.visualstudio.com/DevDiv/_workitems/edit/1851677
It's possible to create a provisioning profile for Mac Catalyst that
doesn't allow dylibs in the app. It seems a significant number of people run
into this problem when publishing their apps, so avoid it by linking Mono and
Xamarin statically by default instead.
The downside is that build time might increase a little bit.
An upside however is that the app size might decrease somewhat.
Fixes https://github.com/xamarin/xamarin-macios/issues/14686.
If an assembly changes, then we must AOT compile that assembly again (which we already
did), in addition to any assembly that references the modified assembly (which we
didn't do).
So rework the AOTCompile target: remove the Inputs and Outputs (because the dependency
tracking is too complicated for MSBuild to resolve), and instead move the logic to
detect if an assembly must be AOT-compiled again into the AOTCompile task.
Note that this PR has a custom port to .NET 8: #18518.
Fixes https://github.com/xamarin/xamarin-macios/issues/17708.
---------
Co-authored-by: Alex Soto <alex@alexsoto.me>
It seems this target has more problems than at first I thought, so make
it easier to opt-out of it by just setting a property in the csproj.
More investigation is needed, but I'm keeping the target on by default
for now, since it solves a real-world problem as well.
Ref: https://github.com/xamarin/xamarin-macios/issues/18445
Add public targets to compute the mlaunch command lines for installing
and launching mobile apps.
These new targets are:
* ComputeMlaunchInstallArguments
* ComputeMlaunchRunArguments
As part of this change, also create a few new public properties:
* MlaunchPath
* MlaunchRunArguments
* MlaunchInstallArguments
* MlaunchRunScript
* MlaunchInstallScript
If the *Script variables are set, the corresponding target will create a
script file with the path to mlaunch + the corresponding arguments.
Otherwise, it's also possible to get the arguments directly from the
build log.
Fixes https://github.com/xamarin/xamarin-macios/issues/18359.
# Description
This PR reduces the application's SOD (size on disk) by making
`__LINKEDIT Export Info` section smaller in the stripped Mach-O
binaries.
The feature is controlled by `_ExportSymbolsExplicitly` MSBuild property
and can be disabled by specifying: `-p:_ExportSymbolsExplicitly=true`
Fixes#18332
# Initial problem
It has been noticed that during stripping, the strip tool does not
resize the export info section after it removes the symbols. Instead it
only zeroes out the entries (achieved by calling `prune_trie` function):
- https://github.com/apple-oss-distributions/cctools/blob/cctools-986/misc/strip.c
- https://github.com/apple-oss-distributions/ld64/blob/ld64-711/src/other/PruneTrie.cpp
Thanks @lambdageek for helping to track this down.
# Approach
As Xamarin build process already collects all the [required symbols][1] needed
for the application to run and preserves them during the strip phase, we can
use the same file to instruct clang toolchain to export only those symbols via
the command line options: `-exported_symbols_list <file>` ([source][2]).
This will make the export info section only include what is necessary for the
runtime - and at the same time eliminate the problem of the `strip` tool which
does not resize stripped symbols.
# Investigation setup
The issue is observable by building and inspecting the test application:
https://github.com/xamarin/xamarin-macios/blob/main/tests/dotnet/MySingleView/MySingleView.csproj
and targeting iOS platform in Release mode.
## Results:
| Measure | MySingleView - main | MySingleView - this PR | Diff (%) |
| :--- | ---: | ---: | ---: |
| SOD (bytes) | 13668940 | 13458476 | -1.5% |
| .ipa (bytes) | 4829368 | 4827928 | -0.03% |
Even though zeroes are compressed well, the SOD is still affected and
unused section takes around 1.5% of the most simplistic app size.
Much bigger impact has been noted when trying out a MAUI iOS template
app with NativeAOT where the `__LINKEDIT Export Info` zeroes take up to
20MB of the SOD, but also with the regular macOS applications:
https://github.com/dotnet/runtime/issues/86707
### Repro current state of MySingleView.app with stripped binary
1. Build the app (you can ignore the need to run the sample, I just did it to
make sure the changes do not break anything)
```bash
make run-device
```
2. Print the load commands - [load_cmds_strip.list][3]
```bash
otool -l bin/Release/net7.0-ios/ios-arm64/MySingleView.app/MySingleView > load_cmds_strip.list
```
- We are interested in the export info section:
```
cmd LC_DYLD_INFO_ONLY
...
export_off 5942960
export_size 207712
```
3. Create a hex dump of the export info section - [hex_dump_strip.list][4]
``` bash
xxd -s 5942960 -l 207712 bin/Release/net7.0-ios/ios-arm64/MySingleView.app/MySingleView > hex_dump_strip.list
```
- NOTE: Notice around ~200kb of zeroes from ~0x005ab490 to ~0x005dda00
4. Verify exported symbols are correct - [dyld_info_strip.list][5]
``` bash
dyld_info -exports bin/Release/net7.0-ios/ios-arm64/MySingleView.app/MySingleView > dyld_info_strip.list
```
### Repro current state of MySingleView.app with unstripped binary
1. Build the app (the make target preserves the symbols)
```bash
make run-device-no-strip
```
2. Print the load commands - [load_cmds_nostrip.list][6]
```bash
otool -l bin/Release/net7.0-ios/ios-arm64/MySingleView.app/MySingleView > load_cmds_nostrip.list
```
- We are interested in the export info section:
```
cmd LC_DYLD_INFO_ONLY
...
export_off 5942960
export_size 207712
```
3. Create a hex dump of the export info section - [hex_dump_nostrip.list][7]
``` bash
xxd -s 5942960 -l 207712 bin/Release/net7.0-ios/ios-arm64/MySingleView.app/MySingleView > hex_dump_nostrip.list
```
- Notice that the range: ~ 0x005ab490 to ~ 0x005dda00 now includes exported symbol entries
4. Verify exported symbols are correct - [dyld_info_nostrip.list][8]
``` bash
dyld_info -exports bin/Release/net7.0-ios/ios-arm64/MySingleView.app/MySingleView > dyld_info_nostrip.list
```
### Repro the new approach
1. Build the app (the make target uses the new approach)
```bash
make run-device-export-syms
```
2. Print the load commands - [load_cmds_export.list][9]
```bash
otool -l bin/Release/net7.0-ios/ios-arm64/MySingleView.app/MySingleView > load_cmds_export.list
```
- We are interested in the export info section ***notice the reduced size of the section***:
```
cmd LC_DYLD_INFO_ONLY
...
export_off 5942432
export_size 1048
```
3. Create a hex dump of the export info section - [hex_dump_export.list][10]
``` bash
xxd -s 5942432 -l 1048 bin/Release/net7.0-ios/ios-arm64/MySingleView.app/MySingleView > hex_dump_export.list
```
4. Verify exported symbols are correct - [dyld_info_export.list][11]
``` bash
dyld_info -exports bin/Release/net7.0-ios/ios-arm64/MySingleView.app/MySingleView > dyld_info_export.list
```
---
## Additional benefits
With this approach we could also switch the way strip tool is invoked to
always strip all debug and local symbols via `strip -S -x` instead of passing
the file with symbols to preserve. This would remove the warning that we are
currently getting (which is being ignored):
```
/Applications/Xcode_14.3.0.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/strip: warning: removing global symbols from a final linked no longer supported. Use -exported_symbols_list at link time when building...
```
## Other references:
- https://github.com/qyang-nj/llios/blob/main/exported_symbol/README.md
[1]: 11e7883da0/tools/dotnet-linker/Steps/GenerateReferencesStep.cs (L38-L44)
[2]: https://developer.apple.com/library/archive/documentation/DeveloperTools/Conceptual/DynamicLibraries/100-Articles/DynamicLibraryDesignGuidelines.html
[3]: https://gist.github.com/ivanpovazan/d53f8d10be5e4ea9f39a41ea540aa7fa
[4]: https://gist.github.com/ivanpovazan/60637422f3ff8cb5f437ddd06a21d9c1
[5]: https://gist.github.com/ivanpovazan/352595ad15c2ac02f38dcb3bd4130642
[6]: https://gist.github.com/ivanpovazan/bf700161f2f3691d1d7381c98d4fa0be
[7]: https://gist.github.com/ivanpovazan/44269e4fff5ebd58a4d181451e5c106f
[8]: https://gist.github.com/ivanpovazan/38c5afe076502d514a77420af0e10b01
[9]: https://gist.github.com/ivanpovazan/3f663c3c630005f5a578605d48ba807e
[10]: https://gist.github.com/ivanpovazan/0bb84f64281d05ab20438aeaed64f13c
[11]: https://gist.github.com/ivanpovazan/78b3ba2288f53a2316b9bc46964e7e4f
---------
Co-authored-by: Rolf Bjarne Kvinge <rolf@xamarin.com>
This property will be required when building for a net7.0-* target framework
using .NET 8 (preview 6 - preview 5 does not need this fix)
Backport of #18411
Fixes#17707
The error target for when there is a conflict of interest in defining
both the runtime identifier and runtime identifiers is called during the
multi-rid builds, but not sure if the placement is the most ideal..
---------
Co-authored-by: GitHub Actions Autoformatter <github-actions-autoformatter@xamarin.com>
Co-authored-by: Rolf Bjarne Kvinge <rolf@xamarin.com>
Co-authored-by: Alex Soto <alex@alexsoto.me>
* Add support for forwarding our debugging msbuild properties to their
corresponding environment variables (the XamarinDebug* properties).
* Add support for passing --stdout/--stderr/--stdin to open to redirect
to/from a file. This is particularly useful for debugging debugging.
* Add support for passing -a (to always create a new instance of the app).
This is useful when debugging (when the developer would always want a new
instance, instead of opening an existing instance).
* Also add support for any other argument using the 'OpenArguments' property.
Fixes https://devdiv.visualstudio.com/DevDiv/_workitems/edit/1825427.
This is the pull request automatically created by the OneLocBuild task
in the build process to check-in localized files generated based upon
translation source files (.lcl files) handed-back from the downstream
localization pipeline. If there are issues in translations, visit
https://aka.ms/icxLocBug and log bugs for fixes. The OneLocBuild wiki is
https://aka.ms/onelocbuild and the localization process in general is
documented at https://aka.ms/AllAboutLoc.
Add a new version of the static registrar (called the managed static
registrar), which most notably doesn't use metadata tokens (because NativeAOT
doesn't support metadata tokens). In addition, the new registrar also takes
advantage of new features in both C# and the runtime, in order to be more
performant.
I won't go into detail about everything here, because it would be rather long,
but I've added documentation for the new registrar (the first commit, so start
reviewing there).
Fixes https://github.com/xamarin/xamarin-macios/issues/17324.
Given the following truths:
* A task will (try to) connect to a Mac if its SessionId property isn't empty.
* The BuildSessionId property is always set on Windows when building from an IDE (even if not connected to a remote Mac).
It stands to reason that we can't use BuildSessionId to distinguish between
connected/not conected status on Windows. Instead introduce a new property,
BuildSessionIdIfConnected, which is only set if connected to a Mac (i.e. if
'IsMacEnabled=true').
Fixes https://devdiv.visualstudio.com/DevDiv/_workitems/edit/1808448 (second attempt).
The AOT compiler does not exist on Windows, so this check was making the
cache never work there. We don't really need to check if the compiler
does still exist, since the cache can be deleted by rebuilding the
project if anything fails.
This change didn't really get rid of the non-breaking space and caused
extra changes after we built xamarin-macios.
Co-authored-by: tj-devel709 <tjlambert@microsoft.com>
This is the pull request automatically created by the OneLocBuild task
in the build process to check-in localized files generated based upon
translation source files (.lcl files) handed-back from the downstream
localization pipeline. If there are issues in translations, visit
https://aka.ms/icxLocBug and log bugs for fixes. The OneLocBuild wiki is
https://aka.ms/onelocbuild and the localization process in general is
documented at https://aka.ms/AllAboutLoc.
Set/fix the PublishFolderType and RelativePath metata for returned items in the ComputeBundleLocation
and ResolveNativeReferences task, so that the new ComputeHotRestartBundleContents
task has enough (and the correct) information to do the right thing with items that
are to be copied to the app bundle.
Rework Hot Restart builds to use as much as possible of the normal build logic, because
this is the easiest way to make sure the Hot Restart build is as close as possible
to normal builds (and we don't end up missing features).
This is done by executing selected parts of a normal build, and a the end we have
a new task that computes where each file goes in the various output directories Hot
Restart uses (HotRestartAppBundlePath, HotRestartContentDir, HotRestartAppContentDir,
etc.)
This makes it unnecessary to special-case this file for it to copied
correctly when building on Windows (once we've fixed the Windows build to use
ResolvedFileToPublish as the source of truth, like we do on macOS).
This is the first part of a fix for
https://github.com/xamarin/xamarin-macios/issues/17579.
We mark all types that derive from NSObject when we find them in user
assemblies, so that these types may be used in storyboards (where the linker
can't see them), without having to go through hoops to make sure the linker
doesn't remove these types (which would make the storyboard fail to load,
because it would reference types that were linked away).
However, not everybody uses storyboards, so in some cases it may make sense to
link away as much as possible, so make it opt-in to skip this custom marking.
This is an experimental feature, and will break at least some apps. It may
break most apps, but if someone wants to try it out, they're welcome!
It can be turned on by passing `--skip-marking-nsobjects-in-user-assemblies=true` to mtouch/mmp:
```xml
<PropertyGroup>
<MtouchExtraArgs>--skip-marking-nsobjects-in-user-assemblies=true</MtouchExtraArgs>
</PropertyGroup>
```
Fixes#15723.
This way we can make the extraction work on Windows for Hot Restart, since ditto
doesn't exist on Windows.
This requires a few other changes:
* Move the Unzip task from the HotRestart tasks to be available everywhere.
* Change the Unzip task to use our existing decompression logic, which calls 'unzip'
on non-Windows platforms in order to correctly support symlinks.
The timeout can be given:
* By setting the __XAMARIN_DEBUG_CONNECT_TIMEOUT__ environment variable for the app when launching it.
* By passing the XamarinDebugConnectTimeout MSBuild property to 'dotnet run' or 'dotnet build /t:Run'.
* By setting the IOSDebugConnectTimeout MSBuild property at build time.
Fixes https://devdiv.visualstudio.com/DevDiv/_workitems/edit/1778177.
This is the pull request automatically created by the OneLocBuild task
in the build process to check-in localized files generated based upon
translation source files (.lcl files) handed-back from the downstream
localization pipeline. If there are issues in translations, visit
https://aka.ms/ceLocBug and log bugs for fixes. The OneLocBuild wiki is
https://aka.ms/onelocbuild and the localization process in general is
documented at https://aka.ms/AllAboutLoc.
Previously, we'd do this:
* Collect all possible native references.
* Extract any compressed native references (*.framework.zip, *.xcframework.zip,
*.resources.zip) to disk.
* Resolve the resulting native references.
This doesn't work very well on Windows (in non-connected/Hot Restart mode),
because some compressed files may contain symlinks (in particular compressed
xcframeworks). If those symlinks are for any other platform than the one we're
building for, they shouldn't matter, but if we extract the entire compressed
xcframework before figuring out what we need from it, we'd run into symlinks
and not knowing whether they should be ignored or not.
So rework the process to:
* Collect all possible native references.
* Resolve the resulting native references, peeking into zip files if need be.
* Extract any compressed native references, but only the parts of the zip we need.
This way we won't run into any symlinks unless we really need them, and it
should also improve build performance slightly, even on macOS, since we're not
extracting files we won't need (which can be significant for xcframeworks).
Additionally:
* Add support for unzipping on Windows by using System.IO.Compression.
* Show an error if attempting to extract a symlink in the last step in the
reworked process on Windows.
* Some tests had to be updated (since they poked into internals of the
ResolveNativeReferences task, and those internals have changed).
If the `RecursiveDir` metadata is empty, the GetDirectoryName method
throws an error because it isn't a valid path. This can happen on VS
design time builds.
This fixes a warning when documentation is enabled for a project:
> The file '~/.nuget/packages/fsharp.core/6.0.0/contentFiles/any/netstandard2.1/FSharp.Core.xml' does not specify a 'PublishFolderType' metadata, and a default value could not be calculated. The file will not be copied to the app bundle.
This doesn't change any behavior (as the warning says, the file wasn't copied
to the app bundle before either), but it makes the behavior explicitly
documented and silences the warning.
Fixes https://github.com/xamarin/xamarin-macios/issues/14939.
Fixes https://github.com/xamarin/xamarin-macios/issues/15897.
Added some information to the MacCatalyst templates based on comments from the community
on how to publish MacCatalyst apps on the App Store.
Fixes#17591
Set the GenerateRuntimeConfigurationFiles (GRCF) property to true
to avoid warnings at build time + add test for change.
Diving deeper into the fix...
- This warning only occurs with .NET apps which is why GRCF
is only updated in the dotnet directory and not msbuild (legacy)
- After examining the binlog (see issue), it was found that the GRCF
was contingent upon the HasRuntimeOutput property, which is only
defined for executable projects. And in this case, the user's project
output type is library thus both the RuntimeOutput and consequently
GRCF properties were not enabled.
- By setting the GRCF to true we can address the original warning of
concern while ensuring the rest of the projects's behavior is not
altered
in mysterious ways (i.e. by touching the RuntimeOutput property or the
project output type instead, these changes could have extraneous
effects).
Fixes#17543
---------
Co-authored-by: Rolf Bjarne Kvinge <rolf@xamarin.com>
With a project structure like this:
* Executable project references a library project.
* The library project references a binding project (or assembly).
The binding project's assembly will be copied to the library project's
output directory during the build. Unless we also make sure any binding
resource packages are copied as well, the executable project won't find those,
and the final app won't contain any native bits from the binding project.
The solution is to add any binding resource packages to the list of
files to be copied to the library's output directory.
Fixes https://github.com/xamarin/xamarin-macios/issues/13910.
Also don't hardcode the trailing directory separator as a forward slash,
because we might be executing on Windows (which is the real purpose behind
this change).
Rolf Bjarne Kvinge
dustin-wojciechowski
Mauro Agnoletti
Meri Khamoyan
Ivan Povazan
VS MobileTools Engineering Service 2
Haritha Mohan
Emanuel Fernandez Dell'Oca
TJ Lambert
Filip Navara
Peter Collins