Detect if a url we use in our tests actually works, and if not, save the
results and ignore any subsequent test that tries to use that url.
---------
Co-authored-by: Haritha Mohan <harithamohan@microsoft.com>
Closes#18356
In the static UnmanagedCallersOnly methods we don't know the generic
parameters of the type we're working with and we need to use this trick
to be able to call methods on the generic type without using reflection.
When we call a non-generic interface method implemented on a generic
class, the .NET runtime will resolve the generic parameters for us. In
the implementation of the interface method, we can simply use the
generic parameters and generate the same code we usually generate in the
UnmanagedCallersOnly callback method.
This is an example of the code we generate in addition to user code:
```csharp
internal interface __IRegistrarGenericTypeProxy__CustomNSObject_1__
{
void __IRegistrarGenericTypeProxy__CustomNSObject_1____SomeMethod (IntPtr p0);
}
public class CustomNSObject<T> : NSObject, __IRegistrarGenericTypeProxy__CustomNSObject_1__
where T : NSObject
{
[Export ("someMethod:")]
public void SomeMethod (T someInput)
{
// ...
}
// generated implementation of the proxy interface:
public void __IRegistrarGenericTypeProxy__CustomNSObject_1____SomeMethod (IntPtr sel, IntPtr p0, IntPtr* exception_gchandle)
{
try {
var obj0 = (T) Runtime.GetNSObject<T> (p0);
SomeMethod (obj0);
} catch (Exception ex) {
*exception_gchandle = Runtime.AllocGCHandle (ex);
}
}
// generated registrar callbacks:
private static class __Registrar_Callbacks__
{
[UnmanagedCallersOnly (EntryPoint = "_callback_1_CustomNSObject_1_SomeMethod")]
public unsafe static void callback_1_CustomNSObject_1_SomeMethod (IntPtr pobj, IntPtr sel, IntPtr p0, IntPtr* exception_gchandle)
{
var proxy = (__IRegistrarGenericTypeProxy__CustomNSObject_1__)Runtime.GetNSObject (pobj);
proxy.__IRegistrarGenericTypeProxy__CustomNSObject_1____SomeMethod (sel, p0, exception_gchandle);
}
}
}
```
```csharp
// regular non-generic class for comparison:
public class CustomNSObject : NSObject
where T : NSObject
{
[Export ("someMethod:")]
public void SomeMethod (NSSet someInput)
{
// ...
}
private static class __Registrar_Callbacks__
{
[UnmanagedCallersOnly (EntryPoint = "_callback_1_CustomNSObject_1_SomeMethod")]
public unsafe static void callback_1_CustomNSObject_1_SomeMethod (IntPtr pobj, IntPtr sel, IntPtr p0, IntPtr* exception_gchandle)
{
try {
NSSet obj0 = Runtime.GetNSObject<NSSet> (p0);
SomeMethod (obj0);
} catch (Exception ex) {
*exception_gchandle = Runtime.AllocGCHandle (ex);
}
}
}
}
```
---------
Co-authored-by: Alex Soto <alex@alexsoto.me>
For a given dylib named '/path/to/libMyLibrary.dylib', we pass this to the native linker:
-L/path/to -lMyLibrary
however, that doesn't work unless the dylib's name starts with 'lib'.
So detect this, and if the dylib doesn't start with 'lib' (say it's just
'MyLibrary.dylib'), then just pass the path to the dylib as-is to the native
linker:
/path/to/MyLibrary.dylib
Fixes https://github.com/xamarin/xamarin-macios/issues/15044.
This has a few advantages:
* A step towards simplifying the generator.
* A step towards being able to build binding projects on Windows, since it's easier
to build C# code in MSBuild using the Csc task rather than figuring out how to
call csc as an external program (which we've already done on macOS, but having
a single solution that works on all platforms is preferrable).
Note that this is only implemented for .NET, doing this for legacy Xamarin required a lot more work.
Added the following new checks:
```csharp
// dotnet_style_coalesce_expression = true
var v = x ?? y;
// dotnet_style_coalesce_expression = false
var v = x != null ? x : y; // or
var v = x == null ? y : x;
```
```csharp
// dotnet_style_null_propagation = true
var v = o?.ToString();
// dotnet_style_null_propagation = false
var v = o == null ? null : o.ToString(); // or
var v = o != null ? o.String() : null;
```
```csharp
// dotnet_style_prefer_is_null_check_over_reference_equality_method = true
if (value is null)
return;
// dotnet_style_prefer_is_null_check_over_reference_equality_method = false
if (object.ReferenceEquals(value, null))
return;
// dotnet_style_prefer_is_null_check_over_reference_equality_method = false
if ((object)o == null)
return;
```
There's no purpose to catching an exception just to throw it, so remove
the entire try/catch handler.
Fixes this compiler warning:
src/bgen/Generator.cs(1284,4): warning CA2200: Re-throwing caught exception changes stack information
* The 'copyCGImageAtTime:actualTime:error:' selector is deprecated, so
replicate that.
* Bind the 'generateCGImageAsynchronouslyForTime:completionHandler:'
selector.
Fixes parts 1 and 2 of https://github.com/xamarin/xamarin-macios/issues/18452.
Also don't create the instance directly from third-party bindings, the
required constructor might not be public - use Runtime.GetINativeObject
instead.
Fixes part 4 of https://github.com/xamarin/xamarin-macios/issues/18452.
The zip file might be used later directly on Windows (for instance it
might be embedded inside a binding assembly as an embedded resource), so make
sure to copy it back to Windows.
Fixes https://github.com/xamarin/xamarin-macios/issues/18402 part 2.
It seems this target has more problems than at first I thought, so make
it easier to opt-out of it by just setting a property in the csproj.
More investigation is needed, but I'm keeping the target on by default
for now, since it solves a real-world problem as well.
Ref: https://github.com/xamarin/xamarin-macios/issues/18445
Add public targets to compute the mlaunch command lines for installing
and launching mobile apps.
These new targets are:
* ComputeMlaunchInstallArguments
* ComputeMlaunchRunArguments
As part of this change, also create a few new public properties:
* MlaunchPath
* MlaunchRunArguments
* MlaunchInstallArguments
* MlaunchRunScript
* MlaunchInstallScript
If the *Script variables are set, the corresponding target will create a
script file with the path to mlaunch + the corresponding arguments.
Otherwise, it's also possible to get the arguments directly from the
build log.
Fixes https://github.com/xamarin/xamarin-macios/issues/18359.
# Description
This PR reduces the application's SOD (size on disk) by making
`__LINKEDIT Export Info` section smaller in the stripped Mach-O
binaries.
The feature is controlled by `_ExportSymbolsExplicitly` MSBuild property
and can be disabled by specifying: `-p:_ExportSymbolsExplicitly=true`
Fixes#18332
# Initial problem
It has been noticed that during stripping, the strip tool does not
resize the export info section after it removes the symbols. Instead it
only zeroes out the entries (achieved by calling `prune_trie` function):
- https://github.com/apple-oss-distributions/cctools/blob/cctools-986/misc/strip.c
- https://github.com/apple-oss-distributions/ld64/blob/ld64-711/src/other/PruneTrie.cpp
Thanks @lambdageek for helping to track this down.
# Approach
As Xamarin build process already collects all the [required symbols][1] needed
for the application to run and preserves them during the strip phase, we can
use the same file to instruct clang toolchain to export only those symbols via
the command line options: `-exported_symbols_list <file>` ([source][2]).
This will make the export info section only include what is necessary for the
runtime - and at the same time eliminate the problem of the `strip` tool which
does not resize stripped symbols.
# Investigation setup
The issue is observable by building and inspecting the test application:
https://github.com/xamarin/xamarin-macios/blob/main/tests/dotnet/MySingleView/MySingleView.csproj
and targeting iOS platform in Release mode.
## Results:
| Measure | MySingleView - main | MySingleView - this PR | Diff (%) |
| :--- | ---: | ---: | ---: |
| SOD (bytes) | 13668940 | 13458476 | -1.5% |
| .ipa (bytes) | 4829368 | 4827928 | -0.03% |
Even though zeroes are compressed well, the SOD is still affected and
unused section takes around 1.5% of the most simplistic app size.
Much bigger impact has been noted when trying out a MAUI iOS template
app with NativeAOT where the `__LINKEDIT Export Info` zeroes take up to
20MB of the SOD, but also with the regular macOS applications:
https://github.com/dotnet/runtime/issues/86707
### Repro current state of MySingleView.app with stripped binary
1. Build the app (you can ignore the need to run the sample, I just did it to
make sure the changes do not break anything)
```bash
make run-device
```
2. Print the load commands - [load_cmds_strip.list][3]
```bash
otool -l bin/Release/net7.0-ios/ios-arm64/MySingleView.app/MySingleView > load_cmds_strip.list
```
- We are interested in the export info section:
```
cmd LC_DYLD_INFO_ONLY
...
export_off 5942960
export_size 207712
```
3. Create a hex dump of the export info section - [hex_dump_strip.list][4]
``` bash
xxd -s 5942960 -l 207712 bin/Release/net7.0-ios/ios-arm64/MySingleView.app/MySingleView > hex_dump_strip.list
```
- NOTE: Notice around ~200kb of zeroes from ~0x005ab490 to ~0x005dda00
4. Verify exported symbols are correct - [dyld_info_strip.list][5]
``` bash
dyld_info -exports bin/Release/net7.0-ios/ios-arm64/MySingleView.app/MySingleView > dyld_info_strip.list
```
### Repro current state of MySingleView.app with unstripped binary
1. Build the app (the make target preserves the symbols)
```bash
make run-device-no-strip
```
2. Print the load commands - [load_cmds_nostrip.list][6]
```bash
otool -l bin/Release/net7.0-ios/ios-arm64/MySingleView.app/MySingleView > load_cmds_nostrip.list
```
- We are interested in the export info section:
```
cmd LC_DYLD_INFO_ONLY
...
export_off 5942960
export_size 207712
```
3. Create a hex dump of the export info section - [hex_dump_nostrip.list][7]
``` bash
xxd -s 5942960 -l 207712 bin/Release/net7.0-ios/ios-arm64/MySingleView.app/MySingleView > hex_dump_nostrip.list
```
- Notice that the range: ~ 0x005ab490 to ~ 0x005dda00 now includes exported symbol entries
4. Verify exported symbols are correct - [dyld_info_nostrip.list][8]
``` bash
dyld_info -exports bin/Release/net7.0-ios/ios-arm64/MySingleView.app/MySingleView > dyld_info_nostrip.list
```
### Repro the new approach
1. Build the app (the make target uses the new approach)
```bash
make run-device-export-syms
```
2. Print the load commands - [load_cmds_export.list][9]
```bash
otool -l bin/Release/net7.0-ios/ios-arm64/MySingleView.app/MySingleView > load_cmds_export.list
```
- We are interested in the export info section ***notice the reduced size of the section***:
```
cmd LC_DYLD_INFO_ONLY
...
export_off 5942432
export_size 1048
```
3. Create a hex dump of the export info section - [hex_dump_export.list][10]
``` bash
xxd -s 5942432 -l 1048 bin/Release/net7.0-ios/ios-arm64/MySingleView.app/MySingleView > hex_dump_export.list
```
4. Verify exported symbols are correct - [dyld_info_export.list][11]
``` bash
dyld_info -exports bin/Release/net7.0-ios/ios-arm64/MySingleView.app/MySingleView > dyld_info_export.list
```
---
## Additional benefits
With this approach we could also switch the way strip tool is invoked to
always strip all debug and local symbols via `strip -S -x` instead of passing
the file with symbols to preserve. This would remove the warning that we are
currently getting (which is being ignored):
```
/Applications/Xcode_14.3.0.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/strip: warning: removing global symbols from a final linked no longer supported. Use -exported_symbols_list at link time when building...
```
## Other references:
- https://github.com/qyang-nj/llios/blob/main/exported_symbol/README.md
[1]: 11e7883da0/tools/dotnet-linker/Steps/GenerateReferencesStep.cs (L38-L44)
[2]: https://developer.apple.com/library/archive/documentation/DeveloperTools/Conceptual/DynamicLibraries/100-Articles/DynamicLibraryDesignGuidelines.html
[3]: https://gist.github.com/ivanpovazan/d53f8d10be5e4ea9f39a41ea540aa7fa
[4]: https://gist.github.com/ivanpovazan/60637422f3ff8cb5f437ddd06a21d9c1
[5]: https://gist.github.com/ivanpovazan/352595ad15c2ac02f38dcb3bd4130642
[6]: https://gist.github.com/ivanpovazan/bf700161f2f3691d1d7381c98d4fa0be
[7]: https://gist.github.com/ivanpovazan/44269e4fff5ebd58a4d181451e5c106f
[8]: https://gist.github.com/ivanpovazan/38c5afe076502d514a77420af0e10b01
[9]: https://gist.github.com/ivanpovazan/3f663c3c630005f5a578605d48ba807e
[10]: https://gist.github.com/ivanpovazan/0bb84f64281d05ab20438aeaed64f13c
[11]: https://gist.github.com/ivanpovazan/78b3ba2288f53a2316b9bc46964e7e4f
---------
Co-authored-by: Rolf Bjarne Kvinge <rolf@xamarin.com>
This fixes a build problem on windows when a project has a project reference
to a binding project. The binding resource package from the binding project
would lack the manifest, and thus the main project wouldn't detect it as a
binding resource package, effectively not seeing any native resource from the
binding project.
Fixes https://github.com/xamarin/xamarin-macios/issues/18402.
We need to update the `System.Diagnostics.Tracer` package to version
`2.1.0-alpha` due to an issue with the `ChecksumAlgorithm` property of
older versions.
Cherry-pick: https://github.com/xamarin/xamarin-macios/pull/18310
This property will be required when building for a net7.0-* target framework
using .NET 8 (preview 6 - preview 5 does not need this fix)
Backport of #18411
Fixes#17707
The error target for when there is a conflict of interest in defining
both the runtime identifier and runtime identifiers is called during the
multi-rid builds, but not sure if the placement is the most ideal..
---------
Co-authored-by: GitHub Actions Autoformatter <github-actions-autoformatter@xamarin.com>
Co-authored-by: Rolf Bjarne Kvinge <rolf@xamarin.com>
Co-authored-by: Alex Soto <alex@alexsoto.me>
* Move all the RunAsync logic to the TestRuntime class, instead of having some
in TestRuntime and some in AppDelegate.
* Create a unified Task-based implementation for all platforms, optionally showing
a UI on the platforms where we can do that.
* Remove all the overloads that took a DateTime timeout, and instead only use a
TimeSpan timeout. This simplified some of the code.
* The new Task-based implementation will capture any exceptions (and rethrow most
of them) from the tasks we're waiting for, so no need to do that in each RunAsync
caller. This simplifies the testing code a lot for some tests.
* Add a new TryRunAsync method that will return (instead of rethrowing) any exceptions.
This simplifies some of the testing code (which verifies the correct exception,
or ignores the test in case of some exceptions).
* The new Task-based implementation will bubble up any NUnit exceptions, which
means that the tasks we're waiting for can call NUnit's Assert and the right thing
happens (in particular Assert.Ignore will actually ignore the test).
Bumps rolfbjarne/autoformat from 0.2.2 to 0.2.99.
## Commits
* Add support for a custom .NET executable and a custom working directory.
* Add branding.
* Change name so that it can be published to the marketplace.
* Quote script. Fixes issue 8.
See full diff: https://github.com/rolfbjarne/autoformat/compare/v0.2.2...v0.2.99
Fixes:
MonoTouchFixtures.AVFoundation.CaptureMetadataOutputTest
[PASS] Defaults
[PASS] Flags
[FAIL] MetadataObjectTypesTest : System.NotImplementedException : The method or operation is not implemented.
at TestRuntime.CheckXcodeVersion(Int32 major, Int32 minor, Int32 build) in xamarin-macios/tests/common/TestRuntime.cs:line 483
at MonoTouchFixtures.AVFoundation.CaptureMetadataOutputTest.MetadataObjectTypesTest() in xamarin-macios/tests/monotouch-test/AVFoundation/CaptureMetadataOutputTest.cs:line 131
at System.Reflection.MethodInvoker.InterpretedInvoke(Object , Span`1 , BindingFlags )
Fixes these warnings in Azure DevOps:
* Please upgrade the version of this build task in your build to major version: 2. For more information, please see: https://aka.ms/gdn-v1-deprecation
Governance Checks • Governance Checks • PoliCheck
* Please upgrade the version of this build task in your build to major version: 2. For more information, please see: https://aka.ms/gdn-v1-deprecation
Governance Checks • Governance Checks • Post Analysis
---------
Co-authored-by: Alex Soto <alex@alexsoto.me>
Filip Navara
Emanuel Fernandez Dell'Oca
Rolf Bjarne Kvinge
Šimon Rozsíval
Meri Khamoyan
Manuel de la Pena
Steve Hawley
EmilienDup
Mike Bond
Ivan Povazan
James Parsons
VS MobileTools Engineering Service 2
Haritha Mohan
dependabot[bot]
Alex Soto