Fixes: https://github.com/dotnet/runtime/issues/83893
Context: https://github.com/dotnet/runtime/pull/72717
In .NET 8, `System.Reflection.{ConstructorInfo,MethodInfo}.Invoke()`
will use `System.Reflection.Emit` when called more than once.
This impacts startup in mobile applications, so it may not be a
desirable feature.
Unfortunately, this appears to happen quite easily in Android apps;
some examples (using a custom dotnet/runtime build for extra output):
* https://gist.github.com/ivanpovazan/2563ea9d2fea320e6425cfcc58da3ee5
* https://gist.github.com/ivanpovazan/d2546d4abad17900d4366cc29e1689b2
The primary situation in which this happens is that all Java-originated
`Java.Lang.Object` subclass constructor invocations always hit
`ConstructorInfo.Invoke()`; see `TypeManager.Activate()`.
To solve this problem, we can set:
<ItemGroup>
<RuntimeHostConfigurationOption
Include="Switch.System.Reflection.ForceInterpretedInvoke"
Value="$(_SystemReflectionForceInterpretedInvoke)"
Trim="true"
/>
</ItemGroup>
Setting the `Switch.System.Reflection.ForceInterpretedInvoke` switch
to True causes the `System.Reflection.Emit` codepath to be *skipped*.
We can set `$(_SystemReflectionForceInterpretedInvoke)` to test
out the setting in various apps.
I added a test to verify the "private" switch is actually set.
I also updated the `.aotprofile` to verify that all
`System.Reflection.Emit` code paths disappear from
`dotnet new android` applications.
Fixes: https://github.com/xamarin/xamarin-android/issues/7953
Context: 11f0e1bf64
When a timezone changes in a `Release` config app, it can crash with:
[monodroid] Unable to find Android.Runtime.AndroidEnvironment.NotifyTimeZoneChanged()!
In commit 11f0e1b, we removed the line:
<?xml version="1.0" encoding="utf-8" ?>
<linker>
<assembly fullname="Mono.Android">
-- <type fullname="Android.Runtime.AndroidEnvironment" />
Unfortunately, `AndroidEnvironment.NotifyTimeZoneChanged()` is called
from non-managed code, via:
* The `NotifyTimeZoneChanges` broadcast receiver
(in `src/java-runtime/java/mono/android/app/NotifyTimeZoneChanges.java`)
which calls-
* The `Rutime.notifyTimeZoneChanged()` `native` method
(in `src/java-runtime/java/mono/android/Runtime.java`),
implemented in-
* `Java_mono_android_Runtime_notifyTimeZoneChanged()`
(in`src/monodroid/jni/timezones.cc`).
The managed linker cannot "see" any of these references, so we
need to *always* preserve this method, as it is always callable.
Update `src/Microsoft.Android.Sdk.ILLink/PreserveLists/Mono.Android.xml`
so that `Android.Runtime.AndroidEnvironment.NotifyTimeZoneChanged()`
is always preserved.
Added a test for this scenario.
TODO: we may want to audit all `mono_class_get_method_from_name()`
calls and add more tests cases.
I also cleaned up the tests a bit with a `getResource()` local function.
Context: https://github.com/xamarin/xamarin-android/issues/7794
Convert `tests/CodeGen-Binding/Xamarin.Android.JcwGen-Tests` to .NET.
This ports 35 of the 41 tests in `Xamarin.Android.JcwGen-Tests`.
The remaining tests require additional projects to be ported to .NET
to run successfully or require additional custom MSBuild logic from
`Xamarin.Android.JcwGen-Tests` to be ported.
We are choosing to not prioritize the remaining tests at this point
and committing what we already have.
The unported tests are tracked in #7794.
Context: 610ade7f97
Context: a389cd1873
Remove where we use the [ILRepack NuGet package][0] to package
`pdb2mdb.exe` into `Xamarin.Android.Build.Tasks.dll`, and remove the
`<ConvertDebuggingFiles/>` MSBuild task. These are not used in
.NET 6+, and can be removed in xamarin-android/main.
I also removed:
* Code or tests around `.mdb` files
* Tests around `$(DebugType)=Full`
In future PRs, I think we can remove:
* `ILRepack` completely
* `Xamarin.Android.Legacy.targets`
* Other unused MSBuild tasks
[0]: https://www.nuget.org/packages/ILRepack/
Tests #7622
Localisation assemblies were not being fast deployed. Commit xamarin/monodroid@1f52d58
fixed that. However we need to add a unit test to make sure that these files are deployed and
actually produce the required results.
So lets use Humanizer to handle the localisation. This PR adds a test which checks that the
app on the device outputs the expected text which Humanizer is supposed to generate.
It also updates existing packaging tests to make sure that the required satellite assemblies are
packaged.
We are no longer running the `net472` versions of our desktop tests,
and we can save a bit of time by not building them either.
I've removed the `net472` version of `xaprepare` as well, as we have
a hard dependency on .NET 7+.
The `run-msbuild-device-tests.yaml` template was unused and is removed.
Context: 112c8328a6
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=14968
Context: 5e923e3693
Context: 2e5f7bb4f7
For historical purposes…
<https://bugzilla.xamarin.com/show_bug.cgi?id=14968> is (was?) a bug
wherein accessing an "invalid" URL such as
<http://example.com/?query&foo|bar> would result in a
`URISyntaxException` at runtime:
Exception of type 'Java.Net.URISyntaxException' was thrown.
at Android.Runtime.JNIEnv.NewObject (IntPtr jclass, IntPtr jmethod, Android.Runtime.JValue[] parms)
at Java.Net.URI..ctor (System.String uri)
at Android.Runtime.AndroidEnvironment+_Proxy.IsBypassed (System.Uri host)
at System.Net.ServicePointManager.FindServicePoint (System.Uri address, IWebProxy proxy)
at System.Net.HttpWebRequest.GetServicePoint ()
at System.Net.HttpWebRequest.BeginGetResponse (System.AsyncCallback callback, System.Object state)
at System.Net.HttpWebRequest.GetResponse ()
at Xamarin.Android.RuntimeTests.ProxyTest.QuoteInvalidQuoteUrlsShouldWork ()
at (wrapper managed-to-native) System.Reflection.MonoMethod:InternalInvoke (System.Reflection.MonoMethod,object,object[],System.Exception&)
at System.Reflection.MonoMethod.Invoke (System.Object obj, BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture)
--- End of managed exception stack trace ---
java.net.URISyntaxException: Illegal character in query at index 29: http://example.com/?query&foo|bar
at libcore.net.UriCodec.validate(UriCodec.java:63)
at java.net.URI.parseURI(URI.java:406)
at java.net.URI.<init>(URI.java:204)
at xamarin.android.nunitlite.TestSuiteInstrumentation.n_onStart(Native Method)
at xamarin.android.nunitlite.TestSuiteInstrumentation.onStart(TestSuiteInstrumentation.java:39)
at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1701)
Index 29 is `|` (zero-based!).
The "problem" here that using `|` within the query string is accepted
under .NET & Mono, which resulted in a "breakage of expectations" for
Xamarin.Android customers ("this URL works on .NET!").
The "fix" was to use [`Uri.AbsoluteUri`][5], which replaces `|` with
`%7C`, hex-escaping the offending value and adding the
`ProxyTest.QuoteInvalidQuoteUrlsShouldWork()` unit test.
That background done, commit 112c8328 updated
`ProxyTest.QuoteInvalidQuoteUrlsShouldWork()` to hit
<https://bing.com/?query&foo|bar> instead of
<http://example.com/?query&foo|bar>, on the assumption that it would
be "better" if we *didn't* use non-Microsoft URLs as part of our unit
tests.
The problem is that this didn't work (eep?! not sure how PR #7909
was green here!):
System.Net.WebException : net_http_ssl_connection_failed
----> System.Net.Http.HttpRequestException : net_http_ssl_connection_failed
----> System.Security.Authentication.AuthenticationException : net_auth_SSPI
----> Interop+AndroidCrypto+SslException : Exception_WasThrown, Interop+AndroidCrypto+SslException
Update `ProxyTest.QuoteInvalidQuoteUrlsShouldWork()` to instead hit
<http://www.msftconnecttest.com/connecttest.txt?query&foo|bar>, which
is a Microsoft-owned server -- no possible DOS'ing of
<http://example.com> -- which *passes* for this particular test.
[0]: https://developer.android.com/reference/java/net/URI#URI(java.lang.String)
[1]: https://www.ietf.org/rfc/rfc2396.txt
[2]: https://www.ietf.org/rfc/rfc3986.html#section-3.4
[3]: https://android.googlesource.com/platform/libcore/+/jb-mr2-release/luni/src/main/java/libcore/net/UriCodec.java#46
[4]: https://android.googlesource.com/platform/libcore/+/jb-mr2-release/luni/src/main/java/java/net/URI.java#349
[5]: https://learn.microsoft.com/en-us/dotnet/api/system.uri.absoluteuri?view=net-7.0
Disabling marshal method generation via
`$(AndroidEnableMarshalMethods)`=False turns off a lot of native code
generation in the `MarshalMethodsNativeAssemblyGenerator` class, but
the class is also responsible for outputting a correctly sized cache
area (an array of *X* pointers) to be used by the native runtime to
cache pointers to `MonoImage` instances.
`libmonodroid.so` trusts that `Xamarin.Android.Build.Tasks.dll` et al
will generate correct code, and thus does not verify the size of
generated array. This trust, unfortunately, was broken because with
marshal methods disabled, the native code generator created an output
cache array that was 0 entries in size, thus leading to segfault when
attempting to run the application.
Fix the issue and also parametrize one of the on-device tests to be
built twice, with marshal methods explicitly disabled and explicitly
enabled.
Context: https://dotnet.microsoft.com/platform/support/policy/maui
> A major version of .NET MAUI receives support for a minimum of 6
> months after a successor (the next major release) ships. For
> example, .NET MAUI 6.0 will be supported for 6 months after .NET
> MAUI 7.0 ships. Similarly, .NET MAUI 7.0 will receive support for 6
> months after .NET MAUI 8.0 ships.
By the time .NET 8 GA ships, .NET 6 MAUI projects will not be supported.
Remove .NET 6 support from the .NET 8 `android` workload.
I removed `net6.0-android` parameters from various tests, making sure
we still have .NET 7 and .NET 8 tests.
Our SSL test can fail quite often with a HTTP-429 Too Many Requests
error. This makes the CI very unstable as we are constantly having
to wait and retry the tests.
Improve this by putting that logic into the test itself. If we get
an HTTP-429 we should try some other SSL site.
Context: 346a933018
Commit 346a9330 made it possible to build and link AOT shared libraries
without having to have an Android NDK available. However, it seems
that the produced shared libraries do not depend on the `libc.so` and
`libm.so ` standard libraries, leading to runtime errors similar to:
D monodroid-assembly: monodroid_dlopen: hash match found, DSO name is 'libaot-Mono.Android.dll.so'
I monodroid-assembly: Trying to load shared library '/data/app/~~4kGTzxWG6HKO7nzyg6ryBg==/com.companyname.hellomaui.old-h8Jlutcvf8Dzfpuuq1ouUA==/lib/arm64/libaot-Mono.Android.dll.so'
I monodroid-assembly: Failed to load shared library '/data/app/~~4kGTzxWG6HKO7nzyg6ryBg==/com.companyname.hellomaui.old-h8Jlutcvf8Dzfpuuq1ouUA==/lib/arm64/libaot-Mono.Android.dll.so'. dlopen failed: cannot locate symbol "memset" referenced by "/data/app/~~4kGTzxWG6HKO7nzyg6ryBg==/com.companyname.hellomaui.old-h8Jlutcvf8Dzfpuuq1ouUA==/lib/arm64/libaot-Mono.Android.dll.so"...
This is caused by us not passing the `-lc` and `-lm` flags to the
linker (or full paths to the `libc.so` and `libm.so` libraries).
However, without the NDK available we do not have either of these
standard libraries to link against.
Fortunately, ELF shared libraries by default will resolve symbols
from anywhere in the process address space, so there is no need to
provide a `libc.so` which contains *all* public symbols. We can
instead build and use a "dummy"/"stub" version of `libc.so` and
`libm.so`, existing only so that the linker will add the appropriate
`NEEDED` entries and resolve symbols from those libraries.
Update `src/monodroid` to also produce "stub" `libc.so` and `libm.so`
native libraries, and update the `<Aot/>` task to add the equivalent
of `-L path/to/libstubs -lc -lm` to the AOT command-line.
This ensures that the resulting `.so` files now require `libc.so`:
$ llvm-readelf -a Mono.Android.dll.so | grep NEEDED
0x0000000000000001 (NEEDED) Shared library: [libc.so]
0x0000000000000001 (NEEDED) Shared library: [libm.so]
We occasionally see the following error on CI:
Unhandled error launching soft debugger
System.AggregateException: One or more errors occurred. (DWP Handshake failed.)
---> System.IO.IOException: DWP Handshake failed.
at Mono.Debugger.Soft.Connection.Connect() in /Users/builder/azdo/_work/1/s/xamarin-android/external/debugger-libs/Mono.Debugger.Soft/Mono.Debugger.Soft/Connection.cs:line 1361
at Mono.Debugger.Soft.VirtualMachine.connect() in /Users/builder/azdo/_work/1/s/xamarin-android/external/debugger-libs/Mono.Debugger.Soft/Mono.Debugger.Soft/VirtualMachine.cs:line 381
at Mono.Debugger.Soft.VirtualMachineManager.Connect(Connection transport, StreamReader standardOutput, StreamReader standardError) in /Users/builder/azdo/_work/1/s/xamarin-android/external/debugger-libs/Mono.Debugger.Soft/Mono.Debugger.Soft/VirtualMachineManager.cs:line 353
at Mono.Debugger.Soft.VirtualMachineManager.ConnectInternalAsync(Socket dbg_sock, Socket con_sock, IPEndPoint dbg_ep, IPEndPoint con_ep) in /Users/builder/azdo/_work/1/s/xamarin-android/external/debugger-libs/Mono.Debugger.Soft/Mono.Debugger.Soft/VirtualMachineManager.cs:line 309
--- End of inner exception stack trace ---
What is supposed to happen is the test should be retried. However,
according to [the NUnit docs][1], the `[Retry]` attribute won't work
if an unhandled exception occurs.
Fix this by catching all exceptions and calling `Assert.Fail()` when
an exception is caught. This allows the `[Retry]` attribute to do
its job and retry the test.
[1]: https://docs.nunit.org/articles/nunit/writing-tests/attributes/retry.html
Context: https://github.com/xamarin/xamarin-android/pull/7850#discussion_r1130001948
With the re-parallelization of our test suites (https://github.com/xamarin/xamarin-android/pull/7804, https://github.com/xamarin/xamarin-android/pull/7850), combined with the fact that most PR's end up running the full test suite anyways, the separate "Smoke Tests" jobs aren't really saving us much, if any, CI time. We feel we would be better off using those additional agents to run the full test suite faster.
This PR removes the separate "Smoke Tests" jobs and runs all tests in the same jobs.
Additionally, as the `[Category ("SmokeTests")]` NUnit attribute is now only used by the Windows build stage to run in-tree tests, it has been cut back to the absolute bare minimum, saving ~45 minutes. Note that all the tests will continue to be run against the Windows installer, this only runs fewer tests for the Windows in-tree tests.
Bring the AzDO parallelization from https://github.com/xamarin/xamarin-android/pull/7804 and environment setup improvements from https://github.com/xamarin/xamarin-android/pull/7832 to `Xamarin.Android.Build.Tests.csproj` based test suites. This includes both the main `MSBuild` stage and the `Smoke Tests` jobs.
Increases parallelization of all jobs as many were approaching ~90 minutes.
As there is no longer a place in the `MSBuild` stage to run `Xamarin.Android.Tools.Aidl-Tests` tests, it was moved to the `macOS > Tests > APKs .NET` stage. This suite should be fine to only run on Mac and not Windows. (Note this test assembly was also updated to .NET 7. This required moving it from `Xamarin.Android-Tests.sln` which is currently built with Mono which cannot build .NET 7+ projects. It now is built via `Xamarin.Android.sln`.)
The following changes attempt to improve the reliability and performance
of our MSBuild test jobs. All nightly tests have been updated to run
against our .NET build/test artifacts rather than classic XA.
The setup execution time for non-device tests has been improved by
moving a handful of adb commands from BaseTest to DeviceTest setup.
These would run for nearly every test run and result in a lot of waiting
due to some RunProcess failures and timeouts.
The output from the shell command we run to check if a device is online
is now cached and only refreshed when required by certain tests/asserts.
The `AssertHasDevices` check has been moved into `DeviceTest` setup,
and removed from individual tests in most cases.
Attempts to restart the emulator if a test determines that it is
inaccessible have been fixed, and emulator data will not be reset in
this case.
I played around with a handful of emulator launch settings and found
that removing the no-boot-anim and headless UI options improved the
reliability of nightly launch tests, and resulted in faster boot
times locally. These options are now disabled for those tests.
The `DeploymentTest` class has been replaced by `TimeZoneInfoTests` and
`LocalizationTests`, and the other tests in that class have been moved
to `InstallAndRunTests`. `TimeZoneInfoTests` and `LocalizationTests`
will now validate command line output rather than trying to press a
button and read a UI element. This should make them more reliable and
faster.
The `TimeZoneInfoTests` and `LocalizationTests` suites have been moved
into separate test stages in the nightly test job.
An issue that caused LocalizationTests nodes 11 and 12 to not contain
any tests has been fixed.
An issue that caused LocalizationTests node 1 to also run tests from
node 11 and node 12 has been fixed.
Test result attachments have been fixed for `TimeZoneInfoTests` and
`LocalizationTests`, ensuring that we capture the right logcat and build
files for each test variant.
An issue causing binlog files to be overwritten by tests that use
multiple build targets has been fixed by naming the binlog file after
the build log file.
Console output verbosity has been set to normal for dotnet test
invocations, which should help with debugging.
Test NUnit packages have been updated to their latest versions.
Context: 5d46685050
Context: 7b2e172829
Commit 5d466850 -- via [7b2e17][0] -- added usage of the NUnit3
[`RetryAttribute`][1] to some of our on-device unit tests.
Unfortunately, `RetryAttribute` doesn't exist in NUnitLite, resulting
in build failures such as:
…/tests/Mono.Android-Tests/Xamarin.Android.Net/AndroidMessageHandlerTests.cs(42,4):
error CS0246: The type or namespace name 'RetryAttribute' could not be found (are you missing a using directive or an assembly reference?)
Oops.
This failure wasn't caught because we've trained ourselves to
partially ignore various failures in the **Tests** tab -- networking
is hard, mkay? -- but completely missed the *compilation* failures,
which don't appear in the **Tests** tab and are harder to see.
Double oops.
Update `AndroidMessageHandlerTests.cs` to remove usage of `[Retry]`
and instead retry things "manually"
[0]: 7b2e172829
[1]: https://docs.nunit.org/articles/nunit/writing-tests/attributes/retry.html
Changes: e3ab0b5...632ddca
Changes: ff7c19f...2bdc3cb
Updates:
* Microsoft.Dotnet.Sdk.Internal: from 8.0.100-preview.2.23123.10 to 8.0.100-preview.3.23128.1
* Microsoft.NET.ILLink.Tasks: from 8.0.0-preview.2.23123.4 to 8.0.0-preview.2.23127.4
* Microsoft.NETCore.App.Ref: from 8.0.0-preview.2.23123.4 to 8.0.0-preview.2.23127.4
~~ Other changes ~~
* Keep `$(DotNetMonoManifestVersionBand)` on preview.2
* Keep `$(DotNetEmscriptenManifestVersionBand)` on preview.2
* Changes for the linker's new dependency: `Microsoft.DotNet.Cecil`
* Update `BuildReleaseArm64XFormsDotNet.apkdesc`
apkdiff: File 'assemblies/System.IO.Compression.dll' has changed by -875 bytes (-5.47 %). This exceeds the threshold of 5.00 %.
* Update to MSBuild.StructuredLogger 2.1.787
System.NotSupportedException : Unsupported log file format. Latest supported version is 15, the log file has version 16.
Co-authored-by: Jonathan Peppers <jonathan.peppers@microsoft.com>
Context: https://github.com/xamarin/xamarin-android/issues/7230
Context: https://github.com/dotnet/runtime/issues/80935
When a WCF application invokes an endpoint which returns compressed
content, and `AndroidMessageHandler` is doing the network requests
([the default when `$(UseNativeHttpHandler)`=True][0]):
var soapClient = new WebServiceSoapClient(WebServiceSoapClient.EndpointConfiguration.WebServiceSoap);
//Async test
var helloResponse = await soapClient.HelloWorldAsync();
then the method will throw:
The formatter threw an exception while trying to deserialize the message: There was an error while trying to deserialize parameter http://tempuri.org/:HelloWorldResponse.
---> There was an error deserializing the object of type ServiceReference1.HelloWorldResponseBody. Unexpected end of file. Following elements are not closed: HelloWorldResult, HelloWorldResponse, Body, Envelope. Line 1, position 298.
at System.Runtime.Serialization.XmlObjectSerializer.ReadObjectHandleExceptions(XmlReaderDelegator reader, Boolean verifyObjectName, DataContractResolver dataContractResolver)
at System.Runtime.Serialization.XmlObjectSerializer.ReadObjectHandleExceptions(XmlReaderDelegator reader, Boolean verifyObjectName)
at System.Runtime.Serialization.DataContractSerializer.ReadObject(XmlDictionaryReader reader, Boolean verifyObjectName)
at System.ServiceModel.Dispatcher.DataContractSerializerOperationFormatter.PartInfo.ReadObject(XmlDictionaryReader reader, XmlObjectSerializer serializer) in /_/src/System.Private.ServiceModel/src/System/ServiceModel/Dispatcher/DataContractSerializerOperationFormatter.cs:line 657
at System.ServiceModel.Dispatcher.DataContractSerializerOperationFormatter.PartInfo.ReadObject(XmlDictionaryReader reader) in /_/src/System.Private.ServiceModel/src/System/ServiceModel/Dispatcher/DataContractSerializerOperationFormatter.cs:line 652
at System.ServiceModel.Dispatcher.DataContractSerializerOperationFormatter.DeserializeParameterPart(XmlDictionaryReader reader, PartInfo part, Boolean isRequest) in /_/src/System.Private.ServiceModel/src/System/ServiceModel/Dispatcher/DataContractSerializerOperationFormatter.cs:line 521
The reason for this is that when `AndroidMessageHandler` creates a
wrapping decompression stream, it does not update `Content-Length` to
match the length of the decoded content, because it doesn't have a
way to know what the length is without first reading the stream to the
end, and that might prevent the end user to read the content.
(Additionally, I think the `Content-Length` header should reflect the
*original* content length, for the end user to be able to
interpret the response as it was sent.)
WCF, on the other hand, looks at the `Content-Length` header and, if
found, takes the value and reads only that many bytes from the content
stream and no more, which will almost always result in short reads and
failure to correctly interpret the response.
Workaround this issue by making `AndroidMessageHandler` behave the
same way as other handlers implemented in the BCL. What they do in
this situation is remove the `Content-Length` header, making WCF
read the stream to the end. Additionally, the clients remove the
compressed content encoding identifier from the `Content-Encoding`
header.
var handler = new AndroidMessageHandler {
AutomaticDecompression = DecompressionMethods.All
};
var client = new HttpClient (handler);
var response = await client.GetAsync ("https://httpbin.org/gzip");
// response.Content.Headers won't contain Content-Length,
// and response.Content.Headers.ContentEncoding won't contain `gzip`.
As a bonus, also adds support for decompression of responses
compressed with the `Brotli` compression which use the `br` encoding
ID in the `Content-Encoding` header.
[0]: https://learn.microsoft.com/en-us/dotnet/core/deploying/trimming/trimming-options?pivots=dotnet-7-0
Fixes: https://github.com/xamarin/xamarin-android/issues/7764
Context: d236af5453
Context: https://github.com/xamarin/AndroidX/issues/64#issuecomment-609970584
.NET Android dropped support for `$(BundleAssemblies)`=True; see also
commit d236af54 and [a comment on xamarin/AndroidX#64][0]:
> After internal discussion, we are strongly considering removing
> support for `$(BundleAssemblies)` in .NET 5, as it increases
> on-device memory use and slows down app startup times.
However, this "dropping of support" was incomplete; setting it would
result in broken apps!
% dotnet new android -n android-ba
% cd android-ba
% dotnet build -p:BundleAssemblies=true
# run the app, or just look at the .apk contents
% unzip -l bin/Debug/net7.0-android/*-Signed.apk | grep assem
660 02-03-2023 14:13 assemblies/rc.bin
# Note: `assemblies/assemblies.blob` is not present, but is required!
Fix this by *removing* most usage of the `$(BundleAssemblies)` MSBuild
property as well as all the code affected by it.
- Stop running bundle test apps on CI
- Remove bundle unit tests
- Remove the `CodeGen-MkBundle` project from the tests solution
- Remove the `Xamarin.Android.MakeBundle-Tests` from the tests
solution
- Remove all native code (and configuration) related to mkbundle
- Warn about `$(BundleAssemblies)` usage in .NET projects.
A XA1035 warning will be emitted when `$(BundleAssemblies)`=True
in .NET Android apps.
Preserve a couple of related properties in the `<BuildApk/>` and
`<GeneratePackageManagerJava/>` tasks, because they are still
used by the debugger targets from `xamarin/monodroid`.
[0]: https://github.com/xamarin/AndroidX/issues/64#issuecomment-6099
Fixes: https://github.com/xamarin/xamarin-android/issues/7819
Our native runtime uses a cache of pointers to loaded managed assembly
images (essentially an array of the native `struct MonoImage` pointers)
which is pre-allocated at build time and placed in the native
`libxamarin-app.so` library.
While generating the cache, we also generate hashes for a number of
assembly name permutations (currently two per assembly: with and
without the extension). Only unique assembly names are considered when
generating the cache (it's possible to have duplicate names, because we
package more than one copy of some assemblies - those which are
architecture specific).
This algorithm had a bug which made it ignore culture prefix in
satellite assembly names (e.g. `en/MyAssembly.resources.dll`);
instead of several entries for each culture, we generated only two
entries (e.g. `MyAssembly.resources.dll` and `MyAssembly.resources`)
but we still counted each culture-prefixed assembly and stored that
number in `libxamarin-app.so` to be used at runtime to calculate number
of entries in the cache.
This made the array storing cached `MonoImage*` pointers to be smaller
than the number of actual assemblies in the APK times 2 and in some
cases we failed to look up pointer to some images and, as the result,
passed a `NULL` pointer to MonoVM which then caused a segmentation fault
trying to dereference the pointer:
F DEBUG : signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x000000000000001c
F DEBUG : Cause: null pointer dereference
…
F DEBUG : #00 pc 00000000000bdad8 /data/app/~~7gCkYmQcKwTyS9NmxfgKxA==/com.companyname.bubby-PJIkJ0Lv0RiQhEVLeWi4wg==/split_config.arm64_v8a.apk!libmonosgen-2.0.so (mono_class_get_checked+24) (BuildId: 80b786675a56824331f363bf29a2b54f454cf006)
Update the `<BuildApk/>` task to stop ignoring the culture prefix for
satellite assemblies in order to avoid the situation. Additionally,
since the previous assumption that MonoVM will validate all pointers
passed to its APIs turned out to be unwarranted, we now check more
carefully for `NULL` pointers when trying to obtain a native function
pointer from the MonoVM runtime.
Add a test for the issue, based on the
`MissingSatelliteAssemblyInLibrary()` packaging test which, when it
fails, will result in a `SIGABRT`:
D monodroid-assembly: assembly_store_open_from_bundles: looking for bundled name: 'System.Private.CoreLib' (hash 0x6b0ff375198b9c17)
F monodroid-assembly: Invalid assembly index 19, exceeds the maximum index of 11
F libc : Fatal signal 6 (SIGABRT), code -1 (SI_QUEUE) in tid 17957 (semblyinlibrary), pid 17957 (semblyinlibrary)
Enhance the packaging test `MissingSatelliteAssemblyInLibrary()` by
adding more languages, to see if they're all packaged correctly.
Changes: bbaeda6f69...77800dda83
* xamarin/java.interop@77800dda: [Java.Interop.Tools.Expressions] Add Java.Interop.Tools.Expressions (xamarin/java.interop#1046)
Remember `jnimarshalmethod-gen.exe`? (0140ab83, d5b2ece6, 106a6211, …)
It never made it to completion, was never stable enough to be used,
even though we did add a "public" *documented*
`$(AndroidGenerateJniMarshalMethods)` MSBuild property to control it…
`jnimarshalmethod-gen.exe` never made it to .NET Android, as it
required .NET Framework features which didn't make it to .NET Core.
xamarin/java.interop@77800dda updates `jnimarshalmethod-gen` to drop
support for building on .NET Framework 4.7.2, and adds support to
build it for .NET 7.
Update `build-tools/installers/create-installers.targets` so that
`jnimarshalmethod-gen.exe` is no longer included in the Classic
installers (which are increasingly moot anyway; see also 618bd4ab).
Remove generation of `Java.Runtime.Environment.dll.config`, as
that file was only supported when using Mono, which won't be the case
under .NET 7.
Update the `_GenerateJniMarshalMethods` target so that it `<Error/>`s
when `$(AndroidGenerateJniMarshalMethods)` is True. While
`jnimarshalmethod-gen.dll` may run on .NET 7 now, @jonpryor doesn't
want to deal with the *integration* work to see if it is usable on
.NET Android, especially considering that the Classic version didn't
work either! (That integration effort is for "later".)
[Previously](https://github.com/xamarin/xamarin-android/pull/6360), we split our `MSBuildIntegration` unit tests to run across multiple CI test agents. While a huge improvement, there are a few downsides to the approach we went with at the time:
- The number of agents is hardcoded by copy/pasting steps in the CI YAML.
- The tests must be manually load-balanced across agents with `[Category ("Node-X")]` in code which is cumbersome and hard to keep updated.
As we are at the point where our tests have expanded well past the 1 hour target we are faced with needing to increase our parallelization.
First, we can remove the YAML duplication by using AZDO's built-in `parallel` strategy, allowing us to control the number of agents to use with a single number:
```yaml
- job:
strategy:
parallel: 4
```
This leaves us the issue of automatically splitting our tests among an unknown number of test agents. A clever solution is [provided in the AZDO docs](https://learn.microsoft.com/en-us/azure/devops/pipelines/test/parallel-testing-any-test-runner?view=azure-devops):
- Use `dotnet test --list-tests` to find all the tests
- Use a script to calculate which tests the agent should run, based on `$(System.JobPositionInPhase)` and `$(System.TotalJobsInPhase)`
- Pass those test names into `dotnet test` as a filter
Unfortunately there are issues with the provided approach:
- `dotnet test --list-tests` is not compatible with `--filter` so you have to run *all* tests in the test assembly which is not desirable for us. (https://github.com/dotnet/sdk/issues/8643)
- Passing test names (including test parameters) on the command line hits limitations with escaping certain characters and argument limits.
So the approach is good, but we have to do all the work ourselves. Enter [dotnet-test-slicer](https://github.com/jpobst/dotnet-test-slicer).
This dotnet global tool:
- Uses the `NUnit.Engine` NuGet package to find all tests in a test assembly using the desired filter query.
- Slices the test list for the current test agent.
- Outputs the desired tests into an NUnit-specific `.runsettings` file that can be passed to `dotnet test --settings foo.runsettings`, bypassing command line arguments.
Voila! Now we can automatically scale our test agents up or down as needed by changing a single variable in our YAML file.
Limitation: The tests are sliced in a round robin fashion. This can produce uneven results if test durations are uneven. A future optimization would be to store approximate test durations so tests can be load balanced more intelligently.
Changes: 9e0a4690ad...bbaeda6f69
* xamarin/java.interop@bbaeda6f: [Java.Interop] Support Desugar + interface static methods (xamarin/java.interop#1077)
Context: bbaeda6f69
Context: 1f27ab552d
Context: f6f11a5a79
[Desugaring][0] is the process of rewriting Java bytecode so that
Java 8+ constructs can be used on Android pre-7.0 (API-24), as
API-24 is the Android version which added native support for Java 8
features such as [interface default methods][1].
One of the implications of desugaring is that methods can "move";
consider this Java interface:
package example;
public interface StaticMethodsInterface {
static int getValue() { return 3; }
}
Java.Interop bindings will attempt to invoke the `getValue().I`
method on the type `example/StaticMethodsInterface`:
public partial interface IStaticMethodsInterface : IJavaObject, IJavaPeerable {
private static readonly JniPeerMembers _members = new XAPeerMembers ("example/StaticMethodsInterface", typeof (IStaticMethodsInterface), isInterface: true);
static unsafe int Value {
get {
const string __id = "getValue.()I";
var __rm = _members.StaticMethods.InvokeInt32Method (__id, null);
return __rm;
}
}
}
The problem is that, after Desugaring, the Java side *actually* looks
like this:
package example;
public interface StaticMethodsInterface {
}
public class StaticMethodsInterface$-CC {
public static int getValue() {return 3;}
}
Commits xamarin/java.interop@1f27ab55 and f6f11a5a added partial
runtime support for this scenario via
`AndroidTypeManager.GetStaticMethodFallbackTypesCore()`, which would
attempt to lookup types with a `$-CC` suffix.
While this was a good start, it wasn't ever actually *tested*
end-to-end. Consequently, instead of *working*, this would instead
cause the process to *abort*:
JNI DETECTED ERROR IN APPLICATION: can't call static int example.StaticMethodsInterface$-CC.getValue() with class java.lang.Class<example.StaticMethodsInterface>
in call to CallStaticIntMethodA
from void crc….MainActivity.n_onCreate(android.os.Bundle)
Oops.
xamarin/java.interop@bbaeda6f improves our runtime support for
invoking *`static`* methods on interfaces.
Add a new `XASdkDeployTests.SupportDesugaringStaticInterfaceMethods()`
test which performs an on-device, end-to-end invocation of a static
method on a Java interface, to ensure that things *actually* work.
*Note*: if `$(SupportedOSPlatformVersion)` is 24 or higher, this test
will work even without xamarin/java.interop#1077, as Desugaring is
*disabled* in that case.
The test `JniPeerMembersTests.DesugarInterfaceStaticMethod()` added
in xamarin/java.interop@bbaeda6f attempts to "fake" a Desugar
environment for testing on Desktop Java, and this "fakery" doesn't
work in the Android environment. Fix execution on Android by updating
`AndroidRuntime.GetStaticMethodFallbackTypesCore()` to support type
remapping (f6f11a5a) -- which was overlooked/not considered --
such that the types returned are *after* calling
`AndroidRuntime.GetReplacementTypeCore()`, which looks up
`<replace-type/>` values. This allows us to remap
`AndroidInterface` to `DesugarAndroidInterface$_CC`, allowing the
`DesugarInterfaceStaticMethod()` test to pass.
Update the `BuildTestJarFile` target so that it properly builds files
that contain `$` on Unixy platforms.
[0]: https://developer.android.com/studio/write/java8-support#library-desugaring
[1]: https://docs.oracle.com/javase/tutorial/java/IandI/defaultmethods.html
Context: c339d1f298
Before we start building API-34 previews for `Mono.Android.dll`,
update `tests/api-compatibility/reference/Mono.Android.zip` & related
to match the `Mono.Android.dll` currently generated by our build, in
particular including the changes of c339d1f2. This will help make
clearer any breakages for API-34.
Mainly this updates for all of the `[ObsoleteOSPlatform]` fixes we
have made over the past few months.
Additionally, the reference file is now based on a .NET Android
`Mono.Android.dll` instead of a Classic `Mono.Android.dll`, so
additional framework dependent changes will no longer have to be
tracked in "acceptable breakages".
As a result, there would need to be fixups made to the Classic
`api-compat` breakage file, but as we are in the process of removing
Classic from `main`, we simply disable the `api-compat` check when
building Classic `Mono.Android.dll` for now.
Fixes: https://github.com/dotnet/maui/issues/11345
For context users can mark their package as "testOnly" in the `AndroidManifest.xml`. When they do this `adb` also needs an additonal flag to install the package.
PR https://github.com/xamarin/monodroid/pull/1279 adds the ability to pass this additional flag to `adb`.
This PR adds the ability to read the flag from the `AndroidManifest.xml`, so it can be used by other tasks.
A unit test has also been added to the Device tests to make sure we can install `testOnly` packages.
The following test case is duplicated in `MSBuildDeviceIntegration.XASdkDeployTests (...)`:
new object[] {
/* isRelease */ true,
/* xamarinForms */ false,
/* targetFramework*/ "net8.0-android",
},
This was likely a c/p error and the correct `targetFramework` is `net7.0-android`.
Fixes: https://dev.azure.com/DevDiv/DevDiv/_workitems/edit/1398544
It is quite common for users to switch between Debug and Release
configurations in order to test the app. However, if the `Release`
build is using a custom signing keystore you will generally see this
warning and error:
warning MSB6006: "adb" exited with code 1.
[BT : 1.8.1] error : Installation of the app failed.
This is not entirely helpful, since you often need to dig into the
diagnostic log to figure out what the actual issue is.
This warning is produced when we try to run
`adb uninstall -k @PACKAGE_NAME@`:
adb uninstall -k com.xamarin.example
The -k option uninstalls the application while retaining the data/cache.
At the moment, there is no way to remove the remaining data.
You will have to reinstall the application with the same signature, and fully uninstall it.
If you truly wish to continue, execute 'adb shell cmd package uninstall -k'.
We are not entirely sure why the application gets into this state,
but once it does you have to completely uninstall it. We currently
ignore this error but that then results in the following error:
Error: INSTALL_FAILED_UPDATE_INCOMPATIBLE: Existing package com.xamarin.example signatures do not match newer version; ignoring!
This is because the apps used different signing keystores. As a
result they are incompatible. The only option once you get this issue
is to uninstall the app manually and try again. However the error
messaging is not obvious so users generally have no idea what to do.
Fix a few things in this area. Introduce a new `<AndroidAdb/>` task
which is responsible for making the calls to `adb`. We previously used
`<Exec/>`, which made it hard to make any customizations around error
messaging. We will check the result of the `<AndroidAdb/>` task when
calling `adb uninstall -k @PACKAGE_NAME@` and if the output contains
`adb shell cmd package uninstall` then we will automatically call
`adb shell cmd package uninstall @PACKAGE_NAME@`. This will completely
remove the app from the device and will allow the later `bundletool`
invocation to work.
We have also updated `<InstallApkSet/>` to look for error messages from
`bundletool` and report them, so users will get better information.
We now generate error messages like:
[BT : 1.8.1] error : Installation of the app failed.
obj/Release/android/bin/com.xamarin.toggledebugreleasewithsigning.apks : java error BT0000: The APKs have been extracted in the directory: /var/folders/5p/10yqy2kx6r9dnmnxh_nt6s0r0000gn/T/1389125984700138671
obj/Release/android/bin/com.xamarin.toggledebugreleasewithsigning.apks : java error BT0000: 01:54:24 E/SplitApkInstallerBase: Failed to commit install session 1426763565 with command cmd package install-commit 1426763565. Error: INSTALL_FAILED_UPDATE_INCOMPATIBLE: Existing package com.xamarin.toggledebugreleasewithsigning signatures do not match newer version; ignoring!
obj/Release/android/bin/com.xamarin.toggledebugreleasewithsigning.apks : java error BT0000: [BT:1.8.1] Error: Installation of the app failed.
obj/Release/android/bin/com.xamarin.toggledebugreleasewithsigning.apks : java error BT0000: com.android.tools.build.bundletool.model.exceptions.CommandExecutionException
obj/Release/android/bin/com.xamarin.toggledebugreleasewithsigning.apks : java error BT0000: Installation of the app failed.
obj/Release/android/bin/com.xamarin.toggledebugreleasewithsigning.apks : java error BT0000: at com.android.tools.build.bundletool.model.exceptions.InternalExceptionBuilder.build(InternalExceptionBuilder.java:57)
obj/Release/android/bin/com.xamarin.toggledebugreleasewithsigning.apks : java error BT0000: at com.android.tools.build.bundletool.device.DdmlibDevice.installApks(DdmlibDevice.java:192)
obj/Release/android/bin/com.xamarin.toggledebugreleasewithsigning.apks : java error BT0000: at com.android.tools.build.bundletool.commands.InstallApksCommand.lambda$execute$2(InstallApksCommand.java:214)
obj/Release/android/bin/com.xamarin.toggledebugreleasewithsigning.apks : java error BT0000: at com.android.tools.build.bundletool.device.AdbRunner.run(AdbRunner.java:81)
obj/Release/android/bin/com.xamarin.toggledebugreleasewithsigning.apks : java error BT0000: at com.android.tools.build.bundletool.device.AdbRunner.run(AdbRunner.java:43)
obj/Release/android/bin/com.xamarin.toggledebugreleasewithsigning.apks : java error BT0000: at com.android.tools.build.bundletool.commands.InstallApksCommand.execute(InstallApksCommand.java:214)
obj/Release/android/bin/com.xamarin.toggledebugreleasewithsigning.apks : java error BT0000: at com.android.tools.build.bundletool.BundleToolMain.main(BundleToolMain.java:91)
obj/Release/android/bin/com.xamarin.toggledebugreleasewithsigning.apks : java error BT0000: at com.android.tools.build.bundletool.BundleToolMain.main(BundleToolMain.java:49)
obj/Release/android/bin/com.xamarin.toggledebugreleasewithsigning.apks : java error BT0000: Caused by: com.android.ddmlib.InstallException: Failed to commit install session 1426763565 with command cmd package install-commit 1426763565. Error: INSTALL_FAILED_UPDATE_INCOMPATIBLE: Existing package com.xamarin.toggledebugreleasewithsigning signatures do not match newer version; ignoring!
obj/Release/android/bin/com.xamarin.toggledebugreleasewithsigning.apks : java error BT0000: at com.android.ddmlib.SplitApkInstallerBase.installCommit(SplitApkInstallerBase.java:99)
obj/Release/android/bin/com.xamarin.toggledebugreleasewithsigning.apks : java error BT0000: at com.android.ddmlib.SplitApkInstaller.install(SplitApkInstaller.java:85)
obj/Release/android/bin/com.xamarin.toggledebugreleasewithsigning.apks : java error BT0000: at com.android.ddmlib.internal.DeviceImpl.installPackages(DeviceImpl.java:1166)
obj/Release/android/bin/com.xamarin.toggledebugreleasewithsigning.apks : java error BT0000: at com.android.tools.build.bundletool.device.DdmlibDevice.installApks(DdmlibDevice.java:176)
obj/Release/android/bin/com.xamarin.toggledebugreleasewithsigning.apks : java error BT0000: ... 6 more
Unit Test are added.
Fixes: https://github.com/xamarin/xamarin-android/issues/2472
Context: ebb6d5dfdf
Enumification of API-23 in xamarin/monodroid@ebb6d5df incorrectly
enumified [`android.telecom.InCallService.setAudioRoute(int)`][0],
enumifying the `int` parameter as `Android.Telecom.VideoQuality`
instead of `Android.Telecom.CallAudioRoute`.
Fix this by changing `src/Mono.Android/methodmap.csv` to use
`Android.Telecom.CallAudioRoute`, which results in the binding:
namespace Android.Telecom {
partial class InCallServices {
[global::System.Runtime.Versioning.SupportedOSPlatformAttribute ("android23.0")]
[Register ("setAudioRoute", "(I)V", "", ApiSince = 23)]
public unsafe void SetAudioRoute ([global::Android.Runtime.GeneratedEnum] Android.Telecom.CallAudioRoute route) => …
}
}
For backwards compatibility, we need to add an `[Obsolete]` overload
containing the incorrect enumification:
namespace Android.Telecom {
partial class InCallServices {
[Obsolete ("Incorrect enum parameter, use the overload that takes a CallAudioRoute paramter instead.")]
[global::System.Runtime.Versioning.SupportedOSPlatformAttribute ("android23.0")]
public void SetAudioRoute ([global::Android.Runtime.GeneratedEnum] Android.Telecom.VideoQuality route) =>
SetAudioRoute ((CallAudioRoute) route);
}
}
The change to `acceptable-breakages*.txt` is due to the `[Register]`
attribute being moved from the old overload to the corrected overload.
[0]: https://developer.android.com/reference/android/telecom/InCallService#setAudioRoute(int)
Fixes: https://github.com/xamarin/xamarin-android/issues/7650
`AndroidMessageHandler` was incorrectly validating SSL certificate
after a redirect to another domain if and only if
`AndroidMessageHandler.ServerCertificateCustomValidationCallback`
was set; consider:
const string url = "https://httpbin.org/redirect-to?url=https://www.microsoft.com/";
using var handler = new AndroidMessageHandler() {
ServerCertificateCustomValidationCallback = (message, certificate, chain, errors) => errors == SslPolicyErrors.None
};
using var client = new HttpClient(handler);
await client.GetAsync(url);
The expectation is that
`AndroidMessageHandler.ServerCertificateCustomValidationCallback`
will be called *twice*:
1. Once with `message.RequestUri` set to
`https://httpbin.org/redirect-to?…`, and
2. Once again with `message.RequestUri` set to
`https://www.microsoft.com`.
The problem was that in actuality the 2nd invocation was unchanged
from (1), with `message.RequestUri` set to
`https://httpbin.org/redirect-to…?…`, and `errors` set to
`SslPolicyErrors.RemoteCertificateNameMismatch` (!).
The problem was that `AndroidMessageHandler.DoSendAsync()` didn't
update `HttpRequestMessage.RequestUri` to the redirected URI, causing
the `errors` enum value to contain an error, and the second
`ServerCertificateCustomValidationCallback` invocation to have the
wrong information.
Update `AndroidMessageHandler.DoSendAsync()` so that when dealing
with an HTTP redirect, `message.RequestUri` always contains the
"current" value in the redirect chain. This keeps `RequestUri` in
sync with the redirect status to avoid this problem.
Fixes: https://github.com/xamarin/xamarin-android/issues/7558
Context: 16338f9956
Context: a073d99a83
Context: 938c349725
Currently when `api-merge` loops through type members, its first check
is to see if there is any existing member with the same name. If there
isn't, it can short-circuit, add the new member, and move to the next
one. If there is, we compare the existing API level member to the new
API level one to see if it has been marked deprecated in this level,
and if so we can add the `deprecated-since` attribute.
However, this initial existing member check was solely done on member
*name*, not signature. 😱
If the member is a method, a later check finds the exact matching
method with the same signature. We were performing the
`deprecated-since` logic against the first method with a matching name
rather than the method with the same signature.
Consider [`PackageManager.getPackageInfo(String, int)`][0], which
was deprecated in *API-33*. Yet our binding emitted `[Obsolete]`,
meaning it has been deprecated since API-21 or before!
namespace Android.Content.PM;
partial class PackageManager {
[global::System.Obsolete (@"deprecated")]
[Register ("getPackageInfo", "(Ljava/lang/String;I)Landroid/content/pm/PackageInfo;", "GetGetPackageInfo_Ljava_lang_String_IHandler")]
public override unsafe Android.Content.PM.PackageInfo? GetPackageInfo (string packageName, [global::Android.Runtime.GeneratedEnum] Android.Content.PM.PackageInfoFlags flags) => …
}
This was wrong; it should instead have used `[ObsoletedOSPlatform]`:
namespace Android.Content.PM;
partial class PackageManager {
[global::System.Runtime.Versioning.ObsoletedOSPlatform ("android33.0")]
[Register ("getPackageInfo", "(Ljava/lang/String;I)Landroid/content/pm/PackageInfo;", "GetGetPackageInfo_Ljava_lang_String_IHandler")]
public override unsafe Android.Content.PM.PackageInfo? GetPackageInfo (string packageName, [global::Android.Runtime.GeneratedEnum] Android.Content.PM.PackageInfoFlags flags) => …
}
Move the `UpdateDeprecatedSince()` invocations to be called *after*
we have the method with the matching signature.
The `acceptable-breakages` changes reflect additional `[Obsolete]`
attributes that are now being converted to
`[ObsoletedOSPlatform ("android-XX.0")]` attributes.
[0]: https://developer.android.com/reference/android/content/pm/PackageManager#getPackageInfo(java.lang.String,%20int)
Fixes: https://github.com/xamarin/xamarin-android/issues/6310
Context: 60d9b98938
Context: https://github.com/dotnet/fsharp/issues/12640
Context: 103b5a755c Optimize ResourceIdManager.UpdateIdValues() invocations
Context: 9e6ce03ca2 Adds $(AndroidLinkResource)
Context: 522d7fb61f
Context: 9c0437866c (AndroidEnablePreloadAssemblies crash)
Context: d521ac0280 (Styleables array values)
Replace the existing `Resource.designer.cs` generation code with a
new system that relies on Reference Assemblies. This results in
smaller apps and faster startup.
~~ Bind `@(AndroidResource)` values as fields ~~
The original approach to binding `@(AndroidResource)` values was to
Do What Java Does™: there are two "styles" of `Resource.designer.cs`
files, one for Library projects, and one for App projects.
`Resource.designer.cs` for Library projects involves mutable read/write
fields:
[assembly: Android.Runtime.ResourceDesignerAttribute ("ExampleLib.Resource", IsApplication=false)]
namespace ExampleLib;
partial class Resource {
partial class String {
public static int app_name = 2130771968;
static String() {
global::Android.Runtime.ResourceIdManager.UpdateIdValues();
}
}
partial class Styleable {
public static int[] MyLibraryWidget = new int[]{…};
static Styleable() {
global::Android.Runtime.ResourceIdManager.UpdateIdValues();
}
}
}
`Resource.designer.cs` for App projects involves *`const`* fields:
[assembly: Android.Runtime.ResourceDesignerAttribute ("App.Resource", IsApplication=true)]
namespace App;
partial class Resource {
partial class String {
public const int app_name = 2130968576;
static String() {
global::Android.Runtime.ResourceIdManager.UpdateIdValues();
}
}
partial class Styleable {
public static int[] MyLibraryWidget = new int[]{…}; // still read+write, not const
static Styleable() {
global::Android.Runtime.ResourceIdManager.UpdateIdValues();
}
}
}
There is a field each Android `resource` in the project *and* any
`resource`s declared in a referenced assembly or `.aar` files.
This can result in 1000's of fields ending up in each `Resource` class.
Because we only know the final `Id` values at app packaging time,
library projects could not know those values at build time. This meant
that we needed to update those library values at startup with the ones
that were compiled into the final application project. This is handled
by the `Resource.UpdateIdValues()` method. This method is called by
reflection on app startup and contains code to set the read/write
fields for *all* `Resource` types from *all referenced assemblies*:
partial class Resource {
public static void UpdateIdValues() {
global::ExampleLib.Resource.String.app_name = String.app_name;
// plus all other resources
}
}
**Pros**:
* It's a "known good" construct, as it's what Java does!
(Or *did*, circa 12 years ago…)
**Cons**:
* There is a semantic difference between the use of the `Resource`
types between Library and App projects: in an App project, you
can use Resource IDs in switch `case`s, e.g.
`case Resource.String.app_name: …`.
This is not possible in Library projects.
* As the App `Resource.UpdateIdValues()` method references *all*
fields from all referenced libraries, the linker is not able to
remove any of the fields. This pattern is linker hostile.
This results in larger `.apk` sizes, though this can be optimized
via [`$(AndroidLinkResources)`][0] (9e6ce03c, d521ac02).
* As the App `Resource.UpdateIdValues()` method references *all*
fields from all referenced libraries, the method can be *huge*;
it depends on how many resources the App and all dependencies
pull in. We have seen cases where the size of
`Resource.UpdateIdValues()` would cause the interpreter to crash,
breaking certain Hot Reload scenarios.
(Fixed in dotnet/runtime@60d9b989).
* The `Resource.UpdateIdValues()` method needs to be invoked during
process startup, *before* any assemblies try to use their
`Resource.…` values, and the method is looked up via *Reflection*.
This means System.Reflection is part of the app startup path,
which has overheads.
(This overhead is also removed via `$(AndroidLinkResources)`.)
~~ Bind `@(AndroidRoesource)` values as properties ~~
Replace the "bind resources as fields" approach with a new system
with significant differences:
1. Android resource ids are bound as read-only *properties*, and
2. The `Resource` class is placed into a *separate assembly*,
`_Microsoft.Android.Resource.Designer.dll`.
The new `$(AndroidUseDesignerAssembly)` MSBuild property controls
which Android resource approach is used; if True -- the default for
.NET 8 -- then `_Microsoft.Android.Resource.Designer.dll` will be
used. If False, then the previous "bind resource ids as fields"
approach will be used. This property is only valid for Library
projects; App projects must use the property-oriented approach.
This new approach takes advantage of [Reference Assemblies][1].
Reference Assemblies are designed to be replaced at runtime, and are
generally used to provide placeholder API's which can be swapped out
later.
Library projects will generate a Reference Assembly for
`_Microsoft.Android.Resource.Designer.dll` which contains read-only
properties for each `@(AndroidResource)` within the project and all
dependencies. This is otherwise identical to the "fields" approach,
*except* that the namespace is predefined, its a new assembly, and
properties are used instead of fields, *as if* it contained:
// _Microsoft.Android.Resource.Designer.dll for Library project
[assembly: System.Runtime.CompilerServices.ReferenceAssemblyAttribute]
namespace Microsoft.Android.Resource.Designer;
public partial class Resource {
public partial class String {
public static int app_name => 0;
}
public partial class Styleable {
public static int[] MyLibraryWidget => nullptr;
}
}
Also note that `_Microsoft.Android.Resource.Designer.dll` is produced
*with Mono.Cecil* as a pre-build action; no C# source is generated.
The Library assembly references the generated
`_Microsoft.Android.Resource.Designer.dll`.
The generated `_Microsoft.Android.Resource.Designer.dll` should
***NOT*** be shipped with NuGet packages.
App projects will generate the "real"
`_Microsoft.Android.Resource.Designer.dll`, also as a pre-build step,
and the "real" assembly will contain actual values for resource ids.
The App-built `_Microsoft.Android.Resource.Designer.dll` will also
have `[assembly:InternalsVisibleToAttribute]` to the App assembly:
// _Microsoft.Android.Resource.Designer.dll for App project
[assembly: System.Runtime.CompilerServices.InternalsVisibleToAttribute ("App…")]
namespace Microsoft.Android.Resource.Designer;
public partial class Resource {
public partial class String {
public static int app_name => 2130968576;
}
public partial class Styleable {
static int[] MyLibraryWidget = new[]{…};
public static int[] MyLibraryWidget => MyLibraryWidget;
}
}
This approach has a number of benefits
1. All the property declarations are in one place and are not
duplicated (-ish… more on that later).
As a result the size of the app will be reduced.
2. Because we no longer need the `Resource.UpdateIdValues()` method,
start up time will be reduced.
3. The linker can now do its job and properly link out unused
properties. This further reduces application size.
4. F# is now fully supported. See also: dotnet/fsharp#12640.
~~ Styleable Arrays ~~
Styleable resources may be arrays; see e.g. d521ac02. Via the power
of Cecil (and not using C# as an intermediate codegen), the binding
of styleable arrays in the "Bind `@(AndroidRoesource)` values as
properties" world order involves a static field containing the array
data, and a public property which returns the private field, which
has the same name:
public partial class Resource {
public partial class Styleable {
static int[] MyLibraryWidget = new[]{…};
public static int[] MyLibraryWidget => MyLibraryWidget;
}
}
CIL-wise, *yes*, the field and the property have the same name (?!),
but because properties actually have `get_` method prefix, there will
actually be a `MyLibraryWidget` field and a `get_MyLibraryWidget()`
method, so there are no name collisions.
*Note*: ***The styleable array is not copied***. This means it is
global mutable data, i.e. one can do this:
Microsoft.Android.Resource.Designer.Resource.Styleable.MyLibraryWidget[0] = 42;
***DO NOT DO THIS***. It will introduce runtime errors.
The e.g. `Resource.Styleable.MyLibraryWidget` property must be an
`int[]` in order to maintain compatibility, as these are often
passed to methods which take `int[]` as the parameter type. We thus
cannot instead use e.g. `IEnumeragble<int>` as the property type.
Additionally, the array isn't copied for performance reasons.
We do not think that this will be a problem in practice, as the
previous "Bind `@(AndroidRoesource)` values as fields" strategy
*also* had mutable `int[]` fields, and suffers from the same
safety concerns, and the world hasn't ended…
~~ Source Compatibility ~~
In the "bind resource ids as fields" approach, the `Resource` class
was in the default namespace for the Library project, set via the
[`$(RootNamespace)`][2] MSBuild property. In order to maintain
source compatibility, Library projects will have a generated
`__Microsoft.Android.Resource.Designer.cs` file which contains a new
`Resource` declaration which *inherits* from the `Resource` type in
`_Microsoft.Android.Resource.Designer.dll`:
// Generated __Microsoft.Android.Resource.Designer.cs in Library projects
namespace ExampleLib;
public class Resource : Microsoft.Android.Resource.Designer.Resource {
}
This allows existing code such as `ExampleLib.Resource.String.app_name`
to continue to compile.
App projects also expect a `Resource` class in `$(RootNamespace)`,
*and* expect the values to be `const`. To support this, the generated
`_Microsoft.Android.Resource.Designer.dll` *actually* has two sets
of `Resource` types, one with properties, and an *`internal`*
`ResourceConstant` type:
// _Microsoft.Android.Resource.Designer.dll for Library project
[assembly: System.Runtime.CompilerServices.InternalsVisibleToAttribute ("App…")]
namespace Microsoft.Android.Resource.Designer;
internal partial class ResourceConstant {
public partial class String {
public const int app_name = 2130968576;
}
}
public partial class Resource {
public partial class String {
public static int app_name => ResourceConstant.String.app_name;
}
}
App projects *also* have a generated
`__Microsoft.Android.Resource.Designer.cs`, which has a `Resource` type
which inherits from `ResourceConstant`. This is why the App-built
`_Microsoft.Android.Resource.Designer.dll` needs
`[assembly: InternalsVisibleToAttribute]`:
// Generated __Microsoft.Android.Resource.Designer.cs in App projects
namespace App;
public class Resource : Microsoft.Android.Resource.Designer.ResourceConstant {
}
This allows existing App code to use `App.Resource.String.app_name`
in `case` statements.
~~ Binary Compatibility ~~
Binary compatibility is maintained via a new
`MonoDroid.Tuner.FixLegacyResourceDesignerStep` linker step.
`FixLegacyResourceDesignerStep` rewrites Library assemblies to replace
`Resource.…` field access with property access to
`Microsoft.Android.Resource.Designer.Resource.…` in
`_Microsoft.Android.Resource.Designer.dll`. Much of this code
overlaps with the existing logic of `$(AndroidLinkResources)`, and
allows existing Library assemblies to participate in the property-
oriented system.
~~ Internals ~~
The new build system introduces a number of new Tasks and Targets to
bring this all together. It also unify's some code between the field-
oriented and property-oriented approaches which would otherwise be
duplicated. The field-oriented system will be maintained for now for
backward compatibility, however the property-oriented system will be
enabled by default for .net 8.
The property-oriented system is mostly contained in
`Xamarin.Android.Resource.Designer.targets`. The entry point for this
set of targets is `_BuildResourceDesigner`, which will only be run if
the `$(AndroidUseDesignerAssembly)` MSBuild property is `True`, as it
will be for .NET 8+.
New tasks are as follows.
- `<GenerateRtxt/>` is responsible for scanning the resource
directory and generating an `aapt2`-compatible `R.txt` file.
This will be used by `<GenerateResourceDesignerAssembly/>`.
- `<GenerateResourceCaseMap/>` is responsible for generating a
`casemap.txt` file which will map the all lower case android
resources to the casing required for the C# code. Android requires
ALL resources be lower case, but our system allows the user to
define the case using any system then want. This task handles
generating this mapping between what the android system needs and
what the user is expecting. Its output is used by the
`<GenerateResourceDesignerAssembly/>` task when generating the IL
in `_Microsoft.Android.Resource.Designer.dll`.
It is also used by the old system to generate the same file.
- `<GenerateResourceDesignerIntermediateClass/>` is responsible for
generating the `__Microsoft.Android.Resource.Designer.cs` file in
`$(IntermediateOutputPath)`.
- `<GenerateResourceDesignerAssembly/>` is the key to the whole
property-oriented approach. This task will read the `R.xt` file
and generate a `_Microsoft.Android.Resource.Designer.dll` assembly
in `$(IntermediateOutputPath)`. This task is called in two places.
The first is in `_GenerateResourceDesignerAssembly`, this is called
as part of the build which happens just before `CoreCompile` and
only for design time builds.
It is also called in `_UpdateAndroidResgen` which happens as part
of the build and runs just after `aapt2` is called. This ensures
we always use the most up to date version of `R.txt` to generate
the new assembly.
Because we are using the `R.txt` file to drive the generation of the
new assembly, we needed some way for that to work when `aapt2` was
not being run. This usually happens on a first time design time build.
The field-oriented approach has a `<GenerateResourceDesigner/>` task
which is responsible for both scanning the resources and generating a
design time `Resource.designer.cs` file. While we could have
duplicated the code it made more sense to split out the resource
scanner into its own class. We now have a new `<GenerateRtxt/>` task
which is responsible for scanning the resources and generating an
`R.txt` file. This is only used when we are not doing a full build
with `aapt2`. This new task lets us generate the needed `R.txt` which
can then be used by both the old and new system to generate their
respective outputs.
As part of this we have two other classes: `RtxtReader` and
`RtxtWriter`. The `RtxtReader` unify's the code which was used to read
the values of the `R.txt` into one class which can be used by both
approaches. The `RtxtWriter` is responsible for writing the `R.txt`
file for design time builds. Again it will be used by both the old
and new system.
The `_AddResourceDesignerFiles` target is responsible for ensuring that
the new assembly and `__Microsoft.Android.Resource.Designer.cs` get
added to the correct item groups. These are `@(ReferencePath)` for the
assembly and `@(Compile)` for the source file. In the case of F# the
`__Microsoft.Android.Resource.Designer.fs` file which gets generated
has to be added to the `@(CompileBefore)` ItemGroup, this is so that
the types are resolved in the correct order.
To ensure that the new assembly is added to the final application we
have to introduce the `_AddResourceDesignerToPublishFiles` target.
This target makes sure that the new assembly is added to the
`@(ResolvedFileToPublish)` ItemGroup. It also adds the require
MetaData items such as `%(IsTrimmable)` and `%(PostprocessAssembly)`
which are required to get the assembly linked correctly.
~~ Results ~~
Results are most visible when lots of Android Resources are used.
For a [Sample app][3] app which uses lots of resources, we see the
following improvements to the **ActivityTaskManager: Displayed** time:
| Before (ms) | After (ms) | Δ (%) | Notes |
| ----------: | ----------: | --------: | ------------------------------------ |
| 340.500 | 313.250 | -8.00% ✓ | defaults; 64-bit build |
| 341.950 | 316.200 | -7.53% ✓ | defaults; profiled AOT; 64-bit build |
| 345.950 | 324.600 | -6.17% ✓ | defaults; 32-bit build |
| 341.000 | 323.050 | -5.26% ✓ | defaults; profiled AOT; 32-bit build |
[0]: https://learn.microsoft.com/en-us/xamarin/android/deploy-test/building-apps/build-properties#androidlinkresources
[1]: https://learn.microsoft.com/en-us/dotnet/standard/assembly/reference-assemblies
[2]: https://learn.microsoft.com/en-us/visualstudio/msbuild/common-msbuild-project-properties?view=vs-2022
[3]: https://github.com/dellis1972/DotNetAndroidTest
Changes: 167a4ed...47a747f
Changes: 27ce032...2db841f
Updates:
* Microsoft.Dotnet.Sdk.Internal: from 8.0.100-alpha.1.22611.1 to 8.0.100-alpha.1.22616.7
* Microsoft.NET.ILLink.Tasks: from 8.0.100-1.22609.1 to 8.0.100-1.22612.2
* Update to MSBuild.StructuredLogger 2.1.758
Context: https://github.com/KirillOsenkov/MSBuildStructuredLog/releases/tag/v2.1.758
Context: https://github.com/KirillOsenkov/MSBuildStructuredLog/pull/644
A few tests were failing with:
Failed Install_CSharp_Change
Error Message:
System.NotSupportedException : Unsupported log file format. Latest supported version is 14, the log file has version 15.
Co-authored-by: Jonathan Peppers <jonathan.peppers@microsoft.com>
Context: 4da27921d7
Context: d3ea180cd4
Context: 15c8879761
Changes: 149d70fea4...f8d77faf55
* xamarin/java.interop@f8d77faf: [generator] Better support deprecated property getter/setters. (xamarin/java.interop#1062)
* xamarin/java.interop@5e6209ea: [generator] Obsolete&SupportedOSPlatform attributes on enum members (xamarin/java.interop#1066)
* xamarin/java.interop@15c88797: [generator] Use decl type's @deprecated-since if < member's (xamarin/java.interop#1068)
* xamarin/java.interop@525a45d5: [Java.Interop.Dynamic-Tests] Use Microsoft.CSharp NuGet package (xamarin/java.interop#1067)
Background: member deprecations can be "historically weird" in Android.
For example, in API-21 [`android/app/ActionBar.TabListener`][0] was
deprecated:
@Deprecated
/* partial */ interface TabListener {
void onTabReselected(ActionBar.Tab tab, FragmentTransaction ft);
// …
}
The type being deprecated means that its members are *implicitly*
deprecated.
In API-29 the members were *explicitly* deprecated:
@Deprecated
/* partial */ interface TabListener {
@Deprecated
void onTabReselected(ActionBar.Tab tab, FragmentTransaction ft);
// …
}
Before xamarin/Java.Interop@d3ea180c, this resulted in the binding:
[Obsolete]
partial interface ITabListener {
[Obsolete]
void OnTabReselected(ActionBar.Tab? tab, FragmentTransaction? ft);
// …
}
Commit xamarin/Java.Interop@d3ea180c added support for
`[ObsoletedOSPlatform]` to `generator`, which *changed* the binding to:
[Obsolete] // because it was deprecated in our min-supported API-21
partial interface ITabListener {
[ObsoletedOSPlatform ("android29.0")]
void OnTabReselected(ActionBar.Tab? tab, FragmentTransaction? ft);
// …
}
This resulted in *lots* of changes in commit 4da27921 to
`tests/api-compatibility/acceptable-breakages-vReference-net7.0.txt`
because `[Obsolete]` was were replaced by `[ObsoletedOSPlatform]`:
CannotRemoveAttribute : Attribute 'System.ObsoleteAttribute' exists on 'Android.App.ActionBar.ITabListener.OnTabReselected(Android.App.ActionBar.Tab, Android.App.FragmentTransaction)' in the contract but not the implementation
Commit xamarin/Java.Interop@15c88797 updates type members to have the
same deprecation status as their declaring type, when the member was
deprecated *after* the type was deprecated. This means we *now*
bind `ITabListener` as:
[Obsolete] // because it was deprecated in our min-supported API-21
partial interface ITabListener {
[Obsolete]
void OnTabReselected(ActionBar.Tab? tab, FragmentTransaction? ft);
// …
}
which matches the state of things pre- xamarin/Java.Interop@d3ea180c,
which means we no longer need to ignore all those
`CannotRemoveAttribute` messages. xamarin/Java.Interop@15c88797 thus
"unintentionally partially reverts" 4da27921 (yay?), simply because
the prior state of affairs in which a member was deprecated *after*
the declaring type, while valid, didn't make any sense.
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Jonathan Pobst <jonathan.pobst@microsoft.com>
[0]: https://developer.android.com/reference/android/app/ActionBar.TabListener
Fixes: https://github.com/xamarin/xamarin-android/issues/7532
Whenever any MonoVM APIs are called, the current thread of execution
must be attached to the runtime or we risk native code crashes, e.g.:
Fatal signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x40 in tid 1860 (jg_fr_pool_thre), pid 1792 (yname.sampleapp)
backtrace:
00 pc 0011588e /data/app/~~lwgdvtQhIfK0II0Nd0mtag==/com.companyname.sampleapp-8yF54k9YuWCWpsuGRJvVQw==/lib/x86/libmonosgen-2.0.so
01 pc 001156ae /data/app/~~lwgdvtQhIfK0II0Nd0mtag==/com.companyname.sampleapp-8yF54k9YuWCWpsuGRJvVQw==/lib/x86/libmonosgen-2.0.so
02 pc 00115575 /data/app/~~lwgdvtQhIfK0II0Nd0mtag==/com.companyname.sampleapp-8yF54k9YuWCWpsuGRJvVQw==/lib/x86/libmonosgen-2.0.so (mono_threads_enter_gc_unsafe_region_internal+53)
03 pc 000973c3 /data/app/~~lwgdvtQhIfK0II0Nd0mtag==/com.companyname.sampleapp-8yF54k9YuWCWpsuGRJvVQw==/lib/x86/libmonosgen-2.0.so (mono_runtime_invoke+51) (BuildId: c54662bbf82dbdadd595ca9d3e31dce29735885f)
04 pc 00027ace /data/app/~~lwgdvtQhIfK0II0Nd0mtag==/com.companyname.sampleapp-8yF54k9YuWCWpsuGRJvVQw==/lib/x86/libmonodroid.so (xamarin::android::internal::MonodroidRuntime::Java_mono_android_Runtime_register(_JNIEnv*, _jstring*, _jclass*, _jstring*)+286)
05 pc 000279a0 /data/app/~~lwgdvtQhIfK0II0Nd0mtag==/com.companyname.sampleapp-8yF54k9YuWCWpsuGRJvVQw==/lib/x86/libmonodroid.so (Java_mono_android_Runtime_register+48)
The above trace translates to the following locations in the Mono runtime:
copy_stack_data_internal
/__w/1/s/src/mono/mono/utils/mono-threads-coop.c:191
copy_stack_data
/__w/1/s/src/mono/mono/utils/mono-threads-coop.c:246
mono_threads_enter_gc_unsafe_region_unbalanced_with_info
/__w/1/s/src/mono/mono/utils/mono-threads-coop.c:476
mono_runtime_invoke
/__w/1/s/src/mono/mono/metadata/object.c:2442
In this case, a pointer to Mono thread information structure
(`MonoThreadInfo*`) is null in `copy_stack_data()` which, in turn,
causes the segfault when the pointer is dereferenced.
In the case of issue #7532, `Java_mono_android_Runtime_register()` is
called on behalf of a 3rd party library on a thread that is, most
likely, created by that library and thus not (yet) attached to the
runtime by the time the registration attempt is made.
Attach thread to the runtime by calling
`mono_jit_thread_attach(nullptr)` in .NET builds to fix the issue.
`nullptr` is used because .NET only has a single AppDomain and
there's no need to pass around pointers to it.
TODO: Try to create a test case. Our attempted unit tests don't
cause an app crash when the fix isn't applied -- i.e. when
`mono_jit_thread_attach(nullptr)` *isn't* called -- which suggests
that we don't fully understand the reported bug report.
Add some unit tests for localization. These need to run on an
emulator like our timezone tests.
Run these tests on a nightly basis as they take a long time to run.
Fixes: https://github.com/xamarin/xamarin-android/issues/7490
Add API-33 to the nightly build tests. Note that the `default` image
is not available so we have to use the `google_apis` image instead.
Also it seems that under API-33 dotnet is unable to get the hardware
address of the network adapters. As a result the tests which compare
these were failing.
Ignore the hardware address check on API-33, while continuing to
check other aspects of the network adapters.
API-34+ will *not* ignore the hardware address check. This will be
reconsidered when API-34 actually exists.
(It's just NuGet package bumps! How hard could it be? 🙃)
Changes: 59cac909fa...fa3711b7dd
* amarin/xamarin-android-tools@fa3711b: [build] Update NuGet package versions (amarin/xamarin-android-tools#196)
What *are* the NuGet package version bumps in
xamarin/xamarin-android-tools@fa3711b?
> NuGet Package Version Bumps:
>
> * Microsoft.Build : `16.10.0` -> `17.3.2`
> * Microsoft.Build.Framework : `16.10.0` -> `17.3.2`
> * Microsoft.Build.Tasks.Core : `16.10.0` -> `17.3.2`
> * Microsoft.Build.Utilities.Core : `16.10.0` -> `17.3.2`
> * Microsoft.NET.Test.Sdk : `16.5.0` -> `17.5.0-preview-20221003-04`
> * nunit : `3.12.0` -> `3.13.2`
> * NUnit3TestAdapter : `3.16.1` -> `4.0.0`
The `nunit` and `NUnit3TestAdapter` versions differed from what
`Xamarin.Android.Tools.Aidl-Tests.csproj` was using, which resulted
in some tests failing with:
NUnit.Engine.NUnitEngineException : An exception occurred in the driver while loading tests.
----> System.IO.FileLoadException : Could not load file or assembly 'nunit.framework, Version=3.13.2.0, Culture=neutral, PublicKeyToken=2638cd05610744eb' or one of its dependencies.
The located assembly's manifest definition does not match the assembly reference. (Exception from HRESULT: 0x80131040)
--NUnitEngineException
An exception occurred in the driver while loading tests.
Fix by harmonizing `nunit` and `NUnit3TestAdapter` package versions.
Then came a fair bit of time trying to get the
`SixColors.ImageSharp` NuGet package working with the updated
packages, in particular the new `Microsoft.Build*` packages:
System.TypeInitializationException : The type initializer for 'Xamarin.ProjectTools.XamarinAndroidCommonProject' threw an exception.
----> System.TypeInitializationException : The type initializer for 'SixLabors.ImageSharp.Configuration' threw an exception.
----> System.IO.FileLoadException : Could not load file or assembly 'System.Memory, Version=4.0.1.1, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51' or one of its dependencies. The located assembly's manifest definition does not match the assembly reference. (Exception from HRESULT: 0x80131040)
at Xamarin.ProjectTools.XamarinAndroidCommonProject..ctor(String debugConfigurationName, String releaseConfigurationName)
at Xamarin.ProjectTools.XamarinAndroidApplicationProject..ctor(String debugConfigurationName, String releaseConfigurationName, String packageName) in C:\a\_work\1\s\src\Xamarin.Android.Build.Tasks\Tests\Xamarin.ProjectTools\Android\XamarinAndroidApplicationProject.cs:line 41
at Xamarin.Android.Build.Tests.PackagingTest.CheckSignApk(Boolean useApkSigner, Boolean perAbiApk) in C:\a\_work\1\s\src\Xamarin.Android.Build.Tasks\Tests\Xamarin.Android.Build.Tests\PackagingTest.cs:line 381
--TypeInitializationException
at SixLabors.ImageSharp.Image.Load(Stream stream, IImageFormat& format)
at Xamarin.ProjectTools.XamarinAndroidCommonProject..cctor() in C:\a\_work\1\s\src\Xamarin.Android.Build.Tasks\Tests\Xamarin.ProjectTools\Android\XamarinAndroidCommonProject.cs:line 40
--FileLoadException
at SixLabors.ImageSharp.Configuration..ctor(IConfigurationModule[] configurationModules)
at SixLabors.ImageSharp.Configuration.CreateDefaultInstance()
at System.Lazy`1.CreateValue()
at System.Lazy`1.LazyInitValue()
at SixLabors.ImageSharp.Configuration..cctor()
I could not figure out a way to make `SixColors.ImageSharp` work with
the other newer NuGet package versions.
*Remove* use of `SixColors.ImageSharp`, and instead have
`Xamarin.ProjectTools.dll` reference the various `.png` resources
contained in the `dotnet new android` template, in
`src\Microsoft.Android.Templates\android\Resources\**\appicon.png`.
In order to ensure sane `%(LogicalName)` values, `%(RecursiveDir)`
is used as part of the logical name, resulting in resource names
on macOS such as:
* `mipmap-hdpi/appicon.png`
* `mipmap-mdpi/appicon.png`
* `mipmap-xhdpi/appicon.png`
* `mipmap-xxhdpi/appicon.png`
* `mipmap-xxxhdpi/appicon.png`
Xamarin.ProjectTools.dll *also* looks for a resource name with
`/` replaced by `\` (`Path.DirectorySeparatorChar`) so that this
logic works on Windows as well.
Using the same icons as the `dotnet new android` template sounded
like a good idea, except those resources caused XA0102 warnings when
validating with lint (?!):
dotnet new android -n lint-check
cd lint-check
dotnet build -p:AndroidLintEnabled=True \
"-p:AndroidLintDisabledIssues=StaticFieldLeak%2cObsoleteSdkInt%2cAllowBackup%2cExportedReceiver%2cRedundantLabel"
results in:
warning XA0102: Launcher icons should not fill every pixel of their square region; see the design guide for details [IconLauncherShape]
warning XA0102: Launcher icons should not fill every pixel of their square region; see the design guide for details [IconLauncherShape]
warning XA0102: Launcher icons should not fill every pixel of their square region; see the design guide for details [IconLauncherShape]
warning XA0102: Launcher icons should not fill every pixel of their square region; see the design guide for details [IconLauncherShape]
(Note use of hex-encoded `,` as `%2c` so that
`$(AndroidLintDisabledIssues)` can be specified on the command-line.)
Update the `appicon.png` resources in
`src/Microsoft.Android.Templates/android` so that these lint warnings
are no longer generated. This in turn requires updating various
`.apkdesc` files, as the new `appicon.png` files are smaller than the
images produced by `SixColors.ImageSharp`, resulting in smaller
`.apk` files in our unit tests.
Then we hit a set of assembly resolution errors around
`System.Runtime.CompilerServices.Unsafe`:
System.IO.FileLoadException : Could not load file or assembly 'System.Runtime.CompilerServices.Unsafe, Version=5.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The located assembly's manifest definition does not match the assembly reference. (Exception from HRESULT: 0x80131040)
----> System.IO.FileLoadException : Could not load file or assembly 'System.Runtime.CompilerServices.Unsafe, Version=4.0.4.1, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The located assembly's manifest definition does not match the assembly reference. (Exception from HRESULT: 0x80131040)
at System.MemoryExtensions.AsSpan(String text)
at Microsoft.Build.Evaluation.Expander`2.Function`1.ExtractPropertyFunction(String expressionFunction, IElementLocation elementLocation, Object propertyValue, UsedUninitializedProperties usedUnInitializedProperties, IFileSystem fileSystem)
at Microsoft.Build.Evaluation.Expander`2.PropertyExpander`1.ExpandPropertyBody(String propertyBody, Object propertyValue, IPropertyProvider`1 properties, ExpanderOptions options, IElementLocation elementLocation, UsedUninitializedProperties usedUninitializedProperties, IFileSystem fileSystem)
at Microsoft.Build.Evaluation.Expander`2.PropertyExpander`1.ExpandPropertiesLeaveTypedAndEscaped(String expression, IPropertyProvider`1 properties, ExpanderOptions options, IElementLocation elementLocation, UsedUninitializedProperties usedUninitializedProperties, IFileSystem fileSystem)
at Microsoft.Build.Evaluation.Expander`2.PropertyExpander`1.ExpandPropertiesLeaveEscaped(String expression, IPropertyProvider`1 properties, ExpanderOptions options, IElementLocation elementLocation, UsedUninitializedProperties usedUninitializedProperties, IFileSystem fileSystem)
at Microsoft.Build.Evaluation.Expander`2.ExpandIntoStringLeaveEscaped(String expression, ExpanderOptions options, IElementLocation elementLocation)
at Microsoft.Build.Evaluation.ToolsetReader.ExpandPropertyUnescaped(ToolsetPropertyDefinition property, Expander`2 expander)
at Microsoft.Build.Evaluation.ToolsetReader.EvaluateAndSetProperty(ToolsetPropertyDefinition property, PropertyDictionary`1 properties, PropertyDictionary`1 globalProperties, PropertyDictionary`1 initialProperties, Boolean accumulateProperties, String& toolsPath, String& binPath, Expander`2& expander)
at Microsoft.Build.Evaluation.ToolsetReader.ReadToolset(ToolsetPropertyDefinition toolsVersion, PropertyDictionary`1 globalProperties, PropertyDictionary`1 initialProperties, Boolean accumulateProperties)
at Microsoft.Build.Evaluation.ToolsetReader.ReadEachToolset(Dictionary`2 toolsets, PropertyDictionary`1 globalProperties, PropertyDictionary`1 initialProperties, Boolean accumulateProperties)
at Microsoft.Build.Evaluation.ToolsetReader.ReadToolsets(Dictionary`2 toolsets, PropertyDictionary`1 globalProperties, PropertyDictionary`1 initialProperties, Boolean accumulateProperties, String& msBuildOverrideTasksPath, String& defaultOverrideToolsVersion)
at Microsoft.Build.Evaluation.ToolsetReader.<ReadAllToolsets>g__ReadConfigToolset|12_0(<>c__DisplayClass12_0& )
at Microsoft.Build.Evaluation.ToolsetReader.ReadAllToolsets(Dictionary`2 toolsets, ToolsetRegistryReader registryReader, ToolsetConfigurationReader configurationReader, PropertyDictionary`1 environmentProperties, PropertyDictionary`1 globalProperties, ToolsetDefinitionLocations locations)
at Microsoft.Build.Evaluation.ProjectCollection.InitializeToolsetCollection(ToolsetRegistryReader registryReader, ToolsetConfigurationReader configReader)
at Microsoft.Build.Evaluation.ProjectCollection..ctor(IDictionary`2 globalProperties, IEnumerable`1 loggers, IEnumerable`1 remoteLoggers, ToolsetDefinitionLocations toolsetDefinitionLocations, Int32 maxNodeCount, Boolean onlyLogCriticalEvents, Boolean loadProjectsReadOnly, Boolean useAsynchronousLogging)
at Microsoft.Build.Evaluation.ProjectCollection.get_GlobalProjectCollection()
at Xamarin.ProjectTools.DotNetXamarinProject.Construct() in C:\a\_work\1\s\src\Xamarin.Android.Build.Tasks\Tests\Xamarin.ProjectTools\Common\DotNetXamarinProject.cs:line 101
at Xamarin.ProjectTools.XamarinAndroidCommonProject.Construct() in C:\a\_work\1\s\src\Xamarin.Android.Build.Tasks\Tests\Xamarin.ProjectTools\Android\XamarinAndroidCommonProject.cs:line 53
at Xamarin.ProjectTools.DotNetXamarinProject.SaveProject() in C:\a\_work\1\s\src\Xamarin.Android.Build.Tasks\Tests\Xamarin.ProjectTools\Common\DotNetXamarinProject.cs:line 141
at Xamarin.ProjectTools.XamarinProject.Save(Boolean saveProject) in C:\a\_work\
Resolve these errors by updating the various `App.config` files so
that they use `<bindingRedirect/>` versions which match
`MSBuild.exe.config` in Visual Studio, and ensure that the referenced
NuGet packages use the same versions as the `<bindingRedirect/>`.
NuGet Package Version Bumps:
* Microsoft.NET.Test.Sdk : `16.*` -> `17.5.0-preview-20221003-04`
* Microsoft.Win32.Registry : `4.7.0` -> `5.0.0`
* nunit : `3.12.0` -> `3.13.2`
* NUnit.ConsoleRunner : `3.11.1` -> `3.12.0`
* NUnit3TestAdapter : `3.16.1` -> `4.0.0`
* System.CodeDom : `4.7.0` -> `6.0.0`
* System.Configuration.ConfigurationManager : `4.7.0` -> `6.0.1`
* System.Reflection.Metdata : `1.8.0` -> `6.0.1`
Co-authored-by: Jonathan Peppers <jonathan.peppers@microsoft.com>
Context: 234cf2c022
Commit 234cf2c0 updated the
`AndroidHandlerTestBase.Redirect_Without_Protocol_Works()` and
`AndroidHandlerTestBase.Redirect_POST_With_Content_Works()` tests to
use <httpbingo.org> to test HTTP redirect behavior.
Recently, <httpbingo.org> started returning HTTP-403 for our
redirection test, whereas we expected it to send HTTP-302:
% curl -D - 'https://httpbingo.org/redirect-to?url=https://github.com/xamarin/xamarin-android'
HTTP/2 403
access-control-allow-credentials: true
access-control-allow-origin: *
content-type: text/plain; charset=utf-8
x-content-type-options: nosniff
date: Wed, 16 Nov 2022 15:38:02 GMT
content-length: 51
server: Fly/5f2bf728 (2022-11-11)
via: 2 fly.io
fly-request-id: 01GJ0HE398V7F0BKFHY0D2N8YE-iad
Forbidden redirect URL. Be careful with this link.
This caused the `Redirect_Without_Protocol_Works()` and
`Redirect_POST_With_Content_Works()` unit tests to start failing:
System.Net.Http.HttpRequestException : Response status code does not indicate success: 403 (Forbidden).
at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode ()
at Xamarin.Android.NetTests.AndroidHandlerTestBase.Redirect_POST_With_Content_Works ()
Update the tests to *stop* using <httpbingo.org> and instead use
<httpbin.org>. This allows our tests to work as expected:
% curl -D - 'https://httpbin.org/redirect-to?url=https://github.com/xamarin/xamarin-android'
HTTP/2 302
date: Wed, 16 Nov 2022 15:38:28 GMT
content-type: text/html; charset=utf-8
content-length: 0
location: https://github.com/xamarin/xamarin-android
server: gunicorn/19.9.0
access-control-allow-origin: *
access-control-allow-credentials: true
Our `https://xamjenkinsartifact.azureedge.net` site for some of the
unit test data seems to have disappeared.
So lets use a github repo to store this data instead. The unit tests
can download the raw files directly from github. The repo will be
public as well so there won't be any need for authorization.
Update `DeleteBinObjTest` to use `7z`
Changes: 11f6b8f712...d552037815
Changes: 5f9bfd94d9...493ce626f3
Changes: 6d10e4c8bc...b8d49801fe
Context: https://github.com/dotnet/runtime/issues/77273
Context: e46523032d
Context: https://github.com/dotnet/runtime/issues/77385
Build and run under .NET 8!
.NET 8 is used by default for the default build.
Running unit tests also requires installing .NET 6 and .NET 7 as well.
Configure dotnet6 + dotnet7 + dotnet8 NuGet feeds.
Run `darc update-dependencies --id 152596` to update the
dotnet/installer version in `eng/Version.Details.xml`.
(ID values come from [Maestro][0].)
Update `AutoImport.props` to only evaluate default android items
in .NET 8.
Update `$(DotNetTargetFrameworkVersion)` to 8.0, and
`$(DotNetStableTargetFramework)` to `net7.0`. (Technically .NET 7
isn't stable *yet*, but it will be soon enough…)
Update `*.apkdesc` files so that unit tests pass.
Update `PackagingTest.CheckIncludedAssemblies()` test to remove
assemblies which are no longer included in .NET 8-based apps.
Update `use-dot-net.yaml` to take a new `quality` parameter, and
install .NET Core 3.1 quality=GA, not 3.1.417 specifically.
Avoids the error:
dotnet-install: Failed to locate the latest version in the channel '3.1.417' with 'preview' quality for 'dotnet-sdk', os: 'win', architecture: 'x64'.
Update `src/Mono.Android.Export` to pass
`MethodAttributes.Static | MethodAttributes.Public` to the
`DynamicMethod` constructor, not just `MethodAttributes.Static`.
This avoids a `NotSupportedException` under .NET 8;
see also dotnet/runtime@e4652303 and dotnet/runtime#77273:
System.NotSupportedException: Wrong MethodAttributes or CallingConventions for DynamicMethod. Only public, static, standard supported
at System.Reflection.Emit.DynamicMethod.CheckConsistency(MethodAttributes attributes, CallingConventions callingConvention)
at System.Reflection.Emit.DynamicMethod.Init(String name, MethodAttributes attributes, CallingConventions callingConvention, Type returnType, Type[] signature, Type owner, Module m, Boolean skipVisibility, Boolean transparentMethod)
at System.Reflection.Emit.DynamicMethod..ctor(String name, MethodAttributes attributes, CallingConventions callingConvention, Type returnType, Type[] parameterTypes, Module m, Boolean skipVisibility)
at Java.Interop.DynamicCallbackCodeGenerator.GenerateNativeCallbackDelegate()
at Java.Interop.DynamicCallbackCodeGenerator.GetCallback()
at Java.Interop.DynamicCallbackCodeGenerator.Create(MethodInfo method)
at System.Reflection.MethodInvoker.InterpretedInvoke(Object obj, Span`1 args, BindingFlags invokeAttr)
Update `XASdkTests.DotNetPublish()` for .NET 8. With 745214deb0
we asserted that builds using the *non-* latest target framework
would use a `Mono.Android.dll` reference assembly *from the build*,
not from an existing published/NuGet-provided `Microsoft.Android.Ref`
package. This started failing because
`$(TargetFramework)`=net7.0-android is no longer the latest framework
and thus *should* be using published packages, not in-tree bits:
Build should be using C:\a_work\1\s\xamarin-android\bin\Release\dotnet\packs\Microsoft.Android.Ref.33\34.0.0-ci.pr.gh7451.52\ref\net7.0\Mono.Android.dll
Expected: True
But was: False
Update the test to instead make this assertion only for the latest
target framework (`net8.0-android`), skipping .NET 7.
Remove an assertion of no build warnings under `net7.0`; this isn't
currently possible, due to dotnet/runtime#77385.
Update the `Microsoft.NETCore.App.Runtime.AOT.Cross` SDK package names
when `$(AotAssemblies)`=True; the names do not (yet?) include the
.NET version in the package names.
When installing multiple .NET SDKs into a single folder with the
`dotnet-install.{.sh,ps1}` script -- needed to run the unit tests --
you can hit an issue where `dotnet` no longer runs:
dotnet --info
Exited with code: 137
The problem is that the `dotnet` binary can get overwritten by an
older .NET, and be completely broken.
The solution is:
1. Install the newest .NET first, followed by any older versions
2. Use `dotnet-install.ps1 -SkipNonVersionedFiles` or
`dotnet-install.sh --skip-non-versioned-files` so the `dotnet`
binary isn't overwritten
This results in the newest .NET SDK, with side-by-side older .NET SDKs
installed as well.
Rename `tests/api-compatibility/acceptable-breakages-vReference-net7.0.txt`
to use `-net8.0`, as the default `$(TargetFramework)` for
`Mono.Android.dll` changed.
[0]: https://maestro-prod.westus2.cloudapp.azure.com/3074/https:%2F%2Fgithub.com%2Fdotnet%2Finstaller/latest/graph
Context: https://github.com/dotnet/runtime/pull/68610
Context: 0be567a991
In .NET 8, a test using `SpecialFolder.Personal` failed with:
[FAIL] Create table attempt failed!
SQLite.SQLiteException: Could not open database file: /data/user/0/com.xamarin.customlinkdescriptionpreserve/files/Documents/TaskDB.db3 (CannotOpen)
at SQLite.SQLiteConnection..ctor(SQLiteConnectionString )
at SQLite.SQLiteConnectionWithLock..ctor(SQLiteConnectionString )
at SQLite.SQLiteConnectionPool.Entry..ctor(SQLiteConnectionString )
at SQLite.SQLiteConnectionPool.GetConnectionAndTransactionLock(SQLiteConnectionString , Object& )
at SQLite.SQLiteConnectionPool.GetConnection(SQLiteConnectionString )
at SQLite.SQLiteAsyncConnection.GetConnection()
at SQLite.SQLiteAsyncConnection.<>c__DisplayClass33_0`1[[SQLite.CreateTableResult, SQLite-net, Version=1.7.335.0, Culture=neutral, PublicKeyToken=null]].<WriteAsync>b__0()
at System.Threading.Tasks.Task`1[[SQLite.CreateTableResult, SQLite-net, Version=1.7.335.0, Culture=neutral, PublicKeyToken=null]].InnerInvoke()
at System.Threading.Tasks.Task.<>c.<.cctor>b__273_0(Object )
at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread , ExecutionContext , ContextCallback , Object )
--- End of stack trace from previous location ---
at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread , ExecutionContext , ContextCallback , Object )
at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& , Thread )
--- End of stack trace from previous location ---
at LinkTestLib.Bug35195.AttemptCreateTable()
All regression tests completed.
In .NET 8+, we should use `SpecialFolder.UserProfile` instead.
Context: 5271f3e109
Context: e1af9587bb
Context: 186a9fcfac
Context: 903ba37ce7
Context: a760281bb1
Context: https://github.com/xamarin/xamarin-android/wiki/Blueprint#java-type-registration
Complete the LLVM Marshal Methods effort sketched out in e1af9587.
LLVM Marshal Methods are only supported in .NET Android, *not*
Xamarin.Android.
A *Marshal Method* is a JNI Callable C function (pointer) which has
[parameter types and return types which comply with the JNI ABI][0].
[`generator`][1] emits marshal methods as part of the binding, which
are turned into Delegate instances at runtime as part of
[Java Type Registration][2].
*LLVM Marshal Methods* turn this runtime operation -- looking up
`generator`-emitted marshal methods and registering those methods
with Java -- into a *build-time* operation, using LLVM-IR to generate
[JNI Native Method Names][3] which will then be contained within
`libxamarin-app.so`. LLVM Marshal Methods will also *remove* the
previous Reflection-based infrastructure from relevant types.
LLVM Marshal Methods are *enabled by default* for ***Release***
configuration builds in .NET 8, and disabled by default for Debug
builds. The new `$(AndroidEnableMarshalMethods)` MSBuild property
explicitly controls whether or not LLVM Marshal Methods are used.
LLVM Marshal Methods are *not* available in Classic Xamarin.Android.
~~ Build Phase: Scanning for Compatible Types ~~
During the application build, all `Java.Lang.Object` and
`Java.Lang.Throwable` subclasses are scanned as part of
[Java Callable Wrapper generation][4], looking for "un-bound"
(user-written) types which override `abstract` or `virtual`
methods, or implement interface members. This is done to emit
Java Callable Wrappers, Java code which "mirrors" the C# code with
an appropriate base class, interface implementation list, and
Java `native` method declarations for "virtual" member overrides.
This scanning process is updated for LLVM Marshal Methods to classify
each type to see if it requires the legacy Delegate-based
registration mechanism, as constructs such as
`[Java.Interop.ExportAttribute]` cannot (yet) be used with
LLVM Marshal Methods.
~~ Build Phase: Java Callable Wrapper Generation ~~
For example, given the C# type:
// C#
public partial class MainActivity : Activity {
protected override void OnCreate (Bundle? state) => …
}
Then the resulting Java Callable Wrapper *without* LLVM Marshal
Methods enabled will be:
// Java + No LLVM Marshal Methods
public /* partial */ class MainActivity extends Activity {
static {
String __md_methods =
"n_onCreate:(Landroid/os/Bundle;)V:GetOnCreate_Landroid_os_Bundle_Handler\n";
mono.android.Runtime.register ("Example.MainActivity, ExampleAssembly", MainActivity.class, __md_methods);
}
public void onCreate (android.os.Bundle p0) {n_onCreate(p0);}
private native void n_onCreate (android.os.Bundle p0);
}
When LLVM Marshal Methods are enabled, the Java Callable Wrapper
has no static constructor, nor any call to `Runtime.register()`.
~~ Build Phase: Marshal Method Wrapper ~~
Consider the binding infrastructure code that `generator` emits for
`Android.App.Activity.OnCreate()`:
namespace Android.App {
public partial class Activity {
static Delegate? cb_onCreate_Landroid_os_Bundle_;
#pragma warning disable 0169
static Delegate GetOnCreate_Landroid_os_Bundle_Handler ()
{
if (cb_onCreate_Landroid_os_Bundle_ == null)
cb_onCreate_Landroid_os_Bundle_ = JNINativeWrapper.CreateDelegate ((_JniMarshal_PPL_V) n_OnCreate_Landroid_os_Bundle_);
return cb_onCreate_Landroid_os_Bundle_;
}
static void n_OnCreate_Landroid_os_Bundle_ (IntPtr jnienv, IntPtr native__this, IntPtr native_savedInstanceState)
{
var __this = global::Java.Lang.Object.GetObject<Android.App.Activity> (jnienv, native__this, JniHandleOwnership.DoNotTransfer)!;
var savedInstanceState = global::Java.Lang.Object.GetObject<Android.OS.Bundle> (native_savedInstanceState, JniHandleOwnership.DoNotTransfer);
__this.OnCreate (savedInstanceState);
}
#pragma warning restore 0169
[Register ("onCreate", "(Landroid/os/Bundle;)V", "GetOnCreate_Landroid_os_Bundle_Handler")]
protected virtual unsafe void OnCreate (Android.OS.Bundle? savedInstanceState)
{
const string __id = "onCreate.(Landroid/os/Bundle;)V";
try {
JniArgumentValue* __args = stackalloc JniArgumentValue [1];
__args [0] = new JniArgumentValue ((savedInstanceState == null) ? IntPtr.Zero : ((global::Java.Lang.Object) savedInstanceState).Handle);
_members.InstanceMethods.InvokeVirtualVoidMethod (__id, this, __args);
} finally {
global::System.GC.KeepAlive (savedInstanceState);
}
}
}
}
When LLVM Marshal Methods are enabled, the following IL
transformations are performed:
* The `static Delegate? cb_…` field is removed.
* The `static Delegate Get…Handler()` method is removed.
* A new `static … n_…_mm_wrapper()` method is added.
The `n_…_mm_wrapper()` method is responsible for exception marshaling
and for `bool` marshaling. The `n_…_mm_wrapper()` method has the
[`UnmanagedCallersOnlyAttribute`][5], and works by calling the
existing `n_…()` method:
namespace Android.App {
public partial class Activity {
// Added
[UnmanagedCallersOnly]
static void n_OnCreate_Landroid_os_Bundle__mm_wrapper (IntPtr jnienv, IntPtr native__this, IntPtr native_savedInstanceState)
{
try {
n_OnCreate_Landroid_os_Bundle_ (jnienv, native__this, native_savedInstanceState);
}
catch (Exception __e) {
Android.Runtime.AndroidEnvironmentInternal.UnhandledException (__e);
}
}
}
}
~~ Build Phase: LLVM-IR Marshal Method Generation ~~
For each Java `native` method declaration contained in Java Callable
Wrappers which support LLVM Marshal Methods, LLVM-IR is used to
generate the JNI Native Method with the `Java_…` symbol name:
using android_app_activity_on_create_bundle_fn = void (*) (JNIEnv *env, jclass klass, jobject savedInstanceState);
static android_app_activity_on_create_bundle_fn android_app_activity_on_create_bundle = nullptr;
extern "C" JNIEXPORT void
JNICALL Java_helloandroid_MainActivity_n_1onCreate__Landroid_os_Bundle_2 (JNIEnv *env, jclass klass, jobject savedInstanceState) noexcept
{
if (android_app_activity_on_create_bundle == nullptr) {
get_function_pointer (
16, // mono image index; computed at build time
0, // class index; computed at build time
0x0600055B, // method token; computed at build time
reinterpret_cast<void*&>(android_app_activity_on_create_bundle) // target pointer
);
}
android_app_activity_on_create_bundle (env, klass, savedInstanceState);
}
~~ Other Changes ~~
The new `Android.Runtime.JNIEnvInit` type was split out of the
`Android.Runtime.JNIEnv` type to further reduce startup overhead, as
there are fewer fields to initialize.
The `Mono.Android.Runtime.dll` assembly is added because the
Marshal Method Wrapper needs to be able to invoke what *was*
`AndroidEnvironment.UnhandledException()`, *while also* updating
`Mono.Android.dll`! `Mono.Android.Runtime.dll` allows the marshal
method wrappers to reliably use
`Android.Runtime.AndroidEnvironmentInternal.UnhandledException()`,
which will *never* be changed by the marshal method wrapper
infrastructure.
~~ Results ~~
Marshal methods make application startup around 3.2% faster (the
bigger the app the more performance gains), with a bit room for
future improvements (by eliminating wrapper methods and other
optimizations):
[.NET Podcasts][6] app test results:
| Before | After | Δ | Notes |
| ------- | ------- | -------- | ---------------------------------------------- |
| 868.500 | 840.400 | -3.24% ✓ | preload disabled; 32-bit build; no compression |
| 863.700 | 837.600 | -3.02% ✓ | preload disabled; 64-bit build; no compression |
| 872.500 | 850.100 | -2.57% ✓ | preload enabled; 64-bit build |
| 877.000 | 854.800 | -2.53% ✓ | preload disabled; 64-bit build |
| 859.300 | 839.800 | -2.27% ✓ | preload enabled; 64-bit build; no compression |
| 871.700 | 853.100 | -2.13% ✓ | preload enabled; 32-bit build |
| 860.600 | 842.300 | -2.13% ✓ | preload enabled; 32-bit build; no compression |
| 869.500 | 852.500 | -1.96% ✓ | preload disabled; 32-bit build |
Maui Hello World app test results:
| Before | After | Δ | Notes |
| ------- | ------- | -------- | ---------------------------------------------- |
| 374.800 | 365.500 | -2.48% ✓ | preload disabled; 64-bit build |
| 374.100 | 365.600 | -2.27% ✓ | preload disabled; 32-bit build |
| 369.100 | 364.400 | -1.27% ✓ | preload enabled; 32-bit build |
| 364.300 | 360.600 | -1.02% ✓ | preload enabled; 32-bit build; no compression |
| 368.900 | 365.400 | -0.95% ✓ | preload enabled; 64-bit build |
| 362.500 | 359.400 | -0.86% ✓ | preload disabled; 32-bit build; no compression |
| 361.100 | 361.600 | +0.14% ✗ | preload enabled; 64-bit build; no compression |
| 359.200 | 368.000 | +2.39% ✗ | preload disabled; 64-bit build; no compression |
[0]: https://docs.oracle.com/javase/8/docs/technotes/guides/jni/spec/design.html#native_method_arguments
[1]: https://github.com/xamarin/xamarin-android/wiki/Blueprint#generator
[2]: https://github.com/xamarin/xamarin-android/wiki/Blueprint#java-type-registration
[3]: https://docs.oracle.com/javase/8/docs/technotes/guides/jni/spec/design.html#resolving_native_method_names
[4]: https://github.com/xamarin/xamarin-android/wiki/Blueprint#java-callable-wrapper-generator
[5]: https://learn.microsoft.com/en-us/dotnet/api/system.runtime.interopservices.unmanagedcallersonlyattribute?view=net-7.0
[6]: https://github.com/microsoft/dotnet-podcasts/tree/net7.0
Fixes: https://github.com/xamarin/xamarin-android/issues/7178
Using this combination on .NET 6+:
<PropertyGroup Condition=" '$(Configuration)' == 'Release' ">
<RunAOTCompilation>true</RunAOTCompilation>
<PublishTrimmed>false</PublishTrimmed>
</PropertyGroup>
Causes apps to crash at runtime with:
D Mono : AOT: module Microsoft.Maui.dll.so is unusable (GUID of dependent assembly Xamarin.AndroidX.AppCompat doesn't match (expected '1FC81757-8A70-4D56-93E6-8A635E2C23DE', got 'FD3821D1-CBF4-4956-B930-EA2A5379E18D')).
AOT runs right after the `<ILLink/>` MSBuild task, and so this
combination currently doesn't work when trimming is disabled.
For now, solve this by emitting an `XA1030` error and fail the build.
This way you at least get a reasonable error at build time instead of
at runtime.
I do not know of a scenario when it would be useful to disable
trimming completely during a `Release` + AOT build. If you are
hitting a trimming/linker issue, you would be better off preserving
a single assembly, type, etc.
Fixes: https://github.com/xamarin/xamarin-android/issues/7097
Context: a619cbea33
Context: 4787e0179b
Usage of a [`Google.Android.Material.TextField.TextInputEditText`][0]:
var filterBox = FindViewById<TextInputEditText>(Resource.Id.filterBox);
filterBox.TextChanged += (s, e) => { };
may crash at runtime with:
android.runtime.JavaProxyThrowable: System.TypeLoadException: Could not load type '{0}' from assembly '{1}'., Android.Text.ITextWatcherInvoker, Mono.Android, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null
at System.RuntimeTypeHandle.GetTypeByName(String , Boolean , Boolean , StackCrawlMark& , Boolean )
at System.RuntimeType.GetType(String , Boolean , Boolean , StackCrawlMark& )
at System.Type.GetType(String , Boolean )
at Android.Runtime.AndroidTypeManager.RegisterNativeMembers(JniType , Type , ReadOnlySpan`1 )
--- End of stack trace from previous location ---
at Java.Interop.JniEnvironment.StaticMethods.CallStaticObjectMethod(JniObjectReference , JniMethodInfo , JniArgumentValue* )
at Android.Runtime.JNIEnv.FindClass(String )
at Android.Runtime.JNIEnv.AllocObject(String )
at Android.Runtime.JNIEnv.StartCreateInstance(String , String , JValue* )
at Android.Runtime.JNIEnv.StartCreateInstance(String , String , JValue[] )
at Android.Text.TextWatcherImplementor..ctor(Object , EventHandler`1 , EventHandler`1 , EventHandler`1 )
at Android.Widget.TextView.add_TextChanged(EventHandler`1 )
at AndroidApp1.MainActivity.OnCreate(Bundle savedInstanceState)
at Android.App.Activity.n_OnCreate_Landroid_os_Bundle_(IntPtr , IntPtr , IntPtr )
at Android.Runtime.JNINativeWrapper.Wrap_JniMarshal_PPL_V(_JniMarshal_PPL_V , IntPtr , IntPtr , IntPtr )
at crc64a6e0c00971f6cd91.MainActivity.n_onCreate(Native Method)
at crc64a6e0c00971f6cd91.MainActivity.onCreate(MainActivity.java:29)
The problem is that the .NET 6 linker *completely* removed the
`Android.Text.ITextWatcher` interface, which involved updating the
post-linked `Android.Text.TextWatcherImplementor` type to no longer
implement the `ITextWatcher` interface.
[`Android.Text.TextWatcherImplementor`][1] in turn is a hand-written
type within `Mono.Android.dll` which we expected (required!) to
always implement the `ITextWatcher` interface, so that we could pass
instances of it to Java code.
The *cause* of the `TypeLoadException` is through the confluence of
multiple factors:
1. Java Callable Wrappers for `Mono.Android.dll` are generated at
*xamarin-android* build time, *not* App build time. This was
originally done to reduce App build times, but also means that
the Java Callable Wrapper for `TextWatcherImplementor` mentions
the `ITextWatcherInvoker` type, which is only preserved if the
`ITextWatcher` interface is preserved.
/* partial */ class TextWatcherImplementor {
static {
__md_methods =
"n_afterTextChanged:(Landroid/text/Editable;)V:GetAfterTextChanged_Landroid_text_Editable_Handler:Android.Text.ITextWatcherInvoker, Mono.Android, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null\n" +
"n_beforeTextChanged:(Ljava/lang/CharSequence;III)V:GetBeforeTextChanged_Ljava_lang_CharSequence_IIIHandler:Android.Text.ITextWatcherInvoker, Mono.Android, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null\n" +
"n_onTextChanged:(Ljava/lang/CharSequence;III)V:GetOnTextChanged_Ljava_lang_CharSequence_IIIHandler:Android.Text.ITextWatcherInvoker, Mono.Android, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null\n" +
"";
mono.android.Runtime.register ("Android.Text.TextWatcherImplementor, Mono.Android", TextWatcherImplementor.class, __md_methods);
}
}
2. The .NET linker determines that the `ITextWatcher` interface is
not actually used, and removes it from `Mono.Android.dll`.
Without `ITextWatcher`, the `ITextWatcherInvoker` type is also
removed from `Mono.Android.dll`.
3. At runtime when subscribing to the `TextView.TextChanged` event,
an instance of `TextWatcherImplementor` is created. This in turn
cases the Java peer `TextWatcherImplementor` type to be created,
triggering the [Java Type Registration process][2], which
includes the `Runtime.register()` invocation mentioning a method
which involves the `ITextWatcherInvoker` type, e.g.
"n_afterTextChanged:(Landroid/text/Editable;)V:GetAfterTextChanged_Landroid_text_Editable_Handler:Android.Text.ITextWatcherInvoker, Mono.Android\n"
4. `AndroidTypeManager.RegisterNativeMembers()` eventually attempts
`Type.GetType("Android.Text.ITextWatcherInvoker, Mono.Android")`.
This throws the `TypeLoadException`, as that type doesn't exist.
Commits 4d8c28f6 and d762aa99 fixed a very similar scenario in
Classic Xamarin.Android. The difference now is that the .NET 6+
linker is getting *so* good that more types are now eligible for
removal by the linker.
Oversimplifying, the "real" cause of the crash is that when:
1. We have a "non-bound" type which implements a Java interface,
such as `TextWatcherImplementor`, *and*
2. The linker decides that no IL-visible code uses that Java
interface, and decides to remove that interface
then we're going to be in a world of hurt.
The fix is to update `MonoDroid.Tuner.MarkJavaObject` so that if
we're preserving a type, we also explicitly preserve all Java
interfaces implemented by the type as well, *if the type is not bound*.
This allows a linked `TextWatcherImplementor` to continue to implement
`ITextWatcher`, which in turn causes `ITextWatcherInvoker` to be
preserved, which prevents the `TypeLoadException` from occurring.
We determine that a type is "bound" by looking for:
* `[Android.Runtime.RegisterAttribute (…, DoNotGenerateAcw=true)]`, or
* `[Java.Interop.JniTypeSignatureAttribute (…, GenerateJavaPeer=false)]`
A `Java.Lang.Object` subclass which *isn't* "bound" is considered to
be "non-bound").
We don't want to preserve all Java interfaces for all types,
including bound types, because that isn't necessary and causes app
sizes to balloon.
[0]: https://developer.android.com/reference/com/google/android/material/textfield/TextInputEditText
[1]: 619420ae1d/src/Mono.Android/Android.Text/ITextWatcher.cs (L52-L93)
[2]: https://github.com/xamarin/xamarin-android/wiki/Blueprint#java-type-registration
Fixes: https://github.com/xamarin/xamarin-android/issues/7194
Context: https://github.com/dotnet/maui/pull/7038
The initial version of `$(AndroidLinkResources)` (9e6ce03c) was too
broad in its removal of Resource classes and fields. Certain fields
such as `Styleable` arrays were not called using the IL `stsfld`
opcode. As a result they could not be easily replaced with constant
usage.
However, the linker removed *all* the fields from the `Resource`
nested types. This would result in the following error at runtime:
System.BadImageFormatException: 'Could not resolve field token 0x0400000b'
This was because the `int[]` fields were removed as part of the
linking process.
Fix this by leaving the `int[]` fields in the `Resource` nested types
instead of removing them.
We can still remove all the other `int` fields.
We now also need to fix up the `Resource` nested type constructors
to replace the `int` field access with the constant values like we do
for the rest of the app. This was not required previously because
these constructors were removed, but now we have to keep them because
the static array initialization takes place in these constructors.
It appears that an Android NDK installation is no longer needed when
using Aot with LLVM. Projects which enable Aot and LLVM will no longer
attempt to use the NDK unless it is explicitly requested by setting
`$(AndroidNdkDirectory)` to a valid NDK path in the project file.
This also enables tests for LLVM with a few categories turned off for now:
* `InetAccess`: https://github.com/dotnet/runtime/issues/73304
* `NetworkInterfaces`: https://github.com/dotnet/runtime/issues/75155
Fixes: https://github.com/xamarin/xamarin-android/issues/7149
Context: 48540d6933
Context: https://docs.microsoft.com/en-us/answers/questions/885827/error-hostname-not-verified-when-sending-request-o.html
Context: https://github.com/dotnet/docs-maui/issues/600
Context: https://github.com/dotnet/maui/discussions/8131
Context: https://github.com/dotnet/runtime/issues/70434
Commit 48540d69 added support to
`AndroidMessageHandler.ServerCertificateCustomValidationCallback` for
self-signed certificates. However, it didn't account for mismatches
between the server's hostname and the certificate's hostname.
This can cause issues when e.g. you use 10.0.2.2 to send requests to
a local server with a self-signed dev certificate in ASP.NET:
var handler = new AndroidMessageHandler {
// Allow any cert; DO NOT USE IN PRODUCTION
ServerCertificateCustomValidationCallback =
(httpRequestMessage, cert, cetChain, policyErrors) => true,
// …
};
var client = new HttpClient(handler);
var response = await client.SendAsync(message);
This could result an exception similar to:
System.Net.WebException: Hostname EXAMPLE.DOMAIN not verified:
certificate: sha1/EXAMPLE_CERT_SHA1
DN: CN=EXAMPLE_CN
subjectAltNames: [EXAMPLE_CN]
---> Javax.Net.Ssl.SSLPeerUnverifiedException: Hostname EXAMPLE.DOMAIN not verified:
certificate: sha1/EXAMPLE_CERT_SHA1
DN: CN=EXAMPLE_CN
subjectAltNames: [EXAMPLE_CN]
at Java.Interop.JniEnvironment.InstanceMethods.CallVoidMethod(JniObjectReference instance, JniMethodInfo method, JniArgumentValue* args)
at Java.Interop.JniPeerMembers.JniInstanceMethods.InvokeAbstractVoidMethod(String encodedMember, IJavaPeerable self, JniArgumentValue* parameters)
at Javax.Net.Ssl.HttpsURLConnectionInvoker.Connect()
at Xamarin.Android.Net.AndroidMessageHandler.<>c__DisplayClass125_0.<ConnectAsync>b__0()
at System.Threading.Tasks.Task.InnerInvoke()
at System.Threading.Tasks.Task.<>c.<.cctor>b__272_0(Object obj)
at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state)
--- End of managed Javax.Net.Ssl.SSLPeerUnverifiedException stack trace ---
javax.net.ssl.SSLPeerUnverifiedException: Hostname EXAMPLE.DOMAIN not verified:
certificate: sha1/EXAMPLE_CERT_SHA1
DN: CN=EXAMPLE_CN
subjectAltNames: [EXAMPLE_CN]
at com.android.okhttp.internal.io.RealConnection.connectTls(RealConnection.java:205)
at com.android.okhttp.internal.io.RealConnection.connectSocket(RealConnection.java:153)
at com.android.okhttp.internal.io.RealConnection.connect(RealConnection.java:116)
at com.android.okhttp.internal.http.StreamAllocation.findConnection(StreamAllocation.java:186)
at com.android.okhttp.internal.http.StreamAllocation.findHealthyConnection(StreamAllocation.java:128)
at com.android.okhttp.internal.http.StreamAllocation.newStream(StreamAllocation.java:97)
at com.android.okhttp.internal.http.HttpEngine.connect(HttpEngine.java:289)
at com.android.okhttp.internal.http.HttpEngine.sendRequest(HttpEngine.java:232)
at com.android.okhttp.internal.huc.HttpURLConnectionImpl.execute(HttpURLConnectionImpl.java:465)
at com.android.okhttp.internal.huc.HttpURLConnectionImpl.connect(HttpURLConnectionImpl.java:131)
at com.android.okhttp.internal.huc.DelegatingHttpsURLConnection.connect(DelegatingHttpsURLConnection.java:90)
at com.android.okhttp.internal.huc.HttpsURLConnectionImpl.connect(HttpsURLConnectionImpl.java:30)
--- End of inner exception stack trace ---
at Xamarin.Android.Net.AndroidMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
Address this problem and aligns the behavior of
`AndroidMessageHandler` with other platforms, where hostname
verification is part of the server certificate custom validation
callback.
There are two significant changes:
- The trust manager that wraps the custom callback now verifies the
certificate hostname
- Java's separate hostname verification is skipped
There is only one aspect which is a little hacky: the
[`javax.net.ssl.HostnameVerifier` interface][0] requires an instance
of [`javax.net.ssl.SSLSession`][1], which we don't have access to in
the trust manager. The default hostname verifier instead uses just a
single getter to access the chain of certificates so it seems OK to me
to have a dummy class that implements the interface that holds just
the peer certificates.
(I tried to find access to the session from `HttpsURLConnection` but
there doesn't seem to be a way to access the `SSLEngine` and its
`HandshakeSesion` property.)
[0]: https://developer.android.com/reference/javax/net/ssl/HostnameVerifier
[1]: https://developer.android.com/reference/javax/net/ssl/SSLSession
Frequently these three performance tests fail with times such as:
* Build_CSharp_Change
Exceeded expected time of 3850ms, actual 3912.98ms
* Build_AndroidManifest_Change
Exceeded expected time of 4250ms, actual 4873.595ms
* Install_CSharp_Change
Exceeded expected time of 4500ms, actual 4693.08ms
Let's increase these times to make our CI runs less noisy.
At some point, we could also do some build performance work to get
these times lower. Right now these time differences are pretty small,
and I believe due to environmental changes in the CI machines, etc.
Jonathan Peppers
Jonathan Pobst
Dean Ellis
Peter Collins
Marek Habersack
dotnet-maestro[bot]
Jonathan Pryor
Šimon Rozsíval
dependabot[bot]