Taking a first pass at deleting MSBuild-related files that are not
used in .NET 8 and only existed to support "classic" Xamarin.Android.
Deleting:
* `Xamarin.Android.Legacy.targets`
* `Xamarin.Android.Aapt.targets`
* `Xamarin.Android.Wear.targets`
* Associated MSBuild task `.cs` files that are now unused
After these changes, I got the error from
`Xamarin.Android.NUnitLite.csproj`:
Xamarin.Android.Tooling.targets(52,31): error MSB4057: The target "_GetReferenceAssemblyPaths" does not exist in the project.
I removed this project from `Xamarin.Android.sln` and `GeneratedMakeRulesFile.cs`
for now, but it will be a good target for removal in the future.
Fixes: https://github.com/xamarin/xamarin-android/issues/6307
This reverts c50df1c5.
In the past, we had to redistribute `apksigner.jar`, because it
required JDK 11. This happened for build-tools 30.0.0 and was fixed in
built-tools 30.0.3. At the time we were nowhere close able to use JDK
11, and so we built it ourselves for JDK 1.8 and redistributed it.
However, .NET 6+ now *requires* JDK 11, because targets API-31 and
higher. "Classic" Xamarin.Android requires JDK 11 if you are targeting
API-31.
We can remove `apksigner` from `main` going forward in .NET 8.
Other changes:
* Update `XASdkTests.cs` to account for `*.apk.sig` files.
Quick win to remove some components that we aren't shipping with .NET from `main`:
- OpenTK
- sqlite-xamarin
- System.EnterpriseServices
Additionally, this allows us to remove 2 `git` submodules: `mono/opentk` and `xamarin/sqlite`.
Bring the AzDO parallelization from https://github.com/xamarin/xamarin-android/pull/7804 and environment setup improvements from https://github.com/xamarin/xamarin-android/pull/7832 to `Xamarin.Android.Build.Tests.csproj` based test suites. This includes both the main `MSBuild` stage and the `Smoke Tests` jobs.
Increases parallelization of all jobs as many were approaching ~90 minutes.
As there is no longer a place in the `MSBuild` stage to run `Xamarin.Android.Tools.Aidl-Tests` tests, it was moved to the `macOS > Tests > APKs .NET` stage. This suite should be fine to only run on Mac and not Windows. (Note this test assembly was also updated to .NET 7. This required moving it from `Xamarin.Android-Tests.sln` which is currently built with Mono which cannot build .NET 7+ projects. It now is built via `Xamarin.Android.sln`.)
Context: 5271f3e109
Context: e1af9587bb
Context: 186a9fcfac
Context: 903ba37ce7
Context: a760281bb1
Context: https://github.com/xamarin/xamarin-android/wiki/Blueprint#java-type-registration
Complete the LLVM Marshal Methods effort sketched out in e1af9587.
LLVM Marshal Methods are only supported in .NET Android, *not*
Xamarin.Android.
A *Marshal Method* is a JNI Callable C function (pointer) which has
[parameter types and return types which comply with the JNI ABI][0].
[`generator`][1] emits marshal methods as part of the binding, which
are turned into Delegate instances at runtime as part of
[Java Type Registration][2].
*LLVM Marshal Methods* turn this runtime operation -- looking up
`generator`-emitted marshal methods and registering those methods
with Java -- into a *build-time* operation, using LLVM-IR to generate
[JNI Native Method Names][3] which will then be contained within
`libxamarin-app.so`. LLVM Marshal Methods will also *remove* the
previous Reflection-based infrastructure from relevant types.
LLVM Marshal Methods are *enabled by default* for ***Release***
configuration builds in .NET 8, and disabled by default for Debug
builds. The new `$(AndroidEnableMarshalMethods)` MSBuild property
explicitly controls whether or not LLVM Marshal Methods are used.
LLVM Marshal Methods are *not* available in Classic Xamarin.Android.
~~ Build Phase: Scanning for Compatible Types ~~
During the application build, all `Java.Lang.Object` and
`Java.Lang.Throwable` subclasses are scanned as part of
[Java Callable Wrapper generation][4], looking for "un-bound"
(user-written) types which override `abstract` or `virtual`
methods, or implement interface members. This is done to emit
Java Callable Wrappers, Java code which "mirrors" the C# code with
an appropriate base class, interface implementation list, and
Java `native` method declarations for "virtual" member overrides.
This scanning process is updated for LLVM Marshal Methods to classify
each type to see if it requires the legacy Delegate-based
registration mechanism, as constructs such as
`[Java.Interop.ExportAttribute]` cannot (yet) be used with
LLVM Marshal Methods.
~~ Build Phase: Java Callable Wrapper Generation ~~
For example, given the C# type:
// C#
public partial class MainActivity : Activity {
protected override void OnCreate (Bundle? state) => …
}
Then the resulting Java Callable Wrapper *without* LLVM Marshal
Methods enabled will be:
// Java + No LLVM Marshal Methods
public /* partial */ class MainActivity extends Activity {
static {
String __md_methods =
"n_onCreate:(Landroid/os/Bundle;)V:GetOnCreate_Landroid_os_Bundle_Handler\n";
mono.android.Runtime.register ("Example.MainActivity, ExampleAssembly", MainActivity.class, __md_methods);
}
public void onCreate (android.os.Bundle p0) {n_onCreate(p0);}
private native void n_onCreate (android.os.Bundle p0);
}
When LLVM Marshal Methods are enabled, the Java Callable Wrapper
has no static constructor, nor any call to `Runtime.register()`.
~~ Build Phase: Marshal Method Wrapper ~~
Consider the binding infrastructure code that `generator` emits for
`Android.App.Activity.OnCreate()`:
namespace Android.App {
public partial class Activity {
static Delegate? cb_onCreate_Landroid_os_Bundle_;
#pragma warning disable 0169
static Delegate GetOnCreate_Landroid_os_Bundle_Handler ()
{
if (cb_onCreate_Landroid_os_Bundle_ == null)
cb_onCreate_Landroid_os_Bundle_ = JNINativeWrapper.CreateDelegate ((_JniMarshal_PPL_V) n_OnCreate_Landroid_os_Bundle_);
return cb_onCreate_Landroid_os_Bundle_;
}
static void n_OnCreate_Landroid_os_Bundle_ (IntPtr jnienv, IntPtr native__this, IntPtr native_savedInstanceState)
{
var __this = global::Java.Lang.Object.GetObject<Android.App.Activity> (jnienv, native__this, JniHandleOwnership.DoNotTransfer)!;
var savedInstanceState = global::Java.Lang.Object.GetObject<Android.OS.Bundle> (native_savedInstanceState, JniHandleOwnership.DoNotTransfer);
__this.OnCreate (savedInstanceState);
}
#pragma warning restore 0169
[Register ("onCreate", "(Landroid/os/Bundle;)V", "GetOnCreate_Landroid_os_Bundle_Handler")]
protected virtual unsafe void OnCreate (Android.OS.Bundle? savedInstanceState)
{
const string __id = "onCreate.(Landroid/os/Bundle;)V";
try {
JniArgumentValue* __args = stackalloc JniArgumentValue [1];
__args [0] = new JniArgumentValue ((savedInstanceState == null) ? IntPtr.Zero : ((global::Java.Lang.Object) savedInstanceState).Handle);
_members.InstanceMethods.InvokeVirtualVoidMethod (__id, this, __args);
} finally {
global::System.GC.KeepAlive (savedInstanceState);
}
}
}
}
When LLVM Marshal Methods are enabled, the following IL
transformations are performed:
* The `static Delegate? cb_…` field is removed.
* The `static Delegate Get…Handler()` method is removed.
* A new `static … n_…_mm_wrapper()` method is added.
The `n_…_mm_wrapper()` method is responsible for exception marshaling
and for `bool` marshaling. The `n_…_mm_wrapper()` method has the
[`UnmanagedCallersOnlyAttribute`][5], and works by calling the
existing `n_…()` method:
namespace Android.App {
public partial class Activity {
// Added
[UnmanagedCallersOnly]
static void n_OnCreate_Landroid_os_Bundle__mm_wrapper (IntPtr jnienv, IntPtr native__this, IntPtr native_savedInstanceState)
{
try {
n_OnCreate_Landroid_os_Bundle_ (jnienv, native__this, native_savedInstanceState);
}
catch (Exception __e) {
Android.Runtime.AndroidEnvironmentInternal.UnhandledException (__e);
}
}
}
}
~~ Build Phase: LLVM-IR Marshal Method Generation ~~
For each Java `native` method declaration contained in Java Callable
Wrappers which support LLVM Marshal Methods, LLVM-IR is used to
generate the JNI Native Method with the `Java_…` symbol name:
using android_app_activity_on_create_bundle_fn = void (*) (JNIEnv *env, jclass klass, jobject savedInstanceState);
static android_app_activity_on_create_bundle_fn android_app_activity_on_create_bundle = nullptr;
extern "C" JNIEXPORT void
JNICALL Java_helloandroid_MainActivity_n_1onCreate__Landroid_os_Bundle_2 (JNIEnv *env, jclass klass, jobject savedInstanceState) noexcept
{
if (android_app_activity_on_create_bundle == nullptr) {
get_function_pointer (
16, // mono image index; computed at build time
0, // class index; computed at build time
0x0600055B, // method token; computed at build time
reinterpret_cast<void*&>(android_app_activity_on_create_bundle) // target pointer
);
}
android_app_activity_on_create_bundle (env, klass, savedInstanceState);
}
~~ Other Changes ~~
The new `Android.Runtime.JNIEnvInit` type was split out of the
`Android.Runtime.JNIEnv` type to further reduce startup overhead, as
there are fewer fields to initialize.
The `Mono.Android.Runtime.dll` assembly is added because the
Marshal Method Wrapper needs to be able to invoke what *was*
`AndroidEnvironment.UnhandledException()`, *while also* updating
`Mono.Android.dll`! `Mono.Android.Runtime.dll` allows the marshal
method wrappers to reliably use
`Android.Runtime.AndroidEnvironmentInternal.UnhandledException()`,
which will *never* be changed by the marshal method wrapper
infrastructure.
~~ Results ~~
Marshal methods make application startup around 3.2% faster (the
bigger the app the more performance gains), with a bit room for
future improvements (by eliminating wrapper methods and other
optimizations):
[.NET Podcasts][6] app test results:
| Before | After | Δ | Notes |
| ------- | ------- | -------- | ---------------------------------------------- |
| 868.500 | 840.400 | -3.24% ✓ | preload disabled; 32-bit build; no compression |
| 863.700 | 837.600 | -3.02% ✓ | preload disabled; 64-bit build; no compression |
| 872.500 | 850.100 | -2.57% ✓ | preload enabled; 64-bit build |
| 877.000 | 854.800 | -2.53% ✓ | preload disabled; 64-bit build |
| 859.300 | 839.800 | -2.27% ✓ | preload enabled; 64-bit build; no compression |
| 871.700 | 853.100 | -2.13% ✓ | preload enabled; 32-bit build |
| 860.600 | 842.300 | -2.13% ✓ | preload enabled; 32-bit build; no compression |
| 869.500 | 852.500 | -1.96% ✓ | preload disabled; 32-bit build |
Maui Hello World app test results:
| Before | After | Δ | Notes |
| ------- | ------- | -------- | ---------------------------------------------- |
| 374.800 | 365.500 | -2.48% ✓ | preload disabled; 64-bit build |
| 374.100 | 365.600 | -2.27% ✓ | preload disabled; 32-bit build |
| 369.100 | 364.400 | -1.27% ✓ | preload enabled; 32-bit build |
| 364.300 | 360.600 | -1.02% ✓ | preload enabled; 32-bit build; no compression |
| 368.900 | 365.400 | -0.95% ✓ | preload enabled; 64-bit build |
| 362.500 | 359.400 | -0.86% ✓ | preload disabled; 32-bit build; no compression |
| 361.100 | 361.600 | +0.14% ✗ | preload enabled; 64-bit build; no compression |
| 359.200 | 368.000 | +2.39% ✗ | preload disabled; 64-bit build; no compression |
[0]: https://docs.oracle.com/javase/8/docs/technotes/guides/jni/spec/design.html#native_method_arguments
[1]: https://github.com/xamarin/xamarin-android/wiki/Blueprint#generator
[2]: https://github.com/xamarin/xamarin-android/wiki/Blueprint#java-type-registration
[3]: https://docs.oracle.com/javase/8/docs/technotes/guides/jni/spec/design.html#resolving_native_method_names
[4]: https://github.com/xamarin/xamarin-android/wiki/Blueprint#java-callable-wrapper-generator
[5]: https://learn.microsoft.com/en-us/dotnet/api/system.runtime.interopservices.unmanagedcallersonlyattribute?view=net-7.0
[6]: https://github.com/microsoft/dotnet-podcasts/tree/net7.0
Context: 018121239b
Commit 01812123 removed support for `dx` and ProGuard, turning
attempted use of `dx` into an XA1023 error. It also mentioned:
> TODO:
>
> Remove support for `dx` and ProGuard.
Complete this TODO; remove code related to `dx` and ProGuard.
This removes a lot of parameterized tests, as well as build and
package less stuff.
Context: 5432886562
*One* issue with running `dotnet build Xamarin.Android.sln` without
`-m:1` (54328865) is that it attempts to run our Android API
extraction process multiple times simultaneously. This results in
file locking issues:
C:\code\xamarin-android\src\Mono.Android\Mono.Android.targets(409,5): "C:\code\xamarin-android\src\Mono.Android\..\..\bin\Debug\lib\xamarin.android\xbuild\Xamarin\Android\class-parse.exe C:\…\android-toolchain\sdk\platforms\android-26\android.jar -platform=26 -parameter-names="C:\code\xamarin-android\src\Mono.Android\..\..\src\Mono.Android\Profiles\api-26.params.txt" -o="C:\code\xamarin-android\src\Mono.Android\..\..\bin\BuildDebug\api\api-26.xml.class-parse"" failed with code: -532462766 Error output:
Unhandled Exception: System.IO.IOException: The process cannot access the file 'C:\code\xamarin-android\bin\BuildDebug\api\api-26.xml.class-parse' because it is being used by another process.
at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
at System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost)
at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, FileOptions options, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost)
at System.IO.StreamWriter.CreateFile(String path, Boolean append, Boolean checkHost)
at System.IO.StreamWriter..ctor(String path, Boolean append, Encoding encoding, Int32 bufferSize, Boolean checkHost)
at System.IO.StreamWriter..ctor(String path, Boolean append, Encoding encoding)
at Xamarin.Android.Tools.App.Main(String[] args) in C:\code\xamarin-android\external\Java.Interop\tools\class-parse\Program.cs:line 68 [C:\code\xamarin-android\src\Mono.Android\Mono.Android.csproj]
This seems to be a result of running the process as an
`AfterTargets="Build"` target in `api-xml-adjuster.csproj`.
By moving it to a `NoTargets` project, we can eliminate MSBuild trying
to run it multiple times.
Move the `class-parse` execution to a new `create-android-api.csproj`
project.
Also, move the `api-merge` step into `create-android-api.csproj`,
from `Mono.Android.targets`. Currently the `pai-merge` process runs
twice, once for `monoandroid10` and once for `net7.0`. However, the
output is not dependent on `TargetFramework`. Instead of generating:
- `src\Mono.Android\obj\Debug\monoandroid10\android-32\mcw\api.xml`
- `src\Mono.Android\obj\Debug\net7.0\android-32\mcw\api.xml`
we will now generate:
- `bin\Build$(Configuration)\api\api-32.xml`
Calling `api-merge` only once saves ~25s from build time.
Note that `generator` writes intermediate files (eg: `api.xml.fixed`)
to the directory containing `api.xml`, so `Mono.Android.targets` will
copy the `api-X.xml` file to its eg: `obj\Debug\net7.0\android-32\mcw`
directory to work from. Without this, both `monoadroid10` and
`net7.0` would write to the common location, causing possible
file sharing conflicts.
Additionally, let VS2022 rewrite `Xamarin.Android.sln` to its
preferred format in order to add the new project.
The tools (opt, llc, cross*) that we download and install from the mono
bundle already seem to have the executable flag set, so this project
should no longer be needed.
Changes: 7dc270dbb8...cbd966686e
* xamarin/Java.Interop@cbd96668: [Java.Interop] $(Version) depends on TargetFramework (#952)
* xamarin/Java.Interop@e587cf6b: [build] Remove the `make prepare-core` targets (#951)
* xamarin/Java.Interop@e02d8572: [build] Remove globalPackagesFolder override (#948)
Of particular note is xamarin/Java.Interop@cbd96668, which changes
the version of `Java.Interop.dll` in the net6.0 profile from 0.1.0.0
to 6.0.0.0, and the version of "utility libs" like
`Xamarin.SourceWriter.dll` from being identical to their non-net6.0
counterparts, to instead having `6.0.0.*` version numbers.
This change in turn necessitates:
* Updating `build-tools/create-packs` and `build-tools/installers`
so that the "utility libs" are grabbed from their appropriate
build directory, so that we don't distribute e.g. "legacy"
`Xamarin.SourceWriter, Version=0.2.0.0` in the .net 6 package.
* Updating `tests/**/*.csproj` to specify a "full"
`$(TargetFramework)`=net6.0-android32 instead of
`$(TargetFramework)`=net6.0-android. If we don't use the "full"
version, we'll use the net6.0-android31.0 targeting pack, which
has the wrong version for `Java.Interop.dll`, possibly resulting
in compilation errors:
error CS0433: The type 'JavaObject' exists in both
'Java.Interop, Version=0.1.0.0, Culture=neutral, PublicKeyToken=84e04ff9cfb79065' and
'Java.Interop, Version=6.0.0.0, Culture=neutral, PublicKeyToken=84e04ff9cfb79065'.
xamarin/Java.Interop@e587cf6b simplified the Java.Interop build
system, necessitating changes to `build-tools/xaprepare` --
`make prepare-core` is no longer needed -- but also required adding
`Java.Runtime.Environment.csproj` to `Xamarin.Android.sln`, so that
it's build-time Target could properly create
`Java.Runtime.Environment.dll.config`.
TODO: Better deal with the `Java.Interop.dll` version change that is
now part of `net6.0-android32.0`. We should consider "removing" the
`net6.0-android31.0` targeting pack, and make `net6.0-android` be an
alias for `net6.0-android32.0`.
Context: ae6c427450
Context: https://docs.microsoft.com/xamarin/android/release-notes/12/12.1
Add a utility to *assist* with writing release notes:
git log -p --cherry-pick --right-only 46e867efc69ac5b62881277e2580d1b6e0347b52...d7ce643351b248777a50ea63b3b13b54d95aa6c5 \
| dotnet run --project tools/relnote-gen
`tools/relnote-gen` parses the `git log -p` output, looking for:
* The `commit COMMIT` header
* The commit message "summary", which optionally contains a
"component" (b9ba3763) and PR number
* The commit message text, which may contain `Fixes: ` URLs
* Release notes (via `git log -p`; any additions to the
`Documentation/release-notes` directory)
All commits with the same component are coalesced, and
Markdown-formatted text is written containing:
* The components found, as a `### Component` header.
* The commits, as a `-` bulleted list.
* For each commit, links to the Fixes, PR, and commit.
* The commit message body within an HTML <!-- … --> comment block
This kinda/sorta allows editing *all* commits at once, with most
relevant information, in one stream. This is "grist" for the commit
message, to be edited down before final publishing.
Update `Documentation/release-notes/README.md`, to reduce the number
of `#`s used for section headers. We're aiming for a smaller number
of sub-sections, as e.g. an `<h4/>` is nigh-invisible.
Current release notes template (kinda/sorta):
## Xamarin.Android Version [Version Number]
> Released [release date], included in Visual Studio [version]
- [Component](#component))
- …
<a name="component"></a>
### Component
- …
Changes: 974ad32ee4...79744f61b9
* xamarin/java.interop@79744f61: [generator] Fix for fixing invalid annotation XML. (#897)
* xamarin/java.interop@220b87fb: [tests] rework JavaObjectTest, use FinalizerHelper from mono/mono (#899)
* xamarin/java.interop@f658ab26: [generator] Add Java.Interop.Tools.JavaTypeSystem (#849)
Add `Java.Interop.Tools.JavaTypeSystem.csproj` to
`Xamarin.Android.sln`, and add `Java.Interop.Tools.JavaTypeSystem.dll`
to the installers.
Add a temporary MSBuild property `$(_AndroidUseJavaLegacyResolver)`,
which when set to `true` will cause `generator` to use the legacy
Java type resolver (`ApiXmlAdjuster`) instead of the new
`JavaTypeSystem` one, in case there are unexpected issues.
What do we want? Faster (Release) App Startup!
How do we get that? Assembly Stores!
"In the beginning", assemblies were stored in the `assemblies`
directory within the `.apk`. App startup would open the `.apk`,
traverse all entries within the `.apk` looking for `assemblies/*.dll`,
`assemblies/*.dll.config`, and `assemblies/*.pdb` files. When a
"supported" `assemblies/*` entry was encountered, the entry would be
**mmap**(2)'d so that it could be used; see also commit c1956835bd.
Of particular note is:
1. The need to enumerate *all* entries within the `.apk`, as there
is no guarantee of entry ordering, and
2. The need for *N* `mmap()` invocations, one per assembly included
in the app, *plus* additional `mmap()` invocations for the `.pdb`
and `.dll.config` files, if present.
Useful contextual note: a "modern" AndroidX-using app could pull
in dozens to over 200 assemblies without really trying.
There will be *lots* of `mmap()` invocations.
Instead of adding (compressed! d236af54) data for each assembly
separately, instead add a small set of "Assembly Store" files which
contain the assembly & related data to use within the app:
* `assemblies/assemblies.blob`
* `assemblies/assemblies.[ARCHITECTURE].blob`
`assemblies.[ARCHITECTURE].blob` contains architecture-specific
assemblies, e.g. `System.Private.CoreLib.dll` built for x86 would be
placed within `assemblies.x86.blob`. `ARCHITECTURE` is one of `x86`,
`x86_64`, `armeabi_v7a`, or `arm64_v8a`; note use of `_` instead of
`-`, which is different from the `lib/ARCHITECTURE` convention within
`.apk` files. This is done because this is apparently what Android
and `bundletool` do, e.g. creating `split_config.armeabi_v7a.apk`.
Once the architecture-neutral `assemblies.blob` and appropriate
(singular!) `assemblies.[ARCHITECTURE].blob` for the current
architecture is found and `mmap()`'d, `.apk` entry traversal can end.
There is no longer a need to parse the entire `.apk` during startup.
The reduction in the number of `mmap()` system calls required can
have a noticeable impact on process startup, particularly with
.NET SDK for Android & MAUI; see below for timing details.
The assembly store format uses the followings structures:
struct AssemblyStoreHeader {
uint32_t magic, version;
uint32_t local_entry_count; // Number of AssemblyStoreAssemblyDescriptor entries
uint32_t global_entry_count; // Number of AssemblyStoreAssemblyDescriptor entries in entire app, across all *.blob files
uint32_t store_id;
};
struct AssemblyStoreAssemblyDescriptor {
uint32_t data_offset, data_size; // Offset from beginning of file for .dll data
uint32_t debug_data_offset, debug_data_size; // Offset from beginning of file for .pdb data
uint32_t config_data_offset, config_data_size; // Offset from beginning of file for .dll.config data
};
struct AssemblyStoreHashEntry {
union {
uint64_t hash64; // 64-bit xxhash of assembly filename
uint32_t hash64; // 32-bit xxhash of assembly filename
};
uint32_t mapping_index, local_store_index, store_id;
};
The assembly store format is roughly as follows:
AssemblyStoreHeader header {…};
AssemblyStoreAssemblyDescriptor assemblies [header.local_entry_count];
// The following two entries exist only when header.store_id == 0
AssemblyStoreHashEntry hashes32[header.global_entry_count];
AssemblyStoreHashEntry hashes64[header.global_entry_count];
uint8_t data[];
Note that `AssemblyStoreFileFormat::hashes32` and
`AssemblyStoreFileFormat::hashes64` are *sorted by their hash*.
Further note that assembly *filenames* are not present.
`EmbeddedAssemblies::blob_assemblies_open_from_bundles()` will hash
the filename, then binary search the appropriate `hashes*` array to
get the appropriate assembly information.
As the assembly store format doesn't include assembly names, `.apk`
and `.aab` files will also contain an `assemblies.manifest` file,
which contains the assembly names and other information in a human-
readable format; it is also used by `assembly-store-reader`:
Hash 32 Hash 64 Blob ID Blob idx Name
0xa2e0939b 0x4288cfb749e4c631 000 0000 Xamarin.AndroidX.Activity
…
0xad6f1e8a 0x6b0ff375198b9c17 001 0000 System.Private.CoreLib
Add a new `tools/assembly-store-reader` utility which can read the
new `assemblies*.blob` files:
% tools/scripts/read-assembly-store path/to/app.apk
Store set 'base_assemblies':
Is complete set? yes
Number of stores in the set: 5
Assemblies:
0:
Name: Xamarin.AndroidX.Activity
Store ID: 0 (shared)
Hashes: 32-bit == 0xa2e0939b; 64-bit == 0x4288cfb749e4c631
Assembly image: offset == 1084; size == 14493
Debug data: absent
Config file: absent
…
16:
Name: System.Private.CoreLib
Store ID: 1 (x86)
Hashes: 32-bit == 0xad6f1e8a; 64-bit == 0x6b0ff375198b9c17
Assembly image: offset == 44; size == 530029
Debug data: absent
Config file: absent
…
On a Pixel 3 XL (arm64-v8a) running Android 12 with MAUI
6.0.101-preview.10.1952, we observe:
~~ MAUI: Displayed Time ~~
| Before ms | After ms | Δ | Notes |
| ---------:| --------: | -----------: | ------------------------------------- |
| 1016.800 | 892.600 | -12.21% ✓ | defaults; profiled AOT; 32-bit build |
| 1016.100 | 894.700 | -11.95% ✓ | defaults; profiled AOT; 64-bit build |
| 1104.200 | 922.000 | -16.50% ✓ | defaults; full AOT+LLVM; 64-bit build |
| 1102.700 | 926.100 | -16.02% ✓ | defaults; full AOT; 32-bit build |
| 1108.400 | 932.600 | -15.86% ✓ | defaults; full AOT; 64-bit build |
| 1106.300 | 932.600 | -15.70% ✓ | defaults; full AOT+LLVM; 32-bit build |
| 1292.000 | 1271.800 | -1.56% ✓ | defaults; 64-bit build |
| 1307.000 | 1275.400 | -2.42% ✓ | defaults; 32-bit build |
Displayed time reduces by ~12% when Profiled AOT is used.
It is interesting to note that **Displayed time** is nearly identical
for the default (JIT) settings case. It's most probably caused by the
amount of JIT-ed code between `OnCreate()` and the time when the
application screen is presented, most likely the time is spent JIT-ing
MAUI rendering code.
~~ MAUI: Total native init time (before `OnCreate()`) ~~
| Before ms | After ms | Δ | Notes |
| --------: | --------: | -----------: | ------------------------------------- |
| 96.727 | 88.921 | -8.07% ✓ | defaults; 32-bit build |
| 97.236 | 89.693 | -7.76% ✓ | defaults; 64-bit build |
| 169.315 | 108.845 | -35.71% ✓ | defaults; profiled AOT; 32-bit build |
| 170.061 | 109.071 | -35.86% ✓ | defaults; profiled AOT; 64-bit build |
| 363.864 | 208.949 | -42.57% ✓ | defaults; full AOT; 64-bit build |
| 363.629 | 209.092 | -42.50% ✓ | defaults; full AOT; 32-bit build |
| 373.203 | 218.289 | -41.51% ✓ | defaults; full AOT+LLVM; 64-bit build |
| 372.783 | 219.003 | -41.25% ✓ | defaults; full AOT+LLVM; 32-bit build |
Note that "native init time" includes running `JNIEnv.Initialize()`,
which requires loading `Mono.Android.dll` + dependencies such as
`System.Private.CoreLib.dll`, which in turn means that the AOT DSOs
such as `libaot-System.Private.CoreLib.dll.so` must *also* be loaded.
The loading of the AOT DSOs is why JIT is fastest here (no AOT DSOs),
and why Profiled AOT is faster than Full AOT (smaller DSOs).
~~ Plain Xamarin.Android: Displayed Time ~~
| Before ms | After ms | Δ | Notes |
| --------: | --------: | -----------: | ------------------------------------- |
| 289.300 | 251.000 | -13.24% ✓ | defaults; full AOT+LLVM; 64-bit build |
| 286.300 | 252.900 | -11.67% ✓ | defaults; full AOT; 64-bit build |
| 285.700 | 255.300 | -10.64% ✓ | defaults; profiled AOT; 32-bit build |
| 282.900 | 255.800 | -9.58% ✓ | defaults; full AOT+LLVM; 32-bit build |
| 286.100 | 256.500 | -10.35% ✓ | defaults; full AOT; 32-bit build |
| 286.100 | 258.000 | -9.82% ✓ | defaults; profiled AOT; 64-bit build |
| 328.900 | 310.600 | -5.56% ✓ | defaults; 32-bit build |
| 319.300 | 313.000 | -1.97% ✓ | defaults; 64-bit build |
~~ Plain Xamarin.Android: Total native init time (before `OnCreate()`) ~~
| Before ms | After ms | Δ | Notes |
| --------: | --------: | -----------: | ------------------------------------- |
| 59.768 | 42.694 | -28.57% ✓ | defaults; profiled AOT; 64-bit build |
| 60.056 | 42.990 | -28.42% ✓ | defaults; profiled AOT; 32-bit build |
| 65.829 | 48.684 | -26.05% ✓ | defaults; full AOT; 64-bit build |
| 65.688 | 48.713 | -25.84% ✓ | defaults; full AOT; 32-bit build |
| 67.159 | 49.938 | -25.64% ✓ | defaults; full AOT+LLVM; 64-bit build |
| 67.514 | 50.465 | -25.25% ✓ | defaults; full AOT+LLVM; 32-bit build |
| 66.758 | 62.531 | -6.33% ✓ | defaults; 32-bit build |
| 67.252 | 62.829 | -6.58% ✓ | defaults; 64-bit build |
The desktop mono version has been bumped to include a major MSBuild
update from 16.6 to 16.9. This MSBuild version supports net5.0, and
as such it allows us to add `Microsoft.Android.Sdk.ILLink` to the
`Xamarin.Android.sln` build.
Context: https://github.com/xamarin/xamarin-android/pull/5529
Commit dc1f7ba enabled ESRP signing for all .NET 6 artifacts, and
we've been asked to apply these changes to our stable `.pkg` installer
which is using a deprecated Apple Developer account and signing logic.
The macOS build job has been updated to sign and harden all Mach-O
files included in both our stable and .NET 6 `.pkg` files. This was
previously happening in two different passes, the first for the
stable `.pkg` content during the build job and the second for the
.NET 6 `.pkg` content during the `dotnet_create_pkg` job.
Signing of the stable `.pkg` has been migrated to use the ESRP
signing service via the tasks/targets imported by the new MicroBuild
package reference.
Our previous notarization preparation logic required us to unpack
`bundletool.jar`, and sign the version of `aapt2` that is bundled
inside. We now instead remove all embedded versions of `aapt2`,
as we pass in the path our own bundled `aapt2` via
`bundletool.jar --aapt2 /path/to/our/aapt2`.
Commits 864bfd53, 315c0d0c, 761ff535, and others, have updated various
"long-form" projects to SDK-style projects.
Visual Studio uses a different GUID for SDK-style projects.
Similar to commit 1fa49e37, allow Visual studio to update
`Xamarin.Android.sln`, so that the correct project type GUIDs are
used, replacing `FAE04EC0-301F-11D3-BF4B-00C04F79EFBC` (long form) to
`9A19103F-16F7-4668-BE54-9A1E7A4F7556` (SDK-style).
These were the final projects that needed to be migrated to
SDK-style, so this should be the final churn on the `.sln`.
Create a new Windows build lane that Prepares and builds
`Xamarin.Android.sln` using `dotnet build` instead of using the
.NET Framework.
Currently this is only implemented for Windows.
Run `dotnet xaprepare.dll` instead of `xaprepare.exe`.
Remove `tools/xabuild/xabuild.csproj` from `Xamarin.Android.sln` and
instead build it as a separate step using .Net Framework/Mono.
This project has issues due to its extensive "creative" use of MSBuild.
Some fixups to `create-vsix`: by default .Net Core only serializes
string resources. Added MSBuild prop and needed NuGet to support
other resource types.
By default .Net Core contains fewer encoding standards.
Add `System.Text.Encoding.CodePages` to `xaprepare` to add support
for `iso-8859-4`, needed by `Kajabity`.
Future work needed:
- This is everything needed to build `Xamarin.Android.sln`, but not
everything needed by CI. We still call:
- `msbuild create-vsix`
- `msbuild Xamarin.Android.sln /t:PackDotNet` (requires .NET 6)
- `msbuild xabuild`
- `xabuild Xamarin.Android-Tests.sln`
- Some executables that we build and run like `api-xml-adjuster`
are still running their `.exe` form.
- macOS support. macOS still uses `mono` and not `dotnet`.
Use the latest `6.0.0-alpha.1.21109.1` version of `Microsoft.NET.ILLink`
linker SDK.
Target `net6.0` when building our custom steps.
We cannot build it with `msbuild` on Mac as we don't have version 16.8
or newer. So instead build the `Microsoft.Android.Sdk.ILLink` project
with `dotnet build`, just before packaging our net6 nugets.
Add a new utility which can dump the typemap information found within
native libraries and `.apk` files.
The new utility is accessible within the build tree as
`bin/$(Configuration)/bin/tmt`, e.g. `bin/Debug/bin/tmt`.
This utility is *not* part of the installer; it's currently
intended primarily for internal use.
See `tools/tmt/README.md` for usage details.
Context: https://github.com/xamarin/xamarin-android-tools/pull/101
Context: https://github.com/xamarin/monodroid/pull/1145
We would like to eventually remove the dependency that xamarin/monodroid
has on xamarin/xamarin-android. As a first step towards this effort,
shared MSBuild code is being moved to xamarin/xamarin-android-tools.
`Xamarin.Android.Build.Tasks` has been updated to use the new shared
`Microsoft.Android.Build.BaseTasks` sources. All of the files that were
moved to xamarin/xamarin-android-tools have been deleted.
The inverted monodroid build has been replaced with the new monodroid
build system, see https://github.com/xamarin/monodroid/pull/1145 for
more context. The one build task test that was still being ran from the
xamarin/monodroid repo has been moved into `DebuggingTasksTests.cs`,
which is conditionally included in `Xamarin.Android.Build.Tests`.
Tests associated with the build task source that was moved to
xamarin/xamarin-android-tools have also been moved to a new
`Microsoft.Android.Build.BaseTasks-Tests` assembly in that repo.
Our solution files no longer match what Visual Studio 2019 generates
when it saves the file. Although either version of the solution files
work, they cause the local tree to be dirty, and can obscure actual
intended changes to the solution file.
Load the current `.sln` files within Visual Studio 2019 16.8, and
save them. This "canonicalizes" the file contents.
The main changes are project types changing from
`FAE04EC0-301F-11D3-BF4B-00C04F79EFBC` to
`9A19103F-16F7-4668-BE54-9A1E7A4F7556`. The former is the project type
for long form C# projects and the latter is the project type for
SDK-style projects.
There are also some additional items Visual Studio wants to specify
regarding Shared Projects.
Commit d236af54 causes assemblies within `.apk` files to be
*separately* compressed. Consequently, if you try to extract an
assembly from a `.apk` and read it, it fails:
% cd samples/HelloWorld'
% msbuild /t:SignAndroidPackage /p:Configuration=Release HelloWorld.csproj
% unzip bin/Release/com.xamarin.android.helloworld-Signed.apk assemblies/HelloWorld.dll
% ikdasm assemblies/HelloWorld.dll
Unhandled Exception:
IKVM.Reflection.BadImageFormatException: Exception of type 'IKVM.Reflection.BadImageFormatException' was thrown.
% file assemblies/HelloWorld.dll
assemblies/HelloWorld.dll: Sony PlayStation Audio
(The `file` output is rather humorous…)
The `assemblies/*.dll` files still contain IL, they're just LZ4
compressed following a `CompressedAssemblyHeader`.
A way to read the assemblies is to instead use the build tree,
using e.g. `obj/Release/android/assets/HelloWorld.dll`.
Instead of requiring the build tree, add a new `decompress-assemblies`
utility which is able to uncompress the assemblies.
Usage:
decompress-assemblies FILES*
where `FILES` are `.dll` files, e.g. as extracted via `unzip` (above),
`.apk` files, or `.aab` files. If `.apk` or `.aab` files are provided,
then all `assemblies/*.dll` files within the `.apk` or `.aab` will be
extracted and uncompressed.
The uncompressed result is stored in the current directory with the
original assembly name modified to contain the `uncompressed-` prefix,
in order to prevent accidental overwriting of assemblies found in the
current directory.
Fixes: https://github.com/xamarin/xamarin-android/issues/4789
Context: 7574f16600
Context: https://github.com/xamarin/xamarin-android/issues/5200
What do we want? Updated documentation! Better Bindings!
How do we get that? Uh…
Historically, [Xamarin.Android API docs][0] were produced by parsing
Android's HTML documentation from `docs-24_r01.zip` and converting it
into [**mdoc**(5) documentation][1] via `tools/javadoc2mdoc`.
The problem is that Google hasn't released an updated `docs*.zip`
package since API-24 (~6 years ago), and thus our documentation is
woefully out of date.
We could have scraped developer.android.com/reference within that
time frame, but web scraping is annoying, and we'd still have to deal
with the pain of parsing HTML.
There is an alternative, though: with API-30, there is a new `sources`
package which contains the Java source code used for the API level,
which in turn contains Javadoc source code comments. Previous API
levels similarly contained an `android-stubs-src.jar` file which
likewise contained Java source code containing Javadoc comments.
The new approach is to:
1. Update `build-tools/xaprepare` to install the `sources` package.
2. Use [`java-source-tool.jar`][2] to parse the Java source code,
creating an `android-javadoc.xml` file.
3. [Update `generator`][3] to consume `android-javadoc.xml` and
convert the Javadocs into [C# XML documentation comments][4].
4. Update `src/Mono.Android` so that `generator` will emit
C# XML doc comments, and then produce a `Mono.Android.xml` file.
The result is a `Mono.Android.xml` file which contains imported
Android Javadoc documentation, e.g.
<doc>
<assembly><name>Mono.Android</name></assembly>
<members>
<member name="T:Java.Lang.Object">
<summary>Class <c>Object</c> is the root of the class hierarchy.</summary>
<remarks>
<para>Class <c>Object</c> is the root of the class hierarchy.
…
"Later", `Mono.Android.xml` can then be used alongside
[`mdoc update --import=Mono.Android.xml`][5] to update our published
API documentation.
Generation of `Mono.Android.xml` is disabled by default on CI PR
builds, as generating `Mono.Android.xml` increases `src/Mono.Android`
build times by an unacceptable amount.
With the inclusion of `java-source-tools.jar`, we can now fix the
TODO mentioned in commit 380e95e3, and re-add support for
`@(JavaSourceJar)` when running under JDK 11.
Update `Xamarin.Android.Bindings.Core.targets` so that before
invoking `generator`, we first run `java-source-utils.jar` on the
`@(JavaSourceJar)` files, producing Javadoc XML files.
The `<BindingsGenerator/>` task in turn is updated to accept a new
`BindingsGenerator.JavadocXml` property, which is converted into a
`generator --with-javadoc-xml=FILE` option.
The `@(JavaSourceJar)` item group is "extended" to support the
following item metadata:
* `%(CopyrightFile)`: A path to a file that contains copyright
information for the Javadoc contents, which will be appended to
all imported documentation.
* `%(UrlPrefix)`: A URL prefix to support linking to online
documentation within imported documentation.
* `%(UrlStyle)`: The "style" of URLs to generate when linking to
online documentation. Only one style is currently supported:
`developer.android.com/reference@2020-Nov`.
For .NET 6 ("One .NET") integration purposes, provide a default item
group of `@(JavaSourceJar)` which includes `**\*-source*.jar` and
`**\*-src.jar`. This will allow `dotnet build` of an "Android Java
Library Binding" project (`dotnet new android-bindinglib`) to
automatically process `.java` source code for documentation
translation purposes.
Add a new `$(AndroidJavadocVerbosity)` MSBuild property, which
controls "how much" of the Javadoc comments are converted into C#
XML documentation. Supported values are:
* `full`: Convert as much Javadoc as possible.
* `intellisense`: Only emit XML documentation for IDE-useful
constructs: summary, parameters, returns, exceptions.
The difference between the two is *build time* impact: `full` takes
longer, and thus may not always be desirable.
Finally, add a new `$(_UseLegacyJavadocImport)` MSBuild property
which *disables* use of `java-source-utils.jar` for Javadoc importing,
and instead use the previous, legacy, doesn't work on JDK 11,
approach of using `javadoc` and HTML parsing. This shouldn't be
needed, but just in case it *is* needed…
[0]: https://github.com/xamarin/android-api-docs
[1]: http://docs.go-mono.com/?link=man%3amdoc(5)
[2]: 69e1b80afd
[3]: https://github.com/xamarin/java.interop/pull/687
[4]: https://docs.microsoft.com/dotnet/csharp/codedoc
[5]: http://docs.go-mono.com/?link=man%3amdoc-update(1)
Fixes? https://work.azdo.io/1148474
Fixes? https://developercommunity.visualstudio.com/content/problem/1086793/我用xamarin-开发android-后进行打包发布apk但升级后打包报密码错误.html
Context: 380e95e340
Context: https://issuetracker.google.com/issues/150189789
Context: 0031300d95
Context: 3f51412d5f
Visual Studio has a [Xamarin Android SDK Manager][0], which allows
installing and updating the various Android SDK packages.
Historically, two "repositories" were available: the Microsoft
repository, and the Google repository. The Microsoft repository was
curated, containing only "known good" package versions, while the
Google repository was whatever Google was providing "now", which could
contain "bad" package versions.
"Bad" package versions such as Build-tools r30+, which requires
JDK 11; see also <https://issuetracker.google.com/issues/150189789>.
Visual Studio 16.7 (Windows) and 8.7 (macOS) updated the SDK Manager
to *remove* the repository selection, showing *only* the Google repo.
The result is that if someone goes to the SDK Manager window and
clicks the **Tools** tab, they'll (currently) be presented with the
options:
- [ ] Android SDK Build-Tools 30.0.2
- [ ] Android SDK Build-Tools 30.0.1
- [ ] Android SDK Build-Tools 30
- …
Should that someone proceed to (1) install the
Android SDK Build-Tools 30+ package, and (2) *uninstall* all older
package versions, they will have broken their Xamarin.Android build,
because Build-Tools 30+ requires JDK 11, while most (all?) customers
will have JDK 8 installed and *not* JDK 11 installed. The result:
Task "AndroidApkSigner"
Task Parameter:ApkSignerJar=%HOME%\android-toolchain\sdk\build-tools\30.0.0-rc4\lib\apksigner.jar
…
Task Parameter:ToolPath=C:\Program Files\Android\jdk\microsoft_dist_openjdk_1.8.0.25\bin
Task Parameter:ManifestFile=obj\Release\android\AndroidManifest.xml
C:\Program Files\Android\jdk\microsoft_dist_openjdk_1.8.0.25\bin\java.exe -jar %HOME%\android-toolchain\sdk\build-tools\30.0.0-rc4\lib\apksigner.jar sign --ks "%HOME%\AppData\Local\Xamarin\Mono for Android\debug.keystore" --ks-pass pass:android --ks-key-alias androiddebugkey --key-pass pass:android --min-sdk-version 21 --max-sdk-version 29 "C:\A\vs2019xam00000Y-1\_work\1\s\bin\TestRelease\temp\BuildAotApplication AndÜmläüts_x86_64_True_True\bin\Release\UnnamedProject.UnnamedProject-Signed.apk"
java.lang.UnsupportedClassVersionError: com/android/apksigner/ApkSignerTool has been compiled by a more recent version of the Java Runtime (class file version 53.0), this version of the Java Runtime only recognizes class file versions up to 52.0
at java.lang.ClassLoader.defineClass1(Native Method)
…
…\Xamarin.Android.Common.targets(2559,2): error MSB6006: "java.exe" exited with code 1.
How do we fix this?
We are discussing bringing back the Microsoft repository, so that we
can (once again) have a set of Known Good, Won't Break My Machine
Android SDK package versions.
In the meantime, a shorter-term fix is to bundle an `apksigner.jar`
which can run on JDK 8, and use that where needed.
Add a Git Submodule to [apksig][1], which contains the `apksigner.jar`
source code, and update the xamarin-android build so that
`apksigner.jar` is built within a JDK 8 environment.
Then I changed the `<ResolveAndroidTooling/>` task so it no longer was
responsible for setting `$(ApkSignerJar)` and `$(AndroidUseApkSigner)`.
I set up these properties in a similar way as we do for aapt2, r8, etc.
TODO:
* I need to make changes to xamarin/androidtools, so it can find
and use the `apksigner.jar` bundled with Xamarin.Android.
[0]: https://docs.microsoft.com/en-us/xamarin/android/get-started/installation/android-sdk?tabs=windows
[1]: https://android.googlesource.com/platform/tools/apksig
instead of building it.
As the [apkdiff](https://github.com/radekdoulik/apkdiff) is now distributed as
[nuget](https://www.nuget.org/packages/apkdiff/) and can be installed as dotnet tool, it doesn't
make sense to have it in the XA repo anymore.
The CI yaml was updated to install and use the `apkdiff` through the `dotnet` tool. The sources were
removed from the tree and `xaprepare` updated.
Implements https://github.com/xamarin/xamarin-android/issues/4707
With .NET5 we are switching from XA linker implemented as msbuild task to ILLink tool with [custom steps](https://github.com/mono/linker/blob/master/docs/custom-steps.md) provided in an assembly.
The new assembly with our custom steps is named `Microsoft.Android.Sdk.ILLink` and the source code is located in `src/Microsoft.Android.Sdk.ILLink`. It is used during build by `ILLink` tool.
The initial support is already able to link and run simple XA and XA/XF samples.
Future work
* replace reflection usage in SetupStep https://github.com/xamarin/xamarin-android/issues/4709
* audit reflection usage in our code https://github.com/xamarin/xamarin-android/issues/4708
* add more customs steps - we are still missing MonoDroidMarkStep
Notes
* the Profile API is not available in the linker [public API](https://github.com/mono/linker/tree/master/src/linker/ref). we work around it by adding `src/Microsoft.Android.Sdk.ILLink/Profile.cs` with missing pieces.
* `TypeDefinition.GetMethods` extension method from `Mono.Tuner.MethodBodyRocks` is not public. It is very simple so we are inlining it.
* the ILLink is now enabled in `Release` configuration defaults.
* tests were updated to reference more packages, to not miss second level dependencies.
Initial results:
apkdiff output summary for HelloAndroid sample, comparing `Debug` and `Release` apk's
```
- 7,300 Davik executables -2.34% (of 311,436)
- 14,733,944 Assemblies -31.85% (of 46,265,064)
- 2,492,452 Shared libraries -11.43% (of 21,803,164)
- 15,528,456 Package size difference -28.70% (of 54,114,462)
```
Context:
* `_RunILLink` target https://github.com/dotnet/sdk/blob/master/src/Tasks/Microsoft.NET.Build.Tasks/targets/Microsoft.NET.ILLink.targets
* linker documentation https://github.com/mono/linker/tree/master/docs
Co-authored-by: Jonathan Peppers <jonathan.peppers@microsoft.com>
Co-authored-by: Jonathan Pryor <jonpryor@vt.edu>
Context: b00e644e0d
Changes: 186174c037...b00e644e0d
* xamarin/java.interop@b00e644: [Java.Interop.Tools.Generator] Create a v2 version of map.csv (#646)
* xamarin/java.interop@1708d8a: [XAT.Bytecode] Bind Kotlin internal interfaces as package-private (#645)
In xamarin/java.interop@b00e644e, we added support for a "v2" version
of the `map.csv` file format used to bundle Java constant fields into
enumerations.
Convert `src/Mono.Android/map.csv` to use this new format to make
enumification easier going forward.
This does not add any new enumifications.
Additionally, the new enum code standardizes on using `$` to denote
nested classes, e.g. `java/lang/Thread$State`, not
`java/lang/Thread.State`. `$` is correct JNI syntax or nested types.
~Half of this existing file used `$` and half used `.`. This resulted
in needing `acceptable-breakage` fixes as this signature changed in the
`[IntDefinition]` attribute.
Changes: 1a086ffd51...56c92c7088
* xamarin/Java.Interop@56c92c7: [build] Remove cecil submodule (#597)
* xamarin/Java.Interop@3091274: [build] Provide a default $(Configuration) value (#612)
* xamarin/Java.Interop@cf3e7c2: [generator] Don't process duplicate reference assemblies (#611)
* xamarin/Java.Interop@f5fa462: [jnienv-gen] Convert to SDK-style (#608)
Of particular note is [xamarin/Java.Interop@56c92c7][0], which
replaces the `mono/cecil` submodule within Java.Interop with the
[`Mono.Cecil` NuGet package][1] in an effort to simplify the
Java.Interop build system.
This simplifies the Java.Interop repo, and we *thought* that since
xamarin-android *doesn't even use* Java.Interop's cecil submodule-built
`Mono.Cecil.dll` -- instead the `Mono.Cecil.dll` from the
"mono archive" is "renamed" to `Xamarin.Android.Cecil.dll` during
`make prepare` (0c9f83b7) -- surely this would be a simple change.
The removal of the cecil submodule also required changing
`ThirdPartyNotice.txt` generation so that the LICENSE for Cecil was
obtained from the mono archive instead of from Java.Interop.
Unfortunately, the integration was a tad more complicated than
anticipated. With the ongoing adoption of MSBuild multi-targeting
and builds against the `netcoreapp3.1` target framework -- commit
e2854ee7 and numerous commits in Java.Interop -- we encountered a
problem with MSBuild semantics: If two `$(TargetFramework)` builds
share the same output directory, the `IncrementalClean` target will
*remove files created by previous builds*, e.g. when e.g.
`Java.Interop/tools/generator.csproj` builds the `netcoreapp3.1`
framework, it will *delete* the `generator.exe` built by the `net472`
framework, which results in subsequent build breaks.
The only path to sanity is to *ensure* that different
`$(TargetFramework)` builds have *completely separate* `$(OutputPath)`
values. The "normal" approach to doing this is for `$(OutputPath)`
to end with `$(TargetFramework)`, which is the case when
`$(AppendTargetFrameworkToOutputPath)`=True (the default).
Unfortunately in xamarin-android we don't want `$(OutputPath)` to end
with `$(TargetFramework)`; we want the build tree structure to mirror
the installation directory structure, which -- at present -- doesn't
mention `$(TargetFramework)` at all.
The solution here is to use "non-overlapping" directories. For
example, in e2854ee7 there are "two" `$(OutputPath)` values:
* `MonoAndroid10.0`: `bin/Debug/lib/xamarin.android/xbuild-frameworks/MonoAndroid/v10.0/Mono.Android.dll`
* `netcoreapp3.1`: `bin/Debug/lib/xamarin.android/xbuild-frameworks/Xamarin.Android.App/netcoreapp3.1/Mono.Android.dll`
The same "non-overlapping directories" strategy needs to be extended
to *all* multi-targeted projects from Java.Interop, *including*
dependencies. Dependencies such as `Xamarin.Android.Cecil.dll`.
Define a new `$(UtilityOutputFullPathCoreApps)` MSBuild property so
that Java.Interop "utility" project builds, when building for the
`netcoreapp3.1` framework, use a *different* `Xamarin.Android.Cecil.dll`
than is used with the `net472`-related builds.
Update `xaprepare` to *create* this new `netcoreapp3.1`-correlated
`Xamarin.Android.Cecil.dll`. It's the same file, just in a different
directory, to prevent "accidental" deletes by `IncrementalClean`.
Even with all that, MSBuild still had other ideas. In particular,
MSBuild wasn't particularly happy about our attempt to use the
`$(UtilityOutputFullPath)` property to "switch" between using a
`@(PackageReference)` to the Mono.Cecil NuGet package vs. using a
`@(Reference)` to the `Xamarin.Android.Cecil.dll` assembly, because
MSBuild *caches* this information somewhere within `obj` directories.
To get MSBuild to re-evaluate it's assembly reference choices, we must
instead replace `msbuild` with `msbuild /restore`.
Which still isn't enough, because some of our MSBuild invocations are
via the `<MSBuild/>` task, within `msbuild`. To get *that* working,
we need to explicitly invoke the `Restore` target through a *separate*
`<MSBuild/>` task invocation. You ***CANNOT*** use
`<MSBuild Targets="Restore;Build" />`, as "obvious" as that may be,
because it [doesn't work reliably][2]. ([Yet.][3])
[0]: 56c92c7088
[1]: https://www.nuget.org/packages/Mono.Cecil/0.11.2
[2]: https://github.com/microsoft/msbuild/issues/3000#issuecomment-417675215
[3]: https://github.com/microsoft/msbuild/issues/2811
`check-boot-times` is a tool to be used by the Hyper-V team to be
able to compare performance improvement progress.
Example of a run:
Testing emulator startup times for 1 execution(s). This may take several minutes.
Acceleration type: 0, WHPX (10.0.19041) is installed and usable.
Android emulator version 29.3.5.0 (build_id 6120838) (CL:N/A)
XamarinPerfTest Average Hot Boot Time for 1 run(s) out of 1 request(s): 4224 ms
XamarinPerfTestPixelWithSkin Average Cold Boot Time for 1 run(s) out of 1 request(s): 159600 ms
XamarinPerfTestPixelWithSkin Average Hot Boot Time for 1 run(s) out of 1 request(s): 2695 ms
Testing emulator startup times for 1 execution(s). This may take several minutes.
Acceleration type: 0, HAXM version 7.5.4 (4) is installed and usable.
Android emulator version 29.3.5.0 (build_id 6120838) (CL:N/A)
XamarinPerfTest Average Cold Boot Time for 1 run(s) out of 1 request(s): 23834 ms
XamarinPerfTest Average Hot Boot Time for 1 run(s) out of 1 request(s): 4021 ms
XamarinPerfTestPixelWithSkin Average Cold Boot Time for 1 run(s) out of 1 request(s): 41912 ms
XamarinPerfTestPixelWithSkin Average Hot Boot Time for 1 run(s) out of 1 request(s): 4584 ms
It also helps us track percentage of times emulator failed to boot.
Updates `Xamarin.Android.Build.Tests.csproj` to a short-form style
project targeting `net472`. The `Xamarin.Android.Build.Tests.Shared`
project files have been removed, as they only appeared to be referenced
by `Xamarin.Android.Build.Tests.csproj`. The `Compile` items in that
project also no longer need to be explicitly defined, so there doesn't
seem to be a reason to keep the project around anymore.
Removes the `create-bundle.csproj` reference from `Xamarin.Android.sln`.
Removes an empty `packages.config` file from `xabuild`.
Removes the empty and unused `Xamarin.ProjectTools.Windows.projitems`.
Use the Google Manifest Merger tooling to merge library
`AndroidManifest.xml` files into the App's `AndroidManifest.xml`.
This should allow developers to override and remove some unwanted
permissions which are currently automatically imported from some of
the support libraries.
Documentation on how to use the new manifest merger tooling is at:
https://developer.android.com/studio/build/manifest-merge
Because of the limitations in Command line argument lengths under
Windows we have to build our own entry point for the manifest merger
tool. Our entry point takes a single "response" file as an input,
with each argument needs to be on a separate line for example:
--main
AndroidManifest.xml
--libs
obj\Debug\lp\0\ji\AndroidManifest.xml:obj\Debug\lp\2\ji\AndroidManifest.xml
--out
android\AndroidManifest.xml
We have to put each argument on its own line because of the way the
built in Manifest Merger Argument parser works.
Xamarin.Android.sln has two issues that make `Xamarin.Android.sln`
not work very well in VS Windows:
* It builds MSBuild tasks and uses them within the same solution,
such as `xa-prep-tasks.csproj` and
`Xamarin.Android.Tools.BootstrapTasks.csproj`.
* We have conditional `@(ProjectReference)`s.
To improve things, I completely removed the `$(XAIntegratedTests)`
MSBuild property (1b72a184) and removed any `@(ProjectReference)`
that were conditional.
I moved the two bootstrap `.csproj` files to a new
`Xamarin.Android.BootstrapTasks.sln` which gets built during
`make prepare` or `msbuild Xamarin.Android.sln /t:Prepare`.
Visual Studio on Windows was also complaining about the
`CreateManifestResourceNames` target missing for projects that merely
import `Microsoft.Common.targets` such as `r8.csproj`. I added an
empty target in `Configuration.props` to fix all cases of this.
Unfortunately, things are still not perfect, as you can hit errors
such as:
Unable to copy file "obj\Debug\Xamarin.Android.Build.Tasks.dll" to "C:\src\xamarin-android\bin\Debug\lib\xamarin.android\xbuild\Xamarin\Android\Xamarin.Android.Build.Tasks.dll".
The process cannot access the file 'C:\src\xamarin-android\bin\Debug\lib\xamarin.android\xbuild\Xamarin\Android\Xamarin.Android.Build.Tasks.dll' because it is being used by another process.
Could not copy "obj\Debug\Xamarin.Android.Build.Tasks.dll" to "C:\src\xamarin-android\bin\Debug\lib\xamarin.android\xbuild\Xamarin\Android\Xamarin.Android.Build.Tasks.dll".
Exceeded retry count of 10. Failed. The file is locked by: "Microsoft Visual Studio 2019 (26660), MSBuild.exe (18800), MSBuild.exe (17416)" Xamarin.Android.Build.Tasks
This occurs if you modify `Xamarin.Android.Build.Tasks.csproj`,
because VS Windows keeps MSBuild nodes alive, and if they have
previously loaded `Xamarin.Android.Build.Tasks.dll` then the assembly
is locked.
I am not yet sure if we can get around this remaining problem, but we
can revisit in the future.
Changes: 63bbeb076d...ba0a823f07
Changes: 2fb054e...d78c6b2
Changes: 4c4daa5281...c092183e87
Bumps to xamarin/monodroid/master@ba0a823f:
* Bumps to xamarin/android-sdk-installer/master@c092183e
* Bumps to xamarin/androidtools/master@d78c6b2
The [`LibZipSharp` repo][0] is now available as the
[Xamarin.LibZipSharp NuGet package][1].
Remove the submodules for libzip and LibZipSharp in favour of using
the nuget package.
The `Xamarin.LibZipSharp` package contains the native binaries for
`libzip`, however by default the NuGet package does NOT copy these
into `$(OutputPath)`. In order for that to happen the following
MSBuild property needs to be set:
<LibZipSharpBundleAllNativeLibraries>True</LibZipSharpBundleAllNativeLibraries>
When `$(LibZipSharpBundleAllNativeLibraries)`=True, the native
binaries will be copied to the `$(OutputPath)\@PLATFORM@` directories
where `@PLATFORM@` will be `Darwin`, `Linux` or `Windows`.
This means we need to update our build system to make sure we include
the files from the correct locations. We also need to make sure we
call the `Restore` target on projects using `LibZipSharp` now as the
package will need to be restored.
Another change is that we have removed the versioning from the
`libzip` native libraries: they used to be e.g. `libzip.5.0.dylib`
but are now just `libzip.dylib`. This was done upstream in the
`Xamarin.LibZipSharp` package itself to allow for support for .NET
Core based `DllImport` which do NOT use a `.dll.config` file, but
instead use only the dll name (e.g `libzip`).
We are also including the `Linux` version of the binaries in the
installer.
[0]: https://github.com/xamarin/LibZipSharp
[1]: https://www.nuget.org/packages/Xamarin.LibZipSharp/
Changes: be581591d0...60e85b0424
Context: https://github.com/xamarin/java.interop/pull/459
Updates `generator` so that all bound Java interfaces also implement
`IJavaPeerable` in addition to `IJavaObject`, for eventual future
C#8 Default Interface Member support.
[generator] Remove extraneous slash when creating `.projitems`.
[generator] Always use `XAPeerMembers` for `XAJavaInterop1`
Drop dependency on DylibMono when building for Xamarin.Android (#3223)
[jnienv-gen] fix p/invoke usage for .NET framework
Add `jnimarshalmethod-gen.exe -r ASSEMBLY` option.
Improve support for binding package-private interfaces.
Parse `EnclosingMethod`, `SourceFile` attribute blobs.
Emit events for `addListener(Listener,Handler)` pattern.
Fix `jnimarshalmethod-gen.exe`-related build error introduced by
having bound interfaces implement `IJavaPeerable`:
Instance property 'PeerReference' is not defined for type 'Android.Widget.IListAdapter'
Parameter name: propertyName
System.ArgumentException: Instance property 'PeerReference' is not defined for type 'Android.Widget.IListAdapter'
Parameter name: propertyName
at System.Linq.Expressions.Expression.Property (System.Linq.Expressions.Expression expression, System.String propertyName)
at Java.Interop.JavaPeerableValueMarshaler.CreateIntermediaryExpressionFromManagedExpression (Java.Interop.Expressions.JniValueMarshalerContext context, System.Linq.Expressions.ParameterExpression sourceValue)
at Java.Interop.JavaPeerableValueMarshaler.CreateReturnValueFromManagedExpression (Java.Interop.Expressions.JniValueMarshalerContext context, System.Linq.Expressions.ParameterExpression sourceValue)
Added `external/Java.Interop/build-tools/jnienv-gen.csproj` to
`Xamarin.Android.sln` so that it builds properly.
Use 7zip to compress our mono bundle. With this change the size goes
down a LOT:
750524441 Jun 7 20:11 'bundle-v21-h9f52cd0-Debug-Linux-libzip=b95cf3f,mono=c6edaa6.zip'
160771876 Jun 7 22:12 'bundle-v21-h9f52cd0-Debug-Linux-libzip=b95cf3f,mono=c6edaa6.7z'
Using 7z results in a file 21% the size of the original!
Additionally, remove the `build-tools/download-bundle` project which
is no longer needed as of 9302d514 and, in the case of this PR, gets
in the way because it doesn't know how to process `.7z` files.
Fixes: https://github.com/xamarin/xamarin-android/issues/2562
The `make prepare` **make** target and `msbuild /t:Prepare` MSBuild
target are used to "prepare" a `xamarin-android` checkout so that
the subsequent `make all` or `msbuild /t:Build` invocation will work.
The preparation step involves ensuring that all known required
dependencies exist or are provisioned (54a7a029), such as the build
utilities (`autoconf`/etc.), cross-compilers (MinGW), the Android SDK
and NDK, and building "enough" to allow `Xamarin.Android.sln` to
build (see also 7343965a0).
Unfortunately, the Glorious Idea™ of using MSBuild for significant
portions of this process is "baroque", making it difficult to
understand and maintain, and the requirement on `msbuild` means
that things break in really bizarre and obscure ways on Linux and
macOS if the installed `mono` version is "too old"...and the
determination of "too old" was being done via `mono` & `msbuild`!
Furthermore, there was duplication and repetition of various bits of
build configuration on Linux and macOS between Makefile and the
MSBuild project system, which made it easier to make mistakes.
(Then there's the occasional bizarro failures that would result from
attempting to evaluate `$(PRODUCT_VERSION)` within `make`; it would
fail when `MONO_LOG_LEVEL=info` was set. Then there's...)
Instead of having `make prepare` be implemented as an unholy mess of
`make` and `msbuild`, introduce the new `build-tools/xaprepare` app.
`xaprepare` will be built and executed as part of `make prepare`,
and takes over responsibility for repo build preparation.
See also: build-tools/xaprepare/README.md
TODO: Windows support
Xamarin.Android needs a lot of information in the runtime in order to
do its tasks. Among this is are two bits of information that are
used during startup: type mappings (java-to-managed `.jm` files and
managed-to-java `.mj` files) and environment settings (variables and
system properties). Historically, Xamarin.Android used these bits of
data (generated on the application build time) by writing them into
separate files (`*.mj`, `*.jm`, `environment`), storing them in the
`.apk`, and reading these files dynamically during startup.
an `environment`
However, none of this data is *actually* dynamic - it's known
beforehand and could be stored as static data right in the `.apk`.
The problem with this idea was that we had no way to put that
information into the application native bits at build time: it would
require the presence of a compiler (or native assembler) as well as
the native linker for the target platforms. This would have required
having the Android NDK installed, which is a hefty requirement.
That said, the Android SDK currently ships a native linker within its
`build-tools` package, and we can also bundle within our packages the
`gas` native assembler for all the architectures we support. This
allows us to generate simple native assembler code, compile it
and link into a shared library ***when packaging*** the application.
The way it works is that during the Xamarin.Android *SDK* build we
build a "stub" `libxamarin-app.so` library which the Xamarin.Android
`libmono-android.*.so` neé `libmonodroid.so` runtime libraries are
linked against. The stub `libxamarin-app.so` library is *not*
distributed with the Xamarin.Android SDK; instead, the "real" version
is generated at build/packaging time and packaged with the runtime
within the `.apk`. Since the "real" version is ABI-compatible with
the stub version, the dynamic loader doesn't care and loads and
resolves symbols just fine.
This allows us to dispense with memory allocation, loading
(potentially large) files from the `.apk` (lots of I/O, slow),
parsing text files to set environment variables (additional memory
allocation), etc. All of the data is loaded for us by Android and we
simply access global variables from the `libxamarin-app.so` library.
The process, however, can be time consuming (sort of...) and in order
to allow fast deployment to work as quickly as possible, the debug
(and desktop/host - for the benefit of the UI Designer) builds of the
runtime also contain code to load both type maps and environment
variables from files stored inside one of the override directories.
The changes provide a speed-up of ~10ms, reducing the startup time
down to around 233ms for a release build of a Xamarin.Forms app on a
Pixel 3 XL.
Context: https://github.com/xamarin/xamarin-android/issues/2608
Bumps to mono/debugger-libs/master@b45303.
Bumps to icsharpcode/nrefactory/master@0607a4.
Add support for testing that the app launches on device *and* can be
interacted with. It does this by making use of some Android shell
features and commands
`adb shell input` allows us to send taps and clicks to the device.
It simulates user input. This can be used to e.g. click a button.
`adb exec-out uiautomator` allows obtaining the complete UI layout
for the currently active Activity. For example, to dump the current
Activity UI layout to the console:
adb exec-out uiautomator dump /dev/tty
This returns a XML:
<hierarchy rotation="0">...
We can parse this XML looking for any ui element we want. We can
then get the bounds of the control and pass that information to
`adb shell input tap`.
Since we are now able to read the displayed UI, we can also add some
`TimeZoneInfo` tests, to ensure that when the timezome is changed:
adb shell su root setprop persist.sys.timezone "America/New_York"
the UI is correctly updated to display the appropriate timezone name.
Finally, add tests to check that the debugger -- via
mono/debugger-libs -- can actually connect to the Application.
The debugger-related tests only run if the commercial bits of the
Xamarin.Android SDK are available.
As all of these new tests *require* that an emulator or device be
available, they have been split out into a new
`tests/MSBuildDeviceIntegration` project. This is built as part of
the main solution. However, it is run as part of the
`make run-apk-tests` target. This is so we can be sure the emulator
is running.
In the future any MSBuild related tests that require a device should
be placed in this new project.
Context: https://github.com/mono/mono/issues/11397
Stop building `netstandard.dll` within the xamarin-android build, and
instead use the `netstandard.dll` which is built by mono and included
in the mono archive (333b98b).
Context: https://github.com/mono/mono/issues/11397
Stop building `System.Drawing.Primitives.dll` within the
xamarin-android build, and instead use the
`System.Drawing.Primitives.dll` which is built by mono and included in
the mono archive (333b98b).
Related: https://github.com/xamarin/designer/issues/1597
There are a few files that *every* project contains, that are present
*in this repo*, and don't really need to be built as part of any
other project; they can instead be built and included with the
installer, thus reducing the need to run `javac` and `dx` during
packaging.
These files include, but are not limited to:
* `IncrementalClassLoader.java`
* `MonkeyPatcher.java`
* `MonoPackageManager.java`
* `MultiDexLoader.java`
* `NotifyTimeZoneChanges.java`
* `ResourcePatcher.java`
Instead of storing these and other files as `@(EmbeddedResource)`s
within `Xamarin.Android.Build.Tasks.dll`, which then need to be
extracted and compiled as part of the app project, compile them into
`java_runtime.jar` and `java_runtime_fastdev.jar` files.
Add a new `src/java-runtime` project which creates the new
`java_runtime.jar` and `java_runtime_fastdev.jar` files. It will
also create the `.dex` files for each of these.
A note about "circular dependencies" and `MonoPackageManager`:
previously, `MonoPackageManager.java` was stored as an
`@(EmbeddedResource)`, extracted, then *appended to* so that we could
generate the `MonoPackageManager_Resources` type, which is referenced
by `MonoPackageManager`. However, `MonoPackageManager` is a type
we'd like to avoid compiling as part of the app project. In order to
support this, we build `MonoPackageManager` against a "dummy"
`MonoPackageManager_Resources` type, then *delete*
`MonoPackageManager_Resources.class` before creating `java_runtime.jar`.
`MonoPackageManager_Resources` is stil generated at build time, and
must be compiled alongside other Java assets such as Java Callable
Wrappers.
There is also an issue with the `Dexes` fast deployment system.
On certain emulators the ART runtime complains if a `.class` is in
multiple `.dex` files. This results in the following error
java.lang.IllegalAccessError: Class ref in pre-verified class resolved to unexpected implementation
and the app will fail to start. This was caused because the classes
`Runtime.class` and IGCUserPeer.class` ended up in both the final
`classes.dex` as well as the `mono.android.dex`. This PR moves the
`Runtime.java` and IGCUserPeer.java` into the new `java_runtime.dex`
and out of `mono.android.dex` so we can avoid this issue. The fast
deployment system will make use of the `java_runtime_fastdev.dex`
as the main `classes.dex` in a fast dev apk. This means we only
include only the required files in the apk to get the app running.
All other files will be resolved from the fast deployment directory.
Using `java_runtime.jar` saves ~190ms from App builds, as the types
within `java_runtime.jar` don't need to be separately compiled.
These are the task execution times over 3 builds of the
`Xamarin.Forms-Performance-Integration` test app:
[Before PR#2536]
2578 ms Javac 1 calls
2474 ms Javac 1 calls
2928 ms Javac 1 calls
Avg: 2660
[With `java_runtime.jar`]
2497 ms Javac 1 calls
2500 ms Javac 1 calls
2424 ms Javac 1 calls
Avg: 2473
One other addition in this PR is the ability to change the name of a
`.dex` file within the Apk. One of the problems we face is the first
`.dex` file in an apk MUST be called `classes.dex`. Our current
`<BuildApk/>` system uses the item Filename for the apk entry. This
works normally because we compile all the java code into
`classes.zip` and then into `classes.dex`. However for the new fast
dev system we will just be using the `java_runtime.dex` file directly
so we need to ability to alter its name in the final apk. Rather
than hardcoding this ability we just need to add the `%(ApkName)`
metadata to the appropriate item in MSBuild and the `<BuildApk/>`
task will now use that for the apk rather than the filename.
Fixes: https://github.com/xamarin/xamarin-android/issues/2627
There are several things broken when building with MSBuild 16.0
and/or using [Visual Studio 2019][0].
~~ TargetFrameworkVersion ~~
Anything referencing MSBuild assemblies must now use
`$(TargetFrameworkVersion)`=v4.7.2, since the MSBuild assemblies are
now targeting v4.7.2.
We should take this as an opportunity to consolidate all
`$(TargetFrameworkVersion)` values, so I moved this property into
`Configuration.props`. I was able to remove the property in many
projects, and the IDE still seemed to work fine.
The only hang up is that some versions of Mono don't have v4.7.2 yet.
I had to use the following trick, which use
[`ToolLocationHelper.GetPathToStandardLibraries()`][1]:
<_StandardLibraryPath>$([Microsoft.Build.Utilities.ToolLocationHelper]::GetPathToStandardLibraries('.NETFramework', 'v4.7.2', ''))</_StandardLibraryPath>
<TargetFrameworkVersion Condition=" '$(TargetFrameworkVersion)' == '' And '$(_StandardLibraryPath)' != '' ">v4.7.2</TargetFrameworkVersion>
<TargetFrameworkVersion Condition=" '$(TargetFrameworkVersion)' == '' ">v4.7.1</TargetFrameworkVersion>
This allows us to fallback to v4.7.1 if we have to.
~~ Prepend<T> ~~
With `$(TargetFrameworkVersion)` changing, this method:
static IEnumerable<T> Prepend<T> (this IEnumerable<T> l, T another)
Now exists in the BCL, so we can remove it.
~~ MSBuild.exe location ~~
In VS 2017, MSBuild is located at:
%VsInstallDir%\MSBuild\15.0\Bin\MSBuild.exe
In VS 2019, it is now located in:
%VsInstallDir%\MSBuild\Current\Bin\MSBuild.exe
Right now we have a bit of code that "finds" Visual Studio, MSBuild,
etc. so we should take this opportunity to improve it.
We originally thought about using MSBuildLocator:
https://www.nuget.org/packages/Microsoft.Build.Locator/https://github.com/Microsoft/MSBuildLocator
But the licensing of the library was concerning... We were uncertain
if we could redistribute MSBuildLocator as part of an OSS product.
For now we can just execute `vswhere.exe`:
https://github.com/Microsoft/vswhere/wiki/Find-MSBuild
I made a simple `vswhere.csproj` we can reference where this
functionality is needed. Currently `xabuild` and
`Xamarin.ProjectTools` need to locate the Visual Studio directory.
~~ Other breakage in xabuild.exe ~~
In VS 2019, the path to Roslyn is a bit odd:
%VsInstallDir%\MSBuild\15.0\Bin\Roslyn
I had to rework things to still work when combined with a different
MSBuild location:
%VsInstallDir%\MSBuild\Current\Bin\MSBuild.exe
Additionally, our binding redirects weren't working in this project at
all. `App.config` was *not* in the csproj!
~~ NUnit ~~
There does not appear to be an NUnit extension available for VS 2019
yet. I frequently use the `Test Explorer` in VS to individually run
tests.
However, it looks like they are switching to a different model for
test frameworks. Each test framework ships its own "adapter" that
enables the testing UIs in Visual Studio. These are shipped on
NuGet, and so they can work without installing any extra extensions.
So we just need this NuGet package:
https://www.nuget.org/packages/NUnit3TestAdapter/
This package should not affect anything on non-Windows platforms.
~~ ILRepack ~~
With the usage of `$(TargetFrameworkVersion)`=v4.7.2 we were hitting:
"src/Xamarin.Android.Build.Tasks/Xamarin.Android.Build.Tasks.csproj" (default target) (1) ->
(ILRepacker target) ->
src/Xamarin.Android.Build.Tasks/ILRepack.targets(24,3): error : Failed to resolve assembly: 'netstandard, Version=2.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51' [src/Xamarin.Android.Build.Tasks/Xamarin.Android.Build.Tasks.csproj]
We need to specify `$(TargetPlatformDirectory)`; path logic in
`<ILRepack/>` appears to be failing, see:
72c4c3bd05/ILRepack.Lib.MSBuild/ILRepack.cs (L148-L151)18698fddc4/ILRepack/ILRepack.cs (L450-L462)
~~ Other repositories ~~
We will need to switch to `$(TargetFrameworkVersion)`=v4.7.2 and use
the NUnit adapter in other repos:
- Downstream in `monodroid`, things will likely break without these
changes.
- `Java.Interop` is currently *working*, but should be updated.
- `xamarin-android-tools` is currently *working*, but should be
updated.
[0]: https://web.archive.org/web/20190307085042/https://visualstudio.microsoft.com/vs/preview/
[1]: e03296fe14/src/Utilities/ToolLocationHelper.cs (L1736-L1747)
In order to support things like `AppBundle` we cannot use the version
of `aapt2` that is shipped with the Android SDK. Google recommends
[downloading it from maven][1].
If we are shipping our own version of `aapt2` then we no longer need
to support using the Android SDK version. If a user really wants to
use the Android SDK version they can set the `$(Aapt2ToolPath)`
MSBuild property manually.
We should also download `bundletool` and ship that too. This will
mean we can rely on both of these being available when we start
looking at `AppBundle` support.
[1]: https://developer.android.com/studio/build/building-cmdline#download_aapt
Fixes: https://github.com/xamarin/monodroid/issues/862
Fixes: https://github.com/xamarin/xamarin-android/issues/2686
Introduce a new `build-tools/create-pkg` project next to
`build-tools/create-vsix` (to be used in a similar way) which creates
a macOS `.pkg` installer for xamarin-android.
The generated `.pkg` file follows the same naming convention as the
`.vsix` package file:
Xamarin.Android.Sdk-OSS-$(VERSION)_$(BRANCH)_$(COMMIT).pkg
`build-tools/installers/create-installers.targets` has been added to
keep track of all of the content we require in both our macOS and
Windows installers. These shared items, properties, and targets are
used by both installer projects to ensure uniformity between the two
installers we produce.
The `ConstructInstallerItems` target determines the majority of the
files which should end up in the installer. Behavior will vary
depending on the existence of certain files or the values of certain
MSBuild properties. This was done intentionally to be able to
support creating installers for PR builds, open source packages, and
commercial packages.
The `$(AndroidFirstFrameworkVersion)`,
`$(AndroidLatestStableFrameworkVersion)`, and
`$(AndroidLatestFrameworkVersion)` MSBuild properties (along with
some hardcoded values) are used to determine which API levels the
installer will support.
The `$(AndroidSupportedTargetJitAbi)` MSBuild property is used to
determine which Android ABIs the installer will support.
The `$(IncludeMonoBundleComponents)` MSBuild property is used to
determine whether or not the components which come from a full mono
integration build should be included in the installers.
A new `make create-installers` target will attempt to package all of
the relevant files for every ABI specified in the `$(ALL_JIT_ABIS)`
variable in `build-tools/scripts/BuildEverything.mk`. This behavior
can be overridden by specifying an empty value for `$(_MSBUILD_ARGS)`
when calling the `create-installers` make rule (as `build.groovy`
does), or by providing an alternate value for
`/p:AndroidSupportedTargetJitAbis` when building either installer
directly from MSBuild.
The `build.groovy` script has been updated to handle creating
different installer types for PR builds and full builds. Our macOS
PR builds will now build a mingw+Windows flavored version of
`libmono-android.debug.dll`, so that it can be included in a `.vsix`
installer and used to extend test coverage for our PR builds in the
future. PR builds will also package a limited set of Android ABI
support as briefly mentioned above.
A new `.pkg` preinstall script has also been added to fix a long-
standing issue in which older Xamarin.Android installations were
never removed, thus causing
`/Library/Frameworks/Xamarin.Android.framework/Versions` to
constantly grow in size. Now, the macOS installer will attempt to
remove the version currently symlinked to
`/Library/Frameworks/Xamarin.Android.framework/Versions/Current`
before running. This will *not* remove any older versions; these
will still need to be manually removed, if desired.
Note further that the directory referenced by the `Current` symlink
will only be removed when the following directory exists:
/Library/Frameworks/Xamarin.Android.framework/Versions/$(readlink Current)
The practical implication is that if the `Current` link points
anywhere "outside" the `Xamarin.Android.framework/Versions` directory
(e.g. `$HOME/xamarin-android/bin/Debug`), the directory will NOT be
removed. This is intentional, to prevent accidental loss of
developer build trees.
Jonathan Peppers
Jonathan Pobst
Marek Habersack
Peter Collins
Jonathan Pryor
Radek Doulik
Jo Shields
Radek Doulik
Gustavo Varo
Dean Ellis
Jonathan Peppers