Context: 059c2c07f0
Commits a7b57688 and 6d2a4bb2 added submodules and updated
`.gitmodules` in a manner *not* in accordance with 059c2c07:
* They used tabs instead of spaces, which means @MSylvia's branch
tooling will cause larger commit diffs for this file
* The entries weren't sorted.
* Branch names weren't specified.
Reformat the entries within `.gitmodules`, sorting the names,
replacing tabs with spaces, and ensuring that all submodules have
a branch name specified.
Context: https://github.com/xamarin/xamarin-android/pull/8800
Context: 8bc7a3e84f
Context: 68368189d6
Context: https://github.com/libunwind/libunwind/issues/702
Begin adding native method call tracing infrastructure based on
[libunwind][0], initially used in tracing the reasons for why
[Blazor][1] apps break when LLVM marshal methods are enabled
(8bc7a3e8, 68368189). The utility of such infrastructure, however,
is beyond just that particular task; it should be available for
general use by both us and applications.
Method call tracing support is not enabled by default.
To enable it, set the `$(_AndroidEnableNativeStackTracing)` MSBuild
property to `true`. This will enable the ability to print native,
managed and Java stack traces by invoking the `monodroid_log_traces`
function from either managed or native application code.
Additionally, change the directory layout for native code.
Previously, `src/monodroid` would produce *five* native libs:
* `libmono-android.debug.so`
* `libmono-android.release.so`
* `libxa-internal-api.so`
* `libxamarin-app.so`
* `libxamarin-debug-app-helper.so`
This made for a large `src/monodroid/CMakeLists.txt`, complicating
maintenance. We had considered trying to move these into separate
`src/LIBRARY-NAME` directories, but it makes things easier with
CMake if we introduce an intermediate directory.
Introduce a new `src/native` directory, to hold native binaries:
* `src/native/monodroid`: new location for `libmono-android.*.so`
* `src/native/xamarin-debug-app-helper`: source for
`libxamarin-debug-app-helper.so`, containing code moved from
previous `src/monodroid`
* `src/native/xamarin-app-stub`: source for the "stub"
`libxamarin-app.so`; the "real" one is produced in the app build.
Some `src/native/*` directories produce libraries (`.so` files). and
some produce *static* archives (`.a` files) to simplify using code
across native libraries.
A [cmake-presets file][2] is processed by `xaprepare`, to create a
`src/native/CMakePresets.json` which contains all the version-specific
bits replacements such as Android API levels, paths to utilities etc.
Developers can create a `src/native/CMakeUserPresets.json` file to
provide local modifications to config options within
`src/native/CMakePresets.json.in`.
Other changes:
* `src/native/libunwind/fixes/aarch64/Gos-linux.c` is an altered
copy of [`libunwind/src/aarch64/Gos-linux.c`][3] in order to
workaround libunwind/libunwind#702.
* *Begin* turning some instance member functions into static member
functions, as we don't otherwise need instances of those classes.
* Begin using `std::source_location` for better crash messages,
instead of using `__FILE__` and `__LINE__`.
* Fix endless recursion from a typo in `Util::ends_with()` overload.
[0]: https://github.com/libunwind/libunwind
[1]: https://learn.microsoft.com/aspnet/core/blazor/hybrid/tutorials/maui?view=aspnetcore-8.0
[2]: https://cmake.org/cmake/help/latest/manual/cmake-presets.7.html
[3]: 9cc4d98b22/src/aarch64/Gos-linux.c
Context: https://xxhash.com/
XXH3 is an improved xxHash algorithm for 64-bit machines, which offers around twice
as much performance over the XXH2 variant (see the link above), especially when processing
small data, at the same time keeping the hash quality intact.
In our case, the processing speed up isn't that dramatic, of course, because we're running
real world applications and not synthetic tests, but we do see startup time improvements.
Performance improvements compared to `main`, tests on Pixel 6 Pro running Android 14:
* **Displayed time**: 2.17% faster
* **Native-to-managed init**: 1.4% faster
* **Total native init**: 1.15% faster
Changes: c53676b080...84c5c67f3e
% git diff --shortstat c53676b0...84c5c67f
48 files changed, 2402 insertions(+), 2175 deletions(-)
Of particular interest:
* mono/debugger-libs@20dce7a: Ported evaluator to roslyn.
mono/debugger-libs@20dce7a updated mono/debugger-libs to use the
Roslyn expression evaluator instead of NRefactory.
Bumping to mono/debugger-libs@84c5c67f allow us to remove the
NRefactory submodule.
Note: we only use `debugger-libs` for tests, we do not ship it or
`NRefactory`.
Context: 6ba99924c9
Updates xaprepare and other relevant build tools to remove installation
of and reference to dependencies required to compile Host OS runtimes
such as mingw and mman-win32. These runtime builds were disabled in
commit 6ba9992, and the dependencies are no longer required.
Fixes: https://github.com/xamarin/xamarin-android/issues/6307
This reverts c50df1c5.
In the past, we had to redistribute `apksigner.jar`, because it
required JDK 11. This happened for build-tools 30.0.0 and was fixed in
built-tools 30.0.3. At the time we were nowhere close able to use JDK
11, and so we built it ourselves for JDK 1.8 and redistributed it.
However, .NET 6+ now *requires* JDK 11, because targets API-31 and
higher. "Classic" Xamarin.Android requires JDK 11 if you are targeting
API-31.
We can remove `apksigner` from `main` going forward in .NET 8.
Other changes:
* Update `XASdkTests.cs` to account for `*.apk.sig` files.
Quick win to remove some components that we aren't shipping with .NET from `main`:
- OpenTK
- sqlite-xamarin
- System.EnterpriseServices
Additionally, this allows us to remove 2 `git` submodules: `mono/opentk` and `xamarin/sqlite`.
* Bump to Guardsquare/proguard@b4ab30a5 [v7.1.1]
Bump from ProGuard v7.0.1 to v7.1.1.
Changes: 912d149394...b4ab30a5b7
Attempts to build proguard without a cache in place have been failing:
FAILURE: Build failed with an exception.
* What went wrong:
Could not determine the dependencies of task ':base:fatJar'.
> Could not resolve all dependencies for configuration ':base:runtime'.
> Could not resolve com.guardsquare:proguard-core:7.0.1.
Required by:
project :base
> Could not resolve com.guardsquare:proguard-core:7.0.1.
> Could not get resource 'https://jcenter.bintray.com/com/guardsquare/proguard-core/7.0.1/proguard-core-7.0.1.module'.
> Could not GET 'https://jcenter.bintray.com/com/guardsquare/proguard-core/7.0.1/proguard-core-7.0.1.module'.
> Read timed out
Proguard v7.1.0 [moves from jcenter to Maven Central][0], which should
fix this issue.
[0]: 87000a49ec
* Remove proguard submodule, use nupkg
* Remove external/proguard references
* Fix line endings
Context: https://sqlite.org/releaselog/3_35_0.html
Context: https://github.com/xamarin/sqlite/compare/3.34.1..3.35.0
The most important upstream changes are:
* Fix a potential NULL pointer dereference when processing a
syntactically incorrect SELECT statement with a correlated WHERE
clause and a "HAVING 0" clause. (Also fixed in the 3.34.1 patch
release.)
* Fix a bug in the IN-operator optimization of version 3.33.0 that
can cause an incorrect answer.
* Fix incorrect answers from the LIKE operator if the pattern ends
with "%" and there is an "ESCAPE '_'" clause.
Context: https://github.com/dotnet/runtime/issues/48416
Context: https://github.com/dotnet/runtime/pull/44505
Context: https://github.com/xamarin/xamarin-android/issues/5537
Update the `src/monodroid` build system to build the native
Xamarin.Android runtime libraries (e.g. `libmono-android.debug.so`)
against the MonoVM shipped from the `dotnet/runtime` repository in
the `Microsoft.NETCore.App.Runtime.*` NuGet packages. These NuGet
packages also contain the header files needed to access and use
MonoVM's embedding API.
We thus ~double the number of libraries built by `src/monodroid`,
with one set of builds against the "legacy" Mono, using Mono from the
SDK archives, and another set of builds against .NET 6 MonoVM.
This change requires the introduction of new target directories in
`$(XAInstallPrefix)xbuild/Xamarin/Android/lib` (e.g.
`bin/Debug/lib/xamarin.android/xbuild/Xamarin/Android/lib`):
* `arm64-v8a-net6`
* `armeabi-v7a-net6`
* `x86_64-net6`
* `x86-net6`
These directories now contain only the shared libraries together
comprising the Xamarin.Android native runtime libraries:
* `libmono-android.*.so`
* `libxamarin-debug-app-helper*.so`
* `libxamarin-app.so`
Mono runtime libraries are *not* copied from their respective NuGet
packages; the .NET 6 NuGet packages are required to obtain them.
Compared to the "legacy" builds, we are missing the
libMonoPosixHelper.so` library. This helper lib will eventually be
placed in a `Mono.Posix` NuGet, once we are able to separate
`Mono.Posix` files from the Mono sources and place them in a
[separate repository][0].
`xaprepare` is modified to generate MSBuild and cmake fragments
responsible for building net6 runtime binaries.
The `libmono-android.*.so` native libraries no longer need to export
any entry points for P/Invoke purposes. Instead, the new
`PINVOKE_OVERRIDE` [runtime property][1] is used to hook into the
P/Invoke lookup process. This allows us to not only hide the public
symbols, but also to remove the `libxa-internal-api.so`
library (d583b7c2) completely.
[0]: https://github.com/mono/mono.posix
[1]: https://docs.microsoft.com/en-us/dotnet/core/tutorials/netcore-hosting#step-3---prepare-runtime-properties
Changes: 0607a4ad96...495405a6ff
* xamarin/NRefactory@495405a6: Merge pull request #2 from jpobst/sdk-style
* xamarin/NRefactory@e4d5997e: Update some project files to SDK-style.
* xamarin/NRefactory@60ac8e99: Merge pull request #1 from xamarin/cecilPackageReference
* xamarin/NRefactory@c2c07caa: Switch Cecil reference to PackageReference.
We are working on using `dotnet build` to build `Xamarin.Android.sln`
and `Xamarin.Android-Tests.sln`. `dotnet build` requires that all
projects be SDK-style short-form projects.
The `tests/MSBuildDeviceIntegration` integration tests use NRefactory
as part of the debugger tests, to ensure that the debugger can attach
to an app on-device. In order for `tests/MSBuildDeviceIntegration`
to be buildable with `dotnet build`, NRefactory must have SDK-style
`.csproj` files.
Unfortunately, the [icsharpcode/NRefactory][0] repo only contains
"long-form" projects, has been archived, and is now read-only.
Switch to the [xamarin/NRefactory][1] repo, which is *not* archived
and is maintained by the VSMac team, and bump to a commit which
includes xamarin/NRefactory@e4d5997e, which migrates many of the
`.csproj` files to SDK-style short-form projects.
This allows us to continue our `dotnet build` explorations.
[0]: https://github.com/icsharpcode/NRefactory
[1]: https://github.com/xamarin/NRefactory
Bump from lz4 v1.9.2 to v1.9.3.
Changes: fdf2ef5809...d44371841a
% git diff --shortstat fdf2ef58...d4437184
94 files changed, 2917 insertions(+), 1377 deletions(-)
Note: lz4/lz4 has no `master` branch. Release tags are for commits
on the `release` branch. Update `.gitmodules` accordingly.
Fixes: https://devdiv.visualstudio.com/DevDiv/_workitems/edit/1139578
Context: https://github.com/xamarin/java.interop/pull/691
Context: https://liquid.microsoft.com/Web/Object/Read/ms.security/Requirements/Microsoft.Security.SystemsADM.10039#guide
Changes: 007b35b489...dcaf794d26
* xamarin/Java.Interop@dcaf794d: [generator] Fix error and warning codes values (#697)
* xamarin/Java.Interop@08ff4db1: [java-interop, Java.Interop] Securely load native libs (#691)
* xamarin/Java.Interop@09c68911: [generator] Clean up generated whitespace (#692)
* xamarin/Java.Interop@d664e90e: [generator] Make warning and error messages localizable. (#689)
For improved security, the guidance is that when loading `.dll` files
on Windows, the default [`LoadLibrary()`][0] behavior is to be
*avoided*, as the [Dynamic-Link Library Search Order][1] includes the
"current working directory" and directories listed in `%PATH%`, both
of which are under control of an "attacker" and not the application.
Instead, [`LoadLibraryEx()`][2] should be used, providing a set of
`LOAD_LIBRARY_SEARCH_*` values which constrain the set of directories
that will be searched for the library (if a full path is not used)
and/or the library's dependencies.
Historically, Xamarin.Android hasn't directly called any
`LoadLibrary()` function. Instead, xamarin-android called
[**dlopen**(3)][3], then used dlfcn-win32/dlfcn-win32@ef7e412d to
implement `dlopen()` in terms of `LoadLibraryEx()`.
Unfortunately, dlfcn-win32/dlfcn-win32@ef7e412d calls `LoadLibraryEx()`
with `LOAD_WITH_ALTERED_SEARCH_PATH`, which while a security
improvement, does not go far enough for our internal requirements.
Migrate away from dlfcn-win32 and instead use the new
`java-interop-dlfcn.h` family of functions added in
xamarin/java.interop@08ff4db1. These new functions appropriately use
`LoadLibraryEx()` with a constrained set of `LOAD_LIBRARY_SEARCH_*`
flags.
*Non-obvious semantic note*: On Mono for all platforms, P/Invoking
`__Internal` is equivalent to trying to load the path `NULL`, e.g.
on macOS `dlopen(NULL, …)`. However, only Android allows a custom
"`loader_func`" to intercept the attempted load of `NULL` and "do
something" with it, which xamarin-android *does* intercept, "remapping"
`NULL` to `libxa-internal-api.so`. On all other platforms,
`loader_func` cannot intercept this invocation.
…which made for an "interesting" interaction: on macOS, if we used:
api_dso_handle = java_interop_lib_load (dso_path.get (), JAVA_INTEROP_LIB_LOAD_LOCALLY, nullptr);
then the macOS Designer integration tests would fail!
Renderer >> 4 [monodroid] Calling into managed runtime init
Renderer (error) >>
Renderer (error) >> Unhandled Exception:
Renderer (error) >> System.EntryPointNotFoundException: java_interop_jnienv_get_java_vm assembly:<unknown assembly> type:<unknown type> member:(null)
Renderer (error) >> at (wrapper managed-to-native) Java.Interop.NativeMethods.java_interop_jnienv_get_java_vm(intptr,intptr&)
Renderer (error) >> at Java.Interop.JniEnvironment+References.GetJavaVM (System.IntPtr jnienv, System.IntPtr& vm) [0x00000] in <0f003a4904fd44d0a8cc6a63962ab40b>:0
Renderer (error) >> at Java.Interop.JniEnvironmentInfo.set_EnvironmentPointer (System.IntPtr value) [0x00037] in <0f003a4904fd44d0a8cc6a63962ab40b>:0
Renderer (error) >> at Java.Interop.JniEnvironmentInfo..ctor (System.IntPtr environmentPointer, Java.Interop.JniRuntime runtime) [0x00006] in <0f003a4904fd44d0a8cc6a63962ab40b>:0
Renderer (error) >> at Java.Interop.JniRuntime..ctor (Java.Interop.JniRuntime+CreationOptions options) [0x0017b] in <0f003a4904fd44d0a8cc6a63962ab40b>:0
Renderer (error) >> at Android.Runtime.AndroidRuntime..ctor (System.IntPtr jnienv, System.IntPtr vm, System.Boolean allocNewObjectSupported, System.IntPtr classLoader, System.IntPtr classLoader_loadClass, System.Boolean jniAddNativeMethodRegistrationAttributePresent) [0x00000] in /Users/builder/azdo/_work/4/s/xamarin-android/src/Mono.Android/Android.Runtime/AndroidRuntime.cs:25
In order for the macOS Designer integration tests to succeed,
`libxa-internal-api.dylib` *must* be loaded as `RTLD_GLOBAL`, which is
done via `JAVA_INTEROP_LIB_LOAD_GLOBALLY`.
[0]: https://docs.microsoft.com/en-us/windows/win32/api/libloaderapi/nf-libloaderapi-loadlibrarya
[1]: https://docs.microsoft.com/en-us/windows/win32/dlls/dynamic-link-library-search-order
[2]: https://docs.microsoft.com/en-us/windows/win32/api/libloaderapi/nf-libloaderapi-loadlibraryexa
[3]: https://linux.die.net/man/3/dlopen
Fixes? https://work.azdo.io/1148474
Fixes? https://developercommunity.visualstudio.com/content/problem/1086793/我用xamarin-开发android-后进行打包发布apk但升级后打包报密码错误.html
Context: 380e95e340
Context: https://issuetracker.google.com/issues/150189789
Context: 0031300d95
Context: 3f51412d5f
Visual Studio has a [Xamarin Android SDK Manager][0], which allows
installing and updating the various Android SDK packages.
Historically, two "repositories" were available: the Microsoft
repository, and the Google repository. The Microsoft repository was
curated, containing only "known good" package versions, while the
Google repository was whatever Google was providing "now", which could
contain "bad" package versions.
"Bad" package versions such as Build-tools r30+, which requires
JDK 11; see also <https://issuetracker.google.com/issues/150189789>.
Visual Studio 16.7 (Windows) and 8.7 (macOS) updated the SDK Manager
to *remove* the repository selection, showing *only* the Google repo.
The result is that if someone goes to the SDK Manager window and
clicks the **Tools** tab, they'll (currently) be presented with the
options:
- [ ] Android SDK Build-Tools 30.0.2
- [ ] Android SDK Build-Tools 30.0.1
- [ ] Android SDK Build-Tools 30
- …
Should that someone proceed to (1) install the
Android SDK Build-Tools 30+ package, and (2) *uninstall* all older
package versions, they will have broken their Xamarin.Android build,
because Build-Tools 30+ requires JDK 11, while most (all?) customers
will have JDK 8 installed and *not* JDK 11 installed. The result:
Task "AndroidApkSigner"
Task Parameter:ApkSignerJar=%HOME%\android-toolchain\sdk\build-tools\30.0.0-rc4\lib\apksigner.jar
…
Task Parameter:ToolPath=C:\Program Files\Android\jdk\microsoft_dist_openjdk_1.8.0.25\bin
Task Parameter:ManifestFile=obj\Release\android\AndroidManifest.xml
C:\Program Files\Android\jdk\microsoft_dist_openjdk_1.8.0.25\bin\java.exe -jar %HOME%\android-toolchain\sdk\build-tools\30.0.0-rc4\lib\apksigner.jar sign --ks "%HOME%\AppData\Local\Xamarin\Mono for Android\debug.keystore" --ks-pass pass:android --ks-key-alias androiddebugkey --key-pass pass:android --min-sdk-version 21 --max-sdk-version 29 "C:\A\vs2019xam00000Y-1\_work\1\s\bin\TestRelease\temp\BuildAotApplication AndÜmläüts_x86_64_True_True\bin\Release\UnnamedProject.UnnamedProject-Signed.apk"
java.lang.UnsupportedClassVersionError: com/android/apksigner/ApkSignerTool has been compiled by a more recent version of the Java Runtime (class file version 53.0), this version of the Java Runtime only recognizes class file versions up to 52.0
at java.lang.ClassLoader.defineClass1(Native Method)
…
…\Xamarin.Android.Common.targets(2559,2): error MSB6006: "java.exe" exited with code 1.
How do we fix this?
We are discussing bringing back the Microsoft repository, so that we
can (once again) have a set of Known Good, Won't Break My Machine
Android SDK package versions.
In the meantime, a shorter-term fix is to bundle an `apksigner.jar`
which can run on JDK 8, and use that where needed.
Add a Git Submodule to [apksig][1], which contains the `apksigner.jar`
source code, and update the xamarin-android build so that
`apksigner.jar` is built within a JDK 8 environment.
Then I changed the `<ResolveAndroidTooling/>` task so it no longer was
responsible for setting `$(ApkSignerJar)` and `$(AndroidUseApkSigner)`.
I set up these properties in a similar way as we do for aapt2, r8, etc.
TODO:
* I need to make changes to xamarin/androidtools, so it can find
and use the `apksigner.jar` bundled with Xamarin.Android.
[0]: https://docs.microsoft.com/en-us/xamarin/android/get-started/installation/android-sdk?tabs=windows
[1]: https://android.googlesource.com/platform/tools/apksig
Context: 059c2c07f0
Commit d236af54 added the `external/lz4` submodule and updated
`.gitmodules` in a manner *not* in accordance with 059c2c07:
* It used tabs instead of spaces, which means @MSylvia's branch
tooling will cause larger commit diffs for this file
* It used a submodule name of "lz4", instead of "conventional"
name mirroring the checkout location of "external/lz4".
Reformat the `lz4` entry within `.gitmodules`, replacing tabs with
spaces and renaming to `external/lz4`, and sort the entry
appropriately.
Currently, Xamarin.Android supports compression of managed assemblies
within the `.apk` if the app is built with
[`$(BundleAssemblies)`=True][0], with the compressed assembly data
stored within `libmonodroid_bundle_app.so` using gzip compression and
placed in an array inside the data section of the shared library.
There are two problems with this approach:
1. `mkbundle` emits C code, which requires a C compiler which requires
the full Android NDK, and thus requires Visual Studio Enterprise.
2. Reliance on Mono's `mkbundle` results in possible issues around
[filename globbing][1] such that
`Xamarin.AndroidX.AppCompat.Resources.dll` is improperly treated
as a [satellite assembly][2].
Because of (2), we are planning on [removing support][3] for
`$(BundleAssemblies)` in .NET 6 ([née .NET 5][4]), which resulted in
[some pushback][5] because `.apk` size is very important for some
customers, and the startup overheads we believed to be inherent to
`$(BundleAssemblies)` turned out to be somewhat over-estimated.
To resolve the above issues, add an assembly compression mechanism
that doesn't rely on `mkbundle` and the NDK: separately compress the
assemblies and store the compressed data within the `.apk`.
Compression is performed using the [managed implementation][6] of the
excellent [LZ4][7] algorithm. This gives us a decent compression ratio
and a much faster (de)compression speed than gzip/zlib offer. Also,
assemblies are stored directly in the APK in their usual directory,
which allows us to [**mmap**(2)][8] them in the runtime directly from
the `.apk`. The build process calculates the size required to store
the decompressed assemblies and adds a data section to
`libxamarin-app.so` which causes *Android* to allocate all the required
memory when the DSO is loaded, thus removing the need of dynamic memory
allocation and making the startup faster.
Compression is supported only in `Release` builds and is enabled by
default, but it can be turned off by setting the
`$(AndroidEnableAssemblyCompression)` MSBuild property to `False`.
Compression can be disabled for an individual assembly by setting the
`%(AndroidSkipCompression)` MSBuild item metadata to True for the
assembly in question, e.g. via:
<AndroidCustomMetaDataForReferences Include="MyAssembly.dll">
<AndroidSkipCompression>true</AssemblySkipCompression>
</AndroidCustomMetaDataForReferences>
The compressed assemblies still use their original name, e.g.
`Mono.Android.dll`, so that we don't have to perform any string
matching on the runtime in order to detect whether the assembly we are
asked to load is compressed or not. Instead, the compression code
*prepends* a short header to each `.dll` file (in pseudo C code):
struct CompressedAssemblyHeader {
uint32_t magic; // 0x5A4C4158; 'XALZ', little-endian
uint32_t descriptor_index; // Index into an internal assembly descriptor table
uint32_t uncompressed_length; // Size of assembly, uncompressed
};
The decompression code looks at the `mmap`ed data and checks whether
the above header is present. If yes, the assembly is decompressed,
otherwise it's loaded as-is.
It is important to remember that the assemblies are compressed at
build time using LZ4 block compression, which requires assembly data
to be entirely loaded into memory before compression; we do this
instead of using the LZ4 frame format to make decompression at runtime
faster. The compression output also requires a separate buffer, thus
memory consumption at *build* time will be roughly 1.5x the size of the
largest assembly, which is reused across all assemblies.
~~ Application Size ~~
A Xamarin.Forms "Hello World" application `.apk` shrinks by 27% with
this approach for a single ABI:
| Before (bytes) | LZ4 (bytes) | Δ |
|------------------:|--------------:|:---------:|
| 23,305,194 | 16,813,034 | -27.85% |
Size comparison between this commit and `.apk`s created with
`$(BundleAssemblies)` =True depends on the number of enabled ABI
targets in the application. For each ABI, `$(BundleAssemblies)`=True
creates a separate shared library, so the amount of space consumed
increases by the size of the bundle shared library.
The new compression scheme shares the compressed assemblies among all
the enabled ABIs, thus effectively creating smaller multi-ABI `.apk`s.
In the tables below, `mkbundle` refers to the APK created with
`$(BundleAssemblies)`=True, `lz4` refers to the `.apk` build with
the new compression scheme:
| ABIs | mkbundle (bytes) | LZ4 (bytes) | Δ |
|--------------------------------------:|------------------:|--------------:|---------|
| armeabi-v7a, arm64-v8a, x86, x86_64 | 27,130,240 | 16,813,034 | -38.03% |
| arm64-v8a | 7,783,449 | 8,746,878 | +11.01% |
The single API case is ~11% larger because gzip offers better
compression, at the cost of higher runtime startup overhead.
~~ Startup Performance ~~
When launching the Xamarin.Forms "Hello World" application on a
Pixel 3 XL, the use of LZ4-compressed assemblies has at worst a ~1.58%
increase in the Activity Displayed time (64-bit app w/ assembly
preload enabled), while slightly faster on 32-bit apps, but is *always*
faster than the mkbundle startup time for all configurations:
| | | | | LZ4 vs | LZ4 vs |
| Description | None (ms) | mkbundle (ms) | LZ4 (ms) | None Δ | mkbundle Δ |
|----------------------------------:|----------:|--------------:|----------:|:--------:|:----------:|
| preload enabled; 32-bit build | 795.8 | 855.6 | 783.8 | -0.25% ✓ | -7.22% ✓ |
| preload disabled; 32-bit build | 777.1 | 843.0 | 780.5 | +0.44% ✗ | -7.41% ✓ |
| preload enabled; 64-bit build | 779.0 | 843.0 | 791.5 | +1.58% ✗ | -6.82% ✓ |
| preload disabled; 64-bit build | 776.0 | 841.6 | 781.5 | +0.69% ✗ | -7.15% ✓ |
[0]: https://docs.microsoft.com/en-us/xamarin/android/deploy-test/release-prep/?tabs=windows#bundle-assemblies-into-native-code
[1]: https://github.com/xamarin/AndroidX/issues/64
[2]: 9b4736d4c2/mcs/tools/mkbundle/mkbundle.cs (L1315-L1317)
[3]: https://github.com/xamarin/AndroidX/issues/64#issuecomment-609970584
[4]: https://devblogs.microsoft.com/dotnet/announcing-net-5-preview-4-and-our-journey-to-one-net/
[5]: https://github.com/xamarin/AndroidX/issues/64#issuecomment-610002467
[6]: https://www.nuget.org/packages/K4os.Compression.LZ4/
[7]: https://github.com/lz4/lz4
[8]: https://linux.die.net/man/2/mmap
Context: 8f30933a89
Changes: 3226a4b57a...8f30933a89
* xamarin/Java.Interop@8f30933: [generator] Mark some Obsolete fields as errors (#568)
* xamarin/Java.Interop@47201c4: [generator] Change protected members in final class to private (#569)
* xamarin/Java.Interop@c5815fb: [generator] Be smarter about when members with same names need "new" (#567)
* xamarin/Java.Interop@bfc0273: [generator] Ensure property setter parameter name is "value" (#566)
This Java.Interop bump includes two changes which "break" API:
* Certain fields are changed from `[Obsolete]` to
`[Obsolete(error:true)]`, turning *use* of the field within C#
code into a CS0619 error.
* The C# compiler has long warned about the presence of `protected`
members within `sealed` types, as they cannot be used. However,
they were still emitted, resulting in CS0628 warnings when
compiling `Mono.Android.dll`.
We consider changing `[Obsolete]` to `[Obsolete(error:true)]` to be
acceptable because this is being done to `const` fields, so this
won't break ABI, as there are no IL references to `const` fields, and
because (1) there are "replacement" members which can be used, and
(2) these fields have been obsolete for *years*.
The "problem" here is that `Microsoft.DotNet.ApiCompat.exe`, which
the `<CheckApiCompatibility/>` task uses from
`src/Mono.Android/Mono.Android.targets`, sees every such change as
an API break. This in and of itself is plausibly sensible, but the
error message *itself* is bananas, e.g.:
namespace Android.AccessibilityServices {
public abstract partial class AccessibilityService : Android.App.Service {
[Register ("GESTURE_SWIPE_DOWN", ApiSince = 16)]
[Obsolete ("This constant will be removed in the future version. Use Android.AccessibilityServices.AccessibilityGesture enum directly instead of this field.", error: true)]
public const Android.AccessibilityServices.AccessibilityGesture GestureSwipeDown = (Android.AccessibilityServices.AccessibilityGesture) 2;
}
}
results in:
CannotRemoveAttribute : Attribute 'System.ObsoleteAttribute' exists on 'Android.AccessibilityServices.AccessibilityGesture Android.AccessibilityServices.AccessibilityService.GestureSwipeDown' in the contract but not the implementation.
The message implies that `[Obsolete]` was removed, which isn't
strictly true! `[Obsolete]` is still there! It's just that the
constructor parameters have changed.
This results in a ginormous change to
`tests/api-compatibility/acceptable-breakages-vReference.txt`.
The "`protected` members within `sealed` types" fix for CS0628 also
resulted in errors from the Tests > API Compatibility job, which runs
via `external/xamarin-android-api-compatibility`.
Given that commit 07e74771 was intended to *replace* the
Tests > API Compatibility job, *remove* the
`external/xamarin-android-api-compatibility` submodule reference.
We will no longer be using that submodule for API compat checks.
Co-authored-by: Jonathan Pobst <monkey@jpobst.com>
Fixes: https://github.com/xamarin/xamarin-android/issues/3086
Bumps to xamarin/android-api-docs/master@25aa18c2
The [Xamarin.Android API docs were open-sourced][0]!
This means that the public `xamarin-android` repo can now easily
reference the API docs, which in turn means that the OSS "flavors" of
the [macOS `.pkg` and Windows `.vsix` installers][1] can likewise
include and redistribute documentation.
Migrate the `make assemble-docs` build target and its dependencies
from the `docs.make` file in monodroid to a set of MSBuild targets in
xamarin-android.
Add a trailing slash on the value of `$(FrameworkSrcDir)` so that the
new targets can follow the convention that the value of a directory
path property always includes a trailing slash.
Add new MSBuild targets to replace the old `docs.make` targets.
These targets run [`mdoc`][2] to generate ["msxdoc"][3] Microsoft XML
Documentation format files that are used in the `.vsix` installer for
Windows as well, as an [`mdoc assemble`][4]-formatted
`MonoAndroid-lib.zip` archive and `.tree` sfile that are used in the
`.pkg` installer for macOS.
Bring in xamarin/android-api-docs repo as a submodule, set at the
same commit as the corresponding submodule in monodroid.
There are a few intentional differences compared to the original
`docs.make` targets:
* The Microsoft XML Documentation files are output to the directory
of the latest *stable* framework version rather than the latest
*known* framework version. This is useful because it means the
MSBuild targets don't need access to the `$(FRAMEWORKS)` value.
* The OpenTK-1.0 docs are only generated for the `mdoc assemble`
step and not for the `mdoc export-msxdoc` step.
`mdoc export-msxdoc` output isn't needed at the moment because
the `.vsix` installer instead uses the `.xml` file that's
generated when `Mono.Android.csproj` builds `OpenTK-1.0.csproj`
for `$(TargetFrameworkVersion)`=`v4.4`.
* The OpenTK docs are omitted as OpenTK has been deprecated and
removed from Xamarin.Android.
* The `make pimpup-docs` target is omitted. I checked the build
logs for a recent commercial build and saw that while the build
does compile `doc-relcontent-merger.exe` and
`recipe-relcontent-generator.exe`, it never runs either one.
Starting in [monodroid@c83b93e2][5], the `$(PIMP_STAMP)` target
that used to run `doc-relcontent-merger.exe` was no longer needed
and was disabled.
* The `docs/netdocs.zip` and `docs/mono-libs.zip` targets are also
intentionally omitted. Although commercial builds do still run
those targets, the outputs are not used for anything.
Based on the contents of the `push-docs` target in `docs.make` and
the fact that that target was last edited in 2012, I believe the
`netdocs.zip` and `mono-libs.zip` archives were for an old online
documentation system that was retired long ago.
I verified that the `Mono.Android.xml` Microsoft XML Documentation
file generated by these new targets was identical to the version
generated by the `docs.make` targets. Similarly, I verified that the
new monodoc `MonoAndroid-lib.zip` archive generated by these targets
contained the same list of `xml.summary.*` files as the `.zip`
archive generated by the `docs.make` targets.
Additionally, include `ICSharpCode.SharpZipLib.dll` into the mono
bundle (fbfd676c, 29568117) and the macOS `.pkg` and Windows `.vsix`
installers, as `mdoc assemble` requires it.
Add a trailing slash on the value of `$(MSBuildSrcDir)` so that the
new targets can follow the convention that the value of a directory
path property always includes a trailing slash. One reason for this
convention is that if the value of a path item does not have a
trailing slash, then the last part of the path is assigned to the
`%(Filename)` metadata rather than being part of the `%(Directory)`
metadata.
Rename `%(SourceFolder)` to `%(SourceDir)`, because "Dir" is more
common than "Folder" both in `create-installers.targets` and in the
built-in MSBuild tasks like `<MakeDir/>` and `<RemoveDir/>`.
Finally, include an extra `\` at the end of `%(SourceDir)` in the
`<Exec/>` tasks. Without this, `mdoc.exe` would hit an
`Illegal characters in path` exception in
`System.IO.Path.LegacyNormalizePath()` on Windows, because
[`CommandLineToArgvW()`][6] treats `\` as an escape character when it
precedes `"`. Adding an additional `\` is a little easier than using
property functions to remove the original `\`.
[0]: https://devblogs.microsoft.com/xamarin/xamarin-api-docs-open-source-available-now/
[1]: 02f3258ade/Documentation/workflow/JenkinsBuildArtifacts.md
[2]: http://docs.go-mono.com/?link=man%3amdoc%281%29
[3]: https://docs.microsoft.com/en-us/dotnet/csharp/language-reference/compiler-options/doc-compiler-option
[4]: http://docs.go-mono.com/?link=man%3amdoc-assemble%281%29
[5]: c83b93e20b
[6]: https://docs.microsoft.com/windows/win32/api/shellapi/nf-shellapi-commandlinetoargvw
Changes: 63bbeb076d...ba0a823f07
Changes: 2fb054e...d78c6b2
Changes: 4c4daa5281...c092183e87
Bumps to xamarin/monodroid/master@ba0a823f:
* Bumps to xamarin/android-sdk-installer/master@c092183e
* Bumps to xamarin/androidtools/master@d78c6b2
The [`LibZipSharp` repo][0] is now available as the
[Xamarin.LibZipSharp NuGet package][1].
Remove the submodules for libzip and LibZipSharp in favour of using
the nuget package.
The `Xamarin.LibZipSharp` package contains the native binaries for
`libzip`, however by default the NuGet package does NOT copy these
into `$(OutputPath)`. In order for that to happen the following
MSBuild property needs to be set:
<LibZipSharpBundleAllNativeLibraries>True</LibZipSharpBundleAllNativeLibraries>
When `$(LibZipSharpBundleAllNativeLibraries)`=True, the native
binaries will be copied to the `$(OutputPath)\@PLATFORM@` directories
where `@PLATFORM@` will be `Darwin`, `Linux` or `Windows`.
This means we need to update our build system to make sure we include
the files from the correct locations. We also need to make sure we
call the `Restore` target on projects using `LibZipSharp` now as the
package will need to be restored.
Another change is that we have removed the versioning from the
`libzip` native libraries: they used to be e.g. `libzip.5.0.dylib`
but are now just `libzip.dylib`. This was done upstream in the
`Xamarin.LibZipSharp` package itself to allow for support for .NET
Core based `DllImport` which do NOT use a `.dll.config` file, but
instead use only the dll name (e.g `libzip`).
We are also including the `Linux` version of the binaries in the
installer.
[0]: https://github.com/xamarin/LibZipSharp
[1]: https://www.nuget.org/packages/Xamarin.LibZipSharp/
Context: https://github.com/xamarin/xamarin-android/projects/10
One of the goals behind the Mono SDK archive was to enable product
builds without having the entire Mono source tree present, which
means it should be possible to remove the Mono git submodule and
build the entire xamarin-android repo without any issues.
At the same time, we want to keep the ability to build Mono from
source, if necessary, for situations when a developer works on the
BCL or the runtime itself and there's no appropriate binary archive
present.
To support these two requirements:
* Remove the Mono submodule
* Put Mono repo commit reference into the `.external` file
* Make distinction between "commercial" and "regular" bits in the
`.external` file so that we don't attempt to check out everything
mentioned in there in situations when a developer doesn't have
access to the commercial bits.
* Use Mono linker sources from the archive
* Remove `zlib-helper.c` from the `libmonodroid` build - the API
implemented in there is no longer used by Mono.
* `Xamarin.Android.Cecil.dll` and `Xamarin.Android.Cecil.Mdb.dll`
are no longer built from source. Instead we use Cecil to reshape
`Mono.Cecil` and conjure the two renamed and re-signed assemblies.
* All the code which used `Xamarin.Android.Cecil` project reference
now simply references the conjured assemblies (which are also
added to the bundle)
* Third Party Notices are generated from licenses found in the
Mono archive.
* Windows build downloads also the Darwin archive - in order to be
able to use Mono Linker sources as well as licenses contained in
this archive.
Bump to mono/2019-06 branch and set mono required versions.
[cecil] Fix the cecil build by reverting "Add a solution file for remap-assembly-ref (#3248)"
Using the solution instead of csproj broke the cecil build. I didn't
traced all the details, but we ended up with compiling
`Mono.Cecil.dll` without any references.
Use 7z when downloading Mono Archives
[xaprepare] Drop llvm32 cross compilers
Mono has just dropped the llvm32 cross compilers because it is now possible to
cross compile to 32-bit using the 64-bit compiler.
Remove requirement for 32-bit llvm runtime in preparation stage.
[tests] Updated Xamarin.Android.Net tests
Update Properties_Invalid test to reflect current implementation
This was testing part of our old HttpClientHandler implementation
introduced here:
f97e03f284
The current corefx implementation doesn't throw when `UseProxy` is
false. The documentation doesn't specify such behavior, so I assume it
was specific for our old implementaion.
Context:
https://docs.microsoft.com/en-us/dotnet/api/system.net.http.httpclienthandler.proxy?view=netframework-4.8https://docs.microsoft.com/en-us/dotnet/api/system.net.http.httpclienthandler.useproxy?view=netframework-4.89354dfd8d3/src/System.Net.Http/src/System/Net/Http/HttpClientHandler.Unix.cs (L242)9354dfd8d3/src/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/SocketsHttpHandler.cs (L75)9354dfd8d3/src/System.Net.Http/src/System/Net/Http/CurlHandler/CurlHandler.cs (L234)
Remove broken HttpClient test
Context: https://github.com/xamarin/xamarin-android/pull/3155#issuecomment-506856300
[linker] Disable linker's UnreachableBodies optimization
Disable it for now til we find out a fix.
Context: https://github.com/xamarin/xamarin-android/issues/3202
[jnimarshalmethod-gen] Introduce MonoAndroidLibDirectory property
As we have switched to dynamically linker mono recently, we need to have
libmonosgen in dyld path. So we set `DYLD_LIBRARY_PATH` for `mono`
when running `jnimarshalmethod-gen`
[tests] Exclude network tests which hang.
Context: https://github.com/mono/mono/issues/14938
Context: https://github.com/xamarin/xamarin-macios/pull/6228
[xamarin-android-api-compatibility] API changes
System.IO.Enumeration.FileSystemName and
System.Security.Cryptography.CryptographicOperations classes were
removed from System.xml as they were duplicates of mscorlib ones. They
were removed as part of NS2.1 work in mono.
They were added by mistake and removed later. Because they were
already present in 16.1 and 16.2 respectively, mono team added type
forwarders to `System.dll` to avoid/lessen API breakage.
System.dll now contains:
.class extern forwarder System.IO.Enumeration.FileSystemName
{
.assembly extern mscorlib
}
.class extern forwarder System.Security.Cryptography.CryptographicOperations
{
.assembly extern mscorlib
}
Context:
ba719acaa2https://github.com/xamarin/xamarin-android/pull/3155#issuecomment-510142890
3 extension methods extracted from
`System.Threading.Tasks.TaskExtensions` to a new
`System.Threading.Tasks.TaskAsyncEnumerableExtensions` class.
They stay in the same namespace and being extension methods, they
should not break anything.
public static System.Runtime.CompilerServices.ConfiguredCancelableAsyncEnumerable<T> ConfigureAwait<T> (this System.Collections.Generic.IAsyncEnumerable<T>, bool);
public static System.Runtime.CompilerServices.ConfiguredAsyncDisposable ConfigureAwait (this System.IAsyncDisposable, bool);
public static System.Runtime.CompilerServices.ConfiguredCancelableAsyncEnumerable<T> WithCancellation<T> (this System.Collections.Generic.IAsyncEnumerable<T>, System.Threading.CancellationToken);
Context:
ebeec50871 (diff-93ef3714cd10edf17d6d8045cf3406b8)https://github.com/xamarin/xamarin-android/pull/3155#issuecomment-502672705
We should prefer using mono's `llvm` submodule instead of providing
our own, as:
1. Having *both* `external/llvm` *and* `external/mono/external/llvm`
is confusing (which one "wins"?), and
2. The one that *actually* matters ("wins") is mono's `llvm`, as
that's the one included in the mono archive, which is now used by
default (333b98b3).
Simplify life. Remove `external/llvm`.
Context: https://github.com/xamarin/xamarin-android/issues/2608
Bumps to mono/debugger-libs/master@b45303.
Bumps to icsharpcode/nrefactory/master@0607a4.
Add support for testing that the app launches on device *and* can be
interacted with. It does this by making use of some Android shell
features and commands
`adb shell input` allows us to send taps and clicks to the device.
It simulates user input. This can be used to e.g. click a button.
`adb exec-out uiautomator` allows obtaining the complete UI layout
for the currently active Activity. For example, to dump the current
Activity UI layout to the console:
adb exec-out uiautomator dump /dev/tty
This returns a XML:
<hierarchy rotation="0">...
We can parse this XML looking for any ui element we want. We can
then get the bounds of the control and pass that information to
`adb shell input tap`.
Since we are now able to read the displayed UI, we can also add some
`TimeZoneInfo` tests, to ensure that when the timezome is changed:
adb shell su root setprop persist.sys.timezone "America/New_York"
the UI is correctly updated to display the appropriate timezone name.
Finally, add tests to check that the debugger -- via
mono/debugger-libs -- can actually connect to the Application.
The debugger-related tests only run if the commercial bits of the
Xamarin.Android SDK are available.
As all of these new tests *require* that an emulator or device be
available, they have been split out into a new
`tests/MSBuildDeviceIntegration` project. This is built as part of
the main solution. However, it is run as part of the
`make run-apk-tests` target. This is so we can be sure the emulator
is running.
In the future any MSBuild related tests that require a device should
be placed in this new project.
* Bump to mono:2018-08
* Bump to mono:2018-10
* apply 2018-06 changes
* bump mono
* revert some submodules
* Update dependencies.projitems
* bump mono
* Update dependencies.projitems
* fix xunit path
* bump mono
* undo xamarin-android-api-compatibility
* Bump to mono/2018-08/04a6e829
Commit list for mono/mono:
* mono/mono@04a6e8294e [2018-08] Bump xunit (#11085)
* mono/mono@f001e691ca [2018-08] Bump msbuild (#11087)
* mono/mono@7001d39bb2 Always use InvariantCulture calendars in X509Certificate on Mobile
Diff: e257971d3c...04a6e8294e
* Bump to mono/2018-08/7805ccc4
Commit list for mono/mono:
* mono/mono@7805ccc43d [sdks] Fix llvm-llvm32 build (#11117)
Diff: 04a6e8294e...7805ccc43d
* Bump to mono/2018-08/657f915e
Commit list for mono/mono:
* mono/mono@657f915eb6 [2018-08] [sdks] Create archive targets for pre-building on CI (#10932)
* mono/mono@bd1f64acc3 [sdks] Add cmake toolchain file for LLVM MXE builds (#11055)
Diff: 7805ccc43d...657f915eb6
* Bump to mono/2018-08/2712292c
Commit list for mono/mono:
* mono/mono@2712292c56 [sdks] Add Groovy script to build the SDKs archives for the products (#11192)
* mono/mono@839f49d3b2 [sdks] futimens and futimes symbols are missing on anything earlier than 10.13 (#11176)
Diff: 657f915eb6...2712292c56
* Bump to mono/2018-10/8890b0c6
Commit list for mono/mono:
* mono/mono@8890b0c6c4 Take fast path only for delegate types not Delegate/MulticastDelegate. (#11195)
* mono/mono@dcaef0b2ca Bump to llvm/release_60/117a508c (#11120)
* mono/mono@9ff6d8f918 [interp] Implement interpreter entry trampolines on amd64 (#10978) (#11163)
* mono/mono@e708752f14 [sdks] Add Groovy script to build the SDKs archives for the products (#11191)
* mono/mono@2a052ba0c9 [sdks] futimens and futimes symbols are missing on anything earlier than 10.13 (#11177)
* mono/mono@76be3b3bc8 Initial Mono.Native Support. (#10941)
* mono/mono@7e7efe488f [2018-10] [sdks] Build tests for BCL targets (#11140)
* mono/mono@72a81cc6d2 Use passwordless import in the `X509Certificate(byte[])` constructor.
* mono/mono@f00fa49bb9 [sdks] Add cmake toolchain file for LLVM MXE builds (#11054)
* mono/mono@fc9fc38926 [2018-08] Bump msbuild (#11097)
* mono/mono@d33c213354 Always use InvariantCulture calendars in X509Certificate on Mobile
* mono/mono@8c7aca3316 [2018-10] Bump xunit (#11084)
* mono/mono@923afca889 [2018-10] Bump cecil (#11022)
* mono/mono@3dd1602a1a Bump msbuild to track xplat-master
* mono/mono@edc6fc4e82 [2018-10] [interp] fixes for native type (#10999)
* mono/mono@98b85ceaec Bump corert
* mono/mono@7d4ca00db4 Bump corert
* mono/mono@810ced6d0b Extract case for verifying several object array parameters to separate test method.
* mono/mono@98c33e716d Add unit tests to verify custom attribute ctor with object array as parameter
* mono/mono@a5016f7898 [ci] Always pass sha1 from Mac to Windows packaging build
* mono/mono@6deb114ce5 [corlib] Simplify confusing "SSPI" error message and align with corefx (#10976)
* mono/mono@ec0ae343f0 [sdks/ios] Build libMonoPosixHelper for device architectures, so that zlib-helper.o is created. (#10959)
* mono/mono@921642ea72 Put xunit tests in mcs/class/lib/PROFILE/tests too
* mono/mono@5e6ea6c806 Merge pull request #10909 from monojenkins/backport-pr-10899-to-2018-10
* mono/mono@aa3762656b [sdks] Ensure LLVM is cloned before trying to download (#10933)
* mono/mono@7f3b574ca7 Bump linker
* mono/mono@31b7ce52ad [runtime] Fix undefined pthread_main_np (#10930)
* mono/mono@be44769e17 [Reflection] Fix issue with finding types in module using an asterisk as filter criteria
* mono/mono@1bcf14ab41 [w32handle] Add coop state transitions in mono_w32handle_lock_handles (#10911)
* mono/mono@f362bfe257 [runtime] Disable stack guard for main thread on osx
Diff: a1c34dea01...8890b0c6c4
* bump mono
* Bump to mono/2018-08/010e741b
Commit list for mono/mono:
* mono/mono@010e741bac [2018-08] Crash Reporter V2 (#11161)
* mono/mono@ec1a736266 [corlib] Ensure that Console Streams are thread-safe
Diff: a9b1acbee1...010e741bac
* Bump to mono/2018-08/bc9d709e
Commit list for mono/mono:
* mono/mono@bc9d709e70 [sdks] Package what's missing for XA (#11256)
* mono/mono@31a6b877f6 [pipeline] Ensure only 1 build per-product is executed per-node (#11249)
* mono/mono@4b343e98f3 [interp] Implement interpreter entry trampolines on amd64 (#10978) (#11164)
* mono/mono@2bc1ed09a3 [aot] Ensure shared got entries are initialized before loading methods (#11226)
Diff: 010e741bac...bc9d709e70
* Bump to system mono/2018-08/bc9d709e
* Bump to mono/2018-10/f743d425
Commit list for mono/mono:
* mono/mono@f743d4250c [2018-10] Use Mono.Android as a reference for netstandard facade (#10974)
* mono/mono@dc9771601a [sdks] Package what's missing for XA (#11254)
* mono/mono@3986341b65 [tests] Embed Mono.Data.Sqlite test resource (#11245)
* mono/mono@0a919874d1 [pipeline] Ensure only 1 build per-product is executed per-node (#11248)
* mono/mono@4577597ce1 [aot] Ensure shared got entries are initialized before loading methods (#11224)
* mono/mono@59b1d81525 [2018-10] Crash Reporter V2 (#11160)
* mono/mono@980aef41b9 [ci] Move OSX .pkg build to a separate bot pool
* mono/mono@429bb0bae0 [corlib] Ensure that Console Streams are thread-safe (#11230)
* mono/mono@c0212dac5f Remove overflow exception catch from #MOBILE
* mono/mono@4f700983df [Reflection] Handle overflow exception on constructor invocation
* mono/mono@790f412275 [pkg] Add preinstall which removes existing Mono of the same version (#11207)
* mono/mono@3b5cacc15c [2018-10] [System]: Reenable native Apple Certificate support on WatchOS. (#11198)
* mono/mono@b261706d88 [sdks] Fix Archive Jenkins job (#11218)
Diff: 8890b0c6c4...f743d4250c
* Fix build
* Remove _Autogen target, it's part of _Build already
* bump mono
* Removes wrong main entry point name for mono-api-info
* Bump to mono/2018-08/0522076f
Commit list for mono/mono:
* mono/mono@0522076f0b [interp] throw overflow exception when {float,double}.NaN is converted to an int type (#11411)
* mono/mono@6034c589df [interp] enable more BCL tests (#11399)
* mono/mono@aebff6ce01 [SDKS, LLVM] Disable detection and use of libxml2 (#11340)
* mono/mono@818a27a386 [2018-08] [aot] add unbox_arbitrary_trampoline for fullaot/interp mixed mode (#11363)
* mono/mono@b5e8de9182 [2018-08][corlib] Adds back accidentaly removed legacy COM API to Binder (#11368)
* mono/mono@205b8129d5 [tests] lower recursion depth of PerformNoPinAction (#11323)
* mono/mono@c2c764dfc6 [2018-08] Replace mono_object_get_class with mono_object_class to avoid gc transitions. (#11338)
* mono/mono@b778499454 [wasm] Add missing include (#11337)
Diff: b7ad1c6923...0522076f0b
* Bump mono to tip of 2018-10
* [build] Download correct mono archive URL (#2366)
Context: 8e0cbf1044/scripts/ci/pipeline/sdks-archive.groovy (L84)
Context: https://jenkins.mono-project.com/view/Per-commit%20packages/job/archive-mono/
The mono archives have a different filename format than that used in
commit f970cd50. As per [`sdks-archive.groovy`][0], the filename to
download *should* be:
${product}-${configuration}-${platform}-${commitHash}.zip
Update `$(_MonoArchiveName)` accordingly, and fix the output paths so
that the mono archive can actually be used. (As noted in f970cd50,
*actual* support for building with the mono archive hadn't been
tested, as one didn't exist for the mono xamarin-android uses.)
[0]: 8e0cbf1044/scripts/ci/pipeline/sdks-archive.groovy
* Bump to mono/2018-10/d02b6c05
Commit list for mono/mono:
* mono/mono@d02b6c0583 [crash] Fix async setting for crash reporter
Diff: 3d170b9afe...d02b6c0583
* Bump to mono/2018-10/24e7d7d5
Commit list for mono/mono:
* mono/mono@24e7d7d5e8 [csproj] Update project files
* mono/mono@a65b7d8b44 [csproj] Update project files
* mono/mono@6bbc707eb3 "Mono.Native - Halloween Edition"
* mono/mono@114f88aaf1 [corlib] Remove incorrectly converted silverlight check
* mono/mono@dc57395fd6 [2018-10] Add .NET 4.7.2 reference assemblies (#11391) (#11503)
* mono/mono@86ba8ff964 [System]`lock` protect `SystemCertificateProvider.EnsureInitialized()`.
Diff: d02b6c0583...24e7d7d5e8
* Bump to mono/2018-10/3c2b25ce
Commit list for mono/mono:
* mono/mono@3c2b25ce23 [System]`lock` protect `SystemCertificateProvider.EnsureInitialized()`.
* mono/mono@5fedefe43e [tests] reduce recursion depth
* mono/mono@53a21a0e84 [crash] Fix async setting for crash reporter
* mono/mono@04bdf8852b [2018-08] [crash] Fix merp invocation (#11468)
* mono/mono@8df60a29bf [delegates] do not use CEE_CALLVIRT for non-virtual methods (#11451)
* mono/mono@d90fbe3a0a [llvm] Fix passing vtypes received by value as arguments, they need to be copied. Fixes https://github.com/mono/mono/issues/11378. (#11429)
* mono/mono@ea740d2bc8 Disable hybrid suspend default for 2018-08
Diff: 0522076f0b...3c2b25ce23
* Revert "[build] Download correct mono archive URL (#2366)"
This reverts commit c2f6f46399.
* Bump to mono/2018-10/e45b97c6
Commit list for mono/mono:
* mono/mono@e45b97c68e [ci] Switch to new syntax of Azure Storage plugin
Diff: 24e7d7d5e8...e45b97c68e
* Bump to mono/2018-10/5e1b1153
Commit list for mono/mono:
* mono/mono@5e1b11535d [llvm] bump for armhf callingconv fix (#11608)
* mono/mono@f28a0c04db [sdks] Exit with non-zero exit code if the simulator test fails on ios. (#11547)
* mono/mono@ae5ff1b4bc [android sdk] Package resource assemblies into app bundle (#11583)
* mono/mono@0e6a3e0c85 [ios sdk] Package resource assemblies into app bundle (#11575)
* mono/mono@2d04a8f274 Bump API snapshot submodule
* mono/mono@2f9dcaf1fc [2018-10] Bump corefx
* mono/mono@39ee7a08f9 [ios] remove DISABLE_POLICY_EVIDENCE (#11582)
* mono/mono@36cac59485 Bump roslyn-binaries submodule to pull in System.Xml.dll file name fix in vbc.rsp (#11573)
* mono/mono@f5e27e160c Added missing cts.Cancel() in HttpClient Dispose() method (#11539)
* mono/mono@6d2ea1b004 [interp] transform delegate invoke in method body too (#11550)
* mono/mono@4023a6233e [2018-10] Fix ref return for structs in runtime invoke wrapper. (#11555)
* mono/mono@78662419df [ci] Fix Windows .msi build
* mono/mono@35d1ff89a0 Fix the issue when ParameterInfo.RawDefaultValue returns Missing.Value for non-null default values of DateTime?` and decimal? parameters (#11537)
* mono/mono@267470152a [bcl] avoid false pinning in GCTest (#11540)
* mono/mono@aab7377d4f [interp] accept typed-by-ref as return type from a JIT call (#11535)
* mono/mono@bdf42f000e Make the test compile on mobile. (#11544)
Diff: e45b97c68e...5e1b11535d
* Bump to mono/2018-08/32d4b6f2
Commit list for mono/mono:
* mono/mono@32d4b6f22a [llvm] bump for armhf callingconv fix (#11609)
* mono/mono@925e0c0d0c [ios] remove DISABLE_POLICY_EVIDENCE (#11581)
* mono/mono@65cd16c227 Bump roslyn-binaries submodule to pull in System.Xml.dll file name fix in vbc.rsp (#11573)
* mono/mono@b5891e25ac [ci] Fix Windows .msi build
* mono/mono@d4d1d1ec24 [bcl] avoid false pinning in GCTest (#11536)
* mono/mono@3837f0c276 [interp] accept typed-by-ref as return type from a JIT call (#11533)
* mono/mono@2e5c0d8f1e [interp] transform delegate invoke in method body too (#11551)
* mono/mono@60c21beaeb [ci] Switch to new syntax of Azure Storage plugin
Diff: 3c2b25ce23...32d4b6f22a
* [build] Download correct mono archive URL (#2389)
Context: 8e0cbf1044/scripts/ci/pipeline/sdks-archive.groovy (L84)
Context: https://jenkins.mono-project.com/view/Per-commit%20packages/job/archive-mono/
The mono archives have a different filename format than that used in
commit f970cd50. As per [`sdks-archive.groovy`][0], the filename to
download *should* be:
${product}-${configuration}-${platform}-${commitHash}.zip
Update `$(_MonoArchiveName)` accordingly, and fix the output paths so
that the mono archive can actually be used. (As noted in f970cd50,
*actual* support for building with the mono archive hadn't been
tested, as one didn't exist for the mono xamarin-android uses.)
[0]: 8e0cbf1044/scripts/ci/pipeline/sdks-archive.groovy
* Remove debug spew
* Bump to mono/2018-10/64c566d8
Commit list for mono/mono:
* mono/mono@64c566d8da [sdks] Compile support/{zlib-helper,nl}.c so XI/XM/XA does not have to cherry-pick them (#11676)
* mono/mono@d51ed697a0 [ci] Clean acceptance-tests/external before build
* mono/mono@4d29428741 [2018-10][System.Data]: GSS is not available on tvOS and watchOS. (#11677)
* mono/mono@f285f82c8d Bumping reference assemblies generated with latest GenAPI (#11633)
Diff: 5e1b11535d...64c566d8da
* [monodroid] Remove dependency on `$(MonoSourceFullPath)/support/{zlib-helper,nl}.c` (#2431)
* Revert unintended changes in previous merge
```
git diff 42656af718 $THIS_COMMIT
diff --git a/.gitmodules b/.gitmodules
index b305840f..7f835877 100644
--- a/.gitmodules
+++ b/.gitmodules
@@ -25,7 +25,7 @@
[submodule "external/mono"]
path = external/mono
url = https://github.com/mono/mono.git
- branch = 2018-08
+ branch = 2018-10
[submodule "external/mxe"]
path = external/mxe
url = https://github.com/xamarin/mxe.git
diff --git a/Configuration.props b/Configuration.props
index a1785e4d..803438ab 100644
--- a/Configuration.props
+++ b/Configuration.props
@@ -73,8 +73,8 @@
<JavaInteropSourceDirectory Condition=" '$(JavaInteropSourceDirectory)' == '' ">$(MSBuildThisFileDirectory)external\Java.Interop</JavaInteropSourceDirectory>
<LlvmSourceDirectory Condition=" '$(LlvmSourceDirectory)' == '' ">$(MSBuildThisFileDirectory)external\llvm</LlvmSourceDirectory>
<MonoSourceDirectory>$(MSBuildThisFileDirectory)external\mono</MonoSourceDirectory>
- <MonoRequiredMinimumVersion Condition=" '$(MonoRequiredMinimumVersion)' == '' ">5.18.0</MonoRequiredMinimumVersion>
- <MonoRequiredMaximumVersion Condition=" '$(MonoRequiredMaximumVersion)' == '' ">5.19.0</MonoRequiredMaximumVersion>
+ <MonoRequiredMinimumVersion Condition=" '$(MonoRequiredMinimumVersion)' == '' ">5.20.0</MonoRequiredMinimumVersion>
+ <MonoRequiredMaximumVersion Condition=" '$(MonoRequiredMaximumVersion)' == '' ">5.21.0</MonoRequiredMaximumVersion>
<IgnoreMaxMonoVersion Condition=" '$(IgnoreMaxMonoVersion)' == '' ">True</IgnoreMaxMonoVersion>
<MonoRequiredDarwinMinimumVersion>$(MonoRequiredMinimumVersion).0</MonoRequiredDarwinMinimumVersion>
<LinkerSourceDirectory>$(MSBuildThisFileDirectory)external\mono\external\linker</LinkerSourceDirectory>
diff --git a/build-tools/dependencies/dependencies.projitems b/build-tools/dependencies/dependencies.projitems
index c9a26b9e..37612ea3 100644
--- a/build-tools/dependencies/dependencies.projitems
+++ b/build-tools/dependencies/dependencies.projitems
@@ -1,7 +1,7 @@
<?xml version="1.0" encoding="utf-8"?>
<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<PropertyGroup>
- <_DarwinMonoFramework>MonoFramework-MDK-5.18.0.162.macos10.xamarin.universal.pkg</_DarwinMonoFramework>
+ <_DarwinMonoFramework>MonoFramework-MDK-5.20.0.81.macos10.xamarin.universal.pkg</_DarwinMonoFramework>
<_AptGetInstall>apt-get -f -u install</_AptGetInstall>
</PropertyGroup>
<ItemGroup>
@@ -59,7 +59,7 @@
<MaximumVersion Condition=" '$(IgnoreMaxMonoVersion)' == '' Or '$(IgnoreMaxMonoVersion)' == 'False' " >$(MonoRequiredMaximumVersion)</MaximumVersion>
<DarwinMinimumVersion>$(MonoRequiredDarwinMinimumVersion)</DarwinMinimumVersion>
<CurrentVersionCommand>$(MSBuildThisFileDirectory)..\scripts\mono-version</CurrentVersionCommand>
- <DarwinMinimumUrl>bc9d709e70/$(_DarwinMonoFramework)</DarwinMinimumUrl>
+ <DarwinMinimumUrl>3d170b9afe/$(_DarwinMonoFramework)</DarwinMinimumUrl>
<DarwinInstall>installer -pkg "$(AndroidToolchainCacheDirectory)\$(_DarwinMonoFramework)" -target /</DarwinInstall>
</RequiredProgram>
</ItemGroup>
diff --git a/external/mono b/external/mono
index 51c4f454..64c566d8 160000
--- a/external/mono
+++ b/external/mono
@@ -1 +1 @@
-Subproject commit 51c4f454022c7a94115a82998556e92d4b8724a2
+Subproject commit 64c566d8da21a29215c8abfc90fafe9e825404bf
```
* [mono] Change source path for System.IO.Compression test assets
* [mono] Bump to include libmono-native.so
This is needed because Android on Mono 2018-10 uses System.Native CoreFX code for the first time, via System.IO.FileSystemWatcher
* Bump to mono/mono/2018-10@e632442
* Fix build
This commit (9f08ff5f32) was somehow lost or reverted(!) So it had to be re-applied.
* Bump to mono/mono/2018-10@15fb740
* Run XUnit tests for System.dll
These tests include operational tests for the CoreFX FileSystemWatcher and so are also a canary for System.Native issues (at 2018-10, mscorlib.dll still does not use System.Native/libmono-native code)
* Revert "Revert unintended changes in previous merge"
This reverts commit 5d65346dff.
* Revert "Revert "Revert unintended changes in previous merge""
(So, re-apply "Revert unintended changes in previous merge")
This reverts commit e1a31da77f.
This commit broke the build (NUnit test suites crash).
* [mono-runtimes] Bundle libmono-native
* Include 'libmono-native.so' in .apk
* Bump to mono/mono/2018-10@4a7563ff852
* Test bump
* Test bump
* Test bump
* Remove some of Mono.Android's System.Drawing types which for monodroid is now available via Mono's System.Drawing.Common assembly + CoreFX implementation
The types removed from this codebase's side were the ones now conflicting with the System.Drawing.Common assembly as of 003bf5dd60
* Add assembly ref to replace removed types
* More of the 'replacing types'
* Another bump for:
[monodroid] Forward the Mono.Android.dll System.Drawing types from System.Drawing.dll too, as that's where Xamarin.Forms.Core.dll (as used by Xamarin Android) expects them to be.
* Another bump for:
[netstandard] Update for Mono.Android change
* Bump 'external/xamarin-android-api-compatibility' for c704b2a928
* Update Mono Mac SDK package version used
* Complete 'Update Mono Mac SDK package version used'
* [Mono.Android] Add type forwarders for the removed System.Drawing types
* [tests] repro breaking change with System.Drawing.Common
This test was originally failing to build with:
The "LinkAssemblies" task failed unexpectedly.
Java.Interop.Tools.Diagnostics.XamarinAndroidException: error XA2006: Could not resolve reference to 'System.Drawing.Color' (defined in assembly 'Acr.UserDialogs.Interface, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null') with scope 'Splat, Version=2.0.0.0, Culture=neutral, PublicKeyToken=null'. When the scope is different from the defining assembly, it usually means that the type is forwarded. ---> Mono.Cecil.ResolutionException: Failed to resolve System.Drawing.Color
at Mono.Linker.Steps.MarkStep.HandleUnresolvedType (Mono.Cecil.TypeReference reference) [0x0001a] in <aa2861304e8349a69bf6f3dda2b1b5bd>:0
at Mono.Linker.Steps.MarkStep.MarkType (Mono.Cecil.TypeReference reference) [0x00055] in <aa2861304e8349a69bf6f3dda2b1b5bd>:0
at MonoDroid.Tuner.MonoDroidMarkStep.MarkType (Mono.Cecil.TypeReference reference) [0x00001] in <aa2861304e8349a69bf6f3dda2b1b5bd>:0
at Mono.Linker.Steps.MarkStep.MarkGenericArguments (Mono.Cecil.IGenericInstance instance) [0x00018] in <aa2861304e8349a69bf6f3dda2b1b5bd>:0
at Mono.Linker.Steps.MarkStep.GetOriginalType (Mono.Cecil.TypeReference type) [0x00013] in <aa2861304e8349a69bf6f3dda2b1b5bd>:0
at Mono.Linker.Steps.MarkStep.MarkType (Mono.Cecil.TypeReference reference) [0x00010] in <aa2861304e8349a69bf6f3dda2b1b5bd>:0
at MonoDroid.Tuner.MonoDroidMarkStep.MarkType (Mono.Cecil.TypeReference reference) [0x00001] in <aa2861304e8349a69bf6f3dda2b1b5bd>:0
at Mono.Linker.Steps.MarkStep.MarkField (Mono.Cecil.FieldReference reference) [0x00058] in <aa2861304e8349a69bf6f3dda2b1b5bd>:0
at Mono.Linker.Steps.MarkStep.InitializeFields (Mono.Cecil.TypeDefinition type) [0x00028] in <aa2861304e8349a69bf6f3dda2b1b5bd>:0
at Mono.Linker.Steps.MarkStep.InitializeType (Mono.Cecil.TypeDefinition type) [0x0006f] in <aa2861304e8349a69bf6f3dda2b1b5bd>:0
at Mono.Linker.Steps.MarkStep.InitializeAssembly (Mono.Cecil.AssemblyDefinition assembly) [0x00034] in <aa2861304e8349a69bf6f3dda2b1b5bd>:0
at Mono.Linker.Steps.MarkStep.Initialize () [0x00016] in <aa2861304e8349a69bf6f3dda2b1b5bd>:0
at Mono.Linker.Steps.MarkStep.Process (Mono.Linker.LinkContext context) [0x00008] in <aa2861304e8349a69bf6f3dda2b1b5bd>:0
at MonoDroid.Tuner.MonoDroidMarkStep.Process (Mono.Linker.LinkContext context) [0x0000d] in <aa2861304e8349a69bf6f3dda2b1b5bd>:0
at Mono.Linker.Pipeline.ProcessStep (Mono.Linker.LinkContext context, Mono.Linker.Steps.IStep step) [0x0000f] in <aa2861304e8349a69bf6f3dda2b1b5bd>:0
at Mono.Linker.Pipeline.Process (Mono.Linker.LinkContext context) [0x00011] in <aa2861304e8349a69bf6f3dda2b1b5bd>:0
at MonoDroid.Tuner.Linker.Run (Mono.Linker.Pipeline pipeline, Mono.Linker.LinkContext context) [0x00001] in <aa2861304e8349a69bf6f3dda2b1b5bd>:0
at MonoDroid.Tuner.Linker.Process (MonoDroid.Tuner.LinkerOptions options, Mono.Linker.ILogger logger, Mono.Linker.LinkContext& context) [0x0007c] in <aa2861304e8349a69bf6f3dda2b1b5bd>:0
at Xamarin.Android.Tasks.LinkAssemblies.Execute (Java.Interop.Tools.Cecil.DirectoryAssemblyResolver res) [0x0030c] in <aa2861304e8349a69bf6f3dda2b1b5bd>:0
--- End of inner exception stack trace ---
at Java.Interop.Tools.Diagnostics.Diagnostic.Error (System.Int32 code, System.Exception innerException, System.String message, System.Object[] args) [0x0000a] in <8df3e814509d4dc7bbff2ee4d062e14e>:0
at Xamarin.Android.Tasks.LinkAssemblies.Execute (Java.Interop.Tools.Cecil.DirectoryAssemblyResolver res) [0x004cf] in <aa2861304e8349a69bf6f3dda2b1b5bd>:0
at Xamarin.Android.Tasks.LinkAssemblies.Execute () [0x0001e] in <aa2861304e8349a69bf6f3dda2b1b5bd>:0
at Microsoft.Build.BackEnd.TaskExecutionHost.Microsoft.Build.BackEnd.ITaskExecutionHost.Execute () [0x00029] in <1c1ea286ad81486b85647b7901984451>:0
at Microsoft.Build.BackEnd.TaskBuilder.ExecuteInstantiatedTask (Microsoft.Build.BackEnd.ITaskExecutionHost taskExecutionHost, Microsoft.Build.BackEnd.Logging.TaskLoggingContext taskLoggingContext, Microsoft.Build.BackEnd.TaskHost taskHost, Microsoft.Build.BackEnd.ItemBucket bucket, Microsoft.Build.BackEnd.TaskExecutionMode howToExecuteTask) [0x00212] in <1c1ea286ad81486b85647b7901984451>:0
We should go ahead and add the test to catch this breakage sooner.
* [Mono.Android] add missing type foward to System.Drawing.Color
* Bump mono
Commit list for mono/mono:
* mono/mono@78e155f252 [2018-10] Fix mono conc hashtable lookup endless loop (#13293)
* mono/mono@af19147d0b [Android] Fix runtime loading of DSOs for 64-bit processes
* mono/mono@e752907c5b [2018-10] [interp] mixed mode fixes for delegates (#13260)
* mono/mono@68fb0e0c9c [mini] Inline Selector.GetHandle for tvOS and watchOS assemblies as well. (#13255)
* mono/mono@1f81468bd5 Update csproj file for System.Drawing.Common facade
* mono/mono@74cfbd2bb4 [sdks] Use throttle() instead of lock()
* mono/mono@f08a9ab1d2 Revert "[sdks] Make sure to run only one build per type per node"
* mono/mono@3151b5cea8 [Facades] Add System.Drawing.Common back to monotouch etc profiles
* mono/mono@91e8acb924 [2018-10] [interp] compute size of generic valuetype correctly (#13227)
* mono/mono@5849bb2f22 [aot] Resolve profile against compiled assembly (#13182)
* mono/mono@f16d2d5d44 [2018-10] [aot] abort lookup early when a method isn\u0027t available in AOT image (#13179)
* mono/mono@da38bdc0e4 [ci] Use dl.internalx.com when building on private Jenkins
* mono/mono@ea3e93b191 [2018-10] AOT fixes for interp mixed mode (#13172)
* mono/mono@b4b86276a4 [System] Bump SocketTest.TestSelect1 socket timeout to 1ms (#13152)
Diff: 5bd3f0b5b4...78e155f252
* Bump mono
* Bump timeout for a step that has timed out
* Fix copy-paste mistake in Xamarin.Android.Common.targets
* Bump mono
Context: https://sqlite.org/releaselog/3_27_0.html
Upstream changes:
SQLite Release 3.27.1 On 2019-02-08
1. Fix a bug in the query optimizer: an adverse interaction between
the [OR optimization][0] and the optimization that tries to use
values read directly from an [expression index][1] instead of
recomputing the expression. Ticket [4e8e4857d32d401f][2]
Changes carried forward from version 3.27.0 (2019-02-07):
* Added the `VACUUM INTO` command
* Issue an SQLITE_WARNING message on the error log if a double-
quoted string literal is used.
* The `sqlite3_normalized_sql()` interface works on any prepared
statement created using `sqlite3_prepare_v2()` or
`sqlite3_prepare_v3()`. It is no longer necessary to use
`sqlite3_prepare_v3()` with `SQLITE_PREPARE_NORMALIZE` in order
to use `sqlite3_normalized_sql()`.
* Added the remove_diacritics=2 option to FTS3 and FTS5.
* Added the `SQLITE_PREPARE_NO_VTAB` option to
`sqlite3_prepare_v3()`. Use that option to prevent circular
references to shadow tables from causing resource leaks.
* Enhancements to the `sqlite3_deserialize()` interface:
* Add the `SQLITE_FCNTL_SIZE_LIMIT` file-control for setting an
upper bound on the size of the in-memory database created by
`sqlite3_deserialize`. The default upper bound is 1GiB, or
whatever value is specified by
`sqlite3_config(SQLITE_CONFIG_MEMDB_MAXSIZE)` and/or
`SQLITE_MEMDB_DEFAULT_MAXSIZE`.
* Honor the `SQLITE_DESERIALIZE_READONLY` flag, which was
previously described in the documentation, but was previously
a no-op.
* Enhance the "deserialize" command of the TCL Interface to
give it new "--maxsize N" and "--readonly BOOLEAN" options.
* Enhancements to the CLI, mostly to support testing and debugging
of the SQLite library itself:
* Add support for ".open --hexdb". The "dbtotxt" utility program
used to generate the text for the "hexdb" is added to the
source tree.
* Add support for the "--maxsize N" option on
".open --deserialize".
* Add the "--memtrace" command-line option, to show all memory
allocations and deallocations.
* Add the ".eqp trace" option on builds with `SQLITE_DEBUG`, to
enable bytecode program listing with indentation and PRAGMA
`vdbe_trace` all in one step.
* Add the ".progress" command for accessing the
`sqlite3_progress_handler()` interface.
* Add the "--async" option to the ".backup" command.
* Add options "--expanded", "--normalized", "--plain",
"--profile", "--row", "--stmt", and "--close" to the ".trace"
command.
* Increased robustness against malicious SQL that is run against a maliciously
corrupted database.
Bug fixes:
* Do not use a partial index to do a table scan on an IN operator.
* Fix the query flattener so that it works on queries that contain
subqueries that use window functions.
* Ensure that ALTER TABLE modifies table and column names embedded
in WITH clauses that are part of views and triggers.
* Fix a parser bug that prevented the use of parentheses around
table-valued functions.
* Fix a problem with the OR optimization on indexes on expressions.
* Fix a problem with the LEFT JOIN strength reduction optimization
in which the optimization was being applied inappropriately due to
an IS NOT NULL operator.
* Fix the REPLACE command so that it is no longer able to sneak a
NULL value into a NOT NULL column even if the NOT NULL column has
a default value of NULL.
* Fix a problem with the use of window functions used within
correlated subqueries.
* Fix the ALTER TABLE RENAME COLUMN command so that it works for
tables that have redundant UNIQUE constraints.
* Fix a bug that caused zeroblob values to be truncated when
inserted into a table that uses an expression index.
[0]: https://sqlite.org/optoverview.html#or_opt
[1]: https://sqlite.org/expridx.html
[2]: https://www.sqlite.org/src/info/4e8e4857d32d401f
Context: https://www.reddit.com/r/androiddev/comments/a4wpuk/for_anyone_wondering_about_todays_failed_builds/
Context: https://issuetracker.google.com/issues/120759347
Context: https://mvnrepository.com/artifact/com.android.tools/r8/1.2.52
We are currently building r8 from source, which has been failing on
occasion:
_BuildR8:
python tools/gradle.py r8 --stacktrace --no-daemon
Running: download_from_google_storage -n -b r8-deps -u -s /mnt/jenkins/workspace/xamarin-anroid-linux-pr-builder/xamarin-android/external/r8/third_party/gradle/gradle.tar.gz.sha1
0> Downloading /mnt/jenkins/workspace/xamarin-anroid-linux-pr-builder/xamarin-android/external/r8/third_party/gradle/gradle.tar.gz...
0> Extracting 196 entries from /mnt/jenkins/workspace/xamarin-anroid-linux-pr-builder/xamarin-android/external/r8/third_party/gradle/gradle.tar.gz to /mnt/jenkins/workspace/xamarin-anroid-linux-pr-builder/xamarin-android/external/r8/third_party/gradle/gradle
Downloading 1 files took 7.041677 second(s)
Running: /mnt/jenkins/workspace/xamarin-anroid-linux-pr-builder/xamarin-android/external/r8/third_party/gradle/gradle/bin/gradle r8 --stacktrace --no-daemon
:buildSrc:clean UP-TO-DATE
:buildSrc:compileJava
:buildSrc:compileGroovy NO-SOURCE
:buildSrc:processResources NO-SOURCE
:buildSrc:classes
:buildSrc:jar
:buildSrc:assemble
:buildSrc:compileTestJava NO-SOURCE
:buildSrc:compileTestGroovy NO-SOURCE
:buildSrc:processTestResources NO-SOURCE
:buildSrc:testClasses UP-TO-DATE
:buildSrc:test NO-SOURCE
:buildSrc:check UP-TO-DATE
:buildSrc:build
FAILURE: Build failed with an exception.
* What went wrong:
A problem occurred configuring root project 'r8'.
> Could not resolve all dependencies for configuration ':classpath'.
> Could not find com.android.tools:common:25.3.1.
Searched in the following locations:
https://repo1.maven.org/maven2/com/android/tools/common/25.3.1/common-25.3.1.pomhttps://repo1.maven.org/maven2/com/android/tools/common/25.3.1/common-25.3.1.jarhttps://jcenter.bintray.com/com/android/tools/common/25.3.1/common-25.3.1.pomhttps://jcenter.bintray.com/com/android/tools/common/25.3.1/common-25.3.1.jarhttps://plugins.gradle.org/m2/com/android/tools/common/25.3.1/common-25.3.1.pomhttps://plugins.gradle.org/m2/com/android/tools/common/25.3.1/common-25.3.1.jar
Required by:
project : > com.cookpad.android.licensetools:license-tools-plugin:0.23.0 > com.android.tools.build:gradle:2.3.1 > com.android.tools.build:gradle-core:2.3.1 > com.android.tools.build:builder:2.3.1
project : > com.cookpad.android.licensetools:license-tools-plugin:0.23.0 > com.android.tools.build:gradle:2.3.1 > com.android.tools.build:gradle-core:2.3.1 > com.android.tools.build:builder:2.3.1 > com.android.tools.build:manifest-merger:25.3.1
project : > com.cookpad.android.licensetools:license-tools-plugin:0.23.0 > com.android.tools.build:gradle:2.3.1 > com.android.tools.build:gradle-core:2.3.1 > com.android.tools.build:builder:2.3.1 > com.android.tools.ddms:ddmlib:25.3.1
project : > com.cookpad.android.licensetools:license-tools-plugin:0.23.0 > com.android.tools.build:gradle:2.3.1 > com.android.tools.build:gradle-core:2.3.1 > com.android.tools.build:builder:2.3.1 > com.android.tools.analytics-library:shared:25.3.1
project : > com.cookpad.android.licensetools:license-tools-plugin:0.23.0 > com.android.tools.build:gradle:2.3.1 > com.android.tools.build:gradle-core:2.3.1 > com.android.tools.build:builder:2.3.1 > com.android.tools.analytics-library:tracker:25.3.1
project : > com.cookpad.android.licensetools:license-tools-plugin:0.23.0 > com.android.tools.build:gradle:2.3.1 > com.android.tools.build:gradle-core:2.3.1 > com.android.tools.build:builder:2.3.1 > com.android.tools:sdklib:25.3.1 > com.android.tools.layoutlib:layoutlib-api:25.3.1
project : > com.cookpad.android.licensetools:license-tools-plugin:0.23.0 > com.android.tools.build:gradle:2.3.1 > com.android.tools.build:gradle-core:2.3.1 > com.android.tools.build:builder:2.3.1 > com.android.tools:sdklib:25.3.1 > com.android.tools:dvlib:25.3.1
project : > com.cookpad.android.licensetools:license-tools-plugin:0.23.0 > com.android.tools.build:gradle:2.3.1 > com.android.tools.build:gradle-core:2.3.1 > com.android.tools.build:builder:2.3.1 > com.android.tools:sdklib:25.3.1 > com.android.tools:repository:25.3.1
* Try:
Run with --info or --debug option to get more log output.
* Exception is:
org.gradle.api.ProjectConfigurationException: A problem occurred configuring root project 'r8'.
I have seen this on Linux and Windows PR builds, which we think is
caused by Google [removing `com.android.tools:common:25.3.1`][0] from
maven. Some of our local machines are still working, likely due to
local caching by gradle. I don't think r8 as-is will build on a
clean machine any longer!
In either case, we've been wanting to use `r8.jar` from maven. It is
a little concerning that Google has pulled packages from there, but
something that is working *now* is better than not working at all.
Since we no longer build `r8` from source, I was able to delete a lot
of code. We don't need to download Chromium's `depot_tools` any more.
`r8` from maven also lists a different license:
https://opensource.org/licenses/BSD-3-Clause
I copied the contents to `src/r8/LICENSE` and moved `r8.tpnitems` to a
more appropriate location.
~~ gradle ~~
When downloading the file directly from maven, if you tried to use it,
you would run into:
_CompileToDalvikWithD8:
C:\Program Files (x86)\Java\jdk1.8.0_161\bin\java.exe -Xmx1G -classpath C:\Users\myuser\Desktop\Git\xamarin-android\bin\Debug\lib\xamarin.android\xbuild\Xamarin\Android\r8.jar com.android.tools.r8.D8 --debug --min-api 21 --output obj\Debug\android\bin --lib C:\Users\myuser\android-toolchain\sdk\platforms\android-28\android.jar obj\Debug\android\bin\classes.zip C:\Users\myuser\Desktop\Git\xamarin-android\bin\Debug\lib\xamarin.android\xbuild-frameworks\MonoAndroid\v9.0\mono.android.jar
Exception in thread "main" java.lang.NoClassDefFoundError: com/google/common/collect/ImmutableList
at com.android.tools.r8.D8CommandParser$OrderedClassFileResourceProvider$Builder.<init>(D8CommandParser.java:30)
at com.android.tools.r8.D8CommandParser$OrderedClassFileResourceProvider.builder(D8CommandParser.java:58)
at com.android.tools.r8.D8CommandParser.parse(D8CommandParser.java:143)
at com.android.tools.r8.D8CommandParser.parse(D8CommandParser.java:120)
at com.android.tools.r8.D8Command.parse(D8Command.java:177)
at com.android.tools.r8.D8.run(D8.java:98)
at com.android.tools.r8.D8.lambda$main$3(D8.java:122)
at com.android.tools.r8.utils.ExceptionUtils.withMainProgramHandler(ExceptionUtils.java:75)
at com.android.tools.r8.D8.main(D8.java:122)
Caused by: java.lang.ClassNotFoundException: com.google.common.collect.ImmutableList
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:338)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 9 more
r8 has ~9 dependencies, which are not included in `r8.jar` itself.
So to workaround this:
1. Created a "dummy" java project using gradle.
2. Setup a dependency on:
`group: 'com.android.tools', name: 'r8', version: '1.2.52'`
3. Setup the `jar` gradle task to zip all dependencies into a
self-contained jar file:
from {
configurations.compile.collect { it.isDirectory() ? it : zipTree(it) }
}
When running:
.\gradlew jar
A `build\libs\r8.jar` file is produced, containing all dependencies.
I also consolidated our existing use of gradle:
- Created a new `build-tools\gradle` directory.
- Keep one copy of `gradlew`, `gradlew.bat`, and `gradle-wrapper.jar`.
- Added new properties to `Configuration.props`: `$(GradleHome)`,
`$(GradleWPath)`, and `$(GradleArgs)`.
- Places that call gradle now uses these new properties and set
`APP_HOME=$(GradleHome)`.
[0]: https://issuetracker.google.com/issues/120759347
Fixes: https://github.com/xamarin/xamarin-android/issues/1423
Fixes: https://github.com/xamarin/xamarin-android/issues/2040
Bumps to r8/d8-1.2@125b72.
~~ Overview ~~
This enables `d8` and `r8` integration for Xamarin.Android.
`d8` is a "next-generation" dex compiler (over `dx`), and
`r8` is a replacement for ProGuard. For full details on the feature,
see `Documentation/guides/D8andR8.md`.
~~ MSBuild targets changes ~~
New MSBuild properties include:
- `$(AndroidDexTool)` - an enum-style property with options `dx`
and `d8`. Defaults to `dx`.
- `$(AndroidLinkTool)` - an enum-style property, that can be left
blank to disable code shrinking.
Valid values are `proguard` and `r8`.
Existing MSBuild properties still retain the old behavior:
- `$(AndroidEnableProguard)` will set `$(AndroidLinkTool)` to
`proguard`, although it is no longer used internally by
Xamarin.Android targets.
- `$(AndroidEnableDesugar)` will default to `true` if `d8` or `r8`
are used.
New MSBuild tasks include:
- `<D8/>`: runs `d8.jar` with the required options for
Xamarin.Android.
- `<R8/>`: subclasses `<D8/>`, and adds functionality for multi-dex
and code-shrinking.
Additionally, any new MSBuild targets are placed in a new
`Xamarin.Android.D8.targets` file. This is good first step to make
`Xamarin.Android.Common.targets` smaller.
Additionally:
* `build.props` is now invalidated via the `$(AndroidDexTool)`
and `$(AndroidLinkTool)` properties, instead of
`$(AndroidEnableProguard)`
* `build.props` is invalidated via `$(AndroidEnableDesugar)`
* Refactored `$(IntermediateOutputPath)_dex_stamp` stamp file in
`_CompileToDalvikWithDx` and `_CompileToDalvikWithD8` to match the
new convention of `$(_AndroidStampDirectory)_CompileToDalvik.stamp`
* `*.dex` files weren't in `@(FileWrites)`?!
* `<CompileToDalvik/>` had a `DexOutputs` output property that was
completely unused, so I removed it. Also removed extra log messages.
~~ Test changes ~~
Tests that need to validate various combinations of properties are
now using parameters such as:
[Values ("dx", "d8")] string dexTool, [Values ("", "proguard", "r8")] string linkTool
Set on the `XamarinAndroidApplicationProject` such as:
var proj = new XamarinAndroidApplicationProject {
DexTool = dexTool,
LinkTool = linkTool,
};
In other cases, a simple `useD8` flag was added to set `DexTool="d8"`.
Since adding test parameters, exponentially causes our test cases to
expand, I removed some non-essential parameters:
- `BuildProguardEnabledProject()` dropped `useLatestSdk`, as it does
not seem applicable here (and is deprecated). Otherwise would
have 24 test cases...
- `BuildApplicationWithSpacesInPath()` dropped `isRelease` and
defaulted it to `true`. We aren't going to likely encounter
issues with spaces in a path that happen *only* in a `Debug`
build. Otherwise we would have 24 test cases here...
- `Desugar()` dropped `enableDesugar` because it is certain this
application project will not build *without* desugar. We don't
need to test this, and would have 24 test cases otherwise...
Also dropped some `[TestCaseSource]` attributes where the `[Values]`
parameter was much cleaner.
~~ Changes to test/sample projects ~~
`HelloWorld` - `$(AndroidDexTool)` set to `d8` if unspecified,
so we can track the performance benefit.
`Xamarin.Forms Integration` - uses `d8` and `$(AndroidLinkTool)` set
to `r8`, using a `proguard.cfg` file for Xamarin.Forms. Will help us
track startup performance benefits of Java code shrinking and build
performance.
`Mono.Android-Tests` - uses `d8` and `$(AndroidLinkTool)` set to
`r8`, to verify these on-device tests pass.
`Runtime-MultiDex` - `$(AndroidDexTool)` set to `d8` if unspecified,
to validate multi-dex is working properly with `d8`.
~~ xamarin-android build changes ~~
The `<DownloadUri/>` MSBuild task now has an optional `HashHeader`
property. When set, an http HEAD request is made to the URL, and the
destination file path gets a suffix added for the value.
`DownloadUri.DestinationFiles` is changed to an `[Output]` property,
as the destination paths could change due to the `HashHeader`.
Chromium's `depot_tools` are downloaded the same as all of our other
dependencies. As the `depot_tools` distribution filename is
unversioned, we use the `x-goog-hash` header to derive a "version" so
that we don't needlessly download the file.
~~ Deployment changes ~~
Three new files will need to be included in Xamarin.Android
installers:
- `d8.jar`
- `r8.jar`
- `Xamarin.Android.D8.targets`
~~ General changes ~~
* Fixed doc for `xa4304` error code
* Added `xa4305` error code for `CompileToDalvik`, `R8`, and
`<CreateMultiDexMainDexClassList/>`
* Removed log messages in `<CreateMultiDexMainDexClassList/>`
~~ Performance Comparison ~~
| MSBuild Target | Options Enabled | Time | APK size (bytes) | dex size (bytes) |
| --- | --- | ---: | ---: | ---: |
| _CompileToDalvikWithDx | n/a | 11074ms | 13378157 | 3894720 |
| _CompileToDalvikWithD8 | d8, (desugar enabled) | 8543ms | 13124205 | 3314064 |
| _CompileToDalvikWithD8 | d8, (desugar disabled) | 9550ms | 13124205 | 3314064 |
| _CompileToDalvikWithDx | multi-dex | 15632ms | 13390498 | 3916496 |
| _CompileToDalvikWithD8 | d8, multi-dex | 25979ms | 13054626 | 3264096 |
| _CompileToDalvikWithDx | proguard | 11903ms | 12804717 | 2446964 |
| _CompileToDalvikWithD8 | d8, r8 | 13799ms | 12513901 | 1835588 |
| _CompileToDalvikWithDx | multi-dex, proguard | 17279ms | 12804770 | 2449512 |
| _CompileToDalvikWithD8 | d8, multi-dex, r8 | 13792ms | 12513954 | 1837588 |
*NOTE: desugar is enabled by default with d8/r8*
I timed this builds with [this script][powershell_script], with a
"Hello World" Xamarin.Forms app.
Build logs here: [d8andr8.zip][d8andr8_zip]
One can draw their own conclusions on which options are faster,
better, smaller. See further detail in `D8andR8.md`.
[powershell_script]: 39e2854f6c/build.ps1
[d8andr8_zip]: https://github.com/xamarin/xamarin-android/files/2470385/d8andr8.zip
Co-authored-by: Atsushi Eno <atsushieno@gmail.com>
The full list of changes since our previous release which contained
SQLite 3.9.2 can be found here (it's "only" nearly 3 years of fixes):
https://sqlite.org/changes.html
Additionally, bump the base NDK API platform we use to build native
code to API-14. This is in line with the API level Mono runtime is
built with as well as with the minimum supported API in newer NDKs.
Making the change part of this commit because it also affects the way
SQLite is built.
Context: 2b863c3106
Context: 7f9a3657b6
Context: acdbe90fd8
Context: 29d778a8db
Context: ...
Every time @MSylvia branches our products for release, we get a
unnecessarily large change to `.gitmodules`, for two reasons:
1. @MSylvia's tooling uses spaces ` ` instead of tabs for
indentation, and
2. @MSylvia's tooling *sorts submodules*.
This turns a conceptually simple change -- update branch names -- into
a very *noisy* change, as the `.gitmodules` contents are re-ordered
and re-formatted.
Update `.gitmodules` so that we *start* with what the tooling expects:
sorted submodule names, and spaces for indentation.
This should allow *future* branch commits to be smaller and saner
to review.
Peter Collins
Marek Habersack
Jonathan Pobst
Jonathan Peppers
Jonathan Pryor
Gustavo Varo
Andrey Kurdyumov
Brendan Zagaeski
Dean Ellis
Jo Shields
Alexis Christoforides
Jonathan Peppers
Ludovic Henry