[Xamarin.Android.Build.Tasks] Add $(AndroidEnableAssemblyCompression) (#4686)
Currently, Xamarin.Android supports compression of managed assemblies
within the `.apk` if the app is built with
[`$(BundleAssemblies)`=True][0], with the compressed assembly data
stored within `libmonodroid_bundle_app.so` using gzip compression and
placed in an array inside the data section of the shared library.
There are two problems with this approach:
1. `mkbundle` emits C code, which requires a C compiler which requires
the full Android NDK, and thus requires Visual Studio Enterprise.
2. Reliance on Mono's `mkbundle` results in possible issues around
[filename globbing][1] such that
`Xamarin.AndroidX.AppCompat.Resources.dll` is improperly treated
as a [satellite assembly][2].
Because of (2), we are planning on [removing support][3] for
`$(BundleAssemblies)` in .NET 6 ([née .NET 5][4]), which resulted in
[some pushback][5] because `.apk` size is very important for some
customers, and the startup overheads we believed to be inherent to
`$(BundleAssemblies)` turned out to be somewhat over-estimated.
To resolve the above issues, add an assembly compression mechanism
that doesn't rely on `mkbundle` and the NDK: separately compress the
assemblies and store the compressed data within the `.apk`.
Compression is performed using the [managed implementation][6] of the
excellent [LZ4][7] algorithm. This gives us a decent compression ratio
and a much faster (de)compression speed than gzip/zlib offer. Also,
assemblies are stored directly in the APK in their usual directory,
which allows us to [**mmap**(2)][8] them in the runtime directly from
the `.apk`. The build process calculates the size required to store
the decompressed assemblies and adds a data section to
`libxamarin-app.so` which causes *Android* to allocate all the required
memory when the DSO is loaded, thus removing the need of dynamic memory
allocation and making the startup faster.
Compression is supported only in `Release` builds and is enabled by
default, but it can be turned off by setting the
`$(AndroidEnableAssemblyCompression)` MSBuild property to `False`.
Compression can be disabled for an individual assembly by setting the
`%(AndroidSkipCompression)` MSBuild item metadata to True for the
assembly in question, e.g. via:
<AndroidCustomMetaDataForReferences Include="MyAssembly.dll">
<AndroidSkipCompression>true</AssemblySkipCompression>
</AndroidCustomMetaDataForReferences>
The compressed assemblies still use their original name, e.g.
`Mono.Android.dll`, so that we don't have to perform any string
matching on the runtime in order to detect whether the assembly we are
asked to load is compressed or not. Instead, the compression code
*prepends* a short header to each `.dll` file (in pseudo C code):
struct CompressedAssemblyHeader {
uint32_t magic; // 0x5A4C4158; 'XALZ', little-endian
uint32_t descriptor_index; // Index into an internal assembly descriptor table
uint32_t uncompressed_length; // Size of assembly, uncompressed
};
The decompression code looks at the `mmap`ed data and checks whether
the above header is present. If yes, the assembly is decompressed,
otherwise it's loaded as-is.
It is important to remember that the assemblies are compressed at
build time using LZ4 block compression, which requires assembly data
to be entirely loaded into memory before compression; we do this
instead of using the LZ4 frame format to make decompression at runtime
faster. The compression output also requires a separate buffer, thus
memory consumption at *build* time will be roughly 1.5x the size of the
largest assembly, which is reused across all assemblies.
~~ Application Size ~~
A Xamarin.Forms "Hello World" application `.apk` shrinks by 27% with
this approach for a single ABI:
| Before (bytes) | LZ4 (bytes) | Δ |
|------------------:|--------------:|:---------:|
| 23,305,194 | 16,813,034 | -27.85% |
Size comparison between this commit and `.apk`s created with
`$(BundleAssemblies)` =True depends on the number of enabled ABI
targets in the application. For each ABI, `$(BundleAssemblies)`=True
creates a separate shared library, so the amount of space consumed
increases by the size of the bundle shared library.
The new compression scheme shares the compressed assemblies among all
the enabled ABIs, thus effectively creating smaller multi-ABI `.apk`s.
In the tables below, `mkbundle` refers to the APK created with
`$(BundleAssemblies)`=True, `lz4` refers to the `.apk` build with
the new compression scheme:
| ABIs | mkbundle (bytes) | LZ4 (bytes) | Δ |
|--------------------------------------:|------------------:|--------------:|---------|
| armeabi-v7a, arm64-v8a, x86, x86_64 | 27,130,240 | 16,813,034 | -38.03% |
| arm64-v8a | 7,783,449 | 8,746,878 | +11.01% |
The single API case is ~11% larger because gzip offers better
compression, at the cost of higher runtime startup overhead.
~~ Startup Performance ~~
When launching the Xamarin.Forms "Hello World" application on a
Pixel 3 XL, the use of LZ4-compressed assemblies has at worst a ~1.58%
increase in the Activity Displayed time (64-bit app w/ assembly
preload enabled), while slightly faster on 32-bit apps, but is *always*
faster than the mkbundle startup time for all configurations:
| | | | | LZ4 vs | LZ4 vs |
| Description | None (ms) | mkbundle (ms) | LZ4 (ms) | None Δ | mkbundle Δ |
|----------------------------------:|----------:|--------------:|----------:|:--------:|:----------:|
| preload enabled; 32-bit build | 795.8 | 855.6 | 783.8 | -0.25% ✓ | -7.22% ✓ |
| preload disabled; 32-bit build | 777.1 | 843.0 | 780.5 | +0.44% ✗ | -7.41% ✓ |
| preload enabled; 64-bit build | 779.0 | 843.0 | 791.5 | +1.58% ✗ | -6.82% ✓ |
| preload disabled; 64-bit build | 776.0 | 841.6 | 781.5 | +0.69% ✗ | -7.15% ✓ |
[0]: https://docs.microsoft.com/en-us/xamarin/android/deploy-test/release-prep/?tabs=windows#bundle-assemblies-into-native-code
[1]: https://github.com/xamarin/AndroidX/issues/64
[2]: 9b4736d4c2/mcs/tools/mkbundle/mkbundle.cs (L1315-L1317)
[3]: https://github.com/xamarin/AndroidX/issues/64#issuecomment-609970584
[4]: https://devblogs.microsoft.com/dotnet/announcing-net-5-preview-4-and-our-journey-to-one-net/
[5]: https://github.com/xamarin/AndroidX/issues/64#issuecomment-610002467
[6]: https://www.nuget.org/packages/K4os.Compression.LZ4/
[7]: https://github.com/lz4/lz4
[8]: https://linux.die.net/man/2/mmap
This commit is contained in:
Marek Habersack
GitHub
Родитель
5c6599c937
Коммит
d236af5453
|
|
@ -39,3 +39,7 @@
|
|||
path = external/xamarin-android-tools
|
||||
url = https://github.com/xamarin/xamarin-android-tools
|
||||
branch = master
|
||||
[submodule "lz4"]
|
||||
path = external/lz4
|
||||
url = https://github.com/lz4/lz4.git
|
||||
branch = master
|
||||
|
|
|
|||
|
|
@ -158,3 +158,7 @@ Overridable MSBuild properties include:
|
|||
Xamarin.Android install to the local build output. This enables proprietary
|
||||
features such as debugging and fast deployment. Since a "normal" OSS build would
|
||||
not include proprietary files, this flag also emits a warning when enabled.
|
||||
|
||||
* `$(AndroidEnableAssemblyCompression)`: Defaults to `True`. When enabled, all the
|
||||
assemblies placed in the APK will be compressed in `Release` builds. `Debug`
|
||||
builds are not affected.
|
||||
|
|
|
|||
|
|
@ -0,0 +1,66 @@
|
|||
### Smaller app package sizes
|
||||
|
||||
This version introduces compression of managed assemblies by default for Release
|
||||
configuration builds, resulting in significantly smaller APK and App Bundle
|
||||
sizes. Assemblies are compressed with the [LZ4][lz4] algorithm during builds
|
||||
and then decompressed on device during app startup.
|
||||
|
||||
For a small example Xamarin.Forms application, this reduced the APK size from
|
||||
about 23 megabytes to about 17 megabytes while only increasing the time to
|
||||
display the first page of the app from about 780 milliseconds to about 790
|
||||
milliseconds in the least favorable configuration.
|
||||
|
||||
If needed, the new behavior can been disabled for a particular project by
|
||||
setting the `AndroidEnableAssemblyCompression` MSBuild property to `false` in
|
||||
the _.csproj_ file:
|
||||
|
||||
```xml
|
||||
<PropertyGroup>
|
||||
<AndroidEnableAssemblyCompression>false</AndroidEnableAssemblyCompression>
|
||||
</PropertyGroup>
|
||||
```
|
||||
|
||||
> [!NOTE]
|
||||
> This feature is intended to replace the older **Bundle assemblies into native
|
||||
> code** Visual Studio Enterprise feature for purposes of app size savings. The
|
||||
> `AndroidEnableAssemblyCompression` property takes precedence if both features
|
||||
> are enabled. Project authors who no longer need the **Bundle assemblies into
|
||||
> native code** feature enabled can now disable it or remove the
|
||||
> `BundleAssemblies` MSBuild property from the _.csproj_ file:
|
||||
>
|
||||
> ```diff
|
||||
> <PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Release|AnyCPU' ">
|
||||
> <DebugSymbols>True</DebugSymbols>
|
||||
> <DebugType>portable</DebugType>
|
||||
> <Optimize>True</Optimize>
|
||||
> <OutputPath>bin\Release\</OutputPath>
|
||||
> <DefineConstants>TRACE</DefineConstants>
|
||||
> <ErrorReport>prompt</ErrorReport>
|
||||
> <WarningLevel>4</WarningLevel>
|
||||
> <AndroidManagedSymbols>true</AndroidManagedSymbols>
|
||||
> <AndroidUseSharedRuntime>False</AndroidUseSharedRuntime>
|
||||
> <AndroidLinkMode>SdkOnly</AndroidLinkMode>
|
||||
> <EmbedAssembliesIntoApk>True</EmbedAssembliesIntoApk>
|
||||
> - <BundleAssemblies>true</BundleAssemblies>
|
||||
> </PropertyGroup>
|
||||
> ```
|
||||
|
||||
#### Background information
|
||||
|
||||
For comparison, for the small test Xamarin.Forms application, the **Bundle
|
||||
assemblies into native code** feature increases the APK size from about 23
|
||||
megabytes to about 26 megabytes while increasing the time to display the
|
||||
first page of the app from about 780 milliseconds to about 850 milliseconds
|
||||
in the least favorable configuration. Size comparison is for an application
|
||||
which enables **all** of the ABIs supported by Xamarin.Android. It needs to
|
||||
be noted that in this scenario, the **Bundle assemblies into native code**
|
||||
feature will add a copy of the shared library with compressed assemblies per
|
||||
ABI, thus the size increase. The new compression shares all the compressed
|
||||
assemblies between all the ABIs, thus enabling a new one won't cause the APK
|
||||
to grow. If sizes are compared for application shipping just a single ABI, the
|
||||
**Bundle assemblies into native code** feature decreases the the APK size from
|
||||
around 16 megabytes to around 7.5 megabytes, while the new scheme decreases the
|
||||
size to around 8 megabytes. Display times are not affected by the number of ABIs
|
||||
present in the APK.
|
||||
|
||||
[lz4]: https://github.com/lz4/lz4
|
||||
|
|
@ -257,6 +257,7 @@
|
|||
<_MSBuildFiles Include="$(MSBuildSrcDir)\Xamarin.Android.VisualBasic.targets" />
|
||||
<_MSBuildFiles Include="$(MSBuildSrcDir)\Xamarin.Build.AsyncTask.dll" />
|
||||
<_MSBuildFiles Include="$(MSBuildSrcDir)\Xamarin.Build.AsyncTask.pdb" />
|
||||
<_MSBuildFiles Include="$(MSBuildSrcDir)\K4os.Compression.LZ4.dll" />
|
||||
</ItemGroup>
|
||||
<ItemGroup>
|
||||
<_MSBuildTargetsSrcFiles Include="$(MSBuildTargetsSrcDir)\Xamarin.Android.Bindings.After.targets" />
|
||||
|
|
|
|||
|
|
@ -0,0 +1,40 @@
|
|||
using System;
|
||||
using System.Collections.Generic;
|
||||
using System.IO;
|
||||
|
||||
namespace Xamarin.Android.Prepare
|
||||
{
|
||||
[TPN]
|
||||
class MiloszKrajewski_K4os_Compression_LZ4_TPN : ThirdPartyNotice
|
||||
{
|
||||
static readonly Uri url = new Uri ("https://github.com/MiloszKrajewski/K4os.Compression.LZ4/");
|
||||
|
||||
public override string LicenseFile => String.Empty;
|
||||
public override string Name => "MiloszKrajewski/K4os.Compression.LZ4";
|
||||
public override Uri SourceUrl => url;
|
||||
public override string LicenseText => @"
|
||||
MIT License
|
||||
|
||||
Copyright (c) 2017 Milosz Krajewski
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the ""Software""), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED ""AS IS"", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
";
|
||||
public override bool Include (bool includeExternalDeps, bool includeBuildDeps) => includeExternalDeps;
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,20 @@
|
|||
using System;
|
||||
using System.Collections.Generic;
|
||||
using System.IO;
|
||||
|
||||
namespace Xamarin.Android.Prepare
|
||||
{
|
||||
[TPN]
|
||||
class lz4_lz4_TPN : ThirdPartyNotice
|
||||
{
|
||||
static readonly Uri url = new Uri ("https://github.com/lz4/lz4/");
|
||||
static readonly string licenseFile = Path.Combine (Configurables.Paths.ExternalDir, "lz4", "lib", "LICENSE");
|
||||
|
||||
public override string LicenseFile => licenseFile;
|
||||
public override string Name => "lz4/lz4";
|
||||
public override Uri SourceUrl => url;
|
||||
public override string LicenseText => String.Empty;
|
||||
|
||||
public override bool Include (bool includeExternalDeps, bool includeBuildDeps) => includeExternalDeps;
|
||||
}
|
||||
}
|
||||
|
|
@ -157,6 +157,8 @@
|
|||
<Compile Include="ThirdPartyNotices\aapt2.cs" />
|
||||
<Compile Include="ThirdPartyNotices\bundletool.cs" />
|
||||
<Compile Include="ThirdPartyNotices\Java.Interop.cs" />
|
||||
<Compile Include="ThirdPartyNotices\K4os.Compression.LZ4.cs" />
|
||||
<Compile Include="ThirdPartyNotices\lz4.cs" />
|
||||
<Compile Include="ThirdPartyNotices\mono.cs" />
|
||||
<Compile Include="ThirdPartyNotices\opentk.cs" />
|
||||
<Compile Include="ThirdPartyNotices\proguard.cs" />
|
||||
|
|
|
|||
|
|
@ -0,0 +1 @@
|
|||
Subproject commit fdf2ef5809ca875c454510610764d9125ef2ebbd
|
||||
|
|
@ -84,6 +84,10 @@ namespace Xamarin.Android.Tasks
|
|||
public string UncompressedFileExtensions { get; set; }
|
||||
public bool InterpreterEnabled { get; set; }
|
||||
|
||||
// Make it required after https://github.com/xamarin/monodroid/pull/1094 is merged
|
||||
//[Required]
|
||||
public bool EnableCompression { get; set; }
|
||||
|
||||
[Output]
|
||||
public ITaskItem[] OutputFiles { get; set; }
|
||||
|
||||
|
|
@ -289,7 +293,15 @@ namespace Xamarin.Android.Tasks
|
|||
private void AddAssemblies (ZipArchiveEx apk)
|
||||
{
|
||||
bool debug = _Debug;
|
||||
bool compress = !debug && EnableCompression;
|
||||
bool use_shared_runtime = String.Equals (UseSharedRuntime, "true", StringComparison.OrdinalIgnoreCase);
|
||||
string sourcePath;
|
||||
AssemblyCompression.AssemblyData compressedAssembly = null;
|
||||
IDictionary<string, CompressedAssemblyInfo> compressedAssembliesInfo = null;
|
||||
|
||||
if (compress) {
|
||||
compressedAssembliesInfo = BuildEngine4.GetRegisteredTaskObject (CompressedAssemblyInfo.CompressedAssembliesInfoKey, RegisteredTaskObjectLifetime.Build) as IDictionary<string, CompressedAssemblyInfo>;
|
||||
}
|
||||
|
||||
int count = 0;
|
||||
foreach (ITaskItem assembly in ResolvedUserAssemblies) {
|
||||
|
|
@ -300,8 +312,11 @@ namespace Xamarin.Android.Tasks
|
|||
if (MonoAndroidHelper.IsReferenceAssembly (assembly.ItemSpec)) {
|
||||
Log.LogCodedWarning ("XA0107", assembly.ItemSpec, 0, Properties.Resources.XA0107, assembly.ItemSpec);
|
||||
}
|
||||
|
||||
sourcePath = CompressAssembly (assembly);
|
||||
|
||||
// Add assembly
|
||||
AddFileToArchiveIfNewer (apk, assembly.ItemSpec, GetTargetDirectory (assembly.ItemSpec) + "/" + Path.GetFileName (assembly.ItemSpec), compressionMethod: UncompressedMethod);
|
||||
AddFileToArchiveIfNewer (apk, sourcePath, GetTargetDirectory (assembly.ItemSpec) + "/" + Path.GetFileName (assembly.ItemSpec), compressionMethod: UncompressedMethod);
|
||||
|
||||
// Try to add config if exists
|
||||
var config = Path.ChangeExtension (assembly.ItemSpec, "dll.config");
|
||||
|
|
@ -336,10 +351,13 @@ namespace Xamarin.Android.Tasks
|
|||
Log.LogDebugMessage ($"Skipping {assembly.ItemSpec} due to 'AndroidSkipAddToPackage' == 'true' ");
|
||||
continue;
|
||||
}
|
||||
|
||||
if (MonoAndroidHelper.IsReferenceAssembly (assembly.ItemSpec)) {
|
||||
Log.LogCodedWarning ("XA0107", assembly.ItemSpec, 0, Properties.Resources.XA0107, assembly.ItemSpec);
|
||||
}
|
||||
AddFileToArchiveIfNewer (apk, assembly.ItemSpec, AssembliesPath + Path.GetFileName (assembly.ItemSpec), compressionMethod: UncompressedMethod);
|
||||
|
||||
sourcePath = CompressAssembly (assembly);
|
||||
AddFileToArchiveIfNewer (apk, sourcePath, AssembliesPath + Path.GetFileName (assembly.ItemSpec), compressionMethod: UncompressedMethod);
|
||||
var config = Path.ChangeExtension (assembly.ItemSpec, "dll.config");
|
||||
AddAssemblyConfigEntry (apk, config);
|
||||
// Try to add symbols if Debug
|
||||
|
|
@ -360,6 +378,50 @@ namespace Xamarin.Android.Tasks
|
|||
count = 0;
|
||||
}
|
||||
}
|
||||
|
||||
void EnsureCompressedAssemblyData (string sourcePath, uint descriptorIndex)
|
||||
{
|
||||
if (compressedAssembly == null)
|
||||
compressedAssembly = new AssemblyCompression.AssemblyData (sourcePath, descriptorIndex);
|
||||
else
|
||||
compressedAssembly.SetData (sourcePath, descriptorIndex);
|
||||
}
|
||||
|
||||
string CompressAssembly (ITaskItem assembly)
|
||||
{
|
||||
if (!compress) {
|
||||
return assembly.ItemSpec;
|
||||
}
|
||||
|
||||
if (bool.TryParse (assembly.GetMetadata ("AndroidSkipCompression"), out bool value) && value) {
|
||||
Log.LogDebugMessage ($"Skipping compression of {assembly.ItemSpec} due to 'AndroidSkipCompression' == 'true' ");
|
||||
return assembly.ItemSpec;
|
||||
}
|
||||
|
||||
if (compressedAssembliesInfo.TryGetValue (Path.GetFileName (assembly.ItemSpec), out CompressedAssemblyInfo info) && info != null) {
|
||||
EnsureCompressedAssemblyData (assembly.ItemSpec, info.DescriptorIndex);
|
||||
AssemblyCompression.CompressionResult result = AssemblyCompression.Compress (compressedAssembly);
|
||||
if (result != AssemblyCompression.CompressionResult.Success) {
|
||||
switch (result) {
|
||||
case AssemblyCompression.CompressionResult.EncodingFailed:
|
||||
Log.LogMessage ($"Failed to compress {assembly.ItemSpec}");
|
||||
break;
|
||||
|
||||
case AssemblyCompression.CompressionResult.InputTooBig:
|
||||
Log.LogMessage ($"Input assembly {assembly.ItemSpec} exceeds maximum input size");
|
||||
break;
|
||||
|
||||
default:
|
||||
Log.LogMessage ($"Unknown error compressing {assembly.ItemSpec}");
|
||||
break;
|
||||
}
|
||||
return assembly.ItemSpec;
|
||||
}
|
||||
return compressedAssembly.DestinationPath;
|
||||
}
|
||||
|
||||
return assembly.ItemSpec;
|
||||
}
|
||||
}
|
||||
|
||||
bool AddFileToArchiveIfNewer (ZipArchiveEx apk, string file, string inArchivePath, CompressionMethod compressionMethod = CompressionMethod.Default)
|
||||
|
|
|
|||
|
|
@ -0,0 +1,86 @@
|
|||
using System;
|
||||
using System.Collections.Generic;
|
||||
using System.IO;
|
||||
using Microsoft.Build.Framework;
|
||||
|
||||
namespace Xamarin.Android.Tasks
|
||||
{
|
||||
public class GenerateCompressedAssembliesNativeSourceFiles : AndroidTask
|
||||
{
|
||||
public override string TaskPrefix => "GCANSF";
|
||||
|
||||
[Required]
|
||||
public ITaskItem[] ResolvedAssemblies { get; set; }
|
||||
|
||||
[Required]
|
||||
public string [] SupportedAbis { get; set; }
|
||||
|
||||
[Required]
|
||||
public string EnvironmentOutputDirectory { get; set; }
|
||||
|
||||
[Required]
|
||||
public bool Debug { get; set; }
|
||||
|
||||
[Required]
|
||||
public bool EnableCompression { get; set; }
|
||||
|
||||
public override bool RunTask ()
|
||||
{
|
||||
GenerateCompressedAssemblySources ();
|
||||
return !Log.HasLoggedErrors;
|
||||
}
|
||||
|
||||
void GenerateCompressedAssemblySources ()
|
||||
{
|
||||
if (Debug || !EnableCompression) {
|
||||
Generate (null);
|
||||
return;
|
||||
}
|
||||
|
||||
var assemblies = new SortedDictionary<string, CompressedAssemblyInfo> (StringComparer.Ordinal);
|
||||
foreach (ITaskItem assembly in ResolvedAssemblies) {
|
||||
if (bool.TryParse (assembly.GetMetadata ("AndroidSkipAddToPackage"), out bool value) && value) {
|
||||
continue;
|
||||
}
|
||||
|
||||
if (assemblies.ContainsKey (assembly.ItemSpec)) {
|
||||
continue;
|
||||
}
|
||||
|
||||
var fi = new FileInfo (assembly.ItemSpec);
|
||||
if (!fi.Exists) {
|
||||
Log.LogError ($"Assembly {assembly.ItemSpec} does not exist");
|
||||
continue;
|
||||
}
|
||||
|
||||
assemblies.Add (Path.GetFileName (assembly.ItemSpec), new CompressedAssemblyInfo (checked((uint)fi.Length)));
|
||||
}
|
||||
|
||||
uint index = 0;
|
||||
foreach (var kvp in assemblies) {
|
||||
kvp.Value.DescriptorIndex = index++;
|
||||
}
|
||||
|
||||
BuildEngine4.RegisterTaskObject (CompressedAssemblyInfo.CompressedAssembliesInfoKey, assemblies, RegisteredTaskObjectLifetime.Build, false);
|
||||
Generate (assemblies);
|
||||
|
||||
void Generate (IDictionary<string, CompressedAssemblyInfo> dict)
|
||||
{
|
||||
foreach (string abi in SupportedAbis) {
|
||||
NativeAssemblerTargetProvider asmTargetProvider = GeneratePackageManagerJava.GetAssemblyTargetProvider (abi);
|
||||
string baseAsmFilePath = Path.Combine (EnvironmentOutputDirectory, $"compressed_assemblies.{abi.ToLowerInvariant ()}");
|
||||
string asmFilePath = $"{baseAsmFilePath}.s";
|
||||
var asmgen = new CompressedAssembliesNativeAssemblyGenerator (dict, asmTargetProvider, baseAsmFilePath);
|
||||
|
||||
using (var sw = MemoryStreamPool.Shared.CreateStreamWriter ()) {
|
||||
asmgen.Write (sw);
|
||||
sw.Flush ();
|
||||
if (MonoAndroidHelper.CopyIfStreamChanged (sw.BaseStream, asmFilePath)) {
|
||||
Log.LogDebugMessage ($"File {asmFilePath} was regenerated");
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -140,6 +140,26 @@ namespace Xamarin.Android.Tasks
|
|||
return !Log.HasLoggedErrors;
|
||||
}
|
||||
|
||||
static internal NativeAssemblerTargetProvider GetAssemblyTargetProvider (string abi)
|
||||
{
|
||||
switch (abi.Trim ()) {
|
||||
case "armeabi-v7a":
|
||||
return new ARMNativeAssemblerTargetProvider (false);
|
||||
|
||||
case "arm64-v8a":
|
||||
return new ARMNativeAssemblerTargetProvider (true);
|
||||
|
||||
case "x86":
|
||||
return new X86NativeAssemblerTargetProvider (false);
|
||||
|
||||
case "x86_64":
|
||||
return new X86NativeAssemblerTargetProvider (true);
|
||||
|
||||
default:
|
||||
throw new InvalidOperationException ($"Unknown ABI {abi}");
|
||||
}
|
||||
}
|
||||
|
||||
static readonly string[] defaultLogLevel = {"MONO_LOG_LEVEL", "info"};
|
||||
static readonly string[] defaultMonoDebug = {"MONO_DEBUG", "gen-compact-seq-points"};
|
||||
static readonly string[] defaultHttpMessageHandler = {"XA_HTTP_CLIENT_HANDLER_TYPE", "System.Net.Http.HttpClientHandler, System.Net.Http"};
|
||||
|
|
@ -251,29 +271,9 @@ namespace Xamarin.Android.Tasks
|
|||
|
||||
var appConfState = BuildEngine4.GetRegisteredTaskObject (ApplicationConfigTaskState.RegisterTaskObjectKey, RegisteredTaskObjectLifetime.Build) as ApplicationConfigTaskState;
|
||||
foreach (string abi in SupportedAbis) {
|
||||
NativeAssemblerTargetProvider asmTargetProvider;
|
||||
NativeAssemblerTargetProvider asmTargetProvider = GetAssemblyTargetProvider (abi);
|
||||
string baseAsmFilePath = Path.Combine (EnvironmentOutputDirectory, $"environment.{abi.ToLowerInvariant ()}");
|
||||
string asmFilePath = $"{baseAsmFilePath}.s";
|
||||
switch (abi.Trim ()) {
|
||||
case "armeabi-v7a":
|
||||
asmTargetProvider = new ARMNativeAssemblerTargetProvider (false);
|
||||
break;
|
||||
|
||||
case "arm64-v8a":
|
||||
asmTargetProvider = new ARMNativeAssemblerTargetProvider (true);
|
||||
break;
|
||||
|
||||
case "x86":
|
||||
asmTargetProvider = new X86NativeAssemblerTargetProvider (false);
|
||||
break;
|
||||
|
||||
case "x86_64":
|
||||
asmTargetProvider = new X86NativeAssemblerTargetProvider (true);
|
||||
break;
|
||||
|
||||
default:
|
||||
throw new InvalidOperationException ($"Unknown ABI {abi}");
|
||||
}
|
||||
|
||||
var asmgen = new ApplicationConfigNativeAssemblyGenerator (asmTargetProvider, baseAsmFilePath, environmentVariables, systemProperties) {
|
||||
IsBundledApp = IsBundledApplication,
|
||||
|
|
|
|||
|
|
@ -12,6 +12,7 @@ namespace Xamarin.Android.Tasks
|
|||
const string ArmV7a = "armeabi-v7a";
|
||||
const string TypeMapBase = "typemaps";
|
||||
const string EnvBase = "environment";
|
||||
const string CompressedAssembliesBase = "compressed_assemblies";
|
||||
|
||||
public override string TaskPrefix => "PAI";
|
||||
|
||||
|
|
@ -22,7 +23,7 @@ namespace Xamarin.Android.Tasks
|
|||
public string NativeSourcesDir { get; set; }
|
||||
|
||||
[Required]
|
||||
public bool TypeMapMode { get; set; }
|
||||
public string Mode { get; set; }
|
||||
|
||||
[Required]
|
||||
public bool Debug { get; set; }
|
||||
|
|
@ -42,10 +43,24 @@ namespace Xamarin.Android.Tasks
|
|||
var includes = new List<ITaskItem> ();
|
||||
bool haveSharedSource = false;
|
||||
bool haveArmV7SharedSource = false;
|
||||
string baseName = TypeMapMode ? TypeMapBase : EnvBase;
|
||||
bool typeMapMode = false;
|
||||
string baseName;
|
||||
|
||||
if (String.Compare ("typemap", Mode, StringComparison.OrdinalIgnoreCase) == 0) {
|
||||
baseName = TypeMapBase;
|
||||
typeMapMode = true;
|
||||
} else if (String.Compare ("environment", Mode, StringComparison.OrdinalIgnoreCase) == 0) {
|
||||
baseName = EnvBase;
|
||||
} else if (String.Compare ("compressed", Mode, StringComparison.OrdinalIgnoreCase) == 0) {
|
||||
baseName = CompressedAssembliesBase;
|
||||
} else {
|
||||
Log.LogError ($"Unknown mode: {Mode}");
|
||||
return false;
|
||||
}
|
||||
|
||||
TaskItem item;
|
||||
foreach (string abi in BuildTargetAbis) {
|
||||
if (TypeMapMode) {
|
||||
if (typeMapMode) {
|
||||
if (String.Compare (ArmV7a, abi, StringComparison.Ordinal) == 0)
|
||||
haveArmV7SharedSource = true;
|
||||
else
|
||||
|
|
@ -56,7 +71,7 @@ namespace Xamarin.Android.Tasks
|
|||
item.SetMetadata ("abi", abi);
|
||||
sources.Add (item);
|
||||
|
||||
if (!TypeMapMode)
|
||||
if (!typeMapMode)
|
||||
continue;
|
||||
|
||||
if (!InstantRunEnabled && !Debug) {
|
||||
|
|
|
|||
|
|
@ -917,23 +917,32 @@ namespace Lib2
|
|||
Assert.Ignore ($"Cross compiler for {supportedAbis} was not available");
|
||||
if (!b.GetSupportedRuntimes ().Any (x => supportedAbis == x.Abi))
|
||||
Assert.Ignore ($"Runtime for {supportedAbis} was not available.");
|
||||
b.BuildLogFile = "first.log";
|
||||
b.CleanupAfterSuccessfulBuild = false;
|
||||
b.CleanupOnDispose = false;
|
||||
b.ThrowOnBuildFailure = false;
|
||||
b.Verbosity = LoggerVerbosity.Diagnostic;
|
||||
Assert.AreEqual (expectedResult, b.Build (proj), "Build should have {0}.", expectedResult ? "succeeded" : "failed");
|
||||
Assert.AreEqual (expectedResult, b.Build (proj, doNotCleanupOnUpdate: true), "Build should have {0}.", expectedResult ? "succeeded" : "failed");
|
||||
if (!expectedResult)
|
||||
return;
|
||||
foreach (var target in targets) {
|
||||
Assert.IsFalse (b.Output.IsTargetSkipped (target), $"`{target}` should *not* be skipped on first build!");
|
||||
}
|
||||
|
||||
Assert.IsTrue (b.Build (proj), "Second build should have succeeded.");
|
||||
|
||||
b.BuildLogFile = "second.log";
|
||||
b.CleanupAfterSuccessfulBuild = false;
|
||||
b.CleanupOnDispose = false;
|
||||
Assert.IsTrue (b.Build (proj, doNotCleanupOnUpdate: true), "Second build should have succeeded.");
|
||||
foreach (var target in targets) {
|
||||
Assert.IsTrue (b.Output.IsTargetSkipped (target), $"`{target}` should be skipped on second build!");
|
||||
}
|
||||
|
||||
proj.Touch ("MainActivity.cs");
|
||||
|
||||
Assert.IsTrue (b.Build (proj), "Third build should have succeeded.");
|
||||
b.BuildLogFile = "third.log";
|
||||
b.CleanupAfterSuccessfulBuild = false;
|
||||
b.CleanupOnDispose = false;
|
||||
Assert.IsTrue (b.Build (proj, doNotCleanupOnUpdate: true), "Third build should have succeeded.");
|
||||
foreach (var target in targets) {
|
||||
Assert.IsFalse (b.Output.IsTargetSkipped (target), $"`{target}` should *not* be skipped on third build!");
|
||||
}
|
||||
|
|
|
|||
|
|
@ -27,6 +27,8 @@ namespace Xamarin.Android.Tasks
|
|||
{
|
||||
if (typeof(T) == typeof(Int32) || typeof(T) == typeof(UInt32))
|
||||
return Is64Bit ? ".word" : ".long";
|
||||
if (typeof(T) == typeof(Int64) || typeof(T) == typeof(UInt64))
|
||||
return Is64Bit ? ".xword" : ".long";
|
||||
return base.MapType <T> ();
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -0,0 +1,101 @@
|
|||
using System;
|
||||
using System.Buffers;
|
||||
using System.IO;
|
||||
|
||||
using K4os.Compression.LZ4;
|
||||
|
||||
namespace Xamarin.Android.Tasks
|
||||
{
|
||||
class AssemblyCompression
|
||||
{
|
||||
public enum CompressionResult
|
||||
{
|
||||
Success,
|
||||
InputTooBig,
|
||||
EncodingFailed,
|
||||
}
|
||||
|
||||
public sealed class AssemblyData
|
||||
{
|
||||
public string SourcePath { get; internal set; }
|
||||
public uint DescriptorIndex { get; internal set; }
|
||||
|
||||
public string DestinationPath;
|
||||
public uint SourceSize;
|
||||
public uint DestinationSize;
|
||||
|
||||
public AssemblyData (string sourcePath, uint descriptorIndex)
|
||||
{
|
||||
SetData (sourcePath, descriptorIndex);
|
||||
}
|
||||
|
||||
public void SetData (string sourcePath, uint descriptorIndex)
|
||||
{
|
||||
if (String.IsNullOrEmpty (sourcePath))
|
||||
throw new ArgumentException ("must not be null or empty", nameof (sourcePath));
|
||||
SourcePath = sourcePath;
|
||||
DescriptorIndex = descriptorIndex;
|
||||
}
|
||||
}
|
||||
|
||||
const uint CompressedDataMagic = 0x5A4C4158; // 'XALZ', little-endian
|
||||
|
||||
// TODO: consider making it configurable via an MSBuild property, would be more flexible this way
|
||||
//
|
||||
// Arbitrary limit of the input assembly size, to clamp down on memory allocation. Our unlinked Mono.Android.dll
|
||||
// assembly (the biggest one we have) is currently (May 2020) around 27MB, so let's bump the value to 30MB times
|
||||
// two - it should be more than enough for most needs.
|
||||
//public const ulong InputAssemblySizeLimit = 60 * 1024 * 1024;
|
||||
|
||||
static readonly ArrayPool<byte> bytePool = ArrayPool<byte>.Shared;
|
||||
|
||||
public static CompressionResult Compress (AssemblyData data)
|
||||
{
|
||||
if (data == null)
|
||||
throw new ArgumentNullException (nameof (data));
|
||||
|
||||
var fi = new FileInfo (data.SourcePath);
|
||||
if (!fi.Exists)
|
||||
throw new InvalidOperationException ($"File '{data.SourcePath}' does not exist");
|
||||
// if ((ulong)fi.Length > InputAssemblySizeLimit) {
|
||||
// return CompressionResult.InputTooBig;
|
||||
// }
|
||||
|
||||
data.DestinationPath = $"{data.SourcePath}.lz4";
|
||||
data.SourceSize = (uint)fi.Length;
|
||||
|
||||
byte[] sourceBytes = null;
|
||||
byte[] destBytes = null;
|
||||
try {
|
||||
sourceBytes = bytePool.Rent (checked((int)fi.Length));
|
||||
using (var fs = File.Open (data.SourcePath, FileMode.Open, FileAccess.Read, FileShare.Read)) {
|
||||
fs.Read (sourceBytes, 0, (int)fi.Length);
|
||||
}
|
||||
|
||||
destBytes = bytePool.Rent (LZ4Codec.MaximumOutputSize (sourceBytes.Length));
|
||||
int encodedLength = LZ4Codec.Encode (sourceBytes, 0, checked((int)fi.Length), destBytes, 0, destBytes.Length, LZ4Level.L09_HC);
|
||||
if (encodedLength < 0)
|
||||
return CompressionResult.EncodingFailed;
|
||||
|
||||
data.DestinationSize = (uint)encodedLength;
|
||||
using (var fs = File.Open (data.DestinationPath, FileMode.Create, FileAccess.Write, FileShare.Read)) {
|
||||
using (var bw = new BinaryWriter (fs)) {
|
||||
bw.Write (CompressedDataMagic); // magic
|
||||
bw.Write (data.DescriptorIndex); // index into runtime array of descriptors
|
||||
bw.Write (checked((uint)fi.Length)); // file size before compression
|
||||
|
||||
bw.Write (destBytes, 0, encodedLength);
|
||||
bw.Flush ();
|
||||
}
|
||||
}
|
||||
} finally {
|
||||
if (sourceBytes != null)
|
||||
bytePool.Return (sourceBytes);
|
||||
if (destBytes != null)
|
||||
bytePool.Return (destBytes);
|
||||
}
|
||||
|
||||
return CompressionResult.Success;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,104 @@
|
|||
using System.Collections.Generic;
|
||||
using System.IO;
|
||||
|
||||
namespace Xamarin.Android.Tasks
|
||||
{
|
||||
class CompressedAssembliesNativeAssemblyGenerator : NativeAssemblyGenerator
|
||||
{
|
||||
const string CompressedAssembliesField = "compressed_assemblies";
|
||||
const string DescriptorsField = "compressed_assembly_descriptors";
|
||||
|
||||
IDictionary<string, CompressedAssemblyInfo> assemblies;
|
||||
string dataIncludeFile;
|
||||
|
||||
public CompressedAssembliesNativeAssemblyGenerator (IDictionary<string, CompressedAssemblyInfo> assemblies, NativeAssemblerTargetProvider targetProvider, string baseFilePath)
|
||||
: base (targetProvider, baseFilePath)
|
||||
{
|
||||
this.assemblies = assemblies;
|
||||
dataIncludeFile = $"{baseFilePath}-data.inc";
|
||||
}
|
||||
|
||||
protected override void WriteSymbols (StreamWriter output)
|
||||
{
|
||||
if (assemblies == null || assemblies.Count == 0) {
|
||||
WriteCompressedAssembliesStructure (output, 0, null);
|
||||
return;
|
||||
}
|
||||
|
||||
string label = MakeLocalLabel (DescriptorsField);
|
||||
using (var dataOutput = MemoryStreamPool.Shared.CreateStreamWriter (output.Encoding)) {
|
||||
uint size = 0;
|
||||
|
||||
output.Write (Indent);
|
||||
output.Write (".include");
|
||||
output.Write (Indent);
|
||||
output.Write ('"');
|
||||
output.Write (Path.GetFileName (dataIncludeFile));
|
||||
output.WriteLine ('"');
|
||||
output.WriteLine ();
|
||||
|
||||
WriteDataSection (output, DescriptorsField);
|
||||
WriteStructureSymbol (output, label, alignBits: TargetProvider.MapModulesAlignBits, isGlobal: false);
|
||||
foreach (var kvp in assemblies) {
|
||||
string assemblyName = kvp.Key;
|
||||
CompressedAssemblyInfo info = kvp.Value;
|
||||
|
||||
string dataLabel = GetAssemblyDataLabel (info.DescriptorIndex);
|
||||
WriteCommSymbol (dataOutput, dataLabel, info.FileSize, 16);
|
||||
dataOutput.WriteLine ();
|
||||
|
||||
size += WriteStructure (output, packed: false, structureWriter: () => WriteDescriptor (output, assemblyName, info, dataLabel));
|
||||
}
|
||||
WriteStructureSize (output, label, size);
|
||||
|
||||
dataOutput.Flush ();
|
||||
MonoAndroidHelper.CopyIfStreamChanged (dataOutput.BaseStream, dataIncludeFile);
|
||||
}
|
||||
|
||||
WriteCompressedAssembliesStructure (output, (uint)assemblies.Count, label);
|
||||
}
|
||||
|
||||
uint WriteDescriptor (StreamWriter output, string assemblyName, CompressedAssemblyInfo info, string dataLabel)
|
||||
{
|
||||
WriteCommentLine (output, $"{info.DescriptorIndex}: {assemblyName}");
|
||||
|
||||
WriteCommentLine (output, "uncompressed_file_size");
|
||||
uint size = WriteData (output, info.FileSize);
|
||||
|
||||
WriteCommentLine (output, "loaded");
|
||||
size += WriteData (output, false);
|
||||
|
||||
WriteCommentLine (output, "data");
|
||||
size += WritePointer (output, dataLabel);
|
||||
|
||||
output.WriteLine ();
|
||||
return size;
|
||||
}
|
||||
|
||||
string GetAssemblyDataLabel (uint index)
|
||||
{
|
||||
return $"compressed_assembly_data_{index}";
|
||||
}
|
||||
|
||||
void WriteCompressedAssembliesStructure (StreamWriter output, uint count, string descriptorsLabel)
|
||||
{
|
||||
WriteDataSection (output, CompressedAssembliesField);
|
||||
WriteSymbol (output, CompressedAssembliesField, TargetProvider.GetStructureAlignment (true), packed: false, isGlobal: true, alwaysWriteSize: true, structureWriter: () => {
|
||||
// Order of fields and their type must correspond *exactly* to that in
|
||||
// src/monodroid/jni/xamarin-app.h CompressedAssemblies structure
|
||||
WriteCommentLine (output, "count");
|
||||
uint size = WriteData (output, count);
|
||||
|
||||
WriteCommentLine (output, "descriptors");
|
||||
size += WritePointer (output, descriptorsLabel);
|
||||
|
||||
return size;
|
||||
});
|
||||
}
|
||||
|
||||
void WriteDataSection (StreamWriter output, string tag)
|
||||
{
|
||||
WriteSection (output, $".data.{tag}", hasStrings: false, writable: true);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,16 @@
|
|||
namespace Xamarin.Android.Tasks
|
||||
{
|
||||
class CompressedAssemblyInfo
|
||||
{
|
||||
public const string CompressedAssembliesInfoKey = "__CompressedAssembliesInfo";
|
||||
|
||||
public uint FileSize { get; }
|
||||
public uint DescriptorIndex { get; set; }
|
||||
|
||||
public CompressedAssemblyInfo (uint fileSize)
|
||||
{
|
||||
FileSize = fileSize;
|
||||
DescriptorIndex = 0;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -239,6 +239,27 @@ namespace Xamarin.Android.Tasks
|
|||
return $"{value}";
|
||||
}
|
||||
|
||||
protected void WriteCommSymbol (StreamWriter output, string symbolName, ulong size, uint alignment)
|
||||
{
|
||||
output.Write (Indent);
|
||||
output.Write (".type");
|
||||
output.Write (Indent);
|
||||
output.Write (symbolName);
|
||||
output.Write (", ");
|
||||
output.Write (TargetProvider.TypePrefix);
|
||||
output.WriteLine ("object");
|
||||
|
||||
output.Write (Indent);
|
||||
output.Write (".local");
|
||||
output.Write (Indent);
|
||||
output.WriteLine (symbolName);
|
||||
|
||||
output.Write (Indent);
|
||||
output.Write (".comm");
|
||||
output.Write (Indent);
|
||||
output.WriteLine ($"{symbolName},{size},{alignment}");
|
||||
}
|
||||
|
||||
protected uint WritePointer (StreamWriter output, string targetName = null, string label = null, bool isGlobal = false)
|
||||
{
|
||||
uint fieldSize = UpdateSize (output, targetName ?? String.Empty);
|
||||
|
|
|
|||
|
|
@ -25,6 +25,8 @@ namespace Xamarin.Android.Tasks
|
|||
{
|
||||
if (typeof(T) == typeof(Int32) || typeof(T) == typeof(UInt32))
|
||||
return ".long";
|
||||
if (typeof(T) == typeof(Int64) || typeof(T) == typeof(UInt64))
|
||||
return ".quad";
|
||||
return base.MapType <T> ();
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -54,6 +54,7 @@
|
|||
<PackageReference Include="System.Runtime" Version="4.3.1" />
|
||||
<PackageReference Include="System.Runtime.InteropServices" Version="4.3.0" />
|
||||
<PackageReference Include="XliffTasks" Version="1.0.0-beta.19252.1" PrivateAssets="all" />
|
||||
<PackageReference Include="K4os.Compression.LZ4" Version="1.1.11" />
|
||||
</ItemGroup>
|
||||
|
||||
<ItemGroup>
|
||||
|
|
|
|||
|
|
@ -103,7 +103,7 @@ Copyright (C) 2011-2012 Xamarin. All rights reserved.
|
|||
<UsingTask TaskName="Xamarin.Android.Tasks.LinkApplicationSharedLibraries" AssemblyFile="Xamarin.Android.Build.Tasks.dll" />
|
||||
<UsingTask TaskName="Xamarin.Android.Tasks.PrepareAbiItems" AssemblyFile="Xamarin.Android.Build.Tasks.dll" />
|
||||
<UsingTask TaskName="Xamarin.Android.Tasks.WriteLockFile" AssemblyFile="Xamarin.Android.Build.Tasks.dll" />
|
||||
|
||||
<UsingTask TaskName="GenerateCompressedAssembliesNativeSourceFiles" AssemblyFile="Xamarin.Android.Build.Tasks.dll" />
|
||||
<!--
|
||||
*******************************************
|
||||
Extensibility hook that allows VS to
|
||||
|
|
@ -207,6 +207,7 @@ Copyright (C) 2011-2012 Xamarin. All rights reserved.
|
|||
<AndroidBuildApplicationPackage Condition=" '$(AndroidBuildApplicationPackage)' == ''">False</AndroidBuildApplicationPackage>
|
||||
<AndroidGenerateLayoutBindings Condition=" '$(AndroidGenerateLayoutBindings)' == '' ">False</AndroidGenerateLayoutBindings>
|
||||
<AndroidFragmentType Condition=" '$(AndroidFragmentType)' == '' ">Android.App.Fragment</AndroidFragmentType>
|
||||
<AndroidEnableAssemblyCompression Condition=" '$(AndroidEnableAssemblyCompression)' == '' ">True</AndroidEnableAssemblyCompression>
|
||||
|
||||
<!-- Currently only C# is supported -->
|
||||
<AndroidGenerateLayoutBindings Condition=" '$(Language)' != 'C#' ">False</AndroidGenerateLayoutBindings>
|
||||
|
|
@ -1586,7 +1587,7 @@ because xbuild doesn't support framework reference assemblies.
|
|||
NativeSourcesDir="$(_NativeAssemblySourceDir)"
|
||||
InstantRunEnabled="$(_InstantRunEnabled)"
|
||||
Debug="$(AndroidIncludeDebugSymbols)"
|
||||
TypeMapMode="True">
|
||||
Mode="typemap">
|
||||
<Output TaskParameter="AssemblySources" ItemName="_TypeMapAssemblySource" />
|
||||
<Output TaskParameter="AssemblyIncludes" ItemName="_TypeMapAssemblyInclude" />
|
||||
</PrepareAbiItems>
|
||||
|
|
@ -1707,9 +1708,17 @@ because xbuild doesn't support framework reference assemblies.
|
|||
NativeSourcesDir="$(_NativeAssemblySourceDir)"
|
||||
InstantRunEnabled="$(_InstantRunEnabled)"
|
||||
Debug="$(AndroidIncludeDebugSymbols)"
|
||||
TypeMapMode="false">
|
||||
Mode="environment">
|
||||
<Output TaskParameter="AssemblySources" ItemName="_EnvironmentAssemblySource" />
|
||||
</PrepareAbiItems>
|
||||
<PrepareAbiItems
|
||||
BuildTargetAbis="@(_BuildTargetAbis)"
|
||||
NativeSourcesDir="$(_NativeAssemblySourceDir)"
|
||||
InstantRunEnabled="$(_InstantRunEnabled)"
|
||||
Debug="$(AndroidIncludeDebugSymbols)"
|
||||
Mode="compressed">
|
||||
<Output TaskParameter="AssemblySources" ItemName="_CompressedAssembliesAssemblySource" />
|
||||
</PrepareAbiItems>
|
||||
</Target>
|
||||
|
||||
<Target Name="_GenerateEnvironmentFiles" DependsOnTargets="_ReadAndroidManifest">
|
||||
|
|
@ -2080,15 +2089,34 @@ because xbuild doesn't support framework reference assemblies.
|
|||
<_NativeAssemblyTarget Include="@(_EnvironmentAssemblySource->Replace('.s', '.o'))">
|
||||
<abi>%(_EnvironmentAssemblySource.abi)</abi>
|
||||
</_NativeAssemblyTarget>
|
||||
<_NativeAssemblyTarget Include="@(_CompressedAssembliesAssemblySource->Replace('.s', '.o'))">
|
||||
<abi>%(_CompressedAssembliesAssemblySource.abi)</abi>
|
||||
</_NativeAssemblyTarget>
|
||||
<_CompressedNativeAssemblyTarget Include="@(_CompressedAssembliesAssemblySource->Replace('.s', '.o'))">
|
||||
<abi>%(_CompressedAssembliesAssemblySource.abi)</abi>
|
||||
</_CompressedNativeAssemblyTarget>
|
||||
</ItemGroup>
|
||||
</Target>
|
||||
|
||||
<Target Name="_GenerateCompressedAssembliesNativeSourceFiles">
|
||||
<GenerateCompressedAssembliesNativeSourceFiles
|
||||
ResolvedAssemblies="@(_ResolvedUserAssemblies);@(_ResolvedFrameworkAssemblies)"
|
||||
EnvironmentOutputDirectory="$(IntermediateOutputPath)android"
|
||||
SupportedAbis="@(_BuildTargetAbis)"
|
||||
Debug="$(AndroidIncludeDebugSymbols)"
|
||||
EnableCompression="$(AndroidEnableAssemblyCompression)"
|
||||
/>
|
||||
<ItemGroup>
|
||||
<FileWrites Include="@(_CompressedAssembliesAssemblySource)" />
|
||||
</ItemGroup>
|
||||
</Target>
|
||||
|
||||
<Target Name="_CompileNativeAssemblySources"
|
||||
DependsOnTargets="_PrepareNativeAssemblyItems"
|
||||
Inputs="@(_TypeMapAssemblySource);@(_TypeMapAssemblyInclude);@(_EnvironmentAssemblySource)"
|
||||
DependsOnTargets="_PrepareNativeAssemblyItems;_GenerateCompressedAssembliesNativeSourceFiles"
|
||||
Inputs="@(_TypeMapAssemblySource);@(_TypeMapAssemblyInclude);@(_EnvironmentAssemblySource);@(_CompressedAssembliesAssemblySource)"
|
||||
Outputs="@(_NativeAssemblyTarget)">
|
||||
<CompileNativeAssembly
|
||||
Sources="@(_TypeMapAssemblySource);@(_EnvironmentAssemblySource)"
|
||||
Sources="@(_TypeMapAssemblySource);@(_EnvironmentAssemblySource);@(_CompressedAssembliesAssemblySource)"
|
||||
DebugBuild="$(AndroidIncludeDebugSymbols)"
|
||||
WorkingDirectory="$(_NativeAssemblySourceDir)"
|
||||
AndroidBinUtilsDirectory="$(AndroidBinUtilsDirectory)"
|
||||
|
|
@ -2260,6 +2288,7 @@ because xbuild doesn't support framework reference assemblies.
|
|||
CreatePackagePerAbi="$(AndroidCreatePackagePerAbi)"
|
||||
UseSharedRuntime="$(AndroidUseSharedRuntime)"
|
||||
Debug="$(AndroidIncludeDebugSymbols)"
|
||||
EnableCompression="$(AndroidEnableAssemblyCompression)"
|
||||
JavaSourceFiles="@(AndroidJavaSource)"
|
||||
JavaLibraries="@(AndroidJavaLibrary)"
|
||||
AndroidSequencePointsMode="$(_SequencePointsMode)"
|
||||
|
|
@ -2289,6 +2318,7 @@ because xbuild doesn't support framework reference assemblies.
|
|||
CreatePackagePerAbi="False"
|
||||
UseSharedRuntime="$(AndroidUseSharedRuntime)"
|
||||
Debug="$(AndroidIncludeDebugSymbols)"
|
||||
EnableCompression="$(AndroidEnableAssemblyCompression)"
|
||||
JavaSourceFiles="@(AndroidJavaSource)"
|
||||
JavaLibraries="@(AndroidJavaLibrary)"
|
||||
AndroidSequencePointsMode="$(_SequencePointsMode)"
|
||||
|
|
|
|||
|
|
@ -21,6 +21,13 @@ include(CheckCXXCompilerFlag)
|
|||
include("../../build-tools/cmake/xa_macros.cmake")
|
||||
|
||||
set(JAVA_INTEROP_SRC_PATH "../../external/Java.Interop/src/java-interop")
|
||||
|
||||
set(LZ4_SRC_DIR "../../external/lz4/lib")
|
||||
set(LZ4_INCLUDE_DIR ${LZ4_SRC_DIR})
|
||||
set(LZ4_SOURCES
|
||||
"${LZ4_SRC_DIR}/lz4.c"
|
||||
)
|
||||
|
||||
string(REPLACE "\\" "/" TOP_DIR ${CMAKE_SOURCE_DIR})
|
||||
set(SOURCES_DIR ${TOP_DIR}/jni)
|
||||
|
||||
|
|
@ -129,6 +136,8 @@ if(STRIP_DEBUG)
|
|||
endif()
|
||||
|
||||
if(ENABLE_NDK)
|
||||
include_directories(${LZ4_INCLUDE_DIR})
|
||||
add_definitions("-DHAVE_LZ4")
|
||||
add_definitions("-DPLATFORM_ANDROID")
|
||||
add_definitions("-DANDROID")
|
||||
add_definitions("-DLINUX -Dlinux -D__linux__")
|
||||
|
|
@ -282,6 +291,13 @@ set(MONODROID_SOURCES
|
|||
${JAVA_INTEROP_SRC_PATH}/java-interop-util.cc
|
||||
)
|
||||
|
||||
if(ENABLE_NDK)
|
||||
set(MONODROID_SOURCES
|
||||
${MONODROID_SOURCES}
|
||||
${LZ4_SOURCES}
|
||||
)
|
||||
endif()
|
||||
|
||||
if(UNIX)
|
||||
set(MONODROID_SOURCES
|
||||
${MONODROID_SOURCES}
|
||||
|
|
|
|||
|
|
@ -27,6 +27,11 @@ const TypeMapModule map_modules[] = {};
|
|||
const TypeMapJava map_java[] = {};
|
||||
#endif
|
||||
|
||||
CompressedAssemblies compressed_assemblies = {
|
||||
/*.count = */ 0,
|
||||
/*.descriptors = */ nullptr,
|
||||
};
|
||||
|
||||
ApplicationConfig application_config = {
|
||||
/*.uses_mono_llvm =*/ false,
|
||||
/*.uses_mono_aot =*/ false,
|
||||
|
|
|
|||
|
|
@ -45,7 +45,37 @@ typedef struct dirent monodroid_dirent_t;
|
|||
#endif
|
||||
|
||||
#define DEFAULT_DIRECTORY_MODE S_IRWXU | S_IRGRP | S_IXGRP | S_IROTH | S_IXOTH
|
||||
|
||||
#if defined (_MSC_VER)
|
||||
#define inline __inline
|
||||
#define force_inline __forceinline
|
||||
#elif defined (__GNUC__)
|
||||
#ifndef XA_LIKELY
|
||||
#define XA_LIKELY(expr) (__builtin_expect ((expr) != 0, 1))
|
||||
#endif
|
||||
|
||||
#ifndef XA_UNLIKELY
|
||||
#define XA_UNLIKELY(expr) (__builtin_expect ((expr) != 0, 0))
|
||||
#endif
|
||||
|
||||
#define force_inline inline __attribute__((always_inline))
|
||||
#endif
|
||||
|
||||
#ifndef force_inline
|
||||
#define force_inline inline
|
||||
#endif
|
||||
|
||||
#ifndef inline
|
||||
#define inline inline
|
||||
#endif
|
||||
|
||||
#ifndef XA_LIKELY
|
||||
#define XA_LIKELY(expr) (expr)
|
||||
#endif
|
||||
|
||||
#ifndef XA_UNLIKELY
|
||||
#define XA_UNLIKELY(expr) (expr)
|
||||
#endif
|
||||
|
||||
namespace xamarin::android
|
||||
{
|
||||
|
|
|
|||
|
|
@ -13,6 +13,10 @@
|
|||
#include <unistd.h>
|
||||
#include <climits>
|
||||
|
||||
#if defined (HAVE_LZ4)
|
||||
#include <lz4.h>
|
||||
#endif
|
||||
|
||||
#include <mono/metadata/assembly.h>
|
||||
#include <mono/metadata/image.h>
|
||||
#include <mono/metadata/mono-config.h>
|
||||
|
|
@ -61,6 +65,73 @@ void EmbeddedAssemblies::set_assemblies_prefix (const char *prefix)
|
|||
assemblies_prefix_override = prefix != nullptr ? utils.strdup_new (prefix) : nullptr;
|
||||
}
|
||||
|
||||
force_inline void
|
||||
EmbeddedAssemblies::get_assembly_data (const MonoBundledAssembly *e, char*& assembly_data, uint32_t& assembly_data_size)
|
||||
{
|
||||
#if defined (ANDROID) && defined (HAVE_LZ4) && defined (RELEASE)
|
||||
auto header = reinterpret_cast<const CompressedAssemblyHeader*>(e->data);
|
||||
if (header->magic == COMPRESSED_DATA_MAGIC) {
|
||||
if (XA_UNLIKELY (compressed_assemblies.descriptors == nullptr)) {
|
||||
log_fatal (LOG_ASSEMBLY, "Compressed assembly found but no descriptor defined");
|
||||
exit (FATAL_EXIT_MISSING_ASSEMBLY);
|
||||
}
|
||||
if (XA_UNLIKELY (header->descriptor_index >= compressed_assemblies.count)) {
|
||||
log_fatal (LOG_ASSEMBLY, "Invalid compressed assembly descriptor index %u", header->descriptor_index);
|
||||
exit (FATAL_EXIT_MISSING_ASSEMBLY);
|
||||
}
|
||||
|
||||
CompressedAssemblyDescriptor &cad = compressed_assemblies.descriptors[header->descriptor_index];
|
||||
assembly_data_size = e->size - sizeof(CompressedAssemblyHeader);
|
||||
if (!cad.loaded) {
|
||||
if (XA_UNLIKELY (cad.data == nullptr)) {
|
||||
log_fatal (LOG_ASSEMBLY, "Invalid compressed assembly descriptor at %u: no data", header->descriptor_index);
|
||||
exit (FATAL_EXIT_MISSING_ASSEMBLY);
|
||||
}
|
||||
|
||||
timing_period decompress_time;
|
||||
if (XA_UNLIKELY (utils.should_log (LOG_TIMING))) {
|
||||
decompress_time.mark_start ();
|
||||
}
|
||||
|
||||
if (header->uncompressed_length != cad.uncompressed_file_size) {
|
||||
if (header->uncompressed_length > cad.uncompressed_file_size) {
|
||||
log_fatal (LOG_ASSEMBLY, "Compressed assembly '%s' is larger than when the application was built (expected at most %u, got %u). Assemblies don't grow just like that!", e->name, cad.uncompressed_file_size, header->uncompressed_length);
|
||||
exit (FATAL_EXIT_MISSING_ASSEMBLY);
|
||||
} else {
|
||||
log_debug (LOG_ASSEMBLY, "Compressed assembly '%s' is smaller than when the application was built. Adjusting accordingly.", e->name);
|
||||
}
|
||||
cad.uncompressed_file_size = header->uncompressed_length;
|
||||
}
|
||||
|
||||
const char *data_start = reinterpret_cast<const char*>(e->data + sizeof(CompressedAssemblyHeader));
|
||||
int ret = LZ4_decompress_safe (data_start, reinterpret_cast<char*>(cad.data), static_cast<int>(assembly_data_size), static_cast<int>(cad.uncompressed_file_size));
|
||||
|
||||
if (XA_UNLIKELY (utils.should_log (LOG_TIMING))) {
|
||||
decompress_time.mark_end ();
|
||||
TIMING_LOG_INFO (decompress_time, "%s LZ4 decompression time", e->name);
|
||||
}
|
||||
|
||||
if (ret < 0) {
|
||||
log_fatal (LOG_ASSEMBLY, "Decompression of assembly %s failed with code %d", e->name, ret);
|
||||
exit (FATAL_EXIT_MISSING_ASSEMBLY);
|
||||
}
|
||||
|
||||
if (static_cast<uint64_t>(ret) != cad.uncompressed_file_size) {
|
||||
log_debug (LOG_ASSEMBLY, "Decompression of assembly %s yielded a different size (expected %lu, got %u)", e->name, cad.uncompressed_file_size, static_cast<uint32_t>(ret));
|
||||
exit (FATAL_EXIT_MISSING_ASSEMBLY);
|
||||
}
|
||||
cad.loaded = true;
|
||||
}
|
||||
assembly_data = reinterpret_cast<char*>(cad.data);
|
||||
assembly_data_size = cad.uncompressed_file_size;
|
||||
} else
|
||||
#endif
|
||||
{
|
||||
assembly_data = reinterpret_cast<char*>(const_cast<unsigned char*>(e->data));
|
||||
assembly_data_size = e->size;
|
||||
}
|
||||
}
|
||||
|
||||
MonoAssembly*
|
||||
EmbeddedAssemblies::open_from_bundles (MonoAssemblyName* aname, bool ref_only)
|
||||
{
|
||||
|
|
@ -96,10 +167,17 @@ EmbeddedAssemblies::open_from_bundles (MonoAssemblyName* aname, bool ref_only)
|
|||
MonoImage *image = nullptr;
|
||||
MonoImageOpenStatus status;
|
||||
const MonoBundledAssembly *e = *p;
|
||||
char *assembly_data = nullptr;
|
||||
uint32_t assembly_data_size;
|
||||
|
||||
if (strcmp (e->name, name) == 0 &&
|
||||
(image = mono_image_open_from_data_with_name ((char*) e->data, e->size, 0, nullptr, ref_only, name)) != nullptr &&
|
||||
(a = mono_assembly_load_from_full (image, name, &status, ref_only)) != nullptr) {
|
||||
if (strcmp (e->name, name) != 0) {
|
||||
continue;
|
||||
}
|
||||
|
||||
get_assembly_data (e, assembly_data, assembly_data_size);
|
||||
|
||||
if ((image = mono_image_open_from_data_with_name (assembly_data, assembly_data_size, 0, nullptr, ref_only, name)) != nullptr &&
|
||||
(a = mono_assembly_load_from_full (image, name, &status, ref_only)) != nullptr) {
|
||||
mono_config_for_assembly (image);
|
||||
break;
|
||||
}
|
||||
|
|
|
|||
|
|
@ -88,6 +88,7 @@ namespace xamarin::android::internal {
|
|||
|
||||
static MonoAssembly* open_from_bundles_full (MonoAssemblyName *aname, char **assemblies_path, void *user_data);
|
||||
static MonoAssembly* open_from_bundles_refonly (MonoAssemblyName *aname, char **assemblies_path, void *user_data);
|
||||
static void get_assembly_data (const MonoBundledAssembly *e, char*& assembly_data, uint32_t& assembly_data_size);
|
||||
|
||||
void zip_load_entries (int fd, const char *apk_name, monodroid_should_register should_register);
|
||||
bool zip_read_cd_info (int fd, uint32_t& cd_offset, uint32_t& cd_size, uint16_t& cd_entries);
|
||||
|
|
|
|||
|
|
@ -8,6 +8,7 @@
|
|||
|
||||
#include "monodroid.h"
|
||||
|
||||
static constexpr uint32_t COMPRESSED_DATA_MAGIC = 0x5A4C4158; // 'XALZ', little-endian
|
||||
static constexpr uint32_t MODULE_MAGIC_NAMES = 0x53544158; // 'XATS', little-endian
|
||||
static constexpr uint32_t MODULE_INDEX_MAGIC = 0x49544158; // 'XATI', little-endian
|
||||
static constexpr uint8_t MODULE_FORMAT_VERSION = 2; // Keep in sync with the value in src/Xamarin.Android.Build.Tasks/Utilities/TypeMapGenerator.cs
|
||||
|
|
@ -74,6 +75,26 @@ struct TypeMapJava
|
|||
};
|
||||
#endif
|
||||
|
||||
struct CompressedAssemblyHeader
|
||||
{
|
||||
uint32_t magic; // COMPRESSED_DATA_MAGIC
|
||||
uint32_t descriptor_index;
|
||||
uint32_t uncompressed_length;
|
||||
};
|
||||
|
||||
struct CompressedAssemblyDescriptor
|
||||
{
|
||||
uint32_t uncompressed_file_size;
|
||||
bool loaded;
|
||||
uint8_t *data;
|
||||
};
|
||||
|
||||
struct CompressedAssemblies
|
||||
{
|
||||
uint32_t count;
|
||||
CompressedAssemblyDescriptor *descriptors;
|
||||
};
|
||||
|
||||
struct ApplicationConfig
|
||||
{
|
||||
bool uses_mono_llvm;
|
||||
|
|
@ -100,6 +121,7 @@ MONO_API const TypeMapModule map_modules[];
|
|||
MONO_API const TypeMapJava map_java[];
|
||||
#endif
|
||||
|
||||
MONO_API CompressedAssemblies compressed_assemblies;
|
||||
MONO_API ApplicationConfig application_config;
|
||||
MONO_API const char* app_environment_variables[];
|
||||
MONO_API const char* app_system_properties[];
|
||||
|
|
|
|||
|
|
@ -5,13 +5,13 @@
|
|||
"Size": 3684
|
||||
},
|
||||
"classes.dex": {
|
||||
"Size": 2200624
|
||||
"Size": 2198984
|
||||
},
|
||||
"lib/armeabi-v7a/libmono-btls-shared.so": {
|
||||
"Size": 856580
|
||||
},
|
||||
"lib/armeabi-v7a/libmonodroid.so": {
|
||||
"Size": 170808
|
||||
"Size": 203664
|
||||
},
|
||||
"lib/armeabi-v7a/libmonodroid_bundle_app.so": {
|
||||
"Size": 4607636
|
||||
|
|
@ -23,13 +23,13 @@
|
|||
"Size": 3816228
|
||||
},
|
||||
"lib/armeabi-v7a/libxamarin-app.so": {
|
||||
"Size": 98020
|
||||
"Size": 105736
|
||||
},
|
||||
"lib/x86/libmono-btls-shared.so": {
|
||||
"Size": 1311172
|
||||
},
|
||||
"lib/x86/libmonodroid.so": {
|
||||
"Size": 203048
|
||||
"Size": 271140
|
||||
},
|
||||
"lib/x86/libmonodroid_bundle_app.so": {
|
||||
"Size": 4607272
|
||||
|
|
@ -41,7 +41,7 @@
|
|||
"Size": 3799820
|
||||
},
|
||||
"lib/x86/libxamarin-app.so": {
|
||||
"Size": 97764
|
||||
"Size": 105416
|
||||
},
|
||||
"META-INF/android.arch.core_runtime.version": {
|
||||
"Size": 6
|
||||
|
|
@ -2990,5 +2990,5 @@
|
|||
"Size": 493672
|
||||
}
|
||||
},
|
||||
"PackageSize": 16058252
|
||||
"PackageSize": 16119692
|
||||
}
|
||||
Загрузка…
Ссылка в новой задаче