Context: https://developer.android.com/about/versions/14
Context: https://android-developers.googleblog.com/2023/06/android-14-beta-3-and-platform-stability.html
Android 14 Developer Beta 3 has been released.
* [API diff vs. API-33][0]
* [API diff vs. API-UpsideDownCake Beta 2][1] (3c1a9851)
The Android 14 Developer Preview Program Overview
[Timeline and updates][2] section suggests the following timeline:
* Feb/Mar: Developer Previews
* April/May: Unstable Betas
* June: Beta 3 with "First [Platform Stability][3] milestone
including final APIs and behaviors. Play publishing also opens."
* July: Near-final builds for final testing
* ???: Final
~~ Acceptable Breakages ~~
Many methods in `Android.Content.PM.PackageManager` and the
`Dalvik.SystemInterop.DexFile` type were un-deprecated.
Interface method `Java.Lang.Reflect.IAnnotatedElement.IsAnnotationPresent`
was given a default implementation, which we enabled in 3c1a9851.
`RequiresPermission` was removed from `AlarmManager.setExact(…)`
in `annotations.zip`.
`RequiresPermission` for `WallpaperManager.*` was changed:
<!-- API-33 -->
<val name="value" val=""android.permission.READ_EXTERNAL_STORAGE"" />
<!-- API-34 -->
<val name="anyOf" val="{"android.permission.MANAGE_EXTERNAL_STORAGE", "android.permission.READ_WALLPAPER_INTERNAL"}" />
We have [previously decided][4] that we do not use these values for
anything and are not updating this.
~~ Enumification? ~~
While reviewing, we discovered that the *past four years* of
enumification was incomplete: we did the work, but the enums weren't
applied. See e.g. [`TextView.LineBreakWordStyle Property`][5],
which is of type `int`, but *should* have been of enum type
`Android.Graphics.Text.LineBreakWordStyle`, via df6c716e41:
33,android/widget,TextView,setLineBreakWordStyle,lineBreakWordStyle,Android.Graphics.Text.LineBreakWordStyle
The reason for this is that `methodmap.csv` column 2 should be a
*Java* `.`d package name, but we used the JNI `/`d name.
We can't fix these (API and ABI breaks are bad), and we are still
exploring what we can do to address this.
This oversight was caught in time that API-34 members are correctly
enumified.
[0]: https://developer.android.com/sdk/api_diff/34/changes
[1]: https://developer.android.com/sdk/api_diff/34-incr/changes
[2]: https://web.archive.org/web/20230616200934/https://developer.android.com/about/versions/14/overview#timeline
[3]: https://web.archive.org/web/20230616200934/https://developer.android.com/about/versions/14/overview#platform_stability
[4]: https://github.com/xamarin/xamarin-android/issues/6775
[5]: https://learn.microsoft.com/dotnet/api/android.widget.textview.LineBreakWordStyle?view=xamarin-android-sdk-13
Many of our projects in `/tools` and `/build-tools` target `net472`. Update them to instead target `$(DotNetStableTargetFramework)`.
A side-effect of this is that there is no longer an `.exe` built, so we also need to update everything in our build system that calls these tools to use `dotnet foo.dll` instead of `mono foo.exe`.
Heads-up: xamarin-android/main is *becoming* .NET Android-only;
Classic Xamarin.Android is only supported [through 2024-May-1][0];
all current and future feature work is oriented toward .NET Android.
Classic Xamarin.Android will only be fully buildable/supportable on
the [d17-5 release branch][1].
Update the Classic Xamarin.Android build *on the main branch* to only
build and package `$(AndroidLatestStableFrameworkVersion)` (v13.0/API-33)
*not* `$(AndroidFirstApiLevel)` (v4.4/API-19) *through* API-33.
Build timing from main:
make jenkins 219.78s user 62.79s system 21% cpu 21:56.92 total
Build timing from the branch for PR #7786:
make jenkins 178.66s user 47.24s system 36% cpu 10:22.81 total
It looks like this will shave off about 10 minutes from the Linux
build and 10-15 minutes from the macOS build in CI as well.
This does not (yet!) remove building the Classic Xamarin.Android
installers entirely from main; instead, the installers will only
include bindings for Android v13.0 (API-33). This seems like a good
incremental step towards obsoletion.
The Designer integration tests have been removed as they rely on API-30
or lower. Additionally, the Designer integration tests *require* the
Classic Xamarin.Android runtime, which is not long for this world/
The Android Wear test job has been moved to the emulator test stage
rather than running in its own stage.
TODO: Multiple API levels are still used in the following projects,
and could be cleaned up in the future:
* build-tools/api-merge
* build-tools/create-android-api
[0]: https://dotnet.microsoft.com/en-us/platform/support/policy/xamarin
[1]: https://github.com/xamarin/xamarin-android/commits/d17-5
Context: https://developer.android.com/about/versions/13
Context: https://android-developers.googleblog.com/2022/06/android-13-beta-3-platform-stability.html
Android 13 Beta 3 has been released.
* [API diff vs. API-32][0]
* [API diff vs. API-Tiramisu Beta 2][1]
The Android 13 Developer Preview Program Overview
[Timeline and updates][2] section suggests the following timeline:
* May: Beta 2
* June: Beta 3: First Platform Stability milestone.
Final APIs and behaviors.
* July: Beta 4: Near-final build for final testing.
* ???: Final
In particular, Beta 3 has "Final APIs". We now have API-33!
Enumify API-33, and declare it stable.
API-33 will be available as:
* Classic Xamarin.Android: `$(TargetFrameworkVersion)`=v13.0
* .NET 6: `$(TargetFramework)`=net6.0-android33.0
* .NET 7: `$(TargetFramework)`=net7.0-android33.0
The default API-level for `net6.0-android` remains API-31.
The default API-level for `net7.0-android` will be API-33.
~~ Enumification Issue ~~
The following methods look like they should be enumified, however the
constants referenced by their [documentation][3] do not exist.
Perhaps this will be fixed later and we should revisit them? For now,
they are not enumified.
?,33,android/net/wifi/aware,PublishConfig,getInstantCommunicationBand,return,
?,33,android/net/wifi/aware,PublishConfig$Builder,setInstantCommunicationModeEnabled,band,
?,33,android/net/wifi/aware,SubscribeConfig,getInstantCommunicationBand,return,
?,33,android/net/wifi/aware,SubscribeConfig$Builder,setInstantCommunicationModeEnabled,band,
?,33,android/net/wifi,MloLink,getBand,return,
~~ API Compat Changes ~~
We now ignore `[Android.Runtime.StringDefAttribute]` and
`[Android.Runtime.IntDefAttribute]` in `<CheckApiCompatibility/>`.
These attributes frequently churn on each preview and release, causing
lots of noise in our `acceptable-breakages*` files. As we do not
consume these attributes, we don't really need to track changes.
Relevant existing entries from `acceptable-breakages*` files were removed.
~~ Update `CodeDiffGen` ~~
The logic for running `Microsoft.DotNet.GenAPI` was failing with:
/Users/builder/azdo/_work/1/s/xamarin-android/src/Mono.Android/Mono.Android.targets(270,5):
error MSB4018: System.ComponentModel.Win32Exception (8): An error occurred trying to start process
'/Users/builder/.nuget/packages/microsoft.dotnet.genapi/7.0.0-beta.22103.1/tools/net472/Microsoft.DotNet.GenAPI.exe'
with working directory '/Users/builder/azdo/_work/1/s/xamarin-android/src/Mono.Android'.
Exec format error
Fix this by using the `netcoreapp3.1/Microsoft.DotNet.GenAPI.dll`
version when running under `dotnet`.
Co-authored-by: Jonathan Peppers <jonathan.peppers@microsoft.com>
[0]: https://developer.android.com/sdk/api_diff/33/changes
[1]: https://developer.android.com/sdk/api_diff/33-incr/changes
[2]: https://web.archive.org/web/20220627195145/https://developer.android.com/about/versions/13/overview#platform_stability
[3]: https://developer.android.com/reference/kotlin/android/net/wifi/aware/PublishConfig#getinstantcommunicationband
Context: 5432886562
*One* issue with running `dotnet build Xamarin.Android.sln` without
`-m:1` (54328865) is that it attempts to run our Android API
extraction process multiple times simultaneously. This results in
file locking issues:
C:\code\xamarin-android\src\Mono.Android\Mono.Android.targets(409,5): "C:\code\xamarin-android\src\Mono.Android\..\..\bin\Debug\lib\xamarin.android\xbuild\Xamarin\Android\class-parse.exe C:\…\android-toolchain\sdk\platforms\android-26\android.jar -platform=26 -parameter-names="C:\code\xamarin-android\src\Mono.Android\..\..\src\Mono.Android\Profiles\api-26.params.txt" -o="C:\code\xamarin-android\src\Mono.Android\..\..\bin\BuildDebug\api\api-26.xml.class-parse"" failed with code: -532462766 Error output:
Unhandled Exception: System.IO.IOException: The process cannot access the file 'C:\code\xamarin-android\bin\BuildDebug\api\api-26.xml.class-parse' because it is being used by another process.
at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
at System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost)
at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, FileOptions options, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost)
at System.IO.StreamWriter.CreateFile(String path, Boolean append, Boolean checkHost)
at System.IO.StreamWriter..ctor(String path, Boolean append, Encoding encoding, Int32 bufferSize, Boolean checkHost)
at System.IO.StreamWriter..ctor(String path, Boolean append, Encoding encoding)
at Xamarin.Android.Tools.App.Main(String[] args) in C:\code\xamarin-android\external\Java.Interop\tools\class-parse\Program.cs:line 68 [C:\code\xamarin-android\src\Mono.Android\Mono.Android.csproj]
This seems to be a result of running the process as an
`AfterTargets="Build"` target in `api-xml-adjuster.csproj`.
By moving it to a `NoTargets` project, we can eliminate MSBuild trying
to run it multiple times.
Move the `class-parse` execution to a new `create-android-api.csproj`
project.
Also, move the `api-merge` step into `create-android-api.csproj`,
from `Mono.Android.targets`. Currently the `pai-merge` process runs
twice, once for `monoandroid10` and once for `net7.0`. However, the
output is not dependent on `TargetFramework`. Instead of generating:
- `src\Mono.Android\obj\Debug\monoandroid10\android-32\mcw\api.xml`
- `src\Mono.Android\obj\Debug\net7.0\android-32\mcw\api.xml`
we will now generate:
- `bin\Build$(Configuration)\api\api-32.xml`
Calling `api-merge` only once saves ~25s from build time.
Note that `generator` writes intermediate files (eg: `api.xml.fixed`)
to the directory containing `api.xml`, so `Mono.Android.targets` will
copy the `api-X.xml` file to its eg: `obj\Debug\net7.0\android-32\mcw`
directory to work from. Without this, both `monoadroid10` and
`net7.0` would write to the common location, causing possible
file sharing conflicts.
Additionally, let VS2022 rewrite `Xamarin.Android.sln` to its
preferred format in order to add the new project.
Context: https://developer.android.com/about/versions/12/12L
Context: https://android-developers.googleblog.com/2021/12/beta-1-update-for-12l-feature-drop.html
Android 12L Developer Beta 1 has been released.
* [API diff vs. API-31][0]
* [API diff vs. Sv2 DP1][1]
The Android 12L Developer Preview Program Overview
[Timeline and updates][2] section suggests the following timeline
(which differs from ef011213):
* October: Developer Preview 1
* December: Beta 1; Final APIs (previously expected in Beta 2)
* January: Beta 2; Incremental beta update
* February: Beta 3; Incremental beta update
* Q1 2022: Final release
In particular, Beta 1 has "Final APIs". We now have API-32!
Enumify API-32, and declare it stable.
As a change from "upstream Google", Classic Xamarin.Android uses
`$(TargetFrameworkVersion)`=v12.1 for API-32, as there is no way to
express "12L" in a `System.Version`. We have decreed 12.1 as 12L.
.NET SDK for Android uses `$(TargetFramework)`=net6.0-android32.0.
However, we don't want to change the default API level for .NET 6
projects; the default will remain `net6.0-android31.0` (API-31),
using the [31.0.101-preview.11.117 binaries on NuGet.org][3].
This requires various changes to the unit test infrastructure and
.NET SDK workflow package creation.
Additionally, `d8` doesn't appear to support API-32 yet (?!);
building an app with an `AndroidManifest.xml` containing:
<uses-sdk android:minSdkVersion="32" android:targetSdkVersion="32" />
would use `d8 --min-api 32`, which results in a warning:
D8 : warning : An API level of 32 is not supported by this compiler.
Please use an API level of 31 or earlier
Update the `XASdkTests.SupportedOSPlatformVersion()` test so that it
only uses API-31 and not API-32. This avoids the above warning.
Finally, we had to update xaprepare to install the new
Android SDK Build-tools version 32.0.0, as `aapt` from previous
Build-tools versions would emit a warning when building for API-32:
Xamarin.Android.Aapt2.targets(157,3): warning APT2000: aapt2 W 01-05 23:58:06 8352 43857 LoadedArsc.cpp:657] Unknown chunk type '200'.
Unfortunately, installing Build-tools 32.0.0 promptly broke unit
tests using `$(AndroidDexTool)`=dx, with:
COMPILETODALVIK : error : Unable to access jarfile C:\…\android-toolchain\sdk\build-tools\32.0.0\lib\dx.jar
as Build-tools 31 and later no longer contains `lib/dx.jar`. Update
xaprepare to install *both* Build-tools 30.0.3 *and* 32.0.0, so that
tests which require `dx.jar` continue to run, and we can get *some*
test coverage against Build-tools 32.
Co-authored-by: Jonathan Peppers <jonathan.peppers@microsoft.com>
[0]: https://developer.android.com/sdk/api_diff/32/changes
[1]: https://developer.android.com/sdk/api_diff/32-incr/changes
[2]: https://web.archive.org/web/20220113142518/https://developer.android.com/about/versions/12/12L/overview
[3]: https://www.nuget.org/packages/Microsoft.Android.Ref.31/31.0.101-preview.11.117
Context: https://developer.android.com/about/versions/12
API-31 has been enumified (88438081, cf2a39b2, cf2a39b2).
Time to declare it stable!
Set API-S as API-31, `$(TargetFrameworkVersion)`=v12.0,
`$(AndroidPackVersion)`=31.0.100, and declare as stable.
Update unit test files so `$(TargetFrameworkVersion)`= v12.0 (API-31)
*or* `$(TargetFrameworkVersion)` isn't needed, a'la 8140991e.
Rename `src/Mono.Android/Profiles/api-S.params.txt` to
`api-31.params.txt`, to match the stable API-level ID.
Updated `tests/api-compatibility/acceptable-breakages-vReference.txt`
because the default API compare will now be against `API-31`, not
`API-30`, and we deliberately removed the `[Category]` custom attribute
(e5a4beef), remove `[DataContractAttribute]` (101fea2c), and fix the
value of `Android.OS.BuildVersionCodes.R` (1b1ec097).
Support `$(AndroidUseLatestPlatformSdk)` in binding projects: once
API-31 is stable, binding projects with
`$(TargetFrameworkVersion)`=v11.0 and
`$(AndroidUseLatestPlatformSdk)`=true fail to build with:
Task ResolveNuGetPackageAssets
...
Microsoft.NuGet.targets(198,5): Your project does not reference "MonoAndroid,Version=v12.0" framework. Add a reference to "MonoAndroid,Version=v12.0" in the "TargetFrameworks" property of your project file and then re-run NuGet restore.
What happened is that `$(TargetFrameworkVersion)` was changed to 12.0
during `Build`, but not during `Restore`. Thus NuGet restored with
`MonoAndroid,Version=v11.0` instead of `MonoAndroid,Version=v12.0`.
I found that we have this setup for other project types, but not
binding projects:
<!--
NOTE:
This target runs during Restore, and there is no $(RestoreDependsOn) property.
There appears to be no other way to do this other than use BeforeTargets.
-->
<Target Name="_SetLatestTargetFrameworkVersionForPackageReference"
Condition=" '$(AndroidUseLatestPlatformSdk)' == 'True' "
BeforeTargets="_GetRestoreTargetFrameworksOutput"
DependsOnTargets="_SetLatestTargetFrameworkVersion">
</Target>
If we move this target to `Xamarin.Android.Legacy.targets`, it will be
imported by binding projects.
We also get the added benefit of it not being in .NET 6.
`$(AndroidUseLatestPlatformSdk)` is not supported in .NET 6, so we
should not have this MSBuild target there.
Co-authored-by: Jonathan Peppers <jonathan.peppers@microsoft.com>
Context: https://developer.android.com/about/versions/12/overview
Context: https://android-developers.googleblog.com/2021/02/android-12-dp1.html
Context: https://github.com/xamarin/java.interop/issues/810
Android 12 Developer Preview 1 has been released. The Android 12
Developer Preview Program Overview [Timeline and updates][0] section
suggests the following timeline:
* February: Developer Preview 1
* March: Developer Preview 2
* April: Developer Preview 3
* May: Beta 1
* June-July: Beta 2, Beta 3
* August: Beta 4
This contains the final APIs and official SDK.
Add support for binding API-S as API-31, with
`$(TargetFrameworkVersion)` v11.0.99.
Breaking changes:
* Two new methods were added to the `Java.Lang.IDeprecated`
annotation interface.
Given that this is an annotation, this should not be used as a
regular interface and shouldn't break anything.
That said, this appearance shows that we have more work to do
in preserving API (source) compatibility.
See also xamarin/java.interop#810.
[0]: https://developer.android.com/about/versions/12/overview#timeline
Context: https://developer.android.com/android11
Changes: 6d05c4929d...fb5542fa19
* xamarin/monodroid@fb5542fa1 [Make.config] Update API-R to API-30 (#1102)
API-30 has been enumified (8f9c6382). Time to declare it stable!
Set API-R as API-30, v11.0, and stable.
Update unit test files so `$(TargetFrameworkVersion)`= v11.0 (API-30).
Rename `src/Mono.Android/Profiles/api-R.params.txt` to
`api-30.params.txt`, to match the stable API-level ID.
Updated `tests/api-compatibility/acceptable-breakages-vReference.txt`
because the default API compare will now be against `API-30`, not
`API-29`, and we deliberately removed the `XxxConst` classes which
were `[Obsolete]`d 5+ years ago in API-30, resulting in an intentional
API break.
TODO:
* `jnimarshalmethod-gen.exe` tests were disabled as they ran into a
[runtime crash][0]. Once the runtime crash is fixed, we need to
update `src/Mono.Android/Test/Mono.Android-Tests.csproj` so that
`$(AndroidGenerateJniMarshalMethods)` is True when
`'$(HostOS)' == 'Darwin'`.
[0]: https://github.com/dotnet/runtime/issues/34389
Previously, we built the 64-bit targets (`x86-64` and `arm64-v8a`)
against NDK API-21 and the32-bit targets (`x86` and `armeabi-v7a`)
against NDK API-16.
Update so that all targets build against API-21.
This gives us the ability to speed up on-device app startup of Debug-
configuration builds on 32-bit platforms by using POSIX APIs for
directory traversal that are available starting from API-21 onwards
([**openat**(2)][0] and [**fstatat**(2)][1]).
The switch to API-21 was initially planned for .NET5 but there's no
significant harm in making the change now.
In addition, this commit updates versions of several components:
* Bump emulator version to 30.0.5
* Bump NDK version to 21b (21.1.6352462)
* Bump x86 system image revision to 7
Also, `XABuildConfig.NDKMinimumApiAvailable` is now set to the minimum
supported NDK API level instead of the minimum API level *installed*
by the NDK.
TODO: This commit knowingly breaks some unit tests, e.g.
`Xamarin.Android.Build.Tests.BuildTest.BuildMultiDexApplication()`.
These will be fixed in a later commit.
[0]: https://linux.die.net/man/2/openat
[1]: https://linux.die.net/man/2/fstatat
Context: 485e39b173
Context: eb08bb4645
...and doubtless others...
Three MSBuild properties control the `android.jar` which is bound and
the `$(TargetFrameworkVersion)` of `Mono.Android.dll`:
* `$(AndroidApiLevel)`: The API level that is bound. Must be an int.
* `$(AndroidFrameworkVersion)`: The `$(TargetFrameworkVersion)` of
the generated `Mono.Android.dll`. Must be *mostly* parseable by
`System.Version` except with a leading `v`, e.g. `v10.0`.
* `$(AndroidPlatformId)`: The "ID" of the API level.
*Most* of the time, `$(AndroidApiLevel)` and `$(AndroidPlatformId)`
will be *identical*: for API-29, they're both `29`.
Where they differ is for new *preview* API levels, such as API-R:
`$(AndroidApiLevel)` will be 30, but `$(AndroidPlatformId)` is `R`.
The distinction is important because various filesystem paths within
the Android SDK use the "id" and *not* the API level when they differ,
e.g. the API-R `android.jar` is installed into:
$(AndroidSdkDirectory)/platforms/android-R/android.jar
We thus need to be *careful* when distinguishing between
`$(AndroidApiLevel)` and `$(AndroidPlatformId)`, using the former when
an integer is *required*, and using the latter whenever it refers to
filesystem paths.
Unfortunately, we *haven't* been careful, because these values really
only differ for ~4 months out of the year, and for only one
`$(TargetFrameworkVersion)` version.
Start bringing some sanity...and finding bugs while we do so:
`api-xml-adjuster.targets` should use `%(AndroidApiInfo.Id)` and *not*
`%(AndroidApiLevel.Level)`, as it references filesystem locations.
Consequently, `src/Mono.Android/Profiles/api-30.params.txt` must be
renamed to `src/Mono.Android/Profiles/api-R.params.txt` so that it
correctly embeds the `$(AndroidPlatformId)` value.
`Mono.Android.targets` should likewise use `$(AndroidPlatformId)` and
not `$(AndroidApiLevel)` when using filesystem paths from the SDK.
For good measure, `Mono.Android.csproj` now overrides
`$(IntermediateOutputPath)` to contain `$(AndroidPlatformId)`, because
why not (MOAR CONSISTENCY!).
These changes, unfortunately, introduce breakage, which will need to
be addressed:
*Because* API-R was installed into
`$(AndroidSdkDirectory)/platforms/android-R`, `api-versions.xml`
*was not previously used* because `Mono.Android.targets` was using
`$(AndroidApiLevel)`, and `platforms/android-30/data/api-version.xml`
does not yet exist. (It will come June! But not now.) As it didn't
exist, it hit the fallback path and used
`platform-tools/api/api-versions.xml` (4cd20605c). You would *think*
this wouldn't be a problem, but the API-R `api-versions.xml` is
*missing* members relative to platform-tools, resulting in members
*missing* `RegisterAttribute.ApiSince` values, which
`Microsoft.DotNet.ApiCompat.exe` reports, e.g.:
CannotChangeAttribute : Attribute 'Android.Runtime.RegisterAttribute' on 'Java.Lang.StringBuilder.TrimToSize()' changed from '[RegisterAttribute("trimToSize", "()V", "", ApiSince=9)]' in the contract to '[RegisterAttribute("trimToSize", "()V", "")]' in the implementation
xamarin/java.interop@568d24ac added support to allow
`generator --apiversions` to be specified multiple times. Take
advantage of this new support to pass in the `api-versions.xml` files
from *both* `platforms/android-R` *and* `platform-tools/api` when
binding API levels > API-29. (Attempting to do this for *all*
versions which have both resulted in bizarre API compat errors, as
the `RegisterAttribute.ApiSince` value was *cleared*. ?!)
This works around the deficiency in API-R's `api-versions.xml` and
allows us to retain correct `RegisterAttribute.ApiSince` values.
Aside: to manually build the API-R binding, use:
msbuild /p:AndroidPlatformId=R /p:AndroidApiLevel=30 /p:AndroidFrameworkVersion=v10.0.99 src/Mono.Android/Mono.Android.csproj /v:diag > b.txt
Context: https://github.com/xamarin/monodroid/pull/1048
Context: https://github.com/xamarin/xamarin-android/pull/3928
Attempt to clean up and modernize various projects to help break up
the scope of change required to convert them to .NET Standard libs.
Bump the .NET Core version that `build.groovy` and
`azure-pipelines.yaml` install to 3.1.100. This is to obtain various
NuGet support fixes.
Updates the shared `MSBuildReferences.projitems` file to instead
include NuGet `@(PackageReference)`s to the required
`Microsoft.Build.*.dll` assemblies. The `Mono.Posix.NETStandard`,
`Xamarin.Build.AsyncTask`, and `Xamarin.LibZipSharp` NuGet references
have also been moved into this file to make it easier to update
versions in one place as needed. We're only using the compile time
assets from the MSBuild NuGet packages, which ensures that their
content is not copied to the output directory. An additional `@(None)`
item group has been added to this file to copy the required assemblies
from the system to the output directory so that
`Xamarin.Android.Build.Tests.dll` can run against them. These changes
are required because the MSBuild assemblies coming from NuGet do not
yet fully support running on Mono.
All projects which reference `MSBuildReferences.projitems` have been
converted from the old `packages.config` to the new
`@(PackageReference)` NuGet reference style.
The Azure Pipeline build job has been updated to completely build
`Xamarin.Android-Tests.sln` with `xabuild`, rather than only building
select projects. This was done to mirror our Jenkins build and to
ensure that `make all-tests` continues to work moving forward.
.NET Core and `NuGetCommand` tasks have been bumped to bring in a
newer version of NuGet with better `@(PackageReference)` support.
NuGet restore will be skipped if any issues occur when calculating the
dependency graph for the solution or project that it is operating on.
A lot of our sources attempt to import `.props`/`.targets`/`.projitems`
files that are generated by the build, and these should be
conditionally imported so that NuGet restore attempts can succeed
before these files are generated. This should also help when loading
these projects in VS or VS Mac.
Example NuGet restore error:
src/Xamarin.Android.Build.Tasks/Xamarin.Android.Build.Tasks.targets(8,3): error MSB4019: The imported project "bin/BuildRelease/ProfileAssemblies.projitems" was not found. Confirm that the expression in the Import declaration "../../bin/BuildRelease/ProfileAssemblies.projitems" is correct, and that the file exists on disk.
/var/folders/pp/x88nqyvd0h18ttl996vv2c3w0000gn/T/NuGetScratch/ok90j1ok.3t3.nugetrestore.targets(266,5): warning : Skipping restore for project 'src/Xamarin.Android.Build.Tasks/Xamarin.Android.Build.Tasks.csproj'. The project file may be invalid or missing targets required for restore.
I've also moved the `NuGet.exe` that is copied and used by tests into
its own folder to avoid a crash on macOS which would occur when
`NuGet.exe` is invoked from the same folder as
`Xunit.NetCore.Extensions.dll`:
mono bin/TestRelease/NuGet.exe
VTable setup of type Xunit.NetCore.Extensions.SkippedTestCase failed
System.TypeLoadException: VTable setup of type Xunit.NetCore.Extensions.SkippedTestCase failed
at (wrapper managed-to-native) System.RuntimeType.GetPropertiesByName_native(System.RuntimeType,intptr,System.Reflection.BindingFlags,System.RuntimeType/MemberListType)
at System.RuntimeType.GetPropertiesByName (System.String name, System.Reflection.BindingFlags bindingAttr, System.RuntimeType+MemberListType listType, System.RuntimeType reflectedType)
at System.RuntimeType.GetPropertyCandidates (System.String name, System.Reflection.BindingFlags bindingAttr, System.Type[] types, System.Boolean allowPrefixLookup)
at System.RuntimeType.GetProperties (System.Reflection.BindingFlags bindingAttr)
at System.ComponentModel.Composition.AttributedModel.AttributedPartCreationInfo+<GetExportMembers>d__32.MoveNext ()
at System.Linq.Enumerable.Any[TSource] (System.Collections.Generic.IEnumerable1[T] source)
...
Fixes: https://github.com/xamarin/xamarin-android/issues/2957
Context: https://blog.paranoidcoding.com/2016/04/05/deterministic-builds-in-roslyn.html
Since Visual Studio 2019, [`$(Deterministic)`][0] is set in the .NET
framework class library project template:
<PropertyGroup>
<Deterministic>True</Deterministic>
<!-- ... -->
</PropertyGroup>
It is also the default for SDK-style projects.
What this setting does is enable a stable MVID for the produced .NET
assembly. If you do `touch Foo.cs && msbuild`, the resulting
assembly will have its contents unchanged. However, the timestamp of
the file *will* change. There may be some narrow performance gains
we can get there.
The only drawback of `$(Determinitic)` is that the following will
produce a CS8357 build error:
[assembly: AssemblyVersion ("1.0.*")]
// error CS8357: The specified version string contains wildcards,
// which are not compatible with determinism.
// Either remove wildcards from the version string,
// or disable determinism for this compilation
Since `1.0.*` implies that the assembly changes on incremental
builds, `$(Deterministic)`=True cannot support it. So we can't just
make it default without breaking someone.
~~ What should we do? ~~
We should start building Xamarin.Android using `$(Deterministic)`=True
and add a few tests for it. We had some usage of
`[assembly:AssemblyVersion]` wildcards that seemed unnecessary.
When `$(Deterministic)`=True, any places using Mono.Cecil need to use
the appropriate `WriterParameters.DeterministicMvid` setting when
writing assemblies. The linker has an equivalent
`LinkContext.DeterministicOutput` property.
After this change makes it in, we should consider putting
`<Deterministic>True</Deterministic>` in the Xamarin templates.
We would also need to remove some comments in `AssemblyInfo.cs` that
suggests to use `[assembly: AssemblyVersion ("1.0.*")]`.
~~ Other changes ~~
I added a few general fixes to `Xamarin.ProjectTools`:
* Made properties so `AndroidUseSharedRuntime` and
`EmbedAssembliesIntoApk` are strongly typed.
* Added a `Deterministic` property
* In `BaseTest`, `CreateApkBuilder()` and `CreateDllBuilder()` now
have the `directory` parameter optional.
[0]: https://docs.microsoft.com/en-us/dotnet/visual-basic/reference/command-line-compiler/deterministic
Instead of having `src/Mono.Android/Mono.Android.projitems`, which
must be manually updated whenever new API level is added, *generate*
a `bin/Build$(Configuration)/Mono.Android.Apis.projitems` file based
on the contents of `xaprepare/ConfigAndData/BuildAndroidPlatforms.cs`,
as `BuildAndroidPlatforms.cs` must be updated *anyway*.
The idea here is to have fewer moving parts to be updated every time
there is a new API level available.
* Rename `Mono.Android.projitem` to `Mono.Android.Apis.projitem` so
we don't have the same file name as the `Mono.Android.pojitem`
under `src/Mono.Android/obj/$(Configuration)/android-*/mcw`.
* Modify `AndroidPlatform.cs` within `xaprepare` to support all
fields we need to generate the `.projitems` file.
* Create a new `xaprepare` step that will flush the data to the
`.projitem` xml file
* Update `BuildAndroidPlatforms.cs` with new fields and API info.
Note that although `Framework` field and `Include` field look
similar, we cannot reuse the `Framework` field as the `Include`
value because `Framework` corresponds to the `$(FRAMEWORKS)`
**make** variable, and should only contain values which we
actually support. The `Include` field, meanwhile, will contain
"historical" versions that are no longer supported, e.g. API-4.
* Remove `Mono.Android.projitems`.
* Update all references to point to the new location.
API-29/v10.0 is now stable.
Bumps to xamarin/xamarin-android-api-compatibility/master@336eedaa.
Bump Targets/`$(TargetSdkVersion)` to v10.0 (API-29).
Add `src/Mono.Android/Profiles/api-29.params.txt` for parameter names.
Add new enums and fix method signatures (`map.csv`, `methodmap.csv`).
Fix metadata whitespace formatting.
Update Android Emulator to use 29 image
Bump test timeouts.
Fix test issues:
* [build-tools] Update the logcat timing patterns for API-29
* [logcat processing] Update activity displayed detection
* Fixing some deploy/undeploy apks to emulator
* Ignore exit code for `adb logcat -c`
* Allow BuildTest.BuildHasNoWarnings to have up to 1 warning due to
be incompatible with Api level 29
Remove `#if !ANDROID_29` condition from `StringBuffer.cs` and
`StringBuilder.cs`, added in 2cace0b3, to preserve API compatibility.
By default, `mono app.exe` doesn't provide filename or line number
information in stack traces for unhandled exceptions, which
complicates the diagnosis of build failures.
`mono --debug app.exe` is required in order to get filename and line
number information, assuming that debug symbols are present.
Update `$(ManagedRuntime)` invocations so that
`$(ManagedRuntimeArgs)` is consistently provided. This will allow us
to reliably use `mono --debug=casts`, which will provide filename and
line number information in stack traces.
Context: https://developer.android.com/preview/overview [^1]
Android Q Beta 1 has been released. The API-Q Program Overview
[Timeline and updates][0] section suggests the following:
* Beta 2 in early April
* Beta 3 in early May
* Beta 4 in early June.
This contains the final APIs and official SDK.
* Beta 5, 6, and final release sometime in Q3, 2019.
Add support for binding API-Q as API-29, with
`$(TargetFrameworkVersion)` v9.0.99.
Note: The Android SDK **Build-tools** package version 29-rc1 appears
to be busted, in that `apksigner.jar` generates invalid `.apk` files.
Our [PR build][1] had errors attempting to install `.apk` files:
AndroidApkSigner:
ApkSignerJar: …/android-toolchain/sdk/build-tools/29.0.0-rc1/lib/apksigner.jar
ApkToSign: …/bin/TestRelease/Xamarin.Android.Locale_Tests-Signed.apk
ManifestFile: obj/Release/android/AndroidManifest.xml
AdditionalArguments:
java -jar …/android-toolchain/sdk/build-tools/29.0.0-rc1/lib/apksigner.jar sign --ks "…/.local/share/Xamarin/Mono for Android/debug.keystore" --ks-pass pass:android --ks-key-alias androiddebugkey --key-pass pass:android --min-sdk-version 16 --max-sdk-version 22 …/bin/TestRelease/Xamarin.Android.Locale_Tests-Signed.apk
...
Executing: …/android-toolchain/sdk/platform-tools/adb -s emulator-5570 install "…/bin/TestRelease/Xamarin.Android.Locale_Tests-Signed.apk"
…/build-tools/scripts/TestApks.targets(154,5): warning : adb: failed to install …/bin/TestRelease/Xamarin.Android.Locale_Tests-Signed.apk: Failure [INSTALL_PARSE_FAILED_NOT_APK: Failed to parse /data/app/vmdl242845261.tmp/base.apk]
Finally, remove `Animation` events from set~ method only after Q API.
Without this change, there was API compatibility breakage:
<h3>Type Changed: Android.Views.Animations.Animation</h3>
<p>Removed events:</p>
<pre>
<span class='removed removed-event breaking' data-is-breaking>public event System.EventHandler<Animation.AnimationEndEventArgs> AnimationEnd;</span>
<span class='removed removed-event breaking' data-is-breaking>public event System.EventHandler<Animation.AnimationRepeatEventArgs> AnimationRepeat;</span>
<span class='removed removed-event breaking' data-is-breaking>public event System.EventHandler<Animation.AnimationStartEventArgs> AnimationStart;</span>
</pre>
[0]: https://developer.android.com/preview/overview#timeline
[1]: https://jenkins.mono-project.com/job/xamarin-android-pr-pipeline-release/151/
[^1]: No `archive.org` URL because one doesn't exist yet. :-(
Fixes: https://github.com/xamarin/xamarin-android/issues/2627
There are several things broken when building with MSBuild 16.0
and/or using [Visual Studio 2019][0].
~~ TargetFrameworkVersion ~~
Anything referencing MSBuild assemblies must now use
`$(TargetFrameworkVersion)`=v4.7.2, since the MSBuild assemblies are
now targeting v4.7.2.
We should take this as an opportunity to consolidate all
`$(TargetFrameworkVersion)` values, so I moved this property into
`Configuration.props`. I was able to remove the property in many
projects, and the IDE still seemed to work fine.
The only hang up is that some versions of Mono don't have v4.7.2 yet.
I had to use the following trick, which use
[`ToolLocationHelper.GetPathToStandardLibraries()`][1]:
<_StandardLibraryPath>$([Microsoft.Build.Utilities.ToolLocationHelper]::GetPathToStandardLibraries('.NETFramework', 'v4.7.2', ''))</_StandardLibraryPath>
<TargetFrameworkVersion Condition=" '$(TargetFrameworkVersion)' == '' And '$(_StandardLibraryPath)' != '' ">v4.7.2</TargetFrameworkVersion>
<TargetFrameworkVersion Condition=" '$(TargetFrameworkVersion)' == '' ">v4.7.1</TargetFrameworkVersion>
This allows us to fallback to v4.7.1 if we have to.
~~ Prepend<T> ~~
With `$(TargetFrameworkVersion)` changing, this method:
static IEnumerable<T> Prepend<T> (this IEnumerable<T> l, T another)
Now exists in the BCL, so we can remove it.
~~ MSBuild.exe location ~~
In VS 2017, MSBuild is located at:
%VsInstallDir%\MSBuild\15.0\Bin\MSBuild.exe
In VS 2019, it is now located in:
%VsInstallDir%\MSBuild\Current\Bin\MSBuild.exe
Right now we have a bit of code that "finds" Visual Studio, MSBuild,
etc. so we should take this opportunity to improve it.
We originally thought about using MSBuildLocator:
https://www.nuget.org/packages/Microsoft.Build.Locator/https://github.com/Microsoft/MSBuildLocator
But the licensing of the library was concerning... We were uncertain
if we could redistribute MSBuildLocator as part of an OSS product.
For now we can just execute `vswhere.exe`:
https://github.com/Microsoft/vswhere/wiki/Find-MSBuild
I made a simple `vswhere.csproj` we can reference where this
functionality is needed. Currently `xabuild` and
`Xamarin.ProjectTools` need to locate the Visual Studio directory.
~~ Other breakage in xabuild.exe ~~
In VS 2019, the path to Roslyn is a bit odd:
%VsInstallDir%\MSBuild\15.0\Bin\Roslyn
I had to rework things to still work when combined with a different
MSBuild location:
%VsInstallDir%\MSBuild\Current\Bin\MSBuild.exe
Additionally, our binding redirects weren't working in this project at
all. `App.config` was *not* in the csproj!
~~ NUnit ~~
There does not appear to be an NUnit extension available for VS 2019
yet. I frequently use the `Test Explorer` in VS to individually run
tests.
However, it looks like they are switching to a different model for
test frameworks. Each test framework ships its own "adapter" that
enables the testing UIs in Visual Studio. These are shipped on
NuGet, and so they can work without installing any extra extensions.
So we just need this NuGet package:
https://www.nuget.org/packages/NUnit3TestAdapter/
This package should not affect anything on non-Windows platforms.
~~ ILRepack ~~
With the usage of `$(TargetFrameworkVersion)`=v4.7.2 we were hitting:
"src/Xamarin.Android.Build.Tasks/Xamarin.Android.Build.Tasks.csproj" (default target) (1) ->
(ILRepacker target) ->
src/Xamarin.Android.Build.Tasks/ILRepack.targets(24,3): error : Failed to resolve assembly: 'netstandard, Version=2.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51' [src/Xamarin.Android.Build.Tasks/Xamarin.Android.Build.Tasks.csproj]
We need to specify `$(TargetPlatformDirectory)`; path logic in
`<ILRepack/>` appears to be failing, see:
72c4c3bd05/ILRepack.Lib.MSBuild/ILRepack.cs (L148-L151)18698fddc4/ILRepack/ILRepack.cs (L450-L462)
~~ Other repositories ~~
We will need to switch to `$(TargetFrameworkVersion)`=v4.7.2 and use
the NUnit adapter in other repos:
- Downstream in `monodroid`, things will likely break without these
changes.
- `Java.Interop` is currently *working*, but should be updated.
- `xamarin-android-tools` is currently *working*, but should be
updated.
[0]: https://web.archive.org/web/20190307085042/https://visualstudio.microsoft.com/vs/preview/
[1]: e03296fe14/src/Utilities/ToolLocationHelper.cs (L1736-L1747)
Since c93a8af changed some projects to netstandard 2.0, we also need
`build-tools/api-xml-adjuster/api-xml-adjuster.csproj` to be v4.6.1:
Microsoft.Common.CurrentVersion.targets(2110,5): Warning MSB3274: The primary reference "E:\A\_work\34\s\external\Java.Interop\bin\Debug\Xamarin.Android.Tools.ApiXmlAdjuster.dll" could not be resolved because it was built against the ".NETFramework,Version=v4.6.1" framework. This is a higher version than the currently targeted framework ".NETFramework,Version=v4.5".
Which causes the error:
build-tools\api-xml-adjuster\Program.cs(12,18): Error CS0246: The type or namespace name 'JavaApi' could not be found (are you missing a using directive or an assembly reference?)
This doesn't seem to be a problem on Mono, just .NET framework.
Context: https://github.com/mono/mono/issues/7107
Context: e44c2520e3/src/Tasks/Microsoft.NET.Build.Tasks/targets/Microsoft.NET.Sdk.props (L78)
Since around the Mono 5.0 timeframe, projects built with
`$(DebugType)`=full or `$(DebugType)`=pdbonly are generating
"portable" PDB files next to the output assembly.
This is when using Mono.
However, building with full .NET framework / MSBuild on Windows,
`$(DebugType)`=full and `$(DebugType)`=pdbonly generates
"classic PDB"/full-PDB files. This is an older, Windows-only format
that Mono does not fully support.
This causes a problem for projects targeting the `MonoAndroid`
profile, since Xamarin.Android's build process will run the following
target if it encounters full-PDB files on Windows:
ConvertDebuggingFiles
Parameters
Files
samples\HelloWorld\bin\Debug\HelloWorld.pdb
bin\Debug\lib\xamarin.android\xbuild-frameworks\MonoAndroid\v9.0\Mono.Android.pdb
OutputItems
_ConvertedDebuggingFiles
samples\HelloWorld\bin\Debug\HelloWorld.dll
bin\Debug\lib\xamarin.android\xbuild-frameworks\MonoAndroid\v9.0\Mono.Android.dll
This is *seriously* messing with me as I try to profile builds on
Windows! This task runs once on `Mono.Android.dll` after building
`xamarin-android` for the first time, and it takes ~4.5 seconds!
When building `xamarin-android` on Windows, we were generating these
symbol files:
- `Mono.Android.dll.mdb` and many other profile assemblies
- `HelloWorld.dll.mdb` if you built the sample
These `.mdb` files were generated because Mono doesn't understand
full-PDB files, and the `<ConvertDebuggingFiles>` task converts
full-PDB files into `.mdb` files for use by mono.
To better align the outputs of our build between macOS and Windows,
I've made the following changes:
- Set `$(DebugType)=portable in `Configuration.props`
- Import `Configuration.props` if we weren't already
- Remove any instances of `<DebugType>portable</DebugType>` or
`<DebugType>pdbonly</DebugType>`
- Places that were using `DebugType=none` or `<DebugType></DebugType>`
were left alone
- One exception here is `tests/CodeBehind`, the way they are copied to
a directory such as
`bin\TestDebug\CodeBehind\SuccessfulBuildFew\Debug`, I thought it
simpler to hardcode `$(DebugType)`=portable.
In fact, this is the same approach taken by SDK-style projects as seen
in `Microsoft.NET.Sdk.props`:
<DebugType Condition=" '$(DebugType)' == '' ">portable</DebugType>
`$(DebugType)`=portable works for both `Debug` and `Release` builds.
Android P Developer Preview 3 has been released, which contains the
*final* API-28 Java API.
Provide an *initial* (not *stable*) API binding for API-28,
`$(TargetFrameworkversion)`=v9.0 (not v8.1.99). This initial binding
is not stable because enumification hasn't been performed yet.
Bumps to mono/2018-02/6f9bef3b
Related change: [API-28 removes the `Canvas.save(int)` method][0],
which caused `Xamarin.Android.McwGen-Tests` to not build:
[0]: https://developer.android.com/sdk/api_diff/p-dp2/changes
"javac" -source 1.5 -target 1.6 -d "obj/Debug/__CreateTestJarFile-bin" -cp "…/android-toolchain/sdk/platforms/android-28/android.jar" "java/com/xamarin/android/Bxc4288.java" "java/com/xamarin/android/Bxc58383.java" "java/com/xamarin/android/CallMethodFromCtor.java" "java/com/xamarin/android/CallNonvirtualBase.java" "java/com/xamarin/android/CallNonvirtualDerived.java" "java/com/xamarin/android/DataHandler.java" "java/com/xamarin/android/DataListener.java" "java/com/xamarin/android/Logger.java" "java/com/xamarin/android/MarkerIgnoringBase.java" "java/com/xamarin/android/MyCanvas.java" "java/com/xamarin/android/Timing.java" "java/com/xamarin/android/Bxc9446.java" "java/com/xamarin/android/Bxc7634.java" "java/com/xamarin/android/Bxc37706Throwable.java"
java/com/xamarin/android/MyCanvas.java:12: error: method does not override or implement a method from a supertype
@Override
^ (TaskId:1188)
java/com/xamarin/android/MyCanvas.java:15: error: method save in class Canvas cannot be applied to given types;
return super.save (saveFlags);
^
required: no arguments
found: int
reason: actual and formal argument lists differ in length
2 errors
Fix `MyCanvas.java` to instead override `Canvas.save()`.
`xbuild` is "wonky" in all manner of ways -- for better or worse
(648aff99, e3abe4b8, 069c65ae).
Worse, our unit tests has a tendency to throw
`ArgumentOutOfRangeException` when running under `xbuild`, e.g. this
recent [`BuildAotApplicationAndBundle("armeabi",True,True)` error][0]
which produces a (temporary, Workspace-local) [log file][1] with:
[0]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android/955/testReport/Xamarin.Android.Build.Tests/BuildTest/BuildAotApplicationAndBundle__armeabi__True_True_/
[1]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android/ws/xamarin-android/bin/TestRelease/temp/BuildAotApplicationAndBundle_armeabi_True_True/build.log
…/xamarin-android/bin/Release/lib/xamarin.android/xbuild/Xamarin/Android/Xamarin.Android.Common.targets: error : Error executing task BuildApk: Non-negative number required.
Parameter name: srcOffset
Error executing task BuildApk: System.ArgumentOutOfRangeException: Non-negative number required.
Parameter name: srcOffset
at System.Buffer.BlockCopy (System.Array src, System.Int32 srcOffset, System.Array dst, System.Int32 dstOffset, System.Int32 count) [0x0009d] in /Users/builder/jenkins/workspace/build-package-osx-mono/2017-12/external/bockbuild/builds/mono-x64/mcs/class/corlib/ReferenceSources/Buffer.cs:65
at System.IO.FileStream.WriteSegment (System.Byte[] src, System.Int32 src_offset, System.Int32 count) [0x00023] in /Users/builder/jenkins/workspace/build-package-osx-mono/2017-12/external/bockbuild/builds/mono-x64/mcs/class/corlib/System.IO/FileStream.cs:995
at System.IO.FileStream.WriteInternal (System.Byte[] src, System.Int32 offset, System.Int32 count) [0x00099] in /Users/builder/jenkins/workspace/build-package-osx-mono/2017-12/external/bockbuild/builds/mono-x64/mcs/class/corlib/System.IO/FileStream.cs:647
at System.IO.FileStream.Write (System.Byte[] array, System.Int32 offset, System.Int32 count) [0x00090] in /Users/builder/jenkins/workspace/build-package-osx-mono/2017-12/external/bockbuild/builds/mono-x64/mcs/class/corlib/System.IO/FileStream.cs:614
at System.IO.StreamWriter.Flush (System.Boolean flushStream, System.Boolean flushEncoder) [0x0007e] in /Users/builder/jenkins/workspace/build-package-osx-mono/2017-12/external/bockbuild/builds/mono-x64/mcs/class/referencesource/mscorlib/system/io/streamwriter.cs:318
at System.IO.StreamWriter.Write (System.Char[] buffer, System.Int32 index, System.Int32 count) [0x00078] in /Users/builder/jenkins/workspace/build-package-osx-mono/2017-12/external/bockbuild/builds/mono-x64/mcs/class/referencesource/mscorlib/system/io/streamwriter.cs:411
at System.IO.TextWriter.WriteLine (System.String value) [0x00070] in /Users/builder/jenkins/workspace/build-package-osx-mono/2017-12/external/bockbuild/builds/mono-x64/mcs/class/referencesource/mscorlib/system/io/textwriter.cs:490
at Microsoft.Build.BuildEngine.ConsoleLogger+BuildRecord.WriteLine (System.String message) [0x00051] in /Users/builder/jenkins/workspace/build-package-osx-mono/2017-12/external/bockbuild/builds/mono-x64/mcs/class/Microsoft.Build.Engine/Microsoft.Build.BuildEngine/ConsoleLogger.cs:856
at Microsoft.Build.BuildEngine.ConsoleLogger+BuildRecord.MessageHandler (Microsoft.Build.Framework.BuildMessageEventArgs args) [0x0001c] in /Users/builder/jenkins/workspace/build-package-osx-mono/2017-12/external/bockbuild/builds/mono-x64/mcs/class/Microsoft.Build.Engine/Microsoft.Build.BuildEngine/ConsoleLogger.cs:708
at Microsoft.Build.BuildEngine.ConsoleLogger.MessageHandler (System.Object sender, Microsoft.Build.Framework.BuildMessageEventArgs e) [0x00000] in /Users/builder/jenkins/workspace/build-package-osx-mono/2017-12/external/bockbuild/builds/mono-x64/mcs/class/Microsoft.Build.Engine/Microsoft.Build.BuildEngine/ConsoleLogger.cs:344
at Microsoft.Build.BuildEngine.EventSource.FireMessageRaised (System.Object sender, Microsoft.Build.Framework.BuildMessageEventArgs bmea) [0x00008] in /Users/builder/jenkins/workspace/build-package-osx-mono/2017-12/external/bockbuild/builds/mono-x64/mcs/class/Microsoft.Build.Engine/Microsoft.Build.BuildEngine/EventSource.cs:69
at Microsoft.Build.BuildEngine.BuildEngine.LogMessageEvent (Microsoft.Build.Framework.BuildMessageEventArgs e) [0x00000] in /Users/builder/jenkins/workspace/build-package-osx-mono/2017-12/external/bockbuild/builds/mono-x64/mcs/class/Microsoft.Build.Engine/Microsoft.Build.BuildEngine/BuildEngine.cs:130
at Microsoft.Build.Utilities.TaskLoggingHelper.LogMessageFromText (System.String lineOfText, Microsoft.Build.Framework.MessageImportance messageImportance) [0x00031] in /Users/builder/jenkins/workspace/build-package-osx-mono/2017-12/external/bockbuild/builds/mono-x64/mcs/class/Microsoft.Build.Utilities/Microsoft.Build.Utilities/TaskLoggingHelper.cs:314
at Microsoft.Build.Utilities.TaskLoggingHelper.LogMessage (Microsoft.Build.Framework.MessageImportance importance, System.String message, System.Object[] messageArgs) [0x0000e] in /Users/builder/jenkins/workspace/build-package-osx-mono/2017-12/external/bockbuild/builds/mono-x64/mcs/class/Microsoft.Build.Utilities/Microsoft.Build.Utilities/TaskLoggingHelper.cs:248
at (wrapper remoting-invoke-with-check) Microsoft.Build.Utilities.TaskLoggingHelper.LogMessage(Microsoft.Build.Framework.MessageImportance,string,object[])
at Xamarin.Android.Tasks.MSBuildExtensions.LogDebugTaskItems (Microsoft.Build.Utilities.TaskLoggingHelper log, System.String message, Microsoft.Build.Framework.ITaskItem[] items) [0x00030] in <d1b2bbe8a91444ef81018dda8acfe92a>:0
at Xamarin.Android.Tasks.BuildApk.Execute () [0x001da] in <d1b2bbe8a91444ef81018dda8acfe92a>:0
at Microsoft.Build.BuildEngine.TaskEngine.Execute () [0x00000] in /Users/builder/jenkins/workspace/build-package-osx-mono/2017-12/external/bockbuild/builds/mono-x64/mcs/class/Microsoft.Build.Engine/Microsoft.Build.BuildEngine/TaskEngine.cs:134
at Microsoft.Build.BuildEngine.BuildTask.Execute () [0x0008d] in /Users/builder/jenkins/workspace/build-package-osx-mono/2017-12/external/bockbuild/builds/mono-x64/mcs/class/Microsoft.Build.Engine/Microsoft.Build.BuildEngine/BuildTask.cs:101
It's hard to have a "green" build when the build system randomly
fails. Furthermore, `xbuild` is no longer maintained, so whatever
bug we're triggering in `xbuild` *will not be fixed*, `xbuild` itself
is ***deprecated***, and the #mono team wants us to use `msbuild`.
Update the build system so that we use `msbuild` by default instead.
Additionally, repair remaining `xbuild`-isms, in particular:
In `msbuild`, item groups are "cumulative". For example, because both
the `RunNUnitTests` and `RunJavaInteropTests` targets updated the
`@(_RenameTestCasesGlob)` item group, when `RunJavaInteropTests` was
executed it contained items from the `RunNUnitTests` execution, which
was unexpected and resulted in a test execution failure:
…/build-tools/scripts/TestApks.targets(221,5): error : Could not find file "/Users/builder/jenkins/workspace/xamarin-android-msbuild-pr-builder/TestResult-Xamarin.Android.Build.Tests.xml"
`TestResult-Xamarin.Android.Build.Tests.xml` didn't exist during the
execution of the `RunJavaInteropTests` because it was deleted/renamed
as part of the `RunNUnitTests` target.
There is also an unintended `make`-ism: `$(USE_MSBUILD)` is used by
various parts of the build system *and unit tests* to determine
whether or not `msbuild` or `xbuild` is being used, and change
behavior accordingly. Unfortunately, `$(USE_MSBUILD)` wasn't being
[`export`ed][2], and thus wasn't visible to child processes.
Consequently, even though `msbuild` was being used for the build
and unit test execution, the unit tests thought that `xbuild`-based
outputs should be used for checks. This resulted in at least three
`Xamarin.Android.Build.Tests` failures:
[2]: https://www.gnu.org/software/make/manual/html_node/Variables_002fRecursion.html#Variables_002fRecursion
* `Xamarin.Android.Build.Tests.AndroidUpdateResourcesTest.BuildAppWithManagedResourceParserAndLibraries / Debug`
* `Xamarin.Android.Build.Tests.BuildTest.CheckJavaError / Debug`
* `Xamarin.Android.Build.Tests.BuildTest.BuildAMassiveApp / Debug`
If necessary, `xbuild` can continue to be used to build things:
make all MSBUILD=xbuild
However, we make no guarantees for how long `xbuild` will continue to
work going forward.
Commit e3abe4b8 broke caching of the
`bin/Build$(Configuration)/api/api-*.xml.*` files, reintroducing the
scenario that commit 5c46ee39 fixed. The result is that *minutes* may
be spent needlessly re-generating API XML.
The cause of the breakage is that `@(_ApiParameterDescription)`
(introduced in e3abe4b8) will contain entries that we don't care
about, e.g. API-4, for which no longer generate bindings. As such, the
`_ClassParse` target Inputs were looking for files which will never
exist, thus causing the `_ClassParse` target to always execute.
This *could* be fixed by making `@(_ApiParameterDescription)`/etc.
entries conditional on on the file
`%(_ApiParameterDescription.ParameterDescription)` refers to actually
existing, a'la:
<ItemGroup>
<_ApiParameterDescription
Condition="Exists('%(ApiFileDefinition.ParameterDescription)')"
Include="%(ApiFileDefinition.ParameterDescription)"
/>
</ItemGroup>
However, this is suboptimal, as it means everything must still be
rebuilt if only one file changes:
$ touch src/Mono.Android/Profiles/api-27.params.txt
$ (cd build-tools/api-xml-adjuster ; time msbuild )
real 2m20.205s
# eep!
What we really want is [MSBuild Target Batching][0], but our use of
item groups *without* item metadata prevents target batching!
[0]: https://msdn.microsoft.com/en-us/library/ms171473.aspx
Which leads to a realization: the rationalization for commit e3abe4b8
was incomplete: The problem wasn't using metadata within Target Inputs
and Outputs; the problem was intermixing strings with metadata.
Meaning the pre-e3abe4b8 `api-xml-adjuster.targets` was *correct*
(except for the bit about it mentioning files which didn't exist,
causing needless rebuilds):
<!-- Good! Permits Target Batching -->
<Target Name="_ClassParse"
Inputs="%(ApiFileDefinition.ParameterDescription)"
Outputs="%(ApiFileDefinition.ApiAdjustedXml)"
...
vs. what e3abe4b8 was attempting to fix:
<!-- BAD! Intermixes metadata with strings -->
<Target Name="_Make"
Inputs="$(IntermediateOutputPath)\%(_LibZipTarget.Identity)\Makefile"
...
Reviewing e3abe4b8, `api-xml-adjuster.targets` is the only file that
was *already* properly using item medata within Target Inputs & Outputs.
Fix `class-parse.exe` and `api-xml-adjuster.exe` rebuilds by properly
using MSBuild Target Batching, *and* by *filtering*
`@(ApiFileDefinition)` so that it only contains values for which we
have a `src/Mono.Android/Profiles/api-*.params.txt` file.
With these changes in place, single-file rebuilds are much faster:
$ touch src/Mono.Android/Profiles/api-27.params.txt
$ (cd build-tools/api-xml-adjuster ; time msbuild )
real 0m26.662s
Whenever `external/mono` is bumped, the
[`xamarin-android-msbuild` job breaks][0]:
[0]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android-msbuild/838/
Building target "_Make" completely.
...
Task "Touch" (TaskId:1642)
Task Parameter:
Files=
…/bin/Debug/lib/xamarin.android/xbuild/Xamarin/Android/x64/libzip.dll
CMake=/Users/builder/android-toolchain/mxe-b9cbb53/bin/x86_64-w64-mingw32.static-cmake
CMakeFlags=
CopyToOutputDirectory=Always
OutputLibrary=x64/libzip.dll
OutputLibraryPath=lib/libzip.dll
…/bin/Debug/lib/xamarin.android/xbuild/Xamarin/Android/libzip.dll
CMake=/Users/builder/android-toolchain/mxe-b9cbb53/bin/i686-w64-mingw32.static-cmake
CMakeFlags=
CopyToOutputDirectory=Always
OutputLibrary=libzip.dll
OutputLibraryPath=lib/libzip.dll (TaskId:1642)
Touching "…/bin/Debug/lib/xamarin.android/xbuild/Xamarin/Android/x64/libzip.dll". (TaskId:1642)
…/build-tools/libzip/libzip.targets(52,5): error MSB3375: The file "…/bin/Debug/lib/xamarin.android/xbuild/Xamarin/Android/libzip.dll" does not exist. […/build-tools/libzip-windows/libzip-windows.csproj]
Done executing task "Touch" -- FAILED. (TaskId:1642)
This is due to an "`xbuild`-ism" we were inadvertently using which
`msbuild` doesn't like: when item metadata is used in
`//Target/@Inputs`, [MSBuild Target Batching][1] is *not* used.
[1]: https://msdn.microsoft.com/en-us/library/ms171473.aspx
Case in point, the `_Make` target:
<!-- BAD! -->
<Target Name="_Make"
Condition=" '@(_LibZipTarget)' != '' "
Inputs="$(IntermediateOutputPath)\%(_LibZipTarget.Identity)\Makefile"
Outputs="@(Content)">
Because this used "inline" item metadata, the target wasn't being
batched as intended (and as `xbuild` does). Consequently,
`@(_LibZipTarget)` was only being built for mxe-Win64, *not*
mxe-Win32, which is why the `<Touch/>` use within the `_Make` target
failed.
The fix? Don't Do That™. Instead, use a new `@(_LibZipTargetMakefile)`
item group for the `//Target/@Inputs`, which allows `msbuild` to
properly batch the `_Make` target contents:
<ItemGroup>
<_LibZipTargetMakefile Include="$(IntermediateOutputPath)\%(_LibZipTarget.Identity)\Makefile" />
</ItemGroup>
<Target Name="_Make"
Condition=" '@(_LibZipTarget)' != '' "
Inputs="@(_LibZipTargetMakefile)"
Outputs="@(Content)">
Review other files matching `git grep 'Inputs=.*%'` and fix them too.
Ideally, project rebuilds when nothing has changed should be *fast*.
`api-xml-adjuster` isn't:
$ time (cd build-tools/api-xml-adjuster ; xbuild)
real 2m1.084s
user 1m55.916s
sys 0m8.853s
# and the rebuild!
$ time (cd build-tools/api-xml-adjuster ; xbuild)
real 2m0.824s
user 1m56.140s
sys 0m8.600s
A *minimum* two minute+ rebuild -- when *nothing* has changed -- is a
surefire way to get really annoyed.
With diagnostic logging, we start to see the culprit:
Target _ClassParse needs to be built as input file '@(ApiFileDefinition -> /Volumes/Seagate4TB/work/xamarin-android/build-tools/api-xml-adjuster/../../src/Mono.Android/Profiles/api-27.params.txt)' does not exist.
This in turn causes `class-parse.exe` and `api-xml-adjuster.exe` to be
*re-executed* on *every* `android.jar` on *every* build.
Fix this by correcting the `//Target/@Inputs` and `//Target/@Outputs`
for the `_ClassParse` and `_AdjustApiXml` tasks. After which,
no-change rebuilds are *significantly* faster:
$ time (cd build-tools/api-xml-adjuster ; xbuild)
real 0m5.308s
user 0m6.042s
sys 0m1.237s
Since 7d705bf, the Windows builds on VSTS have seemed to be failing. I
tested this locally, and noticed the use of command line arguments such
as:
-parameter-names='%(SomeVariable)'
Unfortunately, this isn't working on Windows due to the single quote. It
is more appropriately expressed as:
-parameter-names="%(SomeVariable)"
This isn't very pretty, but it should work on all platforms.
I also fixed all the tabs I saw in this file--in favor of spaces, and
fixed other XML code conventions.
I will now return to my regularly scheduled baby duty.
Part of the XA build process is download and installation of Android SDK, NDK as
well as creation of native compilation toolchains. However, it appears that at
some point we regressed and stopped performing those tasks and, thus, the builds
fail when adjusting API description files with api-xml-adjuster since at this
point no SDK, NDK or toolchains are found. The download and installation of
those components is handled by the android-toolchain project which is dependent
upon by several other projects in XA that require the SDK and the NDK to be
present. android-xml-adjuster needs them as well but it hasn't dependent on
android-toolchain until this commit which adds a reference to the toolchain
project and fixes the build.
Bumps to Java.Interop/master/e1ad5066.
When adding a new API level, instead of using `class-parse -docspath`
to parse JavaDoc/AndroidDoc/etc., use `class-parse --parameter-names`
with a "JavaApiParameterNames" file generated by
[`xamarin-android-docimporter-ng`][import].
[import]: https://github.com/xamarin/java.interop/tree/e1ad506606/build-tools/xamarin-android-docimporter-ng/xamarin-android-docimporter-ng
Eventually we'll remove `src/Mono.Android/Profiles/api*.xml.in`
and instead generate those files at build-time by processing
`android.jar`.
Update `build-tools/api-xml-adjuster` to use the
JavaApiParameterNames files and generate `api-*.xml.in`, which now
takes only a few minutes.
Regenerate `src/Mono.Android/Profiles/api-*.xml.in` using the new
`api-xml-adjuster`. This results in some fixes, e.g.
`NEGATIVE_INFINITY` should be `(-1.0f / 0.0f)`, not `-Infinity`,
and some "parameter name regressions," e.g. in `api-24.xml.in`
some parameter names were changed *to* `p0` and other meaningless
names. *This is not breakage*; `api-merge` uses the previous
API level parameter names when it encounters `p0`/etc. names,
so these changes don't result in any API breakage.
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58994
When loading `Xamarin.Android.sln` into Visual Studio for Mac, the
Solution Panel indicates an error on the `Mono.Android` and
`Mono.Android.Export` projects:
Project does not support framework 'MonoAndroid,v1.0'.
Fix this warning by adding `$(ProjectTypeGuids)` to
`Mono.Android.csproj` and `Mono.Android.Export.csproj`.
Additionally, some projects such as
`src/Xamarin.Android.Build.Tasks/Xamarin.Android.Build.Tasks.csproj`
*do not build* within Visual Studio for Mac, because Visual Studio for
Mac doesn't appear to support the use of MSBuild properties within
`<Import/>`s -- at least not *our* property use. Specifically, this:
<Import
Project="$(JavaInteropFullPath)\src\Java.Interop.Tools.TypeNameMappings\Java.Interop.Tools.TypeNameMappings.projitems"
Label="Shared"
Condition="Exists('$(JavaInteropFullPath)\src\Java.Interop.Tools.TypeNameMappings\Java.Interop.Tools.TypeNameMappings.projitems')"
/>
is *completely ignored*, which results in multiple C# compiler errors
as types which are provided by that shared project cannot be found.
Appease Visual Studio for Mac by *removing* use of such properties
within `.csproj` files, and instead hardcoding relative paths.
Unfortunately this kills the idea from commit d205cab2:
> Allow the Java.Interop checkout directory to be specified by
> overriding the `$(JavaInteropSourceDirectory)` MSBuild property.
> ...
> Normally `$(JavaInteropSourceDirectory)` wouldn't need to be
> overridden; the current use case is to allow a CI-like environment
> which grabs the latest commit of every referenced module to make sure
> they all work together. (Note: such a "CI-like environment"
> DOES NOT (yet) EXIST. This commit is to help *permit* such a thing.)
Such a "CI-like environment" still doesn't exist, and is arguably
moot/silly, as Jenkins has an option to
**Update tracking submodules to tip of branch**, which is more-or-
less what was being advocated for here.
Using e.g. `$(JavaInteropFullPath)` -- the value of which was based
off of `$(JavaInteropSourceDirectory)` -- actively prevents
Visual Studio for Mac from building projects, so remove that use.
Finally, Visual Studio for Mac is *still* unable to do a toplevel
Build, even after all of these changes: Visual Studio for Mac doesn't
like `@(ProjectReference)` items which are `Condition`al, and we have
those all over the place (for now).
The changes in this commit *improve*, but **do not** "fix",
Visual Studio for Mac use.
Just build projects individually, with Command+K. :-)
Context: https://github.com/xamarin/xamarin-android/pull/253#discussion_r82862993
The *intention* is that Jenkins-produced `oss-xamarin.android*.zip`
artifacts be usable on Windows, so that side-by-side testing can be
performed without replacing the system installation. Usage is in
[UsingJenkinsBuildArtifacts.md](Documentation/UsingJenkinsBuildArtifacts).
This isn't *entirely* the case. It was *apparently* the case at the
time of commit 87ca2737, but things have bitrotten since. For
example, following the 87ca2737 instructions would currently result
in an XA0020 `Could not find monodroid` error, because
`class-parse.exe` wasn't where Windows expects it to be (it was in
`lib/mandroid`, not `lib/xbuild/Xamarin/Android`).
This needs to be fixed.
Additionally, PR #253 mentions that, for filesystem organization, it
would be useful if the macOS/Linux directory structure --
`$prefix/bin`, `$prefix/lib/mandroid`,
`$prefix/lib/xbuild/Xamarin/Android`, `$prefix/lib/xbuild-frameworks`
-- more closely resembled the Windows directory structure of
`$MSBuild` and `$ReferenceAssemblies` (as seen in `.vsix` files).
This would turn macOS/Linux into using `$xa_prefix/xbuild` and
`$xa_prefix/xbuild-frameworks` directories.
`$prefix/bin` would only contain `xabuild`. What is currently in
`$prefix/lib/mandroid` would be merged with
`$xa_prefix/xbuild/Xamarin/Android`.
`$xa_prefix` would `$prefix/lib/xamarin.android`.
This would turn the current macOS structure:
$prefix/bin/xabuild
$prefix/bin/generator
$prefix/bin/cross-arm
$prefix/lib/mandroid/generator.exe
$prefix/lib/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll
$prefix/lib/xbuild/Xamarin/Android/Xamarin.Android.Common.targets
Into:
$prefix/bin/xabuild
$prefix/lib/xamarin.android/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll
$prefix/lib/xamarin.android/xbuild/Xamarin/Android/generator.exe
$prefix/lib/xamarin.android/xbuild/Xamarin/Android/Xamarin.Android.Common.targets
$prefix/lib/xamarin.android/xbuild/Xamarin/Android/Darwin/cross-arm
Other notes:
* The `bundle-*.zip` filename has been has been changed to include a
*hash* of the contents of various files, in particular
`build-tools\mono-runtimes.*`. This was instigated via a
conversation with @kumpera about making the filename more
"idiot-proof": `mono-runtimes.props` contains *compiler flags*,
and if any of those change, then *logically* the bundle should
differ as well, as the mono runtimes may differ in significant
ways. In theory, the `-vXX` version should be used to track this,
but this is a manual change, easy to overlook. The new `-hHASH`
part of the filename should be more automagic.
The new `<HashFileContents/>` task in `xa-prep-tasks.dll` is
responsible for creating the hash value.
* `Configuration.Java.Interop.Override.props` was moved into
`build-tools/scripts`, because that would cleanup the root
directory a bit.
* OS-specific binaries are now placed into
`$prefix/lib/xamarin.android/xbuild/Xamarin/Android/$(HostOS)`.
On a theoretical plus side, this means that the same build
directory can contain OS-specific binaries for multiple operating
systems. (I don't know if anyone shares a build tree between e.g.
macOS and Linux, but if anyone *does*...)
Unfortunately this requires a workaround for an `xbuild` bug:
`%(_LlvmRuntime.InstallPath)` and
`%(_MonoCrossRuntime.InstallPath)` *cannot* use MSBuild
properties, e.g. `<InstallPath>$(HostOS)/</InstallPath>` doesn't
work as desired; it's instead treated literally.
Special-case `%(InstallPath)` until we fully migrate to MSBuild.
* `$(MonoAndroidToolsDirectory)` should be considered *dead*, along
with `$(MonoAndroidBinDirectory)`, as these should now *always*
be the same directory as where `Xamarin.Android.Build.Tasks.dll`
is located, or a `$(HostOS)` sub-directory.
* `Xamarin.ProjectTools.csproj` needed to be updated to ensure that
the build order was correct.
* Remove all `[Obsolete]` and unreferenced members from
`Xamarin.Android.Build.Utilities.dll`. There's too much in there,
and it makes my head hurt trying to understand the
interrelationships between it all. If it's not used, it's gone.
* The changes to `src/monodroid/jni/Android.mk` are...weird. The
removal of `-I$(topdir)/libmonodroid/zip`/etc. is to reduce use of
non-existent paths, as `$(topdir)` isn't defined, so that's
*actually* `-I/libmonodroid/zip`, which is nonsensical. So far,
so good. What's *odd* is the addition of `$(LOCAL_PATH)` to
`$(LOCAL_C_INCLUDES)`: This is needed so that
`external/mono/support/zlib-helper.c` exports `CreateZStream` and
related symbols, otherwise we get a unit test failure in
`GzipStreamTest.Compression` due to an
`EntryPointNotFoundException`, because `CreateZStream` isn't
exported/public.
What's odd here is that I don't understand what caused this
behavior to change. Previous builds exported `CreateZStream`,
otherwise the tests would fail, and I don't understand how any of
the other changes in this PR would be at fault, though that's
certainly the most plausible explanation.
Regardless, `-Ijni` *first* (due to adding `$(LOCAL_PATH)` to
`$(LOCAL_C_INCLUDES)`) is the desired behavior, so that
`jni/config.h` is included, thus ensuring that `MONO_API` has the
required definition when building `zlib-helper.c`.
Note that API Level 26 (O) is not built by default. To build it, you'll
have to specify extra `make` arguments:
# as usual....
make prepare all
# for API-O:
make framework-assemblies API_LEVELS=26
# -or-
xbuild src/Mono.Android/Mono.Android.csproj \
/p:AndroidLatestApiLevel=26 \
/p:AndroidLatestFrameworkVersion=v7.99.0 \
/p:AndroidPlatformId=O
Alternatively, if you want to build *only* API-O, you can override
the `$(AndroidLatestApiLevel)`, `$(AndroidLatestFrameworkVersion)`,
and `$(AndroidPlatformId)` MSBuild properties within
`Configuration.Override.props`.
First step towards support for Android-O Preview API.
The changes are big due to the structural changes in Android SDK,
particularly that "android" command line tool is gone and they had to
be rewritten. Samples that depended on "create" or "update" command
had to be checked in as already-created state so that they don't have
to bring further dependencies.
Changes in the library (Mono.Android) itself is not very big at all.
`src/Mono.Android/Profiles/api-O.xml.in` comes from API-O DP 1's
`android.jar`.
Jonathan Pobst
Peter Collins
Radek Doulik
Marek Habersack
Radek Doulik
Jonathan Pryor
Gustavo Varo
Jonathan Peppers
Atsushi Eno
Jonathan Peppers