xamarin
/
xamarin-android
зеркало из https://github.com/xamarin/xamarin-android.git
1
0
Форкнуть 0
Код Задачи Пакеты Проекты Релизы Вики Активность
xamarin-android/build-tools/mono-runtimes/mono-runtimes.targets

660 строки
33 KiB
Plaintext
Исходник Обычный вид История

[mono-runtimes] Build required mono runtimes. Add a new build-tools/mono-runtimes project to build the required Mono runtimes for Xamarin.Android app execution on Android devices. Xamarin.Android 6.0 ("cycle 6") includes Mono runtimes compiled for five architectures: * armeabi (armv5) * armeabi-v7a (armv7) * arm64-v8a (aarch64) * x86 * x86_64 An additional "host" architecture is also built to build the "monodroid" profile assemblies such as mscorlib.dll. In the interest of expediency and not requiring that *six* different runtimes be built -- which can greatly increase build times and storage requirements -- the initial xamarin-android open-source build will only build the armeabi-v7a (armv7) and "host" architectures. Additionally, the <UnzipDirectoryChildren/> task needed to be altered to use unzip(1) on OS X instead of using System.IO.Compression.ZipFile, because the ZipFile implementation included with Mono 4.4 doesn't respect file attributes such as the +x, causing all files to be extracted as 644 (rw-r--r--), resulting in an NDK toolchain which wouldn't work. To resolve this issue, execution on OS X and Linux now uses unzip(1). ~~ Adding new architectures ~~ To build Mono for a new architecture, edit build-tools/mono-runtimes/mono-runtimes.projitems and add a new @(_MonoRuntime) entry for the desired architecture. TODO: Figure out if there's a reasonable way to "opt-in" or "opt-out" of CPU architectures so that it isn't an "all or nothing" prospect.
2016-04-20 04:30:00 +03:00
<?xml version="1.0" encoding="utf-8"?>
<Project DefaultTargets="Build" ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
[xa-prep-tasks] Add <GitCommitTime/> (#734) Context: #181 (in comments) In several places throughout the build, we are running a command such as `<Exec Command="touch -m -t `git log -1 --format=%25cd --date=format-local:%25Y%25m%25d%25H%25M.%25S` Makefile" />`, which will not work on Windows. The solution is to create an MSBuild task in xa-prep-tasks named `GitCommitTime` that will work cross-platform. `GitCommitTime` returns a string value that can be passed to the `Touch` MSBuild task.
2017-08-10 22:43:02 +03:00
<UsingTask AssemblyFile="$(MSBuildThisFileDirectory)..\..\bin\Build$(Configuration)\xa-prep-tasks.dll" TaskName="Xamarin.Android.BuildTools.PrepTasks.GitCommitTime" />
[bundle] Create the bundle (#206) Using the mono bundle (29568117, 57b18c27, ffe39776) didn't work as well as intended, because *using* the mono bundle requires that the bundle (1) *exist*, and (2) contain expected outputs. Neither of which is true: $ curl -o x.zip https://xamjenkinsartifact.blob.core.windows.net/xamarin-android/xamarin-android/bin/BuildDebug/bundle-v3-Debug-Darwin-libzip=1d8b1ac,llvm=8b1520c,mono=2c13a95.zip $ file x.zip x.zip: XML document text $ cat x.zip <?xml version="1.0" encoding="utf-8"?> <Error> <Code>BlobNotFound</Code> <Message>The specified blob does not exist. RequestId:f1722d1f-0001-0098-3fdd-fe4670000000 Time:2016-08-25T14:31:09.4102289Z </Message> </Error> That is, the URL "exists", but it doesn't contain anything *useful*. The bundle doesn't exist, in turn, because [*it isn't created*][0]: Project "/Users/builder/jenkins/workspace/xamarin-android/xamarin-android/build-tools/bundle/bundle.mdproj" (default target(s)): ... Done building project "/Users/builder/jenkins/workspace/xamarin-android/xamarin-android/build-tools/bundle/bundle.mdproj". What's *missing* from the log file is any mention of the `CreateBundle` target, which *creates the bundle*. Since the `CreateBundle` target isn't invoked, *there is no bundle*, and thus there's *nothing to upload*, and thus there's *nothing to download*, either. Why isn't the `CreateBundle` target executed? Because of property group "overwrites". But first, a step back. Remember 57b18c27? > I *assume* that this is due to `<CallTarget/>` being "flakey" within > xbuild (it passes the file containing the target to `<MSBuild/>`!), > and *somehow* in that "flakiness" `$(OutputPath)` doesn't end with > `\`, *which it should*, because `Microsoft.Common.targets` will > automatically append `\` to `$(OutputPath)`. Let's elaborate on the "flakiness" of xbuild: The `<CallTarget/>` task doesn't *just* invoke the specified target. Rather, xbuild will *reload the file containing the target*, "stand-alone", and execute the target. It's "as if" `<CallTarget/>` were implemented as: <MSBuild Projects="*File-Containing-The-Target-To-Invoke*" Targets="...Targets..." /> Therein lies the problem: by convention, ~everything is using the `$(BuildDependsOn)` property, and 29568117 altered `mono-runtimes.targets` so that `$(BuildDependsOn)` would be explicitly specified, but `mono-runtimes.targets` is `<Import/>`ed by `bundle.targets`, thus *replacing* the *intended* definition of `$(BuildDependsOn)` from `bundle.mdproj`. Thus, `$(BuildDependsOn)` *didn't* include the `CreateBundle` target, so the `CreateBundle` target was never executed, thus *no bundle was created or uploaded*. Doh! The fix? Remove all use of `<CallTarget/>`. `<CallTarget/>` is a BUG. Instead, *always* use `<MSBuild/>`, which allows specifying the correct project file to build from (the `.mdproj`, in our case), so that the correct "deferred" `ForceBuild` target can be invoked. Unfortunately, [using the `<MSBuild/> task raised *another* issue][1]: > this task uses the same MSBuild process to build the child projects. > The list of already-built targets that can be skipped is shared > between the parent and child builds. This means that the [child build *skips* running the `_GetRuntimesOutputItems` target][2], which means that *everything breaks* because nothing is built, because there are no `Inputs` or `Outputs` for anything. Target _BuildRuntimes: No input files were specified for target _BuildRuntimes, skipping. Done building target "_BuildRuntimes" in project "/Users/builder/jenkins/workspace/xamarin-android-pr-builder/xamarin-android/build-tools/mono-runtimes/mono-runtimes.mdproj". We work around this issue by providing a `$(_ForceXbuildToNotCacheTargets)` property to the `<MSBuild/>` task with the value of `DateTime.Now.Ticks`, i.e. a ~constantly changing value, which should *prevent* the target skipping. (We hope.) *Additionally*, `msbuild` reported a few errors when referencing `%(_LibZipTarget.OutputLibraryPath)` item metadata, so fix those references. Finally, another aspect of `<MSBuild/>` behavior arose: The [`MSBuild.Properties`][1] parameter is equivalent to the `msbuild /p:` parameter, which in turn means that the *value* is *immutable*; it *can't* be changed (or fixed!) from `.targets` files. The result being that `<MSBuild/>` use in `android-toolchain.targets` is *fraught with peril*: <MSBuild Projects="..\..\src\Xamarin.Android.Tools.BootstrapTasks\Xamarin.Android.Tools.BootstrapTasks.csproj" Properties="OutputPath=$(AndroidToolchainDirectory)" /> This *looks* reasonable, but `$(OutputPath)` is expected to end with the directory separator char, and `Microsoft.Common.targets` *tries* to ensure this: <OutputPath Condition="'$(OutputPath)' != '' and !HasTrailingSlash('$(OutputPath)')">$(OutputPath)\</OutputPath> However, since `MSBuild.Properties` values are *immutable*, this attempt to modify the `$(OutputPath)` property to end with a `\` has *no effect*, which results in *bizarre* errors later: error MSB4062: The "Xamarin.Android.BuildTools.PrepTasks.DownloadUri" task could not be loaded from the assembly $HOME/android-toolchainxa-prep-tasks.dll. Note the path, `$HOME/android-toolchainxa-prep-tasks.dll`. The path *should* be `$HOME/android-toolchain/xa-prep-tasks.dll`. What's missing is...the directory separator char after `$(OutputPath)`. The fix? Explicitly provide the directory separator character in the `<MSBuild/>` invocation: <MSBuild Projects="..\..\src\Xamarin.Android.Tools.BootstrapTasks\Xamarin.Android.Tools.BootstrapTasks.csproj" Properties="OutputPath=$(AndroidToolchainDirectory)\" /> [0]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android/51/consoleText [1]: https://msdn.microsoft.com/en-us/library/z7f65y0d.aspx [2]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android-pr-builder/131/consoleText
2016-09-03 00:13:23 +03:00
<Target Name="Build" DependsOnTargets="$(BuildDependsOn)" />
[mono-runtimes] Build required mono runtimes. Add a new build-tools/mono-runtimes project to build the required Mono runtimes for Xamarin.Android app execution on Android devices. Xamarin.Android 6.0 ("cycle 6") includes Mono runtimes compiled for five architectures: * armeabi (armv5) * armeabi-v7a (armv7) * arm64-v8a (aarch64) * x86 * x86_64 An additional "host" architecture is also built to build the "monodroid" profile assemblies such as mscorlib.dll. In the interest of expediency and not requiring that *six* different runtimes be built -- which can greatly increase build times and storage requirements -- the initial xamarin-android open-source build will only build the armeabi-v7a (armv7) and "host" architectures. Additionally, the <UnzipDirectoryChildren/> task needed to be altered to use unzip(1) on OS X instead of using System.IO.Compression.ZipFile, because the ZipFile implementation included with Mono 4.4 doesn't respect file attributes such as the +x, causing all files to be extracted as 644 (rw-r--r--), resulting in an NDK toolchain which wouldn't work. To resolve this issue, execution on OS X and Linux now uses unzip(1). ~~ Adding new architectures ~~ To build Mono for a new architecture, edit build-tools/mono-runtimes/mono-runtimes.projitems and add a new @(_MonoRuntime) entry for the desired architecture. TODO: Figure out if there's a reasonable way to "opt-in" or "opt-out" of CPU architectures so that it isn't an "all or nothing" prospect.
2016-04-20 04:30:00 +03:00
<PropertyGroup>
Copy FSharp support assemblies from nuget (#21) Instead of building F# from sources as we've been doing so far, we'll now use a nuget to get the binaries we require. This makes the build time much shorter.
2016-05-06 00:07:05 +03:00
<_SourceTopDir>..\..</_SourceTopDir>
[structure] Rework installation directory structure (#704) Context: https://github.com/xamarin/xamarin-android/pull/253#discussion_r82862993 The *intention* is that Jenkins-produced `oss-xamarin.android*.zip` artifacts be usable on Windows, so that side-by-side testing can be performed without replacing the system installation. Usage is in [UsingJenkinsBuildArtifacts.md](Documentation/UsingJenkinsBuildArtifacts). This isn't *entirely* the case. It was *apparently* the case at the time of commit 87ca2737, but things have bitrotten since. For example, following the 87ca2737 instructions would currently result in an XA0020 `Could not find monodroid` error, because `class-parse.exe` wasn't where Windows expects it to be (it was in `lib/mandroid`, not `lib/xbuild/Xamarin/Android`). This needs to be fixed. Additionally, PR #253 mentions that, for filesystem organization, it would be useful if the macOS/Linux directory structure -- `$prefix/bin`, `$prefix/lib/mandroid`, `$prefix/lib/xbuild/Xamarin/Android`, `$prefix/lib/xbuild-frameworks` -- more closely resembled the Windows directory structure of `$MSBuild` and `$ReferenceAssemblies` (as seen in `.vsix` files). This would turn macOS/Linux into using `$xa_prefix/xbuild` and `$xa_prefix/xbuild-frameworks` directories. `$prefix/bin` would only contain `xabuild`. What is currently in `$prefix/lib/mandroid` would be merged with `$xa_prefix/xbuild/Xamarin/Android`. `$xa_prefix` would `$prefix/lib/xamarin.android`. This would turn the current macOS structure: $prefix/bin/xabuild $prefix/bin/generator $prefix/bin/cross-arm $prefix/lib/mandroid/generator.exe $prefix/lib/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xbuild/Xamarin/Android/Xamarin.Android.Common.targets Into: $prefix/bin/xabuild $prefix/lib/xamarin.android/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xamarin.android/xbuild/Xamarin/Android/generator.exe $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Xamarin.Android.Common.targets $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Darwin/cross-arm Other notes: * The `bundle-*.zip` filename has been has been changed to include a *hash* of the contents of various files, in particular `build-tools\mono-runtimes.*`. This was instigated via a conversation with @kumpera about making the filename more "idiot-proof": `mono-runtimes.props` contains *compiler flags*, and if any of those change, then *logically* the bundle should differ as well, as the mono runtimes may differ in significant ways. In theory, the `-vXX` version should be used to track this, but this is a manual change, easy to overlook. The new `-hHASH` part of the filename should be more automagic. The new `<HashFileContents/>` task in `xa-prep-tasks.dll` is responsible for creating the hash value. * `Configuration.Java.Interop.Override.props` was moved into `build-tools/scripts`, because that would cleanup the root directory a bit. * OS-specific binaries are now placed into `$prefix/lib/xamarin.android/xbuild/Xamarin/Android/$(HostOS)`. On a theoretical plus side, this means that the same build directory can contain OS-specific binaries for multiple operating systems. (I don't know if anyone shares a build tree between e.g. macOS and Linux, but if anyone *does*...) Unfortunately this requires a workaround for an `xbuild` bug: `%(_LlvmRuntime.InstallPath)` and `%(_MonoCrossRuntime.InstallPath)` *cannot* use MSBuild properties, e.g. `<InstallPath>$(HostOS)/</InstallPath>` doesn't work as desired; it's instead treated literally. Special-case `%(InstallPath)` until we fully migrate to MSBuild. * `$(MonoAndroidToolsDirectory)` should be considered *dead*, along with `$(MonoAndroidBinDirectory)`, as these should now *always* be the same directory as where `Xamarin.Android.Build.Tasks.dll` is located, or a `$(HostOS)` sub-directory. * `Xamarin.ProjectTools.csproj` needed to be updated to ensure that the build order was correct. * Remove all `[Obsolete]` and unreferenced members from `Xamarin.Android.Build.Utilities.dll`. There's too much in there, and it makes my head hurt trying to understand the interrelationships between it all. If it's not used, it's gone. * The changes to `src/monodroid/jni/Android.mk` are...weird. The removal of `-I$(topdir)/libmonodroid/zip`/etc. is to reduce use of non-existent paths, as `$(topdir)` isn't defined, so that's *actually* `-I/libmonodroid/zip`, which is nonsensical. So far, so good. What's *odd* is the addition of `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`: This is needed so that `external/mono/support/zlib-helper.c` exports `CreateZStream` and related symbols, otherwise we get a unit test failure in `GzipStreamTest.Compression` due to an `EntryPointNotFoundException`, because `CreateZStream` isn't exported/public. What's odd here is that I don't understand what caused this behavior to change. Previous builds exported `CreateZStream`, otherwise the tests would fail, and I don't understand how any of the other changes in this PR would be at fault, though that's certainly the most plausible explanation. Regardless, `-Ijni` *first* (due to adding `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`) is the desired behavior, so that `jni/config.h` is included, thus ensuring that `MONO_API` has the required definition when building `zlib-helper.c`.
2017-07-28 16:36:32 +03:00
<_BclFrameworkDir>$(XAInstallPrefix)xbuild-frameworks\MonoAndroid\v1.0\</_BclFrameworkDir>
<_MSBuildDir>$(XAInstallPrefix)xbuild\Xamarin\Android</_MSBuildDir>
<_OutputIncludeDir>..\..\bin\$(Configuration)\include\</_OutputIncludeDir>
[mono-runtimes] Fix bundle*.zip use (#510) Commit 76be1fb1 (6e9a6612) broke `bundle*.zip` use. Consequently, `bundle*.zip` *is not used*, resulting in *yuuuge* build times (nearly 7 hours for full builds on macOS, over 2 hours for PRs!). The `_BuildUnlessCached` target calls the `GetMonoBundleItems` target, to determine whether the `ForceBuild` target -- which invokes `_BuildRuntimes` and many other targets -- should be executed. The `GetMonoBundleItems` returns all files which would be included in `bundle*.zip`. The cause of the break is that commit 6e9a6612 changed the `GetMonoBundleItems` target to depend on the `_GetMonodocItems` target, which in turn depended on the `_BuildRuntimes` target. `_BuildRuntimes` is the target we want to avoid executing; it's the target that builds mono, and thus can take an eternity (particularly on the `xamarin-android` Jenkins lane, which builds **ALL THE ABIS**). Adding `_GetMonodocItems` in this fashion thus means we never use the cache, bceause in order to determine whether or not we can use the cache we need to do all the work that the cache was intended to avoid. Consider the [xamarin-android Build 312 log file][0] Building target "_BuildUnlessCached" in project ".../xamarin-android/build-tools/mono-runtimes/mono-runtimes.mdproj" (".../xamarin-android/build-tools/mono-runtimes/mono-runtimes.targets"); "Build" depends on it. ... Building target "GetMonoBundleItems" in project ".../xamarin-android/build-tools/mono-runtimes/mono-runtimes.mdproj" (".../xamarin-android/build-tools/mono-runtimes/mono-runtimes.targets"); "_BuildUnlessCached" depends on it. ... Building target "_GetMonodocItems" in project ".../xamarin-android/build-tools/mono-runtimes/mono-runtimes.mdproj" (".../xamarin-android/build-tools/mono-runtimes/mono-runtimes.targets"); "GetMonoBundleItems" depends on it. Building target "_BuildRuntimes" in project ".../xamarin-android/build-tools/mono-runtimes/mono-runtimes.mdproj" (".../xamarin-android/build-tools/mono-runtimes/mono-runtimes.targets"); "_GetMonodocItems" depends on it. ...and **BOOM**, the existence of `bundle*.zip` is meaningless. The fix is to partially revert commit 6e9a6612, but the entire reason 6e9a6612 exists is because commit 76be1fb1 changes the default file extension for debug symbols from `.mdb` to `.pdb`: How should we know if we're emitting `.mdb` or `.pdb` symbols? *A* way to know that is to actually build everything and see what is generated, but that's...slow. Slow and undesirable. (That's what 6e9a6612 does!) An alternate ("hacky") way to know is to rely on mono-specific extensions, and spackle, and duct tape: Mono's `msbuild` sets the `$(_DebugFileExt)` MSBuild property to the file extension of debug symbols, with values of `.mdb` on Mono 4.6 through Mono 4.8, and a value of `.pdb` on Mono 4.9. In `build-tools/scripts/msbuild.mk`, we can probe the mono version and attempt to come up with "sane" defaults, setting `$(_DebugFileExt)` for `xbuild` invocations done via `make`. We can thus rely on the `$(_DebugFileExt)` extension instead of costly filesystem probing, allowing `bundle*.zip` to actually be used, instead of ignored. ~~~ **A tale of two `pkg-config`s**: A funny thing happened while "fixing" `msbuild.mk` to set `$(_DebugFileExt)`: Turns Out™ there are (at least?) *two* `pkg-config` files on the Jenkins+macOS machine: 1. `/usr/local/bin/pkg-config` 2. `/Library/Frameworks/Mono.framework/Commands/pkg-config` (1) knows *nothing* about `Mono.framework`, and we're using the `mono` from `Mono.framework`, which distributes (2). Furthermore, we want to use `pkg-config --atleast-version=4.9 mono` in order to know if we're emitting `.pdb` or `.mdb` debug files, so using the correct version is Very Important. So, off I go to fix that so that we always use (2) on macOS, [and macOS+msbuild breaks][1]. Why's this matter? Recall 0eee6734, in which `$(_XAFixMSBuildFrameworkPathOverride)` is set only when (a) `msbuild` is used, and (b) mono prior to 4.8 is used? The check for (b) used the `pkg-config` from (1), meaning *the version check always failed*, meaning when building with `msbuild` from e.g. Mono 4.9, `$(_XAFixMSBuildFrameworkPathOverride)` was still being set, even though, as per 0eee6734, it shouldn't have been. Which means the explanation in 0eee6734 is wrong. Drain the swamp, at least a little bit: Remove `$(_XAFixMSBuildFrameworkPathOverride)`, and *always* set `$(FrameworkPathOverride)` within `MonoAndroidFramework.props`. It appears that attempting to make `$(FrameworkPathOverride)` conditional wasn't necessary, and only complicates matters. [0]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android/312/consoleText [1]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android-msbuild-pr-builder/243/
2017-03-22 17:50:14 +03:00
<_DebugFileExt Condition=" '$(_DebugFileExt)' == '' ">.pdb</_DebugFileExt>
[mono-runtimes] Build required mono runtimes. Add a new build-tools/mono-runtimes project to build the required Mono runtimes for Xamarin.Android app execution on Android devices. Xamarin.Android 6.0 ("cycle 6") includes Mono runtimes compiled for five architectures: * armeabi (armv5) * armeabi-v7a (armv7) * arm64-v8a (aarch64) * x86 * x86_64 An additional "host" architecture is also built to build the "monodroid" profile assemblies such as mscorlib.dll. In the interest of expediency and not requiring that *six* different runtimes be built -- which can greatly increase build times and storage requirements -- the initial xamarin-android open-source build will only build the armeabi-v7a (armv7) and "host" architectures. Additionally, the <UnzipDirectoryChildren/> task needed to be altered to use unzip(1) on OS X instead of using System.IO.Compression.ZipFile, because the ZipFile implementation included with Mono 4.4 doesn't respect file attributes such as the +x, causing all files to be extracted as 644 (rw-r--r--), resulting in an NDK toolchain which wouldn't work. To resolve this issue, execution on OS X and Linux now uses unzip(1). ~~ Adding new architectures ~~ To build Mono for a new architecture, edit build-tools/mono-runtimes/mono-runtimes.projitems and add a new @(_MonoRuntime) entry for the desired architecture. TODO: Figure out if there's a reasonable way to "opt-in" or "opt-out" of CPU architectures so that it isn't an "all or nothing" prospect.
2016-04-20 04:30:00 +03:00
</PropertyGroup>
[mdoc] Add mdoc and dependencies to the bundle Context: https://github.com/xamarin/xamarin-android/commit/5a8e48eba8cc8d05ec4fef25ac074d00606f5a69 `mdoc.exe` is used by the `<MDoc/>` task, used by the `BuildDocumentation` target in `Xamarin.Android.Bindings.targets`. We obtain `mdoc.exe` from the `external/mono` build, but because `mdoc.exe` will be referencing mono's `Mono.Cecil.dll`, we use `remap-assembly-ref` to update `mdoc.exe` to instead use `Xamarin.Android.Cecil.dll`.
2016-12-20 01:18:21 +03:00
<ItemGroup>
<MonoDocCopyItem Include="monodoc.dll" />
[mono-runtimes] Fix bundle*.zip use (#510) Commit 76be1fb1 (6e9a6612) broke `bundle*.zip` use. Consequently, `bundle*.zip` *is not used*, resulting in *yuuuge* build times (nearly 7 hours for full builds on macOS, over 2 hours for PRs!). The `_BuildUnlessCached` target calls the `GetMonoBundleItems` target, to determine whether the `ForceBuild` target -- which invokes `_BuildRuntimes` and many other targets -- should be executed. The `GetMonoBundleItems` returns all files which would be included in `bundle*.zip`. The cause of the break is that commit 6e9a6612 changed the `GetMonoBundleItems` target to depend on the `_GetMonodocItems` target, which in turn depended on the `_BuildRuntimes` target. `_BuildRuntimes` is the target we want to avoid executing; it's the target that builds mono, and thus can take an eternity (particularly on the `xamarin-android` Jenkins lane, which builds **ALL THE ABIS**). Adding `_GetMonodocItems` in this fashion thus means we never use the cache, bceause in order to determine whether or not we can use the cache we need to do all the work that the cache was intended to avoid. Consider the [xamarin-android Build 312 log file][0] Building target "_BuildUnlessCached" in project ".../xamarin-android/build-tools/mono-runtimes/mono-runtimes.mdproj" (".../xamarin-android/build-tools/mono-runtimes/mono-runtimes.targets"); "Build" depends on it. ... Building target "GetMonoBundleItems" in project ".../xamarin-android/build-tools/mono-runtimes/mono-runtimes.mdproj" (".../xamarin-android/build-tools/mono-runtimes/mono-runtimes.targets"); "_BuildUnlessCached" depends on it. ... Building target "_GetMonodocItems" in project ".../xamarin-android/build-tools/mono-runtimes/mono-runtimes.mdproj" (".../xamarin-android/build-tools/mono-runtimes/mono-runtimes.targets"); "GetMonoBundleItems" depends on it. Building target "_BuildRuntimes" in project ".../xamarin-android/build-tools/mono-runtimes/mono-runtimes.mdproj" (".../xamarin-android/build-tools/mono-runtimes/mono-runtimes.targets"); "_GetMonodocItems" depends on it. ...and **BOOM**, the existence of `bundle*.zip` is meaningless. The fix is to partially revert commit 6e9a6612, but the entire reason 6e9a6612 exists is because commit 76be1fb1 changes the default file extension for debug symbols from `.mdb` to `.pdb`: How should we know if we're emitting `.mdb` or `.pdb` symbols? *A* way to know that is to actually build everything and see what is generated, but that's...slow. Slow and undesirable. (That's what 6e9a6612 does!) An alternate ("hacky") way to know is to rely on mono-specific extensions, and spackle, and duct tape: Mono's `msbuild` sets the `$(_DebugFileExt)` MSBuild property to the file extension of debug symbols, with values of `.mdb` on Mono 4.6 through Mono 4.8, and a value of `.pdb` on Mono 4.9. In `build-tools/scripts/msbuild.mk`, we can probe the mono version and attempt to come up with "sane" defaults, setting `$(_DebugFileExt)` for `xbuild` invocations done via `make`. We can thus rely on the `$(_DebugFileExt)` extension instead of costly filesystem probing, allowing `bundle*.zip` to actually be used, instead of ignored. ~~~ **A tale of two `pkg-config`s**: A funny thing happened while "fixing" `msbuild.mk` to set `$(_DebugFileExt)`: Turns Out™ there are (at least?) *two* `pkg-config` files on the Jenkins+macOS machine: 1. `/usr/local/bin/pkg-config` 2. `/Library/Frameworks/Mono.framework/Commands/pkg-config` (1) knows *nothing* about `Mono.framework`, and we're using the `mono` from `Mono.framework`, which distributes (2). Furthermore, we want to use `pkg-config --atleast-version=4.9 mono` in order to know if we're emitting `.pdb` or `.mdb` debug files, so using the correct version is Very Important. So, off I go to fix that so that we always use (2) on macOS, [and macOS+msbuild breaks][1]. Why's this matter? Recall 0eee6734, in which `$(_XAFixMSBuildFrameworkPathOverride)` is set only when (a) `msbuild` is used, and (b) mono prior to 4.8 is used? The check for (b) used the `pkg-config` from (1), meaning *the version check always failed*, meaning when building with `msbuild` from e.g. Mono 4.9, `$(_XAFixMSBuildFrameworkPathOverride)` was still being set, even though, as per 0eee6734, it shouldn't have been. Which means the explanation in 0eee6734 is wrong. Drain the swamp, at least a little bit: Remove `$(_XAFixMSBuildFrameworkPathOverride)`, and *always* set `$(FrameworkPathOverride)` within `MonoAndroidFramework.props`. It appears that attempting to make `$(FrameworkPathOverride)` conditional wasn't necessary, and only complicates matters. [0]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android/312/consoleText [1]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android-msbuild-pr-builder/243/
2017-03-22 17:50:14 +03:00
<MonoDocCopyItem
Condition=" '$(_DebugFileExt)' == '.pdb'"
[bundle] Include mono's debug symbols (#513) Microsoft has (several?) [symbol servers][0], which is a repository "somewhere" which contains debug symbols, so that when a stack trace containing e.g. BCL stack frames is sent to Microsoft, there is a way to correlate that back to filename and line number source for further investigation. Xamarin.Android has not partaken in this mechanism, in large part because we did most of our building on Linux and macOS machines which generate `.mdb` debug symbols, while symbol servers require `.pdb`. That constraint is now behind us; with 76be1fb1 and the use of Mono 4.9 for builds, our builds now generate Portable `.pdb` files, which *should* (hopefully) be usable on Microsoft's symbol servers. However, to transmit them to Microsoft's servers, we need to actually *retain* them. At present, they're not: because of `bundle*.zip` use, the `mscorlib.dll` that we include in the final product may have been built much earlier than the rest of the product, and we don't have any other mechanism to retain the `mscorlib.pdb` file. Let's change that: update `mono-runtimes.targets` so that the created debug symbol files are included in the produced `bundle*.zip`, and bump the corresponding `$(XABundleFileName)` version number. This will ensure that we retain the debug symbols for our BCL assemblies, allowing us to actually have something to upload. [0]: https://msdn.microsoft.com/en-us/library/cc667410.aspx
2017-03-22 22:51:57 +03:00
Include="mdoc.pdb;monodoc.pdb"
[mono-runtimes] Fix bundle*.zip use (#510) Commit 76be1fb1 (6e9a6612) broke `bundle*.zip` use. Consequently, `bundle*.zip` *is not used*, resulting in *yuuuge* build times (nearly 7 hours for full builds on macOS, over 2 hours for PRs!). The `_BuildUnlessCached` target calls the `GetMonoBundleItems` target, to determine whether the `ForceBuild` target -- which invokes `_BuildRuntimes` and many other targets -- should be executed. The `GetMonoBundleItems` returns all files which would be included in `bundle*.zip`. The cause of the break is that commit 6e9a6612 changed the `GetMonoBundleItems` target to depend on the `_GetMonodocItems` target, which in turn depended on the `_BuildRuntimes` target. `_BuildRuntimes` is the target we want to avoid executing; it's the target that builds mono, and thus can take an eternity (particularly on the `xamarin-android` Jenkins lane, which builds **ALL THE ABIS**). Adding `_GetMonodocItems` in this fashion thus means we never use the cache, bceause in order to determine whether or not we can use the cache we need to do all the work that the cache was intended to avoid. Consider the [xamarin-android Build 312 log file][0] Building target "_BuildUnlessCached" in project ".../xamarin-android/build-tools/mono-runtimes/mono-runtimes.mdproj" (".../xamarin-android/build-tools/mono-runtimes/mono-runtimes.targets"); "Build" depends on it. ... Building target "GetMonoBundleItems" in project ".../xamarin-android/build-tools/mono-runtimes/mono-runtimes.mdproj" (".../xamarin-android/build-tools/mono-runtimes/mono-runtimes.targets"); "_BuildUnlessCached" depends on it. ... Building target "_GetMonodocItems" in project ".../xamarin-android/build-tools/mono-runtimes/mono-runtimes.mdproj" (".../xamarin-android/build-tools/mono-runtimes/mono-runtimes.targets"); "GetMonoBundleItems" depends on it. Building target "_BuildRuntimes" in project ".../xamarin-android/build-tools/mono-runtimes/mono-runtimes.mdproj" (".../xamarin-android/build-tools/mono-runtimes/mono-runtimes.targets"); "_GetMonodocItems" depends on it. ...and **BOOM**, the existence of `bundle*.zip` is meaningless. The fix is to partially revert commit 6e9a6612, but the entire reason 6e9a6612 exists is because commit 76be1fb1 changes the default file extension for debug symbols from `.mdb` to `.pdb`: How should we know if we're emitting `.mdb` or `.pdb` symbols? *A* way to know that is to actually build everything and see what is generated, but that's...slow. Slow and undesirable. (That's what 6e9a6612 does!) An alternate ("hacky") way to know is to rely on mono-specific extensions, and spackle, and duct tape: Mono's `msbuild` sets the `$(_DebugFileExt)` MSBuild property to the file extension of debug symbols, with values of `.mdb` on Mono 4.6 through Mono 4.8, and a value of `.pdb` on Mono 4.9. In `build-tools/scripts/msbuild.mk`, we can probe the mono version and attempt to come up with "sane" defaults, setting `$(_DebugFileExt)` for `xbuild` invocations done via `make`. We can thus rely on the `$(_DebugFileExt)` extension instead of costly filesystem probing, allowing `bundle*.zip` to actually be used, instead of ignored. ~~~ **A tale of two `pkg-config`s**: A funny thing happened while "fixing" `msbuild.mk` to set `$(_DebugFileExt)`: Turns Out™ there are (at least?) *two* `pkg-config` files on the Jenkins+macOS machine: 1. `/usr/local/bin/pkg-config` 2. `/Library/Frameworks/Mono.framework/Commands/pkg-config` (1) knows *nothing* about `Mono.framework`, and we're using the `mono` from `Mono.framework`, which distributes (2). Furthermore, we want to use `pkg-config --atleast-version=4.9 mono` in order to know if we're emitting `.pdb` or `.mdb` debug files, so using the correct version is Very Important. So, off I go to fix that so that we always use (2) on macOS, [and macOS+msbuild breaks][1]. Why's this matter? Recall 0eee6734, in which `$(_XAFixMSBuildFrameworkPathOverride)` is set only when (a) `msbuild` is used, and (b) mono prior to 4.8 is used? The check for (b) used the `pkg-config` from (1), meaning *the version check always failed*, meaning when building with `msbuild` from e.g. Mono 4.9, `$(_XAFixMSBuildFrameworkPathOverride)` was still being set, even though, as per 0eee6734, it shouldn't have been. Which means the explanation in 0eee6734 is wrong. Drain the swamp, at least a little bit: Remove `$(_XAFixMSBuildFrameworkPathOverride)`, and *always* set `$(FrameworkPathOverride)` within `MonoAndroidFramework.props`. It appears that attempting to make `$(FrameworkPathOverride)` conditional wasn't necessary, and only complicates matters. [0]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android/312/consoleText [1]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android-msbuild-pr-builder/243/
2017-03-22 17:50:14 +03:00
/>
<MonoDocCopyItem
Condition=" '$(_DebugFileExt)' == '.mdb'"
[bundle] Include mono's debug symbols (#513) Microsoft has (several?) [symbol servers][0], which is a repository "somewhere" which contains debug symbols, so that when a stack trace containing e.g. BCL stack frames is sent to Microsoft, there is a way to correlate that back to filename and line number source for further investigation. Xamarin.Android has not partaken in this mechanism, in large part because we did most of our building on Linux and macOS machines which generate `.mdb` debug symbols, while symbol servers require `.pdb`. That constraint is now behind us; with 76be1fb1 and the use of Mono 4.9 for builds, our builds now generate Portable `.pdb` files, which *should* (hopefully) be usable on Microsoft's symbol servers. However, to transmit them to Microsoft's servers, we need to actually *retain* them. At present, they're not: because of `bundle*.zip` use, the `mscorlib.dll` that we include in the final product may have been built much earlier than the rest of the product, and we don't have any other mechanism to retain the `mscorlib.pdb` file. Let's change that: update `mono-runtimes.targets` so that the created debug symbol files are included in the produced `bundle*.zip`, and bump the corresponding `$(XABundleFileName)` version number. This will ensure that we retain the debug symbols for our BCL assemblies, allowing us to actually have something to upload. [0]: https://msdn.microsoft.com/en-us/library/cc667410.aspx
2017-03-22 22:51:57 +03:00
Include="mdoc.exe.mdb;monodoc.dll.mdb"
[mono-runtimes] Fix bundle*.zip use (#510) Commit 76be1fb1 (6e9a6612) broke `bundle*.zip` use. Consequently, `bundle*.zip` *is not used*, resulting in *yuuuge* build times (nearly 7 hours for full builds on macOS, over 2 hours for PRs!). The `_BuildUnlessCached` target calls the `GetMonoBundleItems` target, to determine whether the `ForceBuild` target -- which invokes `_BuildRuntimes` and many other targets -- should be executed. The `GetMonoBundleItems` returns all files which would be included in `bundle*.zip`. The cause of the break is that commit 6e9a6612 changed the `GetMonoBundleItems` target to depend on the `_GetMonodocItems` target, which in turn depended on the `_BuildRuntimes` target. `_BuildRuntimes` is the target we want to avoid executing; it's the target that builds mono, and thus can take an eternity (particularly on the `xamarin-android` Jenkins lane, which builds **ALL THE ABIS**). Adding `_GetMonodocItems` in this fashion thus means we never use the cache, bceause in order to determine whether or not we can use the cache we need to do all the work that the cache was intended to avoid. Consider the [xamarin-android Build 312 log file][0] Building target "_BuildUnlessCached" in project ".../xamarin-android/build-tools/mono-runtimes/mono-runtimes.mdproj" (".../xamarin-android/build-tools/mono-runtimes/mono-runtimes.targets"); "Build" depends on it. ... Building target "GetMonoBundleItems" in project ".../xamarin-android/build-tools/mono-runtimes/mono-runtimes.mdproj" (".../xamarin-android/build-tools/mono-runtimes/mono-runtimes.targets"); "_BuildUnlessCached" depends on it. ... Building target "_GetMonodocItems" in project ".../xamarin-android/build-tools/mono-runtimes/mono-runtimes.mdproj" (".../xamarin-android/build-tools/mono-runtimes/mono-runtimes.targets"); "GetMonoBundleItems" depends on it. Building target "_BuildRuntimes" in project ".../xamarin-android/build-tools/mono-runtimes/mono-runtimes.mdproj" (".../xamarin-android/build-tools/mono-runtimes/mono-runtimes.targets"); "_GetMonodocItems" depends on it. ...and **BOOM**, the existence of `bundle*.zip` is meaningless. The fix is to partially revert commit 6e9a6612, but the entire reason 6e9a6612 exists is because commit 76be1fb1 changes the default file extension for debug symbols from `.mdb` to `.pdb`: How should we know if we're emitting `.mdb` or `.pdb` symbols? *A* way to know that is to actually build everything and see what is generated, but that's...slow. Slow and undesirable. (That's what 6e9a6612 does!) An alternate ("hacky") way to know is to rely on mono-specific extensions, and spackle, and duct tape: Mono's `msbuild` sets the `$(_DebugFileExt)` MSBuild property to the file extension of debug symbols, with values of `.mdb` on Mono 4.6 through Mono 4.8, and a value of `.pdb` on Mono 4.9. In `build-tools/scripts/msbuild.mk`, we can probe the mono version and attempt to come up with "sane" defaults, setting `$(_DebugFileExt)` for `xbuild` invocations done via `make`. We can thus rely on the `$(_DebugFileExt)` extension instead of costly filesystem probing, allowing `bundle*.zip` to actually be used, instead of ignored. ~~~ **A tale of two `pkg-config`s**: A funny thing happened while "fixing" `msbuild.mk` to set `$(_DebugFileExt)`: Turns Out™ there are (at least?) *two* `pkg-config` files on the Jenkins+macOS machine: 1. `/usr/local/bin/pkg-config` 2. `/Library/Frameworks/Mono.framework/Commands/pkg-config` (1) knows *nothing* about `Mono.framework`, and we're using the `mono` from `Mono.framework`, which distributes (2). Furthermore, we want to use `pkg-config --atleast-version=4.9 mono` in order to know if we're emitting `.pdb` or `.mdb` debug files, so using the correct version is Very Important. So, off I go to fix that so that we always use (2) on macOS, [and macOS+msbuild breaks][1]. Why's this matter? Recall 0eee6734, in which `$(_XAFixMSBuildFrameworkPathOverride)` is set only when (a) `msbuild` is used, and (b) mono prior to 4.8 is used? The check for (b) used the `pkg-config` from (1), meaning *the version check always failed*, meaning when building with `msbuild` from e.g. Mono 4.9, `$(_XAFixMSBuildFrameworkPathOverride)` was still being set, even though, as per 0eee6734, it shouldn't have been. Which means the explanation in 0eee6734 is wrong. Drain the swamp, at least a little bit: Remove `$(_XAFixMSBuildFrameworkPathOverride)`, and *always* set `$(FrameworkPathOverride)` within `MonoAndroidFramework.props`. It appears that attempting to make `$(FrameworkPathOverride)` conditional wasn't necessary, and only complicates matters. [0]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android/312/consoleText [1]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android-msbuild-pr-builder/243/
2017-03-22 17:50:14 +03:00
/>
[mdoc] Add mdoc and dependencies to the bundle Context: https://github.com/xamarin/xamarin-android/commit/5a8e48eba8cc8d05ec4fef25ac074d00606f5a69 `mdoc.exe` is used by the `<MDoc/>` task, used by the `BuildDocumentation` target in `Xamarin.Android.Bindings.targets`. We obtain `mdoc.exe` from the `external/mono` build, but because `mdoc.exe` will be referencing mono's `Mono.Cecil.dll`, we use `remap-assembly-ref` to update `mdoc.exe` to instead use `Xamarin.Android.Cecil.dll`.
2016-12-20 01:18:21 +03:00
<MonoDocCopyItem Include="monodoc.dll.config" />
</ItemGroup>
Copy FSharp support assemblies from nuget (#21) Instead of building F# from sources as we've been doing so far, we'll now use a nuget to get the binaries we require. This makes the build time much shorter.
2016-05-06 00:07:05 +03:00
<UsingTask AssemblyFile="$(_SourceTopDir)\bin\Build$(Configuration)\Xamarin.Android.Tools.BootstrapTasks.dll" TaskName="Xamarin.Android.Tools.BootstrapTasks.GetNugetPackageBasePath" />
[mono-runtimes] Fix bundle*.zip use (#510) Commit 76be1fb1 (6e9a6612) broke `bundle*.zip` use. Consequently, `bundle*.zip` *is not used*, resulting in *yuuuge* build times (nearly 7 hours for full builds on macOS, over 2 hours for PRs!). The `_BuildUnlessCached` target calls the `GetMonoBundleItems` target, to determine whether the `ForceBuild` target -- which invokes `_BuildRuntimes` and many other targets -- should be executed. The `GetMonoBundleItems` returns all files which would be included in `bundle*.zip`. The cause of the break is that commit 6e9a6612 changed the `GetMonoBundleItems` target to depend on the `_GetMonodocItems` target, which in turn depended on the `_BuildRuntimes` target. `_BuildRuntimes` is the target we want to avoid executing; it's the target that builds mono, and thus can take an eternity (particularly on the `xamarin-android` Jenkins lane, which builds **ALL THE ABIS**). Adding `_GetMonodocItems` in this fashion thus means we never use the cache, bceause in order to determine whether or not we can use the cache we need to do all the work that the cache was intended to avoid. Consider the [xamarin-android Build 312 log file][0] Building target "_BuildUnlessCached" in project ".../xamarin-android/build-tools/mono-runtimes/mono-runtimes.mdproj" (".../xamarin-android/build-tools/mono-runtimes/mono-runtimes.targets"); "Build" depends on it. ... Building target "GetMonoBundleItems" in project ".../xamarin-android/build-tools/mono-runtimes/mono-runtimes.mdproj" (".../xamarin-android/build-tools/mono-runtimes/mono-runtimes.targets"); "_BuildUnlessCached" depends on it. ... Building target "_GetMonodocItems" in project ".../xamarin-android/build-tools/mono-runtimes/mono-runtimes.mdproj" (".../xamarin-android/build-tools/mono-runtimes/mono-runtimes.targets"); "GetMonoBundleItems" depends on it. Building target "_BuildRuntimes" in project ".../xamarin-android/build-tools/mono-runtimes/mono-runtimes.mdproj" (".../xamarin-android/build-tools/mono-runtimes/mono-runtimes.targets"); "_GetMonodocItems" depends on it. ...and **BOOM**, the existence of `bundle*.zip` is meaningless. The fix is to partially revert commit 6e9a6612, but the entire reason 6e9a6612 exists is because commit 76be1fb1 changes the default file extension for debug symbols from `.mdb` to `.pdb`: How should we know if we're emitting `.mdb` or `.pdb` symbols? *A* way to know that is to actually build everything and see what is generated, but that's...slow. Slow and undesirable. (That's what 6e9a6612 does!) An alternate ("hacky") way to know is to rely on mono-specific extensions, and spackle, and duct tape: Mono's `msbuild` sets the `$(_DebugFileExt)` MSBuild property to the file extension of debug symbols, with values of `.mdb` on Mono 4.6 through Mono 4.8, and a value of `.pdb` on Mono 4.9. In `build-tools/scripts/msbuild.mk`, we can probe the mono version and attempt to come up with "sane" defaults, setting `$(_DebugFileExt)` for `xbuild` invocations done via `make`. We can thus rely on the `$(_DebugFileExt)` extension instead of costly filesystem probing, allowing `bundle*.zip` to actually be used, instead of ignored. ~~~ **A tale of two `pkg-config`s**: A funny thing happened while "fixing" `msbuild.mk` to set `$(_DebugFileExt)`: Turns Out™ there are (at least?) *two* `pkg-config` files on the Jenkins+macOS machine: 1. `/usr/local/bin/pkg-config` 2. `/Library/Frameworks/Mono.framework/Commands/pkg-config` (1) knows *nothing* about `Mono.framework`, and we're using the `mono` from `Mono.framework`, which distributes (2). Furthermore, we want to use `pkg-config --atleast-version=4.9 mono` in order to know if we're emitting `.pdb` or `.mdb` debug files, so using the correct version is Very Important. So, off I go to fix that so that we always use (2) on macOS, [and macOS+msbuild breaks][1]. Why's this matter? Recall 0eee6734, in which `$(_XAFixMSBuildFrameworkPathOverride)` is set only when (a) `msbuild` is used, and (b) mono prior to 4.8 is used? The check for (b) used the `pkg-config` from (1), meaning *the version check always failed*, meaning when building with `msbuild` from e.g. Mono 4.9, `$(_XAFixMSBuildFrameworkPathOverride)` was still being set, even though, as per 0eee6734, it shouldn't have been. Which means the explanation in 0eee6734 is wrong. Drain the swamp, at least a little bit: Remove `$(_XAFixMSBuildFrameworkPathOverride)`, and *always* set `$(FrameworkPathOverride)` within `MonoAndroidFramework.props`. It appears that attempting to make `$(FrameworkPathOverride)` conditional wasn't necessary, and only complicates matters. [0]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android/312/consoleText [1]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android-msbuild-pr-builder/243/
2017-03-22 17:50:14 +03:00
<ItemGroup>
<_MonoDocCopyItems Include="@(MonoDocCopyItem->'$(_MonoOutputDir)\%(Identity)')" />
</ItemGroup>
<ItemGroup>
[structure] Rework installation directory structure (#704) Context: https://github.com/xamarin/xamarin-android/pull/253#discussion_r82862993 The *intention* is that Jenkins-produced `oss-xamarin.android*.zip` artifacts be usable on Windows, so that side-by-side testing can be performed without replacing the system installation. Usage is in [UsingJenkinsBuildArtifacts.md](Documentation/UsingJenkinsBuildArtifacts). This isn't *entirely* the case. It was *apparently* the case at the time of commit 87ca2737, but things have bitrotten since. For example, following the 87ca2737 instructions would currently result in an XA0020 `Could not find monodroid` error, because `class-parse.exe` wasn't where Windows expects it to be (it was in `lib/mandroid`, not `lib/xbuild/Xamarin/Android`). This needs to be fixed. Additionally, PR #253 mentions that, for filesystem organization, it would be useful if the macOS/Linux directory structure -- `$prefix/bin`, `$prefix/lib/mandroid`, `$prefix/lib/xbuild/Xamarin/Android`, `$prefix/lib/xbuild-frameworks` -- more closely resembled the Windows directory structure of `$MSBuild` and `$ReferenceAssemblies` (as seen in `.vsix` files). This would turn macOS/Linux into using `$xa_prefix/xbuild` and `$xa_prefix/xbuild-frameworks` directories. `$prefix/bin` would only contain `xabuild`. What is currently in `$prefix/lib/mandroid` would be merged with `$xa_prefix/xbuild/Xamarin/Android`. `$xa_prefix` would `$prefix/lib/xamarin.android`. This would turn the current macOS structure: $prefix/bin/xabuild $prefix/bin/generator $prefix/bin/cross-arm $prefix/lib/mandroid/generator.exe $prefix/lib/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xbuild/Xamarin/Android/Xamarin.Android.Common.targets Into: $prefix/bin/xabuild $prefix/lib/xamarin.android/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xamarin.android/xbuild/Xamarin/Android/generator.exe $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Xamarin.Android.Common.targets $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Darwin/cross-arm Other notes: * The `bundle-*.zip` filename has been has been changed to include a *hash* of the contents of various files, in particular `build-tools\mono-runtimes.*`. This was instigated via a conversation with @kumpera about making the filename more "idiot-proof": `mono-runtimes.props` contains *compiler flags*, and if any of those change, then *logically* the bundle should differ as well, as the mono runtimes may differ in significant ways. In theory, the `-vXX` version should be used to track this, but this is a manual change, easy to overlook. The new `-hHASH` part of the filename should be more automagic. The new `<HashFileContents/>` task in `xa-prep-tasks.dll` is responsible for creating the hash value. * `Configuration.Java.Interop.Override.props` was moved into `build-tools/scripts`, because that would cleanup the root directory a bit. * OS-specific binaries are now placed into `$prefix/lib/xamarin.android/xbuild/Xamarin/Android/$(HostOS)`. On a theoretical plus side, this means that the same build directory can contain OS-specific binaries for multiple operating systems. (I don't know if anyone shares a build tree between e.g. macOS and Linux, but if anyone *does*...) Unfortunately this requires a workaround for an `xbuild` bug: `%(_LlvmRuntime.InstallPath)` and `%(_MonoCrossRuntime.InstallPath)` *cannot* use MSBuild properties, e.g. `<InstallPath>$(HostOS)/</InstallPath>` doesn't work as desired; it's instead treated literally. Special-case `%(InstallPath)` until we fully migrate to MSBuild. * `$(MonoAndroidToolsDirectory)` should be considered *dead*, along with `$(MonoAndroidBinDirectory)`, as these should now *always* be the same directory as where `Xamarin.Android.Build.Tasks.dll` is located, or a `$(HostOS)` sub-directory. * `Xamarin.ProjectTools.csproj` needed to be updated to ensure that the build order was correct. * Remove all `[Obsolete]` and unreferenced members from `Xamarin.Android.Build.Utilities.dll`. There's too much in there, and it makes my head hurt trying to understand the interrelationships between it all. If it's not used, it's gone. * The changes to `src/monodroid/jni/Android.mk` are...weird. The removal of `-I$(topdir)/libmonodroid/zip`/etc. is to reduce use of non-existent paths, as `$(topdir)` isn't defined, so that's *actually* `-I/libmonodroid/zip`, which is nonsensical. So far, so good. What's *odd* is the addition of `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`: This is needed so that `external/mono/support/zlib-helper.c` exports `CreateZStream` and related symbols, otherwise we get a unit test failure in `GzipStreamTest.Compression` due to an `EntryPointNotFoundException`, because `CreateZStream` isn't exported/public. What's odd here is that I don't understand what caused this behavior to change. Previous builds exported `CreateZStream`, otherwise the tests would fail, and I don't understand how any of the other changes in this PR would be at fault, though that's certainly the most plausible explanation. Regardless, `-Ijni` *first* (due to adding `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`) is the desired behavior, so that `jni/config.h` is included, thus ensuring that `MONO_API` has the required definition when building `zlib-helper.c`.
2017-07-28 16:36:32 +03:00
<_MonoDocInstalledItems Include="@(MonoDocCopyItem->'$(_MSBuildDir)\%(Identity)')" />
<_MonoDocInstalledItems Include="$(_MSBuildDir)\mdoc.exe" />
[mono-runtimes] Fix bundle*.zip use (#510) Commit 76be1fb1 (6e9a6612) broke `bundle*.zip` use. Consequently, `bundle*.zip` *is not used*, resulting in *yuuuge* build times (nearly 7 hours for full builds on macOS, over 2 hours for PRs!). The `_BuildUnlessCached` target calls the `GetMonoBundleItems` target, to determine whether the `ForceBuild` target -- which invokes `_BuildRuntimes` and many other targets -- should be executed. The `GetMonoBundleItems` returns all files which would be included in `bundle*.zip`. The cause of the break is that commit 6e9a6612 changed the `GetMonoBundleItems` target to depend on the `_GetMonodocItems` target, which in turn depended on the `_BuildRuntimes` target. `_BuildRuntimes` is the target we want to avoid executing; it's the target that builds mono, and thus can take an eternity (particularly on the `xamarin-android` Jenkins lane, which builds **ALL THE ABIS**). Adding `_GetMonodocItems` in this fashion thus means we never use the cache, bceause in order to determine whether or not we can use the cache we need to do all the work that the cache was intended to avoid. Consider the [xamarin-android Build 312 log file][0] Building target "_BuildUnlessCached" in project ".../xamarin-android/build-tools/mono-runtimes/mono-runtimes.mdproj" (".../xamarin-android/build-tools/mono-runtimes/mono-runtimes.targets"); "Build" depends on it. ... Building target "GetMonoBundleItems" in project ".../xamarin-android/build-tools/mono-runtimes/mono-runtimes.mdproj" (".../xamarin-android/build-tools/mono-runtimes/mono-runtimes.targets"); "_BuildUnlessCached" depends on it. ... Building target "_GetMonodocItems" in project ".../xamarin-android/build-tools/mono-runtimes/mono-runtimes.mdproj" (".../xamarin-android/build-tools/mono-runtimes/mono-runtimes.targets"); "GetMonoBundleItems" depends on it. Building target "_BuildRuntimes" in project ".../xamarin-android/build-tools/mono-runtimes/mono-runtimes.mdproj" (".../xamarin-android/build-tools/mono-runtimes/mono-runtimes.targets"); "_GetMonodocItems" depends on it. ...and **BOOM**, the existence of `bundle*.zip` is meaningless. The fix is to partially revert commit 6e9a6612, but the entire reason 6e9a6612 exists is because commit 76be1fb1 changes the default file extension for debug symbols from `.mdb` to `.pdb`: How should we know if we're emitting `.mdb` or `.pdb` symbols? *A* way to know that is to actually build everything and see what is generated, but that's...slow. Slow and undesirable. (That's what 6e9a6612 does!) An alternate ("hacky") way to know is to rely on mono-specific extensions, and spackle, and duct tape: Mono's `msbuild` sets the `$(_DebugFileExt)` MSBuild property to the file extension of debug symbols, with values of `.mdb` on Mono 4.6 through Mono 4.8, and a value of `.pdb` on Mono 4.9. In `build-tools/scripts/msbuild.mk`, we can probe the mono version and attempt to come up with "sane" defaults, setting `$(_DebugFileExt)` for `xbuild` invocations done via `make`. We can thus rely on the `$(_DebugFileExt)` extension instead of costly filesystem probing, allowing `bundle*.zip` to actually be used, instead of ignored. ~~~ **A tale of two `pkg-config`s**: A funny thing happened while "fixing" `msbuild.mk` to set `$(_DebugFileExt)`: Turns Out™ there are (at least?) *two* `pkg-config` files on the Jenkins+macOS machine: 1. `/usr/local/bin/pkg-config` 2. `/Library/Frameworks/Mono.framework/Commands/pkg-config` (1) knows *nothing* about `Mono.framework`, and we're using the `mono` from `Mono.framework`, which distributes (2). Furthermore, we want to use `pkg-config --atleast-version=4.9 mono` in order to know if we're emitting `.pdb` or `.mdb` debug files, so using the correct version is Very Important. So, off I go to fix that so that we always use (2) on macOS, [and macOS+msbuild breaks][1]. Why's this matter? Recall 0eee6734, in which `$(_XAFixMSBuildFrameworkPathOverride)` is set only when (a) `msbuild` is used, and (b) mono prior to 4.8 is used? The check for (b) used the `pkg-config` from (1), meaning *the version check always failed*, meaning when building with `msbuild` from e.g. Mono 4.9, `$(_XAFixMSBuildFrameworkPathOverride)` was still being set, even though, as per 0eee6734, it shouldn't have been. Which means the explanation in 0eee6734 is wrong. Drain the swamp, at least a little bit: Remove `$(_XAFixMSBuildFrameworkPathOverride)`, and *always* set `$(FrameworkPathOverride)` within `MonoAndroidFramework.props`. It appears that attempting to make `$(FrameworkPathOverride)` conditional wasn't necessary, and only complicates matters. [0]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android/312/consoleText [1]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android-msbuild-pr-builder/243/
2017-03-22 17:50:14 +03:00
</ItemGroup>
[build] Add `make jenkins` target. (#116) The `make jenkins` [^0] target is for use by Continuous Integration machines, to build *everything* [^1]. This is expected to take an eternity. Think *hours*. $ time make jenkins ... real 130m11.608s user 97m22.220s sys 18m20.522s Of particular note is that the above "everything" includes *Release configuration builds* of everything, which is something that didn't actually work before. (Oops.) Bump Java.Interop so that it supports building the Release configuration, update Xamarin.Android.sln so that all required projects are part of the Release configuration, and update Xamarin.Android.Build.Tasks.csproj so that `JCW_ONLY_TYPE_NAMES` isn't defined, as this was preventing compilation. Fix **strip**(1) use: `mono-runtimes.targets` was trying to use `strip -S` on macOS, but the value of `%(_MonoRuntime.Strip)` was quoted, and thus attempted to execute `"strip -S" ...`, which failed. Move the `-S` into a new `%(_MonoRuntime.StripFlags)` value. Fixup `mono-runtimes.targets` and related files so that `$(MonoSourceFullPath)` is used instead of a relative path. This helps alleviate the "mental math" of determining the relative path to the Mono checkout. Plus, the Mono checkout is supposed to be overridable, e.g. commit d205cab2, and using `$(MonoSourceFullPath)` supports that. Download and install `android.jar` for all supported API levels. Fix the `Mono.Android.csproj` build so that `Mono.Android.dll` is stored in a per-API-level intermediate directory. Otherwise, if e.g. API-10 is built after API-23, the API-23 version will be installed, but the JCW build will fail. Additionally, API-24 requires using `javac -source 1.8 -target 1.8`, not 1.6. Fix `Mono.Android/metadata` to use the correct `merge.SourceFile` filename of `Profiles/api-24.xml.in`. Without that fix, API-24 won't build because `NumericShaper.GetContextualShaper()` is emitted twice, and the C# compiler doesn't like that. Disable use of `-lz` when building for Windows. Windows doesn't contain a `Z.DLL` to link against. [^0]: https://en.wikipedia.org/wiki/Leeroy_Jenkins [^1]: https://www.youtube.com/watch?v=hooKVstzbz0
2016-07-18 15:42:39 +03:00
<PropertyGroup>
<_MonoProfileDir>$(MonoSourceFullPath)\mcs\class\lib\monodroid</_MonoProfileDir>
Bump to mono/2017-10/7e85cfa5 (#943) Bump to xamarin-android-api-compatibility/master/c050ba3c. Updates expected APIs for BCL assemblies. Use `azureedge.net` instead of `blob.core.windows.net`. It's behind a CDN and way faster outside US. Use the new profiler init symbol name, `mono_profiler_init()`. Avoid dumping the content of `libmono-profiler-log.so` when tests fail. The binary contents aren't meaningful, and needlessly bloat the output log. Fix `eglib` path, as mono changed it's location. Don't skip errors from mono's `make test` target anymore. Stop building Mono's `net_4_x` profile. It isn't required to build the `monodroid` profile and *not* building it saves lots of time. The tools needed by XA are taken from the new `monodroid_tools` profile instead. Update Mono version phrasing in `README.md`. Using an explicit version is just a recipe for outdated info, instead rewrite it to be generic since the build will complain on an outdated Mono anyway. Add more exclusions to `LinkerDescription.xml` for BCL tests, which are used in the new `System.Reflection.Emit.SaveTest()`. Update `.gitignore` for some paths. `Xamarin.Android.Common.props` is generated as part of the build, and the `.rawproto` files created by gradle should be ignored. Context: https://bugzilla.xamarin.com/show_bug.cgi?id=4786 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=5798 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=5804 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=11696 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=17325 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=23153 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=25755 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=36053 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=36634 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=40556 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=41279 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=43172 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=43563 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=44266 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=46683 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=52157 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=52621 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=55603 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=56653 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=56684 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57069 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57160 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57197 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57222 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57232 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57301 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57476 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57488 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57496 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57505 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57528 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57629 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57691 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57744 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57796 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57850 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57851 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57887 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57918 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57930 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57936 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57959 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58100 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58168 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58210 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58291 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58317 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58344 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58361 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58379 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58399 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58411 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58421 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58423 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58454 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58612 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58621 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58728 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58738 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58770 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58782 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58793 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58809 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58829 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58877 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58901 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58911 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58965 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=59039 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=59154 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=59180 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=59235 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=59281 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=59364 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=59436 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=59608 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=59664 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=59824 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=60233
2017-10-31 18:55:44 +03:00
<_MonoOutputDir>$(MonoSourceFullPath)\mcs\class\lib\monodroid_tools</_MonoOutputDir>
[build] Add `make jenkins` target. (#116) The `make jenkins` [^0] target is for use by Continuous Integration machines, to build *everything* [^1]. This is expected to take an eternity. Think *hours*. $ time make jenkins ... real 130m11.608s user 97m22.220s sys 18m20.522s Of particular note is that the above "everything" includes *Release configuration builds* of everything, which is something that didn't actually work before. (Oops.) Bump Java.Interop so that it supports building the Release configuration, update Xamarin.Android.sln so that all required projects are part of the Release configuration, and update Xamarin.Android.Build.Tasks.csproj so that `JCW_ONLY_TYPE_NAMES` isn't defined, as this was preventing compilation. Fix **strip**(1) use: `mono-runtimes.targets` was trying to use `strip -S` on macOS, but the value of `%(_MonoRuntime.Strip)` was quoted, and thus attempted to execute `"strip -S" ...`, which failed. Move the `-S` into a new `%(_MonoRuntime.StripFlags)` value. Fixup `mono-runtimes.targets` and related files so that `$(MonoSourceFullPath)` is used instead of a relative path. This helps alleviate the "mental math" of determining the relative path to the Mono checkout. Plus, the Mono checkout is supposed to be overridable, e.g. commit d205cab2, and using `$(MonoSourceFullPath)` supports that. Download and install `android.jar` for all supported API levels. Fix the `Mono.Android.csproj` build so that `Mono.Android.dll` is stored in a per-API-level intermediate directory. Otherwise, if e.g. API-10 is built after API-23, the API-23 version will be installed, but the JCW build will fail. Additionally, API-24 requires using `javac -source 1.8 -target 1.8`, not 1.6. Fix `Mono.Android/metadata` to use the correct `merge.SourceFile` filename of `Profiles/api-24.xml.in`. Without that fix, API-24 won't build because `NumericShaper.GetContextualShaper()` is emitted twice, and the C# compiler doesn't like that. Disable use of `-lz` when building for Windows. Windows doesn't contain a `Z.DLL` to link against. [^0]: https://en.wikipedia.org/wiki/Leeroy_Jenkins [^1]: https://www.youtube.com/watch?v=hooKVstzbz0
2016-07-18 15:42:39 +03:00
</PropertyGroup>
[xa-prep-tasks] Use the Mono bundle (#162) Context: https://github.com/xamarin/xamarin-android/commit/fbfd676c102c63e4e06e750857b178725e33450c Stage 3 of the cunning plan is to (attempt to) use the mono bundle introduced in commit fbfd676c. This "simple" desire (ha!) re-raises the architectural project dependency issue "solved" in fbfd676c, but first, a simple question: What should download the mono bundle? There are two plausible answers: 1. `make prepare` can (somehow) handle it. 2. MSBuild can (somehow) handle it. Both are plausible. The problem with using `make` targets (1) is there is increased potential for "duplication" -- duplication of the bundle filename, downloading it, and extracting it. Plus, `make` isn't "Windows friendly", in that GNU make isn't (normally) present with Visual Studio. (`NMAKE` is, but the Makefiles in this project are not compatible with `NMAKE`.) Which brings us to MSBuild (2): can it handle the task? To tackle that, we need to be able to have an MSBuild task project which has *no dependencies*, so that it can download and extract the mono bundle *before anything else runs*, as it may be downloading contents which mean that other projects don't *need* to run. The need for a "pre-bootstrap" task assembly -- called `xa-prep-tasks` -- thus "undoes" *some* of the logic regarding `libzip-windows.mdproj` and the `<Zip/>` task from fbfd676c: it isn't *possible* to rely on `libzip` from a "pre-build" state, as `libzip` is one of the things in the mono bundle, so now we need *two* "bootstrap" task assemblies: one without a `libzip` dependency -- `xa-prep-tasks.dll` -- and one *with* a `libzip` dependency -- `Xamarin.Android.Tools.BootstrapTasks.dll` Move tasks which don't currently require `libzip` -- or won't in the future, or laziness -- from `Xamarin.Android.Tools.BootstrapTasks.dll` and move them into `xa-prep-tasks.dll`. With that architectural compromise in place, add `xa-prep-tasks` as a `@(ProjectReference)` to various projects to help ensure it's built *first*, and rearchitect `bundle.mdproj` so that `xa-prep-tasks.targets` and `bundle.targets` can use the same targets to compute the bundle filename, now in `build-tools/bundle/bundle-path.targets`. Add a post-build step to `xa-prep-tasks.csproj` which downloads and extracts the expected mono bundle. One "problem" (feature?) is that the new `<SystemUnzip/>` task doesn't report errors as errors when unzip'ing the file. This turns out to be fine here because when downloading the mono bundle from Azure we don't get a 404 *anyway* -- Azure instead returns an XML document containing an error message (wat?!). We can thus ignore most error handling entirely...though we're *also* ignoring any checking for invalid downloads, which is something we should address in the future. Update the varioous project files so that they won't attempt to rebuild binaries that were present in the mono bundle.
2016-08-16 23:02:48 +03:00
<PropertyGroup>
<ForceBuildDependsOn>
_BuildLlvm;
_InstallLlvm;
_Autogen;
_ConfigureRuntimes;
_BuildRuntimes;
_InstallRuntimes;
_InstallBcl;
[mono-runtimes] Install $prefix/lib/mandroid/cil-strip.exe `cil-strip.exe` is used by the `<CilStrip/>` task in `Xamarin.Android.Build.Tasks.dll`, and is just a renamed copy of `mono-cil-strip.exe`. Additionally, update the `_InstallMonoDoc` target to always `<Touch/>` the created files. This is to keep the `_InstallMonoDoc` task from constantly re-executing for me when running `xbuild /t:ForceBuild`: Target _InstallMonoDoc needs to be built as input file '.../xamarin-android/external/mono/mcs/class/lib/net_4_x/mdoc.exe' is newer than output file '../../bin/Debug//lib/mandroid/monodoc.dll.config'
2017-03-04 01:10:15 +03:00
_InstallCilStrip;
[mdoc] Add mdoc and dependencies to the bundle Context: https://github.com/xamarin/xamarin-android/commit/5a8e48eba8cc8d05ec4fef25ac074d00606f5a69 `mdoc.exe` is used by the `<MDoc/>` task, used by the `BuildDocumentation` target in `Xamarin.Android.Bindings.targets`. We obtain `mdoc.exe` from the `external/mono` build, but because `mdoc.exe` will be referencing mono's `Mono.Cecil.dll`, we use `remap-assembly-ref` to update `mdoc.exe` to instead use `Xamarin.Android.Cecil.dll`.
2016-12-20 01:18:21 +03:00
_InstallMonoDoc;
[bundle] Include mono's debug symbols (#513) Microsoft has (several?) [symbol servers][0], which is a repository "somewhere" which contains debug symbols, so that when a stack trace containing e.g. BCL stack frames is sent to Microsoft, there is a way to correlate that back to filename and line number source for further investigation. Xamarin.Android has not partaken in this mechanism, in large part because we did most of our building on Linux and macOS machines which generate `.mdb` debug symbols, while symbol servers require `.pdb`. That constraint is now behind us; with 76be1fb1 and the use of Mono 4.9 for builds, our builds now generate Portable `.pdb` files, which *should* (hopefully) be usable on Microsoft's symbol servers. However, to transmit them to Microsoft's servers, we need to actually *retain* them. At present, they're not: because of `bundle*.zip` use, the `mscorlib.dll` that we include in the final product may have been built much earlier than the rest of the product, and we don't have any other mechanism to retain the `mscorlib.pdb` file. Let's change that: update `mono-runtimes.targets` so that the created debug symbol files are included in the produced `bundle*.zip`, and bump the corresponding `$(XABundleFileName)` version number. This will ensure that we retain the debug symbols for our BCL assemblies, allowing us to actually have something to upload. [0]: https://msdn.microsoft.com/en-us/library/cc667410.aspx
2017-03-22 22:51:57 +03:00
_InstallMonoUtilities;
[xa-prep-tasks] Use the Mono bundle (#162) Context: https://github.com/xamarin/xamarin-android/commit/fbfd676c102c63e4e06e750857b178725e33450c Stage 3 of the cunning plan is to (attempt to) use the mono bundle introduced in commit fbfd676c. This "simple" desire (ha!) re-raises the architectural project dependency issue "solved" in fbfd676c, but first, a simple question: What should download the mono bundle? There are two plausible answers: 1. `make prepare` can (somehow) handle it. 2. MSBuild can (somehow) handle it. Both are plausible. The problem with using `make` targets (1) is there is increased potential for "duplication" -- duplication of the bundle filename, downloading it, and extracting it. Plus, `make` isn't "Windows friendly", in that GNU make isn't (normally) present with Visual Studio. (`NMAKE` is, but the Makefiles in this project are not compatible with `NMAKE`.) Which brings us to MSBuild (2): can it handle the task? To tackle that, we need to be able to have an MSBuild task project which has *no dependencies*, so that it can download and extract the mono bundle *before anything else runs*, as it may be downloading contents which mean that other projects don't *need* to run. The need for a "pre-bootstrap" task assembly -- called `xa-prep-tasks` -- thus "undoes" *some* of the logic regarding `libzip-windows.mdproj` and the `<Zip/>` task from fbfd676c: it isn't *possible* to rely on `libzip` from a "pre-build" state, as `libzip` is one of the things in the mono bundle, so now we need *two* "bootstrap" task assemblies: one without a `libzip` dependency -- `xa-prep-tasks.dll` -- and one *with* a `libzip` dependency -- `Xamarin.Android.Tools.BootstrapTasks.dll` Move tasks which don't currently require `libzip` -- or won't in the future, or laziness -- from `Xamarin.Android.Tools.BootstrapTasks.dll` and move them into `xa-prep-tasks.dll`. With that architectural compromise in place, add `xa-prep-tasks` as a `@(ProjectReference)` to various projects to help ensure it's built *first*, and rearchitect `bundle.mdproj` so that `xa-prep-tasks.targets` and `bundle.targets` can use the same targets to compute the bundle filename, now in `build-tools/bundle/bundle-path.targets`. Add a post-build step to `xa-prep-tasks.csproj` which downloads and extracts the expected mono bundle. One "problem" (feature?) is that the new `<SystemUnzip/>` task doesn't report errors as errors when unzip'ing the file. This turns out to be fine here because when downloading the mono bundle from Azure we don't get a 404 *anyway* -- Azure instead returns an XML document containing an error message (wat?!). We can thus ignore most error handling entirely...though we're *also* ignoring any checking for invalid downloads, which is something we should address in the future. Update the varioous project files so that they won't attempt to rebuild binaries that were present in the mono bundle.
2016-08-16 23:02:48 +03:00
_ConfigureCrossRuntimes;
_BuildCrossRuntimes;
_InstallCrossRuntimes;
</ForceBuildDependsOn>
</PropertyGroup>
Copy FSharp support assemblies from nuget (#21) Instead of building F# from sources as we've been doing so far, we'll now use a nuget to get the binaries we require. This makes the build time much shorter.
2016-05-06 00:07:05 +03:00
<Import Project="mono-runtimes.props" />
<Import Project="mono-runtimes.projitems" />
[bundle] Allow the bundle to be used. (#213) The bundle package (24b3085a, b1f13708, others) isn't being used, for (at least) two reasons: 1. A Win64 `libzip.dll` isn't in the bundle. 2. `build-tools/mono-runtimes.mdproj` isn't using the bundle. The problem with `libzip.dll` is that when commit b1f13708 introduced the `ForceBuild` targets and using the `<MSBuild/>` task instead of the `<CallTarget/>` task, `libzip.targets` hard-coded `libzip.mdproj` as the project file to build. This meant that the `libzip-windows.mdproj` build did *nothing*, becuase when the `_BuildUnlessCached` target ran within the context of `libzip-windows.mdproj`, it executed the `ForceBuild` target from `libzip.mdproj`, *not* from `libzip-windows.mdproj`. Introduce a new `$(ForceBuildProjectFilePath)` property which `libzip.mdproj` and `libzip-windows.mdproj` can provide so that the `ForceBuild` target can delegate to the correct project. The bundle is supposed to package the mono runtimes. Yet `build-tools/mono-runtimes.mdproj` is building all the runtimes: Target _BuildUnlessCached needs to be built as input file '.../xamarin-android/external/mono/autogen.sh' is newer than output file '.../xamarin-android/bin/Debug/lib/xbuild-frameworks/MonoAndroid/v1.0/Facades/libZipSharp.dll.config' `autogen.sh` is set to the timestamp of the last mono commit -- 2016-Aug-31, in this case -- while `libZipSharp.dll.config` has the timestamp from the libZipSharp git repo, which is 2016-Jun-24. Thus, the message is correct. The question, though, is *why* is `libZipSharp.dll.config` even present within the `Facades` directory in the first place? I don't have a good answer for that question. The diganostic output for the `<Copy/>` task doesn't specify *what* it's copying, which makes it difficult to figure out what project is copying `LibZipSharp.dll.config` into the Facades directory. No matter. The related problem is the use of *file globs*, which are both a blessing (less XML to write!) and a curse (`LibZipSharp.dll.config` shouldn't be there, and certainly shouldn't be preventing the bundle from being used...). Add `build-tools/mono-runtimes/ProfileAssemblies.projitems`, which provides the new `@(MonoFacadeAssembly)` and `@(MonoProfileAssembly)` item groups, *explicitly* specifying the Facade and BCL assemblies for use Xamarin.Android. This should allow the mono bundle to be used. Bump the bundle version to -v5 to signify (hopeful) inclusion of `libzip.dll` for Win64.
2016-09-09 00:14:49 +03:00
<Import Project="ProfileAssemblies.projitems" />
[mono-runtimes] Build mono if monodroid profile is missing (#39) If the mono build was aborted for whatever reason and the monodroid profile wasn't built completely (or at all) the current targets will not fail because they won't see any of the outputs missing and the _InstallBcl target will happily accept empty set for @(_Assemblies). We currently depend on a manually added full list of the monodroid profile assemblies (sans Mono.Data.Sqlite which will is built separately elsewhere and will not be built as part of the profile eventually), but we will implement a way to dynamically obtain the list of assemblies from mono.
2016-05-12 18:23:10 +03:00
<ItemGroup>
[bundle] Allow the bundle to be used. (#213) The bundle package (24b3085a, b1f13708, others) isn't being used, for (at least) two reasons: 1. A Win64 `libzip.dll` isn't in the bundle. 2. `build-tools/mono-runtimes.mdproj` isn't using the bundle. The problem with `libzip.dll` is that when commit b1f13708 introduced the `ForceBuild` targets and using the `<MSBuild/>` task instead of the `<CallTarget/>` task, `libzip.targets` hard-coded `libzip.mdproj` as the project file to build. This meant that the `libzip-windows.mdproj` build did *nothing*, becuase when the `_BuildUnlessCached` target ran within the context of `libzip-windows.mdproj`, it executed the `ForceBuild` target from `libzip.mdproj`, *not* from `libzip-windows.mdproj`. Introduce a new `$(ForceBuildProjectFilePath)` property which `libzip.mdproj` and `libzip-windows.mdproj` can provide so that the `ForceBuild` target can delegate to the correct project. The bundle is supposed to package the mono runtimes. Yet `build-tools/mono-runtimes.mdproj` is building all the runtimes: Target _BuildUnlessCached needs to be built as input file '.../xamarin-android/external/mono/autogen.sh' is newer than output file '.../xamarin-android/bin/Debug/lib/xbuild-frameworks/MonoAndroid/v1.0/Facades/libZipSharp.dll.config' `autogen.sh` is set to the timestamp of the last mono commit -- 2016-Aug-31, in this case -- while `libZipSharp.dll.config` has the timestamp from the libZipSharp git repo, which is 2016-Jun-24. Thus, the message is correct. The question, though, is *why* is `libZipSharp.dll.config` even present within the `Facades` directory in the first place? I don't have a good answer for that question. The diganostic output for the `<Copy/>` task doesn't specify *what* it's copying, which makes it difficult to figure out what project is copying `LibZipSharp.dll.config` into the Facades directory. No matter. The related problem is the use of *file globs*, which are both a blessing (less XML to write!) and a curse (`LibZipSharp.dll.config` shouldn't be there, and certainly shouldn't be preventing the bundle from being used...). Add `build-tools/mono-runtimes/ProfileAssemblies.projitems`, which provides the new `@(MonoFacadeAssembly)` and `@(MonoProfileAssembly)` item groups, *explicitly* specifying the Facade and BCL assemblies for use Xamarin.Android. This should allow the mono bundle to be used. Bump the bundle version to -v5 to signify (hopeful) inclusion of `libzip.dll` for Win64.
2016-09-09 00:14:49 +03:00
<_BclAssembly Include="@(MonoProfileAssembly)" />
[bundle] Include mono's debug symbols (#513) Microsoft has (several?) [symbol servers][0], which is a repository "somewhere" which contains debug symbols, so that when a stack trace containing e.g. BCL stack frames is sent to Microsoft, there is a way to correlate that back to filename and line number source for further investigation. Xamarin.Android has not partaken in this mechanism, in large part because we did most of our building on Linux and macOS machines which generate `.mdb` debug symbols, while symbol servers require `.pdb`. That constraint is now behind us; with 76be1fb1 and the use of Mono 4.9 for builds, our builds now generate Portable `.pdb` files, which *should* (hopefully) be usable on Microsoft's symbol servers. However, to transmit them to Microsoft's servers, we need to actually *retain* them. At present, they're not: because of `bundle*.zip` use, the `mscorlib.dll` that we include in the final product may have been built much earlier than the rest of the product, and we don't have any other mechanism to retain the `mscorlib.pdb` file. Let's change that: update `mono-runtimes.targets` so that the created debug symbol files are included in the produced `bundle*.zip`, and bump the corresponding `$(XABundleFileName)` version number. This will ensure that we retain the debug symbols for our BCL assemblies, allowing us to actually have something to upload. [0]: https://msdn.microsoft.com/en-us/library/cc667410.aspx
2017-03-22 22:51:57 +03:00
<_BclExcludeDebugSymbols Include="System.Windows.dll" />
<_BclExcludeDebugSymbols Include="System.Xml.Serialization.dll" />
[Xamarin.Android.Bcl-Tests] Add BCL test project. (#872) What do we want? (with apologies to 48e3fc26) **MOAR** Unit tests! Specifically, we want to run the BCL unit tests which mono generates: $ cd external/mono/mcs/class/corlib $ make PROFILE=monodroid test # creates `monodroid_corlib_test.dll` Creation of `monodroid_*_test.dll` assemblies and the `make PROFILE=monodroid test` target is a relatively recent development, for which I need to buy the #runtime team some beers. In terms of `mono-runtimes.targets`, we can build *all* of the BCL unit test assemblies with: $ cd external/mono/mcs/class $ make -i do-test PROFILE=monodroid Now that we can create them, how do we *use* them? That's the trickier bit: they need to be built within mono, as part of the existing BCL build process. This in turn means that the BCL unit test assemblies need to be distributed as part of the mono bundle, as we don't want to rebuild the mono repo "from scratch" just for the unit tests. Update `build-tools/mono-runtimes/ProfileAssemblies.projitems` to include a new `@(MonoTestAssembly)` item group which contains all of the BCL unit test assemblies and related files which should be included into `bundle-*.zip`. Additionally, add `ProfileAssemblies.projitems` to `@(VersionFile)` witihin `bundle-path.targets`, so that if anything within `ProfileAssemblies.projitems` changes, we rebuild the bundle. Once we *have* the BCL unit test assemblies, and their dependencies, we need to *run* them. The new `Xamarin.Android.Bcl-Tests.csproj` project is a Xamarin.Android application project which will execute the unit tests. There's just one small problem: Xamarin.Android apps want to use `Xamarin.Android.NUnitLite.dll`. The BCL unit test assemblies instead build against their own `nunitlite.dll`, which has no Xamarin.Android integration or support. How do we use the new test assemblies? *Force* a fix by using `remap-assembly-ref` to "rename" the `nunitlite` assembly reference to `Xamarin.Android.NUnitLite.dll`. This *cannot* be done as part of the `mono-runtimes.mdproj` build, as `Xamarin.Android.NUnitLite.dll` won't yet exist. Instead, remap the assemblies within `Xamarin.Android.Bcl-Tests.targets`, and distribute the remapped assemblies with the application. Finally, address one other "small" problem: not all of the unit tests pass! Some of these are for reasons we don't know, and others will require changes to `mono`. Update `Xamarin.Android.NUnitLite` to allow *filtering* of tests: namespace Xamarin.Android.NUnitLite { partial class TestSuiteActivity { public ITestFilter Filter {get; set;} public virtual void UpdateFilter (); } partial class TestSuiteInstrumentation { public ITestFilter Filter {get; set;} public virtual void UpdateFilter (); } } `TestSuiteActivity.UpdateFilter()` is called by `TestSuiteActivity.OnCreate()`, *after* `GetIncludedCategories()` and `GetExcludedCategories()` are called, to allow subclasses to alter the `ITestFilter` which is used to determine which tests are executed. `TestSuiteInstrumentation.UpdateFilter()` is called by `TestSuiteInstrumentation.OnStart()`, *after* `GetIncludedCategories()` and `GetExcludedCategories()` are called, to allow subclasses to alter the `ITestFilter` which is used to determine which tests are executed. `Xamarin.Android.Bcl_Tests` overrides both of these and updates the `Filter` property so that "known failing" tests are excluded. This allows us to skip failing tests, giving us time to properly fix them in time while allowing the rest of this PR to be merged. The skipped tests include: * MonoTests.System.Reflection.AssemblyTest.GetReferencedAssemblies * MonoTests.System.ServiceModel.Description.WebInvokeAttributeTest.RejectTwoParametersWhenNotWrapped
2017-10-03 22:48:08 +03:00
<_BclTestAssemblySource Include="@(MonoTestAssembly->'$(MonoSourceFullPath)\mcs\class\%(SourcePath)\%(Identity)')" />
<_BclTestAssemblySource Include="@(MonoTestAssembly->'$(MonoSourceFullPath)\mcs\class\%(SourcePath)\%(Filename).pdb')" />
<_BclTestAssemblyDestination Include="@(MonoTestAssembly->'$(XAInstallPrefix)\..\..\bcl-tests\%(Identity)')" />
<_BclTestAssemblyDestination Include="@(MonoTestAssembly->'$(XAInstallPrefix)\..\..\bcl-tests\%(Filename).pdb')" />
</ItemGroup>
<ItemGroup>
<_BclTestOutput Include="@(_BclTestAssemblyDestination)" />
<_BclTestOutput Include="$(XAInstallPrefix)\..\..\bcl-tests\bcl-tests.zip" />
[mono-runtimes] Build mono if monodroid profile is missing (#39) If the mono build was aborted for whatever reason and the monodroid profile wasn't built completely (or at all) the current targets will not fail because they won't see any of the outputs missing and the _InstallBcl target will happily accept empty set for @(_Assemblies). We currently depend on a manually added full list of the monodroid profile assemblies (sans Mono.Data.Sqlite which will is built separately elsewhere and will not be built as part of the profile eventually), but we will implement a way to dynamically obtain the list of assemblies from mono.
2016-05-12 18:23:10 +03:00
</ItemGroup>
<ItemGroup>
<_BclProfileItems Include="@(_BclAssembly->'$(_MonoProfileDir)\%(Identity)')" />
[bundle] Include mono's debug symbols (#513) Microsoft has (several?) [symbol servers][0], which is a repository "somewhere" which contains debug symbols, so that when a stack trace containing e.g. BCL stack frames is sent to Microsoft, there is a way to correlate that back to filename and line number source for further investigation. Xamarin.Android has not partaken in this mechanism, in large part because we did most of our building on Linux and macOS machines which generate `.mdb` debug symbols, while symbol servers require `.pdb`. That constraint is now behind us; with 76be1fb1 and the use of Mono 4.9 for builds, our builds now generate Portable `.pdb` files, which *should* (hopefully) be usable on Microsoft's symbol servers. However, to transmit them to Microsoft's servers, we need to actually *retain* them. At present, they're not: because of `bundle*.zip` use, the `mscorlib.dll` that we include in the final product may have been built much earlier than the rest of the product, and we don't have any other mechanism to retain the `mscorlib.pdb` file. Let's change that: update `mono-runtimes.targets` so that the created debug symbol files are included in the produced `bundle*.zip`, and bump the corresponding `$(XABundleFileName)` version number. This will ensure that we retain the debug symbols for our BCL assemblies, allowing us to actually have something to upload. [0]: https://msdn.microsoft.com/en-us/library/cc667410.aspx
2017-03-22 22:51:57 +03:00
<_BclProfileItems
Condition=" '$(_DebugFileExt)' == '.mdb' "
Include="@(_BclAssembly->'$(_MonoProfileDir)\%(Identity).mdb')"
Exclude="@(_BclExcludeDebugSymbols->'$(_MonoProfileDir)\%(Identity).mdb')"
/>
<_BclProfileItems
Condition=" '$(_DebugFileExt)' == '.pdb' "
Include="@(_BclAssembly->'$(_MonoProfileDir)\%(Filename).pdb')"
Exclude="@(_BclExcludeDebugSymbols->'$(_MonoProfileDir)\%(Filename).pdb')"
/>
[mono-runtimes] Build mono if monodroid profile is missing (#39) If the mono build was aborted for whatever reason and the monodroid profile wasn't built completely (or at all) the current targets will not fail because they won't see any of the outputs missing and the _InstallBcl target will happily accept empty set for @(_Assemblies). We currently depend on a manually added full list of the monodroid profile assemblies (sans Mono.Data.Sqlite which will is built separately elsewhere and will not be built as part of the profile eventually), but we will implement a way to dynamically obtain the list of assemblies from mono.
2016-05-12 18:23:10 +03:00
</ItemGroup>
<ItemGroup>
[structure] Rework installation directory structure (#704) Context: https://github.com/xamarin/xamarin-android/pull/253#discussion_r82862993 The *intention* is that Jenkins-produced `oss-xamarin.android*.zip` artifacts be usable on Windows, so that side-by-side testing can be performed without replacing the system installation. Usage is in [UsingJenkinsBuildArtifacts.md](Documentation/UsingJenkinsBuildArtifacts). This isn't *entirely* the case. It was *apparently* the case at the time of commit 87ca2737, but things have bitrotten since. For example, following the 87ca2737 instructions would currently result in an XA0020 `Could not find monodroid` error, because `class-parse.exe` wasn't where Windows expects it to be (it was in `lib/mandroid`, not `lib/xbuild/Xamarin/Android`). This needs to be fixed. Additionally, PR #253 mentions that, for filesystem organization, it would be useful if the macOS/Linux directory structure -- `$prefix/bin`, `$prefix/lib/mandroid`, `$prefix/lib/xbuild/Xamarin/Android`, `$prefix/lib/xbuild-frameworks` -- more closely resembled the Windows directory structure of `$MSBuild` and `$ReferenceAssemblies` (as seen in `.vsix` files). This would turn macOS/Linux into using `$xa_prefix/xbuild` and `$xa_prefix/xbuild-frameworks` directories. `$prefix/bin` would only contain `xabuild`. What is currently in `$prefix/lib/mandroid` would be merged with `$xa_prefix/xbuild/Xamarin/Android`. `$xa_prefix` would `$prefix/lib/xamarin.android`. This would turn the current macOS structure: $prefix/bin/xabuild $prefix/bin/generator $prefix/bin/cross-arm $prefix/lib/mandroid/generator.exe $prefix/lib/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xbuild/Xamarin/Android/Xamarin.Android.Common.targets Into: $prefix/bin/xabuild $prefix/lib/xamarin.android/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xamarin.android/xbuild/Xamarin/Android/generator.exe $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Xamarin.Android.Common.targets $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Darwin/cross-arm Other notes: * The `bundle-*.zip` filename has been has been changed to include a *hash* of the contents of various files, in particular `build-tools\mono-runtimes.*`. This was instigated via a conversation with @kumpera about making the filename more "idiot-proof": `mono-runtimes.props` contains *compiler flags*, and if any of those change, then *logically* the bundle should differ as well, as the mono runtimes may differ in significant ways. In theory, the `-vXX` version should be used to track this, but this is a manual change, easy to overlook. The new `-hHASH` part of the filename should be more automagic. The new `<HashFileContents/>` task in `xa-prep-tasks.dll` is responsible for creating the hash value. * `Configuration.Java.Interop.Override.props` was moved into `build-tools/scripts`, because that would cleanup the root directory a bit. * OS-specific binaries are now placed into `$prefix/lib/xamarin.android/xbuild/Xamarin/Android/$(HostOS)`. On a theoretical plus side, this means that the same build directory can contain OS-specific binaries for multiple operating systems. (I don't know if anyone shares a build tree between e.g. macOS and Linux, but if anyone *does*...) Unfortunately this requires a workaround for an `xbuild` bug: `%(_LlvmRuntime.InstallPath)` and `%(_MonoCrossRuntime.InstallPath)` *cannot* use MSBuild properties, e.g. `<InstallPath>$(HostOS)/</InstallPath>` doesn't work as desired; it's instead treated literally. Special-case `%(InstallPath)` until we fully migrate to MSBuild. * `$(MonoAndroidToolsDirectory)` should be considered *dead*, along with `$(MonoAndroidBinDirectory)`, as these should now *always* be the same directory as where `Xamarin.Android.Build.Tasks.dll` is located, or a `$(HostOS)` sub-directory. * `Xamarin.ProjectTools.csproj` needed to be updated to ensure that the build order was correct. * Remove all `[Obsolete]` and unreferenced members from `Xamarin.Android.Build.Utilities.dll`. There's too much in there, and it makes my head hurt trying to understand the interrelationships between it all. If it's not used, it's gone. * The changes to `src/monodroid/jni/Android.mk` are...weird. The removal of `-I$(topdir)/libmonodroid/zip`/etc. is to reduce use of non-existent paths, as `$(topdir)` isn't defined, so that's *actually* `-I/libmonodroid/zip`, which is nonsensical. So far, so good. What's *odd* is the addition of `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`: This is needed so that `external/mono/support/zlib-helper.c` exports `CreateZStream` and related symbols, otherwise we get a unit test failure in `GzipStreamTest.Compression` due to an `EntryPointNotFoundException`, because `CreateZStream` isn't exported/public. What's odd here is that I don't understand what caused this behavior to change. Previous builds exported `CreateZStream`, otherwise the tests would fail, and I don't understand how any of the other changes in this PR would be at fault, though that's certainly the most plausible explanation. Regardless, `-Ijni` *first* (due to adding `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`) is the desired behavior, so that `jni/config.h` is included, thus ensuring that `MONO_API` has the required definition when building `zlib-helper.c`.
2017-07-28 16:36:32 +03:00
<_BclInstalledItem Include="@(_BclAssembly->'$(_BclFrameworkDir)%(Identity)')" />
[bundle] Include mono's debug symbols (#513) Microsoft has (several?) [symbol servers][0], which is a repository "somewhere" which contains debug symbols, so that when a stack trace containing e.g. BCL stack frames is sent to Microsoft, there is a way to correlate that back to filename and line number source for further investigation. Xamarin.Android has not partaken in this mechanism, in large part because we did most of our building on Linux and macOS machines which generate `.mdb` debug symbols, while symbol servers require `.pdb`. That constraint is now behind us; with 76be1fb1 and the use of Mono 4.9 for builds, our builds now generate Portable `.pdb` files, which *should* (hopefully) be usable on Microsoft's symbol servers. However, to transmit them to Microsoft's servers, we need to actually *retain* them. At present, they're not: because of `bundle*.zip` use, the `mscorlib.dll` that we include in the final product may have been built much earlier than the rest of the product, and we don't have any other mechanism to retain the `mscorlib.pdb` file. Let's change that: update `mono-runtimes.targets` so that the created debug symbol files are included in the produced `bundle*.zip`, and bump the corresponding `$(XABundleFileName)` version number. This will ensure that we retain the debug symbols for our BCL assemblies, allowing us to actually have something to upload. [0]: https://msdn.microsoft.com/en-us/library/cc667410.aspx
2017-03-22 22:51:57 +03:00
<_BclInstalledItem
Condition=" '$(_DebugFileExt)' == '.mdb' "
[structure] Rework installation directory structure (#704) Context: https://github.com/xamarin/xamarin-android/pull/253#discussion_r82862993 The *intention* is that Jenkins-produced `oss-xamarin.android*.zip` artifacts be usable on Windows, so that side-by-side testing can be performed without replacing the system installation. Usage is in [UsingJenkinsBuildArtifacts.md](Documentation/UsingJenkinsBuildArtifacts). This isn't *entirely* the case. It was *apparently* the case at the time of commit 87ca2737, but things have bitrotten since. For example, following the 87ca2737 instructions would currently result in an XA0020 `Could not find monodroid` error, because `class-parse.exe` wasn't where Windows expects it to be (it was in `lib/mandroid`, not `lib/xbuild/Xamarin/Android`). This needs to be fixed. Additionally, PR #253 mentions that, for filesystem organization, it would be useful if the macOS/Linux directory structure -- `$prefix/bin`, `$prefix/lib/mandroid`, `$prefix/lib/xbuild/Xamarin/Android`, `$prefix/lib/xbuild-frameworks` -- more closely resembled the Windows directory structure of `$MSBuild` and `$ReferenceAssemblies` (as seen in `.vsix` files). This would turn macOS/Linux into using `$xa_prefix/xbuild` and `$xa_prefix/xbuild-frameworks` directories. `$prefix/bin` would only contain `xabuild`. What is currently in `$prefix/lib/mandroid` would be merged with `$xa_prefix/xbuild/Xamarin/Android`. `$xa_prefix` would `$prefix/lib/xamarin.android`. This would turn the current macOS structure: $prefix/bin/xabuild $prefix/bin/generator $prefix/bin/cross-arm $prefix/lib/mandroid/generator.exe $prefix/lib/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xbuild/Xamarin/Android/Xamarin.Android.Common.targets Into: $prefix/bin/xabuild $prefix/lib/xamarin.android/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xamarin.android/xbuild/Xamarin/Android/generator.exe $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Xamarin.Android.Common.targets $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Darwin/cross-arm Other notes: * The `bundle-*.zip` filename has been has been changed to include a *hash* of the contents of various files, in particular `build-tools\mono-runtimes.*`. This was instigated via a conversation with @kumpera about making the filename more "idiot-proof": `mono-runtimes.props` contains *compiler flags*, and if any of those change, then *logically* the bundle should differ as well, as the mono runtimes may differ in significant ways. In theory, the `-vXX` version should be used to track this, but this is a manual change, easy to overlook. The new `-hHASH` part of the filename should be more automagic. The new `<HashFileContents/>` task in `xa-prep-tasks.dll` is responsible for creating the hash value. * `Configuration.Java.Interop.Override.props` was moved into `build-tools/scripts`, because that would cleanup the root directory a bit. * OS-specific binaries are now placed into `$prefix/lib/xamarin.android/xbuild/Xamarin/Android/$(HostOS)`. On a theoretical plus side, this means that the same build directory can contain OS-specific binaries for multiple operating systems. (I don't know if anyone shares a build tree between e.g. macOS and Linux, but if anyone *does*...) Unfortunately this requires a workaround for an `xbuild` bug: `%(_LlvmRuntime.InstallPath)` and `%(_MonoCrossRuntime.InstallPath)` *cannot* use MSBuild properties, e.g. `<InstallPath>$(HostOS)/</InstallPath>` doesn't work as desired; it's instead treated literally. Special-case `%(InstallPath)` until we fully migrate to MSBuild. * `$(MonoAndroidToolsDirectory)` should be considered *dead*, along with `$(MonoAndroidBinDirectory)`, as these should now *always* be the same directory as where `Xamarin.Android.Build.Tasks.dll` is located, or a `$(HostOS)` sub-directory. * `Xamarin.ProjectTools.csproj` needed to be updated to ensure that the build order was correct. * Remove all `[Obsolete]` and unreferenced members from `Xamarin.Android.Build.Utilities.dll`. There's too much in there, and it makes my head hurt trying to understand the interrelationships between it all. If it's not used, it's gone. * The changes to `src/monodroid/jni/Android.mk` are...weird. The removal of `-I$(topdir)/libmonodroid/zip`/etc. is to reduce use of non-existent paths, as `$(topdir)` isn't defined, so that's *actually* `-I/libmonodroid/zip`, which is nonsensical. So far, so good. What's *odd* is the addition of `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`: This is needed so that `external/mono/support/zlib-helper.c` exports `CreateZStream` and related symbols, otherwise we get a unit test failure in `GzipStreamTest.Compression` due to an `EntryPointNotFoundException`, because `CreateZStream` isn't exported/public. What's odd here is that I don't understand what caused this behavior to change. Previous builds exported `CreateZStream`, otherwise the tests would fail, and I don't understand how any of the other changes in this PR would be at fault, though that's certainly the most plausible explanation. Regardless, `-Ijni` *first* (due to adding `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`) is the desired behavior, so that `jni/config.h` is included, thus ensuring that `MONO_API` has the required definition when building `zlib-helper.c`.
2017-07-28 16:36:32 +03:00
Include="@(_BclAssembly->'$(_BclFrameworkDir)%(Identity).mdb')"
Exclude="@(_BclExcludeDebugSymbols->'$(_BclFrameworkDir)%(Identity).mdb')"
[bundle] Include mono's debug symbols (#513) Microsoft has (several?) [symbol servers][0], which is a repository "somewhere" which contains debug symbols, so that when a stack trace containing e.g. BCL stack frames is sent to Microsoft, there is a way to correlate that back to filename and line number source for further investigation. Xamarin.Android has not partaken in this mechanism, in large part because we did most of our building on Linux and macOS machines which generate `.mdb` debug symbols, while symbol servers require `.pdb`. That constraint is now behind us; with 76be1fb1 and the use of Mono 4.9 for builds, our builds now generate Portable `.pdb` files, which *should* (hopefully) be usable on Microsoft's symbol servers. However, to transmit them to Microsoft's servers, we need to actually *retain* them. At present, they're not: because of `bundle*.zip` use, the `mscorlib.dll` that we include in the final product may have been built much earlier than the rest of the product, and we don't have any other mechanism to retain the `mscorlib.pdb` file. Let's change that: update `mono-runtimes.targets` so that the created debug symbol files are included in the produced `bundle*.zip`, and bump the corresponding `$(XABundleFileName)` version number. This will ensure that we retain the debug symbols for our BCL assemblies, allowing us to actually have something to upload. [0]: https://msdn.microsoft.com/en-us/library/cc667410.aspx
2017-03-22 22:51:57 +03:00
/>
<_BclInstalledItem
Condition=" '$(_DebugFileExt)' == '.pdb' "
[structure] Rework installation directory structure (#704) Context: https://github.com/xamarin/xamarin-android/pull/253#discussion_r82862993 The *intention* is that Jenkins-produced `oss-xamarin.android*.zip` artifacts be usable on Windows, so that side-by-side testing can be performed without replacing the system installation. Usage is in [UsingJenkinsBuildArtifacts.md](Documentation/UsingJenkinsBuildArtifacts). This isn't *entirely* the case. It was *apparently* the case at the time of commit 87ca2737, but things have bitrotten since. For example, following the 87ca2737 instructions would currently result in an XA0020 `Could not find monodroid` error, because `class-parse.exe` wasn't where Windows expects it to be (it was in `lib/mandroid`, not `lib/xbuild/Xamarin/Android`). This needs to be fixed. Additionally, PR #253 mentions that, for filesystem organization, it would be useful if the macOS/Linux directory structure -- `$prefix/bin`, `$prefix/lib/mandroid`, `$prefix/lib/xbuild/Xamarin/Android`, `$prefix/lib/xbuild-frameworks` -- more closely resembled the Windows directory structure of `$MSBuild` and `$ReferenceAssemblies` (as seen in `.vsix` files). This would turn macOS/Linux into using `$xa_prefix/xbuild` and `$xa_prefix/xbuild-frameworks` directories. `$prefix/bin` would only contain `xabuild`. What is currently in `$prefix/lib/mandroid` would be merged with `$xa_prefix/xbuild/Xamarin/Android`. `$xa_prefix` would `$prefix/lib/xamarin.android`. This would turn the current macOS structure: $prefix/bin/xabuild $prefix/bin/generator $prefix/bin/cross-arm $prefix/lib/mandroid/generator.exe $prefix/lib/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xbuild/Xamarin/Android/Xamarin.Android.Common.targets Into: $prefix/bin/xabuild $prefix/lib/xamarin.android/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xamarin.android/xbuild/Xamarin/Android/generator.exe $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Xamarin.Android.Common.targets $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Darwin/cross-arm Other notes: * The `bundle-*.zip` filename has been has been changed to include a *hash* of the contents of various files, in particular `build-tools\mono-runtimes.*`. This was instigated via a conversation with @kumpera about making the filename more "idiot-proof": `mono-runtimes.props` contains *compiler flags*, and if any of those change, then *logically* the bundle should differ as well, as the mono runtimes may differ in significant ways. In theory, the `-vXX` version should be used to track this, but this is a manual change, easy to overlook. The new `-hHASH` part of the filename should be more automagic. The new `<HashFileContents/>` task in `xa-prep-tasks.dll` is responsible for creating the hash value. * `Configuration.Java.Interop.Override.props` was moved into `build-tools/scripts`, because that would cleanup the root directory a bit. * OS-specific binaries are now placed into `$prefix/lib/xamarin.android/xbuild/Xamarin/Android/$(HostOS)`. On a theoretical plus side, this means that the same build directory can contain OS-specific binaries for multiple operating systems. (I don't know if anyone shares a build tree between e.g. macOS and Linux, but if anyone *does*...) Unfortunately this requires a workaround for an `xbuild` bug: `%(_LlvmRuntime.InstallPath)` and `%(_MonoCrossRuntime.InstallPath)` *cannot* use MSBuild properties, e.g. `<InstallPath>$(HostOS)/</InstallPath>` doesn't work as desired; it's instead treated literally. Special-case `%(InstallPath)` until we fully migrate to MSBuild. * `$(MonoAndroidToolsDirectory)` should be considered *dead*, along with `$(MonoAndroidBinDirectory)`, as these should now *always* be the same directory as where `Xamarin.Android.Build.Tasks.dll` is located, or a `$(HostOS)` sub-directory. * `Xamarin.ProjectTools.csproj` needed to be updated to ensure that the build order was correct. * Remove all `[Obsolete]` and unreferenced members from `Xamarin.Android.Build.Utilities.dll`. There's too much in there, and it makes my head hurt trying to understand the interrelationships between it all. If it's not used, it's gone. * The changes to `src/monodroid/jni/Android.mk` are...weird. The removal of `-I$(topdir)/libmonodroid/zip`/etc. is to reduce use of non-existent paths, as `$(topdir)` isn't defined, so that's *actually* `-I/libmonodroid/zip`, which is nonsensical. So far, so good. What's *odd* is the addition of `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`: This is needed so that `external/mono/support/zlib-helper.c` exports `CreateZStream` and related symbols, otherwise we get a unit test failure in `GzipStreamTest.Compression` due to an `EntryPointNotFoundException`, because `CreateZStream` isn't exported/public. What's odd here is that I don't understand what caused this behavior to change. Previous builds exported `CreateZStream`, otherwise the tests would fail, and I don't understand how any of the other changes in this PR would be at fault, though that's certainly the most plausible explanation. Regardless, `-Ijni` *first* (due to adding `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`) is the desired behavior, so that `jni/config.h` is included, thus ensuring that `MONO_API` has the required definition when building `zlib-helper.c`.
2017-07-28 16:36:32 +03:00
Include="@(_BclAssembly->'$(_BclFrameworkDir)%(Filename).pdb')"
Exclude="@(_BclExcludeDebugSymbols->'$(_BclFrameworkDir)%(Filename).pdb')"
[bundle] Include mono's debug symbols (#513) Microsoft has (several?) [symbol servers][0], which is a repository "somewhere" which contains debug symbols, so that when a stack trace containing e.g. BCL stack frames is sent to Microsoft, there is a way to correlate that back to filename and line number source for further investigation. Xamarin.Android has not partaken in this mechanism, in large part because we did most of our building on Linux and macOS machines which generate `.mdb` debug symbols, while symbol servers require `.pdb`. That constraint is now behind us; with 76be1fb1 and the use of Mono 4.9 for builds, our builds now generate Portable `.pdb` files, which *should* (hopefully) be usable on Microsoft's symbol servers. However, to transmit them to Microsoft's servers, we need to actually *retain* them. At present, they're not: because of `bundle*.zip` use, the `mscorlib.dll` that we include in the final product may have been built much earlier than the rest of the product, and we don't have any other mechanism to retain the `mscorlib.pdb` file. Let's change that: update `mono-runtimes.targets` so that the created debug symbol files are included in the produced `bundle*.zip`, and bump the corresponding `$(XABundleFileName)` version number. This will ensure that we retain the debug symbols for our BCL assemblies, allowing us to actually have something to upload. [0]: https://msdn.microsoft.com/en-us/library/cc667410.aspx
2017-03-22 22:51:57 +03:00
/>
</ItemGroup>
<ItemGroup>
<_MonoUtility Include="mono-symbolicate.exe" />
[mono-runtimes] add mkbundle.exe to the bundle (#520) - so that we don't have to build it in the monodroid
2017-03-23 21:29:44 +03:00
<_MonoUtility Include="mkbundle.exe" />
[bundle] Include mono's debug symbols (#513) Microsoft has (several?) [symbol servers][0], which is a repository "somewhere" which contains debug symbols, so that when a stack trace containing e.g. BCL stack frames is sent to Microsoft, there is a way to correlate that back to filename and line number source for further investigation. Xamarin.Android has not partaken in this mechanism, in large part because we did most of our building on Linux and macOS machines which generate `.mdb` debug symbols, while symbol servers require `.pdb`. That constraint is now behind us; with 76be1fb1 and the use of Mono 4.9 for builds, our builds now generate Portable `.pdb` files, which *should* (hopefully) be usable on Microsoft's symbol servers. However, to transmit them to Microsoft's servers, we need to actually *retain* them. At present, they're not: because of `bundle*.zip` use, the `mscorlib.dll` that we include in the final product may have been built much earlier than the rest of the product, and we don't have any other mechanism to retain the `mscorlib.pdb` file. Let's change that: update `mono-runtimes.targets` so that the created debug symbol files are included in the produced `bundle*.zip`, and bump the corresponding `$(XABundleFileName)` version number. This will ensure that we retain the debug symbols for our BCL assemblies, allowing us to actually have something to upload. [0]: https://msdn.microsoft.com/en-us/library/cc667410.aspx
2017-03-22 22:51:57 +03:00
</ItemGroup>
<ItemGroup>
<_MonoUtilitySource Include="@(_MonoUtility->'$(_MonoOutputDir)\%(Identity)')" />
[structure] Rework installation directory structure (#704) Context: https://github.com/xamarin/xamarin-android/pull/253#discussion_r82862993 The *intention* is that Jenkins-produced `oss-xamarin.android*.zip` artifacts be usable on Windows, so that side-by-side testing can be performed without replacing the system installation. Usage is in [UsingJenkinsBuildArtifacts.md](Documentation/UsingJenkinsBuildArtifacts). This isn't *entirely* the case. It was *apparently* the case at the time of commit 87ca2737, but things have bitrotten since. For example, following the 87ca2737 instructions would currently result in an XA0020 `Could not find monodroid` error, because `class-parse.exe` wasn't where Windows expects it to be (it was in `lib/mandroid`, not `lib/xbuild/Xamarin/Android`). This needs to be fixed. Additionally, PR #253 mentions that, for filesystem organization, it would be useful if the macOS/Linux directory structure -- `$prefix/bin`, `$prefix/lib/mandroid`, `$prefix/lib/xbuild/Xamarin/Android`, `$prefix/lib/xbuild-frameworks` -- more closely resembled the Windows directory structure of `$MSBuild` and `$ReferenceAssemblies` (as seen in `.vsix` files). This would turn macOS/Linux into using `$xa_prefix/xbuild` and `$xa_prefix/xbuild-frameworks` directories. `$prefix/bin` would only contain `xabuild`. What is currently in `$prefix/lib/mandroid` would be merged with `$xa_prefix/xbuild/Xamarin/Android`. `$xa_prefix` would `$prefix/lib/xamarin.android`. This would turn the current macOS structure: $prefix/bin/xabuild $prefix/bin/generator $prefix/bin/cross-arm $prefix/lib/mandroid/generator.exe $prefix/lib/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xbuild/Xamarin/Android/Xamarin.Android.Common.targets Into: $prefix/bin/xabuild $prefix/lib/xamarin.android/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xamarin.android/xbuild/Xamarin/Android/generator.exe $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Xamarin.Android.Common.targets $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Darwin/cross-arm Other notes: * The `bundle-*.zip` filename has been has been changed to include a *hash* of the contents of various files, in particular `build-tools\mono-runtimes.*`. This was instigated via a conversation with @kumpera about making the filename more "idiot-proof": `mono-runtimes.props` contains *compiler flags*, and if any of those change, then *logically* the bundle should differ as well, as the mono runtimes may differ in significant ways. In theory, the `-vXX` version should be used to track this, but this is a manual change, easy to overlook. The new `-hHASH` part of the filename should be more automagic. The new `<HashFileContents/>` task in `xa-prep-tasks.dll` is responsible for creating the hash value. * `Configuration.Java.Interop.Override.props` was moved into `build-tools/scripts`, because that would cleanup the root directory a bit. * OS-specific binaries are now placed into `$prefix/lib/xamarin.android/xbuild/Xamarin/Android/$(HostOS)`. On a theoretical plus side, this means that the same build directory can contain OS-specific binaries for multiple operating systems. (I don't know if anyone shares a build tree between e.g. macOS and Linux, but if anyone *does*...) Unfortunately this requires a workaround for an `xbuild` bug: `%(_LlvmRuntime.InstallPath)` and `%(_MonoCrossRuntime.InstallPath)` *cannot* use MSBuild properties, e.g. `<InstallPath>$(HostOS)/</InstallPath>` doesn't work as desired; it's instead treated literally. Special-case `%(InstallPath)` until we fully migrate to MSBuild. * `$(MonoAndroidToolsDirectory)` should be considered *dead*, along with `$(MonoAndroidBinDirectory)`, as these should now *always* be the same directory as where `Xamarin.Android.Build.Tasks.dll` is located, or a `$(HostOS)` sub-directory. * `Xamarin.ProjectTools.csproj` needed to be updated to ensure that the build order was correct. * Remove all `[Obsolete]` and unreferenced members from `Xamarin.Android.Build.Utilities.dll`. There's too much in there, and it makes my head hurt trying to understand the interrelationships between it all. If it's not used, it's gone. * The changes to `src/monodroid/jni/Android.mk` are...weird. The removal of `-I$(topdir)/libmonodroid/zip`/etc. is to reduce use of non-existent paths, as `$(topdir)` isn't defined, so that's *actually* `-I/libmonodroid/zip`, which is nonsensical. So far, so good. What's *odd* is the addition of `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`: This is needed so that `external/mono/support/zlib-helper.c` exports `CreateZStream` and related symbols, otherwise we get a unit test failure in `GzipStreamTest.Compression` due to an `EntryPointNotFoundException`, because `CreateZStream` isn't exported/public. What's odd here is that I don't understand what caused this behavior to change. Previous builds exported `CreateZStream`, otherwise the tests would fail, and I don't understand how any of the other changes in this PR would be at fault, though that's certainly the most plausible explanation. Regardless, `-Ijni` *first* (due to adding `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`) is the desired behavior, so that `jni/config.h` is included, thus ensuring that `MONO_API` has the required definition when building `zlib-helper.c`.
2017-07-28 16:36:32 +03:00
<_MonoUtilityDest Include="@(_MonoUtility->'$(_MSBuildDir)\%(Identity)')" />
[bundle] Include mono's debug symbols (#513) Microsoft has (several?) [symbol servers][0], which is a repository "somewhere" which contains debug symbols, so that when a stack trace containing e.g. BCL stack frames is sent to Microsoft, there is a way to correlate that back to filename and line number source for further investigation. Xamarin.Android has not partaken in this mechanism, in large part because we did most of our building on Linux and macOS machines which generate `.mdb` debug symbols, while symbol servers require `.pdb`. That constraint is now behind us; with 76be1fb1 and the use of Mono 4.9 for builds, our builds now generate Portable `.pdb` files, which *should* (hopefully) be usable on Microsoft's symbol servers. However, to transmit them to Microsoft's servers, we need to actually *retain* them. At present, they're not: because of `bundle*.zip` use, the `mscorlib.dll` that we include in the final product may have been built much earlier than the rest of the product, and we don't have any other mechanism to retain the `mscorlib.pdb` file. Let's change that: update `mono-runtimes.targets` so that the created debug symbol files are included in the produced `bundle*.zip`, and bump the corresponding `$(XABundleFileName)` version number. This will ensure that we retain the debug symbols for our BCL assemblies, allowing us to actually have something to upload. [0]: https://msdn.microsoft.com/en-us/library/cc667410.aspx
2017-03-22 22:51:57 +03:00
<_MonoUtilitySource
Condition=" '$(_DebugFileExt)' == '.mdb'"
Include="@(_MonoUtility->'$(_MonoOutputDir)\%(Identity).mdb')"
/>
<_MonoUtilityDest
Condition=" '$(_DebugFileExt)' == '.mdb'"
[structure] Rework installation directory structure (#704) Context: https://github.com/xamarin/xamarin-android/pull/253#discussion_r82862993 The *intention* is that Jenkins-produced `oss-xamarin.android*.zip` artifacts be usable on Windows, so that side-by-side testing can be performed without replacing the system installation. Usage is in [UsingJenkinsBuildArtifacts.md](Documentation/UsingJenkinsBuildArtifacts). This isn't *entirely* the case. It was *apparently* the case at the time of commit 87ca2737, but things have bitrotten since. For example, following the 87ca2737 instructions would currently result in an XA0020 `Could not find monodroid` error, because `class-parse.exe` wasn't where Windows expects it to be (it was in `lib/mandroid`, not `lib/xbuild/Xamarin/Android`). This needs to be fixed. Additionally, PR #253 mentions that, for filesystem organization, it would be useful if the macOS/Linux directory structure -- `$prefix/bin`, `$prefix/lib/mandroid`, `$prefix/lib/xbuild/Xamarin/Android`, `$prefix/lib/xbuild-frameworks` -- more closely resembled the Windows directory structure of `$MSBuild` and `$ReferenceAssemblies` (as seen in `.vsix` files). This would turn macOS/Linux into using `$xa_prefix/xbuild` and `$xa_prefix/xbuild-frameworks` directories. `$prefix/bin` would only contain `xabuild`. What is currently in `$prefix/lib/mandroid` would be merged with `$xa_prefix/xbuild/Xamarin/Android`. `$xa_prefix` would `$prefix/lib/xamarin.android`. This would turn the current macOS structure: $prefix/bin/xabuild $prefix/bin/generator $prefix/bin/cross-arm $prefix/lib/mandroid/generator.exe $prefix/lib/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xbuild/Xamarin/Android/Xamarin.Android.Common.targets Into: $prefix/bin/xabuild $prefix/lib/xamarin.android/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xamarin.android/xbuild/Xamarin/Android/generator.exe $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Xamarin.Android.Common.targets $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Darwin/cross-arm Other notes: * The `bundle-*.zip` filename has been has been changed to include a *hash* of the contents of various files, in particular `build-tools\mono-runtimes.*`. This was instigated via a conversation with @kumpera about making the filename more "idiot-proof": `mono-runtimes.props` contains *compiler flags*, and if any of those change, then *logically* the bundle should differ as well, as the mono runtimes may differ in significant ways. In theory, the `-vXX` version should be used to track this, but this is a manual change, easy to overlook. The new `-hHASH` part of the filename should be more automagic. The new `<HashFileContents/>` task in `xa-prep-tasks.dll` is responsible for creating the hash value. * `Configuration.Java.Interop.Override.props` was moved into `build-tools/scripts`, because that would cleanup the root directory a bit. * OS-specific binaries are now placed into `$prefix/lib/xamarin.android/xbuild/Xamarin/Android/$(HostOS)`. On a theoretical plus side, this means that the same build directory can contain OS-specific binaries for multiple operating systems. (I don't know if anyone shares a build tree between e.g. macOS and Linux, but if anyone *does*...) Unfortunately this requires a workaround for an `xbuild` bug: `%(_LlvmRuntime.InstallPath)` and `%(_MonoCrossRuntime.InstallPath)` *cannot* use MSBuild properties, e.g. `<InstallPath>$(HostOS)/</InstallPath>` doesn't work as desired; it's instead treated literally. Special-case `%(InstallPath)` until we fully migrate to MSBuild. * `$(MonoAndroidToolsDirectory)` should be considered *dead*, along with `$(MonoAndroidBinDirectory)`, as these should now *always* be the same directory as where `Xamarin.Android.Build.Tasks.dll` is located, or a `$(HostOS)` sub-directory. * `Xamarin.ProjectTools.csproj` needed to be updated to ensure that the build order was correct. * Remove all `[Obsolete]` and unreferenced members from `Xamarin.Android.Build.Utilities.dll`. There's too much in there, and it makes my head hurt trying to understand the interrelationships between it all. If it's not used, it's gone. * The changes to `src/monodroid/jni/Android.mk` are...weird. The removal of `-I$(topdir)/libmonodroid/zip`/etc. is to reduce use of non-existent paths, as `$(topdir)` isn't defined, so that's *actually* `-I/libmonodroid/zip`, which is nonsensical. So far, so good. What's *odd* is the addition of `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`: This is needed so that `external/mono/support/zlib-helper.c` exports `CreateZStream` and related symbols, otherwise we get a unit test failure in `GzipStreamTest.Compression` due to an `EntryPointNotFoundException`, because `CreateZStream` isn't exported/public. What's odd here is that I don't understand what caused this behavior to change. Previous builds exported `CreateZStream`, otherwise the tests would fail, and I don't understand how any of the other changes in this PR would be at fault, though that's certainly the most plausible explanation. Regardless, `-Ijni` *first* (due to adding `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`) is the desired behavior, so that `jni/config.h` is included, thus ensuring that `MONO_API` has the required definition when building `zlib-helper.c`.
2017-07-28 16:36:32 +03:00
Include="@(_MonoUtility->'$(_MSBuildDir)\%(Identity).mdb')"
[bundle] Include mono's debug symbols (#513) Microsoft has (several?) [symbol servers][0], which is a repository "somewhere" which contains debug symbols, so that when a stack trace containing e.g. BCL stack frames is sent to Microsoft, there is a way to correlate that back to filename and line number source for further investigation. Xamarin.Android has not partaken in this mechanism, in large part because we did most of our building on Linux and macOS machines which generate `.mdb` debug symbols, while symbol servers require `.pdb`. That constraint is now behind us; with 76be1fb1 and the use of Mono 4.9 for builds, our builds now generate Portable `.pdb` files, which *should* (hopefully) be usable on Microsoft's symbol servers. However, to transmit them to Microsoft's servers, we need to actually *retain* them. At present, they're not: because of `bundle*.zip` use, the `mscorlib.dll` that we include in the final product may have been built much earlier than the rest of the product, and we don't have any other mechanism to retain the `mscorlib.pdb` file. Let's change that: update `mono-runtimes.targets` so that the created debug symbol files are included in the produced `bundle*.zip`, and bump the corresponding `$(XABundleFileName)` version number. This will ensure that we retain the debug symbols for our BCL assemblies, allowing us to actually have something to upload. [0]: https://msdn.microsoft.com/en-us/library/cc667410.aspx
2017-03-22 22:51:57 +03:00
/>
<_MonoUtilitySource
Condition=" '$(_DebugFileExt)' == '.pdb'"
Include="@(_MonoUtility->'$(_MonoOutputDir)\%(Filename).pdb')"
/>
<_MonoUtilityDest
Condition=" '$(_DebugFileExt)' == '.pdb'"
[structure] Rework installation directory structure (#704) Context: https://github.com/xamarin/xamarin-android/pull/253#discussion_r82862993 The *intention* is that Jenkins-produced `oss-xamarin.android*.zip` artifacts be usable on Windows, so that side-by-side testing can be performed without replacing the system installation. Usage is in [UsingJenkinsBuildArtifacts.md](Documentation/UsingJenkinsBuildArtifacts). This isn't *entirely* the case. It was *apparently* the case at the time of commit 87ca2737, but things have bitrotten since. For example, following the 87ca2737 instructions would currently result in an XA0020 `Could not find monodroid` error, because `class-parse.exe` wasn't where Windows expects it to be (it was in `lib/mandroid`, not `lib/xbuild/Xamarin/Android`). This needs to be fixed. Additionally, PR #253 mentions that, for filesystem organization, it would be useful if the macOS/Linux directory structure -- `$prefix/bin`, `$prefix/lib/mandroid`, `$prefix/lib/xbuild/Xamarin/Android`, `$prefix/lib/xbuild-frameworks` -- more closely resembled the Windows directory structure of `$MSBuild` and `$ReferenceAssemblies` (as seen in `.vsix` files). This would turn macOS/Linux into using `$xa_prefix/xbuild` and `$xa_prefix/xbuild-frameworks` directories. `$prefix/bin` would only contain `xabuild`. What is currently in `$prefix/lib/mandroid` would be merged with `$xa_prefix/xbuild/Xamarin/Android`. `$xa_prefix` would `$prefix/lib/xamarin.android`. This would turn the current macOS structure: $prefix/bin/xabuild $prefix/bin/generator $prefix/bin/cross-arm $prefix/lib/mandroid/generator.exe $prefix/lib/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xbuild/Xamarin/Android/Xamarin.Android.Common.targets Into: $prefix/bin/xabuild $prefix/lib/xamarin.android/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xamarin.android/xbuild/Xamarin/Android/generator.exe $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Xamarin.Android.Common.targets $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Darwin/cross-arm Other notes: * The `bundle-*.zip` filename has been has been changed to include a *hash* of the contents of various files, in particular `build-tools\mono-runtimes.*`. This was instigated via a conversation with @kumpera about making the filename more "idiot-proof": `mono-runtimes.props` contains *compiler flags*, and if any of those change, then *logically* the bundle should differ as well, as the mono runtimes may differ in significant ways. In theory, the `-vXX` version should be used to track this, but this is a manual change, easy to overlook. The new `-hHASH` part of the filename should be more automagic. The new `<HashFileContents/>` task in `xa-prep-tasks.dll` is responsible for creating the hash value. * `Configuration.Java.Interop.Override.props` was moved into `build-tools/scripts`, because that would cleanup the root directory a bit. * OS-specific binaries are now placed into `$prefix/lib/xamarin.android/xbuild/Xamarin/Android/$(HostOS)`. On a theoretical plus side, this means that the same build directory can contain OS-specific binaries for multiple operating systems. (I don't know if anyone shares a build tree between e.g. macOS and Linux, but if anyone *does*...) Unfortunately this requires a workaround for an `xbuild` bug: `%(_LlvmRuntime.InstallPath)` and `%(_MonoCrossRuntime.InstallPath)` *cannot* use MSBuild properties, e.g. `<InstallPath>$(HostOS)/</InstallPath>` doesn't work as desired; it's instead treated literally. Special-case `%(InstallPath)` until we fully migrate to MSBuild. * `$(MonoAndroidToolsDirectory)` should be considered *dead*, along with `$(MonoAndroidBinDirectory)`, as these should now *always* be the same directory as where `Xamarin.Android.Build.Tasks.dll` is located, or a `$(HostOS)` sub-directory. * `Xamarin.ProjectTools.csproj` needed to be updated to ensure that the build order was correct. * Remove all `[Obsolete]` and unreferenced members from `Xamarin.Android.Build.Utilities.dll`. There's too much in there, and it makes my head hurt trying to understand the interrelationships between it all. If it's not used, it's gone. * The changes to `src/monodroid/jni/Android.mk` are...weird. The removal of `-I$(topdir)/libmonodroid/zip`/etc. is to reduce use of non-existent paths, as `$(topdir)` isn't defined, so that's *actually* `-I/libmonodroid/zip`, which is nonsensical. So far, so good. What's *odd* is the addition of `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`: This is needed so that `external/mono/support/zlib-helper.c` exports `CreateZStream` and related symbols, otherwise we get a unit test failure in `GzipStreamTest.Compression` due to an `EntryPointNotFoundException`, because `CreateZStream` isn't exported/public. What's odd here is that I don't understand what caused this behavior to change. Previous builds exported `CreateZStream`, otherwise the tests would fail, and I don't understand how any of the other changes in this PR would be at fault, though that's certainly the most plausible explanation. Regardless, `-Ijni` *first* (due to adding `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`) is the desired behavior, so that `jni/config.h` is included, thus ensuring that `MONO_API` has the required definition when building `zlib-helper.c`.
2017-07-28 16:36:32 +03:00
Include="@(_MonoUtility->'$(_MSBuildDir)\%(Filename).pdb')"
[bundle] Include mono's debug symbols (#513) Microsoft has (several?) [symbol servers][0], which is a repository "somewhere" which contains debug symbols, so that when a stack trace containing e.g. BCL stack frames is sent to Microsoft, there is a way to correlate that back to filename and line number source for further investigation. Xamarin.Android has not partaken in this mechanism, in large part because we did most of our building on Linux and macOS machines which generate `.mdb` debug symbols, while symbol servers require `.pdb`. That constraint is now behind us; with 76be1fb1 and the use of Mono 4.9 for builds, our builds now generate Portable `.pdb` files, which *should* (hopefully) be usable on Microsoft's symbol servers. However, to transmit them to Microsoft's servers, we need to actually *retain* them. At present, they're not: because of `bundle*.zip` use, the `mscorlib.dll` that we include in the final product may have been built much earlier than the rest of the product, and we don't have any other mechanism to retain the `mscorlib.pdb` file. Let's change that: update `mono-runtimes.targets` so that the created debug symbol files are included in the produced `bundle*.zip`, and bump the corresponding `$(XABundleFileName)` version number. This will ensure that we retain the debug symbols for our BCL assemblies, allowing us to actually have something to upload. [0]: https://msdn.microsoft.com/en-us/library/cc667410.aspx
2017-03-22 22:51:57 +03:00
/>
[mono-runtimes] Build mono if monodroid profile is missing (#39) If the mono build was aborted for whatever reason and the monodroid profile wasn't built completely (or at all) the current targets will not fail because they won't see any of the outputs missing and the _InstallBcl target will happily accept empty set for @(_Assemblies). We currently depend on a manually added full list of the monodroid profile assemblies (sans Mono.Data.Sqlite which will is built separately elsewhere and will not be built as part of the profile eventually), but we will implement a way to dynamically obtain the list of assemblies from mono.
2016-05-12 18:23:10 +03:00
</ItemGroup>
[mono-runtimes] Rebuild mono when mono changes. (#36) The `mono-runtimes` project isn't resilient to mono changes, in large part because of the `_Autogen` target: `_Autogen` only executes if `$(_MonoPath)\autogen.sh` is newer than `$(_MonoPath)\configure`, and if the `_Autogen` target *doesn't* run, *no other targets* are re-executed either. Meaning if mono is bumped, e.g. as in commit cebaba20, the "newly updated" mono isn't necessarily rebuilt, because as far as the `_Autogen` target is concerned, nothing has changed (unless there is a commit which just happened to touch `autogen.sh`...which is a fairly infrequent occurrence). Add a new `_SetAutogenShTimeToLastCommitTimestamp` target which *always* touch(1)es `autogen.sh` to contain the timestamp of the most recent commit. That way, if (when) mono is updated, the timestamp of `autogen.sh` will be updated to match, allowing mono to be properly rebuilt. If mono *isn't* updated, the timestamp will be set to what it previously was, and no rebuild will occur. Note: `git log --date=format-local:...` must be used so that the resulting timestamp is in the *local* timezone. Otherwise, it's possible that if the current commit was done "in the future" relative to the current local timezone, `autogen.sh` will *always* have a more recent timestamp...until "tomorrow", anyway, and nobody wants to wait for "tomorrow".
2016-05-12 03:08:11 +03:00
<Target Name="_SetAutogenShTimeToLastCommitTimestamp">
[xa-prep-tasks] Add <GitCommitTime/> (#734) Context: #181 (in comments) In several places throughout the build, we are running a command such as `<Exec Command="touch -m -t `git log -1 --format=%25cd --date=format-local:%25Y%25m%25d%25H%25M.%25S` Makefile" />`, which will not work on Windows. The solution is to create an MSBuild task in xa-prep-tasks named `GitCommitTime` that will work cross-platform. `GitCommitTime` returns a string value that can be passed to the `Touch` MSBuild task.
2017-08-10 22:43:02 +03:00
<GitCommitTime
[build] Add `make jenkins` target. (#116) The `make jenkins` [^0] target is for use by Continuous Integration machines, to build *everything* [^1]. This is expected to take an eternity. Think *hours*. $ time make jenkins ... real 130m11.608s user 97m22.220s sys 18m20.522s Of particular note is that the above "everything" includes *Release configuration builds* of everything, which is something that didn't actually work before. (Oops.) Bump Java.Interop so that it supports building the Release configuration, update Xamarin.Android.sln so that all required projects are part of the Release configuration, and update Xamarin.Android.Build.Tasks.csproj so that `JCW_ONLY_TYPE_NAMES` isn't defined, as this was preventing compilation. Fix **strip**(1) use: `mono-runtimes.targets` was trying to use `strip -S` on macOS, but the value of `%(_MonoRuntime.Strip)` was quoted, and thus attempted to execute `"strip -S" ...`, which failed. Move the `-S` into a new `%(_MonoRuntime.StripFlags)` value. Fixup `mono-runtimes.targets` and related files so that `$(MonoSourceFullPath)` is used instead of a relative path. This helps alleviate the "mental math" of determining the relative path to the Mono checkout. Plus, the Mono checkout is supposed to be overridable, e.g. commit d205cab2, and using `$(MonoSourceFullPath)` supports that. Download and install `android.jar` for all supported API levels. Fix the `Mono.Android.csproj` build so that `Mono.Android.dll` is stored in a per-API-level intermediate directory. Otherwise, if e.g. API-10 is built after API-23, the API-23 version will be installed, but the JCW build will fail. Additionally, API-24 requires using `javac -source 1.8 -target 1.8`, not 1.6. Fix `Mono.Android/metadata` to use the correct `merge.SourceFile` filename of `Profiles/api-24.xml.in`. Without that fix, API-24 won't build because `NumericShaper.GetContextualShaper()` is emitted twice, and the C# compiler doesn't like that. Disable use of `-lz` when building for Windows. Windows doesn't contain a `Z.DLL` to link against. [^0]: https://en.wikipedia.org/wiki/Leeroy_Jenkins [^1]: https://www.youtube.com/watch?v=hooKVstzbz0
2016-07-18 15:42:39 +03:00
WorkingDirectory="$(MonoSourceFullPath)"
[xa-prep-tasks] Add <GitCommitTime/> (#734) Context: #181 (in comments) In several places throughout the build, we are running a command such as `<Exec Command="touch -m -t `git log -1 --format=%25cd --date=format-local:%25Y%25m%25d%25H%25M.%25S` Makefile" />`, which will not work on Windows. The solution is to create an MSBuild task in xa-prep-tasks named `GitCommitTime` that will work cross-platform. `GitCommitTime` returns a string value that can be passed to the `Touch` MSBuild task.
2017-08-10 22:43:02 +03:00
ToolPath="$(GitToolPath)"
ToolExe="$(GitToolExe)">
<Output TaskParameter="Time" PropertyName="_MonoCommitTime" />
</GitCommitTime>
<Touch Files="$(MonoSourceFullPath)\autogen.sh" Time="$(_MonoCommitTime)" />
<GitCommitTime
[xa-prep-tasks] Use the Mono bundle (#162) Context: https://github.com/xamarin/xamarin-android/commit/fbfd676c102c63e4e06e750857b178725e33450c Stage 3 of the cunning plan is to (attempt to) use the mono bundle introduced in commit fbfd676c. This "simple" desire (ha!) re-raises the architectural project dependency issue "solved" in fbfd676c, but first, a simple question: What should download the mono bundle? There are two plausible answers: 1. `make prepare` can (somehow) handle it. 2. MSBuild can (somehow) handle it. Both are plausible. The problem with using `make` targets (1) is there is increased potential for "duplication" -- duplication of the bundle filename, downloading it, and extracting it. Plus, `make` isn't "Windows friendly", in that GNU make isn't (normally) present with Visual Studio. (`NMAKE` is, but the Makefiles in this project are not compatible with `NMAKE`.) Which brings us to MSBuild (2): can it handle the task? To tackle that, we need to be able to have an MSBuild task project which has *no dependencies*, so that it can download and extract the mono bundle *before anything else runs*, as it may be downloading contents which mean that other projects don't *need* to run. The need for a "pre-bootstrap" task assembly -- called `xa-prep-tasks` -- thus "undoes" *some* of the logic regarding `libzip-windows.mdproj` and the `<Zip/>` task from fbfd676c: it isn't *possible* to rely on `libzip` from a "pre-build" state, as `libzip` is one of the things in the mono bundle, so now we need *two* "bootstrap" task assemblies: one without a `libzip` dependency -- `xa-prep-tasks.dll` -- and one *with* a `libzip` dependency -- `Xamarin.Android.Tools.BootstrapTasks.dll` Move tasks which don't currently require `libzip` -- or won't in the future, or laziness -- from `Xamarin.Android.Tools.BootstrapTasks.dll` and move them into `xa-prep-tasks.dll`. With that architectural compromise in place, add `xa-prep-tasks` as a `@(ProjectReference)` to various projects to help ensure it's built *first*, and rearchitect `bundle.mdproj` so that `xa-prep-tasks.targets` and `bundle.targets` can use the same targets to compute the bundle filename, now in `build-tools/bundle/bundle-path.targets`. Add a post-build step to `xa-prep-tasks.csproj` which downloads and extracts the expected mono bundle. One "problem" (feature?) is that the new `<SystemUnzip/>` task doesn't report errors as errors when unzip'ing the file. This turns out to be fine here because when downloading the mono bundle from Azure we don't get a 404 *anyway* -- Azure instead returns an XML document containing an error message (wat?!). We can thus ignore most error handling entirely...though we're *also* ignoring any checking for invalid downloads, which is something we should address in the future. Update the varioous project files so that they won't attempt to rebuild binaries that were present in the mono bundle.
2016-08-16 23:02:48 +03:00
WorkingDirectory="$(LlvmSourceFullPath)"
[xa-prep-tasks] Add <GitCommitTime/> (#734) Context: #181 (in comments) In several places throughout the build, we are running a command such as `<Exec Command="touch -m -t `git log -1 --format=%25cd --date=format-local:%25Y%25m%25d%25H%25M.%25S` Makefile" />`, which will not work on Windows. The solution is to create an MSBuild task in xa-prep-tasks named `GitCommitTime` that will work cross-platform. `GitCommitTime` returns a string value that can be passed to the `Touch` MSBuild task.
2017-08-10 22:43:02 +03:00
ToolPath="$(GitToolPath)"
ToolExe="$(GitToolExe)">
<Output TaskParameter="Time" PropertyName="_LlvmCommitTime" />
</GitCommitTime>
<Touch Files="$(LlvmSourceFullPath)\Makefile.config.in" Time="$(_LlvmCommitTime)" />
[mono-runtimes] Rebuild mono when mono changes. (#36) The `mono-runtimes` project isn't resilient to mono changes, in large part because of the `_Autogen` target: `_Autogen` only executes if `$(_MonoPath)\autogen.sh` is newer than `$(_MonoPath)\configure`, and if the `_Autogen` target *doesn't* run, *no other targets* are re-executed either. Meaning if mono is bumped, e.g. as in commit cebaba20, the "newly updated" mono isn't necessarily rebuilt, because as far as the `_Autogen` target is concerned, nothing has changed (unless there is a commit which just happened to touch `autogen.sh`...which is a fairly infrequent occurrence). Add a new `_SetAutogenShTimeToLastCommitTimestamp` target which *always* touch(1)es `autogen.sh` to contain the timestamp of the most recent commit. That way, if (when) mono is updated, the timestamp of `autogen.sh` will be updated to match, allowing mono to be properly rebuilt. If mono *isn't* updated, the timestamp will be set to what it previously was, and no rebuild will occur. Note: `git log --date=format-local:...` must be used so that the resulting timestamp is in the *local* timezone. Otherwise, it's possible that if the current commit was done "in the future" relative to the current local timezone, `autogen.sh` will *always* have a more recent timestamp...until "tomorrow", anyway, and nobody wants to wait for "tomorrow".
2016-05-12 03:08:11 +03:00
</Target>
[mono-runtimes] Build AOT+LLVM cross-compilers (#125) The commit implements building of LLVM and cross-compilers to support Xamarin.Android/Mono AOT. LLVM and cross-compilers can be built for both the host platform (Linux and OS/X at the moment) as well as cross-compiled for 32-bit and 64-bit Windows platforms. Windows builds are done with MXE toolchain on OS/X and with the packaged mingw-w64 toolchain on Linux (tested on Ubuntu 16.04 ONLY). Also introducing a new set of MSBuild properties that contain information about the host system. Some of those properties (HostOS, HostCC, HostCXX for instance) have been moved from Configuration.props to better support auto-detection. A new script, build-tools/scripts/generate-os-info, is invoked as part of `make prepare` to generate file that contains the new properties. The generated file is required for the build to work and is also host-specific (it mustn't be moved between different machines) Cross compiler builds require access to a configured Mono build tree, in order to generate C structure offsets header file that is used by the AOT compilers to properly generate AOT-ed binaries. Therefore, even if a JIT target is not enabled in the configuration, enabling a cross-compiler for some target will configure Mono for that JIT target but it will NOT build it, to save time. To facilitate this, the _MonoRuntimes items defined in build-tools/mono-runtimes/mono-runtimes.projitems gain an additional metadata item called `DoBuild` which will be set to `true` if the runtime actually needs to be built, as opposed to just configured. MXE builds are disabled on Linux as mingw-w64 works just fine. A `make prepare` warning is issued for Linux hosts which have the binfmt_misc module enabled and either Wine of Mono (cli) registered as PE32/PE32+ binary interpreters. In such instance building of the Windows cross-compilers will fail because Autotools determine whether software is being cross compiled by building a test program and attempting to execute it. In normal circumstances such an attempt will fail, but with Windows cross-compilation and either Wine or Mono registered to handle the PE32 executables this attempt will succeed thus causing the cross compilation detection to fail. Currently to build cross compilers on Linux you need to generate the C structure offsets header file on OS/X and copy the resulting headers to appropriate places on Linux. The header files should be placed in build-tools/mono-runtimes/obj/Debug/cross-*/ directories. The header files are: {cross-arm,cross-arm-win}/aarch64-v8a-linux-android.h {cross-arm64,cross-arm64-win}/armv5-none-linux-androideabi.h {cross-x86,cross-x86-win}/i686-none-linux-android.h {cross-x86_64,cross-x86_64-win}/x86_64-none-linux-android.h Offsets header generation doesn't work on Linux atm because of missing support for it in the Mono utility used to generate the offsets. Hopefully this limitation will be removed in the near future and a start-to-end build of everything will be possible on Linux. It is now mandatory to run at least `make prepare-props` before Xamarin.Android can be built. The target generates the OS-specific props file which is required by the build. `make prepare` depends on the target.
2016-07-26 16:27:31 +03:00
<Target Name="_PrepareLlvmItems">
<ItemGroup>
<_LlvmSourceFile Include="$(LlvmSourceFullPath)\lib\**\*.cpp" />
<_LlvmHeaderFile Include="$(LlvmSourceFullPath)\lib\**\*.h" />
<_LlvmMakefileConfig Include="@(_LlvmRuntime->'$(_LlvmOutputDirTop)\%(BuildDir)\Makefile.config')" />
<_LlvmConfigureStamp Include="@(_LlvmRuntime->'$(_LlvmOutputDirTop)\%(BuildDir)\.stamp-configure')" />
</ItemGroup>
<ItemGroup>
<_LlvmArchive Include="$(_LlvmOutputDirTop)\%(_LlvmRuntime.BuildDir)\Release\lib\*.a" />
<_LlvmSourceBinary Include="@(_LlvmRuntime->'$(_LlvmOutputDirTop)\%(BuildDir)\Release\bin\opt%(ExeSuffix)')" Condition=" '%(_LlvmRuntime.InstallBinaries)' == 'true' " />
[structure] Rework installation directory structure (#704) Context: https://github.com/xamarin/xamarin-android/pull/253#discussion_r82862993 The *intention* is that Jenkins-produced `oss-xamarin.android*.zip` artifacts be usable on Windows, so that side-by-side testing can be performed without replacing the system installation. Usage is in [UsingJenkinsBuildArtifacts.md](Documentation/UsingJenkinsBuildArtifacts). This isn't *entirely* the case. It was *apparently* the case at the time of commit 87ca2737, but things have bitrotten since. For example, following the 87ca2737 instructions would currently result in an XA0020 `Could not find monodroid` error, because `class-parse.exe` wasn't where Windows expects it to be (it was in `lib/mandroid`, not `lib/xbuild/Xamarin/Android`). This needs to be fixed. Additionally, PR #253 mentions that, for filesystem organization, it would be useful if the macOS/Linux directory structure -- `$prefix/bin`, `$prefix/lib/mandroid`, `$prefix/lib/xbuild/Xamarin/Android`, `$prefix/lib/xbuild-frameworks` -- more closely resembled the Windows directory structure of `$MSBuild` and `$ReferenceAssemblies` (as seen in `.vsix` files). This would turn macOS/Linux into using `$xa_prefix/xbuild` and `$xa_prefix/xbuild-frameworks` directories. `$prefix/bin` would only contain `xabuild`. What is currently in `$prefix/lib/mandroid` would be merged with `$xa_prefix/xbuild/Xamarin/Android`. `$xa_prefix` would `$prefix/lib/xamarin.android`. This would turn the current macOS structure: $prefix/bin/xabuild $prefix/bin/generator $prefix/bin/cross-arm $prefix/lib/mandroid/generator.exe $prefix/lib/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xbuild/Xamarin/Android/Xamarin.Android.Common.targets Into: $prefix/bin/xabuild $prefix/lib/xamarin.android/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xamarin.android/xbuild/Xamarin/Android/generator.exe $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Xamarin.Android.Common.targets $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Darwin/cross-arm Other notes: * The `bundle-*.zip` filename has been has been changed to include a *hash* of the contents of various files, in particular `build-tools\mono-runtimes.*`. This was instigated via a conversation with @kumpera about making the filename more "idiot-proof": `mono-runtimes.props` contains *compiler flags*, and if any of those change, then *logically* the bundle should differ as well, as the mono runtimes may differ in significant ways. In theory, the `-vXX` version should be used to track this, but this is a manual change, easy to overlook. The new `-hHASH` part of the filename should be more automagic. The new `<HashFileContents/>` task in `xa-prep-tasks.dll` is responsible for creating the hash value. * `Configuration.Java.Interop.Override.props` was moved into `build-tools/scripts`, because that would cleanup the root directory a bit. * OS-specific binaries are now placed into `$prefix/lib/xamarin.android/xbuild/Xamarin/Android/$(HostOS)`. On a theoretical plus side, this means that the same build directory can contain OS-specific binaries for multiple operating systems. (I don't know if anyone shares a build tree between e.g. macOS and Linux, but if anyone *does*...) Unfortunately this requires a workaround for an `xbuild` bug: `%(_LlvmRuntime.InstallPath)` and `%(_MonoCrossRuntime.InstallPath)` *cannot* use MSBuild properties, e.g. `<InstallPath>$(HostOS)/</InstallPath>` doesn't work as desired; it's instead treated literally. Special-case `%(InstallPath)` until we fully migrate to MSBuild. * `$(MonoAndroidToolsDirectory)` should be considered *dead*, along with `$(MonoAndroidBinDirectory)`, as these should now *always* be the same directory as where `Xamarin.Android.Build.Tasks.dll` is located, or a `$(HostOS)` sub-directory. * `Xamarin.ProjectTools.csproj` needed to be updated to ensure that the build order was correct. * Remove all `[Obsolete]` and unreferenced members from `Xamarin.Android.Build.Utilities.dll`. There's too much in there, and it makes my head hurt trying to understand the interrelationships between it all. If it's not used, it's gone. * The changes to `src/monodroid/jni/Android.mk` are...weird. The removal of `-I$(topdir)/libmonodroid/zip`/etc. is to reduce use of non-existent paths, as `$(topdir)` isn't defined, so that's *actually* `-I/libmonodroid/zip`, which is nonsensical. So far, so good. What's *odd* is the addition of `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`: This is needed so that `external/mono/support/zlib-helper.c` exports `CreateZStream` and related symbols, otherwise we get a unit test failure in `GzipStreamTest.Compression` due to an `EntryPointNotFoundException`, because `CreateZStream` isn't exported/public. What's odd here is that I don't understand what caused this behavior to change. Previous builds exported `CreateZStream`, otherwise the tests would fail, and I don't understand how any of the other changes in this PR would be at fault, though that's certainly the most plausible explanation. Regardless, `-Ijni` *first* (due to adding `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`) is the desired behavior, so that `jni/config.h` is included, thus ensuring that `MONO_API` has the required definition when building `zlib-helper.c`.
2017-07-28 16:36:32 +03:00
<_LlvmTargetBinary Include="@(_LlvmRuntime->'$(_MSBuildDir)\%(InstallPath)opt%(ExeSuffix)')" Condition=" '%(_LlvmRuntime.InstallBinaries)' == 'true' "/>
[mono-runtimes] Build AOT+LLVM cross-compilers (#125) The commit implements building of LLVM and cross-compilers to support Xamarin.Android/Mono AOT. LLVM and cross-compilers can be built for both the host platform (Linux and OS/X at the moment) as well as cross-compiled for 32-bit and 64-bit Windows platforms. Windows builds are done with MXE toolchain on OS/X and with the packaged mingw-w64 toolchain on Linux (tested on Ubuntu 16.04 ONLY). Also introducing a new set of MSBuild properties that contain information about the host system. Some of those properties (HostOS, HostCC, HostCXX for instance) have been moved from Configuration.props to better support auto-detection. A new script, build-tools/scripts/generate-os-info, is invoked as part of `make prepare` to generate file that contains the new properties. The generated file is required for the build to work and is also host-specific (it mustn't be moved between different machines) Cross compiler builds require access to a configured Mono build tree, in order to generate C structure offsets header file that is used by the AOT compilers to properly generate AOT-ed binaries. Therefore, even if a JIT target is not enabled in the configuration, enabling a cross-compiler for some target will configure Mono for that JIT target but it will NOT build it, to save time. To facilitate this, the _MonoRuntimes items defined in build-tools/mono-runtimes/mono-runtimes.projitems gain an additional metadata item called `DoBuild` which will be set to `true` if the runtime actually needs to be built, as opposed to just configured. MXE builds are disabled on Linux as mingw-w64 works just fine. A `make prepare` warning is issued for Linux hosts which have the binfmt_misc module enabled and either Wine of Mono (cli) registered as PE32/PE32+ binary interpreters. In such instance building of the Windows cross-compilers will fail because Autotools determine whether software is being cross compiled by building a test program and attempting to execute it. In normal circumstances such an attempt will fail, but with Windows cross-compilation and either Wine or Mono registered to handle the PE32 executables this attempt will succeed thus causing the cross compilation detection to fail. Currently to build cross compilers on Linux you need to generate the C structure offsets header file on OS/X and copy the resulting headers to appropriate places on Linux. The header files should be placed in build-tools/mono-runtimes/obj/Debug/cross-*/ directories. The header files are: {cross-arm,cross-arm-win}/aarch64-v8a-linux-android.h {cross-arm64,cross-arm64-win}/armv5-none-linux-androideabi.h {cross-x86,cross-x86-win}/i686-none-linux-android.h {cross-x86_64,cross-x86_64-win}/x86_64-none-linux-android.h Offsets header generation doesn't work on Linux atm because of missing support for it in the Mono utility used to generate the offsets. Hopefully this limitation will be removed in the near future and a start-to-end build of everything will be possible on Linux. It is now mandatory to run at least `make prepare-props` before Xamarin.Android can be built. The target generates the OS-specific props file which is required by the build. `make prepare` depends on the target.
2016-07-26 16:27:31 +03:00
<_LlvmSourceBinary Include="@(_LlvmRuntime->'$(_LlvmOutputDirTop)\%(BuildDir)\Release\bin\llc%(ExeSuffix)')" Condition=" '%(_LlvmRuntime.InstallBinaries)' == 'true' " />
[structure] Rework installation directory structure (#704) Context: https://github.com/xamarin/xamarin-android/pull/253#discussion_r82862993 The *intention* is that Jenkins-produced `oss-xamarin.android*.zip` artifacts be usable on Windows, so that side-by-side testing can be performed without replacing the system installation. Usage is in [UsingJenkinsBuildArtifacts.md](Documentation/UsingJenkinsBuildArtifacts). This isn't *entirely* the case. It was *apparently* the case at the time of commit 87ca2737, but things have bitrotten since. For example, following the 87ca2737 instructions would currently result in an XA0020 `Could not find monodroid` error, because `class-parse.exe` wasn't where Windows expects it to be (it was in `lib/mandroid`, not `lib/xbuild/Xamarin/Android`). This needs to be fixed. Additionally, PR #253 mentions that, for filesystem organization, it would be useful if the macOS/Linux directory structure -- `$prefix/bin`, `$prefix/lib/mandroid`, `$prefix/lib/xbuild/Xamarin/Android`, `$prefix/lib/xbuild-frameworks` -- more closely resembled the Windows directory structure of `$MSBuild` and `$ReferenceAssemblies` (as seen in `.vsix` files). This would turn macOS/Linux into using `$xa_prefix/xbuild` and `$xa_prefix/xbuild-frameworks` directories. `$prefix/bin` would only contain `xabuild`. What is currently in `$prefix/lib/mandroid` would be merged with `$xa_prefix/xbuild/Xamarin/Android`. `$xa_prefix` would `$prefix/lib/xamarin.android`. This would turn the current macOS structure: $prefix/bin/xabuild $prefix/bin/generator $prefix/bin/cross-arm $prefix/lib/mandroid/generator.exe $prefix/lib/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xbuild/Xamarin/Android/Xamarin.Android.Common.targets Into: $prefix/bin/xabuild $prefix/lib/xamarin.android/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xamarin.android/xbuild/Xamarin/Android/generator.exe $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Xamarin.Android.Common.targets $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Darwin/cross-arm Other notes: * The `bundle-*.zip` filename has been has been changed to include a *hash* of the contents of various files, in particular `build-tools\mono-runtimes.*`. This was instigated via a conversation with @kumpera about making the filename more "idiot-proof": `mono-runtimes.props` contains *compiler flags*, and if any of those change, then *logically* the bundle should differ as well, as the mono runtimes may differ in significant ways. In theory, the `-vXX` version should be used to track this, but this is a manual change, easy to overlook. The new `-hHASH` part of the filename should be more automagic. The new `<HashFileContents/>` task in `xa-prep-tasks.dll` is responsible for creating the hash value. * `Configuration.Java.Interop.Override.props` was moved into `build-tools/scripts`, because that would cleanup the root directory a bit. * OS-specific binaries are now placed into `$prefix/lib/xamarin.android/xbuild/Xamarin/Android/$(HostOS)`. On a theoretical plus side, this means that the same build directory can contain OS-specific binaries for multiple operating systems. (I don't know if anyone shares a build tree between e.g. macOS and Linux, but if anyone *does*...) Unfortunately this requires a workaround for an `xbuild` bug: `%(_LlvmRuntime.InstallPath)` and `%(_MonoCrossRuntime.InstallPath)` *cannot* use MSBuild properties, e.g. `<InstallPath>$(HostOS)/</InstallPath>` doesn't work as desired; it's instead treated literally. Special-case `%(InstallPath)` until we fully migrate to MSBuild. * `$(MonoAndroidToolsDirectory)` should be considered *dead*, along with `$(MonoAndroidBinDirectory)`, as these should now *always* be the same directory as where `Xamarin.Android.Build.Tasks.dll` is located, or a `$(HostOS)` sub-directory. * `Xamarin.ProjectTools.csproj` needed to be updated to ensure that the build order was correct. * Remove all `[Obsolete]` and unreferenced members from `Xamarin.Android.Build.Utilities.dll`. There's too much in there, and it makes my head hurt trying to understand the interrelationships between it all. If it's not used, it's gone. * The changes to `src/monodroid/jni/Android.mk` are...weird. The removal of `-I$(topdir)/libmonodroid/zip`/etc. is to reduce use of non-existent paths, as `$(topdir)` isn't defined, so that's *actually* `-I/libmonodroid/zip`, which is nonsensical. So far, so good. What's *odd* is the addition of `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`: This is needed so that `external/mono/support/zlib-helper.c` exports `CreateZStream` and related symbols, otherwise we get a unit test failure in `GzipStreamTest.Compression` due to an `EntryPointNotFoundException`, because `CreateZStream` isn't exported/public. What's odd here is that I don't understand what caused this behavior to change. Previous builds exported `CreateZStream`, otherwise the tests would fail, and I don't understand how any of the other changes in this PR would be at fault, though that's certainly the most plausible explanation. Regardless, `-Ijni` *first* (due to adding `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`) is the desired behavior, so that `jni/config.h` is included, thus ensuring that `MONO_API` has the required definition when building `zlib-helper.c`.
2017-07-28 16:36:32 +03:00
<_LlvmTargetBinary Include="@(_LlvmRuntime->'$(_MSBuildDir)\%(InstallPath)llc%(ExeSuffix)')" Condition=" '%(_LlvmRuntime.InstallBinaries)' == 'true' " />
[mono-runtimes] Build AOT+LLVM cross-compilers (#125) The commit implements building of LLVM and cross-compilers to support Xamarin.Android/Mono AOT. LLVM and cross-compilers can be built for both the host platform (Linux and OS/X at the moment) as well as cross-compiled for 32-bit and 64-bit Windows platforms. Windows builds are done with MXE toolchain on OS/X and with the packaged mingw-w64 toolchain on Linux (tested on Ubuntu 16.04 ONLY). Also introducing a new set of MSBuild properties that contain information about the host system. Some of those properties (HostOS, HostCC, HostCXX for instance) have been moved from Configuration.props to better support auto-detection. A new script, build-tools/scripts/generate-os-info, is invoked as part of `make prepare` to generate file that contains the new properties. The generated file is required for the build to work and is also host-specific (it mustn't be moved between different machines) Cross compiler builds require access to a configured Mono build tree, in order to generate C structure offsets header file that is used by the AOT compilers to properly generate AOT-ed binaries. Therefore, even if a JIT target is not enabled in the configuration, enabling a cross-compiler for some target will configure Mono for that JIT target but it will NOT build it, to save time. To facilitate this, the _MonoRuntimes items defined in build-tools/mono-runtimes/mono-runtimes.projitems gain an additional metadata item called `DoBuild` which will be set to `true` if the runtime actually needs to be built, as opposed to just configured. MXE builds are disabled on Linux as mingw-w64 works just fine. A `make prepare` warning is issued for Linux hosts which have the binfmt_misc module enabled and either Wine of Mono (cli) registered as PE32/PE32+ binary interpreters. In such instance building of the Windows cross-compilers will fail because Autotools determine whether software is being cross compiled by building a test program and attempting to execute it. In normal circumstances such an attempt will fail, but with Windows cross-compilation and either Wine or Mono registered to handle the PE32 executables this attempt will succeed thus causing the cross compilation detection to fail. Currently to build cross compilers on Linux you need to generate the C structure offsets header file on OS/X and copy the resulting headers to appropriate places on Linux. The header files should be placed in build-tools/mono-runtimes/obj/Debug/cross-*/ directories. The header files are: {cross-arm,cross-arm-win}/aarch64-v8a-linux-android.h {cross-arm64,cross-arm64-win}/armv5-none-linux-androideabi.h {cross-x86,cross-x86-win}/i686-none-linux-android.h {cross-x86_64,cross-x86_64-win}/x86_64-none-linux-android.h Offsets header generation doesn't work on Linux atm because of missing support for it in the Mono utility used to generate the offsets. Hopefully this limitation will be removed in the near future and a start-to-end build of everything will be possible on Linux. It is now mandatory to run at least `make prepare-props` before Xamarin.Android can be built. The target generates the OS-specific props file which is required by the build. `make prepare` depends on the target.
2016-07-26 16:27:31 +03:00
</ItemGroup>
</Target>
<Target Name="_ConfigureLlvm"
DependsOnTargets="_PrepareLlvmItems"
Inputs="$(LlvmSourceFullPath)\Makefile.config.in;$(LlvmSourceFullPath)\configure;@(_LlvmSourceFile);@(_LlvmHeaderFile)"
Outputs="@(_LlvmMakefileConfig);@(_LlvmConfigureStamp)"
Condition=" $(_LlvmNeeded) != '' ">
<MakeDir Directories="$(_LlvmOutputDirTop)\%(_LlvmRuntime.BuildDir)" />
<Exec
Command="%(_LlvmRuntime.ConfigureEnvironment) $(LlvmSourceFullPath)\configure %(_LlvmRuntime.ConfigureFlags)"
[build] Support building *from clean* with `msbuild` (Take 2!) (#391) Commit 7343965a didn't succeed in actually building `xamarin-android` with `msbuild` on our Jenkins bot. :-( Time to take another stab at it. As an added complication, my local `msbuild` behavior (mono 4.8) differs from Jenkins' `msbuild` behavior (mono 4.4), specifically around MSB3644. [On Jenkins][msb-96], it's a warning: warning MSB3644: The reference assemblies for framework ".NETFramework,Version=v4.5" were not found. When I run locally, it's an *error*. Go figure. Fixing the local error requires that I *remove* the `$(TargetFrameworkRootPath)` MSBuild property from `Configuration.props`, added in commit 255a569b. The issue is that `$(TargetFrameworkRootPath)` is an *override*, an "absolute" override. Thus, when e.g. building `Xamarin.Android.Build.Tasks.csproj`, the `.NETFramework,Version=v4.5` framework -- and corresponding `mscorlib.dll` -- cannot be resolved, and everything breaks. (Which raises the wonderful question of whether removing `$(TargetFrameworkRootPath)` will cause the Linux build to break again, but we'll cross that bridge later...) Removing `$(TargetFrameworkRootPath)` in turn requires that I remove the explicit `@(Reference)`s to `System.Runtime` which commit 7343965a added when building under `xbuild`. This also means that we no longer need the `$(_XABuildingWithXBuild)` property. (The "history" eluded to in 7343965a? It's continuing! Argh!) Additionally, fix the logging from the `<UnzipDirectoryChildren/>` task so that it prints the *correct* task name, *not* `Unzip`. (Seeing the wrong task name in log files makes things *confusing*.) With my local `msbuild` faiklures fixed, we can turn our attention to to the [errors reported on Jenkins][msb-96]. They are variations on the same "form": # from build-tools/mono-runtimes .../external/mono/configure LDFLAGS=... ... EXEC : error : /Applications/Xcode8.1-beta1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/libtool: unknown option character `-' in: --config # from src/sqlite-xamarin .../android create project -g -n sqlitesample -p . -a MainActivity -t android-25 --package com.xamarin.sqlitesample -v 1.0.0 EXEC : error : Project folder '.' is not empty. Please consider using 'android update' instead. The issue here is that MSBuild is "better" than xbuild: xbuild only checks the process exit code for errors. The above `android create project` command has en exit code of 0, i.e. it didn't error...according to xbuild. That said, when running [with xbuild][xb-203]: Task "Exec" Executing: .../android create project -g -n sqlitesample -p . -a MainActivity -t android-25 --package com.xamarin.sqlitesample -v 1.0.0 Environment variables being passed to the tool: Error: Project folder '.' is not empty. Please consider using 'android update' instead ... Done executing task "Exec" Task "Copy" ... Rephrased: `android create project ...` is emitting an "Error: ..." *message*, but even when it emits that message the exit status is still 0 (no error). xbuild is only checking the exit status, and thus continues the build. MSBuild not only checks the exit status, but *also* checks the command *output*, looking for the string `error`. Because an "error" message is printed, MSBuild flags this as an error, halting the build. The fix? Set `Exec.IgnoreStandardErrorWarningFormat=True` on all `<Exec/>` tasks which invoke **make**(1) or `configure` within `build-tools/mono-runtimes` and `src/sqlite-xamarin`. Setting `Exec.IgnoreStandardErrorWarningFormat=True` prevents the `<Exec/>` task from scanning the task output for "error", causing `msbuild` to behave like `xbuild` and only use the process exit code for errors. This allows the build to continue. [msb-96]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android-msbuild/96/consoleText [xb-203]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android/203/consoleText
2017-01-20 15:05:02 +03:00
IgnoreStandardErrorWarningFormat="True"
[mono-runtimes] Build AOT+LLVM cross-compilers (#125) The commit implements building of LLVM and cross-compilers to support Xamarin.Android/Mono AOT. LLVM and cross-compilers can be built for both the host platform (Linux and OS/X at the moment) as well as cross-compiled for 32-bit and 64-bit Windows platforms. Windows builds are done with MXE toolchain on OS/X and with the packaged mingw-w64 toolchain on Linux (tested on Ubuntu 16.04 ONLY). Also introducing a new set of MSBuild properties that contain information about the host system. Some of those properties (HostOS, HostCC, HostCXX for instance) have been moved from Configuration.props to better support auto-detection. A new script, build-tools/scripts/generate-os-info, is invoked as part of `make prepare` to generate file that contains the new properties. The generated file is required for the build to work and is also host-specific (it mustn't be moved between different machines) Cross compiler builds require access to a configured Mono build tree, in order to generate C structure offsets header file that is used by the AOT compilers to properly generate AOT-ed binaries. Therefore, even if a JIT target is not enabled in the configuration, enabling a cross-compiler for some target will configure Mono for that JIT target but it will NOT build it, to save time. To facilitate this, the _MonoRuntimes items defined in build-tools/mono-runtimes/mono-runtimes.projitems gain an additional metadata item called `DoBuild` which will be set to `true` if the runtime actually needs to be built, as opposed to just configured. MXE builds are disabled on Linux as mingw-w64 works just fine. A `make prepare` warning is issued for Linux hosts which have the binfmt_misc module enabled and either Wine of Mono (cli) registered as PE32/PE32+ binary interpreters. In such instance building of the Windows cross-compilers will fail because Autotools determine whether software is being cross compiled by building a test program and attempting to execute it. In normal circumstances such an attempt will fail, but with Windows cross-compilation and either Wine or Mono registered to handle the PE32 executables this attempt will succeed thus causing the cross compilation detection to fail. Currently to build cross compilers on Linux you need to generate the C structure offsets header file on OS/X and copy the resulting headers to appropriate places on Linux. The header files should be placed in build-tools/mono-runtimes/obj/Debug/cross-*/ directories. The header files are: {cross-arm,cross-arm-win}/aarch64-v8a-linux-android.h {cross-arm64,cross-arm64-win}/armv5-none-linux-androideabi.h {cross-x86,cross-x86-win}/i686-none-linux-android.h {cross-x86_64,cross-x86_64-win}/x86_64-none-linux-android.h Offsets header generation doesn't work on Linux atm because of missing support for it in the Mono utility used to generate the offsets. Hopefully this limitation will be removed in the near future and a start-to-end build of everything will be possible on Linux. It is now mandatory to run at least `make prepare-props` before Xamarin.Android can be built. The target generates the OS-specific props file which is required by the build. `make prepare` depends on the target.
2016-07-26 16:27:31 +03:00
WorkingDirectory="$(_LlvmOutputDirTop)\%(_LlvmRuntime.BuildDir)"
/>
<Touch
Files="$(_LlvmOutputDirTop)\%(_LlvmRuntime.BuildDir)\Makefile.config"
/>
<Touch
Files="$(_LlvmOutputDirTop)\%(_LlvmRuntime.BuildDir)\.stamp-configure"
AlwaysCreate="true"
/>
</Target>
<Target Name="_BuildLlvm"
DependsOnTargets="_ConfigureLlvm"
Inputs="@(_LlvmConfigureStamp);@(_LlvmSourceFile);@(_LlvmHeaderFile)"
Outputs="@(_LlvmArchive);@(_LlvmSourceBinary)"
Condition=" '$(_LlvmNeeded)' != '' ">
<Exec
Command="%(_LlvmRuntime.BuildEnvironment) make $(MakeConcurrency) all install"
[build] Support building *from clean* with `msbuild` (Take 2!) (#391) Commit 7343965a didn't succeed in actually building `xamarin-android` with `msbuild` on our Jenkins bot. :-( Time to take another stab at it. As an added complication, my local `msbuild` behavior (mono 4.8) differs from Jenkins' `msbuild` behavior (mono 4.4), specifically around MSB3644. [On Jenkins][msb-96], it's a warning: warning MSB3644: The reference assemblies for framework ".NETFramework,Version=v4.5" were not found. When I run locally, it's an *error*. Go figure. Fixing the local error requires that I *remove* the `$(TargetFrameworkRootPath)` MSBuild property from `Configuration.props`, added in commit 255a569b. The issue is that `$(TargetFrameworkRootPath)` is an *override*, an "absolute" override. Thus, when e.g. building `Xamarin.Android.Build.Tasks.csproj`, the `.NETFramework,Version=v4.5` framework -- and corresponding `mscorlib.dll` -- cannot be resolved, and everything breaks. (Which raises the wonderful question of whether removing `$(TargetFrameworkRootPath)` will cause the Linux build to break again, but we'll cross that bridge later...) Removing `$(TargetFrameworkRootPath)` in turn requires that I remove the explicit `@(Reference)`s to `System.Runtime` which commit 7343965a added when building under `xbuild`. This also means that we no longer need the `$(_XABuildingWithXBuild)` property. (The "history" eluded to in 7343965a? It's continuing! Argh!) Additionally, fix the logging from the `<UnzipDirectoryChildren/>` task so that it prints the *correct* task name, *not* `Unzip`. (Seeing the wrong task name in log files makes things *confusing*.) With my local `msbuild` faiklures fixed, we can turn our attention to to the [errors reported on Jenkins][msb-96]. They are variations on the same "form": # from build-tools/mono-runtimes .../external/mono/configure LDFLAGS=... ... EXEC : error : /Applications/Xcode8.1-beta1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/libtool: unknown option character `-' in: --config # from src/sqlite-xamarin .../android create project -g -n sqlitesample -p . -a MainActivity -t android-25 --package com.xamarin.sqlitesample -v 1.0.0 EXEC : error : Project folder '.' is not empty. Please consider using 'android update' instead. The issue here is that MSBuild is "better" than xbuild: xbuild only checks the process exit code for errors. The above `android create project` command has en exit code of 0, i.e. it didn't error...according to xbuild. That said, when running [with xbuild][xb-203]: Task "Exec" Executing: .../android create project -g -n sqlitesample -p . -a MainActivity -t android-25 --package com.xamarin.sqlitesample -v 1.0.0 Environment variables being passed to the tool: Error: Project folder '.' is not empty. Please consider using 'android update' instead ... Done executing task "Exec" Task "Copy" ... Rephrased: `android create project ...` is emitting an "Error: ..." *message*, but even when it emits that message the exit status is still 0 (no error). xbuild is only checking the exit status, and thus continues the build. MSBuild not only checks the exit status, but *also* checks the command *output*, looking for the string `error`. Because an "error" message is printed, MSBuild flags this as an error, halting the build. The fix? Set `Exec.IgnoreStandardErrorWarningFormat=True` on all `<Exec/>` tasks which invoke **make**(1) or `configure` within `build-tools/mono-runtimes` and `src/sqlite-xamarin`. Setting `Exec.IgnoreStandardErrorWarningFormat=True` prevents the `<Exec/>` task from scanning the task output for "error", causing `msbuild` to behave like `xbuild` and only use the process exit code for errors. This allows the build to continue. [msb-96]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android-msbuild/96/consoleText [xb-203]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android/203/consoleText
2017-01-20 15:05:02 +03:00
IgnoreStandardErrorWarningFormat="True"
[mono-runtimes] Build AOT+LLVM cross-compilers (#125) The commit implements building of LLVM and cross-compilers to support Xamarin.Android/Mono AOT. LLVM and cross-compilers can be built for both the host platform (Linux and OS/X at the moment) as well as cross-compiled for 32-bit and 64-bit Windows platforms. Windows builds are done with MXE toolchain on OS/X and with the packaged mingw-w64 toolchain on Linux (tested on Ubuntu 16.04 ONLY). Also introducing a new set of MSBuild properties that contain information about the host system. Some of those properties (HostOS, HostCC, HostCXX for instance) have been moved from Configuration.props to better support auto-detection. A new script, build-tools/scripts/generate-os-info, is invoked as part of `make prepare` to generate file that contains the new properties. The generated file is required for the build to work and is also host-specific (it mustn't be moved between different machines) Cross compiler builds require access to a configured Mono build tree, in order to generate C structure offsets header file that is used by the AOT compilers to properly generate AOT-ed binaries. Therefore, even if a JIT target is not enabled in the configuration, enabling a cross-compiler for some target will configure Mono for that JIT target but it will NOT build it, to save time. To facilitate this, the _MonoRuntimes items defined in build-tools/mono-runtimes/mono-runtimes.projitems gain an additional metadata item called `DoBuild` which will be set to `true` if the runtime actually needs to be built, as opposed to just configured. MXE builds are disabled on Linux as mingw-w64 works just fine. A `make prepare` warning is issued for Linux hosts which have the binfmt_misc module enabled and either Wine of Mono (cli) registered as PE32/PE32+ binary interpreters. In such instance building of the Windows cross-compilers will fail because Autotools determine whether software is being cross compiled by building a test program and attempting to execute it. In normal circumstances such an attempt will fail, but with Windows cross-compilation and either Wine or Mono registered to handle the PE32 executables this attempt will succeed thus causing the cross compilation detection to fail. Currently to build cross compilers on Linux you need to generate the C structure offsets header file on OS/X and copy the resulting headers to appropriate places on Linux. The header files should be placed in build-tools/mono-runtimes/obj/Debug/cross-*/ directories. The header files are: {cross-arm,cross-arm-win}/aarch64-v8a-linux-android.h {cross-arm64,cross-arm64-win}/armv5-none-linux-androideabi.h {cross-x86,cross-x86-win}/i686-none-linux-android.h {cross-x86_64,cross-x86_64-win}/x86_64-none-linux-android.h Offsets header generation doesn't work on Linux atm because of missing support for it in the Mono utility used to generate the offsets. Hopefully this limitation will be removed in the near future and a start-to-end build of everything will be possible on Linux. It is now mandatory to run at least `make prepare-props` before Xamarin.Android can be built. The target generates the OS-specific props file which is required by the build. `make prepare` depends on the target.
2016-07-26 16:27:31 +03:00
WorkingDirectory="$(_LlvmOutputDirTop)\%(_LlvmRuntime.BuildDir)"
/>
<Touch
Files="@(_LlvmArchive);@(_LlvmTargetBinary)"
/>
</Target>
<Target Name="_InstallLlvm"
DependsOnTargets="_BuildLlvm"
Inputs="@(_LlvmSourceBinary)"
Outputs="@(_LlvmTargetBinary)"
Condition=" '$(_LlvmNeeded)' != '' ">
<Copy
SourceFiles="@(_LlvmSourceBinary)"
[mono-runtimes] Use directory separators with `$(OutputPath)`. [The build][0] [is broken][1]: xamarin-android/build-tools/mono-runtimes/mono-runtimes.targets: error : Could not find a part of the path ".../xamarin-android/xamarin-android/bin/Debuglib/xbuild-frameworks/MonoAndroid/v1.0/RedistList/FrameworkList.xml". The error is generated from the `_InstallBcl` target when writing `FrameworkList.xml`, and a close reading of the error message shows that a directory separator is missing: it should be `bin/Debug/lib`, but is instead `bin/Debuglib`. I *assume* that this is due to `<CallTarget/>` being "flakey" within xbuild (it passes the file containing the target to `<MSBuild/>`!), and *somehow* in that "flakiness" `$(OutputPath)` doesn't end with `\`, *which it should*, because `Microsoft.Common.targets` will automatically append `\` to `$(OutputPath)`. To work around this issue, update `mono-runtimes.targets` so that it *always* uses `\` after `$(OutputPath)` and `$(IntermediateOutputPath)`. Additionally, `llc.exe` was being installed into the wrong directory, preventing the bundle from being used: Target _BuildUnlessCached needs to be built as output file '../../bin/Debug/lib/mandroid/llc.exe' does not exist. `llc.exe` was instead installed into `bin/Debug/bin/llc.exe`. Bump the bundle version number to -v3 and correct `mono-runtimes.targets` so that `llc.exe` and `opt.exe` are properly installed into `$prefix/lib/mandroid`. [0]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android/49/ [1]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android/49/consoleText
2016-08-17 04:18:15 +03:00
DestinationFiles="@(_LlvmTargetBinary)" />
[mono-runtimes] Build AOT+LLVM cross-compilers (#125) The commit implements building of LLVM and cross-compilers to support Xamarin.Android/Mono AOT. LLVM and cross-compilers can be built for both the host platform (Linux and OS/X at the moment) as well as cross-compiled for 32-bit and 64-bit Windows platforms. Windows builds are done with MXE toolchain on OS/X and with the packaged mingw-w64 toolchain on Linux (tested on Ubuntu 16.04 ONLY). Also introducing a new set of MSBuild properties that contain information about the host system. Some of those properties (HostOS, HostCC, HostCXX for instance) have been moved from Configuration.props to better support auto-detection. A new script, build-tools/scripts/generate-os-info, is invoked as part of `make prepare` to generate file that contains the new properties. The generated file is required for the build to work and is also host-specific (it mustn't be moved between different machines) Cross compiler builds require access to a configured Mono build tree, in order to generate C structure offsets header file that is used by the AOT compilers to properly generate AOT-ed binaries. Therefore, even if a JIT target is not enabled in the configuration, enabling a cross-compiler for some target will configure Mono for that JIT target but it will NOT build it, to save time. To facilitate this, the _MonoRuntimes items defined in build-tools/mono-runtimes/mono-runtimes.projitems gain an additional metadata item called `DoBuild` which will be set to `true` if the runtime actually needs to be built, as opposed to just configured. MXE builds are disabled on Linux as mingw-w64 works just fine. A `make prepare` warning is issued for Linux hosts which have the binfmt_misc module enabled and either Wine of Mono (cli) registered as PE32/PE32+ binary interpreters. In such instance building of the Windows cross-compilers will fail because Autotools determine whether software is being cross compiled by building a test program and attempting to execute it. In normal circumstances such an attempt will fail, but with Windows cross-compilation and either Wine or Mono registered to handle the PE32 executables this attempt will succeed thus causing the cross compilation detection to fail. Currently to build cross compilers on Linux you need to generate the C structure offsets header file on OS/X and copy the resulting headers to appropriate places on Linux. The header files should be placed in build-tools/mono-runtimes/obj/Debug/cross-*/ directories. The header files are: {cross-arm,cross-arm-win}/aarch64-v8a-linux-android.h {cross-arm64,cross-arm64-win}/armv5-none-linux-androideabi.h {cross-x86,cross-x86-win}/i686-none-linux-android.h {cross-x86_64,cross-x86_64-win}/x86_64-none-linux-android.h Offsets header generation doesn't work on Linux atm because of missing support for it in the Mono utility used to generate the offsets. Hopefully this limitation will be removed in the near future and a start-to-end build of everything will be possible on Linux. It is now mandatory to run at least `make prepare-props` before Xamarin.Android can be built. The target generates the OS-specific props file which is required by the build. `make prepare` depends on the target.
2016-07-26 16:27:31 +03:00
<Touch
Files="@(_LlvmTargetBinary)" />
</Target>
[mono-runtimes] Build required mono runtimes. Add a new build-tools/mono-runtimes project to build the required Mono runtimes for Xamarin.Android app execution on Android devices. Xamarin.Android 6.0 ("cycle 6") includes Mono runtimes compiled for five architectures: * armeabi (armv5) * armeabi-v7a (armv7) * arm64-v8a (aarch64) * x86 * x86_64 An additional "host" architecture is also built to build the "monodroid" profile assemblies such as mscorlib.dll. In the interest of expediency and not requiring that *six* different runtimes be built -- which can greatly increase build times and storage requirements -- the initial xamarin-android open-source build will only build the armeabi-v7a (armv7) and "host" architectures. Additionally, the <UnzipDirectoryChildren/> task needed to be altered to use unzip(1) on OS X instead of using System.IO.Compression.ZipFile, because the ZipFile implementation included with Mono 4.4 doesn't respect file attributes such as the +x, causing all files to be extracted as 644 (rw-r--r--), resulting in an NDK toolchain which wouldn't work. To resolve this issue, execution on OS X and Linux now uses unzip(1). ~~ Adding new architectures ~~ To build Mono for a new architecture, edit build-tools/mono-runtimes/mono-runtimes.projitems and add a new @(_MonoRuntime) entry for the desired architecture. TODO: Figure out if there's a reasonable way to "opt-in" or "opt-out" of CPU architectures so that it isn't an "all or nothing" prospect.
2016-04-20 04:30:00 +03:00
<Target Name="_Autogen"
[mono-runtimes] Rebuild mono when mono changes. (#36) The `mono-runtimes` project isn't resilient to mono changes, in large part because of the `_Autogen` target: `_Autogen` only executes if `$(_MonoPath)\autogen.sh` is newer than `$(_MonoPath)\configure`, and if the `_Autogen` target *doesn't* run, *no other targets* are re-executed either. Meaning if mono is bumped, e.g. as in commit cebaba20, the "newly updated" mono isn't necessarily rebuilt, because as far as the `_Autogen` target is concerned, nothing has changed (unless there is a commit which just happened to touch `autogen.sh`...which is a fairly infrequent occurrence). Add a new `_SetAutogenShTimeToLastCommitTimestamp` target which *always* touch(1)es `autogen.sh` to contain the timestamp of the most recent commit. That way, if (when) mono is updated, the timestamp of `autogen.sh` will be updated to match, allowing mono to be properly rebuilt. If mono *isn't* updated, the timestamp will be set to what it previously was, and no rebuild will occur. Note: `git log --date=format-local:...` must be used so that the resulting timestamp is in the *local* timezone. Otherwise, it's possible that if the current commit was done "in the future" relative to the current local timezone, `autogen.sh` will *always* have a more recent timestamp...until "tomorrow", anyway, and nobody wants to wait for "tomorrow".
2016-05-12 03:08:11 +03:00
DependsOnTargets="_SetAutogenShTimeToLastCommitTimestamp"
[build] Add `make jenkins` target. (#116) The `make jenkins` [^0] target is for use by Continuous Integration machines, to build *everything* [^1]. This is expected to take an eternity. Think *hours*. $ time make jenkins ... real 130m11.608s user 97m22.220s sys 18m20.522s Of particular note is that the above "everything" includes *Release configuration builds* of everything, which is something that didn't actually work before. (Oops.) Bump Java.Interop so that it supports building the Release configuration, update Xamarin.Android.sln so that all required projects are part of the Release configuration, and update Xamarin.Android.Build.Tasks.csproj so that `JCW_ONLY_TYPE_NAMES` isn't defined, as this was preventing compilation. Fix **strip**(1) use: `mono-runtimes.targets` was trying to use `strip -S` on macOS, but the value of `%(_MonoRuntime.Strip)` was quoted, and thus attempted to execute `"strip -S" ...`, which failed. Move the `-S` into a new `%(_MonoRuntime.StripFlags)` value. Fixup `mono-runtimes.targets` and related files so that `$(MonoSourceFullPath)` is used instead of a relative path. This helps alleviate the "mental math" of determining the relative path to the Mono checkout. Plus, the Mono checkout is supposed to be overridable, e.g. commit d205cab2, and using `$(MonoSourceFullPath)` supports that. Download and install `android.jar` for all supported API levels. Fix the `Mono.Android.csproj` build so that `Mono.Android.dll` is stored in a per-API-level intermediate directory. Otherwise, if e.g. API-10 is built after API-23, the API-23 version will be installed, but the JCW build will fail. Additionally, API-24 requires using `javac -source 1.8 -target 1.8`, not 1.6. Fix `Mono.Android/metadata` to use the correct `merge.SourceFile` filename of `Profiles/api-24.xml.in`. Without that fix, API-24 won't build because `NumericShaper.GetContextualShaper()` is emitted twice, and the C# compiler doesn't like that. Disable use of `-lz` when building for Windows. Windows doesn't contain a `Z.DLL` to link against. [^0]: https://en.wikipedia.org/wiki/Leeroy_Jenkins [^1]: https://www.youtube.com/watch?v=hooKVstzbz0
2016-07-18 15:42:39 +03:00
Inputs="$(MonoSourceFullPath)\autogen.sh"
Outputs="$(MonoSourceFullPath)\configure">
[mono-runtimes] Build required mono runtimes. Add a new build-tools/mono-runtimes project to build the required Mono runtimes for Xamarin.Android app execution on Android devices. Xamarin.Android 6.0 ("cycle 6") includes Mono runtimes compiled for five architectures: * armeabi (armv5) * armeabi-v7a (armv7) * arm64-v8a (aarch64) * x86 * x86_64 An additional "host" architecture is also built to build the "monodroid" profile assemblies such as mscorlib.dll. In the interest of expediency and not requiring that *six* different runtimes be built -- which can greatly increase build times and storage requirements -- the initial xamarin-android open-source build will only build the armeabi-v7a (armv7) and "host" architectures. Additionally, the <UnzipDirectoryChildren/> task needed to be altered to use unzip(1) on OS X instead of using System.IO.Compression.ZipFile, because the ZipFile implementation included with Mono 4.4 doesn't respect file attributes such as the +x, causing all files to be extracted as 644 (rw-r--r--), resulting in an NDK toolchain which wouldn't work. To resolve this issue, execution on OS X and Linux now uses unzip(1). ~~ Adding new architectures ~~ To build Mono for a new architecture, edit build-tools/mono-runtimes/mono-runtimes.projitems and add a new @(_MonoRuntime) entry for the desired architecture. TODO: Figure out if there's a reasonable way to "opt-in" or "opt-out" of CPU architectures so that it isn't an "all or nothing" prospect.
2016-04-20 04:30:00 +03:00
<Exec
Command="NOCONFIGURE=1 ./autogen.sh"
[build] Add `make jenkins` target. (#116) The `make jenkins` [^0] target is for use by Continuous Integration machines, to build *everything* [^1]. This is expected to take an eternity. Think *hours*. $ time make jenkins ... real 130m11.608s user 97m22.220s sys 18m20.522s Of particular note is that the above "everything" includes *Release configuration builds* of everything, which is something that didn't actually work before. (Oops.) Bump Java.Interop so that it supports building the Release configuration, update Xamarin.Android.sln so that all required projects are part of the Release configuration, and update Xamarin.Android.Build.Tasks.csproj so that `JCW_ONLY_TYPE_NAMES` isn't defined, as this was preventing compilation. Fix **strip**(1) use: `mono-runtimes.targets` was trying to use `strip -S` on macOS, but the value of `%(_MonoRuntime.Strip)` was quoted, and thus attempted to execute `"strip -S" ...`, which failed. Move the `-S` into a new `%(_MonoRuntime.StripFlags)` value. Fixup `mono-runtimes.targets` and related files so that `$(MonoSourceFullPath)` is used instead of a relative path. This helps alleviate the "mental math" of determining the relative path to the Mono checkout. Plus, the Mono checkout is supposed to be overridable, e.g. commit d205cab2, and using `$(MonoSourceFullPath)` supports that. Download and install `android.jar` for all supported API levels. Fix the `Mono.Android.csproj` build so that `Mono.Android.dll` is stored in a per-API-level intermediate directory. Otherwise, if e.g. API-10 is built after API-23, the API-23 version will be installed, but the JCW build will fail. Additionally, API-24 requires using `javac -source 1.8 -target 1.8`, not 1.6. Fix `Mono.Android/metadata` to use the correct `merge.SourceFile` filename of `Profiles/api-24.xml.in`. Without that fix, API-24 won't build because `NumericShaper.GetContextualShaper()` is emitted twice, and the C# compiler doesn't like that. Disable use of `-lz` when building for Windows. Windows doesn't contain a `Z.DLL` to link against. [^0]: https://en.wikipedia.org/wiki/Leeroy_Jenkins [^1]: https://www.youtube.com/watch?v=hooKVstzbz0
2016-07-18 15:42:39 +03:00
WorkingDirectory="$(MonoSourceFullPath)"
[mono-runtimes] Build required mono runtimes. Add a new build-tools/mono-runtimes project to build the required Mono runtimes for Xamarin.Android app execution on Android devices. Xamarin.Android 6.0 ("cycle 6") includes Mono runtimes compiled for five architectures: * armeabi (armv5) * armeabi-v7a (armv7) * arm64-v8a (aarch64) * x86 * x86_64 An additional "host" architecture is also built to build the "monodroid" profile assemblies such as mscorlib.dll. In the interest of expediency and not requiring that *six* different runtimes be built -- which can greatly increase build times and storage requirements -- the initial xamarin-android open-source build will only build the armeabi-v7a (armv7) and "host" architectures. Additionally, the <UnzipDirectoryChildren/> task needed to be altered to use unzip(1) on OS X instead of using System.IO.Compression.ZipFile, because the ZipFile implementation included with Mono 4.4 doesn't respect file attributes such as the +x, causing all files to be extracted as 644 (rw-r--r--), resulting in an NDK toolchain which wouldn't work. To resolve this issue, execution on OS X and Linux now uses unzip(1). ~~ Adding new architectures ~~ To build Mono for a new architecture, edit build-tools/mono-runtimes/mono-runtimes.projitems and add a new @(_MonoRuntime) entry for the desired architecture. TODO: Figure out if there's a reasonable way to "opt-in" or "opt-out" of CPU architectures so that it isn't an "all or nothing" prospect.
2016-04-20 04:30:00 +03:00
/>
</Target>
<Target Name="_ConfigureRuntimes"
Correct "Exists" logic on monodoc debug files The previous commit's attempts to copy the pdb/mdb files for monodoc had a problem where they would be evaluated too early (probably before the files existed) and in the wrong directory. To address this, the files are instead specified as a MonoDocCopyItemOptional, a new target _GetMonodocItems is created and the ItemGroups for _MonoDocCopyItems/_MonoDocInstalledItems are evaluated in this target with correct Exists logic for the Optional files. Additional DependsOnTargets attributes were distributed through the file to ensure this new target is evaluated at the correct time.
2017-03-17 02:16:53 +03:00
DependsOnTargets="_BuildLlvm;_Autogen"
[build] Add `make jenkins` target. (#116) The `make jenkins` [^0] target is for use by Continuous Integration machines, to build *everything* [^1]. This is expected to take an eternity. Think *hours*. $ time make jenkins ... real 130m11.608s user 97m22.220s sys 18m20.522s Of particular note is that the above "everything" includes *Release configuration builds* of everything, which is something that didn't actually work before. (Oops.) Bump Java.Interop so that it supports building the Release configuration, update Xamarin.Android.sln so that all required projects are part of the Release configuration, and update Xamarin.Android.Build.Tasks.csproj so that `JCW_ONLY_TYPE_NAMES` isn't defined, as this was preventing compilation. Fix **strip**(1) use: `mono-runtimes.targets` was trying to use `strip -S` on macOS, but the value of `%(_MonoRuntime.Strip)` was quoted, and thus attempted to execute `"strip -S" ...`, which failed. Move the `-S` into a new `%(_MonoRuntime.StripFlags)` value. Fixup `mono-runtimes.targets` and related files so that `$(MonoSourceFullPath)` is used instead of a relative path. This helps alleviate the "mental math" of determining the relative path to the Mono checkout. Plus, the Mono checkout is supposed to be overridable, e.g. commit d205cab2, and using `$(MonoSourceFullPath)` supports that. Download and install `android.jar` for all supported API levels. Fix the `Mono.Android.csproj` build so that `Mono.Android.dll` is stored in a per-API-level intermediate directory. Otherwise, if e.g. API-10 is built after API-23, the API-23 version will be installed, but the JCW build will fail. Additionally, API-24 requires using `javac -source 1.8 -target 1.8`, not 1.6. Fix `Mono.Android/metadata` to use the correct `merge.SourceFile` filename of `Profiles/api-24.xml.in`. Without that fix, API-24 won't build because `NumericShaper.GetContextualShaper()` is emitted twice, and the C# compiler doesn't like that. Disable use of `-lz` when building for Windows. Windows doesn't contain a `Z.DLL` to link against. [^0]: https://en.wikipedia.org/wiki/Leeroy_Jenkins [^1]: https://www.youtube.com/watch?v=hooKVstzbz0
2016-07-18 15:42:39 +03:00
Inputs="$(MonoSourceFullPath)\configure"
[mono-runtimes] Build AOT+LLVM cross-compilers (#125) The commit implements building of LLVM and cross-compilers to support Xamarin.Android/Mono AOT. LLVM and cross-compilers can be built for both the host platform (Linux and OS/X at the moment) as well as cross-compiled for 32-bit and 64-bit Windows platforms. Windows builds are done with MXE toolchain on OS/X and with the packaged mingw-w64 toolchain on Linux (tested on Ubuntu 16.04 ONLY). Also introducing a new set of MSBuild properties that contain information about the host system. Some of those properties (HostOS, HostCC, HostCXX for instance) have been moved from Configuration.props to better support auto-detection. A new script, build-tools/scripts/generate-os-info, is invoked as part of `make prepare` to generate file that contains the new properties. The generated file is required for the build to work and is also host-specific (it mustn't be moved between different machines) Cross compiler builds require access to a configured Mono build tree, in order to generate C structure offsets header file that is used by the AOT compilers to properly generate AOT-ed binaries. Therefore, even if a JIT target is not enabled in the configuration, enabling a cross-compiler for some target will configure Mono for that JIT target but it will NOT build it, to save time. To facilitate this, the _MonoRuntimes items defined in build-tools/mono-runtimes/mono-runtimes.projitems gain an additional metadata item called `DoBuild` which will be set to `true` if the runtime actually needs to be built, as opposed to just configured. MXE builds are disabled on Linux as mingw-w64 works just fine. A `make prepare` warning is issued for Linux hosts which have the binfmt_misc module enabled and either Wine of Mono (cli) registered as PE32/PE32+ binary interpreters. In such instance building of the Windows cross-compilers will fail because Autotools determine whether software is being cross compiled by building a test program and attempting to execute it. In normal circumstances such an attempt will fail, but with Windows cross-compilation and either Wine or Mono registered to handle the PE32 executables this attempt will succeed thus causing the cross compilation detection to fail. Currently to build cross compilers on Linux you need to generate the C structure offsets header file on OS/X and copy the resulting headers to appropriate places on Linux. The header files should be placed in build-tools/mono-runtimes/obj/Debug/cross-*/ directories. The header files are: {cross-arm,cross-arm-win}/aarch64-v8a-linux-android.h {cross-arm64,cross-arm64-win}/armv5-none-linux-androideabi.h {cross-x86,cross-x86-win}/i686-none-linux-android.h {cross-x86_64,cross-x86_64-win}/x86_64-none-linux-android.h Offsets header generation doesn't work on Linux atm because of missing support for it in the Mono utility used to generate the offsets. Hopefully this limitation will be removed in the near future and a start-to-end build of everything will be possible on Linux. It is now mandatory to run at least `make prepare-props` before Xamarin.Android can be built. The target generates the OS-specific props file which is required by the build. `make prepare` depends on the target.
2016-07-26 16:27:31 +03:00
Outputs="$(IntermediateOutputPath)\%(_MonoRuntime.Identity)\Makefile">
<MakeDir Directories="$(IntermediateOutputPath)\%(_MonoRuntime.Identity)" />
[mono-runtimes] Build required mono runtimes. Add a new build-tools/mono-runtimes project to build the required Mono runtimes for Xamarin.Android app execution on Android devices. Xamarin.Android 6.0 ("cycle 6") includes Mono runtimes compiled for five architectures: * armeabi (armv5) * armeabi-v7a (armv7) * arm64-v8a (aarch64) * x86 * x86_64 An additional "host" architecture is also built to build the "monodroid" profile assemblies such as mscorlib.dll. In the interest of expediency and not requiring that *six* different runtimes be built -- which can greatly increase build times and storage requirements -- the initial xamarin-android open-source build will only build the armeabi-v7a (armv7) and "host" architectures. Additionally, the <UnzipDirectoryChildren/> task needed to be altered to use unzip(1) on OS X instead of using System.IO.Compression.ZipFile, because the ZipFile implementation included with Mono 4.4 doesn't respect file attributes such as the +x, causing all files to be extracted as 644 (rw-r--r--), resulting in an NDK toolchain which wouldn't work. To resolve this issue, execution on OS X and Linux now uses unzip(1). ~~ Adding new architectures ~~ To build Mono for a new architecture, edit build-tools/mono-runtimes/mono-runtimes.projitems and add a new @(_MonoRuntime) entry for the desired architecture. TODO: Figure out if there's a reasonable way to "opt-in" or "opt-out" of CPU architectures so that it isn't an "all or nothing" prospect.
2016-04-20 04:30:00 +03:00
<Exec
[build] Add `make jenkins` target. (#116) The `make jenkins` [^0] target is for use by Continuous Integration machines, to build *everything* [^1]. This is expected to take an eternity. Think *hours*. $ time make jenkins ... real 130m11.608s user 97m22.220s sys 18m20.522s Of particular note is that the above "everything" includes *Release configuration builds* of everything, which is something that didn't actually work before. (Oops.) Bump Java.Interop so that it supports building the Release configuration, update Xamarin.Android.sln so that all required projects are part of the Release configuration, and update Xamarin.Android.Build.Tasks.csproj so that `JCW_ONLY_TYPE_NAMES` isn't defined, as this was preventing compilation. Fix **strip**(1) use: `mono-runtimes.targets` was trying to use `strip -S` on macOS, but the value of `%(_MonoRuntime.Strip)` was quoted, and thus attempted to execute `"strip -S" ...`, which failed. Move the `-S` into a new `%(_MonoRuntime.StripFlags)` value. Fixup `mono-runtimes.targets` and related files so that `$(MonoSourceFullPath)` is used instead of a relative path. This helps alleviate the "mental math" of determining the relative path to the Mono checkout. Plus, the Mono checkout is supposed to be overridable, e.g. commit d205cab2, and using `$(MonoSourceFullPath)` supports that. Download and install `android.jar` for all supported API levels. Fix the `Mono.Android.csproj` build so that `Mono.Android.dll` is stored in a per-API-level intermediate directory. Otherwise, if e.g. API-10 is built after API-23, the API-23 version will be installed, but the JCW build will fail. Additionally, API-24 requires using `javac -source 1.8 -target 1.8`, not 1.6. Fix `Mono.Android/metadata` to use the correct `merge.SourceFile` filename of `Profiles/api-24.xml.in`. Without that fix, API-24 won't build because `NumericShaper.GetContextualShaper()` is emitted twice, and the C# compiler doesn't like that. Disable use of `-lz` when building for Windows. Windows doesn't contain a `Z.DLL` to link against. [^0]: https://en.wikipedia.org/wiki/Leeroy_Jenkins [^1]: https://www.youtube.com/watch?v=hooKVstzbz0
2016-07-18 15:42:39 +03:00
Command="$(MonoSourceFullPath)\configure LDFLAGS=&quot;%(_MonoRuntime.LdFlags)&quot; CFLAGS=&quot;%(_MonoRuntime.CFlags)&quot; CXXFLAGS=&quot;%(_MonoRuntime.CxxFlags)&quot; CC=&quot;%(_MonoRuntime.Cc)&quot; CXX=&quot;%(_MonoRuntime.Cxx)&quot; CPP=&quot;%(_MonoRuntime.Cpp)&quot; CXXCPP=&quot;%(_MonoRuntime.CxxCpp)&quot; LD=&quot;%(_MonoRuntime.Ld)&quot; AR=&quot;%(_MonoRuntime.Ar)&quot; AS=&quot;%(_MonoRuntime.As)&quot; RANLIB=&quot;%(_MonoRuntime.RanLib)&quot; STRIP=&quot;%(_MonoRuntime.Strip)&quot; DLLTOOL=&quot;%(_MonoRuntime.DllTool)&quot; OBJDUMP=&quot;%(_MonoRuntime.Objdump)&quot; --cache-file=..\%(_MonoRuntime.Identity).config.cache %(_MonoRuntime.ConfigureFlags)"
[build] Support building *from clean* with `msbuild` (Take 2!) (#391) Commit 7343965a didn't succeed in actually building `xamarin-android` with `msbuild` on our Jenkins bot. :-( Time to take another stab at it. As an added complication, my local `msbuild` behavior (mono 4.8) differs from Jenkins' `msbuild` behavior (mono 4.4), specifically around MSB3644. [On Jenkins][msb-96], it's a warning: warning MSB3644: The reference assemblies for framework ".NETFramework,Version=v4.5" were not found. When I run locally, it's an *error*. Go figure. Fixing the local error requires that I *remove* the `$(TargetFrameworkRootPath)` MSBuild property from `Configuration.props`, added in commit 255a569b. The issue is that `$(TargetFrameworkRootPath)` is an *override*, an "absolute" override. Thus, when e.g. building `Xamarin.Android.Build.Tasks.csproj`, the `.NETFramework,Version=v4.5` framework -- and corresponding `mscorlib.dll` -- cannot be resolved, and everything breaks. (Which raises the wonderful question of whether removing `$(TargetFrameworkRootPath)` will cause the Linux build to break again, but we'll cross that bridge later...) Removing `$(TargetFrameworkRootPath)` in turn requires that I remove the explicit `@(Reference)`s to `System.Runtime` which commit 7343965a added when building under `xbuild`. This also means that we no longer need the `$(_XABuildingWithXBuild)` property. (The "history" eluded to in 7343965a? It's continuing! Argh!) Additionally, fix the logging from the `<UnzipDirectoryChildren/>` task so that it prints the *correct* task name, *not* `Unzip`. (Seeing the wrong task name in log files makes things *confusing*.) With my local `msbuild` faiklures fixed, we can turn our attention to to the [errors reported on Jenkins][msb-96]. They are variations on the same "form": # from build-tools/mono-runtimes .../external/mono/configure LDFLAGS=... ... EXEC : error : /Applications/Xcode8.1-beta1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/libtool: unknown option character `-' in: --config # from src/sqlite-xamarin .../android create project -g -n sqlitesample -p . -a MainActivity -t android-25 --package com.xamarin.sqlitesample -v 1.0.0 EXEC : error : Project folder '.' is not empty. Please consider using 'android update' instead. The issue here is that MSBuild is "better" than xbuild: xbuild only checks the process exit code for errors. The above `android create project` command has en exit code of 0, i.e. it didn't error...according to xbuild. That said, when running [with xbuild][xb-203]: Task "Exec" Executing: .../android create project -g -n sqlitesample -p . -a MainActivity -t android-25 --package com.xamarin.sqlitesample -v 1.0.0 Environment variables being passed to the tool: Error: Project folder '.' is not empty. Please consider using 'android update' instead ... Done executing task "Exec" Task "Copy" ... Rephrased: `android create project ...` is emitting an "Error: ..." *message*, but even when it emits that message the exit status is still 0 (no error). xbuild is only checking the exit status, and thus continues the build. MSBuild not only checks the exit status, but *also* checks the command *output*, looking for the string `error`. Because an "error" message is printed, MSBuild flags this as an error, halting the build. The fix? Set `Exec.IgnoreStandardErrorWarningFormat=True` on all `<Exec/>` tasks which invoke **make**(1) or `configure` within `build-tools/mono-runtimes` and `src/sqlite-xamarin`. Setting `Exec.IgnoreStandardErrorWarningFormat=True` prevents the `<Exec/>` task from scanning the task output for "error", causing `msbuild` to behave like `xbuild` and only use the process exit code for errors. This allows the build to continue. [msb-96]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android-msbuild/96/consoleText [xb-203]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android/203/consoleText
2017-01-20 15:05:02 +03:00
IgnoreStandardErrorWarningFormat="True"
[mono-runtimes] Build AOT+LLVM cross-compilers (#125) The commit implements building of LLVM and cross-compilers to support Xamarin.Android/Mono AOT. LLVM and cross-compilers can be built for both the host platform (Linux and OS/X at the moment) as well as cross-compiled for 32-bit and 64-bit Windows platforms. Windows builds are done with MXE toolchain on OS/X and with the packaged mingw-w64 toolchain on Linux (tested on Ubuntu 16.04 ONLY). Also introducing a new set of MSBuild properties that contain information about the host system. Some of those properties (HostOS, HostCC, HostCXX for instance) have been moved from Configuration.props to better support auto-detection. A new script, build-tools/scripts/generate-os-info, is invoked as part of `make prepare` to generate file that contains the new properties. The generated file is required for the build to work and is also host-specific (it mustn't be moved between different machines) Cross compiler builds require access to a configured Mono build tree, in order to generate C structure offsets header file that is used by the AOT compilers to properly generate AOT-ed binaries. Therefore, even if a JIT target is not enabled in the configuration, enabling a cross-compiler for some target will configure Mono for that JIT target but it will NOT build it, to save time. To facilitate this, the _MonoRuntimes items defined in build-tools/mono-runtimes/mono-runtimes.projitems gain an additional metadata item called `DoBuild` which will be set to `true` if the runtime actually needs to be built, as opposed to just configured. MXE builds are disabled on Linux as mingw-w64 works just fine. A `make prepare` warning is issued for Linux hosts which have the binfmt_misc module enabled and either Wine of Mono (cli) registered as PE32/PE32+ binary interpreters. In such instance building of the Windows cross-compilers will fail because Autotools determine whether software is being cross compiled by building a test program and attempting to execute it. In normal circumstances such an attempt will fail, but with Windows cross-compilation and either Wine or Mono registered to handle the PE32 executables this attempt will succeed thus causing the cross compilation detection to fail. Currently to build cross compilers on Linux you need to generate the C structure offsets header file on OS/X and copy the resulting headers to appropriate places on Linux. The header files should be placed in build-tools/mono-runtimes/obj/Debug/cross-*/ directories. The header files are: {cross-arm,cross-arm-win}/aarch64-v8a-linux-android.h {cross-arm64,cross-arm64-win}/armv5-none-linux-androideabi.h {cross-x86,cross-x86-win}/i686-none-linux-android.h {cross-x86_64,cross-x86_64-win}/x86_64-none-linux-android.h Offsets header generation doesn't work on Linux atm because of missing support for it in the Mono utility used to generate the offsets. Hopefully this limitation will be removed in the near future and a start-to-end build of everything will be possible on Linux. It is now mandatory to run at least `make prepare-props` before Xamarin.Android can be built. The target generates the OS-specific props file which is required by the build. `make prepare` depends on the target.
2016-07-26 16:27:31 +03:00
WorkingDirectory="$(IntermediateOutputPath)\%(_MonoRuntime.Identity)"
[mono-runtimes] Build required mono runtimes. Add a new build-tools/mono-runtimes project to build the required Mono runtimes for Xamarin.Android app execution on Android devices. Xamarin.Android 6.0 ("cycle 6") includes Mono runtimes compiled for five architectures: * armeabi (armv5) * armeabi-v7a (armv7) * arm64-v8a (aarch64) * x86 * x86_64 An additional "host" architecture is also built to build the "monodroid" profile assemblies such as mscorlib.dll. In the interest of expediency and not requiring that *six* different runtimes be built -- which can greatly increase build times and storage requirements -- the initial xamarin-android open-source build will only build the armeabi-v7a (armv7) and "host" architectures. Additionally, the <UnzipDirectoryChildren/> task needed to be altered to use unzip(1) on OS X instead of using System.IO.Compression.ZipFile, because the ZipFile implementation included with Mono 4.4 doesn't respect file attributes such as the +x, causing all files to be extracted as 644 (rw-r--r--), resulting in an NDK toolchain which wouldn't work. To resolve this issue, execution on OS X and Linux now uses unzip(1). ~~ Adding new architectures ~~ To build Mono for a new architecture, edit build-tools/mono-runtimes/mono-runtimes.projitems and add a new @(_MonoRuntime) entry for the desired architecture. TODO: Figure out if there's a reasonable way to "opt-in" or "opt-out" of CPU architectures so that it isn't an "all or nothing" prospect.
2016-04-20 04:30:00 +03:00
/>
<Touch
[mono-runtimes] Build AOT+LLVM cross-compilers (#125) The commit implements building of LLVM and cross-compilers to support Xamarin.Android/Mono AOT. LLVM and cross-compilers can be built for both the host platform (Linux and OS/X at the moment) as well as cross-compiled for 32-bit and 64-bit Windows platforms. Windows builds are done with MXE toolchain on OS/X and with the packaged mingw-w64 toolchain on Linux (tested on Ubuntu 16.04 ONLY). Also introducing a new set of MSBuild properties that contain information about the host system. Some of those properties (HostOS, HostCC, HostCXX for instance) have been moved from Configuration.props to better support auto-detection. A new script, build-tools/scripts/generate-os-info, is invoked as part of `make prepare` to generate file that contains the new properties. The generated file is required for the build to work and is also host-specific (it mustn't be moved between different machines) Cross compiler builds require access to a configured Mono build tree, in order to generate C structure offsets header file that is used by the AOT compilers to properly generate AOT-ed binaries. Therefore, even if a JIT target is not enabled in the configuration, enabling a cross-compiler for some target will configure Mono for that JIT target but it will NOT build it, to save time. To facilitate this, the _MonoRuntimes items defined in build-tools/mono-runtimes/mono-runtimes.projitems gain an additional metadata item called `DoBuild` which will be set to `true` if the runtime actually needs to be built, as opposed to just configured. MXE builds are disabled on Linux as mingw-w64 works just fine. A `make prepare` warning is issued for Linux hosts which have the binfmt_misc module enabled and either Wine of Mono (cli) registered as PE32/PE32+ binary interpreters. In such instance building of the Windows cross-compilers will fail because Autotools determine whether software is being cross compiled by building a test program and attempting to execute it. In normal circumstances such an attempt will fail, but with Windows cross-compilation and either Wine or Mono registered to handle the PE32 executables this attempt will succeed thus causing the cross compilation detection to fail. Currently to build cross compilers on Linux you need to generate the C structure offsets header file on OS/X and copy the resulting headers to appropriate places on Linux. The header files should be placed in build-tools/mono-runtimes/obj/Debug/cross-*/ directories. The header files are: {cross-arm,cross-arm-win}/aarch64-v8a-linux-android.h {cross-arm64,cross-arm64-win}/armv5-none-linux-androideabi.h {cross-x86,cross-x86-win}/i686-none-linux-android.h {cross-x86_64,cross-x86_64-win}/x86_64-none-linux-android.h Offsets header generation doesn't work on Linux atm because of missing support for it in the Mono utility used to generate the offsets. Hopefully this limitation will be removed in the near future and a start-to-end build of everything will be possible on Linux. It is now mandatory to run at least `make prepare-props` before Xamarin.Android can be built. The target generates the OS-specific props file which is required by the build. `make prepare` depends on the target.
2016-07-26 16:27:31 +03:00
Files="$(IntermediateOutputPath)\%(_MonoRuntime.Identity)\.stamp"
[mono-runtimes] Build required mono runtimes. Add a new build-tools/mono-runtimes project to build the required Mono runtimes for Xamarin.Android app execution on Android devices. Xamarin.Android 6.0 ("cycle 6") includes Mono runtimes compiled for five architectures: * armeabi (armv5) * armeabi-v7a (armv7) * arm64-v8a (aarch64) * x86 * x86_64 An additional "host" architecture is also built to build the "monodroid" profile assemblies such as mscorlib.dll. In the interest of expediency and not requiring that *six* different runtimes be built -- which can greatly increase build times and storage requirements -- the initial xamarin-android open-source build will only build the armeabi-v7a (armv7) and "host" architectures. Additionally, the <UnzipDirectoryChildren/> task needed to be altered to use unzip(1) on OS X instead of using System.IO.Compression.ZipFile, because the ZipFile implementation included with Mono 4.4 doesn't respect file attributes such as the +x, causing all files to be extracted as 644 (rw-r--r--), resulting in an NDK toolchain which wouldn't work. To resolve this issue, execution on OS X and Linux now uses unzip(1). ~~ Adding new architectures ~~ To build Mono for a new architecture, edit build-tools/mono-runtimes/mono-runtimes.projitems and add a new @(_MonoRuntime) entry for the desired architecture. TODO: Figure out if there's a reasonable way to "opt-in" or "opt-out" of CPU architectures so that it isn't an "all or nothing" prospect.
2016-04-20 04:30:00 +03:00
AlwaysCreate="True"
/>
</Target>
[mxe] Add Windows cross-compiler support. (#55) Certain Xamarin.Android features require that Mono be built for Windows, e.g. the [AOT compilers][aot] require a build of mono that executes on Windows to generate the AOT native libraries. Unfortunately, building Mono on Windows continues to be a massive PITA. (Autotools on Windows requires Cygwin/mingw, running shell scripts on Windows is painfully slow, it's all brittle, etc.) To work around this pain, we instead build the Mono/Windows binaries on OS X, via [MXE][mxe], which produces a gcc-based cross-compiler which generates Windows binaries and is executable from Unix. This in turn requires that we have MXE, so add a `_CreateMxeToolchains` target to `android-toolchain.targets` which will build MXE. The installation prefix for MXE can be overridden via the new `$(AndroidMxeInstallPrefix)` MSBuild property; it defaults to `$HOME/android-toolchain/mxe`. Rework the `$(AndroidSupportedAbis)` MSBuild property so that it must include the "host" ABI, and add support for a new `host-win64` value which will use MXE to generate 64-bit Windows binaries for libmonosgen-2.0.dll and libMonoPosixHelper.dll. We can't always process `host-$(HostOS)` because of an xbuild bug. The scenario is that if you want to just build `host-win64`, the obvious thing to do is: cd build-tools/mono-runtimes xbuild /p:AndroidSupportedAbis=host-win64 Alas, if `host-$(HostOS)` is always processed, this inexplicably causes `host-$(HostOS)` to be re-rebuilt, which (1) is a waste of time, and (2) fails -- inexplicably -- in the `_BuildRuntimes` target because make(1) thinks that the configure flags have somehow changed, which currently makes no sense at all. (When can we move to MSBuild?) Changing `$(AndroidSupportedAbis)` so that `host-$(HostOS)` is explicitly processed instead of implicitly processed allows working around the above xbuild bug, as `host-$(HostOS)` won't be implicitly processed on every build, but only when required. Additionally, we add a new <Which/> MSBuild task so that we can determine if a particular program is in `$PATH`. This is useful because listing requirements within README.md is a road to pain -- e.g. xxd(1) is required to build `src/monodroid` but if it's missing it'll still *build* but you'll instead get a *linker* failure because the `monodroid_config` and `monodroid_machine_config` symbols aren't present. Building MXE requires that even more programs be present within $PATH, so explicitly check for these so that *useful* error messages can be generated instead of obscure ones. Finally, a note about autotools and generating Windows native libraries: creation of `.dll` files *requires* that an appropriate objdump be present so it can determine if e.g. `libkernel32.a` is an import library or an archive. If `x86_64-w64-mingw32.static-objdump` isn't found -- e.g. because $PATH doesn't contain it -- then no `.dll` files will be created, and much head scratching will occur. To rectify this, override the OBJDUMP and DLLTOOL values when invoking `configure` so that that full paths are used and `$PATH` use is reduced. (Who wants `x86_64-w64-mingw32.static-objdump` in `$PATH`?) [aot]: https://developer.xamarin.com/releases/android/xamarin.android_5/xamarin.android_5.1/#AOT_Support [mxe]: http://mxe.cc/
2016-06-07 00:12:49 +03:00
<Target Name="_GetRuntimesOutputItems">
<ItemGroup>
[build] Build the monodroid-profile BCL assemblies reliably (#135) Commit 37b5fd51 broke `make jenkins` (and possible `make all`?) through the introduction of the `%(_MonoRuntime.DoBuild)` metadata: `%(_MonoRuntime.DoBuild)` wasn't added everywhere required. In particular, the `host-Darwin` item didn't gain `<DoBuild/>`, meaning *`host-Darwin` wasn't built*. This in turn meant that the monodroid BCL wasn't built (!) which caused a build break much later within `src/Mono.Android`: Executing: mono .../xamarin-android/external/Java.Interop/bin/Debug/jcw-gen.exe -o ".../xamarin-android/src/Mono.Android/obj/Debug/android-23/jcw/src" -L ".../xamarin-androidg/lib/xbuild-frameworks/MonoAndroid/v6.0/" -L ".../xamarin-android/src/Mono.Android/../../bin/Debug/lib/xbuild-frameworks/MonoAndroid/v6.0/../v1.0/" -L ".../xamarin-android/src/Mono.Android/../../MonoAndroid/v6.0/../v1.0/Facades" ".../xamarin-android/src/Mono.Android/../../bin/Debug/lib/xbuild-frameworks/MonoAndroid/v6.0/Mono.Android.dll" jcw-gen: Could not load assembly 'mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089'. Perhaps it doesn't exist in the Mono for Android profile? `mscorlib.dll` couldn't be found because it wasn't *built*, nor installed into `bin/Debug/lib/xbuild-frameworks/MonoAndroid/v1.0`. Additionally, fully building the AOT cross compilers requires adding `mxe-Win32` to `$(ALL_HOST_ABIS)`, otherwise e.g. `win-armeabi` fails to build because `arm_dpimacros.h` doesn't exist (as `arm_dpimacros.h` is built by `mxe-Win32`): Error parsing '.../xamarin-android/external/mono/mono/metadata/metadata-cross-helpers.c' .../xamarin-android/external/mono/mono/arch/arm/arm-codegen.h(937,10): fatal: 'mono/arch/arm/arm_dpimacros.h' file not found Error parsing '.../xamarin-android/external/mono/mono/mini/mini-cross-helpers.c' .../xamarin-android/external/mono/mono/arch/arm/arm-codegen.h(937,10): fatal: 'mono/arch/arm/arm_dpimacros.h' file not found Unhandled Exception: System.Exception: Expected to find struct definition for MonoObject Fix the `mono-runtimes.mdproj` build so that it doesn't re-execute targets when not necessary, by "rolling" with xbuild limitations and refactoring item groups as appropriate. In particular, xbuild doesn't like referencing item metadata within a `//Target/@Outputs` that in turn references a property group: <PropertyGroup> <V>Value</v> </PropertyGroup> <ItemGroup> <Item Include="Something"> <Metadata>$(V)</Metadata> </Item> </ItemGroup> <Target Name="Example" Inputs="..." Outputs="%(Item.Metadata)"> ... </Target> Whenever the `Example` target is run, it will *always* run, even if `Value` exists, because `xbuild` perceives the `//Target[@Name='Example']/@Outputs` value as being `$(V)`, *not* `Value`, which is wonderfully broken. To compensate, Don't Do That™; in particular, item metadata used within `//Target/@Outputs` *must* be hardcoded and not contain a property reference. This "hardcoding" removes the need for the `$(_Cross*ffsetsHeaderFile)` properties, so remove them. Remove `System.Net.Http.WebRequest.dll` from `@(_BclAssembly)`, as it is no longer built in the monodroid profile; requiring it as an output means that the "host" mono build is *always* built, which isn't needed. Improve the `_BuildRuntimes` and `_InstallRuntimes` targets so that their referenced files aren't *so* repetitive, in particular by "splitting out" `@(_RuntimeLibraries)` into `@(_RuntimeSource)`, `@(_ProfilerSource)`, and `@(_MonoPosixHelperSource)`. This allows more intelligently `strip`ing the native libraries in the Release configuration, while also maintaining generation of the `.d.so` files.
2016-07-28 13:24:18 +03:00
<_RuntimeSource
Condition=" '%(_MonoRuntime.DoBuild)' == 'True' "
[mono-runtimes] Use directory separators with `$(OutputPath)`. [The build][0] [is broken][1]: xamarin-android/build-tools/mono-runtimes/mono-runtimes.targets: error : Could not find a part of the path ".../xamarin-android/xamarin-android/bin/Debuglib/xbuild-frameworks/MonoAndroid/v1.0/RedistList/FrameworkList.xml". The error is generated from the `_InstallBcl` target when writing `FrameworkList.xml`, and a close reading of the error message shows that a directory separator is missing: it should be `bin/Debug/lib`, but is instead `bin/Debuglib`. I *assume* that this is due to `<CallTarget/>` being "flakey" within xbuild (it passes the file containing the target to `<MSBuild/>`!), and *somehow* in that "flakiness" `$(OutputPath)` doesn't end with `\`, *which it should*, because `Microsoft.Common.targets` will automatically append `\` to `$(OutputPath)`. To work around this issue, update `mono-runtimes.targets` so that it *always* uses `\` after `$(OutputPath)` and `$(IntermediateOutputPath)`. Additionally, `llc.exe` was being installed into the wrong directory, preventing the bundle from being used: Target _BuildUnlessCached needs to be built as output file '../../bin/Debug/lib/mandroid/llc.exe' does not exist. `llc.exe` was instead installed into `bin/Debug/bin/llc.exe`. Bump the bundle version number to -v3 and correct `mono-runtimes.targets` so that `llc.exe` and `opt.exe` are properly installed into `$prefix/lib/mandroid`. [0]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android/49/ [1]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android/49/consoleText
2016-08-17 04:18:15 +03:00
Include="@(_MonoRuntime->'$(IntermediateOutputPath)\%(Identity)\mono\mini\.libs\%(OutputRuntimeFilename).%(NativeLibraryExtension)')"
[build] Build the monodroid-profile BCL assemblies reliably (#135) Commit 37b5fd51 broke `make jenkins` (and possible `make all`?) through the introduction of the `%(_MonoRuntime.DoBuild)` metadata: `%(_MonoRuntime.DoBuild)` wasn't added everywhere required. In particular, the `host-Darwin` item didn't gain `<DoBuild/>`, meaning *`host-Darwin` wasn't built*. This in turn meant that the monodroid BCL wasn't built (!) which caused a build break much later within `src/Mono.Android`: Executing: mono .../xamarin-android/external/Java.Interop/bin/Debug/jcw-gen.exe -o ".../xamarin-android/src/Mono.Android/obj/Debug/android-23/jcw/src" -L ".../xamarin-androidg/lib/xbuild-frameworks/MonoAndroid/v6.0/" -L ".../xamarin-android/src/Mono.Android/../../bin/Debug/lib/xbuild-frameworks/MonoAndroid/v6.0/../v1.0/" -L ".../xamarin-android/src/Mono.Android/../../MonoAndroid/v6.0/../v1.0/Facades" ".../xamarin-android/src/Mono.Android/../../bin/Debug/lib/xbuild-frameworks/MonoAndroid/v6.0/Mono.Android.dll" jcw-gen: Could not load assembly 'mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089'. Perhaps it doesn't exist in the Mono for Android profile? `mscorlib.dll` couldn't be found because it wasn't *built*, nor installed into `bin/Debug/lib/xbuild-frameworks/MonoAndroid/v1.0`. Additionally, fully building the AOT cross compilers requires adding `mxe-Win32` to `$(ALL_HOST_ABIS)`, otherwise e.g. `win-armeabi` fails to build because `arm_dpimacros.h` doesn't exist (as `arm_dpimacros.h` is built by `mxe-Win32`): Error parsing '.../xamarin-android/external/mono/mono/metadata/metadata-cross-helpers.c' .../xamarin-android/external/mono/mono/arch/arm/arm-codegen.h(937,10): fatal: 'mono/arch/arm/arm_dpimacros.h' file not found Error parsing '.../xamarin-android/external/mono/mono/mini/mini-cross-helpers.c' .../xamarin-android/external/mono/mono/arch/arm/arm-codegen.h(937,10): fatal: 'mono/arch/arm/arm_dpimacros.h' file not found Unhandled Exception: System.Exception: Expected to find struct definition for MonoObject Fix the `mono-runtimes.mdproj` build so that it doesn't re-execute targets when not necessary, by "rolling" with xbuild limitations and refactoring item groups as appropriate. In particular, xbuild doesn't like referencing item metadata within a `//Target/@Outputs` that in turn references a property group: <PropertyGroup> <V>Value</v> </PropertyGroup> <ItemGroup> <Item Include="Something"> <Metadata>$(V)</Metadata> </Item> </ItemGroup> <Target Name="Example" Inputs="..." Outputs="%(Item.Metadata)"> ... </Target> Whenever the `Example` target is run, it will *always* run, even if `Value` exists, because `xbuild` perceives the `//Target[@Name='Example']/@Outputs` value as being `$(V)`, *not* `Value`, which is wonderfully broken. To compensate, Don't Do That™; in particular, item metadata used within `//Target/@Outputs` *must* be hardcoded and not contain a property reference. This "hardcoding" removes the need for the `$(_Cross*ffsetsHeaderFile)` properties, so remove them. Remove `System.Net.Http.WebRequest.dll` from `@(_BclAssembly)`, as it is no longer built in the monodroid profile; requiring it as an output means that the "host" mono build is *always* built, which isn't needed. Improve the `_BuildRuntimes` and `_InstallRuntimes` targets so that their referenced files aren't *so* repetitive, in particular by "splitting out" `@(_RuntimeLibraries)` into `@(_RuntimeSource)`, `@(_ProfilerSource)`, and `@(_MonoPosixHelperSource)`. This allows more intelligently `strip`ing the native libraries in the Release configuration, while also maintaining generation of the `.d.so` files.
2016-07-28 13:24:18 +03:00
/>
<_InstallRuntimeOutput
Condition=" '%(_MonoRuntime.DoBuild)' == 'True' "
[structure] Rework installation directory structure (#704) Context: https://github.com/xamarin/xamarin-android/pull/253#discussion_r82862993 The *intention* is that Jenkins-produced `oss-xamarin.android*.zip` artifacts be usable on Windows, so that side-by-side testing can be performed without replacing the system installation. Usage is in [UsingJenkinsBuildArtifacts.md](Documentation/UsingJenkinsBuildArtifacts). This isn't *entirely* the case. It was *apparently* the case at the time of commit 87ca2737, but things have bitrotten since. For example, following the 87ca2737 instructions would currently result in an XA0020 `Could not find monodroid` error, because `class-parse.exe` wasn't where Windows expects it to be (it was in `lib/mandroid`, not `lib/xbuild/Xamarin/Android`). This needs to be fixed. Additionally, PR #253 mentions that, for filesystem organization, it would be useful if the macOS/Linux directory structure -- `$prefix/bin`, `$prefix/lib/mandroid`, `$prefix/lib/xbuild/Xamarin/Android`, `$prefix/lib/xbuild-frameworks` -- more closely resembled the Windows directory structure of `$MSBuild` and `$ReferenceAssemblies` (as seen in `.vsix` files). This would turn macOS/Linux into using `$xa_prefix/xbuild` and `$xa_prefix/xbuild-frameworks` directories. `$prefix/bin` would only contain `xabuild`. What is currently in `$prefix/lib/mandroid` would be merged with `$xa_prefix/xbuild/Xamarin/Android`. `$xa_prefix` would `$prefix/lib/xamarin.android`. This would turn the current macOS structure: $prefix/bin/xabuild $prefix/bin/generator $prefix/bin/cross-arm $prefix/lib/mandroid/generator.exe $prefix/lib/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xbuild/Xamarin/Android/Xamarin.Android.Common.targets Into: $prefix/bin/xabuild $prefix/lib/xamarin.android/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xamarin.android/xbuild/Xamarin/Android/generator.exe $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Xamarin.Android.Common.targets $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Darwin/cross-arm Other notes: * The `bundle-*.zip` filename has been has been changed to include a *hash* of the contents of various files, in particular `build-tools\mono-runtimes.*`. This was instigated via a conversation with @kumpera about making the filename more "idiot-proof": `mono-runtimes.props` contains *compiler flags*, and if any of those change, then *logically* the bundle should differ as well, as the mono runtimes may differ in significant ways. In theory, the `-vXX` version should be used to track this, but this is a manual change, easy to overlook. The new `-hHASH` part of the filename should be more automagic. The new `<HashFileContents/>` task in `xa-prep-tasks.dll` is responsible for creating the hash value. * `Configuration.Java.Interop.Override.props` was moved into `build-tools/scripts`, because that would cleanup the root directory a bit. * OS-specific binaries are now placed into `$prefix/lib/xamarin.android/xbuild/Xamarin/Android/$(HostOS)`. On a theoretical plus side, this means that the same build directory can contain OS-specific binaries for multiple operating systems. (I don't know if anyone shares a build tree between e.g. macOS and Linux, but if anyone *does*...) Unfortunately this requires a workaround for an `xbuild` bug: `%(_LlvmRuntime.InstallPath)` and `%(_MonoCrossRuntime.InstallPath)` *cannot* use MSBuild properties, e.g. `<InstallPath>$(HostOS)/</InstallPath>` doesn't work as desired; it's instead treated literally. Special-case `%(InstallPath)` until we fully migrate to MSBuild. * `$(MonoAndroidToolsDirectory)` should be considered *dead*, along with `$(MonoAndroidBinDirectory)`, as these should now *always* be the same directory as where `Xamarin.Android.Build.Tasks.dll` is located, or a `$(HostOS)` sub-directory. * `Xamarin.ProjectTools.csproj` needed to be updated to ensure that the build order was correct. * Remove all `[Obsolete]` and unreferenced members from `Xamarin.Android.Build.Utilities.dll`. There's too much in there, and it makes my head hurt trying to understand the interrelationships between it all. If it's not used, it's gone. * The changes to `src/monodroid/jni/Android.mk` are...weird. The removal of `-I$(topdir)/libmonodroid/zip`/etc. is to reduce use of non-existent paths, as `$(topdir)` isn't defined, so that's *actually* `-I/libmonodroid/zip`, which is nonsensical. So far, so good. What's *odd* is the addition of `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`: This is needed so that `external/mono/support/zlib-helper.c` exports `CreateZStream` and related symbols, otherwise we get a unit test failure in `GzipStreamTest.Compression` due to an `EntryPointNotFoundException`, because `CreateZStream` isn't exported/public. What's odd here is that I don't understand what caused this behavior to change. Previous builds exported `CreateZStream`, otherwise the tests would fail, and I don't understand how any of the other changes in this PR would be at fault, though that's certainly the most plausible explanation. Regardless, `-Ijni` *first* (due to adding `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`) is the desired behavior, so that `jni/config.h` is included, thus ensuring that `MONO_API` has the required definition when building `zlib-helper.c`.
2017-07-28 16:36:32 +03:00
Include="@(_MonoRuntime->'$(_MSBuildDir)\lib\%(Identity)\%(OutputRuntimeFilename).%(NativeLibraryExtension)')"
[build] Build the monodroid-profile BCL assemblies reliably (#135) Commit 37b5fd51 broke `make jenkins` (and possible `make all`?) through the introduction of the `%(_MonoRuntime.DoBuild)` metadata: `%(_MonoRuntime.DoBuild)` wasn't added everywhere required. In particular, the `host-Darwin` item didn't gain `<DoBuild/>`, meaning *`host-Darwin` wasn't built*. This in turn meant that the monodroid BCL wasn't built (!) which caused a build break much later within `src/Mono.Android`: Executing: mono .../xamarin-android/external/Java.Interop/bin/Debug/jcw-gen.exe -o ".../xamarin-android/src/Mono.Android/obj/Debug/android-23/jcw/src" -L ".../xamarin-androidg/lib/xbuild-frameworks/MonoAndroid/v6.0/" -L ".../xamarin-android/src/Mono.Android/../../bin/Debug/lib/xbuild-frameworks/MonoAndroid/v6.0/../v1.0/" -L ".../xamarin-android/src/Mono.Android/../../MonoAndroid/v6.0/../v1.0/Facades" ".../xamarin-android/src/Mono.Android/../../bin/Debug/lib/xbuild-frameworks/MonoAndroid/v6.0/Mono.Android.dll" jcw-gen: Could not load assembly 'mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089'. Perhaps it doesn't exist in the Mono for Android profile? `mscorlib.dll` couldn't be found because it wasn't *built*, nor installed into `bin/Debug/lib/xbuild-frameworks/MonoAndroid/v1.0`. Additionally, fully building the AOT cross compilers requires adding `mxe-Win32` to `$(ALL_HOST_ABIS)`, otherwise e.g. `win-armeabi` fails to build because `arm_dpimacros.h` doesn't exist (as `arm_dpimacros.h` is built by `mxe-Win32`): Error parsing '.../xamarin-android/external/mono/mono/metadata/metadata-cross-helpers.c' .../xamarin-android/external/mono/mono/arch/arm/arm-codegen.h(937,10): fatal: 'mono/arch/arm/arm_dpimacros.h' file not found Error parsing '.../xamarin-android/external/mono/mono/mini/mini-cross-helpers.c' .../xamarin-android/external/mono/mono/arch/arm/arm-codegen.h(937,10): fatal: 'mono/arch/arm/arm_dpimacros.h' file not found Unhandled Exception: System.Exception: Expected to find struct definition for MonoObject Fix the `mono-runtimes.mdproj` build so that it doesn't re-execute targets when not necessary, by "rolling" with xbuild limitations and refactoring item groups as appropriate. In particular, xbuild doesn't like referencing item metadata within a `//Target/@Outputs` that in turn references a property group: <PropertyGroup> <V>Value</v> </PropertyGroup> <ItemGroup> <Item Include="Something"> <Metadata>$(V)</Metadata> </Item> </ItemGroup> <Target Name="Example" Inputs="..." Outputs="%(Item.Metadata)"> ... </Target> Whenever the `Example` target is run, it will *always* run, even if `Value` exists, because `xbuild` perceives the `//Target[@Name='Example']/@Outputs` value as being `$(V)`, *not* `Value`, which is wonderfully broken. To compensate, Don't Do That™; in particular, item metadata used within `//Target/@Outputs` *must* be hardcoded and not contain a property reference. This "hardcoding" removes the need for the `$(_Cross*ffsetsHeaderFile)` properties, so remove them. Remove `System.Net.Http.WebRequest.dll` from `@(_BclAssembly)`, as it is no longer built in the monodroid profile; requiring it as an output means that the "host" mono build is *always* built, which isn't needed. Improve the `_BuildRuntimes` and `_InstallRuntimes` targets so that their referenced files aren't *so* repetitive, in particular by "splitting out" `@(_RuntimeLibraries)` into `@(_RuntimeSource)`, `@(_ProfilerSource)`, and `@(_MonoPosixHelperSource)`. This allows more intelligently `strip`ing the native libraries in the Release configuration, while also maintaining generation of the `.d.so` files.
2016-07-28 13:24:18 +03:00
/>
<_InstallUnstrippedRuntimeOutput
Condition=" '%(_MonoRuntime.DoBuild)' == 'True' "
[structure] Rework installation directory structure (#704) Context: https://github.com/xamarin/xamarin-android/pull/253#discussion_r82862993 The *intention* is that Jenkins-produced `oss-xamarin.android*.zip` artifacts be usable on Windows, so that side-by-side testing can be performed without replacing the system installation. Usage is in [UsingJenkinsBuildArtifacts.md](Documentation/UsingJenkinsBuildArtifacts). This isn't *entirely* the case. It was *apparently* the case at the time of commit 87ca2737, but things have bitrotten since. For example, following the 87ca2737 instructions would currently result in an XA0020 `Could not find monodroid` error, because `class-parse.exe` wasn't where Windows expects it to be (it was in `lib/mandroid`, not `lib/xbuild/Xamarin/Android`). This needs to be fixed. Additionally, PR #253 mentions that, for filesystem organization, it would be useful if the macOS/Linux directory structure -- `$prefix/bin`, `$prefix/lib/mandroid`, `$prefix/lib/xbuild/Xamarin/Android`, `$prefix/lib/xbuild-frameworks` -- more closely resembled the Windows directory structure of `$MSBuild` and `$ReferenceAssemblies` (as seen in `.vsix` files). This would turn macOS/Linux into using `$xa_prefix/xbuild` and `$xa_prefix/xbuild-frameworks` directories. `$prefix/bin` would only contain `xabuild`. What is currently in `$prefix/lib/mandroid` would be merged with `$xa_prefix/xbuild/Xamarin/Android`. `$xa_prefix` would `$prefix/lib/xamarin.android`. This would turn the current macOS structure: $prefix/bin/xabuild $prefix/bin/generator $prefix/bin/cross-arm $prefix/lib/mandroid/generator.exe $prefix/lib/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xbuild/Xamarin/Android/Xamarin.Android.Common.targets Into: $prefix/bin/xabuild $prefix/lib/xamarin.android/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xamarin.android/xbuild/Xamarin/Android/generator.exe $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Xamarin.Android.Common.targets $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Darwin/cross-arm Other notes: * The `bundle-*.zip` filename has been has been changed to include a *hash* of the contents of various files, in particular `build-tools\mono-runtimes.*`. This was instigated via a conversation with @kumpera about making the filename more "idiot-proof": `mono-runtimes.props` contains *compiler flags*, and if any of those change, then *logically* the bundle should differ as well, as the mono runtimes may differ in significant ways. In theory, the `-vXX` version should be used to track this, but this is a manual change, easy to overlook. The new `-hHASH` part of the filename should be more automagic. The new `<HashFileContents/>` task in `xa-prep-tasks.dll` is responsible for creating the hash value. * `Configuration.Java.Interop.Override.props` was moved into `build-tools/scripts`, because that would cleanup the root directory a bit. * OS-specific binaries are now placed into `$prefix/lib/xamarin.android/xbuild/Xamarin/Android/$(HostOS)`. On a theoretical plus side, this means that the same build directory can contain OS-specific binaries for multiple operating systems. (I don't know if anyone shares a build tree between e.g. macOS and Linux, but if anyone *does*...) Unfortunately this requires a workaround for an `xbuild` bug: `%(_LlvmRuntime.InstallPath)` and `%(_MonoCrossRuntime.InstallPath)` *cannot* use MSBuild properties, e.g. `<InstallPath>$(HostOS)/</InstallPath>` doesn't work as desired; it's instead treated literally. Special-case `%(InstallPath)` until we fully migrate to MSBuild. * `$(MonoAndroidToolsDirectory)` should be considered *dead*, along with `$(MonoAndroidBinDirectory)`, as these should now *always* be the same directory as where `Xamarin.Android.Build.Tasks.dll` is located, or a `$(HostOS)` sub-directory. * `Xamarin.ProjectTools.csproj` needed to be updated to ensure that the build order was correct. * Remove all `[Obsolete]` and unreferenced members from `Xamarin.Android.Build.Utilities.dll`. There's too much in there, and it makes my head hurt trying to understand the interrelationships between it all. If it's not used, it's gone. * The changes to `src/monodroid/jni/Android.mk` are...weird. The removal of `-I$(topdir)/libmonodroid/zip`/etc. is to reduce use of non-existent paths, as `$(topdir)` isn't defined, so that's *actually* `-I/libmonodroid/zip`, which is nonsensical. So far, so good. What's *odd* is the addition of `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`: This is needed so that `external/mono/support/zlib-helper.c` exports `CreateZStream` and related symbols, otherwise we get a unit test failure in `GzipStreamTest.Compression` due to an `EntryPointNotFoundException`, because `CreateZStream` isn't exported/public. What's odd here is that I don't understand what caused this behavior to change. Previous builds exported `CreateZStream`, otherwise the tests would fail, and I don't understand how any of the other changes in this PR would be at fault, though that's certainly the most plausible explanation. Regardless, `-Ijni` *first* (due to adding `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`) is the desired behavior, so that `jni/config.h` is included, thus ensuring that `MONO_API` has the required definition when building `zlib-helper.c`.
2017-07-28 16:36:32 +03:00
Include="@(_MonoRuntime->'$(_MSBuildDir)\lib\%(Identity)\%(OutputRuntimeFilename).d.%(NativeLibraryExtension)')"
[build] Build the monodroid-profile BCL assemblies reliably (#135) Commit 37b5fd51 broke `make jenkins` (and possible `make all`?) through the introduction of the `%(_MonoRuntime.DoBuild)` metadata: `%(_MonoRuntime.DoBuild)` wasn't added everywhere required. In particular, the `host-Darwin` item didn't gain `<DoBuild/>`, meaning *`host-Darwin` wasn't built*. This in turn meant that the monodroid BCL wasn't built (!) which caused a build break much later within `src/Mono.Android`: Executing: mono .../xamarin-android/external/Java.Interop/bin/Debug/jcw-gen.exe -o ".../xamarin-android/src/Mono.Android/obj/Debug/android-23/jcw/src" -L ".../xamarin-androidg/lib/xbuild-frameworks/MonoAndroid/v6.0/" -L ".../xamarin-android/src/Mono.Android/../../bin/Debug/lib/xbuild-frameworks/MonoAndroid/v6.0/../v1.0/" -L ".../xamarin-android/src/Mono.Android/../../MonoAndroid/v6.0/../v1.0/Facades" ".../xamarin-android/src/Mono.Android/../../bin/Debug/lib/xbuild-frameworks/MonoAndroid/v6.0/Mono.Android.dll" jcw-gen: Could not load assembly 'mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089'. Perhaps it doesn't exist in the Mono for Android profile? `mscorlib.dll` couldn't be found because it wasn't *built*, nor installed into `bin/Debug/lib/xbuild-frameworks/MonoAndroid/v1.0`. Additionally, fully building the AOT cross compilers requires adding `mxe-Win32` to `$(ALL_HOST_ABIS)`, otherwise e.g. `win-armeabi` fails to build because `arm_dpimacros.h` doesn't exist (as `arm_dpimacros.h` is built by `mxe-Win32`): Error parsing '.../xamarin-android/external/mono/mono/metadata/metadata-cross-helpers.c' .../xamarin-android/external/mono/mono/arch/arm/arm-codegen.h(937,10): fatal: 'mono/arch/arm/arm_dpimacros.h' file not found Error parsing '.../xamarin-android/external/mono/mono/mini/mini-cross-helpers.c' .../xamarin-android/external/mono/mono/arch/arm/arm-codegen.h(937,10): fatal: 'mono/arch/arm/arm_dpimacros.h' file not found Unhandled Exception: System.Exception: Expected to find struct definition for MonoObject Fix the `mono-runtimes.mdproj` build so that it doesn't re-execute targets when not necessary, by "rolling" with xbuild limitations and refactoring item groups as appropriate. In particular, xbuild doesn't like referencing item metadata within a `//Target/@Outputs` that in turn references a property group: <PropertyGroup> <V>Value</v> </PropertyGroup> <ItemGroup> <Item Include="Something"> <Metadata>$(V)</Metadata> </Item> </ItemGroup> <Target Name="Example" Inputs="..." Outputs="%(Item.Metadata)"> ... </Target> Whenever the `Example` target is run, it will *always* run, even if `Value` exists, because `xbuild` perceives the `//Target[@Name='Example']/@Outputs` value as being `$(V)`, *not* `Value`, which is wonderfully broken. To compensate, Don't Do That™; in particular, item metadata used within `//Target/@Outputs` *must* be hardcoded and not contain a property reference. This "hardcoding" removes the need for the `$(_Cross*ffsetsHeaderFile)` properties, so remove them. Remove `System.Net.Http.WebRequest.dll` from `@(_BclAssembly)`, as it is no longer built in the monodroid profile; requiring it as an output means that the "host" mono build is *always* built, which isn't needed. Improve the `_BuildRuntimes` and `_InstallRuntimes` targets so that their referenced files aren't *so* repetitive, in particular by "splitting out" `@(_RuntimeLibraries)` into `@(_RuntimeSource)`, `@(_ProfilerSource)`, and `@(_MonoPosixHelperSource)`. This allows more intelligently `strip`ing the native libraries in the Release configuration, while also maintaining generation of the `.d.so` files.
2016-07-28 13:24:18 +03:00
/>
<_ProfilerSource
Condition=" '%(_MonoRuntime.DoBuild)' == 'True' And '%(_MonoRuntime.OutputProfilerFilename)' != '' "
[mono-runtimes] Use directory separators with `$(OutputPath)`. [The build][0] [is broken][1]: xamarin-android/build-tools/mono-runtimes/mono-runtimes.targets: error : Could not find a part of the path ".../xamarin-android/xamarin-android/bin/Debuglib/xbuild-frameworks/MonoAndroid/v1.0/RedistList/FrameworkList.xml". The error is generated from the `_InstallBcl` target when writing `FrameworkList.xml`, and a close reading of the error message shows that a directory separator is missing: it should be `bin/Debug/lib`, but is instead `bin/Debuglib`. I *assume* that this is due to `<CallTarget/>` being "flakey" within xbuild (it passes the file containing the target to `<MSBuild/>`!), and *somehow* in that "flakiness" `$(OutputPath)` doesn't end with `\`, *which it should*, because `Microsoft.Common.targets` will automatically append `\` to `$(OutputPath)`. To work around this issue, update `mono-runtimes.targets` so that it *always* uses `\` after `$(OutputPath)` and `$(IntermediateOutputPath)`. Additionally, `llc.exe` was being installed into the wrong directory, preventing the bundle from being used: Target _BuildUnlessCached needs to be built as output file '../../bin/Debug/lib/mandroid/llc.exe' does not exist. `llc.exe` was instead installed into `bin/Debug/bin/llc.exe`. Bump the bundle version number to -v3 and correct `mono-runtimes.targets` so that `llc.exe` and `opt.exe` are properly installed into `$prefix/lib/mandroid`. [0]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android/49/ [1]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android/49/consoleText
2016-08-17 04:18:15 +03:00
Include="@(_MonoRuntime->'$(IntermediateOutputPath)\%(Identity)\mono\profiler\.libs\%(OutputProfilerFilename).%(NativeLibraryExtension)')"
[mxe] Add Windows cross-compiler support. (#55) Certain Xamarin.Android features require that Mono be built for Windows, e.g. the [AOT compilers][aot] require a build of mono that executes on Windows to generate the AOT native libraries. Unfortunately, building Mono on Windows continues to be a massive PITA. (Autotools on Windows requires Cygwin/mingw, running shell scripts on Windows is painfully slow, it's all brittle, etc.) To work around this pain, we instead build the Mono/Windows binaries on OS X, via [MXE][mxe], which produces a gcc-based cross-compiler which generates Windows binaries and is executable from Unix. This in turn requires that we have MXE, so add a `_CreateMxeToolchains` target to `android-toolchain.targets` which will build MXE. The installation prefix for MXE can be overridden via the new `$(AndroidMxeInstallPrefix)` MSBuild property; it defaults to `$HOME/android-toolchain/mxe`. Rework the `$(AndroidSupportedAbis)` MSBuild property so that it must include the "host" ABI, and add support for a new `host-win64` value which will use MXE to generate 64-bit Windows binaries for libmonosgen-2.0.dll and libMonoPosixHelper.dll. We can't always process `host-$(HostOS)` because of an xbuild bug. The scenario is that if you want to just build `host-win64`, the obvious thing to do is: cd build-tools/mono-runtimes xbuild /p:AndroidSupportedAbis=host-win64 Alas, if `host-$(HostOS)` is always processed, this inexplicably causes `host-$(HostOS)` to be re-rebuilt, which (1) is a waste of time, and (2) fails -- inexplicably -- in the `_BuildRuntimes` target because make(1) thinks that the configure flags have somehow changed, which currently makes no sense at all. (When can we move to MSBuild?) Changing `$(AndroidSupportedAbis)` so that `host-$(HostOS)` is explicitly processed instead of implicitly processed allows working around the above xbuild bug, as `host-$(HostOS)` won't be implicitly processed on every build, but only when required. Additionally, we add a new <Which/> MSBuild task so that we can determine if a particular program is in `$PATH`. This is useful because listing requirements within README.md is a road to pain -- e.g. xxd(1) is required to build `src/monodroid` but if it's missing it'll still *build* but you'll instead get a *linker* failure because the `monodroid_config` and `monodroid_machine_config` symbols aren't present. Building MXE requires that even more programs be present within $PATH, so explicitly check for these so that *useful* error messages can be generated instead of obscure ones. Finally, a note about autotools and generating Windows native libraries: creation of `.dll` files *requires* that an appropriate objdump be present so it can determine if e.g. `libkernel32.a` is an import library or an archive. If `x86_64-w64-mingw32.static-objdump` isn't found -- e.g. because $PATH doesn't contain it -- then no `.dll` files will be created, and much head scratching will occur. To rectify this, override the OBJDUMP and DLLTOOL values when invoking `configure` so that that full paths are used and `$PATH` use is reduced. (Who wants `x86_64-w64-mingw32.static-objdump` in `$PATH`?) [aot]: https://developer.xamarin.com/releases/android/xamarin.android_5/xamarin.android_5.1/#AOT_Support [mxe]: http://mxe.cc/
2016-06-07 00:12:49 +03:00
/>
[build] Build the monodroid-profile BCL assemblies reliably (#135) Commit 37b5fd51 broke `make jenkins` (and possible `make all`?) through the introduction of the `%(_MonoRuntime.DoBuild)` metadata: `%(_MonoRuntime.DoBuild)` wasn't added everywhere required. In particular, the `host-Darwin` item didn't gain `<DoBuild/>`, meaning *`host-Darwin` wasn't built*. This in turn meant that the monodroid BCL wasn't built (!) which caused a build break much later within `src/Mono.Android`: Executing: mono .../xamarin-android/external/Java.Interop/bin/Debug/jcw-gen.exe -o ".../xamarin-android/src/Mono.Android/obj/Debug/android-23/jcw/src" -L ".../xamarin-androidg/lib/xbuild-frameworks/MonoAndroid/v6.0/" -L ".../xamarin-android/src/Mono.Android/../../bin/Debug/lib/xbuild-frameworks/MonoAndroid/v6.0/../v1.0/" -L ".../xamarin-android/src/Mono.Android/../../MonoAndroid/v6.0/../v1.0/Facades" ".../xamarin-android/src/Mono.Android/../../bin/Debug/lib/xbuild-frameworks/MonoAndroid/v6.0/Mono.Android.dll" jcw-gen: Could not load assembly 'mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089'. Perhaps it doesn't exist in the Mono for Android profile? `mscorlib.dll` couldn't be found because it wasn't *built*, nor installed into `bin/Debug/lib/xbuild-frameworks/MonoAndroid/v1.0`. Additionally, fully building the AOT cross compilers requires adding `mxe-Win32` to `$(ALL_HOST_ABIS)`, otherwise e.g. `win-armeabi` fails to build because `arm_dpimacros.h` doesn't exist (as `arm_dpimacros.h` is built by `mxe-Win32`): Error parsing '.../xamarin-android/external/mono/mono/metadata/metadata-cross-helpers.c' .../xamarin-android/external/mono/mono/arch/arm/arm-codegen.h(937,10): fatal: 'mono/arch/arm/arm_dpimacros.h' file not found Error parsing '.../xamarin-android/external/mono/mono/mini/mini-cross-helpers.c' .../xamarin-android/external/mono/mono/arch/arm/arm-codegen.h(937,10): fatal: 'mono/arch/arm/arm_dpimacros.h' file not found Unhandled Exception: System.Exception: Expected to find struct definition for MonoObject Fix the `mono-runtimes.mdproj` build so that it doesn't re-execute targets when not necessary, by "rolling" with xbuild limitations and refactoring item groups as appropriate. In particular, xbuild doesn't like referencing item metadata within a `//Target/@Outputs` that in turn references a property group: <PropertyGroup> <V>Value</v> </PropertyGroup> <ItemGroup> <Item Include="Something"> <Metadata>$(V)</Metadata> </Item> </ItemGroup> <Target Name="Example" Inputs="..." Outputs="%(Item.Metadata)"> ... </Target> Whenever the `Example` target is run, it will *always* run, even if `Value` exists, because `xbuild` perceives the `//Target[@Name='Example']/@Outputs` value as being `$(V)`, *not* `Value`, which is wonderfully broken. To compensate, Don't Do That™; in particular, item metadata used within `//Target/@Outputs` *must* be hardcoded and not contain a property reference. This "hardcoding" removes the need for the `$(_Cross*ffsetsHeaderFile)` properties, so remove them. Remove `System.Net.Http.WebRequest.dll` from `@(_BclAssembly)`, as it is no longer built in the monodroid profile; requiring it as an output means that the "host" mono build is *always* built, which isn't needed. Improve the `_BuildRuntimes` and `_InstallRuntimes` targets so that their referenced files aren't *so* repetitive, in particular by "splitting out" `@(_RuntimeLibraries)` into `@(_RuntimeSource)`, `@(_ProfilerSource)`, and `@(_MonoPosixHelperSource)`. This allows more intelligently `strip`ing the native libraries in the Release configuration, while also maintaining generation of the `.d.so` files.
2016-07-28 13:24:18 +03:00
<_InstallProfilerOutput
Condition=" '%(_MonoRuntime.DoBuild)' == 'True' And '%(_MonoRuntime.OutputProfilerFilename)' != '' "
[structure] Rework installation directory structure (#704) Context: https://github.com/xamarin/xamarin-android/pull/253#discussion_r82862993 The *intention* is that Jenkins-produced `oss-xamarin.android*.zip` artifacts be usable on Windows, so that side-by-side testing can be performed without replacing the system installation. Usage is in [UsingJenkinsBuildArtifacts.md](Documentation/UsingJenkinsBuildArtifacts). This isn't *entirely* the case. It was *apparently* the case at the time of commit 87ca2737, but things have bitrotten since. For example, following the 87ca2737 instructions would currently result in an XA0020 `Could not find monodroid` error, because `class-parse.exe` wasn't where Windows expects it to be (it was in `lib/mandroid`, not `lib/xbuild/Xamarin/Android`). This needs to be fixed. Additionally, PR #253 mentions that, for filesystem organization, it would be useful if the macOS/Linux directory structure -- `$prefix/bin`, `$prefix/lib/mandroid`, `$prefix/lib/xbuild/Xamarin/Android`, `$prefix/lib/xbuild-frameworks` -- more closely resembled the Windows directory structure of `$MSBuild` and `$ReferenceAssemblies` (as seen in `.vsix` files). This would turn macOS/Linux into using `$xa_prefix/xbuild` and `$xa_prefix/xbuild-frameworks` directories. `$prefix/bin` would only contain `xabuild`. What is currently in `$prefix/lib/mandroid` would be merged with `$xa_prefix/xbuild/Xamarin/Android`. `$xa_prefix` would `$prefix/lib/xamarin.android`. This would turn the current macOS structure: $prefix/bin/xabuild $prefix/bin/generator $prefix/bin/cross-arm $prefix/lib/mandroid/generator.exe $prefix/lib/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xbuild/Xamarin/Android/Xamarin.Android.Common.targets Into: $prefix/bin/xabuild $prefix/lib/xamarin.android/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xamarin.android/xbuild/Xamarin/Android/generator.exe $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Xamarin.Android.Common.targets $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Darwin/cross-arm Other notes: * The `bundle-*.zip` filename has been has been changed to include a *hash* of the contents of various files, in particular `build-tools\mono-runtimes.*`. This was instigated via a conversation with @kumpera about making the filename more "idiot-proof": `mono-runtimes.props` contains *compiler flags*, and if any of those change, then *logically* the bundle should differ as well, as the mono runtimes may differ in significant ways. In theory, the `-vXX` version should be used to track this, but this is a manual change, easy to overlook. The new `-hHASH` part of the filename should be more automagic. The new `<HashFileContents/>` task in `xa-prep-tasks.dll` is responsible for creating the hash value. * `Configuration.Java.Interop.Override.props` was moved into `build-tools/scripts`, because that would cleanup the root directory a bit. * OS-specific binaries are now placed into `$prefix/lib/xamarin.android/xbuild/Xamarin/Android/$(HostOS)`. On a theoretical plus side, this means that the same build directory can contain OS-specific binaries for multiple operating systems. (I don't know if anyone shares a build tree between e.g. macOS and Linux, but if anyone *does*...) Unfortunately this requires a workaround for an `xbuild` bug: `%(_LlvmRuntime.InstallPath)` and `%(_MonoCrossRuntime.InstallPath)` *cannot* use MSBuild properties, e.g. `<InstallPath>$(HostOS)/</InstallPath>` doesn't work as desired; it's instead treated literally. Special-case `%(InstallPath)` until we fully migrate to MSBuild. * `$(MonoAndroidToolsDirectory)` should be considered *dead*, along with `$(MonoAndroidBinDirectory)`, as these should now *always* be the same directory as where `Xamarin.Android.Build.Tasks.dll` is located, or a `$(HostOS)` sub-directory. * `Xamarin.ProjectTools.csproj` needed to be updated to ensure that the build order was correct. * Remove all `[Obsolete]` and unreferenced members from `Xamarin.Android.Build.Utilities.dll`. There's too much in there, and it makes my head hurt trying to understand the interrelationships between it all. If it's not used, it's gone. * The changes to `src/monodroid/jni/Android.mk` are...weird. The removal of `-I$(topdir)/libmonodroid/zip`/etc. is to reduce use of non-existent paths, as `$(topdir)` isn't defined, so that's *actually* `-I/libmonodroid/zip`, which is nonsensical. So far, so good. What's *odd* is the addition of `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`: This is needed so that `external/mono/support/zlib-helper.c` exports `CreateZStream` and related symbols, otherwise we get a unit test failure in `GzipStreamTest.Compression` due to an `EntryPointNotFoundException`, because `CreateZStream` isn't exported/public. What's odd here is that I don't understand what caused this behavior to change. Previous builds exported `CreateZStream`, otherwise the tests would fail, and I don't understand how any of the other changes in this PR would be at fault, though that's certainly the most plausible explanation. Regardless, `-Ijni` *first* (due to adding `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`) is the desired behavior, so that `jni/config.h` is included, thus ensuring that `MONO_API` has the required definition when building `zlib-helper.c`.
2017-07-28 16:36:32 +03:00
Include="@(_MonoRuntime->'$(_MSBuildDir)\lib\%(Identity)\%(OutputProfilerFilename).%(NativeLibraryExtension)')"
[mxe] Add Windows cross-compiler support. (#55) Certain Xamarin.Android features require that Mono be built for Windows, e.g. the [AOT compilers][aot] require a build of mono that executes on Windows to generate the AOT native libraries. Unfortunately, building Mono on Windows continues to be a massive PITA. (Autotools on Windows requires Cygwin/mingw, running shell scripts on Windows is painfully slow, it's all brittle, etc.) To work around this pain, we instead build the Mono/Windows binaries on OS X, via [MXE][mxe], which produces a gcc-based cross-compiler which generates Windows binaries and is executable from Unix. This in turn requires that we have MXE, so add a `_CreateMxeToolchains` target to `android-toolchain.targets` which will build MXE. The installation prefix for MXE can be overridden via the new `$(AndroidMxeInstallPrefix)` MSBuild property; it defaults to `$HOME/android-toolchain/mxe`. Rework the `$(AndroidSupportedAbis)` MSBuild property so that it must include the "host" ABI, and add support for a new `host-win64` value which will use MXE to generate 64-bit Windows binaries for libmonosgen-2.0.dll and libMonoPosixHelper.dll. We can't always process `host-$(HostOS)` because of an xbuild bug. The scenario is that if you want to just build `host-win64`, the obvious thing to do is: cd build-tools/mono-runtimes xbuild /p:AndroidSupportedAbis=host-win64 Alas, if `host-$(HostOS)` is always processed, this inexplicably causes `host-$(HostOS)` to be re-rebuilt, which (1) is a waste of time, and (2) fails -- inexplicably -- in the `_BuildRuntimes` target because make(1) thinks that the configure flags have somehow changed, which currently makes no sense at all. (When can we move to MSBuild?) Changing `$(AndroidSupportedAbis)` so that `host-$(HostOS)` is explicitly processed instead of implicitly processed allows working around the above xbuild bug, as `host-$(HostOS)` won't be implicitly processed on every build, but only when required. Additionally, we add a new <Which/> MSBuild task so that we can determine if a particular program is in `$PATH`. This is useful because listing requirements within README.md is a road to pain -- e.g. xxd(1) is required to build `src/monodroid` but if it's missing it'll still *build* but you'll instead get a *linker* failure because the `monodroid_config` and `monodroid_machine_config` symbols aren't present. Building MXE requires that even more programs be present within $PATH, so explicitly check for these so that *useful* error messages can be generated instead of obscure ones. Finally, a note about autotools and generating Windows native libraries: creation of `.dll` files *requires* that an appropriate objdump be present so it can determine if e.g. `libkernel32.a` is an import library or an archive. If `x86_64-w64-mingw32.static-objdump` isn't found -- e.g. because $PATH doesn't contain it -- then no `.dll` files will be created, and much head scratching will occur. To rectify this, override the OBJDUMP and DLLTOOL values when invoking `configure` so that that full paths are used and `$PATH` use is reduced. (Who wants `x86_64-w64-mingw32.static-objdump` in `$PATH`?) [aot]: https://developer.xamarin.com/releases/android/xamarin.android_5/xamarin.android_5.1/#AOT_Support [mxe]: http://mxe.cc/
2016-06-07 00:12:49 +03:00
/>
[build] Build the monodroid-profile BCL assemblies reliably (#135) Commit 37b5fd51 broke `make jenkins` (and possible `make all`?) through the introduction of the `%(_MonoRuntime.DoBuild)` metadata: `%(_MonoRuntime.DoBuild)` wasn't added everywhere required. In particular, the `host-Darwin` item didn't gain `<DoBuild/>`, meaning *`host-Darwin` wasn't built*. This in turn meant that the monodroid BCL wasn't built (!) which caused a build break much later within `src/Mono.Android`: Executing: mono .../xamarin-android/external/Java.Interop/bin/Debug/jcw-gen.exe -o ".../xamarin-android/src/Mono.Android/obj/Debug/android-23/jcw/src" -L ".../xamarin-androidg/lib/xbuild-frameworks/MonoAndroid/v6.0/" -L ".../xamarin-android/src/Mono.Android/../../bin/Debug/lib/xbuild-frameworks/MonoAndroid/v6.0/../v1.0/" -L ".../xamarin-android/src/Mono.Android/../../MonoAndroid/v6.0/../v1.0/Facades" ".../xamarin-android/src/Mono.Android/../../bin/Debug/lib/xbuild-frameworks/MonoAndroid/v6.0/Mono.Android.dll" jcw-gen: Could not load assembly 'mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089'. Perhaps it doesn't exist in the Mono for Android profile? `mscorlib.dll` couldn't be found because it wasn't *built*, nor installed into `bin/Debug/lib/xbuild-frameworks/MonoAndroid/v1.0`. Additionally, fully building the AOT cross compilers requires adding `mxe-Win32` to `$(ALL_HOST_ABIS)`, otherwise e.g. `win-armeabi` fails to build because `arm_dpimacros.h` doesn't exist (as `arm_dpimacros.h` is built by `mxe-Win32`): Error parsing '.../xamarin-android/external/mono/mono/metadata/metadata-cross-helpers.c' .../xamarin-android/external/mono/mono/arch/arm/arm-codegen.h(937,10): fatal: 'mono/arch/arm/arm_dpimacros.h' file not found Error parsing '.../xamarin-android/external/mono/mono/mini/mini-cross-helpers.c' .../xamarin-android/external/mono/mono/arch/arm/arm-codegen.h(937,10): fatal: 'mono/arch/arm/arm_dpimacros.h' file not found Unhandled Exception: System.Exception: Expected to find struct definition for MonoObject Fix the `mono-runtimes.mdproj` build so that it doesn't re-execute targets when not necessary, by "rolling" with xbuild limitations and refactoring item groups as appropriate. In particular, xbuild doesn't like referencing item metadata within a `//Target/@Outputs` that in turn references a property group: <PropertyGroup> <V>Value</v> </PropertyGroup> <ItemGroup> <Item Include="Something"> <Metadata>$(V)</Metadata> </Item> </ItemGroup> <Target Name="Example" Inputs="..." Outputs="%(Item.Metadata)"> ... </Target> Whenever the `Example` target is run, it will *always* run, even if `Value` exists, because `xbuild` perceives the `//Target[@Name='Example']/@Outputs` value as being `$(V)`, *not* `Value`, which is wonderfully broken. To compensate, Don't Do That™; in particular, item metadata used within `//Target/@Outputs` *must* be hardcoded and not contain a property reference. This "hardcoding" removes the need for the `$(_Cross*ffsetsHeaderFile)` properties, so remove them. Remove `System.Net.Http.WebRequest.dll` from `@(_BclAssembly)`, as it is no longer built in the monodroid profile; requiring it as an output means that the "host" mono build is *always* built, which isn't needed. Improve the `_BuildRuntimes` and `_InstallRuntimes` targets so that their referenced files aren't *so* repetitive, in particular by "splitting out" `@(_RuntimeLibraries)` into `@(_RuntimeSource)`, `@(_ProfilerSource)`, and `@(_MonoPosixHelperSource)`. This allows more intelligently `strip`ing the native libraries in the Release configuration, while also maintaining generation of the `.d.so` files.
2016-07-28 13:24:18 +03:00
<_InstallUnstrippedProfilerOutput
Condition=" '%(_MonoRuntime.DoBuild)' == 'True' And '%(_MonoRuntime.OutputProfilerFilename)' != '' "
[structure] Rework installation directory structure (#704) Context: https://github.com/xamarin/xamarin-android/pull/253#discussion_r82862993 The *intention* is that Jenkins-produced `oss-xamarin.android*.zip` artifacts be usable on Windows, so that side-by-side testing can be performed without replacing the system installation. Usage is in [UsingJenkinsBuildArtifacts.md](Documentation/UsingJenkinsBuildArtifacts). This isn't *entirely* the case. It was *apparently* the case at the time of commit 87ca2737, but things have bitrotten since. For example, following the 87ca2737 instructions would currently result in an XA0020 `Could not find monodroid` error, because `class-parse.exe` wasn't where Windows expects it to be (it was in `lib/mandroid`, not `lib/xbuild/Xamarin/Android`). This needs to be fixed. Additionally, PR #253 mentions that, for filesystem organization, it would be useful if the macOS/Linux directory structure -- `$prefix/bin`, `$prefix/lib/mandroid`, `$prefix/lib/xbuild/Xamarin/Android`, `$prefix/lib/xbuild-frameworks` -- more closely resembled the Windows directory structure of `$MSBuild` and `$ReferenceAssemblies` (as seen in `.vsix` files). This would turn macOS/Linux into using `$xa_prefix/xbuild` and `$xa_prefix/xbuild-frameworks` directories. `$prefix/bin` would only contain `xabuild`. What is currently in `$prefix/lib/mandroid` would be merged with `$xa_prefix/xbuild/Xamarin/Android`. `$xa_prefix` would `$prefix/lib/xamarin.android`. This would turn the current macOS structure: $prefix/bin/xabuild $prefix/bin/generator $prefix/bin/cross-arm $prefix/lib/mandroid/generator.exe $prefix/lib/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xbuild/Xamarin/Android/Xamarin.Android.Common.targets Into: $prefix/bin/xabuild $prefix/lib/xamarin.android/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xamarin.android/xbuild/Xamarin/Android/generator.exe $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Xamarin.Android.Common.targets $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Darwin/cross-arm Other notes: * The `bundle-*.zip` filename has been has been changed to include a *hash* of the contents of various files, in particular `build-tools\mono-runtimes.*`. This was instigated via a conversation with @kumpera about making the filename more "idiot-proof": `mono-runtimes.props` contains *compiler flags*, and if any of those change, then *logically* the bundle should differ as well, as the mono runtimes may differ in significant ways. In theory, the `-vXX` version should be used to track this, but this is a manual change, easy to overlook. The new `-hHASH` part of the filename should be more automagic. The new `<HashFileContents/>` task in `xa-prep-tasks.dll` is responsible for creating the hash value. * `Configuration.Java.Interop.Override.props` was moved into `build-tools/scripts`, because that would cleanup the root directory a bit. * OS-specific binaries are now placed into `$prefix/lib/xamarin.android/xbuild/Xamarin/Android/$(HostOS)`. On a theoretical plus side, this means that the same build directory can contain OS-specific binaries for multiple operating systems. (I don't know if anyone shares a build tree between e.g. macOS and Linux, but if anyone *does*...) Unfortunately this requires a workaround for an `xbuild` bug: `%(_LlvmRuntime.InstallPath)` and `%(_MonoCrossRuntime.InstallPath)` *cannot* use MSBuild properties, e.g. `<InstallPath>$(HostOS)/</InstallPath>` doesn't work as desired; it's instead treated literally. Special-case `%(InstallPath)` until we fully migrate to MSBuild. * `$(MonoAndroidToolsDirectory)` should be considered *dead*, along with `$(MonoAndroidBinDirectory)`, as these should now *always* be the same directory as where `Xamarin.Android.Build.Tasks.dll` is located, or a `$(HostOS)` sub-directory. * `Xamarin.ProjectTools.csproj` needed to be updated to ensure that the build order was correct. * Remove all `[Obsolete]` and unreferenced members from `Xamarin.Android.Build.Utilities.dll`. There's too much in there, and it makes my head hurt trying to understand the interrelationships between it all. If it's not used, it's gone. * The changes to `src/monodroid/jni/Android.mk` are...weird. The removal of `-I$(topdir)/libmonodroid/zip`/etc. is to reduce use of non-existent paths, as `$(topdir)` isn't defined, so that's *actually* `-I/libmonodroid/zip`, which is nonsensical. So far, so good. What's *odd* is the addition of `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`: This is needed so that `external/mono/support/zlib-helper.c` exports `CreateZStream` and related symbols, otherwise we get a unit test failure in `GzipStreamTest.Compression` due to an `EntryPointNotFoundException`, because `CreateZStream` isn't exported/public. What's odd here is that I don't understand what caused this behavior to change. Previous builds exported `CreateZStream`, otherwise the tests would fail, and I don't understand how any of the other changes in this PR would be at fault, though that's certainly the most plausible explanation. Regardless, `-Ijni` *first* (due to adding `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`) is the desired behavior, so that `jni/config.h` is included, thus ensuring that `MONO_API` has the required definition when building `zlib-helper.c`.
2017-07-28 16:36:32 +03:00
Include="@(_MonoRuntime->'$(_MSBuildDir)\lib\%(Identity)\%(OutputProfilerFilename).d.%(NativeLibraryExtension)')"
[build] Build the monodroid-profile BCL assemblies reliably (#135) Commit 37b5fd51 broke `make jenkins` (and possible `make all`?) through the introduction of the `%(_MonoRuntime.DoBuild)` metadata: `%(_MonoRuntime.DoBuild)` wasn't added everywhere required. In particular, the `host-Darwin` item didn't gain `<DoBuild/>`, meaning *`host-Darwin` wasn't built*. This in turn meant that the monodroid BCL wasn't built (!) which caused a build break much later within `src/Mono.Android`: Executing: mono .../xamarin-android/external/Java.Interop/bin/Debug/jcw-gen.exe -o ".../xamarin-android/src/Mono.Android/obj/Debug/android-23/jcw/src" -L ".../xamarin-androidg/lib/xbuild-frameworks/MonoAndroid/v6.0/" -L ".../xamarin-android/src/Mono.Android/../../bin/Debug/lib/xbuild-frameworks/MonoAndroid/v6.0/../v1.0/" -L ".../xamarin-android/src/Mono.Android/../../MonoAndroid/v6.0/../v1.0/Facades" ".../xamarin-android/src/Mono.Android/../../bin/Debug/lib/xbuild-frameworks/MonoAndroid/v6.0/Mono.Android.dll" jcw-gen: Could not load assembly 'mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089'. Perhaps it doesn't exist in the Mono for Android profile? `mscorlib.dll` couldn't be found because it wasn't *built*, nor installed into `bin/Debug/lib/xbuild-frameworks/MonoAndroid/v1.0`. Additionally, fully building the AOT cross compilers requires adding `mxe-Win32` to `$(ALL_HOST_ABIS)`, otherwise e.g. `win-armeabi` fails to build because `arm_dpimacros.h` doesn't exist (as `arm_dpimacros.h` is built by `mxe-Win32`): Error parsing '.../xamarin-android/external/mono/mono/metadata/metadata-cross-helpers.c' .../xamarin-android/external/mono/mono/arch/arm/arm-codegen.h(937,10): fatal: 'mono/arch/arm/arm_dpimacros.h' file not found Error parsing '.../xamarin-android/external/mono/mono/mini/mini-cross-helpers.c' .../xamarin-android/external/mono/mono/arch/arm/arm-codegen.h(937,10): fatal: 'mono/arch/arm/arm_dpimacros.h' file not found Unhandled Exception: System.Exception: Expected to find struct definition for MonoObject Fix the `mono-runtimes.mdproj` build so that it doesn't re-execute targets when not necessary, by "rolling" with xbuild limitations and refactoring item groups as appropriate. In particular, xbuild doesn't like referencing item metadata within a `//Target/@Outputs` that in turn references a property group: <PropertyGroup> <V>Value</v> </PropertyGroup> <ItemGroup> <Item Include="Something"> <Metadata>$(V)</Metadata> </Item> </ItemGroup> <Target Name="Example" Inputs="..." Outputs="%(Item.Metadata)"> ... </Target> Whenever the `Example` target is run, it will *always* run, even if `Value` exists, because `xbuild` perceives the `//Target[@Name='Example']/@Outputs` value as being `$(V)`, *not* `Value`, which is wonderfully broken. To compensate, Don't Do That™; in particular, item metadata used within `//Target/@Outputs` *must* be hardcoded and not contain a property reference. This "hardcoding" removes the need for the `$(_Cross*ffsetsHeaderFile)` properties, so remove them. Remove `System.Net.Http.WebRequest.dll` from `@(_BclAssembly)`, as it is no longer built in the monodroid profile; requiring it as an output means that the "host" mono build is *always* built, which isn't needed. Improve the `_BuildRuntimes` and `_InstallRuntimes` targets so that their referenced files aren't *so* repetitive, in particular by "splitting out" `@(_RuntimeLibraries)` into `@(_RuntimeSource)`, `@(_ProfilerSource)`, and `@(_MonoPosixHelperSource)`. This allows more intelligently `strip`ing the native libraries in the Release configuration, while also maintaining generation of the `.d.so` files.
2016-07-28 13:24:18 +03:00
/>
Bump to mono/mono-4.8.0-branch/9f4abcc3 (#278) * Bump to mono/mono-4.8.0-branch/9f4abcc3 [mono/085b653a][0] splits out the BTLS code into a new, separate, `libmono-btls-shared.so` native library. This allows BTLS inclusion to be *optional*, so that the size increase is only paid when it's used. Bump to mono/9f4abcc3 so that we can use this new feature. [0]: https://github.com/mono/mono/commit/085b653a7425e30516150d2335c39a6906e6cd4d * [mono-runtimes, Xamarin.Android.Build.Tasks] Use separate BTLS lib There was an unintended side effect of commit cebc6c83, which added `--with-btls-android-ndk` to the mono runtime configure command: BTLS was compiled into `libmonosgen-2.0.so`, greatly increasing size: $ ls -lh bin/Release/lib/xbuild/Xamarin/Android/lib/arm64-v8a/libmonosgen-2.0.so /Library/Frameworks/Xamarin.Android.framework/Libraries/xbuild/Xamarin/Android/lib/arm64-v8a/libmonosgen-2.0.so -rwxr-xr-x 1 root wheel 2.8M Sep 23 18:34 /Library/Frameworks/Xamarin.Android.framework/Libraries/xbuild/Xamarin/Android/lib/arm64-v8a/libmonosgen-2.0.so -rwxr-xr-x 1 jon staff 4.6M Oct 5 17:02 bin/Release/lib/xbuild/Xamarin/Android/lib/arm64-v8a/libmonosgen-2.0.so The `/Library/Frameworks/...` is the Xamarin.Android 7.0 file size; the `bin/Release/...` is the size when BTLS is included. BTLS inclusion into `libmonosgen-2.0.so` increases the file size by 1.8 MB. **1.8 MB** That's a ~64% increase in file size, for *all* apps. [mono/085b653a][0] splits out the BTLS code into a new, separate, `libmono-btls-shared.so` native library. This allows BTLS inclusion to be *optional*, so that the size increase is only paid when it's used. Update `build-tools/mono-runtimes` to use `configure --enable-dynamic-btls` for the Android and host builds so that BTLS is placed into the new `libmono-btls-shared.so` library and installed into `bin/$(Configuration)/lib/xbuild/Xamarin/Android/lib/ABI/libmono-btls-shared.so`. (The host needs `--enable-dynamic-btls` so that the monodroid-profile `System.dll` properly P/Invokes `libmono-btls-shared.so`, instead of P/Invoking `__Internal`.) Update the `<BuildApk/>` task so that when `$(AndroidTlsProvider)` has the value `btls`, `libmono-btls-shared.so` is embedded into the `app.apk`. [0]: https://github.com/mono/mono/commit/085b653a7425e30516150d2335c39a6906e6cd4d
2016-10-25 17:58:29 +03:00
<_MonoBtlsSource
Condition=" '%(_MonoRuntime.DoBuild)' == 'True' And '%(_MonoRuntime.OutputMonoBtlsFilename)' != '' "
Include="@(_MonoRuntime->'$(IntermediateOutputPath)\%(Identity)\mono\btls\build-shared\%(OutputMonoBtlsFilename).%(NativeLibraryExtension)')"
/>
<_InstallMonoBtlsOutput
Condition=" '%(_MonoRuntime.DoBuild)' == 'True' And '%(_MonoRuntime.OutputMonoBtlsFilename)' != '' "
[structure] Rework installation directory structure (#704) Context: https://github.com/xamarin/xamarin-android/pull/253#discussion_r82862993 The *intention* is that Jenkins-produced `oss-xamarin.android*.zip` artifacts be usable on Windows, so that side-by-side testing can be performed without replacing the system installation. Usage is in [UsingJenkinsBuildArtifacts.md](Documentation/UsingJenkinsBuildArtifacts). This isn't *entirely* the case. It was *apparently* the case at the time of commit 87ca2737, but things have bitrotten since. For example, following the 87ca2737 instructions would currently result in an XA0020 `Could not find monodroid` error, because `class-parse.exe` wasn't where Windows expects it to be (it was in `lib/mandroid`, not `lib/xbuild/Xamarin/Android`). This needs to be fixed. Additionally, PR #253 mentions that, for filesystem organization, it would be useful if the macOS/Linux directory structure -- `$prefix/bin`, `$prefix/lib/mandroid`, `$prefix/lib/xbuild/Xamarin/Android`, `$prefix/lib/xbuild-frameworks` -- more closely resembled the Windows directory structure of `$MSBuild` and `$ReferenceAssemblies` (as seen in `.vsix` files). This would turn macOS/Linux into using `$xa_prefix/xbuild` and `$xa_prefix/xbuild-frameworks` directories. `$prefix/bin` would only contain `xabuild`. What is currently in `$prefix/lib/mandroid` would be merged with `$xa_prefix/xbuild/Xamarin/Android`. `$xa_prefix` would `$prefix/lib/xamarin.android`. This would turn the current macOS structure: $prefix/bin/xabuild $prefix/bin/generator $prefix/bin/cross-arm $prefix/lib/mandroid/generator.exe $prefix/lib/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xbuild/Xamarin/Android/Xamarin.Android.Common.targets Into: $prefix/bin/xabuild $prefix/lib/xamarin.android/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xamarin.android/xbuild/Xamarin/Android/generator.exe $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Xamarin.Android.Common.targets $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Darwin/cross-arm Other notes: * The `bundle-*.zip` filename has been has been changed to include a *hash* of the contents of various files, in particular `build-tools\mono-runtimes.*`. This was instigated via a conversation with @kumpera about making the filename more "idiot-proof": `mono-runtimes.props` contains *compiler flags*, and if any of those change, then *logically* the bundle should differ as well, as the mono runtimes may differ in significant ways. In theory, the `-vXX` version should be used to track this, but this is a manual change, easy to overlook. The new `-hHASH` part of the filename should be more automagic. The new `<HashFileContents/>` task in `xa-prep-tasks.dll` is responsible for creating the hash value. * `Configuration.Java.Interop.Override.props` was moved into `build-tools/scripts`, because that would cleanup the root directory a bit. * OS-specific binaries are now placed into `$prefix/lib/xamarin.android/xbuild/Xamarin/Android/$(HostOS)`. On a theoretical plus side, this means that the same build directory can contain OS-specific binaries for multiple operating systems. (I don't know if anyone shares a build tree between e.g. macOS and Linux, but if anyone *does*...) Unfortunately this requires a workaround for an `xbuild` bug: `%(_LlvmRuntime.InstallPath)` and `%(_MonoCrossRuntime.InstallPath)` *cannot* use MSBuild properties, e.g. `<InstallPath>$(HostOS)/</InstallPath>` doesn't work as desired; it's instead treated literally. Special-case `%(InstallPath)` until we fully migrate to MSBuild. * `$(MonoAndroidToolsDirectory)` should be considered *dead*, along with `$(MonoAndroidBinDirectory)`, as these should now *always* be the same directory as where `Xamarin.Android.Build.Tasks.dll` is located, or a `$(HostOS)` sub-directory. * `Xamarin.ProjectTools.csproj` needed to be updated to ensure that the build order was correct. * Remove all `[Obsolete]` and unreferenced members from `Xamarin.Android.Build.Utilities.dll`. There's too much in there, and it makes my head hurt trying to understand the interrelationships between it all. If it's not used, it's gone. * The changes to `src/monodroid/jni/Android.mk` are...weird. The removal of `-I$(topdir)/libmonodroid/zip`/etc. is to reduce use of non-existent paths, as `$(topdir)` isn't defined, so that's *actually* `-I/libmonodroid/zip`, which is nonsensical. So far, so good. What's *odd* is the addition of `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`: This is needed so that `external/mono/support/zlib-helper.c` exports `CreateZStream` and related symbols, otherwise we get a unit test failure in `GzipStreamTest.Compression` due to an `EntryPointNotFoundException`, because `CreateZStream` isn't exported/public. What's odd here is that I don't understand what caused this behavior to change. Previous builds exported `CreateZStream`, otherwise the tests would fail, and I don't understand how any of the other changes in this PR would be at fault, though that's certainly the most plausible explanation. Regardless, `-Ijni` *first* (due to adding `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`) is the desired behavior, so that `jni/config.h` is included, thus ensuring that `MONO_API` has the required definition when building `zlib-helper.c`.
2017-07-28 16:36:32 +03:00
Include="@(_MonoRuntime->'$(_MSBuildDir)\lib\%(Identity)\%(OutputMonoBtlsFilename).%(NativeLibraryExtension)')"
Bump to mono/mono-4.8.0-branch/9f4abcc3 (#278) * Bump to mono/mono-4.8.0-branch/9f4abcc3 [mono/085b653a][0] splits out the BTLS code into a new, separate, `libmono-btls-shared.so` native library. This allows BTLS inclusion to be *optional*, so that the size increase is only paid when it's used. Bump to mono/9f4abcc3 so that we can use this new feature. [0]: https://github.com/mono/mono/commit/085b653a7425e30516150d2335c39a6906e6cd4d * [mono-runtimes, Xamarin.Android.Build.Tasks] Use separate BTLS lib There was an unintended side effect of commit cebc6c83, which added `--with-btls-android-ndk` to the mono runtime configure command: BTLS was compiled into `libmonosgen-2.0.so`, greatly increasing size: $ ls -lh bin/Release/lib/xbuild/Xamarin/Android/lib/arm64-v8a/libmonosgen-2.0.so /Library/Frameworks/Xamarin.Android.framework/Libraries/xbuild/Xamarin/Android/lib/arm64-v8a/libmonosgen-2.0.so -rwxr-xr-x 1 root wheel 2.8M Sep 23 18:34 /Library/Frameworks/Xamarin.Android.framework/Libraries/xbuild/Xamarin/Android/lib/arm64-v8a/libmonosgen-2.0.so -rwxr-xr-x 1 jon staff 4.6M Oct 5 17:02 bin/Release/lib/xbuild/Xamarin/Android/lib/arm64-v8a/libmonosgen-2.0.so The `/Library/Frameworks/...` is the Xamarin.Android 7.0 file size; the `bin/Release/...` is the size when BTLS is included. BTLS inclusion into `libmonosgen-2.0.so` increases the file size by 1.8 MB. **1.8 MB** That's a ~64% increase in file size, for *all* apps. [mono/085b653a][0] splits out the BTLS code into a new, separate, `libmono-btls-shared.so` native library. This allows BTLS inclusion to be *optional*, so that the size increase is only paid when it's used. Update `build-tools/mono-runtimes` to use `configure --enable-dynamic-btls` for the Android and host builds so that BTLS is placed into the new `libmono-btls-shared.so` library and installed into `bin/$(Configuration)/lib/xbuild/Xamarin/Android/lib/ABI/libmono-btls-shared.so`. (The host needs `--enable-dynamic-btls` so that the monodroid-profile `System.dll` properly P/Invokes `libmono-btls-shared.so`, instead of P/Invoking `__Internal`.) Update the `<BuildApk/>` task so that when `$(AndroidTlsProvider)` has the value `btls`, `libmono-btls-shared.so` is embedded into the `app.apk`. [0]: https://github.com/mono/mono/commit/085b653a7425e30516150d2335c39a6906e6cd4d
2016-10-25 17:58:29 +03:00
/>
<_InstallUnstrippedMonoBtlsOutput
Condition=" '%(_MonoRuntime.DoBuild)' == 'True' And '%(_MonoRuntime.OutputMonoBtlsFilename)' != '' "
[structure] Rework installation directory structure (#704) Context: https://github.com/xamarin/xamarin-android/pull/253#discussion_r82862993 The *intention* is that Jenkins-produced `oss-xamarin.android*.zip` artifacts be usable on Windows, so that side-by-side testing can be performed without replacing the system installation. Usage is in [UsingJenkinsBuildArtifacts.md](Documentation/UsingJenkinsBuildArtifacts). This isn't *entirely* the case. It was *apparently* the case at the time of commit 87ca2737, but things have bitrotten since. For example, following the 87ca2737 instructions would currently result in an XA0020 `Could not find monodroid` error, because `class-parse.exe` wasn't where Windows expects it to be (it was in `lib/mandroid`, not `lib/xbuild/Xamarin/Android`). This needs to be fixed. Additionally, PR #253 mentions that, for filesystem organization, it would be useful if the macOS/Linux directory structure -- `$prefix/bin`, `$prefix/lib/mandroid`, `$prefix/lib/xbuild/Xamarin/Android`, `$prefix/lib/xbuild-frameworks` -- more closely resembled the Windows directory structure of `$MSBuild` and `$ReferenceAssemblies` (as seen in `.vsix` files). This would turn macOS/Linux into using `$xa_prefix/xbuild` and `$xa_prefix/xbuild-frameworks` directories. `$prefix/bin` would only contain `xabuild`. What is currently in `$prefix/lib/mandroid` would be merged with `$xa_prefix/xbuild/Xamarin/Android`. `$xa_prefix` would `$prefix/lib/xamarin.android`. This would turn the current macOS structure: $prefix/bin/xabuild $prefix/bin/generator $prefix/bin/cross-arm $prefix/lib/mandroid/generator.exe $prefix/lib/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xbuild/Xamarin/Android/Xamarin.Android.Common.targets Into: $prefix/bin/xabuild $prefix/lib/xamarin.android/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xamarin.android/xbuild/Xamarin/Android/generator.exe $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Xamarin.Android.Common.targets $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Darwin/cross-arm Other notes: * The `bundle-*.zip` filename has been has been changed to include a *hash* of the contents of various files, in particular `build-tools\mono-runtimes.*`. This was instigated via a conversation with @kumpera about making the filename more "idiot-proof": `mono-runtimes.props` contains *compiler flags*, and if any of those change, then *logically* the bundle should differ as well, as the mono runtimes may differ in significant ways. In theory, the `-vXX` version should be used to track this, but this is a manual change, easy to overlook. The new `-hHASH` part of the filename should be more automagic. The new `<HashFileContents/>` task in `xa-prep-tasks.dll` is responsible for creating the hash value. * `Configuration.Java.Interop.Override.props` was moved into `build-tools/scripts`, because that would cleanup the root directory a bit. * OS-specific binaries are now placed into `$prefix/lib/xamarin.android/xbuild/Xamarin/Android/$(HostOS)`. On a theoretical plus side, this means that the same build directory can contain OS-specific binaries for multiple operating systems. (I don't know if anyone shares a build tree between e.g. macOS and Linux, but if anyone *does*...) Unfortunately this requires a workaround for an `xbuild` bug: `%(_LlvmRuntime.InstallPath)` and `%(_MonoCrossRuntime.InstallPath)` *cannot* use MSBuild properties, e.g. `<InstallPath>$(HostOS)/</InstallPath>` doesn't work as desired; it's instead treated literally. Special-case `%(InstallPath)` until we fully migrate to MSBuild. * `$(MonoAndroidToolsDirectory)` should be considered *dead*, along with `$(MonoAndroidBinDirectory)`, as these should now *always* be the same directory as where `Xamarin.Android.Build.Tasks.dll` is located, or a `$(HostOS)` sub-directory. * `Xamarin.ProjectTools.csproj` needed to be updated to ensure that the build order was correct. * Remove all `[Obsolete]` and unreferenced members from `Xamarin.Android.Build.Utilities.dll`. There's too much in there, and it makes my head hurt trying to understand the interrelationships between it all. If it's not used, it's gone. * The changes to `src/monodroid/jni/Android.mk` are...weird. The removal of `-I$(topdir)/libmonodroid/zip`/etc. is to reduce use of non-existent paths, as `$(topdir)` isn't defined, so that's *actually* `-I/libmonodroid/zip`, which is nonsensical. So far, so good. What's *odd* is the addition of `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`: This is needed so that `external/mono/support/zlib-helper.c` exports `CreateZStream` and related symbols, otherwise we get a unit test failure in `GzipStreamTest.Compression` due to an `EntryPointNotFoundException`, because `CreateZStream` isn't exported/public. What's odd here is that I don't understand what caused this behavior to change. Previous builds exported `CreateZStream`, otherwise the tests would fail, and I don't understand how any of the other changes in this PR would be at fault, though that's certainly the most plausible explanation. Regardless, `-Ijni` *first* (due to adding `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`) is the desired behavior, so that `jni/config.h` is included, thus ensuring that `MONO_API` has the required definition when building `zlib-helper.c`.
2017-07-28 16:36:32 +03:00
Include="@(_MonoRuntime->'$(_MSBuildDir)\lib\%(Identity)\%(OutputMonoBtlsFilename).d.%(NativeLibraryExtension)')"
Bump to mono/mono-4.8.0-branch/9f4abcc3 (#278) * Bump to mono/mono-4.8.0-branch/9f4abcc3 [mono/085b653a][0] splits out the BTLS code into a new, separate, `libmono-btls-shared.so` native library. This allows BTLS inclusion to be *optional*, so that the size increase is only paid when it's used. Bump to mono/9f4abcc3 so that we can use this new feature. [0]: https://github.com/mono/mono/commit/085b653a7425e30516150d2335c39a6906e6cd4d * [mono-runtimes, Xamarin.Android.Build.Tasks] Use separate BTLS lib There was an unintended side effect of commit cebc6c83, which added `--with-btls-android-ndk` to the mono runtime configure command: BTLS was compiled into `libmonosgen-2.0.so`, greatly increasing size: $ ls -lh bin/Release/lib/xbuild/Xamarin/Android/lib/arm64-v8a/libmonosgen-2.0.so /Library/Frameworks/Xamarin.Android.framework/Libraries/xbuild/Xamarin/Android/lib/arm64-v8a/libmonosgen-2.0.so -rwxr-xr-x 1 root wheel 2.8M Sep 23 18:34 /Library/Frameworks/Xamarin.Android.framework/Libraries/xbuild/Xamarin/Android/lib/arm64-v8a/libmonosgen-2.0.so -rwxr-xr-x 1 jon staff 4.6M Oct 5 17:02 bin/Release/lib/xbuild/Xamarin/Android/lib/arm64-v8a/libmonosgen-2.0.so The `/Library/Frameworks/...` is the Xamarin.Android 7.0 file size; the `bin/Release/...` is the size when BTLS is included. BTLS inclusion into `libmonosgen-2.0.so` increases the file size by 1.8 MB. **1.8 MB** That's a ~64% increase in file size, for *all* apps. [mono/085b653a][0] splits out the BTLS code into a new, separate, `libmono-btls-shared.so` native library. This allows BTLS inclusion to be *optional*, so that the size increase is only paid when it's used. Update `build-tools/mono-runtimes` to use `configure --enable-dynamic-btls` for the Android and host builds so that BTLS is placed into the new `libmono-btls-shared.so` library and installed into `bin/$(Configuration)/lib/xbuild/Xamarin/Android/lib/ABI/libmono-btls-shared.so`. (The host needs `--enable-dynamic-btls` so that the monodroid-profile `System.dll` properly P/Invokes `libmono-btls-shared.so`, instead of P/Invoking `__Internal`.) Update the `<BuildApk/>` task so that when `$(AndroidTlsProvider)` has the value `btls`, `libmono-btls-shared.so` is embedded into the `app.apk`. [0]: https://github.com/mono/mono/commit/085b653a7425e30516150d2335c39a6906e6cd4d
2016-10-25 17:58:29 +03:00
/>
[build] Build the monodroid-profile BCL assemblies reliably (#135) Commit 37b5fd51 broke `make jenkins` (and possible `make all`?) through the introduction of the `%(_MonoRuntime.DoBuild)` metadata: `%(_MonoRuntime.DoBuild)` wasn't added everywhere required. In particular, the `host-Darwin` item didn't gain `<DoBuild/>`, meaning *`host-Darwin` wasn't built*. This in turn meant that the monodroid BCL wasn't built (!) which caused a build break much later within `src/Mono.Android`: Executing: mono .../xamarin-android/external/Java.Interop/bin/Debug/jcw-gen.exe -o ".../xamarin-android/src/Mono.Android/obj/Debug/android-23/jcw/src" -L ".../xamarin-androidg/lib/xbuild-frameworks/MonoAndroid/v6.0/" -L ".../xamarin-android/src/Mono.Android/../../bin/Debug/lib/xbuild-frameworks/MonoAndroid/v6.0/../v1.0/" -L ".../xamarin-android/src/Mono.Android/../../MonoAndroid/v6.0/../v1.0/Facades" ".../xamarin-android/src/Mono.Android/../../bin/Debug/lib/xbuild-frameworks/MonoAndroid/v6.0/Mono.Android.dll" jcw-gen: Could not load assembly 'mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089'. Perhaps it doesn't exist in the Mono for Android profile? `mscorlib.dll` couldn't be found because it wasn't *built*, nor installed into `bin/Debug/lib/xbuild-frameworks/MonoAndroid/v1.0`. Additionally, fully building the AOT cross compilers requires adding `mxe-Win32` to `$(ALL_HOST_ABIS)`, otherwise e.g. `win-armeabi` fails to build because `arm_dpimacros.h` doesn't exist (as `arm_dpimacros.h` is built by `mxe-Win32`): Error parsing '.../xamarin-android/external/mono/mono/metadata/metadata-cross-helpers.c' .../xamarin-android/external/mono/mono/arch/arm/arm-codegen.h(937,10): fatal: 'mono/arch/arm/arm_dpimacros.h' file not found Error parsing '.../xamarin-android/external/mono/mono/mini/mini-cross-helpers.c' .../xamarin-android/external/mono/mono/arch/arm/arm-codegen.h(937,10): fatal: 'mono/arch/arm/arm_dpimacros.h' file not found Unhandled Exception: System.Exception: Expected to find struct definition for MonoObject Fix the `mono-runtimes.mdproj` build so that it doesn't re-execute targets when not necessary, by "rolling" with xbuild limitations and refactoring item groups as appropriate. In particular, xbuild doesn't like referencing item metadata within a `//Target/@Outputs` that in turn references a property group: <PropertyGroup> <V>Value</v> </PropertyGroup> <ItemGroup> <Item Include="Something"> <Metadata>$(V)</Metadata> </Item> </ItemGroup> <Target Name="Example" Inputs="..." Outputs="%(Item.Metadata)"> ... </Target> Whenever the `Example` target is run, it will *always* run, even if `Value` exists, because `xbuild` perceives the `//Target[@Name='Example']/@Outputs` value as being `$(V)`, *not* `Value`, which is wonderfully broken. To compensate, Don't Do That™; in particular, item metadata used within `//Target/@Outputs` *must* be hardcoded and not contain a property reference. This "hardcoding" removes the need for the `$(_Cross*ffsetsHeaderFile)` properties, so remove them. Remove `System.Net.Http.WebRequest.dll` from `@(_BclAssembly)`, as it is no longer built in the monodroid profile; requiring it as an output means that the "host" mono build is *always* built, which isn't needed. Improve the `_BuildRuntimes` and `_InstallRuntimes` targets so that their referenced files aren't *so* repetitive, in particular by "splitting out" `@(_RuntimeLibraries)` into `@(_RuntimeSource)`, `@(_ProfilerSource)`, and `@(_MonoPosixHelperSource)`. This allows more intelligently `strip`ing the native libraries in the Release configuration, while also maintaining generation of the `.d.so` files.
2016-07-28 13:24:18 +03:00
<_MonoPosixHelperSource
Condition=" '%(_MonoRuntime.DoBuild)' == 'True' And '%(_MonoRuntime.OutputMonoPosixHelperFilename)' != '' "
[mono-runtimes] Use directory separators with `$(OutputPath)`. [The build][0] [is broken][1]: xamarin-android/build-tools/mono-runtimes/mono-runtimes.targets: error : Could not find a part of the path ".../xamarin-android/xamarin-android/bin/Debuglib/xbuild-frameworks/MonoAndroid/v1.0/RedistList/FrameworkList.xml". The error is generated from the `_InstallBcl` target when writing `FrameworkList.xml`, and a close reading of the error message shows that a directory separator is missing: it should be `bin/Debug/lib`, but is instead `bin/Debuglib`. I *assume* that this is due to `<CallTarget/>` being "flakey" within xbuild (it passes the file containing the target to `<MSBuild/>`!), and *somehow* in that "flakiness" `$(OutputPath)` doesn't end with `\`, *which it should*, because `Microsoft.Common.targets` will automatically append `\` to `$(OutputPath)`. To work around this issue, update `mono-runtimes.targets` so that it *always* uses `\` after `$(OutputPath)` and `$(IntermediateOutputPath)`. Additionally, `llc.exe` was being installed into the wrong directory, preventing the bundle from being used: Target _BuildUnlessCached needs to be built as output file '../../bin/Debug/lib/mandroid/llc.exe' does not exist. `llc.exe` was instead installed into `bin/Debug/bin/llc.exe`. Bump the bundle version number to -v3 and correct `mono-runtimes.targets` so that `llc.exe` and `opt.exe` are properly installed into `$prefix/lib/mandroid`. [0]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android/49/ [1]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android/49/consoleText
2016-08-17 04:18:15 +03:00
Include="@(_MonoRuntime->'$(IntermediateOutputPath)\%(Identity)\support\.libs\%(OutputMonoPosixHelperFilename).%(NativeLibraryExtension)')"
[build] Build the monodroid-profile BCL assemblies reliably (#135) Commit 37b5fd51 broke `make jenkins` (and possible `make all`?) through the introduction of the `%(_MonoRuntime.DoBuild)` metadata: `%(_MonoRuntime.DoBuild)` wasn't added everywhere required. In particular, the `host-Darwin` item didn't gain `<DoBuild/>`, meaning *`host-Darwin` wasn't built*. This in turn meant that the monodroid BCL wasn't built (!) which caused a build break much later within `src/Mono.Android`: Executing: mono .../xamarin-android/external/Java.Interop/bin/Debug/jcw-gen.exe -o ".../xamarin-android/src/Mono.Android/obj/Debug/android-23/jcw/src" -L ".../xamarin-androidg/lib/xbuild-frameworks/MonoAndroid/v6.0/" -L ".../xamarin-android/src/Mono.Android/../../bin/Debug/lib/xbuild-frameworks/MonoAndroid/v6.0/../v1.0/" -L ".../xamarin-android/src/Mono.Android/../../MonoAndroid/v6.0/../v1.0/Facades" ".../xamarin-android/src/Mono.Android/../../bin/Debug/lib/xbuild-frameworks/MonoAndroid/v6.0/Mono.Android.dll" jcw-gen: Could not load assembly 'mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089'. Perhaps it doesn't exist in the Mono for Android profile? `mscorlib.dll` couldn't be found because it wasn't *built*, nor installed into `bin/Debug/lib/xbuild-frameworks/MonoAndroid/v1.0`. Additionally, fully building the AOT cross compilers requires adding `mxe-Win32` to `$(ALL_HOST_ABIS)`, otherwise e.g. `win-armeabi` fails to build because `arm_dpimacros.h` doesn't exist (as `arm_dpimacros.h` is built by `mxe-Win32`): Error parsing '.../xamarin-android/external/mono/mono/metadata/metadata-cross-helpers.c' .../xamarin-android/external/mono/mono/arch/arm/arm-codegen.h(937,10): fatal: 'mono/arch/arm/arm_dpimacros.h' file not found Error parsing '.../xamarin-android/external/mono/mono/mini/mini-cross-helpers.c' .../xamarin-android/external/mono/mono/arch/arm/arm-codegen.h(937,10): fatal: 'mono/arch/arm/arm_dpimacros.h' file not found Unhandled Exception: System.Exception: Expected to find struct definition for MonoObject Fix the `mono-runtimes.mdproj` build so that it doesn't re-execute targets when not necessary, by "rolling" with xbuild limitations and refactoring item groups as appropriate. In particular, xbuild doesn't like referencing item metadata within a `//Target/@Outputs` that in turn references a property group: <PropertyGroup> <V>Value</v> </PropertyGroup> <ItemGroup> <Item Include="Something"> <Metadata>$(V)</Metadata> </Item> </ItemGroup> <Target Name="Example" Inputs="..." Outputs="%(Item.Metadata)"> ... </Target> Whenever the `Example` target is run, it will *always* run, even if `Value` exists, because `xbuild` perceives the `//Target[@Name='Example']/@Outputs` value as being `$(V)`, *not* `Value`, which is wonderfully broken. To compensate, Don't Do That™; in particular, item metadata used within `//Target/@Outputs` *must* be hardcoded and not contain a property reference. This "hardcoding" removes the need for the `$(_Cross*ffsetsHeaderFile)` properties, so remove them. Remove `System.Net.Http.WebRequest.dll` from `@(_BclAssembly)`, as it is no longer built in the monodroid profile; requiring it as an output means that the "host" mono build is *always* built, which isn't needed. Improve the `_BuildRuntimes` and `_InstallRuntimes` targets so that their referenced files aren't *so* repetitive, in particular by "splitting out" `@(_RuntimeLibraries)` into `@(_RuntimeSource)`, `@(_ProfilerSource)`, and `@(_MonoPosixHelperSource)`. This allows more intelligently `strip`ing the native libraries in the Release configuration, while also maintaining generation of the `.d.so` files.
2016-07-28 13:24:18 +03:00
/>
<_InstallMonoPosixHelperOutput
Condition=" '%(_MonoRuntime.DoBuild)' == 'True' And '%(_MonoRuntime.OutputMonoPosixHelperFilename)' != '' "
[structure] Rework installation directory structure (#704) Context: https://github.com/xamarin/xamarin-android/pull/253#discussion_r82862993 The *intention* is that Jenkins-produced `oss-xamarin.android*.zip` artifacts be usable on Windows, so that side-by-side testing can be performed without replacing the system installation. Usage is in [UsingJenkinsBuildArtifacts.md](Documentation/UsingJenkinsBuildArtifacts). This isn't *entirely* the case. It was *apparently* the case at the time of commit 87ca2737, but things have bitrotten since. For example, following the 87ca2737 instructions would currently result in an XA0020 `Could not find monodroid` error, because `class-parse.exe` wasn't where Windows expects it to be (it was in `lib/mandroid`, not `lib/xbuild/Xamarin/Android`). This needs to be fixed. Additionally, PR #253 mentions that, for filesystem organization, it would be useful if the macOS/Linux directory structure -- `$prefix/bin`, `$prefix/lib/mandroid`, `$prefix/lib/xbuild/Xamarin/Android`, `$prefix/lib/xbuild-frameworks` -- more closely resembled the Windows directory structure of `$MSBuild` and `$ReferenceAssemblies` (as seen in `.vsix` files). This would turn macOS/Linux into using `$xa_prefix/xbuild` and `$xa_prefix/xbuild-frameworks` directories. `$prefix/bin` would only contain `xabuild`. What is currently in `$prefix/lib/mandroid` would be merged with `$xa_prefix/xbuild/Xamarin/Android`. `$xa_prefix` would `$prefix/lib/xamarin.android`. This would turn the current macOS structure: $prefix/bin/xabuild $prefix/bin/generator $prefix/bin/cross-arm $prefix/lib/mandroid/generator.exe $prefix/lib/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xbuild/Xamarin/Android/Xamarin.Android.Common.targets Into: $prefix/bin/xabuild $prefix/lib/xamarin.android/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xamarin.android/xbuild/Xamarin/Android/generator.exe $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Xamarin.Android.Common.targets $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Darwin/cross-arm Other notes: * The `bundle-*.zip` filename has been has been changed to include a *hash* of the contents of various files, in particular `build-tools\mono-runtimes.*`. This was instigated via a conversation with @kumpera about making the filename more "idiot-proof": `mono-runtimes.props` contains *compiler flags*, and if any of those change, then *logically* the bundle should differ as well, as the mono runtimes may differ in significant ways. In theory, the `-vXX` version should be used to track this, but this is a manual change, easy to overlook. The new `-hHASH` part of the filename should be more automagic. The new `<HashFileContents/>` task in `xa-prep-tasks.dll` is responsible for creating the hash value. * `Configuration.Java.Interop.Override.props` was moved into `build-tools/scripts`, because that would cleanup the root directory a bit. * OS-specific binaries are now placed into `$prefix/lib/xamarin.android/xbuild/Xamarin/Android/$(HostOS)`. On a theoretical plus side, this means that the same build directory can contain OS-specific binaries for multiple operating systems. (I don't know if anyone shares a build tree between e.g. macOS and Linux, but if anyone *does*...) Unfortunately this requires a workaround for an `xbuild` bug: `%(_LlvmRuntime.InstallPath)` and `%(_MonoCrossRuntime.InstallPath)` *cannot* use MSBuild properties, e.g. `<InstallPath>$(HostOS)/</InstallPath>` doesn't work as desired; it's instead treated literally. Special-case `%(InstallPath)` until we fully migrate to MSBuild. * `$(MonoAndroidToolsDirectory)` should be considered *dead*, along with `$(MonoAndroidBinDirectory)`, as these should now *always* be the same directory as where `Xamarin.Android.Build.Tasks.dll` is located, or a `$(HostOS)` sub-directory. * `Xamarin.ProjectTools.csproj` needed to be updated to ensure that the build order was correct. * Remove all `[Obsolete]` and unreferenced members from `Xamarin.Android.Build.Utilities.dll`. There's too much in there, and it makes my head hurt trying to understand the interrelationships between it all. If it's not used, it's gone. * The changes to `src/monodroid/jni/Android.mk` are...weird. The removal of `-I$(topdir)/libmonodroid/zip`/etc. is to reduce use of non-existent paths, as `$(topdir)` isn't defined, so that's *actually* `-I/libmonodroid/zip`, which is nonsensical. So far, so good. What's *odd* is the addition of `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`: This is needed so that `external/mono/support/zlib-helper.c` exports `CreateZStream` and related symbols, otherwise we get a unit test failure in `GzipStreamTest.Compression` due to an `EntryPointNotFoundException`, because `CreateZStream` isn't exported/public. What's odd here is that I don't understand what caused this behavior to change. Previous builds exported `CreateZStream`, otherwise the tests would fail, and I don't understand how any of the other changes in this PR would be at fault, though that's certainly the most plausible explanation. Regardless, `-Ijni` *first* (due to adding `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`) is the desired behavior, so that `jni/config.h` is included, thus ensuring that `MONO_API` has the required definition when building `zlib-helper.c`.
2017-07-28 16:36:32 +03:00
Include="@(_MonoRuntime->'$(_MSBuildDir)\lib\%(Identity)\%(OutputMonoPosixHelperFilename).%(NativeLibraryExtension)')"
[build] Build the monodroid-profile BCL assemblies reliably (#135) Commit 37b5fd51 broke `make jenkins` (and possible `make all`?) through the introduction of the `%(_MonoRuntime.DoBuild)` metadata: `%(_MonoRuntime.DoBuild)` wasn't added everywhere required. In particular, the `host-Darwin` item didn't gain `<DoBuild/>`, meaning *`host-Darwin` wasn't built*. This in turn meant that the monodroid BCL wasn't built (!) which caused a build break much later within `src/Mono.Android`: Executing: mono .../xamarin-android/external/Java.Interop/bin/Debug/jcw-gen.exe -o ".../xamarin-android/src/Mono.Android/obj/Debug/android-23/jcw/src" -L ".../xamarin-androidg/lib/xbuild-frameworks/MonoAndroid/v6.0/" -L ".../xamarin-android/src/Mono.Android/../../bin/Debug/lib/xbuild-frameworks/MonoAndroid/v6.0/../v1.0/" -L ".../xamarin-android/src/Mono.Android/../../MonoAndroid/v6.0/../v1.0/Facades" ".../xamarin-android/src/Mono.Android/../../bin/Debug/lib/xbuild-frameworks/MonoAndroid/v6.0/Mono.Android.dll" jcw-gen: Could not load assembly 'mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089'. Perhaps it doesn't exist in the Mono for Android profile? `mscorlib.dll` couldn't be found because it wasn't *built*, nor installed into `bin/Debug/lib/xbuild-frameworks/MonoAndroid/v1.0`. Additionally, fully building the AOT cross compilers requires adding `mxe-Win32` to `$(ALL_HOST_ABIS)`, otherwise e.g. `win-armeabi` fails to build because `arm_dpimacros.h` doesn't exist (as `arm_dpimacros.h` is built by `mxe-Win32`): Error parsing '.../xamarin-android/external/mono/mono/metadata/metadata-cross-helpers.c' .../xamarin-android/external/mono/mono/arch/arm/arm-codegen.h(937,10): fatal: 'mono/arch/arm/arm_dpimacros.h' file not found Error parsing '.../xamarin-android/external/mono/mono/mini/mini-cross-helpers.c' .../xamarin-android/external/mono/mono/arch/arm/arm-codegen.h(937,10): fatal: 'mono/arch/arm/arm_dpimacros.h' file not found Unhandled Exception: System.Exception: Expected to find struct definition for MonoObject Fix the `mono-runtimes.mdproj` build so that it doesn't re-execute targets when not necessary, by "rolling" with xbuild limitations and refactoring item groups as appropriate. In particular, xbuild doesn't like referencing item metadata within a `//Target/@Outputs` that in turn references a property group: <PropertyGroup> <V>Value</v> </PropertyGroup> <ItemGroup> <Item Include="Something"> <Metadata>$(V)</Metadata> </Item> </ItemGroup> <Target Name="Example" Inputs="..." Outputs="%(Item.Metadata)"> ... </Target> Whenever the `Example` target is run, it will *always* run, even if `Value` exists, because `xbuild` perceives the `//Target[@Name='Example']/@Outputs` value as being `$(V)`, *not* `Value`, which is wonderfully broken. To compensate, Don't Do That™; in particular, item metadata used within `//Target/@Outputs` *must* be hardcoded and not contain a property reference. This "hardcoding" removes the need for the `$(_Cross*ffsetsHeaderFile)` properties, so remove them. Remove `System.Net.Http.WebRequest.dll` from `@(_BclAssembly)`, as it is no longer built in the monodroid profile; requiring it as an output means that the "host" mono build is *always* built, which isn't needed. Improve the `_BuildRuntimes` and `_InstallRuntimes` targets so that their referenced files aren't *so* repetitive, in particular by "splitting out" `@(_RuntimeLibraries)` into `@(_RuntimeSource)`, `@(_ProfilerSource)`, and `@(_MonoPosixHelperSource)`. This allows more intelligently `strip`ing the native libraries in the Release configuration, while also maintaining generation of the `.d.so` files.
2016-07-28 13:24:18 +03:00
/>
<_InstallUnstrippedMonoPosixHelperOutput
Condition=" '%(_MonoRuntime.DoBuild)' == 'True' And '%(_MonoRuntime.OutputMonoPosixHelperFilename)' != '' "
[structure] Rework installation directory structure (#704) Context: https://github.com/xamarin/xamarin-android/pull/253#discussion_r82862993 The *intention* is that Jenkins-produced `oss-xamarin.android*.zip` artifacts be usable on Windows, so that side-by-side testing can be performed without replacing the system installation. Usage is in [UsingJenkinsBuildArtifacts.md](Documentation/UsingJenkinsBuildArtifacts). This isn't *entirely* the case. It was *apparently* the case at the time of commit 87ca2737, but things have bitrotten since. For example, following the 87ca2737 instructions would currently result in an XA0020 `Could not find monodroid` error, because `class-parse.exe` wasn't where Windows expects it to be (it was in `lib/mandroid`, not `lib/xbuild/Xamarin/Android`). This needs to be fixed. Additionally, PR #253 mentions that, for filesystem organization, it would be useful if the macOS/Linux directory structure -- `$prefix/bin`, `$prefix/lib/mandroid`, `$prefix/lib/xbuild/Xamarin/Android`, `$prefix/lib/xbuild-frameworks` -- more closely resembled the Windows directory structure of `$MSBuild` and `$ReferenceAssemblies` (as seen in `.vsix` files). This would turn macOS/Linux into using `$xa_prefix/xbuild` and `$xa_prefix/xbuild-frameworks` directories. `$prefix/bin` would only contain `xabuild`. What is currently in `$prefix/lib/mandroid` would be merged with `$xa_prefix/xbuild/Xamarin/Android`. `$xa_prefix` would `$prefix/lib/xamarin.android`. This would turn the current macOS structure: $prefix/bin/xabuild $prefix/bin/generator $prefix/bin/cross-arm $prefix/lib/mandroid/generator.exe $prefix/lib/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xbuild/Xamarin/Android/Xamarin.Android.Common.targets Into: $prefix/bin/xabuild $prefix/lib/xamarin.android/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xamarin.android/xbuild/Xamarin/Android/generator.exe $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Xamarin.Android.Common.targets $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Darwin/cross-arm Other notes: * The `bundle-*.zip` filename has been has been changed to include a *hash* of the contents of various files, in particular `build-tools\mono-runtimes.*`. This was instigated via a conversation with @kumpera about making the filename more "idiot-proof": `mono-runtimes.props` contains *compiler flags*, and if any of those change, then *logically* the bundle should differ as well, as the mono runtimes may differ in significant ways. In theory, the `-vXX` version should be used to track this, but this is a manual change, easy to overlook. The new `-hHASH` part of the filename should be more automagic. The new `<HashFileContents/>` task in `xa-prep-tasks.dll` is responsible for creating the hash value. * `Configuration.Java.Interop.Override.props` was moved into `build-tools/scripts`, because that would cleanup the root directory a bit. * OS-specific binaries are now placed into `$prefix/lib/xamarin.android/xbuild/Xamarin/Android/$(HostOS)`. On a theoretical plus side, this means that the same build directory can contain OS-specific binaries for multiple operating systems. (I don't know if anyone shares a build tree between e.g. macOS and Linux, but if anyone *does*...) Unfortunately this requires a workaround for an `xbuild` bug: `%(_LlvmRuntime.InstallPath)` and `%(_MonoCrossRuntime.InstallPath)` *cannot* use MSBuild properties, e.g. `<InstallPath>$(HostOS)/</InstallPath>` doesn't work as desired; it's instead treated literally. Special-case `%(InstallPath)` until we fully migrate to MSBuild. * `$(MonoAndroidToolsDirectory)` should be considered *dead*, along with `$(MonoAndroidBinDirectory)`, as these should now *always* be the same directory as where `Xamarin.Android.Build.Tasks.dll` is located, or a `$(HostOS)` sub-directory. * `Xamarin.ProjectTools.csproj` needed to be updated to ensure that the build order was correct. * Remove all `[Obsolete]` and unreferenced members from `Xamarin.Android.Build.Utilities.dll`. There's too much in there, and it makes my head hurt trying to understand the interrelationships between it all. If it's not used, it's gone. * The changes to `src/monodroid/jni/Android.mk` are...weird. The removal of `-I$(topdir)/libmonodroid/zip`/etc. is to reduce use of non-existent paths, as `$(topdir)` isn't defined, so that's *actually* `-I/libmonodroid/zip`, which is nonsensical. So far, so good. What's *odd* is the addition of `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`: This is needed so that `external/mono/support/zlib-helper.c` exports `CreateZStream` and related symbols, otherwise we get a unit test failure in `GzipStreamTest.Compression` due to an `EntryPointNotFoundException`, because `CreateZStream` isn't exported/public. What's odd here is that I don't understand what caused this behavior to change. Previous builds exported `CreateZStream`, otherwise the tests would fail, and I don't understand how any of the other changes in this PR would be at fault, though that's certainly the most plausible explanation. Regardless, `-Ijni` *first* (due to adding `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`) is the desired behavior, so that `jni/config.h` is included, thus ensuring that `MONO_API` has the required definition when building `zlib-helper.c`.
2017-07-28 16:36:32 +03:00
Include="@(_MonoRuntime->'$(_MSBuildDir)\lib\%(Identity)\%(OutputMonoPosixHelperFilename).d.%(NativeLibraryExtension)')"
[build] Build the monodroid-profile BCL assemblies reliably (#135) Commit 37b5fd51 broke `make jenkins` (and possible `make all`?) through the introduction of the `%(_MonoRuntime.DoBuild)` metadata: `%(_MonoRuntime.DoBuild)` wasn't added everywhere required. In particular, the `host-Darwin` item didn't gain `<DoBuild/>`, meaning *`host-Darwin` wasn't built*. This in turn meant that the monodroid BCL wasn't built (!) which caused a build break much later within `src/Mono.Android`: Executing: mono .../xamarin-android/external/Java.Interop/bin/Debug/jcw-gen.exe -o ".../xamarin-android/src/Mono.Android/obj/Debug/android-23/jcw/src" -L ".../xamarin-androidg/lib/xbuild-frameworks/MonoAndroid/v6.0/" -L ".../xamarin-android/src/Mono.Android/../../bin/Debug/lib/xbuild-frameworks/MonoAndroid/v6.0/../v1.0/" -L ".../xamarin-android/src/Mono.Android/../../MonoAndroid/v6.0/../v1.0/Facades" ".../xamarin-android/src/Mono.Android/../../bin/Debug/lib/xbuild-frameworks/MonoAndroid/v6.0/Mono.Android.dll" jcw-gen: Could not load assembly 'mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089'. Perhaps it doesn't exist in the Mono for Android profile? `mscorlib.dll` couldn't be found because it wasn't *built*, nor installed into `bin/Debug/lib/xbuild-frameworks/MonoAndroid/v1.0`. Additionally, fully building the AOT cross compilers requires adding `mxe-Win32` to `$(ALL_HOST_ABIS)`, otherwise e.g. `win-armeabi` fails to build because `arm_dpimacros.h` doesn't exist (as `arm_dpimacros.h` is built by `mxe-Win32`): Error parsing '.../xamarin-android/external/mono/mono/metadata/metadata-cross-helpers.c' .../xamarin-android/external/mono/mono/arch/arm/arm-codegen.h(937,10): fatal: 'mono/arch/arm/arm_dpimacros.h' file not found Error parsing '.../xamarin-android/external/mono/mono/mini/mini-cross-helpers.c' .../xamarin-android/external/mono/mono/arch/arm/arm-codegen.h(937,10): fatal: 'mono/arch/arm/arm_dpimacros.h' file not found Unhandled Exception: System.Exception: Expected to find struct definition for MonoObject Fix the `mono-runtimes.mdproj` build so that it doesn't re-execute targets when not necessary, by "rolling" with xbuild limitations and refactoring item groups as appropriate. In particular, xbuild doesn't like referencing item metadata within a `//Target/@Outputs` that in turn references a property group: <PropertyGroup> <V>Value</v> </PropertyGroup> <ItemGroup> <Item Include="Something"> <Metadata>$(V)</Metadata> </Item> </ItemGroup> <Target Name="Example" Inputs="..." Outputs="%(Item.Metadata)"> ... </Target> Whenever the `Example` target is run, it will *always* run, even if `Value` exists, because `xbuild` perceives the `//Target[@Name='Example']/@Outputs` value as being `$(V)`, *not* `Value`, which is wonderfully broken. To compensate, Don't Do That™; in particular, item metadata used within `//Target/@Outputs` *must* be hardcoded and not contain a property reference. This "hardcoding" removes the need for the `$(_Cross*ffsetsHeaderFile)` properties, so remove them. Remove `System.Net.Http.WebRequest.dll` from `@(_BclAssembly)`, as it is no longer built in the monodroid profile; requiring it as an output means that the "host" mono build is *always* built, which isn't needed. Improve the `_BuildRuntimes` and `_InstallRuntimes` targets so that their referenced files aren't *so* repetitive, in particular by "splitting out" `@(_RuntimeLibraries)` into `@(_RuntimeSource)`, `@(_ProfilerSource)`, and `@(_MonoPosixHelperSource)`. This allows more intelligently `strip`ing the native libraries in the Release configuration, while also maintaining generation of the `.d.so` files.
2016-07-28 13:24:18 +03:00
/>
[mono-runtimes] "Install" eglib headers into $prefix/include (#164) Context: https://github.com/xamarin/xamarin-android/pull/162 PR #162 tries to use the mono bundle (fbfd676c) to avoid rebuilding mono on subsequent builds (when the mono commit hasn't changed), in order to speed up the Jenkins build process (currently tracking at ~3.5 *hours* for a non-PR build...) [PR #162 currently fails][0] when building `src/monodroid`: Executing: /Users/builder/android-toolchain/ndk/ndk-build CONFIGURATION=Debug NDK_LIBS_OUT=./libs/Debug NDK_OUT=./obj/Debug V=1 ... [armeabi-v7a] Compile thumb : monodroid <= nl.c /Users/builder/android-toolchain/ndk/toolchains/arm-linux-androideabi-4.9/prebuilt/darwin-x86_64/bin/arm-linux-androideabi-gcc -MMD -MP -MF ./obj/Debug/local/armeabi-v7a/objs/monodroid/__/__/__/external/mono/support/nl.o.d -fpic -ffunction-sections -funwind-tables -fstack-protector-strong -no-canonical-prefixes -march=armv7-a -mfpu=vfpv3-d16 -mfloat-abi=softfp -mthumb -Os -g -DNDEBUG -fomit-frame-pointer -fno-strict-aliasing -finline-limit=64 -Ijni/../../../build-tools/mono-runtimes/obj/Debug/armeabi-v7a -Ijni/../../../build-tools/mono-runtimes/obj/Debug/armeabi-v7a/eglib -Ijni/../../../build-tools/mono-runtimes/obj/Debug/armeabi-v7a/eglib/src -Ijni/../../../external/mono/eglib/src -Ijni/zip -Ijni -DANDROID -ggdb3 -O0 -fno-omit-frame-pointer -std=c99 -DHAVE_LINUX_NETLINK_H=1 -DHAVE_LINUX_RTNETLINK_H=1 -D_REENTRANT -DPLATFORM_ANDROID -DANDROID -DLINUX -Dlinux -D__linux_ -DHAVE_CONFIG_H -DJI_DLL_EXPORT -DMONO_DLL_EXPORT -I/libmonodroid/zip -I/include -I/include/eglib -mandroid -fno-strict-aliasing -ffunction-sections -fomit-frame-pointer -funswitch-loops -finline-limit=300 -fvisibility=hidden -fstack-protector -Wa,--noexecstack -Wformat -Werror=format-security -DDEBUG=1 -Wa,--noexecstack -Wformat -Werror=format-security -isystem /Users/builder/android-toolchain/ndk/platforms/android-9/arch-arm/usr/include -c jni/../../../external/mono/support/nl.c -o ./obj/Debug/local/armeabi-v7a/objs/monodroid/__/__/__/external/mono/support/nl.o In file included from jni/../../../external/mono/support/nl.h:3:0, from jni/../../../external/mono/support/nl.c:14: jni/../../../external/mono/eglib/src/glib.h:18:26: fatal error: eglib-config.h: No such file or directory #include <eglib-config.h> ^ compilation terminated. `src/monodroid` fails to build because it requires build artifacts located in `build-tools/mono-runtimes/obj/$(Configuration)/%(Identity)/eglib`, which doesn't exist when the mono bundle is being used (because we never built mono!) Fix this issue by updating the `_InstallRuntimes` target in `mono-runtimes.targets` to copy the generated eglib header files into `bin/$(Configuration)/include/%(Identity)/eglib`, and update `bundle.targets` so that the `include` directory is also included in the mono bundle. This should allow a future PR #162 fix to build, as all required generated files will be present within the bundle. Additionally, add the `$(HostOS)` and a version number to the bundle filename, now `-v1`, to note that the structure of the bundle has changed, *without* a corresponding mono, llvm, or libzip commit bump. [0]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android-pr-builder/61/
2016-08-14 15:52:04 +03:00
<_RuntimeEglibHeaderSource
Condition=" '%(_MonoRuntime.DoBuild)' == 'True' "
Bump to mono/2017-10/7e85cfa5 (#943) Bump to xamarin-android-api-compatibility/master/c050ba3c. Updates expected APIs for BCL assemblies. Use `azureedge.net` instead of `blob.core.windows.net`. It's behind a CDN and way faster outside US. Use the new profiler init symbol name, `mono_profiler_init()`. Avoid dumping the content of `libmono-profiler-log.so` when tests fail. The binary contents aren't meaningful, and needlessly bloat the output log. Fix `eglib` path, as mono changed it's location. Don't skip errors from mono's `make test` target anymore. Stop building Mono's `net_4_x` profile. It isn't required to build the `monodroid` profile and *not* building it saves lots of time. The tools needed by XA are taken from the new `monodroid_tools` profile instead. Update Mono version phrasing in `README.md`. Using an explicit version is just a recipe for outdated info, instead rewrite it to be generic since the build will complain on an outdated Mono anyway. Add more exclusions to `LinkerDescription.xml` for BCL tests, which are used in the new `System.Reflection.Emit.SaveTest()`. Update `.gitignore` for some paths. `Xamarin.Android.Common.props` is generated as part of the build, and the `.rawproto` files created by gradle should be ignored. Context: https://bugzilla.xamarin.com/show_bug.cgi?id=4786 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=5798 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=5804 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=11696 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=17325 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=23153 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=25755 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=36053 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=36634 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=40556 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=41279 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=43172 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=43563 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=44266 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=46683 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=52157 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=52621 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=55603 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=56653 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=56684 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57069 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57160 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57197 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57222 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57232 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57301 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57476 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57488 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57496 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57505 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57528 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57629 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57691 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57744 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57796 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57850 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57851 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57887 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57918 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57930 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57936 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57959 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58100 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58168 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58210 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58291 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58317 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58344 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58361 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58379 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58399 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58411 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58421 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58423 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58454 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58612 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58621 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58728 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58738 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58770 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58782 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58793 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58809 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58829 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58877 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58901 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58911 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58965 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=59039 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=59154 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=59180 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=59235 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=59281 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=59364 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=59436 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=59608 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=59664 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=59824 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=60233
2017-10-31 18:55:44 +03:00
Include="@(_MonoRuntime->'$(IntermediateOutputPath)\%(Identity)\mono\eglib\eglib-config.h')"
[mono-runtimes] "Install" eglib headers into $prefix/include (#164) Context: https://github.com/xamarin/xamarin-android/pull/162 PR #162 tries to use the mono bundle (fbfd676c) to avoid rebuilding mono on subsequent builds (when the mono commit hasn't changed), in order to speed up the Jenkins build process (currently tracking at ~3.5 *hours* for a non-PR build...) [PR #162 currently fails][0] when building `src/monodroid`: Executing: /Users/builder/android-toolchain/ndk/ndk-build CONFIGURATION=Debug NDK_LIBS_OUT=./libs/Debug NDK_OUT=./obj/Debug V=1 ... [armeabi-v7a] Compile thumb : monodroid <= nl.c /Users/builder/android-toolchain/ndk/toolchains/arm-linux-androideabi-4.9/prebuilt/darwin-x86_64/bin/arm-linux-androideabi-gcc -MMD -MP -MF ./obj/Debug/local/armeabi-v7a/objs/monodroid/__/__/__/external/mono/support/nl.o.d -fpic -ffunction-sections -funwind-tables -fstack-protector-strong -no-canonical-prefixes -march=armv7-a -mfpu=vfpv3-d16 -mfloat-abi=softfp -mthumb -Os -g -DNDEBUG -fomit-frame-pointer -fno-strict-aliasing -finline-limit=64 -Ijni/../../../build-tools/mono-runtimes/obj/Debug/armeabi-v7a -Ijni/../../../build-tools/mono-runtimes/obj/Debug/armeabi-v7a/eglib -Ijni/../../../build-tools/mono-runtimes/obj/Debug/armeabi-v7a/eglib/src -Ijni/../../../external/mono/eglib/src -Ijni/zip -Ijni -DANDROID -ggdb3 -O0 -fno-omit-frame-pointer -std=c99 -DHAVE_LINUX_NETLINK_H=1 -DHAVE_LINUX_RTNETLINK_H=1 -D_REENTRANT -DPLATFORM_ANDROID -DANDROID -DLINUX -Dlinux -D__linux_ -DHAVE_CONFIG_H -DJI_DLL_EXPORT -DMONO_DLL_EXPORT -I/libmonodroid/zip -I/include -I/include/eglib -mandroid -fno-strict-aliasing -ffunction-sections -fomit-frame-pointer -funswitch-loops -finline-limit=300 -fvisibility=hidden -fstack-protector -Wa,--noexecstack -Wformat -Werror=format-security -DDEBUG=1 -Wa,--noexecstack -Wformat -Werror=format-security -isystem /Users/builder/android-toolchain/ndk/platforms/android-9/arch-arm/usr/include -c jni/../../../external/mono/support/nl.c -o ./obj/Debug/local/armeabi-v7a/objs/monodroid/__/__/__/external/mono/support/nl.o In file included from jni/../../../external/mono/support/nl.h:3:0, from jni/../../../external/mono/support/nl.c:14: jni/../../../external/mono/eglib/src/glib.h:18:26: fatal error: eglib-config.h: No such file or directory #include <eglib-config.h> ^ compilation terminated. `src/monodroid` fails to build because it requires build artifacts located in `build-tools/mono-runtimes/obj/$(Configuration)/%(Identity)/eglib`, which doesn't exist when the mono bundle is being used (because we never built mono!) Fix this issue by updating the `_InstallRuntimes` target in `mono-runtimes.targets` to copy the generated eglib header files into `bin/$(Configuration)/include/%(Identity)/eglib`, and update `bundle.targets` so that the `include` directory is also included in the mono bundle. This should allow a future PR #162 fix to build, as all required generated files will be present within the bundle. Additionally, add the `$(HostOS)` and a version number to the bundle filename, now `-v1`, to note that the structure of the bundle has changed, *without* a corresponding mono, llvm, or libzip commit bump. [0]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android-pr-builder/61/
2016-08-14 15:52:04 +03:00
/>
<_RuntimeEglibHeaderOutput
Condition=" '%(_MonoRuntime.DoBuild)' == 'True' "
Bump to mono/2017-10/7e85cfa5 (#943) Bump to xamarin-android-api-compatibility/master/c050ba3c. Updates expected APIs for BCL assemblies. Use `azureedge.net` instead of `blob.core.windows.net`. It's behind a CDN and way faster outside US. Use the new profiler init symbol name, `mono_profiler_init()`. Avoid dumping the content of `libmono-profiler-log.so` when tests fail. The binary contents aren't meaningful, and needlessly bloat the output log. Fix `eglib` path, as mono changed it's location. Don't skip errors from mono's `make test` target anymore. Stop building Mono's `net_4_x` profile. It isn't required to build the `monodroid` profile and *not* building it saves lots of time. The tools needed by XA are taken from the new `monodroid_tools` profile instead. Update Mono version phrasing in `README.md`. Using an explicit version is just a recipe for outdated info, instead rewrite it to be generic since the build will complain on an outdated Mono anyway. Add more exclusions to `LinkerDescription.xml` for BCL tests, which are used in the new `System.Reflection.Emit.SaveTest()`. Update `.gitignore` for some paths. `Xamarin.Android.Common.props` is generated as part of the build, and the `.rawproto` files created by gradle should be ignored. Context: https://bugzilla.xamarin.com/show_bug.cgi?id=4786 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=5798 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=5804 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=11696 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=17325 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=23153 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=25755 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=36053 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=36634 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=40556 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=41279 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=43172 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=43563 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=44266 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=46683 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=52157 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=52621 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=55603 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=56653 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=56684 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57069 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57160 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57197 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57222 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57232 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57301 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57476 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57488 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57496 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57505 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57528 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57629 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57691 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57744 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57796 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57850 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57851 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57887 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57918 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57930 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57936 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57959 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58100 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58168 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58210 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58291 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58317 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58344 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58361 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58379 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58399 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58411 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58421 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58423 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58454 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58612 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58621 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58728 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58738 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58770 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58782 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58793 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58809 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58829 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58877 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58901 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58911 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58965 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=59039 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=59154 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=59180 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=59235 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=59281 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=59364 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=59436 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=59608 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=59664 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=59824 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=60233
2017-10-31 18:55:44 +03:00
Include="@(_MonoRuntime->'$(_OutputIncludeDir)%(Identity)\eglib\eglib-config.h')"
[mono-runtimes] "Install" eglib headers into $prefix/include (#164) Context: https://github.com/xamarin/xamarin-android/pull/162 PR #162 tries to use the mono bundle (fbfd676c) to avoid rebuilding mono on subsequent builds (when the mono commit hasn't changed), in order to speed up the Jenkins build process (currently tracking at ~3.5 *hours* for a non-PR build...) [PR #162 currently fails][0] when building `src/monodroid`: Executing: /Users/builder/android-toolchain/ndk/ndk-build CONFIGURATION=Debug NDK_LIBS_OUT=./libs/Debug NDK_OUT=./obj/Debug V=1 ... [armeabi-v7a] Compile thumb : monodroid <= nl.c /Users/builder/android-toolchain/ndk/toolchains/arm-linux-androideabi-4.9/prebuilt/darwin-x86_64/bin/arm-linux-androideabi-gcc -MMD -MP -MF ./obj/Debug/local/armeabi-v7a/objs/monodroid/__/__/__/external/mono/support/nl.o.d -fpic -ffunction-sections -funwind-tables -fstack-protector-strong -no-canonical-prefixes -march=armv7-a -mfpu=vfpv3-d16 -mfloat-abi=softfp -mthumb -Os -g -DNDEBUG -fomit-frame-pointer -fno-strict-aliasing -finline-limit=64 -Ijni/../../../build-tools/mono-runtimes/obj/Debug/armeabi-v7a -Ijni/../../../build-tools/mono-runtimes/obj/Debug/armeabi-v7a/eglib -Ijni/../../../build-tools/mono-runtimes/obj/Debug/armeabi-v7a/eglib/src -Ijni/../../../external/mono/eglib/src -Ijni/zip -Ijni -DANDROID -ggdb3 -O0 -fno-omit-frame-pointer -std=c99 -DHAVE_LINUX_NETLINK_H=1 -DHAVE_LINUX_RTNETLINK_H=1 -D_REENTRANT -DPLATFORM_ANDROID -DANDROID -DLINUX -Dlinux -D__linux_ -DHAVE_CONFIG_H -DJI_DLL_EXPORT -DMONO_DLL_EXPORT -I/libmonodroid/zip -I/include -I/include/eglib -mandroid -fno-strict-aliasing -ffunction-sections -fomit-frame-pointer -funswitch-loops -finline-limit=300 -fvisibility=hidden -fstack-protector -Wa,--noexecstack -Wformat -Werror=format-security -DDEBUG=1 -Wa,--noexecstack -Wformat -Werror=format-security -isystem /Users/builder/android-toolchain/ndk/platforms/android-9/arch-arm/usr/include -c jni/../../../external/mono/support/nl.c -o ./obj/Debug/local/armeabi-v7a/objs/monodroid/__/__/__/external/mono/support/nl.o In file included from jni/../../../external/mono/support/nl.h:3:0, from jni/../../../external/mono/support/nl.c:14: jni/../../../external/mono/eglib/src/glib.h:18:26: fatal error: eglib-config.h: No such file or directory #include <eglib-config.h> ^ compilation terminated. `src/monodroid` fails to build because it requires build artifacts located in `build-tools/mono-runtimes/obj/$(Configuration)/%(Identity)/eglib`, which doesn't exist when the mono bundle is being used (because we never built mono!) Fix this issue by updating the `_InstallRuntimes` target in `mono-runtimes.targets` to copy the generated eglib header files into `bin/$(Configuration)/include/%(Identity)/eglib`, and update `bundle.targets` so that the `include` directory is also included in the mono bundle. This should allow a future PR #162 fix to build, as all required generated files will be present within the bundle. Additionally, add the `$(HostOS)` and a version number to the bundle filename, now `-v1`, to note that the structure of the bundle has changed, *without* a corresponding mono, llvm, or libzip commit bump. [0]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android-pr-builder/61/
2016-08-14 15:52:04 +03:00
/>
[mono-runtimes] Bundle Consts.cs, use correct opt.exe install location (#165) Context: https://github.com/xamarin/xamarin-android/pull/162 PR #162 tries to use the mono bundle (fbfd676c) to avoid rebuilding mono on subsequent builds (when the mono commit hasn't changed), in order to speed up the Jenkins build process (currently tracking at ~3.5 *hours* for a non-PR build...) [PR #162 currently fails][0] when building `src/Mono.Data.Sqlite`: CSC: error CS2001: Source file `/Users/builder/jenkins/workspace/xamarin-android-pr-builder/xamarin-android/external/mono/mcs/build/common/Consts.cs' could not be found Fix this error by copying `external/mono/mcs/build/common/Consts.cs` into `bin/$(Configuration)/include`, and updating the `Consts.cs` references within `Mono.Data.Sqlite.csproj` and `Mono.Posix.csproj` so that they'll use the `include` copy. This should allow us to use the bundle to build these projects. Additionally, `llc.exe` and `opt.exe` are installed into `$prefix/bin`, which isn't correct; `$prefix/bin` as a construct doesn't exist on Windows. Install these into `$prefix/lib/mandroid`. Update the bundle version to note these bundle changes. [0]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android-pr-builder/67/console
2016-08-15 00:54:15 +03:00
<_MonoConstsSource
Condition=" '%(_MonoRuntime.DoBuild)' == 'True' "
Include="$(MonoSourceFullPath)\mcs\build\common\Consts.cs"
/>
<_MonoConstsOutput
Condition=" '%(_MonoRuntime.DoBuild)' == 'True' "
[structure] Rework installation directory structure (#704) Context: https://github.com/xamarin/xamarin-android/pull/253#discussion_r82862993 The *intention* is that Jenkins-produced `oss-xamarin.android*.zip` artifacts be usable on Windows, so that side-by-side testing can be performed without replacing the system installation. Usage is in [UsingJenkinsBuildArtifacts.md](Documentation/UsingJenkinsBuildArtifacts). This isn't *entirely* the case. It was *apparently* the case at the time of commit 87ca2737, but things have bitrotten since. For example, following the 87ca2737 instructions would currently result in an XA0020 `Could not find monodroid` error, because `class-parse.exe` wasn't where Windows expects it to be (it was in `lib/mandroid`, not `lib/xbuild/Xamarin/Android`). This needs to be fixed. Additionally, PR #253 mentions that, for filesystem organization, it would be useful if the macOS/Linux directory structure -- `$prefix/bin`, `$prefix/lib/mandroid`, `$prefix/lib/xbuild/Xamarin/Android`, `$prefix/lib/xbuild-frameworks` -- more closely resembled the Windows directory structure of `$MSBuild` and `$ReferenceAssemblies` (as seen in `.vsix` files). This would turn macOS/Linux into using `$xa_prefix/xbuild` and `$xa_prefix/xbuild-frameworks` directories. `$prefix/bin` would only contain `xabuild`. What is currently in `$prefix/lib/mandroid` would be merged with `$xa_prefix/xbuild/Xamarin/Android`. `$xa_prefix` would `$prefix/lib/xamarin.android`. This would turn the current macOS structure: $prefix/bin/xabuild $prefix/bin/generator $prefix/bin/cross-arm $prefix/lib/mandroid/generator.exe $prefix/lib/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xbuild/Xamarin/Android/Xamarin.Android.Common.targets Into: $prefix/bin/xabuild $prefix/lib/xamarin.android/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xamarin.android/xbuild/Xamarin/Android/generator.exe $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Xamarin.Android.Common.targets $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Darwin/cross-arm Other notes: * The `bundle-*.zip` filename has been has been changed to include a *hash* of the contents of various files, in particular `build-tools\mono-runtimes.*`. This was instigated via a conversation with @kumpera about making the filename more "idiot-proof": `mono-runtimes.props` contains *compiler flags*, and if any of those change, then *logically* the bundle should differ as well, as the mono runtimes may differ in significant ways. In theory, the `-vXX` version should be used to track this, but this is a manual change, easy to overlook. The new `-hHASH` part of the filename should be more automagic. The new `<HashFileContents/>` task in `xa-prep-tasks.dll` is responsible for creating the hash value. * `Configuration.Java.Interop.Override.props` was moved into `build-tools/scripts`, because that would cleanup the root directory a bit. * OS-specific binaries are now placed into `$prefix/lib/xamarin.android/xbuild/Xamarin/Android/$(HostOS)`. On a theoretical plus side, this means that the same build directory can contain OS-specific binaries for multiple operating systems. (I don't know if anyone shares a build tree between e.g. macOS and Linux, but if anyone *does*...) Unfortunately this requires a workaround for an `xbuild` bug: `%(_LlvmRuntime.InstallPath)` and `%(_MonoCrossRuntime.InstallPath)` *cannot* use MSBuild properties, e.g. `<InstallPath>$(HostOS)/</InstallPath>` doesn't work as desired; it's instead treated literally. Special-case `%(InstallPath)` until we fully migrate to MSBuild. * `$(MonoAndroidToolsDirectory)` should be considered *dead*, along with `$(MonoAndroidBinDirectory)`, as these should now *always* be the same directory as where `Xamarin.Android.Build.Tasks.dll` is located, or a `$(HostOS)` sub-directory. * `Xamarin.ProjectTools.csproj` needed to be updated to ensure that the build order was correct. * Remove all `[Obsolete]` and unreferenced members from `Xamarin.Android.Build.Utilities.dll`. There's too much in there, and it makes my head hurt trying to understand the interrelationships between it all. If it's not used, it's gone. * The changes to `src/monodroid/jni/Android.mk` are...weird. The removal of `-I$(topdir)/libmonodroid/zip`/etc. is to reduce use of non-existent paths, as `$(topdir)` isn't defined, so that's *actually* `-I/libmonodroid/zip`, which is nonsensical. So far, so good. What's *odd* is the addition of `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`: This is needed so that `external/mono/support/zlib-helper.c` exports `CreateZStream` and related symbols, otherwise we get a unit test failure in `GzipStreamTest.Compression` due to an `EntryPointNotFoundException`, because `CreateZStream` isn't exported/public. What's odd here is that I don't understand what caused this behavior to change. Previous builds exported `CreateZStream`, otherwise the tests would fail, and I don't understand how any of the other changes in this PR would be at fault, though that's certainly the most plausible explanation. Regardless, `-Ijni` *first* (due to adding `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`) is the desired behavior, so that `jni/config.h` is included, thus ensuring that `MONO_API` has the required definition when building `zlib-helper.c`.
2017-07-28 16:36:32 +03:00
Include="$(_OutputIncludeDir)Consts.cs"
[mono-runtimes] Bundle Consts.cs, use correct opt.exe install location (#165) Context: https://github.com/xamarin/xamarin-android/pull/162 PR #162 tries to use the mono bundle (fbfd676c) to avoid rebuilding mono on subsequent builds (when the mono commit hasn't changed), in order to speed up the Jenkins build process (currently tracking at ~3.5 *hours* for a non-PR build...) [PR #162 currently fails][0] when building `src/Mono.Data.Sqlite`: CSC: error CS2001: Source file `/Users/builder/jenkins/workspace/xamarin-android-pr-builder/xamarin-android/external/mono/mcs/build/common/Consts.cs' could not be found Fix this error by copying `external/mono/mcs/build/common/Consts.cs` into `bin/$(Configuration)/include`, and updating the `Consts.cs` references within `Mono.Data.Sqlite.csproj` and `Mono.Posix.csproj` so that they'll use the `include` copy. This should allow us to use the bundle to build these projects. Additionally, `llc.exe` and `opt.exe` are installed into `$prefix/bin`, which isn't correct; `$prefix/bin` as a construct doesn't exist on Windows. Install these into `$prefix/lib/mandroid`. Update the bundle version to note these bundle changes. [0]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android-pr-builder/67/console
2016-08-15 00:54:15 +03:00
/>
[mono-runtimes] Use directory separators with `$(OutputPath)`. [The build][0] [is broken][1]: xamarin-android/build-tools/mono-runtimes/mono-runtimes.targets: error : Could not find a part of the path ".../xamarin-android/xamarin-android/bin/Debuglib/xbuild-frameworks/MonoAndroid/v1.0/RedistList/FrameworkList.xml". The error is generated from the `_InstallBcl` target when writing `FrameworkList.xml`, and a close reading of the error message shows that a directory separator is missing: it should be `bin/Debug/lib`, but is instead `bin/Debuglib`. I *assume* that this is due to `<CallTarget/>` being "flakey" within xbuild (it passes the file containing the target to `<MSBuild/>`!), and *somehow* in that "flakiness" `$(OutputPath)` doesn't end with `\`, *which it should*, because `Microsoft.Common.targets` will automatically append `\` to `$(OutputPath)`. To work around this issue, update `mono-runtimes.targets` so that it *always* uses `\` after `$(OutputPath)` and `$(IntermediateOutputPath)`. Additionally, `llc.exe` was being installed into the wrong directory, preventing the bundle from being used: Target _BuildUnlessCached needs to be built as output file '../../bin/Debug/lib/mandroid/llc.exe' does not exist. `llc.exe` was instead installed into `bin/Debug/bin/llc.exe`. Bump the bundle version number to -v3 and correct `mono-runtimes.targets` so that `llc.exe` and `opt.exe` are properly installed into `$prefix/lib/mandroid`. [0]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android/49/ [1]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android/49/consoleText
2016-08-17 04:18:15 +03:00
<_RuntimeBuildStamp Condition=" '%(_MonoRuntime.DoBuild)' == 'true' " Include="@(_MonoRuntime->'$(IntermediateOutputPath)\%(Identity)\.stamp')" />
[mxe] Add Windows cross-compiler support. (#55) Certain Xamarin.Android features require that Mono be built for Windows, e.g. the [AOT compilers][aot] require a build of mono that executes on Windows to generate the AOT native libraries. Unfortunately, building Mono on Windows continues to be a massive PITA. (Autotools on Windows requires Cygwin/mingw, running shell scripts on Windows is painfully slow, it's all brittle, etc.) To work around this pain, we instead build the Mono/Windows binaries on OS X, via [MXE][mxe], which produces a gcc-based cross-compiler which generates Windows binaries and is executable from Unix. This in turn requires that we have MXE, so add a `_CreateMxeToolchains` target to `android-toolchain.targets` which will build MXE. The installation prefix for MXE can be overridden via the new `$(AndroidMxeInstallPrefix)` MSBuild property; it defaults to `$HOME/android-toolchain/mxe`. Rework the `$(AndroidSupportedAbis)` MSBuild property so that it must include the "host" ABI, and add support for a new `host-win64` value which will use MXE to generate 64-bit Windows binaries for libmonosgen-2.0.dll and libMonoPosixHelper.dll. We can't always process `host-$(HostOS)` because of an xbuild bug. The scenario is that if you want to just build `host-win64`, the obvious thing to do is: cd build-tools/mono-runtimes xbuild /p:AndroidSupportedAbis=host-win64 Alas, if `host-$(HostOS)` is always processed, this inexplicably causes `host-$(HostOS)` to be re-rebuilt, which (1) is a waste of time, and (2) fails -- inexplicably -- in the `_BuildRuntimes` target because make(1) thinks that the configure flags have somehow changed, which currently makes no sense at all. (When can we move to MSBuild?) Changing `$(AndroidSupportedAbis)` so that `host-$(HostOS)` is explicitly processed instead of implicitly processed allows working around the above xbuild bug, as `host-$(HostOS)` won't be implicitly processed on every build, but only when required. Additionally, we add a new <Which/> MSBuild task so that we can determine if a particular program is in `$PATH`. This is useful because listing requirements within README.md is a road to pain -- e.g. xxd(1) is required to build `src/monodroid` but if it's missing it'll still *build* but you'll instead get a *linker* failure because the `monodroid_config` and `monodroid_machine_config` symbols aren't present. Building MXE requires that even more programs be present within $PATH, so explicitly check for these so that *useful* error messages can be generated instead of obscure ones. Finally, a note about autotools and generating Windows native libraries: creation of `.dll` files *requires* that an appropriate objdump be present so it can determine if e.g. `libkernel32.a` is an import library or an archive. If `x86_64-w64-mingw32.static-objdump` isn't found -- e.g. because $PATH doesn't contain it -- then no `.dll` files will be created, and much head scratching will occur. To rectify this, override the OBJDUMP and DLLTOOL values when invoking `configure` so that that full paths are used and `$PATH` use is reduced. (Who wants `x86_64-w64-mingw32.static-objdump` in `$PATH`?) [aot]: https://developer.xamarin.com/releases/android/xamarin.android_5/xamarin.android_5.1/#AOT_Support [mxe]: http://mxe.cc/
2016-06-07 00:12:49 +03:00
</ItemGroup>
</Target>
[mono-runtimes] Build required mono runtimes. Add a new build-tools/mono-runtimes project to build the required Mono runtimes for Xamarin.Android app execution on Android devices. Xamarin.Android 6.0 ("cycle 6") includes Mono runtimes compiled for five architectures: * armeabi (armv5) * armeabi-v7a (armv7) * arm64-v8a (aarch64) * x86 * x86_64 An additional "host" architecture is also built to build the "monodroid" profile assemblies such as mscorlib.dll. In the interest of expediency and not requiring that *six* different runtimes be built -- which can greatly increase build times and storage requirements -- the initial xamarin-android open-source build will only build the armeabi-v7a (armv7) and "host" architectures. Additionally, the <UnzipDirectoryChildren/> task needed to be altered to use unzip(1) on OS X instead of using System.IO.Compression.ZipFile, because the ZipFile implementation included with Mono 4.4 doesn't respect file attributes such as the +x, causing all files to be extracted as 644 (rw-r--r--), resulting in an NDK toolchain which wouldn't work. To resolve this issue, execution on OS X and Linux now uses unzip(1). ~~ Adding new architectures ~~ To build Mono for a new architecture, edit build-tools/mono-runtimes/mono-runtimes.projitems and add a new @(_MonoRuntime) entry for the desired architecture. TODO: Figure out if there's a reasonable way to "opt-in" or "opt-out" of CPU architectures so that it isn't an "all or nothing" prospect.
2016-04-20 04:30:00 +03:00
<Target Name="_BuildRuntimes"
Correct "Exists" logic on monodoc debug files The previous commit's attempts to copy the pdb/mdb files for monodoc had a problem where they would be evaluated too early (probably before the files existed) and in the wrong directory. To address this, the files are instead specified as a MonoDocCopyItemOptional, a new target _GetMonodocItems is created and the ItemGroups for _MonoDocCopyItems/_MonoDocInstalledItems are evaluated in this target with correct Exists logic for the Optional files. Additional DependsOnTargets attributes were distributed through the file to ensure this new target is evaluated at the correct time.
2017-03-17 02:16:53 +03:00
DependsOnTargets="_GetRuntimesOutputItems;_ConfigureRuntimes"
[build] Build the monodroid-profile BCL assemblies reliably (#135) Commit 37b5fd51 broke `make jenkins` (and possible `make all`?) through the introduction of the `%(_MonoRuntime.DoBuild)` metadata: `%(_MonoRuntime.DoBuild)` wasn't added everywhere required. In particular, the `host-Darwin` item didn't gain `<DoBuild/>`, meaning *`host-Darwin` wasn't built*. This in turn meant that the monodroid BCL wasn't built (!) which caused a build break much later within `src/Mono.Android`: Executing: mono .../xamarin-android/external/Java.Interop/bin/Debug/jcw-gen.exe -o ".../xamarin-android/src/Mono.Android/obj/Debug/android-23/jcw/src" -L ".../xamarin-androidg/lib/xbuild-frameworks/MonoAndroid/v6.0/" -L ".../xamarin-android/src/Mono.Android/../../bin/Debug/lib/xbuild-frameworks/MonoAndroid/v6.0/../v1.0/" -L ".../xamarin-android/src/Mono.Android/../../MonoAndroid/v6.0/../v1.0/Facades" ".../xamarin-android/src/Mono.Android/../../bin/Debug/lib/xbuild-frameworks/MonoAndroid/v6.0/Mono.Android.dll" jcw-gen: Could not load assembly 'mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089'. Perhaps it doesn't exist in the Mono for Android profile? `mscorlib.dll` couldn't be found because it wasn't *built*, nor installed into `bin/Debug/lib/xbuild-frameworks/MonoAndroid/v1.0`. Additionally, fully building the AOT cross compilers requires adding `mxe-Win32` to `$(ALL_HOST_ABIS)`, otherwise e.g. `win-armeabi` fails to build because `arm_dpimacros.h` doesn't exist (as `arm_dpimacros.h` is built by `mxe-Win32`): Error parsing '.../xamarin-android/external/mono/mono/metadata/metadata-cross-helpers.c' .../xamarin-android/external/mono/mono/arch/arm/arm-codegen.h(937,10): fatal: 'mono/arch/arm/arm_dpimacros.h' file not found Error parsing '.../xamarin-android/external/mono/mono/mini/mini-cross-helpers.c' .../xamarin-android/external/mono/mono/arch/arm/arm-codegen.h(937,10): fatal: 'mono/arch/arm/arm_dpimacros.h' file not found Unhandled Exception: System.Exception: Expected to find struct definition for MonoObject Fix the `mono-runtimes.mdproj` build so that it doesn't re-execute targets when not necessary, by "rolling" with xbuild limitations and refactoring item groups as appropriate. In particular, xbuild doesn't like referencing item metadata within a `//Target/@Outputs` that in turn references a property group: <PropertyGroup> <V>Value</v> </PropertyGroup> <ItemGroup> <Item Include="Something"> <Metadata>$(V)</Metadata> </Item> </ItemGroup> <Target Name="Example" Inputs="..." Outputs="%(Item.Metadata)"> ... </Target> Whenever the `Example` target is run, it will *always* run, even if `Value` exists, because `xbuild` perceives the `//Target[@Name='Example']/@Outputs` value as being `$(V)`, *not* `Value`, which is wonderfully broken. To compensate, Don't Do That™; in particular, item metadata used within `//Target/@Outputs` *must* be hardcoded and not contain a property reference. This "hardcoding" removes the need for the `$(_Cross*ffsetsHeaderFile)` properties, so remove them. Remove `System.Net.Http.WebRequest.dll` from `@(_BclAssembly)`, as it is no longer built in the monodroid profile; requiring it as an output means that the "host" mono build is *always* built, which isn't needed. Improve the `_BuildRuntimes` and `_InstallRuntimes` targets so that their referenced files aren't *so* repetitive, in particular by "splitting out" `@(_RuntimeLibraries)` into `@(_RuntimeSource)`, `@(_ProfilerSource)`, and `@(_MonoPosixHelperSource)`. This allows more intelligently `strip`ing the native libraries in the Release configuration, while also maintaining generation of the `.d.so` files.
2016-07-28 13:24:18 +03:00
Inputs="@(_RuntimeBuildStamp)"
[Xamarin.Android.Bcl-Tests] Add BCL test project. (#872) What do we want? (with apologies to 48e3fc26) **MOAR** Unit tests! Specifically, we want to run the BCL unit tests which mono generates: $ cd external/mono/mcs/class/corlib $ make PROFILE=monodroid test # creates `monodroid_corlib_test.dll` Creation of `monodroid_*_test.dll` assemblies and the `make PROFILE=monodroid test` target is a relatively recent development, for which I need to buy the #runtime team some beers. In terms of `mono-runtimes.targets`, we can build *all* of the BCL unit test assemblies with: $ cd external/mono/mcs/class $ make -i do-test PROFILE=monodroid Now that we can create them, how do we *use* them? That's the trickier bit: they need to be built within mono, as part of the existing BCL build process. This in turn means that the BCL unit test assemblies need to be distributed as part of the mono bundle, as we don't want to rebuild the mono repo "from scratch" just for the unit tests. Update `build-tools/mono-runtimes/ProfileAssemblies.projitems` to include a new `@(MonoTestAssembly)` item group which contains all of the BCL unit test assemblies and related files which should be included into `bundle-*.zip`. Additionally, add `ProfileAssemblies.projitems` to `@(VersionFile)` witihin `bundle-path.targets`, so that if anything within `ProfileAssemblies.projitems` changes, we rebuild the bundle. Once we *have* the BCL unit test assemblies, and their dependencies, we need to *run* them. The new `Xamarin.Android.Bcl-Tests.csproj` project is a Xamarin.Android application project which will execute the unit tests. There's just one small problem: Xamarin.Android apps want to use `Xamarin.Android.NUnitLite.dll`. The BCL unit test assemblies instead build against their own `nunitlite.dll`, which has no Xamarin.Android integration or support. How do we use the new test assemblies? *Force* a fix by using `remap-assembly-ref` to "rename" the `nunitlite` assembly reference to `Xamarin.Android.NUnitLite.dll`. This *cannot* be done as part of the `mono-runtimes.mdproj` build, as `Xamarin.Android.NUnitLite.dll` won't yet exist. Instead, remap the assemblies within `Xamarin.Android.Bcl-Tests.targets`, and distribute the remapped assemblies with the application. Finally, address one other "small" problem: not all of the unit tests pass! Some of these are for reasons we don't know, and others will require changes to `mono`. Update `Xamarin.Android.NUnitLite` to allow *filtering* of tests: namespace Xamarin.Android.NUnitLite { partial class TestSuiteActivity { public ITestFilter Filter {get; set;} public virtual void UpdateFilter (); } partial class TestSuiteInstrumentation { public ITestFilter Filter {get; set;} public virtual void UpdateFilter (); } } `TestSuiteActivity.UpdateFilter()` is called by `TestSuiteActivity.OnCreate()`, *after* `GetIncludedCategories()` and `GetExcludedCategories()` are called, to allow subclasses to alter the `ITestFilter` which is used to determine which tests are executed. `TestSuiteInstrumentation.UpdateFilter()` is called by `TestSuiteInstrumentation.OnStart()`, *after* `GetIncludedCategories()` and `GetExcludedCategories()` are called, to allow subclasses to alter the `ITestFilter` which is used to determine which tests are executed. `Xamarin.Android.Bcl_Tests` overrides both of these and updates the `Filter` property so that "known failing" tests are excluded. This allows us to skip failing tests, giving us time to properly fix them in time while allowing the rest of this PR to be merged. The skipped tests include: * MonoTests.System.Reflection.AssemblyTest.GetReferencedAssemblies * MonoTests.System.ServiceModel.Description.WebInvokeAttributeTest.RejectTwoParametersWhenNotWrapped
2017-10-03 22:48:08 +03:00
Outputs="@(_RuntimeSource);@(_ProfilerSource);@(_MonoPosixHelperSource);@(_BclProfileItems);@(_MonoBtlsSource);@(_BclTestOutput)">
[mono-runtimes] Build required mono runtimes. Add a new build-tools/mono-runtimes project to build the required Mono runtimes for Xamarin.Android app execution on Android devices. Xamarin.Android 6.0 ("cycle 6") includes Mono runtimes compiled for five architectures: * armeabi (armv5) * armeabi-v7a (armv7) * arm64-v8a (aarch64) * x86 * x86_64 An additional "host" architecture is also built to build the "monodroid" profile assemblies such as mscorlib.dll. In the interest of expediency and not requiring that *six* different runtimes be built -- which can greatly increase build times and storage requirements -- the initial xamarin-android open-source build will only build the armeabi-v7a (armv7) and "host" architectures. Additionally, the <UnzipDirectoryChildren/> task needed to be altered to use unzip(1) on OS X instead of using System.IO.Compression.ZipFile, because the ZipFile implementation included with Mono 4.4 doesn't respect file attributes such as the +x, causing all files to be extracted as 644 (rw-r--r--), resulting in an NDK toolchain which wouldn't work. To resolve this issue, execution on OS X and Linux now uses unzip(1). ~~ Adding new architectures ~~ To build Mono for a new architecture, edit build-tools/mono-runtimes/mono-runtimes.projitems and add a new @(_MonoRuntime) entry for the desired architecture. TODO: Figure out if there's a reasonable way to "opt-in" or "opt-out" of CPU architectures so that it isn't an "all or nothing" prospect.
2016-04-20 04:30:00 +03:00
<Exec
[mono-runtimes] Build AOT+LLVM cross-compilers (#125) The commit implements building of LLVM and cross-compilers to support Xamarin.Android/Mono AOT. LLVM and cross-compilers can be built for both the host platform (Linux and OS/X at the moment) as well as cross-compiled for 32-bit and 64-bit Windows platforms. Windows builds are done with MXE toolchain on OS/X and with the packaged mingw-w64 toolchain on Linux (tested on Ubuntu 16.04 ONLY). Also introducing a new set of MSBuild properties that contain information about the host system. Some of those properties (HostOS, HostCC, HostCXX for instance) have been moved from Configuration.props to better support auto-detection. A new script, build-tools/scripts/generate-os-info, is invoked as part of `make prepare` to generate file that contains the new properties. The generated file is required for the build to work and is also host-specific (it mustn't be moved between different machines) Cross compiler builds require access to a configured Mono build tree, in order to generate C structure offsets header file that is used by the AOT compilers to properly generate AOT-ed binaries. Therefore, even if a JIT target is not enabled in the configuration, enabling a cross-compiler for some target will configure Mono for that JIT target but it will NOT build it, to save time. To facilitate this, the _MonoRuntimes items defined in build-tools/mono-runtimes/mono-runtimes.projitems gain an additional metadata item called `DoBuild` which will be set to `true` if the runtime actually needs to be built, as opposed to just configured. MXE builds are disabled on Linux as mingw-w64 works just fine. A `make prepare` warning is issued for Linux hosts which have the binfmt_misc module enabled and either Wine of Mono (cli) registered as PE32/PE32+ binary interpreters. In such instance building of the Windows cross-compilers will fail because Autotools determine whether software is being cross compiled by building a test program and attempting to execute it. In normal circumstances such an attempt will fail, but with Windows cross-compilation and either Wine or Mono registered to handle the PE32 executables this attempt will succeed thus causing the cross compilation detection to fail. Currently to build cross compilers on Linux you need to generate the C structure offsets header file on OS/X and copy the resulting headers to appropriate places on Linux. The header files should be placed in build-tools/mono-runtimes/obj/Debug/cross-*/ directories. The header files are: {cross-arm,cross-arm-win}/aarch64-v8a-linux-android.h {cross-arm64,cross-arm64-win}/armv5-none-linux-androideabi.h {cross-x86,cross-x86-win}/i686-none-linux-android.h {cross-x86_64,cross-x86_64-win}/x86_64-none-linux-android.h Offsets header generation doesn't work on Linux atm because of missing support for it in the Mono utility used to generate the offsets. Hopefully this limitation will be removed in the near future and a start-to-end build of everything will be possible on Linux. It is now mandatory to run at least `make prepare-props` before Xamarin.Android can be built. The target generates the OS-specific props file which is required by the build. `make prepare` depends on the target.
2016-07-26 16:27:31 +03:00
Condition=" '%(_MonoRuntime.DoBuild)' == 'true' "
Command="make $(MakeConcurrency) # %(_MonoRuntime.Identity)"
[build] Support building *from clean* with `msbuild` (Take 2!) (#391) Commit 7343965a didn't succeed in actually building `xamarin-android` with `msbuild` on our Jenkins bot. :-( Time to take another stab at it. As an added complication, my local `msbuild` behavior (mono 4.8) differs from Jenkins' `msbuild` behavior (mono 4.4), specifically around MSB3644. [On Jenkins][msb-96], it's a warning: warning MSB3644: The reference assemblies for framework ".NETFramework,Version=v4.5" were not found. When I run locally, it's an *error*. Go figure. Fixing the local error requires that I *remove* the `$(TargetFrameworkRootPath)` MSBuild property from `Configuration.props`, added in commit 255a569b. The issue is that `$(TargetFrameworkRootPath)` is an *override*, an "absolute" override. Thus, when e.g. building `Xamarin.Android.Build.Tasks.csproj`, the `.NETFramework,Version=v4.5` framework -- and corresponding `mscorlib.dll` -- cannot be resolved, and everything breaks. (Which raises the wonderful question of whether removing `$(TargetFrameworkRootPath)` will cause the Linux build to break again, but we'll cross that bridge later...) Removing `$(TargetFrameworkRootPath)` in turn requires that I remove the explicit `@(Reference)`s to `System.Runtime` which commit 7343965a added when building under `xbuild`. This also means that we no longer need the `$(_XABuildingWithXBuild)` property. (The "history" eluded to in 7343965a? It's continuing! Argh!) Additionally, fix the logging from the `<UnzipDirectoryChildren/>` task so that it prints the *correct* task name, *not* `Unzip`. (Seeing the wrong task name in log files makes things *confusing*.) With my local `msbuild` faiklures fixed, we can turn our attention to to the [errors reported on Jenkins][msb-96]. They are variations on the same "form": # from build-tools/mono-runtimes .../external/mono/configure LDFLAGS=... ... EXEC : error : /Applications/Xcode8.1-beta1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/libtool: unknown option character `-' in: --config # from src/sqlite-xamarin .../android create project -g -n sqlitesample -p . -a MainActivity -t android-25 --package com.xamarin.sqlitesample -v 1.0.0 EXEC : error : Project folder '.' is not empty. Please consider using 'android update' instead. The issue here is that MSBuild is "better" than xbuild: xbuild only checks the process exit code for errors. The above `android create project` command has en exit code of 0, i.e. it didn't error...according to xbuild. That said, when running [with xbuild][xb-203]: Task "Exec" Executing: .../android create project -g -n sqlitesample -p . -a MainActivity -t android-25 --package com.xamarin.sqlitesample -v 1.0.0 Environment variables being passed to the tool: Error: Project folder '.' is not empty. Please consider using 'android update' instead ... Done executing task "Exec" Task "Copy" ... Rephrased: `android create project ...` is emitting an "Error: ..." *message*, but even when it emits that message the exit status is still 0 (no error). xbuild is only checking the exit status, and thus continues the build. MSBuild not only checks the exit status, but *also* checks the command *output*, looking for the string `error`. Because an "error" message is printed, MSBuild flags this as an error, halting the build. The fix? Set `Exec.IgnoreStandardErrorWarningFormat=True` on all `<Exec/>` tasks which invoke **make**(1) or `configure` within `build-tools/mono-runtimes` and `src/sqlite-xamarin`. Setting `Exec.IgnoreStandardErrorWarningFormat=True` prevents the `<Exec/>` task from scanning the task output for "error", causing `msbuild` to behave like `xbuild` and only use the process exit code for errors. This allows the build to continue. [msb-96]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android-msbuild/96/consoleText [xb-203]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android/203/consoleText
2017-01-20 15:05:02 +03:00
IgnoreStandardErrorWarningFormat="True"
[mono-runtimes] Build AOT+LLVM cross-compilers (#125) The commit implements building of LLVM and cross-compilers to support Xamarin.Android/Mono AOT. LLVM and cross-compilers can be built for both the host platform (Linux and OS/X at the moment) as well as cross-compiled for 32-bit and 64-bit Windows platforms. Windows builds are done with MXE toolchain on OS/X and with the packaged mingw-w64 toolchain on Linux (tested on Ubuntu 16.04 ONLY). Also introducing a new set of MSBuild properties that contain information about the host system. Some of those properties (HostOS, HostCC, HostCXX for instance) have been moved from Configuration.props to better support auto-detection. A new script, build-tools/scripts/generate-os-info, is invoked as part of `make prepare` to generate file that contains the new properties. The generated file is required for the build to work and is also host-specific (it mustn't be moved between different machines) Cross compiler builds require access to a configured Mono build tree, in order to generate C structure offsets header file that is used by the AOT compilers to properly generate AOT-ed binaries. Therefore, even if a JIT target is not enabled in the configuration, enabling a cross-compiler for some target will configure Mono for that JIT target but it will NOT build it, to save time. To facilitate this, the _MonoRuntimes items defined in build-tools/mono-runtimes/mono-runtimes.projitems gain an additional metadata item called `DoBuild` which will be set to `true` if the runtime actually needs to be built, as opposed to just configured. MXE builds are disabled on Linux as mingw-w64 works just fine. A `make prepare` warning is issued for Linux hosts which have the binfmt_misc module enabled and either Wine of Mono (cli) registered as PE32/PE32+ binary interpreters. In such instance building of the Windows cross-compilers will fail because Autotools determine whether software is being cross compiled by building a test program and attempting to execute it. In normal circumstances such an attempt will fail, but with Windows cross-compilation and either Wine or Mono registered to handle the PE32 executables this attempt will succeed thus causing the cross compilation detection to fail. Currently to build cross compilers on Linux you need to generate the C structure offsets header file on OS/X and copy the resulting headers to appropriate places on Linux. The header files should be placed in build-tools/mono-runtimes/obj/Debug/cross-*/ directories. The header files are: {cross-arm,cross-arm-win}/aarch64-v8a-linux-android.h {cross-arm64,cross-arm64-win}/armv5-none-linux-androideabi.h {cross-x86,cross-x86-win}/i686-none-linux-android.h {cross-x86_64,cross-x86_64-win}/x86_64-none-linux-android.h Offsets header generation doesn't work on Linux atm because of missing support for it in the Mono utility used to generate the offsets. Hopefully this limitation will be removed in the near future and a start-to-end build of everything will be possible on Linux. It is now mandatory to run at least `make prepare-props` before Xamarin.Android can be built. The target generates the OS-specific props file which is required by the build. `make prepare` depends on the target.
2016-07-26 16:27:31 +03:00
WorkingDirectory="$(IntermediateOutputPath)\%(_MonoRuntime.Identity)"
[mono-runtimes] Build required mono runtimes. Add a new build-tools/mono-runtimes project to build the required Mono runtimes for Xamarin.Android app execution on Android devices. Xamarin.Android 6.0 ("cycle 6") includes Mono runtimes compiled for five architectures: * armeabi (armv5) * armeabi-v7a (armv7) * arm64-v8a (aarch64) * x86 * x86_64 An additional "host" architecture is also built to build the "monodroid" profile assemblies such as mscorlib.dll. In the interest of expediency and not requiring that *six* different runtimes be built -- which can greatly increase build times and storage requirements -- the initial xamarin-android open-source build will only build the armeabi-v7a (armv7) and "host" architectures. Additionally, the <UnzipDirectoryChildren/> task needed to be altered to use unzip(1) on OS X instead of using System.IO.Compression.ZipFile, because the ZipFile implementation included with Mono 4.4 doesn't respect file attributes such as the +x, causing all files to be extracted as 644 (rw-r--r--), resulting in an NDK toolchain which wouldn't work. To resolve this issue, execution on OS X and Linux now uses unzip(1). ~~ Adding new architectures ~~ To build Mono for a new architecture, edit build-tools/mono-runtimes/mono-runtimes.projitems and add a new @(_MonoRuntime) entry for the desired architecture. TODO: Figure out if there's a reasonable way to "opt-in" or "opt-out" of CPU architectures so that it isn't an "all or nothing" prospect.
2016-04-20 04:30:00 +03:00
/>
[mono-runtimes] Fix _BuildRuntimes @Outputs checking. The _BuildRuntimes target was executing unnecessarily: Target _BuildRuntimes needs to be built as input file 'obj/Debug//armeabi-v7a/.stamp' is newer than output file 'obj/host/mono/profiler/.libs/libmono-profiler-log.dylib This would *always* be true, because the output files were never touched once they were built, so e.g. when copying a build tree from one machine to another (Dropbox!), the dependency check would be "wrong". Improve the _BuildRuntimes target to explicitly <Touch/> the output files so that -- short of timezone changes (!) -- the outputs will be forced to be newer than the inputs.
2016-04-23 14:54:47 +03:00
<Touch
[Xamarin.Android.Build.Tasks] Reduce rebuild cascades (#549) Context: https://github.com/xamarin/xamarin-android/commit/518e57ca540fe3a1d4a7228ef19468d674238cf9 The scenario: rebuild `Xamarin.Android.Build.Tasks.csproj`: $ xbuild src/Xamarin.Android.Build.Tasks/Xamarin.Android.Build.Tasks.csproj *If nothing has changed*, the expectation is that this should be reasonably quick, because *nothing has changed*. Unfortunately, that's not the case; a rebuild could take upwards of 30sec on my local machine, becaues of rebuild cascades: Target CoreCompile needs to be built as input file 'Xamarin.Android.Tools.BootstrapTasks/GenerateProfile.cs' is newer than output file 'obj/Debug/Xamarin.Android.Tools.BootstrapTasks.dll' Target _BuildJNIEnv needs to be built as input file '../../bin/BuildDebug/jnienv-gen.exe' is newer than output file 'Android.Runtime/JNIEnv.g.cs' Target CoreCompile needs to be built as input file 'Android.Runtime/JNIEnv.g.cs' is newer than output file 'obj/Debug/android-25/Mono.Android.dll' Target _GenerateMonoAndroidDex18 needs to be built as input file '../../bin/Debug/lib/xbuild-frameworks/MonoAndroid/v7.1/mono.android.jar' is newer than output file '../../bin/Debug/lib/xbuild-frameworks/MonoAndroid/v7.1/mono.android.dex' Target _CopyExtractedMultiDexJar needs to be built as input file '$HOME/android-toolchain/sdk/extras/android/m2repository/com/android/support/multidex/1.0.1/multidex-1.0.1.aar' is newer than output file '../../bin/Debug/lib/xbuild/Xamarin/Android/../../../mandroid/android-support-multidex.jar' ...and if I'm *really* unlucky: Target _BuildUnlessCached needs to be built as input file '.../xamarin-android/external/mono/autogen.sh' is newer than output file '../../bin/Debug//lib/xbuild-frameworks/MonoAndroid/v1.0/FSharp.Core.dll' ...as that means an `external/mono` rebuild (!). Most of these are due to missing `<Touch/>` task use, to ensure that a created/generated file is newer than the `Inputs` of the associated target. The `<GenerateProfile/>` task is slightly more complicated: in that case, we only want the file timestamp to change if the file contents have changed. Alter the `GenerateProfile.Execute()` logic to perform a content diff before writing to the file, resulting in a new timestamp. Finally, fix `GenerateJavaCallableWrappers` target use from `Mono.Android.csproj` by providing a "real" value for `$(JavaCallableWrapperAbsAssembly)` so that the `GenerateJavaCallableWrappers` target's `Inputs` and `Outputs` reference valid (existing) files. The previous value used a `$(JavaCallableWrapperOutputPathAbs)` property, which doesn't appear to have been defined anywhere, and thus was `""`. Performing these changes reduces my typical "no change" `Xamarin.Android.Build.Tasks.csproj` rebuild time from ~30sec down to a more reasonable ~10sec, which is much better (though more than I'd personally like).
2017-04-07 22:11:18 +03:00
Files="@(_RuntimeSource);@(_ProfilerSource);@(_MonoPosixHelperSource);@(_BclProfileItems);@(_MonoBtlsSource)"
[mono-runtimes] Fix _BuildRuntimes @Outputs checking. The _BuildRuntimes target was executing unnecessarily: Target _BuildRuntimes needs to be built as input file 'obj/Debug//armeabi-v7a/.stamp' is newer than output file 'obj/host/mono/profiler/.libs/libmono-profiler-log.dylib This would *always* be true, because the output files were never touched once they were built, so e.g. when copying a build tree from one machine to another (Dropbox!), the dependency check would be "wrong". Improve the _BuildRuntimes target to explicitly <Touch/> the output files so that -- short of timezone changes (!) -- the outputs will be forced to be newer than the inputs.
2016-04-23 14:54:47 +03:00
/>
[Xamarin.Android.Bcl-Tests] Add BCL test project. (#872) What do we want? (with apologies to 48e3fc26) **MOAR** Unit tests! Specifically, we want to run the BCL unit tests which mono generates: $ cd external/mono/mcs/class/corlib $ make PROFILE=monodroid test # creates `monodroid_corlib_test.dll` Creation of `monodroid_*_test.dll` assemblies and the `make PROFILE=monodroid test` target is a relatively recent development, for which I need to buy the #runtime team some beers. In terms of `mono-runtimes.targets`, we can build *all* of the BCL unit test assemblies with: $ cd external/mono/mcs/class $ make -i do-test PROFILE=monodroid Now that we can create them, how do we *use* them? That's the trickier bit: they need to be built within mono, as part of the existing BCL build process. This in turn means that the BCL unit test assemblies need to be distributed as part of the mono bundle, as we don't want to rebuild the mono repo "from scratch" just for the unit tests. Update `build-tools/mono-runtimes/ProfileAssemblies.projitems` to include a new `@(MonoTestAssembly)` item group which contains all of the BCL unit test assemblies and related files which should be included into `bundle-*.zip`. Additionally, add `ProfileAssemblies.projitems` to `@(VersionFile)` witihin `bundle-path.targets`, so that if anything within `ProfileAssemblies.projitems` changes, we rebuild the bundle. Once we *have* the BCL unit test assemblies, and their dependencies, we need to *run* them. The new `Xamarin.Android.Bcl-Tests.csproj` project is a Xamarin.Android application project which will execute the unit tests. There's just one small problem: Xamarin.Android apps want to use `Xamarin.Android.NUnitLite.dll`. The BCL unit test assemblies instead build against their own `nunitlite.dll`, which has no Xamarin.Android integration or support. How do we use the new test assemblies? *Force* a fix by using `remap-assembly-ref` to "rename" the `nunitlite` assembly reference to `Xamarin.Android.NUnitLite.dll`. This *cannot* be done as part of the `mono-runtimes.mdproj` build, as `Xamarin.Android.NUnitLite.dll` won't yet exist. Instead, remap the assemblies within `Xamarin.Android.Bcl-Tests.targets`, and distribute the remapped assemblies with the application. Finally, address one other "small" problem: not all of the unit tests pass! Some of these are for reasons we don't know, and others will require changes to `mono`. Update `Xamarin.Android.NUnitLite` to allow *filtering* of tests: namespace Xamarin.Android.NUnitLite { partial class TestSuiteActivity { public ITestFilter Filter {get; set;} public virtual void UpdateFilter (); } partial class TestSuiteInstrumentation { public ITestFilter Filter {get; set;} public virtual void UpdateFilter (); } } `TestSuiteActivity.UpdateFilter()` is called by `TestSuiteActivity.OnCreate()`, *after* `GetIncludedCategories()` and `GetExcludedCategories()` are called, to allow subclasses to alter the `ITestFilter` which is used to determine which tests are executed. `TestSuiteInstrumentation.UpdateFilter()` is called by `TestSuiteInstrumentation.OnStart()`, *after* `GetIncludedCategories()` and `GetExcludedCategories()` are called, to allow subclasses to alter the `ITestFilter` which is used to determine which tests are executed. `Xamarin.Android.Bcl_Tests` overrides both of these and updates the `Filter` property so that "known failing" tests are excluded. This allows us to skip failing tests, giving us time to properly fix them in time while allowing the rest of this PR to be merged. The skipped tests include: * MonoTests.System.Reflection.AssemblyTest.GetReferencedAssemblies * MonoTests.System.ServiceModel.Description.WebInvokeAttributeTest.RejectTwoParametersWhenNotWrapped
2017-10-03 22:48:08 +03:00
<Exec
Bump to mono/2017-10/7e85cfa5 (#943) Bump to xamarin-android-api-compatibility/master/c050ba3c. Updates expected APIs for BCL assemblies. Use `azureedge.net` instead of `blob.core.windows.net`. It's behind a CDN and way faster outside US. Use the new profiler init symbol name, `mono_profiler_init()`. Avoid dumping the content of `libmono-profiler-log.so` when tests fail. The binary contents aren't meaningful, and needlessly bloat the output log. Fix `eglib` path, as mono changed it's location. Don't skip errors from mono's `make test` target anymore. Stop building Mono's `net_4_x` profile. It isn't required to build the `monodroid` profile and *not* building it saves lots of time. The tools needed by XA are taken from the new `monodroid_tools` profile instead. Update Mono version phrasing in `README.md`. Using an explicit version is just a recipe for outdated info, instead rewrite it to be generic since the build will complain on an outdated Mono anyway. Add more exclusions to `LinkerDescription.xml` for BCL tests, which are used in the new `System.Reflection.Emit.SaveTest()`. Update `.gitignore` for some paths. `Xamarin.Android.Common.props` is generated as part of the build, and the `.rawproto` files created by gradle should be ignored. Context: https://bugzilla.xamarin.com/show_bug.cgi?id=4786 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=5798 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=5804 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=11696 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=17325 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=23153 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=25755 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=36053 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=36634 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=40556 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=41279 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=43172 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=43563 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=44266 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=46683 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=52157 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=52621 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=55603 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=56653 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=56684 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57069 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57160 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57197 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57222 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57232 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57301 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57476 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57488 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57496 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57505 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57528 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57629 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57691 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57744 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57796 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57850 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57851 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57887 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57918 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57930 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57936 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57959 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58100 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58168 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58210 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58291 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58317 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58344 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58361 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58379 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58399 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58411 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58421 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58423 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58454 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58612 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58621 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58728 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58738 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58770 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58782 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58793 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58809 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58829 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58877 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58901 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58911 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58965 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=59039 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=59154 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=59180 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=59235 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=59281 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=59364 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=59436 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=59608 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=59664 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=59824 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=60233
2017-10-31 18:55:44 +03:00
Command="make $(MakeConcurrency) test # %(_MonoRuntime.Identity)"
[Xamarin.Android.Bcl-Tests] Add BCL test project. (#872) What do we want? (with apologies to 48e3fc26) **MOAR** Unit tests! Specifically, we want to run the BCL unit tests which mono generates: $ cd external/mono/mcs/class/corlib $ make PROFILE=monodroid test # creates `monodroid_corlib_test.dll` Creation of `monodroid_*_test.dll` assemblies and the `make PROFILE=monodroid test` target is a relatively recent development, for which I need to buy the #runtime team some beers. In terms of `mono-runtimes.targets`, we can build *all* of the BCL unit test assemblies with: $ cd external/mono/mcs/class $ make -i do-test PROFILE=monodroid Now that we can create them, how do we *use* them? That's the trickier bit: they need to be built within mono, as part of the existing BCL build process. This in turn means that the BCL unit test assemblies need to be distributed as part of the mono bundle, as we don't want to rebuild the mono repo "from scratch" just for the unit tests. Update `build-tools/mono-runtimes/ProfileAssemblies.projitems` to include a new `@(MonoTestAssembly)` item group which contains all of the BCL unit test assemblies and related files which should be included into `bundle-*.zip`. Additionally, add `ProfileAssemblies.projitems` to `@(VersionFile)` witihin `bundle-path.targets`, so that if anything within `ProfileAssemblies.projitems` changes, we rebuild the bundle. Once we *have* the BCL unit test assemblies, and their dependencies, we need to *run* them. The new `Xamarin.Android.Bcl-Tests.csproj` project is a Xamarin.Android application project which will execute the unit tests. There's just one small problem: Xamarin.Android apps want to use `Xamarin.Android.NUnitLite.dll`. The BCL unit test assemblies instead build against their own `nunitlite.dll`, which has no Xamarin.Android integration or support. How do we use the new test assemblies? *Force* a fix by using `remap-assembly-ref` to "rename" the `nunitlite` assembly reference to `Xamarin.Android.NUnitLite.dll`. This *cannot* be done as part of the `mono-runtimes.mdproj` build, as `Xamarin.Android.NUnitLite.dll` won't yet exist. Instead, remap the assemblies within `Xamarin.Android.Bcl-Tests.targets`, and distribute the remapped assemblies with the application. Finally, address one other "small" problem: not all of the unit tests pass! Some of these are for reasons we don't know, and others will require changes to `mono`. Update `Xamarin.Android.NUnitLite` to allow *filtering* of tests: namespace Xamarin.Android.NUnitLite { partial class TestSuiteActivity { public ITestFilter Filter {get; set;} public virtual void UpdateFilter (); } partial class TestSuiteInstrumentation { public ITestFilter Filter {get; set;} public virtual void UpdateFilter (); } } `TestSuiteActivity.UpdateFilter()` is called by `TestSuiteActivity.OnCreate()`, *after* `GetIncludedCategories()` and `GetExcludedCategories()` are called, to allow subclasses to alter the `ITestFilter` which is used to determine which tests are executed. `TestSuiteInstrumentation.UpdateFilter()` is called by `TestSuiteInstrumentation.OnStart()`, *after* `GetIncludedCategories()` and `GetExcludedCategories()` are called, to allow subclasses to alter the `ITestFilter` which is used to determine which tests are executed. `Xamarin.Android.Bcl_Tests` overrides both of these and updates the `Filter` property so that "known failing" tests are excluded. This allows us to skip failing tests, giving us time to properly fix them in time while allowing the rest of this PR to be merged. The skipped tests include: * MonoTests.System.Reflection.AssemblyTest.GetReferencedAssemblies * MonoTests.System.ServiceModel.Description.WebInvokeAttributeTest.RejectTwoParametersWhenNotWrapped
2017-10-03 22:48:08 +03:00
IgnoreStandardErrorWarningFormat="True"
WorkingDirectory="$(IntermediateOutputPath)\%(_MonoRuntime.Identity)\runtime"
/>
<Copy
SourceFiles="@(_BclTestAssemblySource)"
DestinationFiles="@(_BclTestAssemblyDestination)"
/>
<Touch
Files="@(_BclTestAssemblyDestination)"
/>
<ItemGroup>
<_BclTestContent Include="$(MonoSourceFullPath)\mcs\class\%(MonoTestAssembly.SourcePath)\Test\**\*.*" />
<_BclTestContent Remove="$(MonoSourceFullPath)\mcs\class\%(MonoTestAssembly.SourcePath)\Test\**\*.cs" />
<_BclTestContent Remove="$(MonoSourceFullPath)\mcs\class\%(MonoTestAssembly.SourcePath)\Test\**\.gitattributes" />
<_BclTestContentDest Include="@(_BclTestContent->'$(XAInstallPrefix)\..\..\bcl-tests\Test\%(RecursiveDir)%(Filename)%(Extension)')" />
</ItemGroup>
<ItemGroup>
<_BclTestContent Include="$(MonoSourceFullPath)\mcs\class\System.IO.Compression\test.nupkg" />
<_BclTestContentDest Include="$(XAInstallPrefix)\..\..\bcl-tests\test.nupkg" />
<_BclTestContent Include="$(MonoSourceFullPath)\mcs\class\System.IO.Compression\archive.zip" />
<_BclTestContentDest Include="$(XAInstallPrefix)\..\..\bcl-tests\archive.zip" />
</ItemGroup>
<ItemGroup>
<_ZipTestContent Include="$(MonoSourceFullPath)\mcs\class\System.IO.Compression.FileSystem\foo\**\*.*" />
<_ZipTestContentDest Include="@(_ZipTestContent->'$(XAInstallPrefix)\..\..\bcl-tests\foo\%(RecursiveDir)%(Filename)%(Extension)')" />
<_BclTestContent Include="@(_ZipTestContent)" />
<_BclTestContentDest Include="@(_ZipTestContentDest)" />
</ItemGroup>
<Copy
SourceFiles="@(_BclTestContent)"
DestinationFiles="@(_BclTestContentDest)"
/>
<Exec
Condition="!Exists('$(XAInstallPrefix)\..\..\bcl-tests\Test\test.db')"
Command="sqlite3 &quot;$(XAInstallPrefix)\..\..\bcl-tests\Test\test.db&quot; &lt; &quot;$(MonoSourceFullPath)\mcs\class\Mono.Data.Sqlite\Test\test.sql&quot;"
/>
<Exec
Command="zip -r bcl-tests.zip ."
WorkingDirectory="$(XAInstallPrefix)\..\..\bcl-tests"
/>
<Touch
Files="$(XAInstallPrefix)\..\..\bcl-tests\bcl-tests.zip"
/>
[mono-runtimes] Build required mono runtimes. Add a new build-tools/mono-runtimes project to build the required Mono runtimes for Xamarin.Android app execution on Android devices. Xamarin.Android 6.0 ("cycle 6") includes Mono runtimes compiled for five architectures: * armeabi (armv5) * armeabi-v7a (armv7) * arm64-v8a (aarch64) * x86 * x86_64 An additional "host" architecture is also built to build the "monodroid" profile assemblies such as mscorlib.dll. In the interest of expediency and not requiring that *six* different runtimes be built -- which can greatly increase build times and storage requirements -- the initial xamarin-android open-source build will only build the armeabi-v7a (armv7) and "host" architectures. Additionally, the <UnzipDirectoryChildren/> task needed to be altered to use unzip(1) on OS X instead of using System.IO.Compression.ZipFile, because the ZipFile implementation included with Mono 4.4 doesn't respect file attributes such as the +x, causing all files to be extracted as 644 (rw-r--r--), resulting in an NDK toolchain which wouldn't work. To resolve this issue, execution on OS X and Linux now uses unzip(1). ~~ Adding new architectures ~~ To build Mono for a new architecture, edit build-tools/mono-runtimes/mono-runtimes.projitems and add a new @(_MonoRuntime) entry for the desired architecture. TODO: Figure out if there's a reasonable way to "opt-in" or "opt-out" of CPU architectures so that it isn't an "all or nothing" prospect.
2016-04-20 04:30:00 +03:00
</Target>
<Target Name="_InstallRuntimes"
[mxe] Add Windows cross-compiler support. (#55) Certain Xamarin.Android features require that Mono be built for Windows, e.g. the [AOT compilers][aot] require a build of mono that executes on Windows to generate the AOT native libraries. Unfortunately, building Mono on Windows continues to be a massive PITA. (Autotools on Windows requires Cygwin/mingw, running shell scripts on Windows is painfully slow, it's all brittle, etc.) To work around this pain, we instead build the Mono/Windows binaries on OS X, via [MXE][mxe], which produces a gcc-based cross-compiler which generates Windows binaries and is executable from Unix. This in turn requires that we have MXE, so add a `_CreateMxeToolchains` target to `android-toolchain.targets` which will build MXE. The installation prefix for MXE can be overridden via the new `$(AndroidMxeInstallPrefix)` MSBuild property; it defaults to `$HOME/android-toolchain/mxe`. Rework the `$(AndroidSupportedAbis)` MSBuild property so that it must include the "host" ABI, and add support for a new `host-win64` value which will use MXE to generate 64-bit Windows binaries for libmonosgen-2.0.dll and libMonoPosixHelper.dll. We can't always process `host-$(HostOS)` because of an xbuild bug. The scenario is that if you want to just build `host-win64`, the obvious thing to do is: cd build-tools/mono-runtimes xbuild /p:AndroidSupportedAbis=host-win64 Alas, if `host-$(HostOS)` is always processed, this inexplicably causes `host-$(HostOS)` to be re-rebuilt, which (1) is a waste of time, and (2) fails -- inexplicably -- in the `_BuildRuntimes` target because make(1) thinks that the configure flags have somehow changed, which currently makes no sense at all. (When can we move to MSBuild?) Changing `$(AndroidSupportedAbis)` so that `host-$(HostOS)` is explicitly processed instead of implicitly processed allows working around the above xbuild bug, as `host-$(HostOS)` won't be implicitly processed on every build, but only when required. Additionally, we add a new <Which/> MSBuild task so that we can determine if a particular program is in `$PATH`. This is useful because listing requirements within README.md is a road to pain -- e.g. xxd(1) is required to build `src/monodroid` but if it's missing it'll still *build* but you'll instead get a *linker* failure because the `monodroid_config` and `monodroid_machine_config` symbols aren't present. Building MXE requires that even more programs be present within $PATH, so explicitly check for these so that *useful* error messages can be generated instead of obscure ones. Finally, a note about autotools and generating Windows native libraries: creation of `.dll` files *requires* that an appropriate objdump be present so it can determine if e.g. `libkernel32.a` is an import library or an archive. If `x86_64-w64-mingw32.static-objdump` isn't found -- e.g. because $PATH doesn't contain it -- then no `.dll` files will be created, and much head scratching will occur. To rectify this, override the OBJDUMP and DLLTOOL values when invoking `configure` so that that full paths are used and `$PATH` use is reduced. (Who wants `x86_64-w64-mingw32.static-objdump` in `$PATH`?) [aot]: https://developer.xamarin.com/releases/android/xamarin.android_5/xamarin.android_5.1/#AOT_Support [mxe]: http://mxe.cc/
2016-06-07 00:12:49 +03:00
DependsOnTargets="_GetRuntimesOutputItems"
[structure] Rework installation directory structure (#704) Context: https://github.com/xamarin/xamarin-android/pull/253#discussion_r82862993 The *intention* is that Jenkins-produced `oss-xamarin.android*.zip` artifacts be usable on Windows, so that side-by-side testing can be performed without replacing the system installation. Usage is in [UsingJenkinsBuildArtifacts.md](Documentation/UsingJenkinsBuildArtifacts). This isn't *entirely* the case. It was *apparently* the case at the time of commit 87ca2737, but things have bitrotten since. For example, following the 87ca2737 instructions would currently result in an XA0020 `Could not find monodroid` error, because `class-parse.exe` wasn't where Windows expects it to be (it was in `lib/mandroid`, not `lib/xbuild/Xamarin/Android`). This needs to be fixed. Additionally, PR #253 mentions that, for filesystem organization, it would be useful if the macOS/Linux directory structure -- `$prefix/bin`, `$prefix/lib/mandroid`, `$prefix/lib/xbuild/Xamarin/Android`, `$prefix/lib/xbuild-frameworks` -- more closely resembled the Windows directory structure of `$MSBuild` and `$ReferenceAssemblies` (as seen in `.vsix` files). This would turn macOS/Linux into using `$xa_prefix/xbuild` and `$xa_prefix/xbuild-frameworks` directories. `$prefix/bin` would only contain `xabuild`. What is currently in `$prefix/lib/mandroid` would be merged with `$xa_prefix/xbuild/Xamarin/Android`. `$xa_prefix` would `$prefix/lib/xamarin.android`. This would turn the current macOS structure: $prefix/bin/xabuild $prefix/bin/generator $prefix/bin/cross-arm $prefix/lib/mandroid/generator.exe $prefix/lib/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xbuild/Xamarin/Android/Xamarin.Android.Common.targets Into: $prefix/bin/xabuild $prefix/lib/xamarin.android/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xamarin.android/xbuild/Xamarin/Android/generator.exe $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Xamarin.Android.Common.targets $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Darwin/cross-arm Other notes: * The `bundle-*.zip` filename has been has been changed to include a *hash* of the contents of various files, in particular `build-tools\mono-runtimes.*`. This was instigated via a conversation with @kumpera about making the filename more "idiot-proof": `mono-runtimes.props` contains *compiler flags*, and if any of those change, then *logically* the bundle should differ as well, as the mono runtimes may differ in significant ways. In theory, the `-vXX` version should be used to track this, but this is a manual change, easy to overlook. The new `-hHASH` part of the filename should be more automagic. The new `<HashFileContents/>` task in `xa-prep-tasks.dll` is responsible for creating the hash value. * `Configuration.Java.Interop.Override.props` was moved into `build-tools/scripts`, because that would cleanup the root directory a bit. * OS-specific binaries are now placed into `$prefix/lib/xamarin.android/xbuild/Xamarin/Android/$(HostOS)`. On a theoretical plus side, this means that the same build directory can contain OS-specific binaries for multiple operating systems. (I don't know if anyone shares a build tree between e.g. macOS and Linux, but if anyone *does*...) Unfortunately this requires a workaround for an `xbuild` bug: `%(_LlvmRuntime.InstallPath)` and `%(_MonoCrossRuntime.InstallPath)` *cannot* use MSBuild properties, e.g. `<InstallPath>$(HostOS)/</InstallPath>` doesn't work as desired; it's instead treated literally. Special-case `%(InstallPath)` until we fully migrate to MSBuild. * `$(MonoAndroidToolsDirectory)` should be considered *dead*, along with `$(MonoAndroidBinDirectory)`, as these should now *always* be the same directory as where `Xamarin.Android.Build.Tasks.dll` is located, or a `$(HostOS)` sub-directory. * `Xamarin.ProjectTools.csproj` needed to be updated to ensure that the build order was correct. * Remove all `[Obsolete]` and unreferenced members from `Xamarin.Android.Build.Utilities.dll`. There's too much in there, and it makes my head hurt trying to understand the interrelationships between it all. If it's not used, it's gone. * The changes to `src/monodroid/jni/Android.mk` are...weird. The removal of `-I$(topdir)/libmonodroid/zip`/etc. is to reduce use of non-existent paths, as `$(topdir)` isn't defined, so that's *actually* `-I/libmonodroid/zip`, which is nonsensical. So far, so good. What's *odd* is the addition of `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`: This is needed so that `external/mono/support/zlib-helper.c` exports `CreateZStream` and related symbols, otherwise we get a unit test failure in `GzipStreamTest.Compression` due to an `EntryPointNotFoundException`, because `CreateZStream` isn't exported/public. What's odd here is that I don't understand what caused this behavior to change. Previous builds exported `CreateZStream`, otherwise the tests would fail, and I don't understand how any of the other changes in this PR would be at fault, though that's certainly the most plausible explanation. Regardless, `-Ijni` *first* (due to adding `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`) is the desired behavior, so that `jni/config.h` is included, thus ensuring that `MONO_API` has the required definition when building `zlib-helper.c`.
2017-07-28 16:36:32 +03:00
Inputs="@(_RuntimeSource);@(_ProfilerSource);@(_MonoPosixHelperSource);@(_RuntimeEglibHeaderSource)"
Outputs="@(_InstallRuntimeOutput);@(_InstallProfilerOutput);@(_InstallMonoPosixHelperOutput);@(_RuntimeEglibHeaderOutput)">
[mono-runtimes] Build AOT+LLVM cross-compilers (#125) The commit implements building of LLVM and cross-compilers to support Xamarin.Android/Mono AOT. LLVM and cross-compilers can be built for both the host platform (Linux and OS/X at the moment) as well as cross-compiled for 32-bit and 64-bit Windows platforms. Windows builds are done with MXE toolchain on OS/X and with the packaged mingw-w64 toolchain on Linux (tested on Ubuntu 16.04 ONLY). Also introducing a new set of MSBuild properties that contain information about the host system. Some of those properties (HostOS, HostCC, HostCXX for instance) have been moved from Configuration.props to better support auto-detection. A new script, build-tools/scripts/generate-os-info, is invoked as part of `make prepare` to generate file that contains the new properties. The generated file is required for the build to work and is also host-specific (it mustn't be moved between different machines) Cross compiler builds require access to a configured Mono build tree, in order to generate C structure offsets header file that is used by the AOT compilers to properly generate AOT-ed binaries. Therefore, even if a JIT target is not enabled in the configuration, enabling a cross-compiler for some target will configure Mono for that JIT target but it will NOT build it, to save time. To facilitate this, the _MonoRuntimes items defined in build-tools/mono-runtimes/mono-runtimes.projitems gain an additional metadata item called `DoBuild` which will be set to `true` if the runtime actually needs to be built, as opposed to just configured. MXE builds are disabled on Linux as mingw-w64 works just fine. A `make prepare` warning is issued for Linux hosts which have the binfmt_misc module enabled and either Wine of Mono (cli) registered as PE32/PE32+ binary interpreters. In such instance building of the Windows cross-compilers will fail because Autotools determine whether software is being cross compiled by building a test program and attempting to execute it. In normal circumstances such an attempt will fail, but with Windows cross-compilation and either Wine or Mono registered to handle the PE32 executables this attempt will succeed thus causing the cross compilation detection to fail. Currently to build cross compilers on Linux you need to generate the C structure offsets header file on OS/X and copy the resulting headers to appropriate places on Linux. The header files should be placed in build-tools/mono-runtimes/obj/Debug/cross-*/ directories. The header files are: {cross-arm,cross-arm-win}/aarch64-v8a-linux-android.h {cross-arm64,cross-arm64-win}/armv5-none-linux-androideabi.h {cross-x86,cross-x86-win}/i686-none-linux-android.h {cross-x86_64,cross-x86_64-win}/x86_64-none-linux-android.h Offsets header generation doesn't work on Linux atm because of missing support for it in the Mono utility used to generate the offsets. Hopefully this limitation will be removed in the near future and a start-to-end build of everything will be possible on Linux. It is now mandatory to run at least `make prepare-props` before Xamarin.Android can be built. The target generates the OS-specific props file which is required by the build. `make prepare` depends on the target.
2016-07-26 16:27:31 +03:00
<MakeDir
Condition=" '%(_MonoRuntime.DoBuild)' == 'true' "
[structure] Rework installation directory structure (#704) Context: https://github.com/xamarin/xamarin-android/pull/253#discussion_r82862993 The *intention* is that Jenkins-produced `oss-xamarin.android*.zip` artifacts be usable on Windows, so that side-by-side testing can be performed without replacing the system installation. Usage is in [UsingJenkinsBuildArtifacts.md](Documentation/UsingJenkinsBuildArtifacts). This isn't *entirely* the case. It was *apparently* the case at the time of commit 87ca2737, but things have bitrotten since. For example, following the 87ca2737 instructions would currently result in an XA0020 `Could not find monodroid` error, because `class-parse.exe` wasn't where Windows expects it to be (it was in `lib/mandroid`, not `lib/xbuild/Xamarin/Android`). This needs to be fixed. Additionally, PR #253 mentions that, for filesystem organization, it would be useful if the macOS/Linux directory structure -- `$prefix/bin`, `$prefix/lib/mandroid`, `$prefix/lib/xbuild/Xamarin/Android`, `$prefix/lib/xbuild-frameworks` -- more closely resembled the Windows directory structure of `$MSBuild` and `$ReferenceAssemblies` (as seen in `.vsix` files). This would turn macOS/Linux into using `$xa_prefix/xbuild` and `$xa_prefix/xbuild-frameworks` directories. `$prefix/bin` would only contain `xabuild`. What is currently in `$prefix/lib/mandroid` would be merged with `$xa_prefix/xbuild/Xamarin/Android`. `$xa_prefix` would `$prefix/lib/xamarin.android`. This would turn the current macOS structure: $prefix/bin/xabuild $prefix/bin/generator $prefix/bin/cross-arm $prefix/lib/mandroid/generator.exe $prefix/lib/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xbuild/Xamarin/Android/Xamarin.Android.Common.targets Into: $prefix/bin/xabuild $prefix/lib/xamarin.android/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xamarin.android/xbuild/Xamarin/Android/generator.exe $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Xamarin.Android.Common.targets $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Darwin/cross-arm Other notes: * The `bundle-*.zip` filename has been has been changed to include a *hash* of the contents of various files, in particular `build-tools\mono-runtimes.*`. This was instigated via a conversation with @kumpera about making the filename more "idiot-proof": `mono-runtimes.props` contains *compiler flags*, and if any of those change, then *logically* the bundle should differ as well, as the mono runtimes may differ in significant ways. In theory, the `-vXX` version should be used to track this, but this is a manual change, easy to overlook. The new `-hHASH` part of the filename should be more automagic. The new `<HashFileContents/>` task in `xa-prep-tasks.dll` is responsible for creating the hash value. * `Configuration.Java.Interop.Override.props` was moved into `build-tools/scripts`, because that would cleanup the root directory a bit. * OS-specific binaries are now placed into `$prefix/lib/xamarin.android/xbuild/Xamarin/Android/$(HostOS)`. On a theoretical plus side, this means that the same build directory can contain OS-specific binaries for multiple operating systems. (I don't know if anyone shares a build tree between e.g. macOS and Linux, but if anyone *does*...) Unfortunately this requires a workaround for an `xbuild` bug: `%(_LlvmRuntime.InstallPath)` and `%(_MonoCrossRuntime.InstallPath)` *cannot* use MSBuild properties, e.g. `<InstallPath>$(HostOS)/</InstallPath>` doesn't work as desired; it's instead treated literally. Special-case `%(InstallPath)` until we fully migrate to MSBuild. * `$(MonoAndroidToolsDirectory)` should be considered *dead*, along with `$(MonoAndroidBinDirectory)`, as these should now *always* be the same directory as where `Xamarin.Android.Build.Tasks.dll` is located, or a `$(HostOS)` sub-directory. * `Xamarin.ProjectTools.csproj` needed to be updated to ensure that the build order was correct. * Remove all `[Obsolete]` and unreferenced members from `Xamarin.Android.Build.Utilities.dll`. There's too much in there, and it makes my head hurt trying to understand the interrelationships between it all. If it's not used, it's gone. * The changes to `src/monodroid/jni/Android.mk` are...weird. The removal of `-I$(topdir)/libmonodroid/zip`/etc. is to reduce use of non-existent paths, as `$(topdir)` isn't defined, so that's *actually* `-I/libmonodroid/zip`, which is nonsensical. So far, so good. What's *odd* is the addition of `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`: This is needed so that `external/mono/support/zlib-helper.c` exports `CreateZStream` and related symbols, otherwise we get a unit test failure in `GzipStreamTest.Compression` due to an `EntryPointNotFoundException`, because `CreateZStream` isn't exported/public. What's odd here is that I don't understand what caused this behavior to change. Previous builds exported `CreateZStream`, otherwise the tests would fail, and I don't understand how any of the other changes in this PR would be at fault, though that's certainly the most plausible explanation. Regardless, `-Ijni` *first* (due to adding `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`) is the desired behavior, so that `jni/config.h` is included, thus ensuring that `MONO_API` has the required definition when building `zlib-helper.c`.
2017-07-28 16:36:32 +03:00
Directories="$(_MSBuildDir)\lib\%(_MonoRuntime.Identity)"
[build] Build the monodroid-profile BCL assemblies reliably (#135) Commit 37b5fd51 broke `make jenkins` (and possible `make all`?) through the introduction of the `%(_MonoRuntime.DoBuild)` metadata: `%(_MonoRuntime.DoBuild)` wasn't added everywhere required. In particular, the `host-Darwin` item didn't gain `<DoBuild/>`, meaning *`host-Darwin` wasn't built*. This in turn meant that the monodroid BCL wasn't built (!) which caused a build break much later within `src/Mono.Android`: Executing: mono .../xamarin-android/external/Java.Interop/bin/Debug/jcw-gen.exe -o ".../xamarin-android/src/Mono.Android/obj/Debug/android-23/jcw/src" -L ".../xamarin-androidg/lib/xbuild-frameworks/MonoAndroid/v6.0/" -L ".../xamarin-android/src/Mono.Android/../../bin/Debug/lib/xbuild-frameworks/MonoAndroid/v6.0/../v1.0/" -L ".../xamarin-android/src/Mono.Android/../../MonoAndroid/v6.0/../v1.0/Facades" ".../xamarin-android/src/Mono.Android/../../bin/Debug/lib/xbuild-frameworks/MonoAndroid/v6.0/Mono.Android.dll" jcw-gen: Could not load assembly 'mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089'. Perhaps it doesn't exist in the Mono for Android profile? `mscorlib.dll` couldn't be found because it wasn't *built*, nor installed into `bin/Debug/lib/xbuild-frameworks/MonoAndroid/v1.0`. Additionally, fully building the AOT cross compilers requires adding `mxe-Win32` to `$(ALL_HOST_ABIS)`, otherwise e.g. `win-armeabi` fails to build because `arm_dpimacros.h` doesn't exist (as `arm_dpimacros.h` is built by `mxe-Win32`): Error parsing '.../xamarin-android/external/mono/mono/metadata/metadata-cross-helpers.c' .../xamarin-android/external/mono/mono/arch/arm/arm-codegen.h(937,10): fatal: 'mono/arch/arm/arm_dpimacros.h' file not found Error parsing '.../xamarin-android/external/mono/mono/mini/mini-cross-helpers.c' .../xamarin-android/external/mono/mono/arch/arm/arm-codegen.h(937,10): fatal: 'mono/arch/arm/arm_dpimacros.h' file not found Unhandled Exception: System.Exception: Expected to find struct definition for MonoObject Fix the `mono-runtimes.mdproj` build so that it doesn't re-execute targets when not necessary, by "rolling" with xbuild limitations and refactoring item groups as appropriate. In particular, xbuild doesn't like referencing item metadata within a `//Target/@Outputs` that in turn references a property group: <PropertyGroup> <V>Value</v> </PropertyGroup> <ItemGroup> <Item Include="Something"> <Metadata>$(V)</Metadata> </Item> </ItemGroup> <Target Name="Example" Inputs="..." Outputs="%(Item.Metadata)"> ... </Target> Whenever the `Example` target is run, it will *always* run, even if `Value` exists, because `xbuild` perceives the `//Target[@Name='Example']/@Outputs` value as being `$(V)`, *not* `Value`, which is wonderfully broken. To compensate, Don't Do That™; in particular, item metadata used within `//Target/@Outputs` *must* be hardcoded and not contain a property reference. This "hardcoding" removes the need for the `$(_Cross*ffsetsHeaderFile)` properties, so remove them. Remove `System.Net.Http.WebRequest.dll` from `@(_BclAssembly)`, as it is no longer built in the monodroid profile; requiring it as an output means that the "host" mono build is *always* built, which isn't needed. Improve the `_BuildRuntimes` and `_InstallRuntimes` targets so that their referenced files aren't *so* repetitive, in particular by "splitting out" `@(_RuntimeLibraries)` into `@(_RuntimeSource)`, `@(_ProfilerSource)`, and `@(_MonoPosixHelperSource)`. This allows more intelligently `strip`ing the native libraries in the Release configuration, while also maintaining generation of the `.d.so` files.
2016-07-28 13:24:18 +03:00
/>
[mono-runtimes] "Install" eglib headers into $prefix/include (#164) Context: https://github.com/xamarin/xamarin-android/pull/162 PR #162 tries to use the mono bundle (fbfd676c) to avoid rebuilding mono on subsequent builds (when the mono commit hasn't changed), in order to speed up the Jenkins build process (currently tracking at ~3.5 *hours* for a non-PR build...) [PR #162 currently fails][0] when building `src/monodroid`: Executing: /Users/builder/android-toolchain/ndk/ndk-build CONFIGURATION=Debug NDK_LIBS_OUT=./libs/Debug NDK_OUT=./obj/Debug V=1 ... [armeabi-v7a] Compile thumb : monodroid <= nl.c /Users/builder/android-toolchain/ndk/toolchains/arm-linux-androideabi-4.9/prebuilt/darwin-x86_64/bin/arm-linux-androideabi-gcc -MMD -MP -MF ./obj/Debug/local/armeabi-v7a/objs/monodroid/__/__/__/external/mono/support/nl.o.d -fpic -ffunction-sections -funwind-tables -fstack-protector-strong -no-canonical-prefixes -march=armv7-a -mfpu=vfpv3-d16 -mfloat-abi=softfp -mthumb -Os -g -DNDEBUG -fomit-frame-pointer -fno-strict-aliasing -finline-limit=64 -Ijni/../../../build-tools/mono-runtimes/obj/Debug/armeabi-v7a -Ijni/../../../build-tools/mono-runtimes/obj/Debug/armeabi-v7a/eglib -Ijni/../../../build-tools/mono-runtimes/obj/Debug/armeabi-v7a/eglib/src -Ijni/../../../external/mono/eglib/src -Ijni/zip -Ijni -DANDROID -ggdb3 -O0 -fno-omit-frame-pointer -std=c99 -DHAVE_LINUX_NETLINK_H=1 -DHAVE_LINUX_RTNETLINK_H=1 -D_REENTRANT -DPLATFORM_ANDROID -DANDROID -DLINUX -Dlinux -D__linux_ -DHAVE_CONFIG_H -DJI_DLL_EXPORT -DMONO_DLL_EXPORT -I/libmonodroid/zip -I/include -I/include/eglib -mandroid -fno-strict-aliasing -ffunction-sections -fomit-frame-pointer -funswitch-loops -finline-limit=300 -fvisibility=hidden -fstack-protector -Wa,--noexecstack -Wformat -Werror=format-security -DDEBUG=1 -Wa,--noexecstack -Wformat -Werror=format-security -isystem /Users/builder/android-toolchain/ndk/platforms/android-9/arch-arm/usr/include -c jni/../../../external/mono/support/nl.c -o ./obj/Debug/local/armeabi-v7a/objs/monodroid/__/__/__/external/mono/support/nl.o In file included from jni/../../../external/mono/support/nl.h:3:0, from jni/../../../external/mono/support/nl.c:14: jni/../../../external/mono/eglib/src/glib.h:18:26: fatal error: eglib-config.h: No such file or directory #include <eglib-config.h> ^ compilation terminated. `src/monodroid` fails to build because it requires build artifacts located in `build-tools/mono-runtimes/obj/$(Configuration)/%(Identity)/eglib`, which doesn't exist when the mono bundle is being used (because we never built mono!) Fix this issue by updating the `_InstallRuntimes` target in `mono-runtimes.targets` to copy the generated eglib header files into `bin/$(Configuration)/include/%(Identity)/eglib`, and update `bundle.targets` so that the `include` directory is also included in the mono bundle. This should allow a future PR #162 fix to build, as all required generated files will be present within the bundle. Additionally, add the `$(HostOS)` and a version number to the bundle filename, now `-v1`, to note that the structure of the bundle has changed, *without* a corresponding mono, llvm, or libzip commit bump. [0]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android-pr-builder/61/
2016-08-14 15:52:04 +03:00
<MakeDir
Condition=" '%(_MonoRuntime.DoBuild)' == 'true' "
[structure] Rework installation directory structure (#704) Context: https://github.com/xamarin/xamarin-android/pull/253#discussion_r82862993 The *intention* is that Jenkins-produced `oss-xamarin.android*.zip` artifacts be usable on Windows, so that side-by-side testing can be performed without replacing the system installation. Usage is in [UsingJenkinsBuildArtifacts.md](Documentation/UsingJenkinsBuildArtifacts). This isn't *entirely* the case. It was *apparently* the case at the time of commit 87ca2737, but things have bitrotten since. For example, following the 87ca2737 instructions would currently result in an XA0020 `Could not find monodroid` error, because `class-parse.exe` wasn't where Windows expects it to be (it was in `lib/mandroid`, not `lib/xbuild/Xamarin/Android`). This needs to be fixed. Additionally, PR #253 mentions that, for filesystem organization, it would be useful if the macOS/Linux directory structure -- `$prefix/bin`, `$prefix/lib/mandroid`, `$prefix/lib/xbuild/Xamarin/Android`, `$prefix/lib/xbuild-frameworks` -- more closely resembled the Windows directory structure of `$MSBuild` and `$ReferenceAssemblies` (as seen in `.vsix` files). This would turn macOS/Linux into using `$xa_prefix/xbuild` and `$xa_prefix/xbuild-frameworks` directories. `$prefix/bin` would only contain `xabuild`. What is currently in `$prefix/lib/mandroid` would be merged with `$xa_prefix/xbuild/Xamarin/Android`. `$xa_prefix` would `$prefix/lib/xamarin.android`. This would turn the current macOS structure: $prefix/bin/xabuild $prefix/bin/generator $prefix/bin/cross-arm $prefix/lib/mandroid/generator.exe $prefix/lib/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xbuild/Xamarin/Android/Xamarin.Android.Common.targets Into: $prefix/bin/xabuild $prefix/lib/xamarin.android/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xamarin.android/xbuild/Xamarin/Android/generator.exe $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Xamarin.Android.Common.targets $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Darwin/cross-arm Other notes: * The `bundle-*.zip` filename has been has been changed to include a *hash* of the contents of various files, in particular `build-tools\mono-runtimes.*`. This was instigated via a conversation with @kumpera about making the filename more "idiot-proof": `mono-runtimes.props` contains *compiler flags*, and if any of those change, then *logically* the bundle should differ as well, as the mono runtimes may differ in significant ways. In theory, the `-vXX` version should be used to track this, but this is a manual change, easy to overlook. The new `-hHASH` part of the filename should be more automagic. The new `<HashFileContents/>` task in `xa-prep-tasks.dll` is responsible for creating the hash value. * `Configuration.Java.Interop.Override.props` was moved into `build-tools/scripts`, because that would cleanup the root directory a bit. * OS-specific binaries are now placed into `$prefix/lib/xamarin.android/xbuild/Xamarin/Android/$(HostOS)`. On a theoretical plus side, this means that the same build directory can contain OS-specific binaries for multiple operating systems. (I don't know if anyone shares a build tree between e.g. macOS and Linux, but if anyone *does*...) Unfortunately this requires a workaround for an `xbuild` bug: `%(_LlvmRuntime.InstallPath)` and `%(_MonoCrossRuntime.InstallPath)` *cannot* use MSBuild properties, e.g. `<InstallPath>$(HostOS)/</InstallPath>` doesn't work as desired; it's instead treated literally. Special-case `%(InstallPath)` until we fully migrate to MSBuild. * `$(MonoAndroidToolsDirectory)` should be considered *dead*, along with `$(MonoAndroidBinDirectory)`, as these should now *always* be the same directory as where `Xamarin.Android.Build.Tasks.dll` is located, or a `$(HostOS)` sub-directory. * `Xamarin.ProjectTools.csproj` needed to be updated to ensure that the build order was correct. * Remove all `[Obsolete]` and unreferenced members from `Xamarin.Android.Build.Utilities.dll`. There's too much in there, and it makes my head hurt trying to understand the interrelationships between it all. If it's not used, it's gone. * The changes to `src/monodroid/jni/Android.mk` are...weird. The removal of `-I$(topdir)/libmonodroid/zip`/etc. is to reduce use of non-existent paths, as `$(topdir)` isn't defined, so that's *actually* `-I/libmonodroid/zip`, which is nonsensical. So far, so good. What's *odd* is the addition of `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`: This is needed so that `external/mono/support/zlib-helper.c` exports `CreateZStream` and related symbols, otherwise we get a unit test failure in `GzipStreamTest.Compression` due to an `EntryPointNotFoundException`, because `CreateZStream` isn't exported/public. What's odd here is that I don't understand what caused this behavior to change. Previous builds exported `CreateZStream`, otherwise the tests would fail, and I don't understand how any of the other changes in this PR would be at fault, though that's certainly the most plausible explanation. Regardless, `-Ijni` *first* (due to adding `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`) is the desired behavior, so that `jni/config.h` is included, thus ensuring that `MONO_API` has the required definition when building `zlib-helper.c`.
2017-07-28 16:36:32 +03:00
Directories="$(_OutputIncludeDir)%(_MonoRuntime.Identity)\eglib"
[mono-runtimes] "Install" eglib headers into $prefix/include (#164) Context: https://github.com/xamarin/xamarin-android/pull/162 PR #162 tries to use the mono bundle (fbfd676c) to avoid rebuilding mono on subsequent builds (when the mono commit hasn't changed), in order to speed up the Jenkins build process (currently tracking at ~3.5 *hours* for a non-PR build...) [PR #162 currently fails][0] when building `src/monodroid`: Executing: /Users/builder/android-toolchain/ndk/ndk-build CONFIGURATION=Debug NDK_LIBS_OUT=./libs/Debug NDK_OUT=./obj/Debug V=1 ... [armeabi-v7a] Compile thumb : monodroid <= nl.c /Users/builder/android-toolchain/ndk/toolchains/arm-linux-androideabi-4.9/prebuilt/darwin-x86_64/bin/arm-linux-androideabi-gcc -MMD -MP -MF ./obj/Debug/local/armeabi-v7a/objs/monodroid/__/__/__/external/mono/support/nl.o.d -fpic -ffunction-sections -funwind-tables -fstack-protector-strong -no-canonical-prefixes -march=armv7-a -mfpu=vfpv3-d16 -mfloat-abi=softfp -mthumb -Os -g -DNDEBUG -fomit-frame-pointer -fno-strict-aliasing -finline-limit=64 -Ijni/../../../build-tools/mono-runtimes/obj/Debug/armeabi-v7a -Ijni/../../../build-tools/mono-runtimes/obj/Debug/armeabi-v7a/eglib -Ijni/../../../build-tools/mono-runtimes/obj/Debug/armeabi-v7a/eglib/src -Ijni/../../../external/mono/eglib/src -Ijni/zip -Ijni -DANDROID -ggdb3 -O0 -fno-omit-frame-pointer -std=c99 -DHAVE_LINUX_NETLINK_H=1 -DHAVE_LINUX_RTNETLINK_H=1 -D_REENTRANT -DPLATFORM_ANDROID -DANDROID -DLINUX -Dlinux -D__linux_ -DHAVE_CONFIG_H -DJI_DLL_EXPORT -DMONO_DLL_EXPORT -I/libmonodroid/zip -I/include -I/include/eglib -mandroid -fno-strict-aliasing -ffunction-sections -fomit-frame-pointer -funswitch-loops -finline-limit=300 -fvisibility=hidden -fstack-protector -Wa,--noexecstack -Wformat -Werror=format-security -DDEBUG=1 -Wa,--noexecstack -Wformat -Werror=format-security -isystem /Users/builder/android-toolchain/ndk/platforms/android-9/arch-arm/usr/include -c jni/../../../external/mono/support/nl.c -o ./obj/Debug/local/armeabi-v7a/objs/monodroid/__/__/__/external/mono/support/nl.o In file included from jni/../../../external/mono/support/nl.h:3:0, from jni/../../../external/mono/support/nl.c:14: jni/../../../external/mono/eglib/src/glib.h:18:26: fatal error: eglib-config.h: No such file or directory #include <eglib-config.h> ^ compilation terminated. `src/monodroid` fails to build because it requires build artifacts located in `build-tools/mono-runtimes/obj/$(Configuration)/%(Identity)/eglib`, which doesn't exist when the mono bundle is being used (because we never built mono!) Fix this issue by updating the `_InstallRuntimes` target in `mono-runtimes.targets` to copy the generated eglib header files into `bin/$(Configuration)/include/%(Identity)/eglib`, and update `bundle.targets` so that the `include` directory is also included in the mono bundle. This should allow a future PR #162 fix to build, as all required generated files will be present within the bundle. Additionally, add the `$(HostOS)` and a version number to the bundle filename, now `-v1`, to note that the structure of the bundle has changed, *without* a corresponding mono, llvm, or libzip commit bump. [0]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android-pr-builder/61/
2016-08-14 15:52:04 +03:00
/>
<Copy
SourceFiles="@(_RuntimeEglibHeaderSource)"
DestinationFiles="@(_RuntimeEglibHeaderOutput)"
/>
[mono-runtimes] Fix @(BundleItem) timestamps (#307) The *intent* of the `_BuildUnlessCached` target within `mono-runtimes.targets` is to (attempt to!) make building mono "free", or at least, free *enough* so that other project files can have a build-time dependency on `mono-runtimes.mdproj` so that build artifacts can be (reasonably) consistent. Unfortunately, that wasn't the case, with the net result being that: tools/scripts/xabuild /t:SignAndroidPackage src/Mono.Android/Test/*.csproj would *constantly* attempt to re-build `build-tools/mono-runtimes.mdproj`, adding (at least!) one minute as we ran mono's `make`...to do nothing. Why? Target _BuildUnlessCached needs to be built as input file '.../xamarin-android/external/mono/autogen.sh' is newer than output file '../../bin/Debug//include/armeabi-v7a/eglib/config.h' The "problem" is that the `<Copy/>` task copies over the timestamp as well, so even though we're (possibly) creating a new file, the timestamp on the created file will match the source file. Normally this is fine. In this case, we're dealing with `eglib`, which is a "nested" ("delegated"?) `configure` invocation from mono's `configure`. It *appears* that if (when) mono is updated, "normal" mono subdirectories will "properly" honor `--enable-maintainer-mode` and re-run mono's `configure`, which will in turn update the timstamp on any `configure`-time generated files. This doesn't appear to be the case for `mono/eglib`; when `mono` is updated, `mono/eglib` does *not* re-run `configure`, and thus no timestamps are updated to follow suit. Fix this by *explicitly* `<Touch/>`ing all copied files so that we ensure they have a "recent" timestamp ("now"), which should ensure that the file's timestamp is newer than that of e.g. `autogen.sh`. Additionally, there was a typo: Commit a5b324d1 used a `@(_InstallRuntimesOutput)` item group, but there is no such item group; it's actually `@(_InstallRuntimeOutput)` (no `s`). Fixity fix this. Finally, commit a5b324d1 forgot to add `libmono-btls-shared` to `@(BundleItem)`, preventing it from being added to the bundle. Fix this oversight.
2016-11-22 14:11:45 +03:00
<Touch Files="@(_RuntimeEglibHeaderOutput)" />
[mono-runtimes] Bundle Consts.cs, use correct opt.exe install location (#165) Context: https://github.com/xamarin/xamarin-android/pull/162 PR #162 tries to use the mono bundle (fbfd676c) to avoid rebuilding mono on subsequent builds (when the mono commit hasn't changed), in order to speed up the Jenkins build process (currently tracking at ~3.5 *hours* for a non-PR build...) [PR #162 currently fails][0] when building `src/Mono.Data.Sqlite`: CSC: error CS2001: Source file `/Users/builder/jenkins/workspace/xamarin-android-pr-builder/xamarin-android/external/mono/mcs/build/common/Consts.cs' could not be found Fix this error by copying `external/mono/mcs/build/common/Consts.cs` into `bin/$(Configuration)/include`, and updating the `Consts.cs` references within `Mono.Data.Sqlite.csproj` and `Mono.Posix.csproj` so that they'll use the `include` copy. This should allow us to use the bundle to build these projects. Additionally, `llc.exe` and `opt.exe` are installed into `$prefix/bin`, which isn't correct; `$prefix/bin` as a construct doesn't exist on Windows. Install these into `$prefix/lib/mandroid`. Update the bundle version to note these bundle changes. [0]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android-pr-builder/67/console
2016-08-15 00:54:15 +03:00
<Copy
SourceFiles="@(_MonoConstsSource)"
DestinationFiles="@(_MonoConstsOutput)"
/>
[mono-runtimes] Fix @(BundleItem) timestamps (#307) The *intent* of the `_BuildUnlessCached` target within `mono-runtimes.targets` is to (attempt to!) make building mono "free", or at least, free *enough* so that other project files can have a build-time dependency on `mono-runtimes.mdproj` so that build artifacts can be (reasonably) consistent. Unfortunately, that wasn't the case, with the net result being that: tools/scripts/xabuild /t:SignAndroidPackage src/Mono.Android/Test/*.csproj would *constantly* attempt to re-build `build-tools/mono-runtimes.mdproj`, adding (at least!) one minute as we ran mono's `make`...to do nothing. Why? Target _BuildUnlessCached needs to be built as input file '.../xamarin-android/external/mono/autogen.sh' is newer than output file '../../bin/Debug//include/armeabi-v7a/eglib/config.h' The "problem" is that the `<Copy/>` task copies over the timestamp as well, so even though we're (possibly) creating a new file, the timestamp on the created file will match the source file. Normally this is fine. In this case, we're dealing with `eglib`, which is a "nested" ("delegated"?) `configure` invocation from mono's `configure`. It *appears* that if (when) mono is updated, "normal" mono subdirectories will "properly" honor `--enable-maintainer-mode` and re-run mono's `configure`, which will in turn update the timstamp on any `configure`-time generated files. This doesn't appear to be the case for `mono/eglib`; when `mono` is updated, `mono/eglib` does *not* re-run `configure`, and thus no timestamps are updated to follow suit. Fix this by *explicitly* `<Touch/>`ing all copied files so that we ensure they have a "recent" timestamp ("now"), which should ensure that the file's timestamp is newer than that of e.g. `autogen.sh`. Additionally, there was a typo: Commit a5b324d1 used a `@(_InstallRuntimesOutput)` item group, but there is no such item group; it's actually `@(_InstallRuntimeOutput)` (no `s`). Fixity fix this. Finally, commit a5b324d1 forgot to add `libmono-btls-shared` to `@(BundleItem)`, preventing it from being added to the bundle. Fix this oversight.
2016-11-22 14:11:45 +03:00
<Touch Files="@(_MonoConstsOutput)" />
[mono-runtimes] Build required mono runtimes. Add a new build-tools/mono-runtimes project to build the required Mono runtimes for Xamarin.Android app execution on Android devices. Xamarin.Android 6.0 ("cycle 6") includes Mono runtimes compiled for five architectures: * armeabi (armv5) * armeabi-v7a (armv7) * arm64-v8a (aarch64) * x86 * x86_64 An additional "host" architecture is also built to build the "monodroid" profile assemblies such as mscorlib.dll. In the interest of expediency and not requiring that *six* different runtimes be built -- which can greatly increase build times and storage requirements -- the initial xamarin-android open-source build will only build the armeabi-v7a (armv7) and "host" architectures. Additionally, the <UnzipDirectoryChildren/> task needed to be altered to use unzip(1) on OS X instead of using System.IO.Compression.ZipFile, because the ZipFile implementation included with Mono 4.4 doesn't respect file attributes such as the +x, causing all files to be extracted as 644 (rw-r--r--), resulting in an NDK toolchain which wouldn't work. To resolve this issue, execution on OS X and Linux now uses unzip(1). ~~ Adding new architectures ~~ To build Mono for a new architecture, edit build-tools/mono-runtimes/mono-runtimes.projitems and add a new @(_MonoRuntime) entry for the desired architecture. TODO: Figure out if there's a reasonable way to "opt-in" or "opt-out" of CPU architectures so that it isn't an "all or nothing" prospect.
2016-04-20 04:30:00 +03:00
<Copy
[build] Build the monodroid-profile BCL assemblies reliably (#135) Commit 37b5fd51 broke `make jenkins` (and possible `make all`?) through the introduction of the `%(_MonoRuntime.DoBuild)` metadata: `%(_MonoRuntime.DoBuild)` wasn't added everywhere required. In particular, the `host-Darwin` item didn't gain `<DoBuild/>`, meaning *`host-Darwin` wasn't built*. This in turn meant that the monodroid BCL wasn't built (!) which caused a build break much later within `src/Mono.Android`: Executing: mono .../xamarin-android/external/Java.Interop/bin/Debug/jcw-gen.exe -o ".../xamarin-android/src/Mono.Android/obj/Debug/android-23/jcw/src" -L ".../xamarin-androidg/lib/xbuild-frameworks/MonoAndroid/v6.0/" -L ".../xamarin-android/src/Mono.Android/../../bin/Debug/lib/xbuild-frameworks/MonoAndroid/v6.0/../v1.0/" -L ".../xamarin-android/src/Mono.Android/../../MonoAndroid/v6.0/../v1.0/Facades" ".../xamarin-android/src/Mono.Android/../../bin/Debug/lib/xbuild-frameworks/MonoAndroid/v6.0/Mono.Android.dll" jcw-gen: Could not load assembly 'mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089'. Perhaps it doesn't exist in the Mono for Android profile? `mscorlib.dll` couldn't be found because it wasn't *built*, nor installed into `bin/Debug/lib/xbuild-frameworks/MonoAndroid/v1.0`. Additionally, fully building the AOT cross compilers requires adding `mxe-Win32` to `$(ALL_HOST_ABIS)`, otherwise e.g. `win-armeabi` fails to build because `arm_dpimacros.h` doesn't exist (as `arm_dpimacros.h` is built by `mxe-Win32`): Error parsing '.../xamarin-android/external/mono/mono/metadata/metadata-cross-helpers.c' .../xamarin-android/external/mono/mono/arch/arm/arm-codegen.h(937,10): fatal: 'mono/arch/arm/arm_dpimacros.h' file not found Error parsing '.../xamarin-android/external/mono/mono/mini/mini-cross-helpers.c' .../xamarin-android/external/mono/mono/arch/arm/arm-codegen.h(937,10): fatal: 'mono/arch/arm/arm_dpimacros.h' file not found Unhandled Exception: System.Exception: Expected to find struct definition for MonoObject Fix the `mono-runtimes.mdproj` build so that it doesn't re-execute targets when not necessary, by "rolling" with xbuild limitations and refactoring item groups as appropriate. In particular, xbuild doesn't like referencing item metadata within a `//Target/@Outputs` that in turn references a property group: <PropertyGroup> <V>Value</v> </PropertyGroup> <ItemGroup> <Item Include="Something"> <Metadata>$(V)</Metadata> </Item> </ItemGroup> <Target Name="Example" Inputs="..." Outputs="%(Item.Metadata)"> ... </Target> Whenever the `Example` target is run, it will *always* run, even if `Value` exists, because `xbuild` perceives the `//Target[@Name='Example']/@Outputs` value as being `$(V)`, *not* `Value`, which is wonderfully broken. To compensate, Don't Do That™; in particular, item metadata used within `//Target/@Outputs` *must* be hardcoded and not contain a property reference. This "hardcoding" removes the need for the `$(_Cross*ffsetsHeaderFile)` properties, so remove them. Remove `System.Net.Http.WebRequest.dll` from `@(_BclAssembly)`, as it is no longer built in the monodroid profile; requiring it as an output means that the "host" mono build is *always* built, which isn't needed. Improve the `_BuildRuntimes` and `_InstallRuntimes` targets so that their referenced files aren't *so* repetitive, in particular by "splitting out" `@(_RuntimeLibraries)` into `@(_RuntimeSource)`, `@(_ProfilerSource)`, and `@(_MonoPosixHelperSource)`. This allows more intelligently `strip`ing the native libraries in the Release configuration, while also maintaining generation of the `.d.so` files.
2016-07-28 13:24:18 +03:00
SourceFiles="@(_RuntimeSource)"
DestinationFiles="@(_InstallRuntimeOutput)"
[mono-runtimes] Build required mono runtimes. Add a new build-tools/mono-runtimes project to build the required Mono runtimes for Xamarin.Android app execution on Android devices. Xamarin.Android 6.0 ("cycle 6") includes Mono runtimes compiled for five architectures: * armeabi (armv5) * armeabi-v7a (armv7) * arm64-v8a (aarch64) * x86 * x86_64 An additional "host" architecture is also built to build the "monodroid" profile assemblies such as mscorlib.dll. In the interest of expediency and not requiring that *six* different runtimes be built -- which can greatly increase build times and storage requirements -- the initial xamarin-android open-source build will only build the armeabi-v7a (armv7) and "host" architectures. Additionally, the <UnzipDirectoryChildren/> task needed to be altered to use unzip(1) on OS X instead of using System.IO.Compression.ZipFile, because the ZipFile implementation included with Mono 4.4 doesn't respect file attributes such as the +x, causing all files to be extracted as 644 (rw-r--r--), resulting in an NDK toolchain which wouldn't work. To resolve this issue, execution on OS X and Linux now uses unzip(1). ~~ Adding new architectures ~~ To build Mono for a new architecture, edit build-tools/mono-runtimes/mono-runtimes.projitems and add a new @(_MonoRuntime) entry for the desired architecture. TODO: Figure out if there's a reasonable way to "opt-in" or "opt-out" of CPU architectures so that it isn't an "all or nothing" prospect.
2016-04-20 04:30:00 +03:00
/>
<Copy
[build] Build the monodroid-profile BCL assemblies reliably (#135) Commit 37b5fd51 broke `make jenkins` (and possible `make all`?) through the introduction of the `%(_MonoRuntime.DoBuild)` metadata: `%(_MonoRuntime.DoBuild)` wasn't added everywhere required. In particular, the `host-Darwin` item didn't gain `<DoBuild/>`, meaning *`host-Darwin` wasn't built*. This in turn meant that the monodroid BCL wasn't built (!) which caused a build break much later within `src/Mono.Android`: Executing: mono .../xamarin-android/external/Java.Interop/bin/Debug/jcw-gen.exe -o ".../xamarin-android/src/Mono.Android/obj/Debug/android-23/jcw/src" -L ".../xamarin-androidg/lib/xbuild-frameworks/MonoAndroid/v6.0/" -L ".../xamarin-android/src/Mono.Android/../../bin/Debug/lib/xbuild-frameworks/MonoAndroid/v6.0/../v1.0/" -L ".../xamarin-android/src/Mono.Android/../../MonoAndroid/v6.0/../v1.0/Facades" ".../xamarin-android/src/Mono.Android/../../bin/Debug/lib/xbuild-frameworks/MonoAndroid/v6.0/Mono.Android.dll" jcw-gen: Could not load assembly 'mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089'. Perhaps it doesn't exist in the Mono for Android profile? `mscorlib.dll` couldn't be found because it wasn't *built*, nor installed into `bin/Debug/lib/xbuild-frameworks/MonoAndroid/v1.0`. Additionally, fully building the AOT cross compilers requires adding `mxe-Win32` to `$(ALL_HOST_ABIS)`, otherwise e.g. `win-armeabi` fails to build because `arm_dpimacros.h` doesn't exist (as `arm_dpimacros.h` is built by `mxe-Win32`): Error parsing '.../xamarin-android/external/mono/mono/metadata/metadata-cross-helpers.c' .../xamarin-android/external/mono/mono/arch/arm/arm-codegen.h(937,10): fatal: 'mono/arch/arm/arm_dpimacros.h' file not found Error parsing '.../xamarin-android/external/mono/mono/mini/mini-cross-helpers.c' .../xamarin-android/external/mono/mono/arch/arm/arm-codegen.h(937,10): fatal: 'mono/arch/arm/arm_dpimacros.h' file not found Unhandled Exception: System.Exception: Expected to find struct definition for MonoObject Fix the `mono-runtimes.mdproj` build so that it doesn't re-execute targets when not necessary, by "rolling" with xbuild limitations and refactoring item groups as appropriate. In particular, xbuild doesn't like referencing item metadata within a `//Target/@Outputs` that in turn references a property group: <PropertyGroup> <V>Value</v> </PropertyGroup> <ItemGroup> <Item Include="Something"> <Metadata>$(V)</Metadata> </Item> </ItemGroup> <Target Name="Example" Inputs="..." Outputs="%(Item.Metadata)"> ... </Target> Whenever the `Example` target is run, it will *always* run, even if `Value` exists, because `xbuild` perceives the `//Target[@Name='Example']/@Outputs` value as being `$(V)`, *not* `Value`, which is wonderfully broken. To compensate, Don't Do That™; in particular, item metadata used within `//Target/@Outputs` *must* be hardcoded and not contain a property reference. This "hardcoding" removes the need for the `$(_Cross*ffsetsHeaderFile)` properties, so remove them. Remove `System.Net.Http.WebRequest.dll` from `@(_BclAssembly)`, as it is no longer built in the monodroid profile; requiring it as an output means that the "host" mono build is *always* built, which isn't needed. Improve the `_BuildRuntimes` and `_InstallRuntimes` targets so that their referenced files aren't *so* repetitive, in particular by "splitting out" `@(_RuntimeLibraries)` into `@(_RuntimeSource)`, `@(_ProfilerSource)`, and `@(_MonoPosixHelperSource)`. This allows more intelligently `strip`ing the native libraries in the Release configuration, while also maintaining generation of the `.d.so` files.
2016-07-28 13:24:18 +03:00
SourceFiles="@(_RuntimeSource)"
DestinationFiles="@(_InstallUnstrippedRuntimeOutput)"
[mono-runtimes] Build required mono runtimes. Add a new build-tools/mono-runtimes project to build the required Mono runtimes for Xamarin.Android app execution on Android devices. Xamarin.Android 6.0 ("cycle 6") includes Mono runtimes compiled for five architectures: * armeabi (armv5) * armeabi-v7a (armv7) * arm64-v8a (aarch64) * x86 * x86_64 An additional "host" architecture is also built to build the "monodroid" profile assemblies such as mscorlib.dll. In the interest of expediency and not requiring that *six* different runtimes be built -- which can greatly increase build times and storage requirements -- the initial xamarin-android open-source build will only build the armeabi-v7a (armv7) and "host" architectures. Additionally, the <UnzipDirectoryChildren/> task needed to be altered to use unzip(1) on OS X instead of using System.IO.Compression.ZipFile, because the ZipFile implementation included with Mono 4.4 doesn't respect file attributes such as the +x, causing all files to be extracted as 644 (rw-r--r--), resulting in an NDK toolchain which wouldn't work. To resolve this issue, execution on OS X and Linux now uses unzip(1). ~~ Adding new architectures ~~ To build Mono for a new architecture, edit build-tools/mono-runtimes/mono-runtimes.projitems and add a new @(_MonoRuntime) entry for the desired architecture. TODO: Figure out if there's a reasonable way to "opt-in" or "opt-out" of CPU architectures so that it isn't an "all or nothing" prospect.
2016-04-20 04:30:00 +03:00
/>
<Exec
[create-vsix] Create `.vsix` files (#541) Visual Studio 2017 uses `.vsix` files for Xamarin.Android SDK support. A `.vsix` file is [ZIP container with additional metadata][0], and the [Microsoft.VSSDK.BuildTools NuGet package][1] contains various MSBuild targets and tools to assist in creating `.vsix` files. Add a new `build-tools/create-vsix/create-vsix.csproj` project to create the `bin/Build$(Configuration)/Xamarin.Android.Sdk*.vsix` file, so that we can plausibly provide per-build OSS Xamarin.Android releases that work with Visual Studio 2017. Unfortunately those tools were written on Windows, and not really well tested on macOS or Linux... In particular, there are case-sensitivity and directory-separator-char issues with the tooling, necessitating that `MONO_IOMAP=all` be exported in order to run them: MONO_IOMAP=all MONO_OPTIONS=--arch=64 msbuild \ build-tools/create-vsix/create-vsix.csproj /p:CreateVsixContainer=True Meanwhile, we're still attempting to allow things to be built with `xbuild` [^3]. Thread this needle by "special-casing" the `$(BuildDependsOn)`, `$(CopyVsixManifestFileDependsOn)`, and `$(DetokenizeVsixManifestFileDependsOn)` MSBuild properties so that when the `$(CreateVsixContainer)` MSBuild property is False -- the default -- no `.vsix` package will be created, and `xbuild` will be able to build the project. Similar needle threading is needed to "support" building on Linux: the Microsoft.VSSDK.BuildTools NuGet package uses case in an inconsistent fashion -- or is it `nuget`? -- which results in [failures when building on Linux][2] because `nuget` extracts e.g.: packages/Microsoft.VSSDK.BuildTools.15.0.26201/tools/vssdk/Microsoft.VsSDK.targets while Microsoft.VSSDK.BuildTools attempts to `<Import/>` the file: packages/Microsoft.VSSDK.BuildTools.15.0.26201/tools/VSSDK/Microsoft.VsSDK.targets `vssdk` != `VSSDK` on case-sensitive filesystems, so this results in an error on Ubuntu (and presumably case-senstive macOS as well). Handle the Linux case by extending the `xbuild` case: *even when* `$(CreateVsixContainer)` is True, we won't actually build the `.vsix` unless the `$(VsSDKInstall)` directory exists, which is the `tools/VSSDK` path. Building with `$(CreateVsixContainer)` set to True will require using `msbuild` with `MONO_IOMAP=all` and `MONO_OPPTIONS=--arch=64` exported, while using a case-insensitive filesystem. The new `make create-vsix CONFIGURATIONS=Debug` target can be used to explicitly create the `Xamarin.Android.Sdk*.vsix` file. One problem with creating `.vsix` files: macOS still defaults to using a 32-bit process for `mono`, and `make create-vsix` regularly fails for me with an `OutOfMemoryException` when attempting to generate a `.vsix` file ~600-700+MB in size. (Why so large? In part because for Debug configuration we're including un-`strip`'d native libraries; `libmonosgen-2.0.dll` is *237MB* in size (!); it's closer to 5MB when `strip`'d, but any un-`strip`'d MXE-generated native libraries to the `.vsix` file quickly results in `OutOfMemoryException`s.) A 64-bit mono can be used by using `mono64` -- which isn't easily done with `msbuild` -- or by using `mono --arch=64`, which *can* be done with `msbuild` by exporting `MONO_OPTIONS=--arch=64`. The `make create-vsix` target does this. Finally, once we have a `.visx` being created, we then examine the *contents* of the `.vsix` file, which is...weird: $MSBuild/Xamarin/Android/Xamarin.Android.CSharp.targets # Desirable $ReferenceAssemblies/Microsoft/Framework/MonoAndroid/v1.0/Xamarin.Android.CSharp.targets # wat? $ReferenceAssemblies/Microsoft/Framework/MonoAndroid/v1.0/Facades/Xamarin.Android.CSharp.targets # really?! $ReferenceAssemblies/Microsoft/Framework/MonoAndroid/v7.1/Xamarin.Android.CSharp.targets # !!!! Turns Out™, the problem was due to `@(ProjectReference)`. We're (ab)using `@(ProjectReference)` to explicitly specify project dependencies, and the (implicit) project build order. Unexpectedly (forgetten?), MSBuild *also* copies the *outputs* of `@(ProjectReference)`s into the `$(OutputPath)` of the current project. Since e.g. `Mono.Posix.csproj` had a reference on `Xamarin.Android.Build.Tasks.csproj`, the result of this is that the `$(OutputPath)` of `Mono.Posix.csproj` -- `$prefix/lib/xbuild-frameworks/MonoAndroid/v1.0` -- also contained the `%(None.CopyToOutputDirectory)` items from `Xamarin.Android.Build.Tasks.csproj`. Oops. The fix is to set `%(ProjectReference.Private)` to False, which disables this copying of additional and undesirable files. [0]: https://msdn.microsoft.com/en-us/library/dd997148.aspx [1]: https://www.nuget.org/packages/Microsoft.VSSDK.BuildTools/ [2]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-anroid-linux-pr-builder/297/consoleText [^3]: But for how much longer?
2017-04-05 18:49:37 +03:00
Condition=" ('$(Configuration)' != 'Debug' Or '%(_MonoRuntime.NativeLibraryExtension)' == 'dll') And '%(_MonoRuntime.DoBuild)' == 'true' "
[structure] Rework installation directory structure (#704) Context: https://github.com/xamarin/xamarin-android/pull/253#discussion_r82862993 The *intention* is that Jenkins-produced `oss-xamarin.android*.zip` artifacts be usable on Windows, so that side-by-side testing can be performed without replacing the system installation. Usage is in [UsingJenkinsBuildArtifacts.md](Documentation/UsingJenkinsBuildArtifacts). This isn't *entirely* the case. It was *apparently* the case at the time of commit 87ca2737, but things have bitrotten since. For example, following the 87ca2737 instructions would currently result in an XA0020 `Could not find monodroid` error, because `class-parse.exe` wasn't where Windows expects it to be (it was in `lib/mandroid`, not `lib/xbuild/Xamarin/Android`). This needs to be fixed. Additionally, PR #253 mentions that, for filesystem organization, it would be useful if the macOS/Linux directory structure -- `$prefix/bin`, `$prefix/lib/mandroid`, `$prefix/lib/xbuild/Xamarin/Android`, `$prefix/lib/xbuild-frameworks` -- more closely resembled the Windows directory structure of `$MSBuild` and `$ReferenceAssemblies` (as seen in `.vsix` files). This would turn macOS/Linux into using `$xa_prefix/xbuild` and `$xa_prefix/xbuild-frameworks` directories. `$prefix/bin` would only contain `xabuild`. What is currently in `$prefix/lib/mandroid` would be merged with `$xa_prefix/xbuild/Xamarin/Android`. `$xa_prefix` would `$prefix/lib/xamarin.android`. This would turn the current macOS structure: $prefix/bin/xabuild $prefix/bin/generator $prefix/bin/cross-arm $prefix/lib/mandroid/generator.exe $prefix/lib/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xbuild/Xamarin/Android/Xamarin.Android.Common.targets Into: $prefix/bin/xabuild $prefix/lib/xamarin.android/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xamarin.android/xbuild/Xamarin/Android/generator.exe $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Xamarin.Android.Common.targets $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Darwin/cross-arm Other notes: * The `bundle-*.zip` filename has been has been changed to include a *hash* of the contents of various files, in particular `build-tools\mono-runtimes.*`. This was instigated via a conversation with @kumpera about making the filename more "idiot-proof": `mono-runtimes.props` contains *compiler flags*, and if any of those change, then *logically* the bundle should differ as well, as the mono runtimes may differ in significant ways. In theory, the `-vXX` version should be used to track this, but this is a manual change, easy to overlook. The new `-hHASH` part of the filename should be more automagic. The new `<HashFileContents/>` task in `xa-prep-tasks.dll` is responsible for creating the hash value. * `Configuration.Java.Interop.Override.props` was moved into `build-tools/scripts`, because that would cleanup the root directory a bit. * OS-specific binaries are now placed into `$prefix/lib/xamarin.android/xbuild/Xamarin/Android/$(HostOS)`. On a theoretical plus side, this means that the same build directory can contain OS-specific binaries for multiple operating systems. (I don't know if anyone shares a build tree between e.g. macOS and Linux, but if anyone *does*...) Unfortunately this requires a workaround for an `xbuild` bug: `%(_LlvmRuntime.InstallPath)` and `%(_MonoCrossRuntime.InstallPath)` *cannot* use MSBuild properties, e.g. `<InstallPath>$(HostOS)/</InstallPath>` doesn't work as desired; it's instead treated literally. Special-case `%(InstallPath)` until we fully migrate to MSBuild. * `$(MonoAndroidToolsDirectory)` should be considered *dead*, along with `$(MonoAndroidBinDirectory)`, as these should now *always* be the same directory as where `Xamarin.Android.Build.Tasks.dll` is located, or a `$(HostOS)` sub-directory. * `Xamarin.ProjectTools.csproj` needed to be updated to ensure that the build order was correct. * Remove all `[Obsolete]` and unreferenced members from `Xamarin.Android.Build.Utilities.dll`. There's too much in there, and it makes my head hurt trying to understand the interrelationships between it all. If it's not used, it's gone. * The changes to `src/monodroid/jni/Android.mk` are...weird. The removal of `-I$(topdir)/libmonodroid/zip`/etc. is to reduce use of non-existent paths, as `$(topdir)` isn't defined, so that's *actually* `-I/libmonodroid/zip`, which is nonsensical. So far, so good. What's *odd* is the addition of `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`: This is needed so that `external/mono/support/zlib-helper.c` exports `CreateZStream` and related symbols, otherwise we get a unit test failure in `GzipStreamTest.Compression` due to an `EntryPointNotFoundException`, because `CreateZStream` isn't exported/public. What's odd here is that I don't understand what caused this behavior to change. Previous builds exported `CreateZStream`, otherwise the tests would fail, and I don't understand how any of the other changes in this PR would be at fault, though that's certainly the most plausible explanation. Regardless, `-Ijni` *first* (due to adding `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`) is the desired behavior, so that `jni/config.h` is included, thus ensuring that `MONO_API` has the required definition when building `zlib-helper.c`.
2017-07-28 16:36:32 +03:00
Command="&quot;%(_MonoRuntime.Strip)&quot; %(_MonoRuntime.StripFlags) &quot;$(_MSBuildDir)\lib\%(_MonoRuntime.Identity)\%(_MonoRuntime.OutputRuntimeFilename).%(_MonoRuntime.NativeLibraryExtension)&quot;"
[mono-runtimes] Build required mono runtimes. Add a new build-tools/mono-runtimes project to build the required Mono runtimes for Xamarin.Android app execution on Android devices. Xamarin.Android 6.0 ("cycle 6") includes Mono runtimes compiled for five architectures: * armeabi (armv5) * armeabi-v7a (armv7) * arm64-v8a (aarch64) * x86 * x86_64 An additional "host" architecture is also built to build the "monodroid" profile assemblies such as mscorlib.dll. In the interest of expediency and not requiring that *six* different runtimes be built -- which can greatly increase build times and storage requirements -- the initial xamarin-android open-source build will only build the armeabi-v7a (armv7) and "host" architectures. Additionally, the <UnzipDirectoryChildren/> task needed to be altered to use unzip(1) on OS X instead of using System.IO.Compression.ZipFile, because the ZipFile implementation included with Mono 4.4 doesn't respect file attributes such as the +x, causing all files to be extracted as 644 (rw-r--r--), resulting in an NDK toolchain which wouldn't work. To resolve this issue, execution on OS X and Linux now uses unzip(1). ~~ Adding new architectures ~~ To build Mono for a new architecture, edit build-tools/mono-runtimes/mono-runtimes.projitems and add a new @(_MonoRuntime) entry for the desired architecture. TODO: Figure out if there's a reasonable way to "opt-in" or "opt-out" of CPU architectures so that it isn't an "all or nothing" prospect.
2016-04-20 04:30:00 +03:00
/>
[mono-runtimes] Fix @(BundleItem) timestamps (#307) The *intent* of the `_BuildUnlessCached` target within `mono-runtimes.targets` is to (attempt to!) make building mono "free", or at least, free *enough* so that other project files can have a build-time dependency on `mono-runtimes.mdproj` so that build artifacts can be (reasonably) consistent. Unfortunately, that wasn't the case, with the net result being that: tools/scripts/xabuild /t:SignAndroidPackage src/Mono.Android/Test/*.csproj would *constantly* attempt to re-build `build-tools/mono-runtimes.mdproj`, adding (at least!) one minute as we ran mono's `make`...to do nothing. Why? Target _BuildUnlessCached needs to be built as input file '.../xamarin-android/external/mono/autogen.sh' is newer than output file '../../bin/Debug//include/armeabi-v7a/eglib/config.h' The "problem" is that the `<Copy/>` task copies over the timestamp as well, so even though we're (possibly) creating a new file, the timestamp on the created file will match the source file. Normally this is fine. In this case, we're dealing with `eglib`, which is a "nested" ("delegated"?) `configure` invocation from mono's `configure`. It *appears* that if (when) mono is updated, "normal" mono subdirectories will "properly" honor `--enable-maintainer-mode` and re-run mono's `configure`, which will in turn update the timstamp on any `configure`-time generated files. This doesn't appear to be the case for `mono/eglib`; when `mono` is updated, `mono/eglib` does *not* re-run `configure`, and thus no timestamps are updated to follow suit. Fix this by *explicitly* `<Touch/>`ing all copied files so that we ensure they have a "recent" timestamp ("now"), which should ensure that the file's timestamp is newer than that of e.g. `autogen.sh`. Additionally, there was a typo: Commit a5b324d1 used a `@(_InstallRuntimesOutput)` item group, but there is no such item group; it's actually `@(_InstallRuntimeOutput)` (no `s`). Fixity fix this. Finally, commit a5b324d1 forgot to add `libmono-btls-shared` to `@(BundleItem)`, preventing it from being added to the bundle. Fix this oversight.
2016-11-22 14:11:45 +03:00
<Touch Files="@(_InstallRuntimeOutput);@(_InstallUnstrippedRuntimeOutput)" />
[build] Build the monodroid-profile BCL assemblies reliably (#135) Commit 37b5fd51 broke `make jenkins` (and possible `make all`?) through the introduction of the `%(_MonoRuntime.DoBuild)` metadata: `%(_MonoRuntime.DoBuild)` wasn't added everywhere required. In particular, the `host-Darwin` item didn't gain `<DoBuild/>`, meaning *`host-Darwin` wasn't built*. This in turn meant that the monodroid BCL wasn't built (!) which caused a build break much later within `src/Mono.Android`: Executing: mono .../xamarin-android/external/Java.Interop/bin/Debug/jcw-gen.exe -o ".../xamarin-android/src/Mono.Android/obj/Debug/android-23/jcw/src" -L ".../xamarin-androidg/lib/xbuild-frameworks/MonoAndroid/v6.0/" -L ".../xamarin-android/src/Mono.Android/../../bin/Debug/lib/xbuild-frameworks/MonoAndroid/v6.0/../v1.0/" -L ".../xamarin-android/src/Mono.Android/../../MonoAndroid/v6.0/../v1.0/Facades" ".../xamarin-android/src/Mono.Android/../../bin/Debug/lib/xbuild-frameworks/MonoAndroid/v6.0/Mono.Android.dll" jcw-gen: Could not load assembly 'mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089'. Perhaps it doesn't exist in the Mono for Android profile? `mscorlib.dll` couldn't be found because it wasn't *built*, nor installed into `bin/Debug/lib/xbuild-frameworks/MonoAndroid/v1.0`. Additionally, fully building the AOT cross compilers requires adding `mxe-Win32` to `$(ALL_HOST_ABIS)`, otherwise e.g. `win-armeabi` fails to build because `arm_dpimacros.h` doesn't exist (as `arm_dpimacros.h` is built by `mxe-Win32`): Error parsing '.../xamarin-android/external/mono/mono/metadata/metadata-cross-helpers.c' .../xamarin-android/external/mono/mono/arch/arm/arm-codegen.h(937,10): fatal: 'mono/arch/arm/arm_dpimacros.h' file not found Error parsing '.../xamarin-android/external/mono/mono/mini/mini-cross-helpers.c' .../xamarin-android/external/mono/mono/arch/arm/arm-codegen.h(937,10): fatal: 'mono/arch/arm/arm_dpimacros.h' file not found Unhandled Exception: System.Exception: Expected to find struct definition for MonoObject Fix the `mono-runtimes.mdproj` build so that it doesn't re-execute targets when not necessary, by "rolling" with xbuild limitations and refactoring item groups as appropriate. In particular, xbuild doesn't like referencing item metadata within a `//Target/@Outputs` that in turn references a property group: <PropertyGroup> <V>Value</v> </PropertyGroup> <ItemGroup> <Item Include="Something"> <Metadata>$(V)</Metadata> </Item> </ItemGroup> <Target Name="Example" Inputs="..." Outputs="%(Item.Metadata)"> ... </Target> Whenever the `Example` target is run, it will *always* run, even if `Value` exists, because `xbuild` perceives the `//Target[@Name='Example']/@Outputs` value as being `$(V)`, *not* `Value`, which is wonderfully broken. To compensate, Don't Do That™; in particular, item metadata used within `//Target/@Outputs` *must* be hardcoded and not contain a property reference. This "hardcoding" removes the need for the `$(_Cross*ffsetsHeaderFile)` properties, so remove them. Remove `System.Net.Http.WebRequest.dll` from `@(_BclAssembly)`, as it is no longer built in the monodroid profile; requiring it as an output means that the "host" mono build is *always* built, which isn't needed. Improve the `_BuildRuntimes` and `_InstallRuntimes` targets so that their referenced files aren't *so* repetitive, in particular by "splitting out" `@(_RuntimeLibraries)` into `@(_RuntimeSource)`, `@(_ProfilerSource)`, and `@(_MonoPosixHelperSource)`. This allows more intelligently `strip`ing the native libraries in the Release configuration, while also maintaining generation of the `.d.so` files.
2016-07-28 13:24:18 +03:00
<Copy
SourceFiles="@(_ProfilerSource)"
DestinationFiles="@(_InstallProfilerOutput)"
/>
<Copy
SourceFiles="@(_ProfilerSource)"
DestinationFiles="@(_InstallUnstrippedProfilerOutput)"
/>
<Exec
Condition=" '$(Configuration)' != 'Debug' And '%(_MonoRuntime.DoBuild)' == 'true' And '%(_MonoRuntime.OutputProfilerFilename)' != '' "
[structure] Rework installation directory structure (#704) Context: https://github.com/xamarin/xamarin-android/pull/253#discussion_r82862993 The *intention* is that Jenkins-produced `oss-xamarin.android*.zip` artifacts be usable on Windows, so that side-by-side testing can be performed without replacing the system installation. Usage is in [UsingJenkinsBuildArtifacts.md](Documentation/UsingJenkinsBuildArtifacts). This isn't *entirely* the case. It was *apparently* the case at the time of commit 87ca2737, but things have bitrotten since. For example, following the 87ca2737 instructions would currently result in an XA0020 `Could not find monodroid` error, because `class-parse.exe` wasn't where Windows expects it to be (it was in `lib/mandroid`, not `lib/xbuild/Xamarin/Android`). This needs to be fixed. Additionally, PR #253 mentions that, for filesystem organization, it would be useful if the macOS/Linux directory structure -- `$prefix/bin`, `$prefix/lib/mandroid`, `$prefix/lib/xbuild/Xamarin/Android`, `$prefix/lib/xbuild-frameworks` -- more closely resembled the Windows directory structure of `$MSBuild` and `$ReferenceAssemblies` (as seen in `.vsix` files). This would turn macOS/Linux into using `$xa_prefix/xbuild` and `$xa_prefix/xbuild-frameworks` directories. `$prefix/bin` would only contain `xabuild`. What is currently in `$prefix/lib/mandroid` would be merged with `$xa_prefix/xbuild/Xamarin/Android`. `$xa_prefix` would `$prefix/lib/xamarin.android`. This would turn the current macOS structure: $prefix/bin/xabuild $prefix/bin/generator $prefix/bin/cross-arm $prefix/lib/mandroid/generator.exe $prefix/lib/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xbuild/Xamarin/Android/Xamarin.Android.Common.targets Into: $prefix/bin/xabuild $prefix/lib/xamarin.android/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xamarin.android/xbuild/Xamarin/Android/generator.exe $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Xamarin.Android.Common.targets $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Darwin/cross-arm Other notes: * The `bundle-*.zip` filename has been has been changed to include a *hash* of the contents of various files, in particular `build-tools\mono-runtimes.*`. This was instigated via a conversation with @kumpera about making the filename more "idiot-proof": `mono-runtimes.props` contains *compiler flags*, and if any of those change, then *logically* the bundle should differ as well, as the mono runtimes may differ in significant ways. In theory, the `-vXX` version should be used to track this, but this is a manual change, easy to overlook. The new `-hHASH` part of the filename should be more automagic. The new `<HashFileContents/>` task in `xa-prep-tasks.dll` is responsible for creating the hash value. * `Configuration.Java.Interop.Override.props` was moved into `build-tools/scripts`, because that would cleanup the root directory a bit. * OS-specific binaries are now placed into `$prefix/lib/xamarin.android/xbuild/Xamarin/Android/$(HostOS)`. On a theoretical plus side, this means that the same build directory can contain OS-specific binaries for multiple operating systems. (I don't know if anyone shares a build tree between e.g. macOS and Linux, but if anyone *does*...) Unfortunately this requires a workaround for an `xbuild` bug: `%(_LlvmRuntime.InstallPath)` and `%(_MonoCrossRuntime.InstallPath)` *cannot* use MSBuild properties, e.g. `<InstallPath>$(HostOS)/</InstallPath>` doesn't work as desired; it's instead treated literally. Special-case `%(InstallPath)` until we fully migrate to MSBuild. * `$(MonoAndroidToolsDirectory)` should be considered *dead*, along with `$(MonoAndroidBinDirectory)`, as these should now *always* be the same directory as where `Xamarin.Android.Build.Tasks.dll` is located, or a `$(HostOS)` sub-directory. * `Xamarin.ProjectTools.csproj` needed to be updated to ensure that the build order was correct. * Remove all `[Obsolete]` and unreferenced members from `Xamarin.Android.Build.Utilities.dll`. There's too much in there, and it makes my head hurt trying to understand the interrelationships between it all. If it's not used, it's gone. * The changes to `src/monodroid/jni/Android.mk` are...weird. The removal of `-I$(topdir)/libmonodroid/zip`/etc. is to reduce use of non-existent paths, as `$(topdir)` isn't defined, so that's *actually* `-I/libmonodroid/zip`, which is nonsensical. So far, so good. What's *odd* is the addition of `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`: This is needed so that `external/mono/support/zlib-helper.c` exports `CreateZStream` and related symbols, otherwise we get a unit test failure in `GzipStreamTest.Compression` due to an `EntryPointNotFoundException`, because `CreateZStream` isn't exported/public. What's odd here is that I don't understand what caused this behavior to change. Previous builds exported `CreateZStream`, otherwise the tests would fail, and I don't understand how any of the other changes in this PR would be at fault, though that's certainly the most plausible explanation. Regardless, `-Ijni` *first* (due to adding `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`) is the desired behavior, so that `jni/config.h` is included, thus ensuring that `MONO_API` has the required definition when building `zlib-helper.c`.
2017-07-28 16:36:32 +03:00
Command="&quot;%(_MonoRuntime.Strip)&quot; %(_MonoRuntime.StripFlags) &quot;$(_MSBuildDir)\lib\%(_MonoRuntime.Identity)\%(_MonoRuntime.OutputProfilerFilename).%(_MonoRuntime.NativeLibraryExtension)&quot;"
[build] Build the monodroid-profile BCL assemblies reliably (#135) Commit 37b5fd51 broke `make jenkins` (and possible `make all`?) through the introduction of the `%(_MonoRuntime.DoBuild)` metadata: `%(_MonoRuntime.DoBuild)` wasn't added everywhere required. In particular, the `host-Darwin` item didn't gain `<DoBuild/>`, meaning *`host-Darwin` wasn't built*. This in turn meant that the monodroid BCL wasn't built (!) which caused a build break much later within `src/Mono.Android`: Executing: mono .../xamarin-android/external/Java.Interop/bin/Debug/jcw-gen.exe -o ".../xamarin-android/src/Mono.Android/obj/Debug/android-23/jcw/src" -L ".../xamarin-androidg/lib/xbuild-frameworks/MonoAndroid/v6.0/" -L ".../xamarin-android/src/Mono.Android/../../bin/Debug/lib/xbuild-frameworks/MonoAndroid/v6.0/../v1.0/" -L ".../xamarin-android/src/Mono.Android/../../MonoAndroid/v6.0/../v1.0/Facades" ".../xamarin-android/src/Mono.Android/../../bin/Debug/lib/xbuild-frameworks/MonoAndroid/v6.0/Mono.Android.dll" jcw-gen: Could not load assembly 'mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089'. Perhaps it doesn't exist in the Mono for Android profile? `mscorlib.dll` couldn't be found because it wasn't *built*, nor installed into `bin/Debug/lib/xbuild-frameworks/MonoAndroid/v1.0`. Additionally, fully building the AOT cross compilers requires adding `mxe-Win32` to `$(ALL_HOST_ABIS)`, otherwise e.g. `win-armeabi` fails to build because `arm_dpimacros.h` doesn't exist (as `arm_dpimacros.h` is built by `mxe-Win32`): Error parsing '.../xamarin-android/external/mono/mono/metadata/metadata-cross-helpers.c' .../xamarin-android/external/mono/mono/arch/arm/arm-codegen.h(937,10): fatal: 'mono/arch/arm/arm_dpimacros.h' file not found Error parsing '.../xamarin-android/external/mono/mono/mini/mini-cross-helpers.c' .../xamarin-android/external/mono/mono/arch/arm/arm-codegen.h(937,10): fatal: 'mono/arch/arm/arm_dpimacros.h' file not found Unhandled Exception: System.Exception: Expected to find struct definition for MonoObject Fix the `mono-runtimes.mdproj` build so that it doesn't re-execute targets when not necessary, by "rolling" with xbuild limitations and refactoring item groups as appropriate. In particular, xbuild doesn't like referencing item metadata within a `//Target/@Outputs` that in turn references a property group: <PropertyGroup> <V>Value</v> </PropertyGroup> <ItemGroup> <Item Include="Something"> <Metadata>$(V)</Metadata> </Item> </ItemGroup> <Target Name="Example" Inputs="..." Outputs="%(Item.Metadata)"> ... </Target> Whenever the `Example` target is run, it will *always* run, even if `Value` exists, because `xbuild` perceives the `//Target[@Name='Example']/@Outputs` value as being `$(V)`, *not* `Value`, which is wonderfully broken. To compensate, Don't Do That™; in particular, item metadata used within `//Target/@Outputs` *must* be hardcoded and not contain a property reference. This "hardcoding" removes the need for the `$(_Cross*ffsetsHeaderFile)` properties, so remove them. Remove `System.Net.Http.WebRequest.dll` from `@(_BclAssembly)`, as it is no longer built in the monodroid profile; requiring it as an output means that the "host" mono build is *always* built, which isn't needed. Improve the `_BuildRuntimes` and `_InstallRuntimes` targets so that their referenced files aren't *so* repetitive, in particular by "splitting out" `@(_RuntimeLibraries)` into `@(_RuntimeSource)`, `@(_ProfilerSource)`, and `@(_MonoPosixHelperSource)`. This allows more intelligently `strip`ing the native libraries in the Release configuration, while also maintaining generation of the `.d.so` files.
2016-07-28 13:24:18 +03:00
/>
[mono-runtimes] Fix @(BundleItem) timestamps (#307) The *intent* of the `_BuildUnlessCached` target within `mono-runtimes.targets` is to (attempt to!) make building mono "free", or at least, free *enough* so that other project files can have a build-time dependency on `mono-runtimes.mdproj` so that build artifacts can be (reasonably) consistent. Unfortunately, that wasn't the case, with the net result being that: tools/scripts/xabuild /t:SignAndroidPackage src/Mono.Android/Test/*.csproj would *constantly* attempt to re-build `build-tools/mono-runtimes.mdproj`, adding (at least!) one minute as we ran mono's `make`...to do nothing. Why? Target _BuildUnlessCached needs to be built as input file '.../xamarin-android/external/mono/autogen.sh' is newer than output file '../../bin/Debug//include/armeabi-v7a/eglib/config.h' The "problem" is that the `<Copy/>` task copies over the timestamp as well, so even though we're (possibly) creating a new file, the timestamp on the created file will match the source file. Normally this is fine. In this case, we're dealing with `eglib`, which is a "nested" ("delegated"?) `configure` invocation from mono's `configure`. It *appears* that if (when) mono is updated, "normal" mono subdirectories will "properly" honor `--enable-maintainer-mode` and re-run mono's `configure`, which will in turn update the timstamp on any `configure`-time generated files. This doesn't appear to be the case for `mono/eglib`; when `mono` is updated, `mono/eglib` does *not* re-run `configure`, and thus no timestamps are updated to follow suit. Fix this by *explicitly* `<Touch/>`ing all copied files so that we ensure they have a "recent" timestamp ("now"), which should ensure that the file's timestamp is newer than that of e.g. `autogen.sh`. Additionally, there was a typo: Commit a5b324d1 used a `@(_InstallRuntimesOutput)` item group, but there is no such item group; it's actually `@(_InstallRuntimeOutput)` (no `s`). Fixity fix this. Finally, commit a5b324d1 forgot to add `libmono-btls-shared` to `@(BundleItem)`, preventing it from being added to the bundle. Fix this oversight.
2016-11-22 14:11:45 +03:00
<Touch Files="@(_InstallProfilerOutput);@(_InstallUnstrippedProfilerOutput)" />
[build] Build the monodroid-profile BCL assemblies reliably (#135) Commit 37b5fd51 broke `make jenkins` (and possible `make all`?) through the introduction of the `%(_MonoRuntime.DoBuild)` metadata: `%(_MonoRuntime.DoBuild)` wasn't added everywhere required. In particular, the `host-Darwin` item didn't gain `<DoBuild/>`, meaning *`host-Darwin` wasn't built*. This in turn meant that the monodroid BCL wasn't built (!) which caused a build break much later within `src/Mono.Android`: Executing: mono .../xamarin-android/external/Java.Interop/bin/Debug/jcw-gen.exe -o ".../xamarin-android/src/Mono.Android/obj/Debug/android-23/jcw/src" -L ".../xamarin-androidg/lib/xbuild-frameworks/MonoAndroid/v6.0/" -L ".../xamarin-android/src/Mono.Android/../../bin/Debug/lib/xbuild-frameworks/MonoAndroid/v6.0/../v1.0/" -L ".../xamarin-android/src/Mono.Android/../../MonoAndroid/v6.0/../v1.0/Facades" ".../xamarin-android/src/Mono.Android/../../bin/Debug/lib/xbuild-frameworks/MonoAndroid/v6.0/Mono.Android.dll" jcw-gen: Could not load assembly 'mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089'. Perhaps it doesn't exist in the Mono for Android profile? `mscorlib.dll` couldn't be found because it wasn't *built*, nor installed into `bin/Debug/lib/xbuild-frameworks/MonoAndroid/v1.0`. Additionally, fully building the AOT cross compilers requires adding `mxe-Win32` to `$(ALL_HOST_ABIS)`, otherwise e.g. `win-armeabi` fails to build because `arm_dpimacros.h` doesn't exist (as `arm_dpimacros.h` is built by `mxe-Win32`): Error parsing '.../xamarin-android/external/mono/mono/metadata/metadata-cross-helpers.c' .../xamarin-android/external/mono/mono/arch/arm/arm-codegen.h(937,10): fatal: 'mono/arch/arm/arm_dpimacros.h' file not found Error parsing '.../xamarin-android/external/mono/mono/mini/mini-cross-helpers.c' .../xamarin-android/external/mono/mono/arch/arm/arm-codegen.h(937,10): fatal: 'mono/arch/arm/arm_dpimacros.h' file not found Unhandled Exception: System.Exception: Expected to find struct definition for MonoObject Fix the `mono-runtimes.mdproj` build so that it doesn't re-execute targets when not necessary, by "rolling" with xbuild limitations and refactoring item groups as appropriate. In particular, xbuild doesn't like referencing item metadata within a `//Target/@Outputs` that in turn references a property group: <PropertyGroup> <V>Value</v> </PropertyGroup> <ItemGroup> <Item Include="Something"> <Metadata>$(V)</Metadata> </Item> </ItemGroup> <Target Name="Example" Inputs="..." Outputs="%(Item.Metadata)"> ... </Target> Whenever the `Example` target is run, it will *always* run, even if `Value` exists, because `xbuild` perceives the `//Target[@Name='Example']/@Outputs` value as being `$(V)`, *not* `Value`, which is wonderfully broken. To compensate, Don't Do That™; in particular, item metadata used within `//Target/@Outputs` *must* be hardcoded and not contain a property reference. This "hardcoding" removes the need for the `$(_Cross*ffsetsHeaderFile)` properties, so remove them. Remove `System.Net.Http.WebRequest.dll` from `@(_BclAssembly)`, as it is no longer built in the monodroid profile; requiring it as an output means that the "host" mono build is *always* built, which isn't needed. Improve the `_BuildRuntimes` and `_InstallRuntimes` targets so that their referenced files aren't *so* repetitive, in particular by "splitting out" `@(_RuntimeLibraries)` into `@(_RuntimeSource)`, `@(_ProfilerSource)`, and `@(_MonoPosixHelperSource)`. This allows more intelligently `strip`ing the native libraries in the Release configuration, while also maintaining generation of the `.d.so` files.
2016-07-28 13:24:18 +03:00
Bump to mono/mono-4.8.0-branch/9f4abcc3 (#278) * Bump to mono/mono-4.8.0-branch/9f4abcc3 [mono/085b653a][0] splits out the BTLS code into a new, separate, `libmono-btls-shared.so` native library. This allows BTLS inclusion to be *optional*, so that the size increase is only paid when it's used. Bump to mono/9f4abcc3 so that we can use this new feature. [0]: https://github.com/mono/mono/commit/085b653a7425e30516150d2335c39a6906e6cd4d * [mono-runtimes, Xamarin.Android.Build.Tasks] Use separate BTLS lib There was an unintended side effect of commit cebc6c83, which added `--with-btls-android-ndk` to the mono runtime configure command: BTLS was compiled into `libmonosgen-2.0.so`, greatly increasing size: $ ls -lh bin/Release/lib/xbuild/Xamarin/Android/lib/arm64-v8a/libmonosgen-2.0.so /Library/Frameworks/Xamarin.Android.framework/Libraries/xbuild/Xamarin/Android/lib/arm64-v8a/libmonosgen-2.0.so -rwxr-xr-x 1 root wheel 2.8M Sep 23 18:34 /Library/Frameworks/Xamarin.Android.framework/Libraries/xbuild/Xamarin/Android/lib/arm64-v8a/libmonosgen-2.0.so -rwxr-xr-x 1 jon staff 4.6M Oct 5 17:02 bin/Release/lib/xbuild/Xamarin/Android/lib/arm64-v8a/libmonosgen-2.0.so The `/Library/Frameworks/...` is the Xamarin.Android 7.0 file size; the `bin/Release/...` is the size when BTLS is included. BTLS inclusion into `libmonosgen-2.0.so` increases the file size by 1.8 MB. **1.8 MB** That's a ~64% increase in file size, for *all* apps. [mono/085b653a][0] splits out the BTLS code into a new, separate, `libmono-btls-shared.so` native library. This allows BTLS inclusion to be *optional*, so that the size increase is only paid when it's used. Update `build-tools/mono-runtimes` to use `configure --enable-dynamic-btls` for the Android and host builds so that BTLS is placed into the new `libmono-btls-shared.so` library and installed into `bin/$(Configuration)/lib/xbuild/Xamarin/Android/lib/ABI/libmono-btls-shared.so`. (The host needs `--enable-dynamic-btls` so that the monodroid-profile `System.dll` properly P/Invokes `libmono-btls-shared.so`, instead of P/Invoking `__Internal`.) Update the `<BuildApk/>` task so that when `$(AndroidTlsProvider)` has the value `btls`, `libmono-btls-shared.so` is embedded into the `app.apk`. [0]: https://github.com/mono/mono/commit/085b653a7425e30516150d2335c39a6906e6cd4d
2016-10-25 17:58:29 +03:00
<Copy
SourceFiles="@(_MonoBtlsSource)"
DestinationFiles="@(_InstallMonoBtlsOutput)"
/>
<Copy
SourceFiles="@(_MonoBtlsSource)"
DestinationFiles="@(_InstallUnstrippedMonoBtlsOutput)"
/>
<Exec
[mono-runtimes] Check for correct filename before `strip`ing Fix a [Build][0] [Break][1]: Executing: ".../mxe/bin/x86_64-w64-mingw32.static-strip" "../host-mxe-Win64/.dll" .../x86_64-w64-mingw32.static-strip: '.../host-mxe-Win64/.dll': No such file This was a typeo introduced in commit a5b324d1 -- among others :-(: <Exec Condition=" '$(Configuration)' != 'Debug' And '%(_MonoRuntime.DoBuild)' == 'true' And '%(_MonoRuntime.OutputMonoPosixHelperFilename)' != '' " Command="&quot;%(_MonoRuntime.Strip)&quot; %(_MonoRuntime.StripFlags) &quot;$(OutputPath)\lib\xbuild\Xamarin\Android\lib\%(_MonoRuntime.Identity)\%(_MonoRuntime.OutputMonoBtlsFilename).%(_MonoRuntime.NativeLibraryExtension)&quot;" /> One of these values is not like the other, and that's the problem. `Condition` checks that `%(_MonoRuntime.OutputMonoPosixHelperFilename)` isn't empty, but the file we **strip** is `%(_MonoRuntime.OutputMonoBtlsFilename)`. When building for `mxe-Win64`, `%(OutputMonoPosixHelperFilename)` is set, but `%(OutputMonoBtlsFilename)` isn't, so we interpolate an empty string, resulting in a filename which doesn't exist. Fix the error by updating the `Condition` to check that `%(OutputMonoBtlsFilename)` isn't empty. [0]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android/140/ [1]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android/140/consoleText
2016-11-10 00:19:21 +03:00
Condition=" '$(Configuration)' != 'Debug' And '%(_MonoRuntime.DoBuild)' == 'true' And '%(_MonoRuntime.OutputMonoBtlsFilename)' != '' "
[structure] Rework installation directory structure (#704) Context: https://github.com/xamarin/xamarin-android/pull/253#discussion_r82862993 The *intention* is that Jenkins-produced `oss-xamarin.android*.zip` artifacts be usable on Windows, so that side-by-side testing can be performed without replacing the system installation. Usage is in [UsingJenkinsBuildArtifacts.md](Documentation/UsingJenkinsBuildArtifacts). This isn't *entirely* the case. It was *apparently* the case at the time of commit 87ca2737, but things have bitrotten since. For example, following the 87ca2737 instructions would currently result in an XA0020 `Could not find monodroid` error, because `class-parse.exe` wasn't where Windows expects it to be (it was in `lib/mandroid`, not `lib/xbuild/Xamarin/Android`). This needs to be fixed. Additionally, PR #253 mentions that, for filesystem organization, it would be useful if the macOS/Linux directory structure -- `$prefix/bin`, `$prefix/lib/mandroid`, `$prefix/lib/xbuild/Xamarin/Android`, `$prefix/lib/xbuild-frameworks` -- more closely resembled the Windows directory structure of `$MSBuild` and `$ReferenceAssemblies` (as seen in `.vsix` files). This would turn macOS/Linux into using `$xa_prefix/xbuild` and `$xa_prefix/xbuild-frameworks` directories. `$prefix/bin` would only contain `xabuild`. What is currently in `$prefix/lib/mandroid` would be merged with `$xa_prefix/xbuild/Xamarin/Android`. `$xa_prefix` would `$prefix/lib/xamarin.android`. This would turn the current macOS structure: $prefix/bin/xabuild $prefix/bin/generator $prefix/bin/cross-arm $prefix/lib/mandroid/generator.exe $prefix/lib/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xbuild/Xamarin/Android/Xamarin.Android.Common.targets Into: $prefix/bin/xabuild $prefix/lib/xamarin.android/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xamarin.android/xbuild/Xamarin/Android/generator.exe $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Xamarin.Android.Common.targets $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Darwin/cross-arm Other notes: * The `bundle-*.zip` filename has been has been changed to include a *hash* of the contents of various files, in particular `build-tools\mono-runtimes.*`. This was instigated via a conversation with @kumpera about making the filename more "idiot-proof": `mono-runtimes.props` contains *compiler flags*, and if any of those change, then *logically* the bundle should differ as well, as the mono runtimes may differ in significant ways. In theory, the `-vXX` version should be used to track this, but this is a manual change, easy to overlook. The new `-hHASH` part of the filename should be more automagic. The new `<HashFileContents/>` task in `xa-prep-tasks.dll` is responsible for creating the hash value. * `Configuration.Java.Interop.Override.props` was moved into `build-tools/scripts`, because that would cleanup the root directory a bit. * OS-specific binaries are now placed into `$prefix/lib/xamarin.android/xbuild/Xamarin/Android/$(HostOS)`. On a theoretical plus side, this means that the same build directory can contain OS-specific binaries for multiple operating systems. (I don't know if anyone shares a build tree between e.g. macOS and Linux, but if anyone *does*...) Unfortunately this requires a workaround for an `xbuild` bug: `%(_LlvmRuntime.InstallPath)` and `%(_MonoCrossRuntime.InstallPath)` *cannot* use MSBuild properties, e.g. `<InstallPath>$(HostOS)/</InstallPath>` doesn't work as desired; it's instead treated literally. Special-case `%(InstallPath)` until we fully migrate to MSBuild. * `$(MonoAndroidToolsDirectory)` should be considered *dead*, along with `$(MonoAndroidBinDirectory)`, as these should now *always* be the same directory as where `Xamarin.Android.Build.Tasks.dll` is located, or a `$(HostOS)` sub-directory. * `Xamarin.ProjectTools.csproj` needed to be updated to ensure that the build order was correct. * Remove all `[Obsolete]` and unreferenced members from `Xamarin.Android.Build.Utilities.dll`. There's too much in there, and it makes my head hurt trying to understand the interrelationships between it all. If it's not used, it's gone. * The changes to `src/monodroid/jni/Android.mk` are...weird. The removal of `-I$(topdir)/libmonodroid/zip`/etc. is to reduce use of non-existent paths, as `$(topdir)` isn't defined, so that's *actually* `-I/libmonodroid/zip`, which is nonsensical. So far, so good. What's *odd* is the addition of `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`: This is needed so that `external/mono/support/zlib-helper.c` exports `CreateZStream` and related symbols, otherwise we get a unit test failure in `GzipStreamTest.Compression` due to an `EntryPointNotFoundException`, because `CreateZStream` isn't exported/public. What's odd here is that I don't understand what caused this behavior to change. Previous builds exported `CreateZStream`, otherwise the tests would fail, and I don't understand how any of the other changes in this PR would be at fault, though that's certainly the most plausible explanation. Regardless, `-Ijni` *first* (due to adding `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`) is the desired behavior, so that `jni/config.h` is included, thus ensuring that `MONO_API` has the required definition when building `zlib-helper.c`.
2017-07-28 16:36:32 +03:00
Command="&quot;%(_MonoRuntime.Strip)&quot; %(_MonoRuntime.StripFlags) &quot;$(_MSBuildDir)\lib\%(_MonoRuntime.Identity)\%(_MonoRuntime.OutputMonoBtlsFilename).%(_MonoRuntime.NativeLibraryExtension)&quot;"
Bump to mono/mono-4.8.0-branch/9f4abcc3 (#278) * Bump to mono/mono-4.8.0-branch/9f4abcc3 [mono/085b653a][0] splits out the BTLS code into a new, separate, `libmono-btls-shared.so` native library. This allows BTLS inclusion to be *optional*, so that the size increase is only paid when it's used. Bump to mono/9f4abcc3 so that we can use this new feature. [0]: https://github.com/mono/mono/commit/085b653a7425e30516150d2335c39a6906e6cd4d * [mono-runtimes, Xamarin.Android.Build.Tasks] Use separate BTLS lib There was an unintended side effect of commit cebc6c83, which added `--with-btls-android-ndk` to the mono runtime configure command: BTLS was compiled into `libmonosgen-2.0.so`, greatly increasing size: $ ls -lh bin/Release/lib/xbuild/Xamarin/Android/lib/arm64-v8a/libmonosgen-2.0.so /Library/Frameworks/Xamarin.Android.framework/Libraries/xbuild/Xamarin/Android/lib/arm64-v8a/libmonosgen-2.0.so -rwxr-xr-x 1 root wheel 2.8M Sep 23 18:34 /Library/Frameworks/Xamarin.Android.framework/Libraries/xbuild/Xamarin/Android/lib/arm64-v8a/libmonosgen-2.0.so -rwxr-xr-x 1 jon staff 4.6M Oct 5 17:02 bin/Release/lib/xbuild/Xamarin/Android/lib/arm64-v8a/libmonosgen-2.0.so The `/Library/Frameworks/...` is the Xamarin.Android 7.0 file size; the `bin/Release/...` is the size when BTLS is included. BTLS inclusion into `libmonosgen-2.0.so` increases the file size by 1.8 MB. **1.8 MB** That's a ~64% increase in file size, for *all* apps. [mono/085b653a][0] splits out the BTLS code into a new, separate, `libmono-btls-shared.so` native library. This allows BTLS inclusion to be *optional*, so that the size increase is only paid when it's used. Update `build-tools/mono-runtimes` to use `configure --enable-dynamic-btls` for the Android and host builds so that BTLS is placed into the new `libmono-btls-shared.so` library and installed into `bin/$(Configuration)/lib/xbuild/Xamarin/Android/lib/ABI/libmono-btls-shared.so`. (The host needs `--enable-dynamic-btls` so that the monodroid-profile `System.dll` properly P/Invokes `libmono-btls-shared.so`, instead of P/Invoking `__Internal`.) Update the `<BuildApk/>` task so that when `$(AndroidTlsProvider)` has the value `btls`, `libmono-btls-shared.so` is embedded into the `app.apk`. [0]: https://github.com/mono/mono/commit/085b653a7425e30516150d2335c39a6906e6cd4d
2016-10-25 17:58:29 +03:00
/>
[mono-runtimes] Fix @(BundleItem) timestamps (#307) The *intent* of the `_BuildUnlessCached` target within `mono-runtimes.targets` is to (attempt to!) make building mono "free", or at least, free *enough* so that other project files can have a build-time dependency on `mono-runtimes.mdproj` so that build artifacts can be (reasonably) consistent. Unfortunately, that wasn't the case, with the net result being that: tools/scripts/xabuild /t:SignAndroidPackage src/Mono.Android/Test/*.csproj would *constantly* attempt to re-build `build-tools/mono-runtimes.mdproj`, adding (at least!) one minute as we ran mono's `make`...to do nothing. Why? Target _BuildUnlessCached needs to be built as input file '.../xamarin-android/external/mono/autogen.sh' is newer than output file '../../bin/Debug//include/armeabi-v7a/eglib/config.h' The "problem" is that the `<Copy/>` task copies over the timestamp as well, so even though we're (possibly) creating a new file, the timestamp on the created file will match the source file. Normally this is fine. In this case, we're dealing with `eglib`, which is a "nested" ("delegated"?) `configure` invocation from mono's `configure`. It *appears* that if (when) mono is updated, "normal" mono subdirectories will "properly" honor `--enable-maintainer-mode` and re-run mono's `configure`, which will in turn update the timstamp on any `configure`-time generated files. This doesn't appear to be the case for `mono/eglib`; when `mono` is updated, `mono/eglib` does *not* re-run `configure`, and thus no timestamps are updated to follow suit. Fix this by *explicitly* `<Touch/>`ing all copied files so that we ensure they have a "recent" timestamp ("now"), which should ensure that the file's timestamp is newer than that of e.g. `autogen.sh`. Additionally, there was a typo: Commit a5b324d1 used a `@(_InstallRuntimesOutput)` item group, but there is no such item group; it's actually `@(_InstallRuntimeOutput)` (no `s`). Fixity fix this. Finally, commit a5b324d1 forgot to add `libmono-btls-shared` to `@(BundleItem)`, preventing it from being added to the bundle. Fix this oversight.
2016-11-22 14:11:45 +03:00
<Touch Files="@(_InstallMonoBtlsOutput);@(_InstallUnstrippedMonoBtlsOutput)" />
Bump to mono/mono-4.8.0-branch/9f4abcc3 (#278) * Bump to mono/mono-4.8.0-branch/9f4abcc3 [mono/085b653a][0] splits out the BTLS code into a new, separate, `libmono-btls-shared.so` native library. This allows BTLS inclusion to be *optional*, so that the size increase is only paid when it's used. Bump to mono/9f4abcc3 so that we can use this new feature. [0]: https://github.com/mono/mono/commit/085b653a7425e30516150d2335c39a6906e6cd4d * [mono-runtimes, Xamarin.Android.Build.Tasks] Use separate BTLS lib There was an unintended side effect of commit cebc6c83, which added `--with-btls-android-ndk` to the mono runtime configure command: BTLS was compiled into `libmonosgen-2.0.so`, greatly increasing size: $ ls -lh bin/Release/lib/xbuild/Xamarin/Android/lib/arm64-v8a/libmonosgen-2.0.so /Library/Frameworks/Xamarin.Android.framework/Libraries/xbuild/Xamarin/Android/lib/arm64-v8a/libmonosgen-2.0.so -rwxr-xr-x 1 root wheel 2.8M Sep 23 18:34 /Library/Frameworks/Xamarin.Android.framework/Libraries/xbuild/Xamarin/Android/lib/arm64-v8a/libmonosgen-2.0.so -rwxr-xr-x 1 jon staff 4.6M Oct 5 17:02 bin/Release/lib/xbuild/Xamarin/Android/lib/arm64-v8a/libmonosgen-2.0.so The `/Library/Frameworks/...` is the Xamarin.Android 7.0 file size; the `bin/Release/...` is the size when BTLS is included. BTLS inclusion into `libmonosgen-2.0.so` increases the file size by 1.8 MB. **1.8 MB** That's a ~64% increase in file size, for *all* apps. [mono/085b653a][0] splits out the BTLS code into a new, separate, `libmono-btls-shared.so` native library. This allows BTLS inclusion to be *optional*, so that the size increase is only paid when it's used. Update `build-tools/mono-runtimes` to use `configure --enable-dynamic-btls` for the Android and host builds so that BTLS is placed into the new `libmono-btls-shared.so` library and installed into `bin/$(Configuration)/lib/xbuild/Xamarin/Android/lib/ABI/libmono-btls-shared.so`. (The host needs `--enable-dynamic-btls` so that the monodroid-profile `System.dll` properly P/Invokes `libmono-btls-shared.so`, instead of P/Invoking `__Internal`.) Update the `<BuildApk/>` task so that when `$(AndroidTlsProvider)` has the value `btls`, `libmono-btls-shared.so` is embedded into the `app.apk`. [0]: https://github.com/mono/mono/commit/085b653a7425e30516150d2335c39a6906e6cd4d
2016-10-25 17:58:29 +03:00
[build] Build the monodroid-profile BCL assemblies reliably (#135) Commit 37b5fd51 broke `make jenkins` (and possible `make all`?) through the introduction of the `%(_MonoRuntime.DoBuild)` metadata: `%(_MonoRuntime.DoBuild)` wasn't added everywhere required. In particular, the `host-Darwin` item didn't gain `<DoBuild/>`, meaning *`host-Darwin` wasn't built*. This in turn meant that the monodroid BCL wasn't built (!) which caused a build break much later within `src/Mono.Android`: Executing: mono .../xamarin-android/external/Java.Interop/bin/Debug/jcw-gen.exe -o ".../xamarin-android/src/Mono.Android/obj/Debug/android-23/jcw/src" -L ".../xamarin-androidg/lib/xbuild-frameworks/MonoAndroid/v6.0/" -L ".../xamarin-android/src/Mono.Android/../../bin/Debug/lib/xbuild-frameworks/MonoAndroid/v6.0/../v1.0/" -L ".../xamarin-android/src/Mono.Android/../../MonoAndroid/v6.0/../v1.0/Facades" ".../xamarin-android/src/Mono.Android/../../bin/Debug/lib/xbuild-frameworks/MonoAndroid/v6.0/Mono.Android.dll" jcw-gen: Could not load assembly 'mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089'. Perhaps it doesn't exist in the Mono for Android profile? `mscorlib.dll` couldn't be found because it wasn't *built*, nor installed into `bin/Debug/lib/xbuild-frameworks/MonoAndroid/v1.0`. Additionally, fully building the AOT cross compilers requires adding `mxe-Win32` to `$(ALL_HOST_ABIS)`, otherwise e.g. `win-armeabi` fails to build because `arm_dpimacros.h` doesn't exist (as `arm_dpimacros.h` is built by `mxe-Win32`): Error parsing '.../xamarin-android/external/mono/mono/metadata/metadata-cross-helpers.c' .../xamarin-android/external/mono/mono/arch/arm/arm-codegen.h(937,10): fatal: 'mono/arch/arm/arm_dpimacros.h' file not found Error parsing '.../xamarin-android/external/mono/mono/mini/mini-cross-helpers.c' .../xamarin-android/external/mono/mono/arch/arm/arm-codegen.h(937,10): fatal: 'mono/arch/arm/arm_dpimacros.h' file not found Unhandled Exception: System.Exception: Expected to find struct definition for MonoObject Fix the `mono-runtimes.mdproj` build so that it doesn't re-execute targets when not necessary, by "rolling" with xbuild limitations and refactoring item groups as appropriate. In particular, xbuild doesn't like referencing item metadata within a `//Target/@Outputs` that in turn references a property group: <PropertyGroup> <V>Value</v> </PropertyGroup> <ItemGroup> <Item Include="Something"> <Metadata>$(V)</Metadata> </Item> </ItemGroup> <Target Name="Example" Inputs="..." Outputs="%(Item.Metadata)"> ... </Target> Whenever the `Example` target is run, it will *always* run, even if `Value` exists, because `xbuild` perceives the `//Target[@Name='Example']/@Outputs` value as being `$(V)`, *not* `Value`, which is wonderfully broken. To compensate, Don't Do That™; in particular, item metadata used within `//Target/@Outputs` *must* be hardcoded and not contain a property reference. This "hardcoding" removes the need for the `$(_Cross*ffsetsHeaderFile)` properties, so remove them. Remove `System.Net.Http.WebRequest.dll` from `@(_BclAssembly)`, as it is no longer built in the monodroid profile; requiring it as an output means that the "host" mono build is *always* built, which isn't needed. Improve the `_BuildRuntimes` and `_InstallRuntimes` targets so that their referenced files aren't *so* repetitive, in particular by "splitting out" `@(_RuntimeLibraries)` into `@(_RuntimeSource)`, `@(_ProfilerSource)`, and `@(_MonoPosixHelperSource)`. This allows more intelligently `strip`ing the native libraries in the Release configuration, while also maintaining generation of the `.d.so` files.
2016-07-28 13:24:18 +03:00
<Copy
SourceFiles="@(_MonoPosixHelperSource)"
DestinationFiles="@(_InstallMonoPosixHelperOutput)"
/>
<Copy
SourceFiles="@(_MonoPosixHelperSource)"
DestinationFiles="@(_InstallUnstrippedMonoPosixHelperOutput)"
/>
<Exec
Condition=" '$(Configuration)' != 'Debug' And '%(_MonoRuntime.DoBuild)' == 'true' And '%(_MonoRuntime.OutputMonoPosixHelperFilename)' != '' "
[structure] Rework installation directory structure (#704) Context: https://github.com/xamarin/xamarin-android/pull/253#discussion_r82862993 The *intention* is that Jenkins-produced `oss-xamarin.android*.zip` artifacts be usable on Windows, so that side-by-side testing can be performed without replacing the system installation. Usage is in [UsingJenkinsBuildArtifacts.md](Documentation/UsingJenkinsBuildArtifacts). This isn't *entirely* the case. It was *apparently* the case at the time of commit 87ca2737, but things have bitrotten since. For example, following the 87ca2737 instructions would currently result in an XA0020 `Could not find monodroid` error, because `class-parse.exe` wasn't where Windows expects it to be (it was in `lib/mandroid`, not `lib/xbuild/Xamarin/Android`). This needs to be fixed. Additionally, PR #253 mentions that, for filesystem organization, it would be useful if the macOS/Linux directory structure -- `$prefix/bin`, `$prefix/lib/mandroid`, `$prefix/lib/xbuild/Xamarin/Android`, `$prefix/lib/xbuild-frameworks` -- more closely resembled the Windows directory structure of `$MSBuild` and `$ReferenceAssemblies` (as seen in `.vsix` files). This would turn macOS/Linux into using `$xa_prefix/xbuild` and `$xa_prefix/xbuild-frameworks` directories. `$prefix/bin` would only contain `xabuild`. What is currently in `$prefix/lib/mandroid` would be merged with `$xa_prefix/xbuild/Xamarin/Android`. `$xa_prefix` would `$prefix/lib/xamarin.android`. This would turn the current macOS structure: $prefix/bin/xabuild $prefix/bin/generator $prefix/bin/cross-arm $prefix/lib/mandroid/generator.exe $prefix/lib/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xbuild/Xamarin/Android/Xamarin.Android.Common.targets Into: $prefix/bin/xabuild $prefix/lib/xamarin.android/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xamarin.android/xbuild/Xamarin/Android/generator.exe $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Xamarin.Android.Common.targets $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Darwin/cross-arm Other notes: * The `bundle-*.zip` filename has been has been changed to include a *hash* of the contents of various files, in particular `build-tools\mono-runtimes.*`. This was instigated via a conversation with @kumpera about making the filename more "idiot-proof": `mono-runtimes.props` contains *compiler flags*, and if any of those change, then *logically* the bundle should differ as well, as the mono runtimes may differ in significant ways. In theory, the `-vXX` version should be used to track this, but this is a manual change, easy to overlook. The new `-hHASH` part of the filename should be more automagic. The new `<HashFileContents/>` task in `xa-prep-tasks.dll` is responsible for creating the hash value. * `Configuration.Java.Interop.Override.props` was moved into `build-tools/scripts`, because that would cleanup the root directory a bit. * OS-specific binaries are now placed into `$prefix/lib/xamarin.android/xbuild/Xamarin/Android/$(HostOS)`. On a theoretical plus side, this means that the same build directory can contain OS-specific binaries for multiple operating systems. (I don't know if anyone shares a build tree between e.g. macOS and Linux, but if anyone *does*...) Unfortunately this requires a workaround for an `xbuild` bug: `%(_LlvmRuntime.InstallPath)` and `%(_MonoCrossRuntime.InstallPath)` *cannot* use MSBuild properties, e.g. `<InstallPath>$(HostOS)/</InstallPath>` doesn't work as desired; it's instead treated literally. Special-case `%(InstallPath)` until we fully migrate to MSBuild. * `$(MonoAndroidToolsDirectory)` should be considered *dead*, along with `$(MonoAndroidBinDirectory)`, as these should now *always* be the same directory as where `Xamarin.Android.Build.Tasks.dll` is located, or a `$(HostOS)` sub-directory. * `Xamarin.ProjectTools.csproj` needed to be updated to ensure that the build order was correct. * Remove all `[Obsolete]` and unreferenced members from `Xamarin.Android.Build.Utilities.dll`. There's too much in there, and it makes my head hurt trying to understand the interrelationships between it all. If it's not used, it's gone. * The changes to `src/monodroid/jni/Android.mk` are...weird. The removal of `-I$(topdir)/libmonodroid/zip`/etc. is to reduce use of non-existent paths, as `$(topdir)` isn't defined, so that's *actually* `-I/libmonodroid/zip`, which is nonsensical. So far, so good. What's *odd* is the addition of `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`: This is needed so that `external/mono/support/zlib-helper.c` exports `CreateZStream` and related symbols, otherwise we get a unit test failure in `GzipStreamTest.Compression` due to an `EntryPointNotFoundException`, because `CreateZStream` isn't exported/public. What's odd here is that I don't understand what caused this behavior to change. Previous builds exported `CreateZStream`, otherwise the tests would fail, and I don't understand how any of the other changes in this PR would be at fault, though that's certainly the most plausible explanation. Regardless, `-Ijni` *first* (due to adding `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`) is the desired behavior, so that `jni/config.h` is included, thus ensuring that `MONO_API` has the required definition when building `zlib-helper.c`.
2017-07-28 16:36:32 +03:00
Command="&quot;%(_MonoRuntime.Strip)&quot; %(_MonoRuntime.StripFlags) &quot;$(_MSBuildDir)\lib\%(_MonoRuntime.Identity)\%(_MonoRuntime.OutputMonoPosixHelperFilename).%(_MonoRuntime.NativeLibraryExtension)&quot;"
[build] Build the monodroid-profile BCL assemblies reliably (#135) Commit 37b5fd51 broke `make jenkins` (and possible `make all`?) through the introduction of the `%(_MonoRuntime.DoBuild)` metadata: `%(_MonoRuntime.DoBuild)` wasn't added everywhere required. In particular, the `host-Darwin` item didn't gain `<DoBuild/>`, meaning *`host-Darwin` wasn't built*. This in turn meant that the monodroid BCL wasn't built (!) which caused a build break much later within `src/Mono.Android`: Executing: mono .../xamarin-android/external/Java.Interop/bin/Debug/jcw-gen.exe -o ".../xamarin-android/src/Mono.Android/obj/Debug/android-23/jcw/src" -L ".../xamarin-androidg/lib/xbuild-frameworks/MonoAndroid/v6.0/" -L ".../xamarin-android/src/Mono.Android/../../bin/Debug/lib/xbuild-frameworks/MonoAndroid/v6.0/../v1.0/" -L ".../xamarin-android/src/Mono.Android/../../MonoAndroid/v6.0/../v1.0/Facades" ".../xamarin-android/src/Mono.Android/../../bin/Debug/lib/xbuild-frameworks/MonoAndroid/v6.0/Mono.Android.dll" jcw-gen: Could not load assembly 'mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089'. Perhaps it doesn't exist in the Mono for Android profile? `mscorlib.dll` couldn't be found because it wasn't *built*, nor installed into `bin/Debug/lib/xbuild-frameworks/MonoAndroid/v1.0`. Additionally, fully building the AOT cross compilers requires adding `mxe-Win32` to `$(ALL_HOST_ABIS)`, otherwise e.g. `win-armeabi` fails to build because `arm_dpimacros.h` doesn't exist (as `arm_dpimacros.h` is built by `mxe-Win32`): Error parsing '.../xamarin-android/external/mono/mono/metadata/metadata-cross-helpers.c' .../xamarin-android/external/mono/mono/arch/arm/arm-codegen.h(937,10): fatal: 'mono/arch/arm/arm_dpimacros.h' file not found Error parsing '.../xamarin-android/external/mono/mono/mini/mini-cross-helpers.c' .../xamarin-android/external/mono/mono/arch/arm/arm-codegen.h(937,10): fatal: 'mono/arch/arm/arm_dpimacros.h' file not found Unhandled Exception: System.Exception: Expected to find struct definition for MonoObject Fix the `mono-runtimes.mdproj` build so that it doesn't re-execute targets when not necessary, by "rolling" with xbuild limitations and refactoring item groups as appropriate. In particular, xbuild doesn't like referencing item metadata within a `//Target/@Outputs` that in turn references a property group: <PropertyGroup> <V>Value</v> </PropertyGroup> <ItemGroup> <Item Include="Something"> <Metadata>$(V)</Metadata> </Item> </ItemGroup> <Target Name="Example" Inputs="..." Outputs="%(Item.Metadata)"> ... </Target> Whenever the `Example` target is run, it will *always* run, even if `Value` exists, because `xbuild` perceives the `//Target[@Name='Example']/@Outputs` value as being `$(V)`, *not* `Value`, which is wonderfully broken. To compensate, Don't Do That™; in particular, item metadata used within `//Target/@Outputs` *must* be hardcoded and not contain a property reference. This "hardcoding" removes the need for the `$(_Cross*ffsetsHeaderFile)` properties, so remove them. Remove `System.Net.Http.WebRequest.dll` from `@(_BclAssembly)`, as it is no longer built in the monodroid profile; requiring it as an output means that the "host" mono build is *always* built, which isn't needed. Improve the `_BuildRuntimes` and `_InstallRuntimes` targets so that their referenced files aren't *so* repetitive, in particular by "splitting out" `@(_RuntimeLibraries)` into `@(_RuntimeSource)`, `@(_ProfilerSource)`, and `@(_MonoPosixHelperSource)`. This allows more intelligently `strip`ing the native libraries in the Release configuration, while also maintaining generation of the `.d.so` files.
2016-07-28 13:24:18 +03:00
/>
[mono-runtimes] Fix @(BundleItem) timestamps (#307) The *intent* of the `_BuildUnlessCached` target within `mono-runtimes.targets` is to (attempt to!) make building mono "free", or at least, free *enough* so that other project files can have a build-time dependency on `mono-runtimes.mdproj` so that build artifacts can be (reasonably) consistent. Unfortunately, that wasn't the case, with the net result being that: tools/scripts/xabuild /t:SignAndroidPackage src/Mono.Android/Test/*.csproj would *constantly* attempt to re-build `build-tools/mono-runtimes.mdproj`, adding (at least!) one minute as we ran mono's `make`...to do nothing. Why? Target _BuildUnlessCached needs to be built as input file '.../xamarin-android/external/mono/autogen.sh' is newer than output file '../../bin/Debug//include/armeabi-v7a/eglib/config.h' The "problem" is that the `<Copy/>` task copies over the timestamp as well, so even though we're (possibly) creating a new file, the timestamp on the created file will match the source file. Normally this is fine. In this case, we're dealing with `eglib`, which is a "nested" ("delegated"?) `configure` invocation from mono's `configure`. It *appears* that if (when) mono is updated, "normal" mono subdirectories will "properly" honor `--enable-maintainer-mode` and re-run mono's `configure`, which will in turn update the timstamp on any `configure`-time generated files. This doesn't appear to be the case for `mono/eglib`; when `mono` is updated, `mono/eglib` does *not* re-run `configure`, and thus no timestamps are updated to follow suit. Fix this by *explicitly* `<Touch/>`ing all copied files so that we ensure they have a "recent" timestamp ("now"), which should ensure that the file's timestamp is newer than that of e.g. `autogen.sh`. Additionally, there was a typo: Commit a5b324d1 used a `@(_InstallRuntimesOutput)` item group, but there is no such item group; it's actually `@(_InstallRuntimeOutput)` (no `s`). Fixity fix this. Finally, commit a5b324d1 forgot to add `libmono-btls-shared` to `@(BundleItem)`, preventing it from being added to the bundle. Fix this oversight.
2016-11-22 14:11:45 +03:00
<Touch Files="@(_InstallMonoPosixHelperOutput);@(_InstallUnstrippedMonoPosixHelperOutput)" />
[mono-runtimes] Build required mono runtimes. Add a new build-tools/mono-runtimes project to build the required Mono runtimes for Xamarin.Android app execution on Android devices. Xamarin.Android 6.0 ("cycle 6") includes Mono runtimes compiled for five architectures: * armeabi (armv5) * armeabi-v7a (armv7) * arm64-v8a (aarch64) * x86 * x86_64 An additional "host" architecture is also built to build the "monodroid" profile assemblies such as mscorlib.dll. In the interest of expediency and not requiring that *six* different runtimes be built -- which can greatly increase build times and storage requirements -- the initial xamarin-android open-source build will only build the armeabi-v7a (armv7) and "host" architectures. Additionally, the <UnzipDirectoryChildren/> task needed to be altered to use unzip(1) on OS X instead of using System.IO.Compression.ZipFile, because the ZipFile implementation included with Mono 4.4 doesn't respect file attributes such as the +x, causing all files to be extracted as 644 (rw-r--r--), resulting in an NDK toolchain which wouldn't work. To resolve this issue, execution on OS X and Linux now uses unzip(1). ~~ Adding new architectures ~~ To build Mono for a new architecture, edit build-tools/mono-runtimes/mono-runtimes.projitems and add a new @(_MonoRuntime) entry for the desired architecture. TODO: Figure out if there's a reasonable way to "opt-in" or "opt-out" of CPU architectures so that it isn't an "all or nothing" prospect.
2016-04-20 04:30:00 +03:00
</Target>
[bundle] Include mono's debug symbols (#513) Microsoft has (several?) [symbol servers][0], which is a repository "somewhere" which contains debug symbols, so that when a stack trace containing e.g. BCL stack frames is sent to Microsoft, there is a way to correlate that back to filename and line number source for further investigation. Xamarin.Android has not partaken in this mechanism, in large part because we did most of our building on Linux and macOS machines which generate `.mdb` debug symbols, while symbol servers require `.pdb`. That constraint is now behind us; with 76be1fb1 and the use of Mono 4.9 for builds, our builds now generate Portable `.pdb` files, which *should* (hopefully) be usable on Microsoft's symbol servers. However, to transmit them to Microsoft's servers, we need to actually *retain* them. At present, they're not: because of `bundle*.zip` use, the `mscorlib.dll` that we include in the final product may have been built much earlier than the rest of the product, and we don't have any other mechanism to retain the `mscorlib.pdb` file. Let's change that: update `mono-runtimes.targets` so that the created debug symbol files are included in the produced `bundle*.zip`, and bump the corresponding `$(XABundleFileName)` version number. This will ensure that we retain the debug symbols for our BCL assemblies, allowing us to actually have something to upload. [0]: https://msdn.microsoft.com/en-us/library/cc667410.aspx
2017-03-22 22:51:57 +03:00
<ItemGroup>
<_MonoCilStripSource Include="$(_MonoOutputDir)\mono-cil-strip.exe" />
[structure] Rework installation directory structure (#704) Context: https://github.com/xamarin/xamarin-android/pull/253#discussion_r82862993 The *intention* is that Jenkins-produced `oss-xamarin.android*.zip` artifacts be usable on Windows, so that side-by-side testing can be performed without replacing the system installation. Usage is in [UsingJenkinsBuildArtifacts.md](Documentation/UsingJenkinsBuildArtifacts). This isn't *entirely* the case. It was *apparently* the case at the time of commit 87ca2737, but things have bitrotten since. For example, following the 87ca2737 instructions would currently result in an XA0020 `Could not find monodroid` error, because `class-parse.exe` wasn't where Windows expects it to be (it was in `lib/mandroid`, not `lib/xbuild/Xamarin/Android`). This needs to be fixed. Additionally, PR #253 mentions that, for filesystem organization, it would be useful if the macOS/Linux directory structure -- `$prefix/bin`, `$prefix/lib/mandroid`, `$prefix/lib/xbuild/Xamarin/Android`, `$prefix/lib/xbuild-frameworks` -- more closely resembled the Windows directory structure of `$MSBuild` and `$ReferenceAssemblies` (as seen in `.vsix` files). This would turn macOS/Linux into using `$xa_prefix/xbuild` and `$xa_prefix/xbuild-frameworks` directories. `$prefix/bin` would only contain `xabuild`. What is currently in `$prefix/lib/mandroid` would be merged with `$xa_prefix/xbuild/Xamarin/Android`. `$xa_prefix` would `$prefix/lib/xamarin.android`. This would turn the current macOS structure: $prefix/bin/xabuild $prefix/bin/generator $prefix/bin/cross-arm $prefix/lib/mandroid/generator.exe $prefix/lib/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xbuild/Xamarin/Android/Xamarin.Android.Common.targets Into: $prefix/bin/xabuild $prefix/lib/xamarin.android/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xamarin.android/xbuild/Xamarin/Android/generator.exe $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Xamarin.Android.Common.targets $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Darwin/cross-arm Other notes: * The `bundle-*.zip` filename has been has been changed to include a *hash* of the contents of various files, in particular `build-tools\mono-runtimes.*`. This was instigated via a conversation with @kumpera about making the filename more "idiot-proof": `mono-runtimes.props` contains *compiler flags*, and if any of those change, then *logically* the bundle should differ as well, as the mono runtimes may differ in significant ways. In theory, the `-vXX` version should be used to track this, but this is a manual change, easy to overlook. The new `-hHASH` part of the filename should be more automagic. The new `<HashFileContents/>` task in `xa-prep-tasks.dll` is responsible for creating the hash value. * `Configuration.Java.Interop.Override.props` was moved into `build-tools/scripts`, because that would cleanup the root directory a bit. * OS-specific binaries are now placed into `$prefix/lib/xamarin.android/xbuild/Xamarin/Android/$(HostOS)`. On a theoretical plus side, this means that the same build directory can contain OS-specific binaries for multiple operating systems. (I don't know if anyone shares a build tree between e.g. macOS and Linux, but if anyone *does*...) Unfortunately this requires a workaround for an `xbuild` bug: `%(_LlvmRuntime.InstallPath)` and `%(_MonoCrossRuntime.InstallPath)` *cannot* use MSBuild properties, e.g. `<InstallPath>$(HostOS)/</InstallPath>` doesn't work as desired; it's instead treated literally. Special-case `%(InstallPath)` until we fully migrate to MSBuild. * `$(MonoAndroidToolsDirectory)` should be considered *dead*, along with `$(MonoAndroidBinDirectory)`, as these should now *always* be the same directory as where `Xamarin.Android.Build.Tasks.dll` is located, or a `$(HostOS)` sub-directory. * `Xamarin.ProjectTools.csproj` needed to be updated to ensure that the build order was correct. * Remove all `[Obsolete]` and unreferenced members from `Xamarin.Android.Build.Utilities.dll`. There's too much in there, and it makes my head hurt trying to understand the interrelationships between it all. If it's not used, it's gone. * The changes to `src/monodroid/jni/Android.mk` are...weird. The removal of `-I$(topdir)/libmonodroid/zip`/etc. is to reduce use of non-existent paths, as `$(topdir)` isn't defined, so that's *actually* `-I/libmonodroid/zip`, which is nonsensical. So far, so good. What's *odd* is the addition of `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`: This is needed so that `external/mono/support/zlib-helper.c` exports `CreateZStream` and related symbols, otherwise we get a unit test failure in `GzipStreamTest.Compression` due to an `EntryPointNotFoundException`, because `CreateZStream` isn't exported/public. What's odd here is that I don't understand what caused this behavior to change. Previous builds exported `CreateZStream`, otherwise the tests would fail, and I don't understand how any of the other changes in this PR would be at fault, though that's certainly the most plausible explanation. Regardless, `-Ijni` *first* (due to adding `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`) is the desired behavior, so that `jni/config.h` is included, thus ensuring that `MONO_API` has the required definition when building `zlib-helper.c`.
2017-07-28 16:36:32 +03:00
<_MonoCilStripDest Include="$(_MSBuildDir)\cil-strip.exe" />
[bundle] Include mono's debug symbols (#513) Microsoft has (several?) [symbol servers][0], which is a repository "somewhere" which contains debug symbols, so that when a stack trace containing e.g. BCL stack frames is sent to Microsoft, there is a way to correlate that back to filename and line number source for further investigation. Xamarin.Android has not partaken in this mechanism, in large part because we did most of our building on Linux and macOS machines which generate `.mdb` debug symbols, while symbol servers require `.pdb`. That constraint is now behind us; with 76be1fb1 and the use of Mono 4.9 for builds, our builds now generate Portable `.pdb` files, which *should* (hopefully) be usable on Microsoft's symbol servers. However, to transmit them to Microsoft's servers, we need to actually *retain* them. At present, they're not: because of `bundle*.zip` use, the `mscorlib.dll` that we include in the final product may have been built much earlier than the rest of the product, and we don't have any other mechanism to retain the `mscorlib.pdb` file. Let's change that: update `mono-runtimes.targets` so that the created debug symbol files are included in the produced `bundle*.zip`, and bump the corresponding `$(XABundleFileName)` version number. This will ensure that we retain the debug symbols for our BCL assemblies, allowing us to actually have something to upload. [0]: https://msdn.microsoft.com/en-us/library/cc667410.aspx
2017-03-22 22:51:57 +03:00
<_MonoCilStripSource
Condition=" '$(_DebugFileExt)' == '.mdb' "
Include="$(_MonoOutputDir)\mono-cil-strip.exe.mdb"
/>
<_MonoCilStripDest
Condition=" '$(_DebugFileExt)' == '.mdb' "
Include="$(_MonoOutputDir)\cil-strip.exe.mdb"
/>
<_MonoCilStripSource
Condition=" '$(_DebugFileExt)' == '.pdb' "
Include="$(_MonoOutputDir)\mono-cil-strip.pdb"
/>
<_MonoCilStripDest
Condition=" '$(_DebugFileExt)' == '.pdb' "
[structure] Rework installation directory structure (#704) Context: https://github.com/xamarin/xamarin-android/pull/253#discussion_r82862993 The *intention* is that Jenkins-produced `oss-xamarin.android*.zip` artifacts be usable on Windows, so that side-by-side testing can be performed without replacing the system installation. Usage is in [UsingJenkinsBuildArtifacts.md](Documentation/UsingJenkinsBuildArtifacts). This isn't *entirely* the case. It was *apparently* the case at the time of commit 87ca2737, but things have bitrotten since. For example, following the 87ca2737 instructions would currently result in an XA0020 `Could not find monodroid` error, because `class-parse.exe` wasn't where Windows expects it to be (it was in `lib/mandroid`, not `lib/xbuild/Xamarin/Android`). This needs to be fixed. Additionally, PR #253 mentions that, for filesystem organization, it would be useful if the macOS/Linux directory structure -- `$prefix/bin`, `$prefix/lib/mandroid`, `$prefix/lib/xbuild/Xamarin/Android`, `$prefix/lib/xbuild-frameworks` -- more closely resembled the Windows directory structure of `$MSBuild` and `$ReferenceAssemblies` (as seen in `.vsix` files). This would turn macOS/Linux into using `$xa_prefix/xbuild` and `$xa_prefix/xbuild-frameworks` directories. `$prefix/bin` would only contain `xabuild`. What is currently in `$prefix/lib/mandroid` would be merged with `$xa_prefix/xbuild/Xamarin/Android`. `$xa_prefix` would `$prefix/lib/xamarin.android`. This would turn the current macOS structure: $prefix/bin/xabuild $prefix/bin/generator $prefix/bin/cross-arm $prefix/lib/mandroid/generator.exe $prefix/lib/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xbuild/Xamarin/Android/Xamarin.Android.Common.targets Into: $prefix/bin/xabuild $prefix/lib/xamarin.android/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xamarin.android/xbuild/Xamarin/Android/generator.exe $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Xamarin.Android.Common.targets $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Darwin/cross-arm Other notes: * The `bundle-*.zip` filename has been has been changed to include a *hash* of the contents of various files, in particular `build-tools\mono-runtimes.*`. This was instigated via a conversation with @kumpera about making the filename more "idiot-proof": `mono-runtimes.props` contains *compiler flags*, and if any of those change, then *logically* the bundle should differ as well, as the mono runtimes may differ in significant ways. In theory, the `-vXX` version should be used to track this, but this is a manual change, easy to overlook. The new `-hHASH` part of the filename should be more automagic. The new `<HashFileContents/>` task in `xa-prep-tasks.dll` is responsible for creating the hash value. * `Configuration.Java.Interop.Override.props` was moved into `build-tools/scripts`, because that would cleanup the root directory a bit. * OS-specific binaries are now placed into `$prefix/lib/xamarin.android/xbuild/Xamarin/Android/$(HostOS)`. On a theoretical plus side, this means that the same build directory can contain OS-specific binaries for multiple operating systems. (I don't know if anyone shares a build tree between e.g. macOS and Linux, but if anyone *does*...) Unfortunately this requires a workaround for an `xbuild` bug: `%(_LlvmRuntime.InstallPath)` and `%(_MonoCrossRuntime.InstallPath)` *cannot* use MSBuild properties, e.g. `<InstallPath>$(HostOS)/</InstallPath>` doesn't work as desired; it's instead treated literally. Special-case `%(InstallPath)` until we fully migrate to MSBuild. * `$(MonoAndroidToolsDirectory)` should be considered *dead*, along with `$(MonoAndroidBinDirectory)`, as these should now *always* be the same directory as where `Xamarin.Android.Build.Tasks.dll` is located, or a `$(HostOS)` sub-directory. * `Xamarin.ProjectTools.csproj` needed to be updated to ensure that the build order was correct. * Remove all `[Obsolete]` and unreferenced members from `Xamarin.Android.Build.Utilities.dll`. There's too much in there, and it makes my head hurt trying to understand the interrelationships between it all. If it's not used, it's gone. * The changes to `src/monodroid/jni/Android.mk` are...weird. The removal of `-I$(topdir)/libmonodroid/zip`/etc. is to reduce use of non-existent paths, as `$(topdir)` isn't defined, so that's *actually* `-I/libmonodroid/zip`, which is nonsensical. So far, so good. What's *odd* is the addition of `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`: This is needed so that `external/mono/support/zlib-helper.c` exports `CreateZStream` and related symbols, otherwise we get a unit test failure in `GzipStreamTest.Compression` due to an `EntryPointNotFoundException`, because `CreateZStream` isn't exported/public. What's odd here is that I don't understand what caused this behavior to change. Previous builds exported `CreateZStream`, otherwise the tests would fail, and I don't understand how any of the other changes in this PR would be at fault, though that's certainly the most plausible explanation. Regardless, `-Ijni` *first* (due to adding `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`) is the desired behavior, so that `jni/config.h` is included, thus ensuring that `MONO_API` has the required definition when building `zlib-helper.c`.
2017-07-28 16:36:32 +03:00
Include="$(_MSBuildDir)\cil-strip.pdb"
[bundle] Include mono's debug symbols (#513) Microsoft has (several?) [symbol servers][0], which is a repository "somewhere" which contains debug symbols, so that when a stack trace containing e.g. BCL stack frames is sent to Microsoft, there is a way to correlate that back to filename and line number source for further investigation. Xamarin.Android has not partaken in this mechanism, in large part because we did most of our building on Linux and macOS machines which generate `.mdb` debug symbols, while symbol servers require `.pdb`. That constraint is now behind us; with 76be1fb1 and the use of Mono 4.9 for builds, our builds now generate Portable `.pdb` files, which *should* (hopefully) be usable on Microsoft's symbol servers. However, to transmit them to Microsoft's servers, we need to actually *retain* them. At present, they're not: because of `bundle*.zip` use, the `mscorlib.dll` that we include in the final product may have been built much earlier than the rest of the product, and we don't have any other mechanism to retain the `mscorlib.pdb` file. Let's change that: update `mono-runtimes.targets` so that the created debug symbol files are included in the produced `bundle*.zip`, and bump the corresponding `$(XABundleFileName)` version number. This will ensure that we retain the debug symbols for our BCL assemblies, allowing us to actually have something to upload. [0]: https://msdn.microsoft.com/en-us/library/cc667410.aspx
2017-03-22 22:51:57 +03:00
/>
</ItemGroup>
[mono-runtimes] Install $prefix/lib/mandroid/cil-strip.exe `cil-strip.exe` is used by the `<CilStrip/>` task in `Xamarin.Android.Build.Tasks.dll`, and is just a renamed copy of `mono-cil-strip.exe`. Additionally, update the `_InstallMonoDoc` target to always `<Touch/>` the created files. This is to keep the `_InstallMonoDoc` task from constantly re-executing for me when running `xbuild /t:ForceBuild`: Target _InstallMonoDoc needs to be built as input file '.../xamarin-android/external/mono/mcs/class/lib/net_4_x/mdoc.exe' is newer than output file '../../bin/Debug//lib/mandroid/monodoc.dll.config'
2017-03-04 01:10:15 +03:00
<Target Name="_InstallCilStrip"
[bundle] Include mono's debug symbols (#513) Microsoft has (several?) [symbol servers][0], which is a repository "somewhere" which contains debug symbols, so that when a stack trace containing e.g. BCL stack frames is sent to Microsoft, there is a way to correlate that back to filename and line number source for further investigation. Xamarin.Android has not partaken in this mechanism, in large part because we did most of our building on Linux and macOS machines which generate `.mdb` debug symbols, while symbol servers require `.pdb`. That constraint is now behind us; with 76be1fb1 and the use of Mono 4.9 for builds, our builds now generate Portable `.pdb` files, which *should* (hopefully) be usable on Microsoft's symbol servers. However, to transmit them to Microsoft's servers, we need to actually *retain* them. At present, they're not: because of `bundle*.zip` use, the `mscorlib.dll` that we include in the final product may have been built much earlier than the rest of the product, and we don't have any other mechanism to retain the `mscorlib.pdb` file. Let's change that: update `mono-runtimes.targets` so that the created debug symbol files are included in the produced `bundle*.zip`, and bump the corresponding `$(XABundleFileName)` version number. This will ensure that we retain the debug symbols for our BCL assemblies, allowing us to actually have something to upload. [0]: https://msdn.microsoft.com/en-us/library/cc667410.aspx
2017-03-22 22:51:57 +03:00
Inputs="@(_MonoCilStripSource)"
Outputs="@(_MonoCilStripDest)">
[structure] Rework installation directory structure (#704) Context: https://github.com/xamarin/xamarin-android/pull/253#discussion_r82862993 The *intention* is that Jenkins-produced `oss-xamarin.android*.zip` artifacts be usable on Windows, so that side-by-side testing can be performed without replacing the system installation. Usage is in [UsingJenkinsBuildArtifacts.md](Documentation/UsingJenkinsBuildArtifacts). This isn't *entirely* the case. It was *apparently* the case at the time of commit 87ca2737, but things have bitrotten since. For example, following the 87ca2737 instructions would currently result in an XA0020 `Could not find monodroid` error, because `class-parse.exe` wasn't where Windows expects it to be (it was in `lib/mandroid`, not `lib/xbuild/Xamarin/Android`). This needs to be fixed. Additionally, PR #253 mentions that, for filesystem organization, it would be useful if the macOS/Linux directory structure -- `$prefix/bin`, `$prefix/lib/mandroid`, `$prefix/lib/xbuild/Xamarin/Android`, `$prefix/lib/xbuild-frameworks` -- more closely resembled the Windows directory structure of `$MSBuild` and `$ReferenceAssemblies` (as seen in `.vsix` files). This would turn macOS/Linux into using `$xa_prefix/xbuild` and `$xa_prefix/xbuild-frameworks` directories. `$prefix/bin` would only contain `xabuild`. What is currently in `$prefix/lib/mandroid` would be merged with `$xa_prefix/xbuild/Xamarin/Android`. `$xa_prefix` would `$prefix/lib/xamarin.android`. This would turn the current macOS structure: $prefix/bin/xabuild $prefix/bin/generator $prefix/bin/cross-arm $prefix/lib/mandroid/generator.exe $prefix/lib/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xbuild/Xamarin/Android/Xamarin.Android.Common.targets Into: $prefix/bin/xabuild $prefix/lib/xamarin.android/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xamarin.android/xbuild/Xamarin/Android/generator.exe $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Xamarin.Android.Common.targets $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Darwin/cross-arm Other notes: * The `bundle-*.zip` filename has been has been changed to include a *hash* of the contents of various files, in particular `build-tools\mono-runtimes.*`. This was instigated via a conversation with @kumpera about making the filename more "idiot-proof": `mono-runtimes.props` contains *compiler flags*, and if any of those change, then *logically* the bundle should differ as well, as the mono runtimes may differ in significant ways. In theory, the `-vXX` version should be used to track this, but this is a manual change, easy to overlook. The new `-hHASH` part of the filename should be more automagic. The new `<HashFileContents/>` task in `xa-prep-tasks.dll` is responsible for creating the hash value. * `Configuration.Java.Interop.Override.props` was moved into `build-tools/scripts`, because that would cleanup the root directory a bit. * OS-specific binaries are now placed into `$prefix/lib/xamarin.android/xbuild/Xamarin/Android/$(HostOS)`. On a theoretical plus side, this means that the same build directory can contain OS-specific binaries for multiple operating systems. (I don't know if anyone shares a build tree between e.g. macOS and Linux, but if anyone *does*...) Unfortunately this requires a workaround for an `xbuild` bug: `%(_LlvmRuntime.InstallPath)` and `%(_MonoCrossRuntime.InstallPath)` *cannot* use MSBuild properties, e.g. `<InstallPath>$(HostOS)/</InstallPath>` doesn't work as desired; it's instead treated literally. Special-case `%(InstallPath)` until we fully migrate to MSBuild. * `$(MonoAndroidToolsDirectory)` should be considered *dead*, along with `$(MonoAndroidBinDirectory)`, as these should now *always* be the same directory as where `Xamarin.Android.Build.Tasks.dll` is located, or a `$(HostOS)` sub-directory. * `Xamarin.ProjectTools.csproj` needed to be updated to ensure that the build order was correct. * Remove all `[Obsolete]` and unreferenced members from `Xamarin.Android.Build.Utilities.dll`. There's too much in there, and it makes my head hurt trying to understand the interrelationships between it all. If it's not used, it's gone. * The changes to `src/monodroid/jni/Android.mk` are...weird. The removal of `-I$(topdir)/libmonodroid/zip`/etc. is to reduce use of non-existent paths, as `$(topdir)` isn't defined, so that's *actually* `-I/libmonodroid/zip`, which is nonsensical. So far, so good. What's *odd* is the addition of `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`: This is needed so that `external/mono/support/zlib-helper.c` exports `CreateZStream` and related symbols, otherwise we get a unit test failure in `GzipStreamTest.Compression` due to an `EntryPointNotFoundException`, because `CreateZStream` isn't exported/public. What's odd here is that I don't understand what caused this behavior to change. Previous builds exported `CreateZStream`, otherwise the tests would fail, and I don't understand how any of the other changes in this PR would be at fault, though that's certainly the most plausible explanation. Regardless, `-Ijni` *first* (due to adding `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`) is the desired behavior, so that `jni/config.h` is included, thus ensuring that `MONO_API` has the required definition when building `zlib-helper.c`.
2017-07-28 16:36:32 +03:00
<MakeDir Directories="$(_MSBuildDir)" />
[mono-runtimes] Install $prefix/lib/mandroid/cil-strip.exe `cil-strip.exe` is used by the `<CilStrip/>` task in `Xamarin.Android.Build.Tasks.dll`, and is just a renamed copy of `mono-cil-strip.exe`. Additionally, update the `_InstallMonoDoc` target to always `<Touch/>` the created files. This is to keep the `_InstallMonoDoc` task from constantly re-executing for me when running `xbuild /t:ForceBuild`: Target _InstallMonoDoc needs to be built as input file '.../xamarin-android/external/mono/mcs/class/lib/net_4_x/mdoc.exe' is newer than output file '../../bin/Debug//lib/mandroid/monodoc.dll.config'
2017-03-04 01:10:15 +03:00
<Copy
[bundle] Include mono's debug symbols (#513) Microsoft has (several?) [symbol servers][0], which is a repository "somewhere" which contains debug symbols, so that when a stack trace containing e.g. BCL stack frames is sent to Microsoft, there is a way to correlate that back to filename and line number source for further investigation. Xamarin.Android has not partaken in this mechanism, in large part because we did most of our building on Linux and macOS machines which generate `.mdb` debug symbols, while symbol servers require `.pdb`. That constraint is now behind us; with 76be1fb1 and the use of Mono 4.9 for builds, our builds now generate Portable `.pdb` files, which *should* (hopefully) be usable on Microsoft's symbol servers. However, to transmit them to Microsoft's servers, we need to actually *retain* them. At present, they're not: because of `bundle*.zip` use, the `mscorlib.dll` that we include in the final product may have been built much earlier than the rest of the product, and we don't have any other mechanism to retain the `mscorlib.pdb` file. Let's change that: update `mono-runtimes.targets` so that the created debug symbol files are included in the produced `bundle*.zip`, and bump the corresponding `$(XABundleFileName)` version number. This will ensure that we retain the debug symbols for our BCL assemblies, allowing us to actually have something to upload. [0]: https://msdn.microsoft.com/en-us/library/cc667410.aspx
2017-03-22 22:51:57 +03:00
SourceFiles="@(_MonoCilStripSource)"
DestinationFiles="@(_MonoCilStripDest)"
[mono-runtimes] Install $prefix/lib/mandroid/cil-strip.exe `cil-strip.exe` is used by the `<CilStrip/>` task in `Xamarin.Android.Build.Tasks.dll`, and is just a renamed copy of `mono-cil-strip.exe`. Additionally, update the `_InstallMonoDoc` target to always `<Touch/>` the created files. This is to keep the `_InstallMonoDoc` task from constantly re-executing for me when running `xbuild /t:ForceBuild`: Target _InstallMonoDoc needs to be built as input file '.../xamarin-android/external/mono/mcs/class/lib/net_4_x/mdoc.exe' is newer than output file '../../bin/Debug//lib/mandroid/monodoc.dll.config'
2017-03-04 01:10:15 +03:00
/>
</Target>
[mdoc] Add mdoc and dependencies to the bundle Context: https://github.com/xamarin/xamarin-android/commit/5a8e48eba8cc8d05ec4fef25ac074d00606f5a69 `mdoc.exe` is used by the `<MDoc/>` task, used by the `BuildDocumentation` target in `Xamarin.Android.Bindings.targets`. We obtain `mdoc.exe` from the `external/mono` build, but because `mdoc.exe` will be referencing mono's `Mono.Cecil.dll`, we use `remap-assembly-ref` to update `mdoc.exe` to instead use `Xamarin.Android.Cecil.dll`.
2016-12-20 01:18:21 +03:00
<Target Name="_InstallMonoDoc"
Inputs="@(_MonoDocCopyItems);$(_MonoOutputDir)\mdoc.exe"
[mono-runtimes] Fix bundle*.zip use (#510) Commit 76be1fb1 (6e9a6612) broke `bundle*.zip` use. Consequently, `bundle*.zip` *is not used*, resulting in *yuuuge* build times (nearly 7 hours for full builds on macOS, over 2 hours for PRs!). The `_BuildUnlessCached` target calls the `GetMonoBundleItems` target, to determine whether the `ForceBuild` target -- which invokes `_BuildRuntimes` and many other targets -- should be executed. The `GetMonoBundleItems` returns all files which would be included in `bundle*.zip`. The cause of the break is that commit 6e9a6612 changed the `GetMonoBundleItems` target to depend on the `_GetMonodocItems` target, which in turn depended on the `_BuildRuntimes` target. `_BuildRuntimes` is the target we want to avoid executing; it's the target that builds mono, and thus can take an eternity (particularly on the `xamarin-android` Jenkins lane, which builds **ALL THE ABIS**). Adding `_GetMonodocItems` in this fashion thus means we never use the cache, bceause in order to determine whether or not we can use the cache we need to do all the work that the cache was intended to avoid. Consider the [xamarin-android Build 312 log file][0] Building target "_BuildUnlessCached" in project ".../xamarin-android/build-tools/mono-runtimes/mono-runtimes.mdproj" (".../xamarin-android/build-tools/mono-runtimes/mono-runtimes.targets"); "Build" depends on it. ... Building target "GetMonoBundleItems" in project ".../xamarin-android/build-tools/mono-runtimes/mono-runtimes.mdproj" (".../xamarin-android/build-tools/mono-runtimes/mono-runtimes.targets"); "_BuildUnlessCached" depends on it. ... Building target "_GetMonodocItems" in project ".../xamarin-android/build-tools/mono-runtimes/mono-runtimes.mdproj" (".../xamarin-android/build-tools/mono-runtimes/mono-runtimes.targets"); "GetMonoBundleItems" depends on it. Building target "_BuildRuntimes" in project ".../xamarin-android/build-tools/mono-runtimes/mono-runtimes.mdproj" (".../xamarin-android/build-tools/mono-runtimes/mono-runtimes.targets"); "_GetMonodocItems" depends on it. ...and **BOOM**, the existence of `bundle*.zip` is meaningless. The fix is to partially revert commit 6e9a6612, but the entire reason 6e9a6612 exists is because commit 76be1fb1 changes the default file extension for debug symbols from `.mdb` to `.pdb`: How should we know if we're emitting `.mdb` or `.pdb` symbols? *A* way to know that is to actually build everything and see what is generated, but that's...slow. Slow and undesirable. (That's what 6e9a6612 does!) An alternate ("hacky") way to know is to rely on mono-specific extensions, and spackle, and duct tape: Mono's `msbuild` sets the `$(_DebugFileExt)` MSBuild property to the file extension of debug symbols, with values of `.mdb` on Mono 4.6 through Mono 4.8, and a value of `.pdb` on Mono 4.9. In `build-tools/scripts/msbuild.mk`, we can probe the mono version and attempt to come up with "sane" defaults, setting `$(_DebugFileExt)` for `xbuild` invocations done via `make`. We can thus rely on the `$(_DebugFileExt)` extension instead of costly filesystem probing, allowing `bundle*.zip` to actually be used, instead of ignored. ~~~ **A tale of two `pkg-config`s**: A funny thing happened while "fixing" `msbuild.mk` to set `$(_DebugFileExt)`: Turns Out™ there are (at least?) *two* `pkg-config` files on the Jenkins+macOS machine: 1. `/usr/local/bin/pkg-config` 2. `/Library/Frameworks/Mono.framework/Commands/pkg-config` (1) knows *nothing* about `Mono.framework`, and we're using the `mono` from `Mono.framework`, which distributes (2). Furthermore, we want to use `pkg-config --atleast-version=4.9 mono` in order to know if we're emitting `.pdb` or `.mdb` debug files, so using the correct version is Very Important. So, off I go to fix that so that we always use (2) on macOS, [and macOS+msbuild breaks][1]. Why's this matter? Recall 0eee6734, in which `$(_XAFixMSBuildFrameworkPathOverride)` is set only when (a) `msbuild` is used, and (b) mono prior to 4.8 is used? The check for (b) used the `pkg-config` from (1), meaning *the version check always failed*, meaning when building with `msbuild` from e.g. Mono 4.9, `$(_XAFixMSBuildFrameworkPathOverride)` was still being set, even though, as per 0eee6734, it shouldn't have been. Which means the explanation in 0eee6734 is wrong. Drain the swamp, at least a little bit: Remove `$(_XAFixMSBuildFrameworkPathOverride)`, and *always* set `$(FrameworkPathOverride)` within `MonoAndroidFramework.props`. It appears that attempting to make `$(FrameworkPathOverride)` conditional wasn't necessary, and only complicates matters. [0]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android/312/consoleText [1]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android-msbuild-pr-builder/243/
2017-03-22 17:50:14 +03:00
Outputs="@(_MonoDocInstalledItems)">
[structure] Rework installation directory structure (#704) Context: https://github.com/xamarin/xamarin-android/pull/253#discussion_r82862993 The *intention* is that Jenkins-produced `oss-xamarin.android*.zip` artifacts be usable on Windows, so that side-by-side testing can be performed without replacing the system installation. Usage is in [UsingJenkinsBuildArtifacts.md](Documentation/UsingJenkinsBuildArtifacts). This isn't *entirely* the case. It was *apparently* the case at the time of commit 87ca2737, but things have bitrotten since. For example, following the 87ca2737 instructions would currently result in an XA0020 `Could not find monodroid` error, because `class-parse.exe` wasn't where Windows expects it to be (it was in `lib/mandroid`, not `lib/xbuild/Xamarin/Android`). This needs to be fixed. Additionally, PR #253 mentions that, for filesystem organization, it would be useful if the macOS/Linux directory structure -- `$prefix/bin`, `$prefix/lib/mandroid`, `$prefix/lib/xbuild/Xamarin/Android`, `$prefix/lib/xbuild-frameworks` -- more closely resembled the Windows directory structure of `$MSBuild` and `$ReferenceAssemblies` (as seen in `.vsix` files). This would turn macOS/Linux into using `$xa_prefix/xbuild` and `$xa_prefix/xbuild-frameworks` directories. `$prefix/bin` would only contain `xabuild`. What is currently in `$prefix/lib/mandroid` would be merged with `$xa_prefix/xbuild/Xamarin/Android`. `$xa_prefix` would `$prefix/lib/xamarin.android`. This would turn the current macOS structure: $prefix/bin/xabuild $prefix/bin/generator $prefix/bin/cross-arm $prefix/lib/mandroid/generator.exe $prefix/lib/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xbuild/Xamarin/Android/Xamarin.Android.Common.targets Into: $prefix/bin/xabuild $prefix/lib/xamarin.android/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xamarin.android/xbuild/Xamarin/Android/generator.exe $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Xamarin.Android.Common.targets $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Darwin/cross-arm Other notes: * The `bundle-*.zip` filename has been has been changed to include a *hash* of the contents of various files, in particular `build-tools\mono-runtimes.*`. This was instigated via a conversation with @kumpera about making the filename more "idiot-proof": `mono-runtimes.props` contains *compiler flags*, and if any of those change, then *logically* the bundle should differ as well, as the mono runtimes may differ in significant ways. In theory, the `-vXX` version should be used to track this, but this is a manual change, easy to overlook. The new `-hHASH` part of the filename should be more automagic. The new `<HashFileContents/>` task in `xa-prep-tasks.dll` is responsible for creating the hash value. * `Configuration.Java.Interop.Override.props` was moved into `build-tools/scripts`, because that would cleanup the root directory a bit. * OS-specific binaries are now placed into `$prefix/lib/xamarin.android/xbuild/Xamarin/Android/$(HostOS)`. On a theoretical plus side, this means that the same build directory can contain OS-specific binaries for multiple operating systems. (I don't know if anyone shares a build tree between e.g. macOS and Linux, but if anyone *does*...) Unfortunately this requires a workaround for an `xbuild` bug: `%(_LlvmRuntime.InstallPath)` and `%(_MonoCrossRuntime.InstallPath)` *cannot* use MSBuild properties, e.g. `<InstallPath>$(HostOS)/</InstallPath>` doesn't work as desired; it's instead treated literally. Special-case `%(InstallPath)` until we fully migrate to MSBuild. * `$(MonoAndroidToolsDirectory)` should be considered *dead*, along with `$(MonoAndroidBinDirectory)`, as these should now *always* be the same directory as where `Xamarin.Android.Build.Tasks.dll` is located, or a `$(HostOS)` sub-directory. * `Xamarin.ProjectTools.csproj` needed to be updated to ensure that the build order was correct. * Remove all `[Obsolete]` and unreferenced members from `Xamarin.Android.Build.Utilities.dll`. There's too much in there, and it makes my head hurt trying to understand the interrelationships between it all. If it's not used, it's gone. * The changes to `src/monodroid/jni/Android.mk` are...weird. The removal of `-I$(topdir)/libmonodroid/zip`/etc. is to reduce use of non-existent paths, as `$(topdir)` isn't defined, so that's *actually* `-I/libmonodroid/zip`, which is nonsensical. So far, so good. What's *odd* is the addition of `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`: This is needed so that `external/mono/support/zlib-helper.c` exports `CreateZStream` and related symbols, otherwise we get a unit test failure in `GzipStreamTest.Compression` due to an `EntryPointNotFoundException`, because `CreateZStream` isn't exported/public. What's odd here is that I don't understand what caused this behavior to change. Previous builds exported `CreateZStream`, otherwise the tests would fail, and I don't understand how any of the other changes in this PR would be at fault, though that's certainly the most plausible explanation. Regardless, `-Ijni` *first* (due to adding `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`) is the desired behavior, so that `jni/config.h` is included, thus ensuring that `MONO_API` has the required definition when building `zlib-helper.c`.
2017-07-28 16:36:32 +03:00
<MakeDir Directories="$(_MSBuildDir)" />
[mdoc] Add mdoc and dependencies to the bundle Context: https://github.com/xamarin/xamarin-android/commit/5a8e48eba8cc8d05ec4fef25ac074d00606f5a69 `mdoc.exe` is used by the `<MDoc/>` task, used by the `BuildDocumentation` target in `Xamarin.Android.Bindings.targets`. We obtain `mdoc.exe` from the `external/mono` build, but because `mdoc.exe` will be referencing mono's `Mono.Cecil.dll`, we use `remap-assembly-ref` to update `mdoc.exe` to instead use `Xamarin.Android.Cecil.dll`.
2016-12-20 01:18:21 +03:00
<Copy
SourceFiles="@(_MonoDocCopyItems)"
[structure] Rework installation directory structure (#704) Context: https://github.com/xamarin/xamarin-android/pull/253#discussion_r82862993 The *intention* is that Jenkins-produced `oss-xamarin.android*.zip` artifacts be usable on Windows, so that side-by-side testing can be performed without replacing the system installation. Usage is in [UsingJenkinsBuildArtifacts.md](Documentation/UsingJenkinsBuildArtifacts). This isn't *entirely* the case. It was *apparently* the case at the time of commit 87ca2737, but things have bitrotten since. For example, following the 87ca2737 instructions would currently result in an XA0020 `Could not find monodroid` error, because `class-parse.exe` wasn't where Windows expects it to be (it was in `lib/mandroid`, not `lib/xbuild/Xamarin/Android`). This needs to be fixed. Additionally, PR #253 mentions that, for filesystem organization, it would be useful if the macOS/Linux directory structure -- `$prefix/bin`, `$prefix/lib/mandroid`, `$prefix/lib/xbuild/Xamarin/Android`, `$prefix/lib/xbuild-frameworks` -- more closely resembled the Windows directory structure of `$MSBuild` and `$ReferenceAssemblies` (as seen in `.vsix` files). This would turn macOS/Linux into using `$xa_prefix/xbuild` and `$xa_prefix/xbuild-frameworks` directories. `$prefix/bin` would only contain `xabuild`. What is currently in `$prefix/lib/mandroid` would be merged with `$xa_prefix/xbuild/Xamarin/Android`. `$xa_prefix` would `$prefix/lib/xamarin.android`. This would turn the current macOS structure: $prefix/bin/xabuild $prefix/bin/generator $prefix/bin/cross-arm $prefix/lib/mandroid/generator.exe $prefix/lib/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xbuild/Xamarin/Android/Xamarin.Android.Common.targets Into: $prefix/bin/xabuild $prefix/lib/xamarin.android/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xamarin.android/xbuild/Xamarin/Android/generator.exe $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Xamarin.Android.Common.targets $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Darwin/cross-arm Other notes: * The `bundle-*.zip` filename has been has been changed to include a *hash* of the contents of various files, in particular `build-tools\mono-runtimes.*`. This was instigated via a conversation with @kumpera about making the filename more "idiot-proof": `mono-runtimes.props` contains *compiler flags*, and if any of those change, then *logically* the bundle should differ as well, as the mono runtimes may differ in significant ways. In theory, the `-vXX` version should be used to track this, but this is a manual change, easy to overlook. The new `-hHASH` part of the filename should be more automagic. The new `<HashFileContents/>` task in `xa-prep-tasks.dll` is responsible for creating the hash value. * `Configuration.Java.Interop.Override.props` was moved into `build-tools/scripts`, because that would cleanup the root directory a bit. * OS-specific binaries are now placed into `$prefix/lib/xamarin.android/xbuild/Xamarin/Android/$(HostOS)`. On a theoretical plus side, this means that the same build directory can contain OS-specific binaries for multiple operating systems. (I don't know if anyone shares a build tree between e.g. macOS and Linux, but if anyone *does*...) Unfortunately this requires a workaround for an `xbuild` bug: `%(_LlvmRuntime.InstallPath)` and `%(_MonoCrossRuntime.InstallPath)` *cannot* use MSBuild properties, e.g. `<InstallPath>$(HostOS)/</InstallPath>` doesn't work as desired; it's instead treated literally. Special-case `%(InstallPath)` until we fully migrate to MSBuild. * `$(MonoAndroidToolsDirectory)` should be considered *dead*, along with `$(MonoAndroidBinDirectory)`, as these should now *always* be the same directory as where `Xamarin.Android.Build.Tasks.dll` is located, or a `$(HostOS)` sub-directory. * `Xamarin.ProjectTools.csproj` needed to be updated to ensure that the build order was correct. * Remove all `[Obsolete]` and unreferenced members from `Xamarin.Android.Build.Utilities.dll`. There's too much in there, and it makes my head hurt trying to understand the interrelationships between it all. If it's not used, it's gone. * The changes to `src/monodroid/jni/Android.mk` are...weird. The removal of `-I$(topdir)/libmonodroid/zip`/etc. is to reduce use of non-existent paths, as `$(topdir)` isn't defined, so that's *actually* `-I/libmonodroid/zip`, which is nonsensical. So far, so good. What's *odd* is the addition of `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`: This is needed so that `external/mono/support/zlib-helper.c` exports `CreateZStream` and related symbols, otherwise we get a unit test failure in `GzipStreamTest.Compression` due to an `EntryPointNotFoundException`, because `CreateZStream` isn't exported/public. What's odd here is that I don't understand what caused this behavior to change. Previous builds exported `CreateZStream`, otherwise the tests would fail, and I don't understand how any of the other changes in this PR would be at fault, though that's certainly the most plausible explanation. Regardless, `-Ijni` *first* (due to adding `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`) is the desired behavior, so that `jni/config.h` is included, thus ensuring that `MONO_API` has the required definition when building `zlib-helper.c`.
2017-07-28 16:36:32 +03:00
DestinationFolder="$(_MSBuildDir)"
[mdoc] Add mdoc and dependencies to the bundle Context: https://github.com/xamarin/xamarin-android/commit/5a8e48eba8cc8d05ec4fef25ac074d00606f5a69 `mdoc.exe` is used by the `<MDoc/>` task, used by the `BuildDocumentation` target in `Xamarin.Android.Bindings.targets`. We obtain `mdoc.exe` from the `external/mono` build, but because `mdoc.exe` will be referencing mono's `Mono.Cecil.dll`, we use `remap-assembly-ref` to update `mdoc.exe` to instead use `Xamarin.Android.Cecil.dll`.
2016-12-20 01:18:21 +03:00
/>
<Exec
[structure] Rework installation directory structure (#704) Context: https://github.com/xamarin/xamarin-android/pull/253#discussion_r82862993 The *intention* is that Jenkins-produced `oss-xamarin.android*.zip` artifacts be usable on Windows, so that side-by-side testing can be performed without replacing the system installation. Usage is in [UsingJenkinsBuildArtifacts.md](Documentation/UsingJenkinsBuildArtifacts). This isn't *entirely* the case. It was *apparently* the case at the time of commit 87ca2737, but things have bitrotten since. For example, following the 87ca2737 instructions would currently result in an XA0020 `Could not find monodroid` error, because `class-parse.exe` wasn't where Windows expects it to be (it was in `lib/mandroid`, not `lib/xbuild/Xamarin/Android`). This needs to be fixed. Additionally, PR #253 mentions that, for filesystem organization, it would be useful if the macOS/Linux directory structure -- `$prefix/bin`, `$prefix/lib/mandroid`, `$prefix/lib/xbuild/Xamarin/Android`, `$prefix/lib/xbuild-frameworks` -- more closely resembled the Windows directory structure of `$MSBuild` and `$ReferenceAssemblies` (as seen in `.vsix` files). This would turn macOS/Linux into using `$xa_prefix/xbuild` and `$xa_prefix/xbuild-frameworks` directories. `$prefix/bin` would only contain `xabuild`. What is currently in `$prefix/lib/mandroid` would be merged with `$xa_prefix/xbuild/Xamarin/Android`. `$xa_prefix` would `$prefix/lib/xamarin.android`. This would turn the current macOS structure: $prefix/bin/xabuild $prefix/bin/generator $prefix/bin/cross-arm $prefix/lib/mandroid/generator.exe $prefix/lib/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xbuild/Xamarin/Android/Xamarin.Android.Common.targets Into: $prefix/bin/xabuild $prefix/lib/xamarin.android/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xamarin.android/xbuild/Xamarin/Android/generator.exe $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Xamarin.Android.Common.targets $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Darwin/cross-arm Other notes: * The `bundle-*.zip` filename has been has been changed to include a *hash* of the contents of various files, in particular `build-tools\mono-runtimes.*`. This was instigated via a conversation with @kumpera about making the filename more "idiot-proof": `mono-runtimes.props` contains *compiler flags*, and if any of those change, then *logically* the bundle should differ as well, as the mono runtimes may differ in significant ways. In theory, the `-vXX` version should be used to track this, but this is a manual change, easy to overlook. The new `-hHASH` part of the filename should be more automagic. The new `<HashFileContents/>` task in `xa-prep-tasks.dll` is responsible for creating the hash value. * `Configuration.Java.Interop.Override.props` was moved into `build-tools/scripts`, because that would cleanup the root directory a bit. * OS-specific binaries are now placed into `$prefix/lib/xamarin.android/xbuild/Xamarin/Android/$(HostOS)`. On a theoretical plus side, this means that the same build directory can contain OS-specific binaries for multiple operating systems. (I don't know if anyone shares a build tree between e.g. macOS and Linux, but if anyone *does*...) Unfortunately this requires a workaround for an `xbuild` bug: `%(_LlvmRuntime.InstallPath)` and `%(_MonoCrossRuntime.InstallPath)` *cannot* use MSBuild properties, e.g. `<InstallPath>$(HostOS)/</InstallPath>` doesn't work as desired; it's instead treated literally. Special-case `%(InstallPath)` until we fully migrate to MSBuild. * `$(MonoAndroidToolsDirectory)` should be considered *dead*, along with `$(MonoAndroidBinDirectory)`, as these should now *always* be the same directory as where `Xamarin.Android.Build.Tasks.dll` is located, or a `$(HostOS)` sub-directory. * `Xamarin.ProjectTools.csproj` needed to be updated to ensure that the build order was correct. * Remove all `[Obsolete]` and unreferenced members from `Xamarin.Android.Build.Utilities.dll`. There's too much in there, and it makes my head hurt trying to understand the interrelationships between it all. If it's not used, it's gone. * The changes to `src/monodroid/jni/Android.mk` are...weird. The removal of `-I$(topdir)/libmonodroid/zip`/etc. is to reduce use of non-existent paths, as `$(topdir)` isn't defined, so that's *actually* `-I/libmonodroid/zip`, which is nonsensical. So far, so good. What's *odd* is the addition of `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`: This is needed so that `external/mono/support/zlib-helper.c` exports `CreateZStream` and related symbols, otherwise we get a unit test failure in `GzipStreamTest.Compression` due to an `EntryPointNotFoundException`, because `CreateZStream` isn't exported/public. What's odd here is that I don't understand what caused this behavior to change. Previous builds exported `CreateZStream`, otherwise the tests would fail, and I don't understand how any of the other changes in this PR would be at fault, though that's certainly the most plausible explanation. Regardless, `-Ijni` *first* (due to adding `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`) is the desired behavior, so that `jni/config.h` is included, thus ensuring that `MONO_API` has the required definition when building `zlib-helper.c`.
2017-07-28 16:36:32 +03:00
Command="$(RemapAssemblyRefTool) &quot;$(_MonoOutputDir)\mdoc.exe&quot; &quot;$(_MSBuildDir)\mdoc.exe&quot; Mono.Cecil &quot;..\..\bin\Build$(Configuration)\Xamarin.Android.Cecil.dll&quot;"
[mdoc] Add mdoc and dependencies to the bundle Context: https://github.com/xamarin/xamarin-android/commit/5a8e48eba8cc8d05ec4fef25ac074d00606f5a69 `mdoc.exe` is used by the `<MDoc/>` task, used by the `BuildDocumentation` target in `Xamarin.Android.Bindings.targets`. We obtain `mdoc.exe` from the `external/mono` build, but because `mdoc.exe` will be referencing mono's `Mono.Cecil.dll`, we use `remap-assembly-ref` to update `mdoc.exe` to instead use `Xamarin.Android.Cecil.dll`.
2016-12-20 01:18:21 +03:00
/>
[mono-runtimes] Install $prefix/lib/mandroid/cil-strip.exe `cil-strip.exe` is used by the `<CilStrip/>` task in `Xamarin.Android.Build.Tasks.dll`, and is just a renamed copy of `mono-cil-strip.exe`. Additionally, update the `_InstallMonoDoc` target to always `<Touch/>` the created files. This is to keep the `_InstallMonoDoc` task from constantly re-executing for me when running `xbuild /t:ForceBuild`: Target _InstallMonoDoc needs to be built as input file '.../xamarin-android/external/mono/mcs/class/lib/net_4_x/mdoc.exe' is newer than output file '../../bin/Debug//lib/mandroid/monodoc.dll.config'
2017-03-04 01:10:15 +03:00
<Touch
Files="@(_MonoDocInstalledItems)"
/>
[mdoc] Add mdoc and dependencies to the bundle Context: https://github.com/xamarin/xamarin-android/commit/5a8e48eba8cc8d05ec4fef25ac074d00606f5a69 `mdoc.exe` is used by the `<MDoc/>` task, used by the `BuildDocumentation` target in `Xamarin.Android.Bindings.targets`. We obtain `mdoc.exe` from the `external/mono` build, but because `mdoc.exe` will be referencing mono's `Mono.Cecil.dll`, we use `remap-assembly-ref` to update `mdoc.exe` to instead use `Xamarin.Android.Cecil.dll`.
2016-12-20 01:18:21 +03:00
</Target>
[bundle] Include mono's debug symbols (#513) Microsoft has (several?) [symbol servers][0], which is a repository "somewhere" which contains debug symbols, so that when a stack trace containing e.g. BCL stack frames is sent to Microsoft, there is a way to correlate that back to filename and line number source for further investigation. Xamarin.Android has not partaken in this mechanism, in large part because we did most of our building on Linux and macOS machines which generate `.mdb` debug symbols, while symbol servers require `.pdb`. That constraint is now behind us; with 76be1fb1 and the use of Mono 4.9 for builds, our builds now generate Portable `.pdb` files, which *should* (hopefully) be usable on Microsoft's symbol servers. However, to transmit them to Microsoft's servers, we need to actually *retain* them. At present, they're not: because of `bundle*.zip` use, the `mscorlib.dll` that we include in the final product may have been built much earlier than the rest of the product, and we don't have any other mechanism to retain the `mscorlib.pdb` file. Let's change that: update `mono-runtimes.targets` so that the created debug symbol files are included in the produced `bundle*.zip`, and bump the corresponding `$(XABundleFileName)` version number. This will ensure that we retain the debug symbols for our BCL assemblies, allowing us to actually have something to upload. [0]: https://msdn.microsoft.com/en-us/library/cc667410.aspx
2017-03-22 22:51:57 +03:00
<Target Name="_InstallMonoUtilities"
Inputs="@(_MonoUtilitySource)"
Outputs="@(_MonoUtilityDest)">
[structure] Rework installation directory structure (#704) Context: https://github.com/xamarin/xamarin-android/pull/253#discussion_r82862993 The *intention* is that Jenkins-produced `oss-xamarin.android*.zip` artifacts be usable on Windows, so that side-by-side testing can be performed without replacing the system installation. Usage is in [UsingJenkinsBuildArtifacts.md](Documentation/UsingJenkinsBuildArtifacts). This isn't *entirely* the case. It was *apparently* the case at the time of commit 87ca2737, but things have bitrotten since. For example, following the 87ca2737 instructions would currently result in an XA0020 `Could not find monodroid` error, because `class-parse.exe` wasn't where Windows expects it to be (it was in `lib/mandroid`, not `lib/xbuild/Xamarin/Android`). This needs to be fixed. Additionally, PR #253 mentions that, for filesystem organization, it would be useful if the macOS/Linux directory structure -- `$prefix/bin`, `$prefix/lib/mandroid`, `$prefix/lib/xbuild/Xamarin/Android`, `$prefix/lib/xbuild-frameworks` -- more closely resembled the Windows directory structure of `$MSBuild` and `$ReferenceAssemblies` (as seen in `.vsix` files). This would turn macOS/Linux into using `$xa_prefix/xbuild` and `$xa_prefix/xbuild-frameworks` directories. `$prefix/bin` would only contain `xabuild`. What is currently in `$prefix/lib/mandroid` would be merged with `$xa_prefix/xbuild/Xamarin/Android`. `$xa_prefix` would `$prefix/lib/xamarin.android`. This would turn the current macOS structure: $prefix/bin/xabuild $prefix/bin/generator $prefix/bin/cross-arm $prefix/lib/mandroid/generator.exe $prefix/lib/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xbuild/Xamarin/Android/Xamarin.Android.Common.targets Into: $prefix/bin/xabuild $prefix/lib/xamarin.android/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xamarin.android/xbuild/Xamarin/Android/generator.exe $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Xamarin.Android.Common.targets $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Darwin/cross-arm Other notes: * The `bundle-*.zip` filename has been has been changed to include a *hash* of the contents of various files, in particular `build-tools\mono-runtimes.*`. This was instigated via a conversation with @kumpera about making the filename more "idiot-proof": `mono-runtimes.props` contains *compiler flags*, and if any of those change, then *logically* the bundle should differ as well, as the mono runtimes may differ in significant ways. In theory, the `-vXX` version should be used to track this, but this is a manual change, easy to overlook. The new `-hHASH` part of the filename should be more automagic. The new `<HashFileContents/>` task in `xa-prep-tasks.dll` is responsible for creating the hash value. * `Configuration.Java.Interop.Override.props` was moved into `build-tools/scripts`, because that would cleanup the root directory a bit. * OS-specific binaries are now placed into `$prefix/lib/xamarin.android/xbuild/Xamarin/Android/$(HostOS)`. On a theoretical plus side, this means that the same build directory can contain OS-specific binaries for multiple operating systems. (I don't know if anyone shares a build tree between e.g. macOS and Linux, but if anyone *does*...) Unfortunately this requires a workaround for an `xbuild` bug: `%(_LlvmRuntime.InstallPath)` and `%(_MonoCrossRuntime.InstallPath)` *cannot* use MSBuild properties, e.g. `<InstallPath>$(HostOS)/</InstallPath>` doesn't work as desired; it's instead treated literally. Special-case `%(InstallPath)` until we fully migrate to MSBuild. * `$(MonoAndroidToolsDirectory)` should be considered *dead*, along with `$(MonoAndroidBinDirectory)`, as these should now *always* be the same directory as where `Xamarin.Android.Build.Tasks.dll` is located, or a `$(HostOS)` sub-directory. * `Xamarin.ProjectTools.csproj` needed to be updated to ensure that the build order was correct. * Remove all `[Obsolete]` and unreferenced members from `Xamarin.Android.Build.Utilities.dll`. There's too much in there, and it makes my head hurt trying to understand the interrelationships between it all. If it's not used, it's gone. * The changes to `src/monodroid/jni/Android.mk` are...weird. The removal of `-I$(topdir)/libmonodroid/zip`/etc. is to reduce use of non-existent paths, as `$(topdir)` isn't defined, so that's *actually* `-I/libmonodroid/zip`, which is nonsensical. So far, so good. What's *odd* is the addition of `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`: This is needed so that `external/mono/support/zlib-helper.c` exports `CreateZStream` and related symbols, otherwise we get a unit test failure in `GzipStreamTest.Compression` due to an `EntryPointNotFoundException`, because `CreateZStream` isn't exported/public. What's odd here is that I don't understand what caused this behavior to change. Previous builds exported `CreateZStream`, otherwise the tests would fail, and I don't understand how any of the other changes in this PR would be at fault, though that's certainly the most plausible explanation. Regardless, `-Ijni` *first* (due to adding `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`) is the desired behavior, so that `jni/config.h` is included, thus ensuring that `MONO_API` has the required definition when building `zlib-helper.c`.
2017-07-28 16:36:32 +03:00
<MakeDir Directories="$(_MSBuildDir)" />
[mono-runtimes] Build mono-symbolicate.exe via mono Building `mono-symbolicate.exe` is fun [^0]. Commit 0147bc79 attempted to build `mono-symbolicate` by using `mono/mcs/tools/mono-symbolicate/monosymbolicate.csproj`. This didn't actually work, though, as the resulting `mono-symbolicate.exe` had an assembly reference to `Mono.Cecil.dll`, which we don't distribute for [historical reasons][1]. This resulted in commit 9b63344a, which attempted to "replicate" what mono's build system did within the constructs of an MSBuild target and `<Csc/>` task invocation. While this "worked," it was brittle: it involved reading `mono-symbolicate.exe.sources`, which can change -- necessitating yet more chanegs (commit 1b023bde) -- and even that is inadequate, as `mono-symbolicate.exe.sources` can contain *file globs*, which our "read lines into a `<TaskItem/>`" voodoo doesn't support. We want something that will more reliably build `mono-symbolicate.exe`, something maintained by the #runtime team. We *have* such a mechanism: `mono-symbolicate/Makefile`. Why can't we use that? We *couldn't* use it because, as mentioned previously, Mono's normal build mechanisms will cause `mono-symbolicate.exe` to reference `Mono.Cecil.dll`. However, we have an alternative now that we didn't have in the time of commit 9b63344a: [`remap-assembly-ref.exe`][1]. Thus a new, hopefully more maintainable approach: Use mono's build system to build `mono-symbolicate.exe`, then use `remap-assembly-ref.exe` to remap `Mono.Cecil` to `Xamarin.Android.Cecil.dll`. [1]: https://github.com/xamarin/xamarin-android/commit/5a8e48eba8cc8d05ec4fef25ac074d00606f5a69 [^0]: Note: Not actually fun.
2017-02-07 21:16:50 +03:00
<Copy
[bundle] Include mono's debug symbols (#513) Microsoft has (several?) [symbol servers][0], which is a repository "somewhere" which contains debug symbols, so that when a stack trace containing e.g. BCL stack frames is sent to Microsoft, there is a way to correlate that back to filename and line number source for further investigation. Xamarin.Android has not partaken in this mechanism, in large part because we did most of our building on Linux and macOS machines which generate `.mdb` debug symbols, while symbol servers require `.pdb`. That constraint is now behind us; with 76be1fb1 and the use of Mono 4.9 for builds, our builds now generate Portable `.pdb` files, which *should* (hopefully) be usable on Microsoft's symbol servers. However, to transmit them to Microsoft's servers, we need to actually *retain* them. At present, they're not: because of `bundle*.zip` use, the `mscorlib.dll` that we include in the final product may have been built much earlier than the rest of the product, and we don't have any other mechanism to retain the `mscorlib.pdb` file. Let's change that: update `mono-runtimes.targets` so that the created debug symbol files are included in the produced `bundle*.zip`, and bump the corresponding `$(XABundleFileName)` version number. This will ensure that we retain the debug symbols for our BCL assemblies, allowing us to actually have something to upload. [0]: https://msdn.microsoft.com/en-us/library/cc667410.aspx
2017-03-22 22:51:57 +03:00
SourceFiles="@(_MonoUtilitySource)"
DestinationFiles="@(_MonoUtilityDest)"
[mono-runtimes] Build mono-symbolicate.exe via mono Building `mono-symbolicate.exe` is fun [^0]. Commit 0147bc79 attempted to build `mono-symbolicate` by using `mono/mcs/tools/mono-symbolicate/monosymbolicate.csproj`. This didn't actually work, though, as the resulting `mono-symbolicate.exe` had an assembly reference to `Mono.Cecil.dll`, which we don't distribute for [historical reasons][1]. This resulted in commit 9b63344a, which attempted to "replicate" what mono's build system did within the constructs of an MSBuild target and `<Csc/>` task invocation. While this "worked," it was brittle: it involved reading `mono-symbolicate.exe.sources`, which can change -- necessitating yet more chanegs (commit 1b023bde) -- and even that is inadequate, as `mono-symbolicate.exe.sources` can contain *file globs*, which our "read lines into a `<TaskItem/>`" voodoo doesn't support. We want something that will more reliably build `mono-symbolicate.exe`, something maintained by the #runtime team. We *have* such a mechanism: `mono-symbolicate/Makefile`. Why can't we use that? We *couldn't* use it because, as mentioned previously, Mono's normal build mechanisms will cause `mono-symbolicate.exe` to reference `Mono.Cecil.dll`. However, we have an alternative now that we didn't have in the time of commit 9b63344a: [`remap-assembly-ref.exe`][1]. Thus a new, hopefully more maintainable approach: Use mono's build system to build `mono-symbolicate.exe`, then use `remap-assembly-ref.exe` to remap `Mono.Cecil` to `Xamarin.Android.Cecil.dll`. [1]: https://github.com/xamarin/xamarin-android/commit/5a8e48eba8cc8d05ec4fef25ac074d00606f5a69 [^0]: Note: Not actually fun.
2017-02-07 21:16:50 +03:00
/>
</Target>
[mono-runtimes] Build required mono runtimes. Add a new build-tools/mono-runtimes project to build the required Mono runtimes for Xamarin.Android app execution on Android devices. Xamarin.Android 6.0 ("cycle 6") includes Mono runtimes compiled for five architectures: * armeabi (armv5) * armeabi-v7a (armv7) * arm64-v8a (aarch64) * x86 * x86_64 An additional "host" architecture is also built to build the "monodroid" profile assemblies such as mscorlib.dll. In the interest of expediency and not requiring that *six* different runtimes be built -- which can greatly increase build times and storage requirements -- the initial xamarin-android open-source build will only build the armeabi-v7a (armv7) and "host" architectures. Additionally, the <UnzipDirectoryChildren/> task needed to be altered to use unzip(1) on OS X instead of using System.IO.Compression.ZipFile, because the ZipFile implementation included with Mono 4.4 doesn't respect file attributes such as the +x, causing all files to be extracted as 644 (rw-r--r--), resulting in an NDK toolchain which wouldn't work. To resolve this issue, execution on OS X and Linux now uses unzip(1). ~~ Adding new architectures ~~ To build Mono for a new architecture, edit build-tools/mono-runtimes/mono-runtimes.projitems and add a new @(_MonoRuntime) entry for the desired architecture. TODO: Figure out if there's a reasonable way to "opt-in" or "opt-out" of CPU architectures so that it isn't an "all or nothing" prospect.
2016-04-20 04:30:00 +03:00
<Target Name="_InstallBcl"
[mono-runtimes] Build mono if monodroid profile is missing (#39) If the mono build was aborted for whatever reason and the monodroid profile wasn't built completely (or at all) the current targets will not fail because they won't see any of the outputs missing and the _InstallBcl target will happily accept empty set for @(_Assemblies). We currently depend on a manually added full list of the monodroid profile assemblies (sans Mono.Data.Sqlite which will is built separately elsewhere and will not be built as part of the profile eventually), but we will implement a way to dynamically obtain the list of assemblies from mono.
2016-05-12 18:23:10 +03:00
Inputs="@(_BclProfileItems)"
[structure] Rework installation directory structure (#704) Context: https://github.com/xamarin/xamarin-android/pull/253#discussion_r82862993 The *intention* is that Jenkins-produced `oss-xamarin.android*.zip` artifacts be usable on Windows, so that side-by-side testing can be performed without replacing the system installation. Usage is in [UsingJenkinsBuildArtifacts.md](Documentation/UsingJenkinsBuildArtifacts). This isn't *entirely* the case. It was *apparently* the case at the time of commit 87ca2737, but things have bitrotten since. For example, following the 87ca2737 instructions would currently result in an XA0020 `Could not find monodroid` error, because `class-parse.exe` wasn't where Windows expects it to be (it was in `lib/mandroid`, not `lib/xbuild/Xamarin/Android`). This needs to be fixed. Additionally, PR #253 mentions that, for filesystem organization, it would be useful if the macOS/Linux directory structure -- `$prefix/bin`, `$prefix/lib/mandroid`, `$prefix/lib/xbuild/Xamarin/Android`, `$prefix/lib/xbuild-frameworks` -- more closely resembled the Windows directory structure of `$MSBuild` and `$ReferenceAssemblies` (as seen in `.vsix` files). This would turn macOS/Linux into using `$xa_prefix/xbuild` and `$xa_prefix/xbuild-frameworks` directories. `$prefix/bin` would only contain `xabuild`. What is currently in `$prefix/lib/mandroid` would be merged with `$xa_prefix/xbuild/Xamarin/Android`. `$xa_prefix` would `$prefix/lib/xamarin.android`. This would turn the current macOS structure: $prefix/bin/xabuild $prefix/bin/generator $prefix/bin/cross-arm $prefix/lib/mandroid/generator.exe $prefix/lib/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xbuild/Xamarin/Android/Xamarin.Android.Common.targets Into: $prefix/bin/xabuild $prefix/lib/xamarin.android/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xamarin.android/xbuild/Xamarin/Android/generator.exe $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Xamarin.Android.Common.targets $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Darwin/cross-arm Other notes: * The `bundle-*.zip` filename has been has been changed to include a *hash* of the contents of various files, in particular `build-tools\mono-runtimes.*`. This was instigated via a conversation with @kumpera about making the filename more "idiot-proof": `mono-runtimes.props` contains *compiler flags*, and if any of those change, then *logically* the bundle should differ as well, as the mono runtimes may differ in significant ways. In theory, the `-vXX` version should be used to track this, but this is a manual change, easy to overlook. The new `-hHASH` part of the filename should be more automagic. The new `<HashFileContents/>` task in `xa-prep-tasks.dll` is responsible for creating the hash value. * `Configuration.Java.Interop.Override.props` was moved into `build-tools/scripts`, because that would cleanup the root directory a bit. * OS-specific binaries are now placed into `$prefix/lib/xamarin.android/xbuild/Xamarin/Android/$(HostOS)`. On a theoretical plus side, this means that the same build directory can contain OS-specific binaries for multiple operating systems. (I don't know if anyone shares a build tree between e.g. macOS and Linux, but if anyone *does*...) Unfortunately this requires a workaround for an `xbuild` bug: `%(_LlvmRuntime.InstallPath)` and `%(_MonoCrossRuntime.InstallPath)` *cannot* use MSBuild properties, e.g. `<InstallPath>$(HostOS)/</InstallPath>` doesn't work as desired; it's instead treated literally. Special-case `%(InstallPath)` until we fully migrate to MSBuild. * `$(MonoAndroidToolsDirectory)` should be considered *dead*, along with `$(MonoAndroidBinDirectory)`, as these should now *always* be the same directory as where `Xamarin.Android.Build.Tasks.dll` is located, or a `$(HostOS)` sub-directory. * `Xamarin.ProjectTools.csproj` needed to be updated to ensure that the build order was correct. * Remove all `[Obsolete]` and unreferenced members from `Xamarin.Android.Build.Utilities.dll`. There's too much in there, and it makes my head hurt trying to understand the interrelationships between it all. If it's not used, it's gone. * The changes to `src/monodroid/jni/Android.mk` are...weird. The removal of `-I$(topdir)/libmonodroid/zip`/etc. is to reduce use of non-existent paths, as `$(topdir)` isn't defined, so that's *actually* `-I/libmonodroid/zip`, which is nonsensical. So far, so good. What's *odd* is the addition of `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`: This is needed so that `external/mono/support/zlib-helper.c` exports `CreateZStream` and related symbols, otherwise we get a unit test failure in `GzipStreamTest.Compression` due to an `EntryPointNotFoundException`, because `CreateZStream` isn't exported/public. What's odd here is that I don't understand what caused this behavior to change. Previous builds exported `CreateZStream`, otherwise the tests would fail, and I don't understand how any of the other changes in this PR would be at fault, though that's certainly the most plausible explanation. Regardless, `-Ijni` *first* (due to adding `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`) is the desired behavior, so that `jni/config.h` is included, thus ensuring that `MONO_API` has the required definition when building `zlib-helper.c`.
2017-07-28 16:36:32 +03:00
Outputs="@(_BclInstalledItem);$(_BclFrameworkDir)RedistList\FrameworkList.xml">
Copy FSharp support assemblies from nuget (#21) Instead of building F# from sources as we've been doing so far, we'll now use a nuget to get the binaries we require. This makes the build time much shorter.
2016-05-06 00:07:05 +03:00
<MakeDir Directories="$(_BclFrameworkDir)" />
[structure] Rework installation directory structure (#704) Context: https://github.com/xamarin/xamarin-android/pull/253#discussion_r82862993 The *intention* is that Jenkins-produced `oss-xamarin.android*.zip` artifacts be usable on Windows, so that side-by-side testing can be performed without replacing the system installation. Usage is in [UsingJenkinsBuildArtifacts.md](Documentation/UsingJenkinsBuildArtifacts). This isn't *entirely* the case. It was *apparently* the case at the time of commit 87ca2737, but things have bitrotten since. For example, following the 87ca2737 instructions would currently result in an XA0020 `Could not find monodroid` error, because `class-parse.exe` wasn't where Windows expects it to be (it was in `lib/mandroid`, not `lib/xbuild/Xamarin/Android`). This needs to be fixed. Additionally, PR #253 mentions that, for filesystem organization, it would be useful if the macOS/Linux directory structure -- `$prefix/bin`, `$prefix/lib/mandroid`, `$prefix/lib/xbuild/Xamarin/Android`, `$prefix/lib/xbuild-frameworks` -- more closely resembled the Windows directory structure of `$MSBuild` and `$ReferenceAssemblies` (as seen in `.vsix` files). This would turn macOS/Linux into using `$xa_prefix/xbuild` and `$xa_prefix/xbuild-frameworks` directories. `$prefix/bin` would only contain `xabuild`. What is currently in `$prefix/lib/mandroid` would be merged with `$xa_prefix/xbuild/Xamarin/Android`. `$xa_prefix` would `$prefix/lib/xamarin.android`. This would turn the current macOS structure: $prefix/bin/xabuild $prefix/bin/generator $prefix/bin/cross-arm $prefix/lib/mandroid/generator.exe $prefix/lib/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xbuild/Xamarin/Android/Xamarin.Android.Common.targets Into: $prefix/bin/xabuild $prefix/lib/xamarin.android/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xamarin.android/xbuild/Xamarin/Android/generator.exe $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Xamarin.Android.Common.targets $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Darwin/cross-arm Other notes: * The `bundle-*.zip` filename has been has been changed to include a *hash* of the contents of various files, in particular `build-tools\mono-runtimes.*`. This was instigated via a conversation with @kumpera about making the filename more "idiot-proof": `mono-runtimes.props` contains *compiler flags*, and if any of those change, then *logically* the bundle should differ as well, as the mono runtimes may differ in significant ways. In theory, the `-vXX` version should be used to track this, but this is a manual change, easy to overlook. The new `-hHASH` part of the filename should be more automagic. The new `<HashFileContents/>` task in `xa-prep-tasks.dll` is responsible for creating the hash value. * `Configuration.Java.Interop.Override.props` was moved into `build-tools/scripts`, because that would cleanup the root directory a bit. * OS-specific binaries are now placed into `$prefix/lib/xamarin.android/xbuild/Xamarin/Android/$(HostOS)`. On a theoretical plus side, this means that the same build directory can contain OS-specific binaries for multiple operating systems. (I don't know if anyone shares a build tree between e.g. macOS and Linux, but if anyone *does*...) Unfortunately this requires a workaround for an `xbuild` bug: `%(_LlvmRuntime.InstallPath)` and `%(_MonoCrossRuntime.InstallPath)` *cannot* use MSBuild properties, e.g. `<InstallPath>$(HostOS)/</InstallPath>` doesn't work as desired; it's instead treated literally. Special-case `%(InstallPath)` until we fully migrate to MSBuild. * `$(MonoAndroidToolsDirectory)` should be considered *dead*, along with `$(MonoAndroidBinDirectory)`, as these should now *always* be the same directory as where `Xamarin.Android.Build.Tasks.dll` is located, or a `$(HostOS)` sub-directory. * `Xamarin.ProjectTools.csproj` needed to be updated to ensure that the build order was correct. * Remove all `[Obsolete]` and unreferenced members from `Xamarin.Android.Build.Utilities.dll`. There's too much in there, and it makes my head hurt trying to understand the interrelationships between it all. If it's not used, it's gone. * The changes to `src/monodroid/jni/Android.mk` are...weird. The removal of `-I$(topdir)/libmonodroid/zip`/etc. is to reduce use of non-existent paths, as `$(topdir)` isn't defined, so that's *actually* `-I/libmonodroid/zip`, which is nonsensical. So far, so good. What's *odd* is the addition of `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`: This is needed so that `external/mono/support/zlib-helper.c` exports `CreateZStream` and related symbols, otherwise we get a unit test failure in `GzipStreamTest.Compression` due to an `EntryPointNotFoundException`, because `CreateZStream` isn't exported/public. What's odd here is that I don't understand what caused this behavior to change. Previous builds exported `CreateZStream`, otherwise the tests would fail, and I don't understand how any of the other changes in this PR would be at fault, though that's certainly the most plausible explanation. Regardless, `-Ijni` *first* (due to adding `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`) is the desired behavior, so that `jni/config.h` is included, thus ensuring that `MONO_API` has the required definition when building `zlib-helper.c`.
2017-07-28 16:36:32 +03:00
<MakeDir Directories="$(_BclFrameworkDir)RedistList" />
<MakeDir Directories="$(_BclFrameworkDir)Facades" />
[mono-runtimes] Build required mono runtimes. Add a new build-tools/mono-runtimes project to build the required Mono runtimes for Xamarin.Android app execution on Android devices. Xamarin.Android 6.0 ("cycle 6") includes Mono runtimes compiled for five architectures: * armeabi (armv5) * armeabi-v7a (armv7) * arm64-v8a (aarch64) * x86 * x86_64 An additional "host" architecture is also built to build the "monodroid" profile assemblies such as mscorlib.dll. In the interest of expediency and not requiring that *six* different runtimes be built -- which can greatly increase build times and storage requirements -- the initial xamarin-android open-source build will only build the armeabi-v7a (armv7) and "host" architectures. Additionally, the <UnzipDirectoryChildren/> task needed to be altered to use unzip(1) on OS X instead of using System.IO.Compression.ZipFile, because the ZipFile implementation included with Mono 4.4 doesn't respect file attributes such as the +x, causing all files to be extracted as 644 (rw-r--r--), resulting in an NDK toolchain which wouldn't work. To resolve this issue, execution on OS X and Linux now uses unzip(1). ~~ Adding new architectures ~~ To build Mono for a new architecture, edit build-tools/mono-runtimes/mono-runtimes.projitems and add a new @(_MonoRuntime) entry for the desired architecture. TODO: Figure out if there's a reasonable way to "opt-in" or "opt-out" of CPU architectures so that it isn't an "all or nothing" prospect.
2016-04-20 04:30:00 +03:00
<ItemGroup>
Copy FSharp support assemblies from nuget (#21) Instead of building F# from sources as we've been doing so far, we'll now use a nuget to get the binaries we require. This makes the build time much shorter.
2016-05-06 00:07:05 +03:00
<_PackageConfigFiles Include="$(_SourceTopDir)\src\Xamarin.Android.Build.Tasks\packages.config" />
</ItemGroup>
<ItemGroup>
[mono-runtimes] Build mono if monodroid profile is missing (#39) If the mono build was aborted for whatever reason and the monodroid profile wasn't built completely (or at all) the current targets will not fail because they won't see any of the outputs missing and the _InstallBcl target will happily accept empty set for @(_Assemblies). We currently depend on a manually added full list of the monodroid profile assemblies (sans Mono.Data.Sqlite which will is built separately elsewhere and will not be built as part of the profile eventually), but we will implement a way to dynamically obtain the list of assemblies from mono.
2016-05-12 18:23:10 +03:00
<_Facades Include="$(_MonoProfileDir)\Facades\*.dll" />
[mono-runtimes] Build required mono runtimes. Add a new build-tools/mono-runtimes project to build the required Mono runtimes for Xamarin.Android app execution on Android devices. Xamarin.Android 6.0 ("cycle 6") includes Mono runtimes compiled for five architectures: * armeabi (armv5) * armeabi-v7a (armv7) * arm64-v8a (aarch64) * x86 * x86_64 An additional "host" architecture is also built to build the "monodroid" profile assemblies such as mscorlib.dll. In the interest of expediency and not requiring that *six* different runtimes be built -- which can greatly increase build times and storage requirements -- the initial xamarin-android open-source build will only build the armeabi-v7a (armv7) and "host" architectures. Additionally, the <UnzipDirectoryChildren/> task needed to be altered to use unzip(1) on OS X instead of using System.IO.Compression.ZipFile, because the ZipFile implementation included with Mono 4.4 doesn't respect file attributes such as the +x, causing all files to be extracted as 644 (rw-r--r--), resulting in an NDK toolchain which wouldn't work. To resolve this issue, execution on OS X and Linux now uses unzip(1). ~~ Adding new architectures ~~ To build Mono for a new architecture, edit build-tools/mono-runtimes/mono-runtimes.projitems and add a new @(_MonoRuntime) entry for the desired architecture. TODO: Figure out if there's a reasonable way to "opt-in" or "opt-out" of CPU architectures so that it isn't an "all or nothing" prospect.
2016-04-20 04:30:00 +03:00
</ItemGroup>
<Copy
[bundle] Include mono's debug symbols (#513) Microsoft has (several?) [symbol servers][0], which is a repository "somewhere" which contains debug symbols, so that when a stack trace containing e.g. BCL stack frames is sent to Microsoft, there is a way to correlate that back to filename and line number source for further investigation. Xamarin.Android has not partaken in this mechanism, in large part because we did most of our building on Linux and macOS machines which generate `.mdb` debug symbols, while symbol servers require `.pdb`. That constraint is now behind us; with 76be1fb1 and the use of Mono 4.9 for builds, our builds now generate Portable `.pdb` files, which *should* (hopefully) be usable on Microsoft's symbol servers. However, to transmit them to Microsoft's servers, we need to actually *retain* them. At present, they're not: because of `bundle*.zip` use, the `mscorlib.dll` that we include in the final product may have been built much earlier than the rest of the product, and we don't have any other mechanism to retain the `mscorlib.pdb` file. Let's change that: update `mono-runtimes.targets` so that the created debug symbol files are included in the produced `bundle*.zip`, and bump the corresponding `$(XABundleFileName)` version number. This will ensure that we retain the debug symbols for our BCL assemblies, allowing us to actually have something to upload. [0]: https://msdn.microsoft.com/en-us/library/cc667410.aspx
2017-03-22 22:51:57 +03:00
SourceFiles="@(_BclProfileItems)"
DestinationFiles="@(_BclInstalledItem)"
[mono-runtimes] Build required mono runtimes. Add a new build-tools/mono-runtimes project to build the required Mono runtimes for Xamarin.Android app execution on Android devices. Xamarin.Android 6.0 ("cycle 6") includes Mono runtimes compiled for five architectures: * armeabi (armv5) * armeabi-v7a (armv7) * arm64-v8a (aarch64) * x86 * x86_64 An additional "host" architecture is also built to build the "monodroid" profile assemblies such as mscorlib.dll. In the interest of expediency and not requiring that *six* different runtimes be built -- which can greatly increase build times and storage requirements -- the initial xamarin-android open-source build will only build the armeabi-v7a (armv7) and "host" architectures. Additionally, the <UnzipDirectoryChildren/> task needed to be altered to use unzip(1) on OS X instead of using System.IO.Compression.ZipFile, because the ZipFile implementation included with Mono 4.4 doesn't respect file attributes such as the +x, causing all files to be extracted as 644 (rw-r--r--), resulting in an NDK toolchain which wouldn't work. To resolve this issue, execution on OS X and Linux now uses unzip(1). ~~ Adding new architectures ~~ To build Mono for a new architecture, edit build-tools/mono-runtimes/mono-runtimes.projitems and add a new @(_MonoRuntime) entry for the desired architecture. TODO: Figure out if there's a reasonable way to "opt-in" or "opt-out" of CPU architectures so that it isn't an "all or nothing" prospect.
2016-04-20 04:30:00 +03:00
/>
<Copy
SourceFiles="@(_Facades)"
[structure] Rework installation directory structure (#704) Context: https://github.com/xamarin/xamarin-android/pull/253#discussion_r82862993 The *intention* is that Jenkins-produced `oss-xamarin.android*.zip` artifacts be usable on Windows, so that side-by-side testing can be performed without replacing the system installation. Usage is in [UsingJenkinsBuildArtifacts.md](Documentation/UsingJenkinsBuildArtifacts). This isn't *entirely* the case. It was *apparently* the case at the time of commit 87ca2737, but things have bitrotten since. For example, following the 87ca2737 instructions would currently result in an XA0020 `Could not find monodroid` error, because `class-parse.exe` wasn't where Windows expects it to be (it was in `lib/mandroid`, not `lib/xbuild/Xamarin/Android`). This needs to be fixed. Additionally, PR #253 mentions that, for filesystem organization, it would be useful if the macOS/Linux directory structure -- `$prefix/bin`, `$prefix/lib/mandroid`, `$prefix/lib/xbuild/Xamarin/Android`, `$prefix/lib/xbuild-frameworks` -- more closely resembled the Windows directory structure of `$MSBuild` and `$ReferenceAssemblies` (as seen in `.vsix` files). This would turn macOS/Linux into using `$xa_prefix/xbuild` and `$xa_prefix/xbuild-frameworks` directories. `$prefix/bin` would only contain `xabuild`. What is currently in `$prefix/lib/mandroid` would be merged with `$xa_prefix/xbuild/Xamarin/Android`. `$xa_prefix` would `$prefix/lib/xamarin.android`. This would turn the current macOS structure: $prefix/bin/xabuild $prefix/bin/generator $prefix/bin/cross-arm $prefix/lib/mandroid/generator.exe $prefix/lib/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xbuild/Xamarin/Android/Xamarin.Android.Common.targets Into: $prefix/bin/xabuild $prefix/lib/xamarin.android/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xamarin.android/xbuild/Xamarin/Android/generator.exe $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Xamarin.Android.Common.targets $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Darwin/cross-arm Other notes: * The `bundle-*.zip` filename has been has been changed to include a *hash* of the contents of various files, in particular `build-tools\mono-runtimes.*`. This was instigated via a conversation with @kumpera about making the filename more "idiot-proof": `mono-runtimes.props` contains *compiler flags*, and if any of those change, then *logically* the bundle should differ as well, as the mono runtimes may differ in significant ways. In theory, the `-vXX` version should be used to track this, but this is a manual change, easy to overlook. The new `-hHASH` part of the filename should be more automagic. The new `<HashFileContents/>` task in `xa-prep-tasks.dll` is responsible for creating the hash value. * `Configuration.Java.Interop.Override.props` was moved into `build-tools/scripts`, because that would cleanup the root directory a bit. * OS-specific binaries are now placed into `$prefix/lib/xamarin.android/xbuild/Xamarin/Android/$(HostOS)`. On a theoretical plus side, this means that the same build directory can contain OS-specific binaries for multiple operating systems. (I don't know if anyone shares a build tree between e.g. macOS and Linux, but if anyone *does*...) Unfortunately this requires a workaround for an `xbuild` bug: `%(_LlvmRuntime.InstallPath)` and `%(_MonoCrossRuntime.InstallPath)` *cannot* use MSBuild properties, e.g. `<InstallPath>$(HostOS)/</InstallPath>` doesn't work as desired; it's instead treated literally. Special-case `%(InstallPath)` until we fully migrate to MSBuild. * `$(MonoAndroidToolsDirectory)` should be considered *dead*, along with `$(MonoAndroidBinDirectory)`, as these should now *always* be the same directory as where `Xamarin.Android.Build.Tasks.dll` is located, or a `$(HostOS)` sub-directory. * `Xamarin.ProjectTools.csproj` needed to be updated to ensure that the build order was correct. * Remove all `[Obsolete]` and unreferenced members from `Xamarin.Android.Build.Utilities.dll`. There's too much in there, and it makes my head hurt trying to understand the interrelationships between it all. If it's not used, it's gone. * The changes to `src/monodroid/jni/Android.mk` are...weird. The removal of `-I$(topdir)/libmonodroid/zip`/etc. is to reduce use of non-existent paths, as `$(topdir)` isn't defined, so that's *actually* `-I/libmonodroid/zip`, which is nonsensical. So far, so good. What's *odd* is the addition of `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`: This is needed so that `external/mono/support/zlib-helper.c` exports `CreateZStream` and related symbols, otherwise we get a unit test failure in `GzipStreamTest.Compression` due to an `EntryPointNotFoundException`, because `CreateZStream` isn't exported/public. What's odd here is that I don't understand what caused this behavior to change. Previous builds exported `CreateZStream`, otherwise the tests would fail, and I don't understand how any of the other changes in this PR would be at fault, though that's certainly the most plausible explanation. Regardless, `-Ijni` *first* (due to adding `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`) is the desired behavior, so that `jni/config.h` is included, thus ensuring that `MONO_API` has the required definition when building `zlib-helper.c`.
2017-07-28 16:36:32 +03:00
DestinationFolder="$(_BclFrameworkDir)Facades"
Copy FSharp support assemblies from nuget (#21) Instead of building F# from sources as we've been doing so far, we'll now use a nuget to get the binaries we require. This makes the build time much shorter.
2016-05-06 00:07:05 +03:00
/>
[mono-runtimes] Build required mono runtimes. Add a new build-tools/mono-runtimes project to build the required Mono runtimes for Xamarin.Android app execution on Android devices. Xamarin.Android 6.0 ("cycle 6") includes Mono runtimes compiled for five architectures: * armeabi (armv5) * armeabi-v7a (armv7) * arm64-v8a (aarch64) * x86 * x86_64 An additional "host" architecture is also built to build the "monodroid" profile assemblies such as mscorlib.dll. In the interest of expediency and not requiring that *six* different runtimes be built -- which can greatly increase build times and storage requirements -- the initial xamarin-android open-source build will only build the armeabi-v7a (armv7) and "host" architectures. Additionally, the <UnzipDirectoryChildren/> task needed to be altered to use unzip(1) on OS X instead of using System.IO.Compression.ZipFile, because the ZipFile implementation included with Mono 4.4 doesn't respect file attributes such as the +x, causing all files to be extracted as 644 (rw-r--r--), resulting in an NDK toolchain which wouldn't work. To resolve this issue, execution on OS X and Linux now uses unzip(1). ~~ Adding new architectures ~~ To build Mono for a new architecture, edit build-tools/mono-runtimes/mono-runtimes.projitems and add a new @(_MonoRuntime) entry for the desired architecture. TODO: Figure out if there's a reasonable way to "opt-in" or "opt-out" of CPU architectures so that it isn't an "all or nothing" prospect.
2016-04-20 04:30:00 +03:00
<Touch
[android-toolchains, mono-runtimes] Build armeabi, arm64-v8a, x86, x86_64 (#40) Commit 38dbfcaf mentions that commercial Xamarin.Android 6.0 provides Mono for five architectures, but "[i]n the interest of expediency" only adds support to build *one* architecture: armeabi-v7a (32-bit ARM v7). It's time to fix that: add build system support for armeabi, arm64-v8a, x86, and x86_64. *However*, it takes *time* to build all those ABIs: on a 2013 6-core Mac Pro, it takes ~29 minutes to build all five of those ABIs plus the "host" ABI (for BCL assemblies), which is presumably 29 minutes that very few people want to spend, and will be even longer in a variety of build environments (virtual machines, slower hardware, etc.). Which means we don't want to require that they all be built. To better support this, add a new `$(AndroidSupportedAbis)` MSBuild property which contains a comma-separated list of ABIs to support. This allows manually overriding the ABIs on the command-line: # build everything! $ xbuild /p:AndroidSupportedAbis=armeabi,armeabi-v7a,arm64-v8a,x86,x86_64 or setting a value within `Configuration.Override.props`: <PropertyGroup> <!-- only build x86 --> <AndroidSupportedAbis>x86,x86_64</AndroidSupportedAbis> </PropertyGroup> The *default* continues to be just armeabi-v7a.
2016-05-13 22:50:23 +03:00
Files="@(_BclInstalledItem)"
[mono-runtimes] Build required mono runtimes. Add a new build-tools/mono-runtimes project to build the required Mono runtimes for Xamarin.Android app execution on Android devices. Xamarin.Android 6.0 ("cycle 6") includes Mono runtimes compiled for five architectures: * armeabi (armv5) * armeabi-v7a (armv7) * arm64-v8a (aarch64) * x86 * x86_64 An additional "host" architecture is also built to build the "monodroid" profile assemblies such as mscorlib.dll. In the interest of expediency and not requiring that *six* different runtimes be built -- which can greatly increase build times and storage requirements -- the initial xamarin-android open-source build will only build the armeabi-v7a (armv7) and "host" architectures. Additionally, the <UnzipDirectoryChildren/> task needed to be altered to use unzip(1) on OS X instead of using System.IO.Compression.ZipFile, because the ZipFile implementation included with Mono 4.4 doesn't respect file attributes such as the +x, causing all files to be extracted as 644 (rw-r--r--), resulting in an NDK toolchain which wouldn't work. To resolve this issue, execution on OS X and Linux now uses unzip(1). ~~ Adding new architectures ~~ To build Mono for a new architecture, edit build-tools/mono-runtimes/mono-runtimes.projitems and add a new @(_MonoRuntime) entry for the desired architecture. TODO: Figure out if there's a reasonable way to "opt-in" or "opt-out" of CPU architectures so that it isn't an "all or nothing" prospect.
2016-04-20 04:30:00 +03:00
/>
[xabuild] *Properly* package the .apk. Commit e20863ea adds an `xabuild` script, which allows us to build Android .apk application packages. And lo, it was good! *Unfortunately*, while a .apk was *generated*, the app on-device would immediately crash with a bizarre error. Further investigation pointed to the actual problem: While xamarin-android was generating the .apk, the framework assemblies that were bundled into the app were coming from the local system Xamarin.Android install. (Oops.) Two things are required in order to cause the correct assemblies to be bundled into the app.apk: 1. MSBuild/xbuild require that the bin/$(Configuration)/lib/xbuild-frameworks/MonoAndroid/v* directories contain a `RedistList/FrameworkList.xml` file, otherwise the directory isn't recognized as a valid location for MSBuild frameworks. Since these files were missing, the built assemblies weren't considered for use when compiling the app. Update the build system to generate these files. 2. Xamarin.Android.Common.targets was overriding the $(CscToolExe) and $(CscToolPath) MSBuild properties when running on OS X to refer to an `smcs` script, which (i) doesn't exist in the xamarin-android repo, so (ii) the system-installed smcs script was used instead. The smcs script is no longer necessary, and hasn't actually been required for *years* (behold! laziness!), so just remove these property overrides. With these two changes, `xabuild` is not only able to generate an Android .apk file, but the resulting .apk will actually run on-device! Woo-hoo!
2016-04-26 16:28:37 +03:00
<ItemGroup>
<FrameworkList Include="&lt;FileList Redist=&quot;MonoAndroid&quot; Name=&quot;Xamarin.Android Base Class Libraries&quot;&gt;" />
<FrameworkList Include="&lt;/FileList&gt;" />
</ItemGroup>
<WriteLinesToFile
[structure] Rework installation directory structure (#704) Context: https://github.com/xamarin/xamarin-android/pull/253#discussion_r82862993 The *intention* is that Jenkins-produced `oss-xamarin.android*.zip` artifacts be usable on Windows, so that side-by-side testing can be performed without replacing the system installation. Usage is in [UsingJenkinsBuildArtifacts.md](Documentation/UsingJenkinsBuildArtifacts). This isn't *entirely* the case. It was *apparently* the case at the time of commit 87ca2737, but things have bitrotten since. For example, following the 87ca2737 instructions would currently result in an XA0020 `Could not find monodroid` error, because `class-parse.exe` wasn't where Windows expects it to be (it was in `lib/mandroid`, not `lib/xbuild/Xamarin/Android`). This needs to be fixed. Additionally, PR #253 mentions that, for filesystem organization, it would be useful if the macOS/Linux directory structure -- `$prefix/bin`, `$prefix/lib/mandroid`, `$prefix/lib/xbuild/Xamarin/Android`, `$prefix/lib/xbuild-frameworks` -- more closely resembled the Windows directory structure of `$MSBuild` and `$ReferenceAssemblies` (as seen in `.vsix` files). This would turn macOS/Linux into using `$xa_prefix/xbuild` and `$xa_prefix/xbuild-frameworks` directories. `$prefix/bin` would only contain `xabuild`. What is currently in `$prefix/lib/mandroid` would be merged with `$xa_prefix/xbuild/Xamarin/Android`. `$xa_prefix` would `$prefix/lib/xamarin.android`. This would turn the current macOS structure: $prefix/bin/xabuild $prefix/bin/generator $prefix/bin/cross-arm $prefix/lib/mandroid/generator.exe $prefix/lib/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xbuild/Xamarin/Android/Xamarin.Android.Common.targets Into: $prefix/bin/xabuild $prefix/lib/xamarin.android/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xamarin.android/xbuild/Xamarin/Android/generator.exe $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Xamarin.Android.Common.targets $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Darwin/cross-arm Other notes: * The `bundle-*.zip` filename has been has been changed to include a *hash* of the contents of various files, in particular `build-tools\mono-runtimes.*`. This was instigated via a conversation with @kumpera about making the filename more "idiot-proof": `mono-runtimes.props` contains *compiler flags*, and if any of those change, then *logically* the bundle should differ as well, as the mono runtimes may differ in significant ways. In theory, the `-vXX` version should be used to track this, but this is a manual change, easy to overlook. The new `-hHASH` part of the filename should be more automagic. The new `<HashFileContents/>` task in `xa-prep-tasks.dll` is responsible for creating the hash value. * `Configuration.Java.Interop.Override.props` was moved into `build-tools/scripts`, because that would cleanup the root directory a bit. * OS-specific binaries are now placed into `$prefix/lib/xamarin.android/xbuild/Xamarin/Android/$(HostOS)`. On a theoretical plus side, this means that the same build directory can contain OS-specific binaries for multiple operating systems. (I don't know if anyone shares a build tree between e.g. macOS and Linux, but if anyone *does*...) Unfortunately this requires a workaround for an `xbuild` bug: `%(_LlvmRuntime.InstallPath)` and `%(_MonoCrossRuntime.InstallPath)` *cannot* use MSBuild properties, e.g. `<InstallPath>$(HostOS)/</InstallPath>` doesn't work as desired; it's instead treated literally. Special-case `%(InstallPath)` until we fully migrate to MSBuild. * `$(MonoAndroidToolsDirectory)` should be considered *dead*, along with `$(MonoAndroidBinDirectory)`, as these should now *always* be the same directory as where `Xamarin.Android.Build.Tasks.dll` is located, or a `$(HostOS)` sub-directory. * `Xamarin.ProjectTools.csproj` needed to be updated to ensure that the build order was correct. * Remove all `[Obsolete]` and unreferenced members from `Xamarin.Android.Build.Utilities.dll`. There's too much in there, and it makes my head hurt trying to understand the interrelationships between it all. If it's not used, it's gone. * The changes to `src/monodroid/jni/Android.mk` are...weird. The removal of `-I$(topdir)/libmonodroid/zip`/etc. is to reduce use of non-existent paths, as `$(topdir)` isn't defined, so that's *actually* `-I/libmonodroid/zip`, which is nonsensical. So far, so good. What's *odd* is the addition of `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`: This is needed so that `external/mono/support/zlib-helper.c` exports `CreateZStream` and related symbols, otherwise we get a unit test failure in `GzipStreamTest.Compression` due to an `EntryPointNotFoundException`, because `CreateZStream` isn't exported/public. What's odd here is that I don't understand what caused this behavior to change. Previous builds exported `CreateZStream`, otherwise the tests would fail, and I don't understand how any of the other changes in this PR would be at fault, though that's certainly the most plausible explanation. Regardless, `-Ijni` *first* (due to adding `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`) is the desired behavior, so that `jni/config.h` is included, thus ensuring that `MONO_API` has the required definition when building `zlib-helper.c`.
2017-07-28 16:36:32 +03:00
File="$(_BclFrameworkDir)RedistList\FrameworkList.xml"
[xabuild] *Properly* package the .apk. Commit e20863ea adds an `xabuild` script, which allows us to build Android .apk application packages. And lo, it was good! *Unfortunately*, while a .apk was *generated*, the app on-device would immediately crash with a bizarre error. Further investigation pointed to the actual problem: While xamarin-android was generating the .apk, the framework assemblies that were bundled into the app were coming from the local system Xamarin.Android install. (Oops.) Two things are required in order to cause the correct assemblies to be bundled into the app.apk: 1. MSBuild/xbuild require that the bin/$(Configuration)/lib/xbuild-frameworks/MonoAndroid/v* directories contain a `RedistList/FrameworkList.xml` file, otherwise the directory isn't recognized as a valid location for MSBuild frameworks. Since these files were missing, the built assemblies weren't considered for use when compiling the app. Update the build system to generate these files. 2. Xamarin.Android.Common.targets was overriding the $(CscToolExe) and $(CscToolPath) MSBuild properties when running on OS X to refer to an `smcs` script, which (i) doesn't exist in the xamarin-android repo, so (ii) the system-installed smcs script was used instead. The smcs script is no longer necessary, and hasn't actually been required for *years* (behold! laziness!), so just remove these property overrides. With these two changes, `xabuild` is not only able to generate an Android .apk file, but the resulting .apk will actually run on-device! Woo-hoo!
2016-04-26 16:28:37 +03:00
Lines="@(FrameworkList)"
Overwrite="True"
/>
[mono-runtimes] Build required mono runtimes. Add a new build-tools/mono-runtimes project to build the required Mono runtimes for Xamarin.Android app execution on Android devices. Xamarin.Android 6.0 ("cycle 6") includes Mono runtimes compiled for five architectures: * armeabi (armv5) * armeabi-v7a (armv7) * arm64-v8a (aarch64) * x86 * x86_64 An additional "host" architecture is also built to build the "monodroid" profile assemblies such as mscorlib.dll. In the interest of expediency and not requiring that *six* different runtimes be built -- which can greatly increase build times and storage requirements -- the initial xamarin-android open-source build will only build the armeabi-v7a (armv7) and "host" architectures. Additionally, the <UnzipDirectoryChildren/> task needed to be altered to use unzip(1) on OS X instead of using System.IO.Compression.ZipFile, because the ZipFile implementation included with Mono 4.4 doesn't respect file attributes such as the +x, causing all files to be extracted as 644 (rw-r--r--), resulting in an NDK toolchain which wouldn't work. To resolve this issue, execution on OS X and Linux now uses unzip(1). ~~ Adding new architectures ~~ To build Mono for a new architecture, edit build-tools/mono-runtimes/mono-runtimes.projitems and add a new @(_MonoRuntime) entry for the desired architecture. TODO: Figure out if there's a reasonable way to "opt-in" or "opt-out" of CPU architectures so that it isn't an "all or nothing" prospect.
2016-04-20 04:30:00 +03:00
</Target>
[mono-runtimes] Build AOT+LLVM cross-compilers (#125) The commit implements building of LLVM and cross-compilers to support Xamarin.Android/Mono AOT. LLVM and cross-compilers can be built for both the host platform (Linux and OS/X at the moment) as well as cross-compiled for 32-bit and 64-bit Windows platforms. Windows builds are done with MXE toolchain on OS/X and with the packaged mingw-w64 toolchain on Linux (tested on Ubuntu 16.04 ONLY). Also introducing a new set of MSBuild properties that contain information about the host system. Some of those properties (HostOS, HostCC, HostCXX for instance) have been moved from Configuration.props to better support auto-detection. A new script, build-tools/scripts/generate-os-info, is invoked as part of `make prepare` to generate file that contains the new properties. The generated file is required for the build to work and is also host-specific (it mustn't be moved between different machines) Cross compiler builds require access to a configured Mono build tree, in order to generate C structure offsets header file that is used by the AOT compilers to properly generate AOT-ed binaries. Therefore, even if a JIT target is not enabled in the configuration, enabling a cross-compiler for some target will configure Mono for that JIT target but it will NOT build it, to save time. To facilitate this, the _MonoRuntimes items defined in build-tools/mono-runtimes/mono-runtimes.projitems gain an additional metadata item called `DoBuild` which will be set to `true` if the runtime actually needs to be built, as opposed to just configured. MXE builds are disabled on Linux as mingw-w64 works just fine. A `make prepare` warning is issued for Linux hosts which have the binfmt_misc module enabled and either Wine of Mono (cli) registered as PE32/PE32+ binary interpreters. In such instance building of the Windows cross-compilers will fail because Autotools determine whether software is being cross compiled by building a test program and attempting to execute it. In normal circumstances such an attempt will fail, but with Windows cross-compilation and either Wine or Mono registered to handle the PE32 executables this attempt will succeed thus causing the cross compilation detection to fail. Currently to build cross compilers on Linux you need to generate the C structure offsets header file on OS/X and copy the resulting headers to appropriate places on Linux. The header files should be placed in build-tools/mono-runtimes/obj/Debug/cross-*/ directories. The header files are: {cross-arm,cross-arm-win}/aarch64-v8a-linux-android.h {cross-arm64,cross-arm64-win}/armv5-none-linux-androideabi.h {cross-x86,cross-x86-win}/i686-none-linux-android.h {cross-x86_64,cross-x86_64-win}/x86_64-none-linux-android.h Offsets header generation doesn't work on Linux atm because of missing support for it in the Mono utility used to generate the offsets. Hopefully this limitation will be removed in the near future and a start-to-end build of everything will be possible on Linux. It is now mandatory to run at least `make prepare-props` before Xamarin.Android can be built. The target generates the OS-specific props file which is required by the build. `make prepare` depends on the target.
2016-07-26 16:27:31 +03:00
<!-- The condition below is to work around a bug in xbuild which attempts to batch
even if there are no _MonoCrossRuntime items and the Copy task fails
-->
<Target Name="_ConfigureCrossRuntimes"
DependsOnTargets="_BuildLlvm"
Inputs="$(MonoSourceFullPath)\configure"
[mono-runtimes] Use directory separators with `$(OutputPath)`. [The build][0] [is broken][1]: xamarin-android/build-tools/mono-runtimes/mono-runtimes.targets: error : Could not find a part of the path ".../xamarin-android/xamarin-android/bin/Debuglib/xbuild-frameworks/MonoAndroid/v1.0/RedistList/FrameworkList.xml". The error is generated from the `_InstallBcl` target when writing `FrameworkList.xml`, and a close reading of the error message shows that a directory separator is missing: it should be `bin/Debug/lib`, but is instead `bin/Debuglib`. I *assume* that this is due to `<CallTarget/>` being "flakey" within xbuild (it passes the file containing the target to `<MSBuild/>`!), and *somehow* in that "flakiness" `$(OutputPath)` doesn't end with `\`, *which it should*, because `Microsoft.Common.targets` will automatically append `\` to `$(OutputPath)`. To work around this issue, update `mono-runtimes.targets` so that it *always* uses `\` after `$(OutputPath)` and `$(IntermediateOutputPath)`. Additionally, `llc.exe` was being installed into the wrong directory, preventing the bundle from being used: Target _BuildUnlessCached needs to be built as output file '../../bin/Debug/lib/mandroid/llc.exe' does not exist. `llc.exe` was instead installed into `bin/Debug/bin/llc.exe`. Bump the bundle version number to -v3 and correct `mono-runtimes.targets` so that `llc.exe` and `opt.exe` are properly installed into `$prefix/lib/mandroid`. [0]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android/49/ [1]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android/49/consoleText
2016-08-17 04:18:15 +03:00
Outputs="$(IntermediateOutputPath)\%(_MonoCrossRuntime.Identity)\Makefile;$(IntermediateOutputPath)\%(_MonoCrossRuntime.Identity)\.stamp"
[mono-runtimes] Build AOT+LLVM cross-compilers (#125) The commit implements building of LLVM and cross-compilers to support Xamarin.Android/Mono AOT. LLVM and cross-compilers can be built for both the host platform (Linux and OS/X at the moment) as well as cross-compiled for 32-bit and 64-bit Windows platforms. Windows builds are done with MXE toolchain on OS/X and with the packaged mingw-w64 toolchain on Linux (tested on Ubuntu 16.04 ONLY). Also introducing a new set of MSBuild properties that contain information about the host system. Some of those properties (HostOS, HostCC, HostCXX for instance) have been moved from Configuration.props to better support auto-detection. A new script, build-tools/scripts/generate-os-info, is invoked as part of `make prepare` to generate file that contains the new properties. The generated file is required for the build to work and is also host-specific (it mustn't be moved between different machines) Cross compiler builds require access to a configured Mono build tree, in order to generate C structure offsets header file that is used by the AOT compilers to properly generate AOT-ed binaries. Therefore, even if a JIT target is not enabled in the configuration, enabling a cross-compiler for some target will configure Mono for that JIT target but it will NOT build it, to save time. To facilitate this, the _MonoRuntimes items defined in build-tools/mono-runtimes/mono-runtimes.projitems gain an additional metadata item called `DoBuild` which will be set to `true` if the runtime actually needs to be built, as opposed to just configured. MXE builds are disabled on Linux as mingw-w64 works just fine. A `make prepare` warning is issued for Linux hosts which have the binfmt_misc module enabled and either Wine of Mono (cli) registered as PE32/PE32+ binary interpreters. In such instance building of the Windows cross-compilers will fail because Autotools determine whether software is being cross compiled by building a test program and attempting to execute it. In normal circumstances such an attempt will fail, but with Windows cross-compilation and either Wine or Mono registered to handle the PE32 executables this attempt will succeed thus causing the cross compilation detection to fail. Currently to build cross compilers on Linux you need to generate the C structure offsets header file on OS/X and copy the resulting headers to appropriate places on Linux. The header files should be placed in build-tools/mono-runtimes/obj/Debug/cross-*/ directories. The header files are: {cross-arm,cross-arm-win}/aarch64-v8a-linux-android.h {cross-arm64,cross-arm64-win}/armv5-none-linux-androideabi.h {cross-x86,cross-x86-win}/i686-none-linux-android.h {cross-x86_64,cross-x86_64-win}/x86_64-none-linux-android.h Offsets header generation doesn't work on Linux atm because of missing support for it in the Mono utility used to generate the offsets. Hopefully this limitation will be removed in the near future and a start-to-end build of everything will be possible on Linux. It is now mandatory to run at least `make prepare-props` before Xamarin.Android can be built. The target generates the OS-specific props file which is required by the build. `make prepare` depends on the target.
2016-07-26 16:27:31 +03:00
Condition=" '@(_MonoCrossRuntime)' != '' ">
[mono-runtimes] Build LLVM cross-compilers on Linux (#616) Adds support to build all of the LLVM-based cross compilers on Linux and thus makes it possible to use AOT on Linux. Windows cross compilers can be built as well, but build of libmonodroid or libzip-windows won't work because Linux doesn't use mxe and mingw packaged for Linux doesn't come with a handful of libraries required for the above builds to complete (mman, dlfcn, cmake and friends)
2017-06-01 18:36:53 +03:00
<Message Text="Configuring %(_MonoCrossRuntime.Identity) in $(IntermediateOutputPath)\%(_MonoCrossRuntime.Identity)"/>
[mono-runtimes] Build AOT+LLVM cross-compilers (#125) The commit implements building of LLVM and cross-compilers to support Xamarin.Android/Mono AOT. LLVM and cross-compilers can be built for both the host platform (Linux and OS/X at the moment) as well as cross-compiled for 32-bit and 64-bit Windows platforms. Windows builds are done with MXE toolchain on OS/X and with the packaged mingw-w64 toolchain on Linux (tested on Ubuntu 16.04 ONLY). Also introducing a new set of MSBuild properties that contain information about the host system. Some of those properties (HostOS, HostCC, HostCXX for instance) have been moved from Configuration.props to better support auto-detection. A new script, build-tools/scripts/generate-os-info, is invoked as part of `make prepare` to generate file that contains the new properties. The generated file is required for the build to work and is also host-specific (it mustn't be moved between different machines) Cross compiler builds require access to a configured Mono build tree, in order to generate C structure offsets header file that is used by the AOT compilers to properly generate AOT-ed binaries. Therefore, even if a JIT target is not enabled in the configuration, enabling a cross-compiler for some target will configure Mono for that JIT target but it will NOT build it, to save time. To facilitate this, the _MonoRuntimes items defined in build-tools/mono-runtimes/mono-runtimes.projitems gain an additional metadata item called `DoBuild` which will be set to `true` if the runtime actually needs to be built, as opposed to just configured. MXE builds are disabled on Linux as mingw-w64 works just fine. A `make prepare` warning is issued for Linux hosts which have the binfmt_misc module enabled and either Wine of Mono (cli) registered as PE32/PE32+ binary interpreters. In such instance building of the Windows cross-compilers will fail because Autotools determine whether software is being cross compiled by building a test program and attempting to execute it. In normal circumstances such an attempt will fail, but with Windows cross-compilation and either Wine or Mono registered to handle the PE32 executables this attempt will succeed thus causing the cross compilation detection to fail. Currently to build cross compilers on Linux you need to generate the C structure offsets header file on OS/X and copy the resulting headers to appropriate places on Linux. The header files should be placed in build-tools/mono-runtimes/obj/Debug/cross-*/ directories. The header files are: {cross-arm,cross-arm-win}/aarch64-v8a-linux-android.h {cross-arm64,cross-arm64-win}/armv5-none-linux-androideabi.h {cross-x86,cross-x86-win}/i686-none-linux-android.h {cross-x86_64,cross-x86_64-win}/x86_64-none-linux-android.h Offsets header generation doesn't work on Linux atm because of missing support for it in the Mono utility used to generate the offsets. Hopefully this limitation will be removed in the near future and a start-to-end build of everything will be possible on Linux. It is now mandatory to run at least `make prepare-props` before Xamarin.Android can be built. The target generates the OS-specific props file which is required by the build. `make prepare` depends on the target.
2016-07-26 16:27:31 +03:00
<MakeDir Directories="$(IntermediateOutputPath)\%(_MonoCrossRuntime.Identity)" />
<Exec
Command="%(_MonoCrossRuntime.ConfigureEnvironment) $(MonoSourceFullPath)\configure LDFLAGS=&quot;%(_MonoCrossRuntime.LdFlags)&quot; CFLAGS=&quot;%(_MonoCrossRuntime.CFlags)&quot; CXXFLAGS=&quot;%(_MonoCrossRuntime.CxxFlags)&quot; CC=&quot;%(_MonoCrossRuntime.Cc)&quot; CXX=&quot;%(_MonoCrossRuntime.Cxx)&quot; CPP=&quot;%(_MonoCrossRuntime.Cpp)&quot; CXXCPP=&quot;%(_MonoCrossRuntime.CxxCpp)&quot; LD=&quot;%(_MonoCrossRuntime.Ld)&quot; AR=&quot;%(_MonoCrossRuntime.Ar)&quot; AS=&quot;%(_MonoCrossRuntime.As)&quot; RANLIB=&quot;%(_MonoCrossRuntime.RanLib)&quot; STRIP=&quot;%(_MonoCrossRuntime.Strip)&quot; DLLTOOL=&quot;%(_MonoCrossRuntime.DllTool)&quot; OBJDUMP=&quot;%(_MonoCrossRuntime.Objdump)&quot; --cache-file=..\%(_MonoCrossRuntime.Identity).config.cache %(_MonoCrossRuntime.ConfigureFlags)"
[build] Support building *from clean* with `msbuild` (Take 2!) (#391) Commit 7343965a didn't succeed in actually building `xamarin-android` with `msbuild` on our Jenkins bot. :-( Time to take another stab at it. As an added complication, my local `msbuild` behavior (mono 4.8) differs from Jenkins' `msbuild` behavior (mono 4.4), specifically around MSB3644. [On Jenkins][msb-96], it's a warning: warning MSB3644: The reference assemblies for framework ".NETFramework,Version=v4.5" were not found. When I run locally, it's an *error*. Go figure. Fixing the local error requires that I *remove* the `$(TargetFrameworkRootPath)` MSBuild property from `Configuration.props`, added in commit 255a569b. The issue is that `$(TargetFrameworkRootPath)` is an *override*, an "absolute" override. Thus, when e.g. building `Xamarin.Android.Build.Tasks.csproj`, the `.NETFramework,Version=v4.5` framework -- and corresponding `mscorlib.dll` -- cannot be resolved, and everything breaks. (Which raises the wonderful question of whether removing `$(TargetFrameworkRootPath)` will cause the Linux build to break again, but we'll cross that bridge later...) Removing `$(TargetFrameworkRootPath)` in turn requires that I remove the explicit `@(Reference)`s to `System.Runtime` which commit 7343965a added when building under `xbuild`. This also means that we no longer need the `$(_XABuildingWithXBuild)` property. (The "history" eluded to in 7343965a? It's continuing! Argh!) Additionally, fix the logging from the `<UnzipDirectoryChildren/>` task so that it prints the *correct* task name, *not* `Unzip`. (Seeing the wrong task name in log files makes things *confusing*.) With my local `msbuild` faiklures fixed, we can turn our attention to to the [errors reported on Jenkins][msb-96]. They are variations on the same "form": # from build-tools/mono-runtimes .../external/mono/configure LDFLAGS=... ... EXEC : error : /Applications/Xcode8.1-beta1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/libtool: unknown option character `-' in: --config # from src/sqlite-xamarin .../android create project -g -n sqlitesample -p . -a MainActivity -t android-25 --package com.xamarin.sqlitesample -v 1.0.0 EXEC : error : Project folder '.' is not empty. Please consider using 'android update' instead. The issue here is that MSBuild is "better" than xbuild: xbuild only checks the process exit code for errors. The above `android create project` command has en exit code of 0, i.e. it didn't error...according to xbuild. That said, when running [with xbuild][xb-203]: Task "Exec" Executing: .../android create project -g -n sqlitesample -p . -a MainActivity -t android-25 --package com.xamarin.sqlitesample -v 1.0.0 Environment variables being passed to the tool: Error: Project folder '.' is not empty. Please consider using 'android update' instead ... Done executing task "Exec" Task "Copy" ... Rephrased: `android create project ...` is emitting an "Error: ..." *message*, but even when it emits that message the exit status is still 0 (no error). xbuild is only checking the exit status, and thus continues the build. MSBuild not only checks the exit status, but *also* checks the command *output*, looking for the string `error`. Because an "error" message is printed, MSBuild flags this as an error, halting the build. The fix? Set `Exec.IgnoreStandardErrorWarningFormat=True` on all `<Exec/>` tasks which invoke **make**(1) or `configure` within `build-tools/mono-runtimes` and `src/sqlite-xamarin`. Setting `Exec.IgnoreStandardErrorWarningFormat=True` prevents the `<Exec/>` task from scanning the task output for "error", causing `msbuild` to behave like `xbuild` and only use the process exit code for errors. This allows the build to continue. [msb-96]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android-msbuild/96/consoleText [xb-203]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android/203/consoleText
2017-01-20 15:05:02 +03:00
IgnoreStandardErrorWarningFormat="True"
[mono-runtimes] Use directory separators with `$(OutputPath)`. [The build][0] [is broken][1]: xamarin-android/build-tools/mono-runtimes/mono-runtimes.targets: error : Could not find a part of the path ".../xamarin-android/xamarin-android/bin/Debuglib/xbuild-frameworks/MonoAndroid/v1.0/RedistList/FrameworkList.xml". The error is generated from the `_InstallBcl` target when writing `FrameworkList.xml`, and a close reading of the error message shows that a directory separator is missing: it should be `bin/Debug/lib`, but is instead `bin/Debuglib`. I *assume* that this is due to `<CallTarget/>` being "flakey" within xbuild (it passes the file containing the target to `<MSBuild/>`!), and *somehow* in that "flakiness" `$(OutputPath)` doesn't end with `\`, *which it should*, because `Microsoft.Common.targets` will automatically append `\` to `$(OutputPath)`. To work around this issue, update `mono-runtimes.targets` so that it *always* uses `\` after `$(OutputPath)` and `$(IntermediateOutputPath)`. Additionally, `llc.exe` was being installed into the wrong directory, preventing the bundle from being used: Target _BuildUnlessCached needs to be built as output file '../../bin/Debug/lib/mandroid/llc.exe' does not exist. `llc.exe` was instead installed into `bin/Debug/bin/llc.exe`. Bump the bundle version number to -v3 and correct `mono-runtimes.targets` so that `llc.exe` and `opt.exe` are properly installed into `$prefix/lib/mandroid`. [0]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android/49/ [1]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android/49/consoleText
2016-08-17 04:18:15 +03:00
WorkingDirectory="$(IntermediateOutputPath)\%(_MonoCrossRuntime.Identity)"
[mono-runtimes] Build AOT+LLVM cross-compilers (#125) The commit implements building of LLVM and cross-compilers to support Xamarin.Android/Mono AOT. LLVM and cross-compilers can be built for both the host platform (Linux and OS/X at the moment) as well as cross-compiled for 32-bit and 64-bit Windows platforms. Windows builds are done with MXE toolchain on OS/X and with the packaged mingw-w64 toolchain on Linux (tested on Ubuntu 16.04 ONLY). Also introducing a new set of MSBuild properties that contain information about the host system. Some of those properties (HostOS, HostCC, HostCXX for instance) have been moved from Configuration.props to better support auto-detection. A new script, build-tools/scripts/generate-os-info, is invoked as part of `make prepare` to generate file that contains the new properties. The generated file is required for the build to work and is also host-specific (it mustn't be moved between different machines) Cross compiler builds require access to a configured Mono build tree, in order to generate C structure offsets header file that is used by the AOT compilers to properly generate AOT-ed binaries. Therefore, even if a JIT target is not enabled in the configuration, enabling a cross-compiler for some target will configure Mono for that JIT target but it will NOT build it, to save time. To facilitate this, the _MonoRuntimes items defined in build-tools/mono-runtimes/mono-runtimes.projitems gain an additional metadata item called `DoBuild` which will be set to `true` if the runtime actually needs to be built, as opposed to just configured. MXE builds are disabled on Linux as mingw-w64 works just fine. A `make prepare` warning is issued for Linux hosts which have the binfmt_misc module enabled and either Wine of Mono (cli) registered as PE32/PE32+ binary interpreters. In such instance building of the Windows cross-compilers will fail because Autotools determine whether software is being cross compiled by building a test program and attempting to execute it. In normal circumstances such an attempt will fail, but with Windows cross-compilation and either Wine or Mono registered to handle the PE32 executables this attempt will succeed thus causing the cross compilation detection to fail. Currently to build cross compilers on Linux you need to generate the C structure offsets header file on OS/X and copy the resulting headers to appropriate places on Linux. The header files should be placed in build-tools/mono-runtimes/obj/Debug/cross-*/ directories. The header files are: {cross-arm,cross-arm-win}/aarch64-v8a-linux-android.h {cross-arm64,cross-arm64-win}/armv5-none-linux-androideabi.h {cross-x86,cross-x86-win}/i686-none-linux-android.h {cross-x86_64,cross-x86_64-win}/x86_64-none-linux-android.h Offsets header generation doesn't work on Linux atm because of missing support for it in the Mono utility used to generate the offsets. Hopefully this limitation will be removed in the near future and a start-to-end build of everything will be possible on Linux. It is now mandatory to run at least `make prepare-props` before Xamarin.Android can be built. The target generates the OS-specific props file which is required by the build. `make prepare` depends on the target.
2016-07-26 16:27:31 +03:00
/>
<Touch
[mono-runtimes] Use directory separators with `$(OutputPath)`. [The build][0] [is broken][1]: xamarin-android/build-tools/mono-runtimes/mono-runtimes.targets: error : Could not find a part of the path ".../xamarin-android/xamarin-android/bin/Debuglib/xbuild-frameworks/MonoAndroid/v1.0/RedistList/FrameworkList.xml". The error is generated from the `_InstallBcl` target when writing `FrameworkList.xml`, and a close reading of the error message shows that a directory separator is missing: it should be `bin/Debug/lib`, but is instead `bin/Debuglib`. I *assume* that this is due to `<CallTarget/>` being "flakey" within xbuild (it passes the file containing the target to `<MSBuild/>`!), and *somehow* in that "flakiness" `$(OutputPath)` doesn't end with `\`, *which it should*, because `Microsoft.Common.targets` will automatically append `\` to `$(OutputPath)`. To work around this issue, update `mono-runtimes.targets` so that it *always* uses `\` after `$(OutputPath)` and `$(IntermediateOutputPath)`. Additionally, `llc.exe` was being installed into the wrong directory, preventing the bundle from being used: Target _BuildUnlessCached needs to be built as output file '../../bin/Debug/lib/mandroid/llc.exe' does not exist. `llc.exe` was instead installed into `bin/Debug/bin/llc.exe`. Bump the bundle version number to -v3 and correct `mono-runtimes.targets` so that `llc.exe` and `opt.exe` are properly installed into `$prefix/lib/mandroid`. [0]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android/49/ [1]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android/49/consoleText
2016-08-17 04:18:15 +03:00
Files="$(IntermediateOutputPath)\%(_MonoCrossRuntime.Identity)\.stamp"
[mono-runtimes] Build AOT+LLVM cross-compilers (#125) The commit implements building of LLVM and cross-compilers to support Xamarin.Android/Mono AOT. LLVM and cross-compilers can be built for both the host platform (Linux and OS/X at the moment) as well as cross-compiled for 32-bit and 64-bit Windows platforms. Windows builds are done with MXE toolchain on OS/X and with the packaged mingw-w64 toolchain on Linux (tested on Ubuntu 16.04 ONLY). Also introducing a new set of MSBuild properties that contain information about the host system. Some of those properties (HostOS, HostCC, HostCXX for instance) have been moved from Configuration.props to better support auto-detection. A new script, build-tools/scripts/generate-os-info, is invoked as part of `make prepare` to generate file that contains the new properties. The generated file is required for the build to work and is also host-specific (it mustn't be moved between different machines) Cross compiler builds require access to a configured Mono build tree, in order to generate C structure offsets header file that is used by the AOT compilers to properly generate AOT-ed binaries. Therefore, even if a JIT target is not enabled in the configuration, enabling a cross-compiler for some target will configure Mono for that JIT target but it will NOT build it, to save time. To facilitate this, the _MonoRuntimes items defined in build-tools/mono-runtimes/mono-runtimes.projitems gain an additional metadata item called `DoBuild` which will be set to `true` if the runtime actually needs to be built, as opposed to just configured. MXE builds are disabled on Linux as mingw-w64 works just fine. A `make prepare` warning is issued for Linux hosts which have the binfmt_misc module enabled and either Wine of Mono (cli) registered as PE32/PE32+ binary interpreters. In such instance building of the Windows cross-compilers will fail because Autotools determine whether software is being cross compiled by building a test program and attempting to execute it. In normal circumstances such an attempt will fail, but with Windows cross-compilation and either Wine or Mono registered to handle the PE32 executables this attempt will succeed thus causing the cross compilation detection to fail. Currently to build cross compilers on Linux you need to generate the C structure offsets header file on OS/X and copy the resulting headers to appropriate places on Linux. The header files should be placed in build-tools/mono-runtimes/obj/Debug/cross-*/ directories. The header files are: {cross-arm,cross-arm-win}/aarch64-v8a-linux-android.h {cross-arm64,cross-arm64-win}/armv5-none-linux-androideabi.h {cross-x86,cross-x86-win}/i686-none-linux-android.h {cross-x86_64,cross-x86_64-win}/x86_64-none-linux-android.h Offsets header generation doesn't work on Linux atm because of missing support for it in the Mono utility used to generate the offsets. Hopefully this limitation will be removed in the near future and a start-to-end build of everything will be possible on Linux. It is now mandatory to run at least `make prepare-props` before Xamarin.Android can be built. The target generates the OS-specific props file which is required by the build. `make prepare` depends on the target.
2016-07-26 16:27:31 +03:00
AlwaysCreate="True"
/>
</Target>
<Target Name="_GenerateCrossOffsetHeaderFiles"
Inputs="$(MonoSourceFullPath)\configure"
[mono-runtimes] Use directory separators with `$(OutputPath)`. [The build][0] [is broken][1]: xamarin-android/build-tools/mono-runtimes/mono-runtimes.targets: error : Could not find a part of the path ".../xamarin-android/xamarin-android/bin/Debuglib/xbuild-frameworks/MonoAndroid/v1.0/RedistList/FrameworkList.xml". The error is generated from the `_InstallBcl` target when writing `FrameworkList.xml`, and a close reading of the error message shows that a directory separator is missing: it should be `bin/Debug/lib`, but is instead `bin/Debuglib`. I *assume* that this is due to `<CallTarget/>` being "flakey" within xbuild (it passes the file containing the target to `<MSBuild/>`!), and *somehow* in that "flakiness" `$(OutputPath)` doesn't end with `\`, *which it should*, because `Microsoft.Common.targets` will automatically append `\` to `$(OutputPath)`. To work around this issue, update `mono-runtimes.targets` so that it *always* uses `\` after `$(OutputPath)` and `$(IntermediateOutputPath)`. Additionally, `llc.exe` was being installed into the wrong directory, preventing the bundle from being used: Target _BuildUnlessCached needs to be built as output file '../../bin/Debug/lib/mandroid/llc.exe' does not exist. `llc.exe` was instead installed into `bin/Debug/bin/llc.exe`. Bump the bundle version number to -v3 and correct `mono-runtimes.targets` so that `llc.exe` and `opt.exe` are properly installed into `$prefix/lib/mandroid`. [0]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android/49/ [1]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android/49/consoleText
2016-08-17 04:18:15 +03:00
Outputs="$(IntermediateOutputPath)\%(_MonoCrossRuntime.Identity)\%(_MonoCrossRuntime.TargetAbi).h"
[mono-runtimes] Build LLVM cross-compilers on Linux (#616) Adds support to build all of the LLVM-based cross compilers on Linux and thus makes it possible to use AOT on Linux. Windows cross compilers can be built as well, but build of libmonodroid or libzip-windows won't work because Linux doesn't use mxe and mingw packaged for Linux doesn't come with a handful of libraries required for the above builds to complete (mman, dlfcn, cmake and friends)
2017-06-01 18:36:53 +03:00
Condition=" '@(_MonoCrossRuntime)' != '' ">
[mono-runtimes] Build AOT+LLVM cross-compilers (#125) The commit implements building of LLVM and cross-compilers to support Xamarin.Android/Mono AOT. LLVM and cross-compilers can be built for both the host platform (Linux and OS/X at the moment) as well as cross-compiled for 32-bit and 64-bit Windows platforms. Windows builds are done with MXE toolchain on OS/X and with the packaged mingw-w64 toolchain on Linux (tested on Ubuntu 16.04 ONLY). Also introducing a new set of MSBuild properties that contain information about the host system. Some of those properties (HostOS, HostCC, HostCXX for instance) have been moved from Configuration.props to better support auto-detection. A new script, build-tools/scripts/generate-os-info, is invoked as part of `make prepare` to generate file that contains the new properties. The generated file is required for the build to work and is also host-specific (it mustn't be moved between different machines) Cross compiler builds require access to a configured Mono build tree, in order to generate C structure offsets header file that is used by the AOT compilers to properly generate AOT-ed binaries. Therefore, even if a JIT target is not enabled in the configuration, enabling a cross-compiler for some target will configure Mono for that JIT target but it will NOT build it, to save time. To facilitate this, the _MonoRuntimes items defined in build-tools/mono-runtimes/mono-runtimes.projitems gain an additional metadata item called `DoBuild` which will be set to `true` if the runtime actually needs to be built, as opposed to just configured. MXE builds are disabled on Linux as mingw-w64 works just fine. A `make prepare` warning is issued for Linux hosts which have the binfmt_misc module enabled and either Wine of Mono (cli) registered as PE32/PE32+ binary interpreters. In such instance building of the Windows cross-compilers will fail because Autotools determine whether software is being cross compiled by building a test program and attempting to execute it. In normal circumstances such an attempt will fail, but with Windows cross-compilation and either Wine or Mono registered to handle the PE32 executables this attempt will succeed thus causing the cross compilation detection to fail. Currently to build cross compilers on Linux you need to generate the C structure offsets header file on OS/X and copy the resulting headers to appropriate places on Linux. The header files should be placed in build-tools/mono-runtimes/obj/Debug/cross-*/ directories. The header files are: {cross-arm,cross-arm-win}/aarch64-v8a-linux-android.h {cross-arm64,cross-arm64-win}/armv5-none-linux-androideabi.h {cross-x86,cross-x86-win}/i686-none-linux-android.h {cross-x86_64,cross-x86_64-win}/x86_64-none-linux-android.h Offsets header generation doesn't work on Linux atm because of missing support for it in the Mono utility used to generate the offsets. Hopefully this limitation will be removed in the near future and a start-to-end build of everything will be possible on Linux. It is now mandatory to run at least `make prepare-props` before Xamarin.Android can be built. The target generates the OS-specific props file which is required by the build. `make prepare` depends on the target.
2016-07-26 16:27:31 +03:00
<Exec
Command="make $(_AotOffsetsDumperName)"
WorkingDirectory="$(_AotOffsetsDumperSourceDir)"
/>
[mono-runtimes] Use correct MONO_PATH for MonoAotOffsetsDumper.exe (#328) Commit 3428975c [broke the macOS build][0] Executing: .../MonoAotOffsetsDumper.exe ... Unhandled Exception: System.IO.FileNotFoundException: Could not load file or assembly 'CppSharp.Generator, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null' or one of its dependencies. The cause of the error is [`CppSharpBinaries/6466d5f7`][1], which altered the directory structure of the CppSharpBinaries/60f763a9 *branch* (Yes, it's a branch!) from: CppSharpBinaries/CppSharp.Generator.dll to: CppSharpBinaries/osx_32/CppSharp.Generator.dll Since our `MONO_PATH` value wasn't updated, mono thus couldn't find `CppSharp.Generator.dll`, hence the error. Update our `MONO_PATH` value so that `CppSharp.Generator.dll` and other dependencies can be found. [0]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android/170/ [1]: https://github.com/xamarin/CppSharpBinaries/commit/6466d5f7c70212fc56aac1a3d96f997935ba7dd7
2016-12-07 22:48:49 +03:00
<PropertyGroup>
<_CppAbiDir Condition="$(AndroidSupportedHostJitAbisForConditionalChecks.Contains (':Darwin:'))">osx_32</_CppAbiDir>
[build] Fix mono version check on Linux Commits 721c4335, 469e628d, and 986869bc attempted to fix the macOS Jenkins build by using `mono --arch=32`/`mono32` to run `MonoAotOffsetsDumper.exe`, and by altering the mono version check to require mono 5.2.0.114, which was needed in order for macOS to provide the `mono32` binary in a location included in `$PATH`. Those commits *also* [broke the Linux+Jenkins build][0], by requiring a binary (`mono32`) that (1) doesn't exist on Linux, and (2) isn't needed on Linux (Linux requires a 64-bit process to invoke `MonoAotOffsetsDumper.exe`!): error : Missing dependency detected. For Linux we do not know how to install program `mono`, version >= 5.2.0.114 Mono 5.2.0 is pre-installed on that machine, but *not* 5.2.0.114. Update the mono version check so that 5.2.0.114 is only required on macOS/Darwin; other platforms just require 5.2.0. Update the `MonoAotOffsetsDumper.exe` invocation so that `mono --arch=32` is only used on macOS, *not* Linux. [0]: https://jenkins.mono-project.com/job/xamarin-anroid-linux-pr-builder/444/
2017-05-16 17:21:00 +03:00
<_CppAbiArch Condition="$(AndroidSupportedHostJitAbisForConditionalChecks.Contains (':Darwin:'))">--arch=32</_CppAbiArch>
[mono-runtimes] Use correct MONO_PATH for MonoAotOffsetsDumper.exe (#328) Commit 3428975c [broke the macOS build][0] Executing: .../MonoAotOffsetsDumper.exe ... Unhandled Exception: System.IO.FileNotFoundException: Could not load file or assembly 'CppSharp.Generator, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null' or one of its dependencies. The cause of the error is [`CppSharpBinaries/6466d5f7`][1], which altered the directory structure of the CppSharpBinaries/60f763a9 *branch* (Yes, it's a branch!) from: CppSharpBinaries/CppSharp.Generator.dll to: CppSharpBinaries/osx_32/CppSharp.Generator.dll Since our `MONO_PATH` value wasn't updated, mono thus couldn't find `CppSharp.Generator.dll`, hence the error. Update our `MONO_PATH` value so that `CppSharp.Generator.dll` and other dependencies can be found. [0]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android/170/ [1]: https://github.com/xamarin/CppSharpBinaries/commit/6466d5f7c70212fc56aac1a3d96f997935ba7dd7
2016-12-07 22:48:49 +03:00
<_CppAbiDir Condition="$(AndroidSupportedHostJitAbisForConditionalChecks.Contains (':Linux:'))">linux_64</_CppAbiDir>
[build] Fix mono version check on Linux Commits 721c4335, 469e628d, and 986869bc attempted to fix the macOS Jenkins build by using `mono --arch=32`/`mono32` to run `MonoAotOffsetsDumper.exe`, and by altering the mono version check to require mono 5.2.0.114, which was needed in order for macOS to provide the `mono32` binary in a location included in `$PATH`. Those commits *also* [broke the Linux+Jenkins build][0], by requiring a binary (`mono32`) that (1) doesn't exist on Linux, and (2) isn't needed on Linux (Linux requires a 64-bit process to invoke `MonoAotOffsetsDumper.exe`!): error : Missing dependency detected. For Linux we do not know how to install program `mono`, version >= 5.2.0.114 Mono 5.2.0 is pre-installed on that machine, but *not* 5.2.0.114. Update the mono version check so that 5.2.0.114 is only required on macOS/Darwin; other platforms just require 5.2.0. Update the `MonoAotOffsetsDumper.exe` invocation so that `mono --arch=32` is only used on macOS, *not* Linux. [0]: https://jenkins.mono-project.com/job/xamarin-anroid-linux-pr-builder/444/
2017-05-16 17:21:00 +03:00
<_CppAbiArch Condition="$(AndroidSupportedHostJitAbisForConditionalChecks.Contains (':Linux:'))"></_CppAbiArch>
[mono-runtimes] Use correct MONO_PATH for MonoAotOffsetsDumper.exe (#328) Commit 3428975c [broke the macOS build][0] Executing: .../MonoAotOffsetsDumper.exe ... Unhandled Exception: System.IO.FileNotFoundException: Could not load file or assembly 'CppSharp.Generator, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null' or one of its dependencies. The cause of the error is [`CppSharpBinaries/6466d5f7`][1], which altered the directory structure of the CppSharpBinaries/60f763a9 *branch* (Yes, it's a branch!) from: CppSharpBinaries/CppSharp.Generator.dll to: CppSharpBinaries/osx_32/CppSharp.Generator.dll Since our `MONO_PATH` value wasn't updated, mono thus couldn't find `CppSharp.Generator.dll`, hence the error. Update our `MONO_PATH` value so that `CppSharp.Generator.dll` and other dependencies can be found. [0]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android/170/ [1]: https://github.com/xamarin/CppSharpBinaries/commit/6466d5f7c70212fc56aac1a3d96f997935ba7dd7
2016-12-07 22:48:49 +03:00
</PropertyGroup>
[mono-runtimes] Build AOT+LLVM cross-compilers (#125) The commit implements building of LLVM and cross-compilers to support Xamarin.Android/Mono AOT. LLVM and cross-compilers can be built for both the host platform (Linux and OS/X at the moment) as well as cross-compiled for 32-bit and 64-bit Windows platforms. Windows builds are done with MXE toolchain on OS/X and with the packaged mingw-w64 toolchain on Linux (tested on Ubuntu 16.04 ONLY). Also introducing a new set of MSBuild properties that contain information about the host system. Some of those properties (HostOS, HostCC, HostCXX for instance) have been moved from Configuration.props to better support auto-detection. A new script, build-tools/scripts/generate-os-info, is invoked as part of `make prepare` to generate file that contains the new properties. The generated file is required for the build to work and is also host-specific (it mustn't be moved between different machines) Cross compiler builds require access to a configured Mono build tree, in order to generate C structure offsets header file that is used by the AOT compilers to properly generate AOT-ed binaries. Therefore, even if a JIT target is not enabled in the configuration, enabling a cross-compiler for some target will configure Mono for that JIT target but it will NOT build it, to save time. To facilitate this, the _MonoRuntimes items defined in build-tools/mono-runtimes/mono-runtimes.projitems gain an additional metadata item called `DoBuild` which will be set to `true` if the runtime actually needs to be built, as opposed to just configured. MXE builds are disabled on Linux as mingw-w64 works just fine. A `make prepare` warning is issued for Linux hosts which have the binfmt_misc module enabled and either Wine of Mono (cli) registered as PE32/PE32+ binary interpreters. In such instance building of the Windows cross-compilers will fail because Autotools determine whether software is being cross compiled by building a test program and attempting to execute it. In normal circumstances such an attempt will fail, but with Windows cross-compilation and either Wine or Mono registered to handle the PE32 executables this attempt will succeed thus causing the cross compilation detection to fail. Currently to build cross compilers on Linux you need to generate the C structure offsets header file on OS/X and copy the resulting headers to appropriate places on Linux. The header files should be placed in build-tools/mono-runtimes/obj/Debug/cross-*/ directories. The header files are: {cross-arm,cross-arm-win}/aarch64-v8a-linux-android.h {cross-arm64,cross-arm64-win}/armv5-none-linux-androideabi.h {cross-x86,cross-x86-win}/i686-none-linux-android.h {cross-x86_64,cross-x86_64-win}/x86_64-none-linux-android.h Offsets header generation doesn't work on Linux atm because of missing support for it in the Mono utility used to generate the offsets. Hopefully this limitation will be removed in the near future and a start-to-end build of everything will be possible on Linux. It is now mandatory to run at least `make prepare-props` before Xamarin.Android can be built. The target generates the OS-specific props file which is required by the build. `make prepare` depends on the target.
2016-07-26 16:27:31 +03:00
<Exec
[build] Fix mono version check on Linux Commits 721c4335, 469e628d, and 986869bc attempted to fix the macOS Jenkins build by using `mono --arch=32`/`mono32` to run `MonoAotOffsetsDumper.exe`, and by altering the mono version check to require mono 5.2.0.114, which was needed in order for macOS to provide the `mono32` binary in a location included in `$PATH`. Those commits *also* [broke the Linux+Jenkins build][0], by requiring a binary (`mono32`) that (1) doesn't exist on Linux, and (2) isn't needed on Linux (Linux requires a 64-bit process to invoke `MonoAotOffsetsDumper.exe`!): error : Missing dependency detected. For Linux we do not know how to install program `mono`, version >= 5.2.0.114 Mono 5.2.0 is pre-installed on that machine, but *not* 5.2.0.114. Update the mono version check so that 5.2.0.114 is only required on macOS/Darwin; other platforms just require 5.2.0. Update the `MonoAotOffsetsDumper.exe` invocation so that `mono --arch=32` is only used on macOS, *not* Linux. [0]: https://jenkins.mono-project.com/job/xamarin-anroid-linux-pr-builder/444/
2017-05-16 17:21:00 +03:00
Command="MONO_PATH=$(_AotOffsetsDumperSourceDir)\CppSharp\$(_CppAbiDir) $(ManagedRuntime) $(_CppAbiArch) $(_AotOffsetsDumperSourceDir)\$(_AotOffsetsDumperName) --android-ndk=&quot;$(AndroidNdkFullPath)&quot; --mono=&quot;$(MonoSourceFullPath)&quot; --monodroid=&quot;$(XamarinAndroidSourcePath)&quot; --abi=&quot;%(_MonoCrossRuntime.TargetAbi)&quot; --targetdir=&quot;$(MSBuildThisFileDirectory)$(IntermediateOutputPath)\%(_MonoCrossRuntime.JitArch)&quot; --out=&quot;$(MSBuildThisFileDirectory)$(IntermediateOutputPath)\%(_MonoCrossRuntime.Identity)&quot;"
[mono-runtimes] Use directory separators with `$(OutputPath)`. [The build][0] [is broken][1]: xamarin-android/build-tools/mono-runtimes/mono-runtimes.targets: error : Could not find a part of the path ".../xamarin-android/xamarin-android/bin/Debuglib/xbuild-frameworks/MonoAndroid/v1.0/RedistList/FrameworkList.xml". The error is generated from the `_InstallBcl` target when writing `FrameworkList.xml`, and a close reading of the error message shows that a directory separator is missing: it should be `bin/Debug/lib`, but is instead `bin/Debuglib`. I *assume* that this is due to `<CallTarget/>` being "flakey" within xbuild (it passes the file containing the target to `<MSBuild/>`!), and *somehow* in that "flakiness" `$(OutputPath)` doesn't end with `\`, *which it should*, because `Microsoft.Common.targets` will automatically append `\` to `$(OutputPath)`. To work around this issue, update `mono-runtimes.targets` so that it *always* uses `\` after `$(OutputPath)` and `$(IntermediateOutputPath)`. Additionally, `llc.exe` was being installed into the wrong directory, preventing the bundle from being used: Target _BuildUnlessCached needs to be built as output file '../../bin/Debug/lib/mandroid/llc.exe' does not exist. `llc.exe` was instead installed into `bin/Debug/bin/llc.exe`. Bump the bundle version number to -v3 and correct `mono-runtimes.targets` so that `llc.exe` and `opt.exe` are properly installed into `$prefix/lib/mandroid`. [0]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android/49/ [1]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android/49/consoleText
2016-08-17 04:18:15 +03:00
WorkingDirectory="$(IntermediateOutputPath)\%(_MonoCrossRuntime.Identity)"
[mono-runtimes] Build AOT+LLVM cross-compilers (#125) The commit implements building of LLVM and cross-compilers to support Xamarin.Android/Mono AOT. LLVM and cross-compilers can be built for both the host platform (Linux and OS/X at the moment) as well as cross-compiled for 32-bit and 64-bit Windows platforms. Windows builds are done with MXE toolchain on OS/X and with the packaged mingw-w64 toolchain on Linux (tested on Ubuntu 16.04 ONLY). Also introducing a new set of MSBuild properties that contain information about the host system. Some of those properties (HostOS, HostCC, HostCXX for instance) have been moved from Configuration.props to better support auto-detection. A new script, build-tools/scripts/generate-os-info, is invoked as part of `make prepare` to generate file that contains the new properties. The generated file is required for the build to work and is also host-specific (it mustn't be moved between different machines) Cross compiler builds require access to a configured Mono build tree, in order to generate C structure offsets header file that is used by the AOT compilers to properly generate AOT-ed binaries. Therefore, even if a JIT target is not enabled in the configuration, enabling a cross-compiler for some target will configure Mono for that JIT target but it will NOT build it, to save time. To facilitate this, the _MonoRuntimes items defined in build-tools/mono-runtimes/mono-runtimes.projitems gain an additional metadata item called `DoBuild` which will be set to `true` if the runtime actually needs to be built, as opposed to just configured. MXE builds are disabled on Linux as mingw-w64 works just fine. A `make prepare` warning is issued for Linux hosts which have the binfmt_misc module enabled and either Wine of Mono (cli) registered as PE32/PE32+ binary interpreters. In such instance building of the Windows cross-compilers will fail because Autotools determine whether software is being cross compiled by building a test program and attempting to execute it. In normal circumstances such an attempt will fail, but with Windows cross-compilation and either Wine or Mono registered to handle the PE32 executables this attempt will succeed thus causing the cross compilation detection to fail. Currently to build cross compilers on Linux you need to generate the C structure offsets header file on OS/X and copy the resulting headers to appropriate places on Linux. The header files should be placed in build-tools/mono-runtimes/obj/Debug/cross-*/ directories. The header files are: {cross-arm,cross-arm-win}/aarch64-v8a-linux-android.h {cross-arm64,cross-arm64-win}/armv5-none-linux-androideabi.h {cross-x86,cross-x86-win}/i686-none-linux-android.h {cross-x86_64,cross-x86_64-win}/x86_64-none-linux-android.h Offsets header generation doesn't work on Linux atm because of missing support for it in the Mono utility used to generate the offsets. Hopefully this limitation will be removed in the near future and a start-to-end build of everything will be possible on Linux. It is now mandatory to run at least `make prepare-props` before Xamarin.Android can be built. The target generates the OS-specific props file which is required by the build. `make prepare` depends on the target.
2016-07-26 16:27:31 +03:00
/>
<Touch
[mono-runtimes] Use directory separators with `$(OutputPath)`. [The build][0] [is broken][1]: xamarin-android/build-tools/mono-runtimes/mono-runtimes.targets: error : Could not find a part of the path ".../xamarin-android/xamarin-android/bin/Debuglib/xbuild-frameworks/MonoAndroid/v1.0/RedistList/FrameworkList.xml". The error is generated from the `_InstallBcl` target when writing `FrameworkList.xml`, and a close reading of the error message shows that a directory separator is missing: it should be `bin/Debug/lib`, but is instead `bin/Debuglib`. I *assume* that this is due to `<CallTarget/>` being "flakey" within xbuild (it passes the file containing the target to `<MSBuild/>`!), and *somehow* in that "flakiness" `$(OutputPath)` doesn't end with `\`, *which it should*, because `Microsoft.Common.targets` will automatically append `\` to `$(OutputPath)`. To work around this issue, update `mono-runtimes.targets` so that it *always* uses `\` after `$(OutputPath)` and `$(IntermediateOutputPath)`. Additionally, `llc.exe` was being installed into the wrong directory, preventing the bundle from being used: Target _BuildUnlessCached needs to be built as output file '../../bin/Debug/lib/mandroid/llc.exe' does not exist. `llc.exe` was instead installed into `bin/Debug/bin/llc.exe`. Bump the bundle version number to -v3 and correct `mono-runtimes.targets` so that `llc.exe` and `opt.exe` are properly installed into `$prefix/lib/mandroid`. [0]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android/49/ [1]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android/49/consoleText
2016-08-17 04:18:15 +03:00
Files="$(IntermediateOutputPath)\%(_MonoCrossRuntime.Identity)\%(_MonoCrossRuntime.OffsetsHeaderFile)"
[mono-runtimes] Build AOT+LLVM cross-compilers (#125) The commit implements building of LLVM and cross-compilers to support Xamarin.Android/Mono AOT. LLVM and cross-compilers can be built for both the host platform (Linux and OS/X at the moment) as well as cross-compiled for 32-bit and 64-bit Windows platforms. Windows builds are done with MXE toolchain on OS/X and with the packaged mingw-w64 toolchain on Linux (tested on Ubuntu 16.04 ONLY). Also introducing a new set of MSBuild properties that contain information about the host system. Some of those properties (HostOS, HostCC, HostCXX for instance) have been moved from Configuration.props to better support auto-detection. A new script, build-tools/scripts/generate-os-info, is invoked as part of `make prepare` to generate file that contains the new properties. The generated file is required for the build to work and is also host-specific (it mustn't be moved between different machines) Cross compiler builds require access to a configured Mono build tree, in order to generate C structure offsets header file that is used by the AOT compilers to properly generate AOT-ed binaries. Therefore, even if a JIT target is not enabled in the configuration, enabling a cross-compiler for some target will configure Mono for that JIT target but it will NOT build it, to save time. To facilitate this, the _MonoRuntimes items defined in build-tools/mono-runtimes/mono-runtimes.projitems gain an additional metadata item called `DoBuild` which will be set to `true` if the runtime actually needs to be built, as opposed to just configured. MXE builds are disabled on Linux as mingw-w64 works just fine. A `make prepare` warning is issued for Linux hosts which have the binfmt_misc module enabled and either Wine of Mono (cli) registered as PE32/PE32+ binary interpreters. In such instance building of the Windows cross-compilers will fail because Autotools determine whether software is being cross compiled by building a test program and attempting to execute it. In normal circumstances such an attempt will fail, but with Windows cross-compilation and either Wine or Mono registered to handle the PE32 executables this attempt will succeed thus causing the cross compilation detection to fail. Currently to build cross compilers on Linux you need to generate the C structure offsets header file on OS/X and copy the resulting headers to appropriate places on Linux. The header files should be placed in build-tools/mono-runtimes/obj/Debug/cross-*/ directories. The header files are: {cross-arm,cross-arm-win}/aarch64-v8a-linux-android.h {cross-arm64,cross-arm64-win}/armv5-none-linux-androideabi.h {cross-x86,cross-x86-win}/i686-none-linux-android.h {cross-x86_64,cross-x86_64-win}/x86_64-none-linux-android.h Offsets header generation doesn't work on Linux atm because of missing support for it in the Mono utility used to generate the offsets. Hopefully this limitation will be removed in the near future and a start-to-end build of everything will be possible on Linux. It is now mandatory to run at least `make prepare-props` before Xamarin.Android can be built. The target generates the OS-specific props file which is required by the build. `make prepare` depends on the target.
2016-07-26 16:27:31 +03:00
/>
</Target>
<!-- The condition below is to work around a bug in xbuild which attempts to batch
even if there are no _MonoCrossRuntime items and the Copy task fails
-->
<Target Name="_BuildCrossRuntimes"
DependsOnTargets="_GenerateCrossOffsetHeaderFiles"
[mono-runtimes] Use directory separators with `$(OutputPath)`. [The build][0] [is broken][1]: xamarin-android/build-tools/mono-runtimes/mono-runtimes.targets: error : Could not find a part of the path ".../xamarin-android/xamarin-android/bin/Debuglib/xbuild-frameworks/MonoAndroid/v1.0/RedistList/FrameworkList.xml". The error is generated from the `_InstallBcl` target when writing `FrameworkList.xml`, and a close reading of the error message shows that a directory separator is missing: it should be `bin/Debug/lib`, but is instead `bin/Debuglib`. I *assume* that this is due to `<CallTarget/>` being "flakey" within xbuild (it passes the file containing the target to `<MSBuild/>`!), and *somehow* in that "flakiness" `$(OutputPath)` doesn't end with `\`, *which it should*, because `Microsoft.Common.targets` will automatically append `\` to `$(OutputPath)`. To work around this issue, update `mono-runtimes.targets` so that it *always* uses `\` after `$(OutputPath)` and `$(IntermediateOutputPath)`. Additionally, `llc.exe` was being installed into the wrong directory, preventing the bundle from being used: Target _BuildUnlessCached needs to be built as output file '../../bin/Debug/lib/mandroid/llc.exe' does not exist. `llc.exe` was instead installed into `bin/Debug/bin/llc.exe`. Bump the bundle version number to -v3 and correct `mono-runtimes.targets` so that `llc.exe` and `opt.exe` are properly installed into `$prefix/lib/mandroid`. [0]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android/49/ [1]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android/49/consoleText
2016-08-17 04:18:15 +03:00
Inputs="$(IntermediateOutputPath)\%(_MonoCrossRuntime.Identity)\.stamp"
Outputs="$(IntermediateOutputPath)\%(_MonoCrossRuntime.Identity)\mono\mini\mono-sgen%(_MonoCrossRuntime.ExeSuffix)"
[mono-runtimes] Build AOT+LLVM cross-compilers (#125) The commit implements building of LLVM and cross-compilers to support Xamarin.Android/Mono AOT. LLVM and cross-compilers can be built for both the host platform (Linux and OS/X at the moment) as well as cross-compiled for 32-bit and 64-bit Windows platforms. Windows builds are done with MXE toolchain on OS/X and with the packaged mingw-w64 toolchain on Linux (tested on Ubuntu 16.04 ONLY). Also introducing a new set of MSBuild properties that contain information about the host system. Some of those properties (HostOS, HostCC, HostCXX for instance) have been moved from Configuration.props to better support auto-detection. A new script, build-tools/scripts/generate-os-info, is invoked as part of `make prepare` to generate file that contains the new properties. The generated file is required for the build to work and is also host-specific (it mustn't be moved between different machines) Cross compiler builds require access to a configured Mono build tree, in order to generate C structure offsets header file that is used by the AOT compilers to properly generate AOT-ed binaries. Therefore, even if a JIT target is not enabled in the configuration, enabling a cross-compiler for some target will configure Mono for that JIT target but it will NOT build it, to save time. To facilitate this, the _MonoRuntimes items defined in build-tools/mono-runtimes/mono-runtimes.projitems gain an additional metadata item called `DoBuild` which will be set to `true` if the runtime actually needs to be built, as opposed to just configured. MXE builds are disabled on Linux as mingw-w64 works just fine. A `make prepare` warning is issued for Linux hosts which have the binfmt_misc module enabled and either Wine of Mono (cli) registered as PE32/PE32+ binary interpreters. In such instance building of the Windows cross-compilers will fail because Autotools determine whether software is being cross compiled by building a test program and attempting to execute it. In normal circumstances such an attempt will fail, but with Windows cross-compilation and either Wine or Mono registered to handle the PE32 executables this attempt will succeed thus causing the cross compilation detection to fail. Currently to build cross compilers on Linux you need to generate the C structure offsets header file on OS/X and copy the resulting headers to appropriate places on Linux. The header files should be placed in build-tools/mono-runtimes/obj/Debug/cross-*/ directories. The header files are: {cross-arm,cross-arm-win}/aarch64-v8a-linux-android.h {cross-arm64,cross-arm64-win}/armv5-none-linux-androideabi.h {cross-x86,cross-x86-win}/i686-none-linux-android.h {cross-x86_64,cross-x86_64-win}/x86_64-none-linux-android.h Offsets header generation doesn't work on Linux atm because of missing support for it in the Mono utility used to generate the offsets. Hopefully this limitation will be removed in the near future and a start-to-end build of everything will be possible on Linux. It is now mandatory to run at least `make prepare-props` before Xamarin.Android can be built. The target generates the OS-specific props file which is required by the build. `make prepare` depends on the target.
2016-07-26 16:27:31 +03:00
Condition=" '@(_MonoCrossRuntime)' != '' ">
[mono-runtimes] Use directory separators with `$(OutputPath)`. [The build][0] [is broken][1]: xamarin-android/build-tools/mono-runtimes/mono-runtimes.targets: error : Could not find a part of the path ".../xamarin-android/xamarin-android/bin/Debuglib/xbuild-frameworks/MonoAndroid/v1.0/RedistList/FrameworkList.xml". The error is generated from the `_InstallBcl` target when writing `FrameworkList.xml`, and a close reading of the error message shows that a directory separator is missing: it should be `bin/Debug/lib`, but is instead `bin/Debuglib`. I *assume* that this is due to `<CallTarget/>` being "flakey" within xbuild (it passes the file containing the target to `<MSBuild/>`!), and *somehow* in that "flakiness" `$(OutputPath)` doesn't end with `\`, *which it should*, because `Microsoft.Common.targets` will automatically append `\` to `$(OutputPath)`. To work around this issue, update `mono-runtimes.targets` so that it *always* uses `\` after `$(OutputPath)` and `$(IntermediateOutputPath)`. Additionally, `llc.exe` was being installed into the wrong directory, preventing the bundle from being used: Target _BuildUnlessCached needs to be built as output file '../../bin/Debug/lib/mandroid/llc.exe' does not exist. `llc.exe` was instead installed into `bin/Debug/bin/llc.exe`. Bump the bundle version number to -v3 and correct `mono-runtimes.targets` so that `llc.exe` and `opt.exe` are properly installed into `$prefix/lib/mandroid`. [0]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android/49/ [1]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android/49/consoleText
2016-08-17 04:18:15 +03:00
<Message Text="Building %(_MonoCrossRuntime.Identity) in $(IntermediateOutputPath)\%(_MonoCrossRuntime.Identity)"/>
[mono-runtimes] Build AOT+LLVM cross-compilers (#125) The commit implements building of LLVM and cross-compilers to support Xamarin.Android/Mono AOT. LLVM and cross-compilers can be built for both the host platform (Linux and OS/X at the moment) as well as cross-compiled for 32-bit and 64-bit Windows platforms. Windows builds are done with MXE toolchain on OS/X and with the packaged mingw-w64 toolchain on Linux (tested on Ubuntu 16.04 ONLY). Also introducing a new set of MSBuild properties that contain information about the host system. Some of those properties (HostOS, HostCC, HostCXX for instance) have been moved from Configuration.props to better support auto-detection. A new script, build-tools/scripts/generate-os-info, is invoked as part of `make prepare` to generate file that contains the new properties. The generated file is required for the build to work and is also host-specific (it mustn't be moved between different machines) Cross compiler builds require access to a configured Mono build tree, in order to generate C structure offsets header file that is used by the AOT compilers to properly generate AOT-ed binaries. Therefore, even if a JIT target is not enabled in the configuration, enabling a cross-compiler for some target will configure Mono for that JIT target but it will NOT build it, to save time. To facilitate this, the _MonoRuntimes items defined in build-tools/mono-runtimes/mono-runtimes.projitems gain an additional metadata item called `DoBuild` which will be set to `true` if the runtime actually needs to be built, as opposed to just configured. MXE builds are disabled on Linux as mingw-w64 works just fine. A `make prepare` warning is issued for Linux hosts which have the binfmt_misc module enabled and either Wine of Mono (cli) registered as PE32/PE32+ binary interpreters. In such instance building of the Windows cross-compilers will fail because Autotools determine whether software is being cross compiled by building a test program and attempting to execute it. In normal circumstances such an attempt will fail, but with Windows cross-compilation and either Wine or Mono registered to handle the PE32 executables this attempt will succeed thus causing the cross compilation detection to fail. Currently to build cross compilers on Linux you need to generate the C structure offsets header file on OS/X and copy the resulting headers to appropriate places on Linux. The header files should be placed in build-tools/mono-runtimes/obj/Debug/cross-*/ directories. The header files are: {cross-arm,cross-arm-win}/aarch64-v8a-linux-android.h {cross-arm64,cross-arm64-win}/armv5-none-linux-androideabi.h {cross-x86,cross-x86-win}/i686-none-linux-android.h {cross-x86_64,cross-x86_64-win}/x86_64-none-linux-android.h Offsets header generation doesn't work on Linux atm because of missing support for it in the Mono utility used to generate the offsets. Hopefully this limitation will be removed in the near future and a start-to-end build of everything will be possible on Linux. It is now mandatory to run at least `make prepare-props` before Xamarin.Android can be built. The target generates the OS-specific props file which is required by the build. `make prepare` depends on the target.
2016-07-26 16:27:31 +03:00
<Exec
Command="%(_MonoCrossRuntime.BuildEnvironment) make $(MakeConcurrency)"
[build] Support building *from clean* with `msbuild` (Take 2!) (#391) Commit 7343965a didn't succeed in actually building `xamarin-android` with `msbuild` on our Jenkins bot. :-( Time to take another stab at it. As an added complication, my local `msbuild` behavior (mono 4.8) differs from Jenkins' `msbuild` behavior (mono 4.4), specifically around MSB3644. [On Jenkins][msb-96], it's a warning: warning MSB3644: The reference assemblies for framework ".NETFramework,Version=v4.5" were not found. When I run locally, it's an *error*. Go figure. Fixing the local error requires that I *remove* the `$(TargetFrameworkRootPath)` MSBuild property from `Configuration.props`, added in commit 255a569b. The issue is that `$(TargetFrameworkRootPath)` is an *override*, an "absolute" override. Thus, when e.g. building `Xamarin.Android.Build.Tasks.csproj`, the `.NETFramework,Version=v4.5` framework -- and corresponding `mscorlib.dll` -- cannot be resolved, and everything breaks. (Which raises the wonderful question of whether removing `$(TargetFrameworkRootPath)` will cause the Linux build to break again, but we'll cross that bridge later...) Removing `$(TargetFrameworkRootPath)` in turn requires that I remove the explicit `@(Reference)`s to `System.Runtime` which commit 7343965a added when building under `xbuild`. This also means that we no longer need the `$(_XABuildingWithXBuild)` property. (The "history" eluded to in 7343965a? It's continuing! Argh!) Additionally, fix the logging from the `<UnzipDirectoryChildren/>` task so that it prints the *correct* task name, *not* `Unzip`. (Seeing the wrong task name in log files makes things *confusing*.) With my local `msbuild` faiklures fixed, we can turn our attention to to the [errors reported on Jenkins][msb-96]. They are variations on the same "form": # from build-tools/mono-runtimes .../external/mono/configure LDFLAGS=... ... EXEC : error : /Applications/Xcode8.1-beta1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/libtool: unknown option character `-' in: --config # from src/sqlite-xamarin .../android create project -g -n sqlitesample -p . -a MainActivity -t android-25 --package com.xamarin.sqlitesample -v 1.0.0 EXEC : error : Project folder '.' is not empty. Please consider using 'android update' instead. The issue here is that MSBuild is "better" than xbuild: xbuild only checks the process exit code for errors. The above `android create project` command has en exit code of 0, i.e. it didn't error...according to xbuild. That said, when running [with xbuild][xb-203]: Task "Exec" Executing: .../android create project -g -n sqlitesample -p . -a MainActivity -t android-25 --package com.xamarin.sqlitesample -v 1.0.0 Environment variables being passed to the tool: Error: Project folder '.' is not empty. Please consider using 'android update' instead ... Done executing task "Exec" Task "Copy" ... Rephrased: `android create project ...` is emitting an "Error: ..." *message*, but even when it emits that message the exit status is still 0 (no error). xbuild is only checking the exit status, and thus continues the build. MSBuild not only checks the exit status, but *also* checks the command *output*, looking for the string `error`. Because an "error" message is printed, MSBuild flags this as an error, halting the build. The fix? Set `Exec.IgnoreStandardErrorWarningFormat=True` on all `<Exec/>` tasks which invoke **make**(1) or `configure` within `build-tools/mono-runtimes` and `src/sqlite-xamarin`. Setting `Exec.IgnoreStandardErrorWarningFormat=True` prevents the `<Exec/>` task from scanning the task output for "error", causing `msbuild` to behave like `xbuild` and only use the process exit code for errors. This allows the build to continue. [msb-96]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android-msbuild/96/consoleText [xb-203]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android/203/consoleText
2017-01-20 15:05:02 +03:00
IgnoreStandardErrorWarningFormat="True"
[mono-runtimes] Use directory separators with `$(OutputPath)`. [The build][0] [is broken][1]: xamarin-android/build-tools/mono-runtimes/mono-runtimes.targets: error : Could not find a part of the path ".../xamarin-android/xamarin-android/bin/Debuglib/xbuild-frameworks/MonoAndroid/v1.0/RedistList/FrameworkList.xml". The error is generated from the `_InstallBcl` target when writing `FrameworkList.xml`, and a close reading of the error message shows that a directory separator is missing: it should be `bin/Debug/lib`, but is instead `bin/Debuglib`. I *assume* that this is due to `<CallTarget/>` being "flakey" within xbuild (it passes the file containing the target to `<MSBuild/>`!), and *somehow* in that "flakiness" `$(OutputPath)` doesn't end with `\`, *which it should*, because `Microsoft.Common.targets` will automatically append `\` to `$(OutputPath)`. To work around this issue, update `mono-runtimes.targets` so that it *always* uses `\` after `$(OutputPath)` and `$(IntermediateOutputPath)`. Additionally, `llc.exe` was being installed into the wrong directory, preventing the bundle from being used: Target _BuildUnlessCached needs to be built as output file '../../bin/Debug/lib/mandroid/llc.exe' does not exist. `llc.exe` was instead installed into `bin/Debug/bin/llc.exe`. Bump the bundle version number to -v3 and correct `mono-runtimes.targets` so that `llc.exe` and `opt.exe` are properly installed into `$prefix/lib/mandroid`. [0]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android/49/ [1]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android/49/consoleText
2016-08-17 04:18:15 +03:00
WorkingDirectory="$(IntermediateOutputPath)\%(_MonoCrossRuntime.Identity)"
[mono-runtimes] Build AOT+LLVM cross-compilers (#125) The commit implements building of LLVM and cross-compilers to support Xamarin.Android/Mono AOT. LLVM and cross-compilers can be built for both the host platform (Linux and OS/X at the moment) as well as cross-compiled for 32-bit and 64-bit Windows platforms. Windows builds are done with MXE toolchain on OS/X and with the packaged mingw-w64 toolchain on Linux (tested on Ubuntu 16.04 ONLY). Also introducing a new set of MSBuild properties that contain information about the host system. Some of those properties (HostOS, HostCC, HostCXX for instance) have been moved from Configuration.props to better support auto-detection. A new script, build-tools/scripts/generate-os-info, is invoked as part of `make prepare` to generate file that contains the new properties. The generated file is required for the build to work and is also host-specific (it mustn't be moved between different machines) Cross compiler builds require access to a configured Mono build tree, in order to generate C structure offsets header file that is used by the AOT compilers to properly generate AOT-ed binaries. Therefore, even if a JIT target is not enabled in the configuration, enabling a cross-compiler for some target will configure Mono for that JIT target but it will NOT build it, to save time. To facilitate this, the _MonoRuntimes items defined in build-tools/mono-runtimes/mono-runtimes.projitems gain an additional metadata item called `DoBuild` which will be set to `true` if the runtime actually needs to be built, as opposed to just configured. MXE builds are disabled on Linux as mingw-w64 works just fine. A `make prepare` warning is issued for Linux hosts which have the binfmt_misc module enabled and either Wine of Mono (cli) registered as PE32/PE32+ binary interpreters. In such instance building of the Windows cross-compilers will fail because Autotools determine whether software is being cross compiled by building a test program and attempting to execute it. In normal circumstances such an attempt will fail, but with Windows cross-compilation and either Wine or Mono registered to handle the PE32 executables this attempt will succeed thus causing the cross compilation detection to fail. Currently to build cross compilers on Linux you need to generate the C structure offsets header file on OS/X and copy the resulting headers to appropriate places on Linux. The header files should be placed in build-tools/mono-runtimes/obj/Debug/cross-*/ directories. The header files are: {cross-arm,cross-arm-win}/aarch64-v8a-linux-android.h {cross-arm64,cross-arm64-win}/armv5-none-linux-androideabi.h {cross-x86,cross-x86-win}/i686-none-linux-android.h {cross-x86_64,cross-x86_64-win}/x86_64-none-linux-android.h Offsets header generation doesn't work on Linux atm because of missing support for it in the Mono utility used to generate the offsets. Hopefully this limitation will be removed in the near future and a start-to-end build of everything will be possible on Linux. It is now mandatory to run at least `make prepare-props` before Xamarin.Android can be built. The target generates the OS-specific props file which is required by the build. `make prepare` depends on the target.
2016-07-26 16:27:31 +03:00
/>
<Touch
[mono-runtimes] Use directory separators with `$(OutputPath)`. [The build][0] [is broken][1]: xamarin-android/build-tools/mono-runtimes/mono-runtimes.targets: error : Could not find a part of the path ".../xamarin-android/xamarin-android/bin/Debuglib/xbuild-frameworks/MonoAndroid/v1.0/RedistList/FrameworkList.xml". The error is generated from the `_InstallBcl` target when writing `FrameworkList.xml`, and a close reading of the error message shows that a directory separator is missing: it should be `bin/Debug/lib`, but is instead `bin/Debuglib`. I *assume* that this is due to `<CallTarget/>` being "flakey" within xbuild (it passes the file containing the target to `<MSBuild/>`!), and *somehow* in that "flakiness" `$(OutputPath)` doesn't end with `\`, *which it should*, because `Microsoft.Common.targets` will automatically append `\` to `$(OutputPath)`. To work around this issue, update `mono-runtimes.targets` so that it *always* uses `\` after `$(OutputPath)` and `$(IntermediateOutputPath)`. Additionally, `llc.exe` was being installed into the wrong directory, preventing the bundle from being used: Target _BuildUnlessCached needs to be built as output file '../../bin/Debug/lib/mandroid/llc.exe' does not exist. `llc.exe` was instead installed into `bin/Debug/bin/llc.exe`. Bump the bundle version number to -v3 and correct `mono-runtimes.targets` so that `llc.exe` and `opt.exe` are properly installed into `$prefix/lib/mandroid`. [0]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android/49/ [1]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android/49/consoleText
2016-08-17 04:18:15 +03:00
Files="$(IntermediateOutputPath)\%(_MonoCrossRuntime.Identity)\mono\mini\mono-sgen%(_MonoCrossRuntime.ExeSuffix)"
[mono-runtimes] Build AOT+LLVM cross-compilers (#125) The commit implements building of LLVM and cross-compilers to support Xamarin.Android/Mono AOT. LLVM and cross-compilers can be built for both the host platform (Linux and OS/X at the moment) as well as cross-compiled for 32-bit and 64-bit Windows platforms. Windows builds are done with MXE toolchain on OS/X and with the packaged mingw-w64 toolchain on Linux (tested on Ubuntu 16.04 ONLY). Also introducing a new set of MSBuild properties that contain information about the host system. Some of those properties (HostOS, HostCC, HostCXX for instance) have been moved from Configuration.props to better support auto-detection. A new script, build-tools/scripts/generate-os-info, is invoked as part of `make prepare` to generate file that contains the new properties. The generated file is required for the build to work and is also host-specific (it mustn't be moved between different machines) Cross compiler builds require access to a configured Mono build tree, in order to generate C structure offsets header file that is used by the AOT compilers to properly generate AOT-ed binaries. Therefore, even if a JIT target is not enabled in the configuration, enabling a cross-compiler for some target will configure Mono for that JIT target but it will NOT build it, to save time. To facilitate this, the _MonoRuntimes items defined in build-tools/mono-runtimes/mono-runtimes.projitems gain an additional metadata item called `DoBuild` which will be set to `true` if the runtime actually needs to be built, as opposed to just configured. MXE builds are disabled on Linux as mingw-w64 works just fine. A `make prepare` warning is issued for Linux hosts which have the binfmt_misc module enabled and either Wine of Mono (cli) registered as PE32/PE32+ binary interpreters. In such instance building of the Windows cross-compilers will fail because Autotools determine whether software is being cross compiled by building a test program and attempting to execute it. In normal circumstances such an attempt will fail, but with Windows cross-compilation and either Wine or Mono registered to handle the PE32 executables this attempt will succeed thus causing the cross compilation detection to fail. Currently to build cross compilers on Linux you need to generate the C structure offsets header file on OS/X and copy the resulting headers to appropriate places on Linux. The header files should be placed in build-tools/mono-runtimes/obj/Debug/cross-*/ directories. The header files are: {cross-arm,cross-arm-win}/aarch64-v8a-linux-android.h {cross-arm64,cross-arm64-win}/armv5-none-linux-androideabi.h {cross-x86,cross-x86-win}/i686-none-linux-android.h {cross-x86_64,cross-x86_64-win}/x86_64-none-linux-android.h Offsets header generation doesn't work on Linux atm because of missing support for it in the Mono utility used to generate the offsets. Hopefully this limitation will be removed in the near future and a start-to-end build of everything will be possible on Linux. It is now mandatory to run at least `make prepare-props` before Xamarin.Android can be built. The target generates the OS-specific props file which is required by the build. `make prepare` depends on the target.
2016-07-26 16:27:31 +03:00
/>
</Target>
<!-- The condition below is to work around a bug in xbuild which attempts to batch
even if there are no _MonoCrossRuntime items and the Copy task fails
-->
<Target Name="_InstallCrossRuntimes"
[mono-runtimes] Use directory separators with `$(OutputPath)`. [The build][0] [is broken][1]: xamarin-android/build-tools/mono-runtimes/mono-runtimes.targets: error : Could not find a part of the path ".../xamarin-android/xamarin-android/bin/Debuglib/xbuild-frameworks/MonoAndroid/v1.0/RedistList/FrameworkList.xml". The error is generated from the `_InstallBcl` target when writing `FrameworkList.xml`, and a close reading of the error message shows that a directory separator is missing: it should be `bin/Debug/lib`, but is instead `bin/Debuglib`. I *assume* that this is due to `<CallTarget/>` being "flakey" within xbuild (it passes the file containing the target to `<MSBuild/>`!), and *somehow* in that "flakiness" `$(OutputPath)` doesn't end with `\`, *which it should*, because `Microsoft.Common.targets` will automatically append `\` to `$(OutputPath)`. To work around this issue, update `mono-runtimes.targets` so that it *always* uses `\` after `$(OutputPath)` and `$(IntermediateOutputPath)`. Additionally, `llc.exe` was being installed into the wrong directory, preventing the bundle from being used: Target _BuildUnlessCached needs to be built as output file '../../bin/Debug/lib/mandroid/llc.exe' does not exist. `llc.exe` was instead installed into `bin/Debug/bin/llc.exe`. Bump the bundle version number to -v3 and correct `mono-runtimes.targets` so that `llc.exe` and `opt.exe` are properly installed into `$prefix/lib/mandroid`. [0]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android/49/ [1]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android/49/consoleText
2016-08-17 04:18:15 +03:00
Inputs="$(IntermediateOutputPath)\%(_MonoCrossRuntime.Identity)\mono\mini\mono-sgen%(_MonoCrossRuntime.ExeSuffix)"
[structure] Rework installation directory structure (#704) Context: https://github.com/xamarin/xamarin-android/pull/253#discussion_r82862993 The *intention* is that Jenkins-produced `oss-xamarin.android*.zip` artifacts be usable on Windows, so that side-by-side testing can be performed without replacing the system installation. Usage is in [UsingJenkinsBuildArtifacts.md](Documentation/UsingJenkinsBuildArtifacts). This isn't *entirely* the case. It was *apparently* the case at the time of commit 87ca2737, but things have bitrotten since. For example, following the 87ca2737 instructions would currently result in an XA0020 `Could not find monodroid` error, because `class-parse.exe` wasn't where Windows expects it to be (it was in `lib/mandroid`, not `lib/xbuild/Xamarin/Android`). This needs to be fixed. Additionally, PR #253 mentions that, for filesystem organization, it would be useful if the macOS/Linux directory structure -- `$prefix/bin`, `$prefix/lib/mandroid`, `$prefix/lib/xbuild/Xamarin/Android`, `$prefix/lib/xbuild-frameworks` -- more closely resembled the Windows directory structure of `$MSBuild` and `$ReferenceAssemblies` (as seen in `.vsix` files). This would turn macOS/Linux into using `$xa_prefix/xbuild` and `$xa_prefix/xbuild-frameworks` directories. `$prefix/bin` would only contain `xabuild`. What is currently in `$prefix/lib/mandroid` would be merged with `$xa_prefix/xbuild/Xamarin/Android`. `$xa_prefix` would `$prefix/lib/xamarin.android`. This would turn the current macOS structure: $prefix/bin/xabuild $prefix/bin/generator $prefix/bin/cross-arm $prefix/lib/mandroid/generator.exe $prefix/lib/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xbuild/Xamarin/Android/Xamarin.Android.Common.targets Into: $prefix/bin/xabuild $prefix/lib/xamarin.android/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xamarin.android/xbuild/Xamarin/Android/generator.exe $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Xamarin.Android.Common.targets $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Darwin/cross-arm Other notes: * The `bundle-*.zip` filename has been has been changed to include a *hash* of the contents of various files, in particular `build-tools\mono-runtimes.*`. This was instigated via a conversation with @kumpera about making the filename more "idiot-proof": `mono-runtimes.props` contains *compiler flags*, and if any of those change, then *logically* the bundle should differ as well, as the mono runtimes may differ in significant ways. In theory, the `-vXX` version should be used to track this, but this is a manual change, easy to overlook. The new `-hHASH` part of the filename should be more automagic. The new `<HashFileContents/>` task in `xa-prep-tasks.dll` is responsible for creating the hash value. * `Configuration.Java.Interop.Override.props` was moved into `build-tools/scripts`, because that would cleanup the root directory a bit. * OS-specific binaries are now placed into `$prefix/lib/xamarin.android/xbuild/Xamarin/Android/$(HostOS)`. On a theoretical plus side, this means that the same build directory can contain OS-specific binaries for multiple operating systems. (I don't know if anyone shares a build tree between e.g. macOS and Linux, but if anyone *does*...) Unfortunately this requires a workaround for an `xbuild` bug: `%(_LlvmRuntime.InstallPath)` and `%(_MonoCrossRuntime.InstallPath)` *cannot* use MSBuild properties, e.g. `<InstallPath>$(HostOS)/</InstallPath>` doesn't work as desired; it's instead treated literally. Special-case `%(InstallPath)` until we fully migrate to MSBuild. * `$(MonoAndroidToolsDirectory)` should be considered *dead*, along with `$(MonoAndroidBinDirectory)`, as these should now *always* be the same directory as where `Xamarin.Android.Build.Tasks.dll` is located, or a `$(HostOS)` sub-directory. * `Xamarin.ProjectTools.csproj` needed to be updated to ensure that the build order was correct. * Remove all `[Obsolete]` and unreferenced members from `Xamarin.Android.Build.Utilities.dll`. There's too much in there, and it makes my head hurt trying to understand the interrelationships between it all. If it's not used, it's gone. * The changes to `src/monodroid/jni/Android.mk` are...weird. The removal of `-I$(topdir)/libmonodroid/zip`/etc. is to reduce use of non-existent paths, as `$(topdir)` isn't defined, so that's *actually* `-I/libmonodroid/zip`, which is nonsensical. So far, so good. What's *odd* is the addition of `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`: This is needed so that `external/mono/support/zlib-helper.c` exports `CreateZStream` and related symbols, otherwise we get a unit test failure in `GzipStreamTest.Compression` due to an `EntryPointNotFoundException`, because `CreateZStream` isn't exported/public. What's odd here is that I don't understand what caused this behavior to change. Previous builds exported `CreateZStream`, otherwise the tests would fail, and I don't understand how any of the other changes in this PR would be at fault, though that's certainly the most plausible explanation. Regardless, `-Ijni` *first* (due to adding `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`) is the desired behavior, so that `jni/config.h` is included, thus ensuring that `MONO_API` has the required definition when building `zlib-helper.c`.
2017-07-28 16:36:32 +03:00
Outputs="$(_MSBuildDir)\%(_MonoCrossRuntime.InstallPath)%(_MonoCrossRuntime.CrossMonoName)%(_MonoCrossRuntime.ExeSuffix)"
[mono-runtimes] Build AOT+LLVM cross-compilers (#125) The commit implements building of LLVM and cross-compilers to support Xamarin.Android/Mono AOT. LLVM and cross-compilers can be built for both the host platform (Linux and OS/X at the moment) as well as cross-compiled for 32-bit and 64-bit Windows platforms. Windows builds are done with MXE toolchain on OS/X and with the packaged mingw-w64 toolchain on Linux (tested on Ubuntu 16.04 ONLY). Also introducing a new set of MSBuild properties that contain information about the host system. Some of those properties (HostOS, HostCC, HostCXX for instance) have been moved from Configuration.props to better support auto-detection. A new script, build-tools/scripts/generate-os-info, is invoked as part of `make prepare` to generate file that contains the new properties. The generated file is required for the build to work and is also host-specific (it mustn't be moved between different machines) Cross compiler builds require access to a configured Mono build tree, in order to generate C structure offsets header file that is used by the AOT compilers to properly generate AOT-ed binaries. Therefore, even if a JIT target is not enabled in the configuration, enabling a cross-compiler for some target will configure Mono for that JIT target but it will NOT build it, to save time. To facilitate this, the _MonoRuntimes items defined in build-tools/mono-runtimes/mono-runtimes.projitems gain an additional metadata item called `DoBuild` which will be set to `true` if the runtime actually needs to be built, as opposed to just configured. MXE builds are disabled on Linux as mingw-w64 works just fine. A `make prepare` warning is issued for Linux hosts which have the binfmt_misc module enabled and either Wine of Mono (cli) registered as PE32/PE32+ binary interpreters. In such instance building of the Windows cross-compilers will fail because Autotools determine whether software is being cross compiled by building a test program and attempting to execute it. In normal circumstances such an attempt will fail, but with Windows cross-compilation and either Wine or Mono registered to handle the PE32 executables this attempt will succeed thus causing the cross compilation detection to fail. Currently to build cross compilers on Linux you need to generate the C structure offsets header file on OS/X and copy the resulting headers to appropriate places on Linux. The header files should be placed in build-tools/mono-runtimes/obj/Debug/cross-*/ directories. The header files are: {cross-arm,cross-arm-win}/aarch64-v8a-linux-android.h {cross-arm64,cross-arm64-win}/armv5-none-linux-androideabi.h {cross-x86,cross-x86-win}/i686-none-linux-android.h {cross-x86_64,cross-x86_64-win}/x86_64-none-linux-android.h Offsets header generation doesn't work on Linux atm because of missing support for it in the Mono utility used to generate the offsets. Hopefully this limitation will be removed in the near future and a start-to-end build of everything will be possible on Linux. It is now mandatory to run at least `make prepare-props` before Xamarin.Android can be built. The target generates the OS-specific props file which is required by the build. `make prepare` depends on the target.
2016-07-26 16:27:31 +03:00
Condition=" '@(_MonoCrossRuntime)' != '' ">
<Copy
[mono-runtimes] Use directory separators with `$(OutputPath)`. [The build][0] [is broken][1]: xamarin-android/build-tools/mono-runtimes/mono-runtimes.targets: error : Could not find a part of the path ".../xamarin-android/xamarin-android/bin/Debuglib/xbuild-frameworks/MonoAndroid/v1.0/RedistList/FrameworkList.xml". The error is generated from the `_InstallBcl` target when writing `FrameworkList.xml`, and a close reading of the error message shows that a directory separator is missing: it should be `bin/Debug/lib`, but is instead `bin/Debuglib`. I *assume* that this is due to `<CallTarget/>` being "flakey" within xbuild (it passes the file containing the target to `<MSBuild/>`!), and *somehow* in that "flakiness" `$(OutputPath)` doesn't end with `\`, *which it should*, because `Microsoft.Common.targets` will automatically append `\` to `$(OutputPath)`. To work around this issue, update `mono-runtimes.targets` so that it *always* uses `\` after `$(OutputPath)` and `$(IntermediateOutputPath)`. Additionally, `llc.exe` was being installed into the wrong directory, preventing the bundle from being used: Target _BuildUnlessCached needs to be built as output file '../../bin/Debug/lib/mandroid/llc.exe' does not exist. `llc.exe` was instead installed into `bin/Debug/bin/llc.exe`. Bump the bundle version number to -v3 and correct `mono-runtimes.targets` so that `llc.exe` and `opt.exe` are properly installed into `$prefix/lib/mandroid`. [0]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android/49/ [1]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android/49/consoleText
2016-08-17 04:18:15 +03:00
SourceFiles="$(IntermediateOutputPath)\%(_MonoCrossRuntime.Identity)\mono\mini\mono-sgen%(_MonoCrossRuntime.ExeSuffix)"
[structure] Rework installation directory structure (#704) Context: https://github.com/xamarin/xamarin-android/pull/253#discussion_r82862993 The *intention* is that Jenkins-produced `oss-xamarin.android*.zip` artifacts be usable on Windows, so that side-by-side testing can be performed without replacing the system installation. Usage is in [UsingJenkinsBuildArtifacts.md](Documentation/UsingJenkinsBuildArtifacts). This isn't *entirely* the case. It was *apparently* the case at the time of commit 87ca2737, but things have bitrotten since. For example, following the 87ca2737 instructions would currently result in an XA0020 `Could not find monodroid` error, because `class-parse.exe` wasn't where Windows expects it to be (it was in `lib/mandroid`, not `lib/xbuild/Xamarin/Android`). This needs to be fixed. Additionally, PR #253 mentions that, for filesystem organization, it would be useful if the macOS/Linux directory structure -- `$prefix/bin`, `$prefix/lib/mandroid`, `$prefix/lib/xbuild/Xamarin/Android`, `$prefix/lib/xbuild-frameworks` -- more closely resembled the Windows directory structure of `$MSBuild` and `$ReferenceAssemblies` (as seen in `.vsix` files). This would turn macOS/Linux into using `$xa_prefix/xbuild` and `$xa_prefix/xbuild-frameworks` directories. `$prefix/bin` would only contain `xabuild`. What is currently in `$prefix/lib/mandroid` would be merged with `$xa_prefix/xbuild/Xamarin/Android`. `$xa_prefix` would `$prefix/lib/xamarin.android`. This would turn the current macOS structure: $prefix/bin/xabuild $prefix/bin/generator $prefix/bin/cross-arm $prefix/lib/mandroid/generator.exe $prefix/lib/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xbuild/Xamarin/Android/Xamarin.Android.Common.targets Into: $prefix/bin/xabuild $prefix/lib/xamarin.android/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xamarin.android/xbuild/Xamarin/Android/generator.exe $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Xamarin.Android.Common.targets $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Darwin/cross-arm Other notes: * The `bundle-*.zip` filename has been has been changed to include a *hash* of the contents of various files, in particular `build-tools\mono-runtimes.*`. This was instigated via a conversation with @kumpera about making the filename more "idiot-proof": `mono-runtimes.props` contains *compiler flags*, and if any of those change, then *logically* the bundle should differ as well, as the mono runtimes may differ in significant ways. In theory, the `-vXX` version should be used to track this, but this is a manual change, easy to overlook. The new `-hHASH` part of the filename should be more automagic. The new `<HashFileContents/>` task in `xa-prep-tasks.dll` is responsible for creating the hash value. * `Configuration.Java.Interop.Override.props` was moved into `build-tools/scripts`, because that would cleanup the root directory a bit. * OS-specific binaries are now placed into `$prefix/lib/xamarin.android/xbuild/Xamarin/Android/$(HostOS)`. On a theoretical plus side, this means that the same build directory can contain OS-specific binaries for multiple operating systems. (I don't know if anyone shares a build tree between e.g. macOS and Linux, but if anyone *does*...) Unfortunately this requires a workaround for an `xbuild` bug: `%(_LlvmRuntime.InstallPath)` and `%(_MonoCrossRuntime.InstallPath)` *cannot* use MSBuild properties, e.g. `<InstallPath>$(HostOS)/</InstallPath>` doesn't work as desired; it's instead treated literally. Special-case `%(InstallPath)` until we fully migrate to MSBuild. * `$(MonoAndroidToolsDirectory)` should be considered *dead*, along with `$(MonoAndroidBinDirectory)`, as these should now *always* be the same directory as where `Xamarin.Android.Build.Tasks.dll` is located, or a `$(HostOS)` sub-directory. * `Xamarin.ProjectTools.csproj` needed to be updated to ensure that the build order was correct. * Remove all `[Obsolete]` and unreferenced members from `Xamarin.Android.Build.Utilities.dll`. There's too much in there, and it makes my head hurt trying to understand the interrelationships between it all. If it's not used, it's gone. * The changes to `src/monodroid/jni/Android.mk` are...weird. The removal of `-I$(topdir)/libmonodroid/zip`/etc. is to reduce use of non-existent paths, as `$(topdir)` isn't defined, so that's *actually* `-I/libmonodroid/zip`, which is nonsensical. So far, so good. What's *odd* is the addition of `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`: This is needed so that `external/mono/support/zlib-helper.c` exports `CreateZStream` and related symbols, otherwise we get a unit test failure in `GzipStreamTest.Compression` due to an `EntryPointNotFoundException`, because `CreateZStream` isn't exported/public. What's odd here is that I don't understand what caused this behavior to change. Previous builds exported `CreateZStream`, otherwise the tests would fail, and I don't understand how any of the other changes in this PR would be at fault, though that's certainly the most plausible explanation. Regardless, `-Ijni` *first* (due to adding `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`) is the desired behavior, so that `jni/config.h` is included, thus ensuring that `MONO_API` has the required definition when building `zlib-helper.c`.
2017-07-28 16:36:32 +03:00
DestinationFiles="$(_MSBuildDir)\%(_MonoCrossRuntime.InstallPath)%(_MonoCrossRuntime.CrossMonoName)%(_MonoCrossRuntime.ExeSuffix)"
[mono-runtimes] Build AOT+LLVM cross-compilers (#125) The commit implements building of LLVM and cross-compilers to support Xamarin.Android/Mono AOT. LLVM and cross-compilers can be built for both the host platform (Linux and OS/X at the moment) as well as cross-compiled for 32-bit and 64-bit Windows platforms. Windows builds are done with MXE toolchain on OS/X and with the packaged mingw-w64 toolchain on Linux (tested on Ubuntu 16.04 ONLY). Also introducing a new set of MSBuild properties that contain information about the host system. Some of those properties (HostOS, HostCC, HostCXX for instance) have been moved from Configuration.props to better support auto-detection. A new script, build-tools/scripts/generate-os-info, is invoked as part of `make prepare` to generate file that contains the new properties. The generated file is required for the build to work and is also host-specific (it mustn't be moved between different machines) Cross compiler builds require access to a configured Mono build tree, in order to generate C structure offsets header file that is used by the AOT compilers to properly generate AOT-ed binaries. Therefore, even if a JIT target is not enabled in the configuration, enabling a cross-compiler for some target will configure Mono for that JIT target but it will NOT build it, to save time. To facilitate this, the _MonoRuntimes items defined in build-tools/mono-runtimes/mono-runtimes.projitems gain an additional metadata item called `DoBuild` which will be set to `true` if the runtime actually needs to be built, as opposed to just configured. MXE builds are disabled on Linux as mingw-w64 works just fine. A `make prepare` warning is issued for Linux hosts which have the binfmt_misc module enabled and either Wine of Mono (cli) registered as PE32/PE32+ binary interpreters. In such instance building of the Windows cross-compilers will fail because Autotools determine whether software is being cross compiled by building a test program and attempting to execute it. In normal circumstances such an attempt will fail, but with Windows cross-compilation and either Wine or Mono registered to handle the PE32 executables this attempt will succeed thus causing the cross compilation detection to fail. Currently to build cross compilers on Linux you need to generate the C structure offsets header file on OS/X and copy the resulting headers to appropriate places on Linux. The header files should be placed in build-tools/mono-runtimes/obj/Debug/cross-*/ directories. The header files are: {cross-arm,cross-arm-win}/aarch64-v8a-linux-android.h {cross-arm64,cross-arm64-win}/armv5-none-linux-androideabi.h {cross-x86,cross-x86-win}/i686-none-linux-android.h {cross-x86_64,cross-x86_64-win}/x86_64-none-linux-android.h Offsets header generation doesn't work on Linux atm because of missing support for it in the Mono utility used to generate the offsets. Hopefully this limitation will be removed in the near future and a start-to-end build of everything will be possible on Linux. It is now mandatory to run at least `make prepare-props` before Xamarin.Android can be built. The target generates the OS-specific props file which is required by the build. `make prepare` depends on the target.
2016-07-26 16:27:31 +03:00
/>
[create-vsix] Create `.vsix` files (#541) Visual Studio 2017 uses `.vsix` files for Xamarin.Android SDK support. A `.vsix` file is [ZIP container with additional metadata][0], and the [Microsoft.VSSDK.BuildTools NuGet package][1] contains various MSBuild targets and tools to assist in creating `.vsix` files. Add a new `build-tools/create-vsix/create-vsix.csproj` project to create the `bin/Build$(Configuration)/Xamarin.Android.Sdk*.vsix` file, so that we can plausibly provide per-build OSS Xamarin.Android releases that work with Visual Studio 2017. Unfortunately those tools were written on Windows, and not really well tested on macOS or Linux... In particular, there are case-sensitivity and directory-separator-char issues with the tooling, necessitating that `MONO_IOMAP=all` be exported in order to run them: MONO_IOMAP=all MONO_OPTIONS=--arch=64 msbuild \ build-tools/create-vsix/create-vsix.csproj /p:CreateVsixContainer=True Meanwhile, we're still attempting to allow things to be built with `xbuild` [^3]. Thread this needle by "special-casing" the `$(BuildDependsOn)`, `$(CopyVsixManifestFileDependsOn)`, and `$(DetokenizeVsixManifestFileDependsOn)` MSBuild properties so that when the `$(CreateVsixContainer)` MSBuild property is False -- the default -- no `.vsix` package will be created, and `xbuild` will be able to build the project. Similar needle threading is needed to "support" building on Linux: the Microsoft.VSSDK.BuildTools NuGet package uses case in an inconsistent fashion -- or is it `nuget`? -- which results in [failures when building on Linux][2] because `nuget` extracts e.g.: packages/Microsoft.VSSDK.BuildTools.15.0.26201/tools/vssdk/Microsoft.VsSDK.targets while Microsoft.VSSDK.BuildTools attempts to `<Import/>` the file: packages/Microsoft.VSSDK.BuildTools.15.0.26201/tools/VSSDK/Microsoft.VsSDK.targets `vssdk` != `VSSDK` on case-sensitive filesystems, so this results in an error on Ubuntu (and presumably case-senstive macOS as well). Handle the Linux case by extending the `xbuild` case: *even when* `$(CreateVsixContainer)` is True, we won't actually build the `.vsix` unless the `$(VsSDKInstall)` directory exists, which is the `tools/VSSDK` path. Building with `$(CreateVsixContainer)` set to True will require using `msbuild` with `MONO_IOMAP=all` and `MONO_OPPTIONS=--arch=64` exported, while using a case-insensitive filesystem. The new `make create-vsix CONFIGURATIONS=Debug` target can be used to explicitly create the `Xamarin.Android.Sdk*.vsix` file. One problem with creating `.vsix` files: macOS still defaults to using a 32-bit process for `mono`, and `make create-vsix` regularly fails for me with an `OutOfMemoryException` when attempting to generate a `.vsix` file ~600-700+MB in size. (Why so large? In part because for Debug configuration we're including un-`strip`'d native libraries; `libmonosgen-2.0.dll` is *237MB* in size (!); it's closer to 5MB when `strip`'d, but any un-`strip`'d MXE-generated native libraries to the `.vsix` file quickly results in `OutOfMemoryException`s.) A 64-bit mono can be used by using `mono64` -- which isn't easily done with `msbuild` -- or by using `mono --arch=64`, which *can* be done with `msbuild` by exporting `MONO_OPTIONS=--arch=64`. The `make create-vsix` target does this. Finally, once we have a `.visx` being created, we then examine the *contents* of the `.vsix` file, which is...weird: $MSBuild/Xamarin/Android/Xamarin.Android.CSharp.targets # Desirable $ReferenceAssemblies/Microsoft/Framework/MonoAndroid/v1.0/Xamarin.Android.CSharp.targets # wat? $ReferenceAssemblies/Microsoft/Framework/MonoAndroid/v1.0/Facades/Xamarin.Android.CSharp.targets # really?! $ReferenceAssemblies/Microsoft/Framework/MonoAndroid/v7.1/Xamarin.Android.CSharp.targets # !!!! Turns Out™, the problem was due to `@(ProjectReference)`. We're (ab)using `@(ProjectReference)` to explicitly specify project dependencies, and the (implicit) project build order. Unexpectedly (forgetten?), MSBuild *also* copies the *outputs* of `@(ProjectReference)`s into the `$(OutputPath)` of the current project. Since e.g. `Mono.Posix.csproj` had a reference on `Xamarin.Android.Build.Tasks.csproj`, the result of this is that the `$(OutputPath)` of `Mono.Posix.csproj` -- `$prefix/lib/xbuild-frameworks/MonoAndroid/v1.0` -- also contained the `%(None.CopyToOutputDirectory)` items from `Xamarin.Android.Build.Tasks.csproj`. Oops. The fix is to set `%(ProjectReference.Private)` to False, which disables this copying of additional and undesirable files. [0]: https://msdn.microsoft.com/en-us/library/dd997148.aspx [1]: https://www.nuget.org/packages/Microsoft.VSSDK.BuildTools/ [2]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-anroid-linux-pr-builder/297/consoleText [^3]: But for how much longer?
2017-04-05 18:49:37 +03:00
<Copy
[structure] Rework installation directory structure (#704) Context: https://github.com/xamarin/xamarin-android/pull/253#discussion_r82862993 The *intention* is that Jenkins-produced `oss-xamarin.android*.zip` artifacts be usable on Windows, so that side-by-side testing can be performed without replacing the system installation. Usage is in [UsingJenkinsBuildArtifacts.md](Documentation/UsingJenkinsBuildArtifacts). This isn't *entirely* the case. It was *apparently* the case at the time of commit 87ca2737, but things have bitrotten since. For example, following the 87ca2737 instructions would currently result in an XA0020 `Could not find monodroid` error, because `class-parse.exe` wasn't where Windows expects it to be (it was in `lib/mandroid`, not `lib/xbuild/Xamarin/Android`). This needs to be fixed. Additionally, PR #253 mentions that, for filesystem organization, it would be useful if the macOS/Linux directory structure -- `$prefix/bin`, `$prefix/lib/mandroid`, `$prefix/lib/xbuild/Xamarin/Android`, `$prefix/lib/xbuild-frameworks` -- more closely resembled the Windows directory structure of `$MSBuild` and `$ReferenceAssemblies` (as seen in `.vsix` files). This would turn macOS/Linux into using `$xa_prefix/xbuild` and `$xa_prefix/xbuild-frameworks` directories. `$prefix/bin` would only contain `xabuild`. What is currently in `$prefix/lib/mandroid` would be merged with `$xa_prefix/xbuild/Xamarin/Android`. `$xa_prefix` would `$prefix/lib/xamarin.android`. This would turn the current macOS structure: $prefix/bin/xabuild $prefix/bin/generator $prefix/bin/cross-arm $prefix/lib/mandroid/generator.exe $prefix/lib/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xbuild/Xamarin/Android/Xamarin.Android.Common.targets Into: $prefix/bin/xabuild $prefix/lib/xamarin.android/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xamarin.android/xbuild/Xamarin/Android/generator.exe $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Xamarin.Android.Common.targets $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Darwin/cross-arm Other notes: * The `bundle-*.zip` filename has been has been changed to include a *hash* of the contents of various files, in particular `build-tools\mono-runtimes.*`. This was instigated via a conversation with @kumpera about making the filename more "idiot-proof": `mono-runtimes.props` contains *compiler flags*, and if any of those change, then *logically* the bundle should differ as well, as the mono runtimes may differ in significant ways. In theory, the `-vXX` version should be used to track this, but this is a manual change, easy to overlook. The new `-hHASH` part of the filename should be more automagic. The new `<HashFileContents/>` task in `xa-prep-tasks.dll` is responsible for creating the hash value. * `Configuration.Java.Interop.Override.props` was moved into `build-tools/scripts`, because that would cleanup the root directory a bit. * OS-specific binaries are now placed into `$prefix/lib/xamarin.android/xbuild/Xamarin/Android/$(HostOS)`. On a theoretical plus side, this means that the same build directory can contain OS-specific binaries for multiple operating systems. (I don't know if anyone shares a build tree between e.g. macOS and Linux, but if anyone *does*...) Unfortunately this requires a workaround for an `xbuild` bug: `%(_LlvmRuntime.InstallPath)` and `%(_MonoCrossRuntime.InstallPath)` *cannot* use MSBuild properties, e.g. `<InstallPath>$(HostOS)/</InstallPath>` doesn't work as desired; it's instead treated literally. Special-case `%(InstallPath)` until we fully migrate to MSBuild. * `$(MonoAndroidToolsDirectory)` should be considered *dead*, along with `$(MonoAndroidBinDirectory)`, as these should now *always* be the same directory as where `Xamarin.Android.Build.Tasks.dll` is located, or a `$(HostOS)` sub-directory. * `Xamarin.ProjectTools.csproj` needed to be updated to ensure that the build order was correct. * Remove all `[Obsolete]` and unreferenced members from `Xamarin.Android.Build.Utilities.dll`. There's too much in there, and it makes my head hurt trying to understand the interrelationships between it all. If it's not used, it's gone. * The changes to `src/monodroid/jni/Android.mk` are...weird. The removal of `-I$(topdir)/libmonodroid/zip`/etc. is to reduce use of non-existent paths, as `$(topdir)` isn't defined, so that's *actually* `-I/libmonodroid/zip`, which is nonsensical. So far, so good. What's *odd* is the addition of `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`: This is needed so that `external/mono/support/zlib-helper.c` exports `CreateZStream` and related symbols, otherwise we get a unit test failure in `GzipStreamTest.Compression` due to an `EntryPointNotFoundException`, because `CreateZStream` isn't exported/public. What's odd here is that I don't understand what caused this behavior to change. Previous builds exported `CreateZStream`, otherwise the tests would fail, and I don't understand how any of the other changes in this PR would be at fault, though that's certainly the most plausible explanation. Regardless, `-Ijni` *first* (due to adding `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`) is the desired behavior, so that `jni/config.h` is included, thus ensuring that `MONO_API` has the required definition when building `zlib-helper.c`.
2017-07-28 16:36:32 +03:00
SourceFiles="$(_MSBuildDir)\%(_MonoCrossRuntime.InstallPath)%(_MonoCrossRuntime.CrossMonoName)%(_MonoCrossRuntime.ExeSuffix)"
DestinationFiles="$(_MSBuildDir)\%(_MonoCrossRuntime.InstallPath)%(_MonoCrossRuntime.CrossMonoName).d%(_MonoCrossRuntime.ExeSuffix)"
[create-vsix] Create `.vsix` files (#541) Visual Studio 2017 uses `.vsix` files for Xamarin.Android SDK support. A `.vsix` file is [ZIP container with additional metadata][0], and the [Microsoft.VSSDK.BuildTools NuGet package][1] contains various MSBuild targets and tools to assist in creating `.vsix` files. Add a new `build-tools/create-vsix/create-vsix.csproj` project to create the `bin/Build$(Configuration)/Xamarin.Android.Sdk*.vsix` file, so that we can plausibly provide per-build OSS Xamarin.Android releases that work with Visual Studio 2017. Unfortunately those tools were written on Windows, and not really well tested on macOS or Linux... In particular, there are case-sensitivity and directory-separator-char issues with the tooling, necessitating that `MONO_IOMAP=all` be exported in order to run them: MONO_IOMAP=all MONO_OPTIONS=--arch=64 msbuild \ build-tools/create-vsix/create-vsix.csproj /p:CreateVsixContainer=True Meanwhile, we're still attempting to allow things to be built with `xbuild` [^3]. Thread this needle by "special-casing" the `$(BuildDependsOn)`, `$(CopyVsixManifestFileDependsOn)`, and `$(DetokenizeVsixManifestFileDependsOn)` MSBuild properties so that when the `$(CreateVsixContainer)` MSBuild property is False -- the default -- no `.vsix` package will be created, and `xbuild` will be able to build the project. Similar needle threading is needed to "support" building on Linux: the Microsoft.VSSDK.BuildTools NuGet package uses case in an inconsistent fashion -- or is it `nuget`? -- which results in [failures when building on Linux][2] because `nuget` extracts e.g.: packages/Microsoft.VSSDK.BuildTools.15.0.26201/tools/vssdk/Microsoft.VsSDK.targets while Microsoft.VSSDK.BuildTools attempts to `<Import/>` the file: packages/Microsoft.VSSDK.BuildTools.15.0.26201/tools/VSSDK/Microsoft.VsSDK.targets `vssdk` != `VSSDK` on case-sensitive filesystems, so this results in an error on Ubuntu (and presumably case-senstive macOS as well). Handle the Linux case by extending the `xbuild` case: *even when* `$(CreateVsixContainer)` is True, we won't actually build the `.vsix` unless the `$(VsSDKInstall)` directory exists, which is the `tools/VSSDK` path. Building with `$(CreateVsixContainer)` set to True will require using `msbuild` with `MONO_IOMAP=all` and `MONO_OPPTIONS=--arch=64` exported, while using a case-insensitive filesystem. The new `make create-vsix CONFIGURATIONS=Debug` target can be used to explicitly create the `Xamarin.Android.Sdk*.vsix` file. One problem with creating `.vsix` files: macOS still defaults to using a 32-bit process for `mono`, and `make create-vsix` regularly fails for me with an `OutOfMemoryException` when attempting to generate a `.vsix` file ~600-700+MB in size. (Why so large? In part because for Debug configuration we're including un-`strip`'d native libraries; `libmonosgen-2.0.dll` is *237MB* in size (!); it's closer to 5MB when `strip`'d, but any un-`strip`'d MXE-generated native libraries to the `.vsix` file quickly results in `OutOfMemoryException`s.) A 64-bit mono can be used by using `mono64` -- which isn't easily done with `msbuild` -- or by using `mono --arch=64`, which *can* be done with `msbuild` by exporting `MONO_OPTIONS=--arch=64`. The `make create-vsix` target does this. Finally, once we have a `.visx` being created, we then examine the *contents* of the `.vsix` file, which is...weird: $MSBuild/Xamarin/Android/Xamarin.Android.CSharp.targets # Desirable $ReferenceAssemblies/Microsoft/Framework/MonoAndroid/v1.0/Xamarin.Android.CSharp.targets # wat? $ReferenceAssemblies/Microsoft/Framework/MonoAndroid/v1.0/Facades/Xamarin.Android.CSharp.targets # really?! $ReferenceAssemblies/Microsoft/Framework/MonoAndroid/v7.1/Xamarin.Android.CSharp.targets # !!!! Turns Out™, the problem was due to `@(ProjectReference)`. We're (ab)using `@(ProjectReference)` to explicitly specify project dependencies, and the (implicit) project build order. Unexpectedly (forgetten?), MSBuild *also* copies the *outputs* of `@(ProjectReference)`s into the `$(OutputPath)` of the current project. Since e.g. `Mono.Posix.csproj` had a reference on `Xamarin.Android.Build.Tasks.csproj`, the result of this is that the `$(OutputPath)` of `Mono.Posix.csproj` -- `$prefix/lib/xbuild-frameworks/MonoAndroid/v1.0` -- also contained the `%(None.CopyToOutputDirectory)` items from `Xamarin.Android.Build.Tasks.csproj`. Oops. The fix is to set `%(ProjectReference.Private)` to False, which disables this copying of additional and undesirable files. [0]: https://msdn.microsoft.com/en-us/library/dd997148.aspx [1]: https://www.nuget.org/packages/Microsoft.VSSDK.BuildTools/ [2]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-anroid-linux-pr-builder/297/consoleText [^3]: But for how much longer?
2017-04-05 18:49:37 +03:00
/>
<Exec
Condition=" '$(Configuration)' != 'Debug' Or '%(_MonoCrossRuntime.ExeSuffix)' == '.exe' "
[structure] Rework installation directory structure (#704) Context: https://github.com/xamarin/xamarin-android/pull/253#discussion_r82862993 The *intention* is that Jenkins-produced `oss-xamarin.android*.zip` artifacts be usable on Windows, so that side-by-side testing can be performed without replacing the system installation. Usage is in [UsingJenkinsBuildArtifacts.md](Documentation/UsingJenkinsBuildArtifacts). This isn't *entirely* the case. It was *apparently* the case at the time of commit 87ca2737, but things have bitrotten since. For example, following the 87ca2737 instructions would currently result in an XA0020 `Could not find monodroid` error, because `class-parse.exe` wasn't where Windows expects it to be (it was in `lib/mandroid`, not `lib/xbuild/Xamarin/Android`). This needs to be fixed. Additionally, PR #253 mentions that, for filesystem organization, it would be useful if the macOS/Linux directory structure -- `$prefix/bin`, `$prefix/lib/mandroid`, `$prefix/lib/xbuild/Xamarin/Android`, `$prefix/lib/xbuild-frameworks` -- more closely resembled the Windows directory structure of `$MSBuild` and `$ReferenceAssemblies` (as seen in `.vsix` files). This would turn macOS/Linux into using `$xa_prefix/xbuild` and `$xa_prefix/xbuild-frameworks` directories. `$prefix/bin` would only contain `xabuild`. What is currently in `$prefix/lib/mandroid` would be merged with `$xa_prefix/xbuild/Xamarin/Android`. `$xa_prefix` would `$prefix/lib/xamarin.android`. This would turn the current macOS structure: $prefix/bin/xabuild $prefix/bin/generator $prefix/bin/cross-arm $prefix/lib/mandroid/generator.exe $prefix/lib/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xbuild/Xamarin/Android/Xamarin.Android.Common.targets Into: $prefix/bin/xabuild $prefix/lib/xamarin.android/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xamarin.android/xbuild/Xamarin/Android/generator.exe $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Xamarin.Android.Common.targets $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Darwin/cross-arm Other notes: * The `bundle-*.zip` filename has been has been changed to include a *hash* of the contents of various files, in particular `build-tools\mono-runtimes.*`. This was instigated via a conversation with @kumpera about making the filename more "idiot-proof": `mono-runtimes.props` contains *compiler flags*, and if any of those change, then *logically* the bundle should differ as well, as the mono runtimes may differ in significant ways. In theory, the `-vXX` version should be used to track this, but this is a manual change, easy to overlook. The new `-hHASH` part of the filename should be more automagic. The new `<HashFileContents/>` task in `xa-prep-tasks.dll` is responsible for creating the hash value. * `Configuration.Java.Interop.Override.props` was moved into `build-tools/scripts`, because that would cleanup the root directory a bit. * OS-specific binaries are now placed into `$prefix/lib/xamarin.android/xbuild/Xamarin/Android/$(HostOS)`. On a theoretical plus side, this means that the same build directory can contain OS-specific binaries for multiple operating systems. (I don't know if anyone shares a build tree between e.g. macOS and Linux, but if anyone *does*...) Unfortunately this requires a workaround for an `xbuild` bug: `%(_LlvmRuntime.InstallPath)` and `%(_MonoCrossRuntime.InstallPath)` *cannot* use MSBuild properties, e.g. `<InstallPath>$(HostOS)/</InstallPath>` doesn't work as desired; it's instead treated literally. Special-case `%(InstallPath)` until we fully migrate to MSBuild. * `$(MonoAndroidToolsDirectory)` should be considered *dead*, along with `$(MonoAndroidBinDirectory)`, as these should now *always* be the same directory as where `Xamarin.Android.Build.Tasks.dll` is located, or a `$(HostOS)` sub-directory. * `Xamarin.ProjectTools.csproj` needed to be updated to ensure that the build order was correct. * Remove all `[Obsolete]` and unreferenced members from `Xamarin.Android.Build.Utilities.dll`. There's too much in there, and it makes my head hurt trying to understand the interrelationships between it all. If it's not used, it's gone. * The changes to `src/monodroid/jni/Android.mk` are...weird. The removal of `-I$(topdir)/libmonodroid/zip`/etc. is to reduce use of non-existent paths, as `$(topdir)` isn't defined, so that's *actually* `-I/libmonodroid/zip`, which is nonsensical. So far, so good. What's *odd* is the addition of `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`: This is needed so that `external/mono/support/zlib-helper.c` exports `CreateZStream` and related symbols, otherwise we get a unit test failure in `GzipStreamTest.Compression` due to an `EntryPointNotFoundException`, because `CreateZStream` isn't exported/public. What's odd here is that I don't understand what caused this behavior to change. Previous builds exported `CreateZStream`, otherwise the tests would fail, and I don't understand how any of the other changes in this PR would be at fault, though that's certainly the most plausible explanation. Regardless, `-Ijni` *first* (due to adding `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`) is the desired behavior, so that `jni/config.h` is included, thus ensuring that `MONO_API` has the required definition when building `zlib-helper.c`.
2017-07-28 16:36:32 +03:00
Command="&quot;%(_MonoCrossRuntime.Strip)&quot; %(_MonoCrossRuntime.StripFlags) &quot;$(_MSBuildDir)\%(_MonoCrossRuntime.InstallPath)%(_MonoCrossRuntime.CrossMonoName)%(_MonoCrossRuntime.ExeSuffix)&quot;"
[create-vsix] Create `.vsix` files (#541) Visual Studio 2017 uses `.vsix` files for Xamarin.Android SDK support. A `.vsix` file is [ZIP container with additional metadata][0], and the [Microsoft.VSSDK.BuildTools NuGet package][1] contains various MSBuild targets and tools to assist in creating `.vsix` files. Add a new `build-tools/create-vsix/create-vsix.csproj` project to create the `bin/Build$(Configuration)/Xamarin.Android.Sdk*.vsix` file, so that we can plausibly provide per-build OSS Xamarin.Android releases that work with Visual Studio 2017. Unfortunately those tools were written on Windows, and not really well tested on macOS or Linux... In particular, there are case-sensitivity and directory-separator-char issues with the tooling, necessitating that `MONO_IOMAP=all` be exported in order to run them: MONO_IOMAP=all MONO_OPTIONS=--arch=64 msbuild \ build-tools/create-vsix/create-vsix.csproj /p:CreateVsixContainer=True Meanwhile, we're still attempting to allow things to be built with `xbuild` [^3]. Thread this needle by "special-casing" the `$(BuildDependsOn)`, `$(CopyVsixManifestFileDependsOn)`, and `$(DetokenizeVsixManifestFileDependsOn)` MSBuild properties so that when the `$(CreateVsixContainer)` MSBuild property is False -- the default -- no `.vsix` package will be created, and `xbuild` will be able to build the project. Similar needle threading is needed to "support" building on Linux: the Microsoft.VSSDK.BuildTools NuGet package uses case in an inconsistent fashion -- or is it `nuget`? -- which results in [failures when building on Linux][2] because `nuget` extracts e.g.: packages/Microsoft.VSSDK.BuildTools.15.0.26201/tools/vssdk/Microsoft.VsSDK.targets while Microsoft.VSSDK.BuildTools attempts to `<Import/>` the file: packages/Microsoft.VSSDK.BuildTools.15.0.26201/tools/VSSDK/Microsoft.VsSDK.targets `vssdk` != `VSSDK` on case-sensitive filesystems, so this results in an error on Ubuntu (and presumably case-senstive macOS as well). Handle the Linux case by extending the `xbuild` case: *even when* `$(CreateVsixContainer)` is True, we won't actually build the `.vsix` unless the `$(VsSDKInstall)` directory exists, which is the `tools/VSSDK` path. Building with `$(CreateVsixContainer)` set to True will require using `msbuild` with `MONO_IOMAP=all` and `MONO_OPPTIONS=--arch=64` exported, while using a case-insensitive filesystem. The new `make create-vsix CONFIGURATIONS=Debug` target can be used to explicitly create the `Xamarin.Android.Sdk*.vsix` file. One problem with creating `.vsix` files: macOS still defaults to using a 32-bit process for `mono`, and `make create-vsix` regularly fails for me with an `OutOfMemoryException` when attempting to generate a `.vsix` file ~600-700+MB in size. (Why so large? In part because for Debug configuration we're including un-`strip`'d native libraries; `libmonosgen-2.0.dll` is *237MB* in size (!); it's closer to 5MB when `strip`'d, but any un-`strip`'d MXE-generated native libraries to the `.vsix` file quickly results in `OutOfMemoryException`s.) A 64-bit mono can be used by using `mono64` -- which isn't easily done with `msbuild` -- or by using `mono --arch=64`, which *can* be done with `msbuild` by exporting `MONO_OPTIONS=--arch=64`. The `make create-vsix` target does this. Finally, once we have a `.visx` being created, we then examine the *contents* of the `.vsix` file, which is...weird: $MSBuild/Xamarin/Android/Xamarin.Android.CSharp.targets # Desirable $ReferenceAssemblies/Microsoft/Framework/MonoAndroid/v1.0/Xamarin.Android.CSharp.targets # wat? $ReferenceAssemblies/Microsoft/Framework/MonoAndroid/v1.0/Facades/Xamarin.Android.CSharp.targets # really?! $ReferenceAssemblies/Microsoft/Framework/MonoAndroid/v7.1/Xamarin.Android.CSharp.targets # !!!! Turns Out™, the problem was due to `@(ProjectReference)`. We're (ab)using `@(ProjectReference)` to explicitly specify project dependencies, and the (implicit) project build order. Unexpectedly (forgetten?), MSBuild *also* copies the *outputs* of `@(ProjectReference)`s into the `$(OutputPath)` of the current project. Since e.g. `Mono.Posix.csproj` had a reference on `Xamarin.Android.Build.Tasks.csproj`, the result of this is that the `$(OutputPath)` of `Mono.Posix.csproj` -- `$prefix/lib/xbuild-frameworks/MonoAndroid/v1.0` -- also contained the `%(None.CopyToOutputDirectory)` items from `Xamarin.Android.Build.Tasks.csproj`. Oops. The fix is to set `%(ProjectReference.Private)` to False, which disables this copying of additional and undesirable files. [0]: https://msdn.microsoft.com/en-us/library/dd997148.aspx [1]: https://www.nuget.org/packages/Microsoft.VSSDK.BuildTools/ [2]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-anroid-linux-pr-builder/297/consoleText [^3]: But for how much longer?
2017-04-05 18:49:37 +03:00
/>
[mono-runtimes] Build AOT+LLVM cross-compilers (#125) The commit implements building of LLVM and cross-compilers to support Xamarin.Android/Mono AOT. LLVM and cross-compilers can be built for both the host platform (Linux and OS/X at the moment) as well as cross-compiled for 32-bit and 64-bit Windows platforms. Windows builds are done with MXE toolchain on OS/X and with the packaged mingw-w64 toolchain on Linux (tested on Ubuntu 16.04 ONLY). Also introducing a new set of MSBuild properties that contain information about the host system. Some of those properties (HostOS, HostCC, HostCXX for instance) have been moved from Configuration.props to better support auto-detection. A new script, build-tools/scripts/generate-os-info, is invoked as part of `make prepare` to generate file that contains the new properties. The generated file is required for the build to work and is also host-specific (it mustn't be moved between different machines) Cross compiler builds require access to a configured Mono build tree, in order to generate C structure offsets header file that is used by the AOT compilers to properly generate AOT-ed binaries. Therefore, even if a JIT target is not enabled in the configuration, enabling a cross-compiler for some target will configure Mono for that JIT target but it will NOT build it, to save time. To facilitate this, the _MonoRuntimes items defined in build-tools/mono-runtimes/mono-runtimes.projitems gain an additional metadata item called `DoBuild` which will be set to `true` if the runtime actually needs to be built, as opposed to just configured. MXE builds are disabled on Linux as mingw-w64 works just fine. A `make prepare` warning is issued for Linux hosts which have the binfmt_misc module enabled and either Wine of Mono (cli) registered as PE32/PE32+ binary interpreters. In such instance building of the Windows cross-compilers will fail because Autotools determine whether software is being cross compiled by building a test program and attempting to execute it. In normal circumstances such an attempt will fail, but with Windows cross-compilation and either Wine or Mono registered to handle the PE32 executables this attempt will succeed thus causing the cross compilation detection to fail. Currently to build cross compilers on Linux you need to generate the C structure offsets header file on OS/X and copy the resulting headers to appropriate places on Linux. The header files should be placed in build-tools/mono-runtimes/obj/Debug/cross-*/ directories. The header files are: {cross-arm,cross-arm-win}/aarch64-v8a-linux-android.h {cross-arm64,cross-arm64-win}/armv5-none-linux-androideabi.h {cross-x86,cross-x86-win}/i686-none-linux-android.h {cross-x86_64,cross-x86_64-win}/x86_64-none-linux-android.h Offsets header generation doesn't work on Linux atm because of missing support for it in the Mono utility used to generate the offsets. Hopefully this limitation will be removed in the near future and a start-to-end build of everything will be possible on Linux. It is now mandatory to run at least `make prepare-props` before Xamarin.Android can be built. The target generates the OS-specific props file which is required by the build. `make prepare` depends on the target.
2016-07-26 16:27:31 +03:00
<Touch
[structure] Rework installation directory structure (#704) Context: https://github.com/xamarin/xamarin-android/pull/253#discussion_r82862993 The *intention* is that Jenkins-produced `oss-xamarin.android*.zip` artifacts be usable on Windows, so that side-by-side testing can be performed without replacing the system installation. Usage is in [UsingJenkinsBuildArtifacts.md](Documentation/UsingJenkinsBuildArtifacts). This isn't *entirely* the case. It was *apparently* the case at the time of commit 87ca2737, but things have bitrotten since. For example, following the 87ca2737 instructions would currently result in an XA0020 `Could not find monodroid` error, because `class-parse.exe` wasn't where Windows expects it to be (it was in `lib/mandroid`, not `lib/xbuild/Xamarin/Android`). This needs to be fixed. Additionally, PR #253 mentions that, for filesystem organization, it would be useful if the macOS/Linux directory structure -- `$prefix/bin`, `$prefix/lib/mandroid`, `$prefix/lib/xbuild/Xamarin/Android`, `$prefix/lib/xbuild-frameworks` -- more closely resembled the Windows directory structure of `$MSBuild` and `$ReferenceAssemblies` (as seen in `.vsix` files). This would turn macOS/Linux into using `$xa_prefix/xbuild` and `$xa_prefix/xbuild-frameworks` directories. `$prefix/bin` would only contain `xabuild`. What is currently in `$prefix/lib/mandroid` would be merged with `$xa_prefix/xbuild/Xamarin/Android`. `$xa_prefix` would `$prefix/lib/xamarin.android`. This would turn the current macOS structure: $prefix/bin/xabuild $prefix/bin/generator $prefix/bin/cross-arm $prefix/lib/mandroid/generator.exe $prefix/lib/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xbuild/Xamarin/Android/Xamarin.Android.Common.targets Into: $prefix/bin/xabuild $prefix/lib/xamarin.android/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xamarin.android/xbuild/Xamarin/Android/generator.exe $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Xamarin.Android.Common.targets $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Darwin/cross-arm Other notes: * The `bundle-*.zip` filename has been has been changed to include a *hash* of the contents of various files, in particular `build-tools\mono-runtimes.*`. This was instigated via a conversation with @kumpera about making the filename more "idiot-proof": `mono-runtimes.props` contains *compiler flags*, and if any of those change, then *logically* the bundle should differ as well, as the mono runtimes may differ in significant ways. In theory, the `-vXX` version should be used to track this, but this is a manual change, easy to overlook. The new `-hHASH` part of the filename should be more automagic. The new `<HashFileContents/>` task in `xa-prep-tasks.dll` is responsible for creating the hash value. * `Configuration.Java.Interop.Override.props` was moved into `build-tools/scripts`, because that would cleanup the root directory a bit. * OS-specific binaries are now placed into `$prefix/lib/xamarin.android/xbuild/Xamarin/Android/$(HostOS)`. On a theoretical plus side, this means that the same build directory can contain OS-specific binaries for multiple operating systems. (I don't know if anyone shares a build tree between e.g. macOS and Linux, but if anyone *does*...) Unfortunately this requires a workaround for an `xbuild` bug: `%(_LlvmRuntime.InstallPath)` and `%(_MonoCrossRuntime.InstallPath)` *cannot* use MSBuild properties, e.g. `<InstallPath>$(HostOS)/</InstallPath>` doesn't work as desired; it's instead treated literally. Special-case `%(InstallPath)` until we fully migrate to MSBuild. * `$(MonoAndroidToolsDirectory)` should be considered *dead*, along with `$(MonoAndroidBinDirectory)`, as these should now *always* be the same directory as where `Xamarin.Android.Build.Tasks.dll` is located, or a `$(HostOS)` sub-directory. * `Xamarin.ProjectTools.csproj` needed to be updated to ensure that the build order was correct. * Remove all `[Obsolete]` and unreferenced members from `Xamarin.Android.Build.Utilities.dll`. There's too much in there, and it makes my head hurt trying to understand the interrelationships between it all. If it's not used, it's gone. * The changes to `src/monodroid/jni/Android.mk` are...weird. The removal of `-I$(topdir)/libmonodroid/zip`/etc. is to reduce use of non-existent paths, as `$(topdir)` isn't defined, so that's *actually* `-I/libmonodroid/zip`, which is nonsensical. So far, so good. What's *odd* is the addition of `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`: This is needed so that `external/mono/support/zlib-helper.c` exports `CreateZStream` and related symbols, otherwise we get a unit test failure in `GzipStreamTest.Compression` due to an `EntryPointNotFoundException`, because `CreateZStream` isn't exported/public. What's odd here is that I don't understand what caused this behavior to change. Previous builds exported `CreateZStream`, otherwise the tests would fail, and I don't understand how any of the other changes in this PR would be at fault, though that's certainly the most plausible explanation. Regardless, `-Ijni` *first* (due to adding `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`) is the desired behavior, so that `jni/config.h` is included, thus ensuring that `MONO_API` has the required definition when building `zlib-helper.c`.
2017-07-28 16:36:32 +03:00
Files="$(_MSBuildDir)\%(_MonoCrossRuntime.InstallPath)%(_MonoCrossRuntime.CrossMonoName)%(_MonoCrossRuntime.ExeSuffix)"
[mono-runtimes] Build AOT+LLVM cross-compilers (#125) The commit implements building of LLVM and cross-compilers to support Xamarin.Android/Mono AOT. LLVM and cross-compilers can be built for both the host platform (Linux and OS/X at the moment) as well as cross-compiled for 32-bit and 64-bit Windows platforms. Windows builds are done with MXE toolchain on OS/X and with the packaged mingw-w64 toolchain on Linux (tested on Ubuntu 16.04 ONLY). Also introducing a new set of MSBuild properties that contain information about the host system. Some of those properties (HostOS, HostCC, HostCXX for instance) have been moved from Configuration.props to better support auto-detection. A new script, build-tools/scripts/generate-os-info, is invoked as part of `make prepare` to generate file that contains the new properties. The generated file is required for the build to work and is also host-specific (it mustn't be moved between different machines) Cross compiler builds require access to a configured Mono build tree, in order to generate C structure offsets header file that is used by the AOT compilers to properly generate AOT-ed binaries. Therefore, even if a JIT target is not enabled in the configuration, enabling a cross-compiler for some target will configure Mono for that JIT target but it will NOT build it, to save time. To facilitate this, the _MonoRuntimes items defined in build-tools/mono-runtimes/mono-runtimes.projitems gain an additional metadata item called `DoBuild` which will be set to `true` if the runtime actually needs to be built, as opposed to just configured. MXE builds are disabled on Linux as mingw-w64 works just fine. A `make prepare` warning is issued for Linux hosts which have the binfmt_misc module enabled and either Wine of Mono (cli) registered as PE32/PE32+ binary interpreters. In such instance building of the Windows cross-compilers will fail because Autotools determine whether software is being cross compiled by building a test program and attempting to execute it. In normal circumstances such an attempt will fail, but with Windows cross-compilation and either Wine or Mono registered to handle the PE32 executables this attempt will succeed thus causing the cross compilation detection to fail. Currently to build cross compilers on Linux you need to generate the C structure offsets header file on OS/X and copy the resulting headers to appropriate places on Linux. The header files should be placed in build-tools/mono-runtimes/obj/Debug/cross-*/ directories. The header files are: {cross-arm,cross-arm-win}/aarch64-v8a-linux-android.h {cross-arm64,cross-arm64-win}/armv5-none-linux-androideabi.h {cross-x86,cross-x86-win}/i686-none-linux-android.h {cross-x86_64,cross-x86_64-win}/x86_64-none-linux-android.h Offsets header generation doesn't work on Linux atm because of missing support for it in the Mono utility used to generate the offsets. Hopefully this limitation will be removed in the near future and a start-to-end build of everything will be possible on Linux. It is now mandatory to run at least `make prepare-props` before Xamarin.Android can be built. The target generates the OS-specific props file which is required by the build. `make prepare` depends on the target.
2016-07-26 16:27:31 +03:00
/>
</Target>
[xa-prep-tasks] Use the Mono bundle (#162) Context: https://github.com/xamarin/xamarin-android/commit/fbfd676c102c63e4e06e750857b178725e33450c Stage 3 of the cunning plan is to (attempt to) use the mono bundle introduced in commit fbfd676c. This "simple" desire (ha!) re-raises the architectural project dependency issue "solved" in fbfd676c, but first, a simple question: What should download the mono bundle? There are two plausible answers: 1. `make prepare` can (somehow) handle it. 2. MSBuild can (somehow) handle it. Both are plausible. The problem with using `make` targets (1) is there is increased potential for "duplication" -- duplication of the bundle filename, downloading it, and extracting it. Plus, `make` isn't "Windows friendly", in that GNU make isn't (normally) present with Visual Studio. (`NMAKE` is, but the Makefiles in this project are not compatible with `NMAKE`.) Which brings us to MSBuild (2): can it handle the task? To tackle that, we need to be able to have an MSBuild task project which has *no dependencies*, so that it can download and extract the mono bundle *before anything else runs*, as it may be downloading contents which mean that other projects don't *need* to run. The need for a "pre-bootstrap" task assembly -- called `xa-prep-tasks` -- thus "undoes" *some* of the logic regarding `libzip-windows.mdproj` and the `<Zip/>` task from fbfd676c: it isn't *possible* to rely on `libzip` from a "pre-build" state, as `libzip` is one of the things in the mono bundle, so now we need *two* "bootstrap" task assemblies: one without a `libzip` dependency -- `xa-prep-tasks.dll` -- and one *with* a `libzip` dependency -- `Xamarin.Android.Tools.BootstrapTasks.dll` Move tasks which don't currently require `libzip` -- or won't in the future, or laziness -- from `Xamarin.Android.Tools.BootstrapTasks.dll` and move them into `xa-prep-tasks.dll`. With that architectural compromise in place, add `xa-prep-tasks` as a `@(ProjectReference)` to various projects to help ensure it's built *first*, and rearchitect `bundle.mdproj` so that `xa-prep-tasks.targets` and `bundle.targets` can use the same targets to compute the bundle filename, now in `build-tools/bundle/bundle-path.targets`. Add a post-build step to `xa-prep-tasks.csproj` which downloads and extracts the expected mono bundle. One "problem" (feature?) is that the new `<SystemUnzip/>` task doesn't report errors as errors when unzip'ing the file. This turns out to be fine here because when downloading the mono bundle from Azure we don't get a 404 *anyway* -- Azure instead returns an XML document containing an error message (wat?!). We can thus ignore most error handling entirely...though we're *also* ignoring any checking for invalid downloads, which is something we should address in the future. Update the varioous project files so that they won't attempt to rebuild binaries that were present in the mono bundle.
2016-08-16 23:02:48 +03:00
<Target Name="GetMonoBundleItems"
[mono-runtimes] Fix bundle*.zip use (#510) Commit 76be1fb1 (6e9a6612) broke `bundle*.zip` use. Consequently, `bundle*.zip` *is not used*, resulting in *yuuuge* build times (nearly 7 hours for full builds on macOS, over 2 hours for PRs!). The `_BuildUnlessCached` target calls the `GetMonoBundleItems` target, to determine whether the `ForceBuild` target -- which invokes `_BuildRuntimes` and many other targets -- should be executed. The `GetMonoBundleItems` returns all files which would be included in `bundle*.zip`. The cause of the break is that commit 6e9a6612 changed the `GetMonoBundleItems` target to depend on the `_GetMonodocItems` target, which in turn depended on the `_BuildRuntimes` target. `_BuildRuntimes` is the target we want to avoid executing; it's the target that builds mono, and thus can take an eternity (particularly on the `xamarin-android` Jenkins lane, which builds **ALL THE ABIS**). Adding `_GetMonodocItems` in this fashion thus means we never use the cache, bceause in order to determine whether or not we can use the cache we need to do all the work that the cache was intended to avoid. Consider the [xamarin-android Build 312 log file][0] Building target "_BuildUnlessCached" in project ".../xamarin-android/build-tools/mono-runtimes/mono-runtimes.mdproj" (".../xamarin-android/build-tools/mono-runtimes/mono-runtimes.targets"); "Build" depends on it. ... Building target "GetMonoBundleItems" in project ".../xamarin-android/build-tools/mono-runtimes/mono-runtimes.mdproj" (".../xamarin-android/build-tools/mono-runtimes/mono-runtimes.targets"); "_BuildUnlessCached" depends on it. ... Building target "_GetMonodocItems" in project ".../xamarin-android/build-tools/mono-runtimes/mono-runtimes.mdproj" (".../xamarin-android/build-tools/mono-runtimes/mono-runtimes.targets"); "GetMonoBundleItems" depends on it. Building target "_BuildRuntimes" in project ".../xamarin-android/build-tools/mono-runtimes/mono-runtimes.mdproj" (".../xamarin-android/build-tools/mono-runtimes/mono-runtimes.targets"); "_GetMonodocItems" depends on it. ...and **BOOM**, the existence of `bundle*.zip` is meaningless. The fix is to partially revert commit 6e9a6612, but the entire reason 6e9a6612 exists is because commit 76be1fb1 changes the default file extension for debug symbols from `.mdb` to `.pdb`: How should we know if we're emitting `.mdb` or `.pdb` symbols? *A* way to know that is to actually build everything and see what is generated, but that's...slow. Slow and undesirable. (That's what 6e9a6612 does!) An alternate ("hacky") way to know is to rely on mono-specific extensions, and spackle, and duct tape: Mono's `msbuild` sets the `$(_DebugFileExt)` MSBuild property to the file extension of debug symbols, with values of `.mdb` on Mono 4.6 through Mono 4.8, and a value of `.pdb` on Mono 4.9. In `build-tools/scripts/msbuild.mk`, we can probe the mono version and attempt to come up with "sane" defaults, setting `$(_DebugFileExt)` for `xbuild` invocations done via `make`. We can thus rely on the `$(_DebugFileExt)` extension instead of costly filesystem probing, allowing `bundle*.zip` to actually be used, instead of ignored. ~~~ **A tale of two `pkg-config`s**: A funny thing happened while "fixing" `msbuild.mk` to set `$(_DebugFileExt)`: Turns Out™ there are (at least?) *two* `pkg-config` files on the Jenkins+macOS machine: 1. `/usr/local/bin/pkg-config` 2. `/Library/Frameworks/Mono.framework/Commands/pkg-config` (1) knows *nothing* about `Mono.framework`, and we're using the `mono` from `Mono.framework`, which distributes (2). Furthermore, we want to use `pkg-config --atleast-version=4.9 mono` in order to know if we're emitting `.pdb` or `.mdb` debug files, so using the correct version is Very Important. So, off I go to fix that so that we always use (2) on macOS, [and macOS+msbuild breaks][1]. Why's this matter? Recall 0eee6734, in which `$(_XAFixMSBuildFrameworkPathOverride)` is set only when (a) `msbuild` is used, and (b) mono prior to 4.8 is used? The check for (b) used the `pkg-config` from (1), meaning *the version check always failed*, meaning when building with `msbuild` from e.g. Mono 4.9, `$(_XAFixMSBuildFrameworkPathOverride)` was still being set, even though, as per 0eee6734, it shouldn't have been. Which means the explanation in 0eee6734 is wrong. Drain the swamp, at least a little bit: Remove `$(_XAFixMSBuildFrameworkPathOverride)`, and *always* set `$(FrameworkPathOverride)` within `MonoAndroidFramework.props`. It appears that attempting to make `$(FrameworkPathOverride)` conditional wasn't necessary, and only complicates matters. [0]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android/312/consoleText [1]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android-msbuild-pr-builder/243/
2017-03-22 17:50:14 +03:00
DependsOnTargets="_GetRuntimesOutputItems;_PrepareLlvmItems">
[xa-prep-tasks] Use the Mono bundle (#162) Context: https://github.com/xamarin/xamarin-android/commit/fbfd676c102c63e4e06e750857b178725e33450c Stage 3 of the cunning plan is to (attempt to) use the mono bundle introduced in commit fbfd676c. This "simple" desire (ha!) re-raises the architectural project dependency issue "solved" in fbfd676c, but first, a simple question: What should download the mono bundle? There are two plausible answers: 1. `make prepare` can (somehow) handle it. 2. MSBuild can (somehow) handle it. Both are plausible. The problem with using `make` targets (1) is there is increased potential for "duplication" -- duplication of the bundle filename, downloading it, and extracting it. Plus, `make` isn't "Windows friendly", in that GNU make isn't (normally) present with Visual Studio. (`NMAKE` is, but the Makefiles in this project are not compatible with `NMAKE`.) Which brings us to MSBuild (2): can it handle the task? To tackle that, we need to be able to have an MSBuild task project which has *no dependencies*, so that it can download and extract the mono bundle *before anything else runs*, as it may be downloading contents which mean that other projects don't *need* to run. The need for a "pre-bootstrap" task assembly -- called `xa-prep-tasks` -- thus "undoes" *some* of the logic regarding `libzip-windows.mdproj` and the `<Zip/>` task from fbfd676c: it isn't *possible* to rely on `libzip` from a "pre-build" state, as `libzip` is one of the things in the mono bundle, so now we need *two* "bootstrap" task assemblies: one without a `libzip` dependency -- `xa-prep-tasks.dll` -- and one *with* a `libzip` dependency -- `Xamarin.Android.Tools.BootstrapTasks.dll` Move tasks which don't currently require `libzip` -- or won't in the future, or laziness -- from `Xamarin.Android.Tools.BootstrapTasks.dll` and move them into `xa-prep-tasks.dll`. With that architectural compromise in place, add `xa-prep-tasks` as a `@(ProjectReference)` to various projects to help ensure it's built *first*, and rearchitect `bundle.mdproj` so that `xa-prep-tasks.targets` and `bundle.targets` can use the same targets to compute the bundle filename, now in `build-tools/bundle/bundle-path.targets`. Add a post-build step to `xa-prep-tasks.csproj` which downloads and extracts the expected mono bundle. One "problem" (feature?) is that the new `<SystemUnzip/>` task doesn't report errors as errors when unzip'ing the file. This turns out to be fine here because when downloading the mono bundle from Azure we don't get a 404 *anyway* -- Azure instead returns an XML document containing an error message (wat?!). We can thus ignore most error handling entirely...though we're *also* ignoring any checking for invalid downloads, which is something we should address in the future. Update the varioous project files so that they won't attempt to rebuild binaries that were present in the mono bundle.
2016-08-16 23:02:48 +03:00
<ItemGroup>
<BundleItem Include="@(_BclInstalledItem)" />
[mdoc] Add mdoc and dependencies to the bundle Context: https://github.com/xamarin/xamarin-android/commit/5a8e48eba8cc8d05ec4fef25ac074d00606f5a69 `mdoc.exe` is used by the `<MDoc/>` task, used by the `BuildDocumentation` target in `Xamarin.Android.Bindings.targets`. We obtain `mdoc.exe` from the `external/mono` build, but because `mdoc.exe` will be referencing mono's `Mono.Cecil.dll`, we use `remap-assembly-ref` to update `mdoc.exe` to instead use `Xamarin.Android.Cecil.dll`.
2016-12-20 01:18:21 +03:00
<BundleItem Include="@(_MonoDocInstalledItems)" />
[bundle] Include mono's debug symbols (#513) Microsoft has (several?) [symbol servers][0], which is a repository "somewhere" which contains debug symbols, so that when a stack trace containing e.g. BCL stack frames is sent to Microsoft, there is a way to correlate that back to filename and line number source for further investigation. Xamarin.Android has not partaken in this mechanism, in large part because we did most of our building on Linux and macOS machines which generate `.mdb` debug symbols, while symbol servers require `.pdb`. That constraint is now behind us; with 76be1fb1 and the use of Mono 4.9 for builds, our builds now generate Portable `.pdb` files, which *should* (hopefully) be usable on Microsoft's symbol servers. However, to transmit them to Microsoft's servers, we need to actually *retain* them. At present, they're not: because of `bundle*.zip` use, the `mscorlib.dll` that we include in the final product may have been built much earlier than the rest of the product, and we don't have any other mechanism to retain the `mscorlib.pdb` file. Let's change that: update `mono-runtimes.targets` so that the created debug symbol files are included in the produced `bundle*.zip`, and bump the corresponding `$(XABundleFileName)` version number. This will ensure that we retain the debug symbols for our BCL assemblies, allowing us to actually have something to upload. [0]: https://msdn.microsoft.com/en-us/library/cc667410.aspx
2017-03-22 22:51:57 +03:00
<BundleItem Include="@(_MonoCilStripDest)" />
<BundleItem Include="@(_MonoUtilityDest)" />
[structure] Rework installation directory structure (#704) Context: https://github.com/xamarin/xamarin-android/pull/253#discussion_r82862993 The *intention* is that Jenkins-produced `oss-xamarin.android*.zip` artifacts be usable on Windows, so that side-by-side testing can be performed without replacing the system installation. Usage is in [UsingJenkinsBuildArtifacts.md](Documentation/UsingJenkinsBuildArtifacts). This isn't *entirely* the case. It was *apparently* the case at the time of commit 87ca2737, but things have bitrotten since. For example, following the 87ca2737 instructions would currently result in an XA0020 `Could not find monodroid` error, because `class-parse.exe` wasn't where Windows expects it to be (it was in `lib/mandroid`, not `lib/xbuild/Xamarin/Android`). This needs to be fixed. Additionally, PR #253 mentions that, for filesystem organization, it would be useful if the macOS/Linux directory structure -- `$prefix/bin`, `$prefix/lib/mandroid`, `$prefix/lib/xbuild/Xamarin/Android`, `$prefix/lib/xbuild-frameworks` -- more closely resembled the Windows directory structure of `$MSBuild` and `$ReferenceAssemblies` (as seen in `.vsix` files). This would turn macOS/Linux into using `$xa_prefix/xbuild` and `$xa_prefix/xbuild-frameworks` directories. `$prefix/bin` would only contain `xabuild`. What is currently in `$prefix/lib/mandroid` would be merged with `$xa_prefix/xbuild/Xamarin/Android`. `$xa_prefix` would `$prefix/lib/xamarin.android`. This would turn the current macOS structure: $prefix/bin/xabuild $prefix/bin/generator $prefix/bin/cross-arm $prefix/lib/mandroid/generator.exe $prefix/lib/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xbuild/Xamarin/Android/Xamarin.Android.Common.targets Into: $prefix/bin/xabuild $prefix/lib/xamarin.android/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xamarin.android/xbuild/Xamarin/Android/generator.exe $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Xamarin.Android.Common.targets $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Darwin/cross-arm Other notes: * The `bundle-*.zip` filename has been has been changed to include a *hash* of the contents of various files, in particular `build-tools\mono-runtimes.*`. This was instigated via a conversation with @kumpera about making the filename more "idiot-proof": `mono-runtimes.props` contains *compiler flags*, and if any of those change, then *logically* the bundle should differ as well, as the mono runtimes may differ in significant ways. In theory, the `-vXX` version should be used to track this, but this is a manual change, easy to overlook. The new `-hHASH` part of the filename should be more automagic. The new `<HashFileContents/>` task in `xa-prep-tasks.dll` is responsible for creating the hash value. * `Configuration.Java.Interop.Override.props` was moved into `build-tools/scripts`, because that would cleanup the root directory a bit. * OS-specific binaries are now placed into `$prefix/lib/xamarin.android/xbuild/Xamarin/Android/$(HostOS)`. On a theoretical plus side, this means that the same build directory can contain OS-specific binaries for multiple operating systems. (I don't know if anyone shares a build tree between e.g. macOS and Linux, but if anyone *does*...) Unfortunately this requires a workaround for an `xbuild` bug: `%(_LlvmRuntime.InstallPath)` and `%(_MonoCrossRuntime.InstallPath)` *cannot* use MSBuild properties, e.g. `<InstallPath>$(HostOS)/</InstallPath>` doesn't work as desired; it's instead treated literally. Special-case `%(InstallPath)` until we fully migrate to MSBuild. * `$(MonoAndroidToolsDirectory)` should be considered *dead*, along with `$(MonoAndroidBinDirectory)`, as these should now *always* be the same directory as where `Xamarin.Android.Build.Tasks.dll` is located, or a `$(HostOS)` sub-directory. * `Xamarin.ProjectTools.csproj` needed to be updated to ensure that the build order was correct. * Remove all `[Obsolete]` and unreferenced members from `Xamarin.Android.Build.Utilities.dll`. There's too much in there, and it makes my head hurt trying to understand the interrelationships between it all. If it's not used, it's gone. * The changes to `src/monodroid/jni/Android.mk` are...weird. The removal of `-I$(topdir)/libmonodroid/zip`/etc. is to reduce use of non-existent paths, as `$(topdir)` isn't defined, so that's *actually* `-I/libmonodroid/zip`, which is nonsensical. So far, so good. What's *odd* is the addition of `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`: This is needed so that `external/mono/support/zlib-helper.c` exports `CreateZStream` and related symbols, otherwise we get a unit test failure in `GzipStreamTest.Compression` due to an `EntryPointNotFoundException`, because `CreateZStream` isn't exported/public. What's odd here is that I don't understand what caused this behavior to change. Previous builds exported `CreateZStream`, otherwise the tests would fail, and I don't understand how any of the other changes in this PR would be at fault, though that's certainly the most plausible explanation. Regardless, `-Ijni` *first* (due to adding `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`) is the desired behavior, so that `jni/config.h` is included, thus ensuring that `MONO_API` has the required definition when building `zlib-helper.c`.
2017-07-28 16:36:32 +03:00
<BundleItem Include="@(MonoFacadeAssembly->'$(_BclFrameworkDir)Facades\%(Identity)')" />
<BundleItem Include="$(_BclFrameworkDir)RedistList\FrameworkList.xml" />
[xa-prep-tasks] Use the Mono bundle (#162) Context: https://github.com/xamarin/xamarin-android/commit/fbfd676c102c63e4e06e750857b178725e33450c Stage 3 of the cunning plan is to (attempt to) use the mono bundle introduced in commit fbfd676c. This "simple" desire (ha!) re-raises the architectural project dependency issue "solved" in fbfd676c, but first, a simple question: What should download the mono bundle? There are two plausible answers: 1. `make prepare` can (somehow) handle it. 2. MSBuild can (somehow) handle it. Both are plausible. The problem with using `make` targets (1) is there is increased potential for "duplication" -- duplication of the bundle filename, downloading it, and extracting it. Plus, `make` isn't "Windows friendly", in that GNU make isn't (normally) present with Visual Studio. (`NMAKE` is, but the Makefiles in this project are not compatible with `NMAKE`.) Which brings us to MSBuild (2): can it handle the task? To tackle that, we need to be able to have an MSBuild task project which has *no dependencies*, so that it can download and extract the mono bundle *before anything else runs*, as it may be downloading contents which mean that other projects don't *need* to run. The need for a "pre-bootstrap" task assembly -- called `xa-prep-tasks` -- thus "undoes" *some* of the logic regarding `libzip-windows.mdproj` and the `<Zip/>` task from fbfd676c: it isn't *possible* to rely on `libzip` from a "pre-build" state, as `libzip` is one of the things in the mono bundle, so now we need *two* "bootstrap" task assemblies: one without a `libzip` dependency -- `xa-prep-tasks.dll` -- and one *with* a `libzip` dependency -- `Xamarin.Android.Tools.BootstrapTasks.dll` Move tasks which don't currently require `libzip` -- or won't in the future, or laziness -- from `Xamarin.Android.Tools.BootstrapTasks.dll` and move them into `xa-prep-tasks.dll`. With that architectural compromise in place, add `xa-prep-tasks` as a `@(ProjectReference)` to various projects to help ensure it's built *first*, and rearchitect `bundle.mdproj` so that `xa-prep-tasks.targets` and `bundle.targets` can use the same targets to compute the bundle filename, now in `build-tools/bundle/bundle-path.targets`. Add a post-build step to `xa-prep-tasks.csproj` which downloads and extracts the expected mono bundle. One "problem" (feature?) is that the new `<SystemUnzip/>` task doesn't report errors as errors when unzip'ing the file. This turns out to be fine here because when downloading the mono bundle from Azure we don't get a 404 *anyway* -- Azure instead returns an XML document containing an error message (wat?!). We can thus ignore most error handling entirely...though we're *also* ignoring any checking for invalid downloads, which is something we should address in the future. Update the varioous project files so that they won't attempt to rebuild binaries that were present in the mono bundle.
2016-08-16 23:02:48 +03:00
<BundleItem Include="@(_InstallRuntimeOutput)" />
<BundleItem Include="@(_InstallUnstrippedRuntimeOutput)" />
<BundleItem Include="@(_InstallProfilerOutput)" />
<BundleItem Include="@(_InstallUnstrippedProfilerOutput)" />
[mono-runtimes] Fix @(BundleItem) timestamps (#307) The *intent* of the `_BuildUnlessCached` target within `mono-runtimes.targets` is to (attempt to!) make building mono "free", or at least, free *enough* so that other project files can have a build-time dependency on `mono-runtimes.mdproj` so that build artifacts can be (reasonably) consistent. Unfortunately, that wasn't the case, with the net result being that: tools/scripts/xabuild /t:SignAndroidPackage src/Mono.Android/Test/*.csproj would *constantly* attempt to re-build `build-tools/mono-runtimes.mdproj`, adding (at least!) one minute as we ran mono's `make`...to do nothing. Why? Target _BuildUnlessCached needs to be built as input file '.../xamarin-android/external/mono/autogen.sh' is newer than output file '../../bin/Debug//include/armeabi-v7a/eglib/config.h' The "problem" is that the `<Copy/>` task copies over the timestamp as well, so even though we're (possibly) creating a new file, the timestamp on the created file will match the source file. Normally this is fine. In this case, we're dealing with `eglib`, which is a "nested" ("delegated"?) `configure` invocation from mono's `configure`. It *appears* that if (when) mono is updated, "normal" mono subdirectories will "properly" honor `--enable-maintainer-mode` and re-run mono's `configure`, which will in turn update the timstamp on any `configure`-time generated files. This doesn't appear to be the case for `mono/eglib`; when `mono` is updated, `mono/eglib` does *not* re-run `configure`, and thus no timestamps are updated to follow suit. Fix this by *explicitly* `<Touch/>`ing all copied files so that we ensure they have a "recent" timestamp ("now"), which should ensure that the file's timestamp is newer than that of e.g. `autogen.sh`. Additionally, there was a typo: Commit a5b324d1 used a `@(_InstallRuntimesOutput)` item group, but there is no such item group; it's actually `@(_InstallRuntimeOutput)` (no `s`). Fixity fix this. Finally, commit a5b324d1 forgot to add `libmono-btls-shared` to `@(BundleItem)`, preventing it from being added to the bundle. Fix this oversight.
2016-11-22 14:11:45 +03:00
<BundleItem Include="@(_InstallMonoBtlsOutput)" />
<BundleItem Include="@(_InstallUnstrippedMonoBtlsOutput)" />
[xa-prep-tasks] Use the Mono bundle (#162) Context: https://github.com/xamarin/xamarin-android/commit/fbfd676c102c63e4e06e750857b178725e33450c Stage 3 of the cunning plan is to (attempt to) use the mono bundle introduced in commit fbfd676c. This "simple" desire (ha!) re-raises the architectural project dependency issue "solved" in fbfd676c, but first, a simple question: What should download the mono bundle? There are two plausible answers: 1. `make prepare` can (somehow) handle it. 2. MSBuild can (somehow) handle it. Both are plausible. The problem with using `make` targets (1) is there is increased potential for "duplication" -- duplication of the bundle filename, downloading it, and extracting it. Plus, `make` isn't "Windows friendly", in that GNU make isn't (normally) present with Visual Studio. (`NMAKE` is, but the Makefiles in this project are not compatible with `NMAKE`.) Which brings us to MSBuild (2): can it handle the task? To tackle that, we need to be able to have an MSBuild task project which has *no dependencies*, so that it can download and extract the mono bundle *before anything else runs*, as it may be downloading contents which mean that other projects don't *need* to run. The need for a "pre-bootstrap" task assembly -- called `xa-prep-tasks` -- thus "undoes" *some* of the logic regarding `libzip-windows.mdproj` and the `<Zip/>` task from fbfd676c: it isn't *possible* to rely on `libzip` from a "pre-build" state, as `libzip` is one of the things in the mono bundle, so now we need *two* "bootstrap" task assemblies: one without a `libzip` dependency -- `xa-prep-tasks.dll` -- and one *with* a `libzip` dependency -- `Xamarin.Android.Tools.BootstrapTasks.dll` Move tasks which don't currently require `libzip` -- or won't in the future, or laziness -- from `Xamarin.Android.Tools.BootstrapTasks.dll` and move them into `xa-prep-tasks.dll`. With that architectural compromise in place, add `xa-prep-tasks` as a `@(ProjectReference)` to various projects to help ensure it's built *first*, and rearchitect `bundle.mdproj` so that `xa-prep-tasks.targets` and `bundle.targets` can use the same targets to compute the bundle filename, now in `build-tools/bundle/bundle-path.targets`. Add a post-build step to `xa-prep-tasks.csproj` which downloads and extracts the expected mono bundle. One "problem" (feature?) is that the new `<SystemUnzip/>` task doesn't report errors as errors when unzip'ing the file. This turns out to be fine here because when downloading the mono bundle from Azure we don't get a 404 *anyway* -- Azure instead returns an XML document containing an error message (wat?!). We can thus ignore most error handling entirely...though we're *also* ignoring any checking for invalid downloads, which is something we should address in the future. Update the varioous project files so that they won't attempt to rebuild binaries that were present in the mono bundle.
2016-08-16 23:02:48 +03:00
<BundleItem Include="@(_InstallMonoPosixHelperOutput)" />
<BundleItem Include="@(_InstallUnstrippedMonoPosixHelperOutput)" />
<BundleItem Include="@(_RuntimeEglibHeaderOutput)" />
[bundle] Bundle Consts.cs (#207) Commit 29568117 inadvertently stopped bundling `include/Consts.cs`, which is causing a [build break][0]: CSC: error CS2001: Source file `../../bin/Debug/lib/xbuild-frameworks/MonoAndroid/v1.0/../../../../include/Consts.cs' could not be found Fix the `GetMonoBundleItems` target to include `Consts.cs`, and bump the bundle version within `GetBundleFileName` so that a new bundle is created and uploaded. [0]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android/78/
2016-09-03 16:08:24 +03:00
<BundleItem Include="@(_MonoConstsOutput)" />
[mono-runtimes] Use directory separators with `$(OutputPath)`. [The build][0] [is broken][1]: xamarin-android/build-tools/mono-runtimes/mono-runtimes.targets: error : Could not find a part of the path ".../xamarin-android/xamarin-android/bin/Debuglib/xbuild-frameworks/MonoAndroid/v1.0/RedistList/FrameworkList.xml". The error is generated from the `_InstallBcl` target when writing `FrameworkList.xml`, and a close reading of the error message shows that a directory separator is missing: it should be `bin/Debug/lib`, but is instead `bin/Debuglib`. I *assume* that this is due to `<CallTarget/>` being "flakey" within xbuild (it passes the file containing the target to `<MSBuild/>`!), and *somehow* in that "flakiness" `$(OutputPath)` doesn't end with `\`, *which it should*, because `Microsoft.Common.targets` will automatically append `\` to `$(OutputPath)`. To work around this issue, update `mono-runtimes.targets` so that it *always* uses `\` after `$(OutputPath)` and `$(IntermediateOutputPath)`. Additionally, `llc.exe` was being installed into the wrong directory, preventing the bundle from being used: Target _BuildUnlessCached needs to be built as output file '../../bin/Debug/lib/mandroid/llc.exe' does not exist. `llc.exe` was instead installed into `bin/Debug/bin/llc.exe`. Bump the bundle version number to -v3 and correct `mono-runtimes.targets` so that `llc.exe` and `opt.exe` are properly installed into `$prefix/lib/mandroid`. [0]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android/49/ [1]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android/49/consoleText
2016-08-17 04:18:15 +03:00
<BundleItem Include="@(_LlvmTargetBinary)" />
[structure] Rework installation directory structure (#704) Context: https://github.com/xamarin/xamarin-android/pull/253#discussion_r82862993 The *intention* is that Jenkins-produced `oss-xamarin.android*.zip` artifacts be usable on Windows, so that side-by-side testing can be performed without replacing the system installation. Usage is in [UsingJenkinsBuildArtifacts.md](Documentation/UsingJenkinsBuildArtifacts). This isn't *entirely* the case. It was *apparently* the case at the time of commit 87ca2737, but things have bitrotten since. For example, following the 87ca2737 instructions would currently result in an XA0020 `Could not find monodroid` error, because `class-parse.exe` wasn't where Windows expects it to be (it was in `lib/mandroid`, not `lib/xbuild/Xamarin/Android`). This needs to be fixed. Additionally, PR #253 mentions that, for filesystem organization, it would be useful if the macOS/Linux directory structure -- `$prefix/bin`, `$prefix/lib/mandroid`, `$prefix/lib/xbuild/Xamarin/Android`, `$prefix/lib/xbuild-frameworks` -- more closely resembled the Windows directory structure of `$MSBuild` and `$ReferenceAssemblies` (as seen in `.vsix` files). This would turn macOS/Linux into using `$xa_prefix/xbuild` and `$xa_prefix/xbuild-frameworks` directories. `$prefix/bin` would only contain `xabuild`. What is currently in `$prefix/lib/mandroid` would be merged with `$xa_prefix/xbuild/Xamarin/Android`. `$xa_prefix` would `$prefix/lib/xamarin.android`. This would turn the current macOS structure: $prefix/bin/xabuild $prefix/bin/generator $prefix/bin/cross-arm $prefix/lib/mandroid/generator.exe $prefix/lib/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xbuild/Xamarin/Android/Xamarin.Android.Common.targets Into: $prefix/bin/xabuild $prefix/lib/xamarin.android/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xamarin.android/xbuild/Xamarin/Android/generator.exe $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Xamarin.Android.Common.targets $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Darwin/cross-arm Other notes: * The `bundle-*.zip` filename has been has been changed to include a *hash* of the contents of various files, in particular `build-tools\mono-runtimes.*`. This was instigated via a conversation with @kumpera about making the filename more "idiot-proof": `mono-runtimes.props` contains *compiler flags*, and if any of those change, then *logically* the bundle should differ as well, as the mono runtimes may differ in significant ways. In theory, the `-vXX` version should be used to track this, but this is a manual change, easy to overlook. The new `-hHASH` part of the filename should be more automagic. The new `<HashFileContents/>` task in `xa-prep-tasks.dll` is responsible for creating the hash value. * `Configuration.Java.Interop.Override.props` was moved into `build-tools/scripts`, because that would cleanup the root directory a bit. * OS-specific binaries are now placed into `$prefix/lib/xamarin.android/xbuild/Xamarin/Android/$(HostOS)`. On a theoretical plus side, this means that the same build directory can contain OS-specific binaries for multiple operating systems. (I don't know if anyone shares a build tree between e.g. macOS and Linux, but if anyone *does*...) Unfortunately this requires a workaround for an `xbuild` bug: `%(_LlvmRuntime.InstallPath)` and `%(_MonoCrossRuntime.InstallPath)` *cannot* use MSBuild properties, e.g. `<InstallPath>$(HostOS)/</InstallPath>` doesn't work as desired; it's instead treated literally. Special-case `%(InstallPath)` until we fully migrate to MSBuild. * `$(MonoAndroidToolsDirectory)` should be considered *dead*, along with `$(MonoAndroidBinDirectory)`, as these should now *always* be the same directory as where `Xamarin.Android.Build.Tasks.dll` is located, or a `$(HostOS)` sub-directory. * `Xamarin.ProjectTools.csproj` needed to be updated to ensure that the build order was correct. * Remove all `[Obsolete]` and unreferenced members from `Xamarin.Android.Build.Utilities.dll`. There's too much in there, and it makes my head hurt trying to understand the interrelationships between it all. If it's not used, it's gone. * The changes to `src/monodroid/jni/Android.mk` are...weird. The removal of `-I$(topdir)/libmonodroid/zip`/etc. is to reduce use of non-existent paths, as `$(topdir)` isn't defined, so that's *actually* `-I/libmonodroid/zip`, which is nonsensical. So far, so good. What's *odd* is the addition of `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`: This is needed so that `external/mono/support/zlib-helper.c` exports `CreateZStream` and related symbols, otherwise we get a unit test failure in `GzipStreamTest.Compression` due to an `EntryPointNotFoundException`, because `CreateZStream` isn't exported/public. What's odd here is that I don't understand what caused this behavior to change. Previous builds exported `CreateZStream`, otherwise the tests would fail, and I don't understand how any of the other changes in this PR would be at fault, though that's certainly the most plausible explanation. Regardless, `-Ijni` *first* (due to adding `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`) is the desired behavior, so that `jni/config.h` is included, thus ensuring that `MONO_API` has the required definition when building `zlib-helper.c`.
2017-07-28 16:36:32 +03:00
<BundleItem Include="$(_MSBuildDir)\%(_MonoCrossRuntime.InstallPath)%(_MonoCrossRuntime.CrossMonoName).d%(_MonoCrossRuntime.ExeSuffix)"
[bundle] Include `cross*.d*` in the bundle Commit a08cd80a updated `mono-runtimes.targets` to copy e.g. `cross-arm64.exe` to `cross-arm64.d.exe`, then to *strip* `cross-arm64.exe` in an effort to (eventually) reduce `.vsix` size. While admirable, there was one oversight: the original un-`strip`ped `cross-arm64.d.exe` *wasn't* added to the bundle, meaning the copy with debug symbols is *lost*. Oops. Correct this oversight and add `cross-*.d*` to the bundle.
2017-04-06 17:00:08 +03:00
Condition=" '@(_MonoCrossRuntime)' != '' "
/>
[structure] Rework installation directory structure (#704) Context: https://github.com/xamarin/xamarin-android/pull/253#discussion_r82862993 The *intention* is that Jenkins-produced `oss-xamarin.android*.zip` artifacts be usable on Windows, so that side-by-side testing can be performed without replacing the system installation. Usage is in [UsingJenkinsBuildArtifacts.md](Documentation/UsingJenkinsBuildArtifacts). This isn't *entirely* the case. It was *apparently* the case at the time of commit 87ca2737, but things have bitrotten since. For example, following the 87ca2737 instructions would currently result in an XA0020 `Could not find monodroid` error, because `class-parse.exe` wasn't where Windows expects it to be (it was in `lib/mandroid`, not `lib/xbuild/Xamarin/Android`). This needs to be fixed. Additionally, PR #253 mentions that, for filesystem organization, it would be useful if the macOS/Linux directory structure -- `$prefix/bin`, `$prefix/lib/mandroid`, `$prefix/lib/xbuild/Xamarin/Android`, `$prefix/lib/xbuild-frameworks` -- more closely resembled the Windows directory structure of `$MSBuild` and `$ReferenceAssemblies` (as seen in `.vsix` files). This would turn macOS/Linux into using `$xa_prefix/xbuild` and `$xa_prefix/xbuild-frameworks` directories. `$prefix/bin` would only contain `xabuild`. What is currently in `$prefix/lib/mandroid` would be merged with `$xa_prefix/xbuild/Xamarin/Android`. `$xa_prefix` would `$prefix/lib/xamarin.android`. This would turn the current macOS structure: $prefix/bin/xabuild $prefix/bin/generator $prefix/bin/cross-arm $prefix/lib/mandroid/generator.exe $prefix/lib/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xbuild/Xamarin/Android/Xamarin.Android.Common.targets Into: $prefix/bin/xabuild $prefix/lib/xamarin.android/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xamarin.android/xbuild/Xamarin/Android/generator.exe $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Xamarin.Android.Common.targets $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Darwin/cross-arm Other notes: * The `bundle-*.zip` filename has been has been changed to include a *hash* of the contents of various files, in particular `build-tools\mono-runtimes.*`. This was instigated via a conversation with @kumpera about making the filename more "idiot-proof": `mono-runtimes.props` contains *compiler flags*, and if any of those change, then *logically* the bundle should differ as well, as the mono runtimes may differ in significant ways. In theory, the `-vXX` version should be used to track this, but this is a manual change, easy to overlook. The new `-hHASH` part of the filename should be more automagic. The new `<HashFileContents/>` task in `xa-prep-tasks.dll` is responsible for creating the hash value. * `Configuration.Java.Interop.Override.props` was moved into `build-tools/scripts`, because that would cleanup the root directory a bit. * OS-specific binaries are now placed into `$prefix/lib/xamarin.android/xbuild/Xamarin/Android/$(HostOS)`. On a theoretical plus side, this means that the same build directory can contain OS-specific binaries for multiple operating systems. (I don't know if anyone shares a build tree between e.g. macOS and Linux, but if anyone *does*...) Unfortunately this requires a workaround for an `xbuild` bug: `%(_LlvmRuntime.InstallPath)` and `%(_MonoCrossRuntime.InstallPath)` *cannot* use MSBuild properties, e.g. `<InstallPath>$(HostOS)/</InstallPath>` doesn't work as desired; it's instead treated literally. Special-case `%(InstallPath)` until we fully migrate to MSBuild. * `$(MonoAndroidToolsDirectory)` should be considered *dead*, along with `$(MonoAndroidBinDirectory)`, as these should now *always* be the same directory as where `Xamarin.Android.Build.Tasks.dll` is located, or a `$(HostOS)` sub-directory. * `Xamarin.ProjectTools.csproj` needed to be updated to ensure that the build order was correct. * Remove all `[Obsolete]` and unreferenced members from `Xamarin.Android.Build.Utilities.dll`. There's too much in there, and it makes my head hurt trying to understand the interrelationships between it all. If it's not used, it's gone. * The changes to `src/monodroid/jni/Android.mk` are...weird. The removal of `-I$(topdir)/libmonodroid/zip`/etc. is to reduce use of non-existent paths, as `$(topdir)` isn't defined, so that's *actually* `-I/libmonodroid/zip`, which is nonsensical. So far, so good. What's *odd* is the addition of `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`: This is needed so that `external/mono/support/zlib-helper.c` exports `CreateZStream` and related symbols, otherwise we get a unit test failure in `GzipStreamTest.Compression` due to an `EntryPointNotFoundException`, because `CreateZStream` isn't exported/public. What's odd here is that I don't understand what caused this behavior to change. Previous builds exported `CreateZStream`, otherwise the tests would fail, and I don't understand how any of the other changes in this PR would be at fault, though that's certainly the most plausible explanation. Regardless, `-Ijni` *first* (due to adding `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`) is the desired behavior, so that `jni/config.h` is included, thus ensuring that `MONO_API` has the required definition when building `zlib-helper.c`.
2017-07-28 16:36:32 +03:00
<BundleItem Include="$(_MSBuildDir)\%(_MonoCrossRuntime.InstallPath)%(_MonoCrossRuntime.CrossMonoName)%(_MonoCrossRuntime.ExeSuffix)"
[xa-prep-tasks] Use the Mono bundle (#162) Context: https://github.com/xamarin/xamarin-android/commit/fbfd676c102c63e4e06e750857b178725e33450c Stage 3 of the cunning plan is to (attempt to) use the mono bundle introduced in commit fbfd676c. This "simple" desire (ha!) re-raises the architectural project dependency issue "solved" in fbfd676c, but first, a simple question: What should download the mono bundle? There are two plausible answers: 1. `make prepare` can (somehow) handle it. 2. MSBuild can (somehow) handle it. Both are plausible. The problem with using `make` targets (1) is there is increased potential for "duplication" -- duplication of the bundle filename, downloading it, and extracting it. Plus, `make` isn't "Windows friendly", in that GNU make isn't (normally) present with Visual Studio. (`NMAKE` is, but the Makefiles in this project are not compatible with `NMAKE`.) Which brings us to MSBuild (2): can it handle the task? To tackle that, we need to be able to have an MSBuild task project which has *no dependencies*, so that it can download and extract the mono bundle *before anything else runs*, as it may be downloading contents which mean that other projects don't *need* to run. The need for a "pre-bootstrap" task assembly -- called `xa-prep-tasks` -- thus "undoes" *some* of the logic regarding `libzip-windows.mdproj` and the `<Zip/>` task from fbfd676c: it isn't *possible* to rely on `libzip` from a "pre-build" state, as `libzip` is one of the things in the mono bundle, so now we need *two* "bootstrap" task assemblies: one without a `libzip` dependency -- `xa-prep-tasks.dll` -- and one *with* a `libzip` dependency -- `Xamarin.Android.Tools.BootstrapTasks.dll` Move tasks which don't currently require `libzip` -- or won't in the future, or laziness -- from `Xamarin.Android.Tools.BootstrapTasks.dll` and move them into `xa-prep-tasks.dll`. With that architectural compromise in place, add `xa-prep-tasks` as a `@(ProjectReference)` to various projects to help ensure it's built *first*, and rearchitect `bundle.mdproj` so that `xa-prep-tasks.targets` and `bundle.targets` can use the same targets to compute the bundle filename, now in `build-tools/bundle/bundle-path.targets`. Add a post-build step to `xa-prep-tasks.csproj` which downloads and extracts the expected mono bundle. One "problem" (feature?) is that the new `<SystemUnzip/>` task doesn't report errors as errors when unzip'ing the file. This turns out to be fine here because when downloading the mono bundle from Azure we don't get a 404 *anyway* -- Azure instead returns an XML document containing an error message (wat?!). We can thus ignore most error handling entirely...though we're *also* ignoring any checking for invalid downloads, which is something we should address in the future. Update the varioous project files so that they won't attempt to rebuild binaries that were present in the mono bundle.
2016-08-16 23:02:48 +03:00
Condition=" '@(_MonoCrossRuntime)' != '' "
/>
[Xamarin.Android.Bcl-Tests] Add BCL test project. (#872) What do we want? (with apologies to 48e3fc26) **MOAR** Unit tests! Specifically, we want to run the BCL unit tests which mono generates: $ cd external/mono/mcs/class/corlib $ make PROFILE=monodroid test # creates `monodroid_corlib_test.dll` Creation of `monodroid_*_test.dll` assemblies and the `make PROFILE=monodroid test` target is a relatively recent development, for which I need to buy the #runtime team some beers. In terms of `mono-runtimes.targets`, we can build *all* of the BCL unit test assemblies with: $ cd external/mono/mcs/class $ make -i do-test PROFILE=monodroid Now that we can create them, how do we *use* them? That's the trickier bit: they need to be built within mono, as part of the existing BCL build process. This in turn means that the BCL unit test assemblies need to be distributed as part of the mono bundle, as we don't want to rebuild the mono repo "from scratch" just for the unit tests. Update `build-tools/mono-runtimes/ProfileAssemblies.projitems` to include a new `@(MonoTestAssembly)` item group which contains all of the BCL unit test assemblies and related files which should be included into `bundle-*.zip`. Additionally, add `ProfileAssemblies.projitems` to `@(VersionFile)` witihin `bundle-path.targets`, so that if anything within `ProfileAssemblies.projitems` changes, we rebuild the bundle. Once we *have* the BCL unit test assemblies, and their dependencies, we need to *run* them. The new `Xamarin.Android.Bcl-Tests.csproj` project is a Xamarin.Android application project which will execute the unit tests. There's just one small problem: Xamarin.Android apps want to use `Xamarin.Android.NUnitLite.dll`. The BCL unit test assemblies instead build against their own `nunitlite.dll`, which has no Xamarin.Android integration or support. How do we use the new test assemblies? *Force* a fix by using `remap-assembly-ref` to "rename" the `nunitlite` assembly reference to `Xamarin.Android.NUnitLite.dll`. This *cannot* be done as part of the `mono-runtimes.mdproj` build, as `Xamarin.Android.NUnitLite.dll` won't yet exist. Instead, remap the assemblies within `Xamarin.Android.Bcl-Tests.targets`, and distribute the remapped assemblies with the application. Finally, address one other "small" problem: not all of the unit tests pass! Some of these are for reasons we don't know, and others will require changes to `mono`. Update `Xamarin.Android.NUnitLite` to allow *filtering* of tests: namespace Xamarin.Android.NUnitLite { partial class TestSuiteActivity { public ITestFilter Filter {get; set;} public virtual void UpdateFilter (); } partial class TestSuiteInstrumentation { public ITestFilter Filter {get; set;} public virtual void UpdateFilter (); } } `TestSuiteActivity.UpdateFilter()` is called by `TestSuiteActivity.OnCreate()`, *after* `GetIncludedCategories()` and `GetExcludedCategories()` are called, to allow subclasses to alter the `ITestFilter` which is used to determine which tests are executed. `TestSuiteInstrumentation.UpdateFilter()` is called by `TestSuiteInstrumentation.OnStart()`, *after* `GetIncludedCategories()` and `GetExcludedCategories()` are called, to allow subclasses to alter the `ITestFilter` which is used to determine which tests are executed. `Xamarin.Android.Bcl_Tests` overrides both of these and updates the `Filter` property so that "known failing" tests are excluded. This allows us to skip failing tests, giving us time to properly fix them in time while allowing the rest of this PR to be merged. The skipped tests include: * MonoTests.System.Reflection.AssemblyTest.GetReferencedAssemblies * MonoTests.System.ServiceModel.Description.WebInvokeAttributeTest.RejectTwoParametersWhenNotWrapped
2017-10-03 22:48:08 +03:00
<BundleItem Include="@(_BclTestOutput)" />
[xa-prep-tasks] Use the Mono bundle (#162) Context: https://github.com/xamarin/xamarin-android/commit/fbfd676c102c63e4e06e750857b178725e33450c Stage 3 of the cunning plan is to (attempt to) use the mono bundle introduced in commit fbfd676c. This "simple" desire (ha!) re-raises the architectural project dependency issue "solved" in fbfd676c, but first, a simple question: What should download the mono bundle? There are two plausible answers: 1. `make prepare` can (somehow) handle it. 2. MSBuild can (somehow) handle it. Both are plausible. The problem with using `make` targets (1) is there is increased potential for "duplication" -- duplication of the bundle filename, downloading it, and extracting it. Plus, `make` isn't "Windows friendly", in that GNU make isn't (normally) present with Visual Studio. (`NMAKE` is, but the Makefiles in this project are not compatible with `NMAKE`.) Which brings us to MSBuild (2): can it handle the task? To tackle that, we need to be able to have an MSBuild task project which has *no dependencies*, so that it can download and extract the mono bundle *before anything else runs*, as it may be downloading contents which mean that other projects don't *need* to run. The need for a "pre-bootstrap" task assembly -- called `xa-prep-tasks` -- thus "undoes" *some* of the logic regarding `libzip-windows.mdproj` and the `<Zip/>` task from fbfd676c: it isn't *possible* to rely on `libzip` from a "pre-build" state, as `libzip` is one of the things in the mono bundle, so now we need *two* "bootstrap" task assemblies: one without a `libzip` dependency -- `xa-prep-tasks.dll` -- and one *with* a `libzip` dependency -- `Xamarin.Android.Tools.BootstrapTasks.dll` Move tasks which don't currently require `libzip` -- or won't in the future, or laziness -- from `Xamarin.Android.Tools.BootstrapTasks.dll` and move them into `xa-prep-tasks.dll`. With that architectural compromise in place, add `xa-prep-tasks` as a `@(ProjectReference)` to various projects to help ensure it's built *first*, and rearchitect `bundle.mdproj` so that `xa-prep-tasks.targets` and `bundle.targets` can use the same targets to compute the bundle filename, now in `build-tools/bundle/bundle-path.targets`. Add a post-build step to `xa-prep-tasks.csproj` which downloads and extracts the expected mono bundle. One "problem" (feature?) is that the new `<SystemUnzip/>` task doesn't report errors as errors when unzip'ing the file. This turns out to be fine here because when downloading the mono bundle from Azure we don't get a 404 *anyway* -- Azure instead returns an XML document containing an error message (wat?!). We can thus ignore most error handling entirely...though we're *also* ignoring any checking for invalid downloads, which is something we should address in the future. Update the varioous project files so that they won't attempt to rebuild binaries that were present in the mono bundle.
2016-08-16 23:02:48 +03:00
</ItemGroup>
</Target>
<Target Name="ForceBuild"
DependsOnTargets="GetMonoBundleItems;$(ForceBuildDependsOn)"
Inputs="$(MonoSourceFullPath)\autogen.sh;$(LlvmSourceFullPath)\Makefile.config.in"
Outputs="@(BundleItem)">
</Target>
<Target Name="_BuildUnlessCached"
DependsOnTargets="_SetAutogenShTimeToLastCommitTimestamp;GetMonoBundleItems"
Inputs="$(MonoSourceFullPath)\autogen.sh;$(LlvmSourceFullPath)\Makefile.config.in"
Outputs="@(BundleItem)">
[bundle] Create the bundle (#206) Using the mono bundle (29568117, 57b18c27, ffe39776) didn't work as well as intended, because *using* the mono bundle requires that the bundle (1) *exist*, and (2) contain expected outputs. Neither of which is true: $ curl -o x.zip https://xamjenkinsartifact.blob.core.windows.net/xamarin-android/xamarin-android/bin/BuildDebug/bundle-v3-Debug-Darwin-libzip=1d8b1ac,llvm=8b1520c,mono=2c13a95.zip $ file x.zip x.zip: XML document text $ cat x.zip <?xml version="1.0" encoding="utf-8"?> <Error> <Code>BlobNotFound</Code> <Message>The specified blob does not exist. RequestId:f1722d1f-0001-0098-3fdd-fe4670000000 Time:2016-08-25T14:31:09.4102289Z </Message> </Error> That is, the URL "exists", but it doesn't contain anything *useful*. The bundle doesn't exist, in turn, because [*it isn't created*][0]: Project "/Users/builder/jenkins/workspace/xamarin-android/xamarin-android/build-tools/bundle/bundle.mdproj" (default target(s)): ... Done building project "/Users/builder/jenkins/workspace/xamarin-android/xamarin-android/build-tools/bundle/bundle.mdproj". What's *missing* from the log file is any mention of the `CreateBundle` target, which *creates the bundle*. Since the `CreateBundle` target isn't invoked, *there is no bundle*, and thus there's *nothing to upload*, and thus there's *nothing to download*, either. Why isn't the `CreateBundle` target executed? Because of property group "overwrites". But first, a step back. Remember 57b18c27? > I *assume* that this is due to `<CallTarget/>` being "flakey" within > xbuild (it passes the file containing the target to `<MSBuild/>`!), > and *somehow* in that "flakiness" `$(OutputPath)` doesn't end with > `\`, *which it should*, because `Microsoft.Common.targets` will > automatically append `\` to `$(OutputPath)`. Let's elaborate on the "flakiness" of xbuild: The `<CallTarget/>` task doesn't *just* invoke the specified target. Rather, xbuild will *reload the file containing the target*, "stand-alone", and execute the target. It's "as if" `<CallTarget/>` were implemented as: <MSBuild Projects="*File-Containing-The-Target-To-Invoke*" Targets="...Targets..." /> Therein lies the problem: by convention, ~everything is using the `$(BuildDependsOn)` property, and 29568117 altered `mono-runtimes.targets` so that `$(BuildDependsOn)` would be explicitly specified, but `mono-runtimes.targets` is `<Import/>`ed by `bundle.targets`, thus *replacing* the *intended* definition of `$(BuildDependsOn)` from `bundle.mdproj`. Thus, `$(BuildDependsOn)` *didn't* include the `CreateBundle` target, so the `CreateBundle` target was never executed, thus *no bundle was created or uploaded*. Doh! The fix? Remove all use of `<CallTarget/>`. `<CallTarget/>` is a BUG. Instead, *always* use `<MSBuild/>`, which allows specifying the correct project file to build from (the `.mdproj`, in our case), so that the correct "deferred" `ForceBuild` target can be invoked. Unfortunately, [using the `<MSBuild/> task raised *another* issue][1]: > this task uses the same MSBuild process to build the child projects. > The list of already-built targets that can be skipped is shared > between the parent and child builds. This means that the [child build *skips* running the `_GetRuntimesOutputItems` target][2], which means that *everything breaks* because nothing is built, because there are no `Inputs` or `Outputs` for anything. Target _BuildRuntimes: No input files were specified for target _BuildRuntimes, skipping. Done building target "_BuildRuntimes" in project "/Users/builder/jenkins/workspace/xamarin-android-pr-builder/xamarin-android/build-tools/mono-runtimes/mono-runtimes.mdproj". We work around this issue by providing a `$(_ForceXbuildToNotCacheTargets)` property to the `<MSBuild/>` task with the value of `DateTime.Now.Ticks`, i.e. a ~constantly changing value, which should *prevent* the target skipping. (We hope.) *Additionally*, `msbuild` reported a few errors when referencing `%(_LibZipTarget.OutputLibraryPath)` item metadata, so fix those references. Finally, another aspect of `<MSBuild/>` behavior arose: The [`MSBuild.Properties`][1] parameter is equivalent to the `msbuild /p:` parameter, which in turn means that the *value* is *immutable*; it *can't* be changed (or fixed!) from `.targets` files. The result being that `<MSBuild/>` use in `android-toolchain.targets` is *fraught with peril*: <MSBuild Projects="..\..\src\Xamarin.Android.Tools.BootstrapTasks\Xamarin.Android.Tools.BootstrapTasks.csproj" Properties="OutputPath=$(AndroidToolchainDirectory)" /> This *looks* reasonable, but `$(OutputPath)` is expected to end with the directory separator char, and `Microsoft.Common.targets` *tries* to ensure this: <OutputPath Condition="'$(OutputPath)' != '' and !HasTrailingSlash('$(OutputPath)')">$(OutputPath)\</OutputPath> However, since `MSBuild.Properties` values are *immutable*, this attempt to modify the `$(OutputPath)` property to end with a `\` has *no effect*, which results in *bizarre* errors later: error MSB4062: The "Xamarin.Android.BuildTools.PrepTasks.DownloadUri" task could not be loaded from the assembly $HOME/android-toolchainxa-prep-tasks.dll. Note the path, `$HOME/android-toolchainxa-prep-tasks.dll`. The path *should* be `$HOME/android-toolchain/xa-prep-tasks.dll`. What's missing is...the directory separator char after `$(OutputPath)`. The fix? Explicitly provide the directory separator character in the `<MSBuild/>` invocation: <MSBuild Projects="..\..\src\Xamarin.Android.Tools.BootstrapTasks\Xamarin.Android.Tools.BootstrapTasks.csproj" Properties="OutputPath=$(AndroidToolchainDirectory)\" /> [0]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android/51/consoleText [1]: https://msdn.microsoft.com/en-us/library/z7f65y0d.aspx [2]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android-pr-builder/131/consoleText
2016-09-03 00:13:23 +03:00
<PropertyGroup>
<_Now>$([System.DateTime]::Now.Ticks)</_Now>
</PropertyGroup>
<MSBuild
Projects="$(MSBuildThisFileDirectory)\mono-runtimes.mdproj"
Properties="_ForceXbuildToNotCacheTargets=$(_Now)"
Targets="ForceBuild"
/>
[xa-prep-tasks] Use the Mono bundle (#162) Context: https://github.com/xamarin/xamarin-android/commit/fbfd676c102c63e4e06e750857b178725e33450c Stage 3 of the cunning plan is to (attempt to) use the mono bundle introduced in commit fbfd676c. This "simple" desire (ha!) re-raises the architectural project dependency issue "solved" in fbfd676c, but first, a simple question: What should download the mono bundle? There are two plausible answers: 1. `make prepare` can (somehow) handle it. 2. MSBuild can (somehow) handle it. Both are plausible. The problem with using `make` targets (1) is there is increased potential for "duplication" -- duplication of the bundle filename, downloading it, and extracting it. Plus, `make` isn't "Windows friendly", in that GNU make isn't (normally) present with Visual Studio. (`NMAKE` is, but the Makefiles in this project are not compatible with `NMAKE`.) Which brings us to MSBuild (2): can it handle the task? To tackle that, we need to be able to have an MSBuild task project which has *no dependencies*, so that it can download and extract the mono bundle *before anything else runs*, as it may be downloading contents which mean that other projects don't *need* to run. The need for a "pre-bootstrap" task assembly -- called `xa-prep-tasks` -- thus "undoes" *some* of the logic regarding `libzip-windows.mdproj` and the `<Zip/>` task from fbfd676c: it isn't *possible* to rely on `libzip` from a "pre-build" state, as `libzip` is one of the things in the mono bundle, so now we need *two* "bootstrap" task assemblies: one without a `libzip` dependency -- `xa-prep-tasks.dll` -- and one *with* a `libzip` dependency -- `Xamarin.Android.Tools.BootstrapTasks.dll` Move tasks which don't currently require `libzip` -- or won't in the future, or laziness -- from `Xamarin.Android.Tools.BootstrapTasks.dll` and move them into `xa-prep-tasks.dll`. With that architectural compromise in place, add `xa-prep-tasks` as a `@(ProjectReference)` to various projects to help ensure it's built *first*, and rearchitect `bundle.mdproj` so that `xa-prep-tasks.targets` and `bundle.targets` can use the same targets to compute the bundle filename, now in `build-tools/bundle/bundle-path.targets`. Add a post-build step to `xa-prep-tasks.csproj` which downloads and extracts the expected mono bundle. One "problem" (feature?) is that the new `<SystemUnzip/>` task doesn't report errors as errors when unzip'ing the file. This turns out to be fine here because when downloading the mono bundle from Azure we don't get a 404 *anyway* -- Azure instead returns an XML document containing an error message (wat?!). We can thus ignore most error handling entirely...though we're *also* ignoring any checking for invalid downloads, which is something we should address in the future. Update the varioous project files so that they won't attempt to rebuild binaries that were present in the mono bundle.
2016-08-16 23:02:48 +03:00
</Target>
[mono-runtimes] Use $(IntermediateOutputPath), not `obj`. One of the "cute" things in mono-runtimes.props is that $(_CommonCFlags) changes based on $(Configuration): for Debug builds, `-O0 -fno-omit-frame-pointer` is used (no optimizations and make stack traces easier for gdb), while for Release builds `-O2` is used. The natural result of this is that Debug and Release builds *are not the same*, and that's a *good* thing. ...except that previously, both Debug and Release builds used the same `obj\%(Identity)` directories, which meant there weren't separate Debug and Release outputs, there were instead "whatever was built first" outputs, which is *madness*. Fix mono-runtimes.targets so that $(IntermediateOutputPath) is used instead instead of `obj`. $(IntermediateOutputPath) should be `obj\$(Configuration)`, so Debug and Release build artifacts will be kept properly separate. Additionally, fix the `Clean` target, so that the $(IntermediateOutputPath) subdirectories are properly removed.
2016-04-22 21:30:39 +03:00
<Target Name="_CleanRuntimes"
AfterTargets="Clean">
[mono-runtimes] Build AOT+LLVM cross-compilers (#125) The commit implements building of LLVM and cross-compilers to support Xamarin.Android/Mono AOT. LLVM and cross-compilers can be built for both the host platform (Linux and OS/X at the moment) as well as cross-compiled for 32-bit and 64-bit Windows platforms. Windows builds are done with MXE toolchain on OS/X and with the packaged mingw-w64 toolchain on Linux (tested on Ubuntu 16.04 ONLY). Also introducing a new set of MSBuild properties that contain information about the host system. Some of those properties (HostOS, HostCC, HostCXX for instance) have been moved from Configuration.props to better support auto-detection. A new script, build-tools/scripts/generate-os-info, is invoked as part of `make prepare` to generate file that contains the new properties. The generated file is required for the build to work and is also host-specific (it mustn't be moved between different machines) Cross compiler builds require access to a configured Mono build tree, in order to generate C structure offsets header file that is used by the AOT compilers to properly generate AOT-ed binaries. Therefore, even if a JIT target is not enabled in the configuration, enabling a cross-compiler for some target will configure Mono for that JIT target but it will NOT build it, to save time. To facilitate this, the _MonoRuntimes items defined in build-tools/mono-runtimes/mono-runtimes.projitems gain an additional metadata item called `DoBuild` which will be set to `true` if the runtime actually needs to be built, as opposed to just configured. MXE builds are disabled on Linux as mingw-w64 works just fine. A `make prepare` warning is issued for Linux hosts which have the binfmt_misc module enabled and either Wine of Mono (cli) registered as PE32/PE32+ binary interpreters. In such instance building of the Windows cross-compilers will fail because Autotools determine whether software is being cross compiled by building a test program and attempting to execute it. In normal circumstances such an attempt will fail, but with Windows cross-compilation and either Wine or Mono registered to handle the PE32 executables this attempt will succeed thus causing the cross compilation detection to fail. Currently to build cross compilers on Linux you need to generate the C structure offsets header file on OS/X and copy the resulting headers to appropriate places on Linux. The header files should be placed in build-tools/mono-runtimes/obj/Debug/cross-*/ directories. The header files are: {cross-arm,cross-arm-win}/aarch64-v8a-linux-android.h {cross-arm64,cross-arm64-win}/armv5-none-linux-androideabi.h {cross-x86,cross-x86-win}/i686-none-linux-android.h {cross-x86_64,cross-x86_64-win}/x86_64-none-linux-android.h Offsets header generation doesn't work on Linux atm because of missing support for it in the Mono utility used to generate the offsets. Hopefully this limitation will be removed in the near future and a start-to-end build of everything will be possible on Linux. It is now mandatory to run at least `make prepare-props` before Xamarin.Android can be built. The target generates the OS-specific props file which is required by the build. `make prepare` depends on the target.
2016-07-26 16:27:31 +03:00
<RemoveDir Directories="@(_MonoRuntime->'$(IntermediateOutputPath)\%(Identity)');@(_MonoCrossRuntime->'$(IntermediateOutputPath)\%(Identity)');$(_LlvmBuildDir32);$(_LlvmBuildDir64);$(_LlvmBuildDirWin32);$(_LlvmBuildDirWin64)" />
<Delete Files="@(_MonoRuntime->'$(IntermediateOutputPath)\%(Identity).config.cache');$(_CrossOutputPrefix)*.config.cache;$(_CrossOutputDirTop)\llvm*.config.cache" />
[mono-runtimes] Use $(IntermediateOutputPath), not `obj`. One of the "cute" things in mono-runtimes.props is that $(_CommonCFlags) changes based on $(Configuration): for Debug builds, `-O0 -fno-omit-frame-pointer` is used (no optimizations and make stack traces easier for gdb), while for Release builds `-O2` is used. The natural result of this is that Debug and Release builds *are not the same*, and that's a *good* thing. ...except that previously, both Debug and Release builds used the same `obj\%(Identity)` directories, which meant there weren't separate Debug and Release outputs, there were instead "whatever was built first" outputs, which is *madness*. Fix mono-runtimes.targets so that $(IntermediateOutputPath) is used instead instead of `obj`. $(IntermediateOutputPath) should be `obj\$(Configuration)`, so Debug and Release build artifacts will be kept properly separate. Additionally, fix the `Clean` target, so that the $(IntermediateOutputPath) subdirectories are properly removed.
2016-04-22 21:30:39 +03:00
</Target>
[mono-runtimes] Build required mono runtimes. Add a new build-tools/mono-runtimes project to build the required Mono runtimes for Xamarin.Android app execution on Android devices. Xamarin.Android 6.0 ("cycle 6") includes Mono runtimes compiled for five architectures: * armeabi (armv5) * armeabi-v7a (armv7) * arm64-v8a (aarch64) * x86 * x86_64 An additional "host" architecture is also built to build the "monodroid" profile assemblies such as mscorlib.dll. In the interest of expediency and not requiring that *six* different runtimes be built -- which can greatly increase build times and storage requirements -- the initial xamarin-android open-source build will only build the armeabi-v7a (armv7) and "host" architectures. Additionally, the <UnzipDirectoryChildren/> task needed to be altered to use unzip(1) on OS X instead of using System.IO.Compression.ZipFile, because the ZipFile implementation included with Mono 4.4 doesn't respect file attributes such as the +x, causing all files to be extracted as 644 (rw-r--r--), resulting in an NDK toolchain which wouldn't work. To resolve this issue, execution on OS X and Linux now uses unzip(1). ~~ Adding new architectures ~~ To build Mono for a new architecture, edit build-tools/mono-runtimes/mono-runtimes.projitems and add a new @(_MonoRuntime) entry for the desired architecture. TODO: Figure out if there's a reasonable way to "opt-in" or "opt-out" of CPU architectures so that it isn't an "all or nothing" prospect.
2016-04-20 04:30:00 +03:00
</Project>