xamarin
/
xamarin-android
зеркало из https://github.com/xamarin/xamarin-android.git
1
0
Форкнуть 0
Код Задачи Пакеты Проекты Релизы Вики Активность
xamarin-android/Makefile

223 строки
7.1 KiB
Makefile
Исходник Обычный вид История

Build stuff (#17) * Add the V make parameter to enable verbose logging Setting V to anything than 0 (zero) enables verbose logging for {ms,x}build as well as exports MONO_OPTIONS=--debug to enable stack traces should an exception be thrown when running mono. To add/override MONO_OPTIONS simply set it on the command line when invoking make, for instance: make V=1 MONO_OPTIONS="--trace" * File.Move doesn't overwrite targets File.Move will throw an exception whenever the target file exists. Assuming that the task should overwrite the existing target files for smooth operation, this change checks whether the target file exists and, if so, deletes it prior to calling File.Move
2016-05-04 15:03:01 +03:00
V ?= 0
[build] Makefile tweaks (#1870)
2018-06-25 23:07:56 +03:00
prefix = /usr/local
CONFIGURATION = Debug
RUNTIME := $(shell which mono64 2> /dev/null && echo mono64 || echo mono) --debug=casts
SOLUTION = Xamarin.Android.sln
TEST_TARGETS = build-tools/scripts/RunTests.targets
[build] Simplify building an API level other than default (#1353) This commits adds the `$(API_LEVEL)` make variable which, when set, will build Xamarin.Android and `Mono.Android.dll` for the specified API level. It takes advantage of the variables found in `build-tools/scripts/BuildEverything.mk` by indexing the `$(ALL_FRAMEWORKS)` variable with the API level number and setting the MSBuild properties accordingly.
2018-03-01 17:09:01 +03:00
API_LEVEL ?=
[build] Xamarin.Android build preparation utility (#3051) Fixes: https://github.com/xamarin/xamarin-android/issues/2562 The `make prepare` **make** target and `msbuild /t:Prepare` MSBuild target are used to "prepare" a `xamarin-android` checkout so that the subsequent `make all` or `msbuild /t:Build` invocation will work. The preparation step involves ensuring that all known required dependencies exist or are provisioned (54a7a029), such as the build utilities (`autoconf`/etc.), cross-compilers (MinGW), the Android SDK and NDK, and building "enough" to allow `Xamarin.Android.sln` to build (see also 7343965a0). Unfortunately, the Glorious Idea™ of using MSBuild for significant portions of this process is "baroque", making it difficult to understand and maintain, and the requirement on `msbuild` means that things break in really bizarre and obscure ways on Linux and macOS if the installed `mono` version is "too old"...and the determination of "too old" was being done via `mono` & `msbuild`! Furthermore, there was duplication and repetition of various bits of build configuration on Linux and macOS between Makefile and the MSBuild project system, which made it easier to make mistakes. (Then there's the occasional bizarro failures that would result from attempting to evaluate `$(PRODUCT_VERSION)` within `make`; it would fail when `MONO_LOG_LEVEL=info` was set. Then there's...) Instead of having `make prepare` be implemented as an unholy mess of `make` and `msbuild`, introduce the new `build-tools/xaprepare` app. `xaprepare` will be built and executed as part of `make prepare`, and takes over responsibility for repo build preparation. See also: build-tools/xaprepare/README.md TODO: Windows support
2019-06-07 20:58:59 +03:00
PREPARE_ARGS =
PREPARE_BUILD_LOG = bin/Build$(CONFIGURATION)/bootstrap-build.binlog
PREPARE_RESTORE_LOG = bin/Build$(CONFIGURATION)/bootstrap-restore.binlog
PREPARE_SOURCE_DIR = build-tools/xaprepare
PREPARE_SOLUTION = $(PREPARE_SOURCE_DIR)/xaprepare.sln
PREPARE_EXE = $(PREPARE_SOURCE_DIR)/xaprepare/bin/$(CONFIGURATION)/xaprepare.exe
PREPARE_COMMON_MSBUILD_FLAGS = /p:Configuration=$(CONFIGURATION) $(PREPARE_MSBUILD_ARGS) $(MSBUILD_ARGS)
PREPARE_MSBUILD_FLAGS = /binaryLogger:"$(PREPARE_BUILD_LOG)" $(PREPARE_COMMON_MSBUILD_FLAGS)
PREPARE_RESTORE_FLAGS = /binaryLogger:"$(PREPARE_RESTORE_LOG)" $(PREPARE_COMMON_MSBUILD_FLAGS)
PREPARE_SCENARIO =
[build] Simplify CI make and xaprepare invocations (#3279) * [build] Simplify CI make and xaprepare invocations Context: https://github.com/xamarin/xamarin-android/commit/9302d514a1437e5d1977da06dfbbe4e7054f33d5 Context: https://github.com/xamarin/xamarin-android/commit/f9f5b6cd120484b13df74e94638e07411d8693fa We built a lot of hacks into our make file when initially working on the xaprepare tool, as the windows and mac azure pipeline builds could not be modified at the time. We now have these builds defined in yaml, and as such we should be able to simplify the underlying make rules used by all of our builds. * Removes `prepare-build-ci` which no longer appears to be relevant, as the extra args it specifies don't appear to be set before the tool is built. * Removes `prepare-jenkins` and `prepare-commercial`, as we can now rely on the `prepare` rule again. The behavior of `prepare` will change depending on the `_PREPARE_ARGS` varible, which is controlled by different flags passed by our CI systems, `azure-pipelines.yaml`, and `build.groovy` * Introduces a new `prepareFlags` variable to build.groovy which is used to control whether or not the "prepare all" (-a arg) should be passed to xaprepare. This should not be specified when building Jenkins PR builds, as they only build a subset of all supported ABIs and frameworks. This should address the EmbeddedDSO test failures we are encountering in Jenkins PR builds. * Removes the MSBuild Auto Provision and ABI related args from make invocations in azure-pipelines.yaml. Make variables will instead be used to control these options. * Removes the `prepare-update-mono` dependency from `prepare-external-git-dependencies` as we should be able to call this without needing to update mono. * Adds a `prepare` dependency back to `jenkins`, now that the `prepare-jenkins` step is no longer required. * [build] Don't override _MSBUILD_ARGS when creating installers Context: https://github.com/xamarin/xamarin-android/commit/e83ba0ddc159d15abcfac1a5a4bce5a53e165d3f The `_MSBUILD_ARGS` override was initially required because we did not import different `rules.make` based on how our build was configured and prepared. The `xaprepare` tool now better controls the ABIs we should be building and packaging for each build type, so this is no longer needed.
2019-06-26 11:08:00 +03:00
PREPARE_CI_PR ?= 0
[build] Xamarin.Android build preparation utility (#3051) Fixes: https://github.com/xamarin/xamarin-android/issues/2562 The `make prepare` **make** target and `msbuild /t:Prepare` MSBuild target are used to "prepare" a `xamarin-android` checkout so that the subsequent `make all` or `msbuild /t:Build` invocation will work. The preparation step involves ensuring that all known required dependencies exist or are provisioned (54a7a029), such as the build utilities (`autoconf`/etc.), cross-compilers (MinGW), the Android SDK and NDK, and building "enough" to allow `Xamarin.Android.sln` to build (see also 7343965a0). Unfortunately, the Glorious Idea™ of using MSBuild for significant portions of this process is "baroque", making it difficult to understand and maintain, and the requirement on `msbuild` means that things break in really bizarre and obscure ways on Linux and macOS if the installed `mono` version is "too old"...and the determination of "too old" was being done via `mono` & `msbuild`! Furthermore, there was duplication and repetition of various bits of build configuration on Linux and macOS between Makefile and the MSBuild project system, which made it easier to make mistakes. (Then there's the occasional bizarro failures that would result from attempting to evaluate `$(PRODUCT_VERSION)` within `make`; it would fail when `MONO_LOG_LEVEL=info` was set. Then there's...) Instead of having `make prepare` be implemented as an unholy mess of `make` and `msbuild`, introduce the new `build-tools/xaprepare` app. `xaprepare` will be built and executed as part of `make prepare`, and takes over responsibility for repo build preparation. See also: build-tools/xaprepare/README.md TODO: Windows support
2019-06-07 20:58:59 +03:00
PREPARE_CI ?= 0
PREPARE_AUTOPROVISION ?= 0
PREPARE_IGNORE_MONO_VERSION ?= 1
[build] Simplify CI make and xaprepare invocations (#3279) * [build] Simplify CI make and xaprepare invocations Context: https://github.com/xamarin/xamarin-android/commit/9302d514a1437e5d1977da06dfbbe4e7054f33d5 Context: https://github.com/xamarin/xamarin-android/commit/f9f5b6cd120484b13df74e94638e07411d8693fa We built a lot of hacks into our make file when initially working on the xaprepare tool, as the windows and mac azure pipeline builds could not be modified at the time. We now have these builds defined in yaml, and as such we should be able to simplify the underlying make rules used by all of our builds. * Removes `prepare-build-ci` which no longer appears to be relevant, as the extra args it specifies don't appear to be set before the tool is built. * Removes `prepare-jenkins` and `prepare-commercial`, as we can now rely on the `prepare` rule again. The behavior of `prepare` will change depending on the `_PREPARE_ARGS` varible, which is controlled by different flags passed by our CI systems, `azure-pipelines.yaml`, and `build.groovy` * Introduces a new `prepareFlags` variable to build.groovy which is used to control whether or not the "prepare all" (-a arg) should be passed to xaprepare. This should not be specified when building Jenkins PR builds, as they only build a subset of all supported ABIs and frameworks. This should address the EmbeddedDSO test failures we are encountering in Jenkins PR builds. * Removes the MSBuild Auto Provision and ABI related args from make invocations in azure-pipelines.yaml. Make variables will instead be used to control these options. * Removes the `prepare-update-mono` dependency from `prepare-external-git-dependencies` as we should be able to call this without needing to update mono. * Adds a `prepare` dependency back to `jenkins`, now that the `prepare-jenkins` step is no longer required. * [build] Don't override _MSBUILD_ARGS when creating installers Context: https://github.com/xamarin/xamarin-android/commit/e83ba0ddc159d15abcfac1a5a4bce5a53e165d3f The `_MSBUILD_ARGS` override was initially required because we did not import different `rules.make` based on how our build was configured and prepared. The `xaprepare` tool now better controls the ABIs we should be building and packaging for each build type, so this is no longer needed.
2019-06-26 11:08:00 +03:00
_PREPARE_CI_MODE_PR_ARGS = --no-emoji --run-mode=CI
_PREPARE_CI_MODE_ARGS = $(_PREPARE_CI_MODE_PR_ARGS) -a
[build] Xamarin.Android build preparation utility (#3051) Fixes: https://github.com/xamarin/xamarin-android/issues/2562 The `make prepare` **make** target and `msbuild /t:Prepare` MSBuild target are used to "prepare" a `xamarin-android` checkout so that the subsequent `make all` or `msbuild /t:Build` invocation will work. The preparation step involves ensuring that all known required dependencies exist or are provisioned (54a7a029), such as the build utilities (`autoconf`/etc.), cross-compilers (MinGW), the Android SDK and NDK, and building "enough" to allow `Xamarin.Android.sln` to build (see also 7343965a0). Unfortunately, the Glorious Idea™ of using MSBuild for significant portions of this process is "baroque", making it difficult to understand and maintain, and the requirement on `msbuild` means that things break in really bizarre and obscure ways on Linux and macOS if the installed `mono` version is "too old"...and the determination of "too old" was being done via `mono` & `msbuild`! Furthermore, there was duplication and repetition of various bits of build configuration on Linux and macOS between Makefile and the MSBuild project system, which made it easier to make mistakes. (Then there's the occasional bizarro failures that would result from attempting to evaluate `$(PRODUCT_VERSION)` within `make`; it would fail when `MONO_LOG_LEVEL=info` was set. Then there's...) Instead of having `make prepare` be implemented as an unholy mess of `make` and `msbuild`, introduce the new `build-tools/xaprepare` app. `xaprepare` will be built and executed as part of `make prepare`, and takes over responsibility for repo build preparation. See also: build-tools/xaprepare/README.md TODO: Windows support
2019-06-07 20:58:59 +03:00
_PREPARE_ARGS =
[build] improve Xamarin.Android.sln usability in VS Windows (#3877) Xamarin.Android.sln has two issues that make `Xamarin.Android.sln` not work very well in VS Windows: * It builds MSBuild tasks and uses them within the same solution, such as `xa-prep-tasks.csproj` and `Xamarin.Android.Tools.BootstrapTasks.csproj`. * We have conditional `@(ProjectReference)`s. To improve things, I completely removed the `$(XAIntegratedTests)` MSBuild property (1b72a184) and removed any `@(ProjectReference)` that were conditional. I moved the two bootstrap `.csproj` files to a new `Xamarin.Android.BootstrapTasks.sln` which gets built during `make prepare` or `msbuild Xamarin.Android.sln /t:Prepare`. Visual Studio on Windows was also complaining about the `CreateManifestResourceNames` target missing for projects that merely import `Microsoft.Common.targets` such as `r8.csproj`. I added an empty target in `Configuration.props` to fix all cases of this. Unfortunately, things are still not perfect, as you can hit errors such as: Unable to copy file "obj\Debug\Xamarin.Android.Build.Tasks.dll" to "C:\src\xamarin-android\bin\Debug\lib\xamarin.android\xbuild\Xamarin\Android\Xamarin.Android.Build.Tasks.dll". The process cannot access the file 'C:\src\xamarin-android\bin\Debug\lib\xamarin.android\xbuild\Xamarin\Android\Xamarin.Android.Build.Tasks.dll' because it is being used by another process. Could not copy "obj\Debug\Xamarin.Android.Build.Tasks.dll" to "C:\src\xamarin-android\bin\Debug\lib\xamarin.android\xbuild\Xamarin\Android\Xamarin.Android.Build.Tasks.dll". Exceeded retry count of 10. Failed. The file is locked by: "Microsoft Visual Studio 2019 (26660), MSBuild.exe (18800), MSBuild.exe (17416)" Xamarin.Android.Build.Tasks This occurs if you modify `Xamarin.Android.Build.Tasks.csproj`, because VS Windows keeps MSBuild nodes alive, and if they have previously loaded `Xamarin.Android.Build.Tasks.dll` then the assembly is locked. I am not yet sure if we can get around this remaining problem, but we can revisit in the future.
2019-11-12 13:37:20 +03:00
BOOTSTRAP_SOLUTION = Xamarin.Android.BootstrapTasks.sln
BOOTSTRAP_BUILD_LOG = bin/Build$(CONFIGURATION)/bootstrap-build.binlog
[build] Sanitize Microsoft.Build* assembly references (#3995) Context: https://github.com/xamarin/monodroid/pull/1048 Context: https://github.com/xamarin/xamarin-android/pull/3928 Attempt to clean up and modernize various projects to help break up the scope of change required to convert them to .NET Standard libs. Bump the .NET Core version that `build.groovy` and `azure-pipelines.yaml` install to 3.1.100. This is to obtain various NuGet support fixes. Updates the shared `MSBuildReferences.projitems` file to instead include NuGet `@(PackageReference)`s to the required `Microsoft.Build.*.dll` assemblies. The `Mono.Posix.NETStandard`, `Xamarin.Build.AsyncTask`, and `Xamarin.LibZipSharp` NuGet references have also been moved into this file to make it easier to update versions in one place as needed. We're only using the compile time assets from the MSBuild NuGet packages, which ensures that their content is not copied to the output directory. An additional `@(None)` item group has been added to this file to copy the required assemblies from the system to the output directory so that `Xamarin.Android.Build.Tests.dll` can run against them. These changes are required because the MSBuild assemblies coming from NuGet do not yet fully support running on Mono. All projects which reference `MSBuildReferences.projitems` have been converted from the old `packages.config` to the new `@(PackageReference)` NuGet reference style. The Azure Pipeline build job has been updated to completely build `Xamarin.Android-Tests.sln` with `xabuild`, rather than only building select projects. This was done to mirror our Jenkins build and to ensure that `make all-tests` continues to work moving forward. .NET Core and `NuGetCommand` tasks have been bumped to bring in a newer version of NuGet with better `@(PackageReference)` support. NuGet restore will be skipped if any issues occur when calculating the dependency graph for the solution or project that it is operating on. A lot of our sources attempt to import `.props`/`.targets`/`.projitems` files that are generated by the build, and these should be conditionally imported so that NuGet restore attempts can succeed before these files are generated. This should also help when loading these projects in VS or VS Mac. Example NuGet restore error: src/Xamarin.Android.Build.Tasks/Xamarin.Android.Build.Tasks.targets(8,3): error MSB4019: The imported project "bin/BuildRelease/ProfileAssemblies.projitems" was not found. Confirm that the expression in the Import declaration "../../bin/BuildRelease/ProfileAssemblies.projitems" is correct, and that the file exists on disk. /var/folders/pp/x88nqyvd0h18ttl996vv2c3w0000gn/T/NuGetScratch/ok90j1ok.3t3.nugetrestore.targets(266,5): warning : Skipping restore for project 'src/Xamarin.Android.Build.Tasks/Xamarin.Android.Build.Tasks.csproj'. The project file may be invalid or missing targets required for restore. I've also moved the `NuGet.exe` that is copied and used by tests into its own folder to avoid a crash on macOS which would occur when `NuGet.exe` is invoked from the same folder as `Xunit.NetCore.Extensions.dll`: mono bin/TestRelease/NuGet.exe VTable setup of type Xunit.NetCore.Extensions.SkippedTestCase failed System.TypeLoadException: VTable setup of type Xunit.NetCore.Extensions.SkippedTestCase failed at (wrapper managed-to-native) System.RuntimeType.GetPropertiesByName_native(System.RuntimeType,intptr,System.Reflection.BindingFlags,System.RuntimeType/MemberListType) at System.RuntimeType.GetPropertiesByName (System.String name, System.Reflection.BindingFlags bindingAttr, System.RuntimeType+MemberListType listType, System.RuntimeType reflectedType) at System.RuntimeType.GetPropertyCandidates (System.String name, System.Reflection.BindingFlags bindingAttr, System.Type[] types, System.Boolean allowPrefixLookup) at System.RuntimeType.GetProperties (System.Reflection.BindingFlags bindingAttr) at System.ComponentModel.Composition.AttributedModel.AttributedPartCreationInfo+<GetExportMembers>d__32.MoveNext () at System.Linq.Enumerable.Any[TSource] (System.Collections.Generic.IEnumerable1[T] source) ...
2019-12-16 23:43:31 +03:00
BOOTSTRAP_MSBUILD_FLAGS = /t:Restore,Build /binaryLogger:"$(BOOTSTRAP_BUILD_LOG)" $(PREPARE_COMMON_MSBUILD_FLAGS)
[build] improve Xamarin.Android.sln usability in VS Windows (#3877) Xamarin.Android.sln has two issues that make `Xamarin.Android.sln` not work very well in VS Windows: * It builds MSBuild tasks and uses them within the same solution, such as `xa-prep-tasks.csproj` and `Xamarin.Android.Tools.BootstrapTasks.csproj`. * We have conditional `@(ProjectReference)`s. To improve things, I completely removed the `$(XAIntegratedTests)` MSBuild property (1b72a184) and removed any `@(ProjectReference)` that were conditional. I moved the two bootstrap `.csproj` files to a new `Xamarin.Android.BootstrapTasks.sln` which gets built during `make prepare` or `msbuild Xamarin.Android.sln /t:Prepare`. Visual Studio on Windows was also complaining about the `CreateManifestResourceNames` target missing for projects that merely import `Microsoft.Common.targets` such as `r8.csproj`. I added an empty target in `Configuration.props` to fix all cases of this. Unfortunately, things are still not perfect, as you can hit errors such as: Unable to copy file "obj\Debug\Xamarin.Android.Build.Tasks.dll" to "C:\src\xamarin-android\bin\Debug\lib\xamarin.android\xbuild\Xamarin\Android\Xamarin.Android.Build.Tasks.dll". The process cannot access the file 'C:\src\xamarin-android\bin\Debug\lib\xamarin.android\xbuild\Xamarin\Android\Xamarin.Android.Build.Tasks.dll' because it is being used by another process. Could not copy "obj\Debug\Xamarin.Android.Build.Tasks.dll" to "C:\src\xamarin-android\bin\Debug\lib\xamarin.android\xbuild\Xamarin\Android\Xamarin.Android.Build.Tasks.dll". Exceeded retry count of 10. Failed. The file is locked by: "Microsoft Visual Studio 2019 (26660), MSBuild.exe (18800), MSBuild.exe (17416)" Xamarin.Android.Build.Tasks This occurs if you modify `Xamarin.Android.Build.Tasks.csproj`, because VS Windows keeps MSBuild nodes alive, and if they have previously loaded `Xamarin.Android.Build.Tasks.dll` then the assembly is locked. I am not yet sure if we can get around this remaining problem, but we can revisit in the future.
2019-11-12 13:37:20 +03:00
[build] Xamarin.Android build preparation utility (#3051) Fixes: https://github.com/xamarin/xamarin-android/issues/2562 The `make prepare` **make** target and `msbuild /t:Prepare` MSBuild target are used to "prepare" a `xamarin-android` checkout so that the subsequent `make all` or `msbuild /t:Build` invocation will work. The preparation step involves ensuring that all known required dependencies exist or are provisioned (54a7a029), such as the build utilities (`autoconf`/etc.), cross-compilers (MinGW), the Android SDK and NDK, and building "enough" to allow `Xamarin.Android.sln` to build (see also 7343965a0). Unfortunately, the Glorious Idea™ of using MSBuild for significant portions of this process is "baroque", making it difficult to understand and maintain, and the requirement on `msbuild` means that things break in really bizarre and obscure ways on Linux and macOS if the installed `mono` version is "too old"...and the determination of "too old" was being done via `mono` & `msbuild`! Furthermore, there was duplication and repetition of various bits of build configuration on Linux and macOS between Makefile and the MSBuild project system, which made it easier to make mistakes. (Then there's the occasional bizarro failures that would result from attempting to evaluate `$(PRODUCT_VERSION)` within `make`; it would fail when `MONO_LOG_LEVEL=info` was set. Then there's...) Instead of having `make prepare` be implemented as an unholy mess of `make` and `msbuild`, introduce the new `build-tools/xaprepare` app. `xaprepare` will be built and executed as part of `make prepare`, and takes over responsibility for repo build preparation. See also: build-tools/xaprepare/README.md TODO: Windows support
2019-06-07 20:58:59 +03:00
all:
$(call MSBUILD_BINLOG,all,$(_SLN_BUILD)) $(MSBUILD_FLAGS) $(SOLUTION)
-include bin/Build$(CONFIGURATION)/rules.mk
ifeq ($(OS_NAME),)
export OS_NAME := $(shell uname)
endif
ifeq ($(OS_ARCH),)
export OS_ARCH := $(shell uname -m)
endif
export NO_SUDO ?= false
ifneq ($(NO_SUDO),false)
_PREPARE_ARGS += --auto-provisioning-uses-sudo=false
endif
ifneq ($(V),0)
MONO_OPTIONS += --debug
NUGET_VERBOSITY = -Verbosity Detailed
_PREPARE_ARGS += -v:d
endif
[build] Simplify CI make and xaprepare invocations (#3279) * [build] Simplify CI make and xaprepare invocations Context: https://github.com/xamarin/xamarin-android/commit/9302d514a1437e5d1977da06dfbbe4e7054f33d5 Context: https://github.com/xamarin/xamarin-android/commit/f9f5b6cd120484b13df74e94638e07411d8693fa We built a lot of hacks into our make file when initially working on the xaprepare tool, as the windows and mac azure pipeline builds could not be modified at the time. We now have these builds defined in yaml, and as such we should be able to simplify the underlying make rules used by all of our builds. * Removes `prepare-build-ci` which no longer appears to be relevant, as the extra args it specifies don't appear to be set before the tool is built. * Removes `prepare-jenkins` and `prepare-commercial`, as we can now rely on the `prepare` rule again. The behavior of `prepare` will change depending on the `_PREPARE_ARGS` varible, which is controlled by different flags passed by our CI systems, `azure-pipelines.yaml`, and `build.groovy` * Introduces a new `prepareFlags` variable to build.groovy which is used to control whether or not the "prepare all" (-a arg) should be passed to xaprepare. This should not be specified when building Jenkins PR builds, as they only build a subset of all supported ABIs and frameworks. This should address the EmbeddedDSO test failures we are encountering in Jenkins PR builds. * Removes the MSBuild Auto Provision and ABI related args from make invocations in azure-pipelines.yaml. Make variables will instead be used to control these options. * Removes the `prepare-update-mono` dependency from `prepare-external-git-dependencies` as we should be able to call this without needing to update mono. * Adds a `prepare` dependency back to `jenkins`, now that the `prepare-jenkins` step is no longer required. * [build] Don't override _MSBUILD_ARGS when creating installers Context: https://github.com/xamarin/xamarin-android/commit/e83ba0ddc159d15abcfac1a5a4bce5a53e165d3f The `_MSBUILD_ARGS` override was initially required because we did not import different `rules.make` based on how our build was configured and prepared. The `xaprepare` tool now better controls the ABIs we should be building and packaging for each build type, so this is no longer needed.
2019-06-26 11:08:00 +03:00
ifneq ($(PREPARE_CI_PR),0)
_PREPARE_ARGS += $(_PREPARE_CI_MODE_PR_ARGS)
endif
[build] Xamarin.Android build preparation utility (#3051) Fixes: https://github.com/xamarin/xamarin-android/issues/2562 The `make prepare` **make** target and `msbuild /t:Prepare` MSBuild target are used to "prepare" a `xamarin-android` checkout so that the subsequent `make all` or `msbuild /t:Build` invocation will work. The preparation step involves ensuring that all known required dependencies exist or are provisioned (54a7a029), such as the build utilities (`autoconf`/etc.), cross-compilers (MinGW), the Android SDK and NDK, and building "enough" to allow `Xamarin.Android.sln` to build (see also 7343965a0). Unfortunately, the Glorious Idea™ of using MSBuild for significant portions of this process is "baroque", making it difficult to understand and maintain, and the requirement on `msbuild` means that things break in really bizarre and obscure ways on Linux and macOS if the installed `mono` version is "too old"...and the determination of "too old" was being done via `mono` & `msbuild`! Furthermore, there was duplication and repetition of various bits of build configuration on Linux and macOS between Makefile and the MSBuild project system, which made it easier to make mistakes. (Then there's the occasional bizarro failures that would result from attempting to evaluate `$(PRODUCT_VERSION)` within `make`; it would fail when `MONO_LOG_LEVEL=info` was set. Then there's...) Instead of having `make prepare` be implemented as an unholy mess of `make` and `msbuild`, introduce the new `build-tools/xaprepare` app. `xaprepare` will be built and executed as part of `make prepare`, and takes over responsibility for repo build preparation. See also: build-tools/xaprepare/README.md TODO: Windows support
2019-06-07 20:58:59 +03:00
ifneq ($(PREPARE_CI),0)
_PREPARE_ARGS += $(_PREPARE_CI_MODE_ARGS)
endif
ifneq ($(PREPARE_AUTOPROVISION),0)
_PREPARE_ARGS += --auto-provision=yes --auto-provision-uses-sudo=yes
endif
[android-toolchain] Permit zero-configuration builds. This might be a suspect idea, but lets see if we can make this work. [The Joel Test: 12 Steps to Better Code][0] outlines 12 steps to better code. The first two steps are: 1. Do you use source control? 2. Can you make a build in one step? github is being used for source control, so (1) is handled, but how simple can we make (2)? How easy can we make it to build Xamarin.Android upon a fresh checkout? The ideal to strive for is simple: Load Xamarin.Android.sln into your IDE and Build the project. I *know* we're not going to be able to do this, if only because we're going to be using git submodules, which will require a separate `git submodule init` invocation [1]. Knowing we can't reach that level of simplicitly doesn't mean we shouldn't *try* to reach it for all other parts of the build system. Which brings us to the Android NDK and SDK. The Android NDK will be required in order to build native code, such as libmonodroid.so, while the Android SDK will be required in order to compile Java Callable Wrappers (née Android Callable Wrappers [2]) and eventual samples and unit tests. There are three ways we can deal with the Android NDK and SDK: 1. Complicate the "build" process by requiring that developers go to the Android SDK Download Page [3], download and install "somewhere" the required bits, and then configure the Xamarin.Android build to use these bits. 2. Complicate the "build" process by requiring that developers run the Xamarin Unified Installer [4], let it install everything required, then configure the Xamarin.Android build to use those bits. 3. Painstakingly determine which files are actually required, then automatically download and extract those files into a "well-known" location known by the Xamarin.Android build process. (1) and (2) can be infuriating. Let's give (3) a try. :-) Add a Xamarin.Android.Tools.BootstrapTasks project which contains MSBuild tasks to facilitate downloading the Android SDK and NDK files. Add an android-toolchain project which uses Xamarin.Android.Tools.BootstrapTasks to download a painstakingly determined set of files and install them "somewhere". Unfortunately [5] the "somewhere" to download and install these files needs to be in a known absolute path, so I've arbitrary decided to download the files into $(HOME)\android-archives and install them into $(HOME)\android-toolchain. On windows, this is %HOMEDRIVE%%HOMEPATH%\android-archives and %HOMEDRIVE%%HOMEPATH%\android-toolchain. These locations may be modified by creating a Configuration.Override.props file; see README.md for details. TL;DR: This setup is able to magically download the Android NDK and SDK files and install them for later use in a reasonably overridable location, all within MSBuild. [0]: http://www.joelonsoftware.com/articles/fog0000000043.html [1]: Though maybe there's some MSBuild-fu we can use to address that. [2]: https://developer.xamarin.com/guides/android/advanced_topics/java_integration_overview/android_callable_wrappers/ [3]: http://developer.android.com/sdk/index.html [4]: https://www.xamarin.com/download [5]: Because I couldn't find a reliable way to use $(SolutionDir) when only building a project, and relative paths would require an in-tree installation location, which might not work.
2016-04-19 03:33:04 +03:00
Bump to lipzip/rel-1-5-1/b95cf3fd (#1721) Bumps to LibZipSharp/master/4d3e59c8 Main reason is to include fixes for a couple CVEs. Changes: * 1.3.0 * Support bzip2 compressed zip archives * Improve file progress callback code * Fix `zip_fdopen()` * CVE-2017-12858: Fix double `free()`. * CVE-2017-14107: Improve EOCD64 parsing. * 1.3.1 * Install `zipconf.h` into `${PREFIX}/include` * Add `zip_libzip_version()` * Fix AES tests on Linux * 1.3.2 * Fix bug introduced in last: `zip_t` was erroneously freed if `zip_close()` failed. * 1.4.0 * Improve build with `cmake` * Retire autoconf/automake build system * Add `zip_source_buffer_fragment()`. * Add support to clone unchanged beginning of archive (instead of rewriting it). Supported for buffer sources and on Apple File System. * Add support for Microsoft Universal Windows Platform. * 1.5.0 * Use standard cryptographic library instead of custom AES implementation. This also simplifies the license. * Use clang-format to format the source code. * More Windows improvements. * 1.5.1 * Choose format of installed documentation based on available tools. * Fix visibility of symbols. * Fix zipcmp directory support. * Don’t set RPATH on Linux. * Use Libs.private for link dependencies in pkg-config file. * Fix build with LibreSSL. * Various bugfixes. Additionally: * make it possible to build Windows version of libzip on Linux with mingw (no mxe required) * build on macOS targeting Windows with system/brew cmake * Linux "host" builds *must* use the system `cmake`, to prevent it from using any MXE-cross-compiled versions. * export `MACOSX_DEPLOYMENT_TARGET=10.11` in the top-level Makefile to specify the minimum version of macOS we support. Without this it is possible that code built on 10.11 with Xcode targeting a newer version of the system will not work on macOS older than the version targetted by Xcode being used. For instance, if we build on macOS 10.11 with Xcode 8.2 (which targets macOS 10.12) then the code which just built on 10.11 may not work on this very system - because it may use APIs available only starting from 10.12
2018-05-29 15:13:13 +03:00
ifeq ($(OS_NAME),Darwin)
[build] Xamarin.Android build preparation utility (#3051) Fixes: https://github.com/xamarin/xamarin-android/issues/2562 The `make prepare` **make** target and `msbuild /t:Prepare` MSBuild target are used to "prepare" a `xamarin-android` checkout so that the subsequent `make all` or `msbuild /t:Build` invocation will work. The preparation step involves ensuring that all known required dependencies exist or are provisioned (54a7a029), such as the build utilities (`autoconf`/etc.), cross-compilers (MinGW), the Android SDK and NDK, and building "enough" to allow `Xamarin.Android.sln` to build (see also 7343965a0). Unfortunately, the Glorious Idea™ of using MSBuild for significant portions of this process is "baroque", making it difficult to understand and maintain, and the requirement on `msbuild` means that things break in really bizarre and obscure ways on Linux and macOS if the installed `mono` version is "too old"...and the determination of "too old" was being done via `mono` & `msbuild`! Furthermore, there was duplication and repetition of various bits of build configuration on Linux and macOS between Makefile and the MSBuild project system, which made it easier to make mistakes. (Then there's the occasional bizarro failures that would result from attempting to evaluate `$(PRODUCT_VERSION)` within `make`; it would fail when `MONO_LOG_LEVEL=info` was set. Then there's...) Instead of having `make prepare` be implemented as an unholy mess of `make` and `msbuild`, introduce the new `build-tools/xaprepare` app. `xaprepare` will be built and executed as part of `make prepare`, and takes over responsibility for repo build preparation. See also: build-tools/xaprepare/README.md TODO: Windows support
2019-06-07 20:58:59 +03:00
ifeq ($(HOMEBREW_PREFIX),)
[build] Use Homebrew mingw-w64, not MXE (#2758) MXE, the [M cross environment][mxe], is used to build Windows binary code on Linux and macOS so that we don't need a Windows machine to produce our Windows `.vsix` files. This has many pros and cons, including: * **Pro**: Building *mono* on Windows has historically been difficult. While there is much better support *now* for building mono for Windows, we also need to build mono *targeting Android*, built via the Android NDK toolchain, and @jonpryor *suspects*, but has not verified, that such a cross-compilation scenario would still be a terrible experience on Windows. (This is why Windows builds require a mono bundle; see 29568117.) * **Pro**: Building both the macOS and Windows commercial installers on macOS means that, where possible, the *same* files can be present in both installers. File such as BCL assemblies, `Mono.Android.dll`, MSBuild targets, etc. Separate macOS & Windows build machines could introduce the possibility for "drift" and accidental variations between the systems. * **Con**: MXE *itself* is a giant pain. See also: b28856f4, in which we see that MXE requires Xcode 8.3 to build; the current Xcode version is 10.1, and 8.3 is nearly 2 years old. * Xcode 8.3 means we can't use the latest-and-greatest Mac mini hardware for builds, as it won't run Xcode 8.3. * It can take *tens* of minutes to build MXE... * ...assuming it builds AT ALL; it frequently fails to build. Finally, the compilers that MXE provides are *ancient*, providing gcc 5.4.0 (circa 2015), which is 3-4 years old now. This is crimping @grendello's style in `src/monodroid`, wherein he wants to use C++17 features when reasonable, but gcc 5.4 doesn't support C++17. We also want to use newer compilers on the assumption that they'll generate more/better warnings around unsafe code practices, improved stack overflow checking code, and other security-related concerns. We thus would like to move away from MXE, but to what? Turns Out™, [Homebrew][hb] now has a [mingw-w64 formula][mingw] to install the latest MinGW toolchain with GCC 8.2, which allows us to drop MXE, and we already depend on Homebrew on macOS (82ce2118, 0c073f67f). The downside of using this pre-packaged MinGW -- a very insignificant one -- is that it does not come with any extra libraries or software prebuilt for Windows, and our build requires `zlib` which was previously built as part of MXE. This downside is easily worked around, however: we simply use our own Homebrew tap which has a formula that installs the Win32/Win64 `zlib` binary. This new tap is available in [xamarin/homebrew-xamarin-android-windeps][windep]. This commit drops MXE dependency in Xamarin.Android *only*; it does *not* remove this dependency from Mono. Mono will now provision its own copy of MXE and use it for builds. This will change when [mono/mono#12434][m12434] makes its way into Mono and, subsequently, to xamarin-android. All the libraries we build for Windows that require `zlib` will link a *static* version of the library instead of using the dynamic one. This is safer than using a `.DLL` because we control what version of `zlib` is used and we leave no potential for breakage should a 3rd party replace/remove/etc. `zlib1.dll` from the target system. [mxe]: http://mxe.cc [hb]: https://brew.sh [mingw]: https://formulae.brew.sh/formula/mingw-w64 [m12434]: https://github.com/mono/mono/pull/12434 [windep]: https://github.com/xamarin/homebrew-xamarin-android-windeps
2019-02-28 06:33:36 +03:00
HOMEBREW_PREFIX ?= $(shell brew --prefix)
[build] Xamarin.Android build preparation utility (#3051) Fixes: https://github.com/xamarin/xamarin-android/issues/2562 The `make prepare` **make** target and `msbuild /t:Prepare` MSBuild target are used to "prepare" a `xamarin-android` checkout so that the subsequent `make all` or `msbuild /t:Build` invocation will work. The preparation step involves ensuring that all known required dependencies exist or are provisioned (54a7a029), such as the build utilities (`autoconf`/etc.), cross-compilers (MinGW), the Android SDK and NDK, and building "enough" to allow `Xamarin.Android.sln` to build (see also 7343965a0). Unfortunately, the Glorious Idea™ of using MSBuild for significant portions of this process is "baroque", making it difficult to understand and maintain, and the requirement on `msbuild` means that things break in really bizarre and obscure ways on Linux and macOS if the installed `mono` version is "too old"...and the determination of "too old" was being done via `mono` & `msbuild`! Furthermore, there was duplication and repetition of various bits of build configuration on Linux and macOS between Makefile and the MSBuild project system, which made it easier to make mistakes. (Then there's the occasional bizarro failures that would result from attempting to evaluate `$(PRODUCT_VERSION)` within `make`; it would fail when `MONO_LOG_LEVEL=info` was set. Then there's...) Instead of having `make prepare` be implemented as an unholy mess of `make` and `msbuild`, introduce the new `build-tools/xaprepare` app. `xaprepare` will be built and executed as part of `make prepare`, and takes over responsibility for repo build preparation. See also: build-tools/xaprepare/README.md TODO: Windows support
2019-06-07 20:58:59 +03:00
endif
[build] Use Homebrew mingw-w64, not MXE (#2758) MXE, the [M cross environment][mxe], is used to build Windows binary code on Linux and macOS so that we don't need a Windows machine to produce our Windows `.vsix` files. This has many pros and cons, including: * **Pro**: Building *mono* on Windows has historically been difficult. While there is much better support *now* for building mono for Windows, we also need to build mono *targeting Android*, built via the Android NDK toolchain, and @jonpryor *suspects*, but has not verified, that such a cross-compilation scenario would still be a terrible experience on Windows. (This is why Windows builds require a mono bundle; see 29568117.) * **Pro**: Building both the macOS and Windows commercial installers on macOS means that, where possible, the *same* files can be present in both installers. File such as BCL assemblies, `Mono.Android.dll`, MSBuild targets, etc. Separate macOS & Windows build machines could introduce the possibility for "drift" and accidental variations between the systems. * **Con**: MXE *itself* is a giant pain. See also: b28856f4, in which we see that MXE requires Xcode 8.3 to build; the current Xcode version is 10.1, and 8.3 is nearly 2 years old. * Xcode 8.3 means we can't use the latest-and-greatest Mac mini hardware for builds, as it won't run Xcode 8.3. * It can take *tens* of minutes to build MXE... * ...assuming it builds AT ALL; it frequently fails to build. Finally, the compilers that MXE provides are *ancient*, providing gcc 5.4.0 (circa 2015), which is 3-4 years old now. This is crimping @grendello's style in `src/monodroid`, wherein he wants to use C++17 features when reasonable, but gcc 5.4 doesn't support C++17. We also want to use newer compilers on the assumption that they'll generate more/better warnings around unsafe code practices, improved stack overflow checking code, and other security-related concerns. We thus would like to move away from MXE, but to what? Turns Out™, [Homebrew][hb] now has a [mingw-w64 formula][mingw] to install the latest MinGW toolchain with GCC 8.2, which allows us to drop MXE, and we already depend on Homebrew on macOS (82ce2118, 0c073f67f). The downside of using this pre-packaged MinGW -- a very insignificant one -- is that it does not come with any extra libraries or software prebuilt for Windows, and our build requires `zlib` which was previously built as part of MXE. This downside is easily worked around, however: we simply use our own Homebrew tap which has a formula that installs the Win32/Win64 `zlib` binary. This new tap is available in [xamarin/homebrew-xamarin-android-windeps][windep]. This commit drops MXE dependency in Xamarin.Android *only*; it does *not* remove this dependency from Mono. Mono will now provision its own copy of MXE and use it for builds. This will change when [mono/mono#12434][m12434] makes its way into Mono and, subsequently, to xamarin-android. All the libraries we build for Windows that require `zlib` will link a *static* version of the library instead of using the dynamic one. This is safer than using a `.DLL` because we control what version of `zlib` is used and we leave no potential for breakage should a 3rd party replace/remove/etc. `zlib1.dll` from the target system. [mxe]: http://mxe.cc [hb]: https://brew.sh [mingw]: https://formulae.brew.sh/formula/mingw-w64 [m12434]: https://github.com/mono/mono/pull/12434 [windep]: https://github.com/xamarin/homebrew-xamarin-android-windeps
2019-02-28 06:33:36 +03:00
else
HOMEBREW_PREFIX := $prefix
Bump to lipzip/rel-1-5-1/b95cf3fd (#1721) Bumps to LibZipSharp/master/4d3e59c8 Main reason is to include fixes for a couple CVEs. Changes: * 1.3.0 * Support bzip2 compressed zip archives * Improve file progress callback code * Fix `zip_fdopen()` * CVE-2017-12858: Fix double `free()`. * CVE-2017-14107: Improve EOCD64 parsing. * 1.3.1 * Install `zipconf.h` into `${PREFIX}/include` * Add `zip_libzip_version()` * Fix AES tests on Linux * 1.3.2 * Fix bug introduced in last: `zip_t` was erroneously freed if `zip_close()` failed. * 1.4.0 * Improve build with `cmake` * Retire autoconf/automake build system * Add `zip_source_buffer_fragment()`. * Add support to clone unchanged beginning of archive (instead of rewriting it). Supported for buffer sources and on Apple File System. * Add support for Microsoft Universal Windows Platform. * 1.5.0 * Use standard cryptographic library instead of custom AES implementation. This also simplifies the license. * Use clang-format to format the source code. * More Windows improvements. * 1.5.1 * Choose format of installed documentation based on available tools. * Fix visibility of symbols. * Fix zipcmp directory support. * Don’t set RPATH on Linux. * Use Libs.private for link dependencies in pkg-config file. * Fix build with LibreSSL. * Various bugfixes. Additionally: * make it possible to build Windows version of libzip on Linux with mingw (no mxe required) * build on macOS targeting Windows with system/brew cmake * Linux "host" builds *must* use the system `cmake`, to prevent it from using any MXE-cross-compiled versions. * export `MACOSX_DEPLOYMENT_TARGET=10.11` in the top-level Makefile to specify the minimum version of macOS we support. Without this it is possible that code built on 10.11 with Xcode targeting a newer version of the system will not work on macOS older than the version targetted by Xcode being used. For instance, if we build on macOS 10.11 with Xcode 8.2 (which targets macOS 10.12) then the code which just built on 10.11 may not work on this very system - because it may use APIs available only starting from 10.12
2018-05-29 15:13:13 +03:00
endif
[build] Xamarin.Android build preparation utility (#3051) Fixes: https://github.com/xamarin/xamarin-android/issues/2562 The `make prepare` **make** target and `msbuild /t:Prepare` MSBuild target are used to "prepare" a `xamarin-android` checkout so that the subsequent `make all` or `msbuild /t:Build` invocation will work. The preparation step involves ensuring that all known required dependencies exist or are provisioned (54a7a029), such as the build utilities (`autoconf`/etc.), cross-compilers (MinGW), the Android SDK and NDK, and building "enough" to allow `Xamarin.Android.sln` to build (see also 7343965a0). Unfortunately, the Glorious Idea™ of using MSBuild for significant portions of this process is "baroque", making it difficult to understand and maintain, and the requirement on `msbuild` means that things break in really bizarre and obscure ways on Linux and macOS if the installed `mono` version is "too old"...and the determination of "too old" was being done via `mono` & `msbuild`! Furthermore, there was duplication and repetition of various bits of build configuration on Linux and macOS between Makefile and the MSBuild project system, which made it easier to make mistakes. (Then there's the occasional bizarro failures that would result from attempting to evaluate `$(PRODUCT_VERSION)` within `make`; it would fail when `MONO_LOG_LEVEL=info` was set. Then there's...) Instead of having `make prepare` be implemented as an unholy mess of `make` and `msbuild`, introduce the new `build-tools/xaprepare` app. `xaprepare` will be built and executed as part of `make prepare`, and takes over responsibility for repo build preparation. See also: build-tools/xaprepare/README.md TODO: Windows support
2019-06-07 20:58:59 +03:00
ifeq ($(wildcard Configuration.OperatingSystem.props),)
PREPARE_MSBUILD_FLAGS += "/p:HostHomebrewPrefix=$(HOMEBREW_PREFIX)"
Build stuff (#17) * Add the V make parameter to enable verbose logging Setting V to anything than 0 (zero) enables verbose logging for {ms,x}build as well as exports MONO_OPTIONS=--debug to enable stack traces should an exception be thrown when running mono. To add/override MONO_OPTIONS simply set it on the command line when invoking make, for instance: make V=1 MONO_OPTIONS="--trace" * File.Move doesn't overwrite targets File.Move will throw an exception whenever the target file exists. Assuming that the task should overwrite the existing target files for smooth operation, this change checks whether the target file exists and, if so, deletes it prior to calling File.Move
2016-05-04 15:03:01 +03:00
endif
[build] Xamarin.Android build preparation utility (#3051) Fixes: https://github.com/xamarin/xamarin-android/issues/2562 The `make prepare` **make** target and `msbuild /t:Prepare` MSBuild target are used to "prepare" a `xamarin-android` checkout so that the subsequent `make all` or `msbuild /t:Build` invocation will work. The preparation step involves ensuring that all known required dependencies exist or are provisioned (54a7a029), such as the build utilities (`autoconf`/etc.), cross-compilers (MinGW), the Android SDK and NDK, and building "enough" to allow `Xamarin.Android.sln` to build (see also 7343965a0). Unfortunately, the Glorious Idea™ of using MSBuild for significant portions of this process is "baroque", making it difficult to understand and maintain, and the requirement on `msbuild` means that things break in really bizarre and obscure ways on Linux and macOS if the installed `mono` version is "too old"...and the determination of "too old" was being done via `mono` & `msbuild`! Furthermore, there was duplication and repetition of various bits of build configuration on Linux and macOS between Makefile and the MSBuild project system, which made it easier to make mistakes. (Then there's the occasional bizarro failures that would result from attempting to evaluate `$(PRODUCT_VERSION)` within `make`; it would fail when `MONO_LOG_LEVEL=info` was set. Then there's...) Instead of having `make prepare` be implemented as an unholy mess of `make` and `msbuild`, introduce the new `build-tools/xaprepare` app. `xaprepare` will be built and executed as part of `make prepare`, and takes over responsibility for repo build preparation. See also: build-tools/xaprepare/README.md TODO: Windows support
2019-06-07 20:58:59 +03:00
ifneq ($(PREPARE_SCENARIO),)
_PREPARE_ARGS += -s:"$(PREPARE_SCENARIO)"
Build stuff (#17) * Add the V make parameter to enable verbose logging Setting V to anything than 0 (zero) enables verbose logging for {ms,x}build as well as exports MONO_OPTIONS=--debug to enable stack traces should an exception be thrown when running mono. To add/override MONO_OPTIONS simply set it on the command line when invoking make, for instance: make V=1 MONO_OPTIONS="--trace" * File.Move doesn't overwrite targets File.Move will throw an exception whenever the target file exists. Assuming that the task should overwrite the existing target files for smooth operation, this change checks whether the target file exists and, if so, deletes it prior to calling File.Move
2016-05-04 15:03:01 +03:00
endif
[xaprepare] Add `xaprepare -refresh` option (#3228) Add an `xaprepare -refresh` option which provides a way to force reinstallation of certain programs or dependencies. Usage: xaprepare.exe -refresh=sdk,ndk Usage of `xaprepare -refresh` can be enabled via **make**(1) by setting the `$(XA_FORCE_COMPONENT_REFRESH)` value to `true`: make prepare XA_FORCE_COMPONENT_REFRESH=true A few of our Android SDK components don't specify a package revision. This breaks caching and updating for these components. If a local version of one of these components is already installed we won't attempt to replace it if it is out of date. As a result, we've likely been sitting on out of date `platform-tools` (`adb`?), and `emulator` versions on many of our build/test machines. These caching fixes will hopefully help address some of the emulator performance and `adb` related crashes we've seen.
2019-06-18 17:32:21 +03:00
ifeq ($(XA_FORCE_COMPONENT_REFRESH),true)
_PREPARE_ARGS += -refresh
endif
[build] Xamarin.Android build preparation utility (#3051) Fixes: https://github.com/xamarin/xamarin-android/issues/2562 The `make prepare` **make** target and `msbuild /t:Prepare` MSBuild target are used to "prepare" a `xamarin-android` checkout so that the subsequent `make all` or `msbuild /t:Build` invocation will work. The preparation step involves ensuring that all known required dependencies exist or are provisioned (54a7a029), such as the build utilities (`autoconf`/etc.), cross-compilers (MinGW), the Android SDK and NDK, and building "enough" to allow `Xamarin.Android.sln` to build (see also 7343965a0). Unfortunately, the Glorious Idea™ of using MSBuild for significant portions of this process is "baroque", making it difficult to understand and maintain, and the requirement on `msbuild` means that things break in really bizarre and obscure ways on Linux and macOS if the installed `mono` version is "too old"...and the determination of "too old" was being done via `mono` & `msbuild`! Furthermore, there was duplication and repetition of various bits of build configuration on Linux and macOS between Makefile and the MSBuild project system, which made it easier to make mistakes. (Then there's the occasional bizarro failures that would result from attempting to evaluate `$(PRODUCT_VERSION)` within `make`; it would fail when `MONO_LOG_LEVEL=info` was set. Then there's...) Instead of having `make prepare` be implemented as an unholy mess of `make` and `msbuild`, introduce the new `build-tools/xaprepare` app. `xaprepare` will be built and executed as part of `make prepare`, and takes over responsibility for repo build preparation. See also: build-tools/xaprepare/README.md TODO: Windows support
2019-06-07 20:58:59 +03:00
_PREPARE_ARGS += $(PREPARE_ARGS)
[build] Add build dependencies for Arch (#664) Add `make prepare` support for Arch Linux. Re-factor the `make prepare` logic for Linux into a set of `build-tools/scripts/dependencies/linux-prepare-*.sh` shell scripts, so that instead of having every possible Linux distro combined into `Makefile`, we can split them up in a more sensible manner. Dependency scripts should have one of the names: linux-prepare-$(LINUX_DISTRO).sh linux-prepare-$(LINUX_DISTRO)-$(LINUX_DISTRO_RELEASE).sh where `$(LINUX_DISTRO)` is from `lsb_release -i -s`, and `$(LINUX_DISTRO_RELEASE)` is from `lsb_release -r -s`. The dependency script should install all required dependencies to build the xamarin-android repo, *unless* `$NO_SUDO` is set. If the `$NO_SUDO` environment variable is `true`, then the dependency script should check to see if all dependencies are installed. If they are not installed, it should exit with a non-0 (zero) value. If `$NO_SUDO` is `false`, then `sudo` should be used to install the dependencies.
2017-06-27 23:31:22 +03:00
include build-tools/scripts/msbuild.mk
[Xamarin.Android.Build.Utilities] Add AndroidVersions (#599) Commit 8e7d37bb is a sign of duplication: in order to use a `Mono.Android.dll` binding assembly, not only do we need to *build* the binding assembly, but we *also* need to update `Xamarin.Android.Build.Utilities.dll` to "know" about the new binding version (to map API level to `$(TargetFrameworkVersion)`). Even "better" (worse), if the new API level is a *preview*, there is no *consistent* API level. For example, with API-O, `android.jar` isn't at `$(AndroidSdkDirectory)\platforms\android-@API_LEVEL@\android.jar`, where `@API_LEVEL@` is 26 (because various codepaths require that the "api level" be an integer). Instead, it's installed at `$(AndroidSdkDirectory)\platforms\android-O\android.jar`, where `O` is the "id" of the preview API level. This "id" is "leaky", in turn requiring that `Xamarin.Android.Build.Tasks.dll` *also* be updated to deal with the mappings. Even "better" (worse), if we *forget* to cross all our our 't's and dot all of our 'i's, we'll have a binding assembly which can't be used. (Which is why we needed commit 8e7d37bb; without it, the API-O binding can't be used!) This is all obviously madness. ;-) Clean this mess up: 1. Update `src/Mono.Android` to create a new `AndroidApiInfo.xml` file within the `$(TargetFrameworkVersion)` directory. This will contain all the information needed to map Android API levels to Ids and Android OS versions and `$(TargetFrameworkVersion)` values. ```xml <AndroidApiInfo> <Id>10</Id> <Level>10</Level> <Name>Gingerbread</Name> <Version>v2.3</Version> </AndroidApiInfo> ``` 2. Add a new `Xamarin.Android.Build.Utilities.AndroidVersions` type which looks for and parses these new `AndroidApiInfo.xml` files. 3. Fixup all the other places using `AndroidVersion.KnownVersions` and related members to instead use `AndroidVersions`. 4. Remove all the old APIs which rely on hardcoded data. The advantage to all this is that we can support new API level bindings by just building a new `Mono.Android.dll` and placing an `AndroidApiInfo.xml` into the appropriate location (next to `Mono.Android.dll`). No further code changes would be required. Related: The build system still has a nasy habit of using system-wide directories to resolve files we'd really rather it not, e.g. in [xamarin-android/master build #503][m503]: [m503]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android/503/consoleText Building target "_GetReferenceAssemblyPaths" in project "/Users/builder/jenkins/workspace/xamarin-android/xamarin-android/src/Xamarin.Android.NUnitLite/Xamarin.Android.NUnitLite.csproj" ("/Users/builder/jenkins/workspace/xamarin-android/xamarin-android/bin/Debug/lib/xamarin.android/xbuild/Xamarin/Android/Xamarin.Android.Common.targets"); "_SetLatestTargetFrameworkVersion" depends on it. Target _GetReferenceAssemblyPaths: Task "GetReferenceAssemblyPaths" Using task GetReferenceAssemblyPaths from Microsoft.Build.Tasks.GetReferenceAssemblyPaths, Microsoft.Build.Tasks.Core, Version=14.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a Looking for framework 'MonoAndroid,Version=v1.0' in root path '/Library/Frameworks/Mono.framework/External/xbuild-frameworks' Found framework definition list '/Library/Frameworks/Mono.framework/External/xbuild-frameworks/MonoAndroid/v1.0/RedistList/FrameworkList.xml' for framework 'MonoAndroid,Version=v1.0' Done executing task "GetReferenceAssemblyPaths" Done building target "_GetReferenceAssemblyPaths" in project "/Users/builder/jenkins/workspace/xamarin-android/xamarin-android/src/Xamarin.Android.NUnitLite/Xamarin.Android.NUnitLite.csproj". Done building target "_GetReferenceAssemblyPaths" in project "/Users/builder/jenkins/workspace/xamarin-android/xamarin-android/src/Xamarin.Android.NUnitLite/Xamarin.Android.NUnitLite.csproj" ("/Users/builder/jenkins/workspace/xamarin-android/xamarin-android/bin/Debug/lib/xamarin.android/xbuild/Xamarin/Android/Xamarin.Android.Common.targets"); "_SetLatestTargetFrameworkVersion" depends on it. Target _SetLatestTargetFrameworkVersion: Task "ResolveSdks" ReferenceAssemblyPaths: /Library/Frameworks/Mono.framework/External/xbuild-frameworks/MonoAndroid/v1.0/ Note the use of the path `/Library/Frameworks/Mono.framework/External/xbuild-frameworks/MonoAndroid/v1.0`. This *was* "fine" -- if undesirable -- but with the introduction of `AndroidVersions` and `AndroidApiInfo.xml` files within the frameworks directory, everything falls apart, becuase the system install *does not have* the `AndroidApiInfo.xml` files, so *NO API LEVELS ARE FOUND*. Oops. The only way to fix this is to use `$XBUILD_FRAMEWORK_FOLDERS_PATH`, and the sanest way to do *that* is to use `tools/scripts/xabuild` for the primary build. Unfortunately, using `xabuild` for the primary build breaks the `msbuild`-based build, as it can't resolve the `.NETFramework,Version=v4.6.1` framework. Context: https://github.com/xamarin/xamarin-android/pull/599#issuecomment-319801330 Split the difference here by introducing `$(_SLN_BUILD)`: use `xabuild` when building with `xbuild`, and continue using `msbuild` when building with `msbuild`.
2017-08-23 14:19:11 +03:00
ifeq ($(USE_MSBUILD),1)
_SLN_BUILD = $(MSBUILD)
else # $(MSBUILD) != 1
_SLN_BUILD = MSBUILD="$(MSBUILD)" tools/scripts/xabuild
endif # $(USE_MSBUILD) == 1
[build] Simplify building an API level other than default (#1353) This commits adds the `$(API_LEVEL)` make variable which, when set, will build Xamarin.Android and `Mono.Android.dll` for the specified API level. It takes advantage of the variables found in `build-tools/scripts/BuildEverything.mk` by indexing the `$(ALL_FRAMEWORKS)` variable with the API level number and setting the MSBuild properties accordingly.
2018-03-01 17:09:01 +03:00
ifneq ($(API_LEVEL),)
[build] Xamarin.Android build preparation utility (#3051) Fixes: https://github.com/xamarin/xamarin-android/issues/2562 The `make prepare` **make** target and `msbuild /t:Prepare` MSBuild target are used to "prepare" a `xamarin-android` checkout so that the subsequent `make all` or `msbuild /t:Build` invocation will work. The preparation step involves ensuring that all known required dependencies exist or are provisioned (54a7a029), such as the build utilities (`autoconf`/etc.), cross-compilers (MinGW), the Android SDK and NDK, and building "enough" to allow `Xamarin.Android.sln` to build (see also 7343965a0). Unfortunately, the Glorious Idea™ of using MSBuild for significant portions of this process is "baroque", making it difficult to understand and maintain, and the requirement on `msbuild` means that things break in really bizarre and obscure ways on Linux and macOS if the installed `mono` version is "too old"...and the determination of "too old" was being done via `mono` & `msbuild`! Furthermore, there was duplication and repetition of various bits of build configuration on Linux and macOS between Makefile and the MSBuild project system, which made it easier to make mistakes. (Then there's the occasional bizarro failures that would result from attempting to evaluate `$(PRODUCT_VERSION)` within `make`; it would fail when `MONO_LOG_LEVEL=info` was set. Then there's...) Instead of having `make prepare` be implemented as an unholy mess of `make` and `msbuild`, introduce the new `build-tools/xaprepare` app. `xaprepare` will be built and executed as part of `make prepare`, and takes over responsibility for repo build preparation. See also: build-tools/xaprepare/README.md TODO: Windows support
2019-06-07 20:58:59 +03:00
MSBUILD_FLAGS += /p:AndroidApiLevel=$(API_LEVEL) /p:AndroidFrameworkVersion=$(word $(API_LEVEL), $(ALL_FRAMEWORKS)) /p:AndroidPlatformId=$(word $(a), $(ALL_PLATFORM_IDS))
[build] Simplify building an API level other than default (#1353) This commits adds the `$(API_LEVEL)` make variable which, when set, will build Xamarin.Android and `Mono.Android.dll` for the specified API level. It takes advantage of the variables found in `build-tools/scripts/BuildEverything.mk` by indexing the `$(ALL_FRAMEWORKS)` variable with the API level number and setting the MSBuild properties accordingly.
2018-03-01 17:09:01 +03:00
endif
[Xamarin.Android-Tests] Split out "full" apps (#146) We're trying to get [`make jenkins`][0] working on Jenkins, and [it's failing][1], as one might expect when a particular repo and associated build system has never been run on Jenkins before: Android.App/ApplicationTest.cs(9,7): error CS0246: The type or namespace name `NUnit' could not be found. Are you missing an assembly reference? This error occurs while building `src/Mono.Android/Test/Mono.Android-Tests.csproj`, and happens because the `Xamarin.Android.NUnitLite.dll` assembly isn't referenced... because it isn't *found*: Microsoft.Common.targets: warning : Reference 'Xamarin.Android.NUnitLite' not resolved This returns us to a long-standing issue which I thought was mentioned in a commit somewhere, but I can't find at present: MSBuild doesn't support updating the assembly resolution directories *while MSBuild is running*. For example, we build `Xamarin.Android.NUniteLite.dll` into `bin/$(Configuration/lib/xbuild-frameworks/MonoAndroid/v1.0`, but "normal Xamarin.Android referencing projects" don't use a `@(ProjectReference)` to `Xamarin.Android.NUnitLite.csproj`, but instead a `@(Reference)` to the assembly: <!-- src/Mono.Android/Test/Mono.Android-Tests.csproj --> <Reference Include="Xamarin.Android.NUnitLite" /> The *requirement* for a "proper" Xamarin.Android SDK install is that `Xamarin.Android.NUnitLite.dll` can be found through a normal `@(Reference)`. In order to satisfy this requirement, we need to tell MSBuild where to find it, which can be with `xbuild` via the `$MSBuildExtensionsPath` and `$XBUILD_FRAMEWORK_FOLDERS_PATH` *environment variables*. *Environment variables*. MSBuild doesn't provide a way to update environment variables, short of writing a new Task which calls `Environment.SetEnvironmentVariable()`, and while [this works][2], it doesn't *actually* work [^3]. The short of all this is that it isn't possible, *within a single `xbuild` invocation*, to both build the Xamarin.Android SDK "environment" *and use that environment* as intended for "normal" apps. The fix, as is often the case, is to bend with the wind. Instead of requiring the impossible, move `src/Mono.Android/Test/Mono.Android-Tests.csproj` into a *new* `Xamarin.Android-Tests.sln` solution, *out of* `Xamarin.Android.sln`. This allows building `Xamarin.Android.sln` without error in a pristine environment -- that is, one that doesn't already have a system-wide Xamarin.Android install -- and separately building the tests by using `tools/scripts/xabuild`, which *can* export environment variables to manipulate `xbuild` behavior so that things Just Work™. Building `Mono.Android-Tests.csproj` and similar projects (`HelloWorld.csproj`!) can be done by using the new `make all-tests` make target. [0]: https://github.com/xamarin/xamarin-android/commit/a16673d3eb2c4945c3a74f5f8154603d7658fc9a [1]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android/20/console [2]: https://github.com/xamarin/xamarin-android/pull/147 [^3]: [PR #147][2] isn't viable because of [xbuild's `AsssemblyResolver`][4]. There's no way to clear/invalidate `target_framework_cache`. The idea of PR #147 was to "hack things up" so that `Xamarin.Android.NuniteLite.dll` would be properly resolved during the build of `Mono.Android-Tests.csproj` *when building everything*. The problem is *this can't work*, because `xbuild` has a "target framework cache," with no way to invalidate it, and the cache is populated the first time the target framework is used. Due to build ordering issues, this first use is *before* `Xamarin.Android.NunitLite.dll` was built, and thus it doesn't exist in the cache. The result: Task "ResolveAssemblyReference" .... TargetFrameworkDirectories: /Volumes/Seagate4TB/work/xamarin-android/bin/Debug/lib/xbuild-frameworks/MonoAndroid/v7.0 /Volumes/Seagate4TB/work/xamarin-android/bin/Debug/lib/xbuild-frameworks/MonoAndroid/v1.0 /Volumes/Seagate4TB/work/xamarin-android/bin/Debug/lib/xbuild-frameworks/MonoAndroid/v1.0/Facades/ ... Primary Reference System.Xml Reference System.Xml resolved to /Volumes/Seagate4TB/work/xamarin-android/bin/Debug/lib/xbuild-frameworks/MonoAndroid/v1.0/System.Xml.dll. CopyLocal = False ... Primary Reference Xamarin.Android.NUnitLite Microsoft.Common.targets: warning : Reference 'Xamarin.Android.NUnitLite' not resolved For searchpath {TargetFrameworkDirectory} Considered target framework dir /Volumes/Seagate4TB/work/xamarin-android/bin/Debug/lib/xbuild-frameworks/MonoAndroid/v7.0, assembly named 'Xamarin.Android.NUnitLite' not found. Considered target framework dir /Volumes/Seagate4TB/work/xamarin-android/bin/Debug/lib/xbuild-frameworks/MonoAndroid/v1.0, assembly named 'Xamarin.Android.NUnitLite' not found. Considered target framework dir /Volumes/Seagate4TB/work/xamarin-android/bin/Debug/lib/xbuild-frameworks/MonoAndroid/v1.0/Facades/, assembly named 'Xamarin.Android.NUnitLite' not found. ... Consequently, the `mcs` invocation is missing a `/reference:path/to/Xamarin.Android.NUniteLite.dll`, and compilation fails: Android.App/ApplicationTest.cs(9,7): error CS0246: The type or namespace name `NUnit' could not be found. Are you missing an assembly reference? ...plus 29 others... [4]: https://github.com/mono/mono/blob/dd8aadf/mcs/class/Microsoft.Build.Tasks/Microsoft.Build.Tasks/AssemblyResolver.cs#L131
2016-08-08 23:46:13 +03:00
all-tests::
[build] Normal & Diagnostic Log Verbosity, at once! Fixes: https://github.com/xamarin/xamarin-android/issues/1792 We have a bit of a conundrum: when things break, the diagnostic build logs are *essential* to tracking down *why* something broke. However, the sheer *verbosity* of those diagnostic logs makes it difficult to determine *what* broke; it's very easy to get lost. Additionally, the gigantic log files make the **Console Output** Jenkins link nearly useless, as most web browsers will cease up when attempting to download ~1GB log files. Split the difference: introduce a new `$(call MSBUILD_BINLOG)` syntax which creates an [MSBuild binary log][0] file, which is like diagnostic output *but even moreso*, while *also* overriding the console logging to `/v:normal`. [0]: https://github.com/Microsoft/msbuild/wiki/Binary-Log This should allow the **Console Output** link to be a bit more reasoanble, and if something goes wrong we can download the separate `*.binlog` files for further analysis. MSBuild binary log files can be read with MSBuild itself: msbuild /v:diag bin/BuildDebug/msbuild-*.binlog Note: above command will actually fail; you can only print one log file at a time with that syntax. The created log files follow the pattern `msbuild-EPOCH-NAME`, where EPOCH is the `time_t`/Unix epoch value *at the time the command was executed*. This should allow filenames to be non-colliding, and allow plausibly sane sorting behavior. (Maybe.)
2018-06-12 04:28:38 +03:00
MSBUILD="$(MSBUILD)" $(call MSBUILD_BINLOG,all-tests,tools/scripts/xabuild) $(MSBUILD_FLAGS) Xamarin.Android-Tests.sln
[install] Add `make install` target (#684) The `make install` target can be used on Unixy platforms to install Xamarin.Android into a system-wide location. By default it installs into a prefix of `/usr/local`: make install The prefix may be changed by overriding the `$(prefix)` make variable: # Install system-wide make install prefix=/usr Installation is performed by copying the contents of `bin/$(CONFIGURATION)` into `$(prefix)/lib/xamarin.android`. Only `xabuild` is installed into `$(prefix)/bin` and (presumably) accessible within `$PATH`.
2017-07-03 20:09:22 +03:00
install::
@if [ ! -d "bin/$(CONFIGURATION)" ]; then \
echo "run 'make all' before you execute 'make install'!"; \
exit 1; \
fi
[structure] Rework installation directory structure (#704) Context: https://github.com/xamarin/xamarin-android/pull/253#discussion_r82862993 The *intention* is that Jenkins-produced `oss-xamarin.android*.zip` artifacts be usable on Windows, so that side-by-side testing can be performed without replacing the system installation. Usage is in [UsingJenkinsBuildArtifacts.md](Documentation/UsingJenkinsBuildArtifacts). This isn't *entirely* the case. It was *apparently* the case at the time of commit 87ca2737, but things have bitrotten since. For example, following the 87ca2737 instructions would currently result in an XA0020 `Could not find monodroid` error, because `class-parse.exe` wasn't where Windows expects it to be (it was in `lib/mandroid`, not `lib/xbuild/Xamarin/Android`). This needs to be fixed. Additionally, PR #253 mentions that, for filesystem organization, it would be useful if the macOS/Linux directory structure -- `$prefix/bin`, `$prefix/lib/mandroid`, `$prefix/lib/xbuild/Xamarin/Android`, `$prefix/lib/xbuild-frameworks` -- more closely resembled the Windows directory structure of `$MSBuild` and `$ReferenceAssemblies` (as seen in `.vsix` files). This would turn macOS/Linux into using `$xa_prefix/xbuild` and `$xa_prefix/xbuild-frameworks` directories. `$prefix/bin` would only contain `xabuild`. What is currently in `$prefix/lib/mandroid` would be merged with `$xa_prefix/xbuild/Xamarin/Android`. `$xa_prefix` would `$prefix/lib/xamarin.android`. This would turn the current macOS structure: $prefix/bin/xabuild $prefix/bin/generator $prefix/bin/cross-arm $prefix/lib/mandroid/generator.exe $prefix/lib/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xbuild/Xamarin/Android/Xamarin.Android.Common.targets Into: $prefix/bin/xabuild $prefix/lib/xamarin.android/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xamarin.android/xbuild/Xamarin/Android/generator.exe $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Xamarin.Android.Common.targets $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Darwin/cross-arm Other notes: * The `bundle-*.zip` filename has been has been changed to include a *hash* of the contents of various files, in particular `build-tools\mono-runtimes.*`. This was instigated via a conversation with @kumpera about making the filename more "idiot-proof": `mono-runtimes.props` contains *compiler flags*, and if any of those change, then *logically* the bundle should differ as well, as the mono runtimes may differ in significant ways. In theory, the `-vXX` version should be used to track this, but this is a manual change, easy to overlook. The new `-hHASH` part of the filename should be more automagic. The new `<HashFileContents/>` task in `xa-prep-tasks.dll` is responsible for creating the hash value. * `Configuration.Java.Interop.Override.props` was moved into `build-tools/scripts`, because that would cleanup the root directory a bit. * OS-specific binaries are now placed into `$prefix/lib/xamarin.android/xbuild/Xamarin/Android/$(HostOS)`. On a theoretical plus side, this means that the same build directory can contain OS-specific binaries for multiple operating systems. (I don't know if anyone shares a build tree between e.g. macOS and Linux, but if anyone *does*...) Unfortunately this requires a workaround for an `xbuild` bug: `%(_LlvmRuntime.InstallPath)` and `%(_MonoCrossRuntime.InstallPath)` *cannot* use MSBuild properties, e.g. `<InstallPath>$(HostOS)/</InstallPath>` doesn't work as desired; it's instead treated literally. Special-case `%(InstallPath)` until we fully migrate to MSBuild. * `$(MonoAndroidToolsDirectory)` should be considered *dead*, along with `$(MonoAndroidBinDirectory)`, as these should now *always* be the same directory as where `Xamarin.Android.Build.Tasks.dll` is located, or a `$(HostOS)` sub-directory. * `Xamarin.ProjectTools.csproj` needed to be updated to ensure that the build order was correct. * Remove all `[Obsolete]` and unreferenced members from `Xamarin.Android.Build.Utilities.dll`. There's too much in there, and it makes my head hurt trying to understand the interrelationships between it all. If it's not used, it's gone. * The changes to `src/monodroid/jni/Android.mk` are...weird. The removal of `-I$(topdir)/libmonodroid/zip`/etc. is to reduce use of non-existent paths, as `$(topdir)` isn't defined, so that's *actually* `-I/libmonodroid/zip`, which is nonsensical. So far, so good. What's *odd* is the addition of `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`: This is needed so that `external/mono/support/zlib-helper.c` exports `CreateZStream` and related symbols, otherwise we get a unit test failure in `GzipStreamTest.Compression` due to an `EntryPointNotFoundException`, because `CreateZStream` isn't exported/public. What's odd here is that I don't understand what caused this behavior to change. Previous builds exported `CreateZStream`, otherwise the tests would fail, and I don't understand how any of the other changes in this PR would be at fault, though that's certainly the most plausible explanation. Regardless, `-Ijni` *first* (due to adding `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`) is the desired behavior, so that `jni/config.h` is included, thus ensuring that `MONO_API` has the required definition when building `zlib-helper.c`.
2017-07-28 16:36:32 +03:00
-mkdir -p "$(prefix)/bin"
[install] Add `make install` target (#684) The `make install` target can be used on Unixy platforms to install Xamarin.Android into a system-wide location. By default it installs into a prefix of `/usr/local`: make install The prefix may be changed by overriding the `$(prefix)` make variable: # Install system-wide make install prefix=/usr Installation is performed by copying the contents of `bin/$(CONFIGURATION)` into `$(prefix)/lib/xamarin.android`. Only `xabuild` is installed into `$(prefix)/bin` and (presumably) accessible within `$PATH`.
2017-07-03 20:09:22 +03:00
-mkdir -p "$(prefix)/lib/mono/xbuild-frameworks"
-mkdir -p "$(prefix)/lib/xamarin.android"
-mkdir -p "$(prefix)/lib/mono/xbuild/Xamarin/"
[structure] Rework installation directory structure (#704) Context: https://github.com/xamarin/xamarin-android/pull/253#discussion_r82862993 The *intention* is that Jenkins-produced `oss-xamarin.android*.zip` artifacts be usable on Windows, so that side-by-side testing can be performed without replacing the system installation. Usage is in [UsingJenkinsBuildArtifacts.md](Documentation/UsingJenkinsBuildArtifacts). This isn't *entirely* the case. It was *apparently* the case at the time of commit 87ca2737, but things have bitrotten since. For example, following the 87ca2737 instructions would currently result in an XA0020 `Could not find monodroid` error, because `class-parse.exe` wasn't where Windows expects it to be (it was in `lib/mandroid`, not `lib/xbuild/Xamarin/Android`). This needs to be fixed. Additionally, PR #253 mentions that, for filesystem organization, it would be useful if the macOS/Linux directory structure -- `$prefix/bin`, `$prefix/lib/mandroid`, `$prefix/lib/xbuild/Xamarin/Android`, `$prefix/lib/xbuild-frameworks` -- more closely resembled the Windows directory structure of `$MSBuild` and `$ReferenceAssemblies` (as seen in `.vsix` files). This would turn macOS/Linux into using `$xa_prefix/xbuild` and `$xa_prefix/xbuild-frameworks` directories. `$prefix/bin` would only contain `xabuild`. What is currently in `$prefix/lib/mandroid` would be merged with `$xa_prefix/xbuild/Xamarin/Android`. `$xa_prefix` would `$prefix/lib/xamarin.android`. This would turn the current macOS structure: $prefix/bin/xabuild $prefix/bin/generator $prefix/bin/cross-arm $prefix/lib/mandroid/generator.exe $prefix/lib/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xbuild/Xamarin/Android/Xamarin.Android.Common.targets Into: $prefix/bin/xabuild $prefix/lib/xamarin.android/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xamarin.android/xbuild/Xamarin/Android/generator.exe $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Xamarin.Android.Common.targets $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Darwin/cross-arm Other notes: * The `bundle-*.zip` filename has been has been changed to include a *hash* of the contents of various files, in particular `build-tools\mono-runtimes.*`. This was instigated via a conversation with @kumpera about making the filename more "idiot-proof": `mono-runtimes.props` contains *compiler flags*, and if any of those change, then *logically* the bundle should differ as well, as the mono runtimes may differ in significant ways. In theory, the `-vXX` version should be used to track this, but this is a manual change, easy to overlook. The new `-hHASH` part of the filename should be more automagic. The new `<HashFileContents/>` task in `xa-prep-tasks.dll` is responsible for creating the hash value. * `Configuration.Java.Interop.Override.props` was moved into `build-tools/scripts`, because that would cleanup the root directory a bit. * OS-specific binaries are now placed into `$prefix/lib/xamarin.android/xbuild/Xamarin/Android/$(HostOS)`. On a theoretical plus side, this means that the same build directory can contain OS-specific binaries for multiple operating systems. (I don't know if anyone shares a build tree between e.g. macOS and Linux, but if anyone *does*...) Unfortunately this requires a workaround for an `xbuild` bug: `%(_LlvmRuntime.InstallPath)` and `%(_MonoCrossRuntime.InstallPath)` *cannot* use MSBuild properties, e.g. `<InstallPath>$(HostOS)/</InstallPath>` doesn't work as desired; it's instead treated literally. Special-case `%(InstallPath)` until we fully migrate to MSBuild. * `$(MonoAndroidToolsDirectory)` should be considered *dead*, along with `$(MonoAndroidBinDirectory)`, as these should now *always* be the same directory as where `Xamarin.Android.Build.Tasks.dll` is located, or a `$(HostOS)` sub-directory. * `Xamarin.ProjectTools.csproj` needed to be updated to ensure that the build order was correct. * Remove all `[Obsolete]` and unreferenced members from `Xamarin.Android.Build.Utilities.dll`. There's too much in there, and it makes my head hurt trying to understand the interrelationships between it all. If it's not used, it's gone. * The changes to `src/monodroid/jni/Android.mk` are...weird. The removal of `-I$(topdir)/libmonodroid/zip`/etc. is to reduce use of non-existent paths, as `$(topdir)` isn't defined, so that's *actually* `-I/libmonodroid/zip`, which is nonsensical. So far, so good. What's *odd* is the addition of `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`: This is needed so that `external/mono/support/zlib-helper.c` exports `CreateZStream` and related symbols, otherwise we get a unit test failure in `GzipStreamTest.Compression` due to an `EntryPointNotFoundException`, because `CreateZStream` isn't exported/public. What's odd here is that I don't understand what caused this behavior to change. Previous builds exported `CreateZStream`, otherwise the tests would fail, and I don't understand how any of the other changes in this PR would be at fault, though that's certainly the most plausible explanation. Regardless, `-Ijni` *first* (due to adding `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`) is the desired behavior, so that `jni/config.h` is included, thus ensuring that `MONO_API` has the required definition when building `zlib-helper.c`.
2017-07-28 16:36:32 +03:00
cp -a "bin/$(CONFIGURATION)/lib/xamarin.android/." "$(prefix)/lib/xamarin.android/"
[install] Remove previous Novell dir before installation (#1736) Fixes: https://github.com/xamarin/xamarin-android/issues/1648 Fixes problematic directory re-use with `ln`: $ make install prefix=/devel # works $ make install prefix=/devel ... ln: failed to create symbolic link '/devel/lib/mono/xbuild/Novell/Novell': File exists Makefile:39: recipe for target 'install' failed Delete the `$(prefix)/lib/mono/xbuild/Novell` directory before trying to create it as a symlink, so that the above error is avoided.
2018-05-28 16:59:02 +03:00
-rm -rf "$(prefix)/lib/mono/xbuild/Novell"
[install] Remove previous Xamarin.Android.Sdk.{props,targets} (#1776) Update the `make install` target so that `Xamarin.Android.Sdk.props` and `Xamarin.Android.Sdk.target` are always removed. If these are not removed, it's possible that older versions from a previous `make install` invocation will be present, resulting in an inconsistent installation and related confusion, such as a wrong `$(TargetFrameworkRootPath)` value.
2018-06-01 12:40:34 +03:00
-rm -rf "$(prefix)/lib/mono/xbuild/Xamarin/Xamarin.Android.Sdk.props"
-rm -rf "$(prefix)/lib/mono/xbuild/Xamarin/Xamarin.Android.Sdk.targets"
Fix `make install` target. rm without -rf fails if there isn't the directory. (#686)
2017-07-06 10:05:42 +03:00
-rm -rf "$(prefix)/lib/mono/xbuild/Xamarin/Android"
-rm -rf "$(prefix)/lib/mono/xbuild-frameworks/MonoAndroid"
[structure] Rework installation directory structure (#704) Context: https://github.com/xamarin/xamarin-android/pull/253#discussion_r82862993 The *intention* is that Jenkins-produced `oss-xamarin.android*.zip` artifacts be usable on Windows, so that side-by-side testing can be performed without replacing the system installation. Usage is in [UsingJenkinsBuildArtifacts.md](Documentation/UsingJenkinsBuildArtifacts). This isn't *entirely* the case. It was *apparently* the case at the time of commit 87ca2737, but things have bitrotten since. For example, following the 87ca2737 instructions would currently result in an XA0020 `Could not find monodroid` error, because `class-parse.exe` wasn't where Windows expects it to be (it was in `lib/mandroid`, not `lib/xbuild/Xamarin/Android`). This needs to be fixed. Additionally, PR #253 mentions that, for filesystem organization, it would be useful if the macOS/Linux directory structure -- `$prefix/bin`, `$prefix/lib/mandroid`, `$prefix/lib/xbuild/Xamarin/Android`, `$prefix/lib/xbuild-frameworks` -- more closely resembled the Windows directory structure of `$MSBuild` and `$ReferenceAssemblies` (as seen in `.vsix` files). This would turn macOS/Linux into using `$xa_prefix/xbuild` and `$xa_prefix/xbuild-frameworks` directories. `$prefix/bin` would only contain `xabuild`. What is currently in `$prefix/lib/mandroid` would be merged with `$xa_prefix/xbuild/Xamarin/Android`. `$xa_prefix` would `$prefix/lib/xamarin.android`. This would turn the current macOS structure: $prefix/bin/xabuild $prefix/bin/generator $prefix/bin/cross-arm $prefix/lib/mandroid/generator.exe $prefix/lib/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xbuild/Xamarin/Android/Xamarin.Android.Common.targets Into: $prefix/bin/xabuild $prefix/lib/xamarin.android/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xamarin.android/xbuild/Xamarin/Android/generator.exe $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Xamarin.Android.Common.targets $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Darwin/cross-arm Other notes: * The `bundle-*.zip` filename has been has been changed to include a *hash* of the contents of various files, in particular `build-tools\mono-runtimes.*`. This was instigated via a conversation with @kumpera about making the filename more "idiot-proof": `mono-runtimes.props` contains *compiler flags*, and if any of those change, then *logically* the bundle should differ as well, as the mono runtimes may differ in significant ways. In theory, the `-vXX` version should be used to track this, but this is a manual change, easy to overlook. The new `-hHASH` part of the filename should be more automagic. The new `<HashFileContents/>` task in `xa-prep-tasks.dll` is responsible for creating the hash value. * `Configuration.Java.Interop.Override.props` was moved into `build-tools/scripts`, because that would cleanup the root directory a bit. * OS-specific binaries are now placed into `$prefix/lib/xamarin.android/xbuild/Xamarin/Android/$(HostOS)`. On a theoretical plus side, this means that the same build directory can contain OS-specific binaries for multiple operating systems. (I don't know if anyone shares a build tree between e.g. macOS and Linux, but if anyone *does*...) Unfortunately this requires a workaround for an `xbuild` bug: `%(_LlvmRuntime.InstallPath)` and `%(_MonoCrossRuntime.InstallPath)` *cannot* use MSBuild properties, e.g. `<InstallPath>$(HostOS)/</InstallPath>` doesn't work as desired; it's instead treated literally. Special-case `%(InstallPath)` until we fully migrate to MSBuild. * `$(MonoAndroidToolsDirectory)` should be considered *dead*, along with `$(MonoAndroidBinDirectory)`, as these should now *always* be the same directory as where `Xamarin.Android.Build.Tasks.dll` is located, or a `$(HostOS)` sub-directory. * `Xamarin.ProjectTools.csproj` needed to be updated to ensure that the build order was correct. * Remove all `[Obsolete]` and unreferenced members from `Xamarin.Android.Build.Utilities.dll`. There's too much in there, and it makes my head hurt trying to understand the interrelationships between it all. If it's not used, it's gone. * The changes to `src/monodroid/jni/Android.mk` are...weird. The removal of `-I$(topdir)/libmonodroid/zip`/etc. is to reduce use of non-existent paths, as `$(topdir)` isn't defined, so that's *actually* `-I/libmonodroid/zip`, which is nonsensical. So far, so good. What's *odd* is the addition of `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`: This is needed so that `external/mono/support/zlib-helper.c` exports `CreateZStream` and related symbols, otherwise we get a unit test failure in `GzipStreamTest.Compression` due to an `EntryPointNotFoundException`, because `CreateZStream` isn't exported/public. What's odd here is that I don't understand what caused this behavior to change. Previous builds exported `CreateZStream`, otherwise the tests would fail, and I don't understand how any of the other changes in this PR would be at fault, though that's certainly the most plausible explanation. Regardless, `-Ijni` *first* (due to adding `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`) is the desired behavior, so that `jni/config.h` is included, thus ensuring that `MONO_API` has the required definition when building `zlib-helper.c`.
2017-07-28 16:36:32 +03:00
ln -s "$(prefix)/lib/xamarin.android/xbuild/Xamarin/Android/" "$(prefix)/lib/mono/xbuild/Xamarin/Android"
[linux] Add a Novell symlink to Debian packages (#1638) Before there was Xamarin.Android, it was "Mono for Android", which was a Novell product. As such, the `<Import/>` used by applications differed in a significant way: <Import Project="$(MSBuildExtensionsPath)\Novell\Novell.MonoDroid.CSharp.targets" /> Create a `$(prefix)/lib/mono/xbuild/Novell` symlink so that these legacy projects can still find `Novell.MonoDroid.CSharp.targets`.
2018-05-03 23:09:33 +03:00
ln -s "$(prefix)/lib/xamarin.android/xbuild/Novell/" "$(prefix)/lib/mono/xbuild/Novell"
[structure] Rework installation directory structure (#704) Context: https://github.com/xamarin/xamarin-android/pull/253#discussion_r82862993 The *intention* is that Jenkins-produced `oss-xamarin.android*.zip` artifacts be usable on Windows, so that side-by-side testing can be performed without replacing the system installation. Usage is in [UsingJenkinsBuildArtifacts.md](Documentation/UsingJenkinsBuildArtifacts). This isn't *entirely* the case. It was *apparently* the case at the time of commit 87ca2737, but things have bitrotten since. For example, following the 87ca2737 instructions would currently result in an XA0020 `Could not find monodroid` error, because `class-parse.exe` wasn't where Windows expects it to be (it was in `lib/mandroid`, not `lib/xbuild/Xamarin/Android`). This needs to be fixed. Additionally, PR #253 mentions that, for filesystem organization, it would be useful if the macOS/Linux directory structure -- `$prefix/bin`, `$prefix/lib/mandroid`, `$prefix/lib/xbuild/Xamarin/Android`, `$prefix/lib/xbuild-frameworks` -- more closely resembled the Windows directory structure of `$MSBuild` and `$ReferenceAssemblies` (as seen in `.vsix` files). This would turn macOS/Linux into using `$xa_prefix/xbuild` and `$xa_prefix/xbuild-frameworks` directories. `$prefix/bin` would only contain `xabuild`. What is currently in `$prefix/lib/mandroid` would be merged with `$xa_prefix/xbuild/Xamarin/Android`. `$xa_prefix` would `$prefix/lib/xamarin.android`. This would turn the current macOS structure: $prefix/bin/xabuild $prefix/bin/generator $prefix/bin/cross-arm $prefix/lib/mandroid/generator.exe $prefix/lib/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xbuild/Xamarin/Android/Xamarin.Android.Common.targets Into: $prefix/bin/xabuild $prefix/lib/xamarin.android/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll $prefix/lib/xamarin.android/xbuild/Xamarin/Android/generator.exe $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Xamarin.Android.Common.targets $prefix/lib/xamarin.android/xbuild/Xamarin/Android/Darwin/cross-arm Other notes: * The `bundle-*.zip` filename has been has been changed to include a *hash* of the contents of various files, in particular `build-tools\mono-runtimes.*`. This was instigated via a conversation with @kumpera about making the filename more "idiot-proof": `mono-runtimes.props` contains *compiler flags*, and if any of those change, then *logically* the bundle should differ as well, as the mono runtimes may differ in significant ways. In theory, the `-vXX` version should be used to track this, but this is a manual change, easy to overlook. The new `-hHASH` part of the filename should be more automagic. The new `<HashFileContents/>` task in `xa-prep-tasks.dll` is responsible for creating the hash value. * `Configuration.Java.Interop.Override.props` was moved into `build-tools/scripts`, because that would cleanup the root directory a bit. * OS-specific binaries are now placed into `$prefix/lib/xamarin.android/xbuild/Xamarin/Android/$(HostOS)`. On a theoretical plus side, this means that the same build directory can contain OS-specific binaries for multiple operating systems. (I don't know if anyone shares a build tree between e.g. macOS and Linux, but if anyone *does*...) Unfortunately this requires a workaround for an `xbuild` bug: `%(_LlvmRuntime.InstallPath)` and `%(_MonoCrossRuntime.InstallPath)` *cannot* use MSBuild properties, e.g. `<InstallPath>$(HostOS)/</InstallPath>` doesn't work as desired; it's instead treated literally. Special-case `%(InstallPath)` until we fully migrate to MSBuild. * `$(MonoAndroidToolsDirectory)` should be considered *dead*, along with `$(MonoAndroidBinDirectory)`, as these should now *always* be the same directory as where `Xamarin.Android.Build.Tasks.dll` is located, or a `$(HostOS)` sub-directory. * `Xamarin.ProjectTools.csproj` needed to be updated to ensure that the build order was correct. * Remove all `[Obsolete]` and unreferenced members from `Xamarin.Android.Build.Utilities.dll`. There's too much in there, and it makes my head hurt trying to understand the interrelationships between it all. If it's not used, it's gone. * The changes to `src/monodroid/jni/Android.mk` are...weird. The removal of `-I$(topdir)/libmonodroid/zip`/etc. is to reduce use of non-existent paths, as `$(topdir)` isn't defined, so that's *actually* `-I/libmonodroid/zip`, which is nonsensical. So far, so good. What's *odd* is the addition of `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`: This is needed so that `external/mono/support/zlib-helper.c` exports `CreateZStream` and related symbols, otherwise we get a unit test failure in `GzipStreamTest.Compression` due to an `EntryPointNotFoundException`, because `CreateZStream` isn't exported/public. What's odd here is that I don't understand what caused this behavior to change. Previous builds exported `CreateZStream`, otherwise the tests would fail, and I don't understand how any of the other changes in this PR would be at fault, though that's certainly the most plausible explanation. Regardless, `-Ijni` *first* (due to adding `$(LOCAL_PATH)` to `$(LOCAL_C_INCLUDES)`) is the desired behavior, so that `jni/config.h` is included, thus ensuring that `MONO_API` has the required definition when building `zlib-helper.c`.
2017-07-28 16:36:32 +03:00
ln -s "$(prefix)/lib/xamarin.android/xbuild-frameworks/MonoAndroid/" "$(prefix)/lib/mono/xbuild-frameworks/MonoAndroid"
[install] Don't install `xabuild` when `$(prefix)/bin/mono` exists (#761)
2017-08-19 01:45:10 +03:00
if [ ! -e "$(prefix)/bin/mono" ]; then \
[install] Fix install target in Makefile (#814) Dunno where if-then-fi without ';' worked, but it doesn't on Ubuntu 17.04.
2017-09-05 17:04:53 +03:00
cp tools/scripts/xabuild "$(prefix)/bin/xabuild"; \
[install] Don't install `xabuild` when `$(prefix)/bin/mono` exists (#761)
2017-08-19 01:45:10 +03:00
fi
[install] Add `make install` target (#684) The `make install` target can be used on Unixy platforms to install Xamarin.Android into a system-wide location. By default it installs into a prefix of `/usr/local`: make install The prefix may be changed by overriding the `$(prefix)` make variable: # Install system-wide make install prefix=/usr Installation is performed by copying the contents of `bin/$(CONFIGURATION)` into `$(prefix)/lib/xamarin.android`. Only `xabuild` is installed into `$(prefix)/bin` and (presumably) accessible within `$PATH`.
2017-07-03 20:09:22 +03:00
uninstall::
rm -rf "$(prefix)/lib/xamarin.android/" "$(prefix)/bin/xabuild"
[xabuild] Ensure directory removal with rm `-rf`. (#1407) This is the common build failure, what we see after `make prepare` and `make`: rm: cannot remove '/sources/monodroid/monodroid/external/xamarin-android/tools/scripts/../../bin/Debug/lib/xamarin.android/xbuild/Xamarin': Is a directory This does not show up once you run `make` again, so it's a cosmetic problem. But this should not happen. First-time contributors would get stuck at it and may leave after seeing this. This should be fixed.
2018-03-13 16:44:48 +03:00
rm -rf "$(prefix)/lib/mono/xbuild/Xamarin/Android"
rm -rf "$(prefix)/lib/mono/xbuild-frameworks/MonoAndroid"
[install] Add `make install` target (#684) The `make install` target can be used on Unixy platforms to install Xamarin.Android into a system-wide location. By default it installs into a prefix of `/usr/local`: make install The prefix may be changed by overriding the `$(prefix)` make variable: # Install system-wide make install prefix=/usr Installation is performed by copying the contents of `bin/$(CONFIGURATION)` into `$(prefix)/lib/xamarin.android`. Only `xabuild` is installed into `$(prefix)/bin` and (presumably) accessible within `$PATH`.
2017-07-03 20:09:22 +03:00
[build] ThirdPartyNotices.txt support (#464) `ThirdPartyNotices.txt` is a Microsoft convention to inform "the audience" that third party software is contained within a given "distribution mechanism", where said "distribution mechanism" could be a git repo or installable binary build artifacts. When applied to a source repository, `ThirdPartyNotices.txt` only contains license information for source code contained *within the current repository*, and does ***not*** include source code that may be pulled in via external sources, such as git submodules or NuGet packages. When applied to installable binary build artifacts, `ThirdPartyNotices.txt` should include everything that is part of the binary distribution. In our case, that means anything that winds up in `bin/$(Configuration)`, but *not* build-system support or tests, which are in `bin/Build$(Configuration)` and `bin/Test$(Configuration)`. This results in two separate files of interest: 1. `ThirdPartyNotices.txt`: Contains license information for 3rd party source code which has been *copied* into this repo. 2. `bin/$(Configuration)/lib/xamarin.android/ThirdPartyNotices.txt`: License information for a binary redistribution based on the build tree. This would include license information for both source code that has been copied into this repo, and source code from git submodules. Generation of these two separate files is controlled by two separate MSBuild properties: * `$(TpnIncludeBuildDependencies)`: True when build-time dependencies should be included in the resulting file. * `$(TpnIncludeExternalDependencies)`: True when *external* dependency information should be included in the resulting file. To satisfy these separate needs: 1. Add a bunch of `external/*.tpnitems` and `src/*.tpnnitems` files, which declares `@(ThirdPartyNotice)` values for various sources. Some of these values are Conditional on MSBuild properties, allowing them to be included or excluded based on the build configuration. The primary reason for this is for later "auditing" and reasoning, e.g. if we *remove* an external dependency, it'll be (presumably) easier to remove the corresponding `@(ThirdPartyNotice)` entry. It also provides a pattern for adding additional dependencies. 2. Create a new `<CreateThirdPartyNotices/>` task in `xa-prep-tasks.dll`. This task processes the `@(ThirdPartyNotice)` item group to produce `ThirdPartyNotices.txt`-formatted content. 3. Add a new `build-tools/ThirdPartyNotices/ThirdPartyNotices.csproj` project. This project includes the `external/*.tpnitems` and `src/*.tpnitems` entries and invokes the `<CreateThirdPartyNotices/>` task, creating the file `$(ThirdPartyNoticeFile)`. 4. Update `Makefile` to build `ThirdPartyNotices.csproj` at well defined build points. `make prepare` will update the top-level `ThirdPartyNotices.txt`, and `$(CONFIGURATION)`-specific `bin/$(Configuration)/lib/xamarin.android/ThirdPartyNotices.txt` files for inclusion elsewhere. Additionally, the packaging infrastructure has been updated to ensure that `ThirdPartyNotices.txt` is included: the `Xamarin.Android.Sdk*.vsix` file will contain a top-level `ThirdPartyNotices.txt` file, and the `xamarin.android-oss*.zip` file will contain one as well.
2018-03-02 17:54:35 +03:00
topdir := $(shell pwd)
[build] Add support for commercial builds (#2845) Fixes: https://github.com/xamarin/monodroid/issues/800 Context: https://github.com/xamarin/monodroid/commit/bee5f69a0b41aaeb88fad86ce4d65960a959def9 When the Mono team works on a PR to bump mono (e.g. [PR 1536][0]), they need to deal with two different build systems: 1. The xamarin-android build system, and 2. The monodroid build system. Shocking as this may sound, the OSS xamarin-android build system gets more love from the Xamarin.Android team than the commercial monodroid build system. As such, the monodroid build system is "baroque", not well tested off of a limited number of machines, prone to peculiar breakages, and otherwise adds a level of complexity and annoyance that everyone would rather do without. Unfortunately, *because* the monodroid repo is private, we can't have the xamarin-android repo contain a "proper" git submodule to it, as that would break all external contributors. Thus, historically, the monodroid repo instead had a git submodule reference to the xamarin-android repo, which brings a *different* set of problems, e.g. the commercial builds are *always* older than the OSS builds, and accidental breakage is caught much later than we'd like. The solution? For commercial builds, instead of "starting with" the monodroid repo, which submodules xamarin-android and (not-always- correctly) builds xamarin-android, we instead have the xamarin-android repo "reference" the monodroid repo, and have "hooks" in the xamarin-android build system so that monodroid can be built *from xamarin-android*. This will simplify life for the Mono team, allow for (hopefully) easier and more frequent commercial builds, simplify integration testing of commercial builds, and hopefully result in fewer apologies from @jonpryor about monodroid builds. Now, about that "reference": if we can't use git submodules to link the monodroid repo to the xamarin-android repo, how do we maintain that reference? ~~ .external ~~ There is a (seldom-used) convention among some Xamarin projects: the [`.external`][1] file. This is a plain-text file which allows specifying external dependencies. Blank lines and lines beginning with `#` are ignored, as well as lines which don't match a valid format. The external dependency reference format is: $(RepoOwner)/$(RepoName):$(BranchName)@$(CommitHash) We will use the `.external` file in order to link a specific monodroid commit to the xamarin-android build. This file *may* be used in the future for mono, as part of the [Mono Archive][2] work. The new `make prepare-external-git-dependencies` rule will clone all git submodule references listed in the `.external` file. This rule indirectly uses the new `external-git-dependencies.targets` MSBuild file, which exposes two new targets: * `ParseExternalFile`: Parses `.external` and creates the `@(Externals)` item group, which contains item metadata: * `%(Owner)`: The GitHub owner/organization * `%(Name)`: The git repo name * `%(Branch)`: The git repo branch * `%(Identity)`: The git commit in the repo * `CheckoutExternalGitDependency`: Clones or updates the git repos specified in `@(Externals)` into `$(ExternalSourceDependencyDirectory)`, which defaults to the top-level `external` directory. The new `<ParseExternalGitDependencies/>` task which is used by the `ParseExternalFile` target, while the new `<CheckoutExternalGitDependency/>` task is used by the `CheckoutExternalGitDependency` target. `external-git-dependencies.target` is `<Import/>`ed by `xa-prep-tasks.csproj` to allow us to easily checkout, download, and process the `.external` file. ~~ Build system hooks ~~ The `make jenkins` target has been turned into a [double-colon rule][3], which allows the `make jenkins` target to be specified multiple times, in multiple files, separately. There is also now an optional [**make** `include`][4] to `external/monodroid/xa-integration.mk`. `monodroid/xa-integration.mk` overrides the `make jenkins` target so that it can integrate it's build into the xamarin-android build. ~~ Installers ~~ `OpenTK.dll` and it's related files have been removed from the installers, in an effort to start reducing the interdependencies between the monodroid & xamarin-android repos. Finally, add a way to override a portion of our installer filenames, while still using version information from xamarin-android, by setting `$(UseCommercialInstallerName)`=True. [0]: https://github.com/xamarin/xamarin-android/pull/1836 [1]: https://github.com/xamarin/external [2]: https://github.com/xamarin/xamarin-android/projects/10 [3]: https://www.gnu.org/software/make/manual/html_node/Double_002dColon.html [4]: https://www.gnu.org/software/make/manual/html_node/Include.html
2019-04-10 21:59:39 +03:00
# Used by External XA Build
EXTERNAL_XA_PATH=$(topdir)
EXTERNAL_GIT_PATH=$(topdir)/external
-include $(EXTERNAL_GIT_PATH)/monodroid/xa-integration.mk
[build] Pass variables to monodroid and improve toolchain caching (#3325) Context: xamarin/monodroid#1010 Partially reworks the inverted build so that variables imported from 'rules.mk' can be accessed externally. Improves commercial build support for new Android API levels. Improves provisioning performance when building and testing on the Azure Pipeline macOS hosted agent pools. We'll now use the Android SDK components which are pre-installed on those machines, as most of what we need is already present and this location should be less prone to being cleaned up.
2019-07-10 02:00:50 +03:00
include build-tools/scripts/BuildEverything.mk
# Must be after BuildEverything.mk - it uses variables defined there
include build-tools/scripts/Packaging.mk
[tests] migrate test targets to MSBuild (#949) The end goal here is to enable Windows users to easily run the various types of tests. Commands have been migrated to MSBuild, keeping `make` commands the same for macOS and linux. ~~ Usage ~~ Windows: ``` msbuild Xamarin.Android.sln /t:RunAllTests msbuild Xamarin.Android.sln /t:RunNUnitTests msbuild Xamarin.Android.sln /t:RunJavaInteropTests msbuild Xamarin.Android.sln /t:RunApkTests ``` macOS/linux should remain unchanged: ``` make run-all-tests make run-nunit-tests make run-ji-tests make run-apk-tests ``` ~~ Changes ~~ - Added a new `build-tools/scripts/RunTests.targets` - `Before.Xamarin.Android.sln.targets` includes these test targets - A new `<SetEnvironmentVariable/>` task is needed, added to `xa-prep-tasks` - Ported any environment variables set in Makefile to MSBuild - `Mono.Android-Tests.csproj` needs to remove the `@(ProjectReference)` within `xa-prep-tasks`: - Otherwise `xa-prep-tasks.dll` can become locked on Windows - A nested `xabuild.exe` call will attempt to overwrite it - Update `.gitignore` for `*.rawproto` and the generated `Xamarin.Android.Common.props`
2017-11-08 23:38:27 +03:00
run-all-tests:
[build] Use a system mono/2018-06 debug build (#2440) Context: https://github.com/mono/mono/pull/11706 Context: https://github.com/xamarin/xamarin-android/pull/2357 Context: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android/1239/console Sometimes a Jenkins build will timeout when executing the xamarin-android unit tests. The timeout *appears* to occur when `mono` is used to execute a process, the process "finishes," but the process never actually *exits*, thus "hanging". For example, [consider this build log fragment][0] (with timestamps): 23:37:48 Build succeeded. 23:37:48 0 Warning(s) 23:37:48 0 Error(s) 23:37:48 23:37:48 Time Elapsed 01:23:14.60 02:37:48 Build timed out (after 180 minutes). Marking the build as aborted. The "Build succeeded" message comes from an `msbuild` invocation. Once `msbuild` finished, we would expect it's process to exit. *It didn't*. Instead, the underlying `mono` process is *not* exiting, presumably deadlocked, until the overall job times out and is killed. Attempt to help identify the hang by using a "debug" mono/2018-08 system mono, built from [Mono PR #11706][1], which prints out additional messages around child process use: diff --git a/mono/metadata/w32process-unix.c b/mono/metadata/w32process-unix.c index 2db5dea7a705..fbd4731cacb2 100644 --- a/mono/metadata/w32process-unix.c +++ b/mono/metadata/w32process-unix.c @@ -2096,7 +2095,7 @@ process_create (const gunichar2 *appname, const gunichar2 *cmdline, if (argv) g_strfreev (argv); - mono_trace (G_LOG_LEVEL_DEBUG, MONO_TRACE_IO_LAYER_PROCESS, "%s: returning handle %p for pid %d", __func__, handle, pid); + g_printerr ("XXX-MONO: %s: returning handle %p for pid %d from %d\n", __func__, handle, pid, getpid()); This will hopefully allow @lewurm to continue investigations. [0]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android/1237/console [1]: https://github.com/mono/mono/pull/11706
2018-11-30 19:40:30 +03:00
@echo "PRINTING MONO VERSION"
mono --version
[tests] Consistently run `run-api-compatibility-tests` (#3040) Context: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android/491 Context: https://github.com/xamarin/xamarin-android/pull/3032#issuecomment-487174225 A "funny" thing has happened with the mono/2019-02 bump: it has apparently [broken API compatibility][0], which was not detected or reported in the last [mono/2019-02 PR build][1]. What went wrong is that the `make run-all-tests` step was failing, reporting an error, but because of how `make run-all-tests` was structured, when the `RunAllTests` MSBuild target failed, *no further tests were run*. Further tests such as the API compatibility tests! Fix the `make run-all-tests` target so that even if the `RunAllTests` MSBuild target fails, we *continue executing tests* so that the `make run-api-compatibility-tests` target is consistently executed. The exit value of `make run-all-tests` is the exit value of the *last failed test*. [0]: https://github.com/xamarin/xamarin-android/pull/3032#issuecomment-487174225 [1]: https://jenkins.mono-project.com/job/xamarin-android-pr-pipeline-release/684/
2019-04-29 23:24:40 +03:00
_r=0 ; \
$(call MSBUILD_BINLOG,run-all-tests,,Test) $(TEST_TARGETS) /t:RunAllTests || _r=$$? ; \
exit $$_r
[tests] Add Configuration info to test names Context: 0077d151 Context: d1d9820a A "funny" thing happened when commit e9daf5ea didn't build on Jenkins: I realized that not all tests were run in all configurations. From commit d1d9820a: > Why are some tests Debug-only and some aren't? The answer: time, primarily. Why run tests multiple times, when they can be potentially time-consuming? While tests can be slow, they're not always *that* slow -- except for `Xamarin.Android.Build.Tests` and the BCL tests -- and even there, program behavior can alter between Debug and Release configurations. See in particular commit 0077d151, in which the BCL tests are run only in the Debug configuration because tests *failed* when run in the Release configuration. The desire, then, is to run *all* tests in both Debug and Release configurations. Yes, it'll take longer! So what! (Within reason: `Xamarin.Android.Build.Tests` should only be run once!) However, this raises two problems: 1. Filename collisions 2. Jenkins unit test display Until now, all tests wrote files into a filename that didn't include the Configuration, e.g. `TestResult-Mono.Android_Tests.xml`. If we did run these tests twice, the second test invocation would overwrite the first test invocation. This isn't desirable. Then there's the display on Jenkins: if we did have e.g. `TestResult-Mono.Android_Tests-Debug.xml` and `TestResult-Mono.Android_Tests-Release.xml`, how will Jenkins display that information? I haven't tested, but I would assume that one of two things will occur, assuming reasonable Jenkins behavior: 1. Each test will be listed twice, e.g. ApplicationContextIsApp ApplicationContextIsApp 2. They'll be "merged" into a single entry. Neither of these behaviors is desirable: if Debug passes but Release fails, we need to be able to differentiate between them. Neither of these possible renderings allows us to tell which configuration fails. Solve both of these problems by introducing a new `<RenameTestCases/>` task. This task takes three values of importance: ```xml <RenameTestCases Configuration="CONFIGURATION" SourceFile="SOURCE" DestinationFolder="DESTINATION" /> ``` The `<RenameTestCases/>` task will read in `SOURCE`, and if `SOURCE` is an XML file which we determine is NUnit2-formatted XML (root element of `<test-case/>`), we will update every `//test-case/@name` value so that it ends with ` / CONFIGURATION`. The updated XML is then written to the `DESTINATION` directory, with a filename that contains `CONFIGURATION`, and `SOURCE` is deleted. Thus, if we have a Debug-configuration `TestResult-Mono.Android_Tests.xml` file with XML fragment: ```xml <test-case name="Mono.Android_Tests, Android.AppTests.ApplicationTest.ApplicationContextIsApp" ... /> ``` then `<RenameTestCases/>` will create the file `TestResult-Mono.Android_Tests-Debug.xml` file with XML fragment: ```xml <test-case name="Mono.Android_Tests, Android.AppTests.ApplicationTest.ApplicationContextIsApp / Debug" ... /> ``` This allows us to run tests in both Debug and Release configurations while not inadvertently overwriting the `TestResults*.xml` files that Jenkins reads, and ensuring that the Jenkins test result output is rendered in a meaningfully useful fashion. Aside: when updating `//test-case/@name`, the resulting value *cannot* end in `)`. If it does, then the `(root)` package name issue fixed in commit 23b2642e reappears for the `generator` unit tests. **Completely random aside about the state of `xbuild`**: A development version of `<RenameTestCases/>` was "saner", using `ITaskItem[]` and not string: ```csharp partial class RenameTestCases { public ITaskItem[] SourceFiles {get; set;} // vs. // public string SourceFile {get; set;} } ``` The problem is that the above, while entirely reasonable, did not work at all correctly with `xbuild`: ```xml <RenameTestCases SourceFiles="%(TestApk.ResultsPath)" /> ``` Under `xbuild`, MSBuild properties would not be expanded, e.g. `RenameTestCases.SourceFiles` would get a "bizarro" value of e.g. `$(OutputPath)Mono.Android_Tests-Signed.apk`, which is *useless* and would result in `FileNotFoundException`s. MSBuild proper, of course, worked as desired. TODO: Once this is merged, update the Jenkins Configuration page so that instead of: make run-all-tests V=1 || exit 1 it instead runs both Debug and Release configuration tests: make run-all-tests SKIP_NUNIT_TESTS=1 V=1 || exit 1 make run-all-tests CONFIGURATION=Release V=1 || exit 1 Note that `$(SKIP_NUNIT_TESTS)` is specified so that we only run the lengthy (1+hr!) `Xamarin.Android.Build.Tests` tests in the Release configuration, not the Debug + Release configurations.
2017-10-11 00:59:42 +03:00
[android-toolchain] Permit zero-configuration builds. This might be a suspect idea, but lets see if we can make this work. [The Joel Test: 12 Steps to Better Code][0] outlines 12 steps to better code. The first two steps are: 1. Do you use source control? 2. Can you make a build in one step? github is being used for source control, so (1) is handled, but how simple can we make (2)? How easy can we make it to build Xamarin.Android upon a fresh checkout? The ideal to strive for is simple: Load Xamarin.Android.sln into your IDE and Build the project. I *know* we're not going to be able to do this, if only because we're going to be using git submodules, which will require a separate `git submodule init` invocation [1]. Knowing we can't reach that level of simplicitly doesn't mean we shouldn't *try* to reach it for all other parts of the build system. Which brings us to the Android NDK and SDK. The Android NDK will be required in order to build native code, such as libmonodroid.so, while the Android SDK will be required in order to compile Java Callable Wrappers (née Android Callable Wrappers [2]) and eventual samples and unit tests. There are three ways we can deal with the Android NDK and SDK: 1. Complicate the "build" process by requiring that developers go to the Android SDK Download Page [3], download and install "somewhere" the required bits, and then configure the Xamarin.Android build to use these bits. 2. Complicate the "build" process by requiring that developers run the Xamarin Unified Installer [4], let it install everything required, then configure the Xamarin.Android build to use those bits. 3. Painstakingly determine which files are actually required, then automatically download and extract those files into a "well-known" location known by the Xamarin.Android build process. (1) and (2) can be infuriating. Let's give (3) a try. :-) Add a Xamarin.Android.Tools.BootstrapTasks project which contains MSBuild tasks to facilitate downloading the Android SDK and NDK files. Add an android-toolchain project which uses Xamarin.Android.Tools.BootstrapTasks to download a painstakingly determined set of files and install them "somewhere". Unfortunately [5] the "somewhere" to download and install these files needs to be in a known absolute path, so I've arbitrary decided to download the files into $(HOME)\android-archives and install them into $(HOME)\android-toolchain. On windows, this is %HOMEDRIVE%%HOMEPATH%\android-archives and %HOMEDRIVE%%HOMEPATH%\android-toolchain. These locations may be modified by creating a Configuration.Override.props file; see README.md for details. TL;DR: This setup is able to magically download the Android NDK and SDK files and install them for later use in a reasonably overridable location, all within MSBuild. [0]: http://www.joelonsoftware.com/articles/fog0000000043.html [1]: Though maybe there's some MSBuild-fu we can use to address that. [2]: https://developer.xamarin.com/guides/android/advanced_topics/java_integration_overview/android_callable_wrappers/ [3]: http://developer.android.com/sdk/index.html [4]: https://www.xamarin.com/download [5]: Because I couldn't find a reliable way to use $(SolutionDir) when only building a project, and relative paths would require an in-tree installation location, which might not work.
2016-04-19 03:33:04 +03:00
clean:
[build] Normal & Diagnostic Log Verbosity, at once! Fixes: https://github.com/xamarin/xamarin-android/issues/1792 We have a bit of a conundrum: when things break, the diagnostic build logs are *essential* to tracking down *why* something broke. However, the sheer *verbosity* of those diagnostic logs makes it difficult to determine *what* broke; it's very easy to get lost. Additionally, the gigantic log files make the **Console Output** Jenkins link nearly useless, as most web browsers will cease up when attempting to download ~1GB log files. Split the difference: introduce a new `$(call MSBUILD_BINLOG)` syntax which creates an [MSBuild binary log][0] file, which is like diagnostic output *but even moreso*, while *also* overriding the console logging to `/v:normal`. [0]: https://github.com/Microsoft/msbuild/wiki/Binary-Log This should allow the **Console Output** link to be a bit more reasoanble, and if something goes wrong we can download the separate `*.binlog` files for further analysis. MSBuild binary log files can be read with MSBuild itself: msbuild /v:diag bin/BuildDebug/msbuild-*.binlog Note: above command will actually fail; you can only print one log file at a time with that syntax. The created log files follow the pattern `msbuild-EPOCH-NAME`, where EPOCH is the `time_t`/Unix epoch value *at the time the command was executed*. This should allow filenames to be non-colliding, and allow plausibly sane sorting behavior. (Maybe.)
2018-06-12 04:28:38 +03:00
$(call MSBUILD_BINLOG,clean) /t:Clean Xamarin.Android.sln
[Xamarin.Android-Tests] Split out "full" apps (#146) We're trying to get [`make jenkins`][0] working on Jenkins, and [it's failing][1], as one might expect when a particular repo and associated build system has never been run on Jenkins before: Android.App/ApplicationTest.cs(9,7): error CS0246: The type or namespace name `NUnit' could not be found. Are you missing an assembly reference? This error occurs while building `src/Mono.Android/Test/Mono.Android-Tests.csproj`, and happens because the `Xamarin.Android.NUnitLite.dll` assembly isn't referenced... because it isn't *found*: Microsoft.Common.targets: warning : Reference 'Xamarin.Android.NUnitLite' not resolved This returns us to a long-standing issue which I thought was mentioned in a commit somewhere, but I can't find at present: MSBuild doesn't support updating the assembly resolution directories *while MSBuild is running*. For example, we build `Xamarin.Android.NUniteLite.dll` into `bin/$(Configuration/lib/xbuild-frameworks/MonoAndroid/v1.0`, but "normal Xamarin.Android referencing projects" don't use a `@(ProjectReference)` to `Xamarin.Android.NUnitLite.csproj`, but instead a `@(Reference)` to the assembly: <!-- src/Mono.Android/Test/Mono.Android-Tests.csproj --> <Reference Include="Xamarin.Android.NUnitLite" /> The *requirement* for a "proper" Xamarin.Android SDK install is that `Xamarin.Android.NUnitLite.dll` can be found through a normal `@(Reference)`. In order to satisfy this requirement, we need to tell MSBuild where to find it, which can be with `xbuild` via the `$MSBuildExtensionsPath` and `$XBUILD_FRAMEWORK_FOLDERS_PATH` *environment variables*. *Environment variables*. MSBuild doesn't provide a way to update environment variables, short of writing a new Task which calls `Environment.SetEnvironmentVariable()`, and while [this works][2], it doesn't *actually* work [^3]. The short of all this is that it isn't possible, *within a single `xbuild` invocation*, to both build the Xamarin.Android SDK "environment" *and use that environment* as intended for "normal" apps. The fix, as is often the case, is to bend with the wind. Instead of requiring the impossible, move `src/Mono.Android/Test/Mono.Android-Tests.csproj` into a *new* `Xamarin.Android-Tests.sln` solution, *out of* `Xamarin.Android.sln`. This allows building `Xamarin.Android.sln` without error in a pristine environment -- that is, one that doesn't already have a system-wide Xamarin.Android install -- and separately building the tests by using `tools/scripts/xabuild`, which *can* export environment variables to manipulate `xbuild` behavior so that things Just Work™. Building `Mono.Android-Tests.csproj` and similar projects (`HelloWorld.csproj`!) can be done by using the new `make all-tests` make target. [0]: https://github.com/xamarin/xamarin-android/commit/a16673d3eb2c4945c3a74f5f8154603d7658fc9a [1]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android/20/console [2]: https://github.com/xamarin/xamarin-android/pull/147 [^3]: [PR #147][2] isn't viable because of [xbuild's `AsssemblyResolver`][4]. There's no way to clear/invalidate `target_framework_cache`. The idea of PR #147 was to "hack things up" so that `Xamarin.Android.NuniteLite.dll` would be properly resolved during the build of `Mono.Android-Tests.csproj` *when building everything*. The problem is *this can't work*, because `xbuild` has a "target framework cache," with no way to invalidate it, and the cache is populated the first time the target framework is used. Due to build ordering issues, this first use is *before* `Xamarin.Android.NunitLite.dll` was built, and thus it doesn't exist in the cache. The result: Task "ResolveAssemblyReference" .... TargetFrameworkDirectories: /Volumes/Seagate4TB/work/xamarin-android/bin/Debug/lib/xbuild-frameworks/MonoAndroid/v7.0 /Volumes/Seagate4TB/work/xamarin-android/bin/Debug/lib/xbuild-frameworks/MonoAndroid/v1.0 /Volumes/Seagate4TB/work/xamarin-android/bin/Debug/lib/xbuild-frameworks/MonoAndroid/v1.0/Facades/ ... Primary Reference System.Xml Reference System.Xml resolved to /Volumes/Seagate4TB/work/xamarin-android/bin/Debug/lib/xbuild-frameworks/MonoAndroid/v1.0/System.Xml.dll. CopyLocal = False ... Primary Reference Xamarin.Android.NUnitLite Microsoft.Common.targets: warning : Reference 'Xamarin.Android.NUnitLite' not resolved For searchpath {TargetFrameworkDirectory} Considered target framework dir /Volumes/Seagate4TB/work/xamarin-android/bin/Debug/lib/xbuild-frameworks/MonoAndroid/v7.0, assembly named 'Xamarin.Android.NUnitLite' not found. Considered target framework dir /Volumes/Seagate4TB/work/xamarin-android/bin/Debug/lib/xbuild-frameworks/MonoAndroid/v1.0, assembly named 'Xamarin.Android.NUnitLite' not found. Considered target framework dir /Volumes/Seagate4TB/work/xamarin-android/bin/Debug/lib/xbuild-frameworks/MonoAndroid/v1.0/Facades/, assembly named 'Xamarin.Android.NUnitLite' not found. ... Consequently, the `mcs` invocation is missing a `/reference:path/to/Xamarin.Android.NUniteLite.dll`, and compilation fails: Android.App/ApplicationTest.cs(9,7): error CS0246: The type or namespace name `NUnit' could not be found. Are you missing an assembly reference? ...plus 29 others... [4]: https://github.com/mono/mono/blob/dd8aadf/mcs/class/Microsoft.Build.Tasks/Microsoft.Build.Tasks/AssemblyResolver.cs#L131
2016-08-08 23:46:13 +03:00
tools/scripts/xabuild $(MSBUILD_FLAGS) /t:Clean Xamarin.Android-Tests.sln
[Xamarin.Android.Build.Tasks] First Pass at adding Unit tests for the MSBuild Tasks (#26) This commit adds basic support for the MSBuild Unit Tests. The goal of these tests is to ensure that the build of an android applicaition works consistently on all the supported platforms. The basic layout of a Unit test is as follows [Test] public void BuildReleaseApplication () { var proj = new XamarinAndroidApplicationProject () { IsRelease = true, }; using (var b = CreateApkBuilder (Path.Combine ("temp", TestContext.CurrentContext.Test.Name))) { Assert.IsTrue (b.Build (proj), "Build should have succeeded."); } } It is a standard Unit test. First we create a XamarinAndroidApplicatonProject and set its properties. You can use the proj to add new source files, references, assets, resources and nuget packages. By default you will get a standard "HelloWorld" android application. Once you have the project you will need to create either a Apk or Dll builder via the helper methods CreateApkBuilder CreateDllBuilder CreateApkBuilder will create an apk and by default will execute the `SignAndroidPackage` build target. the CreateDllBuilder will produce a dll. The source files are created in a temp directory relative to the build output of the unit tests. By default this is src/Xamarin.Android.Build.Tasks/UnitTests/Xamarin.Android.Build.Tests/bin/$(Configuraiton)/temp Once you have a builder you can then call Build passing in the project. There are also methods available for Clean and Save. Running any of these will cause the Unit test to shell out to xbuild/msbuild and attempt to build the project. Test results are written to a build.log file. If a unit test passes then the test directory is removed. If a test fails the directory and the build.log will be left in place for investigation.
2016-05-12 21:54:03 +03:00
Add a thorough clean make target `make distclean` will not only run `make clean` but it will also run `git clean -xdff` for the main project as well as the submodules thus giving you a pristine tree. Note that it will remove *all* of the files not in the git index as well as all the build artifacts! Use with care!
2016-06-10 17:07:34 +03:00
distclean:
# It may fail if we're cleaning a half-built tree, no harm done if we ignore it
-$(MAKE) clean
git clean -xdff
git submodule foreach git clean -xdff
[tests] migrate test targets to MSBuild (#949) The end goal here is to enable Windows users to easily run the various types of tests. Commands have been migrated to MSBuild, keeping `make` commands the same for macOS and linux. ~~ Usage ~~ Windows: ``` msbuild Xamarin.Android.sln /t:RunAllTests msbuild Xamarin.Android.sln /t:RunNUnitTests msbuild Xamarin.Android.sln /t:RunJavaInteropTests msbuild Xamarin.Android.sln /t:RunApkTests ``` macOS/linux should remain unchanged: ``` make run-all-tests make run-nunit-tests make run-ji-tests make run-apk-tests ``` ~~ Changes ~~ - Added a new `build-tools/scripts/RunTests.targets` - `Before.Xamarin.Android.sln.targets` includes these test targets - A new `<SetEnvironmentVariable/>` task is needed, added to `xa-prep-tasks` - Ported any environment variables set in Makefile to MSBuild - `Mono.Android-Tests.csproj` needs to remove the `@(ProjectReference)` within `xa-prep-tasks`: - Otherwise `xa-prep-tasks.dll` can become locked on Windows - A nested `xabuild.exe` call will attempt to overwrite it - Update `.gitignore` for `*.rawproto` and the generated `Xamarin.Android.Common.props`
2017-11-08 23:38:27 +03:00
run-nunit-tests:
[build] SKIP_NUNIT_TESTS check is reversed (#950) When working with the `Makefile` I noticed this command doesn’t run the tests: ``` $ make run-nunit-tests make: Nothing to be done for `run-nunit-tests’. ``` And this command runs the tests: ``` $ make run-nunit-tests SKIP_NUNIT_TESTS=1 ``` I suspect the intention is that `ifneq` needs to be `ifeq`.
2017-10-19 22:49:09 +03:00
ifeq ($(SKIP_NUNIT_TESTS),)
[build] Upload unit test results (#2782) There was some unexpected and unforeseen fallout from commit 241d9416: `make package-build-status` is only run once. This doesn't *sound* problematic -- hence the oversight -- but the `msbuild*.binlog` files that the `xa-build-status*.zip` file contained prior to commit 241d9416 contained *unit test execution logs*, e.g. `msbuild-*-Target-RunApkTests.binlog` which contains on-device execution of the `.apk` unit tests. However, these files don't exist until *after* the unit tests are run. Meanwhile, `make package-build-status` is executed *before* unit tests are run, and thus the resulting `xa-build-status*.zip` file no longer contains unit test information! (This makes it rather difficult to actually review anything about unit test execution!) The fix? Update the `$(MSBUILD_BINLOG)` Make `define` so that it now takes an optional 3rd parameter: part of the directory name to place files into, which defaults to `Build`. (This allows overriding things so that unit-test related `.binlog` files now go into `bin/Test$(Configuration)`, not `bin/Build$(Configuration)`.) Additionally, *rename* the `make package-test-errors` rule to `make package-test-results`, have it produce an `xa-test-results*.zip` file, and include the `bin/Test$(Configuration)/msbuild*.binlog` files into `xa-test-results*.zip`. `make package-test-results` is executed after unit tests are executed, so this change allows us the uploaded `xa-test-results.zip` file to contain unit test execution results.
2019-03-02 04:46:00 +03:00
$(call MSBUILD_BINLOG,run-nunit-tests,,Test) $(TEST_TARGETS) /t:RunNUnitTests
[tests] Add Configuration info to test names Context: 0077d151 Context: d1d9820a A "funny" thing happened when commit e9daf5ea didn't build on Jenkins: I realized that not all tests were run in all configurations. From commit d1d9820a: > Why are some tests Debug-only and some aren't? The answer: time, primarily. Why run tests multiple times, when they can be potentially time-consuming? While tests can be slow, they're not always *that* slow -- except for `Xamarin.Android.Build.Tests` and the BCL tests -- and even there, program behavior can alter between Debug and Release configurations. See in particular commit 0077d151, in which the BCL tests are run only in the Debug configuration because tests *failed* when run in the Release configuration. The desire, then, is to run *all* tests in both Debug and Release configurations. Yes, it'll take longer! So what! (Within reason: `Xamarin.Android.Build.Tests` should only be run once!) However, this raises two problems: 1. Filename collisions 2. Jenkins unit test display Until now, all tests wrote files into a filename that didn't include the Configuration, e.g. `TestResult-Mono.Android_Tests.xml`. If we did run these tests twice, the second test invocation would overwrite the first test invocation. This isn't desirable. Then there's the display on Jenkins: if we did have e.g. `TestResult-Mono.Android_Tests-Debug.xml` and `TestResult-Mono.Android_Tests-Release.xml`, how will Jenkins display that information? I haven't tested, but I would assume that one of two things will occur, assuming reasonable Jenkins behavior: 1. Each test will be listed twice, e.g. ApplicationContextIsApp ApplicationContextIsApp 2. They'll be "merged" into a single entry. Neither of these behaviors is desirable: if Debug passes but Release fails, we need to be able to differentiate between them. Neither of these possible renderings allows us to tell which configuration fails. Solve both of these problems by introducing a new `<RenameTestCases/>` task. This task takes three values of importance: ```xml <RenameTestCases Configuration="CONFIGURATION" SourceFile="SOURCE" DestinationFolder="DESTINATION" /> ``` The `<RenameTestCases/>` task will read in `SOURCE`, and if `SOURCE` is an XML file which we determine is NUnit2-formatted XML (root element of `<test-case/>`), we will update every `//test-case/@name` value so that it ends with ` / CONFIGURATION`. The updated XML is then written to the `DESTINATION` directory, with a filename that contains `CONFIGURATION`, and `SOURCE` is deleted. Thus, if we have a Debug-configuration `TestResult-Mono.Android_Tests.xml` file with XML fragment: ```xml <test-case name="Mono.Android_Tests, Android.AppTests.ApplicationTest.ApplicationContextIsApp" ... /> ``` then `<RenameTestCases/>` will create the file `TestResult-Mono.Android_Tests-Debug.xml` file with XML fragment: ```xml <test-case name="Mono.Android_Tests, Android.AppTests.ApplicationTest.ApplicationContextIsApp / Debug" ... /> ``` This allows us to run tests in both Debug and Release configurations while not inadvertently overwriting the `TestResults*.xml` files that Jenkins reads, and ensuring that the Jenkins test result output is rendered in a meaningfully useful fashion. Aside: when updating `//test-case/@name`, the resulting value *cannot* end in `)`. If it does, then the `(root)` package name issue fixed in commit 23b2642e reappears for the `generator` unit tests. **Completely random aside about the state of `xbuild`**: A development version of `<RenameTestCases/>` was "saner", using `ITaskItem[]` and not string: ```csharp partial class RenameTestCases { public ITaskItem[] SourceFiles {get; set;} // vs. // public string SourceFile {get; set;} } ``` The problem is that the above, while entirely reasonable, did not work at all correctly with `xbuild`: ```xml <RenameTestCases SourceFiles="%(TestApk.ResultsPath)" /> ``` Under `xbuild`, MSBuild properties would not be expanded, e.g. `RenameTestCases.SourceFiles` would get a "bizarro" value of e.g. `$(OutputPath)Mono.Android_Tests-Signed.apk`, which is *useless* and would result in `FileNotFoundException`s. MSBuild proper, of course, worked as desired. TODO: Once this is merged, update the Jenkins Configuration page so that instead of: make run-all-tests V=1 || exit 1 it instead runs both Debug and Release configuration tests: make run-all-tests SKIP_NUNIT_TESTS=1 V=1 || exit 1 make run-all-tests CONFIGURATION=Release V=1 || exit 1 Note that `$(SKIP_NUNIT_TESTS)` is specified so that we only run the lengthy (1+hr!) `Xamarin.Android.Build.Tests` tests in the Release configuration, not the Debug + Release configurations.
2017-10-11 00:59:42 +03:00
endif # $(SKIP_NUNIT_TESTS) == ''
[Mono.Android-Test] Import Mono.Android tests from monodroid/9c5b3712 (#32) Import monodroid/tests/runtime from monodroid/9c5b3712. Add a toplevel `make run-apk-tests` target to "full stack" tests, in which a .apk is created, installed, and executed on an attached Android device. `make run-apk-tests` requires that `adb` be in $PATH, and uses GNU make(1) features, and... Additionally, tools/scripts/xabuild *must* be used to execute the `SignAndroidPackage` target, to ensure that the local/"in tree" assemblies are used. There is no "within xbuild" facility to alter where target framework assemblies are resolved from, i.e no MSBuild properties currently control the resolution order, only environment variables, and MSBuild can't *set* environment variables... The $(ADB_TARGET) variable can be used to control on which target Android device the tests will be installed and executed on: # Install & run tests on *only* connected USB device $ make run-apk-tests ADB_TARGET=-d # Install & run tests on *only* connected emulator $ make run-apk-tests ADB_TARGET=-e # Install & run tests on specified device, listed via `adb devices`. $ make run-apk-tests ADB_TARGET="-s 036533824381cfcb" Sorry potential/future Windows developers. *Running* tests will require manual testing or running on OS X or Linux for now... Note: These tests DO NOT PASS. In fact, they *crash*: $ make run-apk-tests ... Target RunTests: Executing: "$HOME/android-toolchain/sdk/platform-tools/adb" shell am instrument -w Mono.Android_Tests/xamarin.android.runtimetests.TestInstrumentation INSTRUMENTATION_RESULT: shortMsg=Process crashed. INSTRUMENTATION_CODE: 0 $ adb logcat ... E mono : Unhandled Exception: E mono : System.ObjectDisposedException: Cannot access a disposed object. E mono : Object name: 'System.Net.Sockets.Socket'. E mono : at System.Net.Sockets.Socket.ThrowIfDisposedAndClosed () <0xa93923f0 + 0x00054> in <filename unknown>:0 E mono : at System.Net.Sockets.Socket.AcceptAsync (System.Net.Sockets.SocketAsyncEventArgs e) <0x9b8f9680 + 0x0001b> in <filename unknown>:0 E mono : at System.Net.EndPointListener.Accept (System.Net.Sockets.Socket socket, System.Net.Sockets.SocketAsyncEventArgs e) <0x9b8f95d0 + 0x0003f> in <filename unknown>:0 E mono : at System.Net.EndPointListener.ProcessAccept (System.Net.Sockets.SocketAsyncEventArgs args) <0x9b8e0340 + 0x0007f> in <filename unknown>:0 E mono : at System.Net.EndPointListener.OnAccept (System.Object sender, System.Net.Sockets.SocketAsyncEventArgs e) <0x9b8e0310 + 0x00017> in <filename unknown>:0 E mono : at System.Net.Sockets.SocketAsyncEventArgs.OnCompleted (System.Net.Sockets.SocketAsyncEventArgs e) <0x9b8e02c8 + 0x0003b> in <filename unknown>:0 E mono : at System.Net.Sockets.SocketAsyncEventArgs.Complete () <0x9b8e02a0 + 0x0001f> in <filename unknown>:0 E mono : at System.Net.Sockets.Socket.<AcceptAsyncCallback>m__0 (System.IAsyncResult ares) <0x9b8dfd40 + 0x002af> in <filename unknown>:0 E mono : at System.Net.Sockets.SocketAsyncResult+<Complete>c__AnonStorey0.<>m__0 (System.Object _) <0xa892f720 + 0x0002b> in <filename unknown>:0 E mono : at System.Threading.QueueUserWorkItemCallback.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem () <0xa938f6b8 + 0x0002f> in <filename unknown>:0 E mono : at System.Threading.ThreadPoolWorkQueue.Dispatch () <0xa938e358 + 0x001bb> in <filename unknown>:0 E mono : at System.Threading._ThreadPoolWaitCallback.PerformWaitCallback () <0xa938e1a8 + 0x00007> in <filename unknown>:0 Looks like a Socket and/or ThreadPool bug in mono.
2016-05-16 21:55:13 +03:00
[tests] Run the Java.Interop unit tests (#565) Bump to Java.Interop/master/8d7060d1 Bump to cecil/master/5f93431f We'd like to run the Java.Interop unit tests as part of the xamarin-android `make run-all-tests` target, for two rasons: 1. An extra layer of sanity checking, and 2. Some of the Java.Interop tests require an Android SDK. In particular, the `ParameterFixupTests.XmlDeclaration_FixedUpFromDocumentation()` test from `Xamarin.Android.Tools.Bytecode-Tests.dll` attempts to read Android documentation from `$ANDROID_SDK_PATH` to test parameter name fixups. However, Java.Interop doesn't install -- and thus can't readily require -- an Android SDK, meaning this test is never executed as part of the Java.Interop Jenkins process. That test *can* be readily executed in xamarin-android, as it *does* install an Android SDK, and thus its presence can be assumed. Add a new `make run-ji-tests` target which runs the Java.Interop unit tests, copying their test output into the same directory that the other `TestResult-*.xml` files are placed, so that Jenkins can pick them up for display. Within the `make run-ji-tests` target, export the `ANDROID_SDK_PATH` environment variable so that *all* the `Xamarin.Android.Tools.Bytecode-Tests.dll` tests can execute. Which introduces two problems: 1. The `Xamarin.Android.Tools.Bytecode-Tests.dll` tests require that `$ANDROID_SDK_PATH` contain documentation...which we're not currently installing. 2. The Java.Interop tests still fail, due to parameter name changes, as those tests were originally using API-18 docs. Result: the `ParameterFixupTest` tests fail. :-( Fix this conundrum by instaslling `docs-24_r01.zip` within `$(AndroidSdkDirectory)`, allowing the tests to execute as intended, and by updating to Java.Interop/master/8d7060d1, which updates the expected parameter name output.
2017-04-21 15:11:40 +03:00
run-ji-tests:
[build] Upload unit test results (#2782) There was some unexpected and unforeseen fallout from commit 241d9416: `make package-build-status` is only run once. This doesn't *sound* problematic -- hence the oversight -- but the `msbuild*.binlog` files that the `xa-build-status*.zip` file contained prior to commit 241d9416 contained *unit test execution logs*, e.g. `msbuild-*-Target-RunApkTests.binlog` which contains on-device execution of the `.apk` unit tests. However, these files don't exist until *after* the unit tests are run. Meanwhile, `make package-build-status` is executed *before* unit tests are run, and thus the resulting `xa-build-status*.zip` file no longer contains unit test information! (This makes it rather difficult to actually review anything about unit test execution!) The fix? Update the `$(MSBUILD_BINLOG)` Make `define` so that it now takes an optional 3rd parameter: part of the directory name to place files into, which defaults to `Build`. (This allows overriding things so that unit-test related `.binlog` files now go into `bin/Test$(Configuration)`, not `bin/Build$(Configuration)`.) Additionally, *rename* the `make package-test-errors` rule to `make package-test-results`, have it produce an `xa-test-results*.zip` file, and include the `bin/Test$(Configuration)/msbuild*.binlog` files into `xa-test-results*.zip`. `make package-test-results` is executed after unit tests are executed, so this change allows us the uploaded `xa-test-results.zip` file to contain unit test execution results.
2019-03-02 04:46:00 +03:00
$(call MSBUILD_BINLOG,run-ji-tests,,Test) $(TEST_TARGETS) /t:RunJavaInteropTests
[tests] Run runtime test in Release & Release+AOT (#748) - besides running in different configurations than default (Debug), we will also use the startup times measurements in the Jenkins plots - also fixes a typo in `RunApkTests.targets`, where supposedly `Condition` was used in place of `Configuration` - modifies logcat timing file names, so that it is based on test results filename instead of just package name. that way we can have multiple timing results for the same test with various configurations
2017-08-16 18:05:39 +03:00
[tests] Allow running only selected apk tests (#2558) Make it possible to run selected apk tests by passing `PACKAGES` variable to `make`. Example usage: make run-apk-tests PACKAGES="Xamarin.Forms_Performance_Integration;Xamarin.Android.Locale_Tests" Or msbuild equivalent: msbuild /t:RunApkTests tests/RunApkTests.targets /p:ApkTests='"Xamarin.Forms_Performance_Integration;Xamarin.Android.Locale_Tests"' The relevant part of the output of the msbuild example above looks like: UndeployTestApks: Executing: /Users/rodo/android-toolchain/sdk/platform-tools/adb uninstall "Xamarin.Android.Locale_Tests" Executing: /Users/rodo/android-toolchain/sdk/platform-tools/adb uninstall "Xamarin.Forms_Performance_Integration" DeployTestApks: Executing: /Users/rodo/android-toolchain/sdk/platform-tools/adb install "/Users/rodo/git/xa5/tests/../bin/TestDebug/Xamarin.Android.Locale_Tests-Signed.apk" Executing: /Users/rodo/android-toolchain/sdk/platform-tools/adb install "/Users/rodo/git/xa5/tests/../bin/TestDebug/Xamarin.Forms_Performance_Integration-Signed.apk" RecordApkSizes: stat -f "stat: %z %N" "/Users/rodo/git/xa5/tests/../bin/TestDebug/Xamarin.Forms_Performance_Integration-Signed.apk" > "/Users/rodo/git/xa5/tests/../bin/TestDebug/apk-sizes-Xamarin.Forms_Performance_Integration-Debug.txt" unzip -l "/Users/rodo/git/xa5/tests/../bin/TestDebug/Xamarin.Forms_Performance_Integration-Signed.apk" >> "/Users/rodo/git/xa5/tests/../bin/TestDebug/apk-sizes-Xamarin.Forms_Performance_Integration-Debug.txt" RunTestApks: Executing: /Users/rodo/android-toolchain/sdk/platform-tools/adb shell pm grant Xamarin.Android.Locale_Tests android.permission.READ_EXTERNAL_STORAGE Executing: /Users/rodo/android-toolchain/sdk/platform-tools/adb shell pm grant Xamarin.Android.Locale_Tests android.permission.WRITE_EXTERNAL_STORAGE Executing: /Users/rodo/android-toolchain/sdk/platform-tools/adb shell am instrument -e "loglevel Verbose" -w "Xamarin.Android.Locale_Tests/xamarin.android.localetests.TestInstrumentation" Executing: /Users/rodo/android-toolchain/sdk/platform-tools/adb logcat -v threadtime -d appending stdout to file: logcat-Debug-Xamarin.Android.Locale_Tests.txt Executing: /Users/rodo/android-toolchain/sdk/platform-tools/adb logcat -c Executing: /Users/rodo/android-toolchain/sdk/platform-tools/adb pull "/storage/emulated/0/Android/data/Xamarin.Android.Locale_Tests/files/Documents/TestResults.xml" "/Users/rodo/git/xa5/tests/../bin/TestDebug/TestResult-Xamarin.Android.Locale_Tests.xml" Executing: /Users/rodo/android-toolchain/sdk/platform-tools/adb shell am start -n "Xamarin.Forms_Performance_Integration/xamarin.forms.performance.integration.MainActivity" Executing: /Users/rodo/android-toolchain/sdk/platform-tools/adb logcat -v threadtime -d appending stdout to file: logcat-Debug-Xamarin.Forms_Performance_Integration.txt Executing: /Users/rodo/android-toolchain/sdk/platform-tools/adb logcat -c
2019-01-02 13:31:26 +03:00
ifneq ($(PACKAGES),)
APK_TESTS_PROP = /p:ApkTests='"$(PACKAGES)"'
endif
[Mono.Android-Test] Import Mono.Android tests from monodroid/9c5b3712 (#32) Import monodroid/tests/runtime from monodroid/9c5b3712. Add a toplevel `make run-apk-tests` target to "full stack" tests, in which a .apk is created, installed, and executed on an attached Android device. `make run-apk-tests` requires that `adb` be in $PATH, and uses GNU make(1) features, and... Additionally, tools/scripts/xabuild *must* be used to execute the `SignAndroidPackage` target, to ensure that the local/"in tree" assemblies are used. There is no "within xbuild" facility to alter where target framework assemblies are resolved from, i.e no MSBuild properties currently control the resolution order, only environment variables, and MSBuild can't *set* environment variables... The $(ADB_TARGET) variable can be used to control on which target Android device the tests will be installed and executed on: # Install & run tests on *only* connected USB device $ make run-apk-tests ADB_TARGET=-d # Install & run tests on *only* connected emulator $ make run-apk-tests ADB_TARGET=-e # Install & run tests on specified device, listed via `adb devices`. $ make run-apk-tests ADB_TARGET="-s 036533824381cfcb" Sorry potential/future Windows developers. *Running* tests will require manual testing or running on OS X or Linux for now... Note: These tests DO NOT PASS. In fact, they *crash*: $ make run-apk-tests ... Target RunTests: Executing: "$HOME/android-toolchain/sdk/platform-tools/adb" shell am instrument -w Mono.Android_Tests/xamarin.android.runtimetests.TestInstrumentation INSTRUMENTATION_RESULT: shortMsg=Process crashed. INSTRUMENTATION_CODE: 0 $ adb logcat ... E mono : Unhandled Exception: E mono : System.ObjectDisposedException: Cannot access a disposed object. E mono : Object name: 'System.Net.Sockets.Socket'. E mono : at System.Net.Sockets.Socket.ThrowIfDisposedAndClosed () <0xa93923f0 + 0x00054> in <filename unknown>:0 E mono : at System.Net.Sockets.Socket.AcceptAsync (System.Net.Sockets.SocketAsyncEventArgs e) <0x9b8f9680 + 0x0001b> in <filename unknown>:0 E mono : at System.Net.EndPointListener.Accept (System.Net.Sockets.Socket socket, System.Net.Sockets.SocketAsyncEventArgs e) <0x9b8f95d0 + 0x0003f> in <filename unknown>:0 E mono : at System.Net.EndPointListener.ProcessAccept (System.Net.Sockets.SocketAsyncEventArgs args) <0x9b8e0340 + 0x0007f> in <filename unknown>:0 E mono : at System.Net.EndPointListener.OnAccept (System.Object sender, System.Net.Sockets.SocketAsyncEventArgs e) <0x9b8e0310 + 0x00017> in <filename unknown>:0 E mono : at System.Net.Sockets.SocketAsyncEventArgs.OnCompleted (System.Net.Sockets.SocketAsyncEventArgs e) <0x9b8e02c8 + 0x0003b> in <filename unknown>:0 E mono : at System.Net.Sockets.SocketAsyncEventArgs.Complete () <0x9b8e02a0 + 0x0001f> in <filename unknown>:0 E mono : at System.Net.Sockets.Socket.<AcceptAsyncCallback>m__0 (System.IAsyncResult ares) <0x9b8dfd40 + 0x002af> in <filename unknown>:0 E mono : at System.Net.Sockets.SocketAsyncResult+<Complete>c__AnonStorey0.<>m__0 (System.Object _) <0xa892f720 + 0x0002b> in <filename unknown>:0 E mono : at System.Threading.QueueUserWorkItemCallback.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem () <0xa938f6b8 + 0x0002f> in <filename unknown>:0 E mono : at System.Threading.ThreadPoolWorkQueue.Dispatch () <0xa938e358 + 0x001bb> in <filename unknown>:0 E mono : at System.Threading._ThreadPoolWaitCallback.PerformWaitCallback () <0xa938e1a8 + 0x00007> in <filename unknown>:0 Looks like a Socket and/or ThreadPool bug in mono.
2016-05-16 21:55:13 +03:00
run-apk-tests:
[tests] Split APK tests into parallel and non-parallel runs (#2990) Context: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android/plot/Tests%20times/ What do we want! "Meaningful" performance metrics! (For various definitions of "meaningful.") Consider the "Xamarin.Forms app startup" times from builds 501, 502 and [504][0]: | Configuration | Build 501 Time | Build 502 Time | Build 504 Time | | ------------------------ | -------------- | -------------- | -------------- | | **Release** | 1,378ms | 1,471ms | 2,425ms | | **Release-AOT** | 5,104ms | 2,138ms | 2,699ms | | **Release-Profiled-Aot** | 1,515ms | 1,686ms | 1,521ms | What we *expect* to see is that Release-Profiled-Aot is fastest, followed by Release-Aot, followed by Release. What we instead see is that sometimes Release (JIT) is fastest, faster than any form of AOT (e.g. builds 501 & 502)! We believe that the cause of this us due to machine load: the `Xamarin.Forms-Performance-Integration` test is run *in parallel* with many of the other on-emulator unit tests. Thus, times fluctuate wildly and inconsistently, depending on what other tests may be running at the same time. The fix? Don't run the `Xamarin.Forms-Performance-Integration` and `locales` tests as part of the other unit tests. Instead, run them separately, *sequentially*, to reduce the likelihood of other processes on the machine creating inconsistent times. (Note that this is probably not going to be a "complete" solution, as e.g. build machine changes, new emulators, etc. will *also* cause build times to fluctuate *between builds*. We just hope that *intra-build* times will be consistent, matching our *expectations* that Release is slowest and some form of AOT is fastest.) The splitting is implemented by introducing new `RunPerformanceApkTests` target and pre-filtering `@(TestApk)`, `@(TestApkInstrumentation)` and `@(TestApkPermission)` item groups for this target and also for `RunApkTests` target. @grendello suggested adding `@(_TypeMapAssemblySource)` to the `Outputs` of the `_GenerateJavaStubs` target to fix this error observed during the build: _CompileNativeAssemblySources: [arm-linux-androideabi-as stderr] Assembler messages: EXEC : [arm-linux-androideabi-as stderr] error : can't open typemap.mj.armeabi-v7a.s for reading: No such file or directory which is caused by skipping `_GenerateJavaStubs` target here: Target "_GenerateJavaStubs: (TargetId:223)" in file "…/bin/Release/lib/xamarin.android/xbuild/Xamarin/Android/Xamarin.Android.Common.targets" from project "…/xamarin-android/tests/Xamarin.Forms-Performance-Integration/Droid/Xamarin.Forms.Performance.Integration.Droid.csproj" (target "_GeneratePackageManagerJava" depends on it): Skipping target "_GenerateJavaStubs" because all output files are up-to-date with respect to the input files. Additionally add `$(AndroidEnableProfiledAot)` to `@(_PropertyCacheItems)` and `$(IntermediateOutputPath)build.props`. This should address: error XA3001: Could not compile native assembly file: typemap.mj.armeabi-v7a.s when rebuilding projects with the value of `$(AndroidEnableProfiledAot)` changed. [0]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android/504/
2019-05-03 19:10:21 +03:00
_r=0 ; \
[Jenkins] remove some duplicate test phases (#3534) As we slowly migrate to Azure DevOps, there are some duplicate phases we are running on Jenkins that mostly produce noise, such as: * MSBuild Tests - we commonly get failures related to NuGet * BCL Tests - enough said... * Networking-related issues on APK-based tests For now we could replace the call: make run-all-tests To run a subset: make run-performance-tests This will keep the existing plots we have working. OSS contributors will still have access to builds, etc. Other changes: * The `run-performance-tests` target should run the APK-based performance tests. * We need to add `Mono.Android-Tests` to the APK performance tests.
2019-08-24 16:14:05 +03:00
$(call MSBUILD_BINLOG,run-apk-tests,,Test) $(TEST_TARGETS) /t:RunApkTests /p:RunApkTestsTarget=RunPerformanceApkTests $(APK_TESTS_PROP) || _r=1 ; \
$(call MSBUILD_BINLOG,run-apk-tests,,Test) $(TEST_TARGETS) /t:RunApkTests $(APK_TESTS_PROP) || _r=1 ; \
[tests] Split APK tests into parallel and non-parallel runs (#2990) Context: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android/plot/Tests%20times/ What do we want! "Meaningful" performance metrics! (For various definitions of "meaningful.") Consider the "Xamarin.Forms app startup" times from builds 501, 502 and [504][0]: | Configuration | Build 501 Time | Build 502 Time | Build 504 Time | | ------------------------ | -------------- | -------------- | -------------- | | **Release** | 1,378ms | 1,471ms | 2,425ms | | **Release-AOT** | 5,104ms | 2,138ms | 2,699ms | | **Release-Profiled-Aot** | 1,515ms | 1,686ms | 1,521ms | What we *expect* to see is that Release-Profiled-Aot is fastest, followed by Release-Aot, followed by Release. What we instead see is that sometimes Release (JIT) is fastest, faster than any form of AOT (e.g. builds 501 & 502)! We believe that the cause of this us due to machine load: the `Xamarin.Forms-Performance-Integration` test is run *in parallel* with many of the other on-emulator unit tests. Thus, times fluctuate wildly and inconsistently, depending on what other tests may be running at the same time. The fix? Don't run the `Xamarin.Forms-Performance-Integration` and `locales` tests as part of the other unit tests. Instead, run them separately, *sequentially*, to reduce the likelihood of other processes on the machine creating inconsistent times. (Note that this is probably not going to be a "complete" solution, as e.g. build machine changes, new emulators, etc. will *also* cause build times to fluctuate *between builds*. We just hope that *intra-build* times will be consistent, matching our *expectations* that Release is slowest and some form of AOT is fastest.) The splitting is implemented by introducing new `RunPerformanceApkTests` target and pre-filtering `@(TestApk)`, `@(TestApkInstrumentation)` and `@(TestApkPermission)` item groups for this target and also for `RunApkTests` target. @grendello suggested adding `@(_TypeMapAssemblySource)` to the `Outputs` of the `_GenerateJavaStubs` target to fix this error observed during the build: _CompileNativeAssemblySources: [arm-linux-androideabi-as stderr] Assembler messages: EXEC : [arm-linux-androideabi-as stderr] error : can't open typemap.mj.armeabi-v7a.s for reading: No such file or directory which is caused by skipping `_GenerateJavaStubs` target here: Target "_GenerateJavaStubs: (TargetId:223)" in file "…/bin/Release/lib/xamarin.android/xbuild/Xamarin/Android/Xamarin.Android.Common.targets" from project "…/xamarin-android/tests/Xamarin.Forms-Performance-Integration/Droid/Xamarin.Forms.Performance.Integration.Droid.csproj" (target "_GeneratePackageManagerJava" depends on it): Skipping target "_GenerateJavaStubs" because all output files are up-to-date with respect to the input files. Additionally add `$(AndroidEnableProfiledAot)` to `@(_PropertyCacheItems)` and `$(IntermediateOutputPath)build.props`. This should address: error XA3001: Could not compile native assembly file: typemap.mj.armeabi-v7a.s when rebuilding projects with the value of `$(AndroidEnableProfiledAot)` changed. [0]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android/504/
2019-05-03 19:10:21 +03:00
exit $$_r
Add a bunch of Make targets to help rebuilding Mono runtime and BCL (#1029) This basically implements the steps described in Documentation/DevelopmentTips.md for the runtime and BCL assembles. Short help is available by running make rebuild-help
2017-11-15 20:32:20 +03:00
[performance] initial MSBuild project timing (#1569) "Performance", in various guises, is an ongoing focus. For example, we track [application startup times][time-plots]. [time-plots]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android/plot/Tests%20times/ The next area we want to concentrate on is build and deployment time, and to do so we need to start collecting information so we know how we're doing, and whether anything is getting worse. The first step in tracking our build times is to track the build times for some of our existing unit tests projects, and plot those on Jenkins. This is done by using a custom MSBuild logger `Xamarin.Android.Tools.BootstrapTasks.TimingLogger` which records detailed timing information into an XML file: <builds> <build start-ticks="636607852701529939" start-threadid="1" id="HelloWorld-FreshBuild" commit="8dcd59e" description="A fresh build after a clean checkout for HelloWorld" end-ticks="636607852899547509" end-threadid="1" elapsed="00:00:19.8017570" succeeded="True"> <toolsets> <toolset version="15.0" path="C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\MSBuild\15.0\Bin" /> </toolsets> <files> <file id="1" path="C:\Users\jopepper\Desktop\Git\xamarin-android\samples\HelloWorld\HelloWorld.csproj" /> </files> <project start-ticks="636607852707145402" start-threadid="6" file-id="1" end-ticks="636607852899326841" end-threadid="6" elapsed="00:00:19.2181439" succeeded="True"> <target start-ticks="636607852707365985" start-threadid="6" name="RedirectMonoAndroidSdkPaths" file-id="2" end-ticks="636607852708932958" end-threadid="6" elapsed="00:00:00.1566973" succeeded="True"> <task start-ticks="636607852708755105" start-threadid="6" name="SetVsMonoAndroidRegistryKey" file-id="2" end-ticks="636607852708895478" end-threadid="6" elapsed="00:00:00.0140373" succeeded="True" /> </target> </project> </build> </builds> These timing XML files from multiple builds are aggregated by the new `<ProcessMSBuildTiming/>` task, which generates a CSV file containing the `//*/@elapsed` attribute values for us by the [Jenkins Plots plugin][jenkins-plots]. [jenkins-plots]: https://wiki.jenkins.io/display/JENKINS/Plot+Plugin The following projects have their build times collected: - `samples/HelloWorld` - `tests/Xamarin.Forms-Performance-Integration` Across four types of builds: - Fresh build (simulated fresh checkout) - Second build (no changes) - Touch C# file, build again - Touch Android resource XML file, build again We are timing the `SignAndroidPackage` and `Install` targets for each of these build types. Results are aggregated into `TestResult-MSBuild-times.csv` for processing on Jenkins. After this gets merged into master, there will be some configuration needed to setup the new plot. Example results, in the aggregated CSV file: | HelloWorld-FreshBuild-Debug | HelloWorld-FreshInstall-Debug | XF-FreshBuild-Debug | XF-FreshInstall-Debug | | ---: | ---: | ---: | ---: | | 00:00:16.3538291 | 00:00:04.7940071 | 00:00:27.5263076 | 00:00:22.6720137 | Also included in the table, will be the `TouchCSharp` and `TouchAndroidResource` builds for both of the `HelloWorld` and `XF` projects for both `SignAndroidPackage` and `Install` targets. Other changes: - I let Visual Studio 2017 re-sort/fixup `Xamarin.Android.sln` - Split up `timing.csproj` into `timing.targets` and `timing.projitems` - Updated `Makefile` to include a `run-performance-tests` target. `TimingLogger` is courtesy of @grendello.
2018-05-02 22:26:50 +03:00
run-performance-tests:
[Jenkins] remove some duplicate test phases (#3534) As we slowly migrate to Azure DevOps, there are some duplicate phases we are running on Jenkins that mostly produce noise, such as: * MSBuild Tests - we commonly get failures related to NuGet * BCL Tests - enough said... * Networking-related issues on APK-based tests For now we could replace the call: make run-all-tests To run a subset: make run-performance-tests This will keep the existing plots we have working. OSS contributors will still have access to builds, etc. Other changes: * The `run-performance-tests` target should run the APK-based performance tests. * We need to add `Mono.Android-Tests` to the APK performance tests.
2019-08-24 16:14:05 +03:00
_r=0 ; \
$(call MSBUILD_BINLOG,run-apk-tests,,Test) $(TEST_TARGETS) /t:RunApkTests /p:RunApkTestsTarget=RunPerformanceApkTests $(APK_TESTS_PROP) || _r=1 ; \
$(call MSBUILD_BINLOG,run-performance-tests,,Test) $(TEST_TARGETS) /t:RunPerformanceTests || _r=1 ; \
exit $$_r
[performance] initial MSBuild project timing (#1569) "Performance", in various guises, is an ongoing focus. For example, we track [application startup times][time-plots]. [time-plots]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android/plot/Tests%20times/ The next area we want to concentrate on is build and deployment time, and to do so we need to start collecting information so we know how we're doing, and whether anything is getting worse. The first step in tracking our build times is to track the build times for some of our existing unit tests projects, and plot those on Jenkins. This is done by using a custom MSBuild logger `Xamarin.Android.Tools.BootstrapTasks.TimingLogger` which records detailed timing information into an XML file: <builds> <build start-ticks="636607852701529939" start-threadid="1" id="HelloWorld-FreshBuild" commit="8dcd59e" description="A fresh build after a clean checkout for HelloWorld" end-ticks="636607852899547509" end-threadid="1" elapsed="00:00:19.8017570" succeeded="True"> <toolsets> <toolset version="15.0" path="C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\MSBuild\15.0\Bin" /> </toolsets> <files> <file id="1" path="C:\Users\jopepper\Desktop\Git\xamarin-android\samples\HelloWorld\HelloWorld.csproj" /> </files> <project start-ticks="636607852707145402" start-threadid="6" file-id="1" end-ticks="636607852899326841" end-threadid="6" elapsed="00:00:19.2181439" succeeded="True"> <target start-ticks="636607852707365985" start-threadid="6" name="RedirectMonoAndroidSdkPaths" file-id="2" end-ticks="636607852708932958" end-threadid="6" elapsed="00:00:00.1566973" succeeded="True"> <task start-ticks="636607852708755105" start-threadid="6" name="SetVsMonoAndroidRegistryKey" file-id="2" end-ticks="636607852708895478" end-threadid="6" elapsed="00:00:00.0140373" succeeded="True" /> </target> </project> </build> </builds> These timing XML files from multiple builds are aggregated by the new `<ProcessMSBuildTiming/>` task, which generates a CSV file containing the `//*/@elapsed` attribute values for us by the [Jenkins Plots plugin][jenkins-plots]. [jenkins-plots]: https://wiki.jenkins.io/display/JENKINS/Plot+Plugin The following projects have their build times collected: - `samples/HelloWorld` - `tests/Xamarin.Forms-Performance-Integration` Across four types of builds: - Fresh build (simulated fresh checkout) - Second build (no changes) - Touch C# file, build again - Touch Android resource XML file, build again We are timing the `SignAndroidPackage` and `Install` targets for each of these build types. Results are aggregated into `TestResult-MSBuild-times.csv` for processing on Jenkins. After this gets merged into master, there will be some configuration needed to setup the new plot. Example results, in the aggregated CSV file: | HelloWorld-FreshBuild-Debug | HelloWorld-FreshInstall-Debug | XF-FreshBuild-Debug | XF-FreshInstall-Debug | | ---: | ---: | ---: | ---: | | 00:00:16.3538291 | 00:00:04.7940071 | 00:00:27.5263076 | 00:00:22.6720137 | Also included in the table, will be the `TouchCSharp` and `TouchAndroidResource` builds for both of the `HelloWorld` and `XF` projects for both `SignAndroidPackage` and `Install` targets. Other changes: - I let Visual Studio 2017 re-sort/fixup `Xamarin.Android.sln` - Split up `timing.csproj` into `timing.targets` and `timing.projitems` - Updated `Makefile` to include a `run-performance-tests` target. `TimingLogger` is courtesy of @grendello.
2018-05-02 22:26:50 +03:00
[build] Add `list-nunit-tests` and docs (#1812) Add the ability to list the unit tests via `make` or `msbuild`. Added docs on how to run single tests.
2018-06-13 20:32:28 +03:00
list-nunit-tests:
$(MSBUILD) $(MSBUILD_FLAGS) $(TEST_TARGETS) /t:ListNUnitTests
Add a bunch of Make targets to help rebuilding Mono runtime and BCL (#1029) This basically implements the steps described in Documentation/DevelopmentTips.md for the runtime and BCL assembles. Short help is available by running make rebuild-help
2017-11-15 20:32:20 +03:00
include build-tools/scripts/runtime-helpers.mk
[build] Xamarin.Android build preparation utility (#3051) Fixes: https://github.com/xamarin/xamarin-android/issues/2562 The `make prepare` **make** target and `msbuild /t:Prepare` MSBuild target are used to "prepare" a `xamarin-android` checkout so that the subsequent `make all` or `msbuild /t:Build` invocation will work. The preparation step involves ensuring that all known required dependencies exist or are provisioned (54a7a029), such as the build utilities (`autoconf`/etc.), cross-compilers (MinGW), the Android SDK and NDK, and building "enough" to allow `Xamarin.Android.sln` to build (see also 7343965a0). Unfortunately, the Glorious Idea™ of using MSBuild for significant portions of this process is "baroque", making it difficult to understand and maintain, and the requirement on `msbuild` means that things break in really bizarre and obscure ways on Linux and macOS if the installed `mono` version is "too old"...and the determination of "too old" was being done via `mono` & `msbuild`! Furthermore, there was duplication and repetition of various bits of build configuration on Linux and macOS between Makefile and the MSBuild project system, which made it easier to make mistakes. (Then there's the occasional bizarro failures that would result from attempting to evaluate `$(PRODUCT_VERSION)` within `make`; it would fail when `MONO_LOG_LEVEL=info` was set. Then there's...) Instead of having `make prepare` be implemented as an unholy mess of `make` and `msbuild`, introduce the new `build-tools/xaprepare` app. `xaprepare` will be built and executed as part of `make prepare`, and takes over responsibility for repo build preparation. See also: build-tools/xaprepare/README.md TODO: Windows support
2019-06-07 20:58:59 +03:00
.PHONY: prepare-build-init
prepare-build-init:
mkdir -p $(dir $(PREPARE_BUILD_LOG))
msbuild $(PREPARE_RESTORE_FLAGS) $(PREPARE_SOLUTION) /t:Restore
.PHONY: prepare-build
prepare-build: prepare-build-init
msbuild $(PREPARE_MSBUILD_FLAGS) $(PREPARE_SOLUTION)
.PHONY: prepare
[build] Pass variables to monodroid and improve toolchain caching (#3325) Context: xamarin/monodroid#1010 Partially reworks the inverted build so that variables imported from 'rules.mk' can be accessed externally. Improves commercial build support for new Android API levels. Improves provisioning performance when building and testing on the Azure Pipeline macOS hosted agent pools. We'll now use the Android SDK components which are pre-installed on those machines, as most of what we need is already present and this location should be less prone to being cleaned up.
2019-07-10 02:00:50 +03:00
prepare: prepare-build
[build] Xamarin.Android build preparation utility (#3051) Fixes: https://github.com/xamarin/xamarin-android/issues/2562 The `make prepare` **make** target and `msbuild /t:Prepare` MSBuild target are used to "prepare" a `xamarin-android` checkout so that the subsequent `make all` or `msbuild /t:Build` invocation will work. The preparation step involves ensuring that all known required dependencies exist or are provisioned (54a7a029), such as the build utilities (`autoconf`/etc.), cross-compilers (MinGW), the Android SDK and NDK, and building "enough" to allow `Xamarin.Android.sln` to build (see also 7343965a0). Unfortunately, the Glorious Idea™ of using MSBuild for significant portions of this process is "baroque", making it difficult to understand and maintain, and the requirement on `msbuild` means that things break in really bizarre and obscure ways on Linux and macOS if the installed `mono` version is "too old"...and the determination of "too old" was being done via `mono` & `msbuild`! Furthermore, there was duplication and repetition of various bits of build configuration on Linux and macOS between Makefile and the MSBuild project system, which made it easier to make mistakes. (Then there's the occasional bizarro failures that would result from attempting to evaluate `$(PRODUCT_VERSION)` within `make`; it would fail when `MONO_LOG_LEVEL=info` was set. Then there's...) Instead of having `make prepare` be implemented as an unholy mess of `make` and `msbuild`, introduce the new `build-tools/xaprepare` app. `xaprepare` will be built and executed as part of `make prepare`, and takes over responsibility for repo build preparation. See also: build-tools/xaprepare/README.md TODO: Windows support
2019-06-07 20:58:59 +03:00
mono --debug $(PREPARE_EXE) $(_PREPARE_ARGS)
[build] improve Xamarin.Android.sln usability in VS Windows (#3877) Xamarin.Android.sln has two issues that make `Xamarin.Android.sln` not work very well in VS Windows: * It builds MSBuild tasks and uses them within the same solution, such as `xa-prep-tasks.csproj` and `Xamarin.Android.Tools.BootstrapTasks.csproj`. * We have conditional `@(ProjectReference)`s. To improve things, I completely removed the `$(XAIntegratedTests)` MSBuild property (1b72a184) and removed any `@(ProjectReference)` that were conditional. I moved the two bootstrap `.csproj` files to a new `Xamarin.Android.BootstrapTasks.sln` which gets built during `make prepare` or `msbuild Xamarin.Android.sln /t:Prepare`. Visual Studio on Windows was also complaining about the `CreateManifestResourceNames` target missing for projects that merely import `Microsoft.Common.targets` such as `r8.csproj`. I added an empty target in `Configuration.props` to fix all cases of this. Unfortunately, things are still not perfect, as you can hit errors such as: Unable to copy file "obj\Debug\Xamarin.Android.Build.Tasks.dll" to "C:\src\xamarin-android\bin\Debug\lib\xamarin.android\xbuild\Xamarin\Android\Xamarin.Android.Build.Tasks.dll". The process cannot access the file 'C:\src\xamarin-android\bin\Debug\lib\xamarin.android\xbuild\Xamarin\Android\Xamarin.Android.Build.Tasks.dll' because it is being used by another process. Could not copy "obj\Debug\Xamarin.Android.Build.Tasks.dll" to "C:\src\xamarin-android\bin\Debug\lib\xamarin.android\xbuild\Xamarin\Android\Xamarin.Android.Build.Tasks.dll". Exceeded retry count of 10. Failed. The file is locked by: "Microsoft Visual Studio 2019 (26660), MSBuild.exe (18800), MSBuild.exe (17416)" Xamarin.Android.Build.Tasks This occurs if you modify `Xamarin.Android.Build.Tasks.csproj`, because VS Windows keeps MSBuild nodes alive, and if they have previously loaded `Xamarin.Android.Build.Tasks.dll` then the assembly is locked. I am not yet sure if we can get around this remaining problem, but we can revisit in the future.
2019-11-12 13:37:20 +03:00
msbuild $(BOOTSTRAP_MSBUILD_FLAGS) $(BOOTSTRAP_SOLUTION)
[build] Xamarin.Android build preparation utility (#3051) Fixes: https://github.com/xamarin/xamarin-android/issues/2562 The `make prepare` **make** target and `msbuild /t:Prepare` MSBuild target are used to "prepare" a `xamarin-android` checkout so that the subsequent `make all` or `msbuild /t:Build` invocation will work. The preparation step involves ensuring that all known required dependencies exist or are provisioned (54a7a029), such as the build utilities (`autoconf`/etc.), cross-compilers (MinGW), the Android SDK and NDK, and building "enough" to allow `Xamarin.Android.sln` to build (see also 7343965a0). Unfortunately, the Glorious Idea™ of using MSBuild for significant portions of this process is "baroque", making it difficult to understand and maintain, and the requirement on `msbuild` means that things break in really bizarre and obscure ways on Linux and macOS if the installed `mono` version is "too old"...and the determination of "too old" was being done via `mono` & `msbuild`! Furthermore, there was duplication and repetition of various bits of build configuration on Linux and macOS between Makefile and the MSBuild project system, which made it easier to make mistakes. (Then there's the occasional bizarro failures that would result from attempting to evaluate `$(PRODUCT_VERSION)` within `make`; it would fail when `MONO_LOG_LEVEL=info` was set. Then there's...) Instead of having `make prepare` be implemented as an unholy mess of `make` and `msbuild`, introduce the new `build-tools/xaprepare` app. `xaprepare` will be built and executed as part of `make prepare`, and takes over responsibility for repo build preparation. See also: build-tools/xaprepare/README.md TODO: Windows support
2019-06-07 20:58:59 +03:00
.PHONY: prepare-help
prepare-help: prepare-build
mono --debug $(PREPARE_EXE) -h
.PHONY: prepare-update-mono
[build] Simplify CI make and xaprepare invocations (#3279) * [build] Simplify CI make and xaprepare invocations Context: https://github.com/xamarin/xamarin-android/commit/9302d514a1437e5d1977da06dfbbe4e7054f33d5 Context: https://github.com/xamarin/xamarin-android/commit/f9f5b6cd120484b13df74e94638e07411d8693fa We built a lot of hacks into our make file when initially working on the xaprepare tool, as the windows and mac azure pipeline builds could not be modified at the time. We now have these builds defined in yaml, and as such we should be able to simplify the underlying make rules used by all of our builds. * Removes `prepare-build-ci` which no longer appears to be relevant, as the extra args it specifies don't appear to be set before the tool is built. * Removes `prepare-jenkins` and `prepare-commercial`, as we can now rely on the `prepare` rule again. The behavior of `prepare` will change depending on the `_PREPARE_ARGS` varible, which is controlled by different flags passed by our CI systems, `azure-pipelines.yaml`, and `build.groovy` * Introduces a new `prepareFlags` variable to build.groovy which is used to control whether or not the "prepare all" (-a arg) should be passed to xaprepare. This should not be specified when building Jenkins PR builds, as they only build a subset of all supported ABIs and frameworks. This should address the EmbeddedDSO test failures we are encountering in Jenkins PR builds. * Removes the MSBuild Auto Provision and ABI related args from make invocations in azure-pipelines.yaml. Make variables will instead be used to control these options. * Removes the `prepare-update-mono` dependency from `prepare-external-git-dependencies` as we should be able to call this without needing to update mono. * Adds a `prepare` dependency back to `jenkins`, now that the `prepare-jenkins` step is no longer required. * [build] Don't override _MSBUILD_ARGS when creating installers Context: https://github.com/xamarin/xamarin-android/commit/e83ba0ddc159d15abcfac1a5a4bce5a53e165d3f The `_MSBUILD_ARGS` override was initially required because we did not import different `rules.make` based on how our build was configured and prepared. The `xaprepare` tool now better controls the ABIs we should be building and packaging for each build type, so this is no longer needed.
2019-06-26 11:08:00 +03:00
prepare-update-mono: prepare-build
Bump to mono/mono/2020-02@d90665a4 (#4219) Changes: https://github.com/mono/api-snapshot/compare/53a841ffd666d32cfa893bcd0a686ec69d91a8eb...5b8247e289d17cef9ff645e21d896ea75bfe6328 $ git diff --shortstat 53a841ff...5b8247e2 9 files changed, 572 insertions(+), 13 deletions(-) Changes: https://github.com/mono/corefx/compare/1cdb9c20946c74a88a7fa507215556fadc2c2b27...7c9e2158925339a5a96ff8d09d21c600a0b27e5a $ git diff --shortstat 1cdb9c20...7c9e2158 24 files changed, 2393 insertions(+), 396 deletions(-) Changes: https://github.com/mono/cecil/compare/a6a7f5c0f48c2cbbec699726bbac0816ae097003...8021f3fbe75715a1762e725594d8c00cce3679d8 $ git diff --shortstat a6a7f5c0...8021f3fb 16 files changed, 98 insertions(+), 25 deletions(-) Changes: https://github.com/mono/linker/compare/e8d054bf3224f6cda6f5ecaf420f7c9d65ade70f...e1c7a72951c86db5c6df5ea8e1864432f7a6b12d $ git diff --shortstat e8d054bf...e1c7a729 220 files changed, 9758 insertions(+), 3165 deletions(-) Changes: https://github.com/mono/mono/compare/2ff898859b8b75f123a5c895959a7e82dc829ea5...d90665a422e9f8d015585b3ca381d74faa033cc4 $ git diff --shortstat 2ff89885...d90665a4 612 files changed, 20193 insertions(+), 10239 deletions(-) Context: https://github.com/mono/mono/issues/9726 Context: https://devdiv.visualstudio.com/DevDiv/_workitems/edit/1048838 Context: https://devdiv.visualstudio.com/DevDiv/_workitems/edit/1050615 Context: https://devdiv.visualstudio.com/DevDiv/_workitems/edit/1069059 Context: https://github.com/mono/mono/issues/10643 Context: https://github.com/mono/mono/issues/10651 Context: https://github.com/mono/mono/issues/12022 Context: https://github.com/mono/mono/issues/12995 Context: https://github.com/mono/mono/issues/15612 Context: https://github.com/mono/mono/issues/16513 Context: https://github.com/mono/mono/issues/16588 Context: https://github.com/mono/mono/issues/16778 Context: https://github.com/mono/mono/issues/16969 Context: https://github.com/mono/mono/issues/17140 Context: https://github.com/mono/mono/issues/17601 Context: https://github.com/mono/mono/issues/17869 Context: https://github.com/mono/mono/issues/17878 Context: https://github.com/mono/mono/issues/17916 Context: https://github.com/mono/mono/issues/17926 Context: https://github.com/mono/mono/issues/17980 Context: https://github.com/mono/mono/issues/18006 Context: https://github.com/mono/mono/issues/18019 Context: https://github.com/mono/mono/issues/18020 Context: https://github.com/mono/mono/issues/18030 Context: https://github.com/mono/mono/issues/18061 Context: https://github.com/mono/mono/issues/18064 Context: https://github.com/mono/mono/issues/18106 Context: https://github.com/mono/mono/issues/18120 Context: https://github.com/mono/mono/issues/18191 Context: https://github.com/mono/mono/issues/18202 Context: https://github.com/mono/mono/issues/18213 Context: https://github.com/mono/mono/issues/18221 Context: https://github.com/mono/mono/issues/18247 Context: https://github.com/mono/mono/issues/18273 Context: https://github.com/mono/mono/issues/18276 Context: https://github.com/mono/mono/issues/18317 Context: https://github.com/mono/mono/issues/18323 Context: https://github.com/mono/mono/issues/18364 Context: https://github.com/mono/mono/issues/18370 Context: https://github.com/mono/mono/issues/18417 Context: https://github.com/mono/mono/issues/18418 Context: https://github.com/mono/mono/issues/18455 Context: https://github.com/mono/mono/issues/18506 Context: https://github.com/mono/mono/issues/18524 Context: https://github.com/mono/mono/issues/18530 Context: https://github.com/mono/mono/issues/18554 Context: https://github.com/mono/mono/issues/18572 Context: https://github.com/mono/mono/issues/18578 Context: https://github.com/mono/mono/issues/18584 Context: https://github.com/mono/mono/issues/18612 Context: https://github.com/mono/mono/issues/18614 Context: https://github.com/mono/mono/issues/18675 Context: https://github.com/mono/mono/issues/18676 Context: https://github.com/mono/mono/issues/18925 Context: https://github.com/mono/mono/issues/19009 Context: https://issuetracker.unity3d.com/issues/unity-physics-collisions-do-not-work-and-errors-are-thrown-when-entering-play-mode Context: https://xamarin.github.io/bugzilla-archives/20/20233/bug.html
2020-03-13 23:12:20 +03:00
pgrep -lfi VBCSCompiler.exe
[build] Simplify CI make and xaprepare invocations (#3279) * [build] Simplify CI make and xaprepare invocations Context: https://github.com/xamarin/xamarin-android/commit/9302d514a1437e5d1977da06dfbbe4e7054f33d5 Context: https://github.com/xamarin/xamarin-android/commit/f9f5b6cd120484b13df74e94638e07411d8693fa We built a lot of hacks into our make file when initially working on the xaprepare tool, as the windows and mac azure pipeline builds could not be modified at the time. We now have these builds defined in yaml, and as such we should be able to simplify the underlying make rules used by all of our builds. * Removes `prepare-build-ci` which no longer appears to be relevant, as the extra args it specifies don't appear to be set before the tool is built. * Removes `prepare-jenkins` and `prepare-commercial`, as we can now rely on the `prepare` rule again. The behavior of `prepare` will change depending on the `_PREPARE_ARGS` varible, which is controlled by different flags passed by our CI systems, `azure-pipelines.yaml`, and `build.groovy` * Introduces a new `prepareFlags` variable to build.groovy which is used to control whether or not the "prepare all" (-a arg) should be passed to xaprepare. This should not be specified when building Jenkins PR builds, as they only build a subset of all supported ABIs and frameworks. This should address the EmbeddedDSO test failures we are encountering in Jenkins PR builds. * Removes the MSBuild Auto Provision and ABI related args from make invocations in azure-pipelines.yaml. Make variables will instead be used to control these options. * Removes the `prepare-update-mono` dependency from `prepare-external-git-dependencies` as we should be able to call this without needing to update mono. * Adds a `prepare` dependency back to `jenkins`, now that the `prepare-jenkins` step is no longer required. * [build] Don't override _MSBUILD_ARGS when creating installers Context: https://github.com/xamarin/xamarin-android/commit/e83ba0ddc159d15abcfac1a5a4bce5a53e165d3f The `_MSBUILD_ARGS` override was initially required because we did not import different `rules.make` based on how our build was configured and prepared. The `xaprepare` tool now better controls the ABIs we should be building and packaging for each build type, so this is no longer needed.
2019-06-26 11:08:00 +03:00
mono --debug $(PREPARE_EXE) $(_PREPARE_ARGS) -s:UpdateMono
Bump to mono/mono/2020-02@d90665a4 (#4219) Changes: https://github.com/mono/api-snapshot/compare/53a841ffd666d32cfa893bcd0a686ec69d91a8eb...5b8247e289d17cef9ff645e21d896ea75bfe6328 $ git diff --shortstat 53a841ff...5b8247e2 9 files changed, 572 insertions(+), 13 deletions(-) Changes: https://github.com/mono/corefx/compare/1cdb9c20946c74a88a7fa507215556fadc2c2b27...7c9e2158925339a5a96ff8d09d21c600a0b27e5a $ git diff --shortstat 1cdb9c20...7c9e2158 24 files changed, 2393 insertions(+), 396 deletions(-) Changes: https://github.com/mono/cecil/compare/a6a7f5c0f48c2cbbec699726bbac0816ae097003...8021f3fbe75715a1762e725594d8c00cce3679d8 $ git diff --shortstat a6a7f5c0...8021f3fb 16 files changed, 98 insertions(+), 25 deletions(-) Changes: https://github.com/mono/linker/compare/e8d054bf3224f6cda6f5ecaf420f7c9d65ade70f...e1c7a72951c86db5c6df5ea8e1864432f7a6b12d $ git diff --shortstat e8d054bf...e1c7a729 220 files changed, 9758 insertions(+), 3165 deletions(-) Changes: https://github.com/mono/mono/compare/2ff898859b8b75f123a5c895959a7e82dc829ea5...d90665a422e9f8d015585b3ca381d74faa033cc4 $ git diff --shortstat 2ff89885...d90665a4 612 files changed, 20193 insertions(+), 10239 deletions(-) Context: https://github.com/mono/mono/issues/9726 Context: https://devdiv.visualstudio.com/DevDiv/_workitems/edit/1048838 Context: https://devdiv.visualstudio.com/DevDiv/_workitems/edit/1050615 Context: https://devdiv.visualstudio.com/DevDiv/_workitems/edit/1069059 Context: https://github.com/mono/mono/issues/10643 Context: https://github.com/mono/mono/issues/10651 Context: https://github.com/mono/mono/issues/12022 Context: https://github.com/mono/mono/issues/12995 Context: https://github.com/mono/mono/issues/15612 Context: https://github.com/mono/mono/issues/16513 Context: https://github.com/mono/mono/issues/16588 Context: https://github.com/mono/mono/issues/16778 Context: https://github.com/mono/mono/issues/16969 Context: https://github.com/mono/mono/issues/17140 Context: https://github.com/mono/mono/issues/17601 Context: https://github.com/mono/mono/issues/17869 Context: https://github.com/mono/mono/issues/17878 Context: https://github.com/mono/mono/issues/17916 Context: https://github.com/mono/mono/issues/17926 Context: https://github.com/mono/mono/issues/17980 Context: https://github.com/mono/mono/issues/18006 Context: https://github.com/mono/mono/issues/18019 Context: https://github.com/mono/mono/issues/18020 Context: https://github.com/mono/mono/issues/18030 Context: https://github.com/mono/mono/issues/18061 Context: https://github.com/mono/mono/issues/18064 Context: https://github.com/mono/mono/issues/18106 Context: https://github.com/mono/mono/issues/18120 Context: https://github.com/mono/mono/issues/18191 Context: https://github.com/mono/mono/issues/18202 Context: https://github.com/mono/mono/issues/18213 Context: https://github.com/mono/mono/issues/18221 Context: https://github.com/mono/mono/issues/18247 Context: https://github.com/mono/mono/issues/18273 Context: https://github.com/mono/mono/issues/18276 Context: https://github.com/mono/mono/issues/18317 Context: https://github.com/mono/mono/issues/18323 Context: https://github.com/mono/mono/issues/18364 Context: https://github.com/mono/mono/issues/18370 Context: https://github.com/mono/mono/issues/18417 Context: https://github.com/mono/mono/issues/18418 Context: https://github.com/mono/mono/issues/18455 Context: https://github.com/mono/mono/issues/18506 Context: https://github.com/mono/mono/issues/18524 Context: https://github.com/mono/mono/issues/18530 Context: https://github.com/mono/mono/issues/18554 Context: https://github.com/mono/mono/issues/18572 Context: https://github.com/mono/mono/issues/18578 Context: https://github.com/mono/mono/issues/18584 Context: https://github.com/mono/mono/issues/18612 Context: https://github.com/mono/mono/issues/18614 Context: https://github.com/mono/mono/issues/18675 Context: https://github.com/mono/mono/issues/18676 Context: https://github.com/mono/mono/issues/18925 Context: https://github.com/mono/mono/issues/19009 Context: https://issuetracker.unity3d.com/issues/unity-physics-collisions-do-not-work-and-errors-are-thrown-when-entering-play-mode Context: https://xamarin.github.io/bugzilla-archives/20/20233/bug.html
2020-03-13 23:12:20 +03:00
pgrep -lfi VBCSCompiler.exe
[build] Xamarin.Android build preparation utility (#3051) Fixes: https://github.com/xamarin/xamarin-android/issues/2562 The `make prepare` **make** target and `msbuild /t:Prepare` MSBuild target are used to "prepare" a `xamarin-android` checkout so that the subsequent `make all` or `msbuild /t:Build` invocation will work. The preparation step involves ensuring that all known required dependencies exist or are provisioned (54a7a029), such as the build utilities (`autoconf`/etc.), cross-compilers (MinGW), the Android SDK and NDK, and building "enough" to allow `Xamarin.Android.sln` to build (see also 7343965a0). Unfortunately, the Glorious Idea™ of using MSBuild for significant portions of this process is "baroque", making it difficult to understand and maintain, and the requirement on `msbuild` means that things break in really bizarre and obscure ways on Linux and macOS if the installed `mono` version is "too old"...and the determination of "too old" was being done via `mono` & `msbuild`! Furthermore, there was duplication and repetition of various bits of build configuration on Linux and macOS between Makefile and the MSBuild project system, which made it easier to make mistakes. (Then there's the occasional bizarro failures that would result from attempting to evaluate `$(PRODUCT_VERSION)` within `make`; it would fail when `MONO_LOG_LEVEL=info` was set. Then there's...) Instead of having `make prepare` be implemented as an unholy mess of `make` and `msbuild`, introduce the new `build-tools/xaprepare` app. `xaprepare` will be built and executed as part of `make prepare`, and takes over responsibility for repo build preparation. See also: build-tools/xaprepare/README.md TODO: Windows support
2019-06-07 20:58:59 +03:00
[build] Simplify CI make and xaprepare invocations (#3279) * [build] Simplify CI make and xaprepare invocations Context: https://github.com/xamarin/xamarin-android/commit/9302d514a1437e5d1977da06dfbbe4e7054f33d5 Context: https://github.com/xamarin/xamarin-android/commit/f9f5b6cd120484b13df74e94638e07411d8693fa We built a lot of hacks into our make file when initially working on the xaprepare tool, as the windows and mac azure pipeline builds could not be modified at the time. We now have these builds defined in yaml, and as such we should be able to simplify the underlying make rules used by all of our builds. * Removes `prepare-build-ci` which no longer appears to be relevant, as the extra args it specifies don't appear to be set before the tool is built. * Removes `prepare-jenkins` and `prepare-commercial`, as we can now rely on the `prepare` rule again. The behavior of `prepare` will change depending on the `_PREPARE_ARGS` varible, which is controlled by different flags passed by our CI systems, `azure-pipelines.yaml`, and `build.groovy` * Introduces a new `prepareFlags` variable to build.groovy which is used to control whether or not the "prepare all" (-a arg) should be passed to xaprepare. This should not be specified when building Jenkins PR builds, as they only build a subset of all supported ABIs and frameworks. This should address the EmbeddedDSO test failures we are encountering in Jenkins PR builds. * Removes the MSBuild Auto Provision and ABI related args from make invocations in azure-pipelines.yaml. Make variables will instead be used to control these options. * Removes the `prepare-update-mono` dependency from `prepare-external-git-dependencies` as we should be able to call this without needing to update mono. * Adds a `prepare` dependency back to `jenkins`, now that the `prepare-jenkins` step is no longer required. * [build] Don't override _MSBUILD_ARGS when creating installers Context: https://github.com/xamarin/xamarin-android/commit/e83ba0ddc159d15abcfac1a5a4bce5a53e165d3f The `_MSBUILD_ARGS` override was initially required because we did not import different `rules.make` based on how our build was configured and prepared. The `xaprepare` tool now better controls the ABIs we should be building and packaging for each build type, so this is no longer needed.
2019-06-26 11:08:00 +03:00
prepare-external-git-dependencies: prepare-build
mono --debug $(PREPARE_EXE) $(_PREPARE_ARGS) -s:PrepareExternalGitDependencies
[tests] Check apk tests sizes to avoid regressions (#3594) Fixes: https://github.com/xamarin/xamarin-android/issues/3593 Added new `<ProcessApkSizes/>` task based on the `<ProcessPlotInput/>` task, which takes additional argument with apk sizes reference file. The file is used to check the current apk sizes and fails when the size difference is over a given threshold; we use 100KiB now. There's one additional threshold to warn about smalled differences, with current threshold 50KiB. The reference files can be update with > make update-apk-sizes-reference CONFIGURATION=<Debug or Release> which copies the current apk sizes files to `tests/apk-sizes-reference` directory, these are then used by `<ProcessApkSizes/>` task. The apk sizes files were split by configuration and contain `$(Configuration)` in the name. The reason for that is an easier update of the references after failed PR build, after the size difference is checked to be valid and desired. This way it is possible to just download the results artifacts file and copy the sizes recorded by CI build into the reference directory.
2019-09-07 02:16:34 +03:00
APK_SIZES_REFERENCE_DIR=tests/apk-sizes-reference
update-apk-sizes-reference:
-mkdir -p $(APK_SIZES_REFERENCE_DIR)
cp -v *values-$(CONFIGURATION).csv $(APK_SIZES_REFERENCE_DIR)/