2020-03-16 11:30:14 +03:00
- i n c l u d e b i n / c o n f i g u r a t i o n . m k
2016-05-04 15:03:01 +03:00
V ?= 0
2018-06-25 23:07:56 +03:00
prefix = /usr/local
2020-03-16 11:30:14 +03:00
CONFIGURATION ?= Debug
2018-06-25 23:07:56 +03:00
RUNTIME := $( shell which mono64 2> /dev/null && echo mono64 || echo mono) --debug= casts
SOLUTION = Xamarin.Android.sln
TEST_TARGETS = build-tools/scripts/RunTests.targets
2018-03-01 17:09:01 +03:00
API_LEVEL ?=
[build] Xamarin.Android build preparation utility (#3051)
Fixes: https://github.com/xamarin/xamarin-android/issues/2562
The `make prepare` **make** target and `msbuild /t:Prepare` MSBuild
target are used to "prepare" a `xamarin-android` checkout so that
the subsequent `make all` or `msbuild /t:Build` invocation will work.
The preparation step involves ensuring that all known required
dependencies exist or are provisioned (54a7a029), such as the build
utilities (`autoconf`/etc.), cross-compilers (MinGW), the Android SDK
and NDK, and building "enough" to allow `Xamarin.Android.sln` to
build (see also 7343965a0).
Unfortunately, the Glorious Idea™ of using MSBuild for significant
portions of this process is "baroque", making it difficult to
understand and maintain, and the requirement on `msbuild` means
that things break in really bizarre and obscure ways on Linux and
macOS if the installed `mono` version is "too old"...and the
determination of "too old" was being done via `mono` & `msbuild`!
Furthermore, there was duplication and repetition of various bits of
build configuration on Linux and macOS between Makefile and the
MSBuild project system, which made it easier to make mistakes.
(Then there's the occasional bizarro failures that would result from
attempting to evaluate `$(PRODUCT_VERSION)` within `make`; it would
fail when `MONO_LOG_LEVEL=info` was set. Then there's...)
Instead of having `make prepare` be implemented as an unholy mess of
`make` and `msbuild`, introduce the new `build-tools/xaprepare` app.
`xaprepare` will be built and executed as part of `make prepare`,
and takes over responsibility for repo build preparation.
See also: build-tools/xaprepare/README.md
TODO: Windows support
2019-06-07 20:58:59 +03:00
PREPARE_ARGS =
PREPARE_BUILD_LOG = bin/Build$( CONFIGURATION) /bootstrap-build.binlog
PREPARE_RESTORE_LOG = bin/Build$( CONFIGURATION) /bootstrap-restore.binlog
PREPARE_SOURCE_DIR = build-tools/xaprepare
PREPARE_SOLUTION = $( PREPARE_SOURCE_DIR) /xaprepare.sln
PREPARE_EXE = $( PREPARE_SOURCE_DIR) /xaprepare/bin/$( CONFIGURATION) /xaprepare.exe
PREPARE_COMMON_MSBUILD_FLAGS = /p:Configuration= $( CONFIGURATION) $( PREPARE_MSBUILD_ARGS) $( MSBUILD_ARGS)
PREPARE_MSBUILD_FLAGS = /binaryLogger:" $( PREPARE_BUILD_LOG) " $( PREPARE_COMMON_MSBUILD_FLAGS)
PREPARE_RESTORE_FLAGS = /binaryLogger:" $( PREPARE_RESTORE_LOG) " $( PREPARE_COMMON_MSBUILD_FLAGS)
PREPARE_SCENARIO =
[build] Simplify CI make and xaprepare invocations (#3279)
* [build] Simplify CI make and xaprepare invocations
Context: https://github.com/xamarin/xamarin-android/commit/9302d514a1437e5d1977da06dfbbe4e7054f33d5
Context: https://github.com/xamarin/xamarin-android/commit/f9f5b6cd120484b13df74e94638e07411d8693fa
We built a lot of hacks into our make file when initially working on the
xaprepare tool, as the windows and mac azure pipeline builds could not
be modified at the time. We now have these builds defined in yaml, and
as such we should be able to simplify the underlying make rules used by
all of our builds.
* Removes `prepare-build-ci` which no longer appears to be relevant,
as the extra args it specifies don't appear to be set before the
tool is built.
* Removes `prepare-jenkins` and `prepare-commercial`, as we can now
rely on the `prepare` rule again. The behavior of `prepare` will
change depending on the `_PREPARE_ARGS` varible, which is controlled
by different flags passed by our CI systems, `azure-pipelines.yaml`,
and `build.groovy`
* Introduces a new `prepareFlags` variable to build.groovy which is
used to control whether or not the "prepare all" (-a arg) should
be passed to xaprepare. This should not be specified when building
Jenkins PR builds, as they only build a subset of all supported ABIs
and frameworks. This should address the EmbeddedDSO test failures we
are encountering in Jenkins PR builds.
* Removes the MSBuild Auto Provision and ABI related args from make
invocations in azure-pipelines.yaml. Make variables will instead be
used to control these options.
* Removes the `prepare-update-mono` dependency from
`prepare-external-git-dependencies` as we should be able to call
this without needing to update mono.
* Adds a `prepare` dependency back to `jenkins`, now that the
`prepare-jenkins` step is no longer required.
* [build] Don't override _MSBUILD_ARGS when creating installers
Context: https://github.com/xamarin/xamarin-android/commit/e83ba0ddc159d15abcfac1a5a4bce5a53e165d3f
The `_MSBUILD_ARGS` override was initially required because we did not
import different `rules.make` based on how our build was configured and
prepared. The `xaprepare` tool now better controls the ABIs we should be
building and packaging for each build type, so this is no longer needed.
2019-06-26 11:08:00 +03:00
PREPARE_CI_PR ?= 0
[build] Xamarin.Android build preparation utility (#3051)
Fixes: https://github.com/xamarin/xamarin-android/issues/2562
The `make prepare` **make** target and `msbuild /t:Prepare` MSBuild
target are used to "prepare" a `xamarin-android` checkout so that
the subsequent `make all` or `msbuild /t:Build` invocation will work.
The preparation step involves ensuring that all known required
dependencies exist or are provisioned (54a7a029), such as the build
utilities (`autoconf`/etc.), cross-compilers (MinGW), the Android SDK
and NDK, and building "enough" to allow `Xamarin.Android.sln` to
build (see also 7343965a0).
Unfortunately, the Glorious Idea™ of using MSBuild for significant
portions of this process is "baroque", making it difficult to
understand and maintain, and the requirement on `msbuild` means
that things break in really bizarre and obscure ways on Linux and
macOS if the installed `mono` version is "too old"...and the
determination of "too old" was being done via `mono` & `msbuild`!
Furthermore, there was duplication and repetition of various bits of
build configuration on Linux and macOS between Makefile and the
MSBuild project system, which made it easier to make mistakes.
(Then there's the occasional bizarro failures that would result from
attempting to evaluate `$(PRODUCT_VERSION)` within `make`; it would
fail when `MONO_LOG_LEVEL=info` was set. Then there's...)
Instead of having `make prepare` be implemented as an unholy mess of
`make` and `msbuild`, introduce the new `build-tools/xaprepare` app.
`xaprepare` will be built and executed as part of `make prepare`,
and takes over responsibility for repo build preparation.
See also: build-tools/xaprepare/README.md
TODO: Windows support
2019-06-07 20:58:59 +03:00
PREPARE_CI ?= 0
PREPARE_AUTOPROVISION ?= 0
PREPARE_IGNORE_MONO_VERSION ?= 1
[build] Simplify CI make and xaprepare invocations (#3279)
* [build] Simplify CI make and xaprepare invocations
Context: https://github.com/xamarin/xamarin-android/commit/9302d514a1437e5d1977da06dfbbe4e7054f33d5
Context: https://github.com/xamarin/xamarin-android/commit/f9f5b6cd120484b13df74e94638e07411d8693fa
We built a lot of hacks into our make file when initially working on the
xaprepare tool, as the windows and mac azure pipeline builds could not
be modified at the time. We now have these builds defined in yaml, and
as such we should be able to simplify the underlying make rules used by
all of our builds.
* Removes `prepare-build-ci` which no longer appears to be relevant,
as the extra args it specifies don't appear to be set before the
tool is built.
* Removes `prepare-jenkins` and `prepare-commercial`, as we can now
rely on the `prepare` rule again. The behavior of `prepare` will
change depending on the `_PREPARE_ARGS` varible, which is controlled
by different flags passed by our CI systems, `azure-pipelines.yaml`,
and `build.groovy`
* Introduces a new `prepareFlags` variable to build.groovy which is
used to control whether or not the "prepare all" (-a arg) should
be passed to xaprepare. This should not be specified when building
Jenkins PR builds, as they only build a subset of all supported ABIs
and frameworks. This should address the EmbeddedDSO test failures we
are encountering in Jenkins PR builds.
* Removes the MSBuild Auto Provision and ABI related args from make
invocations in azure-pipelines.yaml. Make variables will instead be
used to control these options.
* Removes the `prepare-update-mono` dependency from
`prepare-external-git-dependencies` as we should be able to call
this without needing to update mono.
* Adds a `prepare` dependency back to `jenkins`, now that the
`prepare-jenkins` step is no longer required.
* [build] Don't override _MSBUILD_ARGS when creating installers
Context: https://github.com/xamarin/xamarin-android/commit/e83ba0ddc159d15abcfac1a5a4bce5a53e165d3f
The `_MSBUILD_ARGS` override was initially required because we did not
import different `rules.make` based on how our build was configured and
prepared. The `xaprepare` tool now better controls the ABIs we should be
building and packaging for each build type, so this is no longer needed.
2019-06-26 11:08:00 +03:00
_PREPARE_CI_MODE_PR_ARGS = --no-emoji --run-mode= CI
_PREPARE_CI_MODE_ARGS = $( _PREPARE_CI_MODE_PR_ARGS) -a
[build] Xamarin.Android build preparation utility (#3051)
Fixes: https://github.com/xamarin/xamarin-android/issues/2562
The `make prepare` **make** target and `msbuild /t:Prepare` MSBuild
target are used to "prepare" a `xamarin-android` checkout so that
the subsequent `make all` or `msbuild /t:Build` invocation will work.
The preparation step involves ensuring that all known required
dependencies exist or are provisioned (54a7a029), such as the build
utilities (`autoconf`/etc.), cross-compilers (MinGW), the Android SDK
and NDK, and building "enough" to allow `Xamarin.Android.sln` to
build (see also 7343965a0).
Unfortunately, the Glorious Idea™ of using MSBuild for significant
portions of this process is "baroque", making it difficult to
understand and maintain, and the requirement on `msbuild` means
that things break in really bizarre and obscure ways on Linux and
macOS if the installed `mono` version is "too old"...and the
determination of "too old" was being done via `mono` & `msbuild`!
Furthermore, there was duplication and repetition of various bits of
build configuration on Linux and macOS between Makefile and the
MSBuild project system, which made it easier to make mistakes.
(Then there's the occasional bizarro failures that would result from
attempting to evaluate `$(PRODUCT_VERSION)` within `make`; it would
fail when `MONO_LOG_LEVEL=info` was set. Then there's...)
Instead of having `make prepare` be implemented as an unholy mess of
`make` and `msbuild`, introduce the new `build-tools/xaprepare` app.
`xaprepare` will be built and executed as part of `make prepare`,
and takes over responsibility for repo build preparation.
See also: build-tools/xaprepare/README.md
TODO: Windows support
2019-06-07 20:58:59 +03:00
_PREPARE_ARGS =
2019-11-12 13:37:20 +03:00
BOOTSTRAP_SOLUTION = Xamarin.Android.BootstrapTasks.sln
BOOTSTRAP_BUILD_LOG = bin/Build$( CONFIGURATION) /bootstrap-build.binlog
[build] Sanitize Microsoft.Build* assembly references (#3995)
Context: https://github.com/xamarin/monodroid/pull/1048
Context: https://github.com/xamarin/xamarin-android/pull/3928
Attempt to clean up and modernize various projects to help break up
the scope of change required to convert them to .NET Standard libs.
Bump the .NET Core version that `build.groovy` and
`azure-pipelines.yaml` install to 3.1.100. This is to obtain various
NuGet support fixes.
Updates the shared `MSBuildReferences.projitems` file to instead
include NuGet `@(PackageReference)`s to the required
`Microsoft.Build.*.dll` assemblies. The `Mono.Posix.NETStandard`,
`Xamarin.Build.AsyncTask`, and `Xamarin.LibZipSharp` NuGet references
have also been moved into this file to make it easier to update
versions in one place as needed. We're only using the compile time
assets from the MSBuild NuGet packages, which ensures that their
content is not copied to the output directory. An additional `@(None)`
item group has been added to this file to copy the required assemblies
from the system to the output directory so that
`Xamarin.Android.Build.Tests.dll` can run against them. These changes
are required because the MSBuild assemblies coming from NuGet do not
yet fully support running on Mono.
All projects which reference `MSBuildReferences.projitems` have been
converted from the old `packages.config` to the new
`@(PackageReference)` NuGet reference style.
The Azure Pipeline build job has been updated to completely build
`Xamarin.Android-Tests.sln` with `xabuild`, rather than only building
select projects. This was done to mirror our Jenkins build and to
ensure that `make all-tests` continues to work moving forward.
.NET Core and `NuGetCommand` tasks have been bumped to bring in a
newer version of NuGet with better `@(PackageReference)` support.
NuGet restore will be skipped if any issues occur when calculating the
dependency graph for the solution or project that it is operating on.
A lot of our sources attempt to import `.props`/`.targets`/`.projitems`
files that are generated by the build, and these should be
conditionally imported so that NuGet restore attempts can succeed
before these files are generated. This should also help when loading
these projects in VS or VS Mac.
Example NuGet restore error:
src/Xamarin.Android.Build.Tasks/Xamarin.Android.Build.Tasks.targets(8,3): error MSB4019: The imported project "bin/BuildRelease/ProfileAssemblies.projitems" was not found. Confirm that the expression in the Import declaration "../../bin/BuildRelease/ProfileAssemblies.projitems" is correct, and that the file exists on disk.
/var/folders/pp/x88nqyvd0h18ttl996vv2c3w0000gn/T/NuGetScratch/ok90j1ok.3t3.nugetrestore.targets(266,5): warning : Skipping restore for project 'src/Xamarin.Android.Build.Tasks/Xamarin.Android.Build.Tasks.csproj'. The project file may be invalid or missing targets required for restore.
I've also moved the `NuGet.exe` that is copied and used by tests into
its own folder to avoid a crash on macOS which would occur when
`NuGet.exe` is invoked from the same folder as
`Xunit.NetCore.Extensions.dll`:
mono bin/TestRelease/NuGet.exe
VTable setup of type Xunit.NetCore.Extensions.SkippedTestCase failed
System.TypeLoadException: VTable setup of type Xunit.NetCore.Extensions.SkippedTestCase failed
at (wrapper managed-to-native) System.RuntimeType.GetPropertiesByName_native(System.RuntimeType,intptr,System.Reflection.BindingFlags,System.RuntimeType/MemberListType)
at System.RuntimeType.GetPropertiesByName (System.String name, System.Reflection.BindingFlags bindingAttr, System.RuntimeType+MemberListType listType, System.RuntimeType reflectedType)
at System.RuntimeType.GetPropertyCandidates (System.String name, System.Reflection.BindingFlags bindingAttr, System.Type[] types, System.Boolean allowPrefixLookup)
at System.RuntimeType.GetProperties (System.Reflection.BindingFlags bindingAttr)
at System.ComponentModel.Composition.AttributedModel.AttributedPartCreationInfo+<GetExportMembers>d__32.MoveNext ()
at System.Linq.Enumerable.Any[TSource] (System.Collections.Generic.IEnumerable1[T] source)
...
2019-12-16 23:43:31 +03:00
BOOTSTRAP_MSBUILD_FLAGS = /t:Restore,Build /binaryLogger:" $( BOOTSTRAP_BUILD_LOG) " $( PREPARE_COMMON_MSBUILD_FLAGS)
2019-11-12 13:37:20 +03:00
[build] Xamarin.Android build preparation utility (#3051)
Fixes: https://github.com/xamarin/xamarin-android/issues/2562
The `make prepare` **make** target and `msbuild /t:Prepare` MSBuild
target are used to "prepare" a `xamarin-android` checkout so that
the subsequent `make all` or `msbuild /t:Build` invocation will work.
The preparation step involves ensuring that all known required
dependencies exist or are provisioned (54a7a029), such as the build
utilities (`autoconf`/etc.), cross-compilers (MinGW), the Android SDK
and NDK, and building "enough" to allow `Xamarin.Android.sln` to
build (see also 7343965a0).
Unfortunately, the Glorious Idea™ of using MSBuild for significant
portions of this process is "baroque", making it difficult to
understand and maintain, and the requirement on `msbuild` means
that things break in really bizarre and obscure ways on Linux and
macOS if the installed `mono` version is "too old"...and the
determination of "too old" was being done via `mono` & `msbuild`!
Furthermore, there was duplication and repetition of various bits of
build configuration on Linux and macOS between Makefile and the
MSBuild project system, which made it easier to make mistakes.
(Then there's the occasional bizarro failures that would result from
attempting to evaluate `$(PRODUCT_VERSION)` within `make`; it would
fail when `MONO_LOG_LEVEL=info` was set. Then there's...)
Instead of having `make prepare` be implemented as an unholy mess of
`make` and `msbuild`, introduce the new `build-tools/xaprepare` app.
`xaprepare` will be built and executed as part of `make prepare`,
and takes over responsibility for repo build preparation.
See also: build-tools/xaprepare/README.md
TODO: Windows support
2019-06-07 20:58:59 +03:00
all :
Bump to xamarin/Java.Interop/master@56c92c70 (#4376)
Changes: https://github.com/xamarin/Java.Interop/compare/1a086ffd51436ec3bb78467a150e4f9121d57419...56c92c70887c58275917752450126c8c39988aa2
* xamarin/Java.Interop@56c92c7: [build] Remove cecil submodule (#597)
* xamarin/Java.Interop@3091274: [build] Provide a default $(Configuration) value (#612)
* xamarin/Java.Interop@cf3e7c2: [generator] Don't process duplicate reference assemblies (#611)
* xamarin/Java.Interop@f5fa462: [jnienv-gen] Convert to SDK-style (#608)
Of particular note is [xamarin/Java.Interop@56c92c7][0], which
replaces the `mono/cecil` submodule within Java.Interop with the
[`Mono.Cecil` NuGet package][1] in an effort to simplify the
Java.Interop build system.
This simplifies the Java.Interop repo, and we *thought* that since
xamarin-android *doesn't even use* Java.Interop's cecil submodule-built
`Mono.Cecil.dll` -- instead the `Mono.Cecil.dll` from the
"mono archive" is "renamed" to `Xamarin.Android.Cecil.dll` during
`make prepare` (0c9f83b7) -- surely this would be a simple change.
The removal of the cecil submodule also required changing
`ThirdPartyNotice.txt` generation so that the LICENSE for Cecil was
obtained from the mono archive instead of from Java.Interop.
Unfortunately, the integration was a tad more complicated than
anticipated. With the ongoing adoption of MSBuild multi-targeting
and builds against the `netcoreapp3.1` target framework -- commit
e2854ee7 and numerous commits in Java.Interop -- we encountered a
problem with MSBuild semantics: If two `$(TargetFramework)` builds
share the same output directory, the `IncrementalClean` target will
*remove files created by previous builds*, e.g. when e.g.
`Java.Interop/tools/generator.csproj` builds the `netcoreapp3.1`
framework, it will *delete* the `generator.exe` built by the `net472`
framework, which results in subsequent build breaks.
The only path to sanity is to *ensure* that different
`$(TargetFramework)` builds have *completely separate* `$(OutputPath)`
values. The "normal" approach to doing this is for `$(OutputPath)`
to end with `$(TargetFramework)`, which is the case when
`$(AppendTargetFrameworkToOutputPath)`=True (the default).
Unfortunately in xamarin-android we don't want `$(OutputPath)` to end
with `$(TargetFramework)`; we want the build tree structure to mirror
the installation directory structure, which -- at present -- doesn't
mention `$(TargetFramework)` at all.
The solution here is to use "non-overlapping" directories. For
example, in e2854ee7 there are "two" `$(OutputPath)` values:
* `MonoAndroid10.0`: `bin/Debug/lib/xamarin.android/xbuild-frameworks/MonoAndroid/v10.0/Mono.Android.dll`
* `netcoreapp3.1`: `bin/Debug/lib/xamarin.android/xbuild-frameworks/Xamarin.Android.App/netcoreapp3.1/Mono.Android.dll`
The same "non-overlapping directories" strategy needs to be extended
to *all* multi-targeted projects from Java.Interop, *including*
dependencies. Dependencies such as `Xamarin.Android.Cecil.dll`.
Define a new `$(UtilityOutputFullPathCoreApps)` MSBuild property so
that Java.Interop "utility" project builds, when building for the
`netcoreapp3.1` framework, use a *different* `Xamarin.Android.Cecil.dll`
than is used with the `net472`-related builds.
Update `xaprepare` to *create* this new `netcoreapp3.1`-correlated
`Xamarin.Android.Cecil.dll`. It's the same file, just in a different
directory, to prevent "accidental" deletes by `IncrementalClean`.
Even with all that, MSBuild still had other ideas. In particular,
MSBuild wasn't particularly happy about our attempt to use the
`$(UtilityOutputFullPath)` property to "switch" between using a
`@(PackageReference)` to the Mono.Cecil NuGet package vs. using a
`@(Reference)` to the `Xamarin.Android.Cecil.dll` assembly, because
MSBuild *caches* this information somewhere within `obj` directories.
To get MSBuild to re-evaluate it's assembly reference choices, we must
instead replace `msbuild` with `msbuild /restore`.
Which still isn't enough, because some of our MSBuild invocations are
via the `<MSBuild/>` task, within `msbuild`. To get *that* working,
we need to explicitly invoke the `Restore` target through a *separate*
`<MSBuild/>` task invocation. You ***CANNOT*** use
`<MSBuild Targets="Restore;Build" />`, as "obvious" as that may be,
because it [doesn't work reliably][2]. ([Yet.][3])
[0]: https://github.com/xamarin/Java.Interop/commit/56c92c70887c58275917752450126c8c39988aa2
[1]: https://www.nuget.org/packages/Mono.Cecil/0.11.2
[2]: https://github.com/microsoft/msbuild/issues/3000#issuecomment-417675215
[3]: https://github.com/microsoft/msbuild/issues/2811
2020-03-27 15:32:59 +03:00
$( call MSBUILD_BINLOG,all,$( _SLN_BUILD) ) /restore $( MSBUILD_FLAGS) $( SOLUTION)
2021-02-17 20:01:02 +03:00
$( call MSBUILD_BINLOG,all,$( _SLN_BUILD) ) /restore $( MSBUILD_FLAGS) tools/xabuild/xabuild.csproj
[build] Xamarin.Android build preparation utility (#3051)
Fixes: https://github.com/xamarin/xamarin-android/issues/2562
The `make prepare` **make** target and `msbuild /t:Prepare` MSBuild
target are used to "prepare" a `xamarin-android` checkout so that
the subsequent `make all` or `msbuild /t:Build` invocation will work.
The preparation step involves ensuring that all known required
dependencies exist or are provisioned (54a7a029), such as the build
utilities (`autoconf`/etc.), cross-compilers (MinGW), the Android SDK
and NDK, and building "enough" to allow `Xamarin.Android.sln` to
build (see also 7343965a0).
Unfortunately, the Glorious Idea™ of using MSBuild for significant
portions of this process is "baroque", making it difficult to
understand and maintain, and the requirement on `msbuild` means
that things break in really bizarre and obscure ways on Linux and
macOS if the installed `mono` version is "too old"...and the
determination of "too old" was being done via `mono` & `msbuild`!
Furthermore, there was duplication and repetition of various bits of
build configuration on Linux and macOS between Makefile and the
MSBuild project system, which made it easier to make mistakes.
(Then there's the occasional bizarro failures that would result from
attempting to evaluate `$(PRODUCT_VERSION)` within `make`; it would
fail when `MONO_LOG_LEVEL=info` was set. Then there's...)
Instead of having `make prepare` be implemented as an unholy mess of
`make` and `msbuild`, introduce the new `build-tools/xaprepare` app.
`xaprepare` will be built and executed as part of `make prepare`,
and takes over responsibility for repo build preparation.
See also: build-tools/xaprepare/README.md
TODO: Windows support
2019-06-07 20:58:59 +03:00
- i n c l u d e b i n / B u i l d $( CONFIGURATION ) / r u l e s . m k
i f e q ( $( OS_NAME ) , )
export OS_NAME := $( shell uname)
e n d i f
i f e q ( $( OS_ARCH ) , )
export OS_ARCH := $( shell uname -m)
e n d i f
export NO_SUDO ?= false
i f n e q ( $( NO_SUDO ) , f a l s e )
_PREPARE_ARGS += --auto-provisioning-uses-sudo= false
e n d i f
i f n e q ( $( V ) , 0 )
MONO_OPTIONS += --debug
NUGET_VERBOSITY = -Verbosity Detailed
_PREPARE_ARGS += -v:d
e n d i f
[build] Simplify CI make and xaprepare invocations (#3279)
* [build] Simplify CI make and xaprepare invocations
Context: https://github.com/xamarin/xamarin-android/commit/9302d514a1437e5d1977da06dfbbe4e7054f33d5
Context: https://github.com/xamarin/xamarin-android/commit/f9f5b6cd120484b13df74e94638e07411d8693fa
We built a lot of hacks into our make file when initially working on the
xaprepare tool, as the windows and mac azure pipeline builds could not
be modified at the time. We now have these builds defined in yaml, and
as such we should be able to simplify the underlying make rules used by
all of our builds.
* Removes `prepare-build-ci` which no longer appears to be relevant,
as the extra args it specifies don't appear to be set before the
tool is built.
* Removes `prepare-jenkins` and `prepare-commercial`, as we can now
rely on the `prepare` rule again. The behavior of `prepare` will
change depending on the `_PREPARE_ARGS` varible, which is controlled
by different flags passed by our CI systems, `azure-pipelines.yaml`,
and `build.groovy`
* Introduces a new `prepareFlags` variable to build.groovy which is
used to control whether or not the "prepare all" (-a arg) should
be passed to xaprepare. This should not be specified when building
Jenkins PR builds, as they only build a subset of all supported ABIs
and frameworks. This should address the EmbeddedDSO test failures we
are encountering in Jenkins PR builds.
* Removes the MSBuild Auto Provision and ABI related args from make
invocations in azure-pipelines.yaml. Make variables will instead be
used to control these options.
* Removes the `prepare-update-mono` dependency from
`prepare-external-git-dependencies` as we should be able to call
this without needing to update mono.
* Adds a `prepare` dependency back to `jenkins`, now that the
`prepare-jenkins` step is no longer required.
* [build] Don't override _MSBUILD_ARGS when creating installers
Context: https://github.com/xamarin/xamarin-android/commit/e83ba0ddc159d15abcfac1a5a4bce5a53e165d3f
The `_MSBUILD_ARGS` override was initially required because we did not
import different `rules.make` based on how our build was configured and
prepared. The `xaprepare` tool now better controls the ABIs we should be
building and packaging for each build type, so this is no longer needed.
2019-06-26 11:08:00 +03:00
i f n e q ( $( PREPARE_CI_PR ) , 0 )
_PREPARE_ARGS += $( _PREPARE_CI_MODE_PR_ARGS)
e n d i f
[build] Xamarin.Android build preparation utility (#3051)
Fixes: https://github.com/xamarin/xamarin-android/issues/2562
The `make prepare` **make** target and `msbuild /t:Prepare` MSBuild
target are used to "prepare" a `xamarin-android` checkout so that
the subsequent `make all` or `msbuild /t:Build` invocation will work.
The preparation step involves ensuring that all known required
dependencies exist or are provisioned (54a7a029), such as the build
utilities (`autoconf`/etc.), cross-compilers (MinGW), the Android SDK
and NDK, and building "enough" to allow `Xamarin.Android.sln` to
build (see also 7343965a0).
Unfortunately, the Glorious Idea™ of using MSBuild for significant
portions of this process is "baroque", making it difficult to
understand and maintain, and the requirement on `msbuild` means
that things break in really bizarre and obscure ways on Linux and
macOS if the installed `mono` version is "too old"...and the
determination of "too old" was being done via `mono` & `msbuild`!
Furthermore, there was duplication and repetition of various bits of
build configuration on Linux and macOS between Makefile and the
MSBuild project system, which made it easier to make mistakes.
(Then there's the occasional bizarro failures that would result from
attempting to evaluate `$(PRODUCT_VERSION)` within `make`; it would
fail when `MONO_LOG_LEVEL=info` was set. Then there's...)
Instead of having `make prepare` be implemented as an unholy mess of
`make` and `msbuild`, introduce the new `build-tools/xaprepare` app.
`xaprepare` will be built and executed as part of `make prepare`,
and takes over responsibility for repo build preparation.
See also: build-tools/xaprepare/README.md
TODO: Windows support
2019-06-07 20:58:59 +03:00
i f n e q ( $( PREPARE_CI ) , 0 )
_PREPARE_ARGS += $( _PREPARE_CI_MODE_ARGS)
e n d i f
i f n e q ( $( PREPARE_AUTOPROVISION ) , 0 )
_PREPARE_ARGS += --auto-provision= yes --auto-provision-uses-sudo= yes
e n d i f
[android-toolchain] Permit zero-configuration builds.
This might be a suspect idea, but lets see if we can make this work.
[The Joel Test: 12 Steps to Better Code][0] outlines 12 steps
to better code. The first two steps are:
1. Do you use source control?
2. Can you make a build in one step?
github is being used for source control, so (1) is handled, but how
simple can we make (2)? How easy can we make it to build
Xamarin.Android upon a fresh checkout?
The ideal to strive for is simple:
Load Xamarin.Android.sln into your IDE and Build the project.
I *know* we're not going to be able to do this, if only because we're
going to be using git submodules, which will require a separate
`git submodule init` invocation [1].
Knowing we can't reach that level of simplicitly doesn't mean we
shouldn't *try* to reach it for all other parts of the build system.
Which brings us to the Android NDK and SDK. The Android NDK will be
required in order to build native code, such as libmonodroid.so, while
the Android SDK will be required in order to compile
Java Callable Wrappers (née Android Callable Wrappers [2]) and
eventual samples and unit tests.
There are three ways we can deal with the Android NDK and SDK:
1. Complicate the "build" process by requiring that developers go to
the Android SDK Download Page [3], download and install
"somewhere" the required bits, and then configure the
Xamarin.Android build to use these bits.
2. Complicate the "build" process by requiring that developers run
the Xamarin Unified Installer [4], let it install everything
required, then configure the Xamarin.Android build to use those
bits.
3. Painstakingly determine which files are actually required, then
automatically download and extract those files into a "well-known"
location known by the Xamarin.Android build process.
(1) and (2) can be infuriating. Let's give (3) a try. :-)
Add a Xamarin.Android.Tools.BootstrapTasks project which contains
MSBuild tasks to facilitate downloading the Android SDK and NDK files.
Add an android-toolchain project which uses
Xamarin.Android.Tools.BootstrapTasks to download a painstakingly
determined set of files and install them "somewhere".
Unfortunately [5] the "somewhere" to download and install these files
needs to be in a known absolute path, so I've arbitrary decided to
download the files into $(HOME)\android-archives and install them into
$(HOME)\android-toolchain. On windows, this is
%HOMEDRIVE%%HOMEPATH%\android-archives and
%HOMEDRIVE%%HOMEPATH%\android-toolchain.
These locations may be modified by creating a
Configuration.Override.props file; see README.md for details.
TL;DR: This setup is able to magically download the Android NDK and
SDK files and install them for later use in a reasonably overridable
location, all within MSBuild.
[0]: http://www.joelonsoftware.com/articles/fog0000000043.html
[1]: Though maybe there's some MSBuild-fu we can use to address that.
[2]: https://developer.xamarin.com/guides/android/advanced_topics/java_integration_overview/android_callable_wrappers/
[3]: http://developer.android.com/sdk/index.html
[4]: https://www.xamarin.com/download
[5]: Because I couldn't find a reliable way to use $(SolutionDir) when
only building a project, and relative paths would require an
in-tree installation location, which might not work.
2016-04-19 03:33:04 +03:00
2018-05-29 15:13:13 +03:00
i f e q ( $( OS_NAME ) , D a r w i n )
[build] Xamarin.Android build preparation utility (#3051)
Fixes: https://github.com/xamarin/xamarin-android/issues/2562
The `make prepare` **make** target and `msbuild /t:Prepare` MSBuild
target are used to "prepare" a `xamarin-android` checkout so that
the subsequent `make all` or `msbuild /t:Build` invocation will work.
The preparation step involves ensuring that all known required
dependencies exist or are provisioned (54a7a029), such as the build
utilities (`autoconf`/etc.), cross-compilers (MinGW), the Android SDK
and NDK, and building "enough" to allow `Xamarin.Android.sln` to
build (see also 7343965a0).
Unfortunately, the Glorious Idea™ of using MSBuild for significant
portions of this process is "baroque", making it difficult to
understand and maintain, and the requirement on `msbuild` means
that things break in really bizarre and obscure ways on Linux and
macOS if the installed `mono` version is "too old"...and the
determination of "too old" was being done via `mono` & `msbuild`!
Furthermore, there was duplication and repetition of various bits of
build configuration on Linux and macOS between Makefile and the
MSBuild project system, which made it easier to make mistakes.
(Then there's the occasional bizarro failures that would result from
attempting to evaluate `$(PRODUCT_VERSION)` within `make`; it would
fail when `MONO_LOG_LEVEL=info` was set. Then there's...)
Instead of having `make prepare` be implemented as an unholy mess of
`make` and `msbuild`, introduce the new `build-tools/xaprepare` app.
`xaprepare` will be built and executed as part of `make prepare`,
and takes over responsibility for repo build preparation.
See also: build-tools/xaprepare/README.md
TODO: Windows support
2019-06-07 20:58:59 +03:00
i f e q ( $( HOMEBREW_PREFIX ) , )
[build] Use Homebrew mingw-w64, not MXE (#2758)
MXE, the [M cross environment][mxe], is used to build Windows binary
code on Linux and macOS so that we don't need a Windows machine to
produce our Windows `.vsix` files. This has many pros and cons,
including:
* **Pro**: Building *mono* on Windows has historically been
difficult. While there is much better support *now* for building
mono for Windows, we also need to build mono *targeting Android*,
built via the Android NDK toolchain, and @jonpryor *suspects*,
but has not verified, that such a cross-compilation scenario
would still be a terrible experience on Windows.
(This is why Windows builds require a mono bundle; see 29568117.)
* **Pro**: Building both the macOS and Windows commercial
installers on macOS means that, where possible, the *same* files
can be present in both installers. File such as BCL assemblies,
`Mono.Android.dll`, MSBuild targets, etc. Separate macOS &
Windows build machines could introduce the possibility for
"drift" and accidental variations between the systems.
* **Con**: MXE *itself* is a giant pain. See also: b28856f4, in
which we see that MXE requires Xcode 8.3 to build; the current
Xcode version is 10.1, and 8.3 is nearly 2 years old.
* Xcode 8.3 means we can't use the latest-and-greatest Mac mini
hardware for builds, as it won't run Xcode 8.3.
* It can take *tens* of minutes to build MXE...
* ...assuming it builds AT ALL; it frequently fails to build.
Finally, the compilers that MXE provides are *ancient*, providing gcc
5.4.0 (circa 2015), which is 3-4 years old now. This is crimping
@grendello's style in `src/monodroid`, wherein he wants to use C++17
features when reasonable, but gcc 5.4 doesn't support C++17. We also
want to use newer compilers on the assumption that they'll generate
more/better warnings around unsafe code practices, improved stack
overflow checking code, and other security-related concerns.
We thus would like to move away from MXE, but to what?
Turns Out™, [Homebrew][hb] now has a [mingw-w64 formula][mingw] to
install the latest MinGW toolchain with GCC 8.2, which allows us to
drop MXE, and we already depend on Homebrew on macOS (82ce2118,
0c073f67f).
The downside of using this pre-packaged MinGW -- a very insignificant
one -- is that it does not come with any extra libraries or software
prebuilt for Windows, and our build requires `zlib` which was
previously built as part of MXE. This downside is easily worked
around, however: we simply use our own Homebrew tap which has a
formula that installs the Win32/Win64 `zlib` binary. This new tap is
available in [xamarin/homebrew-xamarin-android-windeps][windep].
This commit drops MXE dependency in Xamarin.Android *only*; it does
*not* remove this dependency from Mono. Mono will now provision its
own copy of MXE and use it for builds. This will change when
[mono/mono#12434][m12434] makes its way into Mono and, subsequently,
to xamarin-android.
All the libraries we build for Windows that require `zlib` will link
a *static* version of the library instead of using the dynamic one.
This is safer than using a `.DLL` because we control what version of
`zlib` is used and we leave no potential for breakage should a 3rd
party replace/remove/etc. `zlib1.dll` from the target system.
[mxe]: http://mxe.cc
[hb]: https://brew.sh
[mingw]: https://formulae.brew.sh/formula/mingw-w64
[m12434]: https://github.com/mono/mono/pull/12434
[windep]: https://github.com/xamarin/homebrew-xamarin-android-windeps
2019-02-28 06:33:36 +03:00
HOMEBREW_PREFIX ?= $( shell brew --prefix)
[build] Xamarin.Android build preparation utility (#3051)
Fixes: https://github.com/xamarin/xamarin-android/issues/2562
The `make prepare` **make** target and `msbuild /t:Prepare` MSBuild
target are used to "prepare" a `xamarin-android` checkout so that
the subsequent `make all` or `msbuild /t:Build` invocation will work.
The preparation step involves ensuring that all known required
dependencies exist or are provisioned (54a7a029), such as the build
utilities (`autoconf`/etc.), cross-compilers (MinGW), the Android SDK
and NDK, and building "enough" to allow `Xamarin.Android.sln` to
build (see also 7343965a0).
Unfortunately, the Glorious Idea™ of using MSBuild for significant
portions of this process is "baroque", making it difficult to
understand and maintain, and the requirement on `msbuild` means
that things break in really bizarre and obscure ways on Linux and
macOS if the installed `mono` version is "too old"...and the
determination of "too old" was being done via `mono` & `msbuild`!
Furthermore, there was duplication and repetition of various bits of
build configuration on Linux and macOS between Makefile and the
MSBuild project system, which made it easier to make mistakes.
(Then there's the occasional bizarro failures that would result from
attempting to evaluate `$(PRODUCT_VERSION)` within `make`; it would
fail when `MONO_LOG_LEVEL=info` was set. Then there's...)
Instead of having `make prepare` be implemented as an unholy mess of
`make` and `msbuild`, introduce the new `build-tools/xaprepare` app.
`xaprepare` will be built and executed as part of `make prepare`,
and takes over responsibility for repo build preparation.
See also: build-tools/xaprepare/README.md
TODO: Windows support
2019-06-07 20:58:59 +03:00
e n d i f
[build] Use Homebrew mingw-w64, not MXE (#2758)
MXE, the [M cross environment][mxe], is used to build Windows binary
code on Linux and macOS so that we don't need a Windows machine to
produce our Windows `.vsix` files. This has many pros and cons,
including:
* **Pro**: Building *mono* on Windows has historically been
difficult. While there is much better support *now* for building
mono for Windows, we also need to build mono *targeting Android*,
built via the Android NDK toolchain, and @jonpryor *suspects*,
but has not verified, that such a cross-compilation scenario
would still be a terrible experience on Windows.
(This is why Windows builds require a mono bundle; see 29568117.)
* **Pro**: Building both the macOS and Windows commercial
installers on macOS means that, where possible, the *same* files
can be present in both installers. File such as BCL assemblies,
`Mono.Android.dll`, MSBuild targets, etc. Separate macOS &
Windows build machines could introduce the possibility for
"drift" and accidental variations between the systems.
* **Con**: MXE *itself* is a giant pain. See also: b28856f4, in
which we see that MXE requires Xcode 8.3 to build; the current
Xcode version is 10.1, and 8.3 is nearly 2 years old.
* Xcode 8.3 means we can't use the latest-and-greatest Mac mini
hardware for builds, as it won't run Xcode 8.3.
* It can take *tens* of minutes to build MXE...
* ...assuming it builds AT ALL; it frequently fails to build.
Finally, the compilers that MXE provides are *ancient*, providing gcc
5.4.0 (circa 2015), which is 3-4 years old now. This is crimping
@grendello's style in `src/monodroid`, wherein he wants to use C++17
features when reasonable, but gcc 5.4 doesn't support C++17. We also
want to use newer compilers on the assumption that they'll generate
more/better warnings around unsafe code practices, improved stack
overflow checking code, and other security-related concerns.
We thus would like to move away from MXE, but to what?
Turns Out™, [Homebrew][hb] now has a [mingw-w64 formula][mingw] to
install the latest MinGW toolchain with GCC 8.2, which allows us to
drop MXE, and we already depend on Homebrew on macOS (82ce2118,
0c073f67f).
The downside of using this pre-packaged MinGW -- a very insignificant
one -- is that it does not come with any extra libraries or software
prebuilt for Windows, and our build requires `zlib` which was
previously built as part of MXE. This downside is easily worked
around, however: we simply use our own Homebrew tap which has a
formula that installs the Win32/Win64 `zlib` binary. This new tap is
available in [xamarin/homebrew-xamarin-android-windeps][windep].
This commit drops MXE dependency in Xamarin.Android *only*; it does
*not* remove this dependency from Mono. Mono will now provision its
own copy of MXE and use it for builds. This will change when
[mono/mono#12434][m12434] makes its way into Mono and, subsequently,
to xamarin-android.
All the libraries we build for Windows that require `zlib` will link
a *static* version of the library instead of using the dynamic one.
This is safer than using a `.DLL` because we control what version of
`zlib` is used and we leave no potential for breakage should a 3rd
party replace/remove/etc. `zlib1.dll` from the target system.
[mxe]: http://mxe.cc
[hb]: https://brew.sh
[mingw]: https://formulae.brew.sh/formula/mingw-w64
[m12434]: https://github.com/mono/mono/pull/12434
[windep]: https://github.com/xamarin/homebrew-xamarin-android-windeps
2019-02-28 06:33:36 +03:00
e l s e
HOMEBREW_PREFIX := $prefix
2018-05-29 15:13:13 +03:00
e n d i f
[build] Xamarin.Android build preparation utility (#3051)
Fixes: https://github.com/xamarin/xamarin-android/issues/2562
The `make prepare` **make** target and `msbuild /t:Prepare` MSBuild
target are used to "prepare" a `xamarin-android` checkout so that
the subsequent `make all` or `msbuild /t:Build` invocation will work.
The preparation step involves ensuring that all known required
dependencies exist or are provisioned (54a7a029), such as the build
utilities (`autoconf`/etc.), cross-compilers (MinGW), the Android SDK
and NDK, and building "enough" to allow `Xamarin.Android.sln` to
build (see also 7343965a0).
Unfortunately, the Glorious Idea™ of using MSBuild for significant
portions of this process is "baroque", making it difficult to
understand and maintain, and the requirement on `msbuild` means
that things break in really bizarre and obscure ways on Linux and
macOS if the installed `mono` version is "too old"...and the
determination of "too old" was being done via `mono` & `msbuild`!
Furthermore, there was duplication and repetition of various bits of
build configuration on Linux and macOS between Makefile and the
MSBuild project system, which made it easier to make mistakes.
(Then there's the occasional bizarro failures that would result from
attempting to evaluate `$(PRODUCT_VERSION)` within `make`; it would
fail when `MONO_LOG_LEVEL=info` was set. Then there's...)
Instead of having `make prepare` be implemented as an unholy mess of
`make` and `msbuild`, introduce the new `build-tools/xaprepare` app.
`xaprepare` will be built and executed as part of `make prepare`,
and takes over responsibility for repo build preparation.
See also: build-tools/xaprepare/README.md
TODO: Windows support
2019-06-07 20:58:59 +03:00
i f e q ( $( wildcard Configuration .OperatingSystem .props ) , )
PREPARE_MSBUILD_FLAGS += " /p:HostHomebrewPrefix= $( HOMEBREW_PREFIX) "
2016-05-04 15:03:01 +03:00
e n d i f
[build] Xamarin.Android build preparation utility (#3051)
Fixes: https://github.com/xamarin/xamarin-android/issues/2562
The `make prepare` **make** target and `msbuild /t:Prepare` MSBuild
target are used to "prepare" a `xamarin-android` checkout so that
the subsequent `make all` or `msbuild /t:Build` invocation will work.
The preparation step involves ensuring that all known required
dependencies exist or are provisioned (54a7a029), such as the build
utilities (`autoconf`/etc.), cross-compilers (MinGW), the Android SDK
and NDK, and building "enough" to allow `Xamarin.Android.sln` to
build (see also 7343965a0).
Unfortunately, the Glorious Idea™ of using MSBuild for significant
portions of this process is "baroque", making it difficult to
understand and maintain, and the requirement on `msbuild` means
that things break in really bizarre and obscure ways on Linux and
macOS if the installed `mono` version is "too old"...and the
determination of "too old" was being done via `mono` & `msbuild`!
Furthermore, there was duplication and repetition of various bits of
build configuration on Linux and macOS between Makefile and the
MSBuild project system, which made it easier to make mistakes.
(Then there's the occasional bizarro failures that would result from
attempting to evaluate `$(PRODUCT_VERSION)` within `make`; it would
fail when `MONO_LOG_LEVEL=info` was set. Then there's...)
Instead of having `make prepare` be implemented as an unholy mess of
`make` and `msbuild`, introduce the new `build-tools/xaprepare` app.
`xaprepare` will be built and executed as part of `make prepare`,
and takes over responsibility for repo build preparation.
See also: build-tools/xaprepare/README.md
TODO: Windows support
2019-06-07 20:58:59 +03:00
i f n e q ( $( PREPARE_SCENARIO ) , )
_PREPARE_ARGS += -s:" $( PREPARE_SCENARIO) "
2016-05-04 15:03:01 +03:00
e n d i f
2019-06-18 17:32:21 +03:00
i f e q ( $( XA_FORCE_COMPONENT_REFRESH ) , t r u e )
_PREPARE_ARGS += -refresh
e n d i f
[build] Xamarin.Android build preparation utility (#3051)
Fixes: https://github.com/xamarin/xamarin-android/issues/2562
The `make prepare` **make** target and `msbuild /t:Prepare` MSBuild
target are used to "prepare" a `xamarin-android` checkout so that
the subsequent `make all` or `msbuild /t:Build` invocation will work.
The preparation step involves ensuring that all known required
dependencies exist or are provisioned (54a7a029), such as the build
utilities (`autoconf`/etc.), cross-compilers (MinGW), the Android SDK
and NDK, and building "enough" to allow `Xamarin.Android.sln` to
build (see also 7343965a0).
Unfortunately, the Glorious Idea™ of using MSBuild for significant
portions of this process is "baroque", making it difficult to
understand and maintain, and the requirement on `msbuild` means
that things break in really bizarre and obscure ways on Linux and
macOS if the installed `mono` version is "too old"...and the
determination of "too old" was being done via `mono` & `msbuild`!
Furthermore, there was duplication and repetition of various bits of
build configuration on Linux and macOS between Makefile and the
MSBuild project system, which made it easier to make mistakes.
(Then there's the occasional bizarro failures that would result from
attempting to evaluate `$(PRODUCT_VERSION)` within `make`; it would
fail when `MONO_LOG_LEVEL=info` was set. Then there's...)
Instead of having `make prepare` be implemented as an unholy mess of
`make` and `msbuild`, introduce the new `build-tools/xaprepare` app.
`xaprepare` will be built and executed as part of `make prepare`,
and takes over responsibility for repo build preparation.
See also: build-tools/xaprepare/README.md
TODO: Windows support
2019-06-07 20:58:59 +03:00
_PREPARE_ARGS += $( PREPARE_ARGS)
2017-06-27 23:31:22 +03:00
i n c l u d e b u i l d - t o o l s / s c r i p t s / m s b u i l d . m k
[Xamarin.Android.Build.Utilities] Add AndroidVersions (#599)
Commit 8e7d37bb is a sign of duplication: in order to use a
`Mono.Android.dll` binding assembly, not only do we need to *build*
the binding assembly, but we *also* need to update
`Xamarin.Android.Build.Utilities.dll` to "know" about the new binding
version (to map API level to `$(TargetFrameworkVersion)`).
Even "better" (worse), if the new API level is a *preview*, there is
no *consistent* API level. For example, with API-O, `android.jar`
isn't at
`$(AndroidSdkDirectory)\platforms\android-@API_LEVEL@\android.jar`,
where `@API_LEVEL@` is 26 (because various codepaths require that the
"api level" be an integer). Instead, it's installed at
`$(AndroidSdkDirectory)\platforms\android-O\android.jar`, where
`O` is the "id" of the preview API level.
This "id" is "leaky", in turn requiring that
`Xamarin.Android.Build.Tasks.dll` *also* be updated to deal with the
mappings.
Even "better" (worse), if we *forget* to cross all our our 't's and
dot all of our 'i's, we'll have a binding assembly which can't be
used. (Which is why we needed commit 8e7d37bb; without it, the API-O
binding can't be used!)
This is all obviously madness. ;-)
Clean this mess up:
1. Update `src/Mono.Android` to create a new `AndroidApiInfo.xml`
file within the `$(TargetFrameworkVersion)` directory.
This will contain all the information needed to map Android API
levels to Ids and Android OS versions and
`$(TargetFrameworkVersion)` values.
```xml
<AndroidApiInfo>
<Id>10</Id>
<Level>10</Level>
<Name>Gingerbread</Name>
<Version>v2.3</Version>
</AndroidApiInfo>
```
2. Add a new `Xamarin.Android.Build.Utilities.AndroidVersions` type
which looks for and parses these new `AndroidApiInfo.xml` files.
3. Fixup all the other places using `AndroidVersion.KnownVersions`
and related members to instead use `AndroidVersions`.
4. Remove all the old APIs which rely on hardcoded data.
The advantage to all this is that we can support new API level
bindings by just building a new `Mono.Android.dll` and placing an
`AndroidApiInfo.xml` into the appropriate location (next to
`Mono.Android.dll`). No further code changes would be required.
Related: The build system still has a nasy habit of using system-wide
directories to resolve files we'd really rather it not, e.g. in
[xamarin-android/master build #503][m503]:
[m503]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android/503/consoleText
Building target "_GetReferenceAssemblyPaths" in project "/Users/builder/jenkins/workspace/xamarin-android/xamarin-android/src/Xamarin.Android.NUnitLite/Xamarin.Android.NUnitLite.csproj" ("/Users/builder/jenkins/workspace/xamarin-android/xamarin-android/bin/Debug/lib/xamarin.android/xbuild/Xamarin/Android/Xamarin.Android.Common.targets"); "_SetLatestTargetFrameworkVersion" depends on it.
Target _GetReferenceAssemblyPaths:
Task "GetReferenceAssemblyPaths"
Using task GetReferenceAssemblyPaths from Microsoft.Build.Tasks.GetReferenceAssemblyPaths, Microsoft.Build.Tasks.Core, Version=14.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
Looking for framework 'MonoAndroid,Version=v1.0' in root path '/Library/Frameworks/Mono.framework/External/xbuild-frameworks'
Found framework definition list '/Library/Frameworks/Mono.framework/External/xbuild-frameworks/MonoAndroid/v1.0/RedistList/FrameworkList.xml' for framework 'MonoAndroid,Version=v1.0'
Done executing task "GetReferenceAssemblyPaths"
Done building target "_GetReferenceAssemblyPaths" in project "/Users/builder/jenkins/workspace/xamarin-android/xamarin-android/src/Xamarin.Android.NUnitLite/Xamarin.Android.NUnitLite.csproj".
Done building target "_GetReferenceAssemblyPaths" in project "/Users/builder/jenkins/workspace/xamarin-android/xamarin-android/src/Xamarin.Android.NUnitLite/Xamarin.Android.NUnitLite.csproj" ("/Users/builder/jenkins/workspace/xamarin-android/xamarin-android/bin/Debug/lib/xamarin.android/xbuild/Xamarin/Android/Xamarin.Android.Common.targets"); "_SetLatestTargetFrameworkVersion" depends on it.
Target _SetLatestTargetFrameworkVersion:
Task "ResolveSdks"
ReferenceAssemblyPaths:
/Library/Frameworks/Mono.framework/External/xbuild-frameworks/MonoAndroid/v1.0/
Note the use of the path
`/Library/Frameworks/Mono.framework/External/xbuild-frameworks/MonoAndroid/v1.0`.
This *was* "fine" -- if undesirable -- but with the introduction of
`AndroidVersions` and `AndroidApiInfo.xml` files within the frameworks
directory, everything falls apart, becuase the system install
*does not have* the `AndroidApiInfo.xml` files, so
*NO API LEVELS ARE FOUND*.
Oops.
The only way to fix this is to use `$XBUILD_FRAMEWORK_FOLDERS_PATH`,
and the sanest way to do *that* is to use `tools/scripts/xabuild` for
the primary build.
Unfortunately, using `xabuild` for the primary build breaks the
`msbuild`-based build, as it can't resolve the
`.NETFramework,Version=v4.6.1` framework.
Context: https://github.com/xamarin/xamarin-android/pull/599#issuecomment-319801330
Split the difference here by introducing `$(_SLN_BUILD)`: use
`xabuild` when building with `xbuild`, and continue using `msbuild`
when building with `msbuild`.
2017-08-23 14:19:11 +03:00
i f e q ( $( USE_MSBUILD ) , 1 )
_SLN_BUILD = $( MSBUILD)
e l s e # $(MSBUILD) != 1
_SLN_BUILD = MSBUILD = " $( MSBUILD) " tools/scripts/xabuild
e n d i f # $(USE_MSBUILD) == 1
2018-03-01 17:09:01 +03:00
i f n e q ( $( API_LEVEL ) , )
[build] Xamarin.Android build preparation utility (#3051)
Fixes: https://github.com/xamarin/xamarin-android/issues/2562
The `make prepare` **make** target and `msbuild /t:Prepare` MSBuild
target are used to "prepare" a `xamarin-android` checkout so that
the subsequent `make all` or `msbuild /t:Build` invocation will work.
The preparation step involves ensuring that all known required
dependencies exist or are provisioned (54a7a029), such as the build
utilities (`autoconf`/etc.), cross-compilers (MinGW), the Android SDK
and NDK, and building "enough" to allow `Xamarin.Android.sln` to
build (see also 7343965a0).
Unfortunately, the Glorious Idea™ of using MSBuild for significant
portions of this process is "baroque", making it difficult to
understand and maintain, and the requirement on `msbuild` means
that things break in really bizarre and obscure ways on Linux and
macOS if the installed `mono` version is "too old"...and the
determination of "too old" was being done via `mono` & `msbuild`!
Furthermore, there was duplication and repetition of various bits of
build configuration on Linux and macOS between Makefile and the
MSBuild project system, which made it easier to make mistakes.
(Then there's the occasional bizarro failures that would result from
attempting to evaluate `$(PRODUCT_VERSION)` within `make`; it would
fail when `MONO_LOG_LEVEL=info` was set. Then there's...)
Instead of having `make prepare` be implemented as an unholy mess of
`make` and `msbuild`, introduce the new `build-tools/xaprepare` app.
`xaprepare` will be built and executed as part of `make prepare`,
and takes over responsibility for repo build preparation.
See also: build-tools/xaprepare/README.md
TODO: Windows support
2019-06-07 20:58:59 +03:00
MSBUILD_FLAGS += /p:AndroidApiLevel= $( API_LEVEL) /p:AndroidFrameworkVersion= $( word $( API_LEVEL) , $( ALL_FRAMEWORKS) ) /p:AndroidPlatformId= $( word $( a) , $( ALL_PLATFORM_IDS) )
2018-03-01 17:09:01 +03:00
e n d i f
[Xamarin.Android-Tests] Split out "full" apps (#146)
We're trying to get [`make jenkins`][0] working on Jenkins, and
[it's failing][1], as one might expect when a particular repo and
associated build system has never been run on Jenkins before:
Android.App/ApplicationTest.cs(9,7): error CS0246: The type or namespace name `NUnit' could not be found. Are you missing an assembly reference?
This error occurs while building
`src/Mono.Android/Test/Mono.Android-Tests.csproj`, and happens because
the `Xamarin.Android.NUnitLite.dll` assembly isn't referenced...
because it isn't *found*:
Microsoft.Common.targets: warning : Reference 'Xamarin.Android.NUnitLite' not resolved
This returns us to a long-standing issue which I thought was mentioned
in a commit somewhere, but I can't find at present:
MSBuild doesn't support updating the assembly resolution directories
*while MSBuild is running*.
For example, we build `Xamarin.Android.NUniteLite.dll` into
`bin/$(Configuration/lib/xbuild-frameworks/MonoAndroid/v1.0`,
but "normal Xamarin.Android referencing projects" don't use a
`@(ProjectReference)` to `Xamarin.Android.NUnitLite.csproj`, but
instead a `@(Reference)` to the assembly:
<!-- src/Mono.Android/Test/Mono.Android-Tests.csproj -->
<Reference Include="Xamarin.Android.NUnitLite" />
The *requirement* for a "proper" Xamarin.Android SDK install is that
`Xamarin.Android.NUnitLite.dll` can be found through a normal
`@(Reference)`. In order to satisfy this requirement, we need to tell
MSBuild where to find it, which can be with `xbuild` via the
`$MSBuildExtensionsPath` and `$XBUILD_FRAMEWORK_FOLDERS_PATH`
*environment variables*.
*Environment variables*.
MSBuild doesn't provide a way to update environment variables, short
of writing a new Task which calls
`Environment.SetEnvironmentVariable()`, and while [this works][2], it
doesn't *actually* work [^3].
The short of all this is that it isn't possible,
*within a single `xbuild` invocation*, to both build the
Xamarin.Android SDK "environment" *and use that environment* as
intended for "normal" apps.
The fix, as is often the case, is to bend with the wind. Instead of
requiring the impossible, move
`src/Mono.Android/Test/Mono.Android-Tests.csproj` into a *new*
`Xamarin.Android-Tests.sln` solution, *out of* `Xamarin.Android.sln`.
This allows building `Xamarin.Android.sln` without error in a pristine
environment -- that is, one that doesn't already have a system-wide
Xamarin.Android install -- and separately building the tests by using
`tools/scripts/xabuild`, which *can* export environment variables to
manipulate `xbuild` behavior so that things Just Work™.
Building `Mono.Android-Tests.csproj` and similar projects
(`HelloWorld.csproj`!) can be done by using the new `make all-tests`
make target.
[0]: https://github.com/xamarin/xamarin-android/commit/a16673d3eb2c4945c3a74f5f8154603d7658fc9a
[1]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android/20/console
[2]: https://github.com/xamarin/xamarin-android/pull/147
[^3]: [PR #147][2] isn't viable because of [xbuild's `AsssemblyResolver`][4].
There's no way to clear/invalidate `target_framework_cache`.
The idea of PR #147 was to "hack things up" so that
`Xamarin.Android.NuniteLite.dll` would be properly resolved during
the build of `Mono.Android-Tests.csproj` *when building everything*.
The problem is *this can't work*, because `xbuild` has a
"target framework cache," with no way to invalidate it, and the
cache is populated the first time the target framework is used.
Due to build ordering issues, this first use is *before*
`Xamarin.Android.NunitLite.dll` was built, and thus it doesn't
exist in the cache. The result:
Task "ResolveAssemblyReference"
....
TargetFrameworkDirectories:
/Volumes/Seagate4TB/work/xamarin-android/bin/Debug/lib/xbuild-frameworks/MonoAndroid/v7.0
/Volumes/Seagate4TB/work/xamarin-android/bin/Debug/lib/xbuild-frameworks/MonoAndroid/v1.0
/Volumes/Seagate4TB/work/xamarin-android/bin/Debug/lib/xbuild-frameworks/MonoAndroid/v1.0/Facades/
...
Primary Reference System.Xml
Reference System.Xml resolved to /Volumes/Seagate4TB/work/xamarin-android/bin/Debug/lib/xbuild-frameworks/MonoAndroid/v1.0/System.Xml.dll. CopyLocal = False
...
Primary Reference Xamarin.Android.NUnitLite
Microsoft.Common.targets: warning : Reference 'Xamarin.Android.NUnitLite' not resolved
For searchpath {TargetFrameworkDirectory}
Considered target framework dir /Volumes/Seagate4TB/work/xamarin-android/bin/Debug/lib/xbuild-frameworks/MonoAndroid/v7.0, assembly named 'Xamarin.Android.NUnitLite' not found.
Considered target framework dir /Volumes/Seagate4TB/work/xamarin-android/bin/Debug/lib/xbuild-frameworks/MonoAndroid/v1.0, assembly named 'Xamarin.Android.NUnitLite' not found.
Considered target framework dir /Volumes/Seagate4TB/work/xamarin-android/bin/Debug/lib/xbuild-frameworks/MonoAndroid/v1.0/Facades/, assembly named 'Xamarin.Android.NUnitLite' not found.
...
Consequently, the `mcs` invocation is missing a
`/reference:path/to/Xamarin.Android.NUniteLite.dll`, and
compilation fails:
Android.App/ApplicationTest.cs(9,7): error CS0246: The type or namespace name `NUnit' could not be found. Are you missing an assembly reference?
...plus 29 others...
[4]: https://github.com/mono/mono/blob/dd8aadf/mcs/class/Microsoft.Build.Tasks/Microsoft.Build.Tasks/AssemblyResolver.cs#L131
2016-08-08 23:46:13 +03:00
all-tests ::
2020-04-15 19:17:22 +03:00
MSBUILD = " $( MSBUILD) " $( call MSBUILD_BINLOG,all-tests,tools/scripts/xabuild) /restore $( MSBUILD_FLAGS) Xamarin.Android-Tests.sln
2017-07-03 20:09:22 +03:00
2020-07-21 04:13:47 +03:00
pack-dotnet ::
$( call MSBUILD_BINLOG,pack-dotnet,$( _SLN_BUILD) ) $( MSBUILD_FLAGS) Xamarin.Android.sln /t:PackDotNet
2017-07-03 20:09:22 +03:00
install ::
@if [ ! -d " bin/ $( CONFIGURATION) " ] ; then \
echo "run 'make all' before you execute 'make install'!" ; \
exit 1; \
fi
[structure] Rework installation directory structure (#704)
Context: https://github.com/xamarin/xamarin-android/pull/253#discussion_r82862993
The *intention* is that Jenkins-produced `oss-xamarin.android*.zip`
artifacts be usable on Windows, so that side-by-side testing can be
performed without replacing the system installation. Usage is in
[UsingJenkinsBuildArtifacts.md](Documentation/UsingJenkinsBuildArtifacts).
This isn't *entirely* the case. It was *apparently* the case at the
time of commit 87ca2737, but things have bitrotten since. For
example, following the 87ca2737 instructions would currently result
in an XA0020 `Could not find monodroid` error, because
`class-parse.exe` wasn't where Windows expects it to be (it was in
`lib/mandroid`, not `lib/xbuild/Xamarin/Android`).
This needs to be fixed.
Additionally, PR #253 mentions that, for filesystem organization, it
would be useful if the macOS/Linux directory structure --
`$prefix/bin`, `$prefix/lib/mandroid`,
`$prefix/lib/xbuild/Xamarin/Android`, `$prefix/lib/xbuild-frameworks`
-- more closely resembled the Windows directory structure of
`$MSBuild` and `$ReferenceAssemblies` (as seen in `.vsix` files).
This would turn macOS/Linux into using `$xa_prefix/xbuild` and
`$xa_prefix/xbuild-frameworks` directories.
`$prefix/bin` would only contain `xabuild`. What is currently in
`$prefix/lib/mandroid` would be merged with
`$xa_prefix/xbuild/Xamarin/Android`.
`$xa_prefix` would `$prefix/lib/xamarin.android`.
This would turn the current macOS structure:
$prefix/bin/xabuild
$prefix/bin/generator
$prefix/bin/cross-arm
$prefix/lib/mandroid/generator.exe
$prefix/lib/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll
$prefix/lib/xbuild/Xamarin/Android/Xamarin.Android.Common.targets
Into:
$prefix/bin/xabuild
$prefix/lib/xamarin.android/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll
$prefix/lib/xamarin.android/xbuild/Xamarin/Android/generator.exe
$prefix/lib/xamarin.android/xbuild/Xamarin/Android/Xamarin.Android.Common.targets
$prefix/lib/xamarin.android/xbuild/Xamarin/Android/Darwin/cross-arm
Other notes:
* The `bundle-*.zip` filename has been has been changed to include a
*hash* of the contents of various files, in particular
`build-tools\mono-runtimes.*`. This was instigated via a
conversation with @kumpera about making the filename more
"idiot-proof": `mono-runtimes.props` contains *compiler flags*,
and if any of those change, then *logically* the bundle should
differ as well, as the mono runtimes may differ in significant
ways. In theory, the `-vXX` version should be used to track this,
but this is a manual change, easy to overlook. The new `-hHASH`
part of the filename should be more automagic.
The new `<HashFileContents/>` task in `xa-prep-tasks.dll` is
responsible for creating the hash value.
* `Configuration.Java.Interop.Override.props` was moved into
`build-tools/scripts`, because that would cleanup the root
directory a bit.
* OS-specific binaries are now placed into
`$prefix/lib/xamarin.android/xbuild/Xamarin/Android/$(HostOS)`.
On a theoretical plus side, this means that the same build
directory can contain OS-specific binaries for multiple operating
systems. (I don't know if anyone shares a build tree between e.g.
macOS and Linux, but if anyone *does*...)
Unfortunately this requires a workaround for an `xbuild` bug:
`%(_LlvmRuntime.InstallPath)` and
`%(_MonoCrossRuntime.InstallPath)` *cannot* use MSBuild
properties, e.g. `<InstallPath>$(HostOS)/</InstallPath>` doesn't
work as desired; it's instead treated literally.
Special-case `%(InstallPath)` until we fully migrate to MSBuild.
* `$(MonoAndroidToolsDirectory)` should be considered *dead*, along
with `$(MonoAndroidBinDirectory)`, as these should now *always*
be the same directory as where `Xamarin.Android.Build.Tasks.dll`
is located, or a `$(HostOS)` sub-directory.
* `Xamarin.ProjectTools.csproj` needed to be updated to ensure that
the build order was correct.
* Remove all `[Obsolete]` and unreferenced members from
`Xamarin.Android.Build.Utilities.dll`. There's too much in there,
and it makes my head hurt trying to understand the
interrelationships between it all. If it's not used, it's gone.
* The changes to `src/monodroid/jni/Android.mk` are...weird. The
removal of `-I$(topdir)/libmonodroid/zip`/etc. is to reduce use of
non-existent paths, as `$(topdir)` isn't defined, so that's
*actually* `-I/libmonodroid/zip`, which is nonsensical. So far,
so good. What's *odd* is the addition of `$(LOCAL_PATH)` to
`$(LOCAL_C_INCLUDES)`: This is needed so that
`external/mono/support/zlib-helper.c` exports `CreateZStream` and
related symbols, otherwise we get a unit test failure in
`GzipStreamTest.Compression` due to an
`EntryPointNotFoundException`, because `CreateZStream` isn't
exported/public.
What's odd here is that I don't understand what caused this
behavior to change. Previous builds exported `CreateZStream`,
otherwise the tests would fail, and I don't understand how any of
the other changes in this PR would be at fault, though that's
certainly the most plausible explanation.
Regardless, `-Ijni` *first* (due to adding `$(LOCAL_PATH)` to
`$(LOCAL_C_INCLUDES)`) is the desired behavior, so that
`jni/config.h` is included, thus ensuring that `MONO_API` has the
required definition when building `zlib-helper.c`.
2017-07-28 16:36:32 +03:00
-mkdir -p " $( prefix) /bin "
2017-07-03 20:09:22 +03:00
-mkdir -p " $( prefix) /lib/mono/xbuild-frameworks "
-mkdir -p " $( prefix) /lib/xamarin.android "
-mkdir -p " $( prefix) /lib/mono/xbuild/Xamarin/ "
[structure] Rework installation directory structure (#704)
Context: https://github.com/xamarin/xamarin-android/pull/253#discussion_r82862993
The *intention* is that Jenkins-produced `oss-xamarin.android*.zip`
artifacts be usable on Windows, so that side-by-side testing can be
performed without replacing the system installation. Usage is in
[UsingJenkinsBuildArtifacts.md](Documentation/UsingJenkinsBuildArtifacts).
This isn't *entirely* the case. It was *apparently* the case at the
time of commit 87ca2737, but things have bitrotten since. For
example, following the 87ca2737 instructions would currently result
in an XA0020 `Could not find monodroid` error, because
`class-parse.exe` wasn't where Windows expects it to be (it was in
`lib/mandroid`, not `lib/xbuild/Xamarin/Android`).
This needs to be fixed.
Additionally, PR #253 mentions that, for filesystem organization, it
would be useful if the macOS/Linux directory structure --
`$prefix/bin`, `$prefix/lib/mandroid`,
`$prefix/lib/xbuild/Xamarin/Android`, `$prefix/lib/xbuild-frameworks`
-- more closely resembled the Windows directory structure of
`$MSBuild` and `$ReferenceAssemblies` (as seen in `.vsix` files).
This would turn macOS/Linux into using `$xa_prefix/xbuild` and
`$xa_prefix/xbuild-frameworks` directories.
`$prefix/bin` would only contain `xabuild`. What is currently in
`$prefix/lib/mandroid` would be merged with
`$xa_prefix/xbuild/Xamarin/Android`.
`$xa_prefix` would `$prefix/lib/xamarin.android`.
This would turn the current macOS structure:
$prefix/bin/xabuild
$prefix/bin/generator
$prefix/bin/cross-arm
$prefix/lib/mandroid/generator.exe
$prefix/lib/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll
$prefix/lib/xbuild/Xamarin/Android/Xamarin.Android.Common.targets
Into:
$prefix/bin/xabuild
$prefix/lib/xamarin.android/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll
$prefix/lib/xamarin.android/xbuild/Xamarin/Android/generator.exe
$prefix/lib/xamarin.android/xbuild/Xamarin/Android/Xamarin.Android.Common.targets
$prefix/lib/xamarin.android/xbuild/Xamarin/Android/Darwin/cross-arm
Other notes:
* The `bundle-*.zip` filename has been has been changed to include a
*hash* of the contents of various files, in particular
`build-tools\mono-runtimes.*`. This was instigated via a
conversation with @kumpera about making the filename more
"idiot-proof": `mono-runtimes.props` contains *compiler flags*,
and if any of those change, then *logically* the bundle should
differ as well, as the mono runtimes may differ in significant
ways. In theory, the `-vXX` version should be used to track this,
but this is a manual change, easy to overlook. The new `-hHASH`
part of the filename should be more automagic.
The new `<HashFileContents/>` task in `xa-prep-tasks.dll` is
responsible for creating the hash value.
* `Configuration.Java.Interop.Override.props` was moved into
`build-tools/scripts`, because that would cleanup the root
directory a bit.
* OS-specific binaries are now placed into
`$prefix/lib/xamarin.android/xbuild/Xamarin/Android/$(HostOS)`.
On a theoretical plus side, this means that the same build
directory can contain OS-specific binaries for multiple operating
systems. (I don't know if anyone shares a build tree between e.g.
macOS and Linux, but if anyone *does*...)
Unfortunately this requires a workaround for an `xbuild` bug:
`%(_LlvmRuntime.InstallPath)` and
`%(_MonoCrossRuntime.InstallPath)` *cannot* use MSBuild
properties, e.g. `<InstallPath>$(HostOS)/</InstallPath>` doesn't
work as desired; it's instead treated literally.
Special-case `%(InstallPath)` until we fully migrate to MSBuild.
* `$(MonoAndroidToolsDirectory)` should be considered *dead*, along
with `$(MonoAndroidBinDirectory)`, as these should now *always*
be the same directory as where `Xamarin.Android.Build.Tasks.dll`
is located, or a `$(HostOS)` sub-directory.
* `Xamarin.ProjectTools.csproj` needed to be updated to ensure that
the build order was correct.
* Remove all `[Obsolete]` and unreferenced members from
`Xamarin.Android.Build.Utilities.dll`. There's too much in there,
and it makes my head hurt trying to understand the
interrelationships between it all. If it's not used, it's gone.
* The changes to `src/monodroid/jni/Android.mk` are...weird. The
removal of `-I$(topdir)/libmonodroid/zip`/etc. is to reduce use of
non-existent paths, as `$(topdir)` isn't defined, so that's
*actually* `-I/libmonodroid/zip`, which is nonsensical. So far,
so good. What's *odd* is the addition of `$(LOCAL_PATH)` to
`$(LOCAL_C_INCLUDES)`: This is needed so that
`external/mono/support/zlib-helper.c` exports `CreateZStream` and
related symbols, otherwise we get a unit test failure in
`GzipStreamTest.Compression` due to an
`EntryPointNotFoundException`, because `CreateZStream` isn't
exported/public.
What's odd here is that I don't understand what caused this
behavior to change. Previous builds exported `CreateZStream`,
otherwise the tests would fail, and I don't understand how any of
the other changes in this PR would be at fault, though that's
certainly the most plausible explanation.
Regardless, `-Ijni` *first* (due to adding `$(LOCAL_PATH)` to
`$(LOCAL_C_INCLUDES)`) is the desired behavior, so that
`jni/config.h` is included, thus ensuring that `MONO_API` has the
required definition when building `zlib-helper.c`.
2017-07-28 16:36:32 +03:00
cp -a " bin/ $( CONFIGURATION) /lib/xamarin.android/. " " $( prefix) /lib/xamarin.android/ "
2018-05-28 16:59:02 +03:00
-rm -rf " $( prefix) /lib/mono/xbuild/Novell "
2018-06-01 12:40:34 +03:00
-rm -rf " $( prefix) /lib/mono/xbuild/Xamarin/Xamarin.Android.Sdk.props "
-rm -rf " $( prefix) /lib/mono/xbuild/Xamarin/Xamarin.Android.Sdk.targets "
2017-07-06 10:05:42 +03:00
-rm -rf " $( prefix) /lib/mono/xbuild/Xamarin/Android "
-rm -rf " $( prefix) /lib/mono/xbuild-frameworks/MonoAndroid "
[structure] Rework installation directory structure (#704)
Context: https://github.com/xamarin/xamarin-android/pull/253#discussion_r82862993
The *intention* is that Jenkins-produced `oss-xamarin.android*.zip`
artifacts be usable on Windows, so that side-by-side testing can be
performed without replacing the system installation. Usage is in
[UsingJenkinsBuildArtifacts.md](Documentation/UsingJenkinsBuildArtifacts).
This isn't *entirely* the case. It was *apparently* the case at the
time of commit 87ca2737, but things have bitrotten since. For
example, following the 87ca2737 instructions would currently result
in an XA0020 `Could not find monodroid` error, because
`class-parse.exe` wasn't where Windows expects it to be (it was in
`lib/mandroid`, not `lib/xbuild/Xamarin/Android`).
This needs to be fixed.
Additionally, PR #253 mentions that, for filesystem organization, it
would be useful if the macOS/Linux directory structure --
`$prefix/bin`, `$prefix/lib/mandroid`,
`$prefix/lib/xbuild/Xamarin/Android`, `$prefix/lib/xbuild-frameworks`
-- more closely resembled the Windows directory structure of
`$MSBuild` and `$ReferenceAssemblies` (as seen in `.vsix` files).
This would turn macOS/Linux into using `$xa_prefix/xbuild` and
`$xa_prefix/xbuild-frameworks` directories.
`$prefix/bin` would only contain `xabuild`. What is currently in
`$prefix/lib/mandroid` would be merged with
`$xa_prefix/xbuild/Xamarin/Android`.
`$xa_prefix` would `$prefix/lib/xamarin.android`.
This would turn the current macOS structure:
$prefix/bin/xabuild
$prefix/bin/generator
$prefix/bin/cross-arm
$prefix/lib/mandroid/generator.exe
$prefix/lib/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll
$prefix/lib/xbuild/Xamarin/Android/Xamarin.Android.Common.targets
Into:
$prefix/bin/xabuild
$prefix/lib/xamarin.android/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll
$prefix/lib/xamarin.android/xbuild/Xamarin/Android/generator.exe
$prefix/lib/xamarin.android/xbuild/Xamarin/Android/Xamarin.Android.Common.targets
$prefix/lib/xamarin.android/xbuild/Xamarin/Android/Darwin/cross-arm
Other notes:
* The `bundle-*.zip` filename has been has been changed to include a
*hash* of the contents of various files, in particular
`build-tools\mono-runtimes.*`. This was instigated via a
conversation with @kumpera about making the filename more
"idiot-proof": `mono-runtimes.props` contains *compiler flags*,
and if any of those change, then *logically* the bundle should
differ as well, as the mono runtimes may differ in significant
ways. In theory, the `-vXX` version should be used to track this,
but this is a manual change, easy to overlook. The new `-hHASH`
part of the filename should be more automagic.
The new `<HashFileContents/>` task in `xa-prep-tasks.dll` is
responsible for creating the hash value.
* `Configuration.Java.Interop.Override.props` was moved into
`build-tools/scripts`, because that would cleanup the root
directory a bit.
* OS-specific binaries are now placed into
`$prefix/lib/xamarin.android/xbuild/Xamarin/Android/$(HostOS)`.
On a theoretical plus side, this means that the same build
directory can contain OS-specific binaries for multiple operating
systems. (I don't know if anyone shares a build tree between e.g.
macOS and Linux, but if anyone *does*...)
Unfortunately this requires a workaround for an `xbuild` bug:
`%(_LlvmRuntime.InstallPath)` and
`%(_MonoCrossRuntime.InstallPath)` *cannot* use MSBuild
properties, e.g. `<InstallPath>$(HostOS)/</InstallPath>` doesn't
work as desired; it's instead treated literally.
Special-case `%(InstallPath)` until we fully migrate to MSBuild.
* `$(MonoAndroidToolsDirectory)` should be considered *dead*, along
with `$(MonoAndroidBinDirectory)`, as these should now *always*
be the same directory as where `Xamarin.Android.Build.Tasks.dll`
is located, or a `$(HostOS)` sub-directory.
* `Xamarin.ProjectTools.csproj` needed to be updated to ensure that
the build order was correct.
* Remove all `[Obsolete]` and unreferenced members from
`Xamarin.Android.Build.Utilities.dll`. There's too much in there,
and it makes my head hurt trying to understand the
interrelationships between it all. If it's not used, it's gone.
* The changes to `src/monodroid/jni/Android.mk` are...weird. The
removal of `-I$(topdir)/libmonodroid/zip`/etc. is to reduce use of
non-existent paths, as `$(topdir)` isn't defined, so that's
*actually* `-I/libmonodroid/zip`, which is nonsensical. So far,
so good. What's *odd* is the addition of `$(LOCAL_PATH)` to
`$(LOCAL_C_INCLUDES)`: This is needed so that
`external/mono/support/zlib-helper.c` exports `CreateZStream` and
related symbols, otherwise we get a unit test failure in
`GzipStreamTest.Compression` due to an
`EntryPointNotFoundException`, because `CreateZStream` isn't
exported/public.
What's odd here is that I don't understand what caused this
behavior to change. Previous builds exported `CreateZStream`,
otherwise the tests would fail, and I don't understand how any of
the other changes in this PR would be at fault, though that's
certainly the most plausible explanation.
Regardless, `-Ijni` *first* (due to adding `$(LOCAL_PATH)` to
`$(LOCAL_C_INCLUDES)`) is the desired behavior, so that
`jni/config.h` is included, thus ensuring that `MONO_API` has the
required definition when building `zlib-helper.c`.
2017-07-28 16:36:32 +03:00
ln -s " $( prefix) /lib/xamarin.android/xbuild/Xamarin/Android/ " " $( prefix) /lib/mono/xbuild/Xamarin/Android "
2018-05-03 23:09:33 +03:00
ln -s " $( prefix) /lib/xamarin.android/xbuild/Novell/ " " $( prefix) /lib/mono/xbuild/Novell "
[structure] Rework installation directory structure (#704)
Context: https://github.com/xamarin/xamarin-android/pull/253#discussion_r82862993
The *intention* is that Jenkins-produced `oss-xamarin.android*.zip`
artifacts be usable on Windows, so that side-by-side testing can be
performed without replacing the system installation. Usage is in
[UsingJenkinsBuildArtifacts.md](Documentation/UsingJenkinsBuildArtifacts).
This isn't *entirely* the case. It was *apparently* the case at the
time of commit 87ca2737, but things have bitrotten since. For
example, following the 87ca2737 instructions would currently result
in an XA0020 `Could not find monodroid` error, because
`class-parse.exe` wasn't where Windows expects it to be (it was in
`lib/mandroid`, not `lib/xbuild/Xamarin/Android`).
This needs to be fixed.
Additionally, PR #253 mentions that, for filesystem organization, it
would be useful if the macOS/Linux directory structure --
`$prefix/bin`, `$prefix/lib/mandroid`,
`$prefix/lib/xbuild/Xamarin/Android`, `$prefix/lib/xbuild-frameworks`
-- more closely resembled the Windows directory structure of
`$MSBuild` and `$ReferenceAssemblies` (as seen in `.vsix` files).
This would turn macOS/Linux into using `$xa_prefix/xbuild` and
`$xa_prefix/xbuild-frameworks` directories.
`$prefix/bin` would only contain `xabuild`. What is currently in
`$prefix/lib/mandroid` would be merged with
`$xa_prefix/xbuild/Xamarin/Android`.
`$xa_prefix` would `$prefix/lib/xamarin.android`.
This would turn the current macOS structure:
$prefix/bin/xabuild
$prefix/bin/generator
$prefix/bin/cross-arm
$prefix/lib/mandroid/generator.exe
$prefix/lib/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll
$prefix/lib/xbuild/Xamarin/Android/Xamarin.Android.Common.targets
Into:
$prefix/bin/xabuild
$prefix/lib/xamarin.android/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll
$prefix/lib/xamarin.android/xbuild/Xamarin/Android/generator.exe
$prefix/lib/xamarin.android/xbuild/Xamarin/Android/Xamarin.Android.Common.targets
$prefix/lib/xamarin.android/xbuild/Xamarin/Android/Darwin/cross-arm
Other notes:
* The `bundle-*.zip` filename has been has been changed to include a
*hash* of the contents of various files, in particular
`build-tools\mono-runtimes.*`. This was instigated via a
conversation with @kumpera about making the filename more
"idiot-proof": `mono-runtimes.props` contains *compiler flags*,
and if any of those change, then *logically* the bundle should
differ as well, as the mono runtimes may differ in significant
ways. In theory, the `-vXX` version should be used to track this,
but this is a manual change, easy to overlook. The new `-hHASH`
part of the filename should be more automagic.
The new `<HashFileContents/>` task in `xa-prep-tasks.dll` is
responsible for creating the hash value.
* `Configuration.Java.Interop.Override.props` was moved into
`build-tools/scripts`, because that would cleanup the root
directory a bit.
* OS-specific binaries are now placed into
`$prefix/lib/xamarin.android/xbuild/Xamarin/Android/$(HostOS)`.
On a theoretical plus side, this means that the same build
directory can contain OS-specific binaries for multiple operating
systems. (I don't know if anyone shares a build tree between e.g.
macOS and Linux, but if anyone *does*...)
Unfortunately this requires a workaround for an `xbuild` bug:
`%(_LlvmRuntime.InstallPath)` and
`%(_MonoCrossRuntime.InstallPath)` *cannot* use MSBuild
properties, e.g. `<InstallPath>$(HostOS)/</InstallPath>` doesn't
work as desired; it's instead treated literally.
Special-case `%(InstallPath)` until we fully migrate to MSBuild.
* `$(MonoAndroidToolsDirectory)` should be considered *dead*, along
with `$(MonoAndroidBinDirectory)`, as these should now *always*
be the same directory as where `Xamarin.Android.Build.Tasks.dll`
is located, or a `$(HostOS)` sub-directory.
* `Xamarin.ProjectTools.csproj` needed to be updated to ensure that
the build order was correct.
* Remove all `[Obsolete]` and unreferenced members from
`Xamarin.Android.Build.Utilities.dll`. There's too much in there,
and it makes my head hurt trying to understand the
interrelationships between it all. If it's not used, it's gone.
* The changes to `src/monodroid/jni/Android.mk` are...weird. The
removal of `-I$(topdir)/libmonodroid/zip`/etc. is to reduce use of
non-existent paths, as `$(topdir)` isn't defined, so that's
*actually* `-I/libmonodroid/zip`, which is nonsensical. So far,
so good. What's *odd* is the addition of `$(LOCAL_PATH)` to
`$(LOCAL_C_INCLUDES)`: This is needed so that
`external/mono/support/zlib-helper.c` exports `CreateZStream` and
related symbols, otherwise we get a unit test failure in
`GzipStreamTest.Compression` due to an
`EntryPointNotFoundException`, because `CreateZStream` isn't
exported/public.
What's odd here is that I don't understand what caused this
behavior to change. Previous builds exported `CreateZStream`,
otherwise the tests would fail, and I don't understand how any of
the other changes in this PR would be at fault, though that's
certainly the most plausible explanation.
Regardless, `-Ijni` *first* (due to adding `$(LOCAL_PATH)` to
`$(LOCAL_C_INCLUDES)`) is the desired behavior, so that
`jni/config.h` is included, thus ensuring that `MONO_API` has the
required definition when building `zlib-helper.c`.
2017-07-28 16:36:32 +03:00
ln -s " $( prefix) /lib/xamarin.android/xbuild-frameworks/MonoAndroid/ " " $( prefix) /lib/mono/xbuild-frameworks/MonoAndroid "
2017-08-19 01:45:10 +03:00
if [ ! -e " $( prefix) /bin/mono " ] ; then \
2017-09-05 17:04:53 +03:00
cp tools/scripts/xabuild " $( prefix) /bin/xabuild " ; \
2017-08-19 01:45:10 +03:00
fi
2017-07-03 20:09:22 +03:00
uninstall ::
rm -rf " $( prefix) /lib/xamarin.android/ " " $( prefix) /bin/xabuild "
2018-03-13 16:44:48 +03:00
rm -rf " $( prefix) /lib/mono/xbuild/Xamarin/Android "
rm -rf " $( prefix) /lib/mono/xbuild-frameworks/MonoAndroid "
2017-07-03 20:09:22 +03:00
[build] ThirdPartyNotices.txt support (#464)
`ThirdPartyNotices.txt` is a Microsoft convention to inform
"the audience" that third party software is contained within a given
"distribution mechanism", where said "distribution mechanism" could be
a git repo or installable binary build artifacts.
When applied to a source repository, `ThirdPartyNotices.txt` only
contains license information for source code contained *within the
current repository*, and does ***not*** include source code that may
be pulled in via external sources, such as git submodules or NuGet
packages.
When applied to installable binary build artifacts,
`ThirdPartyNotices.txt` should include everything that is part of the
binary distribution. In our case, that means anything that winds up in
`bin/$(Configuration)`, but *not* build-system support or tests, which
are in `bin/Build$(Configuration)` and `bin/Test$(Configuration)`.
This results in two separate files of interest:
1. `ThirdPartyNotices.txt`: Contains license information for 3rd
party source code which has been *copied* into this repo.
2. `bin/$(Configuration)/lib/xamarin.android/ThirdPartyNotices.txt`:
License information for a binary redistribution based on the build
tree. This would include license information for both source code
that has been copied into this repo, and source code from git
submodules.
Generation of these two separate files is controlled by two separate
MSBuild properties:
* `$(TpnIncludeBuildDependencies)`: True when build-time
dependencies should be included in the resulting file.
* `$(TpnIncludeExternalDependencies)`: True when *external*
dependency information should be included in the resulting file.
To satisfy these separate needs:
1. Add a bunch of `external/*.tpnitems` and `src/*.tpnnitems` files,
which declares `@(ThirdPartyNotice)` values for various sources.
Some of these values are Conditional on MSBuild properties,
allowing them to be included or excluded based on the build
configuration.
The primary reason for this is for later "auditing" and reasoning,
e.g. if we *remove* an external dependency, it'll be (presumably)
easier to remove the corresponding `@(ThirdPartyNotice)` entry.
It also provides a pattern for adding additional dependencies.
2. Create a new `<CreateThirdPartyNotices/>` task in
`xa-prep-tasks.dll`. This task processes the `@(ThirdPartyNotice)`
item group to produce `ThirdPartyNotices.txt`-formatted content.
3. Add a new `build-tools/ThirdPartyNotices/ThirdPartyNotices.csproj`
project. This project includes the `external/*.tpnitems` and
`src/*.tpnitems` entries and invokes the
`<CreateThirdPartyNotices/>` task, creating the file
`$(ThirdPartyNoticeFile)`.
4. Update `Makefile` to build `ThirdPartyNotices.csproj` at well
defined build points. `make prepare` will update the top-level
`ThirdPartyNotices.txt`, and `$(CONFIGURATION)`-specific
`bin/$(Configuration)/lib/xamarin.android/ThirdPartyNotices.txt`
files for inclusion elsewhere.
Additionally, the packaging infrastructure has been updated to ensure
that `ThirdPartyNotices.txt` is included: the
`Xamarin.Android.Sdk*.vsix` file will contain a top-level
`ThirdPartyNotices.txt` file, and the `xamarin.android-oss*.zip` file
will contain one as well.
2018-03-02 17:54:35 +03:00
topdir := $( shell pwd )
2019-07-10 02:00:50 +03:00
i n c l u d e b u i l d - t o o l s / s c r i p t s / B u i l d E v e r y t h i n g . m k
# Must be after BuildEverything.mk - it uses variables defined there
i n c l u d e b u i l d - t o o l s / s c r i p t s / P a c k a g i n g . m k
2017-11-08 23:38:27 +03:00
run-all-tests :
[build] Use a system mono/2018-06 debug build (#2440)
Context: https://github.com/mono/mono/pull/11706
Context: https://github.com/xamarin/xamarin-android/pull/2357
Context: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android/1239/console
Sometimes a Jenkins build will timeout when executing the
xamarin-android unit tests. The timeout *appears* to occur when
`mono` is used to execute a process, the process "finishes," but the
process never actually *exits*, thus "hanging".
For example, [consider this build log fragment][0] (with timestamps):
23:37:48 Build succeeded.
23:37:48 0 Warning(s)
23:37:48 0 Error(s)
23:37:48
23:37:48 Time Elapsed 01:23:14.60
02:37:48 Build timed out (after 180 minutes). Marking the build as aborted.
The "Build succeeded" message comes from an `msbuild` invocation.
Once `msbuild` finished, we would expect it's process to exit.
*It didn't*. Instead, the underlying `mono` process is *not* exiting,
presumably deadlocked, until the overall job times out and is killed.
Attempt to help identify the hang by using a "debug" mono/2018-08
system mono, built from [Mono PR #11706][1], which prints out
additional messages around child process use:
diff --git a/mono/metadata/w32process-unix.c b/mono/metadata/w32process-unix.c
index 2db5dea7a705..fbd4731cacb2 100644
--- a/mono/metadata/w32process-unix.c
+++ b/mono/metadata/w32process-unix.c
@@ -2096,7 +2095,7 @@ process_create (const gunichar2 *appname, const gunichar2 *cmdline,
if (argv)
g_strfreev (argv);
- mono_trace (G_LOG_LEVEL_DEBUG, MONO_TRACE_IO_LAYER_PROCESS, "%s: returning handle %p for pid %d", __func__, handle, pid);
+ g_printerr ("XXX-MONO: %s: returning handle %p for pid %d from %d\n", __func__, handle, pid, getpid());
This will hopefully allow @lewurm to continue investigations.
[0]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android/1237/console
[1]: https://github.com/mono/mono/pull/11706
2018-11-30 19:40:30 +03:00
@echo "PRINTING MONO VERSION"
mono --version
2019-04-29 23:24:40 +03:00
_r = 0 ; \
$( call MSBUILD_BINLOG,run-all-tests,,Test) $( TEST_TARGETS) /t:RunAllTests || _r = $$ ? ; \
exit $$ _r
[tests] Add Configuration info to test names
Context: 0077d151
Context: d1d9820a
A "funny" thing happened when commit e9daf5ea didn't build on Jenkins:
I realized that not all tests were run in all configurations. From
commit d1d9820a:
> Why are some tests Debug-only and some aren't?
The answer: time, primarily. Why run tests multiple times, when they
can be potentially time-consuming?
While tests can be slow, they're not always *that* slow -- except for
`Xamarin.Android.Build.Tests` and the BCL tests -- and even there,
program behavior can alter between Debug and Release configurations.
See in particular commit 0077d151, in which the BCL tests are run only
in the Debug configuration because tests *failed* when run in the
Release configuration.
The desire, then, is to run *all* tests in both Debug and Release
configurations. Yes, it'll take longer! So what! (Within reason:
`Xamarin.Android.Build.Tests` should only be run once!)
However, this raises two problems:
1. Filename collisions
2. Jenkins unit test display
Until now, all tests wrote files into a filename that didn't include
the Configuration, e.g. `TestResult-Mono.Android_Tests.xml`. If we did
run these tests twice, the second test invocation would overwrite the
first test invocation. This isn't desirable.
Then there's the display on Jenkins: if we did have e.g.
`TestResult-Mono.Android_Tests-Debug.xml` and
`TestResult-Mono.Android_Tests-Release.xml`, how will Jenkins display
that information? I haven't tested, but I would assume that one of two
things will occur, assuming reasonable Jenkins behavior:
1. Each test will be listed twice, e.g.
ApplicationContextIsApp
ApplicationContextIsApp
2. They'll be "merged" into a single entry.
Neither of these behaviors is desirable: if Debug passes but Release
fails, we need to be able to differentiate between them. Neither of
these possible renderings allows us to tell which configuration fails.
Solve both of these problems by introducing a new `<RenameTestCases/>`
task. This task takes three values of importance:
```xml
<RenameTestCases
Configuration="CONFIGURATION"
SourceFile="SOURCE"
DestinationFolder="DESTINATION"
/>
```
The `<RenameTestCases/>` task will read in `SOURCE`, and if `SOURCE`
is an XML file which we determine is NUnit2-formatted XML (root
element of `<test-case/>`), we will update every `//test-case/@name`
value so that it ends with ` / CONFIGURATION`. The updated XML is
then written to the `DESTINATION` directory, with a filename that
contains `CONFIGURATION`, and `SOURCE` is deleted.
Thus, if we have a Debug-configuration
`TestResult-Mono.Android_Tests.xml` file with XML fragment:
```xml
<test-case
name="Mono.Android_Tests, Android.AppTests.ApplicationTest.ApplicationContextIsApp"
...
/>
```
then `<RenameTestCases/>` will create the file
`TestResult-Mono.Android_Tests-Debug.xml` file with XML fragment:
```xml
<test-case
name="Mono.Android_Tests, Android.AppTests.ApplicationTest.ApplicationContextIsApp / Debug"
...
/>
```
This allows us to run tests in both Debug and Release configurations
while not inadvertently overwriting the `TestResults*.xml` files that
Jenkins reads, and ensuring that the Jenkins test result output is
rendered in a meaningfully useful fashion.
Aside: when updating `//test-case/@name`, the resulting value *cannot*
end in `)`. If it does, then the `(root)` package name issue fixed in
commit 23b2642e reappears for the `generator` unit tests.
**Completely random aside about the state of `xbuild`**:
A development version of `<RenameTestCases/>` was "saner", using
`ITaskItem[]` and not string:
```csharp
partial class RenameTestCases {
public ITaskItem[] SourceFiles {get; set;}
// vs.
// public string SourceFile {get; set;}
}
```
The problem is that the above, while entirely reasonable, did not work
at all correctly with `xbuild`:
```xml
<RenameTestCases SourceFiles="%(TestApk.ResultsPath)" />
```
Under `xbuild`, MSBuild properties would not be expanded, e.g.
`RenameTestCases.SourceFiles` would get a "bizarro" value of e.g.
`$(OutputPath)Mono.Android_Tests-Signed.apk`, which is *useless* and
would result in `FileNotFoundException`s.
MSBuild proper, of course, worked as desired.
TODO: Once this is merged, update the Jenkins Configuration page so
that instead of:
make run-all-tests V=1 || exit 1
it instead runs both Debug and Release configuration tests:
make run-all-tests SKIP_NUNIT_TESTS=1 V=1 || exit 1
make run-all-tests CONFIGURATION=Release V=1 || exit 1
Note that `$(SKIP_NUNIT_TESTS)` is specified so that we only run the
lengthy (1+hr!) `Xamarin.Android.Build.Tests` tests in the Release
configuration, not the Debug + Release configurations.
2017-10-11 00:59:42 +03:00
[android-toolchain] Permit zero-configuration builds.
This might be a suspect idea, but lets see if we can make this work.
[The Joel Test: 12 Steps to Better Code][0] outlines 12 steps
to better code. The first two steps are:
1. Do you use source control?
2. Can you make a build in one step?
github is being used for source control, so (1) is handled, but how
simple can we make (2)? How easy can we make it to build
Xamarin.Android upon a fresh checkout?
The ideal to strive for is simple:
Load Xamarin.Android.sln into your IDE and Build the project.
I *know* we're not going to be able to do this, if only because we're
going to be using git submodules, which will require a separate
`git submodule init` invocation [1].
Knowing we can't reach that level of simplicitly doesn't mean we
shouldn't *try* to reach it for all other parts of the build system.
Which brings us to the Android NDK and SDK. The Android NDK will be
required in order to build native code, such as libmonodroid.so, while
the Android SDK will be required in order to compile
Java Callable Wrappers (née Android Callable Wrappers [2]) and
eventual samples and unit tests.
There are three ways we can deal with the Android NDK and SDK:
1. Complicate the "build" process by requiring that developers go to
the Android SDK Download Page [3], download and install
"somewhere" the required bits, and then configure the
Xamarin.Android build to use these bits.
2. Complicate the "build" process by requiring that developers run
the Xamarin Unified Installer [4], let it install everything
required, then configure the Xamarin.Android build to use those
bits.
3. Painstakingly determine which files are actually required, then
automatically download and extract those files into a "well-known"
location known by the Xamarin.Android build process.
(1) and (2) can be infuriating. Let's give (3) a try. :-)
Add a Xamarin.Android.Tools.BootstrapTasks project which contains
MSBuild tasks to facilitate downloading the Android SDK and NDK files.
Add an android-toolchain project which uses
Xamarin.Android.Tools.BootstrapTasks to download a painstakingly
determined set of files and install them "somewhere".
Unfortunately [5] the "somewhere" to download and install these files
needs to be in a known absolute path, so I've arbitrary decided to
download the files into $(HOME)\android-archives and install them into
$(HOME)\android-toolchain. On windows, this is
%HOMEDRIVE%%HOMEPATH%\android-archives and
%HOMEDRIVE%%HOMEPATH%\android-toolchain.
These locations may be modified by creating a
Configuration.Override.props file; see README.md for details.
TL;DR: This setup is able to magically download the Android NDK and
SDK files and install them for later use in a reasonably overridable
location, all within MSBuild.
[0]: http://www.joelonsoftware.com/articles/fog0000000043.html
[1]: Though maybe there's some MSBuild-fu we can use to address that.
[2]: https://developer.xamarin.com/guides/android/advanced_topics/java_integration_overview/android_callable_wrappers/
[3]: http://developer.android.com/sdk/index.html
[4]: https://www.xamarin.com/download
[5]: Because I couldn't find a reliable way to use $(SolutionDir) when
only building a project, and relative paths would require an
in-tree installation location, which might not work.
2016-04-19 03:33:04 +03:00
clean :
[build] Normal & Diagnostic Log Verbosity, at once!
Fixes: https://github.com/xamarin/xamarin-android/issues/1792
We have a bit of a conundrum: when things break, the diagnostic build
logs are *essential* to tracking down *why* something broke.
However, the sheer *verbosity* of those diagnostic logs makes it
difficult to determine *what* broke; it's very easy to get lost.
Additionally, the gigantic log files make the **Console Output**
Jenkins link nearly useless, as most web browsers will cease up when
attempting to download ~1GB log files.
Split the difference: introduce a new `$(call MSBUILD_BINLOG)` syntax
which creates an [MSBuild binary log][0] file, which is like
diagnostic output *but even moreso*, while *also* overriding the
console logging to `/v:normal`.
[0]: https://github.com/Microsoft/msbuild/wiki/Binary-Log
This should allow the **Console Output** link to be a bit more
reasoanble, and if something goes wrong we can download the separate
`*.binlog` files for further analysis.
MSBuild binary log files can be read with MSBuild itself:
msbuild /v:diag bin/BuildDebug/msbuild-*.binlog
Note: above command will actually fail; you can only print one log
file at a time with that syntax.
The created log files follow the pattern `msbuild-EPOCH-NAME`, where
EPOCH is the `time_t`/Unix epoch value *at the time the command was
executed*. This should allow filenames to be non-colliding, and allow
plausibly sane sorting behavior. (Maybe.)
2018-06-12 04:28:38 +03:00
$( call MSBUILD_BINLOG,clean) /t:Clean Xamarin.Android.sln
[Xamarin.Android-Tests] Split out "full" apps (#146)
We're trying to get [`make jenkins`][0] working on Jenkins, and
[it's failing][1], as one might expect when a particular repo and
associated build system has never been run on Jenkins before:
Android.App/ApplicationTest.cs(9,7): error CS0246: The type or namespace name `NUnit' could not be found. Are you missing an assembly reference?
This error occurs while building
`src/Mono.Android/Test/Mono.Android-Tests.csproj`, and happens because
the `Xamarin.Android.NUnitLite.dll` assembly isn't referenced...
because it isn't *found*:
Microsoft.Common.targets: warning : Reference 'Xamarin.Android.NUnitLite' not resolved
This returns us to a long-standing issue which I thought was mentioned
in a commit somewhere, but I can't find at present:
MSBuild doesn't support updating the assembly resolution directories
*while MSBuild is running*.
For example, we build `Xamarin.Android.NUniteLite.dll` into
`bin/$(Configuration/lib/xbuild-frameworks/MonoAndroid/v1.0`,
but "normal Xamarin.Android referencing projects" don't use a
`@(ProjectReference)` to `Xamarin.Android.NUnitLite.csproj`, but
instead a `@(Reference)` to the assembly:
<!-- src/Mono.Android/Test/Mono.Android-Tests.csproj -->
<Reference Include="Xamarin.Android.NUnitLite" />
The *requirement* for a "proper" Xamarin.Android SDK install is that
`Xamarin.Android.NUnitLite.dll` can be found through a normal
`@(Reference)`. In order to satisfy this requirement, we need to tell
MSBuild where to find it, which can be with `xbuild` via the
`$MSBuildExtensionsPath` and `$XBUILD_FRAMEWORK_FOLDERS_PATH`
*environment variables*.
*Environment variables*.
MSBuild doesn't provide a way to update environment variables, short
of writing a new Task which calls
`Environment.SetEnvironmentVariable()`, and while [this works][2], it
doesn't *actually* work [^3].
The short of all this is that it isn't possible,
*within a single `xbuild` invocation*, to both build the
Xamarin.Android SDK "environment" *and use that environment* as
intended for "normal" apps.
The fix, as is often the case, is to bend with the wind. Instead of
requiring the impossible, move
`src/Mono.Android/Test/Mono.Android-Tests.csproj` into a *new*
`Xamarin.Android-Tests.sln` solution, *out of* `Xamarin.Android.sln`.
This allows building `Xamarin.Android.sln` without error in a pristine
environment -- that is, one that doesn't already have a system-wide
Xamarin.Android install -- and separately building the tests by using
`tools/scripts/xabuild`, which *can* export environment variables to
manipulate `xbuild` behavior so that things Just Work™.
Building `Mono.Android-Tests.csproj` and similar projects
(`HelloWorld.csproj`!) can be done by using the new `make all-tests`
make target.
[0]: https://github.com/xamarin/xamarin-android/commit/a16673d3eb2c4945c3a74f5f8154603d7658fc9a
[1]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android/20/console
[2]: https://github.com/xamarin/xamarin-android/pull/147
[^3]: [PR #147][2] isn't viable because of [xbuild's `AsssemblyResolver`][4].
There's no way to clear/invalidate `target_framework_cache`.
The idea of PR #147 was to "hack things up" so that
`Xamarin.Android.NuniteLite.dll` would be properly resolved during
the build of `Mono.Android-Tests.csproj` *when building everything*.
The problem is *this can't work*, because `xbuild` has a
"target framework cache," with no way to invalidate it, and the
cache is populated the first time the target framework is used.
Due to build ordering issues, this first use is *before*
`Xamarin.Android.NunitLite.dll` was built, and thus it doesn't
exist in the cache. The result:
Task "ResolveAssemblyReference"
....
TargetFrameworkDirectories:
/Volumes/Seagate4TB/work/xamarin-android/bin/Debug/lib/xbuild-frameworks/MonoAndroid/v7.0
/Volumes/Seagate4TB/work/xamarin-android/bin/Debug/lib/xbuild-frameworks/MonoAndroid/v1.0
/Volumes/Seagate4TB/work/xamarin-android/bin/Debug/lib/xbuild-frameworks/MonoAndroid/v1.0/Facades/
...
Primary Reference System.Xml
Reference System.Xml resolved to /Volumes/Seagate4TB/work/xamarin-android/bin/Debug/lib/xbuild-frameworks/MonoAndroid/v1.0/System.Xml.dll. CopyLocal = False
...
Primary Reference Xamarin.Android.NUnitLite
Microsoft.Common.targets: warning : Reference 'Xamarin.Android.NUnitLite' not resolved
For searchpath {TargetFrameworkDirectory}
Considered target framework dir /Volumes/Seagate4TB/work/xamarin-android/bin/Debug/lib/xbuild-frameworks/MonoAndroid/v7.0, assembly named 'Xamarin.Android.NUnitLite' not found.
Considered target framework dir /Volumes/Seagate4TB/work/xamarin-android/bin/Debug/lib/xbuild-frameworks/MonoAndroid/v1.0, assembly named 'Xamarin.Android.NUnitLite' not found.
Considered target framework dir /Volumes/Seagate4TB/work/xamarin-android/bin/Debug/lib/xbuild-frameworks/MonoAndroid/v1.0/Facades/, assembly named 'Xamarin.Android.NUnitLite' not found.
...
Consequently, the `mcs` invocation is missing a
`/reference:path/to/Xamarin.Android.NUniteLite.dll`, and
compilation fails:
Android.App/ApplicationTest.cs(9,7): error CS0246: The type or namespace name `NUnit' could not be found. Are you missing an assembly reference?
...plus 29 others...
[4]: https://github.com/mono/mono/blob/dd8aadf/mcs/class/Microsoft.Build.Tasks/Microsoft.Build.Tasks/AssemblyResolver.cs#L131
2016-08-08 23:46:13 +03:00
tools/scripts/xabuild $( MSBUILD_FLAGS) /t:Clean Xamarin.Android-Tests.sln
2016-05-12 21:54:03 +03:00
2016-06-10 17:07:34 +03:00
distclean :
# It may fail if we're cleaning a half-built tree, no harm done if we ignore it
-$( MAKE) clean
git clean -xdff
git submodule foreach git clean -xdff
2017-11-08 23:38:27 +03:00
run-nunit-tests :
2017-10-19 22:49:09 +03:00
i f e q ( $( SKIP_NUNIT_TESTS ) , )
[build] Upload unit test results (#2782)
There was some unexpected and unforeseen fallout from commit 241d9416:
`make package-build-status` is only run once.
This doesn't *sound* problematic -- hence the oversight -- but the
`msbuild*.binlog` files that the `xa-build-status*.zip` file contained
prior to commit 241d9416 contained *unit test execution logs*, e.g.
`msbuild-*-Target-RunApkTests.binlog` which contains on-device
execution of the `.apk` unit tests.
However, these files don't exist until *after* the unit tests are run.
Meanwhile, `make package-build-status` is executed *before* unit tests
are run, and thus the resulting `xa-build-status*.zip` file no longer
contains unit test information!
(This makes it rather difficult to actually review anything about unit
test execution!)
The fix? Update the `$(MSBUILD_BINLOG)` Make `define` so that it now
takes an optional 3rd parameter: part of the directory name to place
files into, which defaults to `Build`. (This allows overriding things
so that unit-test related `.binlog` files now go into
`bin/Test$(Configuration)`, not `bin/Build$(Configuration)`.)
Additionally, *rename* the `make package-test-errors` rule to
`make package-test-results`, have it produce an `xa-test-results*.zip`
file, and include the `bin/Test$(Configuration)/msbuild*.binlog` files
into `xa-test-results*.zip`.
`make package-test-results` is executed after unit tests are executed,
so this change allows us the uploaded `xa-test-results.zip` file to
contain unit test execution results.
2019-03-02 04:46:00 +03:00
$( call MSBUILD_BINLOG,run-nunit-tests,,Test) $( TEST_TARGETS) /t:RunNUnitTests
[tests] Add Configuration info to test names
Context: 0077d151
Context: d1d9820a
A "funny" thing happened when commit e9daf5ea didn't build on Jenkins:
I realized that not all tests were run in all configurations. From
commit d1d9820a:
> Why are some tests Debug-only and some aren't?
The answer: time, primarily. Why run tests multiple times, when they
can be potentially time-consuming?
While tests can be slow, they're not always *that* slow -- except for
`Xamarin.Android.Build.Tests` and the BCL tests -- and even there,
program behavior can alter between Debug and Release configurations.
See in particular commit 0077d151, in which the BCL tests are run only
in the Debug configuration because tests *failed* when run in the
Release configuration.
The desire, then, is to run *all* tests in both Debug and Release
configurations. Yes, it'll take longer! So what! (Within reason:
`Xamarin.Android.Build.Tests` should only be run once!)
However, this raises two problems:
1. Filename collisions
2. Jenkins unit test display
Until now, all tests wrote files into a filename that didn't include
the Configuration, e.g. `TestResult-Mono.Android_Tests.xml`. If we did
run these tests twice, the second test invocation would overwrite the
first test invocation. This isn't desirable.
Then there's the display on Jenkins: if we did have e.g.
`TestResult-Mono.Android_Tests-Debug.xml` and
`TestResult-Mono.Android_Tests-Release.xml`, how will Jenkins display
that information? I haven't tested, but I would assume that one of two
things will occur, assuming reasonable Jenkins behavior:
1. Each test will be listed twice, e.g.
ApplicationContextIsApp
ApplicationContextIsApp
2. They'll be "merged" into a single entry.
Neither of these behaviors is desirable: if Debug passes but Release
fails, we need to be able to differentiate between them. Neither of
these possible renderings allows us to tell which configuration fails.
Solve both of these problems by introducing a new `<RenameTestCases/>`
task. This task takes three values of importance:
```xml
<RenameTestCases
Configuration="CONFIGURATION"
SourceFile="SOURCE"
DestinationFolder="DESTINATION"
/>
```
The `<RenameTestCases/>` task will read in `SOURCE`, and if `SOURCE`
is an XML file which we determine is NUnit2-formatted XML (root
element of `<test-case/>`), we will update every `//test-case/@name`
value so that it ends with ` / CONFIGURATION`. The updated XML is
then written to the `DESTINATION` directory, with a filename that
contains `CONFIGURATION`, and `SOURCE` is deleted.
Thus, if we have a Debug-configuration
`TestResult-Mono.Android_Tests.xml` file with XML fragment:
```xml
<test-case
name="Mono.Android_Tests, Android.AppTests.ApplicationTest.ApplicationContextIsApp"
...
/>
```
then `<RenameTestCases/>` will create the file
`TestResult-Mono.Android_Tests-Debug.xml` file with XML fragment:
```xml
<test-case
name="Mono.Android_Tests, Android.AppTests.ApplicationTest.ApplicationContextIsApp / Debug"
...
/>
```
This allows us to run tests in both Debug and Release configurations
while not inadvertently overwriting the `TestResults*.xml` files that
Jenkins reads, and ensuring that the Jenkins test result output is
rendered in a meaningfully useful fashion.
Aside: when updating `//test-case/@name`, the resulting value *cannot*
end in `)`. If it does, then the `(root)` package name issue fixed in
commit 23b2642e reappears for the `generator` unit tests.
**Completely random aside about the state of `xbuild`**:
A development version of `<RenameTestCases/>` was "saner", using
`ITaskItem[]` and not string:
```csharp
partial class RenameTestCases {
public ITaskItem[] SourceFiles {get; set;}
// vs.
// public string SourceFile {get; set;}
}
```
The problem is that the above, while entirely reasonable, did not work
at all correctly with `xbuild`:
```xml
<RenameTestCases SourceFiles="%(TestApk.ResultsPath)" />
```
Under `xbuild`, MSBuild properties would not be expanded, e.g.
`RenameTestCases.SourceFiles` would get a "bizarro" value of e.g.
`$(OutputPath)Mono.Android_Tests-Signed.apk`, which is *useless* and
would result in `FileNotFoundException`s.
MSBuild proper, of course, worked as desired.
TODO: Once this is merged, update the Jenkins Configuration page so
that instead of:
make run-all-tests V=1 || exit 1
it instead runs both Debug and Release configuration tests:
make run-all-tests SKIP_NUNIT_TESTS=1 V=1 || exit 1
make run-all-tests CONFIGURATION=Release V=1 || exit 1
Note that `$(SKIP_NUNIT_TESTS)` is specified so that we only run the
lengthy (1+hr!) `Xamarin.Android.Build.Tests` tests in the Release
configuration, not the Debug + Release configurations.
2017-10-11 00:59:42 +03:00
e n d i f # $(SKIP_NUNIT_TESTS) == ''
[Mono.Android-Test] Import Mono.Android tests from monodroid/9c5b3712 (#32)
Import monodroid/tests/runtime from monodroid/9c5b3712.
Add a toplevel `make run-apk-tests` target to "full stack" tests, in
which a .apk is created, installed, and executed on an attached
Android device. `make run-apk-tests` requires that `adb` be in $PATH,
and uses GNU make(1) features, and...
Additionally, tools/scripts/xabuild *must* be used to execute the
`SignAndroidPackage` target, to ensure that the local/"in tree"
assemblies are used. There is no "within xbuild" facility to alter
where target framework assemblies are resolved from, i.e no MSBuild
properties currently control the resolution order, only environment
variables, and MSBuild can't *set* environment variables...
The $(ADB_TARGET) variable can be used to control on which target
Android device the tests will be installed and executed on:
# Install & run tests on *only* connected USB device
$ make run-apk-tests ADB_TARGET=-d
# Install & run tests on *only* connected emulator
$ make run-apk-tests ADB_TARGET=-e
# Install & run tests on specified device, listed via `adb devices`.
$ make run-apk-tests ADB_TARGET="-s 036533824381cfcb"
Sorry potential/future Windows developers. *Running* tests will
require manual testing or running on OS X or Linux for now...
Note: These tests DO NOT PASS. In fact, they *crash*:
$ make run-apk-tests
...
Target RunTests:
Executing: "$HOME/android-toolchain/sdk/platform-tools/adb" shell am instrument -w Mono.Android_Tests/xamarin.android.runtimetests.TestInstrumentation
INSTRUMENTATION_RESULT: shortMsg=Process crashed.
INSTRUMENTATION_CODE: 0
$ adb logcat
...
E mono : Unhandled Exception:
E mono : System.ObjectDisposedException: Cannot access a disposed object.
E mono : Object name: 'System.Net.Sockets.Socket'.
E mono : at System.Net.Sockets.Socket.ThrowIfDisposedAndClosed () <0xa93923f0 + 0x00054> in <filename unknown>:0
E mono : at System.Net.Sockets.Socket.AcceptAsync (System.Net.Sockets.SocketAsyncEventArgs e) <0x9b8f9680 + 0x0001b> in <filename unknown>:0
E mono : at System.Net.EndPointListener.Accept (System.Net.Sockets.Socket socket, System.Net.Sockets.SocketAsyncEventArgs e) <0x9b8f95d0 + 0x0003f> in <filename unknown>:0
E mono : at System.Net.EndPointListener.ProcessAccept (System.Net.Sockets.SocketAsyncEventArgs args) <0x9b8e0340 + 0x0007f> in <filename unknown>:0
E mono : at System.Net.EndPointListener.OnAccept (System.Object sender, System.Net.Sockets.SocketAsyncEventArgs e) <0x9b8e0310 + 0x00017> in <filename unknown>:0
E mono : at System.Net.Sockets.SocketAsyncEventArgs.OnCompleted (System.Net.Sockets.SocketAsyncEventArgs e) <0x9b8e02c8 + 0x0003b> in <filename unknown>:0
E mono : at System.Net.Sockets.SocketAsyncEventArgs.Complete () <0x9b8e02a0 + 0x0001f> in <filename unknown>:0
E mono : at System.Net.Sockets.Socket.<AcceptAsyncCallback>m__0 (System.IAsyncResult ares) <0x9b8dfd40 + 0x002af> in <filename unknown>:0
E mono : at System.Net.Sockets.SocketAsyncResult+<Complete>c__AnonStorey0.<>m__0 (System.Object _) <0xa892f720 + 0x0002b> in <filename unknown>:0
E mono : at System.Threading.QueueUserWorkItemCallback.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem () <0xa938f6b8 + 0x0002f> in <filename unknown>:0
E mono : at System.Threading.ThreadPoolWorkQueue.Dispatch () <0xa938e358 + 0x001bb> in <filename unknown>:0
E mono : at System.Threading._ThreadPoolWaitCallback.PerformWaitCallback () <0xa938e1a8 + 0x00007> in <filename unknown>:0
Looks like a Socket and/or ThreadPool bug in mono.
2016-05-16 21:55:13 +03:00
[tests] Run the Java.Interop unit tests (#565)
Bump to Java.Interop/master/8d7060d1
Bump to cecil/master/5f93431f
We'd like to run the Java.Interop unit tests as part of the
xamarin-android `make run-all-tests` target, for two rasons:
1. An extra layer of sanity checking, and
2. Some of the Java.Interop tests require an Android SDK.
In particular, the
`ParameterFixupTests.XmlDeclaration_FixedUpFromDocumentation()` test
from `Xamarin.Android.Tools.Bytecode-Tests.dll` attempts to read
Android documentation from `$ANDROID_SDK_PATH` to test parameter name
fixups.
However, Java.Interop doesn't install -- and thus can't readily
require -- an Android SDK, meaning this test is never executed as part
of the Java.Interop Jenkins process.
That test *can* be readily executed in xamarin-android, as it *does*
install an Android SDK, and thus its presence can be assumed.
Add a new `make run-ji-tests` target which runs the Java.Interop unit
tests, copying their test output into the same directory that the
other `TestResult-*.xml` files are placed, so that Jenkins can pick
them up for display.
Within the `make run-ji-tests` target, export the `ANDROID_SDK_PATH`
environment variable so that *all* the
`Xamarin.Android.Tools.Bytecode-Tests.dll` tests can execute.
Which introduces two problems:
1. The `Xamarin.Android.Tools.Bytecode-Tests.dll` tests require that
`$ANDROID_SDK_PATH` contain documentation...which we're not
currently installing.
2. The Java.Interop tests still fail, due to parameter name changes,
as those tests were originally using API-18 docs.
Result: the `ParameterFixupTest` tests fail. :-(
Fix this conundrum by instaslling `docs-24_r01.zip` within
`$(AndroidSdkDirectory)`, allowing the tests to execute as intended,
and by updating to Java.Interop/master/8d7060d1, which updates the
expected parameter name output.
2017-04-21 15:11:40 +03:00
run-ji-tests :
[build] Upload unit test results (#2782)
There was some unexpected and unforeseen fallout from commit 241d9416:
`make package-build-status` is only run once.
This doesn't *sound* problematic -- hence the oversight -- but the
`msbuild*.binlog` files that the `xa-build-status*.zip` file contained
prior to commit 241d9416 contained *unit test execution logs*, e.g.
`msbuild-*-Target-RunApkTests.binlog` which contains on-device
execution of the `.apk` unit tests.
However, these files don't exist until *after* the unit tests are run.
Meanwhile, `make package-build-status` is executed *before* unit tests
are run, and thus the resulting `xa-build-status*.zip` file no longer
contains unit test information!
(This makes it rather difficult to actually review anything about unit
test execution!)
The fix? Update the `$(MSBUILD_BINLOG)` Make `define` so that it now
takes an optional 3rd parameter: part of the directory name to place
files into, which defaults to `Build`. (This allows overriding things
so that unit-test related `.binlog` files now go into
`bin/Test$(Configuration)`, not `bin/Build$(Configuration)`.)
Additionally, *rename* the `make package-test-errors` rule to
`make package-test-results`, have it produce an `xa-test-results*.zip`
file, and include the `bin/Test$(Configuration)/msbuild*.binlog` files
into `xa-test-results*.zip`.
`make package-test-results` is executed after unit tests are executed,
so this change allows us the uploaded `xa-test-results.zip` file to
contain unit test execution results.
2019-03-02 04:46:00 +03:00
$( call MSBUILD_BINLOG,run-ji-tests,,Test) $( TEST_TARGETS) /t:RunJavaInteropTests
2017-08-16 18:05:39 +03:00
2019-01-02 13:31:26 +03:00
i f n e q ( $( PACKAGES ) , )
APK_TESTS_PROP = /p:ApkTests= '"$(PACKAGES)"'
e n d i f
[Mono.Android-Test] Import Mono.Android tests from monodroid/9c5b3712 (#32)
Import monodroid/tests/runtime from monodroid/9c5b3712.
Add a toplevel `make run-apk-tests` target to "full stack" tests, in
which a .apk is created, installed, and executed on an attached
Android device. `make run-apk-tests` requires that `adb` be in $PATH,
and uses GNU make(1) features, and...
Additionally, tools/scripts/xabuild *must* be used to execute the
`SignAndroidPackage` target, to ensure that the local/"in tree"
assemblies are used. There is no "within xbuild" facility to alter
where target framework assemblies are resolved from, i.e no MSBuild
properties currently control the resolution order, only environment
variables, and MSBuild can't *set* environment variables...
The $(ADB_TARGET) variable can be used to control on which target
Android device the tests will be installed and executed on:
# Install & run tests on *only* connected USB device
$ make run-apk-tests ADB_TARGET=-d
# Install & run tests on *only* connected emulator
$ make run-apk-tests ADB_TARGET=-e
# Install & run tests on specified device, listed via `adb devices`.
$ make run-apk-tests ADB_TARGET="-s 036533824381cfcb"
Sorry potential/future Windows developers. *Running* tests will
require manual testing or running on OS X or Linux for now...
Note: These tests DO NOT PASS. In fact, they *crash*:
$ make run-apk-tests
...
Target RunTests:
Executing: "$HOME/android-toolchain/sdk/platform-tools/adb" shell am instrument -w Mono.Android_Tests/xamarin.android.runtimetests.TestInstrumentation
INSTRUMENTATION_RESULT: shortMsg=Process crashed.
INSTRUMENTATION_CODE: 0
$ adb logcat
...
E mono : Unhandled Exception:
E mono : System.ObjectDisposedException: Cannot access a disposed object.
E mono : Object name: 'System.Net.Sockets.Socket'.
E mono : at System.Net.Sockets.Socket.ThrowIfDisposedAndClosed () <0xa93923f0 + 0x00054> in <filename unknown>:0
E mono : at System.Net.Sockets.Socket.AcceptAsync (System.Net.Sockets.SocketAsyncEventArgs e) <0x9b8f9680 + 0x0001b> in <filename unknown>:0
E mono : at System.Net.EndPointListener.Accept (System.Net.Sockets.Socket socket, System.Net.Sockets.SocketAsyncEventArgs e) <0x9b8f95d0 + 0x0003f> in <filename unknown>:0
E mono : at System.Net.EndPointListener.ProcessAccept (System.Net.Sockets.SocketAsyncEventArgs args) <0x9b8e0340 + 0x0007f> in <filename unknown>:0
E mono : at System.Net.EndPointListener.OnAccept (System.Object sender, System.Net.Sockets.SocketAsyncEventArgs e) <0x9b8e0310 + 0x00017> in <filename unknown>:0
E mono : at System.Net.Sockets.SocketAsyncEventArgs.OnCompleted (System.Net.Sockets.SocketAsyncEventArgs e) <0x9b8e02c8 + 0x0003b> in <filename unknown>:0
E mono : at System.Net.Sockets.SocketAsyncEventArgs.Complete () <0x9b8e02a0 + 0x0001f> in <filename unknown>:0
E mono : at System.Net.Sockets.Socket.<AcceptAsyncCallback>m__0 (System.IAsyncResult ares) <0x9b8dfd40 + 0x002af> in <filename unknown>:0
E mono : at System.Net.Sockets.SocketAsyncResult+<Complete>c__AnonStorey0.<>m__0 (System.Object _) <0xa892f720 + 0x0002b> in <filename unknown>:0
E mono : at System.Threading.QueueUserWorkItemCallback.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem () <0xa938f6b8 + 0x0002f> in <filename unknown>:0
E mono : at System.Threading.ThreadPoolWorkQueue.Dispatch () <0xa938e358 + 0x001bb> in <filename unknown>:0
E mono : at System.Threading._ThreadPoolWaitCallback.PerformWaitCallback () <0xa938e1a8 + 0x00007> in <filename unknown>:0
Looks like a Socket and/or ThreadPool bug in mono.
2016-05-16 21:55:13 +03:00
run-apk-tests :
[tests] Split APK tests into parallel and non-parallel runs (#2990)
Context: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android/plot/Tests%20times/
What do we want!
"Meaningful" performance metrics!
(For various definitions of "meaningful.")
Consider the "Xamarin.Forms app startup" times from builds 501, 502
and [504][0]:
| Configuration | Build 501 Time | Build 502 Time | Build 504 Time |
| ------------------------ | -------------- | -------------- | -------------- |
| **Release** | 1,378ms | 1,471ms | 2,425ms |
| **Release-AOT** | 5,104ms | 2,138ms | 2,699ms |
| **Release-Profiled-Aot** | 1,515ms | 1,686ms | 1,521ms |
What we *expect* to see is that Release-Profiled-Aot is fastest,
followed by Release-Aot, followed by Release.
What we instead see is that sometimes Release (JIT) is fastest,
faster than any form of AOT (e.g. builds 501 & 502)!
We believe that the cause of this us due to machine load: the
`Xamarin.Forms-Performance-Integration` test is run *in parallel*
with many of the other on-emulator unit tests. Thus, times fluctuate
wildly and inconsistently, depending on what other tests may be
running at the same time.
The fix? Don't run the `Xamarin.Forms-Performance-Integration`
and `locales` tests as part of the other unit tests. Instead, run
them separately, *sequentially*, to reduce the likelihood of other
processes on the machine creating inconsistent times.
(Note that this is probably not going to be a "complete" solution,
as e.g. build machine changes, new emulators, etc. will *also* cause
build times to fluctuate *between builds*. We just hope that
*intra-build* times will be consistent, matching our *expectations*
that Release is slowest and some form of AOT is fastest.)
The splitting is implemented by introducing new
`RunPerformanceApkTests` target and pre-filtering `@(TestApk)`,
`@(TestApkInstrumentation)` and `@(TestApkPermission)` item groups
for this target and also for `RunApkTests` target.
@grendello suggested adding `@(_TypeMapAssemblySource)` to
the `Outputs` of the `_GenerateJavaStubs` target to fix this error
observed during the build:
_CompileNativeAssemblySources:
[arm-linux-androideabi-as stderr] Assembler messages:
EXEC : [arm-linux-androideabi-as stderr] error : can't open typemap.mj.armeabi-v7a.s for reading: No such file or directory
which is caused by skipping `_GenerateJavaStubs` target here:
Target "_GenerateJavaStubs: (TargetId:223)" in file "…/bin/Release/lib/xamarin.android/xbuild/Xamarin/Android/Xamarin.Android.Common.targets" from project "…/xamarin-android/tests/Xamarin.Forms-Performance-Integration/Droid/Xamarin.Forms.Performance.Integration.Droid.csproj" (target "_GeneratePackageManagerJava" depends on it):
Skipping target "_GenerateJavaStubs" because all output files are up-to-date with respect to the input files.
Additionally add `$(AndroidEnableProfiledAot)` to
`@(_PropertyCacheItems)` and `$(IntermediateOutputPath)build.props`.
This should address:
error XA3001: Could not compile native assembly file: typemap.mj.armeabi-v7a.s
when rebuilding projects with the value of
`$(AndroidEnableProfiledAot)` changed.
[0]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android/504/
2019-05-03 19:10:21 +03:00
_r = 0 ; \
2019-08-24 16:14:05 +03:00
$( call MSBUILD_BINLOG,run-apk-tests,,Test) $( TEST_TARGETS) /t:RunApkTests /p:RunApkTestsTarget= RunPerformanceApkTests $( APK_TESTS_PROP) || _r = 1 ; \
$( call MSBUILD_BINLOG,run-apk-tests,,Test) $( TEST_TARGETS) /t:RunApkTests $( APK_TESTS_PROP) || _r = 1 ; \
[tests] Split APK tests into parallel and non-parallel runs (#2990)
Context: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android/plot/Tests%20times/
What do we want!
"Meaningful" performance metrics!
(For various definitions of "meaningful.")
Consider the "Xamarin.Forms app startup" times from builds 501, 502
and [504][0]:
| Configuration | Build 501 Time | Build 502 Time | Build 504 Time |
| ------------------------ | -------------- | -------------- | -------------- |
| **Release** | 1,378ms | 1,471ms | 2,425ms |
| **Release-AOT** | 5,104ms | 2,138ms | 2,699ms |
| **Release-Profiled-Aot** | 1,515ms | 1,686ms | 1,521ms |
What we *expect* to see is that Release-Profiled-Aot is fastest,
followed by Release-Aot, followed by Release.
What we instead see is that sometimes Release (JIT) is fastest,
faster than any form of AOT (e.g. builds 501 & 502)!
We believe that the cause of this us due to machine load: the
`Xamarin.Forms-Performance-Integration` test is run *in parallel*
with many of the other on-emulator unit tests. Thus, times fluctuate
wildly and inconsistently, depending on what other tests may be
running at the same time.
The fix? Don't run the `Xamarin.Forms-Performance-Integration`
and `locales` tests as part of the other unit tests. Instead, run
them separately, *sequentially*, to reduce the likelihood of other
processes on the machine creating inconsistent times.
(Note that this is probably not going to be a "complete" solution,
as e.g. build machine changes, new emulators, etc. will *also* cause
build times to fluctuate *between builds*. We just hope that
*intra-build* times will be consistent, matching our *expectations*
that Release is slowest and some form of AOT is fastest.)
The splitting is implemented by introducing new
`RunPerformanceApkTests` target and pre-filtering `@(TestApk)`,
`@(TestApkInstrumentation)` and `@(TestApkPermission)` item groups
for this target and also for `RunApkTests` target.
@grendello suggested adding `@(_TypeMapAssemblySource)` to
the `Outputs` of the `_GenerateJavaStubs` target to fix this error
observed during the build:
_CompileNativeAssemblySources:
[arm-linux-androideabi-as stderr] Assembler messages:
EXEC : [arm-linux-androideabi-as stderr] error : can't open typemap.mj.armeabi-v7a.s for reading: No such file or directory
which is caused by skipping `_GenerateJavaStubs` target here:
Target "_GenerateJavaStubs: (TargetId:223)" in file "…/bin/Release/lib/xamarin.android/xbuild/Xamarin/Android/Xamarin.Android.Common.targets" from project "…/xamarin-android/tests/Xamarin.Forms-Performance-Integration/Droid/Xamarin.Forms.Performance.Integration.Droid.csproj" (target "_GeneratePackageManagerJava" depends on it):
Skipping target "_GenerateJavaStubs" because all output files are up-to-date with respect to the input files.
Additionally add `$(AndroidEnableProfiledAot)` to
`@(_PropertyCacheItems)` and `$(IntermediateOutputPath)build.props`.
This should address:
error XA3001: Could not compile native assembly file: typemap.mj.armeabi-v7a.s
when rebuilding projects with the value of
`$(AndroidEnableProfiledAot)` changed.
[0]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android/504/
2019-05-03 19:10:21 +03:00
exit $$ _r
2017-11-15 20:32:20 +03:00
[performance] initial MSBuild project timing (#1569)
"Performance", in various guises, is an ongoing focus. For example,
we track [application startup times][time-plots].
[time-plots]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android/plot/Tests%20times/
The next area we want to concentrate on is build and deployment time,
and to do so we need to start collecting information so we know how
we're doing, and whether anything is getting worse.
The first step in tracking our build times is to track the build
times for some of our existing unit tests projects, and plot those
on Jenkins. This is done by using a custom MSBuild logger
`Xamarin.Android.Tools.BootstrapTasks.TimingLogger` which records
detailed timing information into an XML file:
<builds>
<build
start-ticks="636607852701529939"
start-threadid="1"
id="HelloWorld-FreshBuild"
commit="8dcd59e"
description="A fresh build after a clean checkout for HelloWorld"
end-ticks="636607852899547509"
end-threadid="1"
elapsed="00:00:19.8017570"
succeeded="True">
<toolsets>
<toolset version="15.0" path="C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\MSBuild\15.0\Bin" />
</toolsets>
<files>
<file id="1" path="C:\Users\jopepper\Desktop\Git\xamarin-android\samples\HelloWorld\HelloWorld.csproj" />
</files>
<project
start-ticks="636607852707145402"
start-threadid="6"
file-id="1"
end-ticks="636607852899326841"
end-threadid="6"
elapsed="00:00:19.2181439"
succeeded="True">
<target
start-ticks="636607852707365985"
start-threadid="6"
name="RedirectMonoAndroidSdkPaths"
file-id="2"
end-ticks="636607852708932958"
end-threadid="6"
elapsed="00:00:00.1566973"
succeeded="True">
<task
start-ticks="636607852708755105"
start-threadid="6"
name="SetVsMonoAndroidRegistryKey"
file-id="2"
end-ticks="636607852708895478"
end-threadid="6"
elapsed="00:00:00.0140373"
succeeded="True"
/>
</target>
</project>
</build>
</builds>
These timing XML files from multiple builds are aggregated by the new
`<ProcessMSBuildTiming/>` task, which generates a CSV file containing
the `//*/@elapsed` attribute values for us by the
[Jenkins Plots plugin][jenkins-plots].
[jenkins-plots]: https://wiki.jenkins.io/display/JENKINS/Plot+Plugin
The following projects have their build times collected:
- `samples/HelloWorld`
- `tests/Xamarin.Forms-Performance-Integration`
Across four types of builds:
- Fresh build (simulated fresh checkout)
- Second build (no changes)
- Touch C# file, build again
- Touch Android resource XML file, build again
We are timing the `SignAndroidPackage` and `Install` targets for each
of these build types.
Results are aggregated into `TestResult-MSBuild-times.csv` for
processing on Jenkins. After this gets merged into master, there will
be some configuration needed to setup the new plot.
Example results, in the aggregated CSV file:
| HelloWorld-FreshBuild-Debug | HelloWorld-FreshInstall-Debug | XF-FreshBuild-Debug | XF-FreshInstall-Debug |
| ---: | ---: | ---: | ---: |
| 00:00:16.3538291 | 00:00:04.7940071 | 00:00:27.5263076 | 00:00:22.6720137 |
Also included in the table, will be the `TouchCSharp` and
`TouchAndroidResource` builds for both of the `HelloWorld` and `XF`
projects for both `SignAndroidPackage` and `Install` targets.
Other changes:
- I let Visual Studio 2017 re-sort/fixup `Xamarin.Android.sln`
- Split up `timing.csproj` into `timing.targets` and `timing.projitems`
- Updated `Makefile` to include a `run-performance-tests` target.
`TimingLogger` is courtesy of @grendello.
2018-05-02 22:26:50 +03:00
run-performance-tests :
2019-08-24 16:14:05 +03:00
_r = 0 ; \
$( call MSBUILD_BINLOG,run-apk-tests,,Test) $( TEST_TARGETS) /t:RunApkTests /p:RunApkTestsTarget= RunPerformanceApkTests $( APK_TESTS_PROP) || _r = 1 ; \
$( call MSBUILD_BINLOG,run-performance-tests,,Test) $( TEST_TARGETS) /t:RunPerformanceTests || _r = 1 ; \
exit $$ _r
[performance] initial MSBuild project timing (#1569)
"Performance", in various guises, is an ongoing focus. For example,
we track [application startup times][time-plots].
[time-plots]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android/plot/Tests%20times/
The next area we want to concentrate on is build and deployment time,
and to do so we need to start collecting information so we know how
we're doing, and whether anything is getting worse.
The first step in tracking our build times is to track the build
times for some of our existing unit tests projects, and plot those
on Jenkins. This is done by using a custom MSBuild logger
`Xamarin.Android.Tools.BootstrapTasks.TimingLogger` which records
detailed timing information into an XML file:
<builds>
<build
start-ticks="636607852701529939"
start-threadid="1"
id="HelloWorld-FreshBuild"
commit="8dcd59e"
description="A fresh build after a clean checkout for HelloWorld"
end-ticks="636607852899547509"
end-threadid="1"
elapsed="00:00:19.8017570"
succeeded="True">
<toolsets>
<toolset version="15.0" path="C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\MSBuild\15.0\Bin" />
</toolsets>
<files>
<file id="1" path="C:\Users\jopepper\Desktop\Git\xamarin-android\samples\HelloWorld\HelloWorld.csproj" />
</files>
<project
start-ticks="636607852707145402"
start-threadid="6"
file-id="1"
end-ticks="636607852899326841"
end-threadid="6"
elapsed="00:00:19.2181439"
succeeded="True">
<target
start-ticks="636607852707365985"
start-threadid="6"
name="RedirectMonoAndroidSdkPaths"
file-id="2"
end-ticks="636607852708932958"
end-threadid="6"
elapsed="00:00:00.1566973"
succeeded="True">
<task
start-ticks="636607852708755105"
start-threadid="6"
name="SetVsMonoAndroidRegistryKey"
file-id="2"
end-ticks="636607852708895478"
end-threadid="6"
elapsed="00:00:00.0140373"
succeeded="True"
/>
</target>
</project>
</build>
</builds>
These timing XML files from multiple builds are aggregated by the new
`<ProcessMSBuildTiming/>` task, which generates a CSV file containing
the `//*/@elapsed` attribute values for us by the
[Jenkins Plots plugin][jenkins-plots].
[jenkins-plots]: https://wiki.jenkins.io/display/JENKINS/Plot+Plugin
The following projects have their build times collected:
- `samples/HelloWorld`
- `tests/Xamarin.Forms-Performance-Integration`
Across four types of builds:
- Fresh build (simulated fresh checkout)
- Second build (no changes)
- Touch C# file, build again
- Touch Android resource XML file, build again
We are timing the `SignAndroidPackage` and `Install` targets for each
of these build types.
Results are aggregated into `TestResult-MSBuild-times.csv` for
processing on Jenkins. After this gets merged into master, there will
be some configuration needed to setup the new plot.
Example results, in the aggregated CSV file:
| HelloWorld-FreshBuild-Debug | HelloWorld-FreshInstall-Debug | XF-FreshBuild-Debug | XF-FreshInstall-Debug |
| ---: | ---: | ---: | ---: |
| 00:00:16.3538291 | 00:00:04.7940071 | 00:00:27.5263076 | 00:00:22.6720137 |
Also included in the table, will be the `TouchCSharp` and
`TouchAndroidResource` builds for both of the `HelloWorld` and `XF`
projects for both `SignAndroidPackage` and `Install` targets.
Other changes:
- I let Visual Studio 2017 re-sort/fixup `Xamarin.Android.sln`
- Split up `timing.csproj` into `timing.targets` and `timing.projitems`
- Updated `Makefile` to include a `run-performance-tests` target.
`TimingLogger` is courtesy of @grendello.
2018-05-02 22:26:50 +03:00
2018-06-13 20:32:28 +03:00
list-nunit-tests :
$( MSBUILD) $( MSBUILD_FLAGS) $( TEST_TARGETS) /t:ListNUnitTests
2017-11-15 20:32:20 +03:00
i n c l u d e b u i l d - t o o l s / s c r i p t s / r u n t i m e - h e l p e r s . m k
[build] Xamarin.Android build preparation utility (#3051)
Fixes: https://github.com/xamarin/xamarin-android/issues/2562
The `make prepare` **make** target and `msbuild /t:Prepare` MSBuild
target are used to "prepare" a `xamarin-android` checkout so that
the subsequent `make all` or `msbuild /t:Build` invocation will work.
The preparation step involves ensuring that all known required
dependencies exist or are provisioned (54a7a029), such as the build
utilities (`autoconf`/etc.), cross-compilers (MinGW), the Android SDK
and NDK, and building "enough" to allow `Xamarin.Android.sln` to
build (see also 7343965a0).
Unfortunately, the Glorious Idea™ of using MSBuild for significant
portions of this process is "baroque", making it difficult to
understand and maintain, and the requirement on `msbuild` means
that things break in really bizarre and obscure ways on Linux and
macOS if the installed `mono` version is "too old"...and the
determination of "too old" was being done via `mono` & `msbuild`!
Furthermore, there was duplication and repetition of various bits of
build configuration on Linux and macOS between Makefile and the
MSBuild project system, which made it easier to make mistakes.
(Then there's the occasional bizarro failures that would result from
attempting to evaluate `$(PRODUCT_VERSION)` within `make`; it would
fail when `MONO_LOG_LEVEL=info` was set. Then there's...)
Instead of having `make prepare` be implemented as an unholy mess of
`make` and `msbuild`, introduce the new `build-tools/xaprepare` app.
`xaprepare` will be built and executed as part of `make prepare`,
and takes over responsibility for repo build preparation.
See also: build-tools/xaprepare/README.md
TODO: Windows support
2019-06-07 20:58:59 +03:00
.PHONY : prepare -build -init
prepare-build-init :
mkdir -p $( dir $( PREPARE_BUILD_LOG) )
msbuild $( PREPARE_RESTORE_FLAGS) $( PREPARE_SOLUTION) /t:Restore
.PHONY : prepare -build
prepare-build : prepare -build -init
msbuild $( PREPARE_MSBUILD_FLAGS) $( PREPARE_SOLUTION)
.PHONY : prepare
2019-07-10 02:00:50 +03:00
prepare : prepare -build
[build] Xamarin.Android build preparation utility (#3051)
Fixes: https://github.com/xamarin/xamarin-android/issues/2562
The `make prepare` **make** target and `msbuild /t:Prepare` MSBuild
target are used to "prepare" a `xamarin-android` checkout so that
the subsequent `make all` or `msbuild /t:Build` invocation will work.
The preparation step involves ensuring that all known required
dependencies exist or are provisioned (54a7a029), such as the build
utilities (`autoconf`/etc.), cross-compilers (MinGW), the Android SDK
and NDK, and building "enough" to allow `Xamarin.Android.sln` to
build (see also 7343965a0).
Unfortunately, the Glorious Idea™ of using MSBuild for significant
portions of this process is "baroque", making it difficult to
understand and maintain, and the requirement on `msbuild` means
that things break in really bizarre and obscure ways on Linux and
macOS if the installed `mono` version is "too old"...and the
determination of "too old" was being done via `mono` & `msbuild`!
Furthermore, there was duplication and repetition of various bits of
build configuration on Linux and macOS between Makefile and the
MSBuild project system, which made it easier to make mistakes.
(Then there's the occasional bizarro failures that would result from
attempting to evaluate `$(PRODUCT_VERSION)` within `make`; it would
fail when `MONO_LOG_LEVEL=info` was set. Then there's...)
Instead of having `make prepare` be implemented as an unholy mess of
`make` and `msbuild`, introduce the new `build-tools/xaprepare` app.
`xaprepare` will be built and executed as part of `make prepare`,
and takes over responsibility for repo build preparation.
See also: build-tools/xaprepare/README.md
TODO: Windows support
2019-06-07 20:58:59 +03:00
mono --debug $( PREPARE_EXE) $( _PREPARE_ARGS)
2019-11-12 13:37:20 +03:00
msbuild $( BOOTSTRAP_MSBUILD_FLAGS) $( BOOTSTRAP_SOLUTION)
[build] Xamarin.Android build preparation utility (#3051)
Fixes: https://github.com/xamarin/xamarin-android/issues/2562
The `make prepare` **make** target and `msbuild /t:Prepare` MSBuild
target are used to "prepare" a `xamarin-android` checkout so that
the subsequent `make all` or `msbuild /t:Build` invocation will work.
The preparation step involves ensuring that all known required
dependencies exist or are provisioned (54a7a029), such as the build
utilities (`autoconf`/etc.), cross-compilers (MinGW), the Android SDK
and NDK, and building "enough" to allow `Xamarin.Android.sln` to
build (see also 7343965a0).
Unfortunately, the Glorious Idea™ of using MSBuild for significant
portions of this process is "baroque", making it difficult to
understand and maintain, and the requirement on `msbuild` means
that things break in really bizarre and obscure ways on Linux and
macOS if the installed `mono` version is "too old"...and the
determination of "too old" was being done via `mono` & `msbuild`!
Furthermore, there was duplication and repetition of various bits of
build configuration on Linux and macOS between Makefile and the
MSBuild project system, which made it easier to make mistakes.
(Then there's the occasional bizarro failures that would result from
attempting to evaluate `$(PRODUCT_VERSION)` within `make`; it would
fail when `MONO_LOG_LEVEL=info` was set. Then there's...)
Instead of having `make prepare` be implemented as an unholy mess of
`make` and `msbuild`, introduce the new `build-tools/xaprepare` app.
`xaprepare` will be built and executed as part of `make prepare`,
and takes over responsibility for repo build preparation.
See also: build-tools/xaprepare/README.md
TODO: Windows support
2019-06-07 20:58:59 +03:00
.PHONY : prepare -help
prepare-help : prepare -build
mono --debug $( PREPARE_EXE) -h
Fix build break on newly reimaged bot (#4607)
In the process of creating a new macOS OSS PR workflow, I encountered
an issue where the build would consistently break when executing
`make prepare-update-mono`. The break occurred on the following
command *on a newly re-imaged agent (bot)*:
pgrep -lfi VBCSCompiler.exe
The command returns 0 if the managed compiler server `VBCSCompiler.exe`
is running; otherwise, it returns 1 if the process is not running.
When executing the build pipeline on the newly re-imaged machine, the
process is not running, `pgrep` command returns 1, and the build fails
with the following error:
pgrep -lfi VBCSCompiler.exe
make: *** [prepare-update-mono] Error 1
Turns out that on established build machines in the pool,
`VBCSCompiler.exe` is consistently found to be running at this stage
of the build. If you inspect a build log, you will see that the
`pgrep` command logged that `VBCSCompiler.exe` is running. Here's an
example from a recent build log:
pgrep -lfi VBCSCompiler.exe
36187 /Library/Frameworks/Mono.framework/Versions/6.10.0/bin/mono-sgen64 /Library/Frameworks/Mono.framework/Versions/6.10.0/lib/mono/msbuild/Current/bin/Roslyn/VBCSCompiler.exe -pipename:sX2CFXy8VTADY0R+V+y_1o2brmCa+1ch6aJTNiSJU4w
In discussing the issue with @directhex, the intent of the command is
quite the opposite: the intent is to ensure that `VBCSCompiler.exe` is
not running at this point of time. Based on the purpose of the target
to perform a mono update, we should ensure that processes associated
with the `mono` toolset are *shut down* and not running prior to the
update. Since `VBCSCompiler.exe` is part of the `mono` toolset it
seems to make more logical sense to ensure that it is not running. In
other words, ensure that the next invocation of `MSBuild` starts the
compiler server from the new location if `mono` is updated.
2020-04-24 22:55:43 +03:00
.PHONY : shutdown -compiler -server
shutdown-compiler-server :
# Ensure the VBCSCompiler.exe process isn't running during the mono update
pgrep -lfi VBCSCompiler.exe 2>/dev/null || true
@pid= ` pgrep -lfi VBCSCompiler.exe 2>/dev/null | awk '{ print $$1 }' ` ; \
echo " VBCSCompiler process ID (if running): $$ pid " ; \
if [ [ -n " $$ pid " ] ] ; then \
echo " Terminating the VBCSCompiler ' $$ pid' server process prior to updating mono " ; \
exitCode = 0 ; \
kill -HUP $$ pid 2>/dev/null || exitCode = $$ ? ; \
if [ [ $$ exitCode -eq 0 ] ] ; then \
sleep 2 ; \
pgrep -lfi VBCSCompiler.exe 2>/dev/null&& echo "ERROR: VBCSCompiler server still exists" || echo "Verified that the VBCSCompiler server process no longer exists" ; \
else \
echo " ERROR: Kill command failed with exit code $$ exitCode " ; \
fi \
fi
[build] Xamarin.Android build preparation utility (#3051)
Fixes: https://github.com/xamarin/xamarin-android/issues/2562
The `make prepare` **make** target and `msbuild /t:Prepare` MSBuild
target are used to "prepare" a `xamarin-android` checkout so that
the subsequent `make all` or `msbuild /t:Build` invocation will work.
The preparation step involves ensuring that all known required
dependencies exist or are provisioned (54a7a029), such as the build
utilities (`autoconf`/etc.), cross-compilers (MinGW), the Android SDK
and NDK, and building "enough" to allow `Xamarin.Android.sln` to
build (see also 7343965a0).
Unfortunately, the Glorious Idea™ of using MSBuild for significant
portions of this process is "baroque", making it difficult to
understand and maintain, and the requirement on `msbuild` means
that things break in really bizarre and obscure ways on Linux and
macOS if the installed `mono` version is "too old"...and the
determination of "too old" was being done via `mono` & `msbuild`!
Furthermore, there was duplication and repetition of various bits of
build configuration on Linux and macOS between Makefile and the
MSBuild project system, which made it easier to make mistakes.
(Then there's the occasional bizarro failures that would result from
attempting to evaluate `$(PRODUCT_VERSION)` within `make`; it would
fail when `MONO_LOG_LEVEL=info` was set. Then there's...)
Instead of having `make prepare` be implemented as an unholy mess of
`make` and `msbuild`, introduce the new `build-tools/xaprepare` app.
`xaprepare` will be built and executed as part of `make prepare`,
and takes over responsibility for repo build preparation.
See also: build-tools/xaprepare/README.md
TODO: Windows support
2019-06-07 20:58:59 +03:00
.PHONY : prepare -update -mono
Fix build break on newly reimaged bot (#4607)
In the process of creating a new macOS OSS PR workflow, I encountered
an issue where the build would consistently break when executing
`make prepare-update-mono`. The break occurred on the following
command *on a newly re-imaged agent (bot)*:
pgrep -lfi VBCSCompiler.exe
The command returns 0 if the managed compiler server `VBCSCompiler.exe`
is running; otherwise, it returns 1 if the process is not running.
When executing the build pipeline on the newly re-imaged machine, the
process is not running, `pgrep` command returns 1, and the build fails
with the following error:
pgrep -lfi VBCSCompiler.exe
make: *** [prepare-update-mono] Error 1
Turns out that on established build machines in the pool,
`VBCSCompiler.exe` is consistently found to be running at this stage
of the build. If you inspect a build log, you will see that the
`pgrep` command logged that `VBCSCompiler.exe` is running. Here's an
example from a recent build log:
pgrep -lfi VBCSCompiler.exe
36187 /Library/Frameworks/Mono.framework/Versions/6.10.0/bin/mono-sgen64 /Library/Frameworks/Mono.framework/Versions/6.10.0/lib/mono/msbuild/Current/bin/Roslyn/VBCSCompiler.exe -pipename:sX2CFXy8VTADY0R+V+y_1o2brmCa+1ch6aJTNiSJU4w
In discussing the issue with @directhex, the intent of the command is
quite the opposite: the intent is to ensure that `VBCSCompiler.exe` is
not running at this point of time. Based on the purpose of the target
to perform a mono update, we should ensure that processes associated
with the `mono` toolset are *shut down* and not running prior to the
update. Since `VBCSCompiler.exe` is part of the `mono` toolset it
seems to make more logical sense to ensure that it is not running. In
other words, ensure that the next invocation of `MSBuild` starts the
compiler server from the new location if `mono` is updated.
2020-04-24 22:55:43 +03:00
prepare-update-mono : prepare -build shutdown -compiler -server
[build] Simplify CI make and xaprepare invocations (#3279)
* [build] Simplify CI make and xaprepare invocations
Context: https://github.com/xamarin/xamarin-android/commit/9302d514a1437e5d1977da06dfbbe4e7054f33d5
Context: https://github.com/xamarin/xamarin-android/commit/f9f5b6cd120484b13df74e94638e07411d8693fa
We built a lot of hacks into our make file when initially working on the
xaprepare tool, as the windows and mac azure pipeline builds could not
be modified at the time. We now have these builds defined in yaml, and
as such we should be able to simplify the underlying make rules used by
all of our builds.
* Removes `prepare-build-ci` which no longer appears to be relevant,
as the extra args it specifies don't appear to be set before the
tool is built.
* Removes `prepare-jenkins` and `prepare-commercial`, as we can now
rely on the `prepare` rule again. The behavior of `prepare` will
change depending on the `_PREPARE_ARGS` varible, which is controlled
by different flags passed by our CI systems, `azure-pipelines.yaml`,
and `build.groovy`
* Introduces a new `prepareFlags` variable to build.groovy which is
used to control whether or not the "prepare all" (-a arg) should
be passed to xaprepare. This should not be specified when building
Jenkins PR builds, as they only build a subset of all supported ABIs
and frameworks. This should address the EmbeddedDSO test failures we
are encountering in Jenkins PR builds.
* Removes the MSBuild Auto Provision and ABI related args from make
invocations in azure-pipelines.yaml. Make variables will instead be
used to control these options.
* Removes the `prepare-update-mono` dependency from
`prepare-external-git-dependencies` as we should be able to call
this without needing to update mono.
* Adds a `prepare` dependency back to `jenkins`, now that the
`prepare-jenkins` step is no longer required.
* [build] Don't override _MSBUILD_ARGS when creating installers
Context: https://github.com/xamarin/xamarin-android/commit/e83ba0ddc159d15abcfac1a5a4bce5a53e165d3f
The `_MSBUILD_ARGS` override was initially required because we did not
import different `rules.make` based on how our build was configured and
prepared. The `xaprepare` tool now better controls the ABIs we should be
building and packaging for each build type, so this is no longer needed.
2019-06-26 11:08:00 +03:00
mono --debug $( PREPARE_EXE) $( _PREPARE_ARGS) -s:UpdateMono
[build] Xamarin.Android build preparation utility (#3051)
Fixes: https://github.com/xamarin/xamarin-android/issues/2562
The `make prepare` **make** target and `msbuild /t:Prepare` MSBuild
target are used to "prepare" a `xamarin-android` checkout so that
the subsequent `make all` or `msbuild /t:Build` invocation will work.
The preparation step involves ensuring that all known required
dependencies exist or are provisioned (54a7a029), such as the build
utilities (`autoconf`/etc.), cross-compilers (MinGW), the Android SDK
and NDK, and building "enough" to allow `Xamarin.Android.sln` to
build (see also 7343965a0).
Unfortunately, the Glorious Idea™ of using MSBuild for significant
portions of this process is "baroque", making it difficult to
understand and maintain, and the requirement on `msbuild` means
that things break in really bizarre and obscure ways on Linux and
macOS if the installed `mono` version is "too old"...and the
determination of "too old" was being done via `mono` & `msbuild`!
Furthermore, there was duplication and repetition of various bits of
build configuration on Linux and macOS between Makefile and the
MSBuild project system, which made it easier to make mistakes.
(Then there's the occasional bizarro failures that would result from
attempting to evaluate `$(PRODUCT_VERSION)` within `make`; it would
fail when `MONO_LOG_LEVEL=info` was set. Then there's...)
Instead of having `make prepare` be implemented as an unholy mess of
`make` and `msbuild`, introduce the new `build-tools/xaprepare` app.
`xaprepare` will be built and executed as part of `make prepare`,
and takes over responsibility for repo build preparation.
See also: build-tools/xaprepare/README.md
TODO: Windows support
2019-06-07 20:58:59 +03:00
[build] Simplify CI make and xaprepare invocations (#3279)
* [build] Simplify CI make and xaprepare invocations
Context: https://github.com/xamarin/xamarin-android/commit/9302d514a1437e5d1977da06dfbbe4e7054f33d5
Context: https://github.com/xamarin/xamarin-android/commit/f9f5b6cd120484b13df74e94638e07411d8693fa
We built a lot of hacks into our make file when initially working on the
xaprepare tool, as the windows and mac azure pipeline builds could not
be modified at the time. We now have these builds defined in yaml, and
as such we should be able to simplify the underlying make rules used by
all of our builds.
* Removes `prepare-build-ci` which no longer appears to be relevant,
as the extra args it specifies don't appear to be set before the
tool is built.
* Removes `prepare-jenkins` and `prepare-commercial`, as we can now
rely on the `prepare` rule again. The behavior of `prepare` will
change depending on the `_PREPARE_ARGS` varible, which is controlled
by different flags passed by our CI systems, `azure-pipelines.yaml`,
and `build.groovy`
* Introduces a new `prepareFlags` variable to build.groovy which is
used to control whether or not the "prepare all" (-a arg) should
be passed to xaprepare. This should not be specified when building
Jenkins PR builds, as they only build a subset of all supported ABIs
and frameworks. This should address the EmbeddedDSO test failures we
are encountering in Jenkins PR builds.
* Removes the MSBuild Auto Provision and ABI related args from make
invocations in azure-pipelines.yaml. Make variables will instead be
used to control these options.
* Removes the `prepare-update-mono` dependency from
`prepare-external-git-dependencies` as we should be able to call
this without needing to update mono.
* Adds a `prepare` dependency back to `jenkins`, now that the
`prepare-jenkins` step is no longer required.
* [build] Don't override _MSBUILD_ARGS when creating installers
Context: https://github.com/xamarin/xamarin-android/commit/e83ba0ddc159d15abcfac1a5a4bce5a53e165d3f
The `_MSBUILD_ARGS` override was initially required because we did not
import different `rules.make` based on how our build was configured and
prepared. The `xaprepare` tool now better controls the ABIs we should be
building and packaging for each build type, so this is no longer needed.
2019-06-26 11:08:00 +03:00
prepare-external-git-dependencies : prepare -build
mono --debug $( PREPARE_EXE) $( _PREPARE_ARGS) -s:PrepareExternalGitDependencies
2019-09-07 02:16:34 +03:00
APK_SIZES_REFERENCE_DIR = tests/apk-sizes-reference
update-apk-sizes-reference :
-mkdir -p $( APK_SIZES_REFERENCE_DIR)
cp -v *values-$( CONFIGURATION) .csv $( APK_SIZES_REFERENCE_DIR) /
