Make sure that environment variables (and directories they refer to)
used by Environment.SpecialFolders are created on application bootstrap
time. This avoids situations when code assumes the directories exist and
crashes if they don't. One example of such code are the sqlite tests as
reported in https://bugzilla.xamarin.com/show_bug.cgi?id=40569
The directories and environment variables are created in the bootstrap
code so that they are certain to exist before the actual app launches.
Fixes https://bugzilla.xamarin.com/show_bug.cgi?id=40569
The <DownloadUri/> Task doesn't create the output directory, so the
$(AndroidToolchainCacheDirectory) needs to be created before calling
the <DownloadUri/> task.
Commit e20863ea adds an `xabuild` script, which allows us to build
Android .apk application packages.
And lo, it was good!
*Unfortunately*, while a .apk was *generated*, the app on-device would
immediately crash with a bizarre error. Further investigation pointed
to the actual problem:
While xamarin-android was generating the .apk, the framework
assemblies that were bundled into the app were coming from the local
system Xamarin.Android install.
(Oops.)
Two things are required in order to cause the correct assemblies to
be bundled into the app.apk:
1. MSBuild/xbuild require that the
bin/$(Configuration)/lib/xbuild-frameworks/MonoAndroid/v*
directories contain a `RedistList/FrameworkList.xml` file,
otherwise the directory isn't recognized as a valid location for
MSBuild frameworks.
Since these files were missing, the built assemblies weren't
considered for use when compiling the app.
Update the build system to generate these files.
2. Xamarin.Android.Common.targets was overriding the $(CscToolExe)
and $(CscToolPath) MSBuild properties when running on OS X to
refer to an `smcs` script, which (i) doesn't exist in the
xamarin-android repo, so (ii) the system-installed smcs script was
used instead.
The smcs script is no longer necessary, and hasn't actually been
required for *years* (behold! laziness!), so just remove these
property overrides.
With these two changes, `xabuild` is not only able to generate an
Android .apk file, but the resulting .apk will actually run on-device!
Woo-hoo!
The `_BuildRuntimes` target would constantly re-run, because the
//Build/@Outputs files were wrong, and thus never existed.
Fix the Outputs references so that the `_BuildRuntimes` target doesn't
need to constantly execute.
(IDEs?! We don't need no stinkin' IDEs! [0, 1])
`tools/scripts/xabuild` is an `xbuild` ("MSBuild") -using script to
support building Xamarin.Android projects from the command-line,
without modifying or replacing the "system" Xamarin.Android install.
Also handy if there are multiple development branches available.
`tools/scripts/xabuild` takes the same command-line arguments as
xbuild(1) and MSBuild, allowing projects to be built and application
packages to be created:
${xamarin-android-checkout-path}/tools/scripts/xabuild /t:SignAndroidPackage
adb install bin/Debug/*-Signed.apk
Xamarin.Android.Build.Tasks.dll is updated to look for an alternate,
*available*, path to `libmonosgen-2.0.so`, and to remove the
%(Reference.Private) metadata so that all required assemblies are
installed into $(OutputPath). (Previous versions of Xamarin.Android
would encode the ABI into the filename, but this encoding should no
longer be necessary.)
android-toolchain is altered to provide `_GetAndroidNdkDirectory` and
`_GetAndroidNdkDirectory` targets, so that the xabuild script can
properly determine the path to the $(AndroidToolchainDirectory)
location.
[0]: In the spirit of: https://www.youtube.com/watch?v=VqomZQMZQCQ
[1]: Seriously, we often like to keep the "system" Xamarin.Android
untouched and unchanged, so that we can more easily build filed
bugs/etc., so the easiest way to build a Xamarin.Android project
from a development branch is from an `xbuild` wrapper script and
building from the command line [2].
[2]: https://developer.xamarin.com/guides/android/under_the_hood/build_process/#Build_Targets
Add a build-time @(ProjectReference) on the
Xamarin.Android.Tools.BootstrapTasks project so that
Xamarin.Android.Tools.BootstrapTasks is rebuilt if necessary.
- Xamarin.Android.sln: Disable Java.Interop build from here. It is built
from _BuildJavaInterop target
- Mono.Android.csproj: Add a ProjectReference to mono-runtimes project,
but don't *use* the reference. We just need it be built before
Mono.Android, so that we can resolve against the built mscorlib and
friends here.
- Mono.Android.targets: Mono.Android build was failing because it could
not find the referenced Java.Interop.dll in ResolveAssemblyReferences.
This was because _BuildJavaInterop was set to build
`BeforeTargets=ResolveReferences`. But the *real* resolution of
references is done in targets defined in `ResolveReferencesDependsOn`.
And that runs *before* the `BeforeTargets`! Which means that
_BuildJavaInterop was running too late!
Solution: `BeforeTargets=BeforeResolveReferences` - JonP !
Added Xamarin.Android.Build.Tasks and its supporting Libraries.
Code has also been updated to make uses of the new libraries in
the Java.Interop repository
Note : Xamarin.Android.Build.Tasks has a compile time code
generation step which generates Profile.g.cs. This file contains
the list of SharedRuntimeAssemblies. If building manually you will
need to make sure that Xamarin.Android.Tools.BootstrapTasks is
built first.
Configuration.props was <Import/>ing Configuration.Override.props
without specifying a directory. Unfortunately, this meant that when
e.g. src/Mono.Android/Mono.Android.csproj <Import/>ed
Configuration.props, xbuild/MSBuild could look for
Configuration.Override.props in the wrong directory (e.g.
src/Mono.Android), meaning any overridden MSBuild properties wouldn't
actually be overridden.
Update the Configuration.Override.props <Import/> paths to use
$(MSBuildThisFileDirectory) so that the correct
Configuration.Override.props location is used.
Commit 5777337e noted a SNAFU:
> One SNAFU: currently, Mono.Android.csproj conditinally <Import/>s a
> Mono.Android.projitems file generated by Java.Interop's generator
> tool, which contains a list of all the generated files.
>
> When the project is first loaded, Mono.Android.projitems will not
> exist, so on that initial build, source code will be generated but
> xbuild won't re-read Mono.Android.projitems (once it exists). This
> will result in a failing build.
The fix? Update @(Compile) and $(DefineConstants) after code
generation. This allows everything to compile after a single build,
without requiring two builds.
As it was already discussed in the team you should never use File.Move()
that is known to not work when source and destination partitions (drives
on Windows) are different.
The fix is not cool, it's just using "mv", but but works. (We need working
solution on Windows everywhere later anyways.)
This reverts commit 74ade5477b.
Mono.Android.csproj doens't need to import
Configuration.Override.props because Configuration.props already
<Import/>s it (when it exists).
See Configuration.props line 3:
<Import Project="Configuration.Override.props" Condition="Exists('Configuration.Override.props')" />
If Configuration.Override.props isn't being imported through
Configuration.props, we have bigger problems. <Import/>ing it from
Mono.Android.csproj would just be hiding these bigger problems.
The _BuildRuntimes target was executing unnecessarily:
Target _BuildRuntimes needs to be built as input file
'obj/Debug//armeabi-v7a/.stamp' is newer than output file
'obj/host/mono/profiler/.libs/libmono-profiler-log.dylib
This would *always* be true, because the output files were never
touched once they were built, so e.g. when copying a build tree from
one machine to another (Dropbox!), the dependency check would be
"wrong".
Improve the _BuildRuntimes target to explicitly <Touch/> the output
files so that -- short of timezone changes (!) -- the outputs will be
forced to be newer than the inputs.
- Build _GenerateJavaCallableWrappers *after* CoreBuild instead of
CoreCompile. This is because it depends on Mono.Android.dll being in
the OutputDir, but the *copy* to that directory happens *after*
CoreCompile.
This was hit when Mono.Android.dll would get compiled to
obj/Debug/Mono.Android.dll , but before it can be copied, jcw-gen is
run which expects Mono.Android.dll in the bin/Debug, thus breaking the
build.
- Paths passed to jcw-gen.exe as arguments should be relative to the
directory containing jcw-gen.exe or should be absolute.
This is required because of a bug in xbuild. It tries to look for the
Java.Interop projects which are referenced from the other projects, in
the sln itself, because it needs to figure out a build order for *all*
the projects. It should probably allow projects not being present in sln
and build them via the reference.
`clang` isn't the C compiler everywhere, and `clang++` isn't the
C++ everywhere. For example, Ubuntu provides `clang-3.5`.
Allow the host-native C and C++ compilers to be specified/overridden
via the new $(HostCc) and $(HostCxx) MSBuild properties.
One of the "cute" things in mono-runtimes.props is that
$(_CommonCFlags) changes based on $(Configuration): for Debug builds,
`-O0 -fno-omit-frame-pointer` is used (no optimizations and make stack
traces easier for gdb), while for Release builds `-O2` is used.
The natural result of this is that Debug and Release builds
*are not the same*, and that's a *good* thing.
...except that previously, both Debug and Release builds used the same
`obj\%(Identity)` directories, which meant there weren't separate
Debug and Release outputs, there were instead "whatever was built
first" outputs, which is *madness*.
Fix mono-runtimes.targets so that $(IntermediateOutputPath) is used
instead instead of `obj`. $(IntermediateOutputPath) should be
`obj\$(Configuration)`, so Debug and Release build artifacts will be
kept properly separate.
Additionally, fix the `Clean` target, so that the
$(IntermediateOutputPath) subdirectories are properly removed.
As an ancient "optimization" -- which probably needs rethinking, but
now is not the time for such rethinking -- during .apk generation
time, Mono.Android.dll was *not* processed along with the other
assemblies, the rationale being that it would slow things down.
(I don't think that this is true anymore...)
To optimize things, we instead generated a mono.android.jar during
Xamarin.Android build-time, and the .apk generation process would
include this prebuilt mono.android.jar into the final .apk.
For now, follow suite: Generate $(OutputPath)mono.android.jar after
building $(OutputPath)Mono.Android.dll. mono.android.jar contains the
Java Callable Wrappers for all Java.Lang.Object and
Java.Lang.Throwable subclasses within Mono.Android.dll.
"Promote" $(ApiLevel), in src/Mono.Android.csproj, to
$(AndroidApiLevel), which can now be overridden within
Configuration.Override.props.
Promote $(XAFrameworkVersion) to $(AndroidFrameworkVersion) as well.
This allows easly altering the default Android API level which will be
bound without requiring uncommittible changes.
`api-merge` is required to generate "forward compatible" binding
assemblies; see commit a073d99a for background.
Instead of binding a specific API description withiin Profiles, use
`api-merge` to generate a "merged", "forward compatible" API
description. This allows Mono.Android.dll targeting e.g. API-10 to
properly execute on e.g. API-8 devices.
Consider the MSBuild-usage scenario implied in 11143d18:
<ItemGroup>
<Profile Include="Profiles\api-*.xml" />
</ItemGroup>
<Exec Command="api-merge.exe -o merged.xml -s 'api-*.xml' @(Profile->'%(Identity)', ' ')" />
Assuming the files Profiles/api-10.xml and Profiles/api-15.xml, the
above will execute the command:
api-merge.exe -o merged.xml Profiles/api-10.xml Profiles/api-15.xml -s 'api-*.xml'
This seems fine...until you want to generate a merged XML description
for API-10, *not* API-15. At which point we're not emitting the
correct description, which is a problem.
To fix this, add `api-merge --last-description=NAME`, so that we can
control which is the last file to process. If NAME is found, then
API description processing will stop after encountering that file.
If NAME is *not* found, then all files will be processed.
This allows generating a merged API-10 desciption via:
<Exec Command="api-merge.exe -o merged.xml -s 'api-*.xml' --last-description=Profiles/api-10.xml @(Profile->'%(Identity)', ' ')" />
`api-merge` processes API descriptions in argument list order, and
requires that API descriptions be in ascending order, meaning
this is good:
api-merge.exe -o merged.xml api-1.xml api-2.xml api-3.xml
and this isn't good:
api-merge.exe -o merged.xml api-3.xml api-2.xml api-1.xml
(Descriptions need to be in ascending order so that API additions and
movements can be properly tracked.)
This was fine in the Old World Order™: a make(1) variable ensured that
ordering was ascending:
API_LEVELS = 1 2 3
merged.xml:
mono api-merge.xml -o merged.xml $(API_LEVELS:%=api-%.xml)
In the glorious new MSBuild-oriented order, we don't yet have an
equivalent to $(API_LEVELS), nor am I sure how to sanely represent
that (perhaps as an @(ApiLevels) ItemGroup?).
However, we still need a way to invoke api-merge.exe with files in the
correct order. There are thus two options:
1. Rely on MSBuild file glob references to be sorted correctly:
<ItemGroup>
<Profile Include="Profiles\*.xml.in" />
</ItemGroup>
and hope that @(Profile) is sorted correctly, or
2. Improve api-merge.exe to sort the descriptions.
I'm not keen on (1) -- I don't see why files should be sorted from
MSBuild -- which leaves (2):
Add an `api-merge -s GLOB` option so that description sorting can be
controlled. If `-s GLOB` isn't provided, inputs are not sorted:
# Bad order, descriptions are unsorted
api-merge.exe -o merged.xml api-3.xml api-1.xml
# Bad order, -s provided, descriptions are properly sorted
api-merge.exe -o merged.xml -s 'api-*.xml' api-3.xmlm api-1.xml
Using `-s GLOB` thus removes any dependency on MSBuild (not) sorting
files, while still ensuring that API descriptions are processed in the
correct order.
GLOB sorting is...a bit of a hack. The GLOB is translated into a
regular expression, and each glob character ('*', '?') is turned into
a regular expression group. When sorting, each group is processed
in-turn, we attempt to parse as an int, and if the parse passes, we
sort by the int values.
`api-merge` is a tool to support emitting *forward compatible*
binding assemblies.
Way back in the dark days before Mono for Android 4.2, a binding
assembly was only backward compatible: if Mono.Android.dll was
targeting API-8, the resulting app could run on API-8 *and later*.
The opposite was *not* true: you *might not* be able to use
Mono.Android.dll for API-10 and run it on an API-8 device.
This is *forward compatibility*, the ability for "a system" to accept
"input" intended for a later version [0].
*Java* apps were forward compatible. Mono for Android prior to 4.2
did *not* generate forward compatible apps.
The reason why was that a binding assembly directly bound what was in
the specified API level [1], so if a member "moved" in a
*backward compatible* manner -- moving
android.view.MotionEvent.getDeviceId() to the newly introduced
android.view.InputEvent base class -- it would not be
*forward compatible* for the binding assemblies we emitted, as the
e.g. API-10 binding assembly would attempt to resolve the type
android.view.InputEvent, which didn't exist in API-8, and would result
in an exception.
`api-merge` was the fix: instead of a Mono.Android.dll targeting a
*specific* API level, it would instead take the API descriptions for
the target API level *and all prior API levels*. It would thus have
enough information to track member "movements", and with a bit of
Java.Interop/tools/generator "magic" forward compatibility was
provided starting in Mono for Android 4.2 [2].
[0]: https://en.wikipedia.org/wiki/Forward_compatibility
[1]: http://lists.ximian.com/pipermail/monodroid/2011-November/007350.html
[2]: https://developer.xamarin.com/releases/android/mono_for_android_4/mono_for_android_4.2/#Improved_API_level_support
(*Now* things are starting to come together...)
One of the important infrastructural pieces of Xamarin.Android is
Mono.Android.dll, a "binding" for the Android SDK `android.jar` file.
Bindings are generated by using the `generator` tool from
Java.Interop, along with tools/jnienv-gen, and various other programs.
This in turn requires adding a git submodule for Java.Interop.
Stitch all these pieces together so that we can take an
*API description* (stored in Profiles\api-*.xml.in) and generate a
binding for that API description.
The generated files are located in
obj\$(Configuration)\android-$(ApiLevel), and the resulting
Mono.Android.dll is copied to
$(topdir)\bin\$(Configuration)\xbuild-frameworks\MonoAndroid\$(XAFrameworkVersion).
One SNAFU: currently, Mono.Android.csproj conditinally <Import/>s a
Mono.Android.projitems file generated by Java.Interop's generator
tool, which contains a list of all the generated files.
When the project is first loaded, Mono.Android.projitems will not
exist, so on that initial build, source code will be generated but
xbuild won't re-read Mono.Android.projitems (once it exists). This
will result in a failing build.
Simply rebuild the project to get a valid build, or use the `make all`
Makefile target.
- fix Android SDK Tools location.
- add SDK/NDK items for Linux.
- xbuild condition did not work and downloaded all the items. It was because
condition by metadata didn't work with the complicated expression.
So, split filtering and transformation into different tasks.
- xbuild aborts when the specified metadata %(HostOS) does not exist, so
specify dummy HostOS metadata (which is ugly but not too annoying).
(Linux build verification is still ongoing, not assuring that this fixes
all the possible existing issues.)
Jonathan Pryor
Atsushi Eno
Marek Habersack
Dean Ellis
Ankit Jain