Граф коммитов

1194 Коммитов

Автор SHA1 Сообщение Дата
Atsushi Eno f573af9cb2 [api-xml-adjuster] Don't parse JavaDoc (#982)
Bumps to Java.Interop/master/e1ad5066.

When adding a new API level, instead of using `class-parse -docspath`
to parse JavaDoc/AndroidDoc/etc., use `class-parse --parameter-names`
with a "JavaApiParameterNames" file generated by
[`xamarin-android-docimporter-ng`][import].

[import]: https://github.com/xamarin/java.interop/tree/e1ad506606/build-tools/xamarin-android-docimporter-ng/xamarin-android-docimporter-ng

Eventually we'll remove `src/Mono.Android/Profiles/api*.xml.in`
and instead generate those files at build-time by processing
`android.jar`.

Update `build-tools/api-xml-adjuster` to use the
JavaApiParameterNames files and generate `api-*.xml.in`, which now
takes only a few minutes.

Regenerate `src/Mono.Android/Profiles/api-*.xml.in` using the new
`api-xml-adjuster`. This results in some fixes, e.g.
`NEGATIVE_INFINITY` should be `(-1.0f / 0.0f)`, not `-Infinity`,
and some "parameter name regressions," e.g. in `api-24.xml.in`
some parameter names were changed *to* `p0` and other meaningless
names. *This is not breakage*; `api-merge` uses the previous
API level parameter names when it encounters `p0`/etc. names,
so these changes don't result in any API breakage.
2017-11-16 13:20:24 -05:00
Jonathan Peppers 3750fbf151 Bump to Java.Interop/master/1bcfcc5 (#1027)
Context: f90f00629a

Setting `%ANDROID_SDK_PATH%` within `RunTests.targets` didn't appear
to *actually* set `%ANDROID_SDK_PATH%` on Windows.

Bump to Java.Interop/master/1bcfcc57 so that `$(AndroidSdkDirectory)`
is instead used, which appears to allow many Java.Interop tests to
execute on Windows.

This bump also changes the "Java parameter names" file format to use
`;` as the comment character, not `#`, so that `#ctor` for
constructors works as intended.
2017-11-15 14:34:09 -05:00
Marek Habersack 1e0803c593
Add a bunch of Make targets to help rebuilding Mono runtime and BCL (#1029)
This basically implements the steps described in
Documentation/DevelopmentTips.md for the runtime and BCL assembles. Short help
is available by running

   make rebuild-help
2017-11-15 12:32:20 -05:00
Radek Doulik a1e65419c2 [Tests] Run runtime test in Release/Aot configuration again
Recently the migration of running the apk tests to msbuild
(cdf3bcc11a)
stopped running runtime test with Aot and thus we are missing
performance measurements for Release/Aot.

The reason of it is that `_AotName` property is not set, when calling
MSBuild task to run the apk tests. This patch sets the `AotAssemblies`
property when calling `RunApkTests.targets`
2017-11-15 17:44:58 +01:00
Jonathan Pryor 20c722a94a
[android-toolchain] Provide path to `autopoint` (#1026)
Context: https://devdiv.visualstudio.com/DevDiv/Default/_build/index?buildId=1138882&_a=summary
Context: https://github.com/xamarin/xamarin-android/pull/606#issuecomment-304171702

We [discovered 6 months ago][0] that a mono bump was failing to build
because:

[0]: https://jenkins.mono-project.com/job/xamarin-android-pr-builder/966

 1. The Jenkins machine building the PR has Homebrew 1.2.1 installed.

 2. The Jenkins machine attempting to build the PR has never built
    MXE before, as MXE is only required for *full* builds in which the
    mono runtime has changed.

 3. The MXE build failed, because `autopoint` wasn't in `$PATH`:

        Executing: make MXE_TARGETS="i686-w64-mingw32.static" gcc cmake zlib pthreads dlfcn-win32 mman-win32 PREFIX="/Users/builder/android-toolchain /mxe"
        Missing requirement: autopoint

The result is a very sad panda.

This *could* be fixed by force-linking the `gettext` package:

	$ brew link --force gettext

However, this is considered to be overkill, as there may be multiple
other `autopoint`s in `$PATH`. (Presumably there's *some* reason why
Brew 1.2 doesn't install `autopoint` into `$PATH`...)

This same issue also reappeared while attempting to build
xamarin-android within VSTS, as the VSTS bots similarly haven't run
`brew link --force gettext` either.

Fix this issue by overriding `$PATH` when building MXE so that it
contains the path to `autopoint`, as returned by:

	dirname $(brew list gettext | grep autopoint$)
2017-11-15 06:26:01 -05:00
Alexander Köplinger fd8346295f Bump to mono/2017-10/a3fb0a12 (#1023)
Context: https://github.com/mono/mono/pull/5984

Moves `tests/Xamarin.Android.Bcl-Tests/Resources/LinkerDescription.xml`
into mono.

Mono now has additional tests and embeds the `LinkerDescription.xml`
directly into the test assemblies, so we can remove it here.
2017-11-14 18:05:08 -05:00
Jonathan Peppers 096210c1a8 [tests] Allow include/exclude test categories for .apks (#1024)
In some QA needs to run tests on devices without a network
connection, so an option to disable tests that rely on the network in
`Mono.Android-Tests` is needed.

NUnit has the option to include or exclude tests based on the
`[CategoryAttribute]` custom attribute. Various tests that appear to
use the internet are now decorated with `[Category("InetAccess")]`.
To verify this works, I ran the tests with my internet disabled. The
test run for `Mono.Android-Tests` went down from 106 to 84 tests,
with no failures.

To exclude a category on Windows, specify `$(ExcludeCategories)`

    msbuild Xamarin.Android.sln /t:RunApkTests /p:ExcludeCategories=InetAccess

On MacOS/Linux, set the `EXCLUDECATEGORIES` make variable:

    make run-apk-tests EXCLUDECATEGORIES=InetAccess

If you want to specify multiple categories, use `:` (colon) between
each category. This delimiter is used for various values throughout
Xamarin.Android's build because it works well from the command line
for both MSBuild and make.

Categories can also be explicitly included, by setting the
`$(IncludeCategories)` MSBuild property or the `INCLUDECATEGORIES`
make variable.
2017-11-14 18:02:18 -05:00
Jonathan Pryor f56ca4ab05 [Xamarin.Android.Build.Tasks] Use `adb install -r` (#1025)
Commit 70d9e2ff added a "rudimentary `Install` target". This worked in
that it would install a package, but was less than ideal because it
would *fail* on multiple installs:

	$ msbuild /t:Install App.csproj
	# works
	$ msbuild /t:Install App.csproj
	Failed to install App-Signed.apk: Failure [INSTALL_FAILED_ALREADY_EXISTS: Attempt to re-install App without first uninstalling.]

Use `adb install -r` instead of `adb install` so that we *replace*
existing packages, should there be an existing package.

This should remove a potential headache from using the OSS bits with
Visual Studio for Mac/etc.
2017-11-14 09:57:36 +00:00
Jonathan Peppers 3cf7b8c557 [tests] fix Java.Interop test results copy (#1016)
On Windows, under MSBuild, I noticed a clean build was not copying the
test results from `external/Java.Interop` to the root directory. Two
things were causing this to be overlooked: the `<Copy />` task left the
original files from past test runs, and`@(_JavaInteropTestResults)`
was calculated at the top of the file. I *think* this is
somehow working on xbuild.

To fix this, I moved the location of `@(_JavaInteropTestResults)`
to happen during the target, instead of at the top of the
file. I also added a `<Delete />` task after the `<Copy />` to keep a clean
`external/Java.Interop` directory. It doesn't look like xbuild supports
`<Move />`.
2017-11-13 19:43:24 -05:00
Jonathan Peppers 19bd3de8a0 Bump to Java.Interop/master/a908424 (#1022)
Context: https://github.com/xamarin/java.interop/pull/209

This should resolve the broken unit tests on the master Jenkins job.
2017-11-10 19:00:56 -05:00
Zoltan Varga d80b485bcb Remove support for debugging with gdb, and embedding gdb debugserver files into .apk files. The lldb integration at: (#1021)
https://github.com/mono/lldb-binaries

should be used instead.
2017-11-10 18:08:03 +00:00
Jonathan Peppers e0c0c2bae5 [Mono.Android-Tests] Skip _RecordApkSizes on Windows (#1020)
The `_RecordApkSizes` target is using `unzip` and `stat` Unix
commands. There is not currently a simple way to implement this
cross-platform, so it is simpler to just skip this target on Windows.
2017-11-09 17:28:42 -05:00
Dean Ellis 8efe76feb8 [Xamarin.Android.Build.Tasks] Rework Parellel.ForEach (#1019)
We should also provide the `CancelationToken` and the
`TaskScheduler` for the `Parallel.ForEach()` calls.

Removed a call to `Log.LogErrorFromException()` as the
continuation does NOT run on the UI thread.
2017-11-09 17:27:10 -05:00
Jonathan Pryor 2c1375561c
Bump to Java.Interop/master/25af6875 (#1018)
Adds support for `class-parse --parameter-names=FILE` and support for
the `JavaDocletType.JavaApiParameterNamesXml` format for specifying
Java method parameter names.
2017-11-09 16:19:48 -05:00
Dean Ellis b61782bf20 [Xamarin.Android.Build.Tasks] Aapt MSBuild Task treats no default translation warnings as MSBuild errors (#1006)
Fixes: https://bugzilla.xamarin.com/show_bug.cgi?id=60445

The regex is picking up the warning in

	warning: string 'app_name1' has no default translation.

but the "warning" part is being picked up by the very first
capture group so the current code does not detect its a warning.

Update the regular expression to better support this string,
and add unit tests (yay!) to check our regex capture behavior.
2017-11-09 14:33:24 -05:00
Jonathan Peppers 9d5cf1a68c [Mono.Android-Tests] Support system Xamarin.Android builds (#1017)
`Mono.Android-Tests.csproj` needed to use `$(MSBuildExtensionsPath)`
as part of locating `Xamarin.Android.CSharp.targets`. The other test
`.apk` projects were already doing this.

The other change here is to set `$(XAIntegratedTests)` to False by
default, so that it is not required to be set when building the test
`.apks` with your system Xamarin.Android installation.

With this change, all on-device `.apk`-based unit tests should be
buildable against a system Xamarin.Android install, and not just the
in-tree build artifacts.

*Note*: Xamarin.Android 8.0 (d15-5) or later is required in order to
build the `.apk`-based unit tests due to commit e9daf5ea, which
updates `Xamarin.Android.NUnitLite.dll`.
The `Xamarin.Android.Bcl-Tests` project requires the new members.
2017-11-09 14:29:59 -05:00
Dean Ellis c428ad3482 [Xamarin.Android.Build.Tasks] Cache AndroidIncludeDebugSymbols (#1015)
The VS team have asked if there is a way to know which runtime
is being used by a particular build configuration.

Since we calculate the `AndroidIncludeDebugSymbols` when the project
is loaded is makes sense to store that in the `build.props` file.
It should not change very often unless the user constantly
updated the project. This way the VS team can read the `build.props`
file and look for the value.

We could have added a new target, but at the point this is needed
VS can not call MSBuild targets or read properties.
2017-11-09 14:22:45 -05:00
Atsushi Eno fcdb34704c [msbuild] Add desugar support. (#991)
This should fix bug #60336.
2017-11-09 14:06:25 +00:00
Jonathan Peppers cdf3bcc11a [tests] migrate test targets to MSBuild (#949)
The end goal here is to enable Windows users to easily run the various
types of tests. Commands have been migrated to MSBuild, keeping `make`
commands the same for macOS and linux.

~~ Usage ~~

Windows:

```
msbuild Xamarin.Android.sln /t:RunAllTests
msbuild Xamarin.Android.sln /t:RunNUnitTests
msbuild Xamarin.Android.sln /t:RunJavaInteropTests
msbuild Xamarin.Android.sln /t:RunApkTests
```

macOS/linux should remain unchanged:

```
make run-all-tests
make run-nunit-tests
make run-ji-tests
make run-apk-tests
```

~~ Changes ~~

  - Added a new `build-tools/scripts/RunTests.targets`
  - `Before.Xamarin.Android.sln.targets` includes these test targets
  - A new `<SetEnvironmentVariable/>` task is needed, added to
    `xa-prep-tasks`
  - Ported any environment variables set in Makefile to MSBuild
  - `Mono.Android-Tests.csproj` needs to remove the `@(ProjectReference)`
    within `xa-prep-tasks`:
    - Otherwise `xa-prep-tasks.dll` can become locked on Windows
    - A nested `xabuild.exe` call will attempt to overwrite it
  - Update `.gitignore` for `*.rawproto` and the generated
    `Xamarin.Android.Common.props`
2017-11-08 15:38:27 -05:00
Dean Ellis ed6f18ca0b [Xamarin.ProjectTools] Allow msbuild tests to use system x-a. (#1014)
We want to be able to run the msbuild tests against a system
installed xamarin-android. So we need to fallback to using
`msbuild` or `xbuild` if the required `xabuild` is NOT in place.

So if we do a full build `xabuild` will be in `bin\$(Configuration)\bin`
and the tests will run under the local environment. But if we just build
the tests `xabuild` will not exist since its copied as part of the
XABuild project build process.
2017-11-07 16:26:50 -05:00
Dean Ellis 68d7ed4c1b [Xamarin.Android.Project.Tools] Allow VSfM to build (#1012)
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58994

There is a bug in Visual Studio for Mac which does not like
Conditionals on things ike `@(Content)` or `@(ProjectReference)`.
This commit moves those items into `.projitems` files which then
allows us to use the `<Import/>` with the required Conditional.
This works around the issue.
2017-11-07 13:50:30 -05:00
Jonathan Peppers 2d8c0cff2d [build] AndroidSdkDirectory and AndroidNdkDirectory can be overridden (#1013)
To be able to run tests against your system's setup of the Android SDK,
we need `$(AndroidSdkDirectory)` and `$(AndroidNdkDirectory)` to be able to
be overidden.
2017-11-07 13:48:16 -05:00
Dean Ellis 6bfa199d71 [Xamarin.Android.Build.Tasks] Remove Unsupported Lint Checks (#975)
If a user provides lint checks which are not supported by
the current lint version it will error.

	Invalid id or ategory "StatckFieldLeak"

This is normally ok. But in our build system we sometimes end up
on a bot which has an older version of lint. And in this case
the test fails, which in reality it should ignore the id we
are trying to use.

So this commit addds some validation to the Disabled and Enabled
checks. If its not supported we will remove the check and issue
a warning.
2017-11-07 09:34:40 -05:00
Jonathan Peppers b10508aa25 [Xamarin.Android.Build.Tests] Workaround broken lint.bat (#994)
Context: https://issuetracker.google.com/issues/68753324

The Windows' SDK tools v26 `lint.bat` tool installed to
`$(AndroidToolchainDirectory)\sdk\tools\bin` appears to be broken, so
a all tests which use it are failing.

This can be worked around -- allowing the `lint`-based tests to pass
on Windows -- by setting the `%JAVA_OPTS%` environment variable.
Hopefully this is a temporary workaround until a new version of the
SDK tools comes out that fixes this issue.

One other thing to note here is that the Android SDK installed by
Visual Studio uses the SDK tools version 25.x, which does not appear
to have this issue on windows.
2017-11-06 21:44:52 -05:00
Jonathan Peppers 73121ee699 [xabuild] Build .NET standard projects (#1011)
Context: https://github.com/Microsoft/msbuild/blob/master/src/Shared/BuildEnvironmentHelper.cs#L511

`xabuild.exe` did not support building .NET standard projects. It turns
out a `$MSBuildSDKPaths` environment variable needed to be set for locating
the `Sdks` directory. On Windows, this path is relative to Visual Studio /
MSBuild, but on Mac I had to hardcode the path to .NET Core. It seems to
use the same path on Linux as well.
2017-11-06 21:35:08 -05:00
Bernhard Urban 628672fee7 [mono-runtimes] Disable mono's IL interpreter (#1010)
Context: https://github.com/mono/mono/pull/5925

Mono's IL interpreter will be enabled by default in mono once the
above PR merged. Xamarin.Android doesn't need an IL interpreter --
we have a JIT! -- so disable the interpreter so that it doesn't
impact the size of `libmonosgen-2.0.so`.
2017-11-06 15:52:54 -05:00
Alexander Köplinger 1672ab073c Bump to mono/2017-10/dcbdec80; reenable tests (#1009)
The tests were fixed in Mono by:

- https://github.com/mono/mono/pull/5720
- https://github.com/mono/mono/pull/5942
2017-11-04 17:05:26 -04:00
Alexander Köplinger 8448a43bc8 Bump to mono/2017-10/56195c0c; remove corlib test linker (#999)
Context: 5a5ab38c9e

Add tests for the `*Attributes` which were being linked away by mono.
We should be able to remove the linker exclusions now.
2017-11-03 20:33:33 -04:00
Jonathan Peppers 4ee1c3d5b6 [tests] Fix issue in ProjectBuilder.Cleanup (#1005)
This method was not using the full path to the project directory, and so
it wasn't actually deleting anything. This caused test failures on
Windows in tests such as `ManifestTest.Bug12935`. This particular test
was modifying the contents of `AndroidManifest.xml`, and checking the
results of the build afterward. Since the project files were not
deleted, changes to the `AndroidManifest.xml` were not getting written
to disk during the test.
2017-11-03 22:53:17 +00:00
Jonathan Dick f11b87317d [Xamarin.Android.Build.Tasks] api.xml in @(JavaDocIndex) (#920)
Context: 806082f26b

`generator` will extract method parameter name information from a
variety of sources, including JavaDoc documentation -- via
`$(JavaDocPaths)`, `$(JavaDoc7Paths)`, `$(JavaDoc8Paths)`,
`$(DroidDocPaths)`, and `@(JavaDocJar)` (after extracting the `.jar`
file) -- all in an effort to avoid the default parameter names of
`p0`, `p1`, etc.

Java.Interop@806082f2 updated `generator` so that it would *also*
extract method parameter name information from `api.xml`-formatted
XML files:

```xml
<api>
  <package name="my.example">
    <class name="MyType">
      <method name="myMethod" returns="void">
        <parameter name="value" type="java.lang.String" />
<!-- ... -->
```

What's desired is a way to provide an `api.xml`-formatted file to the
`<ClassParse/>` task so that the generated `api.xml` contains the
desired parameter names.

Remove the `ClassParse.JavaDocPaths`, `ClassParse.Java7DocPaths`,
`ClassParse.Java8DocPaths`, `ClassParse.DroidDocPaths`, and
`ClassParse.JavaDocs` properties, replacing them with a new
`ClassParse.DocumentationPaths` property, and pass *all* of the old
values *alongside* `@(JavaDocIndex)` for
`ClassParse.DocumentationPaths`.

Thus, to manually specify method parameter names via an
`api.xml`-formatted file:

 1. Add the file to the `.csproj`, and
 2. Set the **Build action** to `@(JavaDocIndex)`

The `api.xml`-formatted file will then be provided to `<ClassParse>`,
and the desired names will be used in generated code.

*Note*: `@(JavaDocIndex)` is *only* supported when
`$(AndroidClassParser)` is set to `class-parse`. It cannot be
used with the older `jar2xml` parser, and attempting to do so will
result in an error of some sort. (No idea what it is, we haven't
tested this scenario to see what the error will be.)
2017-11-03 15:55:16 -04:00
Jonathan Peppers fe8d607c11 [Xamarin.Android.Build.Tasks] Fix Profile.g.cs rebuild (#1000)
`obj/Debug/Profile.g.cs` is getting regenerated on every build,
forcing `Xamarin.Android.Build.Tasks.csproj` to rebuild *every* time.
It looks like the `IncrementalClean` target is the culprit, in that
it deleted `obj/Debug/Profile.g.cs` file after the build completed.

The fix here is to add the *absolute path* of `Profile.g.cs` to
`@(FileWrites)`. (Non-absolute paths still result in the
`IncrementalClean` target removing the file.)

Update `$(_GenerateProfileClass)` so that it's an absolute path.
2017-11-03 12:49:15 -04:00
Radek Doulik 10f57e0a29 [tests] Measure NUnit test end times (#1007)
Added `NUnit.results` label to the default time performance definition
file. It matches NUnit line with results, which is printed out at the
end of the NUnit driven test.

This will add `NUnit.results-$(Configuration)` columns to the `Bcl`,
`Jcw`, `Locale`, and `Runtime merged` plots at:

https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android/plot/Tests%20times/
2017-11-03 10:57:48 -04:00
Dean Ellis 6299c6ec5c [Xamarin.Android.Build.Tasks] ApkSigner Failed if minsdk > maxSdk (#1003)
Fixes: https://bugzilla.xamarin.com/show_bug.cgi?id=60444

We should NOT ever pass a `//uses-sdk/@android:minSdkVersion` value
that is greater than the `//uses-sdk/@android:maxSdkVersion` value.
It results in the following error:

	Min API Level (10) > max API Level (8)

We should check the values and use the minimum value for
`//uses-sdk/@android:minSdkVersion`.
2017-11-02 17:20:39 -04:00
Dean Ellis b3e98204e4 [Xamarin.Android.ProjectTools] Fix logic error in native crash detection. (#1002)
Commit 855ad009 added logic to detect if a test had a native crash and to
retry the test. There was a flaw in the logic such that it would
run every test twice...

Fix that.
2017-11-02 12:52:06 -04:00
Jonathan Peppers 63fc278cc4 [Xamarin.Forms-Performance-Integration] Use AndroidFrameworkVersion (#1004)
Other Android projects (such as test APKs), set `$(TargetFrameworkVersion)`
by importing `Configuration.props` and using `$(AndroidFrameworkVersion)`.
`$(AndroidUseLatestPlatformSdk)` should also be set to `false`.

Not sure why this hasn't caused any issues yet, I first noticed this
causing a failure on PR #997.
2017-11-02 12:49:24 -04:00
Marek Habersack e6b85552ef [Mono.Android] Don't optimize URL encoding (#1001)
Fixes: https://bugzilla.xamarin.com/show_bug.cgi?id=60473

In the case when the URL doesn't contain any query elements we
optimized for speed and memory by not using `UrlBuilder` to do the
encoding work on the URL. However, it causes problems with URLs
that are sent pre-encoded.

Remove the optimization and rely on `UrlBuilder` in all cases.
2017-11-02 12:43:42 -04:00
Dean Ellis d628a98c09 [Xamarin.Android.Build.Tasks] Handle unknown version from lint.bat (#996)
Turns out `lint.bat` does not always return a version number.
It returns stuff like

	lint: version unknown
	lint: unknown version

not very helpful. So we need to handle that case and return a
empty verison.
2017-11-01 13:50:14 -04:00
Dean Ellis aeec3ae4dd [Xamarin.Android.Build.Tasks] Speed up ResolveLibraryProjectImports (#955)
Rather than trying to ALWAYS unset the ReadOnly flag, lets only
do it if the directory is read only. Also use `EnumerateFiles()`
rather than `GetFiles()` to get the `.jar` files.
`EnumerateFiles()` is more efficient.
2017-11-01 10:45:08 -04:00
Dean Ellis 7f45a6d20b [Xamarin.Android.Build.Tasks] NRE getting android.jar path (#976)
Fixes: https://bugzilla.xamarin.com/show_bug.cgi?id=60324

If an `android.jar` file does *not* exist for a target platform we
should raise a normal error. Instead we throw a
`NullReferenceException`.

	…/xamarin-android/bin/Debug/lib/xamarin.android/xbuild/Xamarin/Android/Xamarin.Android.Common.targets: error : Error executing task GetJavaPlatformJar: Value cannot be null.
	Parameter name: path1
	Error executing task GetJavaPlatformJar: System.ArgumentNullException: Value cannot be null.
	Parameter name: path1
	  at System.IO.Path.Combine (System.String path1, System.String path2) [0x00003] in <4656b2b94f914437bce672312dd9e44b>:0 
	  at Xamarin.Android.Tasks.GetJavaPlatformJar.Execute () [0x00188] in <3fa51b5c8b554c02b4c7f549bcabde77>:0 
	  at Microsoft.Build.BuildEngine.TaskEngine.Execute () [0x00000] in <cbad9a49d2cc4e87ad2241615dd4666c>:0 
	  at Microsoft.Build.BuildEngine.BuildTask.Execute () [0x0008d] in <cbad9a49d2cc4e87ad2241615dd4666c>:0 

Oops.

The cause is that
`AndroidSdkInfo.TryGetPlatformDirectoryFromApiLevel()` can return
`null`, which is by design. We need to handle that in the places
where it's called, and provide an XA5207 error message when
`TryGetPlatformDirectoryFromApiLevel()` returns `null`.
2017-11-01 10:43:56 -04:00
Radek Doulik 386e6cbe1b [linker] Use mono's linker submodule (#944)
Use mono's `external/linker` submodule reference instead of including
our own, separate, `linker` submodule reference.

This way we are more consistent with xamarin-macios, which also uses
mono's linker submodule.
2017-11-01 10:17:19 -04:00
Dean Ellis 855ad00931 [Xamarin.Android.Build.Tests] Retry builds on native crashes (#993)
Sometimes when running `Xamarin.Android.Build.Tests`, mono crashes:

	Stacktrace:
	
	  at <unknown> <0xffffffff>
	  at (wrapper managed-to-native) object.__icall_wrapper_mono_gc_alloc_obj (intptr,intptr) [0x00000] in <4fdc5ed61a074cafb49fa42deb20d521>:0
	  at (wrapper alloc) object.AllocSmall (intptr,intptr) <0x000fa>
	  at System.IO.FileSystemEnumerableIterator`1<TSource_REF>.CreateSearchResult (System.IO.Directory/SearchData,Microsoft.Win32.Win32Native/WIN32_FIND_DATA) [0x00023] in <4fdc5ed61a074cafb49fa42deb20d521>:0
	# Actual stack trace doesn't *really* matter, and is basically random
	...
	Native stacktrace:
	
		0   mono                                0x000000010d9df481 mono_handle_native_crash + 257
	...
	=================================================================
	Got a SIGSEGV while executing native code. This usually indicates
	a fatal error in the mono runtime or one of the native libraries 
	used by your application.
	=================================================================

This is decidedly Not Good™.

However, a significant percent of the time, the crash *also* isn't
reproducible: a re-run of the build+tests will result in all tests
passing. However, a re-run can take upwards of 2-6 *hours* --
depending mostly on whether the PR has a mono bump -- so it is
undesirable to re-run builds just to see if it's a "random" error.

Instead, update `Builder.BuildInternal()` so that if it determines
that mono crashed -- via the `Got a SIGSEGV…` message -- then we'll
re-try the crashing command. (Just once!)

This should hopefully reduce the number of test failures we need
to investigate.

The *downside* is that the #runtime team would dearly love to know
about these crashes, so *they* can investigate. This is *partially*
supported by emitting the following message into the build output
when a native crash is detected:

	Native crash detected! Running the build for … again.

This is less than ideal, and may be improved in the future.
2017-11-01 10:14:11 -04:00
Alexander Köplinger c43c5e671a Bump to mono/2017-10/7e85cfa5 (#943)
Bump to xamarin-android-api-compatibility/master/c050ba3c. Updates
expected APIs for BCL assemblies.

Use `azureedge.net` instead of `blob.core.windows.net`. It's behind
a CDN and way faster outside US.

Use the new profiler init symbol name, `mono_profiler_init()`.

Avoid dumping the content of `libmono-profiler-log.so` when tests
fail. The binary contents aren't meaningful, and needlessly bloat the
output log.

Fix `eglib` path, as mono changed it's location.

Don't skip errors from mono's `make test` target anymore.

Stop building Mono's `net_4_x` profile. It isn't required to build
the `monodroid` profile and *not* building it saves lots of time.
The tools needed by XA are taken from the new `monodroid_tools`
profile instead.

Update Mono version phrasing in `README.md`. Using an explicit
version is just a recipe for outdated info, instead rewrite it to be
generic since the build will complain on an outdated Mono anyway.

Add more exclusions to `LinkerDescription.xml` for BCL tests, which
are used in the new `System.Reflection.Emit.SaveTest()`.

Update `.gitignore` for some paths. `Xamarin.Android.Common.props` is
generated as part of the build, and the `.rawproto` files created by
gradle should be ignored.

Context: https://bugzilla.xamarin.com/show_bug.cgi?id=4786
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=5798
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=5804
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=11696
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=17325
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=23153
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=25755
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=36053
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=36634
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=40556
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=41279
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=43172
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=43563
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=44266
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=46683
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=52157
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=52621
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=55603
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=56653
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=56684
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57069
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57160
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57197
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57222
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57232
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57301
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57476
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57488
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57496
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57505
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57528
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57629
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57691
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57744
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57796
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57850
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57851
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57887
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57918
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57930
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57936
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=57959
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58100
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58168
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58210
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58291
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58317
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58344
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58361
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58379
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58399
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58411
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58421
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58423
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58454
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58612
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58621
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58728
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58738
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58770
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58782
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58793
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58809
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58829
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58877
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58901
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58911
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=58965
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=59039
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=59154
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=59180
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=59235
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=59281
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=59364
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=59436
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=59608
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=59664
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=59824
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=60233
2017-10-31 11:55:44 -04:00
Dean Ellis 9254af3315 [Xamarin.Android.Build.Tests] Add Build Log to the Exception data. (#961)
If the build fails we get a generic message to look at the
`build.log`. This is a pain because the build log is HUGE! and takes
ages to find the problem.

It seems NUnit does not include the `FailedBuildException.BuildLog`
property. Override the `Exception.StackTrace` property to include the
`FailedBuildException.BuildLog` data.
2017-10-30 08:27:35 -04:00
Atsushi Eno b89302cae6 [msbuild] remove --dex argument and add DxExtraArguments from Dx task. (#990)
Otherwise debugging won't work with d8/r8.
2017-10-30 11:23:24 +00:00
Jonathan Peppers ab48f79d2b [build] fix for JdkInfo task in PrepareWindows.targets (#989)
On Windows after formatting my machine (don't ask),
`PrepareWindows.targets` was failing with:

	System.InvalidOperationException: Could not determine Android SDK location.

Looking into it, there appear to be two things wrong:

  * `PrepareWindows.targets` should import `Configuration.props`
  * The incoming properties are `$(AndroidSdkDirectory)` and
    `$(AndroidNdkDirectory)` not `$(AndroidSdkPath)` and
    `$(AndroidNdkPath)`.

I'm not quite sure how this *ever* worked, unless `AndroidSdkInfo`
was able to find *some* Android SDK on your machine, e.g. from a
prior Xamarin install.
2017-10-28 10:09:56 -04:00
Jonathan Pryor 4d94d34731 [build] Install mono in `make prepare-image-dependencies`
I have been told that the `prepare-image-dependencies.sh` script
should also install `mono`, to ensure that the required mono exists.
Add a `@PKG_URLS@` variable to `prepare-image-dependencies.sh.in` to
support downloading and installing our minimum specified mono version.

Additionally, I discovered that `xbuild` bugs are still around; I just
keep forgetting their specifics. Case in point:
[Jenkins build #712][xa712] uploaded
[`prepare-image-dependencies.sh`][sh], and it's not good:

[xa712]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android/
[sh]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android/712/Azure/processDownloadRequest/xamarin-android/prepare-image-dependencies.sh

	https://dl.google.com/android/repository/build-tools_r26.0.1-macosx.zip sdk/build-tools\$(XABuildToolsFolder)
	...
	https://dl.google.com/android/repository/android-15_r03.zip sdk/platforms\android-15

MSBuild properties contained within item metadata -- specifically
`%(AndroidSdkItem.DestDir)` -- aren't fully expanded, resulting in the
above value of `$(XABuildToolsFolder)`, which *should* be `26.0.1`.
Similarly, some install paths are using `\` (backslash) when they
should contain `/` (forward slash).

Jenkins has been updated to explicitly use `msbuild`:

	$ make prepare-image-dependencies MSBUILD=msbuild

However, `\r` characters now appear in
`prepare-image-dependencies.sh`. Fix this by having the
`PrepareImageDependencies` target emit the MSBuild-generated file into
an intermediate file, then fix it up using **tr**(1).

Finally, correct the `brew install` command so that any errors are
ignored, for if a brew package is already installed, re-installing it
will result in an error.
2017-10-27 16:50:34 -04:00
Jonathan Peppers a5fba94368 [xabuild.exe] Handle AndroidSdkDirectory like xabuild (#988)
Context: https://github.com/xamarin/xamarin-android/blob/master/tools/scripts/xabuild#L105

`xabuild`, the shell script, has some logic to set MSBuild properties
for `AndroidSdkDirectory` and `AndroidNdkDirectory`. The issue here is
on Windows, `xabuild.exe` is not invoked from this shell script. So we
need to replicate the logic in C# in `xabuild.exe`, but not break what
was fixed in #977.

Changes:

  * Check for `ANDROID_SDK_PATH` or `ANDROID_NDK_PATH` environment
    variables
  * Run `Paths.targets` via a new MSBuild process, the same way as
    `tools/scripts/xabuild`
  * Set the `AndroidSdkDirectory` and `AndroidNdkDirectory`
    properties in the config file if they are not blank
  * `AndroidSdkDirectory` and `AndroidNdkDirectory` can still be
    overridden by a user, because incoming values supersede what is
    found in the config file
2017-10-27 15:44:24 -04:00
Dean Ellis bd2640e17e [Xamarin.Android.Build.Tests] Fix BuildAppCheckDebugSymbols in monodroid (#987)
The `BuildAppCheckDebugSymbols()` test does not take into account
that when building in debug mode on monodroid the assemblies
are not embedded in the apk. As a result when trying to get
`Library1.pdb` from the apk we get a `null` back. This causes
a `NullReferenceException` when we try to get a length.

The fix is to look in the `assets` directory if we can't get
the file from the `.apk`. This will allow the test to work in
both xamarin-android and monodroid.
2017-10-27 15:05:08 -04:00
Jonathan Peppers b2cf680146 Bump to Java.Interop/master/00ad8d6 (#978)
Fixes: https://bugzilla.xamarin.com/show_bug.cgi?id=59472
Fixes: https://bugzilla.xamarin.com/show_bug.cgi?id=59770
2017-10-27 14:59:01 -04:00
Jonathan Pryor aa9b33a4d1 [build] Add `make prepare-image-dependencies` target. (#979)
We are investigating building Xamarin.Android atop
Visual Studio Team System (VSTS) in addition to our current Jenkins
setup, and one of the issues we're running into is "bootstrapping":
VSTS is configured to create a "fresh" VM for each build.

**Pro**: It should allow builds to be more reliable, as previous build
artifacts won't be present, and thus won't cause/hide errors.

**Con**: *Previous build artifacts are not present*. Previous build
artifacts such as *downloading and extracting* the Android NDK & SDK,
using `brew` to install dependencies, building MXE...

Ensuring that the dependencies are installed through `make prepare`
can be quite time consuming. What we want is a way to ensure that the
"build image" -- *what's already installed* when the VM boots --
contains all of our desired dependencies.

Furthermore, we *don't* want to have the responsible parties checkout
and build xamarin-android in order to determine what the dependencies
should be.

Attempt to square this circle by adding a new
`make prepare-image-dependencies` target, which processes
`@(AndroidSdkItem)`, `@(AndroidNdkItem)`, `@(AntItem)`, and
`@(RequiredProgram)` to create a `prepare-image-dependencies.sh`
script which will download and install the required dependencies.

The generated shell script does *not* take the state of the machine
running `make prepare-image-dependencies` into consideration. This
allows the target to be executed on one machine, and the output run
on another.

	$ make prepare-image-dependencies
	# creates `prepare-image-dependencies.sh`

*Note*: `make prepare-image-dependencies` does not currently deal
with MXE.  (Building MXE on the VSTS VM is *very* time consuming, so
it's something we need to take care of. It is not *yet* dealt with.)
2017-10-26 20:12:18 -04:00