## Miscellaneous fixes
* Fixed
`/Users/builder/jenkins/workspace/xamarin-macios/xamarin-macios/builds/mono-ios-sdk-destdir/ios-sources/external/linker/src/linker/Linker.Steps/OutputStep.cs(110,15): error CS0246: The type or namespace name ‘OutputException’ could not be found (are you missing a using directive or an assembly reference?) [/Users/builder/jenkins/workspace/xamarin-macios/xamarin-macios/tools/mmp/mmp.csproj]`
* Changed the name of the method that is used from linker. Because of this commit 6be26771b9
* Added `OutputException.cs` file on `mtouch.csproj`.
* Removing enter_gc_safe and exit_gc_safe because now it's already gc_safe in this part of code, after a mono change.
* Added known exceptions to LLVM exception list.
* Needs `ifdef` because of this https://github.com/mono/mono/pull/17260.
* Bump MIN_MONO_VERSION to 6.8.0.41 and point MIN_MONO_URL to the PR.
* Add ENABLE_IOS=1 and ENABLE_MAC=1.
* Added switch to disable packaged mono build
* [Tests] Ignore tests that fail on 32b.
Ignore the test on 32b, and filled issue: https://github.com/mono/mono/issues/17752
* [Tests] Ignore a couple of tests causing OOM.
Hopefully fixes https://github.com/xamarin/maccore/issues/1659 for good.
* Ignore `MM0135` test on Catalina+ because it needs Xcode 9.4.
* [monotouch-test] Add null checks for teardown when test didn't run because of a too early OS version.
* [CFNetwork]: Http 2.0 requires OS X 10.11 or later.
Check whether `_HTTPVersion2_0` is available and fallback to HTTP 1.1 otherwise.
## Bring HttpClient from CoreFX
* #7346
* This bumps Mono to use https://github.com/mono/mono/pull/17645 (which is the 2019-10 backport
of https://github.com/mono/mono/pull/17628).
* The big user-visible change is in regards to certificate validation, everything below are just
some minor adjustments to tests.
### SocketsHttpHandler
CoreFX uses a completely new `HttpClientHandler` implementation called `SocketsHttpHandler`,
which you can find at https://github.com/dotnet/corefx/tree/release/3.0/src/System.Net.Http/src/System/Net/Http/SocketsHttpHandler.
Since this is not based on the web stack anymore, it does not use any of the related APIs such
as `ServicePointManager` or `WebException`.
### Certificate Validation Changes
There is a new API called `HttpClientHandler.ServerCertificateCustomValidationCallback`.
- https://docs.microsoft.com/en-us/dotnet/api/system.net.http.httpclienthandler.servercertificatecustomvalidationcallback?view=netframework-4.8
- c1778515a3/src/System.Net.Http/src/System/Net/Http/HttpClientHandler.Unix.cs (L154)
- c1778515a3/src/System.Net.Http/src/System/Net/Http/HttpClientHandler.Windows.cs (L383)
The `ServicePointManager.ServerCertificateValidationCallback` is no longer invoked and on
certificate validation failure, `AuthenticationException` (from `System.Security.Authentication`)
is thrown instead of `WebException`.
At the moment, the `NSUrlSessionHandler` still uses it's own validation callback and also still
throws `WebException` on failure; we should probably look into making this consistent with the
other handlers.
### Minor adjustments related to internal Mono APIs
* `HttpContent.SerializeToStreamAsync()` is now `protected` (changed from `protected internal`).
- src/Foundation/NSUrlSessionHandler.cs: changed overload accordingly.
- src/System.Net.Http/CFContentStream.cs: likewise.
* `HttpHeaders.GetKnownHeaderKind()` is an internal Mono API.
There is a new internal API called `System.Net.Http.PlatformHelper.IsContentHeader(key)`
which exists in both the old as well as the new implementation.
The correct way of doing it with the CoreFX handler is
`HeaderDescriptor.TryGet (key, out var descriptor) && descriptor.HeaderType == HttpHeaderType.Content`
### Minor adjustments to tests.
* `HttpClientHandler.MaxRequestContentBufferSize` is now longer supported, you can set it to
any non-negative value, the getter will always return 0.
See c1778515a3/src/System.Net.Http/src/System/Net/Http/HttpClientHandler.Core.cs (L18).
- tests/linker/ios/link sdk/HttpClientHandlerTest.cs: removed assertion from test.
* `HttpMessageInvoker.handler` is a `protected private` field - in the CoreFX handler, it is
called `_handler` and `private`. This is accessed via reflection by some of the tests, which are
now using the new name.
- tests/mmptest/src/MMPTest.cs: here
- tests/mtouch/MTouch.cs: here
* tests/monotouch-test/System.Net.Http/MessageHandlers.cs:
Adjust `RejectSslCertificatesServicePointManager` to reflect the certificate validation
changes described above.
- FIXME: There was an `Assert.Ignore()` related to `NSUrlSessionHandler` and macOS 10.10;
I removed that to reenable the test because the description linked to an old issue in
the private repo that was referenced by several "Merged" PR's, so it looked to me that
this might have already been fixed - and I also didn't see why it would fail there.
* [build] fix Makefile around x86-64-slice
a7fd4552dc was slightly wrong, it should have broken all tests around notarization (which are presumably not done for PRs).
Also adding a missing dependency.
* typo
* $(suffix $@) does not work when specified as dependency
* update .gitignore
* Update builds/Makefile
Co-Authored-By: Rolf Bjarne Kvinge <rolf@xamarin.com>
The result of `lipo`ing `.a`s and `.dylib`s together is not well defined.
In this specific case, using `lipo` on `libmonosgen-2.0.a` to inject `x86-64-slice.dylib` resulted into `dsymutil` (run by `mtouch` when building the app) not finding the proper debug information anymore.
Also putting the dummy slice into a file and adding `-ggdb3` to avoid those warnings:
```console
$ dsymutil ./_ios-build/Library/Frameworks/Xamarin.iOS.framework/Versions/git/SDKs/MonoTouch.iphoneos.sdk/usr/lib/libmonosgen-2.0.dylib
warning: (x86_64) /var/folders/p3/5279mmgn1p575bz28j0ngfqw0000gn/T/--19c1b5.o unable to open object file: No such file or directory
warning: no debug symbols in executable (-arch x86_64)
```
See https://github.com/mono/mono/issues/17064.
The 2019-08 version of the fix adds a new assembly with NS2.1 APIs that we stubbed out in 2019-06: System.Data.DataSetExtensions.dll
* [builds] add arm64_32 cross compiler
* [mtouch] force --interpreter on arm64_32 debug mode build
* [mtouch] include debug check for arm64_32 and reflect error codes in documentation
* Drop the Xcode 9.4 dependency.
Also bump mono to get the removal of the mac32 binaries.
New commits in mono/mono:
* mono/mono@70d6903053 [2019-08] [merp] Use a separate program as the hang supervisor. (#16900)
* mono/mono@4bff2b6370 [offsets-tool] Install clang into the user-specific python directory.
* mono/mono@81894ec8ca Implement WriteCore and ReadCore in DeflateStream
* mono/mono@bfbf823ca1 [ci] Remove more XCODE32_DIR usages (#16964)
* mono/mono@ce01b20a4d Add net_4.8.xml to EXTRA_DIST and bump binary-reference-assemblies again
* mono/mono@7a587d7fa6 Add .NET 4.8 reference assemblies (#16912)
* mono/mono@35e454a8f6 [sdks] Remove the mac32 build. (#16936)
* mono/mono@75eb342f53 [2019-08] [System] Make FileSystemWatcher backend non-static (#16926)
* mono/mono@5881981f79 [2019-08] [mini] Add missing membars when initializing rgctx entries (#16909)
* mono/mono@6290b6cd6e Temporarily disable embedded ppdb data decompression (#16911)
* mono/mono@a0e7f9eaf2 [2019-08] [arm64_32] make "Debug Mode" work on Watch series 4 with --interpreter (#16886)
* mono/mono@6275840a7f Rename bundle identifier for the various Mono.frameworks we create for Xamarin.iOS. Fixesxamarin/xamarin-macios#7005. (#16901)
* mono/mono@25f6093283 [corlib] Fix building nunit-lite twice (#16895)
* mono/mono@7ec17ba1be [2019-08] [android sdk] Add aprofutil tool (#16884)
* mono/mono@f755f3b539 [metadata] Fix leaks when handling a few attributes (#16850)
* mono/mono@5f9a2db39b [2019-08] Fix infrequent hangs in test-runner. (#16854)
* mono/mono@f31f5ea1f1 [2019-08] [threads] do not convert NULL thread name (#16828)
* mono/mono@20308e6f87 [aot] Do not wrap tool_prefix path when calling strip (#16820)
* mono/mono@cecda47c48 [aprofutil] Add -p and -f options
* mono/mono@824cc12ac3 Bump to mono/corefx@e79cf5b
* mono/mono@b77dc06a7e [aprofutil] Install the tool correctly (#16112)
* mono/mono@1848d78d60 [aotprof-tool] Initial import of AOT profiler tool (#15384)
* mono/mono@da0086e304 [2019-08] Add RenamedEvent* to FSW sources from CoreFX (#16756)
* mono/mono@0297b21b03 [msbuild][roslyn] Bump msbuild and roslyn to pull in new versions (#16768)
* mono/mono@40631e3b9e [2019-08] [aot] move method_addresses to data.rel.so section to avoid text relocations (#16751)
* mono/mono@68b77674e2 Vtable [i] can be null so this should be check before use it. Fixes#16712
* mono/mono@4a0b4f41ed [mini] publish global patches after JitInfo has been added
* mono/mono@7a1f63fde6 [debugger][android] It was not initialising seq_points on MonoCompile on Android, so when was compiling dynamic methods, seq_points wasn't created and we got the assert when try to single step.
Diff: 29b1ac19c9..70d6903053
* [tests] Add a fat macOS dylib for testing purposes.
Add a binary version of a fat macOS dylib (because we can't create one when we
need it since we can't create 32-bit slice anymore).
It was created like this (in tests/test-libraries):
$ cat test.m
int theUltimateAnswer ()
{
return 42;
}
$ /Applications/Xcode94.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang test.m -olibtest.i386.dylib -shared -isysroot /Applications/Xcode94.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk -framework Foundation -framework CoreLocation -lz -arch i386
$ lipo -create libtest.i386.dylib .libs/macos/libtest.dylib -output libtest-fat.dylib
* [tests] Adjust XM tests to XM not having fat dylibs anymore.
* [tests] Adjust product tests to some libraries not being fat anymore.
* One more test fix.
* Add a dummy x86_64 slice to all our native libraries that don't have one. (#6848)
Apple's notarization tool has a bug where they incorrectly flag Mach-O
binaries without an x86_64 slice, so make sure all our libraries have one.
* Jenkinsfile notarization (#6869)
* Add in notarization script for xamarin.mac/xamarin.iOS
* Flatten the list to get rid of the braces
* Add in keychain password
* Add login.keychain back in to access codesigning certificates
* Always sign pkgs, upload notarized copies
* Enable ios notarization and make notarized pkgs public
* Make notarization non-fatal
* Publish GH statuses for notarized PKGs
* Don't forget to declare URI variables for notarized pkgs
* report proper package links
* [jenkins] Improve package reporting.
* Use dummy function name which our tests won't complain about.
Rename the bundle identifier for the frameworks we ship, because apparently
Apple has added an App Store check to reject apps with multiple frameworks
with the same bundle identifier.
For some reason that includes rejecting frameworks that are both in iOS and
watchOS apps. This feels like a bug on Apple's end, but we can still easily
work around it, so let's do just that.
Strictly speaking it's not necessary to change the bundle identifier for tvOS
frameworks, but it's more consistent that way.
Fixes https://github.com/xamarin/xamarin-macios/issues/6920.
* Use the commonly used casing for `MSBuildSDKsPath` property
Handle "incorrectly" cased msbuild property names
msbuild property names are case insensitive. While generating the custom
app.config, in `SetToolsetProperty(..)` we try to update the property if
it already exists. But the name lookup was case sensitive, thus causing
the lookup to fail, resulting in two entries for the same property name
differing only in case. Eg. `MSBuildSDKsPath` vs `MSBuildSdksPath`.
* [mtouch] Whitelist new Brotli native symbols in Xamarin.Tests.Misc.PublicSymbols test
* [mtouch] Better assert in NoLLVMFailuresInWatchOS() test
We'd list the "LLVM failed" messages before even though the AOT might've crashed and the list is meaningless. Assert the exit code before that.
* [mtouch] Use new LLVM even for 32bit targets
See https://github.com/mono/mono/issues/14841 and https://github.com/mono/mono/issues/9621
* [mtouch] Work around slow LLVM in "don't link" test
See https://github.com/mono/mono/issues/14843
* Remove useless conditional
* Remove LLVM36 from Makefile
* [watch4] set right min version for arm64_32 based watch devices (#6307)
Fixes the confusion around `libmono-native*` (see for example ce5ba1e41d (commitcomment-33834491) ) when building with `MONO_BUILD_FROM_SOURCE=1`.
* reflect watchos64_32_version_min change from mono sdk
* Move mono hash info to mk/mono.mk so that existing scripts work.
* Add Makefile dependency on mono.mk where necessary
With 3e7bc29ade the Mono hash was moved from Make.config to mono.mk.
We need to add a Makefile dependency on this file wherever Make.config was used to track a Mono dependency.
* [tests] Copy mk/mono.mk to the XM test package.
* [tests] Update minOS version test after consolidating min watchOS versions everywhere.
Fixes this mtouch and mmptest failure:
1) Failed : Xamarin.Tests.ProductTests.MinOSVersion(watchOS,MinwatchOS,WatchOSSimulator,False)
Failures
Expected: <empty>
But was: < "Unexpected minOS version (expected 2.0.0, alternatively 2.0.0, found 5.1.0) in /Users/builder/jenkins/workspace/xamarin-macios/xamarin-macios/_ios-build/Library/Frameworks/Xamarin.iOS.framework/Versions/git/SDKs/Xamarin.WatchSimulator.sdk/usr/lib/libxamarin-debug.a (mono-runtime-debug.arm64_32.o).", "Unexpected minOS version (expected 2.0.0, alternatively 2.0.0, found 5.1.0) in /Users/builder/jenkins/workspace/xamarin-macios/xamarin-macios/_ios-build/Library/Frameworks/Xamarin.iOS.framework/Versions/git/SDKs/Xamarin.WatchSimulator.sdk/usr/lib/libxamarin-debug.a (bindings-debug.arm64_32.o).", "Unexpected minOS version (expected 2.0.0, alternatively 2.0.0, found 5.1.0) in /Users/builder/jenkins/workspace/xamarin-macios/xamarin-macios/_ios-build/Library/Frameworks/Xamarin.iOS.framework/Versions/git/SDKs/Xamarin.WatchSimulator.sdk/usr/lib/libxamarin-debug.a (bindings-generated-debug.arm64_32.o).", "Unexpected minOS version (expected 2.0.0, alternatively 2.0.0, found 5.1.0) in /Users/builder/jenkins/workspace/xamarin-macios/xamarin-macios/_ios-build/Library/Frameworks/Xamarin.iOS.framework/Versions/git/SDKs/Xamarin.WatchSimulator.sdk/usr/lib/libxamarin-debug.a (shared-debug.arm64_32.o).", "Unexpected minOS version (expected 2.0.0, alternatively 2.0.0, found 5.1.0) in /Users/builder/jenkins/workspace/xamarin-macios/xamarin-macios/_ios-build/Library/Frameworks/Xamarin.iOS.framework/Versions/git/SDKs/Xamarin.WatchSimulator.sdk/usr/lib/libxamarin-debug.a (runtime-debug.arm64_32.o).", "Unexpected minOS version (expected 2.0.0, alternatively 2.0.0, found 5.1.0) in /Users/builder/jenkins/workspace/xamarin-macios/xamarin-macios/_ios-build/Library/Frameworks/Xamarin.iOS.framework/Versions/git/SDKs/Xamarin.WatchSimulator.sdk/usr/lib/libxamarin-debug.a (trampolines-debug.arm64_32.o).", "Unexpected minOS version (expected 2.0.0, alternatively 2.0.0, found 5.1.0) in /Users/builder/jenkins/workspace/xamarin-macios/xamarin-macios/_ios-build/Library/Frameworks/Xamarin.iOS.framework/Versions/git/SDKs/Xamarin.WatchSimulator.sdk/usr/lib/libxamarin-debug.a (trampolines-invoke-debug.arm64_32.o).", "Unexpected minOS version (expected 2.0.0, alternatively 2.0.0, found 5.1.0) in /Users/builder/jenkins/workspace/xamarin-macios/xamarin-macios/_ios-build/Library/Frameworks/Xamarin.iOS.framework/Versions/git/SDKs/Xamarin.WatchSimulator.sdk/usr/lib/libxamarin-debug.a (xamarin-support-debug.arm64_32.o).", "Unexpected minOS version (expected 2.0.0, alternatively 2.0.0, found 5.1.0) in /Users/builder/jenkins/workspace/xamarin-macios/xamarin-macios/_ios-build/Library/Frameworks/Xamarin.iOS.framework/Versions/git/SDKs/Xamarin.WatchSimulator.sdk/usr/lib/libxamarin-debug.a (nsstring-localization-debug.arm64_32.o).", "Unexpected minOS version (expected 2.0.0, alternatively 2.0.0, found 5.1.0) in /Users/builder/jenkins/workspace/xamarin-macios/xamarin-macios/_ios-build/Library/Frameworks/Xamarin.iOS.framework/Versions/git/SDKs/Xamarin.WatchSimulator.sdk/usr/lib/libxamarin-debug.a (trampolines-varargs-debug.arm64_32.o)."... >
* [mmp] Fix make clean target
It needs an -r to remove directories:
```
rm: bin: is a directory
rm: obj: is a directory
```
* Add new xamarin_timezone_get_local_name() to a few more places
* [builds] Build the BCL test assemblies. Fixes#6261.
Build the BCL test assemblies and copy them where xharness expects them to be.
Fixes https://github.com/xamarin/xamarin-macios/issues/6261.
* [builds] Copy nunitlite.dll to where it's expected too.
Building in non-parallel mode revelead and issue where a required directory
would not exist when needed. Fixed by creating the required directory when
needed.
The .deps.mk target didn't get generated because its dependency was wrong.
Also do not exclude tests from the dependencies, they should be rebuild when changed too.
Fixes#6224
We were hitting this:
```
fatal error: /Applications/Xcode11-beta1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/lipo: can't open input file: /Users/vidondai/Documents/xi-xcode11/xamarin-macios/builds/install/targetwatch64_32/tmp-lib/libmono-native-unified.dylib (No such file or directory)
make[2]: *** [/Users/vidondai/Documents/xi-xcode11/xamarin-macios/_ios-build/Library/Frameworks/Xamarin.iOS.framework/Versions/git/SDKs/Xamarin.WatchOS.sdk/usr/lib/libmono-native-unified.dylib] Error 1
```
* [builds] Add support for using cached downloads of the mono archives from ~/Library/Caches.
* [apidiff] Add support for using cached downloads of apidiff bundle from ~/Library/Caches.
* [builds] Move the facade check targets down in the Makefile.
Move the facade check targets below the declaration of their prerequisite
variables (*_BCL_TARGETS), since otherwise the prerequisite variable will be
empty when the facade check targets are read by make, they end up with no
prerequisites at all, and the targets fail.
* [builds] Make the tools build use mono's packaged logic instead of our own.
Make the 'tools64' build use mono's packaged build logic instead of our own.
This is the first step to consuming the BCL from the mono archive.
Also completely refactor the 'tools64' build by removing everything we don't
need and renaming it to 'bcl' (since that's more representative of what it
does).
* [apidiff] Unzip downloaded files into a temporary subdirectory.
Before we unzip, we remove the target directory. This is a bad idea if the
target directory is also used for other things: in particular it breaks
parallel make if some other target tries to write to the temporary directory.
Instead unzip downloaded files into a subdirectory exclusively used by those
unzipped files, which means we can remove at will before unzipping.
* [apidiff] Use mono-api-[info|html].exe from the mono archive.
* [builds] Fix race condition when building device runtimes.
We need to complete the corresponding cross builds first, because the cross
builds will also try to configure device builds, so we might end up with two
make processes trying to configure the same target simulatenously.
* [jenkins] Add support for building mono from source in pull requests by using labels.
There's no need to use 'true' to create a recipe that does nothing: a single
semi-colon is enough (and for the install-tools* and build-tools64 targets I
don't see a need for an empty recipe in any case). This saves ~2.2k 'true'
subprocesses during a make with everything already built (and a second of
build time; a rebuild in builds/ when everything is already built now takes
~2s on my machine instead of ~3s).
Alexander Köplinger
Jo Shields
Bernhard Urban-Forster
Rolf Bjarne Kvinge
Vincent Dondain