Some of the test assemblies are too large and will be splitted by mono so
that they can be compiled for iOS 32b. In that case, we are using the
following pattern
`
test_assembly_name.dll
`
becomes
`
test_assembly_name.part1.dll
test_assembly_name.part2.dll
`
Perse the only change we need to add to make our life easier is to be
able to mantain a single .ignore file since mantaining more .ignore
files is hard and more error prone. This change simply checks if we are
working with a splitted dll and ensures that the correct .ignore files
are added to the BCL test application.
Co-Authored-By: Pramit Mallick <prmallic@microsoft.com>
* [msbuild] Don't use gdk-sharp, use native functions instead.
There's no netstandard version of gdk-sharp.
* Don't require the source checkout to be named 'xamarin-macios'.
After this commit we will have more application but they will be the
appropiate size so that they can be built with the linker for iOS 32b.
It is important to note that the following apps WILL CONTINUE to fail
since the dlls need to be splitted in mono:
* mscorlib tests
* Mono BCL tests group 5 - Which is monotouch_System.Core_xunit-test.dll
and is too large.
We now need to merge System.Text.Json into the final assembly, which means all
the System.Text.Json dependencies as well (there are quite a few), so adjust
the logic to figure out which assemblies to be merged to include every
library, except:
* MSBuild assemblies
* Resource assemblies
* System assemblies.
The Bug60536 is about showing a good error message if the json in an asset
pack is invalid. It turns out that System.Text.Json is a bit more permissive
than System.Json, so the json that was previously failing to parse is now
parsing fine [1]. So introduce something that is certainly invalid json
everywhere, and update the test itself as well to cope with different error
message and error location.
Also remove workaround for xbuild in this test, since xbuild isn't used anymore.
[1]: Technically the json is still invalid, because according to the json spec
it's not valid to end a list of array elements with a trailing comma. The
problem is that System.Json would in some cases allow trailing commas for
lists of array elements, and sometimes not, which means that we have to ask
System.Text.Json to allow trailing commas as well. And when System.Text.Json
is asked to allow trailing commas, it also successfully parses a case of
trailing comma where System.Json didn't, thus the need for coming up with
something even more invalid.
TouchUnit uses Console.Out as the writer when the TCP connection cannot
be performed. The BCL tests were not doing so and were crashing.
With this change, when ran locally, if the TCP connection is not done,
we will use Console.Out as TouchUnit and the file will be parsed by
xharness correctly.
Updating the msbuild tasks to use netstandard2.0 requires us to bump NUnit to 3+.
This requires:
* A few code changes due to breaking API changes in NUnit.
* Changes in xharness and a makefile to cope with the new location for the
NUnit console runner (I added a helper script to make things slightly
easier).
* [monotouch-test] Improve reachability test.
Give up.
Different versions of OSes report different flags. Trying to
figure out which OS versions have which flags set turned out to
be a never-ending game of whack-a-mole, so just don't assert
that any specific flags are set.
Some of the fixes done for the warnings have breaking changes. Move back
to int and ensure that we do not have any compilation errors (we are
using -Werror).
Fixes: https://github.com/xamarin/xamarin-macios/issues/7509
It's possible for a nuget to have dependencies that will bring satellite
assemblies into a project.
When doing so the satellite assemblies are copied to the right output
(e.g. `Debug`) directory, so they work fine at runtime. However it's
not 100% fine at build time, e.g. msbuild won't detected them as
satellite assemblies.
Something similar happens with `mtouch` (and `mmp`) since the satellite
assemblies won't be found inside a (culture-named) subdirectory from the
original assembly location. In our case this means the assemblies won't
be copied into the .app bundle (since we don't run from `Debug`) and
localization won't work properly.
The solution is to check for both the (culture-named) subdirectories
from the assembly location (like before) and, if nothing is found, also
try to locate them in the _build_ (like `Debug`) directory - so if
something else (nuget or custom build scripts) tried to outsmart the
build logic then we'll still bring those satellite assemblies in the
app bundle we produce.
https://github.com/xamarin/xamarin-macios/issues/7113
The nice, repeatable test case from #7514 pointed out two issues
1. `cache.cs` ignored some changes
It looks like something changed (at some point) and the first _ignored_
line was the `@x.rsp` our response file - which should not be ignored.
This solved the build issue where updating the nuget should have
triggered a rebuild because
> /Users/poupou/.nuget/packages/skiasharp/1.68.0/lib/Xamarin.iOS/SkiaSharp.dll
and
> /Users/poupou/.nuget/packages/skiasharp/1.68.1/lib/Xamarin.iOS/SkiaSharp.dll
are different assemblies (but the same response file).
2. `copyfile` could fail silently
Copying the framework could fail (error 260) and the failure was never
reported so the build succeeded - but without updating (completely) the
framework.
The exact reason it fails is unknown :( but we can recover from it by
deleting the target and copying (everything) back to the expected
(target) location.
Build logs will now indicate when this fails and will try to recover
before reporting a build error. Best case it works :) worse case we'll
be aware something is wrong (which is better than ignoring)
ref: https://github.com/xamarin/xamarin-macios/issues/7514
* Do not call `Marshal.GetLastWin32Error` instead the callback
as the `SetLastError` logic has yet to be executed so we get a bogus
`260` value...
Instead we call it after the `copyfile` call returns but, at this stage,
the callback (i.e. **us**) signaled an error so what we get back (`17`)
is not very helpful - as we aborted (Quit) the logic when copying a file
that existed.
```c
#define EEXIST 17 /* File exists */
```
from Console logs
```
default 14:23:54.771292-0500 mono64 Cannot make directory /Users/poupou/Projects/gh7514/gh7514/bin/iPhoneSimulator/Debug/gh7514.app/Frameworks/libSkiaSharp.framework: File exists
default 14:23:54.771620-0500 mono64 Cannot make directory /Users/poupou/Projects/gh7514/gh7514/bin/iPhoneSimulator/Debug/gh7514.app/Frameworks/libSkiaSharp.framework/_CodeSignature: File exists
default 14:23:54.771850-0500 mono64 open on /Users/poupou/Projects/gh7514/gh7514/bin/iPhoneSimulator/Debug/gh7514.app/Frameworks/libSkiaSharp.framework/_CodeSignature/CodeResources: File exists
```
`copyfile.c` source code (might not be the latest) can be seen from
https://opensource.apple.com/source/copyfile/copyfile-42/copyfile.c
```c
if (mkdir(s->dst, mode) == -1) {
if (errno != EEXIST || (s->flags & COPYFILE_EXCL)) {
copyfile_warn("Cannot make directory %s", s->dst);
```
so `mkdir` fails - but not because of `EEXIST` and from `copyfile.h`
we see that `EXCL` exists but it's not using (or even defined) in our
bindings.
```c
#define COPYFILE_EXCL (1<<17) /* fail if destination exists */
```
So sadly the `Err` condition (inside the callback) does not give us more
detail about the error itself.
1. Instead of fetching the managed response's cookies twice, and then
comparing them to themselves, fetch the managed response's cookies once and
then the native response's cookies once, and compare that. This way the
test does what it's actually supposed to do!
2. Don't pass if we fail to get the managed response's cookies.
3. Improve assertions to verify the actual cookie content as well.
4. Improve diagnostics if an exception was thrown by checking for the
exception first instead of failing due to something else being different
(since the underlying cause of the difference is the exception, that's what
will have the most information about the failure).
5. Use httpbin.org instead of microsoft.com, since microsoft.com does UA
sniffing and returns different results between the http handlers :/
## Miscellaneous fixes
* Fixed
`/Users/builder/jenkins/workspace/xamarin-macios/xamarin-macios/builds/mono-ios-sdk-destdir/ios-sources/external/linker/src/linker/Linker.Steps/OutputStep.cs(110,15): error CS0246: The type or namespace name ‘OutputException’ could not be found (are you missing a using directive or an assembly reference?) [/Users/builder/jenkins/workspace/xamarin-macios/xamarin-macios/tools/mmp/mmp.csproj]`
* Changed the name of the method that is used from linker. Because of this commit 6be26771b9
* Added `OutputException.cs` file on `mtouch.csproj`.
* Removing enter_gc_safe and exit_gc_safe because now it's already gc_safe in this part of code, after a mono change.
* Added known exceptions to LLVM exception list.
* Needs `ifdef` because of this https://github.com/mono/mono/pull/17260.
* Bump MIN_MONO_VERSION to 6.8.0.41 and point MIN_MONO_URL to the PR.
* Add ENABLE_IOS=1 and ENABLE_MAC=1.
* Added switch to disable packaged mono build
* [Tests] Ignore tests that fail on 32b.
Ignore the test on 32b, and filled issue: https://github.com/mono/mono/issues/17752
* [Tests] Ignore a couple of tests causing OOM.
Hopefully fixes https://github.com/xamarin/maccore/issues/1659 for good.
* Ignore `MM0135` test on Catalina+ because it needs Xcode 9.4.
* [monotouch-test] Add null checks for teardown when test didn't run because of a too early OS version.
* [CFNetwork]: Http 2.0 requires OS X 10.11 or later.
Check whether `_HTTPVersion2_0` is available and fallback to HTTP 1.1 otherwise.
## Bring HttpClient from CoreFX
* #7346
* This bumps Mono to use https://github.com/mono/mono/pull/17645 (which is the 2019-10 backport
of https://github.com/mono/mono/pull/17628).
* The big user-visible change is in regards to certificate validation, everything below are just
some minor adjustments to tests.
### SocketsHttpHandler
CoreFX uses a completely new `HttpClientHandler` implementation called `SocketsHttpHandler`,
which you can find at https://github.com/dotnet/corefx/tree/release/3.0/src/System.Net.Http/src/System/Net/Http/SocketsHttpHandler.
Since this is not based on the web stack anymore, it does not use any of the related APIs such
as `ServicePointManager` or `WebException`.
### Certificate Validation Changes
There is a new API called `HttpClientHandler.ServerCertificateCustomValidationCallback`.
- https://docs.microsoft.com/en-us/dotnet/api/system.net.http.httpclienthandler.servercertificatecustomvalidationcallback?view=netframework-4.8
- c1778515a3/src/System.Net.Http/src/System/Net/Http/HttpClientHandler.Unix.cs (L154)
- c1778515a3/src/System.Net.Http/src/System/Net/Http/HttpClientHandler.Windows.cs (L383)
The `ServicePointManager.ServerCertificateValidationCallback` is no longer invoked and on
certificate validation failure, `AuthenticationException` (from `System.Security.Authentication`)
is thrown instead of `WebException`.
At the moment, the `NSUrlSessionHandler` still uses it's own validation callback and also still
throws `WebException` on failure; we should probably look into making this consistent with the
other handlers.
### Minor adjustments related to internal Mono APIs
* `HttpContent.SerializeToStreamAsync()` is now `protected` (changed from `protected internal`).
- src/Foundation/NSUrlSessionHandler.cs: changed overload accordingly.
- src/System.Net.Http/CFContentStream.cs: likewise.
* `HttpHeaders.GetKnownHeaderKind()` is an internal Mono API.
There is a new internal API called `System.Net.Http.PlatformHelper.IsContentHeader(key)`
which exists in both the old as well as the new implementation.
The correct way of doing it with the CoreFX handler is
`HeaderDescriptor.TryGet (key, out var descriptor) && descriptor.HeaderType == HttpHeaderType.Content`
### Minor adjustments to tests.
* `HttpClientHandler.MaxRequestContentBufferSize` is now longer supported, you can set it to
any non-negative value, the getter will always return 0.
See c1778515a3/src/System.Net.Http/src/System/Net/Http/HttpClientHandler.Core.cs (L18).
- tests/linker/ios/link sdk/HttpClientHandlerTest.cs: removed assertion from test.
* `HttpMessageInvoker.handler` is a `protected private` field - in the CoreFX handler, it is
called `_handler` and `private`. This is accessed via reflection by some of the tests, which are
now using the new name.
- tests/mmptest/src/MMPTest.cs: here
- tests/mtouch/MTouch.cs: here
* tests/monotouch-test/System.Net.Http/MessageHandlers.cs:
Adjust `RejectSslCertificatesServicePointManager` to reflect the certificate validation
changes described above.
- FIXME: There was an `Assert.Ignore()` related to `NSUrlSessionHandler` and macOS 10.10;
I removed that to reenable the test because the description linked to an old issue in
the private repo that was referenced by several "Merged" PR's, so it looked to me that
this might have already been fixed - and I also didn't see why it would fail there.
Manuel de la Pena
Rolf Bjarne Kvinge
Alex Soto
Marius Ungureanu
monojenkins
Sebastien Pouliot
Alexander Köplinger
Vincent Dondain
Oleg Demchenko
Jo Shields