Граф коммитов

1420 Коммитов

Автор SHA1 Сообщение Дата
Manuel de la Pena 1c3bd7f114
[XHarness] If there are issues generating the BCL tests for mac dont crash. (#7646) 2019-12-23 19:04:09 -05:00
Manuel de la Pena 4c6181e904
[Test] Bump mono and use the new splited tests. (#7626)
Bump mono to get the new splited test dlls and add them to be ran in
xharness. Special logic is used for mscorlib so we make sure that all
the 'parts' of the test dll do have the same configurations.


Co-Authored-By: Waleed Chaudhry <54864665+wachaudh@users.noreply.github.com>
2019-12-23 18:39:41 -05:00
Vincent Dondain f1b92747c7
[DevOps] Convert our device tests into a pool agnostic yaml pipeline (#7604)
- Needed for the DDFun migration of our tests
- Don't use SSH because it's restricted on the new bots
- Use brand new DevOps feature to clone multiple repos via HTTPS
- Update names of all tasks (clearer)
- Simplify and re-use code
- Update all paths because our cloning strategy is different compared to old pipeline
- Update xharness to print "experimental" titles in GitHub comments
- Update xharness to not add the Html link for tests from the DDFun pipeline
- Fix all timeouts to be relevant to the tests length
- Split in multiple jobs (for GitHub status)
- Move longer scripts to independant files
- Make inline bash prettier
- Use a variable for the pool name
- Fix single space alignment
- Remove unused variable in `vsts-device-tests-set-status.sh` that broke it
- Bump maccore to use the right certificates and profiles
- Bump maccore to target the right pipeline
- Bump maccore to not use ssh in scripts
- Bump maccore and use persistCredentials

Notes:
- Right now the xharness test output can only be seen when downloading the zip file locally (better output soon)
2019-12-23 14:34:41 -05:00
Chris Hamons e316fc38b5
[mmp][mtouch] Drill into wrapped exceptions to produce better linker errors
- https://devdiv.visualstudio.com/DevDiv/_workitems/edit/947932
- It turns out the linker in some cases can wrap expected exceptions in a outer exception. By drilling in we can produce better errors, again.
- Refactor pipeline exception handling to be shared between mmp/mtouch
2019-12-23 09:57:48 -06:00
Sebastien Pouliot 2694579530
[mtouch] Fix case where `copyfile` could fail silently (#7633)
Copying the framework could fail (error 260) and the failure was never
reported so the build succeeded - but without updating (completely) the
framework.

The exact reason it fails is unknown :( but we can recover from it by
deleting the target and copying (everything) back to the expected
(target) location.

Build logs will now indicate when this fails and will try to recover
before reporting a build error. Best case it works :) worse case we'll
be aware something is wrong (which is better than ignoring)

This is half the fix from https://github.com/xamarin/xamarin-macios/pull/7544
which was reverted due to the second half causing a regression (under
investigation).

Reference: https://github.com/xamarin/xamarin-macios/issues/7514 (partial fix)
2019-12-23 09:18:04 -05:00
Rolf Bjarne Kvinge 0ec82e7cca
[sample-tester] Collect app size and build duration statistics. (#7602)
* [sample-tester] Collect app size and build duration statistics.

* Always publish performance data.

* [mmp] Add support for MMP_ENV_OPTIONS to mirror mtouch's MTOUCH_ENV_OPTIONS.

* [sample-tester] Make mmp/mtouch show timing information, and get the diagnostic msbuild log.

* Collect task and target info.

* Add more perf data and a baseline test.

* Redo the base test a bit.

* More tweaks.

* More tweaks.

* Huh?

* Debug stuff.

* diagnostics.

* Better diagnostics.

* cleanup

* tmp

* Use a separate stage to push data.

* Next attempt.

* Unique artifact names.

* Make the right name unique.

* More progress

* Improvements

* Don't need new maccore.

* cleanup

* Remove debug spew.

* Realign stuff.

* duh

* More path fixes.

* zippity zip.

* Show publishing errors.

* dependencies

* Not fake results anymore.

* Tweaks

* Merge xml files

* [sampletester] Bump nuget timeout to 5 minutes.

The TodoREST.iOS test seems to use a lot of packages, and it can apparently
take a while to restore them, so give a few more minutes.

* Improvements.

* Fix glob.

* [sampletester] Fix fetching of sample repository and hash.

* [sampletester] Centralize verbosity for mtouch and mmp.
2019-12-20 16:27:36 +01:00
Manuel de la Pena 002b4dffd2
[xharness] Support the split of test assemblies from the BCL. (#7625)
Some of the test assemblies are too large and will be splitted by mono so
that they can be compiled for iOS 32b. In that case, we are using the
following pattern

`
test_assembly_name.dll
`
becomes
`
test_assembly_name.part1.dll
test_assembly_name.part2.dll
`
Perse the only change we need to add to make our life easier is to be
able to mantain a single .ignore file since mantaining more .ignore
files is hard and more error prone. This change simply checks if we are
working with a splitted dll and ensures that the correct .ignore files
are added to the BCL test application.

Co-Authored-By: Pramit Mallick <prmallic@microsoft.com>
2019-12-19 09:29:43 -05:00
Alex Soto 9ac17bdbcf
Merge remote-tracking branch 'xamarin/xcode11.3' into master-xcode11.3 2019-12-17 19:25:11 -05:00
Manuel de la Pena 867dcfaa77
[Tests] Split groups so that they work on iOS 32b. (#7612)
After this commit we will have more application but they will be the
appropiate size so that they can be built with the linker for iOS 32b.
It is important to note that the following apps WILL CONTINUE to fail
since the dlls need to be splitted in mono:

* mscorlib tests
* Mono BCL tests group 5 - Which is monotouch_System.Core_xunit-test.dll
and is too large.
2019-12-17 17:09:28 -05:00
Rolf Bjarne Kvinge c422869c4b
[msbuild] Update tests to NUnit 3.12. (#7597)
Updating the msbuild tasks to use netstandard2.0 requires us to bump NUnit to 3+.

This requires:

* A few code changes due to breaking API changes in NUnit.
* Changes in xharness and a makefile to cope with the new location for the
  NUnit console runner (I added a helper script to make things slightly
  easier).
2019-12-16 23:55:59 +01:00
Sebastien Pouliot 144c193332
Revert "[mtouch] Fix caching issues. Fix #7514 (#7544)" (#7589)
This reverts commit b3bc04c919.
2019-12-12 18:01:29 -05:00
Sebastien Pouliot ff2d5bb94f
[mtouch][mmp] Include satellite assemblies from nuget dependencies. Fix #7113 (#7548)
It's possible for a nuget to have dependencies that will bring satellite
assemblies into a project.

When doing so the satellite assemblies are copied to the right output
(e.g. `Debug`) directory, so they work fine at runtime. However it's
not 100% fine at build time, e.g. msbuild won't detected them as
satellite assemblies.

Something similar happens with `mtouch` (and `mmp`) since the satellite
assemblies won't be found inside a (culture-named) subdirectory from the
original assembly location. In our case this means the assemblies won't
be copied into the .app bundle (since we don't run from `Debug`) and
localization won't work properly.

The solution is to check for both the (culture-named) subdirectories
from the assembly location (like before) and, if nothing is found, also
try to locate them in the _build_ (like `Debug`) directory - so if
something else (nuget or custom build scripts) tried to outsmart the
build logic then we'll still bring those satellite assemblies in the
app bundle we produce.

https://github.com/xamarin/xamarin-macios/issues/7113
2019-12-10 09:23:31 -05:00
Rolf Bjarne Kvinge 9fe358650b
Use relative paths in scripts. (#7545)
It makes relocating XI/XM easier.
2019-12-10 12:47:30 +01:00
Sebastien Pouliot b3bc04c919
[mtouch] Fix caching issues. Fix #7514 (#7544)
The nice, repeatable test case from #7514 pointed out two issues

1. `cache.cs` ignored some changes

It looks like something changed (at some point) and the first _ignored_
line was the `@x.rsp` our response file - which should not be ignored.

This solved the build issue where updating the nuget should have
triggered a rebuild because
> /Users/poupou/.nuget/packages/skiasharp/1.68.0/lib/Xamarin.iOS/SkiaSharp.dll
and
> /Users/poupou/.nuget/packages/skiasharp/1.68.1/lib/Xamarin.iOS/SkiaSharp.dll
are different assemblies (but the same response file).

2. `copyfile` could fail silently

Copying the framework could fail (error 260) and the failure was never
reported so the build succeeded - but without updating (completely) the
framework.

The exact reason it fails is unknown :( but we can recover from it by
deleting the target and copying (everything) back to the expected
(target) location.

Build logs will now indicate when this fails and will try to recover
before reporting a build error. Best case it works :) worse case we'll
be aware something is wrong (which is better than ignoring)

ref: https://github.com/xamarin/xamarin-macios/issues/7514

* Do not call `Marshal.GetLastWin32Error` instead the callback
as the `SetLastError` logic has yet to be executed so we get a bogus
`260` value...

Instead we call it after the `copyfile` call returns but, at this stage,
the callback (i.e. **us**) signaled an error so what we get back (`17`)
is not very helpful - as we aborted (Quit) the logic when copying a file
that existed.

```c
#define EEXIST          17      /* File exists */
```

from Console logs

```
default	14:23:54.771292-0500	mono64	Cannot make directory /Users/poupou/Projects/gh7514/gh7514/bin/iPhoneSimulator/Debug/gh7514.app/Frameworks/libSkiaSharp.framework: File exists
default	14:23:54.771620-0500	mono64	Cannot make directory /Users/poupou/Projects/gh7514/gh7514/bin/iPhoneSimulator/Debug/gh7514.app/Frameworks/libSkiaSharp.framework/_CodeSignature: File exists
default	14:23:54.771850-0500	mono64	open on /Users/poupou/Projects/gh7514/gh7514/bin/iPhoneSimulator/Debug/gh7514.app/Frameworks/libSkiaSharp.framework/_CodeSignature/CodeResources: File exists
```

`copyfile.c` source code (might not be the latest) can be seen from
https://opensource.apple.com/source/copyfile/copyfile-42/copyfile.c

```c
if (mkdir(s->dst, mode) == -1) {
	if (errno != EEXIST || (s->flags & COPYFILE_EXCL)) {
		copyfile_warn("Cannot make directory %s", s->dst);
```

so `mkdir` fails - but not because of `EEXIST` and from `copyfile.h`
we see that `EXCL` exists but it's not using (or even defined) in our
bindings.

```c
#define COPYFILE_EXCL		(1<<17) /* fail if destination exists */
```

So sadly the `Err` condition (inside the callback) does not give us more
detail about the error itself.
2019-12-05 09:15:11 -05:00
Jo Shields 5d4ada34c7 Bump to mono:2019-10 (#7192)
## Miscellaneous fixes

* Fixed
`/Users/builder/jenkins/workspace/xamarin-macios/xamarin-macios/builds/mono-ios-sdk-destdir/ios-sources/external/linker/src/linker/Linker.Steps/OutputStep.cs(110,15): error CS0246: The type or namespace name ‘OutputException’ could not be found (are you missing a using directive or an assembly reference?) [/Users/builder/jenkins/workspace/xamarin-macios/xamarin-macios/tools/mmp/mmp.csproj]`
* Changed the name of the method that is used from linker. Because of this commit 6be26771b9
* Added `OutputException.cs` file on `mtouch.csproj`.
* Removing enter_gc_safe and exit_gc_safe because now it's already gc_safe in this part of code, after a mono change.
* Added known exceptions to LLVM exception list.
* Needs `ifdef` because of this https://github.com/mono/mono/pull/17260.
* Bump MIN_MONO_VERSION to 6.8.0.41 and point MIN_MONO_URL to the PR.
* Add ENABLE_IOS=1 and ENABLE_MAC=1.
* Added switch to disable packaged mono build
* [Tests] Ignore tests that fail on 32b.
    Ignore the test on 32b, and filled issue: https://github.com/mono/mono/issues/17752
* [Tests] Ignore a couple of tests causing OOM.
    Hopefully fixes https://github.com/xamarin/maccore/issues/1659 for good.
* Ignore `MM0135` test on Catalina+ because it needs Xcode 9.4.
* [monotouch-test] Add null checks for teardown when test didn't run because of a too early OS version.
* [CFNetwork]: Http 2.0 requires OS X 10.11 or later.
    Check whether `_HTTPVersion2_0` is available and fallback to HTTP 1.1 otherwise.

## Bring HttpClient from CoreFX

* #7346 
* This bumps Mono to use https://github.com/mono/mono/pull/17645 (which is the 2019-10 backport
of https://github.com/mono/mono/pull/17628).
* The big user-visible change is in regards to certificate validation, everything below are just
some minor adjustments to tests.

### SocketsHttpHandler

CoreFX uses a completely new `HttpClientHandler` implementation called `SocketsHttpHandler`,
which you can find at https://github.com/dotnet/corefx/tree/release/3.0/src/System.Net.Http/src/System/Net/Http/SocketsHttpHandler.

Since this is not based on the web stack anymore, it does not use any of the related APIs such
as `ServicePointManager` or `WebException`.

### Certificate Validation Changes

There is a new API called `HttpClientHandler.ServerCertificateCustomValidationCallback`.
- https://docs.microsoft.com/en-us/dotnet/api/system.net.http.httpclienthandler.servercertificatecustomvalidationcallback?view=netframework-4.8
- c1778515a3/src/System.Net.Http/src/System/Net/Http/HttpClientHandler.Unix.cs (L154)
- c1778515a3/src/System.Net.Http/src/System/Net/Http/HttpClientHandler.Windows.cs (L383)

The `ServicePointManager.ServerCertificateValidationCallback` is no longer invoked and on
certificate validation failure, `AuthenticationException` (from `System.Security.Authentication`)
is thrown instead of `WebException`.

At the moment, the `NSUrlSessionHandler` still uses it's own validation callback and also still
throws `WebException` on failure; we should probably look into making this consistent with the
other handlers.

### Minor adjustments related to internal Mono APIs

* `HttpContent.SerializeToStreamAsync()` is now `protected` (changed from `protected internal`).
  - src/Foundation/NSUrlSessionHandler.cs: changed overload accordingly.
  - src/System.Net.Http/CFContentStream.cs: likewise.

* `HttpHeaders.GetKnownHeaderKind()` is an internal Mono API.
   There is a new internal API called `System.Net.Http.PlatformHelper.IsContentHeader(key)`
   which exists in both the old as well as the new implementation.
   The correct way of doing it with the CoreFX handler is
   `HeaderDescriptor.TryGet (key, out var descriptor) && descriptor.HeaderType == HttpHeaderType.Content`

### Minor adjustments to tests.

* `HttpClientHandler.MaxRequestContentBufferSize` is now longer supported, you can set it to
  any non-negative value, the getter will always return 0.
  See c1778515a3/src/System.Net.Http/src/System/Net/Http/HttpClientHandler.Core.cs (L18).
  - tests/linker/ios/link sdk/HttpClientHandlerTest.cs: removed assertion from test.

* `HttpMessageInvoker.handler` is a `protected private` field - in the CoreFX handler, it is
  called `_handler` and `private`.  This is accessed via reflection by some of the tests, which are
  now using the new name.
  - tests/mmptest/src/MMPTest.cs: here
  - tests/mtouch/MTouch.cs: here

* tests/monotouch-test/System.Net.Http/MessageHandlers.cs:
  Adjust `RejectSslCertificatesServicePointManager` to reflect the certificate validation
  changes described above.
  - FIXME: There was an `Assert.Ignore()` related to `NSUrlSessionHandler` and macOS 10.10;
    I removed that to reenable the test because the description linked to an old issue in
    the private repo that was referenced by several "Merged" PR's, so it looked to me that
    this might have already been fixed - and I also didn't see why it would fail there.
2019-12-04 15:10:32 -05:00
Rolf Bjarne Kvinge 4586171c4c
[xibuild] Clean up temporary files when done. (#7508) 2019-12-03 14:42:56 +01:00
Sebastien Pouliot 845e7574ae
[linker] Fix "Linked Away" exceptions with Thread.CurrentPrincipal. Fix #7321 (#7510)
Keep 'CallContextSecurityData' around since it's quite small (and the
normal linker logic will be able to deal with it if unused) and allows
the use of `Thread.CurrentPrincipal`

Also add unit test.

Fix https://github.com/xamarin/xamarin-macios/issues/7321
2019-11-28 14:17:00 -05:00
monojenkins fd7a5360a5 [xcode11.3] [linker] Do not mark NSObject subclasses from Xamarin.Forms.Platform.iOS.dll assembly (#7483)
Turn older #7165 prototype into an experimental feature. It can be
enabled by adding `--optimize=experimental-xforms-product-type` to the
**Additional mtouch arguments** of the project.

ref: https://github.com/xamarin/xamarin-macios/pull/7165
2019-11-26 08:32:02 -05:00
Sebastien Pouliot 611d6f37fb
[runtime] Comment two special cases that triggers static analysis tools (#7491) 2019-11-26 08:30:05 -05:00
Manuel de la Pena cb815ecd39
[Runtime] Update cflags and fix warnings accordingly. (#7478)
Update flags:

* Add -Werror so that warnings are errors.
* Move to -Wconversion that adds:
  * -Wbitfield-enum-conversion
  * -Wbool-conversion
  * -Wconstant-conversion
  * -Wenum-conversion
  * -Wfloat-conversion
  * -Wimplicit-float-conversion
  * -Wimplicit-int-conversion
  * -Wint-conversion
  * -Wliteral-conversion
  * -Wnon-literal-null-conversion
  * -Wnull-conversion
  * -Wobjc-literal-conversion
  * -Wshorten-64-to-32
  * -Wsign-conversion
  * -Wstring-conversion
Last work related to: https://github.com/xamarin/xamarin-macios/pull/7405
2019-11-25 16:00:24 -05:00
Sebastien Pouliot 58d325746a
[linker] Do not mark NSObject subclasses from Xamarin.Forms.Platform.iOS.dll assembly (#7473)
Turn older #7165 prototype into an experimental feature. It can be
enabled by adding `--optimize=experimental-xforms-product-type` to the
**Additional mtouch arguments** of the project.

ref: https://github.com/xamarin/xamarin-macios/pull/7165
2019-11-22 15:38:46 +01:00
Rolf Bjarne Kvinge 806404fa51
Keep track of SDK versions and OS versions separately. Fixes xamarin/maccore#2066. (#7454)
The latest SDK version and the latest OS version does not necessarily have to
match (for instance the iOS 13.2 SDK can support both iOS 13.2 and iOS 13.3),
so keep track of them separately.

Also use the latest OS version to determine which simulator to run, instead of
the latest SDK version (Xcode 11.3 ships with the iOS 13.2 SDK but only has an
iOS 13.3 simulator, not an iOS 13.2 simulator).

Fixes https://github.com/xamarin/maccore/issues/2066.
2019-11-15 17:21:10 +01:00
Vincent Dondain 5e52f30c05 Bump for Xcode 11.3 beta 1 (#7431)
* Bump for Xcode 11.3 beta 1

* [system-dependencies] Make it clearer what failed on the bots.

Locally we use colors to distinguish between warnings and failures, but colors
don't show up on the bots, so use text instead.

* Verbose provisioning.

* [system-dependencies] Improve simulator checks a bit.

* Non-verbose provisioning.
2019-11-14 14:32:26 +01:00
Manuel de la Pena 63ab48e679
[Runtime] Enable the -Wsign-conversion warnings flag and fix some of the warnings. (#7405)
The flag adds a lot (A LOT) of warnings. The diff would be huge and
dangerous to review. We enable the flag and fix file by file.
2019-11-12 15:10:16 -05:00
monojenkins 31be34a306 [xcode11.3] [Packaging] Ensure that when we build from source, the srcs go to the correct place. (#7407)
* [Packaging] Ensure that when we build from source, the srcs go to the correct plance.

When building from source, the install-sources command was not moving
the files correctly. This change makes sure that, if we build from
source, we do add the mono sources in the correct location.

Fixes: https://github.com/xamarin/xamarin-macios/issues/7393
2019-11-12 09:05:35 -05:00
Manuel de la Pena e639470892
[Packaging] Ensure that when we build from source, the srcs go to the correct place. (#7394)
* [Packaging] Ensure that when we build from source,  the srcs go to the correct plance.

When building from source, the install-sources command was not moving
the files correctly. This change makes sure that, if we build from
source, we do add the mono sources in the correct location.

Fixes: https://github.com/xamarin/xamarin-macios/issues/7393
2019-11-11 18:18:28 -05:00
monojenkins 6d13a39027 [Install] Ensure that sources from mono are included when building from source. (#7389)
The PathManglerFactory was getting confused when building mono from
source again and since it did not find the sources in the download
directory it considered that the path to modify was part of Xamarin.

Now the factory tests first if we are getting a path from the mono
external submodule, if that is the case we know is a mono path and do
the right thing.

Fixes https://github.com/xamarin/maccore/issues/2053
2019-11-08 16:08:30 -05:00
Manuel de la Pena cf6280b305
[Install] Ensure that sources from mono are included when building from source. (#7385)
The PathManglerFactory was getting confused when building mono from
source again and since it did not find the sources in the download
directory it considered that the path to modify was part of Xamarin.

Now the factory tests first if we are getting a path from the mono
external submodule, if that is the case we know is a mono path and do
the right thing.

Fixes https://github.com/xamarin/maccore/issues/2053
2019-11-08 10:33:20 -05:00
monojenkins 5fa3d8abbb [Install] Do not crash when the directory is not found. (#7382)
This is NOT a fix for https://github.com/xamarin/maccore/issues/2053 but
a nice thing to have to make sure that when the file cannot be copied we
have a useful debugging message. Fixing
https://github.com/xamarin/maccore/issues/2053 should be done in the
path mangler that is doing something wrong with the paths.
2019-11-07 13:25:41 -05:00
Manuel de la Pena 566e5a8d12
[Install] Do not crash when the directory is not found. (#7378)
This is NOT a fix for https://github.com/xamarin/maccore/issues/2053 but
a nice thing to have to make sure that when the file cannot be copied we
have a useful debugging message. Fixing
https://github.com/xamarin/maccore/issues/2053 should be done in the
path mangler that is doing something wrong with the paths.
2019-11-06 16:09:02 -05:00
Vincent Dondain a558d84457 Merge branch 'xcode11.2' into master-merge-xcode11.2 2019-11-05 15:25:49 -05:00
Vincent Dondain ec3c01d131 Merge branch 'xcode11.2' into d16-4-xcode11.2-merge 2019-11-01 17:11:44 -04:00
Sebastien Pouliot f6efbf551e
[registrar] Check parameter pointer before de-referencing it (#7336)
This case was generated (only) twice

1. https://developer.apple.com/documentation/uikit/uipopoverpresentationcontrollerdelegate/1622326-popoverpresentationcontroller?language=objc
where `inView:(inout UIView * _Nonnull *)view;`

2. https://developer.apple.com/documentation/quicklook/qlpreviewcontrollerdelegate/1617007-previewcontroller?language=objc
where `inSourceView:(UIView * _Nullable *)view;`
and `CGRect FrameForPreviewItem (QLPreviewController controller, [Protocolize] QLPreviewItem item, ref UIView view);`

This is not an issue because it won't be `null` in those case
but making the code safer (against future changes/API) is safer.

The output (copy-back) to the pointer already has a null-check.
2019-10-30 14:09:29 -04:00
Sebastien Pouliot bba2ea46df
[simlauncher] Move simlauncher source to .mm file extension (ObjC++) (#7334)
This is already compiled as ObjC++ but this gives an additional (and
required) hint to some tools that it is Objective C++ code.

This avoid a lot of build errors (under static analysis tools),
most of them not really helpful (a trait shared with many C++ compiler
warnings and errors) except for this one:

```
error: invalid argument ‘-std=c++14’ not allowed with ‘Objective-C’
```
2019-10-30 11:40:05 -04:00
William Kent 8425129eeb Add bindings for NSXpcConnection and related types (#7001)
* Add bindings for NSXpcConnection and related types

* Re-add accidentally deleted file

* Typo fix

* Add NSXpcInterface.CreateForType()

* Add MethodInfo-taking overloads to NSXpcInterface

* Add null check

* Mark methods with wrappers as internal

Also fixed a formatting bug that I didn't catch earlier.

* Change NSXPCProxyCreating methods to be strongly typed

I got rid of the NSXpcProxyCreating interface in this change,
because its only user was NSXpcConnection, and I needed to
inline the protocol methods into the class definition so I
could mark them as [Internal].

* Add missing casts

* Add NSXpcConnectionOptions enum

* Convert NSXpcConnection constructor to use new enum

* Remove now-unneeded manual constructor

* Fix bgen warning

* Typo fix

* Fix selector

* Remove incorrect use of BindAsAttribute

Per the docs, this only works for enums backed
by NSNumber and NSValue, not for enums
passed directly as integers.

* Fix duplicated selector errors

* Throw ArgumentException instead of InvalidOperationException

* Extend AppExtension targets to produce XPC services

Rather than create an entirely new set of targets
(that would require VS and VSMac updates to properly
consume), I have decided to use the existing AppExtension
build targets to produce XPC services as well. All the
user must do is set the $(IsXPCService) property to true in
their project file, and the targets will do The Right Thing™.

Note that this support is Mac-only for now; I may need a bit
of help adjusting them to work on for iOS/watchOS/tvOS, as I
am not as familiar with those platforms.

* Copy XPC service bundles into the correct location

* Move IsXPCService property definition to props file

* Don't pass /extension to mmp for XPC service targets

This would cause the XPC service binary to
be linked incorrectly.

* Add NSXpcConnection/NSXpcInterface.cs files to the build

* Fix build

* Fix build

* Add required type parameter requirements

* Fix type parameter requirements

* Fix return type

* Fix return type of NSXpcInterface.CreateForProtocol ()

* Take ownership of the returned object types

* Adjust XPC service mmp invocation

I need to link the XPC service bundle as if it is an app extension, but
I must not use xamarin_mac_extension_main. I added a new flag to make
this possible.

* Change mmp to correctly construct XPC service bundle

* Set the MonoBundleExecutable Info.plist key for XPC services

* Use the runtime to get the protocol

* Make NSXpcInterface.CreateForProtocol() public

The static registrar must be used for Cocoa to accept the protocol
as a valid XPC interface, but that then seems to break resolving
the protocol from the type. I must therefore hard-code the protocol
name in my code, and that requires I make this constructor public.

* Add XpcInterfaceAttribute

See the doc comment in XpcInterfaceAttribute.cs for why
this type is required. The referenced mmp optimizations
will be added in future commits.

* Add XpcInterfaceAttribute to generator build

* Add support for XpcInterfaceAttribute to the generator

* Force static generation of protocols decorated with XpcInterfaceAttribute

* Change how static registrar translates block parameters

Previously, they would always be marshalled as "id".
This would throw off the XPC subsystem, which parses
the block signature to determine the communication
protocol.

* Undo whitespace noise

* Remove unneeded casts

* Add trailing comma

* Use HasAttribute instead of GetCustomAttribute

* Fix style issues

* Bind NSXpcConnection.auditSessionIdentifier

* Address naming feedback

* Make Get/SetAllowedClasses public

IMHO, passing the selector as a string is just as
usable as passing a MethodInfo, and is also less
verbose if you copy/paste the selector string
from the ExportAttribute. There is no reason why
we cannot have both overloads be public.

* Update overload names to match

* Update more overload names to match

* Make mmp --xpc imply --extension

* Reformat if statement

* Fix build

* Conditionalize creation of PlugIns and XPCServices directories

* Add AutoGeneratedName

Co-Authored-By: Rolf Bjarne Kvinge <rolf@xamarin.com>

* Get rid of ProtocolizeAttribute

* Update availability attributes to please xharness

I actually think xharness is wrong here, since the
NSXPCConnection header lists these types as being
available starting in macOS 10.8.

* Update sharpie ignore files to reflect changes

This should fix the xtro-sharpie test failures CI has been reporting.

* Fix MM4105 error generation

* Adjust error message in test to match mmp

I had to change the error text slightly, because the type of the parameter
cannot be determined where the error is thrown anymore. However, the newer
exception message IMO is just as clear.

* Make exception message match test exactly

* Remove outdated copyright header text

* Remove more outdated copyright header text

* Revert changes to MM4105 error generation

I have a more elegant way of fixing this test now.

* Return "id" if Invoke method cannot be found

This fixes the MM4105 error unit test,
without requiring modification to that test.

* Remove redundant availability attributes

* Add DesignatedInitializerAttribute

* Re-add required code to macOS-Foundation.ignore

* Put DesignatedInitializer on the right constructor

* Update xtro-sharpie ignore files
2019-10-22 15:38:01 +02:00
Rolf Bjarne Kvinge f4563c2f92
Bump mono to get archives built with Xcode 11.1. (#7237)
* Bump mono to get archives built with Xcode 11.1.

New commits in mono/mono:

* mono/mono@b601371d5f Update MERP event type to MonoAppCrash
* mono/mono@6184ff007b [2019-08][ci] Use Xcode11.1 and 11.2beta2 for XI/XM Mono SDK builds (#17324)
* mono/mono@8969f2cc99 [2019-08] [merp] Include any managed methods in the 'unmanaged_frames' portion … (#17316)
* mono/mono@3009440108 [2019-08][merp] Don't install SIGTERM handler in EnableMicrosoftTelemetry (#17308)

Diff: df5e13f95d..b601371d5f

* Fix accidental/mysterious/captivating/incorrect merge solution selected by git when merging d16-4/xcode11.1 into master.
2019-10-17 23:10:23 +02:00
Rolf Bjarne Kvinge ab60a11f14
[mmp] Add deployment target to the -fobjc-runtime clang argument. Fixes #7204. (#7227) (#7231)
This is required when also passing the -fobjc-arc argument.

Fixes https://github.com/xamarin/xamarin-macios/issues/7204.

This is a backport of #7227.
2019-10-17 17:44:37 +02:00
Sebastien Pouliot a50c753117
[coreimage] Update for Xcode 11[.2] (#7216)
Apple decided to expose most (but not all) `CIFilter` using protocols
(instead of weakly named dictionaries). Most of this maps well with
our strong bindings but there are cases where we:

* missing some properties (easy, there were added); or
* used a different types [1] and that requires new members / obsoletion

A few other API were also added in Xcode 11 (nothing in 11.1 or 11.2) and
included in this PR.

New introspection tests were also added to minimize the risk that
the API and generator changes produced incorrect code. This lead
to the finding of some missing API (in particular `Output*` properties)
that were added.

[1] Often ours are better (using `float` for a `bool` value is not
optimal) but we do not have `[BindAs]` for protocols :( to _fix_ them

Note: this replace draft PR https://github.com/xamarin/xamarin-macios/pull/7120 but it's has quite a bit of changes in filter generation (inlining protocols) and that affected bindings too.
2019-10-16 09:16:32 -04:00
monojenkins b16f41abd0 [linker] Ensure we can remove NSUrlSessionHandler if unused (#7164)
Moving `NSUrlSessionHandler` into the platform assemblies (e.g.
Xamarin.iOS.dll) instead of System.Net.Http.dll was not optimal as it
prevented the linker to remove it when the application did not use it.

This shows a lot in an "helloworld" type of application (see below)
but in real life most of the removed code gets used by something else
(and is included.

```
Directories / Files                                                              helloworld-d16-4.app helloworld-fixed.app         diff            %
./
        AppIcon60x60@2x.png                                                             2,632        2,632            0       0.00 %
        AppIcon76x76@2x~ipad.png                                                        3,125        3,125            0       0.00 %
        archived-expanded-entitlements.xcent                                              181          181            0       0.00 %
        Assets.car                                                                     75,688       75,688            0       0.00 %
        embedded.mobileprovision                                                        8,456        8,456            0       0.00 %
        helloworld                                                                  4,700,256    3,915,936     -784,320     -16.69 %
        helloworld.aotdata.arm64                                                        1,448        1,432          -16      -1.10 %
        helloworld.exe                                                                  6,144        6,144            0       0.00 %
        Info.plist                                                                      1,712        1,712            0       0.00 %
        mscorlib.aotdata.arm64                                                        406,080      364,104      -41,976     -10.34 %
        mscorlib.dll                                                                  561,664      501,760      -59,904     -10.67 %
        NOTICE                                                                            159          159            0       0.00 %
        PkgInfo                                                                             8            8            0       0.00 %
        System.aotdata.arm64                                                           17,008          936      -16,072     -94.50 %
        System.Core.aotdata.arm64                                                       2,432            0       -2,432    -100.00 %
        System.Core.dll                                                                 4,608            0       -4,608    -100.00 %
        System.dll                                                                     32,768        5,120      -27,648     -84.38 %
        System.Net.Http.aotdata.arm64                                                  31,648            0      -31,648    -100.00 %
        System.Net.Http.dll                                                            29,184            0      -29,184    -100.00 %
        Xamarin.iOS.aotdata.arm64                                                      62,544       33,464      -29,080     -46.50 %
        Xamarin.iOS.dll                                                                92,672       53,248      -39,424     -42.54 %
./_CodeSignature
        CodeResources                                                                   7,575        6,655         -920     -12.15 %
./LaunchScreen.storyboardc
        01J-lp-oVM-view-Ze5-6b-2t3.nib                                                  1,831        1,835            4       0.22 %
        Info.plist                                                                        258          258            0       0.00 %
        UIViewController-01J-lp-oVM.nib                                                   896          896            0       0.00 %
./Main.storyboardc
        BYZ-38-t0r-view-8bC-Xf-vdC.nib                                                  1,836        1,832           -4      -0.22 %
        Info.plist                                                                        258          258            0       0.00 %
        UIViewController-BYZ-38-t0r.nib                                                   916          916            0       0.00 %

Statistics

Native subtotal                                                                     4,700,256    3,915,936     -784,320     -16.69 %
    Executable                                                                      4,700,256    3,915,936     -784,320     -16.69 %
    AOT data *.aotdata                                                                      0            0            0          -

Managed *.dll/exe                                                                     727,040      566,272     -160,768     -22.11 %

TOTAL                                                                               6,053,987    4,986,755   -1,067,232     -17.63 %
```
2019-10-15 17:50:42 -04:00
Vincent Dondain 61d0e1809f Merge remote-tracking branch 'origin/master' into master-merge-d16-4 2019-10-14 18:17:35 -04:00
Vincent Dondain a71292165e Merge branch 'd16-4' into master-merge-d16-4 2019-10-14 18:06:42 -04:00
Rolf Bjarne Kvinge dee19397f7
[mmp] Add deployment target to the -fobjc-runtime clang argument. Fixes #7204. (#7227)
This is required when also passing the -fobjc-arc argument.

Fixes https://github.com/xamarin/xamarin-macios/issues/7204.
2019-10-14 22:25:58 +02:00
Rolf Bjarne Kvinge a8fdf6db6e
Implement a different escaping/quoting algorithm for arguments to System.Diagnostics.Process. (#7177)
* Implement a different escaping/quoting algorithm for arguments to System.Diagnostics.Process.

mono changed how quotes should be escaped when passed to
System.Diagnostic.Process, so we need to change accordingly.

The main difference is that single quotes don't have to be escaped anymore.

This solves problems like this:

    System.ComponentModel.Win32Exception : ApplicationName='nuget', CommandLine='restore '/Users/vsts/agent/2.158.0/work/1/s/tests/sampletester/bin/Debug/repositories/ios-samples/WorkingWithTables/Part 3 - Customizing a Table\'s appearance/3 - CellCustomTable/CellCustomTable.sln' -Verbosity detailed -SolutionDir '/Users/vsts/agent/2.158.0/work/1/s/tests/sampletester/bin/Debug/repositories/ios-samples/WorkingWithTables/Part 3 - Customizing a Table\'s appearance/3 - CellCustomTable'', CurrentDirectory='/Users/vsts/agent/2.158.0/work/1/s/tests/sampletester/bin/Debug/repositories', Native error= Cannot find the specified file
      at System.Diagnostics.Process.StartWithCreateProcess (System.Diagnostics.ProcessStartInfo startInfo) [0x0029f] in /Users/builder/jenkins/workspace/build-package-osx-mono/2019-08/external/bockbuild/builds/mono-x64/mcs/class/System/System.Diagnostics/Process.cs:778

ref: https://github.com/mono/mono/pull/15047

* Rework process arguments to pass arrays/lists around instead of quoted strings.

And then only convert to a string at the very end when we create the Process
instance.

In the future there will be a ProcessStartInfo.ArgumentList property we can
use to give the original array/list of arguments directly to the BCL so that
we can avoid quoting at all. These changes gets us almost all the way there
already (except that the ArgumentList property isn't available quite yet).

We also have to bump to target framework version v4.7.2 from v4.5 in several
places because of 'Array.Empty<T> ()' which is now used in more places.

* Parse linker flags from LinkWith attributes.

* [sampletester] Bump to v4.7.2 for Array.Empty<T> ().

* Fix typo.

* Rename GetVerbosity -> AddVerbosity.

* Remove unnecessary string interpolation.

* Remove unused variable.

* [mtouch] Simplify code a bit.

* Use implicitly typed arrays.
2019-10-14 16:18:46 +02:00
Sebastien Pouliot 445bf4be07
[registrar] Increase code sharing in generated `native_to_managed_trampoline_*` functions (#7221)
Several functions were 100% identical except for their signature. However
the signature were over-specialized and simplifying them allows the
(already) present code merge feature to achieve much better results.

Final size impact will vary based on the API used (both the managed and
native linker can remove some) but the more API you use the more likely
the application included duplicate code.

**Before**

`tools/mtouch/Xamarin.iOS.registrar.ios.x86_64.m` has `native_to_managed_trampoline_[1..379]`

```
find tools/ -name Xamarin.*.a | xargs ls -l
-rw-r--r--  1 poupou  staff  4143148 11 Oct 15:09 tools//mmp/Xamarin.Mac.registrar.full.a
-rw-r--r--  1 poupou  staff  4139052 11 Oct 15:09 tools//mmp/Xamarin.Mac.registrar.full.x86_64.a
-rw-r--r--  1 poupou  staff  4143160 11 Oct 15:09 tools//mmp/Xamarin.Mac.registrar.mobile.a
-rw-r--r--  1 poupou  staff  4139064 11 Oct 15:09 tools//mmp/Xamarin.Mac.registrar.mobile.x86_64.a
-rw-r--r--  1 poupou  staff  4521752 11 Oct 15:09 tools//mtouch/Xamarin.iOS.registrar.ios.i386.a
-rw-r--r--  1 poupou  staff  9240416 11 Oct 15:09 tools//mtouch/Xamarin.iOS.registrar.ios.simulator.a
-rw-r--r--  1 poupou  staff  4714336 11 Oct 15:09 tools//mtouch/Xamarin.iOS.registrar.ios.x86_64.a
```

**After**

`tools/mtouch/Xamarin.iOS.registrar.ios.x86_64.m` has `native_to_managed_trampoline_[1..132]`

```
find tools/ -name Xamarin.*.a | xargs ls -l
-rw-r--r--  1 poupou  staff  3723012 11 Oct 14:57 tools//mmp/Xamarin.Mac.registrar.full.a
-rw-r--r--  1 poupou  staff  3718916 11 Oct 14:57 tools//mmp/Xamarin.Mac.registrar.full.x86_64.a
-rw-r--r--  1 poupou  staff  3723016 11 Oct 14:57 tools//mmp/Xamarin.Mac.registrar.mobile.a
-rw-r--r--  1 poupou  staff  3718920 11 Oct 14:57 tools//mmp/Xamarin.Mac.registrar.mobile.x86_64.a
-rw-r--r--  1 poupou  staff  3995520 11 Oct 14:57 tools//mtouch/Xamarin.iOS.registrar.ios.i386.a
-rw-r--r--  1 poupou  staff  8195748 11 Oct 14:57 tools//mtouch/Xamarin.iOS.registrar.ios.simulator.a
-rw-r--r--  1 poupou  staff  4193956 11 Oct 14:57 tools//mtouch/Xamarin.iOS.registrar.ios.x86_64.a
```
2019-10-11 22:18:58 -04:00
monojenkins 36040b55ca [Tests] Add the new test dll from mono to xharness. (#7189)
Following PR  https://github.com/xamarin/xamarin-macios/pull/7147 we
have a new dll and therefore new tests to run.

Fixes: https://github.com/xamarin/xamarin-macios/issues/7148
2019-10-09 15:37:00 -04:00
Manuel de la Pena 118fac4c2a
[Tests] Add the new test dll from mono to xharness. (#7188)
Following PR  https://github.com/xamarin/xamarin-macios/pull/7147 we
have a new dll and therefore new tests to run.

Fixes: https://github.com/xamarin/xamarin-macios/issues/7148
2019-10-07 21:52:03 -04:00
Vincent Dondain a3076f29a3 Merge branch 'xcode11.1' into d16-4 2019-10-07 21:46:13 -04:00
Rolf Bjarne Kvinge 8128976084
[runtime] Split the build differently: build libraries for simulator and device separately. (#7143)
This has a couple of advantages:

* It makes it easier to add a catalyst version of these libraries (because it
  becomes cumbersome to build for catalyst when the build rules assumes we're
  building for both simulator and device).
* It makes it easier to create an xcframework of our libraries, because the
  contents in an xcframework is split like this.
2019-10-03 16:38:44 +02:00
Sebastien Pouliot 82b84d0c8f
[linker] Ensure we can remove NSUrlSessionHandler if unused (#7151)
Moving `NSUrlSessionHandler` into the platform assemblies (e.g.
Xamarin.iOS.dll) instead of System.Net.Http.dll was not optimal as it
prevented the linker to remove it when the application did not use it.

This shows a lot in an "helloworld" type of application (see below)
but in real life most of the removed code gets used by something else
(and is included.

```
Directories / Files                                                              helloworld-d16-4.app helloworld-fixed.app         diff            %
./
        AppIcon60x60@2x.png                                                             2,632        2,632            0       0.00 %
        AppIcon76x76@2x~ipad.png                                                        3,125        3,125            0       0.00 %
        archived-expanded-entitlements.xcent                                              181          181            0       0.00 %
        Assets.car                                                                     75,688       75,688            0       0.00 %
        embedded.mobileprovision                                                        8,456        8,456            0       0.00 %
        helloworld                                                                  4,700,256    3,915,936     -784,320     -16.69 %
        helloworld.aotdata.arm64                                                        1,448        1,432          -16      -1.10 %
        helloworld.exe                                                                  6,144        6,144            0       0.00 %
        Info.plist                                                                      1,712        1,712            0       0.00 %
        mscorlib.aotdata.arm64                                                        406,080      364,104      -41,976     -10.34 %
        mscorlib.dll                                                                  561,664      501,760      -59,904     -10.67 %
        NOTICE                                                                            159          159            0       0.00 %
        PkgInfo                                                                             8            8            0       0.00 %
        System.aotdata.arm64                                                           17,008          936      -16,072     -94.50 %
        System.Core.aotdata.arm64                                                       2,432            0       -2,432    -100.00 %
        System.Core.dll                                                                 4,608            0       -4,608    -100.00 %
        System.dll                                                                     32,768        5,120      -27,648     -84.38 %
        System.Net.Http.aotdata.arm64                                                  31,648            0      -31,648    -100.00 %
        System.Net.Http.dll                                                            29,184            0      -29,184    -100.00 %
        Xamarin.iOS.aotdata.arm64                                                      62,544       33,464      -29,080     -46.50 %
        Xamarin.iOS.dll                                                                92,672       53,248      -39,424     -42.54 %
./_CodeSignature
        CodeResources                                                                   7,575        6,655         -920     -12.15 %
./LaunchScreen.storyboardc
        01J-lp-oVM-view-Ze5-6b-2t3.nib                                                  1,831        1,835            4       0.22 %
        Info.plist                                                                        258          258            0       0.00 %
        UIViewController-01J-lp-oVM.nib                                                   896          896            0       0.00 %
./Main.storyboardc
        BYZ-38-t0r-view-8bC-Xf-vdC.nib                                                  1,836        1,832           -4      -0.22 %
        Info.plist                                                                        258          258            0       0.00 %
        UIViewController-BYZ-38-t0r.nib                                                   916          916            0       0.00 %

Statistics

Native subtotal                                                                     4,700,256    3,915,936     -784,320     -16.69 %
    Executable                                                                      4,700,256    3,915,936     -784,320     -16.69 %
    AOT data *.aotdata                                                                      0            0            0          -

Managed *.dll/exe                                                                     727,040      566,272     -160,768     -22.11 %

TOTAL                                                                               6,053,987    4,986,755   -1,067,232     -17.63 %
```
2019-10-02 10:10:26 -04:00
monojenkins d45ad36c66 Bump mono to 2019-08@528103728fc2aedb7b6062e11255d39a0ed3f31c to add missing NS2.1 APIs (#7150)
See https://github.com/mono/mono/issues/17064.

The 2019-08 version of the fix adds a new assembly with NS2.1 APIs that we stubbed out in 2019-06: System.Data.DataSetExtensions.dll
2019-10-02 08:08:41 -04:00