Граф коммитов

9971 Коммитов

Автор SHA1 Сообщение Дата
Rolf Bjarne Kvinge b644e23adf
[tests] Don't use the tcp tunnel in the HttpClientHandler tests. Fixes xamarin/maccore#2154. (#10451)
Fixes https://github.com/xamarin/maccore/issues/2154.
2021-01-18 14:27:34 +01:00
Rolf Bjarne Kvinge aa671914f9
[dotnet] Remove workaround for mono/linker#1304, which has now been fixed. (#9862) 2021-01-18 08:21:31 +01:00
Rolf Bjarne Kvinge 18ab7db7c7
[xharness] Fix warning about unknown file wrt default inclusion. (#10438)
Fixes this warning when running xharness:

    Unknown file: fsharplibrary.fsproj (extension: .fsproj). There might be a default inclusion behavior for this file.
2021-01-18 07:28:39 +01:00
Sebastien Pouliot 4eda43b1c9
[msbuild] Fix semi-conflicting options to set codesign timestamp (#10428)
TL&DR:

This effectively change nothing - but prevents (warn) if both options
contradict themselves.

Long Story....

So we have two ways to control the codesign's `--timetamp` option but
they both ignore each other so we can end up with the option being
set more than once at build time.

`DisableTimestamp` was the original one. It was meant for iOS (and
derivative OS) and disable the option (which requires the network)
for simulator or debug builds. IOW we _wanted_ timestamps when doing
release builds for devices.

```
DisableTimestamp="$(_CodesignDisableTimestamp)"
```

```
<_CodesignDisableTimestamp>False</_CodesignDisableTimestamp>
<_CodesignDisableTimestamp Condition="'$(_SdkIsSimulator)' == 'true' Or '$(_BundlerDebug)' == 'true'">True</_CodesignDisableTimestamp>
```

Now disabling the timestamp did not mean it was enabled. We did not ask
for a timestamp, leaving it to the default which from `man codesign`
means:

> If this option is not given at all, a system-specific default behavior is invoked.
> This may result in some but not all code signatures being timestamped.

Then `UseSecureTimestamp` was added for macOS builds. If hardening is
enabled then a secure timestamp is required.

`msbuild/Xamarin.Mac.Tasks/Xamarin.Mac.Common.targets` `_CodesignAppBundle`

```
UseSecureTimestamp="$(UseHardenedRuntime)"
```

However it's also exposed for iOS (shared target) in
`msbuild/Xamarin.Shared/Xamarin.Shared.targets` `__CodesignNativeLibraries`
but it would always be `false` in that case.

Adding this option means there's now always a `--timestamp` option given,
either to enable it (no URL so it means using Apple's server) or to
disable it (`=none`) but since it's controlled by `UseHardenedRuntime`,
which is macOS only, then iOS device builds are never timestamped.

An alternative would be to have `UseSecureTimestamp` as a macOS-only
option - but that would change how we currently sign the iOS applications
and I'd rather not change things that are known to work.
2021-01-15 10:43:11 -05:00
Rolf Bjarne Kvinge 4811246277
[msbuild] Don't try to find frameworks in a directory that doesn't exist. (#10378)
Check if the .app has a Frameworks directory before trying to iterate over any
*.framework directories it may have.
2021-01-15 16:42:24 +01:00
Rolf Bjarne Kvinge 8e1f6e1d7e
[msbuild] Fix code signing logic for Mac Catalyst. (#10408)
Code signing for Mac Catalyst is very much like code signing for macOS, except that:

* If a provisioning profile is required, then we must find a code signing certificate.
* Provide a code signing key even if we don't need a code signing certificate.

This makes the tests in monotouch-test that require a correctly signed app pass.
2021-01-15 16:41:50 +01:00
Rolf Bjarne Kvinge 29727d6a8d
[tests] Preserve a required method in System.Private.CoreLib to work around a bug in .NET 6. (#10426)
Ref: https://github.com/dotnet/runtime/issues/46908.
2021-01-15 16:39:44 +01:00
Rolf Bjarne Kvinge 24331f35dd
[monotouch-test] Adjust CaptiveNetworkTest.TryCopyCurrentNetworkInfo to work on iOS 10.3.3 (#10435)
According to Apple's documentation, an app linked with iOS 12 or later must
enable a specific entitlement, otherwise calls to CNCopyCurrentNetworkInfo
will always return NULL.
2021-01-15 16:14:27 +01:00
Manuel de la Pena d04a86a408
[CI][VSTS] The pkg of the tests fail because there is a path not longer found. (#10432) 2021-01-15 07:53:05 -05:00
Rolf Bjarne Kvinge ef8cdfa64e
[dotnet] Add default exclusions for recursive patterns to not pick up files in the bin/obj directories. (#10422) 2021-01-15 08:25:42 +01:00
Rolf Bjarne Kvinge ef53600f7a
[monotouch-test] Change the url for the TestPACParsingScriptNoProxy test to be the Microsoft uri. (#10423)
All the other *NoProxy tests use the Microsoft uri, which seems to be the uri
that's not supposed to need a proxy (according to what I understand from the
code) - in other words, it looks like this was a c&p error.

Fixes this test failure when running on device:

    [FAIL] TestPACParsingAsyncNoProxy :   Expected: None
        But was:  HTTPS
	        at MonoTouchFixtures.CoreFoundation.ProxyTest.TestPACParsingAsyncNoProxy () [0x000fa] in /Users/rolf/work/maccore/onedotnet/xamarin-macios/tests/monotouch-test/CoreFoundation/ProxyTest.cs:238
2021-01-15 08:25:19 +01:00
Rolf Bjarne Kvinge 48080d2d03
[monotouch-test] Add version checks for new HKCategoryTypeIdentifier fields. (#10424) 2021-01-15 08:24:29 +01:00
Rolf Bjarne Kvinge ca7d562493
[monotouch-test] Add a few Xcode version checks to ColorTest and AvoidOccluderConstraintTest. (#10425)
* [monotouch-test] Adjust ColorTest.CreateByMatchingToColorSpace to be Xcode 11+ only.

This test uses CGColor.CreateSrgb, which is Xcode 11+ only.

* [monotouch-test] Restrict AvoidOccluderConstraintTest to iOS 11+.
2021-01-15 08:24:12 +01:00
Manuel de la Pena 518657ba15
[CI][VSTS] If comments are too large, provide a gist. (#10431)
Github has a limited size for messages in comments. If we did reach that
limit, we create a gist to show all the results.
2021-01-14 18:33:22 -05:00
Manuel de la Pena 4f3b942a78
[CI][VSTS] When webservers are rude, retry (#10411)
Perseverance is failing 19 times and succeeding the 20th.

fix: https://github.com/xamarin/maccore/issues/2361
2021-01-14 12:34:42 -05:00
Sebastien Pouliot 05e28c3713
[macos] Add correct support for producing/archiving `dSYM` (#10409)
TL&DR: This PR

1. Removes the creation of the `.dSYM` based on `Debug Information` [1]

2. Adds dSYM support to XM msbuild (now shared with XI implementation)

3. Archive the `.dSYM` directories (plural) properly, e.g.

```
msbuild -p:Configuration=Release -p:ArchiveOnBuild=true
```

Why ? The long story...

Historically `.dSYM` for Xamarin.Mac have not been very useful, largely
because (most of) the code is JITed so not much is known before runtime.
So they were simply not generated during the builds...

However AOT options were added to Xamarin.Mac, making them potentially
more useful. Also symbols from `libmono` and other native libraries /
frameworks can prove useful when diagnosing application crashes.

Unsurprisingly developers looking to get symbols eventually found _a way_
[1] to get a `.dSYM` for their applications - but it was not quite
correct because:

* setting the debug information option meant that `mmp` would be supplied with `-debug`. This disables several optimizations that are, by default, enabled for release builds. IOW generating symbols should have no effect on the executing code (but it had);

* it was produced when compiling the native launcher, so the symbols coverage was incomplete. How much depends if mono was statically or dynamically linked. However this would not cover any AOTed code nor bundled libraries or user frameworks.

* the .dSYM was produced inside the `x.app/Contents/MacOS/`, side-by-side with the native executable, which makes it part of the **signed** `.app` and also part of the created (and signed) `.pkg`. This had a large impact on the application's, disk and download, size(s). Manually (re)moving the `.dSYM` means re-signing the app and re-creating (and signing) the `.pkg` is not a good solution.

[1] https://forums.xamarin.com/discussion/139705/how-to-symbolicate-a-xam-mac-crash-log

Additional fixes

* Use `Directory.Move` instead of running the `mv` command

While the result is identical there is a cost to spawn several `mv`
processes. Doing it in parallel (might have) helped but that setup
also comes at a cost.

`Directory.Move` the four `.dylib.dSYM` of an app takes 1 ms, while
the existing code took 17 ms to do the same.

* Fix building mmptest since the DeleteDebugSymbolCommand constant is not present (nor used) anymore
2021-01-14 08:42:24 -05:00
Rolf Bjarne Kvinge 471da9f51b
[dotnet] Don't AOT compile resource assemblies. (#10418)
Xamarin.iOS don't AOT compile resource assemblies, so we shouldn't do so in
.NET either (and in any case they end up causing native linking failures due
to duplicate native symbols).
2021-01-14 14:37:50 +01:00
Rolf Bjarne Kvinge db79cd6b4c
[monotouch-test] Fix a few issues when running the .NET version of monotouch-test on device. (#10419)
* [monotouch-test] Fix conditional compilation constants when building for device.

* [tests/monotouch-test] Don't use entitlements when building monotouch-test for device.
2021-01-14 14:37:31 +01:00
Rolf Bjarne Kvinge 7584ada30d
Bump to .NET 6.0.100-alpha.1.21060.3. (#10388)
* Bump to .NET 6.0.100-alpha.1.21060.3.

* Fix dotnet command line arguments.

* dotnet build: the project file must be the first argument.
* dotnet build/publish: use the documented verbosity format.

* Update version number in tests.

* [tests/introspection] Adjust introspection to cope with different library names in the new .NET version.

* [tests/link sdk] Adjust the LinkSdkRegressionTest.SpecialFolder test according to the new version of .NET 6.

* [tests/link sdk] Preserve a required method in System.Private.CoreLib to work around a bug in .NET 6.

Ref: https://github.com/dotnet/runtime/issues/46908.

* Revert "[CI][VSTS] Add the donet 6 pkg as a dependency. (#10348)"

This reverts commit 6de4e717e7.

There's no need to provision .NET 6, it's done automatically.
2021-01-14 14:07:28 +01:00
Rolf Bjarne Kvinge c61f6aeeac
Use the reference assemblies from the final .NET 5.0 release. (#10385) 2021-01-14 07:57:14 +01:00
Rolf Bjarne Kvinge cfb32cba09
[dotnet-linker] Link with GSS when building for iOS/Mac Catalyst. (#10414)
* [dotnet-linker] Add support for writing to the same MSBuild output items multiple times.

* Split parts of LinkerConfiguration.WriteOutputForMSBuild into a FlushOutputForMSBuild
  method (the part that does the actual writing).
* Make WriteOutputForMSBuild just store the items in a dictionary.
* Add a DoneStep that runs at the very end and that writes out the MSBuild output
  items.

* [dotnet-linker] Link with GSS when building for iOS/Mac Catalyst.

Add a ComputeNativeBuildFlagsStep, which computes the flags to pass to the native
compiler + native linker. This is currently a very simple implementation, but it
will become more complex as support for missing features are added.

GSS is required because of libSystem.Net.Security.Native.a:

    Undefined symbols for architecture arm64:
      "___gss_c_nt_hostbased_service_oid_desc", referenced from:
          _NetSecurityNative_ImportPrincipalName in libSystem.Net.Security.Native.a(pal_gssapi.c.o)
      "___gss_c_nt_user_name_oid_desc", referenced from:
          _NetSecurityNative_ImportUserName in libSystem.Net.Security.Native.a(pal_gssapi.c.o)
      "___gss_krb5_cred_no_ci_flags_x_oid_desc", referenced from:
          _NetSecurityNative_InitiateCredSpNego in libSystem.Net.Security.Native.a(pal_gssapi.c.o)
          _NetSecurityNative_InitiateCredWithPassword in libSystem.Net.Security.Native.a(pal_gssapi.c.o)
      "___gss_krb5_mechanism_oid_desc", referenced from:
          _NetSecurityNative_InitSecContext in libSystem.Net.Security.Native.a(pal_gssapi.c.o)
          _NetSecurityNative_InitSecContextEx in libSystem.Net.Security.Native.a(pal_gssapi.c.o)
      "___gss_ntlm_mechanism_oid_desc", referenced from:
          _NetSecurityNative_InitSecContext in libSystem.Net.Security.Native.a(pal_gssapi.c.o)
          _NetSecurityNative_InitSecContextEx in libSystem.Net.Security.Native.a(pal_gssapi.c.o)
          _NetSecurityNative_AcceptSecContext in libSystem.Net.Security.Native.a(pal_gssapi.c.o)
          _NetSecurityNative_IsNtlmInstalled in libSystem.Net.Security.Native.a(pal_gssapi.c.o)
      "___gss_spnego_mechanism_oid_desc", referenced from:
          _NetSecurityNative_InitSecContext in libSystem.Net.Security.Native.a(pal_gssapi.c.o)
          _NetSecurityNative_InitSecContextEx in libSystem.Net.Security.Native.a(pal_gssapi.c.o)
          l___const.NetSecurityNative_AcquireCredSpNego.gss_mech_spnego_OID_set_desc in libSystem.Net.Security.Native.a(pal_gssapi.c.o)
      "_gss_accept_sec_context", referenced from:
          _NetSecurityNative_AcceptSecContext in libSystem.Net.Security.Native.a(pal_gssapi.c.o)
      "_gss_acquire_cred", referenced from:
          _NetSecurityNative_InitiateCredSpNego in libSystem.Net.Security.Native.a(pal_gssapi.c.o)
          _NetSecurityNative_AcquireAcceptorCred in libSystem.Net.Security.Native.a(pal_gssapi.c.o)
      "_gss_acquire_cred_with_password", referenced from:
          _NetSecurityNative_InitiateCredWithPassword in libSystem.Net.Security.Native.a(pal_gssapi.c.o)
      "_gss_delete_sec_context", referenced from:
          _NetSecurityNative_DeleteSecContext in libSystem.Net.Security.Native.a(pal_gssapi.c.o)
      "_gss_display_name", referenced from:
          _NetSecurityNative_GetUser in libSystem.Net.Security.Native.a(pal_gssapi.c.o)
      "_gss_display_status", referenced from:
          _NetSecurityNative_DisplayMinorStatus in libSystem.Net.Security.Native.a(pal_gssapi.c.o)
          _NetSecurityNative_DisplayMajorStatus in libSystem.Net.Security.Native.a(pal_gssapi.c.o)
      "_gss_import_name", referenced from:
          _NetSecurityNative_ImportUserName in libSystem.Net.Security.Native.a(pal_gssapi.c.o)
          _NetSecurityNative_ImportPrincipalName in libSystem.Net.Security.Native.a(pal_gssapi.c.o)
      "_gss_indicate_mechs", referenced from:
          _NetSecurityNative_IsNtlmInstalled in libSystem.Net.Security.Native.a(pal_gssapi.c.o)
      "_gss_init_sec_context", referenced from:
          _NetSecurityNative_InitSecContext in libSystem.Net.Security.Native.a(pal_gssapi.c.o)
          _NetSecurityNative_InitSecContextEx in libSystem.Net.Security.Native.a(pal_gssapi.c.o)
      "_gss_inquire_context", referenced from:
          _NetSecurityNative_GetUser in libSystem.Net.Security.Native.a(pal_gssapi.c.o)
      "_gss_oid_equal", referenced from:
          _NetSecurityNative_InitSecContext in libSystem.Net.Security.Native.a(pal_gssapi.c.o)
          _NetSecurityNative_InitSecContextEx in libSystem.Net.Security.Native.a(pal_gssapi.c.o)
          _NetSecurityNative_AcceptSecContext in libSystem.Net.Security.Native.a(pal_gssapi.c.o)
      "_gss_release_buffer", referenced from:
          _NetSecurityNative_ReleaseGssBuffer in libSystem.Net.Security.Native.a(pal_gssapi.c.o)
      "_gss_release_cred", referenced from:
          _NetSecurityNative_ReleaseCred in libSystem.Net.Security.Native.a(pal_gssapi.c.o)
      "_gss_release_name", referenced from:
          _NetSecurityNative_GetUser in libSystem.Net.Security.Native.a(pal_gssapi.c.o)
          _NetSecurityNative_ReleaseName in libSystem.Net.Security.Native.a(pal_gssapi.c.o)
      "_gss_release_oid_set", referenced from:
          _NetSecurityNative_IsNtlmInstalled in libSystem.Net.Security.Native.a(pal_gssapi.c.o)
      "_gss_set_cred_option", referenced from:
          _NetSecurityNative_InitiateCredSpNego in libSystem.Net.Security.Native.a(pal_gssapi.c.o)
          _NetSecurityNative_InitiateCredWithPassword in libSystem.Net.Security.Native.a(pal_gssapi.c.o)
      "_gss_unwrap", referenced from:
          _NetSecurityNative_Unwrap in libSystem.Net.Security.Native.a(pal_gssapi.c.o)
      "_gss_wrap", referenced from:
          _NetSecurityNative_Wrap in libSystem.Net.Security.Native.a(pal_gssapi.c.o)
    ld: symbol(s) not found for architecture arm64
2021-01-14 07:50:49 +01:00
Rolf Bjarne Kvinge 4181593b15
[msbuild] Merge the iOS and Mac versions of the DetectSigningIdentity task. (#10407)
Mac Catalyst needs much of the Mac signing logic, so merge the iOS and Mac
versions of DetectSigningIdentity so that the Xamarin.iOS.Tasks assembly
contains what it needs for Mac Catalyst.
2021-01-14 07:49:30 +01:00
Manuel de la Pena 9d1ce6130b
[CI][VSTS] Clean .xip files. (#10398)
At some point either due to a manual error or a bug in provisionator a
*.xip file was left behind. Those files make provisioantor fail, for
example: https://devdiv.visualstudio.com/DevDiv/_build/results?buildId=4361631&view=logs&j=f0137cdf-e292-5541-69aa-72303a08a0ba&t=59bea1c0-f907-5818-000e-f6fef9d3ffa2&l=230

We will remove those files to make sure provisionator works and we get
device tests back.


Co-authored-by: Rolf Bjarne Kvinge <rolf@xamarin.com>
2021-01-13 16:12:53 -05:00
Mike Bond 39409c6216
YAML build: Dump environment steps: Include HOSTNAME (#10397) 2021-01-13 09:47:58 -08:00
Rolf Bjarne Kvinge 92ce9972c4
Bump system mono to the latest available 2020-02 package. (#10402)
This makes xamarin-macios build on Apple Silicon, and also seems to get an
updated csc that fixes a problem with nullability warnings/errors.
2021-01-13 14:45:06 +01:00
Sebastien Pouliot d9b0097c3b
[mtouch] Fix embedding Mono (and Xamarin) as frameworks. Fixes #10382 (#10400)
This broke with 1582bf47cc and cause the
frameworks to be nested under another `Frameworks` directory.

See https://github.com/xamarin/xamarin-macios/issues/10382
2021-01-13 08:22:55 -05:00
Rolf Bjarne Kvinge a54f948011
[msbuild] Unify handling of Sdks. (#10375)
* [msbuild] Unify handling of Sdks.

Create an Sdks class in Xamarin.MacDev.Tasks.Core, which handles both Xamarin.iOS
and Xamarin.Mac. Refactor the MacOSXSdks and IPhoneSdks classes to use the new Sdks
class.

This makes it possible to avoid some code duplication in MacOSXSdks and IPhoneSdks,
and also share code elsewhere.

This requires a bump of Xamarin.MacDev.

New commits in xamarin/Xamarin.MacDev:

* xamarin/Xamarin.MacDev@fae0237 [Xamarin.MacDev] Add GetAppleDTSettings and GetSdkSettings to the IAppleSdk interface. (#85)

Diff: f665e3a0fc..fae0237704

* Translate exception message.

* Simplify according to review.

* Update list of non-translated strings.
2021-01-13 11:44:11 +01:00
Rolf Bjarne Kvinge c89b0fac64
[src] Build and ship a very simple implementation-only version of OpenTK-1.0.dll for Mac Catalyst. (#10387)
This version of OpenTK-1.0.dll only contains type forwarders for
System.Drawing.Color and System.Drawing.KnownColor. Eventually it may also
contain stubs that throw PlatformNotSupportedExceptions.

The main problem we're trying to solve is that we want to make the version of
Xamarin.Essentials built for Xamarin.iOS work in Mac Catalyst, and it
references OpenTK-1.0.dll, which means that we'll show an error at build time
if we can't find OpenTK-1.0.dll. This way there will be an OpenTK-1.0.dll, but
it's only available as an implementation assembly, which means that it's not
possible to reference it directly from a Mac Catalyst project.
2021-01-13 07:39:48 +01:00
Rolf Bjarne Kvinge 09ce925df2
[UIKit] Add the same UIBarItem.SetTitleTextAttributes to Mac Catalyst as iOS has. (#10386)
Mac Catalyst will be able to reference assemblies built for Xamarin.iOS, but
without any guarantees that it will actually work.

However, we'll want to make basic scenarios work, and with this change we're
adding an overload of UIBarItem.SetTitleTextAttributes that would not normally
exist in Mac Catalyst, so that existing code built for Xamarin.iOS works just
fine at runtime (in particular Xamarin.Forms runs into this).
2021-01-13 07:34:31 +01:00
Sebastien Pouliot 734b8a7f2a
[msbuild] Simplify resolving xcframeworks (#10376)
TD&LR: This PR simplifies how we refer to user frameworks and fixes both
warnings and non-optimal (app) output.

Much longer story:

Additional testing on macOS showed some build-time warnings and an
[extra (dupe) file](a20f8aba41 (diff-54fd7d9cd5deae57f30195be0a43133eace03c1132401741a317e0ae8d5e13fdR34)).

Logs shows that we referred to the xcframework several times, where once
should have been enough.

```
/native-reference:/Users/poupou/git/spouliot/xcframework/Universal.xcframework
/native-reference:/Users/poupou/git/spouliot/xcframework/Universal.xcframework/macos-arm64_x86_64/Universal.framework
/native-reference:/Users/poupou/git/spouliot/xcframework/Universal.xcframework/macos-arm64_x86_64/Universal.framework/Universal
```

The first `/native-reference` line produced a warning like:

```
MMP warning MM2006: Native library 'Universal.xcframework' was referenced but could not be found.
```

which makes sense as the tools (both `mmp` and `mtouch`) are not, by
design, aware of (unresolved) xcframeworks.

Removing `{NativeReference}` from `Xamarin.Mac.Common.targets` (and
`Xamarin.iOS.Common.targets`) as it has already been processed by
`_ExpandNativeReferences` solves this.

The other part of the issue (next two lines) is because `msbuild` does
not track changes to directories like it does for files - and the
workaround (in `_ExpandNativeReferences`) had to be copied in other
places (both XI and XM `_CompileToNative`) and that was not enough (and
would eventually need to be duplicated again and again).

This could lead to duplicate entries (i msbuild logs) like

```
NativeReferences
../../Universal.xcframework/macos-arm64_x86_64/Universal.framework
../../Universal.xcframework/macos-arm64_x86_64/Universal.framework/Univeral
```
which maps to our extra entries.

In order to simplify things we make the `_ExpandNativeReferences` resolve
the full path to the library name (not the `.framework` directory) which
simplifies both `_CompileToNative` and ensure a single way (at least for
`msbuild`) to provide this data to the tools (`mmp` and `mtouch`).

Using a file, instead of a directory, is also more consistent for the
existing `-framework` option, e.g. we provide the names like:

```
--framework=CoreLocation
--framework=ModelIO
```

So adding a full path that include the name is more appropriate, e.g.

``` --framework=/Users/poupou/git/master/xamarin-macios/tests/xharness/tmp-test-dir/xcframework-test760/bin/AnyCPU/Debug/bindings-xcframework-test.resources/XTest.xcframework/ios-i386_x86_64-simulator/XTest.framework/XTest
```

Finally for macOS applications it turns out we were embedding yet another
copy of the framework's library inside the `MonoBundle`, which is clearly
wrong, because of the last entry.

```
$ l bin/Release/xcf-mac.app/Contents/MonoBundle/Universal
-rwxr-xr-x  1 poupou  staff  167152  2 Dec 16:16 bin/Release/xcf-mac.app/Contents/MonoBundle/Universal
```

The tool now checks if a provided library is inside a framework (or not)
which is a good validation to have anyway when it gets called directly,
i.e. not thru `msbuild`.
2021-01-12 16:02:01 -05:00
Manuel de la Pena 8f25850c20
[CI][VSTS] Add required profiles for mac. (#10366)
fixes: xamarin/maccore#2357
2021-01-12 15:29:49 -05:00
Manuel de la Pena ec38f76ab1
[CI][VSTS] Add links to the created pkgs. (#10360)
Add a list with all the pkgs created to make users life better.

fixes: https://github.com/xamarin/xamarin-macios/issues/10298
2021-01-12 15:20:54 -05:00
Manuel de la Pena bd76b393f2
[CI][VSTS] Add a better title for the comments. (#10383)
Remove the misleading title 'Device tests..' for 'Tests..'. The message
already contains the context of the test execution, it can be:

* Build
* VSTS: device tests iOS32b
* VSTS: device tests tvOS
* VSTS: device tests iOS

Removing the 'Device' word is enough to do not confuse users.

fixes: https://github.com/xamarin/maccore/issues/2358
2021-01-12 12:33:23 -05:00
Manuel de la Pena 3e7464e461
[CI][VSTS] Rename path to fix API diff (#10384)
Since 'packages' is a common dir name that is ignored. Revert the change
in the .gitignore and rename the template path since it just means a one
liner change in the entry yaml file.

fixes: https://github.com/xamarin/maccore/issues/2359
2021-01-12 11:32:26 -05:00
Rolf Bjarne Kvinge 8e2915a773
[src] Use PlatformName.MacCatalyst instead of PlatformName.UIKitForMac. (#10290)
* [src] Use PlatformName.MacCatalyst instead of PlatformName.UIKitForMac.

And remove PlatformName.UIKitForMac.

* Fix breaking change.
2021-01-12 09:56:09 +01:00
Rolf Bjarne Kvinge 08cf9653df
[xtro] Make it possible to override the sharpie executable. (#10373)
Use a variable for the sharpie executable, so that it's easier to override
while doing testing.
2021-01-12 09:55:23 +01:00
Manuel de la Pena 6a6f1f1ff8
[Foundation] Break out of the delegate when we have canceled a task. Fixes #9132 (#10230)
This is an interesting issue that happens in the following scenario:

1. A request is performed that will take some extra time to get a
   response or an error.
2. The request is cancelled BEFORE the delegate methods
   DidReceiveResponse, DidReceiveData or DidCompleteWithError and
   called. Yet we have not yet fully canceled the native request. It
   does take some time to do so, that is why we have a
   'NSURLSessionTaskStateCanceling' and not 'NSURLSessionTaskStateCancelled'

The issue happens because the GetInflightData checks if the task has
been canceled. In that method we have the following:

```csharp
if (inflight.CancellationToken.IsCancellationRequested)
  task?.Cancel ();
return inflight;
```

The call of the Cancel method in the NSData task has as ripple effect which
is the execution of the following callback:

```csharp
cancellationToken.Register (() => {
  RemoveInflightData (dataTask);
  tcs.TrySetCanceled ();
});
```

Which calls:

```csharp
void RemoveInflightData (NSUrlSessionTask task, bool cancel = true)
{
      lock (inflightRequestsLock) {
              if (inflightRequests.TryGetValue (task, out var data)) {
                      if (cancel)
                              data.CancellationTokenSource.Cancel ();
                      data.Dispose ();
                      inflightRequests.Remove (task);
              }
              // do we need to be notified? If we have not inflightData, we do not
              if (inflightRequests.Count == 0)
                      RemoveNotification ();
      }

      if (cancel)
              task?.Cancel ();

      task?.Dispose ();
}
```

The above call does call Dispose and Dipose in the inflight data does:
```csharp
if (disposing) {
    if (CancellationTokenSource != null) {
            CancellationTokenSource.Dispose ();
            CancellationTokenSource = null;
    }
}
```

If we follow these set of events for example in the
DidRecieveResponse:
```chsarp
[Preserve (Conditional = true)]
public override void DidReceiveResponse (NSUrlSession session, NSUrlSessionDataTask dataTask, NSUrlResponse response, Action<NSUrlSessionResponseDisposition> completionHandler)
{
    var inflight = GetInflightData (dataTask);

    if (inflight == null)
            return;

    try {
            var urlResponse = (NSHttpUrlResponse)response;
            var status = (int)urlResponse.StatusCode;

            var content = new NSUrlSessionDataTaskStreamContent (inflight.Stream, () => {
                    if (!inflight.Completed) {
                            dataTask.Cancel ();
                    }

                    inflight.Disposed = true;
                    inflight.Stream.TrySetException (new ObjectDisposedException ("The content stream was disposed."));

                    sessionHandler.RemoveInflightData (dataTask);
            }, inflight.CancellationTokenSource.Token);
```
If can happen that, when we get the delegate call to the method, the
request has been cancelled. That means that we are in the precise state
in which we do not longer care about the response BUT we did get a
infligh data reference.. that HAS BEEN DIPOSED!!! this later gives a NRE
in several places.

The correct way is to return null from the GetInflighData method if we
have been cancelled, this makes sense because if we cancelled we do not
care about the execution of any of the methods that required the data
since it has been disposed!


Co-authored-by: Rolf Bjarne Kvinge <rolf@xamarin.com>
2021-01-11 17:29:51 -05:00
Sebastien Pouliot 0709c8882f
[msbuild] Always codesign the framework directory, not what's inside (#10309)
**Example #1.** Signing a framework binary is the **same** thing as
signing the framework directory.

```
$ codesign -v --force --timestamp=none --sign - bin/iPhone/Release/xcf_ios.app//Frameworks/lame.framework/lame
bin/iPhone/Release/xcf_ios.app//Frameworks/lame.framework/lame: replacing existing signature
bin/iPhone/Release/xcf_ios.app//Frameworks/lame.framework/lame: signed bundle with Mach-O thin (arm64) [io.sourceforge.lame]

$ codesign -v --force --timestamp=none --sign - bin/iPhone/Release/xcf_ios.app//Frameworks/lame.framework
bin/iPhone/Release/xcf_ios.app//Frameworks/lame.framework: replacing existing signature
bin/iPhone/Release/xcf_ios.app//Frameworks/lame.framework: signed bundle with Mach-O thin (arm64) [io.sourceforge.lame]
```

Nice right ? Pretty much until...

**Example #2.** Signing a framework binary is **NOT** the **same** thing
as signing the framework directory.

```
$ codesign -v --force --timestamp=none --sign - bin/iPhone/Release/xcf_ios.app//Frameworks/flac.framework/flac
bin/iPhone/Release/xcf_ios.app//Frameworks/flac.framework/flac: replacing existing signature
bin/iPhone/Release/xcf_ios.app//Frameworks/flac.framework/flac: signed Mach-O thin (arm64) [flac-55554944583d2f02282c33d8bfed082daa857e30]

$ codesign -v --force --timestamp=none --sign - bin/iPhone/Release/xcf_ios.app//Frameworks/flac.framework
bin/iPhone/Release/xcf_ios.app//Frameworks/flac.framework: replacing existing signature
bin/iPhone/Release/xcf_ios.app//Frameworks/flac.framework: signed bundle with Mach-O thin (arm64) [org.xiph.flac]
```

In this case signing the binary `flac` does not produce the
`_CodeSignature` directory and fails our msbuild Codesign task

The fix is to detect if we're signing a framework like `A.framework/A`
and change this to sign `A.framework` as this will always work.
2021-01-11 16:05:35 -05:00
Rolf Bjarne Kvinge 28791b6e2a
[builds] Create the cache directory if needed. (#10372) 2021-01-11 16:25:56 +01:00
Rolf Bjarne Kvinge 0132667419
[Security] Fix the underlying integral type for SslCipherSuite for Mac Catalyst. (#10321)
This difference manifested as a memory corruption (because we were passing an array
of the enum to native code, and the array was half the size it should be, so native
code write into random memory).
2021-01-11 16:11:38 +01:00
Rolf Bjarne Kvinge 4730ef30b8
[msbuild] Unify the EmbedProvisionProfile and EmbedMobileProvision tasks between iOS and macOS. (#10322)
These two tasks did essentially the same thing, so we can just merge them. I
kept the "EmbedProvisionProfile" name, because that sounded like the most
applicable to all platforms.
2021-01-11 14:34:19 +01:00
Sebastien Pouliot 6d143763c3
[msbuild] Use the existing user framework dSYM when available (#10304)
User frameworks comes in different "shapes"

* You can roll your own (like ours are) and they are likely to be similar
  to dynamic libraries. Of interest is the fact that the native binary
  will include debugging symbols. On which our tools will, at a later
  point, run `dsymutil` to create a `dSYM` and `strip` the binary.

* You can create them with a tool, like Xcode, and the build's output
  will give you a `dSYM` and a stripped (no debug information) binary.

Now we can't process the later like the former. Because if we execute
`dsymutil` on the **stripped** framework we'll get an _empty_ (no symbol)
`.dSYM`. The tool will issue a warning like:

> warning: no debug symbols in executable (-arch arm64)

which is easy to miss since it's informational (now promoted to a proper
msbuild warning).

The result is that the archive will have our new, but _empty_ .dSYM,
which won't be of any help to symbolicate crashes later.

It's also quite inefficient since
* we run `dsymutil` way too often (to produce nothing useful)
* we run `strip` on the, already stripped, framework binary

The solution is to check if the user framework comes with a `.dSYM`.
If it does then this is the one we should bundle in the archive and we
can skip the `dsymutil` and `strip` steps.

If it does not have one (dSYM) then the current logic is the best we can
do, i.e. assume there might be debug information inside the binary,
extract it (`dsymutil`) and remove it (`strip`) from shipping code.
2021-01-11 08:32:56 -05:00
Sebastien Pouliot a6866b4864
[msbuild] Remove the old `--rsrc` argument when using `ditto` (#10303)
`--rsrc` has been the default since macOS 10.4
2021-01-11 08:23:35 -05:00
Rolf Bjarne Kvinge 22d65a560d
[tests] Add xtro support for Mac Catalyst. Fixes #10214. (#10289)
* [xtro] Add support for Mac Catalyst. Fixes #10214.

Fixes https://github.com/xamarin/xamarin-macios/issues/10214.

* Update xtro.

* Bump Objective-Sharpie.

* Delete .todo files for frameworks we don't support.

* And another one bites the dust.

* [xtro] Update.
2021-01-11 11:56:58 +01:00
Rolf Bjarne Kvinge fa65fc4bcf
[versions-check.csharp] Improve this script a bit (#10323)
* Pass -s to csharp to tell it the rest of the arguments are the script and its arguments.

    Fixes these warnings printed to the terminal:

        warning CS2002: Source file `7.0' specified multiple times
        warning CS2002: Source file `10.15' specified multiple times

* Add an exception handler and return 1 if there are any exceptions.
2021-01-11 10:47:57 +01:00
Manuel de la Pena 031e14f1a3
[Actions] Add the user to all branches. Remove monojenkins. (#10368) 2021-01-08 16:21:41 -05:00
Přemek Vysoký 1c34588487
[xharness] Bump the shared library version
This is mainly to bring in this fix: dotnet/xharness@841114a
(two parameters were swapped and BundleIdentifier was set to AppPath)

Other changes contained:

We don't log the full XML when listing devices anymore, just the file size
SimulatorLoader has a new method that accepts retryCount
2021-01-08 15:47:56 -05:00
Manuel de la Pena e2db8536f4
[CI][VSTS] Add trigger for xcode branches. (#10349)
fixes: https://github.com/xamarin/maccore/issues/2355
2021-01-07 17:30:50 -05:00
Manuel de la Pena f218681d7b
[Actions] Ensure the new bot is considered. (#10352)
@monojenkins is no more.
2021-01-07 15:41:00 -05:00
Sebastien Pouliot 89482b5d5b
[macos] Handle symlinks in user frameworks (#10281)
User frameworks for macOS often uses symlinks (as Xcode creates them
this way).

This cause problem cause the symlink is on the binary and we expected
the `_CodeSignature` directory to by side-by-side with the binary. This
was missing and cause exceptions when codesigning such frameworks.

A second problem happened because `mmp` use `lipo -thin` to remove
non-required architectures. However when a framework has symlinks, like:

```
├── Frameworks
│   └── Universal.framework
│       ├── Resources -> Versions/Current/Resources
│       ├── Universal -> Versions/Current/Universal
│       └── Versions
│           ├── A
│           │   ├── Resources
│           │   │   └── Info.plist
│           │   ├── Universal
│           │   └── _CodeSignature
│           │       └── CodeResources
│           └── Current -> A
```

then this actually replaced the (very small) symlink with a thin version
of the framework. Which means the original one was still _fat_ and the
whole app was now larger than the original version.

Sample used: https://github.com/spouliot/xcframework/tree/main/xamarin/xcf-mac
2021-01-07 11:20:31 -05:00