Mac Catalyst will be able to reference assemblies built for Xamarin.iOS, but
without any guarantees that it will actually work.
However, we'll want to make basic scenarios work, and with this change we're
adding an overload of UIBarItem.SetTitleTextAttributes that would not normally
exist in Mac Catalyst, so that existing code built for Xamarin.iOS works just
fine at runtime (in particular Xamarin.Forms runs into this).
TD&LR: This PR simplifies how we refer to user frameworks and fixes both
warnings and non-optimal (app) output.
Much longer story:
Additional testing on macOS showed some build-time warnings and an
[extra (dupe) file](a20f8aba41 (diff-54fd7d9cd5deae57f30195be0a43133eace03c1132401741a317e0ae8d5e13fdR34)).
Logs shows that we referred to the xcframework several times, where once
should have been enough.
```
/native-reference:/Users/poupou/git/spouliot/xcframework/Universal.xcframework
/native-reference:/Users/poupou/git/spouliot/xcframework/Universal.xcframework/macos-arm64_x86_64/Universal.framework
/native-reference:/Users/poupou/git/spouliot/xcframework/Universal.xcframework/macos-arm64_x86_64/Universal.framework/Universal
```
The first `/native-reference` line produced a warning like:
```
MMP warning MM2006: Native library 'Universal.xcframework' was referenced but could not be found.
```
which makes sense as the tools (both `mmp` and `mtouch`) are not, by
design, aware of (unresolved) xcframeworks.
Removing `{NativeReference}` from `Xamarin.Mac.Common.targets` (and
`Xamarin.iOS.Common.targets`) as it has already been processed by
`_ExpandNativeReferences` solves this.
The other part of the issue (next two lines) is because `msbuild` does
not track changes to directories like it does for files - and the
workaround (in `_ExpandNativeReferences`) had to be copied in other
places (both XI and XM `_CompileToNative`) and that was not enough (and
would eventually need to be duplicated again and again).
This could lead to duplicate entries (i msbuild logs) like
```
NativeReferences
../../Universal.xcframework/macos-arm64_x86_64/Universal.framework
../../Universal.xcframework/macos-arm64_x86_64/Universal.framework/Univeral
```
which maps to our extra entries.
In order to simplify things we make the `_ExpandNativeReferences` resolve
the full path to the library name (not the `.framework` directory) which
simplifies both `_CompileToNative` and ensure a single way (at least for
`msbuild`) to provide this data to the tools (`mmp` and `mtouch`).
Using a file, instead of a directory, is also more consistent for the
existing `-framework` option, e.g. we provide the names like:
```
--framework=CoreLocation
--framework=ModelIO
```
So adding a full path that include the name is more appropriate, e.g.
``` --framework=/Users/poupou/git/master/xamarin-macios/tests/xharness/tmp-test-dir/xcframework-test760/bin/AnyCPU/Debug/bindings-xcframework-test.resources/XTest.xcframework/ios-i386_x86_64-simulator/XTest.framework/XTest
```
Finally for macOS applications it turns out we were embedding yet another
copy of the framework's library inside the `MonoBundle`, which is clearly
wrong, because of the last entry.
```
$ l bin/Release/xcf-mac.app/Contents/MonoBundle/Universal
-rwxr-xr-x 1 poupou staff 167152 2 Dec 16:16 bin/Release/xcf-mac.app/Contents/MonoBundle/Universal
```
The tool now checks if a provided library is inside a framework (or not)
which is a good validation to have anyway when it gets called directly,
i.e. not thru `msbuild`.
Remove the misleading title 'Device tests..' for 'Tests..'. The message
already contains the context of the test execution, it can be:
* Build
* VSTS: device tests iOS32b
* VSTS: device tests tvOS
* VSTS: device tests iOS
Removing the 'Device' word is enough to do not confuse users.
fixes: https://github.com/xamarin/maccore/issues/2358
Since 'packages' is a common dir name that is ignored. Revert the change
in the .gitignore and rename the template path since it just means a one
liner change in the entry yaml file.
fixes: https://github.com/xamarin/maccore/issues/2359
This is an interesting issue that happens in the following scenario:
1. A request is performed that will take some extra time to get a
response or an error.
2. The request is cancelled BEFORE the delegate methods
DidReceiveResponse, DidReceiveData or DidCompleteWithError and
called. Yet we have not yet fully canceled the native request. It
does take some time to do so, that is why we have a
'NSURLSessionTaskStateCanceling' and not 'NSURLSessionTaskStateCancelled'
The issue happens because the GetInflightData checks if the task has
been canceled. In that method we have the following:
```csharp
if (inflight.CancellationToken.IsCancellationRequested)
task?.Cancel ();
return inflight;
```
The call of the Cancel method in the NSData task has as ripple effect which
is the execution of the following callback:
```csharp
cancellationToken.Register (() => {
RemoveInflightData (dataTask);
tcs.TrySetCanceled ();
});
```
Which calls:
```csharp
void RemoveInflightData (NSUrlSessionTask task, bool cancel = true)
{
lock (inflightRequestsLock) {
if (inflightRequests.TryGetValue (task, out var data)) {
if (cancel)
data.CancellationTokenSource.Cancel ();
data.Dispose ();
inflightRequests.Remove (task);
}
// do we need to be notified? If we have not inflightData, we do not
if (inflightRequests.Count == 0)
RemoveNotification ();
}
if (cancel)
task?.Cancel ();
task?.Dispose ();
}
```
The above call does call Dispose and Dipose in the inflight data does:
```csharp
if (disposing) {
if (CancellationTokenSource != null) {
CancellationTokenSource.Dispose ();
CancellationTokenSource = null;
}
}
```
If we follow these set of events for example in the
DidRecieveResponse:
```chsarp
[Preserve (Conditional = true)]
public override void DidReceiveResponse (NSUrlSession session, NSUrlSessionDataTask dataTask, NSUrlResponse response, Action<NSUrlSessionResponseDisposition> completionHandler)
{
var inflight = GetInflightData (dataTask);
if (inflight == null)
return;
try {
var urlResponse = (NSHttpUrlResponse)response;
var status = (int)urlResponse.StatusCode;
var content = new NSUrlSessionDataTaskStreamContent (inflight.Stream, () => {
if (!inflight.Completed) {
dataTask.Cancel ();
}
inflight.Disposed = true;
inflight.Stream.TrySetException (new ObjectDisposedException ("The content stream was disposed."));
sessionHandler.RemoveInflightData (dataTask);
}, inflight.CancellationTokenSource.Token);
```
If can happen that, when we get the delegate call to the method, the
request has been cancelled. That means that we are in the precise state
in which we do not longer care about the response BUT we did get a
infligh data reference.. that HAS BEEN DIPOSED!!! this later gives a NRE
in several places.
The correct way is to return null from the GetInflighData method if we
have been cancelled, this makes sense because if we cancelled we do not
care about the execution of any of the methods that required the data
since it has been disposed!
Co-authored-by: Rolf Bjarne Kvinge <rolf@xamarin.com>
**Example #1.** Signing a framework binary is the **same** thing as
signing the framework directory.
```
$ codesign -v --force --timestamp=none --sign - bin/iPhone/Release/xcf_ios.app//Frameworks/lame.framework/lame
bin/iPhone/Release/xcf_ios.app//Frameworks/lame.framework/lame: replacing existing signature
bin/iPhone/Release/xcf_ios.app//Frameworks/lame.framework/lame: signed bundle with Mach-O thin (arm64) [io.sourceforge.lame]
$ codesign -v --force --timestamp=none --sign - bin/iPhone/Release/xcf_ios.app//Frameworks/lame.framework
bin/iPhone/Release/xcf_ios.app//Frameworks/lame.framework: replacing existing signature
bin/iPhone/Release/xcf_ios.app//Frameworks/lame.framework: signed bundle with Mach-O thin (arm64) [io.sourceforge.lame]
```
Nice right ? Pretty much until...
**Example #2.** Signing a framework binary is **NOT** the **same** thing
as signing the framework directory.
```
$ codesign -v --force --timestamp=none --sign - bin/iPhone/Release/xcf_ios.app//Frameworks/flac.framework/flac
bin/iPhone/Release/xcf_ios.app//Frameworks/flac.framework/flac: replacing existing signature
bin/iPhone/Release/xcf_ios.app//Frameworks/flac.framework/flac: signed Mach-O thin (arm64) [flac-55554944583d2f02282c33d8bfed082daa857e30]
$ codesign -v --force --timestamp=none --sign - bin/iPhone/Release/xcf_ios.app//Frameworks/flac.framework
bin/iPhone/Release/xcf_ios.app//Frameworks/flac.framework: replacing existing signature
bin/iPhone/Release/xcf_ios.app//Frameworks/flac.framework: signed bundle with Mach-O thin (arm64) [org.xiph.flac]
```
In this case signing the binary `flac` does not produce the
`_CodeSignature` directory and fails our msbuild Codesign task
The fix is to detect if we're signing a framework like `A.framework/A`
and change this to sign `A.framework` as this will always work.
This difference manifested as a memory corruption (because we were passing an array
of the enum to native code, and the array was half the size it should be, so native
code write into random memory).
These two tasks did essentially the same thing, so we can just merge them. I
kept the "EmbedProvisionProfile" name, because that sounded like the most
applicable to all platforms.
User frameworks comes in different "shapes"
* You can roll your own (like ours are) and they are likely to be similar
to dynamic libraries. Of interest is the fact that the native binary
will include debugging symbols. On which our tools will, at a later
point, run `dsymutil` to create a `dSYM` and `strip` the binary.
* You can create them with a tool, like Xcode, and the build's output
will give you a `dSYM` and a stripped (no debug information) binary.
Now we can't process the later like the former. Because if we execute
`dsymutil` on the **stripped** framework we'll get an _empty_ (no symbol)
`.dSYM`. The tool will issue a warning like:
> warning: no debug symbols in executable (-arch arm64)
which is easy to miss since it's informational (now promoted to a proper
msbuild warning).
The result is that the archive will have our new, but _empty_ .dSYM,
which won't be of any help to symbolicate crashes later.
It's also quite inefficient since
* we run `dsymutil` way too often (to produce nothing useful)
* we run `strip` on the, already stripped, framework binary
The solution is to check if the user framework comes with a `.dSYM`.
If it does then this is the one we should bundle in the archive and we
can skip the `dsymutil` and `strip` steps.
If it does not have one (dSYM) then the current logic is the best we can
do, i.e. assume there might be debug information inside the binary,
extract it (`dsymutil`) and remove it (`strip`) from shipping code.
* [xtro] Add support for Mac Catalyst. Fixes #10214.
Fixes https://github.com/xamarin/xamarin-macios/issues/10214.
* Update xtro.
* Bump Objective-Sharpie.
* Delete .todo files for frameworks we don't support.
* And another one bites the dust.
* [xtro] Update.
* Pass -s to csharp to tell it the rest of the arguments are the script and its arguments.
Fixes these warnings printed to the terminal:
warning CS2002: Source file `7.0' specified multiple times
warning CS2002: Source file `10.15' specified multiple times
* Add an exception handler and return 1 if there are any exceptions.
This is mainly to bring in this fix: dotnet/xharness@841114a
(two parameters were swapped and BundleIdentifier was set to AppPath)
Other changes contained:
We don't log the full XML when listing devices anymore, just the file size
SimulatorLoader has a new method that accepts retryCount
User frameworks for macOS often uses symlinks (as Xcode creates them
this way).
This cause problem cause the symlink is on the binary and we expected
the `_CodeSignature` directory to by side-by-side with the binary. This
was missing and cause exceptions when codesigning such frameworks.
A second problem happened because `mmp` use `lipo -thin` to remove
non-required architectures. However when a framework has symlinks, like:
```
├── Frameworks
│ └── Universal.framework
│ ├── Resources -> Versions/Current/Resources
│ ├── Universal -> Versions/Current/Universal
│ └── Versions
│ ├── A
│ │ ├── Resources
│ │ │ └── Info.plist
│ │ ├── Universal
│ │ └── _CodeSignature
│ │ └── CodeResources
│ └── Current -> A
```
then this actually replaced the (very small) symlink with a thin version
of the framework. Which means the original one was still _fat_ and the
whole app was now larger than the original version.
Sample used: https://github.com/spouliot/xcframework/tree/main/xamarin/xcf-mac
The GitHup url to be used to create comments in PRs is diff to the one
for comments. Use the build reason AND the changeID to identify if we
are building due to a PR and use the correct url.
fixes: https://github.com/xamarin/maccore/issues/2356
Co-authored-by: Alex Soto <alex@alexsoto.me>
The prefix of the location of the logs depends on the platform where it
was executed (else we step on them) but that was not added in the env
var use by xharness.
fixes: https://github.com/xamarin/maccore/issues/2349
This shows up in a Xamarin.Forms app, which has a reference to
System.Drawing.RectangleF in Xamarin.iOS.dll, which is a type forwarder to
System.Drawing.Common.dll. That type forwarder must also exist in the
Xamarin.iOS.dll Facadate that Mac Catalyst uses.
* [apitest] Sanitize files by adding missing eols.
* [apitest] Add #if __MACOS__ to all test files.
In preparation for the move into monotouchtest.
* [apitest] Move test files into monotouchtest.
* [tests] Remove the apitest project.
* [monotouch-test] Remove MessagingMac.cs, it's not needed.
* [xammac-tests] Add file PlatformInfo.cs to the build.
* [xammac-tests] Move files into monotouch-test.
* [monotouch-test] Rename test class to not clash with another test class of the same (Objective-C) name.
* [tests] How did this ever work?
Answer: it never did.
* [monotouch-test] Remove duplicated test code.
* [xammac-tests] Define DYNAMIC_REGISTRAR when we're using the dynamic registrar.
* [monotouch-test] Adjust the BundleTest.TestGetBundleId test to cope with having multiple apps for the same bundle id.
* [monotouch-test] Ignore a test that doesn't work with the static registrar (due to a bug in the static registrar).
* [tests] Build test-libraries for Mac Catalyst.
* [msbuild] Add support for Mac Catalyst binding projects.
* [mtouch] Allow frameworks for Mac Catalyst apps.
* [mtouch] Put frameworks in the expected location for Mac Catalyst apps.
* [msbuild] Create the Resources directory before trying to put files in it.
* [src] Generate the string constants for each framework from our list of frameworks.
This makes it less error prone to add new frameworks (one less place to add
them), and easier to add new platforms (only one place to make sure the list
of frameworks is correct).
This also revelead a few issues in our list of frameworks (missing frameworks,
frameworks that have been moved, etc.), which have been fixed.
* [mmp] Generate the library path -> namespace map for the linker to optimize away dlopen statements.
* Fix indentation to be just as broken as the surrounding indentation.
* Remove unused Cecil reference.
* Fix c&p oversight.
* Adjust more Mac Catalyst API.
* CFNetwork is its own top-level framework.
As part of the effort to move away from jenkins, the @monojenkins bot will be retired. This adds an action to be used that does the same job.
The new action can be executed via '@vs-mobiletools-engineering-service2 rebase'
* [Actions] Add the backport action
As part of the effort to move away from jenkins, the @monojenkins bot will be retired. This adds an action to be used that does the same job.
New action can be executed via '/sudo backport $TARGET_BRANCH'
Rework the code to generate an API comparison and a generator comparison: we
now clone xamarin-macios into a different directory, checks out the hash we
want to compare against, build the whole thing, and then compare the results.
This is somewhat slower than the previous implementation (which would checkout
the previous hash and build that on top of the existing build), but since we
don't have to build mono, it shouldn't make too much of a difference (the
biggest chunk of our build is building src/, and we had to do that anyway with
the previous implementation as well).
The upside is that it should be much more reliable.
Fixes https://github.com/xamarin/maccore/issues/2034.
Fixes https://github.com/xamarin/maccore/issues/1467.
Fixes https://github.com/xamarin/maccore/issues/1059.
Maybe https://github.com/xamarin/maccore/issues/1173 too.
Rolf Bjarne Kvinge
Sebastien Pouliot
Manuel de la Pena
Přemek Vysoký
Alex Soto