Share all the code to find and verify Xcode between mtouch and mmp.
Also print out the actual product version when logging which Xcode was used
(to make it easier to detect if a beta version is in use).
- https://github.com/xamarin/xamarin-macios/pull/4980 and the mono branch merge both added it
- However both copies were not the same, one was conditional and one added an extra -u option
- This collapses them into one check
I'm not sure why we have an entry in our list of frameworks claiming that
iTunesLibrary was introduced in macOS 10.9, when we didn't have any bindings
for the library back then.
In any case we also have an entry for iTunesLibrary in our list of frameworks
claiming that iTunesLibrary was introduced in macOS 10.14.
I looked at some of Apple's documentation for the types in iTunesLibrary, and
they all claim to be introduced in macOS 10.13 [2].
And to make matters even more interesting, Apple's documentation for the
framework itself states the library is in
/Library/Frameworks/iTunesLibrary.framework, and ships with iTunes 11.0+ [1]
(which is introduced in 2012).
Then I looked at a macOS 10.14 machine, and the framework is available at
/System/Library/Frameworks/iTunesLibrary.framework, and
/Library/Frameworks/iTunesLibrary.framework is just a symlink there
(/System/Library/Frameworks/iTunesLibrary.framework does not exist on my macOS
10.13 machines, while /Library/Frameworks/iTunesLibrary.framework does). From
this I conclude that the framework was converted into a
system framework in macOS 10.14, and as such our claim that the framework was
introduced in 10.14 is at least somewhat right.
So treat iTunesLibrary as any other framework introduced in macOS 10.14, and
remove our (duplicated) framework entry for 10.9 (for which we didn't have any
bindings anyway).
Also fix the path to the framework, I'm wonder how this got past our tests in
the first place.
[1] https://developer.apple.com/documentation/ituneslibrary: "... located at /Library/Frameworks/iTunesLibrary.framework ... The iTunes Library framework is available to users running iTunes v11.0 or above."
[2] https://developer.apple.com/documentation/ituneslibrary/itlibrary?language=objc
Instead of a generic `MT0000` caused by an exception reading the magic
numbers of the binary framework file.
This was caused be uncompressing an archive, with a symlink, into a
file system that does not support symlinks (on Windows).
ref: https://github.com/xamarin/xamarin-macios/issues/5028
* Refactor type map to have a separate flag field for each type instead of magic location in the array.
Refactor the type map to have a separate flag field for each type instead of a
magic location in the array. This simplifies some code, but also allows us to
introduce more flags in the future (which becomes increasingly complex if the
location in the array is to determine yet another value).
This has neglible performance impacts.
* Add a UserType flag for registered types, and use it to improve the performance for is_user_type.
Reflection in the Objective-C runtime is apparently quite slow, so try to
avoid it by computing the information we need for determining whether a
particular Objective-C type represents a user type or not in the static
registrar.
We store this information in a flag for the type in question in the type map,
and use a binary search to search the type map when needed.
This provides a significant improvement, in particular in the dontlink
scenario (probably because there are many more Objective-C types in the app,
which made Objective-C reflection slow). In fact, it somehow made the dontlink
scenario so fast that it's faster than the linkall scenario (which also
improved, but not nearly as much). While quite inexplicable, it's a consistent
result I've seen over multiple test runs.
Numbers
=======
Test case: 004283d7b6
Fix 1 refers to PR #5009.
Fix 2 refers to PR #5013.
Fix 3 refers to PR #5016.
Fix 4 is this fix.
iPad Air 2
----------
| Configuration | Before | After fix 1 | After fix 2 | After fix 3 | After fix 4 | Improvement from fix 3 to fix 4 | Cumulative improvement |
| ------------------- | ------ | ----------: | -----------: | -----------: | -----------: | ------------------------------: | ---------------------: |
| Release (link all) | 477 ms | 481 ms | 224 ms | 172 ms | 148 ms | 24 ms (14%) | 329 ms (69%) |
| Release (dont link) | 738 ms | 656 ms | 377 ms | 201 ms | 146 ms | 55 ms (27%) | 592 ms (80%) |
iPhone X
--------
| Configuration | Before | After fix 1 | After fix 2 | After fix 3 | After fix 4 | Improvement from fix 3 to fix 4 | Cumulative improvement |
| ------------------- | ------ | ----------: | -----------: | -----------: | -----------: | ------------------------------: | ---------------------: |
| Release (link all) | 98 ms | 99 ms | 42 ms | 31 ms | 29 ms | 2 ms ( 6%) | 69 ms (70%) |
| Release (dont link) | 197 ms | 153 ms | 91 ms | 43 ms | 28 ms | 15 ms (35%) | 169 ms (86%) |
When linking all assemblies, the type map has 24 entries, and when not linking
at all it has 2993 entries.
This is part 4 (the last) of multiple fixes for #4936.
The total speed-up is 69-86% (3-7x faster).
Our type map has two sections: first come all the wrapper types, then all the
custom types. This means we need to sort these two sections separately, since
code elsewhere depends on this split in order to determine if a type is a
custom type or not.
We also need a minor modification in the array of skipped types, since it
contained indexes into the type map, which won't be valid after is has been
sorted. Instead store a type reference for the actual type in the array, and
use that to search the type map for the corresponding Class. This is a little
bit slower, but the results are cached in a dictionary, so it'll only happen
once for each type.
The performance is slightly slower when the type map has very few entries, but
that is repaid many times over when the number of entries go up.
Numbers
=======
Test case: 004283d7b6
iPad Air 2
----------
| Configuration | Before | After | Improvement |
| ------------------- | ------ | ------ | ------------: |
| Release (link all) | 477 ms | 481 ms | -4 ms (-0,8%) |
| Release (dont link) | 738 ms | 656 ms | 82 ms (11%) |
iPhone X
--------
| Configuration | Before | After | Improvement |
| ------------------- | ------ | ------ | ----------: |
| Release (link all) | 98 ms | 99 ms | -1 ms (-1%) |
| Release (dont link) | 197 ms | 153 ms | 44 ms (22%) |
When linking all assemblies, the type map has 24 entries, and when not linking
at all it has 2993 entries.
This is part 1 of multiple fixes for #4936.
Commit 996d90614b fixed the use of XML files. However it did not set the flags so if it's used in both XML and in code then the dictionary is being added twice, which throws an exception like
```
error MT2102: Error processing the method 'System.Boolean SignalGo.Shared.Helpers.ReflectionHelper/d__0::MoveNext()' in the assembly 'SignalGo.Shared.dll': An item with the same key has already been added. Key: System.String System.Reflection.ParameterInfo::get_Name()
```
reference: https://github.com/xamarin/xamarin-macios/issues/4978
* Bump to mono:2018-06
* Bump mono
* Updates compression to work with the public span
* Bump mono
* Fixes pointer check logic in Deflater
* Bump mono
* Fixes pointer check logic in Deflater
* Bump mono
* Bump Mono
* [runtime] always use `mono_jit_set_aot_mode` (#4491)
`mono_jit_set_aot_only` is deprecated and accidentally broke with
https://github.com/mono/mono/pull/7887
This should fix device tests with `mono-2018-06`
* Testing with Zoltan's patch
* Include libmono-system-native on Xamarin.Mac
* Bump Mono
Commit list for mono/mono:
* mono/mono@7bcda192a0 Bump llvm to release_60/fc854b8ec5873d294b80afa3e6cf6a88c5c48886. (#9786). (#9804)
* mono/mono@23e95ec7ad Apply F# portable pdb debug fix for pinvokes & bump (#9797)
* mono/mono@295f6d32af [2018-06] [MacOS] On Mac, use the copyfile API to copy files (#9696)
Diff: 7d5f4b6136...7bcda192a0
* Revert 4bacab3d5c, it doesn't fix the ios aot problems.
* Bump mono
* [tests] Adjust the MT0137 test for mcs change in behavior.
Starting with mono 5.16 mcs will now add assembly references when the assembly
is only used in attributes (this was already the case for csc in both 5.14 and
5.16, so it seems to be a compatibility change).
Adjust the MT0137 test accordingly.
* [msbuild] Fix parsing of json parser errors to handle trailing periods in the error message.
Fixes this test:
1) Test Failure : Xamarin.iOS.Tasks.Bug60536.TestACToolTaskCatchesJsonException
ColumnNumber
Expected: 2
But was: 0
* Bump mono
* [builds] Install the old llvm binaries into the LLVM36 directory and make the 32 bit builds use that.
* Bump mono
* Bump mono
* [jenkins] Don't give VSTS a fake branch. (#4667)
Something in VSTS changed, and now fake branch names don't work anymore.
So instead use real branch names (and for pull requests I've created a
'pull-request' branch we can use).
* Assembly.LoadFile accepts only absolute path
* [linker] Add new Facade (System.Threading.Tasks.Extensions).
Fixes these MTouch test failures:
1. Xamarin.Linker.SdkTest.iOS_Unified : Facades
Expected:
But was: < "System.Threading.Tasks.Extensions" >
2. Xamarin.Linker.SdkTest.tvOS : Facades
Expected:
But was: < "System.Threading.Tasks.Extensions" >
3. Xamarin.Linker.SdkTest.watchOS : Facades
Expected:
But was: < "System.Threading.Tasks.Extensions" >
* [mono-sdks] Necessary changes to unify the LLVM provisioning for both iOS and Android. (#4732)
* Bump Mono
* [mtouch] add mixed-mode support (#4751)
* [mtouch] add --interp-mixed option
When enabling this option, mtouch will AOT compile `mscorlib.dll`. At
runtime that means every method that wasn't AOT'd will be executed by
the runtime interpreter.
* [mtouch] Add support to --interpreter to list the assemblies to (not) interpret.
* [msbuild] Simplify interpreter code to use a single variable.
* Fix whitespace.
* [mtouch] Move mtouch-specific code to mtouch-specific file.
* [msbuild] An empty string is a valid value for 'Interpreter', so make it a non-required property.
* [mtouch] Add sanity check for aot-compiling interpreted assemblies.
* Bump Mono
* [linker] Updates SDKs facades list
* Bump mono
* [msbuild] Adds facades which might override default nuget version to framework list
The collision resolver task reads them from here https://github.com/dotnet/sdk/blob/master/src/Tasks/Common/ConflictResolution/FrameworkListReader.cs
* Bump to a VSfM version that can build XM Classic projects.
This means we'll get an empty diff if there are no generator changes, so
update the logic that detects/reports empty generator diffs.
Also there's no need to ignore mdbs anymore, since we don't create mdbs.
* XI references updated from `xcode10` (XI 12.0)
* XM references are not updated, we continue to compare with `d15.8`
* references_/* were temporary files that should not have been committed (old mistake)
* XI references updated from xcode10 (XI 12.0)
* XM references are not updated, we continue 4.99.x previews
* references_/* were temporary files that should not have been committed (old mistake)
* Add support for delegates as return values in protocol members. Fixes#4102.
This required a few changes:
* The generator now emits the DelegateProxy attribute for property getters in
protocol interfaces.
* The generator now emits the DelegateProxy attribute in ProtocolMember
attributes (and the ProtocolMember attribute has been extended with
additional properties for this purpose).
* The generator now emits the BlockProxy attribute for the parameter in
property setters.
* The generator now emits the BlockProxy attribute in ProtocolMember
attributes for property setters.
* The static registrar now emits the metadata token for the
DelegateProxy.DelegateType property into the generated code so that the
DelegateProxy attribute itself isn't needed at runtime. This is required
when the dynamic registrar has been optimized away.
Fixes https://github.com/xamarin/xamarin-macios/issues/4102.
* [tests] Update MX4105 test to expect new warnings.
* Store the minimum mono version for Xamarin.Mac in one place only (Make.config) and bump it to 5.14. Fixes#4120.
I've verified that we fail at launch of running on 5.12, while 5.14 works fine
(to launch at least), so the minimum system mono version is _at least_ 5.14.
Fixes https://github.com/xamarin/xamarin-macios/issues/4120.
* [mmp] Load mono's version file instead of using pkg-config to get mono's version.
pkg-config will only get three parts of the version, while the version file
has all four parts.
This is important, since we're now verifying the four parts of the version
file, and without loading those four from the system, we'll fail builds like
this:
error MM0001: This version of Xamarin.Mac requires Mono 5.14.0.136 (the current Mono version is 5.14.0).
because the three part version's fourth number is assumed to be 0.
* Only verify mono runtime version when running with system/dynamic mono.
There should be no need to verify the mono runtime version when embedding mono:
* If it's a mono we're shipping, something very bad happened in our
build/package for it to be an invalid mono.
* If it's a system mono that's being embedded, then we verify in mmp at build
time.
In the first scenario (a mono we're shipping), the problem is that the mono
we've built does not report back the full version number (with four parts) [1],
which means we'll fail any check whose requirements are identical for the
first three parts, and non-zero for the last.
[1] The fourth part of the version number is created/calculated when packaging
mono, and we're not packaging it.
* [src] Fix bgen build to support custom output directory for api comparison. Fixes maccore#959.
This broke in 87c27e0c3f, which made bgen
compile using a project file (I forgot to verify that it wouldn't affect the
API comparison, and the PR build didn't complain because problems with the API
comparison typically show up in subsequent PRs).
https://github.com/xamarin/maccore/issues/959
* [src] Fix response file usage to use BUILD_DIR, so that API comparison can redirect as expected.
* [src] Fix generation of generator.csproj's dependency file to use BUILD_DIR, so that API comparison can redirect as expected.
* [src] Fix bgen.exe's dependencies.
* Use full paths to create-makefile-fragment.sh.
This makes it possible to build mmp (the partial static registrar code) with
the wrong system Xcode (which is not supported, but that doesn't mean we can't
try to make it mostly work to ease our lives).
* Make sure mtouch.csproj builds using the same compiler arguments as the makefile used.
* Build mtouch.exe using msbuild in the makefile, and clean up the resulting unused make logic.
Partial fix for https://github.com/xamarin/xamarin-macios/issues/4384.
* Make sure mmp.csproj builds using the same compiler arguments as the makefile used.
* Clean up mmp.csproj by removing stuff that was left over from when it was a Xamarin.Mac project.
* Build mmp.exe using msbuild in the makefile, and clean up the resulting unused make logic.
Partial fix for https://github.com/xamarin/xamarin-macios/issues/4384.
Linking with CoreNFC crash applications on iOS 12 on iPad (and likely
other device not supporting, or supported, for NFC).
This used to work on iOS 11.x (when introduced). The solution is to
always **weak** link CoreNFC (since we can't guess devices)
https://github.com/xamarin/xamarin-macios/issues/4628
* [jenkins] Only XM apps with variations are Classic/32-bit apps, so adjust ignore logic accordingly. Fixes maccore issue 884.
Fixes https://github.com/xamarin/maccore/issues/884.
* [mmp] Fix passing -stdlib=libc++ to clang.
* [tests] Fix 32-bit XM issues.
* [xharness] Add support for building 32-bit XM apps by using Xcode 9.4.
* [xharness] Since xharness can now build 32-bit mac apps, enable them by default.
* Remove debug code.
* Remove unused variable.
* CoreFoundation/DispatchData: avoid possible integer overflow
* Network: move attributes for types introduced in Xcode10 from the
members to the types.
* Network: for callbacks that surface INativeObjects, rather than using
* Hide P/Invokes that are not currently surfaced so xtro tests can track this
* guidelines: Ip -> IP
* SecIdentity2: fix a leak by releasing the returned array, check for handle being null.
* SecTrust2: check for handle being null.
Fixes this problem in the MT4134 test:
Process exited with code 1, command:
/Applications/Xcode83.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang -Wno-receiver-forward-class -Wno-objc-missing-super-calls -gdwarf-2 -I/Users/builder/jenkins/workspace/xamarin-macios/xamarin-macios/_ios-build/Library/Frameworks/Xamarin.iOS.framework/Versions/git/SDKs/MonoTouch.iphonesimulator.sdk/usr/include -isysroot /Applications/Xcode83.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator10.3.sdk -Qunused-arguments -fobjc-legacy-dispatch -fobjc-abi-version=2 -mios-simulator-version-min=11.4 -arch x86_64 -c -o /Users/builder/jenkins/workspace/xamarin-macios/xamarin-macios/tests/mtouch/bin/Debug/tmp-test-dir/Xamarin.Tests.BundlerTool.CreateTemporaryDirectory513/mtouch-test-cache/x86_64/registrar.o -x objective-c++ /Users/builder/jenkins/workspace/xamarin-macios/xamarin-macios/tests/mtouch/bin/Debug/tmp-test-dir/Xamarin.Tests.BundlerTool.CreateTemporaryDirectory513/mtouch-test-cache/registrar.m
In file included from /Users/builder/jenkins/workspace/xamarin-macios/xamarin-macios/tests/mtouch/bin/Debug/tmp-test-dir/Xamarin.Tests.BundlerTool.CreateTemporaryDirectory513/mtouch-test-cache/registrar.m:2:
/Users/builder/jenkins/workspace/xamarin-macios/xamarin-macios/tests/mtouch/bin/Debug/tmp-test-dir/Xamarin.Tests.BundlerTool.CreateTemporaryDirectory513/mtouch-test-cache/registrar.h:47:9: fatal error: 'MetalPerformanceShaders/MetalPerformanceShaders.h' file not found
#import <MetalPerformanceShaders/MetalPerformanceShaders.h>
Note that the error is because the test is building with Xcode 8.3 (and this
is also why the problem wasn't found by the PR bots: they don't have older
Xcodes installed to run this particular test).
* [mtouch] Show warnings when we can't find referenced assemblies.
This would have helped track down #4235.
* Improve MT0137 warning to indicate the type of the attribute causing the warning.
Rolf Bjarne Kvinge
Chris Hamons
Sebastien Pouliot
Sebastien Pouliot
Jo Shields
Vincent Dondain
Manuel de la Pena
Miguel de Icaza