* [tests] Add introspection tests to ensure there are native linking instructions for all frameworks. Fixes#3976
or a good, documented (in test code) reason for not needing it.
https://github.com/xamarin/xamarin-macios/issues/3976
* Fix failures
- static registrar failed on 32bits because one type in PhotoUI missed
its `onlyOn64: true`
- link with the top, not the sub-frameworks
* [tvos] Fix mistakes in tvOS where some extranous types triggered unrequired/unavailable (or missing) frameworks
* [watchos] Fix watchOS frameworks mapping
* Bump to use Xcode 10 beta 1
* Update Versions.plist
* Add a dependency on Xcode 9.4.
* [msbuild] Fix build with Xcode 10 beta 1. (#4182)
Many years ago (in Xcode 7 according to code comment)
Developer/Platforms/iPhoneOS.platform/Developer/usr disappeared, and we coped
by looking at Developer/usr instead (and also the subsequent code to locate
the bin directory was based on the location of the usr directory).
Developer/Platforms/iPhoneOS.platform/Developer/usr reappeared in Xcode 10
beta 1, but it seems useless (for one it doesn't contain a bin directory), so
in order to try to keep things sane don't look for this directory in Xcode 10
and instead go directly for Developer/usr (which is what we've been using as
the usr directory for years anyway).
Fixes this problem when building apps with Xcode 10 beta 1:
/Library/Frameworks/Mono.framework/External/xbuild/Xamarin/iOS/Xamarin.iOS.Common.targets(626,3): error : Could not locate SDK bin directory [/Users/rolf/Projects/TestApp/test-app.csproj]
* [runtime] Build 32-bit mac executables using Xcode 9.4.
* [mtouch] Work around broken tvOS headers in Xcode 10 beta 1.
* [mtouch] Work around build problem with Apple's simd headers in Objective-C++ mode.
* Use version-agnostic paths to sdk directories.
* [tests][xtro] Add todo files (from unclassified) and adjust ignore files to avoid errors
* [macos][security] Re-enable SSL[Get|Set]AlpnProtocols. Fixes#4001 (#4022)
* [macos][security] Re-enable SSL[Get}Set]AlpnProtocols. Fixes#4001
This was fixed in macOS 10.13.4
https://github.com/xamarin/xamarin-macios/issues/4001
* [tests][monotouch-tests] Disable a few test cases (one crasher, other failures). Causes to be verified later
* [xharness] Fix permission dialog suppression in Xcode 10.
* [xharness] Ignore 32-bit macOS tests by default.
* [tests] Execute mmp regression tests with Xcode 9.4 since many of them are 32-bit and needs porting to 64-bit.
* [mmptest] Ignore 32-bit XM tests if we don't have a 32-bit-capable Xcode.
* [registrar] Add workaround for broken headers in Xcode 10 beta 1 (radar 40824697).
* [mtouch] Restrict another workaround for an Xcode 10 beta 1 bug to a specific Xcode version to remove it asap.
* [tests] Fix some protocol changes (public or not) find by introspection tests
* [tests][intro] Fix DefaultCtorAllowed failures
* [Intents] Obsolete several Intents classes in watchOS.
Several existing Intents classes have been marked as unavailable in watchOS in
the headers in Xcode 10 beta 1, and corresponding tests are now failing.
So obsolete the managed wrapper types, and fix tests accordingly.
* Fix xtro wrt previous Ietents/intro changes
* [tests] Minor adjustments to mtouch tests to work with Xcode 10.
* [msbuild] Update tests to cope with additional files produced by the Core ML compiler.
* [msbuild] Xcode 10 doesn't support building watchOS 1 apps, so show a clear error message explaining it.
Also update tests accordingly.
* [coreimage] Stub new filters and exclude ?removed? ones from tests
* Update GameplayKit and SpriteKit NSSecureCoding _upgrade_ and fix other non-public cases (in tests)
* [tests] Ignore some GameKit selectors that don't respond anymore (but seems to be available, at least in header files)
* [tests] Fix intro 32bits testing for filters resutls
* [msbuild] Slightly change error message to be better English.
The PR is not final and cannot be merged until the final Xcode 9.4
release from Apple is available.
Since there's no macOS specific changes (at least up to beta 2) we can
directly merge into the _normal_ milestone branch and avoid having
separate branches to maintain for XI and XM (until 15.8).
watchOS extensions are top-level containers in our build (because we ignore
watchOS apps entirely in mtouch), so treat them as such when computing the
protocol member map.
Fixes issue https://github.com/xamarin/xamarin-macios/issues/3930.
* [tools][api-diff] Add a tool to merge several .md files into a single one (#3814)
This depends on an updated mono-api-html tool (not yet available). This
PR will test the creation (empty headers) and storage of the files on bots.
Reference https://github.com/xamarin/xamarin-macios/issues/3787
* Bump macios-binaries to get latest mono-api-html.exe to produce markdown
* Move Registrar.SanitizeName to StringUtils.SanitizeObjectiveCName.
* [generator] Register models with unique names to not match platform types. Fixes#3875.
* [NSObject] Don't compare against a non-existent protocol.
* [generator] Make it possible to register models like before if the binding developer wishes it.
* [src] Make sure to not declare ObjC classes Apple already defines.
Fixes these warnings at startup:
Class DOMNodeFilter is implemented in both /System/Library/Frameworks/WebKit.framework/Versions/A/Frameworks/WebKitLegacy.framework/Versions/A/WebKitLegacy (0x7fffa944a788) and /Users/builder/data/lanes/6035/0ca02336/source/xamarin-macios/tests/xharness/tmp-test-dir/dont link-mac-unified/bin/x86/Debug-unified/dont link.app/Contents/MacOS/dont link (0x10d8a9958). One of the two will be used. Which one is undefined.
Class WebOpenPanelResultListener is implemented in both /System/Library/Frameworks/WebKit.framework/Versions/A/Frameworks/WebKitLegacy.framework/Versions/A/WebKitLegacy (0x7fffa944e4c8) and /Users/builder/data/lanes/6035/0ca02336/source/xamarin-macios/tests/xharness/tmp-test-dir/dont link-mac-unified/bin/x86/Debug-unified/dont link.app/Contents/MacOS/dont link (0x10d8a9a98). One of the two will be used. Which one is undefined.
Class WebPolicyDecisionListener is implemented in both /System/Library/Frameworks/WebKit.framework/Versions/A/Frameworks/WebKitLegacy.framework/Versions/A/WebKitLegacy (0x7fffa944e838) and /Users/builder/data/lanes/6035/0ca02336/source/xamarin-macios/tests/xharness/tmp-test-dir/dont link-mac-unified/bin/x86/Debug-unified/dont link.app/Contents/MacOS/dont link (0x10d8a9ae8). One of the two will be used. Which one is undefined.
Class MTLCaptureScope is implemented in both /System/Library/Frameworks/Metal.framework/Versions/A/Metal (0x7fffa806f1d0) and /Users/builder/data/lanes/6035/0ca02336/source/xamarin-macios/tests/xharness/tmp-test-dir/dont link-mac-unified/bin/x86/Debug-unified/dont link.app/Contents/MacOS/dont link (0x10d8aa858). One of the two will be used. Which one is undefined.
Class JSExport is implemented in both /System/Library/Frameworks/JavaScriptCore.framework/Versions/A/JavaScriptCore (0x7fffa7eb4f60) and /Users/builder/data/lanes/6035/0ca02336/source/xamarin-macios/tests/xharness/tmp-test-dir/dont link-mac-unified/bin/x86/Debug-unified/dont link.app/Contents/MacOS/dont link (0x10d8aaa38). One of the two will be used. Which one is undefined.
* [ClassKit] Add Xcode 9.4 Beta 1 Bindings
* ClassKit moved to v11.4 or we'd link against a private frmework in 11.3
* Turn CLSPredicateKeyPath into a static class
CLSPredicateKeyPath does not make much sense as an enum, we'll use
a static class instead so we do not have to call GetConstant() and
use the NSString directly.
Sample code using CLSPredicateKeyPath:
```csharp
var store = CLSDataStore.Shared;
var predicate = NSPredicate.FromFormat ("%K = %@", CLSPredicateKeyPath.Parent, store.MainAppContext);
var ctxs = await store.FindContextsMatchingAsync (predicate);
foreach (var ctx in ctxs) {
Console.WriteLine (ctx.Title);
}
```
* [tests] Exclude WeakTopic incorrect check, bound as smart enum
* [jenkins] Create comment file for PR builds. (#3799)
* [jenkins] Create comment file for PR builds.
* [tools] Create stamp file after doing things that might modify files we care about. (#3864)
We have consistency checks to verify that no unexpected files are modified
done when comparing APIs in for a pull request.
Unfortunately the check didn't take into account that checking out the
revision to do the API check against might modify some of the files in the
consistency check itself, thus triggering the consistency check.
Fix this by only verify timestamps of files modified after checkout out the
revision, which is the only thing we care about anyway.
For examples see PR #3855 or PR #3850.
* [jenkins] Don't succeed if something went wrong when creating API or generator diff. (#3865)
Ref: https://github.com/xamarin/xamarin-macios/pull/3855#issuecomment-378441993
Fix https://github.com/xamarin/xamarin-macios/issues/3653 differently as
`UseShellExecute` cannot be used when redirecting output so the original
fix [1] caused an exception which affected it from both macOS and windows
(thru XMA) instead of being an issue only with the later.
It's not clear how the original [1] fix was validated successfully, it's
possible than an older version of mono did not throw (since that
limitation seems windows specific).
[1] https://github.com/xamarin/xamarin-macios/pull/3781
- https://github.com/xamarin/xamarin-macios/issues/3725
- These frameworks "CoreAudioKit Metal MetalKit MetalPerformanceShaders CoreNFC DeviceCheck"
were special cased, but that special case did do an SDK check.
- Create a helper method to share check
- Add test for MM0135
If a class implements a protocol with optional members, and that class also exports methods whose selectors match an optional member, but the signature doesn't match, we must show a useful warning instead of erroring out due to a NullReferenceException.
Fixes this:
System.NullReferenceException: Object reference not set to an instance of an object
at Registrar.StaticRegistrar.GetBlockProxyAttributeMethod (Mono.Cecil.MethodDefinition method, System.Int32 parameter) [0x00001] in /Users/builder/data/lanes/1381/9de35b83/source/xamarin-macios/tools/common/StaticRegistrar.cs:4113
at Registrar.StaticRegistrar.GetBlockWrapperCreator (Registrar.Registrar+ObjCMethod obj_method, System.Int32 parameter) [0x001e1] in /Users/builder/data/lanes/1381/9de35b83/source/xamarin-macios/tools/common/StaticRegistrar.cs:4101
at Registrar.StaticRegistrar.Specialize (Registrar.AutoIndentStringBuilder sb, Registrar.Registrar+ObjCMethod method, System.Collections.Generic.List`1[T] exceptions) [0x0216b] in /Users/builder/data/lanes/1381/9de35b83/source/xamarin-macios/tools/common/StaticRegistrar.cs:3683
when building the msbuild/tests/MyWatchKit2IntentsExtension project.
Fixes this test failure:
1) Failed : Xamarin.MTouch.MT0113_linker
The error 'MT0113: Native code sharing has been disabled for the extension 'testServiceExtension' because the managed linker settings are different between the container app (None) and the extension (All).' was not found in the output:
Message #1 did not match:
actual: 'Native code sharing has been disabled for the extension 'testServiceExtension' because the remove-dynamic-registrar optimization differ between the container app (default) and the extension (false).'
expected: 'Native code sharing has been disabled for the extension 'testServiceExtension' because the managed linker settings are different between the container app (None) and the extension (All).'
which happens because:
* Removing the dynamic registrar requires the linker, so removal of the dynamic registrar is disabled if the linker is not disabled
* This results in the app and appex having different values for the remove-dynamic-registrar option
* Thus the error message.
Technically either error is correct, but I prefer the previous one (about the
linker), because it directly assigns blame (the linker setting). Figuring out
what has to change (the linker setting) when the error message complains about
an optimization is not so straight forward for users.
* [MMP] Revert recursive search dirs changes
* [MMP] Allow resolving assemblies to the ones passed in command line args
This is what actually makes the MonoMacResolver actually resolve to
assemblies given as arguments.
This mirrors the behaviour of Pack, which is called on the other
code-path (that's not --runregistrar)
3113c5d2b5/tools/mmp/driver.cs (L513)
This fixes a startup crash in code shared app extensions due to having the
wrong value set in the runtime (the dynamic registrar was removed, but the
executable didn't know it).
* [static registrar] Optimize creation of delegates for blocks.
Optimize creation of delegates for blocks so that it doesn't require the
dynamic registrar.
This is done by getting the metadata token for the Create method that creates
the delegate, and embed that metadata token in the generated code from the
static registrar.
Also add tests, since this scenario was not covered by tests already.
* [mmptest] Fix test after recent changes.
* [test-libraries] Avoid duplicate symbols.
* [tests] Update according to changes.
At some point the code wasn't able to figure out the framework for linked away
types (because linked away types don't know which assembly they belong to, and
thus we couldn't verify that the type in question was a platform type or not),
so I skipped checking the namespace for such types.
Some time later I implemented support for storing the assembly for a linked
away type separately, so that it can later be looked up, and thus it's not
necessary to exclude linked away types anymore.
This fixes a build problem with the generated registrar code (which happens
only if the INCWidgetProviding interface is linked away) when building the
linkall extension tests:
/work/maccore/master/xamarin-macios/tests/xharness/tmp-test-dir/link all1894/obj/iPhone/Debug64-today-extension/mtouch-cache/registrar.h:319:51: error: no type or protocol named 'NCWidgetProviding'
@interface TodayViewController : UIViewController<NCWidgetProviding> {
^
- https://github.com/xamarin/xamarin-macios/issues/3367
- App Store will now fail builds if you add in a 32-bit dylib
- If you are a 32-bit app you don't need the 64-bit part of your fat
dylib anyway
- Add --optimize=-trim-architectures to allow customization of behavior, as not everyone
uses app store
In addition, while writing tests for this is was noticed that mmp tests did not "really" run Release configuration correctly in most cases. Fixing this turned out to be a bit of a pain, but necessary to correctly test this (and other things).
- Turns out that /p:configuration:debug is not sufficient to tell mmp to
do the right thing
- That, in most projects, sets the DebugSymbols property, which really
is what is checked.
- However, two of our projects did not have that, so we always did
release mmp work.
- Removed configuration property for tests and added real "Release"
configuration option
* [registrar] Fix resolving linked away generic types. Fixes#3523.
Fixes#3523.
* [tests] Use a link all test instead of mtouch test.
It's much faster, since we're already building link all everywhere.
* [tests] Use the static registrar in all linkall simulator configurations.
When code sharing is enabled, only the container app/target will have a
LinkContext.
So make sure the static registrar finds that link context when it needs
information from the linker.
https://github.com/xamarin/xamarin-macios/issues/3514
* [tests] Improve debug spew for the RebuildTest_WithExtensions test.
* [mtouch/mmp] Store/load if the dynamic registrar is removed or not into the cached link results.
Store/load if the dynamic registrar is removed or not into the cached link
results, so that we generate the correct main.m even if cached linker results
are used.
* [mtouch/mmp] The static registrar must not execute if we're loading cached results from the linker.
The static registrar must not execute if we're loading cached results from the
linker, because the static registrar needs information from the linker that's
not restored from the cache.
* [mtouch/mmp] Share Touch code.
* [mtouch/mmp] Make it possible to touch inexistent files (to create them).
* [mtouch/mmp] Fix tracking of whether the static registrar should run again or not.
The recent changes to support optimizing away the dynamic registrar caused the
Xamarin.MTouch.RebuildTest_WithExtensions test to regress.
The problem
-----------
* The linker now collects and stores information the static registrar needs.
* This information is not restored from disk when the linker realizes that it
can reload previously linked assemblies instead of executing again.
* The static registrar runs again (for another reason).
* The information the static registrar needs isn't available, and incorrect
output follows.
So fix 1: show an error if the static registrar runs when the linker loaded
cached results.
The exact scenario the test ran into is this:
* 1st build: everything is new and everything is built.
* 2nd build: contents of .exe changes, the linker runs again, the static
registrar runs again, but sees that the generated output didn't change, so
it doesn't write the new content to disk (this is an optimization to avoid
compiling the registrar.m file again unless needed).
* 3rd build: only the .exe timestamp changes, the linker sees nothing changes
in the contents of the .exe and loads the previously linked assemblies from
disk, the static registrar sees that the .exe's timestamp is newer than
registrar.m's timestamp and run again, but doesn't produce the right result
because it doesn't have the information it needs.
Considered solutions
--------------------
1. Only track timestamps, not file contents. This is not ideal, since it will
result in more work done: in particular for the case above, it would add a
registrar.m compilation in build #2, and linker rerun + static registrar
rerun + registrar.m compilation + final native link in build #3.
2. Always write the output of the static registrar, even if it hasn't changed.
This is not ideal either, since it will also result in more work done: for
the case above, it would add a registrar.m compilation + final native link
in build #3.
3. Always write the output of the static registrar, but track if it changed or
not, and if it didn't, just touch registrar.o instead of recompiling it.
This only means the final native link in build #3 is added (see #5 for why
this is worse than it sounds).
4. Always write the output of the static registrar, but track it it changed or
not, and if it didn't, just touch registrar.o instead of recompiling it,
and track that too, so that the final native link in build #3 isn't needed
anymore. Unfortunately this may result in incorrect behavior, because now
the msbuild tasks will detect that the executable has changed, and may run
dsymutil + strip again. The executable didn't actually change, which means
it would be the previously stripped executable, and thus we'd end up with
an empty .dSYM because we ran dsymtil on an already stripped executable.
5. Idea #4, but write the output of the final link into a temporary directory
instead of the .app, so that we could track whether we should update the
executable in the .app or not. This is not optimal either, because
executables can be *big* (I've seen multi-GB tvOS bitcode executables), and
extra copies of such files should not be taken lightly.
6. Idea #4, but tell the MSBuild tasks that dsymutil/strip doesn't need to be
rerun even if the timestamp of the executable changed. This might actually
work, but now the solution's become quite complex.
Implemented solution
--------------------
Use stamp files to detect whether a file is up-to-date or not.
In particular:
* When we don't write to a file because the new contents are identical to the
old contents, we now touch a .stamp file. This stamp file means "the
accompanying file was determined to be up-to-date when the stamp was
touched."
* When checking whether a file is up-to-date, also check for the presence of a
.stamp file, and if it exists, use the highest timestamp between the stamp
file and the actual file.
Now the test scenario becomes:
* 1st build: everything is new and everything is built.
* 2nd build: contents of .exe changes, the linker runs again, the static
registrar runs again, but sees that the generated output didn't change, so
it doesn't write the new content to disk, but it creates a registrar.m.stamp
file to indicate the point in time when registrar.m was considered up-to-
date.
* 3rd build: only the .exe timestamp changes, the linker sees nothing changes
in the contents of the .exe and loads the previously linked assemblies from
disk, the static registrar sees that the .exe's timestamp is *older* than
registrar.m.stamp's timestamp and doesn't run again.
We only use the stamp file for source code (registrar.[m|h], main.[m|h],
pinvokes.[m|h]), since using it every time has too much potential for running
into other problems (for instance we should never create .stamp files inside
the .app).
Fixes these test failures:
1) Failed : Xamarin.MTouch.RebuildTest_WithExtensions("single","",False,System.String[])
single
Expected: <empty>
But was: < "/Users/builder/data/lanes/5746/4123bf7e/source/xamarin-macios/tests/mtouch/bin/Debug/tmp-test-dir/Xamarin.Tests.BundlerTool.CreateTemporaryDirectory371/testApp.app/testApp is modified, timestamp: 2/15/2018 3:04:11 PM > 2/15/2018 3:04:09 PM" >
2) Failed : Xamarin.MTouch.RebuildTest_WithExtensions("dual","armv7,arm64",False,System.String[])
dual
Expected: <empty>
But was: < "/Users/builder/data/lanes/5746/4123bf7e/source/xamarin-macios/tests/mtouch/bin/Debug/tmp-test-dir/Xamarin.Tests.BundlerTool.CreateTemporaryDirectory375/testApp.app/testApp is modified, timestamp: 2/15/2018 3:06:03 PM > 2/15/2018 3:06:00 PM" >
3) Failed : Xamarin.MTouch.RebuildTest_WithExtensions("llvm","armv7+llvm",False,System.String[])
llvm
Expected: <empty>
But was: < "/Users/builder/data/lanes/5746/4123bf7e/source/xamarin-macios/tests/mtouch/bin/Debug/tmp-test-dir/Xamarin.Tests.BundlerTool.CreateTemporaryDirectory379/testApp.app/testApp is modified, timestamp: 2/15/2018 3:07:14 PM > 2/15/2018 3:07:12 PM" >
4) Failed : Xamarin.MTouch.RebuildTest_WithExtensions("debug","",True,System.String[])
debug
Expected: <empty>
But was: < "/Users/builder/data/lanes/5746/4123bf7e/source/xamarin-macios/tests/mtouch/bin/Debug/tmp-test-dir/Xamarin.Tests.BundlerTool.CreateTemporaryDirectory383/testApp.app/testApp is modified, timestamp: 2/15/2018 3:08:16 PM > 2/15/2018 3:08:13 PM" >
5) Failed : Xamarin.MTouch.RebuildTest_WithExtensions("single-framework","",False,System.String[])
single-framework
Expected: <empty>
But was: < "/Users/builder/data/lanes/5746/4123bf7e/source/xamarin-macios/tests/mtouch/bin/Debug/tmp-test-dir/Xamarin.Tests.BundlerTool.CreateTemporaryDirectory387/testApp.app/testApp is modified, timestamp: 2/15/2018 3:09:18 PM > 2/15/2018 3:09:16 PM" >
Fixes https://github.com/xamarin/maccore/issues/641
The linker might remove interfaces that have already been linked away. Make
sure to look for the TypeDefinition for such interfaces among the types that
have already been linked away.
Fixes https://github.com/xamarin/xamarin-macios/issues/3513.
Fixes this test:
Xamarin.ApiTest.ApiTest.AlwaysOptimizable(iOS) : All methods calling optimizable API must be optimizable
The method System.IntPtr ObjCRuntime.BlockLiteral::GetBlockForDelegate(System.Reflection.MethodInfo,System.Object,System.String) calls System.Boolean ObjCRuntime.Runtime::get_DynamicRegistrationSupported(), but it does not have a [BindingImpl (BindingImplOptions.Optimizable)] attribute.
Also skip the BlockLiteral.SetupBlock optimization for this method, because
it's not optimizable (and thus produces a warning users can't ignore).
Manuel de la Pena
Sebastien Pouliot
Alex Soto
Rolf Bjarne Kvinge
Sebastien Pouliot
Vincent Dondain
Chris Hamons
Zoltan Varga