The [Protocol] attributes the test checks for are not linked away in
monotouch-test (because monotouch-test only links SDK assemblies), but they
are linked away in link all, so adjust the test accordingly.
* [registrar] Fix resolving linked away generic types. Fixes#3523.
Fixes#3523.
* [tests] Use a link all test instead of mtouch test.
It's much faster, since we're already building link all everywhere.
* [tests] Use the static registrar in all linkall simulator configurations.
When code sharing is enabled, only the container app/target will have a
LinkContext.
So make sure the static registrar finds that link context when it needs
information from the linker.
https://github.com/xamarin/xamarin-macios/issues/3514
* [tests] Automatically rebuild 'test.config' if it can't be found.
This condition usually happens when I 'git clean -xfd' in tests/ and then want
to run tests from xharness.
The manual step is to run "make" in tests/, but it's better if we can do it automatically.
* [msbuild] Add file to fix build.
We want to avoid separate `mscorlib.dll` assemblies for 32/64 bits so
the architecture specific code for `n[u]int` and `nfloat` must be
preserved in both cases.
Because a single, slightly larger, assembly is much smaller than two
(slightly smaller) ones.
This is not a common situation since the extraneous preserved API are
often used in the application (or 3rd party code) so, in most cases,
a single `mscorlib.dll` was already used.
This merely close the gap for some cases, like our `link all` application
where this happened.
* [mtouch] Rename temporary directories so the order assemblies move is clearer.
I implemented this myself, but I can never remember in which order assemblies
go from one directory to another during the build.
So number these temporary directories, so that even the most forgetful minds
can understand without having to remember anything.
* [tests] Update test according to temporary directory name change.
* [tests] Improve debug spew for the RebuildTest_WithExtensions test.
* [mtouch/mmp] Store/load if the dynamic registrar is removed or not into the cached link results.
Store/load if the dynamic registrar is removed or not into the cached link
results, so that we generate the correct main.m even if cached linker results
are used.
* [mtouch/mmp] The static registrar must not execute if we're loading cached results from the linker.
The static registrar must not execute if we're loading cached results from the
linker, because the static registrar needs information from the linker that's
not restored from the cache.
* [mtouch/mmp] Share Touch code.
* [mtouch/mmp] Make it possible to touch inexistent files (to create them).
* [mtouch/mmp] Fix tracking of whether the static registrar should run again or not.
The recent changes to support optimizing away the dynamic registrar caused the
Xamarin.MTouch.RebuildTest_WithExtensions test to regress.
The problem
-----------
* The linker now collects and stores information the static registrar needs.
* This information is not restored from disk when the linker realizes that it
can reload previously linked assemblies instead of executing again.
* The static registrar runs again (for another reason).
* The information the static registrar needs isn't available, and incorrect
output follows.
So fix 1: show an error if the static registrar runs when the linker loaded
cached results.
The exact scenario the test ran into is this:
* 1st build: everything is new and everything is built.
* 2nd build: contents of .exe changes, the linker runs again, the static
registrar runs again, but sees that the generated output didn't change, so
it doesn't write the new content to disk (this is an optimization to avoid
compiling the registrar.m file again unless needed).
* 3rd build: only the .exe timestamp changes, the linker sees nothing changes
in the contents of the .exe and loads the previously linked assemblies from
disk, the static registrar sees that the .exe's timestamp is newer than
registrar.m's timestamp and run again, but doesn't produce the right result
because it doesn't have the information it needs.
Considered solutions
--------------------
1. Only track timestamps, not file contents. This is not ideal, since it will
result in more work done: in particular for the case above, it would add a
registrar.m compilation in build #2, and linker rerun + static registrar
rerun + registrar.m compilation + final native link in build #3.
2. Always write the output of the static registrar, even if it hasn't changed.
This is not ideal either, since it will also result in more work done: for
the case above, it would add a registrar.m compilation + final native link
in build #3.
3. Always write the output of the static registrar, but track if it changed or
not, and if it didn't, just touch registrar.o instead of recompiling it.
This only means the final native link in build #3 is added (see #5 for why
this is worse than it sounds).
4. Always write the output of the static registrar, but track it it changed or
not, and if it didn't, just touch registrar.o instead of recompiling it,
and track that too, so that the final native link in build #3 isn't needed
anymore. Unfortunately this may result in incorrect behavior, because now
the msbuild tasks will detect that the executable has changed, and may run
dsymutil + strip again. The executable didn't actually change, which means
it would be the previously stripped executable, and thus we'd end up with
an empty .dSYM because we ran dsymtil on an already stripped executable.
5. Idea #4, but write the output of the final link into a temporary directory
instead of the .app, so that we could track whether we should update the
executable in the .app or not. This is not optimal either, because
executables can be *big* (I've seen multi-GB tvOS bitcode executables), and
extra copies of such files should not be taken lightly.
6. Idea #4, but tell the MSBuild tasks that dsymutil/strip doesn't need to be
rerun even if the timestamp of the executable changed. This might actually
work, but now the solution's become quite complex.
Implemented solution
--------------------
Use stamp files to detect whether a file is up-to-date or not.
In particular:
* When we don't write to a file because the new contents are identical to the
old contents, we now touch a .stamp file. This stamp file means "the
accompanying file was determined to be up-to-date when the stamp was
touched."
* When checking whether a file is up-to-date, also check for the presence of a
.stamp file, and if it exists, use the highest timestamp between the stamp
file and the actual file.
Now the test scenario becomes:
* 1st build: everything is new and everything is built.
* 2nd build: contents of .exe changes, the linker runs again, the static
registrar runs again, but sees that the generated output didn't change, so
it doesn't write the new content to disk, but it creates a registrar.m.stamp
file to indicate the point in time when registrar.m was considered up-to-
date.
* 3rd build: only the .exe timestamp changes, the linker sees nothing changes
in the contents of the .exe and loads the previously linked assemblies from
disk, the static registrar sees that the .exe's timestamp is *older* than
registrar.m.stamp's timestamp and doesn't run again.
We only use the stamp file for source code (registrar.[m|h], main.[m|h],
pinvokes.[m|h]), since using it every time has too much potential for running
into other problems (for instance we should never create .stamp files inside
the .app).
Fixes these test failures:
1) Failed : Xamarin.MTouch.RebuildTest_WithExtensions("single","",False,System.String[])
single
Expected: <empty>
But was: < "/Users/builder/data/lanes/5746/4123bf7e/source/xamarin-macios/tests/mtouch/bin/Debug/tmp-test-dir/Xamarin.Tests.BundlerTool.CreateTemporaryDirectory371/testApp.app/testApp is modified, timestamp: 2/15/2018 3:04:11 PM > 2/15/2018 3:04:09 PM" >
2) Failed : Xamarin.MTouch.RebuildTest_WithExtensions("dual","armv7,arm64",False,System.String[])
dual
Expected: <empty>
But was: < "/Users/builder/data/lanes/5746/4123bf7e/source/xamarin-macios/tests/mtouch/bin/Debug/tmp-test-dir/Xamarin.Tests.BundlerTool.CreateTemporaryDirectory375/testApp.app/testApp is modified, timestamp: 2/15/2018 3:06:03 PM > 2/15/2018 3:06:00 PM" >
3) Failed : Xamarin.MTouch.RebuildTest_WithExtensions("llvm","armv7+llvm",False,System.String[])
llvm
Expected: <empty>
But was: < "/Users/builder/data/lanes/5746/4123bf7e/source/xamarin-macios/tests/mtouch/bin/Debug/tmp-test-dir/Xamarin.Tests.BundlerTool.CreateTemporaryDirectory379/testApp.app/testApp is modified, timestamp: 2/15/2018 3:07:14 PM > 2/15/2018 3:07:12 PM" >
4) Failed : Xamarin.MTouch.RebuildTest_WithExtensions("debug","",True,System.String[])
debug
Expected: <empty>
But was: < "/Users/builder/data/lanes/5746/4123bf7e/source/xamarin-macios/tests/mtouch/bin/Debug/tmp-test-dir/Xamarin.Tests.BundlerTool.CreateTemporaryDirectory383/testApp.app/testApp is modified, timestamp: 2/15/2018 3:08:16 PM > 2/15/2018 3:08:13 PM" >
5) Failed : Xamarin.MTouch.RebuildTest_WithExtensions("single-framework","",False,System.String[])
single-framework
Expected: <empty>
But was: < "/Users/builder/data/lanes/5746/4123bf7e/source/xamarin-macios/tests/mtouch/bin/Debug/tmp-test-dir/Xamarin.Tests.BundlerTool.CreateTemporaryDirectory387/testApp.app/testApp is modified, timestamp: 2/15/2018 3:09:18 PM > 2/15/2018 3:09:16 PM" >
Fixes https://github.com/xamarin/maccore/issues/641
Implement support for optimizing away the dynamic registrar.
* Add a new property (Runtime.DynamicRegistrationSupported) that indicates at
runtime whether dynamic registration is available.
* Check this new property whenever we need dynamic registration.
* Add an optimization to mtouch/mmp that detects whether dynamic registration
is required, and if not, changes Runtime.DynamicRegistrationSupported to
return false.
* Add an optimization to mtouch/mmp that inlines
Runtime.DynamicRegistrationSupported as a constant value.
The end result is that the linker will link away the dynamic registrar if
mtouch/mmp detects that it's not needed.
Benchmark
---------
I've compared the size of entire apps built for device:
|test | Before | After | Diff | % |
|:-----------------------------|-------:|-------:|-------:|------:|
|[monotouch-test/Release][1] | 95.7mb | 95.0mb | -680kb | -0.7% |
|[link sdk/Release][2] | 21.2mb | 20.9mb | -245kb | -1.2% |
|[minimalistic app/Release][3] | 4.58mb | 4.32mb | -259kb | -5.7% |
[1]: https://gist.github.com/rolfbjarne/3871e36d2de8db1a8eee1d9f9276d3d2#monotouch-test-release-wall-optimizations-enabled
[2]: https://gist.github.com/rolfbjarne/3871e36d2de8db1a8eee1d9f9276d3d2#link-sdk
[3]: https://gist.github.com/rolfbjarne/3871e36d2de8db1a8eee1d9f9276d3d2#minimalistic-app
Add a 'register-protocols' optimization that:
Improves static registrar to:
* Generate a new table of protocol -> managed wrapper type. This is required
to find the wrapper type without having the `[Protocol]` attribute around.
* Make the generated code implement protocols from [Adopts] attributes. This
makes it possible to link away the `[Protocol]` attribute, because the
native implementation of `conformsToProtocol:` does the right thing (we
might even be able to link away our complete `ConformsToProtocol` logic when
we remove the dynamic registrar).
Improves linker to:
* Not mark protocol interfaces by the mere virtue of having a type that
implements them. This is implemented by not marking protocol interfaces when
they're implementing a class, but instead when a method implementation is
found to implement a method from a protocol interface.
* Mark the wrapper type for protocols (this allows us to remove the Protocol
attribute, since that's the link between the protocol and its wrapper type).
* Remove the [Protocol], [ProtocolMember] and [Adopts] attributes (but only if
optimizing protocols).
The static registrar still needs some of the information linked away, so a few
changes are required to make it available post linker.
Benchmark
---------
I've compared the size of entire apps built for device:
|test | Before | After | Diff | % |
|:-----------------------------|-------:|-------:|-------:|------:|
|[monotouch-test/Debug][1] | 101mb | 100mb | -888kb | -0.9% |
|[monotouch-test/Release][2] | 99.2mb | 95.4mb | -830kb | -0.9% |
|[minimalistic app/Debug][3] | 10.8mb | 10.4mb | -443kb | -4.1% |
|[minimalistic app/Release][4] | 4.7mb | 4.55mb | -157kb | -3.3% |
[1]: https://gist.github.com/rolfbjarne/0181ab8abe436c34cf4ee68ecfb8cd18#monotouch-test-debug
[2]: https://gist.github.com/rolfbjarne/0181ab8abe436c34cf4ee68ecfb8cd18#monotouch-test-release
[3]: https://gist.github.com/rolfbjarne/0181ab8abe436c34cf4ee68ecfb8cd18#minimal-xi-app-debug
[4]: https://gist.github.com/rolfbjarne/0181ab8abe436c34cf4ee68ecfb8cd18#minimal-xi-app-release
Create a table of interface -> protocol in the static registrar, since we need
to be able to look up a protocol given a managed type without looking at the
(possibly linked away) [Protocol] attribute.
* [tests] Fix relative paths in mmptest and make cloned XM nunit projects work properly.
Fixes this build problem with mmptest:
/Users/builder/jenkins/workspace/xamarin-macios-pr-builder/tests/mmptest/CustomBuildActions.targets: error : Command 'make bin/SimpleClassDylib.dylib bin/SimpleClassStatic.a bin/Mobile-static/MobileBinding.dll' exited with code: 255.
Fixes https://github.com/xamarin/maccore/issues/633.
* [mmp] Improve Driver.IsDefaultMarshalingManagedExceptionMode to work before MarshalManagedException has been initialized.
Fixes this test failure:
1) Failed : Xamarin.MMP.Tests.MMPTests.UnifiedDebugBuilds_ShouldLinkToPartialStatic_UnlessDisabled
Debug build should use partial static registrar
Expected: True
But was: False
This regressed in 0561618460, where we wouldn't
properly select the partial static registrar because the initialization order
changed and we'd do it before selecting the default exception marshaling mode.
Changing Driver.IsDefaultMarshalingManagedExceptionMode to work before
MarshalManagedException has been initialized to its default value fixes the
problem.
Those were missed because xtro did not scan ObjC categories for
`objc_requires_super` attributes.
Fixing the naming mapping (to consider inlined categories) also
uncovered a few API with extraneous [DesignatedInitializer] attributes
Those were deprecated (API) and moved into categories so xtro missed
the designated initializer was removed.
All your `base`, well `super` in ObjC, now belong to us :)
https://github.com/xamarin/xamarin-macios/issues/3253
* [tests][xammac] Fix a few path calculations to not be hardcoded.
* [xharness] Clone XM projects.
Cloning projects before building them will also clone project references,
which will make it possible to build multiple projects in parallel, when those
projects have common project references.
Fixes https://github.com/xamarin/maccore/issues/624.
* [xharness] Fix XM project cloning
* Don't clone Classic projects (they're solutions, and xharness doesn't understand them).
* Clonee projects properly when cloning execution tasks.
* [xharness] GuiUnit needs specialized cloning.
The GuiUnit project uses relative paths to write to files outside the project
directory, which means that multiple GuiUnit project files may write to the
same location.
So special-case GuiUnit cloning to make those paths subdirectories of the
project's directory instead.
* [xharness] Process imported targets when cloning projects.
Also make the msbuild-mac's custom targets file independent of the location of
the project file by making all paths relative to the custom targets file.
Two constants with `__IPHONE_NA` were changed to `__IPHONE_11_3`.
Also decorate with [TV] and [Watch] attributes since they are implicit,
i.e. not in the headers but available in tvOS 11.3+ and WatchOS 4.3+
and, without our attributes, would seems to have been available much
earlier
* [linker] Optimize calls to BlockLiteral.SetupBlock to inject the block signature.
Optimize calls to BlockLiteral.SetupBlock[Unsafe] to calculate the block
signature at build time, and inject it into the call site.
This makes block invocations 10-15x faster (I've added tests that asserts at
least an 8x increase).
It's also required in order to be able to remove the dynamic registrar code in
the future (since calculating the block signature at runtime requires the
dynamic registrar).
* [mtouch/mmp] Add support for reporting errors/warnings that point to the code line causing the error/warning.
Add support for reporting errors/warnings that point to the code line causing
the error/warning by adding ErrorHelper overloads that take the exact
instruction to report (previously we defaulted to the first line/instruction
in a method).
* [tests] Add support for asserting filename/linenumber in warning messages.
* Make all methods that manually create BlockLiterals optimizable.
* [tests] Create a BaseOptimizeGeneratedCodeTest test that's included in both XI's and XM's link all test.
* [tests] Add link all test (for both XI and XM) to test the BlockLiteral.SetupBlock optimization.
* [tests] Add mtouch/mmp tests for the BlockLiteral.SetupBlock optimization.
* [tests][linker] Make the base test class abstract, so tests in the base class aren't executed twice.
* [tests][linker] Don't execute linkall-only tests in linksdk.
The optimization tests only apply when the test assembly is linked, and that
only happens in linkall, so exclude those tests in linksdk.
* [tests][mmptest] Update test according to mmp changes.
Fixes these test failures:
1) Failed : Xamarin.MMP.Tests.MMPTests.MM0132("inline-runtime-arch")
The warning 'MM0132: Unknown optimization: 'inline-runtime-arch'. Valid optimizations are: remove-uithread-checks, dead-code-elimination, inline-isdirectbinding, inline-intptr-size.' was not found in the output:
Message #1 did not match:
actual: 'Unknown optimization: 'inline-runtime-arch'. Valid optimizations are: remove-uithread-checks, dead-code-elimination, inline-isdirectbinding, inline-intptr-size, blockliteral-setupblock.'
expected: 'Unknown optimization: 'inline-runtime-arch'. Valid optimizations are: remove-uithread-checks, dead-code-elimination, inline-isdirectbinding, inline-intptr-size.'
Message #2 did not match:
actual: 'Unknown optimization: 'inline-runtime-arch'. Valid optimizations are: remove-uithread-checks, dead-code-elimination, inline-isdirectbinding, inline-intptr-size, blockliteral-setupblock.'
expected: 'Unknown optimization: 'inline-runtime-arch'. Valid optimizations are: remove-uithread-checks, dead-code-elimination, inline-isdirectbinding, inline-intptr-size.'
2) Failed : Xamarin.MMP.Tests.MMPTests.MM0132("foo")
The warning 'MM0132: Unknown optimization: 'foo'. Valid optimizations are: remove-uithread-checks, dead-code-elimination, inline-isdirectbinding, inline-intptr-size.' was not found in the output:
Message #1 did not match:
actual: 'Unknown optimization: 'foo'. Valid optimizations are: remove-uithread-checks, dead-code-elimination, inline-isdirectbinding, inline-intptr-size, blockliteral-setupblock.'
expected: 'Unknown optimization: 'foo'. Valid optimizations are: remove-uithread-checks, dead-code-elimination, inline-isdirectbinding, inline-intptr-size.'
Message #2 did not match:
actual: 'Unknown optimization: 'foo'. Valid optimizations are: remove-uithread-checks, dead-code-elimination, inline-isdirectbinding, inline-intptr-size, blockliteral-setupblock.'
expected: 'Unknown optimization: 'foo'. Valid optimizations are: remove-uithread-checks, dead-code-elimination, inline-isdirectbinding, inline-intptr-size.'
* [tests][linker] Fix typo.
Fixes this test failure:
1) SetupBlock_CustomDelegate (Linker.Shared.BaseOptimizeGeneratedCodeTest.SetupBlock_CustomDelegate)
Counter
Expected: 1
But was: 2
* [registrar] Minor adjustment to error message to match previous (and better) behavior.
Fixes this test failure:
1) Failed : Xamarin.Registrar.GenericType_WithInvalidParameterTypes
The error 'MT4136: The registrar cannot marshal the parameter type 'System.Collections.Generic.List`1<U>' of the parameter 'arg' in the method 'Open`1.Bar(System.Collections.Generic.List`1<U>)'' was not found in the output:
Message #1 did not match:
actual: 'The registrar cannot marshal the parameter type 'System.Collections.Generic.List`1<Foundation.NSObject>' of the parameter 'arg' in the method 'Open`1.Bar(System.Collections.Generic.List`1<U>)''
expected: 'The registrar cannot marshal the parameter type 'System.Collections.Generic.List`1<U>' of the parameter 'arg' in the method 'Open`1.Bar(System.Collections.Generic.List`1<U>)''
* [docs] mmp shows MM errors/warnings.
* [docs] Improve according to reviews.
* [tests] Fix merge failure causing test duplication.
* [generator] Teach generator about WrapAttribute on Getters and Setters
https://bugzilla.xamarin.com/show_bug.cgi?id=57870
`WrapAttribute` can now be used in property getters and setters,
this allows to Wrap virtually anything the way you need, for example
smart enums, consider the following API definition:
```csharp
// Smart enum.
enum PersonRelationship {
[Field (null)]
None,
[Field ("FMFather", "__Internal")]
Father,
[Field ("FMMother", "__Internal")]
Mother
}
```
```csharp
// Property definition.
[Export ("presenceType")]
NSString _PresenceType { get; set; }
PersonRelationship PresenceType {
[Wrap ("PersonRelationshipExtensions.GetValue (_PresenceType)")]
get;
[Wrap ("_PresenceType = value.GetConstant ()")]
set;
}
```
* Fix Feedback
* Fix doc error
* Update error message
Moving to reference sources added a few, new converters so the existing
logic to preserve them was not complete.
This update the list of converters, sorted like the reference sources
(RS) for easier reviews.
It also adds:
* a canary test in "dont link" that will fail it the RS code change
or is replaced;
* more complete unit tests to ensure all cases works
https://github.com/xamarin/xamarin-macios/issues/3372
* [registrar] Register models in the static registrar.
This also means we need to quiet a few types of warnings:.
* Models declares virtual methods of required protocol members. We don't
export virtual methods (only when they're overridden are they exported),
which results in numerous warnings about protocol members not being
implemented:
Xamarin.Mac.registrar.mobile.x86_64.m:37827:17: warning: method 'deviceBrowserView:selectionDidChange:' in protocol 'IKDeviceBrowserViewDelegate' not implemented [-Wprotocol]
@implementation ImageKit_IKDeviceBrowserView__IKDeviceBrowserViewDelegate {
^
/Applications/Xcode92.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.13.sdk/System/Library/Frameworks/Quartz.framework/Frameworks/ImageKit.framework/Headers/IKDeviceBrowserView.h:29:1: note: method 'deviceBrowserView:selectionDidChange:' declared here
- (void)deviceBrowserView: (IKDeviceBrowserView *)deviceBrowserView selectionDidChange: (ICDevice *)device;
* These two are the same as above, just for properties instead of methods.
Xamarin.Mac.registrar.mobile.x86_64.m:31988:17: warning: auto property synthesis will not synthesize property 'boundingMapRect' declared in protocol 'MKOverlay' [-Wobjc-protocol-property-synthesis]
@implementation MKOverlay {
^
/Applications/Xcode92.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.13.sdk/System/Library/Frameworks/MapKit.framework/Headers/MKOverlay.h:24:43: note: property declared here
@property (nonatomic, readonly) MKMapRect boundingMapRect;
^
Xamarin.Mac.registrar.mobile.x86_64.m:32002:1: note: add a '@synthesize' directive
@end
^
Xamarin.Mac.registrar.mobile.i386.m:28957:17: warning: property 'repeatCount' requires method 'repeatCount' to be defined - use @synthesize, @dynamic or provide a method implementation in this class implementation [-Wobjc-property-implementation]
@implementation CAMediaTiming {
^
/Applications/Xcode92.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.13.sdk/System/Library/Frameworks/QuartzCore.framework/Headers/CAMediaTiming.h:58:17: note: property declared here
@property float repeatCount;
^
* [AVFoundation] Special-case AVCaptureDataOutputSynchronizer[Delegate] in the registrar for macOS.
This class and protocol were incorrectly added to our macOS bindings, but
since we can't remove them because it would break backwards compatibility, we
must skip them manually in the registrar, since the registrar would otherwise
produce uncompilable code:
In file included from Xamarin.Mac.registrar.full.x86_64.m:2:
./Xamarin.Mac.registrar.full.x86_64.h:2929:63: error: 'AVCaptureDataOutputSynchronizerDelegate' is unavailable: not available on macOS
@interface AVCaptureDataOutputSynchronizerDelegate : NSObject<AVCaptureDataOutputSynchronizerDelegate> {
^
/Applications/Xcode92.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.13.sdk/System/Library/Frameworks/AVFoundation.framework/Headers/AVCaptureDataOutputSynchronizer.h:101:11: note: 'AVCaptureDataOutputSynchronizerDelegate' has been explicitly marked unavailable here
@protocol AVCaptureDataOutputSynchronizerDelegate <NSObject>
^
In file included from Xamarin.Mac.registrar.mobile.x86_64.m:2:
./Xamarin.Mac.registrar.mobile.x86_64.h:3370:63: error: 'AVCaptureDataOutputSynchronizerDelegate' is unavailable: not available on macOS
@interface AVCaptureDataOutputSynchronizerDelegate : NSObject<AVCaptureDataOutputSynchronizerDelegate> {
* [AVFoundation] Stub out AVCaptureDataOutputSynchronizer[Delegate] on macOS.
AVCaptureDataOutputSynchronizer[Delegate] were incorrectly added to our macOS
bindings, which makes the static registrar's life difficult. So remove those
bindings, and re-implemented them as normal classes, without any attributes,
which makes the static registrar ignore them (to a certain extent: enough to
not generate uncompilable code at least).
* [registrar] Remove more model exclusion code.
* [xtro] Update ignored entries.
* [tests] Move linker tests to match introspection directory layout.
Move linker tests to match introspectio directory layout: tests/linker/ios and
tests/linker/mac instead of tests/linker-ios and tests/linker-mac.
This creates a logical place for shared linker files (tests/linker).
* [tests] Fix path to GuiUnit_NET_4_5.csproj in sln as well.
This is the path used to find referenced projects when building Classic.
Fixes this build problem:
warning: Referenced project 'GuiUnit_NET_4_5' not found in the solution.
/Users/builder/jenkins/workspace/xamarin-macios-pr-builder/tests/common/mac/MacTestMain.cs(11,7) : error CS0246: The type or namespace name `GuiUnit' could not be found. Are you missing an assembly reference?
/Users/builder/jenkins/workspace/xamarin-macios-pr-builder/tests/common/mac/MacTestMain.cs(64,42) : error CS0246: The type or namespace name `IMainLoopIntegration' could not be found. Are you missing an assembly reference?
/Users/builder/jenkins/workspace/xamarin-macios-pr-builder/tests/common/mac/MacTestMain.cs(75,34) : error CS0246: The type or namespace name `InvokerHelper' could not be found. Are you missing an assembly reference?
* [tests] Build the native test library for macOS and create a binding project for it.
Also add the new binding project to the xammac and link all XM test projects,
which allows us to stop excluding tests that require the native library and
the corresponding bindings.
* [tests] Include more tests in xammac_tests.
* [tests] Correctly ignore the ObjC exception tests in release mode.
* [tests] Share supporting code between the mtouch and mmp tests.
Create a new class 'BundlerTool', which now contains most of the code in
MTouchTool that's also applicable to mmp (and the new MmpTool class).
This will make it easier to share tests between the mtouch and mmp tests.
Some tweaks are still probably required, but this should get us most of the
way.
* [tests] Fix generator tests after changes in shared test code.
* [tests] Add new file to the MSBuild/XM tests.
This file is included in several projects, some projects use property, some
don't (and report the warning). There's no harm in not setting this property
(it's expected), ignore the warning.
This won't affect device tests on the bots (because those already set LLVM
manually when testing Release), but it becomes less confusing when trying to
reproduce any problems locally, since now the project configuration on disk
matches the tested configuration.
Fixes this:
tests/common/mac/ProjectTestHelpers.cs(154,22): warning CS0436: The type 'StringUtils' in 'tests/mmptest/../../tools/common/StringUtils.cs' conflicts with the imported type 'StringUtils' in 'mmp, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null'. Using the type defined in 'tests/mmptest/../../tools/common/StringUtils.cs'.
tests/common/Configuration.cs(162,53): warning CS0436: The type 'StringUtils' in 'tests/mmptest/../../tools/common/StringUtils.cs' conflicts with the imported type 'StringUtils' in 'mmp, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null'. Using the type defined in 'tests/mmptest/../../tools/common/StringUtils.cs'.
* [Foundation] Simplify NSValue code to use [No*] instead of ifdefs.
Also enable the MKCoordinateSpan and CLLocationCoordinate2D values for NSValue
in macOS, since they're now available (since macOS 10.9).
Type Changed: Foundation.NSValue
Added properties:
public virtual MapKit.MKCoordinateSpan CoordinateSpanValue { get; }
public virtual CoreLocation.CLLocationCoordinate2D CoordinateValue { get; }
* [Foundation] Mark the new NSValue selectors as 64-bit only.
* [xharness] Add support for running XI projects with different platform configurations.
* [xharness] Run dont link, link all and link sdk tests in both Debug and Release in the simulator. Fixes#53181.
Fixes https://bugzilla.xamarin.com/show_bug.cgi?id=53181.
* [tests] Fix dont link to always optimize C# code in release mode.
* [tests] No need to weak link GameController anymore in dont link.
This was removed long ago from the Debug configurations, so remove it from the
Release configuration as well.
- Obsolete `ARFaceAnchor`'s default constructor because it's marked as unavailable.
- Obsolete `CreateFaceGeometry` in `ARSCNFaceGeometry` in favor of `Create` (same as `ARSCNPlaneGeometry`).
* [xharness] Ask mono to not attach gdb/ldb when XM tests crash.
Attaching gdb/ldb usually hangs, thus causing the test run to time out instead
of finishing early (due to the crash).
Also bump the timeout for finding crash reports to 60 seconds when the tests
crash, since macOS can be quite slow to create the crash report.
* [xharness] Timestamp mac logs to aid diagnostics.
We must call VerifyRun for each subtest in the aggregated simulator test run
in order to mark those subtests as failed if there's no available device.
Fixes https://github.com/xamarin/maccore/issues/623.
This class and protocol were incorrectly added to our macOS bindings.
The catch is that we can't remove them because it would break backwards
compatibility, so mark them for removal in XAMCORE_4_0.
* [ObjCRuntime] Add a BindingImplAttribute.
* [linker] Make ProviderToString an extension method on ICustomAttributeProvider to make it more discoverable.
* [generator] Use [BindingImpl] instead of [CompilerGenerated].
The entire diff is big (89MB), so it can't be gisted. However, most of it is
either removal of `using System.Runtime.CompilerServices;` or the change from
`[CompilerGenerated]` to `[BindingImpl (...)]` like this:
https://gist.github.com/rolfbjarne/8bfda3ed37b956d0342a1c1e9b079244
If I remove those parts of the diff, there's nothing significant left:
https://gist.github.com/rolfbjarne/4156164d6bdb1376366200394eb8a091
* [linker] Teach the linker about the new [BindingImpl] attribute.
In addition to the existing logic where the linker would optimize some
[CompilerGenerated] code (sometimes with additional requirements), it will now
also optimize all [BindingImpl (Optimizable)] code (without any additional
requirements).
* [tests] Add tests to make sure [BindingImpl (Optimizable)] works as expected.
* [linker] Check for [BindingImpl] before [CompilerGenerated] and stop checking for [CompilerGenerated] in XAMCORE_4_0.
Check for [BindingImpl] before checking for [CompilerGenerated], since the
former is more common.
Also stop checking for [CompilerGenerated] (at least to mean that code is
optimizable) in our next non-compatible evolutionary leap (XAMCORE_4_0):
* [introspection] Impl a better typo check.
Fixes this test failure:
1) Failed : Xamarin.MMP.Tests.MMPTests.UnifiedWithLinking_ShouldHaveFewFrameworkClangLines
Found more framework entries in clang invocation then expected - Foundation AppKit QuartzCore CoreData Quartz CoreFoundation CoreServices Security Carbon CloudKit
xcrun -sdk macosx clang -mmacosx-version-min=10.7 -arch x86_64 -fobjc-runtime=macosx -Wno-unguarded-availability-new -ObjC -framework Foundation -framework AppKit -framework QuartzCore -framework CoreData -framework Quartz -framework CoreFoundation -framework CoreServices -framework Security -framework Carbon -weak_framework CloudKit -u _xamarin_timezone_get_data -u _xamarin_get_block_descriptor /work/maccore/master/xamarin-macios/_mac-build/Library/Frameworks/Xamarin.Mac.framework/Versions/Current/lib/libxammac.a -o /work/maccore/master/xamarin-macios/tests/mmptest/bin/Debug/tmp-test-dir/Xamarin.MMP.Tests.MMPTests.RunMMPTest/bin/Debug/UnifiedExample.app/Contents/MacOS/UnifiedExample -D_THREAD_SAFE -I/work/maccore/master/xamarin-macios/_mac-build/Library/Frameworks/Xamarin.Mac.framework/Versions/Current/lib/pkgconfig/../../include/mono-2.0 /work/maccore/master/xamarin-macios/_mac-build/Library/Frameworks/Xamarin.Mac.framework/Versions/Current/lib/pkgconfig/../../lib/libmonosgen-2.0.a -liconv -x objective-c++ -I/work/maccore/master/xamarin-macios/_mac-build/Library/Frameworks/Xamarin.Mac.framework/Versions/Current/include /work/maccore/master/xamarin-macios/tests/mmptest/bin/Debug/tmp-test-dir/Xamarin.MMP.Tests.MMPTests.RunMMPTest/obj/Debug/mmp-cache/registrar.m -fno-caret-diagnostics -fno-diagnostics-fixit-info -isysroot /Applications/Xcode92.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.13.sdk /work/maccore/master/xamarin-macios/tests/mmptest/bin/Debug/tmp-test-dir/Xamarin.MMP.Tests.MMPTests.RunMMPTest/obj/Debug/mmp-cache/main.m
Expected: less than 10
But was: 10
This regressed in 4da8016db4, where new API is
partially unremovable by the linker, and as such causes XM to link with more
frameworks at build time.
This is expected, so change the test to accept more frameworks (11).
Fixes https://github.com/xamarin/maccore/issues/615.
Fixes this test failure on High Sierra:
Xamarin.Mac.Tests.EveryFrameworkSmokeTests.ExpectedLibrariesAreLoaded: CoreSpotlightLibrary (/System/Library/Frameworks/CoreSpotlight.framework/CoreSpotlight) failed to load but this was not expected
Make sure to package the linker tests when packaging XM tests (this broke
recently because the dontlink tests were moved).
Fixes this problem when running the packaged tests:
make[2]: linker-mac/dont link/bin/x86/Debug/dont link.app/Contents/MacOS/dont link: No such file or directory
make[2]: *** [exec-mac-classic-dont link] Error 1
make[2]: linker-mac/dont link/bin/x86/Debug-unified/dont link.app/Contents/MacOS/dont link: No such file or directory
make[2]: *** [exec-mac-unified-dont link] Error 1
make[2]: linker-mac/dont link/bin/x86/Debug-unifiedXM45/dont link.app/Contents/MacOS/dont link: No such file or directory
make[2]: *** [exec-mac-unifiedXM45-dont link] Error 1
make[2]: linker-mac/dont link/bin/x86/Debug-unified-32/dont link.app/Contents/MacOS/dont link: No such file or directory
make[2]: *** [exec-mac-unified32-dont link] Error 1
make[2]: linker-mac/dont link/bin/x86/Debug-unifiedXM45-32/dont link.app/Contents/MacOS/dont link: No such file or directory
make[2]: *** [exec-mac-unifiedXM4532-dont link] Error 1
exec-mac-classic-dont\ link failed
exec-mac-unified-dont\ link failed
exec-mac-unifiedXM45-dont\ link failed
exec-mac-unified32-dont\ link failed
exec-mac-unifiedXM4532-dont\ link failed
make[1]: *** [exec-mac-dontlink] Error 1
* [mtouch/mmp] Give users more control over optimizations, and share more code between mtouch and mmp.
1. Add an --optimize flag to mtouch/mmp that allows users to select which
optimizations to apply (or not). This makes it easier to add future
optimizations, and allow users to disable any optimization that causes
problems without having to disable many other features.
2. Share as much optimization code as possible between mtouch and mmp. This
immediately gives a benefit to mmp, which has three new optimizations only
mtouch had: NSObject.IsDirectBinding inlining, IntPtr.Size inlining and
dead code elimination.
This results in ~6kb of disk space saved for a linked Xamarin.Mac app:
* link sdk: [Debug][1], [Release][2]
* link all: [Debug][3], [Release][4]
Testing also verifies that monotouchtest ([Debug][5], [Release][6]) has not
changed size at all, which means that no default optimizations have changed
inadvertedly.
[1]: https://gist.github.com/rolfbjarne/6b731e3b5ca6170355662e6505c3d492#link-sdk-mac--debug
[2]: https://gist.github.com/rolfbjarne/6b731e3b5ca6170355662e6505c3d492#link-sdk-mac--release
[3]: https://gist.github.com/rolfbjarne/6b731e3b5ca6170355662e6505c3d492#link-all-mac--debug
[4]: https://gist.github.com/rolfbjarne/6b731e3b5ca6170355662e6505c3d492#link-all-mac--release
[5]: https://gist.github.com/rolfbjarne/6b731e3b5ca6170355662e6505c3d492#monotouchtest-iphonedebug64
[6]: https://gist.github.com/rolfbjarne/6b731e3b5ca6170355662e6505c3d492#monotouchtest-iphonerelease64
* [tools] Don't enable the IsDirectBinding optimization by default for Xamarin.Mac apps, it's not safe.
* Fix whitespace issues.
* [doc] Document optimizations.
* Officially support optimizations by adding them to the Versions.plist.
* [linker] Improve IntPtr.Size inliner + dead code eliminatior and add tests.
* Properly handle operands for the ldc_i4_s instruction (they're sbyte).
* Fix less-than condition to actually do a less-than comparison.
* Make sure to look up the bitness in the Target, not the Application, since
the Application's value will be incorrect when building fat apps (both
Is32Build and Is64Build will be true).
* Remove unnecessary checks for the IntPtr.Size inliner: this optimization
does not depend on other instructions than the IntPtr.get_Size call, so
remove the checks that verify surrounding instructions. This makes the
IntPtr.Size inliner kick in in more scenarios (such as the new tests).
* Add tests.
* [tests] Add mmp tests for optimizations.
* [tests] Fix XM optimization tests.
* [tests] Fix test build error.
* [mtouch] Make sure the xamarin_localized_string_format* functions are available in simlauncher. Fixes#3265.
Fixes https://github.com/xamarin/xamarin-macios/issues/3265.
* [mtouch] Add test for simlauncher symbols, add add more missing symbols.
Fixes this rather puzzling build problem when building monotouch-test using xharness:
obj/iPhoneSimulator/Debug-unified/Xamarin.iOS,Version=v1.0.AssemblyAttribute.cs(2,12): error CS0579: Duplicate 'global::System.Runtime.Versioning.TargetFrameworkAttribute' attribute
It happens only on Wrench (where it consistently fails), but it didn't happen
in the PR when this broke, nor could I reproduce it locally at first.
Initial investigation proved frustrating (searching for the file name
`Xamarin.iOS,Version=v1.0.AssemblyAttribute.cs` in the build log confirmed
that it was included multiple times on Wrench, but not locally), but after a
while I realized that there were multiple files with the same file name in the
build log:
/Users/builder/data/lanes/1381/daf07005/source/xamarin-macios/tests/monotouch-test/obj/iPhone/Release64-unified/Xamarin.iOS,Version=v1.0.AssemblyAttribute.cs
obj/iPhoneSimulator/Debug-unified/Xamarin.iOS,Version=v1.0.AssemblyAttribute.cs
One full path, and one relative, and one is obviously wrong.
Even more so once I realized that the full path's Platform/Configuration
(`iPhone/Release64-unified`) doesn't match the relative path's
Platform/Configuration (`iPhoneSimulator/Debug-unified`).
It's not even a test configuration we execute on Wrench, so how come the file
got there in the first place?
But it made it possible to reproduce locally: I just had to build monotouch-
test for device first, using tests.sln. Then I could reproduce the build error
when executing monotouch-test from xharness, since now I had the extraneous
file on my file system.
xharness will clone project files (to avoid parallel issues when building
multiple projects in parallel, all projects are built in their own
directories), and it turns out we didn't properly fixup all paths.
In particular the following:
<Compile Include="**\*.cs" Exclude="obj\**">
<Link>%(RecursiveDir)%(Filename).cs</Link>
</Compile>
would be transformed into:
<Compile Include="\Users\builder\data\lanes\1381\daf07005\source\xamarin-macios\tests\monotouch-test\**\*.cs" Exclude="obj\**">
<Link>%(RecursiveDir)%(Filename).cs</Link>
</Compile>
which meant that we'd include all files recursively in the monotouch-test
directory, excluding files from the project file's obj sub directory. And
since the project file was cloned, it would be in a different directory, and
thus not able to exclude files from the original monotouch-test/obj directory.
And with this knowledge the simple oneliner to fix it was trivial.
One mystery remained though: how was the extraneous
`Xamarin.iOS,Version=v1.0.AssemblyAttribute.cs` file generated when we don't
execute device tests on Wrench? After a while I remembered that the MTouch
tests build monotouch-test for device, even though they're not executed.
And this explains why the problem happens on wrench only: Jenkins didn't
execute the MTouch tests.
The original, now obsoleted, `LocalizedString` API returned a .net
`string` which does not work in most cases.
Different versions of iOS seems to return different (public or internal)
subclasses of `NSString` that are understood by other API (like NSString
`localizedStringWithFormat:`) for further customization.
Our logic to convert NSString to string is correct but it cannot
recreate the custom, required subclass to continue the localization.
So the new API return an `NSString` publicly (which is actually a
subclass) that can do the required job.
Adding a test in monotouch-test is presently blocked by #3265 [2]
[1] https://bugzilla.xamarin.com/show_bug.cgi?id=41292
[2] https://github.com/xamarin/xamarin-macios/issues/3265
* Add tests for new (and old) NSBundle API and adjust old ones since adding a Base.lproj directories changes things
* Add localization to xammac_tests since it shares the same, updated tests
When determining the color for a collection of tests, first check if the
execution result is identical for all tests in the collection, in which case
just use the corresponding color for a single test.
- Fixes bug #59272: [xtro] Report !missing-protocol-conformance! when protocols are defined in categories
(https://bugzilla.xamarin.com/show_bug.cgi?id=59272)
- Implemented missing protocol conformances based on tool's new data.
- Remove Internal check from VisitObjCCategoryDecl and VisitObjCInterfaceDecl
- Ignore previewItemTitle failure (normal since it's optional)
- Only skip UIStateRestoring for subclasses of UIViewController
- Ignore UIStateRestoring test on watchOS (UIViewController not available)
- Remove protocol conformances that generated wrong availability attributes (https://github.com/xamarin/xamarin-macios/issues/3213)
- Avoid new virtual or virtual when adding protocol conformance (https://github.com/xamarin/xamarin-macios/issues/3217)
xharness needs a solution in order to ask for a nuget restore, so make sure to
provide the path to the solution.
This fixes a build issue where the install source tests would fail to build
due to picking up the system's nunit.framework.dll because the nuget one
wasn't found/restored:
MonoPathManglerTest.cs(8,3): error CS0619: 'TestFixtureAttribute' is obsolete: 'The NUnit framework shipped with Mono is deprecated and will be removed in a future release. It was based on NUnit 2.4 which is long outdated. Please move to the NUnit NuGet package or some other form of acquiring NUnit.'
XamarinSourcesPathManglerTest.cs(8,3): error CS0619: 'TestFixtureAttribute' is obsolete: 'The NUnit framework shipped with Mono is deprecated and will be removed in a future release. It was based on NUnit 2.4 which is long outdated. Please move to the NUnit NuGet package or some other form of acquiring NUnit.'
OpenTKManglerTest.cs(8,3): error CS0619: 'TestFixtureAttribute' is obsolete: 'The NUnit framework shipped with Mono is deprecated and will be removed in a future release. It was based on NUnit 2.4 which is long outdated. Please move to the NUnit NuGet package or some other form of acquiring NUnit.'
PathManclerFactoryTests.cs(8,3): error CS0619: 'TestFixtureAttribute' is obsolete: 'The NUnit framework shipped with Mono is deprecated and will be removed in a future release. It was based on NUnit 2.4 which is long outdated. Please move to the NUnit NuGet package or some other form of acquiring NUnit.'
OpenTKManglerTest.cs(29,4): error CS0616: 'TestCase' is not an attribute class
OpenTKManglerTest.cs(30,4): error CS0616: 'TestCase' is not an attribute class
MonoPathManglerTest.cs(29,4): error CS0616: 'TestCase' is not an attribute class
MonoPathManglerTest.cs(30,4): error CS0616: 'TestCase' is not an attribute class
MonoPathManglerTest.cs(31,4): error CS0616: 'TestCase' is not an attribute class
XamarinSourcesPathManglerTest.cs(33,4): error CS0616: 'TestCase' is not an attribute class
XamarinSourcesPathManglerTest.cs(35,4): error CS0616: 'TestCase' is not an attribute class
XamarinSourcesPathManglerTest.cs(37,4): error CS0616: 'TestCase' is not an attribute class
OpenTKManglerTest.cs(36,4): error CS0616: 'TestCase' is not an attribute class
OpenTKManglerTest.cs(37,4): error CS0616: 'TestCase' is not an attribute class
MonoPathManglerTest.cs(37,4): error CS0616: 'TestCase' is not an attribute class
MonoPathManglerTest.cs(38,4): error CS0616: 'TestCase' is not an attribute class
MonoPathManglerTest.cs(39,4): error CS0616: 'TestCase' is not an attribute class
XamarinSourcesPathManglerTest.cs(47,4): error CS0616: 'TestCase' is not an attribute class
These targets will already be properly escaped (using backslash), so adding
quotes will cause make to treat the backslashes literally (and break).
Fixes an issue with package-tests:
$ make package-tests
[...]
build-mac-classic-dont\ link failed
build-mac-unified-dont\ link failed
build-mac-unifiedXM45-dont\ link failed
build-mac-unified32-dont\ link failed
build-mac-unifiedXM4532-dont\ link failed
build-mac-unified-link\ all failed
build-mac-unified-link\ sdk failed
make[4]: *** [build-mac] Error 1
make[3]: *** [mac-test-package.zip] Error 2
make[2]: *** [package-tests] Error 2
* [xharness] Run monotouch-test both with and without the static registrar in the simulator.
This way we catch (some) problems with the static registrar without having to run on device.
* [xharness] Add comments to explain extra test configurations.
* [xharness] Fix how projects are cloned.
Fix how projects are cloned so that xharness doesn't end up confused when
multiple test tasks share the same build task.
Previously xharness would iterate over all test tasks and modifying their
build tasks, now it will create the build tasks correctly in the first place.
Rolf Bjarne Kvinge
Alex Soto
Chris Hamons
Vincent Dondain
Sebastien Pouliot
Manuel de la Pena
mdbech
Ludovic Henry
Alex Soto