This also centralize other interpreter checks and options in the same
location (making it easier to read / update).
* Warn and switch the REPL if the interpreter is enabled on simulator
Why ? It's confusing to build the same code using different options for
simulator and devices. This is what happens if you try to use features
like `dynamic` or `System.Reflection.Emit`.
So instead of an error, we warn that the interpreter is not supported
and switch to the existing REPL mode.
The JIT remains the only option for the simulator but it allows testing
features without a device.
* Fail early if the interpreter is used on 32bits [1]
The current interpreter only works on 64 bits (so ARM64). However the
error won't be reported, back to the developer, until deployment time.
This temporary [1] fix spot the condition very early and report an error
```
error MT0099 : Internal error : The interpreter is currently only available for 64 bits.
```
instead of the current one at deploy time
```
IncorrectArchitecture: Failed to find matching arch for 32-bit Mach-O input file /private/var/installd/Library/Caches/com.apple.mobile.installd.staging/temp.tNKDlx/extracted/X.app/X
error MT1006: Could not install the application 'X.app' on the device 'Mercure': AMDeviceSecureInstallApplicationBundle returned: 0xe8000087 (kAMDIncorrectArchitectureError).
Application could not be uploaded to the device.
```
[1] https://github.com/mono/mono/issues/9871
* [tests] Fix/renumbered MT0138
The test was using simulator + interpreter which is not _really_
possible, we use REPL in that case - so we're now checking if
assemblies were specified with `--interpreter` to cover both cases.
Also 0138 was already used by `mmp` and the warning was **not**
registered or documented in the errors documents. To avoid
confusion it has been renumbered to 0142 and documented.
* MachO.cs: Support reading LC_BUILD_VERSION
Newer SDKs set this instead of LC_VERSION_MIN_*
* MachO.cs: Add support for reading Mach-O files inside ar archives.
* [tests] Augment ProductTests.MinOSVersion to test static libraries as well.
* Adjust enum field names to match our naming scheme.
* [linker] Remove non-bitcode compatible code, and show a warning.
Remove code not currently compatible with bitcode and replace it with an
exception instead (otherwise we'll assert at runtime).
Also show a warning when we detect this.
This is quite helpful when looking at watch device test runs to filter out
failures we already know about.
This fixes point #2 in #4763.
* Improve documentation.
* Simplify linker code by using a substep.
* Fix whitespace issues.
* Improve reporting.
* Add support for reporting more than one MT2105 at the same time when making
the errors instead of warnings.
* Only report MT2105 for methods that haven't been linked away.
* Format the error message nicer for properties.
* Tweak a bit for warning tests to pass.
* Use ExceptionalSubStep to provide better error information.
* Adjust where linker warnings/errors are reported from to avoid a NullReferenceException.
* Make PreserveSmartEnumConversionsSubStep use error codes that are not
already used elsewhere: this isn't obvious at first, but all
ExceptionalSubStep errors use a range of error numbers (10 numbers from NNN0
to NNN9), so we need to reserve that entire range. In this case there were
other errors using some of the numbers of the range for PreserveSmartEnumConversionsSubStep.
* Make sure there are anchor links for all the variations of each ExceptionalSubStep error.
Before
```
clang : error : no such file or directory: '/Users/poupou/git/xamarin/xamarin-macios/_ios-build/Library/Frameworks/Xamarin.iOS.framework/Versions/git/SDKs/MonoTouch.iphonesimulator.sdk/usr/lib/libmono-ee-interp.a'
clang : error : no such file or directory: '/Users/poupou/git/xamarin/xamarin-macios/_ios-build/Library/Frameworks/Xamarin.iOS.framework/Versions/git/SDKs/MonoTouch.iphonesimulator.sdk/usr/lib/libmono-icall-table.a'
clang : error : no such file or directory: '/Users/poupou/git/xamarin/xamarin-macios/_ios-build/Library/Frameworks/Xamarin.iOS.framework/Versions/git/SDKs/MonoTouch.iphonesimulator.sdk/usr/lib/libmono-ilgen.a'
error MT5209 : Native linking error : clang: error: no such file or directory: '/Users/poupou/git/xamarin/xamarin-macios/_ios-build/Library/Frameworks/Xamarin.iOS.framework/Versions/git/SDKs/MonoTouch.iphonesimulator.sdk/usr/lib/libmono-ee-interp.a'
error MT5209 : Native linking error : clang: error: no such file or directory: '/Users/poupou/git/xamarin/xamarin-macios/_ios-build/Library/Frameworks/Xamarin.iOS.framework/Versions/git/SDKs/MonoTouch.iphonesimulator.sdk/usr/lib/libmono-icall-table.a'
error MT5209 : Native linking error : clang: error: no such file or directory: '/Users/poupou/git/xamarin/xamarin-macios/_ios-build/Library/Frameworks/Xamarin.iOS.framework/Versions/git/SDKs/MonoTouch.iphonesimulator.sdk/usr/lib/libmono-ilgen.a'
MTOUCH : error MT5202: Native linking failed. Please review the build log.
0 Warning(s)
7 Error(s)
```
After
```
MTOUCH : error MT0141: The interpreter is not supported in the simulator. Do not pass --interpreter when building for the simulator.
0 Warning(s)
1 Error(s)
```
Fixesxamarin/xamarin-macios#4107
We currently generate invalid registrar code when we get invalid input
in 'Adopts' attribute, this can happen because 'Adopts' can take a
plain string. We now generate a better error MT4177 instead of
generating invalid registrar.m/h code.
Instead of a generic `MT0000` caused by an exception reading the magic
numbers of the binary framework file.
This was caused be uncompressing an archive, with a symlink, into a
file system that does not support symlinks (on Windows).
ref: https://github.com/xamarin/xamarin-macios/issues/5028
* [runtime] Throw a managed exception instead of trying to call a null function pointer if the runtime tries to call a function that has been linked away.
* [tests] Add new file for Xamarin.Mac tests.
* Direct people to file issues in github.
* Add support for delegates as return values in protocol members. Fixes#4102.
This required a few changes:
* The generator now emits the DelegateProxy attribute for property getters in
protocol interfaces.
* The generator now emits the DelegateProxy attribute in ProtocolMember
attributes (and the ProtocolMember attribute has been extended with
additional properties for this purpose).
* The generator now emits the BlockProxy attribute for the parameter in
property setters.
* The generator now emits the BlockProxy attribute in ProtocolMember
attributes for property setters.
* The static registrar now emits the metadata token for the
DelegateProxy.DelegateType property into the generated code so that the
DelegateProxy attribute itself isn't needed at runtime. This is required
when the dynamic registrar has been optimized away.
Fixes https://github.com/xamarin/xamarin-macios/issues/4102.
* [tests] Update MX4105 test to expect new warnings.
* [mtouch] Show warnings when we can't find referenced assemblies.
This would have helped track down #4235.
* Improve MT0137 warning to indicate the type of the attribute causing the warning.
* Convert it into a MM8027/MT8027 error (with documentation).
* Add information about the selector and managed method that triggered the error.
Now the problem reported in issue #4254 shows up as:
> ObjCRuntime.RuntimeException: Failed to marshal the Objective-C object 0x7f8080412810 (type: UIView). Could not find an existing managed instance for this object, nor was it possible to create a new managed instance (because the type 'UIKit.UIView&' does not have a constructor that takes one IntPtr argument).
> Additional information:
> Selector: popoverController:willRepositionPopoverToRect:inView:
> Method: UIKit.UIPopoverController+_UIPopoverControllerDelegate.WillReposition(UIKit.UIPopoverController, CoreGraphics.CGRect ByRef, UIKit.UIView ByRef)
instead of just:
> ObjCRuntime.RuntimeException: Failed to marshal the Objective-C object 0x7f8080412810 (type: UIView). Could not find an existing managed instance for this object, nor was it possible to create a new managed instance (because the type 'UIKit.UIView&' does not have a constructor that takes one IntPtr argument).
which makes it much easier to understand, track down, and fix/work around,
both for customers and ourselves.
* [mtouch/mmp] Handle invalid types in BlockProxy attributes better. Fixes#4072.
BlockProxy attributes may have types we don't expect, so handle those cases
gracefully by showing a warning when we encounter them:
testApp.cs(11): warning MT4175: The parameter 'completionHandler' in the method 'Issue4072Session.CreateDataTask(Foundation.NSUrl,Foundation.NSUrlSessionResponse)' has an invalid BlockProxy attribute (the type passed to the attribute does not have a 'Create' method).
instead of an ugly MT0000:
MTOUCH : error MT0000: Unexpected error - Please file a bug report at http://bugzilla.xamarin.com
System.InvalidOperationException: Sequence contains no matching element
at System.Linq.Enumerable.First[TSource] (System.Collections.Generic.IEnumerable`1[T] source, System.Func`2[T,TResult] predicate) [0x00012] in /Users/builder/jenkins/workspace/build-package-osx-mono/2017-12/external/bockbuild/builds/mono-x64/external/corefx/src/System.Linq/src/System/Linq/First.cs:30
at Registrar.StaticRegistrar.GetBlockProxyAttributeMethod (Mono.Cecil.MethodDefinition method, System.Int32 parameter) [0x00020] in /Users/builder/data/lanes/5944/7e782c1e/source/xamarin-macios/tools/common/StaticRegistrar.cs:4121
at Registrar.StaticRegistrar.GetBlockWrapperCreator (Registrar.Registrar+ObjCMethod obj_method, System.Int32 parameter) [0x00011] in /Users/builder/data/lanes/5944/7e782c1e/source/xamarin-macios/tools/common/StaticRegistrar.cs:4065
at Registrar.StaticRegistrar.Specialize (Registrar.AutoIndentStringBuilder sb, Registrar.Registrar+ObjCMethod method, System.Collections.Generic.List`1[T] exceptions) [0x0216b] in /Users/builder/data/lanes/5944/7e782c1e/source/xamarin-macios/tools/common/StaticRegistrar.cs:3683
at Registrar.StaticRegistrar.Specialize (Registrar.AutoIndentStringBuilder sb) [0x00f1e] in /Users/builder/data/lanes/5944/7e782c1e/source/xamarin-macios/tools/common/StaticRegistrar.cs:2963
Fixes#4072.
* [docs] Improve text for MT4175.
* [registrar] Comment some code.
- https://github.com/xamarin/xamarin-macios/issues/3725
- These frameworks "CoreAudioKit Metal MetalKit MetalPerformanceShaders CoreNFC DeviceCheck"
were special cased, but that special case did do an SDK check.
- Create a helper method to share check
- Add test for MM0135
- https://github.com/xamarin/xamarin-macios/issues/3725
- These frameworks "CoreAudioKit Metal MetalKit MetalPerformanceShaders CoreNFC DeviceCheck"
were special cased, but that special case did do an SDK check.
- Create a helper method to share check
- Add test for MM0135
The main cause of the warning: "warning MSB6002: The command-line for the 'MTouch' task is too long" was the number of references that can be passed to `mtouch`.
Now all the arguments are written in a response file which in turn is passed to `mtouch`.
- Fixes bug #56501: MSB6002 command-line for MTouch task is too long, > 32000 characters
(https://bugzilla.xamarin.com/show_bug.cgi?id=56501)
- The response file is created in the device specific folder, as `response-file.rsp`. It is re-created every time.
- Added an msbuild test to ensure the response file is created by `GenerateCommandLineCommands` and that it includes all the references.
- Introduce `AddLine` and `AddQuotedLine` which we use to populate the response file (and go to the next line).
- Move to C# 7 syntax for string replacement.
- Update all tests in `MTouchTaskTests` to use the response file since the arguments are no longer passed directly to `mtouch`.
- Update MT0018's documentation for response files.
* [static registrar] Optimize creation of delegates for blocks.
Optimize creation of delegates for blocks so that it doesn't require the
dynamic registrar.
This is done by getting the metadata token for the Create method that creates
the delegate, and embed that metadata token in the generated code from the
static registrar.
Also add tests, since this scenario was not covered by tests already.
* [mmptest] Fix test after recent changes.
* [test-libraries] Avoid duplicate symbols.
* [tests] Update according to changes.
- https://github.com/xamarin/xamarin-macios/issues/3367
- App Store will now fail builds if you add in a 32-bit dylib
- If you are a 32-bit app you don't need the 64-bit part of your fat
dylib anyway
- Add --optimize=-trim-architectures to allow customization of behavior, as not everyone
uses app store
In addition, while writing tests for this is was noticed that mmp tests did not "really" run Release configuration correctly in most cases. Fixing this turned out to be a bit of a pain, but necessary to correctly test this (and other things).
- Turns out that /p:configuration:debug is not sufficient to tell mmp to
do the right thing
- That, in most projects, sets the DebugSymbols property, which really
is what is checked.
- However, two of our projects did not have that, so we always did
release mmp work.
- Removed configuration property for tests and added real "Release"
configuration option
* [static registrar] Optimize creation of delegates for blocks.
Optimize creation of delegates for blocks so that it doesn't require the
dynamic registrar.
This is done by getting the metadata token for the Create method that creates
the delegate, and embed that metadata token in the generated code from the
static registrar.
Also add tests, since this scenario was not covered by tests already.
* [mmptest] Fix test after recent changes.
* [test-libraries] Avoid duplicate symbols.
* [tests] Update according to changes.
- https://github.com/xamarin/xamarin-macios/issues/3367
- App Store will now fail builds if you add in a 32-bit dylib
- If you are a 32-bit app you don't need the 64-bit part of your fat
dylib anyway
- Add --optimize=-trim-architectures to allow customization of behavior, as not everyone
uses app store
In addition, while writing tests for this is was noticed that mmp tests did not "really" run Release configuration correctly in most cases. Fixing this turned out to be a bit of a pain, but necessary to correctly test this (and other things).
- Turns out that /p:configuration:debug is not sufficient to tell mmp to
do the right thing
- That, in most projects, sets the DebugSymbols property, which really
is what is checked.
- However, two of our projects did not have that, so we always did
release mmp work.
- Removed configuration property for tests and added real "Release"
configuration option
Generate table of protocols and their corresponding wrapper type in the static
registrar, so that we don't have to look for this information using the
dynamic registrar at runtime.
* [linker] Optimize calls to BlockLiteral.SetupBlock to inject the block signature.
Optimize calls to BlockLiteral.SetupBlock[Unsafe] to calculate the block
signature at build time, and inject it into the call site.
This makes block invocations 10-15x faster (I've added tests that asserts at
least an 8x increase).
It's also required in order to be able to remove the dynamic registrar code in
the future (since calculating the block signature at runtime requires the
dynamic registrar).
* [mtouch/mmp] Add support for reporting errors/warnings that point to the code line causing the error/warning.
Add support for reporting errors/warnings that point to the code line causing
the error/warning by adding ErrorHelper overloads that take the exact
instruction to report (previously we defaulted to the first line/instruction
in a method).
* [tests] Add support for asserting filename/linenumber in warning messages.
* Make all methods that manually create BlockLiterals optimizable.
* [tests] Create a BaseOptimizeGeneratedCodeTest test that's included in both XI's and XM's link all test.
* [tests] Add link all test (for both XI and XM) to test the BlockLiteral.SetupBlock optimization.
* [tests] Add mtouch/mmp tests for the BlockLiteral.SetupBlock optimization.
* [tests][linker] Make the base test class abstract, so tests in the base class aren't executed twice.
* [tests][linker] Don't execute linkall-only tests in linksdk.
The optimization tests only apply when the test assembly is linked, and that
only happens in linkall, so exclude those tests in linksdk.
* [tests][mmptest] Update test according to mmp changes.
Fixes these test failures:
1) Failed : Xamarin.MMP.Tests.MMPTests.MM0132("inline-runtime-arch")
The warning 'MM0132: Unknown optimization: 'inline-runtime-arch'. Valid optimizations are: remove-uithread-checks, dead-code-elimination, inline-isdirectbinding, inline-intptr-size.' was not found in the output:
Message #1 did not match:
actual: 'Unknown optimization: 'inline-runtime-arch'. Valid optimizations are: remove-uithread-checks, dead-code-elimination, inline-isdirectbinding, inline-intptr-size, blockliteral-setupblock.'
expected: 'Unknown optimization: 'inline-runtime-arch'. Valid optimizations are: remove-uithread-checks, dead-code-elimination, inline-isdirectbinding, inline-intptr-size.'
Message #2 did not match:
actual: 'Unknown optimization: 'inline-runtime-arch'. Valid optimizations are: remove-uithread-checks, dead-code-elimination, inline-isdirectbinding, inline-intptr-size, blockliteral-setupblock.'
expected: 'Unknown optimization: 'inline-runtime-arch'. Valid optimizations are: remove-uithread-checks, dead-code-elimination, inline-isdirectbinding, inline-intptr-size.'
2) Failed : Xamarin.MMP.Tests.MMPTests.MM0132("foo")
The warning 'MM0132: Unknown optimization: 'foo'. Valid optimizations are: remove-uithread-checks, dead-code-elimination, inline-isdirectbinding, inline-intptr-size.' was not found in the output:
Message #1 did not match:
actual: 'Unknown optimization: 'foo'. Valid optimizations are: remove-uithread-checks, dead-code-elimination, inline-isdirectbinding, inline-intptr-size, blockliteral-setupblock.'
expected: 'Unknown optimization: 'foo'. Valid optimizations are: remove-uithread-checks, dead-code-elimination, inline-isdirectbinding, inline-intptr-size.'
Message #2 did not match:
actual: 'Unknown optimization: 'foo'. Valid optimizations are: remove-uithread-checks, dead-code-elimination, inline-isdirectbinding, inline-intptr-size, blockliteral-setupblock.'
expected: 'Unknown optimization: 'foo'. Valid optimizations are: remove-uithread-checks, dead-code-elimination, inline-isdirectbinding, inline-intptr-size.'
* [tests][linker] Fix typo.
Fixes this test failure:
1) SetupBlock_CustomDelegate (Linker.Shared.BaseOptimizeGeneratedCodeTest.SetupBlock_CustomDelegate)
Counter
Expected: 1
But was: 2
* [registrar] Minor adjustment to error message to match previous (and better) behavior.
Fixes this test failure:
1) Failed : Xamarin.Registrar.GenericType_WithInvalidParameterTypes
The error 'MT4136: The registrar cannot marshal the parameter type 'System.Collections.Generic.List`1<U>' of the parameter 'arg' in the method 'Open`1.Bar(System.Collections.Generic.List`1<U>)'' was not found in the output:
Message #1 did not match:
actual: 'The registrar cannot marshal the parameter type 'System.Collections.Generic.List`1<Foundation.NSObject>' of the parameter 'arg' in the method 'Open`1.Bar(System.Collections.Generic.List`1<U>)''
expected: 'The registrar cannot marshal the parameter type 'System.Collections.Generic.List`1<U>' of the parameter 'arg' in the method 'Open`1.Bar(System.Collections.Generic.List`1<U>)''
* [docs] mmp shows MM errors/warnings.
* [docs] Improve according to reviews.
* [tests] Fix merge failure causing test duplication.
* [ObjCRuntime] Add a BindingImplAttribute.
* [linker] Make ProviderToString an extension method on ICustomAttributeProvider to make it more discoverable.
* [generator] Use [BindingImpl] instead of [CompilerGenerated].
The entire diff is big (89MB), so it can't be gisted. However, most of it is
either removal of `using System.Runtime.CompilerServices;` or the change from
`[CompilerGenerated]` to `[BindingImpl (...)]` like this:
https://gist.github.com/rolfbjarne/8bfda3ed37b956d0342a1c1e9b079244
If I remove those parts of the diff, there's nothing significant left:
https://gist.github.com/rolfbjarne/4156164d6bdb1376366200394eb8a091
* [linker] Teach the linker about the new [BindingImpl] attribute.
In addition to the existing logic where the linker would optimize some
[CompilerGenerated] code (sometimes with additional requirements), it will now
also optimize all [BindingImpl (Optimizable)] code (without any additional
requirements).
* [tests] Add tests to make sure [BindingImpl (Optimizable)] works as expected.
* [linker] Check for [BindingImpl] before [CompilerGenerated] and stop checking for [CompilerGenerated] in XAMCORE_4_0.
Check for [BindingImpl] before checking for [CompilerGenerated], since the
former is more common.
Also stop checking for [CompilerGenerated] (at least to mean that code is
optimizable) in our next non-compatible evolutionary leap (XAMCORE_4_0):
* [introspection] Impl a better typo check.
* [mtouch/mmp] Give users more control over optimizations, and share more code between mtouch and mmp.
1. Add an --optimize flag to mtouch/mmp that allows users to select which
optimizations to apply (or not). This makes it easier to add future
optimizations, and allow users to disable any optimization that causes
problems without having to disable many other features.
2. Share as much optimization code as possible between mtouch and mmp. This
immediately gives a benefit to mmp, which has three new optimizations only
mtouch had: NSObject.IsDirectBinding inlining, IntPtr.Size inlining and
dead code elimination.
This results in ~6kb of disk space saved for a linked Xamarin.Mac app:
* link sdk: [Debug][1], [Release][2]
* link all: [Debug][3], [Release][4]
Testing also verifies that monotouchtest ([Debug][5], [Release][6]) has not
changed size at all, which means that no default optimizations have changed
inadvertedly.
[1]: https://gist.github.com/rolfbjarne/6b731e3b5ca6170355662e6505c3d492#link-sdk-mac--debug
[2]: https://gist.github.com/rolfbjarne/6b731e3b5ca6170355662e6505c3d492#link-sdk-mac--release
[3]: https://gist.github.com/rolfbjarne/6b731e3b5ca6170355662e6505c3d492#link-all-mac--debug
[4]: https://gist.github.com/rolfbjarne/6b731e3b5ca6170355662e6505c3d492#link-all-mac--release
[5]: https://gist.github.com/rolfbjarne/6b731e3b5ca6170355662e6505c3d492#monotouchtest-iphonedebug64
[6]: https://gist.github.com/rolfbjarne/6b731e3b5ca6170355662e6505c3d492#monotouchtest-iphonerelease64
* [tools] Don't enable the IsDirectBinding optimization by default for Xamarin.Mac apps, it's not safe.
* Fix whitespace issues.
* [doc] Document optimizations.
* Officially support optimizations by adding them to the Versions.plist.
* [linker] Improve IntPtr.Size inliner + dead code eliminatior and add tests.
* Properly handle operands for the ldc_i4_s instruction (they're sbyte).
* Fix less-than condition to actually do a less-than comparison.
* Make sure to look up the bitness in the Target, not the Application, since
the Application's value will be incorrect when building fat apps (both
Is32Build and Is64Build will be true).
* Remove unnecessary checks for the IntPtr.Size inliner: this optimization
does not depend on other instructions than the IntPtr.get_Size call, so
remove the checks that verify surrounding instructions. This makes the
IntPtr.Size inliner kick in in more scenarios (such as the new tests).
* Add tests.
* [tests] Add mmp tests for optimizations.
* [tests] Fix XM optimization tests.
* [tests] Fix test build error.
* [mtouch][mmp] Report invalid debug symbols files. Fixes#3200
Try to read the assembly with symbols and, if that fails, warn and
fallback to loading them without symbols.
This fixes cases were it's not easy to update or delete (e.g. nuget)
bad symbols files - so this cannot be an error without causing a lot
of pain.
However it needs to be reported, otherwise it wont be fixed (by the
publisher) and it can limit the debugability of the application and
the usefulness of the stacktraces.
Finally merge most of the resolver's code between mtouch and mmp so
we don't have to fix such issue twice anymore.
note: this needs to be slightly updated once we get a version of cecil
that can give us a more precise error message.
Also bring Rolf's tests from
https://github.com/xamarin/xamarin-macios/pull/3079
reference:
https://github.com/xamarin/xamarin-macios/issues/3200