* [mtouch/mmp] Handle invalid types in BlockProxy attributes better. Fixes#4072.
BlockProxy attributes may have types we don't expect, so handle those cases
gracefully by showing a warning when we encounter them:
testApp.cs(11): warning MT4175: The parameter 'completionHandler' in the method 'Issue4072Session.CreateDataTask(Foundation.NSUrl,Foundation.NSUrlSessionResponse)' has an invalid BlockProxy attribute (the type passed to the attribute does not have a 'Create' method).
instead of an ugly MT0000:
MTOUCH : error MT0000: Unexpected error - Please file a bug report at http://bugzilla.xamarin.com
System.InvalidOperationException: Sequence contains no matching element
at System.Linq.Enumerable.First[TSource] (System.Collections.Generic.IEnumerable`1[T] source, System.Func`2[T,TResult] predicate) [0x00012] in /Users/builder/jenkins/workspace/build-package-osx-mono/2017-12/external/bockbuild/builds/mono-x64/external/corefx/src/System.Linq/src/System/Linq/First.cs:30
at Registrar.StaticRegistrar.GetBlockProxyAttributeMethod (Mono.Cecil.MethodDefinition method, System.Int32 parameter) [0x00020] in /Users/builder/data/lanes/5944/7e782c1e/source/xamarin-macios/tools/common/StaticRegistrar.cs:4121
at Registrar.StaticRegistrar.GetBlockWrapperCreator (Registrar.Registrar+ObjCMethod obj_method, System.Int32 parameter) [0x00011] in /Users/builder/data/lanes/5944/7e782c1e/source/xamarin-macios/tools/common/StaticRegistrar.cs:4065
at Registrar.StaticRegistrar.Specialize (Registrar.AutoIndentStringBuilder sb, Registrar.Registrar+ObjCMethod method, System.Collections.Generic.List`1[T] exceptions) [0x0216b] in /Users/builder/data/lanes/5944/7e782c1e/source/xamarin-macios/tools/common/StaticRegistrar.cs:3683
at Registrar.StaticRegistrar.Specialize (Registrar.AutoIndentStringBuilder sb) [0x00f1e] in /Users/builder/data/lanes/5944/7e782c1e/source/xamarin-macios/tools/common/StaticRegistrar.cs:2963
Fixes#4072.
* [docs] Improve text for MT4175.
* [registrar] Comment some code.
- https://github.com/xamarin/xamarin-macios/issues/3725
- These frameworks "CoreAudioKit Metal MetalKit MetalPerformanceShaders CoreNFC DeviceCheck"
were special cased, but that special case did do an SDK check.
- Create a helper method to share check
- Add test for MM0135
- https://github.com/xamarin/xamarin-macios/issues/3725
- These frameworks "CoreAudioKit Metal MetalKit MetalPerformanceShaders CoreNFC DeviceCheck"
were special cased, but that special case did do an SDK check.
- Create a helper method to share check
- Add test for MM0135
The main cause of the warning: "warning MSB6002: The command-line for the 'MTouch' task is too long" was the number of references that can be passed to `mtouch`.
Now all the arguments are written in a response file which in turn is passed to `mtouch`.
- Fixes bug #56501: MSB6002 command-line for MTouch task is too long, > 32000 characters
(https://bugzilla.xamarin.com/show_bug.cgi?id=56501)
- The response file is created in the device specific folder, as `response-file.rsp`. It is re-created every time.
- Added an msbuild test to ensure the response file is created by `GenerateCommandLineCommands` and that it includes all the references.
- Introduce `AddLine` and `AddQuotedLine` which we use to populate the response file (and go to the next line).
- Move to C# 7 syntax for string replacement.
- Update all tests in `MTouchTaskTests` to use the response file since the arguments are no longer passed directly to `mtouch`.
- Update MT0018's documentation for response files.
* [static registrar] Optimize creation of delegates for blocks.
Optimize creation of delegates for blocks so that it doesn't require the
dynamic registrar.
This is done by getting the metadata token for the Create method that creates
the delegate, and embed that metadata token in the generated code from the
static registrar.
Also add tests, since this scenario was not covered by tests already.
* [mmptest] Fix test after recent changes.
* [test-libraries] Avoid duplicate symbols.
* [tests] Update according to changes.
- https://github.com/xamarin/xamarin-macios/issues/3367
- App Store will now fail builds if you add in a 32-bit dylib
- If you are a 32-bit app you don't need the 64-bit part of your fat
dylib anyway
- Add --optimize=-trim-architectures to allow customization of behavior, as not everyone
uses app store
In addition, while writing tests for this is was noticed that mmp tests did not "really" run Release configuration correctly in most cases. Fixing this turned out to be a bit of a pain, but necessary to correctly test this (and other things).
- Turns out that /p:configuration:debug is not sufficient to tell mmp to
do the right thing
- That, in most projects, sets the DebugSymbols property, which really
is what is checked.
- However, two of our projects did not have that, so we always did
release mmp work.
- Removed configuration property for tests and added real "Release"
configuration option
* [static registrar] Optimize creation of delegates for blocks.
Optimize creation of delegates for blocks so that it doesn't require the
dynamic registrar.
This is done by getting the metadata token for the Create method that creates
the delegate, and embed that metadata token in the generated code from the
static registrar.
Also add tests, since this scenario was not covered by tests already.
* [mmptest] Fix test after recent changes.
* [test-libraries] Avoid duplicate symbols.
* [tests] Update according to changes.
- https://github.com/xamarin/xamarin-macios/issues/3367
- App Store will now fail builds if you add in a 32-bit dylib
- If you are a 32-bit app you don't need the 64-bit part of your fat
dylib anyway
- Add --optimize=-trim-architectures to allow customization of behavior, as not everyone
uses app store
In addition, while writing tests for this is was noticed that mmp tests did not "really" run Release configuration correctly in most cases. Fixing this turned out to be a bit of a pain, but necessary to correctly test this (and other things).
- Turns out that /p:configuration:debug is not sufficient to tell mmp to
do the right thing
- That, in most projects, sets the DebugSymbols property, which really
is what is checked.
- However, two of our projects did not have that, so we always did
release mmp work.
- Removed configuration property for tests and added real "Release"
configuration option
Implement support for optimizing away the dynamic registrar.
* Add a new property (Runtime.DynamicRegistrationSupported) that indicates at
runtime whether dynamic registration is available.
* Check this new property whenever we need dynamic registration.
* Add an optimization to mtouch/mmp that detects whether dynamic registration
is required, and if not, changes Runtime.DynamicRegistrationSupported to
return false.
* Add an optimization to mtouch/mmp that inlines
Runtime.DynamicRegistrationSupported as a constant value.
The end result is that the linker will link away the dynamic registrar if
mtouch/mmp detects that it's not needed.
Benchmark
---------
I've compared the size of entire apps built for device:
|test | Before | After | Diff | % |
|:-----------------------------|-------:|-------:|-------:|------:|
|[monotouch-test/Release][1] | 95.7mb | 95.0mb | -680kb | -0.7% |
|[link sdk/Release][2] | 21.2mb | 20.9mb | -245kb | -1.2% |
|[minimalistic app/Release][3] | 4.58mb | 4.32mb | -259kb | -5.7% |
[1]: https://gist.github.com/rolfbjarne/3871e36d2de8db1a8eee1d9f9276d3d2#monotouch-test-release-wall-optimizations-enabled
[2]: https://gist.github.com/rolfbjarne/3871e36d2de8db1a8eee1d9f9276d3d2#link-sdk
[3]: https://gist.github.com/rolfbjarne/3871e36d2de8db1a8eee1d9f9276d3d2#minimalistic-app
Improved linker to:
* Not mark protocol interfaces by the mere virtue of having a type that
implements them. This is implemented by not marking protocol interfaces when
they're implementing a class, but instead when a method implementation is
found to implement a method from a protocol interface.
* Mark the wrapper type for protocols (this allows us to remove the Protocol
attribute, since that's the link between the protocol and its wrapper type).
* Remove the [Protocol], [ProtocolMember] and [Adopts] attributes.
The static registrar still needs some of the information linked away, so a few
changes are required to make it available post linker.
Generate table of protocols and their corresponding wrapper type in the static
registrar, so that we don't have to look for this information using the
dynamic registrar at runtime.
* [linker] Optimize calls to BlockLiteral.SetupBlock to inject the block signature.
Optimize calls to BlockLiteral.SetupBlock[Unsafe] to calculate the block
signature at build time, and inject it into the call site.
This makes block invocations 10-15x faster (I've added tests that asserts at
least an 8x increase).
It's also required in order to be able to remove the dynamic registrar code in
the future (since calculating the block signature at runtime requires the
dynamic registrar).
* [mtouch/mmp] Add support for reporting errors/warnings that point to the code line causing the error/warning.
Add support for reporting errors/warnings that point to the code line causing
the error/warning by adding ErrorHelper overloads that take the exact
instruction to report (previously we defaulted to the first line/instruction
in a method).
* [tests] Add support for asserting filename/linenumber in warning messages.
* Make all methods that manually create BlockLiterals optimizable.
* [tests] Create a BaseOptimizeGeneratedCodeTest test that's included in both XI's and XM's link all test.
* [tests] Add link all test (for both XI and XM) to test the BlockLiteral.SetupBlock optimization.
* [tests] Add mtouch/mmp tests for the BlockLiteral.SetupBlock optimization.
* [tests][linker] Make the base test class abstract, so tests in the base class aren't executed twice.
* [tests][linker] Don't execute linkall-only tests in linksdk.
The optimization tests only apply when the test assembly is linked, and that
only happens in linkall, so exclude those tests in linksdk.
* [tests][mmptest] Update test according to mmp changes.
Fixes these test failures:
1) Failed : Xamarin.MMP.Tests.MMPTests.MM0132("inline-runtime-arch")
The warning 'MM0132: Unknown optimization: 'inline-runtime-arch'. Valid optimizations are: remove-uithread-checks, dead-code-elimination, inline-isdirectbinding, inline-intptr-size.' was not found in the output:
Message #1 did not match:
actual: 'Unknown optimization: 'inline-runtime-arch'. Valid optimizations are: remove-uithread-checks, dead-code-elimination, inline-isdirectbinding, inline-intptr-size, blockliteral-setupblock.'
expected: 'Unknown optimization: 'inline-runtime-arch'. Valid optimizations are: remove-uithread-checks, dead-code-elimination, inline-isdirectbinding, inline-intptr-size.'
Message #2 did not match:
actual: 'Unknown optimization: 'inline-runtime-arch'. Valid optimizations are: remove-uithread-checks, dead-code-elimination, inline-isdirectbinding, inline-intptr-size, blockliteral-setupblock.'
expected: 'Unknown optimization: 'inline-runtime-arch'. Valid optimizations are: remove-uithread-checks, dead-code-elimination, inline-isdirectbinding, inline-intptr-size.'
2) Failed : Xamarin.MMP.Tests.MMPTests.MM0132("foo")
The warning 'MM0132: Unknown optimization: 'foo'. Valid optimizations are: remove-uithread-checks, dead-code-elimination, inline-isdirectbinding, inline-intptr-size.' was not found in the output:
Message #1 did not match:
actual: 'Unknown optimization: 'foo'. Valid optimizations are: remove-uithread-checks, dead-code-elimination, inline-isdirectbinding, inline-intptr-size, blockliteral-setupblock.'
expected: 'Unknown optimization: 'foo'. Valid optimizations are: remove-uithread-checks, dead-code-elimination, inline-isdirectbinding, inline-intptr-size.'
Message #2 did not match:
actual: 'Unknown optimization: 'foo'. Valid optimizations are: remove-uithread-checks, dead-code-elimination, inline-isdirectbinding, inline-intptr-size, blockliteral-setupblock.'
expected: 'Unknown optimization: 'foo'. Valid optimizations are: remove-uithread-checks, dead-code-elimination, inline-isdirectbinding, inline-intptr-size.'
* [tests][linker] Fix typo.
Fixes this test failure:
1) SetupBlock_CustomDelegate (Linker.Shared.BaseOptimizeGeneratedCodeTest.SetupBlock_CustomDelegate)
Counter
Expected: 1
But was: 2
* [registrar] Minor adjustment to error message to match previous (and better) behavior.
Fixes this test failure:
1) Failed : Xamarin.Registrar.GenericType_WithInvalidParameterTypes
The error 'MT4136: The registrar cannot marshal the parameter type 'System.Collections.Generic.List`1<U>' of the parameter 'arg' in the method 'Open`1.Bar(System.Collections.Generic.List`1<U>)'' was not found in the output:
Message #1 did not match:
actual: 'The registrar cannot marshal the parameter type 'System.Collections.Generic.List`1<Foundation.NSObject>' of the parameter 'arg' in the method 'Open`1.Bar(System.Collections.Generic.List`1<U>)''
expected: 'The registrar cannot marshal the parameter type 'System.Collections.Generic.List`1<U>' of the parameter 'arg' in the method 'Open`1.Bar(System.Collections.Generic.List`1<U>)''
* [docs] mmp shows MM errors/warnings.
* [docs] Improve according to reviews.
* [tests] Fix merge failure causing test duplication.
* [generator] Teach generator about WrapAttribute on Getters and Setters
https://bugzilla.xamarin.com/show_bug.cgi?id=57870
`WrapAttribute` can now be used in property getters and setters,
this allows to Wrap virtually anything the way you need, for example
smart enums, consider the following API definition:
```csharp
// Smart enum.
enum PersonRelationship {
[Field (null)]
None,
[Field ("FMFather", "__Internal")]
Father,
[Field ("FMMother", "__Internal")]
Mother
}
```
```csharp
// Property definition.
[Export ("presenceType")]
NSString _PresenceType { get; set; }
PersonRelationship PresenceType {
[Wrap ("PersonRelationshipExtensions.GetValue (_PresenceType)")]
get;
[Wrap ("_PresenceType = value.GetConstant ()")]
set;
}
```
* Fix Feedback
* Fix doc error
* Update error message
* [ObjCRuntime] Add a BindingImplAttribute.
* [linker] Make ProviderToString an extension method on ICustomAttributeProvider to make it more discoverable.
* [generator] Use [BindingImpl] instead of [CompilerGenerated].
The entire diff is big (89MB), so it can't be gisted. However, most of it is
either removal of `using System.Runtime.CompilerServices;` or the change from
`[CompilerGenerated]` to `[BindingImpl (...)]` like this:
https://gist.github.com/rolfbjarne/8bfda3ed37b956d0342a1c1e9b079244
If I remove those parts of the diff, there's nothing significant left:
https://gist.github.com/rolfbjarne/4156164d6bdb1376366200394eb8a091
* [linker] Teach the linker about the new [BindingImpl] attribute.
In addition to the existing logic where the linker would optimize some
[CompilerGenerated] code (sometimes with additional requirements), it will now
also optimize all [BindingImpl (Optimizable)] code (without any additional
requirements).
* [tests] Add tests to make sure [BindingImpl (Optimizable)] works as expected.
* [linker] Check for [BindingImpl] before [CompilerGenerated] and stop checking for [CompilerGenerated] in XAMCORE_4_0.
Check for [BindingImpl] before checking for [CompilerGenerated], since the
former is more common.
Also stop checking for [CompilerGenerated] (at least to mean that code is
optimizable) in our next non-compatible evolutionary leap (XAMCORE_4_0):
* [introspection] Impl a better typo check.
* [mtouch/mmp] Give users more control over optimizations, and share more code between mtouch and mmp.
1. Add an --optimize flag to mtouch/mmp that allows users to select which
optimizations to apply (or not). This makes it easier to add future
optimizations, and allow users to disable any optimization that causes
problems without having to disable many other features.
2. Share as much optimization code as possible between mtouch and mmp. This
immediately gives a benefit to mmp, which has three new optimizations only
mtouch had: NSObject.IsDirectBinding inlining, IntPtr.Size inlining and
dead code elimination.
This results in ~6kb of disk space saved for a linked Xamarin.Mac app:
* link sdk: [Debug][1], [Release][2]
* link all: [Debug][3], [Release][4]
Testing also verifies that monotouchtest ([Debug][5], [Release][6]) has not
changed size at all, which means that no default optimizations have changed
inadvertedly.
[1]: https://gist.github.com/rolfbjarne/6b731e3b5ca6170355662e6505c3d492#link-sdk-mac--debug
[2]: https://gist.github.com/rolfbjarne/6b731e3b5ca6170355662e6505c3d492#link-sdk-mac--release
[3]: https://gist.github.com/rolfbjarne/6b731e3b5ca6170355662e6505c3d492#link-all-mac--debug
[4]: https://gist.github.com/rolfbjarne/6b731e3b5ca6170355662e6505c3d492#link-all-mac--release
[5]: https://gist.github.com/rolfbjarne/6b731e3b5ca6170355662e6505c3d492#monotouchtest-iphonedebug64
[6]: https://gist.github.com/rolfbjarne/6b731e3b5ca6170355662e6505c3d492#monotouchtest-iphonerelease64
* [tools] Don't enable the IsDirectBinding optimization by default for Xamarin.Mac apps, it's not safe.
* Fix whitespace issues.
* [doc] Document optimizations.
* Officially support optimizations by adding them to the Versions.plist.
* [linker] Improve IntPtr.Size inliner + dead code eliminatior and add tests.
* Properly handle operands for the ldc_i4_s instruction (they're sbyte).
* Fix less-than condition to actually do a less-than comparison.
* Make sure to look up the bitness in the Target, not the Application, since
the Application's value will be incorrect when building fat apps (both
Is32Build and Is64Build will be true).
* Remove unnecessary checks for the IntPtr.Size inliner: this optimization
does not depend on other instructions than the IntPtr.get_Size call, so
remove the checks that verify surrounding instructions. This makes the
IntPtr.Size inliner kick in in more scenarios (such as the new tests).
* Add tests.
* [tests] Add mmp tests for optimizations.
* [tests] Fix XM optimization tests.
* [tests] Fix test build error.
https://bugzilla.xamarin.com/show_bug.cgi?id=59294
This enables ImageRepresentation strong dictionary helpers.
--- A note about WrapAttribute
Whenever we use the `WrapAttribute` inside a `Category` we need to
include the `This` parameter as the first argument inside the Wrap
signature, like the following example:
```csharp
[Wrap ("WriteTiffRepresentation (This, image, url, format, colorSpace, options?.Dictionary, out error)")]
bool WriteTiffRepresentation (CIImage image, NSUrl url, CIFormat format, CGColorSpace colorSpace, CIImageRepresentationOptions options, out NSError error);
```
* [mtouch][mmp] Report invalid debug symbols files. Fixes#3200
Try to read the assembly with symbols and, if that fails, warn and
fallback to loading them without symbols.
This fixes cases were it's not easy to update or delete (e.g. nuget)
bad symbols files - so this cannot be an error without causing a lot
of pain.
However it needs to be reported, otherwise it wont be fixed (by the
publisher) and it can limit the debugability of the application and
the usefulness of the stacktraces.
Finally merge most of the resolver's code between mtouch and mmp so
we don't have to fix such issue twice anymore.
note: this needs to be slightly updated once we get a version of cecil
that can give us a more precise error message.
Also bring Rolf's tests from
https://github.com/xamarin/xamarin-macios/pull/3079
reference:
https://github.com/xamarin/xamarin-macios/issues/3200
When a method signature contains any ref/out parameters
it is a hint that this method is not a candidate to be
used with [Async] we now error BI1062 in the generator.
The reason of an error instead of a warning is that we
currently generate not compilable code when ref/out is
used on the method signature or in the signature of the
delegate (completion handler), it is very unlikely we are
breaking someone now that we emit an error instead of broken
code.
* Fix Anchor and Clarify the addition of BI1117 Warning into docs/website/generator-errors.md
BI1117 Warning documentation was missing from docs/website/generator-errors.md
so I added it.
- Add XIA0006: HttpClientAvoidManaged.
- Add documentation on how the rules work and how to activate them.
Also mention that they need to be ran on each active configuration.
- Bump maccore to include XIA 0006.
* [generator] Improve BI1014 - include name of unsupported field and update valid types on docs, fixes bug 57094.
https://bugzilla.xamarin.com/show_bug.cgi?id=57094
* Implement feedback
* fix error message
* More feedback
Detect MT1111 from mlaunch (which means mlaunch won't wait for the app to
exit), and instead use test run completion to determine test completion.
The main drawback is that if the test app crashes, it won't be detected (the
test run will time out, and reported as such), but it's still an improvement
over the current behavior (the tests may complete successfully, and still be
reported as timed out).
This also requires bumping maccore to get an updated mlaunch (one that reports
MT1111).
It does not make sense to create Xamarin.iOS projects that don't reference
Xamarin.iOS.dll, so make this an explicit error.
This fixes a NullReferenceException which could (when building for device, or
when not using simlauncher) occur, and instead shows the MT0123 error.
> MTOUCH : error MT0000: Unexpected error - Please file a bug report at http://bugzilla.xamarin.com
> System.NullReferenceException: Object reference not set to an instance of an object
> at Xamarin.Bundler.Target.GatherFrameworks () [0x00065] in /Users/builder/data/lanes/5024/152b654a/source/xamarin-macios/tools/common/Target.cs:122
> at Xamarin.Bundler.Target.ProcessAssemblies () [0x000c2] in /Users/builder/data/lanes/5024/152b654a/source/xamarin-macios/tools/mtouch/Target.cs:802
> at Xamarin.Bundler.Application.ProcessAssemblies () [0x0002f] in /Users/builder/data/lanes/5024/152b654a/source/xamarin-macios/tools/mtouch/Application.cs:1407
> at Xamarin.Bundler.Application.BuildManaged () [0x00001] in /Users/builder/data/lanes/5024/152b654a/source/xamarin-macios/tools/mtouch/Application.cs:831
> at Xamarin.Bundler.Application+<>c.<BuildAll>b__134_1 (Xamarin.Bundler.Application v) [0x00000] in /Users/builder/data/lanes/5024/152b654a/source/xamarin-macios/tools/mtouch/Application.cs:779
> at System.Collections.Generic.List`1[T].ForEach (System.Action`1[T] action) [0x00024] in <48b95f3df5804531818f80e28ec60191>:0
> at Xamarin.Bundler.Application.BuildAll () [0x00050] in /Users/builder/data/lanes/5024/152b654a/source/xamarin-macios/tools/mtouch/Application.cs:779
> at Xamarin.Bundler.Driver.Main2 (System.String[] args) [0x00481] in /Users/builder/data/lanes/5024/152b654a/source/xamarin-macios/tools/mtouch/mtouch.cs:1420
> at Xamarin.Bundler.Driver.Main (System.String[] args) [0x0000f] in /Users/builder/data/lanes/5024/152b654a/source/xamarin-macios/tools/mtouch/mtouch.cs:945
https://bugzilla.xamarin.com/show_bug.cgi?id=59798
Bug #59570 was a dupe of #58063 but was harder (than it should be) to
diagnose because the error did not point out which assembly had the
issue (requiring a whole, large, customer solution to confirm).
This adds a new MT2103 error which points to the assembly that could
not be read.
E.g. VSfM will now show
> Error MT2103: Error processing assembly 'Shim, Version=2.0.1.0, Culture=neutral, PublicKeyToken=c55ec16d10c4b366': System.NullReferenceException: Object reference not set to an instance of an object (MT2103) (...)
[1] https://bugzilla.xamarin.com/show_bug.cgi?id=59570
[2] https://bugzilla.xamarin.com/show_bug.cgi?id=58063
Vincent Dondain
Rolf Bjarne Kvinge
Brad Umbaugh
Sebastien Pouliot
Chris Hamons
Alex Soto
Marius Ungureanu
Alex Soto
Timothy Risi