xamarin-macios/tools/common/DerivedLinkContext.cs

154 строки
5.3 KiB
C#
Исходник Обычный вид История

using System.Collections.Generic;
using System.Linq;
using Mono.Cecil;
using Mono.Linker;
using Mono.Collections.Generic;
using Registrar;
[mtouch] Improve how we make sure native symbols aren't stripped away. Fixes #51710 and #54417. (#2162) * [mtouch] Improve how we make sure native symbols aren't stripped away. Fixes #51710 and #54417. * Refactor required symbol collection to store more information about each symbol (field, function, Objective-C class), and in general make the code more straight forward. * Implement support for generating source code that references these symbols, and do this whenever we can't ask the native linker to keep these symbols (when using bitcode). Additionally make it possible to do this manually, so that the source code can be generated for non-bitcode platforms too (which is useful if the number of symbols is enormous, in which case we might surpass the maximum command-line length). * Also make it possible to completely ignore native symbols, or ignore them on a per-symbol basis. This provides a fallback for users if we get something right and we try to preserve something that shouldn't be preserved (for instance if it doesn't exist), and the user ends up with unfixable linker errors. * Don't collect Objective-C classes unless they're in an assembly with LinkWith attributes. We don't need to preserve Objective-C classes in any other circumstances. * Implement everything for both Xamarin.iOS and Xamarin.Mac, and share the code between them. * Remove previous workaround for bug #51710, since it's no longer needed. * Add tests. https://bugzilla.xamarin.com/show_bug.cgi?id=54417 https://bugzilla.xamarin.com/show_bug.cgi?id=51710 * [mtouch] Make sure to only keep symbols from the current app when code sharing. This fixes a build problem with the interdependent-binding-projects test when testing in Today Extension mode.
2017-06-02 19:29:19 +03:00
using Xamarin.Bundler;
[mtouch] Make sure native symbols from third-party libraries are preserved in dylibs. Fixes #51548. The native linker treats object files (.o) and static libraries (.a files, which are archives of .o files) differently. The native linker will always include object files into the executable: $ echo "void xxx () {}" > foo.m $ clang -c foo.m -o foo.o -arch x86_64 $ ld foo.o -dylib -o foo.dylib -macosx_version_min 10.12 -arch x86_64 $ nm foo.dylib 0000000000000fe0 T _xxx However, if the object file is inside a static library: $ echo "void xxx () {}" > foo.m $ clang -c foo.m -o foo.o -arch x86_64 $ ar cru foo.a foo.o $ ld foo.a -dylib -o foo.dylib -macosx_version_min 10.12 -arch x86_64 $ nm foo.dylib <no output> This means that our testing library (libtest.a) which is a fat library of _object files_, do not show the problems reported in bug #51548. So: a) I've fixed the creation of libtest.a to be a fat library of _static libraries_. This causes the `FastDev_LinkWithTest` test to fail exactly like in bug #51548. b) I've made mtouch pass `-u <native symbol>` to the native linker, for every native symbol referenced in a managed assembly, when creating a dylib. Amazingly this seems to work fine even with symbols to Objective-C classes (`_OBJC_CLASS_$_<class name>`). c) This also required adding support for collecting the Objective-C names of all managed types registered with Objective-C to the linker. The information is already available in the static registrar, but that would require us to make sure the static registrar is executed before compiling dylibs, which means those two tasks won't be able to run in parallel (also there's no guarantee we'll even run the static registrar). https://bugzilla.xamarin.com/show_bug.cgi?id=51548
2017-01-18 12:25:58 +03:00
namespace Xamarin.Tuner
{
public class DerivedLinkContext : LinkContext
{
[mtouch] Make sure native symbols from third-party libraries are preserved in dylibs. Fixes #51548. The native linker treats object files (.o) and static libraries (.a files, which are archives of .o files) differently. The native linker will always include object files into the executable: $ echo "void xxx () {}" > foo.m $ clang -c foo.m -o foo.o -arch x86_64 $ ld foo.o -dylib -o foo.dylib -macosx_version_min 10.12 -arch x86_64 $ nm foo.dylib 0000000000000fe0 T _xxx However, if the object file is inside a static library: $ echo "void xxx () {}" > foo.m $ clang -c foo.m -o foo.o -arch x86_64 $ ar cru foo.a foo.o $ ld foo.a -dylib -o foo.dylib -macosx_version_min 10.12 -arch x86_64 $ nm foo.dylib <no output> This means that our testing library (libtest.a) which is a fat library of _object files_, do not show the problems reported in bug #51548. So: a) I've fixed the creation of libtest.a to be a fat library of _static libraries_. This causes the `FastDev_LinkWithTest` test to fail exactly like in bug #51548. b) I've made mtouch pass `-u <native symbol>` to the native linker, for every native symbol referenced in a managed assembly, when creating a dylib. Amazingly this seems to work fine even with symbols to Objective-C classes (`_OBJC_CLASS_$_<class name>`). c) This also required adding support for collecting the Objective-C names of all managed types registered with Objective-C to the linker. The information is already available in the static registrar, but that would require us to make sure the static registrar is executed before compiling dylibs, which means those two tasks won't be able to run in parallel (also there's no guarantee we'll even run the static registrar). https://bugzilla.xamarin.com/show_bug.cgi?id=51548
2017-01-18 12:25:58 +03:00
internal StaticRegistrar StaticRegistrar;
[mtouch] Improve how we make sure native symbols aren't stripped away. Fixes #51710 and #54417. (#2162) * [mtouch] Improve how we make sure native symbols aren't stripped away. Fixes #51710 and #54417. * Refactor required symbol collection to store more information about each symbol (field, function, Objective-C class), and in general make the code more straight forward. * Implement support for generating source code that references these symbols, and do this whenever we can't ask the native linker to keep these symbols (when using bitcode). Additionally make it possible to do this manually, so that the source code can be generated for non-bitcode platforms too (which is useful if the number of symbols is enormous, in which case we might surpass the maximum command-line length). * Also make it possible to completely ignore native symbols, or ignore them on a per-symbol basis. This provides a fallback for users if we get something right and we try to preserve something that shouldn't be preserved (for instance if it doesn't exist), and the user ends up with unfixable linker errors. * Don't collect Objective-C classes unless they're in an assembly with LinkWith attributes. We don't need to preserve Objective-C classes in any other circumstances. * Implement everything for both Xamarin.iOS and Xamarin.Mac, and share the code between them. * Remove previous workaround for bug #51710, since it's no longer needed. * Add tests. https://bugzilla.xamarin.com/show_bug.cgi?id=54417 https://bugzilla.xamarin.com/show_bug.cgi?id=51710 * [mtouch] Make sure to only keep symbols from the current app when code sharing. This fixes a build problem with the interdependent-binding-projects test when testing in Today Extension mode.
2017-06-02 19:29:19 +03:00
internal Target Target;
Symbols required_symbols;
// SDK candidates - they will be preserved only if the application (not the SDK) uses it
List<ICustomAttributeProvider> srs_data_contract = new List<ICustomAttributeProvider> ();
List<ICustomAttributeProvider> xml_serialization = new List<ICustomAttributeProvider> ();
HashSet<TypeDefinition> cached_isnsobject;
[linker] Improve inlining of IsDirectBinding check to inline both true and false values. (#3214) Previous behavior ================= * The linker would determine if a class is a generated binding class or not. This was determined by the presence of a constructor taking a single IntPtr argument, and with a [CompilerGenerated] attribute. * If a class was not a generated binding class, then that class and all its superclasses were marked as not optimizable (in the context of inlining the IsDirectBinding check). * The end result was that all classes with a [CompilerGenerated] IntPtr constructor that weren't subclassed, were optimized (the IsDirectBinding value was implied to be 'true'). Unfortunately this does not match how the IsDirectBinding value is actually computed, and is in many cases quite wrong. Background ========== The authorative value for the IsDirectBinding value is the register attribute: ```csharp [Register ("MyClass", true)] // the second parameter specifies the IsDirectBinding value class MyClass : NSObject {} ``` Due to history this second parameter is called `IsWrapper` and not `IsDirectBinding`, but it's the exact same thing. Unfortunately looking up this attribute every time a class is instantiated is slow (since fetching attributes is slow), so we guess this value in NSObject's initialization: if the actual type of the object is in the platform assembly, then we assume IsDirectBinding=true: ```csharp IsDirectBinding = (this.GetType ().Assembly == PlatformAssembly); ``` and any subclasses in the platform assembly which is not a direct binding have to set the correct value in their constructors. New behavior ============ In the linker we now track three states for the IsDirectBinding value for each class: if it can be inlined into a constant true or false, or if it has to be checked at runtime (a nullable bool is used, and null corresponds with this last undetermined state). * The linker will look at the `[Register]` attribute for a class, and: * If the type is CIFilter, store that IsDirectBinding=undetermined. * If IsWrapper=False, store that IsDirectBinding=False for that class, and that IsDirectBinding=undetermined for all super classes where IsWrapper=True. * If IsWrapper=True, tentatively assume IsDirectBinding=True in that class unless already determined to be undetermined (which will be overruled if a non-wrapper subclass is found later). Results ======= For monotouch-test, the changes are as follows: * Classes we can now assume IsDirectBinding=true: https://gist.github.com/rolfbjarne/acd6a8cf1236562a832d6db9400afee9#file-foo-diff-L1-L25 * Classes we previously assumed incorrectly that IsDirectBinding=true: https://gist.github.com/rolfbjarne/acd6a8cf1236562a832d6db9400afee9#file-foo-diff-L27-L645 * Classes we can now assume IsDirectBinding=false: https://gist.github.com/rolfbjarne/acd6a8cf1236562a832d6db9400afee9#file-foo-diff-L647-L2281 There are also minor size improvements (in the iPhone/Debug64 configuration): The executable is 17.632 bytes smaller: -rwxr-xr-x 1 rolf staff 73038384 Jan 12 12:40 /Users/rolf/test/old/monotouchtest.app/monotouchtest -rwxr-xr-x 1 rolf staff 73020752 Jan 12 12:50 /Users/rolf/test/new/monotouchtest.app/monotouchtest Xamarin.iOS.dll is 3.072 bytes smaller (this will probably be 0 on a release (stripped) build). -rw-r--r-- 1 rolf staff 2522624 Jan 12 12:40 /Users/rolf/test/old/monotouchtest.app/Xamarin.iOS.dll -rw-r--r-- 1 rolf staff 2519552 Jan 12 12:50 /Users/rolf/test/new/monotouchtest.app/Xamarin.iOS.dll CIFilter ======== There's a complication with CIFilters [1] [2], their implementation is somewhat special, so we do not want to optimize anything for those classes to not risk getting anything wrong. [1] https://github.com/xamarin/xamarin-macios/pull/3055 [2] https://bugzilla.xamarin.com/show_bug.cgi?id=15465
2018-01-15 14:28:34 +03:00
// Tristate:
// null = don't know, must check at runtime (can't inline)
// true/false = corresponding constant value
Dictionary<TypeDefinition, bool?> isdirectbinding_value;
// Store interfaces the linker has linked away so that the static registrar can access them.
public Dictionary<TypeDefinition, List<TypeDefinition>> ProtocolImplementations { get; private set; } = new Dictionary<TypeDefinition, List<TypeDefinition>> ();
public Application App {
get {
return Target.App;
}
}
public HashSet<TypeDefinition> CachedIsNSObject {
get { return cached_isnsobject; }
set { cached_isnsobject = value; }
}
[linker] Improve inlining of IsDirectBinding check to inline both true and false values. (#3214) Previous behavior ================= * The linker would determine if a class is a generated binding class or not. This was determined by the presence of a constructor taking a single IntPtr argument, and with a [CompilerGenerated] attribute. * If a class was not a generated binding class, then that class and all its superclasses were marked as not optimizable (in the context of inlining the IsDirectBinding check). * The end result was that all classes with a [CompilerGenerated] IntPtr constructor that weren't subclassed, were optimized (the IsDirectBinding value was implied to be 'true'). Unfortunately this does not match how the IsDirectBinding value is actually computed, and is in many cases quite wrong. Background ========== The authorative value for the IsDirectBinding value is the register attribute: ```csharp [Register ("MyClass", true)] // the second parameter specifies the IsDirectBinding value class MyClass : NSObject {} ``` Due to history this second parameter is called `IsWrapper` and not `IsDirectBinding`, but it's the exact same thing. Unfortunately looking up this attribute every time a class is instantiated is slow (since fetching attributes is slow), so we guess this value in NSObject's initialization: if the actual type of the object is in the platform assembly, then we assume IsDirectBinding=true: ```csharp IsDirectBinding = (this.GetType ().Assembly == PlatformAssembly); ``` and any subclasses in the platform assembly which is not a direct binding have to set the correct value in their constructors. New behavior ============ In the linker we now track three states for the IsDirectBinding value for each class: if it can be inlined into a constant true or false, or if it has to be checked at runtime (a nullable bool is used, and null corresponds with this last undetermined state). * The linker will look at the `[Register]` attribute for a class, and: * If the type is CIFilter, store that IsDirectBinding=undetermined. * If IsWrapper=False, store that IsDirectBinding=False for that class, and that IsDirectBinding=undetermined for all super classes where IsWrapper=True. * If IsWrapper=True, tentatively assume IsDirectBinding=True in that class unless already determined to be undetermined (which will be overruled if a non-wrapper subclass is found later). Results ======= For monotouch-test, the changes are as follows: * Classes we can now assume IsDirectBinding=true: https://gist.github.com/rolfbjarne/acd6a8cf1236562a832d6db9400afee9#file-foo-diff-L1-L25 * Classes we previously assumed incorrectly that IsDirectBinding=true: https://gist.github.com/rolfbjarne/acd6a8cf1236562a832d6db9400afee9#file-foo-diff-L27-L645 * Classes we can now assume IsDirectBinding=false: https://gist.github.com/rolfbjarne/acd6a8cf1236562a832d6db9400afee9#file-foo-diff-L647-L2281 There are also minor size improvements (in the iPhone/Debug64 configuration): The executable is 17.632 bytes smaller: -rwxr-xr-x 1 rolf staff 73038384 Jan 12 12:40 /Users/rolf/test/old/monotouchtest.app/monotouchtest -rwxr-xr-x 1 rolf staff 73020752 Jan 12 12:50 /Users/rolf/test/new/monotouchtest.app/monotouchtest Xamarin.iOS.dll is 3.072 bytes smaller (this will probably be 0 on a release (stripped) build). -rw-r--r-- 1 rolf staff 2522624 Jan 12 12:40 /Users/rolf/test/old/monotouchtest.app/Xamarin.iOS.dll -rw-r--r-- 1 rolf staff 2519552 Jan 12 12:50 /Users/rolf/test/new/monotouchtest.app/Xamarin.iOS.dll CIFilter ======== There's a complication with CIFilters [1] [2], their implementation is somewhat special, so we do not want to optimize anything for those classes to not risk getting anything wrong. [1] https://github.com/xamarin/xamarin-macios/pull/3055 [2] https://bugzilla.xamarin.com/show_bug.cgi?id=15465
2018-01-15 14:28:34 +03:00
public Dictionary<TypeDefinition, bool?> IsDirectBindingValue {
get { return isdirectbinding_value; }
set { isdirectbinding_value = value; }
}
public IList<ICustomAttributeProvider> DataContract {
get {
return srs_data_contract;
}
}
public IList<ICustomAttributeProvider> XmlSerialization {
get {
return xml_serialization;
}
}
[mtouch] Improve how we make sure native symbols aren't stripped away. Fixes #51710 and #54417. (#2162) * [mtouch] Improve how we make sure native symbols aren't stripped away. Fixes #51710 and #54417. * Refactor required symbol collection to store more information about each symbol (field, function, Objective-C class), and in general make the code more straight forward. * Implement support for generating source code that references these symbols, and do this whenever we can't ask the native linker to keep these symbols (when using bitcode). Additionally make it possible to do this manually, so that the source code can be generated for non-bitcode platforms too (which is useful if the number of symbols is enormous, in which case we might surpass the maximum command-line length). * Also make it possible to completely ignore native symbols, or ignore them on a per-symbol basis. This provides a fallback for users if we get something right and we try to preserve something that shouldn't be preserved (for instance if it doesn't exist), and the user ends up with unfixable linker errors. * Don't collect Objective-C classes unless they're in an assembly with LinkWith attributes. We don't need to preserve Objective-C classes in any other circumstances. * Implement everything for both Xamarin.iOS and Xamarin.Mac, and share the code between them. * Remove previous workaround for bug #51710, since it's no longer needed. * Add tests. https://bugzilla.xamarin.com/show_bug.cgi?id=54417 https://bugzilla.xamarin.com/show_bug.cgi?id=51710 * [mtouch] Make sure to only keep symbols from the current app when code sharing. This fixes a build problem with the interdependent-binding-projects test when testing in Today Extension mode.
2017-06-02 19:29:19 +03:00
public Symbols RequiredSymbols {
get {
if (required_symbols == null)
[mtouch] Improve how we make sure native symbols aren't stripped away. Fixes #51710 and #54417. (#2162) * [mtouch] Improve how we make sure native symbols aren't stripped away. Fixes #51710 and #54417. * Refactor required symbol collection to store more information about each symbol (field, function, Objective-C class), and in general make the code more straight forward. * Implement support for generating source code that references these symbols, and do this whenever we can't ask the native linker to keep these symbols (when using bitcode). Additionally make it possible to do this manually, so that the source code can be generated for non-bitcode platforms too (which is useful if the number of symbols is enormous, in which case we might surpass the maximum command-line length). * Also make it possible to completely ignore native symbols, or ignore them on a per-symbol basis. This provides a fallback for users if we get something right and we try to preserve something that shouldn't be preserved (for instance if it doesn't exist), and the user ends up with unfixable linker errors. * Don't collect Objective-C classes unless they're in an assembly with LinkWith attributes. We don't need to preserve Objective-C classes in any other circumstances. * Implement everything for both Xamarin.iOS and Xamarin.Mac, and share the code between them. * Remove previous workaround for bug #51710, since it's no longer needed. * Add tests. https://bugzilla.xamarin.com/show_bug.cgi?id=54417 https://bugzilla.xamarin.com/show_bug.cgi?id=51710 * [mtouch] Make sure to only keep symbols from the current app when code sharing. This fixes a build problem with the interdependent-binding-projects test when testing in Today Extension mode.
2017-06-02 19:29:19 +03:00
required_symbols = new Symbols ();
return required_symbols;
}
}
public DerivedLinkContext (Pipeline pipeline, AssemblyResolver resolver)
: base (pipeline, resolver)
{
Bump mono and fix default for linker UserAction (#3113) The linker bump (included in the mono bump) requires setting a default value for UserAction https://github.com/mono/linker/commit/4d2362d8083e3c71560ba1225d468a0080aeea27 Commit list for mono/mono: * mono/mono@ce89e4c5fa4 [threads] Fix leak of gchandle to MonoInternalThread (#6258) * mono/mono@9bd33ec5b67 [android] Android NDK does not contain API level/platform 12 * mono/mono@63e8dc6ea17 Bump cecil * mono/mono@e65bf00e22b Merge pull request #6122 from lewurm/2017-10-interp-aot-mode-fixes * mono/mono@8f0589ae817 [mini] Add missing try holes * mono/mono@ab20369d5f0 [mini] Align stack when resuming to catch handler * mono/mono@3a134a2d8c3 [mini] Add missing try holes * mono/mono@2e775c7e390 [mini] Fix clause try hole checking * mono/mono@a9a4166431e [loader] Don't assert on abstract methods in get_method_constrained * mono/mono@feba66a6ceb [interp] small improvment for error reporting in interp compile method callback * mono/mono@6fc6ca1e189 [aot] encode interp_in wrappers with proper signature * mono/mono@73326908260 [interp] fix copy/paste-typo in n2m macro magic * mono/mono@b64faae88c3 [aot] add more signatures for interp_in wrapper needed for iOS * mono/mono@b3b0613ad38 Bump msbuild to bring in fix for #60770 (#6107) * mono/mono@ddeba6e1bab [interp] fix using conv.u with string * mono/mono@0360f420fe3 Bump API snapshot submodule * mono/mono@2f18e7dd23c Bump cecil & linker to match master. * mono/mono@0f53cb275c4 [interp] allow unsigned i8 in pinvoke signature Diff: https://github.com/mono/mono/compare/c5cd0f1e7fb494cec523757b8d7f29cc95b707c9...ce89e4c5fa46a4d8225f43339fab0b0574f81cfe https://bugzilla.xamarin.com/show_bug.cgi?id=60770
2017-12-19 01:56:12 +03:00
UserAction = AssemblyAction.Link;
}
public Dictionary<IMetadataTokenProvider, object> GetAllCustomAttributes (string storage_name)
{
return Annotations?.GetCustomAnnotations (storage_name);
}
public List<ICustomAttribute> GetCustomAttributes (ICustomAttributeProvider provider, string storage_name)
{
var annotations = Annotations?.GetCustomAnnotations (storage_name);
object storage = null;
if (annotations?.TryGetValue (provider, out storage) != true)
return null;
return (List<ICustomAttribute>) storage;
}
// Stores custom attributes in the link context, so that the attribute can be retrieved and
// inspected even if it's linked away.
public void StoreCustomAttribute (ICustomAttributeProvider provider, CustomAttribute attribute, string storage_name)
{
var dict = Annotations.GetCustomAnnotations (storage_name);
List<ICustomAttribute> attribs;
object attribObjects;
if (!dict.TryGetValue (provider, out attribObjects)) {
attribs = new List<ICustomAttribute> ();
dict [provider] = attribs;
} else {
attribs = (List<ICustomAttribute>) attribObjects;
}
// Make sure the attribute is resolved, since after removing the attribute
// it won't be able to do it. The 'CustomAttribute.Resolve' method is private, but fetching
// any property will cause it to be called.
// We also need to store the constructor's DeclaringType separately, because it may
// be nulled out from the constructor by the linker if the attribute type itself is linked away.
var dummy = attribute.HasConstructorArguments;
attribs.Add (new AttributeStorage { Attribute = attribute, AttributeType = attribute.Constructor.DeclaringType });
}
public List<ICustomAttribute> GetCustomAttributes (ICustomAttributeProvider provider, string @namespace, string name)
{
// The equivalent StoreCustomAttribute method below ignores the namespace (it's not needed so far since all attribute names we care about are unique),
// so we need to retrieve the attributes the same way (using the name only).
return GetCustomAttributes (provider, name);
}
public void StoreCustomAttribute (ICustomAttributeProvider provider, CustomAttribute attribute)
{
StoreCustomAttribute (provider, attribute, attribute.AttributeType.Name);
}
public void StoreProtocolMethods (TypeDefinition type)
{
var attribs = Annotations.GetCustomAnnotations ("ProtocolMethods");
object value;
if (!attribs.TryGetValue (type, out value))
attribs [type] = type.Methods.ToArray (); // Make a copy of the collection, since the linker may remove methods from it.
}
public IList<MethodDefinition> GetProtocolMethods (TypeDefinition type)
{
var attribs = Annotations.GetCustomAnnotations ("ProtocolMethods");
object value;
if (attribs.TryGetValue (type, out value))
return (MethodDefinition []) value;
return null;
}
class AttributeStorage : ICustomAttribute
{
public CustomAttribute Attribute;
public TypeReference AttributeType { get; set; }
public bool HasFields => Attribute.HasFields;
public bool HasProperties => Attribute.HasProperties;
public bool HasConstructorArguments => Attribute.HasConstructorArguments;
public Collection<CustomAttributeNamedArgument> Fields => Attribute.Fields;
public Collection<CustomAttributeNamedArgument> Properties => Attribute.Properties;
public Collection<CustomAttributeArgument> ConstructorArguments => Attribute.ConstructorArguments;
}
}
}