* [mtouch] Don't look for assembly references in attributes in assemblies we ship. Partially fixes#49087.
Don't look for assembly references in attributes in assemblies we ship,
because it takes a significant amount of time to do this, and we can
precompute the fact that there aren't any such assembly references.
Additionally add a test to ensure we catch any changes to this assumption.
For a simple test app this makes rebuilding (without any changes) go from
~1.1s to ~0.4s.
https://bugzilla.xamarin.com/show_bug.cgi?id=49087
* [tests] Simplify tests to not use [TestCaseSource].
Using [TestCaseSource] is nice when running from the IDE, since it shows all
test cases in the test tree.
Unfortunately it causes the console runner to freak out [1], because the method
that lists all the test cases calls Configuration's cctor, which calls
TestContext.CurrentContext.TestDirectory, which is apparently not safe this
early in the test run.
[1] I think 'freak out' is the appropriate term for this behavior, which has
absolutely no direct nor obvious connection to the cause of the problem:
System.Runtime.Remoting.RemotingException: Cannot create channel sink to connect to URL 93a78115_c0da_4b6a_9661_9f9b9d9fb935/6669afd6_4.rem. An appropriate channel has probably not been registered.
Server stack trace:
at System.Runtime.Remoting.RemotingServices.GetClientChannelSinkChain (System.String url, System.Object channelData, System.String& objectUri) [0x00019] in <04300341516a482b9708b764d58af7ca>:0
at System.Runtime.Remoting.RemotingServices.GetOrCreateClientIdentity (System.Runtime.Remoting.ObjRef objRef, System.Type proxyType, System.Object& clientProxy) [0x0001d] in <04300341516a482b9708b764d58af7ca>:0
at System.Runtime.Remoting.RemotingServices.GetRemoteObject (System.Runtime.Remoting.ObjRef objRef, System.Type proxyType) [0x00000] in <04300341516a482b9708b764d58af7ca>:0
at System.Runtime.Remoting.RemotingServices.GetProxyForRemoteObject (System.Runtime.Remoting.ObjRef objref, System.Type classToProxy) [0x0001b] in <04300341516a482b9708b764d58af7ca>:0
at System.Runtime.Remoting.RemotingServices.Unmarshal (System.Runtime.Remoting.ObjRef objectRef, System.Boolean fRefine) [0x0007a] in <04300341516a482b9708b764d58af7ca>:0
at System.Runtime.Remoting.RemotingServices.Unmarshal (System.Runtime.Remoting.ObjRef objectRef) [0x00000] in <04300341516a482b9708b764d58af7ca>:0
at System.Runtime.Remoting.ObjRef.GetRealObject (System.Runtime.Serialization.StreamingContext context) [0x0000f] in <04300341516a482b9708b764d58af7ca>:0
at System.Runtime.Serialization.ObjectManager.ResolveObjectReference (System.Runtime.Serialization.ObjectHolder holder) [0x00010] in <04300341516a482b9708b764d58af7ca>:0
at System.Runtime.Serialization.ObjectManager.DoFixups () [0x0007f] in <04300341516a482b9708b764d58af7ca>:0
at System.Runtime.Serialization.Formatters.Binary.ObjectReader.Deserialize (System.Runtime.Remoting.Messaging.HeaderHandler handler, System.Runtime.Serialization.Formatters.Binary.__BinaryParser serParser, System.Boolean fCheck, System.Boolean isCrossAppDomain, System.Runtime.Remoting.Messaging.IMethodCallMessage methodCallMessage) [0x00077] in <04300341516a482b9708b764d58af7ca>:0
at System.Runtime.Serialization.Formatters.Binary.BinaryFormatter.Deserialize (System.IO.Stream serializationStream, System.Runtime.Remoting.Messaging.HeaderHandler handler, System.Boolean fCheck, System.Boolean isCrossAppDomain, System.Runtime.Remoting.Messaging.IMethodCallMessage methodCallMessage) [0x000a2] in <04300341516a482b9708b764d58af7ca>:0
at System.Runtime.Serialization.Formatters.Binary.BinaryFormatter.Deserialize (System.IO.Stream serializationStream, System.Runtime.Remoting.Messaging.HeaderHandler handler, System.Boolean fCheck, System.Runtime.Remoting.Messaging.IMethodCallMessage methodCallMessage) [0x00000] in <04300341516a482b9708b764d58af7ca>:0
at System.Runtime.Serialization.Formatters.Binary.BinaryFormatter.DeserializeMethodResponse (System.IO.Stream serializationStream, System.Runtime.Remoting.Messaging.HeaderHandler handler, System.Runtime.Remoting.Messaging.IMethodCallMessage methodCallMessage) [0x00000] in <04300341516a482b9708b764d58af7ca>:0
at System.Runtime.Remoting.Channels.BinaryClientFormatterSink.SyncProcessMessage (System.Runtime.Remoting.Messaging.IMessage msg) [0x00083] in <270c90abbc234cde9d33eb198a97cf71>:0
Make sure to instantiate an Assembly instance for every assembly loaded using
the cached list of assemblies.
Fixes the following test failures:
1. Xamarin.MTouch.RebuildTest_WithExtensions("single","",False,System.String[]) : second build
Expected: 0
But was: 1
2. Xamarin.MTouch.RebuildTest_WithExtensions("dual","armv7,arm64",False,System.String[]) : second build
Expected: 0
But was: 1
3. Xamarin.MTouch.RebuildTest_WithExtensions("llvm","armv7+llvm",False,System.String[]) : second build
Expected: 0
But was: 1
4. Xamarin.MTouch.RebuildTest_WithExtensions("debug","",True,System.String[]) : second build
Expected: 0
But was: 1
5. Xamarin.MTouch.RebuildTest_WithExtensions("single-framework","",False,System.String[]) : second build
Expected: 0
But was: 1
* [mtouch] Cache the list of assemblies we computed. Partially fixes#49087.
Computing the list of assemblies can be expensive, so cache it and re-use the
cached list if we can.
https://bugzilla.xamarin.com/show_bug.cgi?id=49087
* [mtouch] Create directory before trying to create files to it.
The linker doesn't allow it when building for bitcode:
> ld: -ignore_optimization_hints and -bitcode_bundle (Xcode setting ENABLE_BITCODE=YES) cannot be used together
> clang: error: linker command failed with exit code 1 (use -v to see invocation)
This fixes the Xamarin.MTouch.StripBitcodeFromFrameworks(tvOS,Marker) mtouch test.
* [runtime] Fix Xamarin-debug.framework's install name.
This makes building to frameworks work in debug mode.
* [mtouch] Fix check to add frameworks to watchKit extensions.
* [mtouch] Never pass -read_only_relocs to the native linker when bitcode is enabled.
* [mtouch] Bitcode requires linking with c++.
This particular case applies to shared libraries/frameworks (we already link
with c++ when building statically).
Of particular importance is if we're building for LLVM or not: this fixes a
bug where we wouldn't pass --llvm to the AOT compiler when compiling
assemblies to frameworks (which we do when sharing code).
https://bugzilla.xamarin.com/show_bug.cgi?id=55555
When collecting a list of roots, it helps adding any roots we find to the list
of roots.
This fixes the MT0023 mtouch tests, since now we actually report MT0023
because we have a list of the root assemblies.
When in embeddinator mode, we must link the generated registrar and pinvoke
code into the framework we create.
This also requires creating the corresponding aot/link tasks after creating
the registrar/pinvoke tasks.
* [mtouch] Only iOS has version 8 (and earlier OS versions that don't support frameworks).
* [mtouch] Update logging to be less confusing when logging about WatchKit extensions.
* [mtouch] Don't compile P/Invoke wrappers for extensions that are sharing code.
The container app already has the P/Invoke wrappers.
- Before this mmp was not adding -framework, -weak_framework consistently on non-static registrar use cases
- GatherFrameworks was previously not ported from mtouch, and did not work as DeploymentTarget was unset in mmp
- Added verbose prints so users can determine why various framework linkages are added
- Fixed an issue where duplicate were being added due to HandleFramework shoving args by hand
- Tested with auto test and https://github.com/chamons/xm-version-regression-test manual test
Previously we copied any equivalent .dylib and ran install_name_tool on the
library to change the library id to make it a framework.
Unfortunately this does not work when the library contains bitcode, because
bitcode embeds linker flags (-install_name for instance), and
install_name_tool does not change those linker flags.
This means that we need to create frameworks by linking with the proper
arguments, since it's much more difficult to fixup the embedded bitcode linker
flags as well.
So change how be build Mono.framework, Xamarin.framework, and any frameworks
built from assemblies to:
* Always link instead of fixup a dylib. For Mono.framework this means
extracting all the object files from libmonosgen-2.0.a and linking those,
for Xamarin.framework this means linking the object files we've already
built.
* Make sure the library is correctly named when linked (once again: bitcode
contains embedded linker flags, so renaming the executable later breaks
stuff as well).
I've also extracted the logic that creates Mono.framework from
libmonosgen-2.0.a to a separate shell script, to deduplicate this logic.
This required a minor change in the mono builds: we need the Mono.framework
when building the `all` target, so make sure that happens.
https://bugzilla.xamarin.com/show_bug.cgi?id=53813
No frameworks should be bundled in WatchKit 1 extensions, they should be
bundled in the container (iOS) app.
This broke when merging the equivalent fix for master into the framework-sdk
branch (i.e. a broken merge).
The cached linker results can have multiple identical input assemblies (for
assemblies that show up in both the app and any app extensions), so make sure
we don't load those more than once.
Since the linker can process multiple apps/appex'es at the same time, it also
means it will put together all the required symbols found in _all_ assemblies.
This means that we need to filter out required symbols for other
apps/appex'es.
Implement support for sharing both code and resources between app extensions
and their container app:
* AOT-compiled code. Each shared assembly is only AOT-compiled once, and if
the assembly is built to a framework or dynamic library, it will also only
be included once in the final app (as a framework or dynamic library in the
container app, referenced directly by the app extension). If the assemblies
are built to static objects there won't be any size improvements in the app,
but the build will be much faster, because the assemblies will only be AOT-
compiled once.
* Any resources related to managed assemblies (debug files, config files,
satellite assemblies) will be put in the container app only.
Since these improvements are significant, code sharing will be enabled by
default.
Test results
============
For an extreme test project with 7 extensions (embedded-frameworks)[1]:
with code sharing cycle 9 difference
build time 1m 47s 3m 33s -1m 46s = ~50% faster
app size 26 MB 131 MB -105 MB = ~80% smaller
For a more normal test project (MyTabbedApplication)[2] - this is a simple application with 1 extension:
with code sharing cycle 9 difference
build time 0m 44s 0m 48s -4s = ~ 8% faster
app size 23 MB 37 MB -15 MB = ~40% smaller
Another tvOS app with one extension also show similar gains (MyTVApp)[3]:
with code sharing cycle 9 difference
build time 0m 22s 0m 48s -26s = ~54% faster
app size 22 MB 62 MB -40 MB = ~65% smaller
[1]: https://github.com/rolfbjarne/embedded-frameworks
[2]: https://github.com/xamarin/xamarin-macios/tree/cycle9/msbuild/tests/MyTabbedApplication
[3]: https://github.com/xamarin/xamarin-macios/tree/cycle9/msbuild/tests/MyTVApp
Warn if mtouch loads an assembly from a different location than requested
(which might be because there are multiple assemblies with the same name).
Also rework the MT0023 check a bit by explicitly loading the root assembly
first, and then detecting if any loaded assemblies matches the root assembly.
This results in code that's a bit more obvious, and it also works correctly
with extensions (previously the entire MT0023 check was skipped for
extensions).
Detect when assemblies have native dependencies between them (which can happen
when there are multiple binding projects, and the native libraries in those
binding projects have dependencies between them), and add the proper link
arguments (this is only required when building to dynamic libraries or
frameworks, since otherwise everything is linked to one big binary and there
are no dependency problems).
https://bugzilla.xamarin.com/show_bug.cgi?id=43689
The previous build system kept a forward-pointing single linked list of tasks
to execute: task X had a list of subsequent tasks to execute. If task X was
up-to-date, it was not created (and the next tasks were directly added to the
list of tasks to execute).
In this world it became complicated to merge output from tasks (for instance
if the output of task X and task Y should be a consumed by a single task
producing a single output, since the corresponding task would end up in both
X's and Y's list of subsequent tasks).
Example: creating a single framework from the aot-compiled output of multiple
assemblies.
So I've reversed the logic: now we keep track of the final output, and then
each task has a list of dependencies that must be built.
This makes it trivial to create merging tasks (for the previous example, there
could for instance be a CreateFrameworkTask, where its dependencies would be
all the corresponding AotTasks).
We also always create every task, and then each task decides when its executed
whether it should do anything or not. This makes it unnecessary to 'forward-
delete' files when creating tasks (say you have three tasks, A, B, C; B
depends on A, and C depends on B; if A's output isn't up-to-date, it has to
delete its own output if it exists, otherwise B would not detect that it would
have to re-execute, because at task *creation* time, B's input hadn't
changed).
Additionally make it based on async/await, since much of the work happens in
externel processes (and we don't need to spin up additional threads just to
run external processes). This makes us have less code run on background
threads, which makes any issues with thread-safety less likely.
Rolf Bjarne Kvinge
Sebastien Pouliot
Chris Hamons
Marek Safar