This change introduces the export of create_classes methods as objc compatible, without enforcing Objective-C++ as the development language for custom registrar embedders by moving the stringbuilder flushing inside the extern "C" block.
Mark the generated linking code as extern "C" too and also change the return type of xamarin_create_classes_Xamarin_Mac to void in mmp generation, as it was mistakenly set to int.
The AVFoundation framework's headers used to be broken in the simulator SDK
([1], [2]) until watchOS 3.2 (Xcode 8.3) fixed it. This means it's now safe to
use AVFoundation.
Additionally set the initial SDK version where this framework was introduced
to 3.2 for simulator builds, which means that if customers try to use it (with
an old Xcode), they will get a nice-ish error:
> MTOUCH : error MT4134: Your application is using the 'AVFoundation' framework, which isn't included in the watchOS SDK you're using to build your app (this framework was introduced in watchOS 3.2, while you're building with the watchOS 3.1 SDK.) Please select a newer SDK in your app's watchOS Build options.
instead of an ugly:
> MTOUCH : error MT4109: Failed to compile the generated registrar code. Please file a bug report at http://bugzilla.xamarin.com
the error message is slightly incorrect (the problem is only with the
simulator SDK, not the device SDK), but this should happen very rarely (it
only occurs if all of the following are true: using AVFoundation + in a
simulator build * the static registrar manually selected), so IMHO a more
accurate error description isn't worth it.
https://bugzilla.xamarin.com/show_bug.cgi?id=56862
[1] 7149661251
[2] https://openradar.appspot.com/29131674
* [runtime] Fix Xamarin-debug.framework's install name.
This makes building to frameworks work in debug mode.
* [mtouch] Fix check to add frameworks to watchKit extensions.
* [mtouch] Never pass -read_only_relocs to the native linker when bitcode is enabled.
* [mtouch] Bitcode requires linking with c++.
This particular case applies to shared libraries/frameworks (we already link
with c++ when building statically).
Put simulator assemblies in MonoBundle/simulator for frameworks, so that we
can have a single framework that contains both device and simulator
assemblies without assemblies conflicting between device and simulator.
The device assemblies continue in the same place, in the MonoBundle directory,
so no additional checks are needed on device.
Also stop using `mdb` as the name for debug symbols and remove
> static MdbReader mdb_reader;
since we're not mkbundl'ing mtouch anymore.
Related to https://github.com/xamarin/xamarin-macios/pull/2002 for mmp
* Use Visual Studio instead of Xamarin Studio.
* VS doesn't have mdtool, it has vstool.
Also there's no need to manually invoke the mdtool.exe executable anymore
(which we did because the mdtool executable had a min macOS version of 10.9,
and we used to build tests on older macOS versions [1]), since now we only run
tests on older macOS versions, we don't build those tests there.
[1] a1932b0ccd
- Before this mmp was not adding -framework, -weak_framework consistently on non-static registrar use cases
- GatherFrameworks was previously not ported from mtouch, and did not work as DeploymentTarget was unset in mmp
- Added verbose prints so users can determine why various framework linkages are added
- Fixed an issue where duplicate were being added due to HandleFramework shoving args by hand
- Tested with auto test and https://github.com/chamons/xm-version-regression-test manual test
When using debug simulator we don't generate main.m so we were not passing the gc options.
The MONO_GC_PARAMS variable is not in app.EnvironmentVariables (which only contains environment variables passed to mtouch using --setenv), which is why the above condition does not trigger.
Don't use the global command line arguments to determine input, because that's
not the input we use for app extensions anymore.
Instead explicitly pass the input arguments when creating the cache.
Implement support for sharing both code and resources between app extensions
and their container app:
* AOT-compiled code. Each shared assembly is only AOT-compiled once, and if
the assembly is built to a framework or dynamic library, it will also only
be included once in the final app (as a framework or dynamic library in the
container app, referenced directly by the app extension). If the assemblies
are built to static objects there won't be any size improvements in the app,
but the build will be much faster, because the assemblies will only be AOT-
compiled once.
* Any resources related to managed assemblies (debug files, config files,
satellite assemblies) will be put in the container app only.
Since these improvements are significant, code sharing will be enabled by
default.
Test results
============
For an extreme test project with 7 extensions (embedded-frameworks)[1]:
with code sharing cycle 9 difference
build time 1m 47s 3m 33s -1m 46s = ~50% faster
app size 26 MB 131 MB -105 MB = ~80% smaller
For a more normal test project (MyTabbedApplication)[2] - this is a simple application with 1 extension:
with code sharing cycle 9 difference
build time 0m 44s 0m 48s -4s = ~ 8% faster
app size 23 MB 37 MB -15 MB = ~40% smaller
Another tvOS app with one extension also show similar gains (MyTVApp)[3]:
with code sharing cycle 9 difference
build time 0m 22s 0m 48s -26s = ~54% faster
app size 22 MB 62 MB -40 MB = ~65% smaller
[1]: https://github.com/rolfbjarne/embedded-frameworks
[2]: https://github.com/xamarin/xamarin-macios/tree/cycle9/msbuild/tests/MyTabbedApplication
[3]: https://github.com/xamarin/xamarin-macios/tree/cycle9/msbuild/tests/MyTVApp
Store the location of every assembly that can't be deduced at runtime (i.e.
all assemblies that are build to frameworks, since there can be multiple
assemblies in each framework, and the framework name can be customized).
According to Vlad it's not necessary to set MONO_GC_PARAMS during AOT-
compilation, since all MONO_GC_PARAMS options can be changed at runtime:
Rolf Kvinge [16:35] @vlad.brezae is this true for all the different options MONO_GC_PARAMS take: https://github.com/xamarin/xamarin-macios/pull/1546#discussion_r97318092?
Rolf Kvinge [16:36] I remember this: https://bugzilla.xamarin.com/show_bug.cgi?id=35414#c14
Rofl Kvinge [16:36] which apparently you changed here, so that it can be changed at runtime: https://bugzilla.xamarin.com/show_bug.cgi?id=35414#c27
Rolf Kvinge [16:36] but I don't know if this is true for all the options you can pass using MONO_GC_PARAMS
Vlad Brezae [16:41] yes, it should be true for all of them, that was a bug
Rolf Kvinge [16:41] ok, that's great news 😄
Performance tests
-----------------
This is for a new watchOS extension project, built for release.
* The default (currently -O2) optimizations: 41s ( baseline ) 30.027.060 bytes ( baseline )
* All optimizations disabled (`--llvm-opt=all=`): 17s (-24s = -59%) 32.978.312 bytes (+2.951.252 = +10%)
* Optimized for size (`--llvm-opt=all=-Os`): 36s ( -5s = -12%) 28.617.408 bytes (-1.409.652 = -5%)
* Optimized for more size (`--llvm-opt=all=-Oz`): 35s ( -6s = -15%) 28.601.016 bytes (-1.426.044 = -5%)
* Optimized slightly (`--llvm-opt=all=-O1`): 35s ( -6s = -15%) 28.666.556 bytes (-1.360.504 = -5%)
* Optimized a lot (`--llvm-opt=all=-O3`): 41s ( 0s = 0%) 30.403.996 bytes (+ 376.936 = +1%)
Conclusions
-----------
* The fastest build by far (less than twice as fast) is if optimizations are
disabled, but this adds a 10% size penalty (~3 MB in this test case),
compared to the baseline, and 15% size penalty (4.3 MB) compared to -Oz.
* -Oz seems to have the best overall results: at least as fast as any other
optimized build, and the smallest app as well.
Caveats
-------
Some optimizations might not work the AOT compiled code. The resulting
binaries have not been tested.
Event sequence:
* mtouch is executed with the linker disabled.
* The linker pipeline copies all input assemblies (since the linker is
disabled the assemblies don't change) into the PreBuild directory. This will
keep the original timestamps of the input assemblies.
* mtouch is executed again, when none of the input assemblies changed.
* The linker pipeline will re-execute, because it will see that at least one
of the input assemblies (at least the .exe) is newer than at least one of
the assemblies in the PreBuild directory (usually a framework assembly,
because those have the original timestamp from their install location).
Fix:
Touch all the assemblies in the PreBuild directory after the linker pipeline
executes the first time. This way the second time mtouch is executed, it will
find that all assemblies in the PreBuild directory have timestamps later than
all the input assemblies, so it will load the cached linked assemblies,
instead of re-executing the linker pipeline.
Build extensions and the container app in the same mtouch process, by storing
all the mtouch arguments when called to build extensions in a text file, and
then reloading those arguments when called to build the main app.
This is required if we want to share code between extensions and the
container.
So that there can be multiple caches in the same process (which we'll have
once mtouch can compile extensions and the container app in the same process).
* [mtouch] Remove most static state and put it on an instance.
Remove most static state, and put it on an instance (of Application) instead.
Soon we'll build multiple apps (app extensions) in the same process, and we
need to make sure we're not inadvertedly (and incorrectly) sharing state
between apps, so remove as much static state as possible to avoid any
problems.
* [mtouch] Rename GetIsUsingClang -> IsUsingClang.
* [mtouch] Always require a SDK version when building.
Technically it was required before too, but the error messages were non-optimal:
it could for instance complain that the user is using an iOS framework that
was introduced in iOS 2.0.
* [mtouch tests] Rewrite MT0060 and MT0061 tests to use MTouchTool.
This makes sure we pass --sdk to mtouch (which MTouchTool does by default), so
that we don't run into MT0025 before the errors we're testing for.
Timothy Risi
Marius Ungureanu
Rolf Bjarne Kvinge
Sebastien Pouliot
Alex Soto
Chris Hamons
Marek Safar
Vlad Brezae