Previously we copied any equivalent .dylib and ran install_name_tool on the
library to change the library id to make it a framework.
Unfortunately this does not work when the library contains bitcode, because
bitcode embeds linker flags (-install_name for instance), and
install_name_tool does not change those linker flags.
This means that we need to create frameworks by linking with the proper
arguments, since it's much more difficult to fixup the embedded bitcode linker
flags as well.
So change how be build Mono.framework, Xamarin.framework, and any frameworks
built from assemblies to:
* Always link instead of fixup a dylib. For Mono.framework this means
extracting all the object files from libmonosgen-2.0.a and linking those,
for Xamarin.framework this means linking the object files we've already
built.
* Make sure the library is correctly named when linked (once again: bitcode
contains embedded linker flags, so renaming the executable later breaks
stuff as well).
I've also extracted the logic that creates Mono.framework from
libmonosgen-2.0.a to a separate shell script, to deduplicate this logic.
This required a minor change in the mono builds: we need the Mono.framework
when building the `all` target, so make sure that happens.
https://bugzilla.xamarin.com/show_bug.cgi?id=53813
* [builds] Improve mono/llvm dependencies.
* Create a list of all the files in the mono and llvm repositories, and save
these lists as a Make variable (in a generated Makefile - .deps.*.mk). We
don't list _all_ the files in each repository, because there are quite a few
(55k for mono), and Make measurably takes a while to check all of them, so
try to limit it to a sane subset, without risking missing changes to files
that actually matters.
* Always create stamp files when we're done with mono builds.
* Modify the mono/llvm builds to depend on all the files in their
repositories.
* Explicitly list the corresponding .stamp-build-* files as dependencies for
various files that are produced by the mono builds, so that make knows how
to build these files.
* Rewrite the *-facade-check targets to depend on the corresponding
*_BCL_TARGETS, so that we can avoid running a submake to the same Makefile
to execute the facade checks.
It now takes a little while (less than a second on my machine, which is
fine) for make to list all dependencies and get their timestamps, but if
executing multiple submakes this adds up to a multi-second timewaste.
So avoid the timewaste by not doing submakes, but instead use dependencies
to enforce the required target execution ordering.
* Don't depend on nicely named intermediate targets, since won't prevent
rebuilds:
build-cross64: setup-cross64
Since the `setup-cross64` file doesn't exist, `build-cross64` will always
execute. Instead depend on the stamp file:
build-cross64: .stamp-configure-cross64
And now `build-cross64` will only rebuild if needed.
* Don't try to list all intermediate files as .SECONDARY dependencies, instead
list none at all, which works as if all files were listed as dependencies.
* Some targets had to move later in the file, since variables used in dependencies:
foo: $(VARIABLE)
must be defined before that point in the file, as opposed to variables used in recipes:
foo:
$(MAKE) $(VARIABLE)
can be defined anywhere in the Makefile.
* Simplify the targets that sign assemblies significantly.
There are a few end results:
* It's now possible to do `make install`, without doing `make all` first. This
might seem weird, but that also ensures the more common `make all install`
works properly.
* Remakes (without any mono/llvm changes) in build/ are much faster, because
we now won't recurse into every mono build:
$ time make all -C builds/ -j8
[...]
real 0m1.873s
This even means that we might be able to make it a habit to remake in the
root directory, which doesn't take forever now:
$ time make all -j8
[...]
real 0m4.521s
Unfortunately adding `make install` to the mix still does some useless
stuff, and it ends up taking ~30 seconds to complete a full build:
$ time make all install -j8
[...]
real 0m32.542s
* [msbuild] Don't verify the xml syntax of targets files unless the files change.
* [build] Don't depend on installed files.
Don't depend on installed files, because that causes a rebuild when installing
to a different directory (i.e. package creation).
* Bump maccore to get build improvements.
Rebuilds are now very fast:
$ make all install -j8
$ time make all install -j8
real 0m5.735s
Less than 6s to figure out that nothing needs to be done.
And strangely flushing the disk cache doesn't make it much slower:
$ sudo purge
$ time make all install -j8
real 0m7.309s
Which probably means that Make mostly reads file metadata, and not actual file
contents (which is good).
* [Debugger] Ensure that we use the correct paths in the mono mdb files to be able to step into the mono code.
Fixes bug 51530.
* Make the mdb paths to be correct in other platforms.
* Ensure mdb paths are also correct in XM.
Use @rpath instead of @executable_path in dylibs, since it allows us to be
more flexible when placing dylibs in the app.
In particular with this change it's trivial to put libmonosgen-2.0.dylib in
the container app, and reference it from extensions.
This is a series of fixes to the dynamic libraries we build / create to remove
any unnecessary bloat (unused architectures, bitcode).
A brand new watchOS app with no changes goes from 35MB to 11MB with these
fixes (with incremental builds disabled, the app size is 10MB).
--------------------------------------------------------------------------------
* [runtime] Split list of architectures into simulator and device-specific lists.
* [runtime] Build separate dylibs for device and simulator.
Build separate dylibs for device and simulator, since we already install these
into different locations.
This makes both the simulator and device builds slightly faster (since the
respective dylibs are smaller, and less data to copy around).
For watchOS apps, this saves ~430kb.
* [runtime] Strip bitcode from dylibs. Fixes#51352.
We know that dylibs will never be shipped to the App Store, so we'll never
need them to have bitcode.
So just strip the bitcode from all our dylibs, since this makes apps with
fastdev significantly smaller (and thus much faster to upload to watch
devices).
For watchOS apps this is a very significant improvement: a branch new watchOS
app without any changes goes from 35MB to 17MB.
https://bugzilla.xamarin.com/show_bug.cgi?id=51352
* [mtouch] Fix dylib compilation to not embed full bitcode.
Facts
=====
a. The output from the AOT compiler is an assembly (.s) file.
b. Clang's assembler does not support -fembed-bitcode-marker (only -fembed-
bitcode), so when we ask clang to -fembed-bitcode-marker, the assembler
receives a -fembed-bitcode argument.
c. This means that the assembled object file does not contain the
__LLVM/__bitcode and __LLVM/__cmdline sections (it does however contain an
__LLVM/__asm section).
d. The native linker will create a bitcode assembly section if none of the
object files passed to the linker contain a __LLVM/__bitcode section and
there's an __LLVM/__asm section present.
e. The end result is that when we build to a dylib, we end up (unexpectedly,
because we ask Clang to -fembed-bitcode-marker) including both armv7k and
bitcode in the dylib, thus bloating the dylib size significantly.
Solution
========
We manually add the __LLVM/__bitcode and __LLVM/__cmdline sections to the .s
file Mono's AOT compiler generated. This way the .o file will have the magic
sections, and the linker will not include bitcode (only the bitcode marker) in
the final library.
An empty watchOS extension with incremental builds is now 6MB smaller (down
to 11MB from 17MB).
* Pass -Wl,-no_weak_imports to the linker so that we don't accidentally use
symbols weakly. This would cause problems on older OSes where the symbol
isn't there, because our code is not prepared to deal with weakly linked
symbols.
* Manually disable fstatat and readlinkat (introduced with Xcode 7 in iOS 8 /
macOS 10.10), found by the above.
* Manually disable __thread support for a few targets, since mono's configure
script doesn't properly detect it's not supported.
* Manually disable clock_nanosleep, since mono's configure script doesn't
properly detect it's not supported.
* [builds] Don't install symlinks to iOS/Classic assemblies in the iOS/Unified profile.
* [fsharp] Don't install symlinks to iOS/Classic assemblies in the iOS/Unified profile.
The symlinks won't quite work once we remove the iOS/Classic assemblies.
* [C8] Bump to latest Mono 4.6.0 commit
Brings in the netstandard updates from https://github.com/mono/mono/pull/3394
We also had to add a new assembly System.IdentityModel.dll to the mobile profiles while doing that work.
* Add System.IdentityModel to Sdk assemblies
Fixes the following mtouch test failure:
```
Xamarin.Linker.SdkTest.iOS_Classic : BCL
Expected:
But was: < "System.IdentityModel" >
at NUnit.Framework.CollectionAssert.IsEmpty (IEnumerable collection, System.String message, System.Object[] args) <0x47bec88 + 0x00047> in :0
at NUnit.Framework.CollectionAssert.IsEmpty (IEnumerable collection, System.String message) <0x47bec58 + 0x0001f> in :0
at Xamarin.Linker.SdkTest.BCL (System.String path) <0x47bccf0 + 0x003f3> in :0
at Xamarin.Linker.SdkTest.iOS_Classic () <0x47bcc50 + 0x0001b> in :0
```
Brings in the netstandard updates from https://github.com/mono/mono/pull/3394
We also had to add a new assembly System.IdentityModel.dll to the mobile profiles while doing that work.
Since eaf2ceb532 there can be .dll.mdb files
for Facades as well which broke the Facades check.
The fix is to only include files ending in .dll and ignore the others.
Instead of having a symlink of the entire Facades directory for
Unified, we must now create a real directory and symlink each Facade
assembly, because there's one Facade assembly that can't be shared
between Unified and Classic (System.Drawing.Primitives.dll) because
it references monotouch.dll/Xamarin.iOS.dll.
If no sdk version (-sdk_version) is passed to the native
linker, it tries to infer the SDK version from the
path to the -syslibroot argument.
In our case we use a versioned path to Xcode, but a general
symlink without the SDK version:
/Applications/Xcode73.app/Contents/Developer/Platforms/AppleTVOS.platform/Developer/SDKs/AppleTVOS.sdk
which means ld picked up the Xcode version as the SDK version,
since that's the first number part of the path [1], so we'd
end up with libraries whose SDK version was 73.
So instead use an SDK path with the SDK version, so that ld
finds the SDK version instead of the Xcode version.
[1] 266f4401b9/src/ld/Options.cpp (L4005-L4020)https://bugzilla.xamarin.com/show_bug.cgi?id=41597
Rolf Bjarne Kvinge
Marek Safar
Manuel de la Pena
Chris Hamons
João Matos
Sebastien Pouliot
Martin Baulig
Zoltan Varga
Sebastien Pouliot
Alexander Köplinger
Aleksey Kliger