* [mtouch] Bring back PreBuildDirectory
Some mtouch unit tests failed because of timestamp checks. It's not clear
it's always a win (it's not in my original test case) not to copy (to
target) as it means we have to copy earlier (PreBuild + Build).
More investigation needed, there's probably a better way to achieve both.
In the mean time this should fix the failing tests cases.
* Fix up / simplify logic and avoid duplicate calls to IsUpdate
The old `legacy` option will now be reported as a warning.
That's by design an warning would require manually editing the .csproj
file (when the UI gets removed, as planned, from the IDE).
This is part of
https://trello.com/c/SrgU38DN/647-only-ship-support-appletls
Note: The BCL changes will happen in later stages.
Earlier versions of Xamarin Studio stored an invalid http message handler in
watchOS project files, which would cause a build error. In addition Xamarin
Studio removed the UI to set the http message handler (since only one value is
valid), which meant that the user had to edit the project file by hand to get
around this build error.
So make it a warning instead (and document what the user has to do to fix the
warning).
https://bugzilla.xamarin.com/show_bug.cgi?id=46552
Use metadata tokens instead of strings to find types and methods.
This makes the code to find methods more compact (a lot less strings in the
executable, and additionally in most cases a compact representation (32-bit
integer) of the corresponding metadata token and additional information can be
used, which results in less executable code (fewer parameters to methods,
etc)), resulting in smaller executables.
Size savings are around 200kb for dont link apps, and 20-60kb for linked apps
(this obviously varies a lot depending on how much has to registered by the
registrar).
| | Before | After | Diff |
|----------------|--------------:|--------------:|------------------:|
| dontlink/32bit | 102.810.144 | 102.609.456 | -200.688 = -0,20% |
| dontlink/64bit | 107.420.576 | 107.221.792 | -198.784 = -0,19% |
| linksdk/32bit | 40.957.296 | 40.936.864 | -20.432 = -0,05% |
| linksdk/64bit | 43.113.136 | 43.093.936 | -19.200 = -0,04% |
| linkall/32bit | 38.410.032 | 38.348.288 | -61.744 = -0,16% |
| linkall/64bit | 40.315.200 | 40.267.344 | -47.856 = -0,12% |
Additionally I've removed the `lazy_map` dictionary, which we populated at
startup and was used to map between Class instances and the corresponding
managed type's FullName, and instead iterate over a native array of Class ->
metadata token mappings whenever we need to look up the managed type for a
certain Class instance.
This is slightly slower for each type we need to look up (for a non-linked app
there might be a 2000-3000 entries in the native array, which would be
iterated instead of using a hashtable lookup), but it's only done once per
type and there's a significant startup memory improvement.
For a non-linked test app I get the following using the Xamarin profiler:
| | Before | After | Diff |
|-------------------|--------:|--------:|----------------:|
| Memory allocated | 2,8 MB | 2,4 MB | -0,4 MB = -14 % |
| Objects allocated | 43678 | 38463 | -5215 = -12 % |
| Private bytes | 26,6 MB | 24,4 MB | -2,2 MB = -8,3% |
| Working set | 26,6 MB | 24,4 MB | -2,2 MB = -8,3% |
Right now the logic exists in a few places, both in and outside the
linker. We recently began to use part of the linker pipeline in normal /
all builds so it's easier to share (and unify) the code now.
The real gain is to avoid copying assemblies, in particular large ones,
more than strictly needed while building.
E.g. a build including a very large 1.3GB assembly, with several
native libraries embedded, save a lot of time avoiding the rewrites
mtouch (before)
Total time: 64202 ms
mtouch (after)
Total time: 34840 ms
* Add XM support for RemoveUserResourcesSubStep
* Tests supplied by @chamons
Run the partial static registrar separately for 32-bit and 64-bit, since this
is required with the upcoming changes to embed metadata tokens inside the
generated output, because the metadata tokens are different between the 32-bit
and 64-bit versions of Xamarin.iOS.dll.
Also make sure to properly resolve the 32-bit and 64-bit assemblies correctly
(by setting the ArchDirectory on the assembly resolver), so that we don't pick
up the reference assembly (which does not have the right metadata tokens).
Additionally stop running the partial static registrar for MonoTouch.Dialog-1,
since with the upcoming changes to use metadata tokens in the generated
output, we won't support registering anything more than once. This shouldn't
make much of an impact, because MonoTouch.Dialog-1 is fairly small, and
doesn't take long to register in the dynamic registrar. For device builds (or
when the static registrar is selected) this has no effect, since in that case
we're registering everythinga anyway.
Link the executable with LinkWith attributes' libraries in the simulator when
doing incremental builds, since we don't create dylibs there.
Fixes several mtouch test failures:
1) Test Failure : Xamarin.MTouch.FastDev_NoFastSim_LinkAll(Unified)
2) Test Failure : Xamarin.MTouch.FastDev_NoFastSim_LinkAll(TVOS)
3) Test Failure : Xamarin.MTouch.FastDev_NoFastSim_LinkSDK(Unified)
4) Test Failure : Xamarin.MTouch.FastDev_NoFastSim_LinkSDK(TVOS)
5) Test Failure : Xamarin.MTouch.FastDev_NoFastSim_NoLink(Unified)
6) Test Failure : Xamarin.MTouch.FastDev_NoFastSim_NoLink(TVOS)
7) Test Failure : Xamarin.MTouch.FastDev_Sim(Unified)
8) Test Failure : Xamarin.MTouch.FastDev_Sim(TVOS)
Update mdb files even if the corresponding assembly didn't change, because the
mdb can change even if the assembly didn't (if whitespace was modified in the
source code, causing code lines to move).
https://bugzilla.xamarin.com/show_bug.cgi?id=39535
If the cache is invalid we print a warning:
> A full rebuild will be performed because the cache is either incomplete or entirely missing.
and set `Driver.Force = true;` (in Application.cs).
This later means that extracting the native code is done on each target:
```
public static bool IsUptodate (string source, string target)
{
if (Driver.Force)
return false;
```
even if this is identical between 32 and 64 bits (targets). That's
inefficient and, for large binding libraries (e.g. > 1GB), has a
noticable impact on build time (see timestamps).
Considering that the cache is cleaned (when detected as invalid) then
this Force condition is not really needed.
E.g. in `IsUptodate`
* the first time (arch) it's called will have to extract the native library;
* if a 2nd arch is built (fat) then it will be found as present and will
not be extracted again
Removing the `Driver.Force = true;` in this condition let the `-f` option
continue to extract it twice, which can be useful in debugging and testing.
As such the check is not removed from `IsUptodate`
Timestamps (before)
Setup: 25 ms
Resolve References: 1605 ms
Extracted native link info: 10465 ms
...
Timestamps (after)
Setup: 24 ms
Resolve References: 1560 ms
Extracted native link info: 5473 ms
...
Total build times (from XS) was around 90-100 seconds so 5 seconds is
about 10%. The actual savings will depend on how much native code needs
to be extracted, but it should help most release builds (almost always
fat builds).
IIRC this used to be needed with a (rather old) version of Cecil.
The current code does nothing as the Load will only hit the cache
and not other properties are changed (compared to the old commented
code)
The SDK does not ship any, non-tool, .exe files. In fact .exe _should_
never need to be resolved since the main .exe is always given as an input
to mtouch.
Still it's possible (if quite uncommon) to refer to other .exe assemblies
just like if they were .dll. However those can only come from the
RootDirectory (and not the other places that ship with the SDK).
This "fix" ensure lookups for *.exe is done only inside the RootDirectory
location, speeding up (a bit) resolving assemblies.
Native libraries are already linked into the dylib for the binding assembly,
which means that if we also link it into the main executable, the native code
ends up twice in the app (which is bad for many reasons).
https://bugzilla.xamarin.com/show_bug.cgi?id=42473
The container app may not reference the same third-party frameworks as
extensions, which means that we must make sure the extension's frameworks are
also included in the app bundle.
So when building extensions save a list of all third-party frameworks, and
then read that list and include those frameworks when building the main app.
https://bugzilla.xamarin.com/show_bug.cgi?id=45800
Managed exception marshaling interferes with the debugger, because it adds
exception handlers to executing code, which makes the Mono runtime think an
exception is handled when logically it's not (although technically it is).
The consequence is that the IDEs will only be notified when we re-throw the
exception after catching it, making it impossible for the IDEs to stop when
the exception is thrown (they will instead stop when we re-throw the
exception).
So disable managed exception marshaling (unless the user changed the default
behavior) when a debugger is attached.
This is the same behavior as Xamarin.Android.
https://bugzilla.xamarin.com/show_bug.cgi?id=45116
This makes it possible to set linker flags per assembly:
[assembly: LinkWith (LinkerFlags = "-lsqlite3")]
Which is required when incremental builds is enabled and a particular assembly
needs special linker flags (because we don't propagate the global -gcc_flags
to each dylib we build when doing incremental builds).
Also add an option to set the dlsym mode for an assembly (using the LinkWith
attribute).
Setting this option prints this to stdout:
> Mono Warning: option gen-compact-seq-points is deprecated.
and it's ignored, so just don't set this option.
Unify the code to determine whether a particular return type requires a stret
signature or not between the generator and platform assemblies.
Also fix the stret detection for armv7k, whose calling convention is not
identical to armv7(s): there's the concept of homogeneous structures, which
contains multiple elements of only one type, and which is sometimes passed in
registers on armv7k.
https://bugzilla.xamarin.com/show_bug.cgi?id=44709
Fix NullReferenceException in PInvoke wrapper generation when incremental
builds are enabled and the linker didn't run because cached results were
found.
https://bugzilla.xamarin.com/show_bug.cgi?id=44763
In the recent Cecil update Cecil changed from loading assemblies in memory to
keep a FileStream around and read whenever necessary [1].
This is problematic for us, because we need all the AssemblyDefinitions in
memory at once, and if there are many assemblies, then we'll run into problems
due to the number of file descriptors in use.
So revert to the behavior for the old Cecil: loading assemblies in memory,
unless the assemblies are big, since in that case we might run out of memory
otherwise.
http://cecil.pe/post/149243207656/mono-cecil-010-beta-1
* [tests] Remove Classic SDK tests.
* Remove XI/Classic support.
This also means we can remove support for the legacy registrars.
* [monotouch-test] Remove legacy registrar tests.
* [tests/mtouch] Remove Classic tests (and legacy registrar logic).
* [tests/scripted] Fix tests to reference Xamarin.iOS.dll.
The MT2001 error is a general, something went bad, in the linker code
base. The stack trace is often enough to track down issues but in some
cases it would be easier to ask customers for a specific assembly
(rather than their complete project) to investigate an issue.
Example:
error MT2103: Binding Optimizer failed processing `System.Void GoogleConversionTracking.Unified.GoogleConversionPing::.ctor()`.
--- inner exception
System.NullReferenceException: Object reference not set to an instance of an object
at MonoTouch.Tuner.OptimizeGeneratedCodeSubStep.ProcessIsDirectBinding (Mono.Cecil.MethodDefinition caller, Mono.Cecil.Cil.Instruction ins) [0x00026] in /Users/poupou/git/xamarin/xamarin-macios/tools/linker/MonoTouch.Tuner/OptimizeGeneratedCodeSubStep.cs:264
at MonoTouch.Tuner.OptimizeGeneratedCodeSubStep.ProcessCalls (Mono.Cecil.MethodDefinition caller, Int32 i) [0x00337] in /Users/poupou/git/xamarin/xamarin-macios/tools/linker/MonoTouch.Tuner/OptimizeGeneratedCodeSubStep.cs:197
at MonoTouch.Tuner.OptimizeGeneratedCodeSubStep.Process (Mono.Cecil.MethodDefinition method) [0x0007b] in /Users/poupou/git/xamarin/xamarin-macios/tools/linker/MonoTouch.Tuner/OptimizeGeneratedCodeSubStep.cs:81
at Xamarin.Linker.StateSubStep.ProcessMethod (Mono.Cecil.MethodDefinition method) [0x00004] in /Users/poupou/git/xamarin/xamarin-macios/tools/linker/CoreOptimizeGeneratedCode.cs:48
---
at Xamarin.Linker.StateSubStep.ProcessMethod (Mono.Cecil.MethodDefinition method) [0x00014] in /Users/poupou/git/xamarin/xamarin-macios/tools/linker/CoreOptimizeGeneratedCode.cs:50
at Mono.Tuner.SubStepDispatcher.DispatchMethod (Mono.Cecil.MethodDefinition method) [0x0001d] in /Users/poupou/git/xamarin/xamarin-macios/external/mono/mcs/tools/tuner/Mono.Tuner/Dispatcher.cs:215
at Mono.Tuner.SubStepDispatcher.BrowseMethods (ICollection methods) [0x0001c] in /Users/poupou/git/xamarin/xamarin-macios/external/mono/mcs/tools/tuner/Mono.Tuner/Dispatcher.cs:167
at Mono.Tuner.SubStepDispatcher.BrowseTypes (ICollection types) [0x0006b] in /Users/poupou/git/xamarin/xamarin-macios/external/mono/mcs/tools/tuner/Mono.Tuner/Dispatcher.cs:145
at Mono.Tuner.SubStepDispatcher.BrowseAssemblies (IEnumerable`1 assemblies) [0x00050] in /Users/poupou/git/xamarin/xamarin-macios/external/mono/mcs/tools/tuner/Mono.Tuner/Dispatcher.cs:123
at Mono.Tuner.SubStepDispatcher.Process (Mono.Linker.LinkContext context) [0x0000f] in /Users/poupou/git/xamarin/xamarin-macios/external/mono/mcs/tools/tuner/Mono.Tuner/Dispatcher.cs:104
at Mono.Linker.Pipeline.Process (Mono.Linker.LinkContext context) [0x00027] in /Users/poupou/git/xamarin/xamarin-macios/external/mono/mcs/tools/linker/Mono.Linker/Pipeline.cs:118
at MonoTouch.Tuner.Linker.Process (MonoTouch.Tuner.LinkerOptions options, MonoTouch.Tuner.MonoTouchLinkContext& context, System.Collections.Generic.List`1& assemblies) [0x000ac] in /Users/poupou/git/xamarin/xamarin-macios/tools/mtouch/Tuning.cs:79
Right now the MT2001 would only include the inner exception, which does
not include any clue to which assembly caused the exception.
Note: The same pattern to be applied to other BaseSubStep subclasses in
separate commits.
Related to (but not the fix for) https://bugzilla.xamarin.com/show_bug.cgi?id=44701
The nested Parallel.ForEach loops don't take into account the outer
MaxDegreeOfParallelism value, causing us to spawn more concurrent tasks than
we want to.
So refactor the code to have one static list of tasks, which each subtask adds
to.
The nested Parallel.ForEach loops don't take into account the outer
MaxDegreeOfParallelism value, causing us to spawn more concurrent tasks than
we want to.
So refactor the code to have one static list of tasks, which each subtask adds
to.
* Fix default http message handler for watchOS.
Fix default http message handler for watchOS to be NSUrlSessionHandler (the
previous attempt at eb7c2fd was quite incomplete), and make sure
HttpClientHandler is never used (show errors if someone tries).
* [tests] Remove explicit http client handler from project files.
Just use the default instead, since the set of valid http client handlers varies between platforms.
Rolf Bjarne Kvinge
Sebastien Pouliot
Chris Hamons
Sebastien Pouliot