The arm64_32 slice for watchOS apps will always use the 'unified' mode, while
the armv7k can be both 'unified' and 'compat' depending on the deployment
target, so we need to keep track of this per Target.
This PR does not change anything related to arm64_32, that will come in a
later PR.
Fixes this NRE:
error MT0000: Unexpected error - Please file a bug report at https://github.com/xamarin/xamarin-macios/issues/new
System.ArgumentNullException: Value cannot be null.
Parameter name: collection
at System.Collections.Generic.List`1[T].InsertRange (System.Int32 index, System.Collections.Generic.IEnumerable`1[T] collection) [0x00003] in <a104f9cbbafd4348bcc580acb0a3f8a8>:0
at System.Collections.Generic.List`1[T].AddRange (System.Collections.Generic.IEnumerable`1[T] collection) [0x00000] in <a104f9cbbafd4348bcc580acb0a3f8a8>:0
at Xamarin.Bundler.Application.BuildApp () [0x00040] in /work/maccore/master/xamarin-macios/tools/mtouch/Application.cs:1541
at Xamarin.Bundler.Application.BuildNative () [0x0001f] in /work/maccore/master/xamarin-macios/tools/mtouch/Application.cs:903
at Xamarin.Bundler.Application+<>c.<BuildAll>b__148_2 (Xamarin.Bundler.Application v) [0x00000] in /work/maccore/master/xamarin-macios/tools/mtouch/Application.cs:840
at System.Collections.Generic.List`1[T].ForEach (System.Action`1[T] action) [0x0001e] in <a104f9cbbafd4348bcc580acb0a3f8a8>:0
at Xamarin.Bundler.Application.BuildAll () [0x00076] in /work/maccore/master/xamarin-macios/tools/mtouch/Application.cs:840
at Xamarin.Bundler.Driver.Main2 (System.String[] args) [0x004a1] in /work/maccore/master/xamarin-macios/tools/mtouch/mtouch.cs:1369
at Xamarin.Bundler.Driver.Main (System.String[] args) [0x00015] in /work/maccore/master/xamarin-macios/tools/mtouch/mtouch.cs:877
According to Rolf it's fine to always add since the native linker will
figure out if it's really needed and so customers don't need to do
anything when using -all_load.
Mono started using System.IO.File from CoreFX and it has a different
behavior for File.Copy() when the target is a directory:
It just puts the file into the directory instead of raising an exception.
In order to fix the MT1015 test we just check ourselves whether the target
is a directory.
This also centralize other interpreter checks and options in the same
location (making it easier to read / update).
* Warn and switch the REPL if the interpreter is enabled on simulator
Why ? It's confusing to build the same code using different options for
simulator and devices. This is what happens if you try to use features
like `dynamic` or `System.Reflection.Emit`.
So instead of an error, we warn that the interpreter is not supported
and switch to the existing REPL mode.
The JIT remains the only option for the simulator but it allows testing
features without a device.
* Fail early if the interpreter is used on 32bits [1]
The current interpreter only works on 64 bits (so ARM64). However the
error won't be reported, back to the developer, until deployment time.
This temporary [1] fix spot the condition very early and report an error
```
error MT0099 : Internal error : The interpreter is currently only available for 64 bits.
```
instead of the current one at deploy time
```
IncorrectArchitecture: Failed to find matching arch for 32-bit Mach-O input file /private/var/installd/Library/Caches/com.apple.mobile.installd.staging/temp.tNKDlx/extracted/X.app/X
error MT1006: Could not install the application 'X.app' on the device 'Mercure': AMDeviceSecureInstallApplicationBundle returned: 0xe8000087 (kAMDIncorrectArchitectureError).
Application could not be uploaded to the device.
```
[1] https://github.com/mono/mono/issues/9871
* [tests] Fix/renumbered MT0138
The test was using simulator + interpreter which is not _really_
possible, we use REPL in that case - so we're now checking if
assemblies were specified with `--interpreter` to cover both cases.
Also 0138 was already used by `mmp` and the warning was **not**
registered or documented in the errors documents. To avoid
confusion it has been renumbered to 0142 and documented.
In both cases we use a different binaries for a few assemblies. They
include support for SRE and a few other things that are not normally
usable on iOS. That's totally fine and not something that can be fixed
(unless you stop using the feature). So this PR simply ignore that
specific case so we don't warn about things that can't be changed (and
are not a problem)
E.g.
```
Warning MT0109: The assembly 'mscorlib.dll' was loaded from a different path than the provided path (provided path: /Users/poupou/git/intr/xamarin-macios/_ios-build/Library/Frameworks/Xamarin.iOS.framework/Versions/Current/lib/mono/Xamarin.iOS/mscorlib.dll, actual path: /Users/poupou/git/intr/xamarin-macios/_ios-build/Library/Frameworks/Xamarin.iOS.framework/Versions/git/lib/mono/Xamarin.iOS/repl/mscorlib.dll). (MT0109) (XXX)
Warning MT0109: The assembly 'System.Core.dll' was loaded from a different path than the provided path (provided path: /Users/poupou/git/intr/xamarin-macios/_ios-build/Library/Frameworks/Xamarin.iOS.framework/Versions/Current/lib/mono/Xamarin.iOS/System.Core.dll, actual path: /Users/poupou/git/intr/xamarin-macios/_ios-build/Library/Frameworks/Xamarin.iOS.framework/Versions/git/lib/mono/Xamarin.iOS/repl/System.Core.dll). (MT0109) (XXX)
Warning MT0109: The assembly 'System.dll' was loaded from a different path than the provided path (provided path: /Users/poupou/git/intr/xamarin-macios/_ios-build/Library/Frameworks/Xamarin.iOS.framework/Versions/Current/lib/mono/Xamarin.iOS/System.dll, actual path: /Users/poupou/git/intr/xamarin-macios/_ios-build/Library/Frameworks/Xamarin.iOS.framework/Versions/git/lib/mono/Xamarin.iOS/repl/System.dll). (MT0109) (XXX)
Warning MT0109: The assembly 'System.Xml.dll' was loaded from a different path than the provided path (provided path: /Users/poupou/git/intr/xamarin-macios/_ios-build/Library/Frameworks/Xamarin.iOS.framework/Versions/Current/lib/mono/Xamarin.iOS/System.Xml.dll, actual path: /Users/poupou/git/intr/xamarin-macios/_ios-build/Library/Frameworks/Xamarin.iOS.framework/Versions/git/lib/mono/Xamarin.iOS/repl/System.Xml.dll). (MT0109) (XXX)
```
Clang will automatically compile .bc files into object code if needed, but
it's done serially. If we do the compilation ourselves, it'll be parallelized.
This makes the dontlink tests build in half the time when building for watchOS
/ Release (it drops from ~15 minutes to ~7 minutes).
* [mtouch] Don't use the native linker to create fat executables.
Don't use the native linker to create fat executables, instead link each
architecture separately, and then manually lipo everything together at the
end. This requires a few changes since we need to keep track of the linker
flags per architecture.
The problem is that bitcode files (.bc) do not correspond with a particular
architecture, so the linker can't distinguish between .bc files for armv7k and
.bc files for arm64_32. So if we pass all together to the linker, the linker
will add all .bc files to both architectures, thus duplicating everything (and
the linking fails with duplicate symbols errors).
* [mtouch] Fix building symlinked simulator executables.
* [mtouch] Fix several assumptions about each Target only producing a single executable.
* Add a Runtime.IsARM64CallingConvention property.
Determining whether we should use the ARM64 calling convention in P/Invokes
gets more complicated with ARM64_32 (which for our purposes is a 32-bit
architecture).
So add a property on the Runtime class to avoid code duplication, and teach
the linker to optimize any calls to this property to a constant value whenever
possible (and the method is marked as optimizable).
Also change our code to use this new property, and make the corresponding
methods optimizable.
Some shuffling in mmp was required, which meant a little bit more code is now
shared between mtouch and mmp.
* Coding style.
* Test tweaks.
* Improve comment.
* Document new optimization
* Move ILReader to shared linker test code location.
* Disable inlining on armv7k.
* Change IsARM64CallingConvention to a read-only field.
We can give the AOT compiler a constant value for a read-only field, so that
the AOT compiler optimizes away the call to the field by using the constant
value.
This commit does not implement this change for the AOT compiler, but using a
read-only field makes it easy to implement it in the future.
* [linker] Remove non-bitcode compatible code, and show a warning.
Remove code not currently compatible with bitcode and replace it with an
exception instead (otherwise we'll assert at runtime).
Also show a warning when we detect this.
This is quite helpful when looking at watch device test runs to filter out
failures we already know about.
This fixes point #2 in #4763.
* Improve documentation.
* Simplify linker code by using a substep.
* Fix whitespace issues.
* Improve reporting.
* Add support for reporting more than one MT2105 at the same time when making
the errors instead of warnings.
* Only report MT2105 for methods that haven't been linked away.
* Format the error message nicer for properties.
* Tweak a bit for warning tests to pass.
* Use ExceptionalSubStep to provide better error information.
* Adjust where linker warnings/errors are reported from to avoid a NullReferenceException.
* Bump to mono:2018-06
* Bump mono
* Updates compression to work with the public span
* Bump mono
* Fixes pointer check logic in Deflater
* Bump mono
* Fixes pointer check logic in Deflater
* Bump mono
* Bump Mono
* [runtime] always use `mono_jit_set_aot_mode` (#4491)
`mono_jit_set_aot_only` is deprecated and accidentally broke with
https://github.com/mono/mono/pull/7887
This should fix device tests with `mono-2018-06`
* Testing with Zoltan's patch
* Include libmono-system-native on Xamarin.Mac
* Bump Mono
Commit list for mono/mono:
* mono/mono@7bcda192a0 Bump llvm to release_60/fc854b8ec5873d294b80afa3e6cf6a88c5c48886. (#9786). (#9804)
* mono/mono@23e95ec7ad Apply F# portable pdb debug fix for pinvokes & bump (#9797)
* mono/mono@295f6d32af [2018-06] [MacOS] On Mac, use the copyfile API to copy files (#9696)
Diff: 7d5f4b6136...7bcda192a0
* Revert 4bacab3d5c, it doesn't fix the ios aot problems.
* Bump mono
* [tests] Adjust the MT0137 test for mcs change in behavior.
Starting with mono 5.16 mcs will now add assembly references when the assembly
is only used in attributes (this was already the case for csc in both 5.14 and
5.16, so it seems to be a compatibility change).
Adjust the MT0137 test accordingly.
* [msbuild] Fix parsing of json parser errors to handle trailing periods in the error message.
Fixes this test:
1) Test Failure : Xamarin.iOS.Tasks.Bug60536.TestACToolTaskCatchesJsonException
ColumnNumber
Expected: 2
But was: 0
* Bump mono
* [builds] Install the old llvm binaries into the LLVM36 directory and make the 32 bit builds use that.
* Bump mono
* Bump mono
* [jenkins] Don't give VSTS a fake branch. (#4667)
Something in VSTS changed, and now fake branch names don't work anymore.
So instead use real branch names (and for pull requests I've created a
'pull-request' branch we can use).
* Assembly.LoadFile accepts only absolute path
* [linker] Add new Facade (System.Threading.Tasks.Extensions).
Fixes these MTouch test failures:
1. Xamarin.Linker.SdkTest.iOS_Unified : Facades
Expected:
But was: < "System.Threading.Tasks.Extensions" >
2. Xamarin.Linker.SdkTest.tvOS : Facades
Expected:
But was: < "System.Threading.Tasks.Extensions" >
3. Xamarin.Linker.SdkTest.watchOS : Facades
Expected:
But was: < "System.Threading.Tasks.Extensions" >
* [mono-sdks] Necessary changes to unify the LLVM provisioning for both iOS and Android. (#4732)
* Bump Mono
* [mtouch] add mixed-mode support (#4751)
* [mtouch] add --interp-mixed option
When enabling this option, mtouch will AOT compile `mscorlib.dll`. At
runtime that means every method that wasn't AOT'd will be executed by
the runtime interpreter.
* [mtouch] Add support to --interpreter to list the assemblies to (not) interpret.
* [msbuild] Simplify interpreter code to use a single variable.
* Fix whitespace.
* [mtouch] Move mtouch-specific code to mtouch-specific file.
* [msbuild] An empty string is a valid value for 'Interpreter', so make it a non-required property.
* [mtouch] Add sanity check for aot-compiling interpreted assemblies.
* Bump Mono
* [linker] Updates SDKs facades list
* Bump mono
* [msbuild] Adds facades which might override default nuget version to framework list
The collision resolver task reads them from here https://github.com/dotnet/sdk/blob/master/src/Tasks/Common/ConflictResolution/FrameworkListReader.cs
* Bump to a VSfM version that can build XM Classic projects.
* [mtouch] Show warnings when we can't find referenced assemblies.
This would have helped track down #4235.
* Improve MT0137 warning to indicate the type of the attribute causing the warning.
* Bump to use Xcode 10 beta 1
* Update Versions.plist
* Add a dependency on Xcode 9.4.
* [msbuild] Fix build with Xcode 10 beta 1. (#4182)
Many years ago (in Xcode 7 according to code comment)
Developer/Platforms/iPhoneOS.platform/Developer/usr disappeared, and we coped
by looking at Developer/usr instead (and also the subsequent code to locate
the bin directory was based on the location of the usr directory).
Developer/Platforms/iPhoneOS.platform/Developer/usr reappeared in Xcode 10
beta 1, but it seems useless (for one it doesn't contain a bin directory), so
in order to try to keep things sane don't look for this directory in Xcode 10
and instead go directly for Developer/usr (which is what we've been using as
the usr directory for years anyway).
Fixes this problem when building apps with Xcode 10 beta 1:
/Library/Frameworks/Mono.framework/External/xbuild/Xamarin/iOS/Xamarin.iOS.Common.targets(626,3): error : Could not locate SDK bin directory [/Users/rolf/Projects/TestApp/test-app.csproj]
* [runtime] Build 32-bit mac executables using Xcode 9.4.
* [mtouch] Work around broken tvOS headers in Xcode 10 beta 1.
* [mtouch] Work around build problem with Apple's simd headers in Objective-C++ mode.
* Use version-agnostic paths to sdk directories.
* [tests][xtro] Add todo files (from unclassified) and adjust ignore files to avoid errors
* [macos][security] Re-enable SSL[Get|Set]AlpnProtocols. Fixes#4001 (#4022)
* [macos][security] Re-enable SSL[Get}Set]AlpnProtocols. Fixes#4001
This was fixed in macOS 10.13.4
https://github.com/xamarin/xamarin-macios/issues/4001
* [tests][monotouch-tests] Disable a few test cases (one crasher, other failures). Causes to be verified later
* [xharness] Fix permission dialog suppression in Xcode 10.
* [xharness] Ignore 32-bit macOS tests by default.
* [tests] Execute mmp regression tests with Xcode 9.4 since many of them are 32-bit and needs porting to 64-bit.
* [mmptest] Ignore 32-bit XM tests if we don't have a 32-bit-capable Xcode.
* [registrar] Add workaround for broken headers in Xcode 10 beta 1 (radar 40824697).
* [mtouch] Restrict another workaround for an Xcode 10 beta 1 bug to a specific Xcode version to remove it asap.
* [tests] Fix some protocol changes (public or not) find by introspection tests
* [tests][intro] Fix DefaultCtorAllowed failures
* [Intents] Obsolete several Intents classes in watchOS.
Several existing Intents classes have been marked as unavailable in watchOS in
the headers in Xcode 10 beta 1, and corresponding tests are now failing.
So obsolete the managed wrapper types, and fix tests accordingly.
* Fix xtro wrt previous Ietents/intro changes
* [tests] Minor adjustments to mtouch tests to work with Xcode 10.
* [msbuild] Update tests to cope with additional files produced by the Core ML compiler.
* [msbuild] Xcode 10 doesn't support building watchOS 1 apps, so show a clear error message explaining it.
Also update tests accordingly.
* [coreimage] Stub new filters and exclude ?removed? ones from tests
* Update GameplayKit and SpriteKit NSSecureCoding _upgrade_ and fix other non-public cases (in tests)
* [tests] Ignore some GameKit selectors that don't respond anymore (but seems to be available, at least in header files)
* [tests] Fix intro 32bits testing for filters resutls
* [msbuild] Slightly change error message to be better English.
* Share the stopwatch code between `mtouch` and `mmp`;
* Add more details on linker steps. Sadly substeps are executed on the
metadata so they can't be reported individually;
* Add a few places where timestamps where missing to get better precision
on the linking part;
On my test project we were calling `IsArchEnabled` 334,713 times which
made the enumerator called enough to show on the memory usage (13MB),
even if each instance is only 40 bytes.
Caching the result makes the method called only 5 times.
* [runtime] build interp-, icalltable- and ilgen libraries so they can be consumed in interpreter configuration
* [mtouch] add --interpreter configuration support
* [xharness] Add support for removing defines from cloned project files.
* [xharness] Run monotouch-test, mscorlib and mini tests with the interpreter too.
They're ignored for now, which means we won't run them on the bots.
* [mtouch] We're using dlsym when using the interpreter.
* [xharness] enable mini regression tests with interpreter on CI
Assemblies that satisfy all of these conditions:
* Are not in the container project.
* Are in multiple app extensions.
can (and should) be bundled in the container app.
Fixes https://bugzilla.xamarin.com/show_bug.cgi?id=58873
When code sharing is enabled, only the container app/target will have a
LinkContext.
So make sure the static registrar finds that link context when it needs
information from the linker.
https://github.com/xamarin/xamarin-macios/issues/3514
When code sharing is enabled, only the container app/target will have a
LinkContext.
So make sure the static registrar finds that link context when it needs
information from the linker.
https://github.com/xamarin/xamarin-macios/issues/3514
* [tests] Improve debug spew for the RebuildTest_WithExtensions test.
* [mtouch/mmp] Store/load if the dynamic registrar is removed or not into the cached link results.
Store/load if the dynamic registrar is removed or not into the cached link
results, so that we generate the correct main.m even if cached linker results
are used.
* [mtouch/mmp] The static registrar must not execute if we're loading cached results from the linker.
The static registrar must not execute if we're loading cached results from the
linker, because the static registrar needs information from the linker that's
not restored from the cache.
* [mtouch/mmp] Share Touch code.
* [mtouch/mmp] Make it possible to touch inexistent files (to create them).
* [mtouch/mmp] Fix tracking of whether the static registrar should run again or not.
The recent changes to support optimizing away the dynamic registrar caused the
Xamarin.MTouch.RebuildTest_WithExtensions test to regress.
The problem
-----------
* The linker now collects and stores information the static registrar needs.
* This information is not restored from disk when the linker realizes that it
can reload previously linked assemblies instead of executing again.
* The static registrar runs again (for another reason).
* The information the static registrar needs isn't available, and incorrect
output follows.
So fix 1: show an error if the static registrar runs when the linker loaded
cached results.
The exact scenario the test ran into is this:
* 1st build: everything is new and everything is built.
* 2nd build: contents of .exe changes, the linker runs again, the static
registrar runs again, but sees that the generated output didn't change, so
it doesn't write the new content to disk (this is an optimization to avoid
compiling the registrar.m file again unless needed).
* 3rd build: only the .exe timestamp changes, the linker sees nothing changes
in the contents of the .exe and loads the previously linked assemblies from
disk, the static registrar sees that the .exe's timestamp is newer than
registrar.m's timestamp and run again, but doesn't produce the right result
because it doesn't have the information it needs.
Considered solutions
--------------------
1. Only track timestamps, not file contents. This is not ideal, since it will
result in more work done: in particular for the case above, it would add a
registrar.m compilation in build #2, and linker rerun + static registrar
rerun + registrar.m compilation + final native link in build #3.
2. Always write the output of the static registrar, even if it hasn't changed.
This is not ideal either, since it will also result in more work done: for
the case above, it would add a registrar.m compilation + final native link
in build #3.
3. Always write the output of the static registrar, but track if it changed or
not, and if it didn't, just touch registrar.o instead of recompiling it.
This only means the final native link in build #3 is added (see #5 for why
this is worse than it sounds).
4. Always write the output of the static registrar, but track it it changed or
not, and if it didn't, just touch registrar.o instead of recompiling it,
and track that too, so that the final native link in build #3 isn't needed
anymore. Unfortunately this may result in incorrect behavior, because now
the msbuild tasks will detect that the executable has changed, and may run
dsymutil + strip again. The executable didn't actually change, which means
it would be the previously stripped executable, and thus we'd end up with
an empty .dSYM because we ran dsymtil on an already stripped executable.
5. Idea #4, but write the output of the final link into a temporary directory
instead of the .app, so that we could track whether we should update the
executable in the .app or not. This is not optimal either, because
executables can be *big* (I've seen multi-GB tvOS bitcode executables), and
extra copies of such files should not be taken lightly.
6. Idea #4, but tell the MSBuild tasks that dsymutil/strip doesn't need to be
rerun even if the timestamp of the executable changed. This might actually
work, but now the solution's become quite complex.
Implemented solution
--------------------
Use stamp files to detect whether a file is up-to-date or not.
In particular:
* When we don't write to a file because the new contents are identical to the
old contents, we now touch a .stamp file. This stamp file means "the
accompanying file was determined to be up-to-date when the stamp was
touched."
* When checking whether a file is up-to-date, also check for the presence of a
.stamp file, and if it exists, use the highest timestamp between the stamp
file and the actual file.
Now the test scenario becomes:
* 1st build: everything is new and everything is built.
* 2nd build: contents of .exe changes, the linker runs again, the static
registrar runs again, but sees that the generated output didn't change, so
it doesn't write the new content to disk, but it creates a registrar.m.stamp
file to indicate the point in time when registrar.m was considered up-to-
date.
* 3rd build: only the .exe timestamp changes, the linker sees nothing changes
in the contents of the .exe and loads the previously linked assemblies from
disk, the static registrar sees that the .exe's timestamp is *older* than
registrar.m.stamp's timestamp and doesn't run again.
We only use the stamp file for source code (registrar.[m|h], main.[m|h],
pinvokes.[m|h]), since using it every time has too much potential for running
into other problems (for instance we should never create .stamp files inside
the .app).
Fixes these test failures:
1) Failed : Xamarin.MTouch.RebuildTest_WithExtensions("single","",False,System.String[])
single
Expected: <empty>
But was: < "/Users/builder/data/lanes/5746/4123bf7e/source/xamarin-macios/tests/mtouch/bin/Debug/tmp-test-dir/Xamarin.Tests.BundlerTool.CreateTemporaryDirectory371/testApp.app/testApp is modified, timestamp: 2/15/2018 3:04:11 PM > 2/15/2018 3:04:09 PM" >
2) Failed : Xamarin.MTouch.RebuildTest_WithExtensions("dual","armv7,arm64",False,System.String[])
dual
Expected: <empty>
But was: < "/Users/builder/data/lanes/5746/4123bf7e/source/xamarin-macios/tests/mtouch/bin/Debug/tmp-test-dir/Xamarin.Tests.BundlerTool.CreateTemporaryDirectory375/testApp.app/testApp is modified, timestamp: 2/15/2018 3:06:03 PM > 2/15/2018 3:06:00 PM" >
3) Failed : Xamarin.MTouch.RebuildTest_WithExtensions("llvm","armv7+llvm",False,System.String[])
llvm
Expected: <empty>
But was: < "/Users/builder/data/lanes/5746/4123bf7e/source/xamarin-macios/tests/mtouch/bin/Debug/tmp-test-dir/Xamarin.Tests.BundlerTool.CreateTemporaryDirectory379/testApp.app/testApp is modified, timestamp: 2/15/2018 3:07:14 PM > 2/15/2018 3:07:12 PM" >
4) Failed : Xamarin.MTouch.RebuildTest_WithExtensions("debug","",True,System.String[])
debug
Expected: <empty>
But was: < "/Users/builder/data/lanes/5746/4123bf7e/source/xamarin-macios/tests/mtouch/bin/Debug/tmp-test-dir/Xamarin.Tests.BundlerTool.CreateTemporaryDirectory383/testApp.app/testApp is modified, timestamp: 2/15/2018 3:08:16 PM > 2/15/2018 3:08:13 PM" >
5) Failed : Xamarin.MTouch.RebuildTest_WithExtensions("single-framework","",False,System.String[])
single-framework
Expected: <empty>
But was: < "/Users/builder/data/lanes/5746/4123bf7e/source/xamarin-macios/tests/mtouch/bin/Debug/tmp-test-dir/Xamarin.Tests.BundlerTool.CreateTemporaryDirectory387/testApp.app/testApp is modified, timestamp: 2/15/2018 3:09:18 PM > 2/15/2018 3:09:16 PM" >
Fixes https://github.com/xamarin/maccore/issues/641
* [mtouch] Rename temporary directories so the order assemblies move is clearer.
I implemented this myself, but I can never remember in which order assemblies
go from one directory to another during the build.
So number these temporary directories, so that even the most forgetful minds
can understand without having to remember anything.
* [tests] Update test according to temporary directory name change.
* [tests] Improve debug spew for the RebuildTest_WithExtensions test.
* [mtouch/mmp] Store/load if the dynamic registrar is removed or not into the cached link results.
Store/load if the dynamic registrar is removed or not into the cached link
results, so that we generate the correct main.m even if cached linker results
are used.
* [mtouch/mmp] The static registrar must not execute if we're loading cached results from the linker.
The static registrar must not execute if we're loading cached results from the
linker, because the static registrar needs information from the linker that's
not restored from the cache.
* [mtouch/mmp] Share Touch code.
* [mtouch/mmp] Make it possible to touch inexistent files (to create them).
* [mtouch/mmp] Fix tracking of whether the static registrar should run again or not.
The recent changes to support optimizing away the dynamic registrar caused the
Xamarin.MTouch.RebuildTest_WithExtensions test to regress.
The problem
-----------
* The linker now collects and stores information the static registrar needs.
* This information is not restored from disk when the linker realizes that it
can reload previously linked assemblies instead of executing again.
* The static registrar runs again (for another reason).
* The information the static registrar needs isn't available, and incorrect
output follows.
So fix 1: show an error if the static registrar runs when the linker loaded
cached results.
The exact scenario the test ran into is this:
* 1st build: everything is new and everything is built.
* 2nd build: contents of .exe changes, the linker runs again, the static
registrar runs again, but sees that the generated output didn't change, so
it doesn't write the new content to disk (this is an optimization to avoid
compiling the registrar.m file again unless needed).
* 3rd build: only the .exe timestamp changes, the linker sees nothing changes
in the contents of the .exe and loads the previously linked assemblies from
disk, the static registrar sees that the .exe's timestamp is newer than
registrar.m's timestamp and run again, but doesn't produce the right result
because it doesn't have the information it needs.
Considered solutions
--------------------
1. Only track timestamps, not file contents. This is not ideal, since it will
result in more work done: in particular for the case above, it would add a
registrar.m compilation in build #2, and linker rerun + static registrar
rerun + registrar.m compilation + final native link in build #3.
2. Always write the output of the static registrar, even if it hasn't changed.
This is not ideal either, since it will also result in more work done: for
the case above, it would add a registrar.m compilation + final native link
in build #3.
3. Always write the output of the static registrar, but track if it changed or
not, and if it didn't, just touch registrar.o instead of recompiling it.
This only means the final native link in build #3 is added (see #5 for why
this is worse than it sounds).
4. Always write the output of the static registrar, but track it it changed or
not, and if it didn't, just touch registrar.o instead of recompiling it,
and track that too, so that the final native link in build #3 isn't needed
anymore. Unfortunately this may result in incorrect behavior, because now
the msbuild tasks will detect that the executable has changed, and may run
dsymutil + strip again. The executable didn't actually change, which means
it would be the previously stripped executable, and thus we'd end up with
an empty .dSYM because we ran dsymtil on an already stripped executable.
5. Idea #4, but write the output of the final link into a temporary directory
instead of the .app, so that we could track whether we should update the
executable in the .app or not. This is not optimal either, because
executables can be *big* (I've seen multi-GB tvOS bitcode executables), and
extra copies of such files should not be taken lightly.
6. Idea #4, but tell the MSBuild tasks that dsymutil/strip doesn't need to be
rerun even if the timestamp of the executable changed. This might actually
work, but now the solution's become quite complex.
Implemented solution
--------------------
Use stamp files to detect whether a file is up-to-date or not.
In particular:
* When we don't write to a file because the new contents are identical to the
old contents, we now touch a .stamp file. This stamp file means "the
accompanying file was determined to be up-to-date when the stamp was
touched."
* When checking whether a file is up-to-date, also check for the presence of a
.stamp file, and if it exists, use the highest timestamp between the stamp
file and the actual file.
Now the test scenario becomes:
* 1st build: everything is new and everything is built.
* 2nd build: contents of .exe changes, the linker runs again, the static
registrar runs again, but sees that the generated output didn't change, so
it doesn't write the new content to disk, but it creates a registrar.m.stamp
file to indicate the point in time when registrar.m was considered up-to-
date.
* 3rd build: only the .exe timestamp changes, the linker sees nothing changes
in the contents of the .exe and loads the previously linked assemblies from
disk, the static registrar sees that the .exe's timestamp is *older* than
registrar.m.stamp's timestamp and doesn't run again.
We only use the stamp file for source code (registrar.[m|h], main.[m|h],
pinvokes.[m|h]), since using it every time has too much potential for running
into other problems (for instance we should never create .stamp files inside
the .app).
Fixes these test failures:
1) Failed : Xamarin.MTouch.RebuildTest_WithExtensions("single","",False,System.String[])
single
Expected: <empty>
But was: < "/Users/builder/data/lanes/5746/4123bf7e/source/xamarin-macios/tests/mtouch/bin/Debug/tmp-test-dir/Xamarin.Tests.BundlerTool.CreateTemporaryDirectory371/testApp.app/testApp is modified, timestamp: 2/15/2018 3:04:11 PM > 2/15/2018 3:04:09 PM" >
2) Failed : Xamarin.MTouch.RebuildTest_WithExtensions("dual","armv7,arm64",False,System.String[])
dual
Expected: <empty>
But was: < "/Users/builder/data/lanes/5746/4123bf7e/source/xamarin-macios/tests/mtouch/bin/Debug/tmp-test-dir/Xamarin.Tests.BundlerTool.CreateTemporaryDirectory375/testApp.app/testApp is modified, timestamp: 2/15/2018 3:06:03 PM > 2/15/2018 3:06:00 PM" >
3) Failed : Xamarin.MTouch.RebuildTest_WithExtensions("llvm","armv7+llvm",False,System.String[])
llvm
Expected: <empty>
But was: < "/Users/builder/data/lanes/5746/4123bf7e/source/xamarin-macios/tests/mtouch/bin/Debug/tmp-test-dir/Xamarin.Tests.BundlerTool.CreateTemporaryDirectory379/testApp.app/testApp is modified, timestamp: 2/15/2018 3:07:14 PM > 2/15/2018 3:07:12 PM" >
4) Failed : Xamarin.MTouch.RebuildTest_WithExtensions("debug","",True,System.String[])
debug
Expected: <empty>
But was: < "/Users/builder/data/lanes/5746/4123bf7e/source/xamarin-macios/tests/mtouch/bin/Debug/tmp-test-dir/Xamarin.Tests.BundlerTool.CreateTemporaryDirectory383/testApp.app/testApp is modified, timestamp: 2/15/2018 3:08:16 PM > 2/15/2018 3:08:13 PM" >
5) Failed : Xamarin.MTouch.RebuildTest_WithExtensions("single-framework","",False,System.String[])
single-framework
Expected: <empty>
But was: < "/Users/builder/data/lanes/5746/4123bf7e/source/xamarin-macios/tests/mtouch/bin/Debug/tmp-test-dir/Xamarin.Tests.BundlerTool.CreateTemporaryDirectory387/testApp.app/testApp is modified, timestamp: 2/15/2018 3:09:18 PM > 2/15/2018 3:09:16 PM" >
Fixes https://github.com/xamarin/maccore/issues/641
There's only one static registrar now, so there's no need for code to select
which one, just create the one and only.
Also unify this code between mtouch and mmp.
You were the preprocessor we wished C# had natively
Removing PMCS requires these changes:
* Remove XamCore from src/
* Remove XamCore from tools/
* Remove XamCore from runtime/
* nint/nuint enum conversion
* _compat_ enum conversion
* NSAction conversion
* Hand fix single API incorrectly converted by PMCS to unbreak compatibility
- Due to a bug in PMCS, the nuint was incorrectly converted in this API.
- However, as that ship as sailed, we must "fix" it until XAMCORE_4_0
* Update readme
* Bump macios-binaries
* [mtouch/mmp] Give users more control over optimizations, and share more code between mtouch and mmp.
1. Add an --optimize flag to mtouch/mmp that allows users to select which
optimizations to apply (or not). This makes it easier to add future
optimizations, and allow users to disable any optimization that causes
problems without having to disable many other features.
2. Share as much optimization code as possible between mtouch and mmp. This
immediately gives a benefit to mmp, which has three new optimizations only
mtouch had: NSObject.IsDirectBinding inlining, IntPtr.Size inlining and
dead code elimination.
This results in ~6kb of disk space saved for a linked Xamarin.Mac app:
* link sdk: [Debug][1], [Release][2]
* link all: [Debug][3], [Release][4]
Testing also verifies that monotouchtest ([Debug][5], [Release][6]) has not
changed size at all, which means that no default optimizations have changed
inadvertedly.
[1]: https://gist.github.com/rolfbjarne/6b731e3b5ca6170355662e6505c3d492#link-sdk-mac--debug
[2]: https://gist.github.com/rolfbjarne/6b731e3b5ca6170355662e6505c3d492#link-sdk-mac--release
[3]: https://gist.github.com/rolfbjarne/6b731e3b5ca6170355662e6505c3d492#link-all-mac--debug
[4]: https://gist.github.com/rolfbjarne/6b731e3b5ca6170355662e6505c3d492#link-all-mac--release
[5]: https://gist.github.com/rolfbjarne/6b731e3b5ca6170355662e6505c3d492#monotouchtest-iphonedebug64
[6]: https://gist.github.com/rolfbjarne/6b731e3b5ca6170355662e6505c3d492#monotouchtest-iphonerelease64
* [tools] Don't enable the IsDirectBinding optimization by default for Xamarin.Mac apps, it's not safe.
* Fix whitespace issues.
* [doc] Document optimizations.
* Officially support optimizations by adding them to the Versions.plist.
* [linker] Improve IntPtr.Size inliner + dead code eliminatior and add tests.
* Properly handle operands for the ldc_i4_s instruction (they're sbyte).
* Fix less-than condition to actually do a less-than comparison.
* Make sure to look up the bitness in the Target, not the Application, since
the Application's value will be incorrect when building fat apps (both
Is32Build and Is64Build will be true).
* Remove unnecessary checks for the IntPtr.Size inliner: this optimization
does not depend on other instructions than the IntPtr.get_Size call, so
remove the checks that verify surrounding instructions. This makes the
IntPtr.Size inliner kick in in more scenarios (such as the new tests).
* Add tests.
* [tests] Add mmp tests for optimizations.
* [tests] Fix XM optimization tests.
* [tests] Fix test build error.
We can't trust Apple's native linker to pick the right (non private)
framework when an older TargetVersion is used. It just prefer what's
available - even if specified with a WeakFramework :(
That was already dealt with for applications. However the native linking
of the Xamarin.Sdk.framework (code sharing with extensions) is done with
the `LinkTask` instead of the `NativeLinkTask` so it did not have the
"auto correct" code.
Unit test added.
reference:
https://bugzilla.xamarin.com/show_bug.cgi?id=61039
Rolf Bjarne Kvinge
Alexander Köplinger
Sebastien Pouliot
Martin Baulig
Jo Shields
Chris Hamons
Vincent Dondain
Sebastien Pouliot
Bernhard Urban