Граф коммитов

143 Коммитов

Автор SHA1 Сообщение Дата
Sebastien Pouliot 2d6ce7459d
Bump to use Xcode 10 beta 1 (#4179)
* Bump to use Xcode 10 beta 1

* Update Versions.plist

* Add a dependency on Xcode 9.4.

* [msbuild] Fix build with Xcode 10 beta 1. (#4182)

Many years ago (in Xcode 7 according to code comment)
Developer/Platforms/iPhoneOS.platform/Developer/usr disappeared, and we coped
by looking at Developer/usr instead (and also the subsequent code to locate
the bin directory was based on the location of the usr directory).

Developer/Platforms/iPhoneOS.platform/Developer/usr reappeared in Xcode 10
beta 1, but it seems useless (for one it doesn't contain a bin directory), so
in order to try to keep things sane don't look for this directory in Xcode 10
and instead go directly for Developer/usr (which is what we've been using as
the usr directory for years anyway).

Fixes this problem when building apps with Xcode 10 beta 1:

      /Library/Frameworks/Mono.framework/External/xbuild/Xamarin/iOS/Xamarin.iOS.Common.targets(626,3): error : Could not locate SDK bin directory [/Users/rolf/Projects/TestApp/test-app.csproj]

* [runtime] Build 32-bit mac executables using Xcode 9.4.

* [mtouch] Work around broken tvOS headers in Xcode 10 beta 1.

* [mtouch] Work around build problem with Apple's simd headers in Objective-C++ mode.

* Use version-agnostic paths to sdk directories.

* [tests][xtro] Add todo files (from unclassified) and adjust ignore files to avoid errors

* [macos][security] Re-enable SSL[Get|Set]AlpnProtocols. Fixes #4001 (#4022)

* [macos][security] Re-enable SSL[Get}Set]AlpnProtocols. Fixes #4001

This was fixed in macOS 10.13.4

https://github.com/xamarin/xamarin-macios/issues/4001

* [tests][monotouch-tests] Disable a few test cases (one crasher, other failures). Causes to be verified later

* [xharness] Fix permission dialog suppression in Xcode 10.

* [xharness] Ignore 32-bit macOS tests by default.

* [tests] Execute mmp regression tests with Xcode 9.4 since many of them are 32-bit and needs porting to 64-bit.

* [mmptest] Ignore 32-bit XM tests if we don't have a 32-bit-capable Xcode.

* [registrar] Add workaround for broken headers in Xcode 10 beta 1 (radar 40824697).

* [mtouch] Restrict another workaround for an Xcode 10 beta 1 bug to a specific Xcode version to remove it asap.

* [tests] Fix some protocol changes (public or not) find by introspection tests

* [tests][intro] Fix DefaultCtorAllowed failures

* [Intents] Obsolete several Intents classes in watchOS.

Several existing Intents classes have been marked as unavailable in watchOS in
the headers in Xcode 10 beta 1, and corresponding tests are now failing.

So obsolete the managed wrapper types, and fix tests accordingly.

* Fix xtro wrt previous Ietents/intro changes

* [tests] Minor adjustments to mtouch tests to work with Xcode 10.

* [msbuild] Update tests to cope with additional files produced by the Core ML compiler.

* [msbuild] Xcode 10 doesn't support building watchOS 1 apps, so show a clear error message explaining it.

Also update tests accordingly.

* [coreimage] Stub new filters and exclude ?removed? ones from tests

* Update GameplayKit and SpriteKit NSSecureCoding _upgrade_ and fix other non-public cases (in tests)

* [tests] Ignore some GameKit selectors that don't respond anymore (but seems to be available, at least in header files)

* [tests] Fix intro 32bits testing for filters resutls

* [msbuild] Slightly change error message to be better English.
2018-06-08 18:45:24 -07:00
Rolf Bjarne Kvinge f4933226fd [mtouch] watchOS extensions are top-level containers. Fixes issue #3930. (#3934) (#3937)
watchOS extensions are top-level containers in our build (because we ignore
watchOS apps entirely in mtouch), so treat them as such when computing the
protocol member map.

Fixes issue https://github.com/xamarin/xamarin-macios/issues/3930.
2018-04-17 21:15:33 -04:00
Rolf Bjarne Kvinge 15a0f94384
[mtouch] When loading cached linked results, we need to propagate the 'RemoveDynamicRegistrar' value to code-sharing extensions. (#3708) (#3711) 2018-03-09 23:15:01 +01:00
Rolf Bjarne Kvinge 664c790b8c
[mtouch] Propagate the computed value for removal of the dynamic registrar to code shared app extensions. (#3622) (#3630)
This fixes a startup crash in code shared app extensions due to having the
wrong value set in the runtime (the dynamic registrar was removed, but the
executable didn't know it).
2018-03-05 07:41:56 +01:00
Rolf Bjarne Kvinge 835f65897f
[static registrar] Optimize creation of delegates for blocks. (#3623) (#3644)
* [static registrar] Optimize creation of delegates for blocks.

Optimize creation of delegates for blocks so that it doesn't require the
dynamic registrar.

This is done by getting the metadata token for the Create method that creates
the delegate, and embed that metadata token in the generated code from the
static registrar.

Also add tests, since this scenario was not covered by tests already.

* [mmptest] Fix test after recent changes.

* [test-libraries] Avoid duplicate symbols.

* [tests] Update according to changes.
2018-03-05 07:41:28 +01:00
Chris Hamons cabc4fbe2c [mmp] Add stripping of 32-bit dylibs to work with new App Store restrictions (#3387) (#3633)
- https://github.com/xamarin/xamarin-macios/issues/3367
- App Store will now fail builds if you add in a 32-bit dylib
- If you are a 32-bit app you don't need the 64-bit part of your fat
dylib anyway
- Add --optimize=-trim-architectures to allow customization of behavior, as not everyone
uses app store

In addition, while writing tests for this is was noticed that mmp tests did not "really" run Release configuration correctly in most cases. Fixing this turned out to be a bit of a pain, but necessary to correctly test this (and other things).

- Turns out that /p:configuration:debug is not sufficient to tell mmp to
do the right thing
- That, in most projects, sets the DebugSymbols property, which really
is what is checked.
- However, two of our projects did not have that, so we always did
release mmp work.
- Removed configuration property for tests and added real "Release"
configuration option
2018-03-02 14:33:27 +01:00
Rolf Bjarne Kvinge 72bc672d6b
[mtouch] Find the link context for code shared app extensions in the static registrar. Fixes #3514. (#3545) (#3557)
When code sharing is enabled, only the container app/target will have a
LinkContext.

So make sure the static registrar finds that link context when it needs
information from the linker.

https://github.com/xamarin/xamarin-macios/issues/3514
2018-02-21 15:10:20 +01:00
Rolf Bjarne Kvinge 2a29d0e0aa
[mtouch/mmp] Fix tracking of whether the static registrar should run again or not. Fixes #641. (#3534) (#3536)
* [tests] Improve debug spew for the RebuildTest_WithExtensions test.

* [mtouch/mmp] Store/load if the dynamic registrar is removed or not into the cached link results.

Store/load if the dynamic registrar is removed or not into the cached link
results, so that we generate the correct main.m even if cached linker results
are used.

* [mtouch/mmp] The static registrar must not execute if we're loading cached results from the linker.

The static registrar must not execute if we're loading cached results from the
linker, because the static registrar needs information from the linker that's
not restored from the cache.

* [mtouch/mmp] Share Touch code.

* [mtouch/mmp] Make it possible to touch inexistent files (to create them).

* [mtouch/mmp] Fix tracking of whether the static registrar should run again or not.

The recent changes to support optimizing away the dynamic registrar caused the
Xamarin.MTouch.RebuildTest_WithExtensions test to regress.

The problem
-----------

* The linker now collects and stores information the static registrar needs.
* This information is not restored from disk when the linker realizes that it
  can reload previously linked assemblies instead of executing again.
* The static registrar runs again (for another reason).
* The information the static registrar needs isn't available, and incorrect
  output follows.

So fix 1: show an error if the static registrar runs when the linker loaded
cached results.

The exact scenario the test ran into is this:

* 1st build: everything is new and everything is built.
* 2nd build: contents of .exe changes, the linker runs again, the static
  registrar runs again, but sees that the generated output didn't change, so
  it doesn't write the new content to disk (this is an optimization to avoid
  compiling the registrar.m file again unless needed).
* 3rd build: only the .exe timestamp changes, the linker sees nothing changes
  in the contents of the .exe and loads the previously linked assemblies from
  disk, the static registrar sees that the .exe's timestamp is newer than
  registrar.m's timestamp and run again, but doesn't produce the right result
  because it doesn't have the information it needs.

Considered solutions
--------------------

1. Only track timestamps, not file contents. This is not ideal, since it will
   result in more work done: in particular for the case above, it would add a
   registrar.m compilation in build #2, and linker rerun + static registrar
   rerun + registrar.m compilation + final native link in build #3.
2. Always write the output of the static registrar, even if it hasn't changed.
   This is not ideal either, since it will also result in more work done: for
   the case above, it would add a registrar.m compilation + final native link
   in build #3.
3. Always write the output of the static registrar, but track if it changed or
   not, and if it didn't, just touch registrar.o instead of recompiling it.
   This only means the final native link in build #3 is added (see #5 for why
   this is worse than it sounds).
4. Always write the output of the static registrar, but track it it changed or
   not, and if it didn't, just touch registrar.o instead of recompiling it,
   and track that too, so that the final native link in build #3 isn't needed
   anymore. Unfortunately this may result in incorrect behavior, because now
   the msbuild tasks will detect that the executable has changed, and may run
   dsymutil + strip again. The executable didn't actually change, which means
   it would be the previously stripped executable, and thus we'd end up with
   an empty .dSYM because we ran dsymtil on an already stripped executable.
5. Idea #4, but write the output of the final link into a temporary directory
   instead of the .app, so that we could track whether we should update the
   executable in the .app or not. This is not optimal either, because
   executables can be *big* (I've seen multi-GB tvOS bitcode executables), and
   extra copies of such files should not be taken lightly.
6. Idea #4, but tell the MSBuild tasks that dsymutil/strip doesn't need to be
   rerun even if the timestamp of the executable changed. This might actually
   work, but now the solution's become quite complex.

Implemented solution
--------------------

Use stamp files to detect whether a file is up-to-date or not.

In particular:

* When we don't write to a file because the new contents are identical to the
  old contents, we now touch a .stamp file. This stamp file means "the
  accompanying file was determined to be up-to-date when the stamp was
  touched."
* When checking whether a file is up-to-date, also check for the presence of a
  .stamp file, and if it exists, use the highest timestamp between the stamp
  file and the actual file.

Now the test scenario becomes:

* 1st build: everything is new and everything is built.
* 2nd build: contents of .exe changes, the linker runs again, the static
  registrar runs again, but sees that the generated output didn't change, so
  it doesn't write the new content to disk, but it creates a registrar.m.stamp
  file to indicate the point in time when registrar.m was considered up-to-
  date.
* 3rd build: only the .exe timestamp changes, the linker sees nothing changes
  in the contents of the .exe and loads the previously linked assemblies from
  disk, the static registrar sees that the .exe's timestamp is *older* than
  registrar.m.stamp's timestamp and doesn't run again.

We only use the stamp file for source code (registrar.[m|h], main.[m|h],
pinvokes.[m|h]), since using it every time has too much potential for running
into other problems (for instance we should never create .stamp files inside
the .app).

Fixes these test failures:

    1) Failed : Xamarin.MTouch.RebuildTest_WithExtensions("single","",False,System.String[])
      single
      Expected: <empty>
      But was:  < "/Users/builder/data/lanes/5746/4123bf7e/source/xamarin-macios/tests/mtouch/bin/Debug/tmp-test-dir/Xamarin.Tests.BundlerTool.CreateTemporaryDirectory371/testApp.app/testApp is modified, timestamp: 2/15/2018 3:04:11 PM > 2/15/2018 3:04:09 PM" >

    2) Failed : Xamarin.MTouch.RebuildTest_WithExtensions("dual","armv7,arm64",False,System.String[])
      dual
      Expected: <empty>
      But was:  < "/Users/builder/data/lanes/5746/4123bf7e/source/xamarin-macios/tests/mtouch/bin/Debug/tmp-test-dir/Xamarin.Tests.BundlerTool.CreateTemporaryDirectory375/testApp.app/testApp is modified, timestamp: 2/15/2018 3:06:03 PM > 2/15/2018 3:06:00 PM" >

    3) Failed : Xamarin.MTouch.RebuildTest_WithExtensions("llvm","armv7+llvm",False,System.String[])
      llvm
      Expected: <empty>
      But was:  < "/Users/builder/data/lanes/5746/4123bf7e/source/xamarin-macios/tests/mtouch/bin/Debug/tmp-test-dir/Xamarin.Tests.BundlerTool.CreateTemporaryDirectory379/testApp.app/testApp is modified, timestamp: 2/15/2018 3:07:14 PM > 2/15/2018 3:07:12 PM" >

    4) Failed : Xamarin.MTouch.RebuildTest_WithExtensions("debug","",True,System.String[])
      debug
      Expected: <empty>
      But was:  < "/Users/builder/data/lanes/5746/4123bf7e/source/xamarin-macios/tests/mtouch/bin/Debug/tmp-test-dir/Xamarin.Tests.BundlerTool.CreateTemporaryDirectory383/testApp.app/testApp is modified, timestamp: 2/15/2018 3:08:16 PM > 2/15/2018 3:08:13 PM" >

    5) Failed : Xamarin.MTouch.RebuildTest_WithExtensions("single-framework","",False,System.String[])
      single-framework
      Expected: <empty>
      But was:  < "/Users/builder/data/lanes/5746/4123bf7e/source/xamarin-macios/tests/mtouch/bin/Debug/tmp-test-dir/Xamarin.Tests.BundlerTool.CreateTemporaryDirectory387/testApp.app/testApp is modified, timestamp: 2/15/2018 3:09:18 PM > 2/15/2018 3:09:16 PM" >

Fixes https://github.com/xamarin/maccore/issues/641
2018-02-20 11:43:23 +01:00
Rolf Bjarne Kvinge a2b49f4bb5 [tests] Fix expected warning message for optimize warning test. 2018-02-15 17:45:02 +01:00
Rolf Bjarne Kvinge 564e433bbd [tests] Adjust tests after new optimization was implemented. 2018-02-14 01:12:26 +01:00
Rolf Bjarne Kvinge 1c77da63cf [tests][mtouch] Update a few tests according to recent changes. 2018-02-13 19:38:34 +01:00
Rolf Bjarne Kvinge 97230c28e1
Optimize calls to BlockLiteral.SetupBlock to inject the block signature. (#3391)
* [linker] Optimize calls to BlockLiteral.SetupBlock to inject the block signature.

Optimize calls to BlockLiteral.SetupBlock[Unsafe] to calculate the block
signature at build time, and inject it into the call site.

This makes block invocations 10-15x faster (I've added tests that asserts at
least an 8x increase).

It's also required in order to be able to remove the dynamic registrar code in
the future (since calculating the block signature at runtime requires the
dynamic registrar).

* [mtouch/mmp] Add support for reporting errors/warnings that point to the code line causing the error/warning.

Add support for reporting errors/warnings that point to the code line causing
the error/warning by adding ErrorHelper overloads that take the exact
instruction to report (previously we defaulted to the first line/instruction
in a method).

* [tests] Add support for asserting filename/linenumber in warning messages.

* Make all methods that manually create BlockLiterals optimizable.

* [tests] Create a BaseOptimizeGeneratedCodeTest test that's included in both XI's and XM's link all test.

* [tests] Add link all test (for both XI and XM) to test the BlockLiteral.SetupBlock optimization.

* [tests] Add mtouch/mmp tests for the BlockLiteral.SetupBlock optimization.

* [tests][linker] Make the base test class abstract, so tests in the base class aren't executed twice.

* [tests][linker] Don't execute linkall-only tests in linksdk.

The optimization tests only apply when the test assembly is linked, and that
only happens in linkall, so exclude those tests in linksdk.

* [tests][mmptest] Update test according to mmp changes.

Fixes these test failures:

    1) Failed : Xamarin.MMP.Tests.MMPTests.MM0132("inline-runtime-arch")
    The warning 'MM0132: Unknown optimization: 'inline-runtime-arch'. Valid optimizations are: remove-uithread-checks, dead-code-elimination, inline-isdirectbinding, inline-intptr-size.' was not found in the output:
    	Message #1 did not match:
    		actual:   'Unknown optimization: 'inline-runtime-arch'. Valid optimizations are: remove-uithread-checks, dead-code-elimination, inline-isdirectbinding, inline-intptr-size, blockliteral-setupblock.'
    		expected: 'Unknown optimization: 'inline-runtime-arch'. Valid optimizations are: remove-uithread-checks, dead-code-elimination, inline-isdirectbinding, inline-intptr-size.'
    	Message #2 did not match:
    		actual:   'Unknown optimization: 'inline-runtime-arch'. Valid optimizations are: remove-uithread-checks, dead-code-elimination, inline-isdirectbinding, inline-intptr-size, blockliteral-setupblock.'
    		expected: 'Unknown optimization: 'inline-runtime-arch'. Valid optimizations are: remove-uithread-checks, dead-code-elimination, inline-isdirectbinding, inline-intptr-size.'

    2) Failed : Xamarin.MMP.Tests.MMPTests.MM0132("foo")
    The warning 'MM0132: Unknown optimization: 'foo'. Valid optimizations are: remove-uithread-checks, dead-code-elimination, inline-isdirectbinding, inline-intptr-size.' was not found in the output:
    	Message #1 did not match:
    		actual:   'Unknown optimization: 'foo'. Valid optimizations are: remove-uithread-checks, dead-code-elimination, inline-isdirectbinding, inline-intptr-size, blockliteral-setupblock.'
    		expected: 'Unknown optimization: 'foo'. Valid optimizations are: remove-uithread-checks, dead-code-elimination, inline-isdirectbinding, inline-intptr-size.'
    	Message #2 did not match:
    		actual:   'Unknown optimization: 'foo'. Valid optimizations are: remove-uithread-checks, dead-code-elimination, inline-isdirectbinding, inline-intptr-size, blockliteral-setupblock.'
    		expected: 'Unknown optimization: 'foo'. Valid optimizations are: remove-uithread-checks, dead-code-elimination, inline-isdirectbinding, inline-intptr-size.'

* [tests][linker] Fix typo.

Fixes this test failure:

    1) SetupBlock_CustomDelegate (Linker.Shared.BaseOptimizeGeneratedCodeTest.SetupBlock_CustomDelegate)
         Counter
      Expected: 1
      But was:  2

* [registrar] Minor adjustment to error message to match previous (and better) behavior.

Fixes this test failure:

    1) Failed : Xamarin.Registrar.GenericType_WithInvalidParameterTypes
    The error 'MT4136: The registrar cannot marshal the parameter type 'System.Collections.Generic.List`1<U>' of the parameter 'arg' in the method 'Open`1.Bar(System.Collections.Generic.List`1<U>)'' was not found in the output:
    	Message #1 did not match:
    		actual:   'The registrar cannot marshal the parameter type 'System.Collections.Generic.List`1<Foundation.NSObject>' of the parameter 'arg' in the method 'Open`1.Bar(System.Collections.Generic.List`1<U>)''
    		expected: 'The registrar cannot marshal the parameter type 'System.Collections.Generic.List`1<U>' of the parameter 'arg' in the method 'Open`1.Bar(System.Collections.Generic.List`1<U>)''

* [docs] mmp shows MM errors/warnings.

* [docs] Improve according to reviews.

* [tests] Fix merge failure causing test duplication.
2018-02-06 07:08:15 +01:00
Rolf Bjarne Kvinge 6b5870d668
[tests] Move linker tests to match introspection directory layout. (#3365)
* [tests] Move linker tests to match introspection directory layout.

Move linker tests to match introspectio directory layout: tests/linker/ios and
tests/linker/mac instead of tests/linker-ios and tests/linker-mac.

This creates a logical place for shared linker files (tests/linker).

* [tests] Fix path to GuiUnit_NET_4_5.csproj in sln as well.

This is the path used to find referenced projects when building Classic.

Fixes this build problem:

    warning: Referenced project 'GuiUnit_NET_4_5' not found in the solution.
    /Users/builder/jenkins/workspace/xamarin-macios-pr-builder/tests/common/mac/MacTestMain.cs(11,7) : error CS0246: The type or namespace name `GuiUnit' could not be found. Are you missing an assembly reference?
    /Users/builder/jenkins/workspace/xamarin-macios-pr-builder/tests/common/mac/MacTestMain.cs(64,42) : error CS0246: The type or namespace name `IMainLoopIntegration' could not be found. Are you missing an assembly reference?
    /Users/builder/jenkins/workspace/xamarin-macios-pr-builder/tests/common/mac/MacTestMain.cs(75,34) : error CS0246: The type or namespace name `InvokerHelper' could not be found. Are you missing an assembly reference?
2018-02-05 15:09:04 +01:00
Rolf Bjarne Kvinge f905427739
[tests] Share supporting code between the mtouch and mmp tests. (#3374)
* [tests] Share supporting code between the mtouch and mmp tests.

Create a new class 'BundlerTool', which now contains most of the code in
MTouchTool that's also applicable to mmp (and the new MmpTool class).

This will make it easier to share tests between the mtouch and mmp tests.

Some tweaks are still probably required, but this should get us most of the
way.

* [tests] Fix generator tests after changes in shared test code.

* [tests] Add new file to the MSBuild/XM tests.
2018-02-02 07:09:04 +01:00
Rolf Bjarne Kvinge 03d78d13f3
[tests][mtouch] Simplify code to fetch symbols from a binary (nm). (#3286)
Simplify/refactor code to fetch symbols from a binary to reuse an existing
function instead of manually calling `nm` every time.
2018-01-24 00:31:32 +01:00
Rolf Bjarne Kvinge af03020eef
[mtouch/mmp] Give users more control over optimizations, and share more code between mtouch and mmp. (#3242)
* [mtouch/mmp] Give users more control over optimizations, and share more code between mtouch and mmp.

1. Add an --optimize flag to mtouch/mmp that allows users to select which
   optimizations to apply (or not). This makes it easier to add future
   optimizations, and allow users to disable any optimization that causes
   problems without having to disable many other features.

2. Share as much optimization code as possible between mtouch and mmp. This
   immediately gives a benefit to mmp, which has three new optimizations only
   mtouch had: NSObject.IsDirectBinding inlining, IntPtr.Size inlining and
   dead code elimination.

   This results in ~6kb of disk space saved for a linked Xamarin.Mac app:

   * link sdk: [Debug][1], [Release][2]
   * link all: [Debug][3], [Release][4]

Testing also verifies that monotouchtest ([Debug][5], [Release][6]) has not
changed size at all, which means that no default optimizations have changed
inadvertedly.

[1]: https://gist.github.com/rolfbjarne/6b731e3b5ca6170355662e6505c3d492#link-sdk-mac--debug
[2]: https://gist.github.com/rolfbjarne/6b731e3b5ca6170355662e6505c3d492#link-sdk-mac--release
[3]: https://gist.github.com/rolfbjarne/6b731e3b5ca6170355662e6505c3d492#link-all-mac--debug
[4]: https://gist.github.com/rolfbjarne/6b731e3b5ca6170355662e6505c3d492#link-all-mac--release
[5]: https://gist.github.com/rolfbjarne/6b731e3b5ca6170355662e6505c3d492#monotouchtest-iphonedebug64
[6]: https://gist.github.com/rolfbjarne/6b731e3b5ca6170355662e6505c3d492#monotouchtest-iphonerelease64

* [tools] Don't enable the IsDirectBinding optimization by default for Xamarin.Mac apps, it's not safe.

* Fix whitespace issues.

* [doc] Document optimizations.

* Officially support optimizations by adding them to the Versions.plist.

* [linker] Improve IntPtr.Size inliner + dead code eliminatior and add tests.

* Properly handle operands for the ldc_i4_s instruction (they're sbyte).
* Fix less-than condition to actually do a less-than comparison.
* Make sure to look up the bitness in the Target, not the Application, since
  the Application's value will be incorrect when building fat apps (both
  Is32Build and Is64Build will be true).
* Remove unnecessary checks for the IntPtr.Size inliner: this optimization
  does not depend on other instructions than the IntPtr.get_Size call, so
  remove the checks that verify surrounding instructions. This makes the
  IntPtr.Size inliner kick in in more scenarios (such as the new tests).
* Add tests.

* [tests] Add mmp tests for optimizations.

* [tests] Fix XM optimization tests.

* [tests] Fix test build error.
2018-01-23 11:33:48 +01:00
Rolf Bjarne Kvinge dadf5ca6e6
[mtouch] Make sure the xamarin_localized_string_format* functions are available in simlauncher. Fixes #3265. (#3269)
* [mtouch] Make sure the xamarin_localized_string_format* functions are available in simlauncher. Fixes #3265.

Fixes https://github.com/xamarin/xamarin-macios/issues/3265.

* [mtouch] Add test for simlauncher symbols, add add more missing symbols.
2018-01-22 14:21:24 +01:00
Sebastien Pouliot 3a851e2a63
[mtouch][mmp] Report invalid debug symbols files. Fixes #3200 (#3203)
* [mtouch][mmp] Report invalid debug symbols files. Fixes #3200

Try to read the assembly with symbols and, if that fails, warn and
fallback to loading them without symbols.

This fixes cases were it's not easy to update or delete (e.g. nuget)
bad symbols files - so this cannot be an error without causing a lot
of pain.

However it needs to be reported, otherwise it wont be fixed (by the
publisher) and it can limit the debugability of the application and
the usefulness of the stacktraces.

Finally merge most of the resolver's code between mtouch and mmp so
we don't have to fix such issue twice anymore.

note: this needs to be slightly updated once we get a version of cecil
that can give us a more precise error message.

Also bring Rolf's tests from
https://github.com/xamarin/xamarin-macios/pull/3079

reference:
https://github.com/xamarin/xamarin-macios/issues/3200
2018-01-12 17:39:38 -05:00
Sebastien Pouliot 4509e66168
[mtouch] Fix Xamarin.Sdk not to link with private frameworks. Fixes #61039 (#3118)
We can't trust Apple's native linker to pick the right (non private)
framework when an older TargetVersion is used. It just prefer what's
available - even if specified with a WeakFramework :(

That was already dealt with for applications. However the native linking
of the Xamarin.Sdk.framework (code sharing with extensions) is done with
the `LinkTask` instead of the `NativeLinkTask` so it did not have the
"auto correct" code.

Unit test added.

reference:
https://bugzilla.xamarin.com/show_bug.cgi?id=61039
2017-12-22 10:28:29 -05:00
Manuel de la Pena 1a7712f595
[MTouch] If we are building a shared code extension, do not build the msym dir. #60415 (#3016)
* [MTouch] If we are building a shared code extension, do not build the msym dir.
2017-11-17 18:24:09 +01:00
Chris Hamons d2d2fcb352
[macos] Error when linking mix-mode assemblies (#2911)
- https://bugzilla.xamarin.com/show_bug.cgi?id=58583
2017-11-16 09:18:18 -06:00
Rolf Bjarne Kvinge e8d16c925b
[mtouch/mmp/bgen] Add support for response files. (#2808)
* [mtouch/mmp/bgen] Add support for response files.

This is the first part of the fix for #56501.

https://bugzilla.xamarin.com/show_bug.cgi?id=56501

* [tests] Make sure no single argument starting with a '@' is passed to mtouch unless it's a response file.

--assembly-build-target takes arguments starting with '@', for instance:

    --assembly-build-target @all=framework

which does not work anymore, because that's interpreted as a response file
(mtouch tries to read the file '@all=framework', which obviously doesn't
exist).

The fix is simple, don't put a space between the two arguments:

    --assembly-build-target=@all=framework

* Add --root-assembly to mtouch/mmp and make the MSBuild tasks use this new option.

This makes it possible to pass root assemblies starting with `@` to mtouch/mmp
without getting mistaken for response files.

* [msbuild] Always use the command-line option that takes an equals or colon.

Always use the command-line option that takes an equals or colon instead of a
space.

Do either of these:

    --foo=something
    --foo:something

instead of this:

    --foo something

so that `something` can start with an at (`@`) sign without being mistaken for
a response file.

* [msbuild] Fix tests according to recent task changes.
2017-11-16 16:02:34 +01:00
Bernhard Urban 20182800a1
[runtime] integrate mono 2017-10 (#2905)
* Update the function name used to initialize libmono-profiler-log, its called mono_profiler_init_log () now.

* [builds] Pass --with-cross-offsets= to crosstv's configure.

* Bump mono to 2017-08.

* Bump mono to 2017-08.

* Force disable 'futimens' and 'utimensat' so that we build with Xcode 9.

This is also needed to build with Xcode 8.3 on High Sierra.

* Remove old AppleTls implementation.

* Bump mono.

* Bump mono to 2017-08.

* Bump mono to 2017-08

* Reenable link-keep-resources-2 test

- This reverts commit 76b759ef22.
- 2017-08 has linker fix

* Bump mono to 2017-10

* Revert "Bump mono to 2017-10"

This reverts commit bb7832724e.

* Bump system mono to 2017-10

* Bump embedded mono to 2017-10

* [runtime] reflect eglib move

9be68f8952

* bump mono

* [btouch] remove Security.Tls usage from test

* [mtouch tests] update the function name used to initialize libmono-profiler-log, its called mono_profiler_init_log () now.

see
ea4e4a9ef6

fixes:

```
1) Failed : Xamarin.MTouch.Profiling(tvOS)
  _mono_profiler_startup_log
  Expected: collection containing "_mono_profiler_startup_log"
  But was:  < "_mono_profiler_init_log" >
at Xamarin.MTouch.Profiling (Xamarin.Profile profile) [0x00106] in <511889694a624cc9a50e0e9b259b05c5>:0

2) Failed : Xamarin.MTouch.Profiling(watchOS)
  _mono_profiler_startup_log
  Expected: collection containing "_mono_profiler_startup_log"
  But was:  < "_xamarin_get_block_descriptor", "_mono_profiler_init_log" >
at Xamarin.MTouch.Profiling (Xamarin.Profile profile) [0x00106] in <511889694a624cc9a50e0e9b259b05c5>:0
```

* [mmptest] update log profiler options.

826558a4af
deprecated the dash prefix for the mlpd path.

`noallocs` or `nocalls` are not needed, neither of them are default anymore.

* [mmptest] fix link-keep-resources-2 test to cope with more corlib resources.

another corlib resource (mscorlib.xml) was added:

https://github.com/mono/mono/commit/11e95169e787#diff-2d1c64decd91d9a6e8842ab0f0e9438d

* Revert "[mmptest] fix link-keep-resources-2 test to cope with more corlib resources."

This reverts commit 350eb3c174.

* [XHarness] Add the Mono.Data.Tds tests.

* Address comments from rolf in the review.

* [mmp regresssion tests] bump mono linker, so mscorlib.xml gets stripped

the test was failing in that way:

> Executing link-keep-resources-2...
> [FAIL]	i18n 4/2 data files present: charinfo.nlp, collation.core.bin, collation.tailoring.bin, mscorlib.xml

also update the output, because it's actually expected at least three
elements.

fixes: https://bugzilla.xamarin.com/show_bug.cgi?id=59277

* bump mono

fixes crash in tvOS: https://github.com/mono/mono/pull/5812

* bump mono for updated BCL tests

see https://github.com/mono/mono/pull/5820

* [mono] set 2017-10 branch in .gitmodules

* [macos] Fix guiunit error on clean builds by depending on correct copy (#2912)

* [macos] Fix guiunit error on clean builds by depending on correct copy

- From a clean build making a BCL test would error due to the non-mobile guiunit not being built
- This was because the Makefile-mac.inc target was incorrect
- This was because xharness assumed that non variation based targets were always Modern
- However, BCL tests are Full, not Modern

* Code review change

* Swap to var to reduce diff

* Revert changes in the paths for GuiUnit.

* [XHarness] Add the System.IO.Compression bcl tests. (#2918)

* [XHarness] Add the System.IO.Compression bcl tests.

*  [XHarness] Add bcl tests for System.IO.Compression.FileSystem. (#2924)

* [XHarness] Add the System.IO.Compression bcl tests.

* Ensure that resources are correctly copied in the bundles.

* [XHarness] Add bcl tests for System.IO.Compression.FileSystem.

* As per review, make the Mac test app name match the tests that are ran.

* [XHarness] Add Mono.CSharp tests on ios. (#2927)

* [XHarness] Add Mono.CSharp tests on ios.

* Bump mono to bring changes in the mono.csharp tests.

* [xtro-sharpie] fix TypeDefinition access due to Cecil change

* Bump mono

* bump mono

fixes
- https://bugzilla.xamarin.com/show_bug.cgi?id=60480
- https://bugzilla.xamarin.com/show_bug.cgi?id=60482

* bump mono

more fixes around conflicting paths when tests are run in parallel.

* Bump for mono/mono@2017-10
2017-11-14 21:30:08 +01:00
Rolf Bjarne Kvinge 407a19b59b
[tests][mtouch] Add test case for bug #59617. (#2926)
* [tests][mtouch] Convert MT4162 to new syntax.

This also means adding support for custom usings in test code, and adding
support for asserting filename/linenumber with error patterns, and custom
pattern syntax.

* [tests][mtouch] Add test case for bug #59617.

This amounts to running the MT4162 test with the linker enabled.

https://bugzilla.xamarin.com/show_bug.cgi?id=59617
2017-10-27 19:14:58 +02:00
Rolf Bjarne Kvinge eeb6256834 [mtouch] Show MT0123 if the executable assembly does not reference the product assembly. Fixes #59798. (#2801)
It does not make sense to create Xamarin.iOS projects that don't reference
Xamarin.iOS.dll, so make this an explicit error.

This fixes a NullReferenceException which could (when building for device, or
when not using simlauncher) occur, and instead shows the MT0123 error.

> MTOUCH : error MT0000: Unexpected error - Please file a bug report at http://bugzilla.xamarin.com
> System.NullReferenceException: Object reference not set to an instance of an object
>   at Xamarin.Bundler.Target.GatherFrameworks () [0x00065] in /Users/builder/data/lanes/5024/152b654a/source/xamarin-macios/tools/common/Target.cs:122
>   at Xamarin.Bundler.Target.ProcessAssemblies () [0x000c2] in /Users/builder/data/lanes/5024/152b654a/source/xamarin-macios/tools/mtouch/Target.cs:802
>   at Xamarin.Bundler.Application.ProcessAssemblies () [0x0002f] in /Users/builder/data/lanes/5024/152b654a/source/xamarin-macios/tools/mtouch/Application.cs:1407
>   at Xamarin.Bundler.Application.BuildManaged () [0x00001] in /Users/builder/data/lanes/5024/152b654a/source/xamarin-macios/tools/mtouch/Application.cs:831
>   at Xamarin.Bundler.Application+<>c.<BuildAll>b__134_1 (Xamarin.Bundler.Application v) [0x00000] in /Users/builder/data/lanes/5024/152b654a/source/xamarin-macios/tools/mtouch/Application.cs:779
>   at System.Collections.Generic.List`1[T].ForEach (System.Action`1[T] action) [0x00024] in <48b95f3df5804531818f80e28ec60191>:0
>   at Xamarin.Bundler.Application.BuildAll () [0x00050] in /Users/builder/data/lanes/5024/152b654a/source/xamarin-macios/tools/mtouch/Application.cs:779
>   at Xamarin.Bundler.Driver.Main2 (System.String[] args) [0x00481] in /Users/builder/data/lanes/5024/152b654a/source/xamarin-macios/tools/mtouch/mtouch.cs:1420
>   at Xamarin.Bundler.Driver.Main (System.String[] args) [0x0000f] in /Users/builder/data/lanes/5024/152b654a/source/xamarin-macios/tools/mtouch/mtouch.cs:945

https://bugzilla.xamarin.com/show_bug.cgi?id=59798
2017-09-28 17:31:55 +02:00
Rolf Bjarne Kvinge 3aabef12dd [mtouch/mmp] Downgrade MT0123 to a verbose log statement instead. (#2701)
* [mtouch/mmp] Downgrade MT0123 to a verbose log statement instead.

It's unlikely to be useful for normal people who've just set LANG for whatever
unrelated reason.

If it's needed to diagnose failures, it'll still show up in verbose logs.

* [docs] Remove MT0123, it's no longer being used, and we never released a version that used it either.

* [tests][mtouch] Update tests according to code changes (MT0123 removal).
2017-09-18 17:14:17 +02:00
Sebastien Pouliot 05eb96d0a7 Merge branch 'xcode9' into xcode9-master-merge 2017-09-15 11:05:30 -04:00
Sebastien Pouliot 6e918098da Merge latest d15-3 fixes into xcode9 (#2660)
* [msbuild] Re-added wildcard (*) expandsion for application-identifier in Entitlements.plist (#2186)

Fixes https://bugzilla.xamarin.com/show_bug.cgi?id=57119

* Bump mono (#2213)

* Framework tests were still binding non-linked Simple class which errors now (#2216) (#2218)

- Improve Makefile to rebuild when projects build with errors

* Bump mono to get cecil fix for bug #56808. (#2222)

https://bugzilla.xamarin.com/show_bug.cgi?id=56808

* [msbuild] Use @(ReferencePath) instead of @(ResolvedFiles) (#2188) (#2214)

This allows things to work on both xbuild and msbuild.

In xbuild, both lists are exactly the same and on msbuild,
only @(ReferencePath) exists.

Fixes https://bugzilla.xamarin.com/show_bug.cgi?id=55147

* NSActivityOptions.IdleDisplaySleepDisabled had wrong value (#2232) (#2239)

This was due to an integer overflow.  The original value was based on Int32
1 << 40 == 256

The correct value should be based on a UInt64.
1UL << 40 == 1099511627776

* [tests] Fix bug 57699 - [iOS]InternalsTest failure (Linkall) tests on device (#2243)

Strip native debugging symbols should not be checked for debug builds

* Bump mono to get fix for bug #57780.

https://bugzilla.xamarin.com/show_bug.cgi?id=57780

* Update .gitmodules

Change branch to d15-3 branch of mono

* Bump maccore to get fix for bug #55064.

https://bugzilla.xamarin.com/show_bug.cgi?id=55064

* [mono] Bump mono to get the head of cecil/mono-2017-04 and fix IsComObject #57919

Also fix #58789 [1], the typo in tools/mtouch/Tuning.cs showing in MT0000 errors
instead MT2102. That's already in master and d15-4

mono bump includes:

[2] commit 2a6502cee0df9de5198eafe7c8b5f6ac25106f34 (HEAD -> d15-3, origin/d15-3)
Merge: 02457c20fcf 5e05cafc6f1
Author: Luis Aguilera <luis.aguilera@xamarin.com>
Date:   Fri Aug 18 10:04:06 2017 -0400

    Merge pull request #5401 from marek-safar/com

    [Marshal.IsComObject] Make this predicate return false instead of thr…

[3] commit 02457c20fcf57c0610e844d638eb1da82b5d1eb0
Merge: da80840ea55 73fd9a1b82e
Author: Luis Aguilera <luis.aguilera@xamarin.com>
Date:   Fri Aug 18 09:59:06 2017 -0400

    Merge pull request #5400 from spouliot/bump-cecil-58834-d15-3

    [cecil] Bump to the head of the mono-2017-04 branch and pick the fix for bug #58834

References
[1] https://bugzilla.xamarin.com/show_bug.cgi?id=58789
[2] https://bugzilla.xamarin.com/show_bug.cgi?id=57919
[3] https://bugzilla.xamarin.com/show_bug.cgi?id=58834

* [mtouch] Put 'mono_profiler_startup_log' in the symbol list. Fixes #58778. (#2501)

We need the 'mono_profiler_startup_log' symbol when profiling is enabled, so
make sure to add the symbol to the correct list of symbols we need.

Previously we were passing `-u _mono_profiler_startup_log` to clang directly,
which is fine, but not complete, since it does not write the symbol to the
symbollist file (--symbollist=file), which means it wouldn't be preserved when
the MSBuild tasks strip the executable.

https://bugzilla.xamarin.com/show_bug.cgi?id=58778

* Bump versions for SR3

https://trello.com/c/EVze08ei

* Bump mono to include HttpClientHandler fix #44027

https://trello.com/c/jYFXadH8/8-systemnethttp-close-request-stream-when-httpclienthandler
https://bugzilla.xamarin.com/show_bug.cgi?id=44027
2017-09-12 13:08:55 -04:00
Rolf Bjarne Kvinge af44978cce [mtouch] Print verbosity using an invariant culture. Fixes #58849. (#2507)
* [mtouch/mmp] Set CultureInfo.CurrentCulture according to LANG.

This makes it easier to run mtouch/mmp under different locales when testing.

Unfortunately mono checks the system locale before checking LANG, which means
that there's no built-in way in macOS/mono to specify the current locale
without changing the system locale.

* [mtouch] Print verbosity using an invariant culture. Fixes #58849.

https://bugzilla.xamarin.com/show_bug.cgi?id=58849

* [mtouch/mmp] Fix error code.

* [mtouch/mmp] Log/warn when we set the current language.
2017-08-22 09:40:26 +02:00
Rolf Bjarne Kvinge f60801bf6b [mtouch] Put 'mono_profiler_startup_log' in the symbol list. Fixes #58778. (#2501)
We need the 'mono_profiler_startup_log' symbol when profiling is enabled, so
make sure to add the symbol to the correct list of symbols we need.

Previously we were passing `-u _mono_profiler_startup_log` to clang directly,
which is fine, but not complete, since it does not write the symbol to the
symbollist file (--symbollist=file), which means it wouldn't be preserved when
the MSBuild tasks strip the executable.

https://bugzilla.xamarin.com/show_bug.cgi?id=58778
2017-08-18 11:20:43 +02:00
Rolf Bjarne Kvinge 81a1d81d38 [tests] Add the HFS+ equivalent of a memory barrier. Fixes #57831. (#2352)
https://bugzilla.xamarin.com/show_bug.cgi?id=57831
2017-07-31 17:13:08 +02:00
Rolf Bjarne Kvinge 772f6d5b43 [mtouch] Fix collecting required internal symbols which aren't in the objc_msgSend family. (#2330)
Fix collecting required internal symbols which aren't in the objc_msgSend
family by not bailing out early for a function which isn't in the objc_msgSend
family.

Also add a test.
2017-07-18 14:26:45 +02:00
Rolf Bjarne Kvinge 981af59058 [mtouch] Don't allow building for 32-bit when deployment target is >= 11. Fixes #57966. (#2303)
* [mtouch] Don't allow building for 32-bit when deployment target is >= 11. Fixes #57966.

Also bump maccore to get an mlaunch error for launching a 32-bit app in a 64-bit-only simulator.

https://bugzilla.xamarin.com/show_bug.cgi?id=57966

* [tests][msbuild] Make sure all Info.plists have deployment targets.

Otherwise we get different behavior (32-bit allowed or not) depending on which
Xcode is used to build.

* [mtouch] Default to 64-bit arch if not specified and targeting iOS 11+.

* [tests] Tweak tests to either specify a deployment target < 11 or not build a 32-bit arch.
2017-07-11 08:54:50 +02:00
Rolf Bjarne Kvinge 22a01fd6f8 [mtouch] Check for __Internal P/Invokes after processing P/Invokes for exception marshaling. Fixes #57833. (#2261)
When we process P/Invokes to add support for exception marshaling, we may
change P/Invokes to be __Internal. This means that we need to move the check
for __Internal P/Invokes to after processing P/Invokes for exception
marshaling.

https://bugzilla.xamarin.com/show_bug.cgi?id=57833
2017-06-28 17:09:15 +02:00
Rolf Bjarne Kvinge 2fa55d410d [mtouch] If a P/Invoke is used by any assembly that looks up the native symbol using dlsym, it's a required symbol. Fixes #57826. (#2260)
Previously the assumption was that if an assembly not using dlsym references a
native symbol, it's not a required symbol. This is true as far as the native
linker goes: the native linker will see that the native symbol is referenced
by the AOT-compiled code, and it won't be removed.

However, we use also this exact logic to create the list of functions we ask
the native strip command to preserve, and in this case we need to include all
symbols needed in all assemblies that looks up native functions using dlsym.

https://bugzilla.xamarin.com/show_bug.cgi?id=57826
2017-06-28 16:26:26 +02:00
Rolf Bjarne Kvinge c1348da5f6 [tests][mtouch] Adjust tests to cope with Xcode 9. Fixes #57601. (#2231)
Fixes:

1. Failed : Xamarin.MTouch.MT0091(tvOS,"tvOS")
    The error 'MT0091: This version of Xamarin.iOS requires the tvOS 11.0 SDK (shipped with Xcode 9.0). Either upgrade Xcode to get the required header files or set the managed linker behaviour to Link Framework SDKs Only (to try to avoid the new APIs).' was not found in the output:
    Message #1 did not match:
        actual:   'This version of Xamarin.iOS requires the tvOS 11.0 SDK (shipped with Xcode 9). Either upgrade Xcode to get the required header files or set the managed linker behaviour to Link Framework SDKs Only (to try to avoid the new APIs).'
        expected: 'This version of Xamarin.iOS requires the tvOS 11.0 SDK (shipped with Xcode 9.0). Either upgrade Xcode to get the required header files or set the managed linker behaviour to Link Framework SDKs Only (to try to avoid the new APIs).'

2. Failed : Xamarin.MTouch.MT0091(iOS,"iOS")
    The error 'MT0091: This version of Xamarin.iOS requires the iOS 11.0 SDK (shipped with Xcode 9.0). Either upgrade Xcode to get the required header files or set the managed linker behaviour to Link Framework SDKs Only (to try to avoid the new APIs).' was not found in the output:
    Message #1 did not match:
        actual:   'This version of Xamarin.iOS requires the iOS 11.0 SDK (shipped with Xcode 9). Either upgrade Xcode to get the required header files or set the managed linker behaviour to Link Framework SDKs Only (to try to avoid the new APIs).'
        expected: 'This version of Xamarin.iOS requires the iOS 11.0 SDK (shipped with Xcode 9.0). Either upgrade Xcode to get the required header files or set the managed linker behaviour to Link Framework SDKs Only (to try to avoid the new APIs).'

3. Failed : Xamarin.Registrar.NoWarnings
    no warnings
    Expected: not String matching "warning:"
    But was:  "/Users/builder/jenkins/workspace/xamarin-macios-pr-builder/tests/mtouch/bin/Debug/tmp-test-dir/Xamarin.MTouchTool.CreateTemporaryDirectory298/mtouch-test-cache/registrar.m:32468:17: warning: method 'paymentAuthorizationViewController:didAuthorizePayment:handler:' in protocol 'PKPaymentAuthorizationViewControllerDelegate' not implemented [-Wprotocol]"

https://bugzilla.xamarin.com/show_bug.cgi?id=57601
2017-06-21 12:07:13 +02:00
Rolf Bjarne Kvinge 257de9c2c0 [mtouch] Normalize strings that refer to assemblies and their paths before comparing them. Fixes #57266. (#2217)
HFS normalizes filenames to Form D when files are stored. This means that an
assembly whose assembly name is stored in Form C might be stored in a file
whose filename is Form D (which you'll get if you use the Form C filename).

However, this is a problem when we've already loaded an assembly and if we
doesn't take normalization into account: we check the cache based on the
filename, but store in the cache based on the assembly name. If those two uses
different normalization schemes, bad things (bug #57266) happen.

So in these scenarios normalize strings before comparing them.

https://bugzilla.xamarin.com/show_bug.cgi?id=57266
2017-06-15 11:24:31 +02:00
Timothy Risi 25468bf617 Merge Quote methods into a single class instead of multiple versions (#2153) 2017-06-06 12:32:25 -08:00
Rolf Bjarne Kvinge d17cb6556a [mtouch] Improve how we make sure native symbols aren't stripped away. Fixes #51710 and #54417. (#2162)
* [mtouch] Improve how we make sure native symbols aren't stripped away. Fixes #51710 and #54417.

* Refactor required symbol collection to store more information about each
  symbol (field, function, Objective-C class), and in general make the code
  more straight forward.
* Implement support for generating source code that references these symbols,
  and do this whenever we can't ask the native linker to keep these symbols
  (when using bitcode). Additionally make it possible to do this manually, so
  that the source code can be generated for non-bitcode platforms too (which
  is useful if the number of symbols is enormous, in which case we might
  surpass the maximum command-line length).
* Also make it possible to completely ignore native symbols, or ignore them on
  a per-symbol basis. This provides a fallback for users if we get something
  right and we try to preserve something that shouldn't be preserved (for
  instance if it doesn't exist), and the user ends up with unfixable linker
  errors.
* Don't collect Objective-C classes unless they're in an assembly with
  LinkWith attributes. We don't need to preserve Objective-C classes in any
  other circumstances.
* Implement everything for both Xamarin.iOS and Xamarin.Mac, and share the
  code between them.
* Remove previous workaround for bug #51710, since it's no longer needed.
* Add tests.

https://bugzilla.xamarin.com/show_bug.cgi?id=54417
https://bugzilla.xamarin.com/show_bug.cgi?id=51710

* [mtouch] Make sure to only keep symbols from the current app when code sharing.

This fixes a build problem with the interdependent-binding-projects test when
testing in Today Extension mode.
2017-06-02 18:29:19 +02:00
Rolf Bjarne Kvinge 1189a5b94c [mtouch] Set the force flag again when the cache is invalid. Fixes #54973. (#2163)
In 11390f119c we stopped setting the force flag
when the cache was invalid, because we'd delete the cache anyway, and it was
determined that deleting the cache was enough.

Unfortunately it's not, because some output is not in the cache, and might not
get correctly updated.

Scenario:

* User builds app.
* User changes some build option (for instance switching off incremental
  builds).
* User does an insignificant change in a source file for the executable
  process.
* User builds app again (without cleaning). This will rebuild the exe, but
  since the change was insignificant, all the IL, except the MVID, would
  remain identical.
* mtouch would see that the command-line options changed, and invalidate the
  cache. This would delete the cache, and everything would be rebuilt,
  including AOT-compiling the assemblies again.
* When the time came for mtouch to copy assemblies to the app directory,
  mtouch would realize that the existing .exe in the app (which was not
  deleted because it's not in the cache, but the actual output directory) was
  only insignificantly different (only the MVID was different, which our cache
  logic knows to ignore when comparing assemblies), so it wouldn't copy the
  .exe to the .app.
* At runtime we'd assert, because the MVID in the aot-compiled code was
  different from the MVID in the assembly:

      error: Failed to load AOT module '(null)' while running in aot-only mode: doesn't match assembly.

* The exact assert varies depending on which build option changed. Other
  variations:

      Failed to load AOT module '(null)' while running in aot-only mode: compiled against GC (4, while the current runtime uses GC sgen)

      * Assertion at /Users/builder/data/lanes/4691/0719ced1/source/xamarin-macios/external/mono/mono/metadata/metadata.c:1118, condition `idx < t->rows' not met

Because of this I'm reverting 11390f119c, and
once again setting the force flag when the cache is invalid. It might be
overkill, but it's the safest option (cache invalidation is after all the only
hard problem in computer science), and bugs are very annoying and
timeconsuming to track down.

https://bugzilla.xamarin.com/show_bug.cgi?id=54973
2017-06-02 15:23:30 +02:00
Rolf Bjarne Kvinge f6599c4437 [tests] Reuse some code between mtouch rebuild tests and improve the output. (#2157)
* [tests] Print mtouch output if verbose output was requested, even if mtouch succeeded.

* [tests] Reuse some code between mtouch rebuild tests.
2017-06-01 17:28:56 +02:00
Andi McClure a6cf8c5edc [Do not merge yet] Update to mono 2017-04 branch (#1960)
* Update to mono 2017-04 branch

* Patch from Zoltan to fix build error with CppSharp.CppParser.dll

* Include new linker files in Makefile, based on mareks commit

* [msbuild] Fix running bgen for Xamarin.Mac.

bgen must be executed with the system mono, not bmac-mobile-mono, and without
the MONO_PATH variable set.

* System.Data tests should act as if they are running on mobile profile

* Add --runtime=mobile to mono flags in Modern

* Move runtime launcher options up

* System.Data tests should use Mobile profile (mac fix)

* Bump 2017-04 to pick up AOT and assembly resolution fixes

* Build fixes for netstandard.dll and System.Drawing.Primitives.dll

The new handling went in with https://github.com/mono/mono/pull/4501.

I also noticed that WatchOS was missing a target for System.Drawing.Primitives.dll, so I added that.

* Add netstandard.dll to 2.1/Facades and System.Drawing.Primitives.dll to WatchOS

* Fix 2.1/Facades/netstandard.dll build

* Fix the netstandard targets

* Bump mono to latest 2017-04 commit

* [xharness] Fix adding defines to csproj by correctly detecting existing defines.

* Bump mono to latest 2017-04 commit

* [mtouch] Update csproj with new files.

* [mtouch] Improve reporting for MarkExceptions from the linker.

* Bump mono to latest 2017-04 commit

* Bump mono to pick up latest 2017-04 branch commit (Fixes #55436)

Fixes https://bugzilla.xamarin.com/show_bug.cgi?id=55436

* Add a missing Makefile dependency

* Chris Hamons patch to apply --runtime=mobile as necessary at AOT time

(It is currently being applied for some configurations at runtime only)

* Bump system mono

* Bump mono for assembly loader changes

* Bump system mono

* Update assemblies list as some where moved to facades

6ca5ec442b
c38e4d9220

* Bump mono to latest 2017-04 commit

* Add another new facade

* Bump mono to tip of 2017-04.

* Bump mono to tip of 2017-04.

* [tests][mtouch] Adjust tests to cope with fewer assemblies being included in linked apps. Fixes #56307 and #56308.

System.dll is now completely linked away unless the app actually uses any
System.dll API.

This is the change that caused this to change: 4960d5d2a2

Previously the following types would always be kept by the linker:

```
$ monodis --typedef System.dll
Typedef Table
1: (null) (flist=1, mlist=1, flags=0x0, extends=0x0)
2: ObjCRuntime.INativeObject (flist=1, mlist=1, flags=0xa0, extends=0x0)
3: Mono.Net.CFObject (flist=1, mlist=2, flags=0x100000, extends=0x5)
4: Mono.Net.CFArray (flist=4, mlist=19, flags=0x100, extends=0xc)
5: Mono.Net.CFNumber (flist=5, mlist=32, flags=0x100100, extends=0xc)
6: Mono.Net.CFRange (flist=5, mlist=41, flags=0x100108, extends=0x25)
7: Mono.Net.CFString (flist=7, mlist=42, flags=0x100100, extends=0xc)
8: Mono.Net.CFData (flist=8, mlist=53, flags=0x100100, extends=0xc)
9: Mono.Net.CFDictionary (flist=8, mlist=63, flags=0x0, extends=0xc)
10: Mono.Net.CFMutableDictionary (flist=10, mlist=75, flags=0x100100, extends=0x24)
11: Mono.Net.CFUrl (flist=10, mlist=80, flags=0x100100, extends=0xc)
12: Mono.Net.CFRunLoop (flist=10, mlist=83, flags=0x100100, extends=0xc)
13: Mono.Net.CFBoolean (flist=10, mlist=94, flags=0x100, extends=0x5)
14: Mono.AppleTls.SecCertificate (flist=13, mlist=106, flags=0x100100, extends=0x5)
15: Mono.AppleTls.SecIdentity (flist=14, mlist=122, flags=0x100, extends=0x5)
16: Mono.AppleTls.SecIdentity/ImportOptions (flist=19, mlist=134, flags=0x100105, extends=0x5)
17: Mono.AppleTls.SecKey (flist=19, mlist=134, flags=0x100100, extends=0x5)
18: Mono.AppleTls.SecStatusCode (flist=21, mlist=141, flags=0x100, extends=0x69)
19: Mono.AppleTls.SecTrustResult (flist=395, mlist=141, flags=0x100, extends=0x69)
20: Mono.AppleTls.SecImportExport (flist=404, mlist=141, flags=0x100100, extends=0x5)
21: Mono.AppleTls.SecImportExport/<>c (flist=404, mlist=144, flags=0x102103, extends=0x5)
22: Mono.AppleTls.SecPolicy (flist=406, mlist=147, flags=0x100100, extends=0x5)
23: Mono.AppleTls.SecTrust (flist=407, mlist=154, flags=0x100100, extends=0x5)
24: System.Security.Cryptography.OidGroup (flist=408, mlist=174, flags=0x101, extends=0x69)
25: System.Security.Cryptography.Oid (flist=420, mlist=174, flags=0x100101, extends=0x5)
26: System.Security.Cryptography.CAPI (flist=423, mlist=176, flags=0x100180, extends=0x5)
27: System.Security.Cryptography.AsnEncodedData (flist=423, mlist=178, flags=0x100101, extends=0x5)
28: System.Security.Cryptography.X509Certificates.X509Utils (flist=424, mlist=179, flags=0x100100, extends=0x5)
29: System.Security.Cryptography.X509Certificates.PublicKey (flist=424, mlist=181, flags=0x100101, extends=0x5)
30: System.Security.Cryptography.X509Certificates.X509Certificate2 (flist=429, mlist=188, flags=0x102101, extends=0x51)
31: System.Security.Cryptography.X509Certificates.X509Certificate2Impl (flist=431, mlist=204, flags=0x100080, extends=0x55)
32: System.Security.Cryptography.X509Certificates.X509CertificateCollection (flist=431, mlist=209, flags=0x102101, extends=0x6d)
33: System.Security.Cryptography.X509Certificates.X509CertificateCollection/X509CertificateEnumerator (flist=431, mlist=212, flags=0x100102, extends=0x5)
34: System.Security.Cryptography.X509Certificates.X509Helper2 (flist=432, mlist=217, flags=0x100180, extends=0x5)
35: <PrivateImplementationDetails> (flist=432, mlist=218, flags=0x100, extends=0x5)
36: <PrivateImplementationDetails>/__StaticArrayInitTypeSize=9 (flist=433, mlist=219, flags=0x113, extends=0x25)
```

Some of the above types from System.dll implemented ObjCRuntime.INativeObject
(from System.dll), which our linker detected as implementing
ObjCRuntime.INativeObject (from Xamarin.iOS.dll), so these types were treated
as custom NSObject subclasses, and the MarkNSObjects linker step would mark
them (which would in turn cause all the other types in the list to be marked).

With that change, these types now implement ObjCRuntimeInternal.INativeObject,
and the linker does not treat them as custom NSObject subclasses anymore.

I think the new behavior is correct: these types do not actually inherit from
the real NSObject/INativeObject, so the linker should not treat them as such.

This may run into different bugs because the linker might now remove more
stuff than before, but that would be a different issue.

This means that the fix is to modify these tests accordingly.

https://bugzilla.xamarin.com/show_bug.cgi?id=56307
https://bugzilla.xamarin.com/show_bug.cgi?id=56308

* Bump mono to latest.

* Fix merge conflict that was missed

* [mtouch] Renumber new error which clashes with an existing error number in master.
2017-05-29 18:39:29 +02:00
Rolf Bjarne Kvinge 1548594e7b [mtouch] Allow code sharing assemblies from multiple locations if they're identical. Fixes #56513. (#2119)
* [mtouch] Allow code sharing assemblies from multiple locations if they're identical. Fixes #56498.

We disallow code sharing when the same assembly (based on name) is referenced
from multiple paths, but poke a hole in this logic by allowing the same
assembly from multiple paths when those assemblies are 100% identical, since
that should be 100% safe.

https://bugzilla.xamarin.com/show_bug.cgi?id=56498

* [tests] Comment out assert that asserts due to another bug.
2017-05-25 07:10:56 +02:00
Rolf Bjarne Kvinge 0c87730ff1 [mtouch] Fix linking with the same third-party framework from both extension and container app. Fixes #56635. (#2112)
If both an extension and the container app (or multiple extensions) reference
the same binding assembly for a framework, then we'd error out with an
internal error when trying to copy the framework from both locations to the
container app.

So instead detect when we're trying to copy multiple identical (by comparing
the on-disk contents) frameworks, and only copy one of them.

We'll still show an error if the frameworks are different, but now a nice
MT1035 error with a proper error description instead of an internal error.

https://bugzilla.xamarin.com/show_bug.cgi?id=56635
2017-05-19 17:05:38 +02:00
Rolf Bjarne Kvinge 03589db5ba [mmp/mtouch/ObjCRuntime] Calculate the path to the runtime-options.plist using NSBundle's ResourcePath. (#2072)
* [mmp/mtouch/ObjCRuntime] Calculate the path to the runtime-options.plist using NSBundle's ResourcePath.

The anatomy of apps and frameworks differ between iOS and macOS:

* iOS: we put runtime-options.plist in the root directory of the app.
* macOS:
  * for apps we put runtime-options in foo.app/Contents/Resources
  * for frameworks we put runtime-options in foo.framework/Versions/Current/A/Resources

Luckily NSBundle's ResourcePath property returns exactly this path, so change
our logic to use this property.

Also calculate the NSBundle using an exported type we know we have (using the
main bundle won't work when we're a framework).

* [tests] Add mmp/mtouch tests to verify the default HttpClientHandler according to build arguments.

* [ObjCRuntime] Use a custom class for finding the bundle.

Use a self-defined custom class to find the bundle where our resources are.
Using the internal NSObject.NSObject_Disposer class doesn't work when this
file (RuntimeOptions.cs) is compiled into System.Net.Http.dll.
2017-05-08 17:58:21 +02:00
Rolf Bjarne Kvinge 1eb6c335a6 [mtouch] Disable incremental builds for the simulator. Fixes #55712. (#2054)
It does not make sense to support incremental builds for the simulator (since
no AOT compilation is done), it just makes the test matrix more complicated.

So simplify things by removing support for incremental builds.

We also ignore any (other) --assembly-build-target arguments, because building
to frameworks doesn't make sense either in the simulator.

https://bugzilla.xamarin.com/show_bug.cgi?id=55712
2017-04-28 13:34:37 -04:00
Sebastien Pouliot 63a67b5287 Merge pull request #2040 from rolfbjarne/bug55555
[mtouch] Don't remove information when collecting all architectures. Fixes #55555.
2017-04-25 21:29:53 -04:00
Rolf Bjarne Kvinge d8d9a96254 [mtouch] Add support for stripping bitcode from frameworks when not needed. Fixes #55256. (#2037)
This dramatically decreases the size of watchOS apps built for debug when
using frameworks, because it ends up removing all the bitcode from
Mono.framework.

https://bugzilla.xamarin.com/show_bug.cgi?id=55256
2017-04-25 17:46:41 +02:00
Rolf Bjarne Kvinge f6cbea8545 [mtouch] Don't remove information when collecting all architectures. Fixes #55555.
Of particular importance is if we're building for LLVM or not: this fixes a
bug where we wouldn't pass --llvm to the AOT compiler when compiling
assemblies to frameworks (which we do when sharing code).

https://bugzilla.xamarin.com/show_bug.cgi?id=55555
2017-04-25 14:32:08 +02:00
Rolf Bjarne Kvinge 6b89100044 Merge pull request #2019 from rolfbjarne/embeddinator-4000
Add embeddinator support.
2017-04-20 10:09:02 +02:00