Make the bgen tests pass in the path to the attribute library, platform
assembly and all the .NET reference assemblies to bgen. This way we execute
these tests using the .NET version of everything.
Some appextension mtouch code had to be moved to shared code. This code is currently
only used for iOS/tvOS/watchOS, but it will eventually be applicable to macOS as
well.
This makes it possible to re-use the registrar code in dotnet-linker.
Fixes these linkall tests:
Linker.Shared.OptimizeGeneratedCodeTest
[FAIL] IsARM64CallingConvention : optimized: no ldsfld instruction
Expected: 0
But was: 1
at Linker.Shared.BaseOptimizeGeneratedCodeTest.IsARM64CallingConvention() in /Users/rolf/work/maccore/main/xamarin-macios/tests/linker/BaseOptimizeGeneratedCodeTest.cs:line 527
[FAIL] SetupBlockPerfTest : At least 6x speedup
Expected: greater than 6
But was: 1.0876440665344851d
at Linker.Shared.BaseOptimizeGeneratedCodeTest.SetupBlockPerfTest() in /Users/rolf/work/maccore/main/xamarin-macios/tests/linker/BaseOptimizeGeneratedCodeTest.cs:line 120
And linkall is now green for .NET/Debug.
Refactor the Optimizations class to have no conditionally compiled code, which makes
it re-usable from our dotnet-linker code.
Also return any errors or warnings instead of showing/throwing them, which makes
the caller able to show them using whatever means is easiest for the caller.
One test needed an update to the list of valid optimizations, because we now have
a per-platform map of valid optimizations, instead of just a iOS/tvOS/watchOS vs
macOS split ('remove-unsupported-il-for-bitcode' is only valid for watchOS, and now
we say so, while we previously said it was a valid optimization for iOS and tvOS
as well, even though we'd warn about it and do nothing if you tried to set it).
The Assembly.IsFrameworkAssembly property is used in two places:
* In Driver.IsBoundAssembly to return early when determining if an assembly has any NSObject subclasses: c1c5b9aac6/tools/mtouch/mtouch.cs (L1155-L1168)
* In Assembly.ExtractNativeLinkInfo to return early when looking for assemblies with LinkWith attributes: c1c5b9aac6/tools/common/Assembly.cs (L150-L154)
In both cases this definition of framework assembly works today and seems likely to work in the future as well.
I also went through and looked at all the usages of Profile.IsSdkAssembly, and it's used to:
* Decide which assemblies are selected for "link sdk"
* Decide which assemblies are considered an 'sdk' assembly for creating a user framework of all the sdk assemblies
* Bail out early when deciding whether:
* An assembly references the product assembly (Xamarin.iOS.dll, etc.)
* An assembly can contain references to UIWebView
* An assembly can contain user resources
* An assembly is a binding project / has third-party native resources
* An assembly needs the dynamic registrar
* An assembly has FieldAttributes whose native fields must be preserved by the native linker
In all cases our .NET definition of 'SDK' seems to work both for now and in the future.
There are also a few usages which does not apply to .NET, so I've ignored them:
* When looking for a few BCL APIs that must be preserved (MobileApplyPreserveAttribute.cs): this is to be done in the upstream .NET linker now, so it doesn't apply to our own code
* When linking away parameter names (MonoTouchMarkStep.cs): this is to be done in the upstream .NET linker now, so it doesn't apply to our own code
* [mmp] Rename LinkMode.All to LinkMode.Full.
So that we can continue to use Enum.Parse<LinkMode> to parse 'Full' as the link mode.
* [dotnet] Implement support for our different link modes.
Tell the managed linker what to do with each input assembly depending the selected
link mode (link all, link sdk, don't link).
Refactor Application creation to happen earlier, and to split out the cache
creation. This way we can create the Application instance before processing
the configuration, and as we process any configuration we can set properties
on the Application instance.
If we can't find the mscorlib assembly in the list of loaded assemblies, try to load
it explicitly. If we still can't find it the mscorlib assembly, look for System.Void
in any assembly. This shouldn't be a performance bottleneck, because we cache the
System.Void type, which means the lookup is only done once.
This makes System.Void lookup work when building with .NET as well, since there's
no mscorlib.dll there.
This is required when running mtouch and mmp to generate the partial static registrar
code for .NET.
* Port the interdependent-binding-projects test to .NET (it's the simplest
test project we have with binding projects).
* Add a lot of the shared source code for mtouch/mmp to dotnet-linker, and
make it compile. Most issues were fixed by adding a few stubbed out classes,
since there are large chunks of the mtouch/mmp code we're not using yet, so
stubbing out while things are being implemented works fine.
* Add a step in dotnet-linker for loading the linker output (the linked
assemblies) into our bundler code.
* Add another step in dotnet-linker to extract native resources from binding
libraries.
* Augment the build process to take into account the native resources we found
in any binding libraries.
This fixes an issue where mtouch would complain about a missing --target-framework argument when it's not actually needed:
/Library/Frameworks/Xamarin.iOS.framework/Versions/Current/bin/mtouch --launchsim bin/iPhoneSimulator/Release/MyApp.app [...]
error MT0086: A target framework (--target-framework) must be specified.
what makes this worse is that passing --target-framework to mtouch makes
mlaunch fail, because mlaunch doesn't accept a --target-framework argument.
Turns out we don't actually _need_ to know, in every case we use this knowledge it's
a performance improvement to not process the framework assemblies, so skip this for
now, since there's no harm done (except to the planet) to do some extra processing
by processing all assemblies in these cases.
Also add a 'None' build target for the BuildTarget enum for when we're
building for neither simulator nor device (i.e. macOS). This means the default
value will change (since 'Simulator' is no longer the first value), but as far
as I can tell we're always assigning a specific value and not relying on the
default, so this should not make any difference.
This will be needed when the .NET code starts using these classes.
* [mtouch/mmp] Add CoreFoundation and GSS to our list of known frameworks.
Putting these frameworks in our known list of frameworks means we won't try to
weak link them unless needed (when the deployment target is earlier than when
they were introduced), because if we encounter a framework we don't know
about, we'll weak link them to be on the safe side.
* GSS was available in at least macOS 10.1
The shared version isn't used by mmp yet as far as I can tell (mmp has its own logic
to copy assemblies), but sharing this code is the first step towards having the same
implementation as well.
A few changes are required to have an Application instance at hand when we need to
get the ProductName from it.
This is necessary for .NET, since there will be a single linker library for all platforms,
which means we can't use a constant.
This solves a rebuild problem if an assembly has an invalid or unsupported symbol
file, where we'd detect that the symbol file exists, and expect it to be copied,
but then the linker would drop it, causing us to always rebuild the app (this is
not the same as when a symbol file is out of date).
This happens for NUnitLite 3.12.0's nunit.framework.dll, which ships with an old-style
pdb.
Also add a warning that is shown when we detect that there's a symbol file, but it
couldn't be loaded for some reason.
Add a GatherFrameworksStep that computes the frameworks an app needs, and
returns the result to the MSBuild tasks.
Then we use that list of frameworks to link the native executable correctly.
* Create a simple Xamarin.Utils.Execution class that can handle all our
process execution needs:
* Captures or streams stdout/stderr (in UTF8).
* Supports async
* Supports a timeout
* Does not depend on any other source file we have, only uses BCL API.
* Have the execution helper classes from mtouch/mmp
(Xamarin.BundlerDriver.RunCommand) and the tests
(Xamarin.Tests.ExecutionHelper) use this new class.
* Some simplifications were made:
* All API that took a string array for the environment now takes a
Dictionary<string, string>.
* The Driver.RunCommand methods were split out to a separate file. This
file also contains a Verbosity field, which is conditioned on not being
in mtouch nor mmp, which makes including this file from other projects
simpler (such as bgen - in particular bgen was modified to use this
Verbosity field instead of its own).
Fixes this:
Process exited with code 1, command:
/Users/builder/jenkins/workspace/xamarin-macios-pr-builder/tools/xibuild/xibuild System.Collections.Generic.List`1[System.String]
Rolf Bjarne Kvinge
Whitney Schmidt