Currently we put the implementation assemblies for all Xamarin.iOS platforms
in the same directory. This makes it impossible to have different
implementations for the same assembly in different platforms: in particular,
we're going to want a special version of Xamarin.iOS.dll for Mac Catalyst
(that will just have type forwarders into Xamarin.MacCatalyst.dll), that that
assembly will go into the Mac Catalyst-specific directory of implementation
assemblies.
* [watchOS] Add x86_64 simulator support
* Build runtime/registrar x86_64 slices
* Produce a 64 bit version of Xamarin.WatchOS.dll
* Allow building x86_64 for watch simulators in mtouch
* Let xharness know about x86_64
* [tests] Add x86_64 arch to test-libraries
* Make dotnet package aware of x64
* [ObjCRuntime] Fix computing if we're calling a stret function or not in a 64-bit watchOS simulator.
* [xharness] Re-enable some watchOS tests.
Co-authored-by: Rolf Bjarne Kvinge <rolf@xamarin.com>
Because just this is kind of useless:
error MT2301: The linker step 'Setup' failed during processing.
now it will say:
error MT2301: The linker step 'Setup' failed during processing: <hopefully something useful here>
* [dotnet-linker] Catch any exceptions from our custom steps and show them using our error reporting logic.
* Letting the linker handle the exceptions will not result in a particularly
good experience, because the linker will crash.
* We can also show better information, since we have more knowledge about many
of the exceptions we raise ourselves.
* [dotnet] Make ConfigurationAwareSubStep inherit from ExceptionalSubStep.
This required a minor modification to ExceptionalSubStep to allow for custom reporting.
* [dotnet] Implement ConfigurationAwareStep's exception handling like it's done in ExceptionalSubStep.
I also changed the error message slightly to hopefully make it a bit more
comprehensible when translated (since the step name won't be translated, we'll
end up with a message that mixes English with the translated string).
* [src] Build the .NET version of our product assemblies using a .NET 5 BCL.
We're not shipping the .NET product assemblies in any stable release, so we
can use a preview version of the .NET 5 BCL.
Also:
* Add all the nuget feeds we need to the top-level NuGet.config, even for .NET
5/6, there shouldn't be any conflicts with stable feeds since we use exact
version numbers.
* Generate a top-level global5.json which is copied to every directory that
needs a .NET 5 global.json (overriding the .NET 3.1 global.json in the root
directory).
* Use the expected dotnet binary during our local build.
* [tests] Fix the bgen tests to use .NET 5.
* [xharness] Set the current directory to the project directory when running .NET tests.
This way we end up using the dotnet version that's configured in global.json for the tests.
* Use the existing information we have in the Frameworks class to determine
whether a particular framework works in the simulator or not.
* Show a warning (MX5223) when we run into such a framework, but only if
the linker is enabled (otherwise we'll often get warnings for API the developer
doesn't use).
Some appextension mtouch code had to be moved to shared code. This code is currently
only used for iOS/tvOS/watchOS, but it will eventually be applicable to macOS as
well.
This makes it possible to re-use the registrar code in dotnet-linker.
Refactor the Optimizations class to have no conditionally compiled code, which makes
it re-usable from our dotnet-linker code.
Also return any errors or warnings instead of showing/throwing them, which makes
the caller able to show them using whatever means is easiest for the caller.
One test needed an update to the list of valid optimizations, because we now have
a per-platform map of valid optimizations, instead of just a iOS/tvOS/watchOS vs
macOS split ('remove-unsupported-il-for-bitcode' is only valid for watchOS, and now
we say so, while we previously said it was a valid optimization for iOS and tvOS
as well, even though we'd warn about it and do nothing if you tried to set it).
This means:
* Move the parts of the ApplyPreserveAttribute step that we don't need for.NET
into a new MobileApplyPreserveAttribute step, and have mtouch and mmp use
that step instead of the ApplyPreserveAttribute step.
* Copy ApplyPreserveAttributeBase into dotnet-linker from the upstream tuner
source (with minor modifications) so that our ApplyPreserveAttribute step
compiles.
* Create a DotNetSubStepDispatcher class that we're going to use as our
substep dispatcher, create an instance of it and insert it into the list of
linker steps.
* Also a workaround for the lack of LinkContext.GetAssemblies (): add a step
that collects all the assemblies and stores them in a list, so that we can
have our own GetAssemblies implementation.
I filed a linker issue to see if we can get LinkContext.GetAssemblies ()
exposed to us: https://github.com/mono/linker/issues/1455.
Fixes this startup crash with the linkall test:
2020-08-26 19:47:10.219697+0200 link all[32709:6065783] Xamarin.iOS: Fatal error: failed to load the method 'ObjCRuntime.Runtime.Initialize'.
=================================================================
Native Crash Reporting
=================================================================
Got a abrt while executing native code. This usually indicates
a fatal error in the mono runtime or one of the native libraries
used by your application.
=================================================================
=================================================================
Native stacktrace:
=================================================================
0x104007b0e - /Users/rolf/Library/Developer/CoreSimulator/Devices/289E372A-501C-4499-A1A6-59C5B3B6A9AE/data/Containers/Bundle/Application/4A5F3968-6980-4E90-88A2-2E77AE039C40/link all.app/libmonosgen-2.0.dylib : mono_dump_native_crash_info
0x103fb4437 - /Users/rolf/Library/Developer/CoreSimulator/Devices/289E372A-501C-4499-A1A6-59C5B3B6A9AE/data/Containers/Bundle/Application/4A5F3968-6980-4E90-88A2-2E77AE039C40/link all.app/libmonosgen-2.0.dylib : mono_handle_native_crash
0x104007365 - /Users/rolf/Library/Developer/CoreSimulator/Devices/289E372A-501C-4499-A1A6-59C5B3B6A9AE/data/Containers/Bundle/Application/4A5F3968-6980-4E90-88A2-2E77AE039C40/link all.app/libmonosgen-2.0.dylib : sigabrt_signal_handler
0x7fff51c005fd - /Library/Developer/CoreSimulator/Profiles/Runtimes/iOS 13.5.simruntime/Contents/Resources/RuntimeRoot/usr/lib/system/libsystem_platform.dylib : _sigtramp
0x0 - Unknown
0x7fff51af0b7c - /Library/Developer/CoreSimulator/Profiles/Runtimes/iOS 13.5.simruntime/Contents/Resources/RuntimeRoot/usr/lib/system/libsystem_c.dylib : abort
0x103daad98 - /Users/rolf/Library/Developer/CoreSimulator/Devices/289E372A-501C-4499-A1A6-59C5B3B6A9AE/data/Containers/Bundle/Application/4A5F3968-6980-4E90-88A2-2E77AE039C40/link all.app/libxamarin-debug.dylib : xamarin_assertion_message
0x103dade77 - /Users/rolf/Library/Developer/CoreSimulator/Devices/289E372A-501C-4499-A1A6-59C5B3B6A9AE/data/Containers/Bundle/Application/4A5F3968-6980-4E90-88A2-2E77AE039C40/link all.app/libxamarin-debug.dylib : xamarin_initialize
0x103dbf80b - /Users/rolf/Library/Developer/CoreSimulator/Devices/289E372A-501C-4499-A1A6-59C5B3B6A9AE/data/Containers/Bundle/Application/4A5F3968-6980-4E90-88A2-2E77AE039C40/link all.app/libxamarin-debug.dylib : xamarin_main
0x103cd2f0d - /Users/rolf/Library/Developer/CoreSimulator/Devices/289E372A-501C-4499-A1A6-59C5B3B6A9AE/data/Containers/Bundle/Application/4A5F3968-6980-4E90-88A2-2E77AE039C40/link all.app/link all : main
0x7fff51a231fd - /Library/Developer/CoreSimulator/Profiles/Runtimes/iOS 13.5.simruntime/Contents/Resources/RuntimeRoot/usr/lib/system/libdyld.dylib : start
This is a partial/modified port of the initial linker support (bc88790201)
Co-authored-by: Sebastien Pouliot <sebastien@xamarin.com>
* Port the interdependent-binding-projects test to .NET (it's the simplest
test project we have with binding projects).
* Add a lot of the shared source code for mtouch/mmp to dotnet-linker, and
make it compile. Most issues were fixed by adding a few stubbed out classes,
since there are large chunks of the mtouch/mmp code we're not using yet, so
stubbing out while things are being implemented works fine.
* Add a step in dotnet-linker for loading the linker output (the linked
assemblies) into our bundler code.
* Add another step in dotnet-linker to extract native resources from binding
libraries.
* Augment the build process to take into account the native resources we found
in any binding libraries.
This fixes an issue where mtouch would complain about a missing --target-framework argument when it's not actually needed:
/Library/Frameworks/Xamarin.iOS.framework/Versions/Current/bin/mtouch --launchsim bin/iPhoneSimulator/Release/MyApp.app [...]
error MT0086: A target framework (--target-framework) must be specified.
what makes this worse is that passing --target-framework to mtouch makes
mlaunch fail, because mlaunch doesn't accept a --target-framework argument.
Turns out we don't actually _need_ to know, in every case we use this knowledge it's
a performance improvement to not process the framework assemblies, so skip this for
now, since there's no harm done (except to the planet) to do some extra processing
by processing all assemblies in these cases.
Also add a 'None' build target for the BuildTarget enum for when we're
building for neither simulator nor device (i.e. macOS). This means the default
value will change (since 'Simulator' is no longer the first value), but as far
as I can tell we're always assigning a specific value and not relying on the
default, so this should not make any difference.
This will be needed when the .NET code starts using these classes.
This works around a build problem that occurs because NUnit ships with a
P/Invoke to a function that doesn't exist on Apple platforms:
MTOUCH : error MT5210: Native linking failed, undefined symbol: _GetVersionEx. Please verify that all the necessary frameworks have been referenced and native libraries are properly linked in. [/Users/xamarinqa/myagent/_work/8/s/xamarin-macios/tests/xharness/tmp-test-dir/monotouch-test58/monotouch-test-tvos.csproj]
MTOUCH : error MT5201: Native linking failed. Please review the build log and the user flags provided to gcc: -fembed-bitcode-marker [/Users/xamarinqa/myagent/_work/8/s/xamarin-macios/tests/xharness/tmp-test-dir/monotouch-test58/monotouch-test-tvos.csproj]
clang : error : linker command failed with exit code 1 (use -v to see invocation) [/Users/xamarinqa/myagent/_work/8/s/xamarin-macios/tests/xharness/tmp-test-dir/monotouch-test58/monotouch-test-tvos.csproj]
Also fix an issue in mtouch where we would overwrite any previous --dlsym
values; they're now accumulative (`--dlsym:foo.dll --dlsym:bar.dll` works
as expected)
Ref: https://github.com/nunit/nunit/issues/3618
The shared version isn't used by mmp yet as far as I can tell (mmp has its own logic
to copy assemblies), but sharing this code is the first step towards having the same
implementation as well.
A few changes are required to have an Application instance at hand when we need to
get the ProductName from it.
This is necessary for .NET, since there will be a single linker library for all platforms,
which means we can't use a constant.
This solves a rebuild problem if an assembly has an invalid or unsupported symbol
file, where we'd detect that the symbol file exists, and expect it to be copied,
but then the linker would drop it, causing us to always rebuild the app (this is
not the same as when a symbol file is out of date).
This happens for NUnitLite 3.12.0's nunit.framework.dll, which ships with an old-style
pdb.
Also add a warning that is shown when we detect that there's a symbol file, but it
couldn't be loaded for some reason.
* Create a simple Xamarin.Utils.Execution class that can handle all our
process execution needs:
* Captures or streams stdout/stderr (in UTF8).
* Supports async
* Supports a timeout
* Does not depend on any other source file we have, only uses BCL API.
* Have the execution helper classes from mtouch/mmp
(Xamarin.BundlerDriver.RunCommand) and the tests
(Xamarin.Tests.ExecutionHelper) use this new class.
* Some simplifications were made:
* All API that took a string array for the environment now takes a
Dictionary<string, string>.
* The Driver.RunCommand methods were split out to a separate file. This
file also contains a Verbosity field, which is conditioned on not being
in mtouch nor mmp, which makes including this file from other projects
simpler (such as bgen - in particular bgen was modified to use this
Verbosity field instead of its own).
Solves build errors like this:
xamarin-macios/tools/dotnet-linker/packages/xlifftasks/1.0.0-beta.20154.1/build/XliffTasks.targets(91,5): error : 'Errors.cs.xlf' is out-of-date with 'Errors.resx'. Run `msbuild /t:UpdateXlf` to update .xlf files or set UpdateXlfOnBuild=true to update them on every build, but note that it is strongly discouraged to set UpdateXlfOnBuild=true in official/CI build environments as they should not modify source code during the build.
See also: https://github.com/xamarin/xamarin-macios/pull/8157
Fixes: https://github.com/xamarin/xamarin-macios/issues/8468
Added missing `<comment/>` fields for:
* BI1033
* BI1077
* MM2007
* MT0073
* MT0074
* MT0112_c
* MT0113_i
* MT4146
* MT4162
I had to split up the `MT4162` error message, introducing:
* `Errors.MT4162_BaseType` - a base type of
* `Errors.MT4162_Parameter` - a parameter in
* `Errors.MT4162_ReturnType` - a return type in
* `Errors.MT4162_PropertyType` - the property type of
This also removed an argument passed into `string.Format`.
* [mtouch/mmp] Share Application.IsDualBuild, Is32Build and Is64Build.
* [mtouch/mmp] Share --tls-provider and --http-message-handler.
* [mtouch/mmp] Share --force.
* [mtouch/mmp] Share --cache.
* [mtouch/mmp] Share --nolink, --linksdkonly, --linkplatform and --linkskip.
* [mtouch/mmp] Share --i18n.
* [mtouch/mmp] Share --xml.
* [mtouch/mmp] Share --registrar and --runregistrar.
* [mtouch/mmp] Share --warn-on-type-ref.
* [mtouch/mmp] Share --sdk.
* [mtouch/mmp] Share --debug.
* [mtouch/mmp] Share --reference, and deprecate -r|--ref and -a|--assembly.
* [mtouch/mmp] Share --targetver, and deprecate mmp's --minos.
* [msbuild] Adjust tests after switching to use --reference instead of -r.
* Update according to review.
* [mmp] Remove --registrar:il.
The IL generator was what MonoMac had before the dynamic/static registrar code
got shared between MonoTouch and MonoMac. The IL registrar been gone for
years, and as far as I know nobody ever used --registrar:il, even though it
was provided as a compatibility option in the beginning (we still had the IL
registrar around for a while after adding the static+dynamic registrars, until
it was completely replaced by the dynamic registrar).
So just remove this option, if anyone ever used it they can replace it with
--registrar:dynamic.
* [mtouch/mmp] Keep bundler-specific code in its corresponding file.
* Move much of ErrorHandler.cs into a partial class in ErrorHandler.tools.cs,
which is referenced by mtouch and mmp (but not our runtime).
* Add ErrorHandler.runtime.cs for runtime-specific bits, including a simpler
version of ErrorHandler.Show. In particular this gets rid of the call to
Environment.Exit, which should never happen at runtime.
* Rename MonoTouchException and MonoMacException to ProductException, which
allows us to remove a lot of ifdefs.
* This required moving Application.LoadSymbols and Target.LoadSymbols to
shared mtouch/mmp code.
* [mmp] Put the custom bundle name in the App instance.
* [mmp] Put the AOT options in the App instance.
* [mmp] Put the DisableLldbAttach and DisableOmitFramePointer options in the App instance.
* [mmp] Remove Driver.Registrar and use App.Registrar instead.
This fixes this mtouch unit test:
Xamarin.MTouch.Architectures_TVOS_Invalid : The error 'MT0075' was not found in the output.
because now we show MT0075 ("Invalid architecture 'ARMv7' for TVOS projects.")
as expected instead of failing to locate the 32-bit platform directory for
tvOS (which doesn't exist).
Fixes https://github.com/xamarin/maccore/issues/2222.
* Files outside the project directory now show up with a much more logical
name in the IDE. External files have a link target which is relative to the
root xamarin-macios directory, and files included from the mono archive show
up as such as well.
* Rename a few mtouch/mmp-specific files to contain 'mtouch' or 'mmp' in the
filename, to avoid having multiple files in the projects with the same name
(it's confusing every time you search for a filename in the IDE and get
multiple filenames where only the directory is different).
* Add a tools.sln that contains only the mtouch and mmp projects. This makes
it easier to work with both mtouch and mmp and the same time, while not
making VSfM unbearably slow by loading many projects in the same solution.
otherwise debug won't work and you'll get a weird error
```
Using Xcode 11.4 (11E146) found in /Applications/Xcode114.app/Contents/Developer
Xamarin.iOS 13.21.0.154 (master): 296eabd9a using framework: /Applications/Xcode114.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS13.4.sdk
A full rebuild will be performed because the cache is either incomplete or entirely missing.
A full rebuild has been forced because the cache for linker is not valid.
error MT2006: Can not load mscorlib.dll from: '/Users/poupou/git/master/xamarin-macios/tools/mtouch/lib/mono/Xamarin.iOS/mscorlib.dll'. Please reinstall Xamarin.iOS.
at Xamarin.Bundler.Target.Initialize (System.Boolean show_warnings) [0x0002c] in /Users/poupou/git/master/xamarin-macios/tools/mtouch/Target.cs:275
at Xamarin.Bundler.Application.Initialize () [0x007c2] in /Users/poupou/git/master/xamarin-macios/tools/mtouch/Application.cs:1104
at Xamarin.Bundler.Application.BuildInitialize () [0x00008] in /Users/poupou/git/master/xamarin-macios/tools/mtouch/Application.cs:684
at Xamarin.Bundler.Application+<>c.<BuildAll>b__135_0 (Xamarin.Bundler.Application v) [0x00000] in /Users/poupou/git/master/xamarin-macios/tools/mtouch/Application.cs:638
at System.Collections.Generic.List`1[T].ForEach (System.Action`1[T] action) [0x0001e] in /Users/builder/jenkins/workspace/build-package-osx-mono/2020-02/external/bockbuild/builds/mono-x64/external/corefx/src/Common/src/CoreLib/System/Collections/Generic/List.cs:578
at Xamarin.Bundler.Application.BuildAll () [0x00023] in /Users/poupou/git/master/xamarin-macios/tools/mtouch/Application.cs:638
at Xamarin.Bundler.Driver.Main2 (System.String[] args) [0x0041c] in /Users/poupou/git/master/xamarin-macios/tools/mtouch/mtouch.cs:1154
at Xamarin.Bundler.Driver.Main (System.String[] args) [0x00015] in /Users/poupou/git/master/xamarin-macios/tools/common/Driver.cs:35
```
[linker] Don't pass information to linker steps by selectively creating them or using constructors.
Instead use the built-in logic to determine if a linker step should light up,
and use information available in the LinkContext to determine how steps should
behave.
This is required for .NET, where linker steps can't have custom constructors.
Several steps have not been modified, because they're not all required in .NET.
Always add `libmono-profiler-log.dylib` if profiling is enabled and we
are building with dynamic libraries. The profiler code is not (meant to
be) shipped so it can be added even without a `Frameworks` directory.
This fix debugging too (if profiler is enabled) since the library was
linked (even if it was not included).
Fix https://github.com/xamarin/xamarin-macios/issues/8470
Co-authored-by: Sebastien Pouliot <sebastien.pouliot@microsoft.com>
Always add `libmono-profiler-log.dylib` if profiling is enabled and we
are building with dynamic libraries. The profiler code is not (meant to
be) shipped so it can be added even without a `Frameworks` directory.
This fix debugging too (if profiler is enabled) since the library was
linked (even if it was not included).
Fix https://github.com/xamarin/xamarin-macios/issues/8470
* Rearrange files in Xamarin.Mac a bit to ease code sharing between mmp and
mtouch, by putting mono's static and dynamic libraries in
/Library/Frameworks/Xamarin.Mac.framework/Versions/Current/Sdks/Xamarin.macOS.sdk
to match how Xamarin.iOS does it.
* Don't use 'usr' as an intermediate directory. This removes another special
case.
* Share many of the functions and properties that return specific directories,
and document (as comments) what each function/property is supposed to
return.
Rolf Bjarne Kvinge
Alex Soto
Manuel de la Pena
Dorothy Tam
Whitney Schmidt
Sebastien Pouliot
Chris Hamons
Jonathan Peppers
Sebastien Pouliot