This test verifies that the build log prints:
"execution started with arguments: actool --errors --warnings --notices --output-format xml1 --output-partial-info-plist "
However, for M1 machines, we don't call actool directly, we go through 'arch'
instead, and this line is printed:
Tool arch execution started with arguments: -arch arm64e /usr/bin/xcrun actool --errors --warnings --notices --output-format xml1 --output-partial-info-plist ...
So adjust the test to cope with both of these behaviors.
* Bump to .NET 6.0.100-preview.3.21152.10
* Bump to 6.0.100-preview.3.21152.10.
* Bump to 6.0.100-preview.3.21161.7.
* Bump to .NET 6.0.100-preview.3.21161.23.
* [dotnet-linker] Bump ILink and use the supported method of getting an assembly's location.
* Bump to MSBuild.StructuredLogger 2.1.364 to get support for newer binlog versions.
* Fix build failure
Fixes:
TestHelpers/BuildEngine.cs(161,24): error CS0433: The type 'ProjectEvaluationFinishedEventArgs' exists in both 'Microsoft.Build.Framework, Version=15.1.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' and 'StructuredLogger, Version=2.1.0.0, Culture=neutral, PublicKeyToken=d4c7181801cb6448' [/Users/builder/azdo/_work/1/s/xamarin-macios/tests/msbuild/Xamarin.MacDev.Tests/Xamarin.MacDev.Tests.csproj]
* Update parsing binlog properties.
* Be more defensive.
* [tests] Make sure the InternalsVisibleToAttribute constructor isn't linked away.
* [tests] Implement better printing of binlogs.
The latest MSBuild.StructuredLogger made some internal changes to the Message
property which confuses some of our test logic. So implement manual printing
of the log entries that we care about to make sure they conform to the
expected format by the tests (the output is mimicing what 'msbuild /v:diag
foo.binlog' would show).
* Add support for Xamarin.Mac arm64
* Add compile product definition task
Xamarin.Mac can be provided with a ProductDefinition file for the generated pkg. Normally, providing a product definition was optional. However, with Apple Silicon, we have an extra issue : `productbuild` needs to know what architectures your package target. If not provided with them, it will guess to the best of its abilities. However, on Catalina and lower, the guess is x86_64, even if you have an arm64 slice. To fix this, we add a new task to compile the product definition and use this file to create the pkg. If you provide your own Product Definition, we can check and warn if the architectures don't match what we expect. If the file doesn't exist or there is no architecture, we set it ourselves based on our target architectures.
* Don't reference dynamic objC_send on arm64
When building in debug, we currently try to link dynamic objC_send symbols when targeting a 64-bit architecture. However, this is actually only defined on Intel architectures, not on arm64, so we end up failing because we're referring symbols that don't exist. Rework the `GetRequiredSymbols` to take an abi, and tag those symbols to only be valid on i386/x86_64, so they don't get referred at all when building on arm64, but still get referred in x86_64.
* Fix improper delete/move with already existing directories
* Fix stret requirement for Xamarin.Mac in arm64.
The generator supposes that we're running in x64 mode, refactor to take into account the possibility of running in arm64.
* Implement OS version generation in Product.plist, based on MinimumSystemVersion of the app
* Re-generalize some mmp registrar rules
`Microsoft.macOS.registrar` was missed by the current rule set
* Fix mmp tests
* Set E7072 as not translated
Tests were failing otherwise
* Rename Xamarin.Mac lib/x86_64 folder to 64bits (currently all targeted archs are the same)
* Fix style issues
* Fix `ToLower` usage for invariant usage
* Fix xtro-sharpie test
TL&DR: This PR
1. Removes the creation of the `.dSYM` based on `Debug Information` [1]
2. Adds dSYM support to XM msbuild (now shared with XI implementation)
3. Archive the `.dSYM` directories (plural) properly, e.g.
```
msbuild -p:Configuration=Release -p:ArchiveOnBuild=true
```
Why ? The long story...
Historically `.dSYM` for Xamarin.Mac have not been very useful, largely
because (most of) the code is JITed so not much is known before runtime.
So they were simply not generated during the builds...
However AOT options were added to Xamarin.Mac, making them potentially
more useful. Also symbols from `libmono` and other native libraries /
frameworks can prove useful when diagnosing application crashes.
Unsurprisingly developers looking to get symbols eventually found _a way_
[1] to get a `.dSYM` for their applications - but it was not quite
correct because:
* setting the debug information option meant that `mmp` would be supplied with `-debug`. This disables several optimizations that are, by default, enabled for release builds. IOW generating symbols should have no effect on the executing code (but it had);
* it was produced when compiling the native launcher, so the symbols coverage was incomplete. How much depends if mono was statically or dynamically linked. However this would not cover any AOTed code nor bundled libraries or user frameworks.
* the .dSYM was produced inside the `x.app/Contents/MacOS/`, side-by-side with the native executable, which makes it part of the **signed** `.app` and also part of the created (and signed) `.pkg`. This had a large impact on the application's, disk and download, size(s). Manually (re)moving the `.dSYM` means re-signing the app and re-creating (and signing) the `.pkg` is not a good solution.
[1] https://forums.xamarin.com/discussion/139705/how-to-symbolicate-a-xam-mac-crash-log
Additional fixes
* Use `Directory.Move` instead of running the `mv` command
While the result is identical there is a cost to spawn several `mv`
processes. Doing it in parallel (might have) helped but that setup
also comes at a cost.
`Directory.Move` the four `.dylib.dSYM` of an app takes 1 ms, while
the existing code took 17 ms to do the same.
* Fix building mmptest since the DeleteDebugSymbolCommand constant is not present (nor used) anymore
1. Use `xcrun` to run `ibtool` (and `actool`) and avoid toolchain mismatches
2. Set `DEVELOPER_DIR` so everyone (well `xcrun`) use the same toolchain
3. Workaround for `macos-arm64` issue (FB8827920)
* Start `ibtool` as an `Apple` [Silicon] process
* This will ensure the `ibtoold` daemon is also running as `Apple`
* If `ibtoold` is run as `Intel` then `ibtool` asserts and build fail
```
LaunchScreen.storyboard : error : 2020-10-26 14:24:52.757 ibtoold[37142:6681382] [MT] DVTAssertions: Warning in /Library/Caches/com.apple.xbs/Sources/IDEInterfaceBuilder/IDEInterfaceBuilder-17506/InterfaceBuilderKit/Utilities/IBAbstractInterfaceBuilderPlatformToolManager.m:481
LaunchScreen.storyboard : error : Details: Failed to attach to IBAgent-iOS with error: Error Domain=com.apple.InterfaceBuilder Code=-1 "Encountered an error communicating with IBAgent-iOS." UserInfo={NSLocalizedFailureReason=IBAgent-iOS (37146) failed to launch and exited with status 10, NSUnderlyingError=0x7fa4e58fc760 {Error Domain=com.apple.InterfaceBuilder Code=-1 "Failed to launch IBAgent-iOS via CoreSimulator spawn" UserInfo={NSLocalizedDescription=Failed to launch IBAgent-iOS via CoreSimulator spawn, NSUnderlyingError=0x7fa4e5e8a900 {Error Domain=com.apple.InterfaceBuilder Code=-1 "Failed to handshake with platform tool" UserInfo={NSLocalizedFailureReason=Failed to open connection over FIFOs with platform tool, NSLocalizedDescription=Failed to handshake with platform tool, NSUnderlyingError=0x7fa4e5e237c0 {Error Domain=com.apple.InterfaceBuilder Code=-1 "" UserInfo=0x7fa4e5e8bba0 (not displayed)}}}}}, NSLocalizedRecoverySuggestion=Please check Console.app for crash reports for "IBAgent-iOS" for further information., NSLocalizedDescription=Encountered an error communicating with IBAgent-iOS.}
LaunchScreen.storyboard : error : Object: <IBCocoaTouchToolManager>
LaunchScreen.storyboard : error : Method: +_THREADSAFE_launchNewToolWithLaunchContext:executionContext:toolProxyClass:proxyDelegate:failureContext:requestingMethod:error:forReason:
LaunchScreen.storyboard : error : Thread: <NSThread: 0x7fa4e341af70>{number = 1, name = main}
LaunchScreen.storyboard : error : Please file a bug at https://feedbackassistant.apple.com with this warning message and any useful information you can provide.
LaunchScreen.storyboard : error : 2020-10-26 14:24:52.766 ibtoold[37142:6681382] [MT] IBPlatformTool: *** Failed to launch tool with description <IBCocoaTouchPlatformToolDescription: 0x7fa4e5f34160> System content for IBCocoaTouchFramework-fourteenAndLater <IBScaleFactorDeviceTypeDescription: 0x7fa4e5f2fb50> scaleFactor=2x, renderMode.identifier=(null): Encountered an error communicating with IBAgent-iOS. (Failure reason: IBAgent-iOS (37146) failed to launch and exited with status 10): Failed to launch IBAgent-iOS via CoreSimulator spawn: Failed to handshake with platform tool (Failure reason: Failed to open connection over FIFOs with platform tool): : Failed to open FIFOs for handshaking with platform tool (Failure reason: IBAgent-iOS exited before we could handshake)
/Library/Frameworks/Mono.framework/External/xbuild/Xamarin/iOS/Xamarin.iOS.Common.targets(1425,3): error : ibtool exited with code 1
/Library/Frameworks/Mono.framework/External/xbuild/Xamarin/iOS/Xamarin.iOS.Common.targets(1425,3): error :
LaunchScreen.storyboard : ibtool error : Encountered an error communicating with IBAgent-iOS.
```
Note: `main` has diverged quite a bit (net6 support) so this pull request
will need to be re-worked (it won't apply)
Fixes https://github.com/xamarin/xamarin-macios/issues/4634
Foreport of #9965 since the source code diverged from `xcode12.2`
* [msbuild] Move msbuild/tests to tests/msbuild to put all the tests together.
* [tests] Move test projects for Xamarin.Mac to tests/common/TestProjects
* [tests] Move test projects for Xamarin.iOS to tests/common/TestProjects
Refactor the Optimizations class to have no conditionally compiled code, which makes
it re-usable from our dotnet-linker code.
Also return any errors or warnings instead of showing/throwing them, which makes
the caller able to show them using whatever means is easiest for the caller.
One test needed an update to the list of valid optimizations, because we now have
a per-platform map of valid optimizations, instead of just a iOS/tvOS/watchOS vs
macOS split ('remove-unsupported-il-for-bitcode' is only valid for watchOS, and now
we say so, while we previously said it was a valid optimization for iOS and tvOS
as well, even though we'd warn about it and do nothing if you tried to set it).
Fixes this warning:
warning MSB9004: ManifestResourceWithNoCulture item type is deprecated. Emit EmbeddedResource items instead, with metadata WithCulture='false', Type='Resx', and optional LogicalName.
There is a slight difference with regards to the warning message: the Type
metadata is set to 'Non-Resx' instead of 'Resx' (because these resources
aren't resx files).
Fixes https://github.com/xamarin/xamarin-macios/issues/3876.
- This commit adds a hook, "AdditionalAppExtensions", to the msbuild to allow
extensions written in other languages, such as Swift, to be embedded and signed in an
Xamarin App bundle easily.
- Example:
<AdditionalAppExtensions Include="$(MSBuildProjectDirectory)/../../native">
<Name>NativeTodayExtension</Name>
<BuildOutput Condition="'$(Platform)' == 'iPhone'">build/Debug-iphoneos</BuildOutput>
<BuildOutput Condition="'$(Platform)' == 'iPhoneSimulator'">build/Debug-iphonesimulator</BuildOutput>
</AdditionalAppExtensions>
- This commit adds a hook, "AdditionalAppExtensions", to the msbuild to allow
extensions written in other languages, such as Swift, to be embedded and signed in an
Xamarin App bundle easily.
- Example:
<AdditionalAppExtensions Include="$(MSBuildProjectDirectory)/../../native">
<Name>NativeTodayExtension</Name>
<BuildOutput Condition="'$(Platform)' == 'iPhone'">build/Debug-iphoneos</BuildOutput>
<BuildOutput Condition="'$(Platform)' == 'iPhoneSimulator'">build/Debug-iphonesimulator</BuildOutput>
</AdditionalAppExtensions>
* [linker] Fix infinite queue found with nullability PR (#8337)
and removed previous workaround
Replace previous attempt https://github.com/xamarin/xamarin-macios/pull/8336
* [linker] Remove code to deal with ExportedTypes (#8632)
This is now supported by upstream mono/linker
* [linker] Remove internal [NullablePublicOnly] attribute from apps (#8568)
I've only seen it with .net5 so far but it's better handled in master
and flow back into the branch
* [linker] Update custom attributes that can be removed (#8535)
Some are no longer part of the SDK (or converted into new ones
at build time), others were new (and missing).
A full list of attributes and their usage frequency in what we ship can
be seen in https://gist.github.com/spouliot/ca03c6da7d4d75670ca77749350eb8a2
Also update tests: no need to check for removals of stuff that does not
exists anymore.
Co-authored-by: Sebastien Pouliot <sebastien.pouliot@gmail.com>
* Create a simple Xamarin.Utils.Execution class that can handle all our
process execution needs:
* Captures or streams stdout/stderr (in UTF8).
* Supports async
* Supports a timeout
* Does not depend on any other source file we have, only uses BCL API.
* Have the execution helper classes from mtouch/mmp
(Xamarin.BundlerDriver.RunCommand) and the tests
(Xamarin.Tests.ExecutionHelper) use this new class.
* Some simplifications were made:
* All API that took a string array for the environment now takes a
Dictionary<string, string>.
* The Driver.RunCommand methods were split out to a separate file. This
file also contains a Verbosity field, which is conditioned on not being
in mtouch nor mmp, which makes including this file from other projects
simpler (such as bgen - in particular bgen was modified to use this
Verbosity field instead of its own).
The build output is very verbose, which means the html reports are just
impossible to navigate due to the amount of text. The build output will be
printed to the terminal anyway, so it'll still be available.
* Rearrange files in Xamarin.Mac a bit to ease code sharing between mmp and
mtouch, by putting mono's static and dynamic libraries in
/Library/Frameworks/Xamarin.Mac.framework/Versions/Current/Sdks/Xamarin.macOS.sdk
to match how Xamarin.iOS does it.
* Don't use 'usr' as an intermediate directory. This removes another special
case.
* Share many of the functions and properties that return specific directories,
and document (as comments) what each function/property is supposed to
return.
Cecil has a fall-back mode where it looks in the GAC / system mono for
assemblies when failing to find them elsewhere. This is not the expected
behavior when using Xamarin.Mac in the Full/XM mode, because then we should
only resolve to assemblies shipped with Xamarin.Mac.
Unfortunately doing so will break apps (our own tests break), so instead
change our resolution to be explicit about where we find assemblies, and if we
find assemblies in the GAC / system mono when we're not supposed to, then show
a warning.
Also add a fall-back mechanism, where we use the old logic instead, in case
the new logic is not 100% compatible with the old one.
This showed up when I tried to port mmp to dotnet, because then Cecil stopped
looking in the GAC / system mono for assemblies (Cecil has a special case when
running on Mono to look in Mono's GAC), and tests started failing.
dotnet doesn't like projects within projects in the file system, because of
default inclusion behavior, where the top project will want to include the
source files for the nested projects.
There are ways around that, but the easiest is to just make sure there aren't
projects within other project directories.
xcrun can print stuff to stderr even if it succeeds and prints what it found
to stdout, which means we have to separate stdout and stderr when capturing
xcrun's output.
This also means that we need a few changes to our RunCommand API:
* Add support for capturing stdout and stderr separately by providing
different callbacks/StringBuilder instances.
* Use overloads instead of default arguments, because with default arguments
we end up with ambiguous matches in a lot of places with the new overloads.
Fixes https://github.com/xamarin/xamarin-macios/issues/8147.
Backport of #8150.
* [mtouch/mmp] Only use stdout from xcrun as the path of the file that was found. Fixes#8147.
xcrun can print stuff to stderr even if it succeeds and prints what it found
to stdout, which means we have to separate stdout and stderr when capturing
xcrun's output.
This also means that we need a few changes to our RunCommand API:
* Add support for capturing stdout and stderr separately by providing
different callbacks/StringBuilder instances.
* Use overloads instead of default arguments, because with default arguments
we end up with ambiguous matches in a lot of places with the new overloads.
Fixes https://github.com/xamarin/xamarin-macios/issues/8147.
* [tests] Fix mmptest build.
* [mtouch/mmp] Fix process log output.
* Unify target framework code between mtouch and mmp.
* Simplify the code in mmp: have three possible valid target frameworks for
most of code, and add special code to handle setting any other valid target
frameworks to redirect to one of those three valid target frameworks (and
warn if given any of those valid, but not "main", target frameworks). Any
other code can then depend on the target framework having exactly one of
those specific values, which means we can make IsUnified* variables
convenience properties instead.
* Unify a bit more of the argument parsing code between mtouch and mmp, since
that made a few other things easier.
* Add TargetFramework.IsValidFramework to have one validation implementation.
* Move the implementation of TargetFramework.MonoFrameworkDirectory to mmp
itself, it's not really related to the target framework.
* Remove Driver.IsUnified and IsClassic from mmp, they're not used anymore.
* Formally deprecate --xamarin-[full|system]-framework in mmp, they've really been deprecated for many years.
* Remove LinkerOptions.TargetFramework, it's not used anymore.
* Get rid of mmp's userTargetFramework fried, it's duplicated with the
targetFramework field.
* Add a few tests, and tweak others a bit.
Breaking changes:
* Both mtouch and mmp require --target-framework now. The only direct
consumers should be the MSBuild tasks, which already pass --target-framework
all the time. This simplifies code, and removes assumptions.
Ignore Xamarin.MMP.Tests.WarningTests.MM0135 on macOS 10.15+, because this
test requires Xcode 9.4, which doesn't work on macOS 10.15+.
Fixes https://github.com/xamarin/maccore/issues/2035.
* Fixed
`/Users/builder/jenkins/workspace/xamarin-macios/xamarin-macios/builds/mono-ios-sdk-destdir/ios-sources/external/linker/src/linker/Linker.Steps/OutputStep.cs(110,15): error CS0246: The type or namespace name ‘OutputException’ could not be found (are you missing a using directive or an assembly reference?) [/Users/builder/jenkins/workspace/xamarin-macios/xamarin-macios/tools/mmp/mmp.csproj]`
* Changed the name of the method that is used from linker. Because of this commit 6be26771b9
* Added `OutputException.cs` file on `mtouch.csproj`.
* Removing enter_gc_safe and exit_gc_safe because now it's already gc_safe in this part of code, after a mono change.
* Added known exceptions to LLVM exception list.
* Needs `ifdef` because of this https://github.com/mono/mono/pull/17260.
* Bump MIN_MONO_VERSION to 6.8.0.41 and point MIN_MONO_URL to the PR.
* Add ENABLE_IOS=1 and ENABLE_MAC=1.
* Added switch to disable packaged mono build
* [Tests] Ignore tests that fail on 32b.
Ignore the test on 32b, and filled issue: https://github.com/mono/mono/issues/17752
* [Tests] Ignore a couple of tests causing OOM.
Hopefully fixes https://github.com/xamarin/maccore/issues/1659 for good.
* Ignore `MM0135` test on Catalina+ because it needs Xcode 9.4.
* [monotouch-test] Add null checks for teardown when test didn't run because of a too early OS version.
* [CFNetwork]: Http 2.0 requires OS X 10.11 or later.
Check whether `_HTTPVersion2_0` is available and fallback to HTTP 1.1 otherwise.
* #7346
* This bumps Mono to use https://github.com/mono/mono/pull/17645 (which is the 2019-10 backport
of https://github.com/mono/mono/pull/17628).
* The big user-visible change is in regards to certificate validation, everything below are just
some minor adjustments to tests.
CoreFX uses a completely new `HttpClientHandler` implementation called `SocketsHttpHandler`,
which you can find at https://github.com/dotnet/corefx/tree/release/3.0/src/System.Net.Http/src/System/Net/Http/SocketsHttpHandler.
Since this is not based on the web stack anymore, it does not use any of the related APIs such
as `ServicePointManager` or `WebException`.
There is a new API called `HttpClientHandler.ServerCertificateCustomValidationCallback`.
- https://docs.microsoft.com/en-us/dotnet/api/system.net.http.httpclienthandler.servercertificatecustomvalidationcallback?view=netframework-4.8
- c1778515a3/src/System.Net.Http/src/System/Net/Http/HttpClientHandler.Unix.cs (L154)
- c1778515a3/src/System.Net.Http/src/System/Net/Http/HttpClientHandler.Windows.cs (L383)
The `ServicePointManager.ServerCertificateValidationCallback` is no longer invoked and on
certificate validation failure, `AuthenticationException` (from `System.Security.Authentication`)
is thrown instead of `WebException`.
At the moment, the `NSUrlSessionHandler` still uses it's own validation callback and also still
throws `WebException` on failure; we should probably look into making this consistent with the
other handlers.
* `HttpContent.SerializeToStreamAsync()` is now `protected` (changed from `protected internal`).
- src/Foundation/NSUrlSessionHandler.cs: changed overload accordingly.
- src/System.Net.Http/CFContentStream.cs: likewise.
* `HttpHeaders.GetKnownHeaderKind()` is an internal Mono API.
There is a new internal API called `System.Net.Http.PlatformHelper.IsContentHeader(key)`
which exists in both the old as well as the new implementation.
The correct way of doing it with the CoreFX handler is
`HeaderDescriptor.TryGet (key, out var descriptor) && descriptor.HeaderType == HttpHeaderType.Content`
* `HttpClientHandler.MaxRequestContentBufferSize` is now longer supported, you can set it to
any non-negative value, the getter will always return 0.
See c1778515a3/src/System.Net.Http/src/System/Net/Http/HttpClientHandler.Core.cs (L18).
- tests/linker/ios/link sdk/HttpClientHandlerTest.cs: removed assertion from test.
* `HttpMessageInvoker.handler` is a `protected private` field - in the CoreFX handler, it is
called `_handler` and `private`. This is accessed via reflection by some of the tests, which are
now using the new name.
- tests/mmptest/src/MMPTest.cs: here
- tests/mtouch/MTouch.cs: here
* tests/monotouch-test/System.Net.Http/MessageHandlers.cs:
Adjust `RejectSslCertificatesServicePointManager` to reflect the certificate validation
changes described above.
- FIXME: There was an `Assert.Ignore()` related to `NSUrlSessionHandler` and macOS 10.10;
I removed that to reenable the test because the description linked to an old issue in
the private repo that was referenced by several "Merged" PR's, so it looked to me that
this might have already been fixed - and I also didn't see why it would fail there.
It's possible for a nuget to have dependencies that will bring satellite
assemblies into a project.
When doing so the satellite assemblies are copied to the right output
(e.g. `Debug`) directory, so they work fine at runtime. However it's
not 100% fine at build time, e.g. msbuild won't detected them as
satellite assemblies.
Something similar happens with `mtouch` (and `mmp`) since the satellite
assemblies won't be found inside a (culture-named) subdirectory from the
original assembly location. In our case this means the assemblies won't
be copied into the .app bundle (since we don't run from `Debug`) and
localization won't work properly.
The solution is to check for both the (culture-named) subdirectories
from the assembly location (like before) and, if nothing is found, also
try to locate them in the _build_ (like `Debug`) directory - so if
something else (nuget or custom build scripts) tried to outsmart the
build logic then we'll still bring those satellite assemblies in the
app bundle we produce.
https://github.com/xamarin/xamarin-macios/issues/7113