The test run can fail even if all the tests pass: in case the process crashes
or deadlocks at exit for instance. Don't overwrite such failures with the unit
test results.
This fixes almost [1] all `extra` API detected from the extrospection
(header-based) tests for MacCatalyst. This is needed to complete the
introspection fixes (without having _too large PR).
[1] except `!extra-null-allowed!` since those are still in progress for
all platforms
Use a private property (prefixed with underscore) for now, until we can decide
on a better/general name.
Also add a variation to xharness to build monotouch-test with CoreCLR
(building works fine, but it crashes at startup, which is expected at this
point).
* [tests] Preserve all test fixtures.
This fixes the wildly diffetent number of tests when running xammac tests with and without linking.
Without linking:
> Tests run: 2446 Passed: 2321 Inconclusive: 9 Failed: 0 Ignored: 125
vs with linking:
> Tests run: 1885 Passed: 1802 Inconclusive: 4 Failed: 0 Ignored: 83
Now we run the same tests either with or without linking.
One test was updated to not fail when linking is enabled.
Also add a test to ensure all future test fixtures are preserved.
* Remove whitespace noise.
The API were already available (iOS) and this helps fixing tests for
MacCatalyst (as it's based on our iOS API but expose the same issue
as macOS wrt extra ctor/init).
* [runtime] Port the is_user_type function from native to managed code.
* This is a straight forward port of native code to managed code, and
shouldn't have any significant side effects.
* Makes it possible to move more code from native to managed for
xamarin_create_managed_ref and xamarin_release_managed_ref in the future.
* Update xtro.
Sometimes msbuild wants to restore packages during this process, which may
take a while.
Hopefully fixes errors like this:
Harness exception for 'Tests for C6353E8B-BFDA-4F54-93D9-FF0F838BF8EE': System.Exception: Unable to evaluate the property OutputPath.
at Xharness.AppBundleLocator.GetPropertyByMSBuildEvaluationAsync (System.Xml.XmlDocument csproj, System.String projectPath, System.String evaluateProperty, System.String dependsOnTargets, System.Collections.Generic.Dictionary`2[TKey,TValue] properties) [0x00327] in /Users/builder/azdo/_work/1/s/xamarin-macios/tests/xharness/AppBundleLocator.cs:109
at Xharness.AppBundleLocator.LocateAppBundle (System.Xml.XmlDocument projectFile, System.String projectFilePath, Microsoft.DotNet.XHarness.iOS.Shared.TestTarget target, System.String buildConfiguration) [0x000d2] in /Users/builder/azdo/_work/1/s/xamarin-macios/tests/xharness/AppBundleLocator.cs:49
at Microsoft.DotNet.XHarness.iOS.Shared.AppBundleInformationParser.ParseFromProject (System.String projectFilePath, Microsoft.DotNet.XHarness.iOS.Shared.TestTarget target, System.String buildConfiguration) [0x00147] in /_/src/Microsoft.DotNet.XHarness.iOS.Shared/AppBundleInformationParser.cs:71
at Xharness.AppRunner.InitializeAsync () [0x00046] in /Users/builder/azdo/_work/1/s/xamarin-macios/tests/xharness/AppRunner.cs:120
at Xharness.Jenkins.TestTasks.RunSimulator.SelectSimulatorAsync () [0x002df] in /Users/builder/azdo/_work/1/s/xamarin-macios/tests/xharness/Jenkins/TestTasks/RunSimulator.cs:108
at Xharness.Jenkins.TestTasks.AggregatedRunSimulatorTask.ExecuteAsync () [0x00335] in /Users/builder/azdo/_work/1/s/xamarin-macios/tests/xharness/Jenkins/TestTasks/AggregatedRunSimulatorTask.cs:63
at Xharness.Jenkins.TestTasks.TestTasks.RunInternalAsync () [0x00226] in /Users/builder/azdo/_work/1/s/xamarin-macios/tests/xharness/Jenkins/TestTasks/TestTask.cs:283
Also
* Fix the Mac Catalyst sample to use the correct contents.json and images sizes
for Mac Catalyst app icons.
* Fix the contents.json and image sizes for the Mac Catalyst test project as well.
* Add unit test for Mac Catalyst that ensures that Assets.car is in the final app.
Fixes https://github.com/xamarin/xamarin-macios/issues/10804.
Assembly.GetType works differently between CoreCLR and Mono for nested types:
Mono accepts both '/' and '+' as the character separating the declaring type
from the nested type, while CoreCLR only accepts '+'.
So just switch to '+', since that works everywhere.
The default marshalling size for characters in .NET is a single byte, and that
is potentially lossy (and causes tests to fail for non-ascii characters).
* [dotnet] Use the expected case for ARM64.
We use case-sensitive comparisons somewhere else, so make sure to use the same casing
for the architecture as other platforms do.
* [tests] Add tests to verify that Mac Catalyst builds successfully for ARM64.
* [msbuild] Fix platform name in condition.
* [msbuild] Pass -isysroot when building Mac Catalyst apps as well.
Ohterwise clang will use the SDK from the command line tools, which may have
nothing to do with the SDK we want to build with.
* [cecil-tests] Add test for MarshalAs attributes for bool return type / parameters in P/Invokes.
This also meant adding support for resolving additional BCL assemblies (to be able to process Action<> and Func<> types).
* [generator] Add [MarshalAs (UnmanagedType.I1)] to bool return types and parameters.
* Fix all manually bound API to use a MarshalAs attribute for boolean return types and parameters in P/Invokes.
and not just methods even if we are generating them on both [1]
That spotted a single error inside `src/Security/SslContext.cs`
which this PR also fix. IOW there's no point in saying the API
was (a long time ago) available since our code to call it was
removed.
[1] which is something we should not need to do on .net since
* the compiler-based tooling does not need it on the getter/setter
* other tools needs to be updated (so it won't break any existing
use case)
* [runtime] Build our runtime for Mac Catalyst/ARM64 for .NET.
* [ObjCRuntime] There's no need for the StartWWAN implementation on Mac Catalyst.
This also fixes a build error:
error MT5214: Native linking failed, undefined symbol: _xamarin_start_wwan. This symbol was referenced by the managed member ObjCRuntime.Runtime.xamarin_start_wwan.
* Only exclude xamarin_start_wwan in the .NET version of Mac Catalyst.
* [tests] Update to not run the StartWWAN test on Mac Catalyst.
* Update conditional logic.
* Fix build with newer make versions.
- this requires small (but tricky) generator changes
- duplicate availability attributes detection is tested by introspection
- it also requires removing dupes in manual bindings (which was the
current, if not the _named_, goal for this PR)
Long story:
It's always been hard to detect and enforce removal of extraneous
availability attributes since the generator inlined them _freely_
when processing ObjC protocols.
That meant introspection could not report dupes and led to many
extra attributes. This was **not** a problem in binding files since we
could ignore them. IOW their presence (in the input) does not mean they
are _all_ dupes in generated code (output).
However dupes in manual bindings were also (forced to be) _ignored_ and
those were part of the assemblies we ship. Again it was not a big deal,
nor a source of much extra metadata/size, so it was ignored.
Forward to today, manual bindings needs to be updated for net6 [1] so any
extra we have requires more (manual) work. Cleaning this problem up in
the generator code reduce the (manual) work we need to do.
It also means introspection can detect dupes (in generated and manual
code) so we don't end up adding more (which would also require more
manual work to support both set of attributes).
[1] https://github.com/xamarin/xamarin-macios/pull/10580
xtro tests are based on Apple's header files and report as _unknown_
bindings to API that are not found in headers - Catalyst in this case.
Removing them is required so the applications can be submitted to the
AppStore.
This required working around internal ctor that uses internal (non
re-generated) types. E.g.
```
build/dotnet/maccatalyst/ref/Xamarin.iOS.cs(9201,64): error CS0234: The type or namespace name 'ABFunc<>' does not exist in the namespace 'AddressBookUI' (are you missing an assembly reference?)
```
We cannot just skip such constructor since the C# compiler will add a
**public** default one when none is present (and that's the case for
the error above),
This also required fixing the ctor chaining code (which can't be shared
with the, previously fixed, override code) so base classes could be
called for some `*EventArg` types.
Follow up to https://github.com/xamarin/xamarin-macios/pull/10658 , https://github.com/xamarin/xamarin-macios/pull/10678 and https://github.com/xamarin/xamarin-macios/pull/10706
This is the last framework that needs removal - but there's still a few
`unknown-*` entries in Catalyst-supported frameworks (more PR coming).
* [tests][sampletester] Add nuget.org single feed to sampletester
Since we no longer upstream to nuget.org our sampletester is not
able to find packages so we create a nuget config file with a single
source to nuget.org to fix it and still be compliant.
* [tests][sampletester] Remove ConferenceVision sample, it no longer exists
* [tests][sampletester] Double TimeOut for a couple of tests
Co-authored-by: Alex Soto <alex@alexsoto.me>
* [tests] Add a .NET/macOS version of monotouch-test.
I made a macOS version of monotouch-test instead of a .NET version of xammac tests,
so that one day we might have only one test suite for all our API tests.
* Add a project file for .NET/macOS
* Fix some code to handle the fact that we're called 'monotouchtest' on macOS (but
only on .NET).
* Ignore exception marshalling tests, those aren't working yet.
* [xharness] Add support for .NET/macOS and add a macOS version of monotouch-test to our tests
* [dotnet-linker] Skip libSystem.Net.Security.Native and libSystem.Native when collecting native methods to preserve for .NET/macOS.
* [tests][sampletester] Add nuget.org single feed to sampletester
Since we no longer upstream to nuget.org our sampletester is not
able to find packages so we create a nuget config file with a single
source to nuget.org to fix it and still be compliant.
* [tests][sampletester] Remove ConferenceVision sample, it no longer exists
* [tests][sampletester] Double TimeOut for a couple of tests
This required fixing the override code so that all (not just one) the
parameter types would match. Fixes
```
build/maccatalyst/reference/Xamarin.iOS.cs(11036,50): error CS0115: 'ARSCNView.HitTest(CGPoint, ARHitTestResultType)': no suitable method found to override
build/maccatalyst/reference/Xamarin.iOS.cs(11703,50): error CS0115: 'ARSKView.HitTest(CGPoint, ARHitTestResultType)': no suitable method found to override
```
Follow up to https://github.com/xamarin/xamarin-macios/pull/10658 and
https://github.com/xamarin/xamarin-macios/pull/10678
Follow up to https://github.com/xamarin/xamarin-macios/pull/10658
This includes a `.ctor` chaining fix for the forwarder tool which
resulted in several errors like:
```
build/dotnet/maccatalyst/ref/Xamarin.iOS.cs(12640,17): error CS1729: 'UIScene' does not contain a constructor that takes 0 arguments
```
There's two different issues blocking ARKit and AddressBookUI that
will be solved in different PR.
When used with Xamarin.MacCatalyst we must still consider Xamarin.iOS.dll
as a platform assembly (and not user code) so the linker will remove more
code (that usual).
This is even more important as the stubs are unusable - so we do not want
to keep any extra pieces there since they would not work anyway.
* Bump .NET to 6.0.100-preview.2.21114.3.
* [dotnet-linker] Several steps are now gone, so load our custom step before the new first step (MarkStep).
* [dotnet-linker] Dump the current steps if we fail to call InsertBefore/InsertAfter.
* [dotnet-linker] Load the CollectAssembliesStep as the first step, and make it load every assembly.
* [dotnet] Set InvariantGlobalization=true because that's the only thing .NET supports for now.
* [dotnet-linker] Use recommended workaround for linker's inability to do load assemblies in custom step.
* [tests] Bump version of MSBuild.StructuredLogger to get support for new log version.
Otherwise this happens in tests that read binary logs:
System.NotSupportedException : Unsupported log file format. Latest supported version is 9, the log file has version 10.
* [introspection] Ignore P/Invokes to QCall for LogThreadPool* P/Invokes.
* [dotnet-linker] Inject a dummy implementation of mono_config_parse_memory as a temporary solution for mono's removal of the same method.
This fixes at least some variations of this failure:
* Protocol_17579: Add
Expected: Success
But was: DuplicateItem
In this test we try to delete any existing keychain items, but it seems that
may fail if the keychain item in question was created by a different app, in
which case we wouldn't have permission to delete it (but adding a new one
would still fail with a duplicate item error). Cleaning up any added keychain
items by the same test (and process) that added them avoids the permissions
problem.
Also clean up in other tests as well, to avoid filling up our keychain with
test stuff.
Those show up with only `!unknown-*` inside extrospection tests.
There's a few more frameworks to disable but they require some changes
inside `src/generate-type-forwarders/Program.cs` and are better done in
separate pull-requests.
* Bump Xamarin.MacDev.
New commits in xamarin/Xamarin.MacDev:
* xamarin/Xamarin.MacDev@1e738e9 [Xamarin.MacDev] Extract the code to convert between Mac Catalyst versions to a separate file. (#89)
* xamarin/Xamarin.MacDev@a3bb12c [Xamarin.MacDev] Add methods to map between iOS and macOS versions for Mac Catalyst. (#88)
Diff: 02d6d05be3..1e738e9f7f
* [msbuild] Use the macOS SDK to build Mac Catalyst apps instead of the iOS SDK
From a native compilation perspective, compilating a Mac Catalyst is the macOS SDK
+ a dash of iOS, so use the native macOS SDK to compile, and then do the corresponding
adjustments elsewhere.
At the same time document which version we want for the sdk version and the deployment
target in mtouch, and adjust the code accordingly (sdk version: macOS version; deployment
target: iOS version).
* Update resource files
* Add new entry to canary test for string localization.
These changes add support for executing iOS and MacDev tasks remotely (on a Mac) when running a build from Windows, and creates a specific .NET6 pack for Windows that's only included in the MSI.
For now this only enables builds for the iOS Simulator, physical devices are not yet supported.
- Each task decides if it should run locally or remotely depending on the SessionId property, which will only have a value on Windows.
- The XMA Build agent is now part of this repo and will be included in the iOS .NET6 Windows pack.
- On this first version we're including some Windows specific tasks and references into the Xamarin.iOS.Tasks project for simplicity, but those will be moved to the Windows specific project.
------------
* [msbuild] Adds support for executing Xamarin.iOS tasks from Windows
* [msbuild] Adds support for executing Xamarin.MacDev tasks from Windows
* Added XMA Build Agent to Xamarin.MacDev.Tasks.sln
* Fixes some MSBuild versioning problems
* Makes the XMA Build agent load Xamarin.iOS tasks
We need to load a type from the iOS tasks assembly so we can run the tasks requested by MSBuild from Windows. We only need to load Xamarin.iOS.Tasks.dll since MacDev.tasks is already embedded in that one.
There's a little trick on the csproj, we can't directly use the Xamarin.iOS.Tasks project ref assemblies because that includes both Xamarin.iOS.Tasks.dll and Xamarin.MacDev.Tasks.dll, so the MacDev tasks will collide. We use the project ref only for build dependency purposes but we add an assembly reference to Xamarin.iOS.Tasks.dll.
* Added Xamarin.iOS.Tasks.Windows project
* Removed unnecessary references on Xamarin.iOS.Tasks.Windows.csproj
* Adds Messaging assemblies when ILRepacking Xamarin Tasks
The Xamarin Task assemblies now depend on Messaging, so we need the Messaging assemblies to be packed into Xamarin.Mac.Tasks and Xamarin.iOS.Tasks. Also had to remove the direct Messaging dependencies from the build agent since those are already contained in Xamarin.iOS.Tasks
* Adds a reference to Messaging.Core targets to the Agent's project
* [msbuild] Adds Xamarin iOS Windows targets
* [msbuild] Adds missing dependencies to Xamarin.iOS.Tasks
This should fix build errors because of missing dependencies. Had to move System.Net.Mqtt.Server from the Build agent project to the tasks one to avoid conflicts with System.Diagnostics.Tracer.
* [dotnet] Creates iOS Windows pack
Creates a new pack for Windows specific (targets, build agent, etc.) files that shouldn't be installed on the Mac. We have a separate package for this to avoid increasing the core pack size with things that are not needed when using it from macOS.
* Fixes type in dotnet makefile
* [dotnet] Fixes the iOS Windows pack generation
- The windows pack should not include the Sdk and Targets folders
- For now we'll just create an iOS pack
- Fixes the path to the files to include on the Windows Sdk pack
* Added reference to the Windows iOS SDK from the Xamarin.iOS.Common.targets
Added a property to navigate to the Windows iOS SDK folder, based on a naming convention that assumes that both packs will always have the same version
* Added reference to the core iOS SDK from the Windows iOS SDK
Added a property to navigate to the core iOS SDK folder, based on a naming convention that assumes that both packs will always have the same version
* Updated Messaging version
* Override MessagingBuildClientAssemblyFile property and correctly imported props from targets
* [dotnet] Make Windows pack using target files from the output dir
We need to take the target files from the output dir to include targets that are part of nuget packages, otherwise we will only include targets from our source
* [dotnet] Adds the Windows Sdk pack to the workload manifest
* [msbuild] Fixes the Windows Sdk pack name
* [dotnet] Merge Mqtt instead of Mqtt.Server
We only need System.Net.Mqtt to be merged into Xamarin.iOS.Tasks
* Updated Messaging version
* [dotnet] Several fixes for the Windows Sdk
- Adds missing task CollectMonotouchReferences
- Merges more dependencies into Xamarin.iOS.Tasks.dll needed by XMA
- Updates the msbuild/Makefile to include files from both the output dir and the source dir
- Overrides the agents directory to look for them on the Windows pack
* [dotnet] Fixes the XMA Build agent
- The build agent is an app so it cannot target ns2.0
- The MSBuild dependencies should be copied into the agent zip file
- Avoids copying all the Xamarin iOS SDK core targets into the build agent, since those are not needed
- Ensures the broker zip file is copied into the Xamarin.iOS.Windows.Tasks output dir so its included in the Windows pack
* Bumps Xamarin.Messaging to 1.2.102
* Adds net6-win branch to trigger builds
* Adds Messaging.Client missing dependency to Xamarin.Mac.Tasks
* Added Xamarin.Messaging.Apple.Tasks project and VerifyXcodeVersion Task
* Fix unloaded Xamarin.Messaging.Build project
* Added Build contracts project and unified Xamarin.Messaigng.Apple.Tasks in Xamarin.iOS.Tasks.Windows
Also added missing tasks and changes .After.targets
* Updated Xamarin.Messaging version
* Build agent - reference MSBuild assemblies from the framework
Since the assemblies will be included in the build agent we need those to be the ones that come from the framework to be compatible with macOS
* [msbuild] Fixes _UpdateDynamicLibraryId target
The tasks con this target need to be executed remotely (when building from Windows).
* Updates resources
* Bump Xamarin.Messaging
Fixes problems when executing Exec task remotely
* [dotnet] Overrides Publish targets to execute them remotely from Windows
The `_CopyResolvedFilesToPublishPreserveNewest` and `_CopyResolvedFilesToPublishAlways` targets essentially copy files into the app bundle. Since those are part of the .NET SDK we need to override those so we can pass to the Copy task the SessionId parameter and then it will be executed remotely when building from Windows.
This is done in a Windows.After.targets file so it won't affect builds on macOS.
* Added ILMerge to Xamarin.iOS.Tasks.Windows
Also modified ILMerge.targets to not include System assemblies because we don't need them on the Windows package
* Bumps Messaging
This new version of messaging fixes a problem when copying task inputs from Windows to the Mac
* [dotnet] Fixes copying files to the Mac when building from Windows
When building from Windows there are .NET SDK targets that copy dynamic libraries from the SDK to the intermediate output directory or other files to the publish directory, since we can't control those we can't run them remotely so we need to copy those files to the Mac to ensure other targets will find those.
* [dotnet] Fixes how files are copied to the output dir
- Before executing `_CopyResolvedFilesToPublishPreserveNewest` and `_CopyResolvedFilesToPublishAlways` we copy the input files for those targets to the Mac
- Then we override the original targets to execute the same copy task as the original ones but on the Mac, so the output files are placed in the right location for the following targets to pick them up.
* Fixes typo on Xamarin.iOS.Common.After.targets
* Bumps Xamarin.Messaging
* [msbuild] Fixes VerifyXcodeVersion and ResolveUTIs tasks
Both tasks were not being able to connect to the Mac mostly because of ILRepack, there were kind of 2 versions of Xamarin.Messaging, one merged into Xamarin.iOS.Tasks and another one merged into Xamarin.iOS.Windows.Tasks. Because of this the build connection object registered on the task could not be casted to the build connection type.
This essentially moves both tasks into the Xamarin.iOS.Tasks assembly to avoid this issue, and as part of that also includes the Messaging contracts into that same project.
* [msbuild] Fixes warnings when building from Windows
* [dotnet] Adds missing assemblies to merge into Xamarin.iOS.Tasks
Those 2 new assemblies will only be used from Windows and we need their implementation instead of the ref assemblies. In the future we will need to find a way of doing this on the Windows only pack insted of doing it on the core Xamarin.iOS.Tasks assembly.
* [dotnet] Compute PublishTrimmed on a target
We need to do this so the property is evaluated after VS on Windows connects to the Mac, otherwise by default IsMacEnabled is false from Windows.
* Bumps Messaging to 1.2.111
* [dotnet] Execute ILLink remotely when building from Windows
- Overrides the ILLink task and _RunILLink target to add the hability to execute it remotely, adding input and output properties so files are copied to the server and output files are created on Windows.
- This "custom" ILLink task will only be executed from the Windows targets so when building from a Mac it will execute the core SDK task.
* [dotnet] Fixes intput/output files creation for linker tasks
- Custom Linker options file should be created on the Mac so we need to execute WriteLinesToFile remotely
- All the *.items files from the linker are created on the Mac so we need to execute ReadItemsFromFile remotely
- CompileNativeCode: fixes the OutputFile metadata path, otherwise the execution fails; also copies all the files in the declared "IncludeDirectories" to the Mac
- Avoids copying input files from Windows to the Mac when running LinkNativeCode since the real input files already exist on the Mac, and Windows contains only empty files just to make MSBuild inputs/outputs check work. If we copy those empty files to the Mac we brake the build.
* [msbuild] Minor fixes after merging from main
* [dotnet] Adds missing output files to the Xamarin.iOS.Tasks.Windows project
The output of this project was missing Messaging build targets and the build agent zip file that are needed to create the dotnet Windows specific pack
* [dotnet] Fixes dotnet Windows specific pack generation
Ensures the Windows projects are built and the files are copied to the dotnet pack directory before creating the package.
It also adds a variable to enable building this pack.
* [dotnet] Adds iOS Windows specific pack to iOS only MSI
There's only a Windows specific pack for iOS available for now, so we should only add it to the iOS SDK MSI
* [dotnet] Create a separate bundle for the iOS Windows MSI
We need to do this to avoid including the Windows specific pack in the pkg. Also for now we'll only create an MSI for iOS since it's the only supported platform from Windows.
* Fixes spacing issues in Xamarin.iOS.Tasks.csproj
* Bumps Touch.Unit back to 05db76
* Fixes formatting problems
* [msbuild] Replaces error E0176 by E0186
Because there's a warning W0176 that will overlap with the error
* [msbuild] Fixes CompileEntitlements task
There were 2 problems:
1- The if statement on the DefaultEntitlementsPath was wrong, because we should return the base value if there's no SessionId (which means the task is running on a Mac)
2- We should copy to the Mac the default entitlements file if no custom file was specified
* Several fixes to cleanup the code to support iOS from Windows
* Apply suggestions from code review
Co-authored-by: Rolf Bjarne Kvinge <rolf@xamarin.com>
* Formatting fixes in Xamarin.Messaging.Build
* Reverted formatting changes in CompileEntitlements.cs
* More formatting fixes
* Update msbuild/Messaging/Xamarin.Messaging.Build/Handlers/ExecuteTaskMessageHandler.cs
Co-authored-by: Rolf Bjarne Kvinge <rolf@xamarin.com>
* Fixes order of MSBuild errors in the resource file
* Add newly added localizable strings to canary test of translated strings.
* Delete tests that ensure theres code only on the abstract tasks
These were needed to ensure all the code was in the base tasks so we could have tasks implementations on Windows to remote those. Now that code is part of this repo (and that is why these tests are failing now) so we do not need them anymore.
* [dotnet] Don't build the Windows SDK pack if not configured to do so.
Co-authored-by: mag <mauro.agnoletti@gmail.com>
Co-authored-by: Rolf Bjarne Kvinge <rolf@xamarin.com>
This requires bumping Xamarin.MacDev.
New commits in xamarin/Xamarin.MacDev:
* xamarin/Xamarin.MacDev@02d6d05 [Xamarin.MacDev] Add an AppleSdkVersion struct which replaces IPhoneSdkVersion and MacOSXSdkVersion. (#87)
* xamarin/Xamarin.MacDev@e7ec7ef [Xamarin.MacDev] Fail gracefully if trying to grab a PList entry from a file that doesn't exist. (#86)
Diff: fae0237704..02d6d05be3
Once you have BigSur installed, to work with xcode12.5, then you
need to bump the minimum simulator versions (to 11.4) and this picked up
a few mistakes in availability and when checking protocol conformance
The bots won't run into those issues, as long as `xcode12.5` is not
merged or while they run on macOS Catalina, but it's nice to have green
tests locally :)
Subset of 8513316015 already in `xcode12.5`
* [monotouch-test] Rework big parts of KeyChainTest. Fixes #xamarin/maccore@2365.
'GenericPassword' keychain items are unique by their Service+Account
properties [1]. This means that changing the Label property will not create a
different 'GenericPassword', which has a few consequences:
* It's possible to filter (and try to delete) using the Label property.
* It's possible to try to delete a 'GenericPassword' item, and have that
deletion attempt fail with 'no item found', and then subsequently trying to
add the same item will fail with a DuplicateItem, because the deletion
was filtered using the Label property.
The change I've made is to:
* Make the Label property much more descriptive, and unique per process. This
makes it easier to figure out where things come from in the Keychain Access
app.
* Make the Service property unique per process. This way these tests are
parallel safe and they won't stomp on eachother.
* Keep the Account property the same (a constant value), so that it's easy to
filter to just these items in the Keychain Access app.
* Remove the Label property from all queries, it doesn't matter anyway. The
Label property is still set when adding items to the keychain.
Finally try to clean up after ourselves as good as possible. This way we don't
fill the keychain with test stuff. This involves removing certificates and
passwords we add to the keychain at the end of tests.
Fixes https://github.com/xamarin/maccore/issues/2365.
[1]: https://stackoverflow.com/a/11672200/183422
* Adjust query for RecordTest.AuthenticationType as well to not include fields that don't determine uniqueness.
Also make the InternetPassword items we add to the keychain more descriptive and labelled helpfully.
* Add debug code.
* Revert "Add debug code."
This reverts commit a3edac8160.
Co-authored-by: Rolf Bjarne Kvinge <rolf@xamarin.com>
Moving to macOS 11.x (BigSur) requires us to bump the minimum/tested
version of simulators to the oldest supported by the required OS.
This means iOS 11.4, tvOS 11.4 and watchOS 5.0 (with iOS 12).
xharness needs to be updated in order for watchOS 5.0, which needs
series 3, to work properly.
Filed as https://github.com/xamarin/xamarin-macios/issues/10593
This version of Xcode requires macOS 11.0 or later.
New commits in xamarin/maccore:
* xamarin/maccore@de616ca53e [mlaunch] Fix dependencies for Xcode 12.5 beta 1 (#2387)
* xamarin/maccore@e93375583c [Actions] Fix rebase trigger.
* xamarin/maccore@0dfeb85ff6 [CI][VSTS] Add information when the external tool fails. (#2377)
* xamarin/maccore@87e165fdbb Remove Vincent as a code owner 😞 (#2368)
* xamarin/maccore@55a700bb5b [build] Add /restore to Xamarin.Analysis build (#2366)
* xamarin/maccore@2d9f8a9685 Bump to xamarin/xamarin-analysis@95245313 (#2362)
Diff: 428964bf32..de616ca53e
* Bump system mono to the latest available 2020-02 package. (#10402)
This makes xamarin-macios build on Apple Silicon, and also seems to get an
updated csc that fixes a problem with nullability warnings/errors.
* [tests] Fix mmp-regression build
```
10:59:46.5383210 Making all in /Users/builder/azdo/_work/1/s/xamarin-macios/tests/mmp-regression
10:59:46.5392610 TargetFrameworkFallbackSearchPaths=/Users/builder/azdo/_work/1/s/xamarin-macios/_mac-build/Library/Frameworks/Mono.framework/External/xbuild-frameworks MSBuildExtensionsPathFallbackPathsOverride=/Users/builder/azdo/_work/1/s/xamarin-macios/_mac-build/Library/Frameworks/Mono.framework/External/xbuild make all
10:59:46.6208220 /Applications/Xcode_12.5.0-beta.app/Contents/Developer/usr/bin/make clean
10:59:46.6956100 /Applications/Xcode_12.5.0-beta.app/Contents/Developer/usr/bin/make build
10:59:46.7422460 GEN dylib/libTest.dylib
10:59:46.8094820 ld: library not found for -lSystem
10:59:46.8097590 clang: error: linker command failed with exit code 1 (use -v to see invocation)
10:59:46.8110310 make[2]: *** [dylib/libTest.dylib] Error 1
10:59:46.8112570 make[1]: *** [run] Error 2
10:59:47.0030120 Process make exited with 2
```
This happens on BigSur as it does not find the default location anymore...
* [tests] Do not test GKLeaderboardViewController on BigSur
The issue is the OS version, not the Xcode version. Update check
to skip that test on macOS 11+
* [tests] Fix MSBuild tests / integration
The test cannot work on BigSur since while you can `dlopen` the library
you can open `fopen` it as a file (and can't embed it for the test).
```
12:31:23.9407820 CSC : error CS1566: Error reading resource 'libz.dylib' -- 'Could not find file "/usr/lib/libz.dylib"' [/Users/builder/azdo/_work/1/s/xamarin-macios/tests/msbuild/Xamarin.MacDev.Tests/bin/Debug/net472/tmp-test-dir/msbuild-tests201/XM45Binding.csproj]
12:31:23.9407900 Done Building Project "/Users/builder/azdo/_work/1/s/xamarin-macios/tests/msbuild/Xamarin.MacDev.Tests/bin/Debug/net472/tmp-test-dir/msbuild-tests201/XM45Binding.csproj" (default targets) -- FAILED.
```
* [tests] Fix MTouch tests failures
* `nm` changed it's output and test was updated
* `clang` linking has a much higher limit for it's command-line length
At least it does not seem to be OS specific since both 10.15 and 11.0
returns the same maximum
```
$ getconf ARG_MAX
1048576
```
* Pass and parse the MtouchInterpreter value to dotnet-linker.
* Move required logic from mtouch to shared code to take advantage of existing
code to compute AOT arguments (which we need to compute the correct AOT
arguments for System.Private.Corelib.dll, which is always AOT-compiled).
* Add a linker task to compute the AOT arguments for .NET.
* Add test projects.
This PR is probably best reviewed commit-by-commit.
Fixes https://github.com/xamarin/xamarin-macios/issues/10292.
* [link sdk/link all] Adjust to compile on Mac Catalyst.
* [tests] Adjust the LinkAllRegressionTest.NoFatCorlib to work on Mac Catalyst.
* [tests] Add version checks to make link sdk green on Mac Catalyst.
* [tests] Make the LinkSdkRegressionTest.SpecialFolder test pass on Mac Catalyst.
* [xharness] link sdk and link all are green on Mac Catalyst now.
* Fix test build.
* [tests] Use the 'Apple Developer' code signing key instead of 'iPhone Developer' for xcframework-test.
* [msbuild] Fix resolving the XCFramework for Mac Catalyst.
* [xharness] Enable xcframework-test by default on Mac Catalyst.
See [HOWTO](https://github.com/xamarin/xamarin-macios/wiki/Nullability)
Also small change to generator so strong dictionaries can accept `null`
`NSDictionary`, which was already accepted by the class itself.
and expose two properties that were not usable before
* [tests] Add test case for single-project properties in .NET.
* [msbuild] Add support for the single-project ApplicationId MSBuild property.
* [msbuild] Add support for the single-project ApplicationTitle, ApplicationVersion and AppleShortVersion MSBuild properties.
* [dotnet] Enable the single-project MSBuild properties by default.
* [dotnet] Add a short doc about single project properties.
* [tests] Fix the GeneratePlistTaskTests.BundleIdentifier test according to bundle identifier changes.
This test asserts that the CFBundleIdentifier value in the Info.plist isn't
overwritten, and does so by calling the CompileAppManifest task, giving it a
different value for the bundle identifier than what's in the Info.plist.
The behavior change is that now we do things in the following manner:
DetectSigningIdentityTask will read the Info.plist, compute a bundle
identifier (which will be the value from the Info.plist if it's there), and
returns it to the MSBuild code. Eventually that value will be passed to the
CompileAppManifestTask, which will write it to the Info.plist.
However, this test doesn't run the DetectSigningIdentityTask, which means that
the initial value for the bundle identifier doesn't come from the Info.plist.
* [monotouch-test] Rework big parts of KeyChainTest. Fixes #xamarin/maccore@2365.
'GenericPassword' keychain items are unique by their Service+Account
properties [1]. This means that changing the Label property will not create a
different 'GenericPassword', which has a few consequences:
* It's possible to filter (and try to delete) using the Label property.
* It's possible to try to delete a 'GenericPassword' item, and have that
deletion attempt fail with 'no item found', and then subsequently trying to
add the same item will fail with a DuplicateItem, because the deletion
was filtered using the Label property.
The change I've made is to:
* Make the Label property much more descriptive, and unique per process. This
makes it easier to figure out where things come from in the Keychain Access
app.
* Make the Service property unique per process. This way these tests are
parallel safe and they won't stomp on eachother.
* Keep the Account property the same (a constant value), so that it's easy to
filter to just these items in the Keychain Access app.
* Remove the Label property from all queries, it doesn't matter anyway. The
Label property is still set when adding items to the keychain.
Finally try to clean up after ourselves as good as possible. This way we don't
fill the keychain with test stuff. This involves removing certificates and
passwords we add to the keychain at the end of tests.
Fixes https://github.com/xamarin/maccore/issues/2365.
[1]: https://stackoverflow.com/a/11672200/183422
* Adjust query for RecordTest.AuthenticationType as well to not include fields that don't determine uniqueness.
Also make the InternetPassword items we add to the keychain more descriptive and labelled helpfully.
* [dotnet] Add support for LinkDescription items.
Also use the LinkDescription item group instead of a custom one
(_BundlerXmlDefinitions), since there's no need for the custom one.
* The world (aka mmp) isn't quite ready for this yet.
* Don't parse into LinkDescription, use a temporary variable instead.
And then only use the temporary variable for .NET.
The problem is that parsing mtouch extra args and adding any --xml arguments
to LinkDescription is that the xml file will be passed again to mtouch, and in
this case any ${ProjectDir} properties won't be evaluated.
Ensure we run sanity checks so the `.todo` files remains up-to-date with
the required work. Updated existing `.todo` files to avoid duplication
errors.
Fix the report. The number of columns varies locally (more details) and
on bots. The former was not right since the original catalyst update.
* [monotouch-test] Use a more descriptive label for items added to the keychain in KeyTest.
* [monotouch-test] Use a more descriptive label for some items in CertificateTest.
Also make generated keys non-permanent so that they're not added to the keychain.
* [monotouch-test] Clean up a few items added to the keychain by CertificateTest.
We need to add the prefix to the xharness logs or we will not be able to
acceess them via a click.
Co-authored-by: Manuel de la Pena <mandel@microsoft.com>
We need to add the prefix to the xharness logs or we will not be able to
acceess them via a click.
Co-authored-by: Manuel de la Pena <mandel@microsoft.com>
* [msbuild] Unify how CFBundleName is calculated.
Previously, CFBundleName would default to:
* iOS: CFBundleDisplayName (if set), otherwise the app bundle name.
* all other platforms: the app bundle name.
Now unify the logic so that we have the same behavior on all platforms.
This is a breaking change under the following conditions:
* Building for iOS
* CFBundleName is not set in the Info.plist
* CFBundleDisplayName is set in the Info.plist
* CFBundleDisplayName from the Info.plist is different from AssemblyName in
the csproj (which is the value used to calculate the app bundle name).
The fix would be to:
* Set CFBundleName in the Info.plist to the desired value.
This change works in previous versions of Xamarin.iOS as well.
* Update tests.
* [dotnet-linker] Add the RemoveUserResources sub step.
This comes with a few changes to the RemoveUserResources sub step as well:
* Bail out earlier if we're in the simulator (no need to do any processing at
all - we know at the very beginning if we're building for the simulator).
* Do a positive simulator check, instead of a negative device check (because
Mac[Catalyst] are neither devices - so they pass the negative device check).
* Remove all the conditional mtouch/mmp code, and figure out at runtime which
resource prefixes we need to check for.
It was also necessary to change where steps are added to the pipeline: we have
to remove resources before the OutputStep, but at the same time we have to do
it after the ExtractBindingLibraries step, otherwise the
ExtractBindingLibraries step won't find any binding libraries to extract. So
move the ExtractBindingLibraries, LoadNonSkippedAssemblies and
ListExportedSymbols to before the OutputStep (to keep their internal order),
and then add the RemoveUserResources after those.
This fixes the following link sdk/link all test when running on device:
BundledResources.ResourcesTest
[FAIL] Bundled : No resources
Expected: 0
But was: 2
* [tests] Update the BuildInterdependentBindingProjects test now that we're removing resources.
Fixes this warning when running xharness:
Unknown file: fsharplibrary.fsproj (extension: .fsproj). There might be a default inclusion behavior for this file.
* Fix system version checks to work properly on Mac Catalyst (which uses the macOS
version as its system version).
* Add the framework-specific defines to the build for monotouch-test.csproj (using
the generated response files), this way we can use them in the tests.
* Sprinkle conditionals in numerous places - I tried using either framework-specific
or XAMCORE_3_0 whenever that made since (instead of using Mac Catalyst as a condition).
* Updated a few tests to use non-deprecated API (because the deprecated API often
doesn't exist in Mac Catalyst).
* Also a few minor API fixes to make any corresponding tests compile.
TL&DR:
This effectively change nothing - but prevents (warn) if both options
contradict themselves.
Long Story....
So we have two ways to control the codesign's `--timetamp` option but
they both ignore each other so we can end up with the option being
set more than once at build time.
`DisableTimestamp` was the original one. It was meant for iOS (and
derivative OS) and disable the option (which requires the network)
for simulator or debug builds. IOW we _wanted_ timestamps when doing
release builds for devices.
```
DisableTimestamp="$(_CodesignDisableTimestamp)"
```
```
<_CodesignDisableTimestamp>False</_CodesignDisableTimestamp>
<_CodesignDisableTimestamp Condition="'$(_SdkIsSimulator)' == 'true' Or '$(_BundlerDebug)' == 'true'">True</_CodesignDisableTimestamp>
```
Now disabling the timestamp did not mean it was enabled. We did not ask
for a timestamp, leaving it to the default which from `man codesign`
means:
> If this option is not given at all, a system-specific default behavior is invoked.
> This may result in some but not all code signatures being timestamped.
Then `UseSecureTimestamp` was added for macOS builds. If hardening is
enabled then a secure timestamp is required.
`msbuild/Xamarin.Mac.Tasks/Xamarin.Mac.Common.targets` `_CodesignAppBundle`
```
UseSecureTimestamp="$(UseHardenedRuntime)"
```
However it's also exposed for iOS (shared target) in
`msbuild/Xamarin.Shared/Xamarin.Shared.targets` `__CodesignNativeLibraries`
but it would always be `false` in that case.
Adding this option means there's now always a `--timestamp` option given,
either to enable it (no URL so it means using Apple's server) or to
disable it (`=none`) but since it's controlled by `UseHardenedRuntime`,
which is macOS only, then iOS device builds are never timestamped.
An alternative would be to have `UseSecureTimestamp` as a macOS-only
option - but that would change how we currently sign the iOS applications
and I'd rather not change things that are known to work.
According to Apple's documentation, an app linked with iOS 12 or later must
enable a specific entitlement, otherwise calls to CNCopyCurrentNetworkInfo
will always return NULL.
All the other *NoProxy tests use the Microsoft uri, which seems to be the uri
that's not supposed to need a proxy (according to what I understand from the
code) - in other words, it looks like this was a c&p error.
Fixes this test failure when running on device:
[FAIL] TestPACParsingAsyncNoProxy : Expected: None
But was: HTTPS
at MonoTouchFixtures.CoreFoundation.ProxyTest.TestPACParsingAsyncNoProxy () [0x000fa] in /Users/rolf/work/maccore/onedotnet/xamarin-macios/tests/monotouch-test/CoreFoundation/ProxyTest.cs:238
* [monotouch-test] Adjust ColorTest.CreateByMatchingToColorSpace to be Xcode 11+ only.
This test uses CGColor.CreateSrgb, which is Xcode 11+ only.
* [monotouch-test] Restrict AvoidOccluderConstraintTest to iOS 11+.
TL&DR: This PR
1. Removes the creation of the `.dSYM` based on `Debug Information` [1]
2. Adds dSYM support to XM msbuild (now shared with XI implementation)
3. Archive the `.dSYM` directories (plural) properly, e.g.
```
msbuild -p:Configuration=Release -p:ArchiveOnBuild=true
```
Why ? The long story...
Historically `.dSYM` for Xamarin.Mac have not been very useful, largely
because (most of) the code is JITed so not much is known before runtime.
So they were simply not generated during the builds...
However AOT options were added to Xamarin.Mac, making them potentially
more useful. Also symbols from `libmono` and other native libraries /
frameworks can prove useful when diagnosing application crashes.
Unsurprisingly developers looking to get symbols eventually found _a way_
[1] to get a `.dSYM` for their applications - but it was not quite
correct because:
* setting the debug information option meant that `mmp` would be supplied with `-debug`. This disables several optimizations that are, by default, enabled for release builds. IOW generating symbols should have no effect on the executing code (but it had);
* it was produced when compiling the native launcher, so the symbols coverage was incomplete. How much depends if mono was statically or dynamically linked. However this would not cover any AOTed code nor bundled libraries or user frameworks.
* the .dSYM was produced inside the `x.app/Contents/MacOS/`, side-by-side with the native executable, which makes it part of the **signed** `.app` and also part of the created (and signed) `.pkg`. This had a large impact on the application's, disk and download, size(s). Manually (re)moving the `.dSYM` means re-signing the app and re-creating (and signing) the `.pkg` is not a good solution.
[1] https://forums.xamarin.com/discussion/139705/how-to-symbolicate-a-xam-mac-crash-log
Additional fixes
* Use `Directory.Move` instead of running the `mv` command
While the result is identical there is a cost to spawn several `mv`
processes. Doing it in parallel (might have) helped but that setup
also comes at a cost.
`Directory.Move` the four `.dylib.dSYM` of an app takes 1 ms, while
the existing code took 17 ms to do the same.
* Fix building mmptest since the DeleteDebugSymbolCommand constant is not present (nor used) anymore
* [monotouch-test] Fix conditional compilation constants when building for device.
* [tests/monotouch-test] Don't use entitlements when building monotouch-test for device.
* Bump to .NET 6.0.100-alpha.1.21060.3.
* Fix dotnet command line arguments.
* dotnet build: the project file must be the first argument.
* dotnet build/publish: use the documented verbosity format.
* Update version number in tests.
* [tests/introspection] Adjust introspection to cope with different library names in the new .NET version.
* [tests/link sdk] Adjust the LinkSdkRegressionTest.SpecialFolder test according to the new version of .NET 6.
* [tests/link sdk] Preserve a required method in System.Private.CoreLib to work around a bug in .NET 6.
Ref: https://github.com/dotnet/runtime/issues/46908.
* Revert "[CI][VSTS] Add the donet 6 pkg as a dependency. (#10348)"
This reverts commit 6de4e717e7.
There's no need to provision .NET 6, it's done automatically.
Mac Catalyst needs much of the Mac signing logic, so merge the iOS and Mac
versions of DetectSigningIdentity so that the Xamarin.iOS.Tasks assembly
contains what it needs for Mac Catalyst.
* [msbuild] Unify handling of Sdks.
Create an Sdks class in Xamarin.MacDev.Tasks.Core, which handles both Xamarin.iOS
and Xamarin.Mac. Refactor the MacOSXSdks and IPhoneSdks classes to use the new Sdks
class.
This makes it possible to avoid some code duplication in MacOSXSdks and IPhoneSdks,
and also share code elsewhere.
This requires a bump of Xamarin.MacDev.
New commits in xamarin/Xamarin.MacDev:
* xamarin/Xamarin.MacDev@fae0237 [Xamarin.MacDev] Add GetAppleDTSettings and GetSdkSettings to the IAppleSdk interface. (#85)
Diff: f665e3a0fc..fae0237704
* Translate exception message.
* Simplify according to review.
* Update list of non-translated strings.
TD&LR: This PR simplifies how we refer to user frameworks and fixes both
warnings and non-optimal (app) output.
Much longer story:
Additional testing on macOS showed some build-time warnings and an
[extra (dupe) file](a20f8aba41 (diff-54fd7d9cd5deae57f30195be0a43133eace03c1132401741a317e0ae8d5e13fdR34)).
Logs shows that we referred to the xcframework several times, where once
should have been enough.
```
/native-reference:/Users/poupou/git/spouliot/xcframework/Universal.xcframework
/native-reference:/Users/poupou/git/spouliot/xcframework/Universal.xcframework/macos-arm64_x86_64/Universal.framework
/native-reference:/Users/poupou/git/spouliot/xcframework/Universal.xcframework/macos-arm64_x86_64/Universal.framework/Universal
```
The first `/native-reference` line produced a warning like:
```
MMP warning MM2006: Native library 'Universal.xcframework' was referenced but could not be found.
```
which makes sense as the tools (both `mmp` and `mtouch`) are not, by
design, aware of (unresolved) xcframeworks.
Removing `{NativeReference}` from `Xamarin.Mac.Common.targets` (and
`Xamarin.iOS.Common.targets`) as it has already been processed by
`_ExpandNativeReferences` solves this.
The other part of the issue (next two lines) is because `msbuild` does
not track changes to directories like it does for files - and the
workaround (in `_ExpandNativeReferences`) had to be copied in other
places (both XI and XM `_CompileToNative`) and that was not enough (and
would eventually need to be duplicated again and again).
This could lead to duplicate entries (i msbuild logs) like
```
NativeReferences
../../Universal.xcframework/macos-arm64_x86_64/Universal.framework
../../Universal.xcframework/macos-arm64_x86_64/Universal.framework/Univeral
```
which maps to our extra entries.
In order to simplify things we make the `_ExpandNativeReferences` resolve
the full path to the library name (not the `.framework` directory) which
simplifies both `_CompileToNative` and ensure a single way (at least for
`msbuild`) to provide this data to the tools (`mmp` and `mtouch`).
Using a file, instead of a directory, is also more consistent for the
existing `-framework` option, e.g. we provide the names like:
```
--framework=CoreLocation
--framework=ModelIO
```
So adding a full path that include the name is more appropriate, e.g.
``` --framework=/Users/poupou/git/master/xamarin-macios/tests/xharness/tmp-test-dir/xcframework-test760/bin/AnyCPU/Debug/bindings-xcframework-test.resources/XTest.xcframework/ios-i386_x86_64-simulator/XTest.framework/XTest
```
Finally for macOS applications it turns out we were embedding yet another
copy of the framework's library inside the `MonoBundle`, which is clearly
wrong, because of the last entry.
```
$ l bin/Release/xcf-mac.app/Contents/MonoBundle/Universal
-rwxr-xr-x 1 poupou staff 167152 2 Dec 16:16 bin/Release/xcf-mac.app/Contents/MonoBundle/Universal
```
The tool now checks if a provided library is inside a framework (or not)
which is a good validation to have anyway when it gets called directly,
i.e. not thru `msbuild`.
Sebastien Pouliot
Rolf Bjarne Kvinge
Alex Soto
Manuel de la Pena
Rachel Kang
Rachel Kang
VS MobileTools Engineering Service 2
Emanuel Fernandez Dell'Oca
Sebastien Pouliot