xamarin-macios/tests/dotnet
Rolf Bjarne Kvinge 7da04184cf
[tests] Improve the CustomizedCodeSigning tests to not depend on file order. (#20889)
The CustomizedCodeSigning test asserts that a certain condition shows a
particular error message from codesign.

There are multiple files in the app bundle that can trigger this particular
message, so change the logic to not assert on a particular file, instead use
assert on the remained of the error message.

Fixes this random test failure:

    Xamarin.Tests.DotNetProjectTest.CustomizedCodeSigning(iOS,"ios-arm64"): Failure when comparing error messages:
        Unexpected error message #0:
            Expected: /usr/bin/codesign exited with code 1:\n/Users/builder/azdo/_work/4/s/xamarin-macios/tests/dotnet/CustomizedCodeSigning/iOS/bin/Debug/net9.0-ios/ios-arm64/CustomizedCodeSigning.app: replacing existing signature\n/Users/builder/azdo/_work/4/s/xamarin-macios/tests/dotnet/CustomizedCodeSigning/iOS/bin/Debug/net9.0-ios/ios-arm64/CustomizedCodeSigning.app: code object is not signed at all\nIn subcomponent: /Users/builder/azdo/_work/4/s/xamarin-macios/tests/dotnet/CustomizedCodeSigning/iOS/bin/Debug/net9.0-ios/ios-arm64/CustomizedCodeSigning.app/System.Diagnostics.DiagnosticSource.dll
            Actual: /usr/bin/codesign exited with code 1:\n/Users/builder/azdo/_work/4/s/xamarin-macios/tests/dotnet/CustomizedCodeSigning/iOS/bin/Debug/net9.0-ios/ios-arm64/CustomizedCodeSigning.app: replacing existing signature\n/Users/builder/azdo/_work/4/s/xamarin-macios/tests/dotnet/CustomizedCodeSigning/iOS/bin/Debug/net9.0-ios/ios-arm64/CustomizedCodeSigning.app: code object is not signed at all\nIn subcomponent: /Users/builder/azdo/_work/4/s/xamarin-macios/tests/dotnet/CustomizedCodeSigning/iOS/bin/Debug/net9.0-ios/ios-arm64/CustomizedCodeSigning.app/System.Collections.NonGeneric.aotdata.arm64
        Unexpected error message #1:
            Expected: Failed to codesign '/Users/builder/azdo/_work/4/s/xamarin-macios/tests/dotnet/CustomizedCodeSigning/iOS/bin/Debug/net9.0-ios/ios-arm64/CustomizedCodeSigning.app': /Users/builder/azdo/_work/4/s/xamarin-macios/tests/dotnet/CustomizedCodeSigning/iOS/bin/Debug/net9.0-ios/ios-arm64/CustomizedCodeSigning.app: replacing existing signature\n/Users/builder/azdo/_work/4/s/xamarin-macios/tests/dotnet/CustomizedCodeSigning/iOS/bin/Debug/net9.0-ios/ios-arm64/CustomizedCodeSigning.app: code object is not signed at all\nIn subcomponent: /Users/builder/azdo/_work/4/s/xamarin-macios/tests/dotnet/CustomizedCodeSigning/iOS/bin/Debug/net9.0-ios/ios-arm64/CustomizedCodeSigning.app/System.Diagnostics.DiagnosticSource.dll
            Actual: Failed to codesign '/Users/builder/azdo/_work/4/s/xamarin-macios/tests/dotnet/CustomizedCodeSigning/iOS/bin/Debug/net9.0-ios/ios-arm64/CustomizedCodeSigning.app': /Users/builder/azdo/_work/4/s/xamarin-macios/tests/dotnet/CustomizedCodeSigning/iOS/bin/Debug/net9.0-ios/ios-arm64/CustomizedCodeSigning.app: replacing existing signature\n/Users/builder/azdo/_work/4/s/xamarin-macios/tests/dotnet/CustomizedCodeSigning/iOS/bin/Debug/net9.0-ios/ios-arm64/CustomizedCodeSigning.app: code object is not signed at all\nIn subcomponent: /Users/builder/azdo/_work/4/s/xamarin-macios/tests/dotnet/CustomizedCodeSigning/iOS/bin/Debug/net9.0-ios/ios-arm64/CustomizedCodeSigning.app/System.Collections.NonGeneric.aotdata.arm64
        All errors:
            /usr/bin/codesign exited with code 1:
                /Users/builder/azdo/_work/4/s/xamarin-macios/tests/dotnet/CustomizedCodeSigning/iOS/bin/Debug/net9.0-ios/ios-arm64/CustomizedCodeSigning.app: replacing existing signature
                /Users/builder/azdo/_work/4/s/xamarin-macios/tests/dotnet/CustomizedCodeSigning/iOS/bin/Debug/net9.0-ios/ios-arm64/CustomizedCodeSigning.app: code object is not signed at all
                In subcomponent: /Users/builder/azdo/_work/4/s/xamarin-macios/tests/dotnet/CustomizedCodeSigning/iOS/bin/Debug/net9.0-ios/ios-arm64/CustomizedCodeSigning.app/System.Collections.NonGeneric.aotdata.arm64
                        Failed to codesign '/Users/builder/azdo/_work/4/s/xamarin-macios/tests/dotnet/CustomizedCodeSigning/iOS/bin/Debug/net9.0-ios/ios-arm64/CustomizedCodeSigning.app': /Users/builder/azdo/_work/4/s/xamarin-macios/tests/dotnet/CustomizedCodeSigning/iOS/bin/Debug/net9.0-ios/ios-arm64/CustomizedCodeSigning.app: replacing existing signature
                /Users/builder/azdo/_work/4/s/xamarin-macios/tests/dotnet/CustomizedCodeSigning/iOS/bin/Debug/net9.0-ios/ios-arm64/CustomizedCodeSigning.app: code object is not signed at all
                In subcomponent: /Users/builder/azdo/_work/4/s/xamarin-macios/tests/dotnet/CustomizedCodeSigning/iOS/bin/Debug/net9.0-ios/ios-arm64/CustomizedCodeSigning.app/System.Collections.NonGeneric.aotdata.arm64

---------

Co-authored-by: Alex Soto <alex@alexsoto.me>
Co-authored-by: Manuel de la Pena <mandel@microsoft.com>
2024-08-12 18:58:17 +02:00
..
AdditionalAppExtensionConsumer [autoformat] Add .NET tests. (#16811) 2022-11-21 09:45:49 +01:00
AppWithGenericLibraryReference [autoformat] Add .NET tests. (#16811) 2022-11-21 09:45:49 +01:00
AppWithNativeDynamicLibrariesInPackageReference [msbuild] Detect any dylibs in NativeReferences. Fixes #19520. (#19560) 2023-12-04 20:11:20 +01:00
AppWithResources [autoformat] Add .NET tests. (#16811) 2022-11-21 09:45:49 +01:00
AppWithWinExeOutputType [msbuild] Validate OutputType property (#16871) 2022-12-22 07:36:04 +01:00
AppWithXCAssets [autoformat] Add .NET tests. (#16811) 2022-11-21 09:45:49 +01:00
AppWithXCFrameworkWithStaticLibraryInPackageReference [msbuild] Fix resolving static libraries inside an XCFramework in a NuGet. (#19502) 2023-11-27 08:54:48 +01:00
AutoDetectEntitlements [autoformat] Add .NET tests. (#16811) 2022-11-21 09:45:49 +01:00
BindingOldStyle [tests] Convert spaces to tabs in numerous test projects. (#15855) 2022-09-05 10:54:33 +02:00
BindingWithDefaultCompileInclude [tests] Convert spaces to tabs in numerous test projects. (#15855) 2022-09-05 10:54:33 +02:00
BindingWithUncompressedResourceBundle [dotnet/msbuild] Copy binding resource packages to output directories together with CopyLocal assemblies. Fixes #13910. (#17806) 2023-03-22 10:49:26 +01:00
BundleStructure [tests] Skip framework-test on iOS and tvOS. (#20590) 2024-05-30 10:34:14 +02:00
CatalystAppOptimizedForMacOS [autoformat] Add .NET tests. (#16811) 2022-11-21 09:45:49 +01:00
CentralPackageVersionsApp [autoformat] Add .NET tests. (#16811) 2022-11-21 09:45:49 +01:00
CustomizedCodeSigning [autoformat] Add .NET tests. (#16811) 2022-11-21 09:45:49 +01:00
Entitlements [autoformat] Add .NET tests. (#16811) 2022-11-21 09:45:49 +01:00
ExceptionalTestApp [runtime] Call mono_unhandled_exception to raise AppDomain.UnhandledException. (#20656) 2024-05-29 20:14:47 +02:00
ExtensionConsumer [autoformat] Add .NET tests. (#16811) 2022-11-21 09:45:49 +01:00
ExtensionConsumerWithFrameworks [dotnet] Point app extensions to any frameworks in the root app bundle. Fixes #17876. (#18913) 2023-09-11 09:52:54 +02:00
ExtensionProject [autoformat] Add .NET tests. (#16811) 2022-11-21 09:45:49 +01:00
ExtensionProjectWithFrameworks [dotnet] Point app extensions to any frameworks in the root app bundle. Fixes #17876. (#18913) 2023-09-11 09:52:54 +02:00
LibraryReferencingBindingLibrary [tests] Skip framework-test on iOS and tvOS. (#20590) 2024-05-30 10:34:14 +02:00
MultiTargetingLibrary [dotnet] Multi target with Xcode 15.0 (#20155) 2024-02-28 09:03:53 +01:00
MyCatalystApp [tests] Use the default RuntimeIdentifier in MyCatalystApp (#19767) 2024-01-10 11:19:40 +01:00
MyClassLibrary [tests] Convert spaces to tabs in numerous test projects. (#15855) 2022-09-05 10:54:33 +02:00
MyClassRedirectApp [dotnet] Test integration (#18543) 2023-10-17 10:29:12 -04:00
MyCocoaApp [dotnet] Create an MSBuild property for the current min OS version. (#17034) 2023-01-10 21:18:39 +01:00
MyInterpretedApp [autoformat] Add .NET tests. (#16811) 2022-11-21 09:45:49 +01:00
MyMauiApp [dotnet] Delay computing trimming options until MAUI has had a chance to change the default. (#20970) 2024-08-07 14:43:53 +02:00
MyPartialAppManifestApp [autoformat] Add .NET tests. (#16811) 2022-11-21 09:45:49 +01:00
MyRegistrarApp [autoformat] Add .NET tests. (#16811) 2022-11-21 09:45:49 +01:00
MySatelliteAssembliesLibrary [tests] Convert spaces to tabs in numerous test projects. (#15855) 2022-09-05 10:54:33 +02:00
MySimpleApp [dotnet] Force 'AppendRuntimeIdentifierToOutputPath=true' for the inner build of universal apps. (#20839) 2024-07-10 13:00:05 +02:00
MySimpleAppWithSatelliteReference [autoformat] Add .NET tests. (#16811) 2022-11-21 09:45:49 +01:00
MySingleView [dotnet] Create an MSBuild property for the current min OS version. (#17034) 2023-01-10 21:18:39 +01:00
MyTVApp [dotnet] Create an MSBuild property for the current min OS version. (#17034) 2023-01-10 21:18:39 +01:00
NativeDynamicLibraryReferencesApp [autoformat] Add .NET tests. (#16811) 2022-11-21 09:45:49 +01:00
NativeFileReferencesApp [autoformat] Add .NET tests. (#16811) 2022-11-21 09:45:49 +01:00
NativeFrameworkReferencesApp [autoformat] Add .NET tests. (#16811) 2022-11-21 09:45:49 +01:00
NativeXCFrameworkReferencesApp [autoformat] Add .NET tests. (#16811) 2022-11-21 09:45:49 +01:00
Net6_0SimpleApp [tests] Hardcode the SupportedOSPlatformVersion for the .NET 6 tests. (#17248) 2023-01-17 17:09:47 +01:00
Net7_0SimpleApp [tests] Disable the NUnitLite and Touch.Unit references for the .NET 7 tests. 2023-06-22 08:44:47 +02:00
RebuildTestAppWithLibraryReference [msbuild] Re-aot referencing assemblies. Fixes #17708. (#18509) 2023-08-07 20:19:30 +02:00
SimpleAppWithOldReferences [autoformat] Add .NET tests. (#16811) 2022-11-21 09:45:49 +01:00
UnitTests [tests] Improve the CustomizedCodeSigning tests to not depend on file order. (#20889) 2024-08-12 18:58:17 +02:00
WidgetConsumer [autoformat] Add .NET tests. (#16811) 2022-11-21 09:45:49 +01:00
WidgetExtension [autoformat] Add .NET tests. (#16811) 2022-11-21 09:45:49 +01:00
Windows [tests] Fix a couple of minor issues. (#20843) 2024-07-10 12:57:30 +02:00
size-comparison [net8.0] Merge main into net8.0. 2023-09-14 07:23:52 +02:00
.gitignore Make our local .NET the default .NET for the build. (#15086) 2022-06-07 10:11:02 +02:00
Makefile [tests] Make the 'run-tests' and 'run-unit-tests' targets equivalent. (#20223) 2024-03-01 09:54:33 +01:00
README.md [appcompare] Remove old/local copy of the tool and update comparison documentation (#14190) 2022-02-18 13:04:28 +01:00

README.md

.net tests

size-comparison

To install the latest appcompare tool do:

$ dotnet tool install --global appcompare

You can update it to the latest version by running:

$ dotnet tool update --global appcompare

The current directory might point to a different and incompatible dotnet SDK, resulting in an error. However running the command from a different location (outside the repo) should work.

Easier Analysis

If you want to read/compare the IL inside the assemblies you need to disable IL stripping.

  • Legacy (oldnet)

Add this option inside the Release|iPhone configuration of size-comparison/MySingleView/oldnet/MySingleView.csproj

<MtouchExtraArgs>--nostrip</MtouchExtraArgs>
  • net6

Build with /p:EnableAssemblyILStripping=false set. The MtouchExtraArgs legacy option is also honored.