Using provisionator could result in the UI prompt being shown which will
make the pipeline hang until there is human interaction.
Move the step to be one of the first ones to ensure provisionator does
not get stuck.
Update the .NET unit tests to parse the binlog instead of standard output for
specific build output, since we're not printing a diagnostic build log to
standard output anymore.
This fixes numerous test failures in the .NET unit tests:
* Xamarin.Tests.DotNetProjectTest.BuildBindingsTest("watchOS"): Assemblies
Expected: not <empty>
But was: <empty>
* Xamarin.Tests.DotNetProjectTest.BuildBundledResources("iOS","monotouch"): Assemblies
Expected: not <empty>
But was: <empty>
* Xamarin.Tests.DotNetProjectTest.BuildEmbeddedResourcesTest("iOS"): Assemblies
Expected: not <empty>
But was: <empty>
* Xamarin.Tests.DotNetProjectTest.BuildBundledResources("watchOS","monotouch"): Assemblies
Expected: not <empty>
But was: <empty>
* Xamarin.Tests.DotNetProjectTest.BuildBindingsTest2("tvOS"): Assemblies
Expected: not <empty>
But was: <empty>
* Xamarin.Tests.DotNetProjectTest.BuildBindingsTest("tvOS"): Assemblies
Expected: not <empty>
But was: <empty>
* Xamarin.Tests.DotNetProjectTest.BuildBindingsTest("iOS"): Assemblies
Expected: not <empty>
But was: <empty>
* Xamarin.Tests.DotNetProjectTest.BuildBundledResources("tvOS","monotouch"): Assemblies
Expected: not <empty>
But was: <empty>
* Xamarin.Tests.DotNetProjectTest.BuildMyTVApp: Linker did not executed as expected.
Expected: String containing "Building target "_RunILLink" completely."
But was: "Microsoft (R) Build Engine version 16.8.0-preview-20475-05+aed5e7ed0 for .NET [...]"
* Xamarin.Tests.DotNetProjectTest.BuildMySingleView: Linker did not executed as expected.
Expected: String containing "Building target "_RunILLink" completely."
But was: "Microsoft (R) Build Engine version 16.8.0-preview-20475-05+aed5e7ed0 for .NET [...]"
* Xamarin.Tests.DotNetProjectTest.BuildInterdependentBindingProjects("iOS"): Assemblies
Expected: not <empty>
But was: <empty>
* Xamarin.Tests.DotNetProjectTest.BuildEmbeddedResourcesTest("tvOS"): Assemblies
Expected: not <empty>
But was: <empty>
* Xamarin.Tests.DotNetProjectTest.BuildBindingsTest2("watchOS"): Assemblies
Expected: not <empty>
But was: <empty>
* Xamarin.Tests.DotNetProjectTest.BuildFSharpLibraryTest("tvOS"): Assemblies
Expected: not <empty>
But was: <empty>
* Xamarin.Tests.DotNetProjectTest.BuildFSharpLibraryTest("watchOS"): Assemblies
Expected: not <empty>
But was: <empty>
* Xamarin.Tests.DotNetProjectTest.BuildEmbeddedResourcesTest("watchOS"): Assemblies
Expected: not <empty>
But was: <empty>
* Xamarin.Tests.DotNetProjectTest.BuildBindingsTest2("iOS"): Assemblies
Expected: not <empty>
But was: <empty>
* Xamarin.Tests.DotNetProjectTest.BuildInterdependentBindingProjects("tvOS"): Assemblies
Expected: not <empty>
But was: <empty>
* Xamarin.Tests.DotNetProjectTest.BuildFSharpLibraryTest("iOS"): Assemblies
Expected: not <empty>
But was: <empty>
We're handling `ObjectDisposedException` a bit inconsistently in our
bindings.
Some, but not all, manual bindings have checks (not all of them are fully
consistent) but there's none for generated bindings, which can lead to
errors wrt nullability.
**Example**
```
[Test]
public void IncompleteNullabilityCheck ()
{
NSString s = new NSString ("bonjour");
s.Dispose ();
ManagedLayer (s);
}
void ManagedLayer (NSString s)
{
// this is similar to the generated binding code
if (s == null)
throw new ArgumentNullException ("s");
NativeCode (s.Handle);
}
void NativeCode (IntPtr p)
{
// let's assume this is native and dereference the pointer
if (p == IntPtr.Zero)
Assert.Fail ("boo");
}
```
This shows that we can, _if disposed_, provide `nil` to a native API for
which we know `nil` is not a valid argument. That can crash a process in
a situation that was possible to detect and throw a (catchable) managed
exception.
Adding dispose checks everywhere could be costly (in size) unless we
share that code with the null check (and a few other optimizations could
be applied too). Sharing the code would also ensure more consistency
across all bindings.
**Impact**
Here's the IL for an existing `CGColor` constructor that *already*
does the dispose check.
```
.method public hidebysig specialname rtspecialname
instance void .ctor (
class CoreGraphics.CGColorSpace colorspace,
valuetype System.nfloat[] components
) cil managed
{
// Method begins at RVA 0x20c834
// Code size 82 (0x52)
.maxstack 3
IL_0000: ldarg.0
IL_0001: call instance void [mscorlib]System.Object::.ctor()
IL_0006: ldarg.2
IL_0007: brtrue.s IL_0014
IL_0009: ldstr "components"
IL_000e: newobj instance void [mscorlib]System.ArgumentNullException::.ctor(string)
IL_0013: throw
IL_0014: ldarg.1
IL_0015: brtrue.s IL_0022
IL_0017: ldstr "colorspace"
IL_001c: newobj instance void [mscorlib]System.ArgumentNullException::.ctor(string)
IL_0021: throw
IL_0022: ldarg.1
IL_0023: ldfld native int CoreGraphics.CGColorSpace::handle
IL_0028: ldsfld native int [mscorlib]System.IntPtr::Zero
IL_002d: call bool [mscorlib]System.IntPtr::op_Equality(native int, native int)
IL_0032: brfalse.s IL_003f
IL_0034: ldstr "colorspace"
IL_0039: newobj instance void [mscorlib]System.ObjectDisposedException::.ctor(string)
IL_003e: throw
IL_003f: ldarg.0
IL_0040: ldarg.1
IL_0041: ldfld native int CoreGraphics.CGColorSpace::handle
IL_0046: ldarg.2
IL_0047: call native int CoreGraphics.CGColor::CGColorCreate(native int, valuetype System.nfloat[])
IL_004c: stfld native int CoreGraphics.CGColor::handle
IL_0051: ret
} // end of method CGColor::.ctor
Here's the IL code (as committed) using a helper extension method.
```
.method public hidebysig specialname rtspecialname
instance void .ctor (
class CoreGraphics.CGColorSpace colorspace,
valuetype System.nfloat[] components
) cil managed
{
// Method begins at RVA 0x210558
// Code size 46 (0x2e)
.maxstack 3
.locals init (
[0] native int handleof_colorspace
)
IL_0000: ldarg.0
IL_0001: call instance void [mscorlib]System.Object::.ctor()
IL_0006: ldarg.2
IL_0007: brtrue.s IL_0014
IL_0009: ldstr "components"
IL_000e: newobj instance void [mscorlib]System.ArgumentNullException::.ctor(string)
IL_0013: throw
IL_0014: ldarg.1
IL_0015: ldstr "colorspace"
IL_001a: call native int ObjCRuntime.NativeObjectHelper::GetNonNullHandle(class ObjCRuntime.INativeObject, string)
IL_001f: stloc.0
IL_0020: ldarg.0
IL_0021: ldloc.0
IL_0022: ldarg.2
IL_0023: call native int CoreGraphics.CGColor::CGColorCreate(native int, valuetype System.nfloat[])
IL_0028: stfld native int CoreGraphics.CGColor::handle
IL_002d: ret
} // end of method CGColor::.ctor
```
So existing checks becomes a lot smaller. Here's another example where
a dispose check is **missing** (on `pattern`). Note that the check for
`colorspace` is not updated so we can see the impact of adding missing
dispose checks in existing code (manual or generated).
```
.method public hidebysig specialname rtspecialname
instance void .ctor (
class CoreGraphics.CGColorSpace colorspace,
class CoreGraphics.CGPattern pattern,
valuetype System.nfloat[] components
) cil managed
{
// Method begins at RVA 0x210660
// Code size 126 (0x7e)
.maxstack 4
IL_0000: ldarg.0
IL_0001: call instance void [mscorlib]System.Object::.ctor()
IL_0006: ldarg.1
IL_0007: brtrue.s IL_0014
IL_0009: ldstr "colorspace"
IL_000e: newobj instance void [mscorlib]System.ArgumentNullException::.ctor(string)
IL_0013: throw
IL_0014: ldarg.1
IL_0015: ldfld native int CoreGraphics.CGColorSpace::handle
IL_001a: ldsfld native int [mscorlib]System.IntPtr::Zero
IL_001f: call bool [mscorlib]System.IntPtr::op_Equality(native int, native int)
IL_0024: brfalse.s IL_0031
IL_0026: ldstr "colorspace"
IL_002b: newobj instance void [mscorlib]System.ObjectDisposedException::.ctor(string)
IL_0030: throw
IL_0031: ldarg.2
IL_0032: brtrue.s IL_003f
IL_0034: ldstr "pattern"
IL_0039: newobj instance void [mscorlib]System.ArgumentNullException::.ctor(string)
IL_003e: throw
IL_003f: ldarg.3
IL_0040: brtrue.s IL_004d
IL_0042: ldstr "components"
IL_0047: newobj instance void [mscorlib]System.ArgumentNullException::.ctor(string)
IL_004c: throw
IL_004d: ldarg.0
IL_004e: ldarg.1
IL_004f: ldfld native int CoreGraphics.CGColorSpace::handle
IL_0054: ldarg.2
IL_0055: callvirt instance native int CoreFoundation.NativeObject::get_Handle()
IL_005a: ldarg.3
IL_005b: call native int CoreGraphics.CGColor::CGColorCreateWithPattern(native int, native int, valuetype System.nfloat[])
IL_0060: stfld native int CoreGraphics.CGColor::handle
IL_0065: ldarg.0
IL_0066: ldfld native int CoreGraphics.CGColor::handle
IL_006b: ldsfld native int [mscorlib]System.IntPtr::Zero
IL_0070: call bool [mscorlib]System.IntPtr::op_Equality(native int, native int)
IL_0075: brfalse.s IL_007d
IL_0077: newobj instance void [mscorlib]System.ArgumentException::.ctor()
IL_007c: throw
IL_007d: ret
} // end of method CGColor::.ctor
```
into
```
.method public hidebysig specialname rtspecialname
instance void .ctor (
class CoreGraphics.CGColorSpace colorspace,
class CoreGraphics.CGPattern pattern,
valuetype System.nfloat[] components
) cil managed
{
// Method begins at RVA 0x210660
// Code size 119 (0x77)
.maxstack 4
.locals init (
[0] native int handleof_pattern
)
IL_0000: ldarg.0
IL_0001: call instance void [mscorlib]System.Object::.ctor()
IL_0006: ldarg.1
IL_0007: brtrue.s IL_0014
IL_0009: ldstr "colorspace"
IL_000e: newobj instance void [mscorlib]System.ArgumentNullException::.ctor(string)
IL_0013: throw
IL_0014: ldarg.1
IL_0015: ldfld native int CoreGraphics.CGColorSpace::handle
IL_001a: ldsfld native int [mscorlib]System.IntPtr::Zero
IL_001f: call bool [mscorlib]System.IntPtr::op_Equality(native int, native int)
IL_0024: brfalse.s IL_0031
IL_0026: ldstr "colorspace"
IL_002b: newobj instance void [mscorlib]System.ObjectDisposedException::.ctor(string)
IL_0030: throw
IL_0031: ldarg.3
IL_0032: brtrue.s IL_003f
IL_0034: ldstr "components"
IL_0039: newobj instance void [mscorlib]System.ArgumentNullException::.ctor(string)
IL_003e: throw
IL_003f: ldarg.2
IL_0040: ldstr "pattern"
IL_0045: call native int ObjCRuntime.NativeObjectHelper::GetNonNullHandle(class ObjCRuntime.INativeObject, string)
IL_004a: stloc.0
IL_004b: ldarg.0
IL_004c: ldarg.1
IL_004d: ldfld native int CoreGraphics.CGColorSpace::handle
IL_0052: ldloc.0
IL_0053: ldarg.3
IL_0054: call native int CoreGraphics.CGColor::CGColorCreateWithPattern(native int, native int, valuetype System.nfloat[])
IL_0059: stfld native int CoreGraphics.CGColor::handle
IL_005e: ldarg.0
IL_005f: ldfld native int CoreGraphics.CGColor::handle
IL_0064: ldsfld native int [mscorlib]System.IntPtr::Zero
IL_0069: call bool [mscorlib]System.IntPtr::op_Equality(native int, native int)
IL_006e: brfalse.s IL_0076
IL_0070: newobj instance void [mscorlib]System.ArgumentException::.ctor()
IL_0075: throw
IL_0076: ret
} // end of method CGColor::.ctor
```
so the difference is smaller, but it's still smaller (7 bytes) than the
existing code (that did not check for a disposed instance).
There are many more optimizations that can be applied on top of this.
However the main/original goal is to have a correct and consistent
handling of disposed managed instances. IOW the size reduction is a nice
side-effect of correctness and actual optimizations can come later :)
Add `ObjectDisposedException` when required on Handle access.
Instead of providing `nil` to native API that are decorated **not**
accepting them.
Also do this using existing and new extension method helpers so the IL
size of the assemblies is (a bit) smaller than the existing code.
The latter has a small, positive impact for Xamarin.Mac (JIT'ing or
requiring IL for, non-full, AOT'ing) and for Xamarin.iOS when the
interpreter is used.
This test will run through all available strings in MSBStrings.Designer.cs in the each of the supported languages.
If any strings should fire in other languages but are still in english, an error will be thrown unless that error code is in an ignore file.
There are two types of ignore files introduced:
LocalizationIgnore/common-Translations.ignore
for error codes that are not yet translated in all languages
LocalizationIgnore/<LOCALE>-Translations.ignore
for error codes that are not yet translated in the specific locale
If an error code is in an ignore file and it turns out it is translated, an error will be thrown to remove the error code from the ignore file.
If there is a non-valid error code in any ignore file, an error will also be thrown.
Wiki will be updated for the team to reflect this test: https://github.com/xamarin/maccore/wiki/Localization
Make sure to redirect the dotnet build output to our custom output directory
when building the previous commit so that it doesn't interfere with the normal
build output.
Also add validation for this.
This is a pretty big refactoring, which:
* Always copies the test projects to a temporary directory before running any tests
that use them.
* Runs all the tests using an out-of-process MSBuild instance.
* Logs to a binlog instead of writing text to stdout.
* Refactors all the code that used the MSBuild assemblies in memory to instead:
* Tests that modified projects in memory now modifies them on disk instead. This
won't affect the working copy because the tests are always working with a copy
of the test projects.
* Tests that inspected projects in memory afterwards now parses the binlog to get
the same information.
* Significantly simplified the code to setup the test projects for testing.
* Continue using our own error handling logic, and print our problems to stderr.
* Also use the linker's messaging facilities to report a more generic error,
in case stderr doesn't show up for some reason.
Because just this is kind of useless:
error MT2301: The linker step 'Setup' failed during processing.
now it will say:
error MT2301: The linker step 'Setup' failed during processing: <hopefully something useful here>
Sometimes we wrap exceptions to add more information to what's happening, but
that may end up worse if we don't print out the wrapped exceptions.
At the same time we don't want to flood the user with information if they
didn't ask for it, so only show nested exceptions if they're something we
raised ourselves.
This is a step towards making Xamarin.MacDev.Tests run tests out-of-process.
It requires adding '_GenerateBundleName' as a dependency for the
'_ValidateAppBundle' target, because we're invoking the '_ValidateAppBundle'
directly, and we can't depend on any other targets executing
'_GenerateBundleName', because there are no other executed targets.
MyiOSFrameworkBinding is in our normal collection of test projects (in
tests/common/TestProjects), which means it doesn't need a lot of special
casing to make it work.
"Is.Not.Null.Or.Empty" is equal to "(value != null) || (value.Length == 0)", which
is the same as "value != null", i.e. "Is.Not.Null", which was clearly not the intention
of this code.
The fact is that the _AppManifest value is not set for library projects, so the correct
assert is to verify that the value is null or empty.
* Remove a lot of dead code.
* Use TestBase.CreateTask<T> to create tasks (and set required properties for all
tasks) instead of instantiating tasks directly. This required subclassing TestBase
in a few places, as well as making a few helper methods instance methods instead
of static methods.
* Bump to net472.
* A few other misc simplifications.
This works fine when executed from xharness, because
MSBuildEnableWorkloadResolver is set by xharness, but without this the dotnet
tests fail when executed from the IDE.
1. Use `xcrun` to run `ibtool` (and `actool`) and avoid toolchain mismatches
2. Set `DEVELOPER_DIR` so everyone (well `xcrun`) use the same toolchain
3. Workaround for `macos-arm64` issue (FB8827920)
* Start `ibtool` as an `Apple` [Silicon] process
* This will ensure the `ibtoold` daemon is also running as `Apple`
* If `ibtoold` is run as `Intel` then `ibtool` asserts and build fail
```
LaunchScreen.storyboard : error : 2020-10-26 14:24:52.757 ibtoold[37142:6681382] [MT] DVTAssertions: Warning in /Library/Caches/com.apple.xbs/Sources/IDEInterfaceBuilder/IDEInterfaceBuilder-17506/InterfaceBuilderKit/Utilities/IBAbstractInterfaceBuilderPlatformToolManager.m:481
LaunchScreen.storyboard : error : Details: Failed to attach to IBAgent-iOS with error: Error Domain=com.apple.InterfaceBuilder Code=-1 "Encountered an error communicating with IBAgent-iOS." UserInfo={NSLocalizedFailureReason=IBAgent-iOS (37146) failed to launch and exited with status 10, NSUnderlyingError=0x7fa4e58fc760 {Error Domain=com.apple.InterfaceBuilder Code=-1 "Failed to launch IBAgent-iOS via CoreSimulator spawn" UserInfo={NSLocalizedDescription=Failed to launch IBAgent-iOS via CoreSimulator spawn, NSUnderlyingError=0x7fa4e5e8a900 {Error Domain=com.apple.InterfaceBuilder Code=-1 "Failed to handshake with platform tool" UserInfo={NSLocalizedFailureReason=Failed to open connection over FIFOs with platform tool, NSLocalizedDescription=Failed to handshake with platform tool, NSUnderlyingError=0x7fa4e5e237c0 {Error Domain=com.apple.InterfaceBuilder Code=-1 "" UserInfo=0x7fa4e5e8bba0 (not displayed)}}}}}, NSLocalizedRecoverySuggestion=Please check Console.app for crash reports for "IBAgent-iOS" for further information., NSLocalizedDescription=Encountered an error communicating with IBAgent-iOS.}
LaunchScreen.storyboard : error : Object: <IBCocoaTouchToolManager>
LaunchScreen.storyboard : error : Method: +_THREADSAFE_launchNewToolWithLaunchContext:executionContext:toolProxyClass:proxyDelegate:failureContext:requestingMethod:error:forReason:
LaunchScreen.storyboard : error : Thread: <NSThread: 0x7fa4e341af70>{number = 1, name = main}
LaunchScreen.storyboard : error : Please file a bug at https://feedbackassistant.apple.com with this warning message and any useful information you can provide.
LaunchScreen.storyboard : error : 2020-10-26 14:24:52.766 ibtoold[37142:6681382] [MT] IBPlatformTool: *** Failed to launch tool with description <IBCocoaTouchPlatformToolDescription: 0x7fa4e5f34160> System content for IBCocoaTouchFramework-fourteenAndLater <IBScaleFactorDeviceTypeDescription: 0x7fa4e5f2fb50> scaleFactor=2x, renderMode.identifier=(null): Encountered an error communicating with IBAgent-iOS. (Failure reason: IBAgent-iOS (37146) failed to launch and exited with status 10): Failed to launch IBAgent-iOS via CoreSimulator spawn: Failed to handshake with platform tool (Failure reason: Failed to open connection over FIFOs with platform tool): : Failed to open FIFOs for handshaking with platform tool (Failure reason: IBAgent-iOS exited before we could handshake)
/Library/Frameworks/Mono.framework/External/xbuild/Xamarin/iOS/Xamarin.iOS.Common.targets(1425,3): error : ibtool exited with code 1
/Library/Frameworks/Mono.framework/External/xbuild/Xamarin/iOS/Xamarin.iOS.Common.targets(1425,3): error :
LaunchScreen.storyboard : ibtool error : Encountered an error communicating with IBAgent-iOS.
```
Note: `main` has diverged quite a bit (net6 support) so this pull request
will need to be re-worked (it won't apply)
Fixes https://github.com/xamarin/xamarin-macios/issues/4634
Foreport of #9965 since the source code diverged from `xcode12.2`
* Add an AOTCompile task that runs the AOT compiler for a given set of input assemblies.
This task will eventually be deprecated by an equivalent task provided by Mono,
but this works for now.
* Add an _AOTCompile target that takes care of running the AOT compiler and compile
the result into object files.
* [dotnet-linker] Catch any exceptions from our custom steps and show them using our error reporting logic.
* Letting the linker handle the exceptions will not result in a particularly
good experience, because the linker will crash.
* We can also show better information, since we have more knowledge about many
of the exceptions we raise ourselves.
* [dotnet] Make ConfigurationAwareSubStep inherit from ExceptionalSubStep.
This required a minor modification to ExceptionalSubStep to allow for custom reporting.
* [dotnet] Implement ConfigurationAwareStep's exception handling like it's done in ExceptionalSubStep.
Rolf Bjarne Kvinge
Manuel de la Pena
Sebastien Pouliot
TJ Lambert
Alex Soto