System.Net.NetworkInformation.Ping.Send doesn't work.
Ref: https://github.com/dotnet/runtime/issues/41355
Fixes this link sdk test failure:
LinkSdk.DllImportTest
[FAIL] LackOfCapget :
Expected: <System.InvalidOperationException>
But was: <System.Net.NetworkInformation.PingException: An exception occurred during a Ping request.
---> System.ComponentModel.Win32Exception (2): No such file or directory
at System.Diagnostics.Process.EnsureInitialized()
at System.Diagnostics.Process.StartCore(ProcessStartInfo startInfo)
at System.Diagnostics.Process.Start()
at System.Net.NetworkInformation.Ping.SendWithPingUtility(IPAddress address, Byte[] buffer, Int32 timeout, PingOptions options)
at System.Net.NetworkInformation.Ping.SendPingCore(IPAddress address, Byte[] buffer, Int32 timeout, PingOptions options)
at System.Net.NetworkInformation.Ping.GetAddressAndSend(String hostNameOrAddress, Int32 timeout, Byte[] buffer, PingOptions options)
--- End of inner exception stack trace ---
at System.Net.NetworkInformation.Ping.GetAddressAndSend(String hostNameOrAddress, Int32 timeout, Byte[] buffer, PingOptions options)
at System.Net.NetworkInformation.Ping.Send(String hostNameOrAddress, Int32 timeout, Byte[] buffer, PingOptions options)
at System.Net.NetworkInformation.Ping.Send(String hostNameOrAddress, Int32 timeout, Byte[] buffer)
at System.Net.NetworkInformation.Ping.Send(String hostNameOrAddress)
at LinkSdk.DllImportTest.<>c__DisplayClass3_0.<LackOfCapget>b__0() in /Users/rolf/work/maccore/whatever/xamarin-macios/tests/linker/ios/link sdk/DllImportTest.cs:line 88
at NUnit.Framework.Assert.Throws(IResolveConstraint expression, TestDelegate code, String message, Object[] args)>
at LinkSdk.DllImportTest.LackOfCapget() in /Users/rolf/work/maccore/whatever/xamarin-macios/tests/linker/ios/link sdk/DllImportTest.cs:line 88
Fixes unhandled exception that makes the app crash:
Unhandled Exception:
System.EntryPointNotFoundException: AppleCryptoNative_SecKeychainItemCopyKeychain assembly:<unknown assembly> type:<unknown type> member:(null)
at Interop.AppleCrypto.SecKeychainItemCopyKeychain(IntPtr item)
at System.Security.Cryptography.Apple.SafeTemporaryKeychainHandle.UntrackItem(IntPtr keychainItem)
at System.Security.Cryptography.Apple.SafeKeychainItemHandle.ReleaseHandle()
at System.Runtime.InteropServices.SafeHandle.InternalRelease(Boolean disposeOrFinalizeOperation)
at System.Runtime.InteropServices.SafeHandle.Dispose(Boolean disposing)
at System.Runtime.InteropServices.SafeHandle.Finalize()
Ref: https://github.com/xamarin/xamarin-macios/issues/8906
This is the third of three steps to fix this test failure:
EmbeddedResources.ResourcesTest
[FAIL] Embedded : en-AU
Expected string length 5 but was 7. Strings differ at index 0.
Expected: "G'day"
But was: "Welcome"
-----------^
at EmbeddedResources.ResourcesTest.Embedded() in [...]/xamarin-macios/tests/EmbeddedResources/ResourcesTest.cs:line 45
.NET doesn't have any globalization support yet.
Ref: https://github.com/xamarin/xamarin-macios/issues/8906
This fixes these tests:
DontLink.Calendars.CalendarTest
[FAIL] Hijri : Calendar
Expected string length 34 but was 38. Strings differ at index 21.
Expected: "System.Globalization.HijriCalendar"
But was: "System.Globalization.GregorianCalendar"
--------------------------------^
at DontLink.Calendars.CalendarTest.Hijri() in [...]/xamarin-macios/tests/linker/ios/dont link/CalendarTest.cs:line 28
[FAIL] ThaiBuddhist : Calendar
Expected string length 41 but was 38. Strings differ at index 21.
Expected: "System.Globalization.ThaiBuddhistCalendar"
But was: "System.Globalization.GregorianCalendar"
--------------------------------^
at DontLink.Calendars.CalendarTest.ThaiBuddhist() in [...]/xamarin-macios/tests/linker/ios/dont link/CalendarTest.cs:line 35
[FAIL] UmAlQura : Calendar
Expected string length 37 but was 38. Strings differ at index 21.
Expected: "System.Globalization.UmAlQuraCalendar"
But was: "System.Globalization.GregorianCalendar"
--------------------------------^
at DontLink.Calendars.CalendarTest.UmAlQura() in [...]/xamarin-macios/tests/linker/ios/dont link/CalendarTest.cs:line 21
Types have moved around in .NET, and in particular
System.ComponentModel.ReflectTypeDescriptionProvider is not in System where it
once was. So change the logic to look in the same assembly
System.ComponentModel.BooleanConverter is, as opposed to hardcoding 'System'.
This fixes this test:
DontLink.CommonDontLinkTest
[FAIL] TypeDescriptorCanary : type
Expected: not null
But was: null
at DontLink.CommonDontLinkTest.TypeDescriptorCanary() in [...]/xamarin-macios/tests/linker/CommonDontLinkTest.cs:line 19
MulticastDelegate.BeginInvoke isn't supported in .NET.
Ref: https://github.com/dotnet/runtime/issues/16312
Fixes this test failure:
DontLink.DontLinkRegressionTests
[FAIL] Bug5354 : System.PlatformNotSupportedException : Operation is not supported on this platform.
at DontLink.DontLinkRegressionTests.Bug5354() in [...]/xamarin-macios/tests/linker/ios/dont link/DontLinkRegressionTests.cs:line 64
at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
This regressed here: d7ab847697
Fixes this failure:
Xamarin.MTouch.MT0113_linker:
The error 'MT0113: Native code sharing has been disabled for the extension 'testServiceExtension' because the managed linker settings are different between the container app (None) and the extension (All).' was not found in the output:
Message #1 did not match:
actual: 'Native code sharing has been disabled for the extension 'testServiceExtension' because the managed linker settings are different between the container app (None) and the extension (Full).'
expected: 'Native code sharing has been disabled for the extension 'testServiceExtension' because the managed linker settings are different between the container app (None) and the extension (All).'
* Port the BundledResources test to .NET.
* Fix a minor issue in the BundledResources test to make sure it works on macOS.
* Add a unit test to make sure resources are bundled as expected.
* Modify the .NET build logic to bundle/unbundle resources.
A System.Drawing.Point test fails because it tests GetHashCode, which returns
a different value in .NET. None of the System.Drawing tests actually test any
of our code (when using .NET), so the fix is to just not include the
System.Drawing tests in the first place.
Fixes this test failure:
MonoTests.System.Drawing.PointTest
[FAIL] GetHashCodeTest : #1
Expected: 32
But was: 1527927283
at MonoTests.System.Drawing.PointTest.GetHashCodeTest() in /Users/rolf/work/maccore/squashed-onedotnet/xamarin-macios/builds/mono-ios-sdk-destdir/ios-sources/mcs/class/System.Drawing/Test/System.Drawing/TestPoint.cs:line 198
Fixes this test failure:
MonoTouchFixtures.ObjCRuntime.RuntimeTest
[FAIL] NSAutoreleasePoolInThreadPool : RC. Iterations: 101
Expected: not greater than 50
But was: 101
at MonoTouchFixtures.ObjCRuntime.RuntimeTest.NSAutoreleasePoolInThreadPool() in /Users/rolf/work/maccore/squashed-onedotnet/xamarin-macios/tests/monotouch-test/ObjCRuntime/RuntimeTest.cs:line 484
While keeping '$(RootTestsDirectory)' as-is works fine when building the
project using MSBuild, xharness itself may end up encountering it (in
particular when listing referenced projects), and at that point will end up
confused. So just always resolve '$(RootTestsDirectory)' to the actual
directory to keep other parts of xharness happy.
Fixes this problem (as seen in PR #9460):
Harness exception for 'interdependent-binding-projects': System.IO.DirectoryNotFoundException: Could not find a part of the path "/Users/builder/jenkins/workspace/xamarin-macios-pr-builder/tests/external/Touch.Unit/Touch.Client/dotnet/iOS/Touch.Client-iOS.dotnet.csproj".
at System.IO.FileStream..ctor (System.String path, System.IO.FileMode mode, System.IO.FileAccess access, System.IO.FileShare share, System.Int32 bufferSize, System.Boolean anonymous, System.IO.FileOptions options) [0x0015e] in /Users/builder/jenkins/workspace/build-package-osx-mono/2020-02/external/bockbuild/builds/mono-x64/mcs/class/corlib/System.IO/FileStream.cs:223
at System.IO.FileStream..ctor (System.String path, System.IO.FileMode mode, System.IO.FileAccess access, System.IO.FileShare share, System.Int32 bufferSize, System.Boolean isAsync, System.Boolean anonymous) [0x00000] in /Users/builder/jenkins/workspace/build-package-osx-mono/2020-02/external/bockbuild/builds/mono-x64/mcs/class/corlib/System.IO/FileStream.cs:149
at System.IO.FileStream..ctor (System.String path, System.IO.FileMode mode, System.IO.FileAccess access) [0x00000] in /Users/builder/jenkins/workspace/build-package-osx-mono/2020-02/external/bockbuild/builds/mono-x64/mcs/class/corlib/System.IO/FileStream.cs:86
at (wrapper remoting-invoke-with-check) System.IO.FileStream..ctor(string,System.IO.FileMode,System.IO.FileAccess)
at Microsoft.DotNet.XHarness.iOS.Shared.Utilities.PListExtensions.LoadWithoutNetworkAccess (System.Xml.XmlDocument doc, System.String filename) [0x00001] in /Users/builder/jenkins/workspace/xamarin-macios-pr-builder/tests/xharness/Microsoft.DotNet.XHarness.iOS.Shared/Utilities/PlistExtensions.cs:17
at Microsoft.DotNet.XHarness.iOS.Shared.TestProject.CreateCopyAsync (Microsoft.DotNet.XHarness.iOS.Shared.Logging.ILog log, Microsoft.DotNet.XHarness.iOS.Shared.Execution.IProcessManager processManager, Microsoft.DotNet.XHarness.iOS.Shared.Tasks.ITestTask test, System.String rootDirectory, System.Collections.Generic.Dictionary`2[TKey,TValue] allProjectReferences) [0x0010c] in /Users/builder/jenkins/workspace/xamarin-macios-pr-builder/tests/xharness/Microsoft.DotNet.XHarness.iOS.Shared/TestProject.cs:98
at Microsoft.DotNet.XHarness.iOS.Shared.TestProject.CreateCopyAsync (Microsoft.DotNet.XHarness.iOS.Shared.Logging.ILog log, Microsoft.DotNet.XHarness.iOS.Shared.Execution.IProcessManager processManager, Microsoft.DotNet.XHarness.iOS.Shared.Tasks.ITestTask test, System.String rootDirectory, System.Collections.Generic.Dictionary`2[TKey,TValue] allProjectReferences) [0x00811] in /Users/builder/jenkins/workspace/xamarin-macios-pr-builder/tests/xharness/Microsoft.DotNet.XHarness.iOS.Shared/TestProject.cs:168
at Microsoft.DotNet.XHarness.iOS.Shared.Tasks.TestTasks.RunInternalAsync () [0x00117] in /Users/builder/jenkins/workspace/xamarin-macios-pr-builder/tests/xharness/Microsoft.DotNet.XHarness.iOS.Shared/Tasks/TestTask.cs:274
* Create a .NET version of the monotouch-test project.
* Make sure it builds and launches (there are test failures, and it eventually
crashes, which will be fixed in a later PR).
* Add it to xharness, in a way that it's possible to run monotouch-test
locally, while at the same time it'll never run on the bots (because it's
not green yet).
* Bump .NET 5 and the linker.
The old linker run into a problem that seems fixed in the new linker, so bump.
* [tests] Add a workaround for a .NET bug in the fsharplibrary tests.
It's now the default to produce reference assemblies, but the F# compiler
doesn't support producing reference assemblies, so when there's no reference
assembly afterwards the MSBuild tasks complain. So explicitly disable
reference assemblies for our F# library.
* [tests] Skip reference assemblies when iterating over produced assemblies in the .NET unit tests.
Also simplify the code a little bit.
This happens because producing reference assemblies is now the default.
* Properties may contain a list of files, separated by semi-colon, so if we find
any semi-colons, treat each entry as a separate path.
* Treat anything that starts with a variable as a full path, because either the
variable is a full path, or it will be updated according to the new project location
and resolve correctly.
Make it possible to make a project ignored by default from its initial definition,
and which takes precedence over any other selection criteria.
This way it's possible to make the .NET version of monotouch-test show up in the
web view, which makes it runnable locally (for testing), while at the same time it
will never be run on CI.
Put NuGet packages in tests/dotnet, instead of a directory next to the project
file, so that default item inclusions / globs don't pick up anything from
them.
This fixes an issue where the build would pick up a Program.cs that ships with NUnitLite,
and include it in the build, which would then cause build failures because that Program.cs
has a Main method, which would conflict with our Main method.
* partial fix for https://github.com/xamarin/xamarin-macios/issues/9267
* simplify notequals
* add monotouch-test case
* add more using per manuel's feedback
Co-authored-by: Whitney Schmidt <whschm@microsoft.com>
* Port the interdependent-binding-projects test to .NET (it's the simplest
test project we have with binding projects).
* Add a lot of the shared source code for mtouch/mmp to dotnet-linker, and
make it compile. Most issues were fixed by adding a few stubbed out classes,
since there are large chunks of the mtouch/mmp code we're not using yet, so
stubbing out while things are being implemented works fine.
* Add a step in dotnet-linker for loading the linker output (the linked
assemblies) into our bundler code.
* Add another step in dotnet-linker to extract native resources from binding
libraries.
* Augment the build process to take into account the native resources we found
in any binding libraries.
Otherwise there could be a problem when installing a test app into the
simulator if there's already an existing app there with the same bundle
identifer, because that doesn't remove existing files, and if there are
existing files in the bundle, the app might end up crashing (in particular
there may be files in a .monotouch-32/64 subdirectory which are not removed,
and those take precedence when looking for assemblies, eventually causing a
conflict).
Problem:
1. interdependent-binding-project references binding-test and bindings-test2.
2. bindings-test2 references bindings-test.
This means bindings-test is reached through 2 projects: both
interdependent-binding-projects and bindings-test2, and previously we'd
process each of these references separately, effectively making two separate
clones of the bindings-test project.
Instead keep track of all referenced projects from the leaf project, and if we
encounter a project we've already cloned, use that directly.
xharness needs to inspect project files, but can't do so through imported
projects, so move some of the logic from the imported shared.csproj to the
main csproj.
We have noticed the following message from Apple when performing
submissions with Xamarin.iOS:
> ITMS-90338: Non-public API usage - The app references non-public
> selectors in WcBc.iOS: behaviorTypes, convolutionState,
> discoverAllContactUserInfosWithCompletionHandler:,
> discoverAllContactsCompletionBlock,
> discoverUserInfoWithEmailAddress:completionHandler:,
> discoverUserInfoWithUserRecordID:completionHandler:,
> discoverUserInfosCompletionBlock, displayContact, drawableResizesAsynchronously,
> encodeToCommandBuffer:sourceImage:convolutionState:,
> encodeToCommandBuffer:sourceImage:destinationImage:state:,
> getProperty:onChannel:responseHandler:, hasProperty:onChannel:responseHandler:,
> initWithEmailAddresses:userRecordIDs:, initWithMIDIEntity:dataReadyHandler:,
> initWithZoneID:options:, initWithZoneID:subscriptionID:options:,
> isPublicDatabase, mouseUpAction, newDrawable, propertyChangedCallback,
> removeAllAppearanceStreams, replaceTextStorage:, retrieveConnectedPeripherals,
> retrievePeripherals:, setDiscoverAllContactsCompletionBlock:,
> setDiscoverUserInfosCompletionBlock:, setDrawableResizesAsynchronously:,
> setEditedMask:, setMouseUpAction:, setMovieControlMode:,
> setProperty:onChannel:responseHandler:, setPropertyChangedCallback:,
> setSocketFamily:, setTemporaryAttributes:forCharacterRange:, setUserRecordIDs:,
> sourceOffset, subscriptionOptions, takeBackgroundColorFrom:, takePasswordFrom:,
> temporalAntialiasingEnabled, userRecordIDs. If method names in your source code
> match the private Apple APIs listed above, altering your method names will help
> prevent this app from being flagged in future submissions. In addition, note
> that one or more of the above APIs may be located in a static library that was
> included with your app. If so, they must be removed. For further information,
> visit the Technical Support Information at http://developer.apple.com/support/technical/
All of them have been removed but without a break in the API excep
"initWithMIDIEntity:dataReadyHandler:" wich does look like an error on
Apples side.
Empty stubs are used as much as possible except on those cases in which
a handler is called or an output variable should be modified (buffer,
out param) to minimize the users surprise at runtime.
This requires passing the root directory around in multiple places, since ResolveAllPaths
doesn't have access to the static class where we define the root directory.
Also call ResolveAllPaths in a few more places to ensure paths everywhere are resolved.
In .NET projects there's a default value for most properties, which means that
there won't necessarily be an AssemblyName property in a csproj. We need to know the
AssemblyName, so calculate it from the csproj filename (which is how .NET does it).
This turned out slighly complicated, because we're pass an XmlDocument around,
and the XmlDocument doesn't know the file from where it was loaded, so we need
to keep that information separately.
This works around a build problem that occurs because NUnit ships with a
P/Invoke to a function that doesn't exist on Apple platforms:
MTOUCH : error MT5210: Native linking failed, undefined symbol: _GetVersionEx. Please verify that all the necessary frameworks have been referenced and native libraries are properly linked in. [/Users/xamarinqa/myagent/_work/8/s/xamarin-macios/tests/xharness/tmp-test-dir/monotouch-test58/monotouch-test-tvos.csproj]
MTOUCH : error MT5201: Native linking failed. Please review the build log and the user flags provided to gcc: -fembed-bitcode-marker [/Users/xamarinqa/myagent/_work/8/s/xamarin-macios/tests/xharness/tmp-test-dir/monotouch-test58/monotouch-test-tvos.csproj]
clang : error : linker command failed with exit code 1 (use -v to see invocation) [/Users/xamarinqa/myagent/_work/8/s/xamarin-macios/tests/xharness/tmp-test-dir/monotouch-test58/monotouch-test-tvos.csproj]
Also fix an issue in mtouch where we would overwrite any previous --dlsym
values; they're now accumulative (`--dlsym:foo.dll --dlsym:bar.dll` works
as expected)
Ref: https://github.com/nunit/nunit/issues/3618
Use two separate output variables (EntitlementsInExecutable/EntitlementsInSignature)
instead of using the same output variable for two different purposes. This makes
the code more self-explanatory.
Also move the simulator check to the C# code, that way it's easier to re-use elsewhere.
* [msbuild] Share the _CompileEntitlements task.
* [msbuild] Create the same class hierarchy for Xamarin.Mac and Xamarin.iOS for the CompileEntitlements task.
CompileEntitlementsTaskBase
└─── iOS/CompileEntitlementsTaskCore
│ └─── iOS/CompileEntitlements
└─── Mac/CompileEntitlementsTaskCore
└─── Mac/CompileEntitlements
This also means we can remove a known failure in the list of MSBuild tasks that don't
conform to our 'no code in final task implementation' requirements.
This fixes numerous P/Invoke test failures like this:
Could not find the symbol 'GlobalizationNative_GetCalendars' in QCall
Could not find the symbol 'GlobalizationNative_GetCalendarInfo' in QCall
Could not find the symbol 'GlobalizationNative_EnumCalendarInfo' in QCall
Could not find the symbol 'GlobalizationNative_GetLatestJapaneseEra' in QCall
[...]
Ref: a56d6a8034
Ref: fae477f34b
This also means that setting 'run-all-tests' enables the .NET tests, which
means they'll now start running on internal Jenkins (as was the intention from
the beginning).
With Mono, Environment.OSVersion returns the macOS version when running in the
simulator. On Catalina, Mono returns 19.* as the Environment.OSVersion, which
means that the check for major version = 15 in the simulator turns out to be a
constant value now (since we don't support running in the simulator on a
4-year-old macOS version).
With .NET, Environment.OSVersion returns the iOS version instead, which means
that the check would have to be reworked.
However, since these version checks have effectively been constant for a few
years now, we can just remove the checks and the resulting unreachable code.
* [Tests] Fix TryGetSupportedInterfaces when we have values. Fixes#2273
Make the test check against the value returned by the native API rather
than expect the result to always be null.
Fixes https://github.com/xamarin/maccore/issues/2273
* Address reviews.
* Touch.Client is a tiny mobile UI/runner for the official NUnitLite. This
means we'll stop using a very old fork of NUnitLite to run our tests, and
instead use the most recent upstream version instead.
* Some test changes were required to use the official NUnitLite, because there
have been breaking changes there.
* NUnitLite is now referenced using a NuGet, which means there are numerous
changes to support this.
* We're not using GuiUnit anymore, so that dependency has been removed.
* Reduces code duplication.
* Makes the macOS versions thread-safe (the iOS versions have been thread-safe
for years: 2c6a5303a7).
* A few parameters names were different in the definitions; I chose to keep the ones in Xamarin.iOS, since they looked better.
* Xamarin.Mac had two methods, SetTextBlocks and SetTextLists, in place of an actual property override (for the mutable setter), this was fixed to be an actual property overload, and compat methods were implemented.
* xtro needed an update to cope with multiple static methods for the same selector.
This solves a rebuild problem if an assembly has an invalid or unsupported symbol
file, where we'd detect that the symbol file exists, and expect it to be copied,
but then the linker would drop it, causing us to always rebuild the app (this is
not the same as when a symbol file is out of date).
This happens for NUnitLite 3.12.0's nunit.framework.dll, which ships with an old-style
pdb.
Also add a warning that is shown when we detect that there's a symbol file, but it
couldn't be loaded for some reason.
We weren't properly reporting all failures when there were multiple failures
in parameterized tests, because we'd incorrectly skip too many xml nodes when
we were looking for the next test failure.
I would have liked to automatically get all the references required by bindings-test.dll,
but that turned out to be not as simple as I first thought (since bindings-test is
a library project, the exact dependencies actually depend on the type of the executable
project that references it, which means it's quite difficult to calculate the final
references given just the binding project).
Instead just hardcode the fact that there's a reference to nunit.framework.dll, and
the corresponding on-disk path (we still look in the actual csproj to get the version
of nunit.framework.dll).
In particular NUnit uses reflection to get a private method, and the linker removes
the corresponding private method:
1c680b4dc8/src/NUnitFramework/framework/Internal/TestExecutionContext.cs (L552)
So add an xml definition to keep this private method, and modify project files to
pass the xml definition to mtouch and mmp.
Some care needs to be taken to make sure xharness is still able to clone these project
files.
These tests now end up testing the NUnitLite and MonoTouch.Dialog assemblies from
NuGet, not the ones we ship, which means they're out of our control to fix.
Rolf Bjarne Kvinge
Whitney Schmidt
monojenkins
Přemek Vysoký
Manuel de la Pena