Looks like we have issues with the internet sharing in the VSTS bots,
this means that now that we always try to parse the XML on CI, we get an
exception, catch it and do not show the results.
The workaround simply tries to read the xml, if possible, we will parse
it, else deal with the text only log. The fix will show the results, but
it is a workaround for a configuration issue in the CI.
Unify the harness properties to just look at InCI and remove all the
other ones. There is no real need to have differences between jenkins
and VSTS and Wrench is gone.
We used to test only on Jenkins, and if the build version had jenkins on
it, rather than doing so, just check if we are in the CI by looking if
BUILD_REVISION is present in the env.
Use the xml parsing helper methods to decide if the xml can be parsed
and if we will be able to generate the report. That way we avoid an
exception that makes the CI noise.
Co-authored-by: Manuel de la Pena <mandel@microsoft.com>
Use the xml parsing helper methods to decide if the xml can be parsed
and if we will be able to generate the report. That way we avoid an
exception that makes the CI noise.
Use the xml parsing helper methods to decide if the xml can be parsed
and if we will be able to generate the report. That way we avoid an
exception that makes the CI noise.
We ping the tcp listener to know that we have a tcp connection, that is
written in the xml logs, which means that parsing will not work. Ignore
the ping, parse xml, and make sure that the xml that will be consume by
vsts is valid.
This PR also fixes https://github.com/xamarin/maccore/issues/827 which does not longer happen.
Looks like we have issues with the internet sharing in the VSTS bots,
this means that now that we always try to parse the XML on CI, we get an
exception, catch it and do not show the results.
The workaround simply tries to read the xml, if possible, we will parse
it, else deal with the text only log. The fix will show the results, but
it is a workaround for a configuration issue in the CI.
Unify the harness properties to just look at InCI and remove all the
other ones. There is no real need to have differences between jenkins
and VSTS and Wrench is gone.
We used to test only on Jenkins, and if the build version had jenkins on
it, rather than doing so, just check if we are in the CI by looking if
BUILD_REVISION is present in the env.
We ping the tcp listener to know that we have a tcp connection, that is
written in the xml logs, which means that parsing will not work. Ignore
the ping, parse xml, and make sure that the xml that will be consume by
vsts is valid.
This PR also fixes https://github.com/xamarin/maccore/issues/827 which does not longer happen.
* Acquire permission for the microphone (the microphone permission dialog is
not a blocking dialog, which is why things have worked so far).
* Set the last_modified field in the TCC database to now instead of 1970.
Apparently some permissions time out (kTCCServiceMediaLibrary), which means
that the last_modified field is important to get right.
Looks like we have issues with the internet sharing in the VSTS bots,
this means that now that we always try to parse the XML on CI, we get an
exception, catch it and do not show the results.
The workaround simply tries to read the xml, if possible, we will parse
it, else deal with the text only log. The fix will show the results, but
it is a workaround for a configuration issue in the CI.
Unify the harness properties to just look at InCI and remove all the
other ones. There is no real need to have differences between jenkins
and VSTS and Wrench is gone.
We used to test only on Jenkins, and if the build version had jenkins on
it, rather than doing so, just check if we are in the CI by looking if
BUILD_REVISION is present in the env.
* Bump Xamarin.MacDev.
New commits in xamarin/Xamarin.MacDev:
* xamarin/Xamarin.MacDev@210c664 Adds net451 to Xamarin.MacDev.csproj
* xamarin/Xamarin.MacDev@64db365 [winios] Changes provisioning profiles default path
* xamarin/Xamarin.MacDev@d34430a Switch to short-form projects and build for both net461 and netstandard2.0. (#68)
Diff: 0f578f51e6..210c664e56
* [msbuild] Update to latest Mono.Cecil.
The older version doesn't support netstandard2.0.
No code changes were required.
* [msbuild] Remove unused usings.
* [msbuild] Make ILMerge work when building for netstandard2.0.
Also unify/deduplicate the ILMerge logic between Xamarin.iOS and Xamarin.Mac.
* [msbuild] Build for netstandard2.0 in addition to net461.
* [msbuild] Use custom project configurations to support running the tests for both netstandard2.0 and net461.
Use custom project configurations to support running the tests for when the
tasks assembly is built for netstandard2.0 and net461.
* [tests] Make command-line based 'make test-ios-tasks' run tests for both netstandard2.0 and net461.
* [xharness] Add test configuration to run iOS MSBuild tests using either netstandard2.0 or net461.
* [msbuild] Make the netstandard2.0-buils task assemblies the default.
* [msbuild] ILRepack lib assemblies, not ref assemblies.
Ask MSBuild to copy lib assemblies to the output folder when building for
netstandard2.0, this way we can easily find the actual implementation
libraries to pass to ILRepack.
* [msbuild] Merge System.Text.Encodings.Web.dll as well.
* [xharness] Fix build of MSBuild tests for iOS.
* [xharness] Fix two compiler warnings.
Fixes these warnings:
Jenkins.cs(538,37): warning CS1998: This async method lacks 'await' operators and will run synchronously. Consider using the 'await' operator to await non-blocking API calls, or 'await Task.Run(...)' to do CPU-bound work on a background thread.
Jenkins.cs(877,14): warning CS1998: This async method lacks 'await' operators and will run synchronously. Consider using the 'await' operator to await non-blocking API calls, or 'await Task.Run(...)' to do CPU-bound work on a background thread.
* [xharness] Simplify async code a bit.
* [msbuild] Convert to short-form csproj.
* [msbuild] Make asserts more useful.
* [msbuild] Make tests ignore the actual location of the test assembly.
* [msbuild] Short-style projects default to deterministic builds, which is not compatible with wildcard versions.
* [msbuild] Adjust test.
* Update .gitignore.
* Bump NUnit.ConsoleRunner version.
* [msbuild] Fix indentation.
* [msbuild] Simplify csproj.
The 32bits **debug** binaries are now too big for Apple's native linker
to process, which gives us (non useful) build errors on the bots.
This will still run the release builds configuration of the tests since
they are smaller and still within the limits of the tooling.
Co-authored-by: Sebastien Pouliot <sebastien.pouliot@gmail.com>
The 32bits **debug** binaries are now too big for Apple's native linker
to process, which gives us (non useful) build errors on the bots.
This will still run the release builds configuration of the tests since
they are smaller and still within the limits of the tooling.
Bump mono to get the new splited test dlls and add them to be ran in
xharness. Special logic is used for mscorlib so we make sure that all
the 'parts' of the test dll do have the same configurations.
Co-authored-by: Manuel de la Pena <mandel@microsoft.com>
Co-authored-by: Waleed Chaudhry <54864665+wachaudh@users.noreply.github.com>
Bump mono to get the new splited test dlls and add them to be ran in
xharness. Special logic is used for mscorlib so we make sure that all
the 'parts' of the test dll do have the same configurations.
Co-Authored-By: Waleed Chaudhry <54864665+wachaudh@users.noreply.github.com>
**Problem**
32 bit tests could only be executed with 64 bit tests. E.g: `run-ios-32-tests` didn't work on its own, only `run-ios-tests,run-ios-32-tests` worked
**Solution**
- Change the logic so that 32 bit tests are only looking for `IncludeiOS32` and 64 bit tests only `IncludeiOS64`
- To keep `run-ios-tests` and `skip-ios-tests` working, hack `SetEnabled` so `IncludeiOS` also sets `IncludeiOS32` and `IncludeiOS64`.
Otherwise the way the code is currently organised we'd have to use `skip-ios-32-tests` and `skip-ios-64-tests`.
Co-authored-by: Vincent Dondain <vidondai@microsoft.com>
After this commit we will have more application but they will be the
appropiate size so that they can be built with the linker for iOS 32b.
It is important to note that the following apps WILL CONTINUE to fail
since the dlls need to be splitted in mono:
* mscorlib tests
* Mono BCL tests group 5 - Which is monotouch_System.Core_xunit-test.dll
and is too large.
After this commit we will have more application but they will be the
appropiate size so that they can be built with the linker for iOS 32b.
It is important to note that the following apps WILL CONTINUE to fail
since the dlls need to be splitted in mono:
* mscorlib tests
* Mono BCL tests group 5 - Which is monotouch_System.Core_xunit-test.dll
and is too large.
After this commit we will have more application but they will be the
appropiate size so that they can be built with the linker for iOS 32b.
It is important to note that the following apps WILL CONTINUE to fail
since the dlls need to be splitted in mono:
* mscorlib tests
* Mono BCL tests group 5 - Which is monotouch_System.Core_xunit-test.dll
and is too large.
Updating the msbuild tasks to use netstandard2.0 requires us to bump NUnit to 3+.
This requires:
* A few code changes due to breaking API changes in NUnit.
* Changes in xharness and a makefile to cope with the new location for the
NUnit console runner (I added a helper script to make things slightly
easier).
**Problem**
32 bit tests could only be executed with 64 bit tests. E.g: `run-ios-32-tests` didn't work on its own, only `run-ios-tests,run-ios-32-tests` worked
**Solution**
- Change the logic so that 32 bit tests are only looking for `IncludeiOS32` and 64 bit tests only `IncludeiOS64`
- To keep `run-ios-tests` and `skip-ios-tests` working, hack `SetEnabled` so `IncludeiOS` also sets `IncludeiOS32` and `IncludeiOS64`.
Otherwise the way the code is currently organised we'd have to use `skip-ios-32-tests` and `skip-ios-64-tests`.
The latest SDK version and the latest OS version does not necessarily have to
match (for instance the iOS 13.2 SDK can support both iOS 13.2 and iOS 13.3),
so keep track of them separately.
Also use the latest OS version to determine which simulator to run, instead of
the latest SDK version (Xcode 11.3 ships with the iOS 13.2 SDK but only has an
iOS 13.3 simulator, not an iOS 13.2 simulator).
Fixes https://github.com/xamarin/maccore/issues/2066.
* Bump mono to a hash with archives and use them.
New commits in mono/mono:
* mono/mono@6af4ae7635 [2019-06][ci] Add Xcode 11.2beta2 for XI/XM Mono SDK builds
Diff: 476d72b9e3..6af4ae7635
* Bump mono to get min iOS version fix.
New commits in mono/mono:
* mono/mono@3775d5ac0a [sdks] Bump min iOS version to 7.0.
Diff: 6af4ae7635..3775d5ac0a
* [xharness] Bump mtouch tests timeout to 3h, we have a couple of new PR bots which are old and slow.
The new PR bots are late 2012 mac minis, so quite slow.
* Implement a different escaping/quoting algorithm for arguments to System.Diagnostics.Process.
mono changed how quotes should be escaped when passed to
System.Diagnostic.Process, so we need to change accordingly.
The main difference is that single quotes don't have to be escaped anymore.
This solves problems like this:
System.ComponentModel.Win32Exception : ApplicationName='nuget', CommandLine='restore '/Users/vsts/agent/2.158.0/work/1/s/tests/sampletester/bin/Debug/repositories/ios-samples/WorkingWithTables/Part 3 - Customizing a Table\'s appearance/3 - CellCustomTable/CellCustomTable.sln' -Verbosity detailed -SolutionDir '/Users/vsts/agent/2.158.0/work/1/s/tests/sampletester/bin/Debug/repositories/ios-samples/WorkingWithTables/Part 3 - Customizing a Table\'s appearance/3 - CellCustomTable'', CurrentDirectory='/Users/vsts/agent/2.158.0/work/1/s/tests/sampletester/bin/Debug/repositories', Native error= Cannot find the specified file
at System.Diagnostics.Process.StartWithCreateProcess (System.Diagnostics.ProcessStartInfo startInfo) [0x0029f] in /Users/builder/jenkins/workspace/build-package-osx-mono/2019-08/external/bockbuild/builds/mono-x64/mcs/class/System/System.Diagnostics/Process.cs:778
ref: https://github.com/mono/mono/pull/15047
* Rework process arguments to pass arrays/lists around instead of quoted strings.
And then only convert to a string at the very end when we create the Process
instance.
In the future there will be a ProcessStartInfo.ArgumentList property we can
use to give the original array/list of arguments directly to the BCL so that
we can avoid quoting at all. These changes gets us almost all the way there
already (except that the ArgumentList property isn't available quite yet).
We also have to bump to target framework version v4.7.2 from v4.5 in several
places because of 'Array.Empty<T> ()' which is now used in more places.
* Parse linker flags from LinkWith attributes.
* [sampletester] Bump to v4.7.2 for Array.Empty<T> ().
* Fix typo.
* Rename GetVerbosity -> AddVerbosity.
* Remove unnecessary string interpolation.
* Remove unused variable.
* [mtouch] Simplify code a bit.
* Use implicitly typed arrays.
This has a couple of advantages:
* It makes it easier to add a catalyst version of these libraries (because it
becomes cumbersome to build for catalyst when the build rules assumes we're
building for both simulator and device).
* It makes it easier to create an xcframework of our libraries, because the
contents in an xcframework is split like this.
* macOS 10.15 starts putting up permission dialogs we can't automatically
dismiss anymore, so start honoring the 'IncludeSystemPermissionTests' option
for macOS tests.
* Improve the 'IncludeSystemPermissionTests' option to have three states: if
set (either true or false), that takes precedence, but if not set, we now
don't run any tests that require permission dialogs on macOS or on device if
we're running in CI. Tests executed locally will still put up dialogs, both
on macOS and on device.
* This needed a few changes to the html report, since the
'IncludeSystemPermissionTests' is exposed in the UI and the code didn't
handle the three different states.
* Update a few tests to check for permission to the contacts.
Fixes https://github.com/xamarin/maccore/issues/1856.
We have tests whose behavior changes when executed on CI, and those tests use
the BUILD_REVISION variable to detect where they're being executed. For this
to work we need to propagate the BUILD_REVISION variable to the test
executable.
Hopefully fixes https://github.com/xamarin/maccore/issues/649 now.
Trying to find a simulator will mark the test as a failure if the simulator
couldn't be found, and we don't want that to happen to ignored tests.
This should fix an issue where xharness seems to try to run the 32-bit
simulator tests when asked to run only device tests.
We have tests whose behavior changes when executed on CI, and those tests use
the BUILD_REVISION variable to detect where they're being executed. For this
to work we need to propagate the BUILD_REVISION variable to the test
executable.
Hopefully fixes https://github.com/xamarin/maccore/issues/649 now.
This boils down to the makefile-generation code having the information it
needs (and that information being correct).
This fixes running of tests on other macOS bots (older/newer), because now we
can build the test package again.
Currently we execute most of the same logic both during the configure phase
and when running tests, and the Harness.Mac value is only set in the configure
phase.
While it doesn't matter right now, this makes sure there aren't any future
surprises in this area, since otherwise we could end up with different
behavior between the configure phase and when running tests.
Harness.AutoConfigureMac now loads all the mac test projects both when
configuring and running tests, the only difference is that the test projects
that must be generated are only generated when configuring. This means that
the Harness.MacTestProject list contains the exact same test projects both
when configuring and when running tests.
This made it possible to remove logic to clone (mac) test projects the Jenkins
class (since Harness.MacTestProjects contains all the test projects already).
Consolidate logic to generate (mac) test projects:
* First we generate BCL and mono-native projects from their templates.
* Then we generate Full/System variations of any project that needs it.
This way we can remove logic to generate Full/System variations from the logic
to generate BCL/mono-native projects, which means less duplicated (and less
confusing) code.
To this purpose, significant changes were required:
* MacTestProject.TargetFrameworkFlavor has been modified to contain a bit mask
of the variations to generate.
* MacMonoNativeInfo has been significantly simplified, and some of the
generated code has been moved to the actual template instead.
* Some project generation (in MacTarget) to make things work as expected.
Fixes https://github.com/xamarin/xamarin-macios/issues/6322.
* Share code with GetInfoPListInclude to find the same Info.plist nodes, so
that FixInfoPListNode also finds existing nodes whose names isn't
"Info.plist".
* Add support for specifying the new Info.plist name.
Rename TestPlatform fields to be more in line with the rest of the code, and
drop the Unified prefix, since everything is Unified now.
* Unified -> Modern
* UnifiedXM45 -> Full
* UnifiedSystem -> System
* Bump VSMac to 8.1.0.2742 to fix msbuild issues (#6279)
* Bump VSMac to 8.1.0.2742 to fix msbuild issues
This is required to get the support for the msbuild `ToolsVersion`
change from `15.0` to `Current`.
* [tests][msbuild] Fix Binding resources test with updated msbuild
Test failure with updated msbuild and vsmac 8.1:
```
Xamarin.iOS.Tasks.NativeReferencesNoEmbedding("iPhone").ShouldNotUnnecessarilyRebuildBindingProject(True)
Binding project build did not create package?
Expected: True
But was: False
at Xamarin.iOS.Tasks.NativeReferencesNoEmbedding.ShouldNotUnnecessarilyRebuildBindingProject (System.Boolean framework) [0x000a0] in <74b8f7d8a53e40109916d305bb4d7403>:0
at (wrapper managed-to-native) System.Reflection.RuntimeMethodInfo.InternalInvoke(System.Reflection.RuntimeMethodInfo,object,object[],System.Exception&)
at System.Reflection.RuntimeMethodInfo.Invoke (System.Object obj, System.Reflection.BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo cul
ture) [0x0006a] in <0519fa732e8845b6a809ce9180f541db>:0
```
The test builds the project multiple times. Before the 3rd build, the project
file's timestamp is updated and expects that the binding package will be
rebuilt. But it is not, because the target `_CreateBindingResourcePackage`
doesn't depend on that project file. So, add that to the target inputs.
* [nuget] Use xibuild to run nuget
Fix errors seen during `nuget restore` for tests:
```
Users/builder/jenkins/workspace/xamarin-macios-pr-builder/tests/xammac_tests/xammac_tests.csproj(213,3): error MSB4024: The imported project file "/Library/Frameworks/Mono.framework/External/xbuild/Xamarin/Mac/Xamarin.Mac.CSharp.targets" could not be loaded. Could not find file "/Library/Frameworks/Mono.framework/External/xbuild/Xamarin/Mac/Xamarin.Mac.CSharp.targets"
```
* [xibuild] Fix incorrect mscorlib.dll being used (#6068)
* [xibuild] Fix incorrect mscorlib.dll being used
The `GuiUnit_NET_4_5` project, when built with `xibuild` uses the wrong `mscorlib.dll`.
From https://github.com/xamarin/xamarin-macios/issues/5760#issuecomment-472457202 :
```
- mscorlib.dll is being used from mono/4.5 and the other system assemblies are from mono/4.5-api
- GuiNet* project is built with xibuild
What is happening here is:
xibuild sets[1] `SetToolsetProperty ("TargetFrameworkRootPath", FrameworksDirectory + Path.DirectorySeparatorChar);`
which points to `/Library/Frameworks/Mono.framework/Versions/Current/lib/mono/xbuild-frameworks`.
This causes $(FrameworkPathOverride) to be set[2] to `/Library/Frameworks/Mono.framework/Versions/Current/lib/mono/xbuild-frameworks/.NETFramework/v4.5`,
but that doesn't have a mscorlib.dll, so it gets reset[3] to /Library/Frameworks/Mono.framework/Versions/5.22.0/lib/mono/4.5/.
If we don't set TargetFrameworkRoothPath, then we get `FrameworkPathOverride = /Library/Frameworks/Mono.framework/Versions/5.22.0/lib/mono/4.5-api`,
causing `_ExplicitReference=/Library/Frameworks/Mono.framework/Versions/5.22.0/lib/mono/4.5-api/mscorlib.dll`(correct one) to be used.
```
Fixes https://github.com/xamarin/xamarin-macios/issues/5760
1. https://github.com/xamarin/xamarin-macios/blob/master/tools/xibuild/Main.cs#L209
2. https://github.com/mono/msbuild/blob/xplat-master/src/Tasks/Microsoft.Common.CurrentVersion.targets#L79
3. https://github.com/mono/msbuild/blob/xplat-master/src/Tasks/Microsoft.Common.CurrentVersion.targets#L84
* Revert "Workaround https://github.com/xamarin/xamarin-macios/issues/5760 in generator csproj"
This reverts commit 9bd927bb7f.
The previous commit for xibuild removes the need for this.
* [xibuild] Handle "incorrectly" cased msbuild property names (#6202)
msbuild property names are case insensitive. While generating the custom
app.config, in `SetToolsetProperty(..)` we try to update the property if
it already exists. But the name lookup was case sensitive, thus causing
the lookup to fail, resulting in two entries for the same property name
differing only in case. Eg. `MSBuildSDKsPath` vs `MSBuildSdksPath`.
Fixed to ignore case.
Fixes https://github.com/mono/mono/issues/14765 .
* Bump VSMac to 8.1.0.2742 to fix msbuild issues
This is required to get the support for the msbuild `ToolsVersion`
change from `15.0` to `Current`.
* [tests][msbuild] Fix Binding resources test with updated msbuild
Test failure with updated msbuild and vsmac 8.1:
```
Xamarin.iOS.Tasks.NativeReferencesNoEmbedding("iPhone").ShouldNotUnnecessarilyRebuildBindingProject(True)
Binding project build did not create package?
Expected: True
But was: False
at Xamarin.iOS.Tasks.NativeReferencesNoEmbedding.ShouldNotUnnecessarilyRebuildBindingProject (System.Boolean framework) [0x000a0] in <74b8f7d8a53e40109916d305bb4d7403>:0
at (wrapper managed-to-native) System.Reflection.RuntimeMethodInfo.InternalInvoke(System.Reflection.RuntimeMethodInfo,object,object[],System.Exception&)
at System.Reflection.RuntimeMethodInfo.Invoke (System.Object obj, System.Reflection.BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo cul
ture) [0x0006a] in <0519fa732e8845b6a809ce9180f541db>:0
```
The test builds the project multiple times. Before the 3rd build, the project
file's timestamp is updated and expects that the binding package will be
rebuilt. But it is not, because the target `_CreateBindingResourcePackage`
doesn't depend on that project file. So, add that to the target inputs.
* [nuget] Use xibuild to run nuget
Fix errors seen during `nuget restore` for tests:
```
Users/builder/jenkins/workspace/xamarin-macios-pr-builder/tests/xammac_tests/xammac_tests.csproj(213,3): error MSB4024: The imported project file "/Library/Frameworks/Mono.framework/External/xbuild/Xamarin/Mac/Xamarin.Mac.CSharp.targets" could not be loaded. Could not find file "/Library/Frameworks/Mono.framework/External/xbuild/Xamarin/Mac/Xamarin.Mac.CSharp.targets"
```
Usually it's when working on beta Xcodes we make mistakes that only show up
when running on older simulators (and devices):
* Missing/wrong availability attributes.
* Tests for new API that don't check the OS if that API is available.
So automatically enable the tests on older simulators for PR builds when using
a beta Xcode.
* [tests] Remove 32-bit Xamarin.Mac tests (both Classic and Unified).
We're removing support for 32-bit Xamarin.Mac apps (#6300), which means we
don't need to run the tests anymore.
This part of the implementation for #6300, I'm starting with the tests because
some of our XM/Classic tests are failing and making the bots unnecessarily red
(since we'll remove XM/Classic support anyway). Also CI will be faster if we
don't run these tests.
* [mmptest] Fix build.
* [tests] Fix build for mono-native-mac.csproj.
* [xharness] Show links to previous test runs in html report in server mode. (#6031)
* [xharness] Change url for server mode.
* [xharness] Show links to previous test runs in html report in server mode.
This makes it much easier to see what failed in a previous test run.
* [XHarness] Show when we have a watch HE0038 error. (#6060)
To simplify the life of the monitorer, if we get a crash with a HE0038
we will propagate the result to the html and will provide a link to the
issue so that it is easy to report it.
* [xharness] Look for HE0038 anywhere in a log file. (#6175)
Most log files now have timestamps, which means that looking for 'HE00380' at
the start of each line won't work anymore.
Fixes an issue where the HE0038 issue isn't detected properly.
This also makes it so that we only parse the log file once (when the test
finishes), instead of parsing every log named 'Run log' every time we produce
the html report.
The change is a pre step to fix https://github.com/xamarin/xamarin-macios/issues/6162
We are removing the not longer child directory bcl-test/BCLTests since
there is not longer any reason for it to exist. The changes make sure
that the tests work as expected in the new directory.
Next step, move all the test apps to their own directory.
Most log files now have timestamps, which means that looking for 'HE00380' at
the start of each line won't work anymore.
Fixes an issue where the HE0038 issue isn't detected properly.
* [Metal] Sprinkle [return: Release] on all 'new*' selectors. Fixes#5941.
Also add tests for all the API I could figure out how to use.
Fixes https://github.com/xamarin/xamarin-macios/issues/5941.
* [tests] MTLFunctionConstantValues didn't have a default ctor until Xcode 9.
* [tests] Use a higher offset when calling MTLBuffer.CreateTexture to try to comply with the requirements for the API.
Hopefully fixes this assertion:
> 07:42:06.7701360 validateStrideTextureParameters:1512: failed assertion `Linear texture: bytesPerRow (64) must be aligned to 256 bytes'
which doesn't happen on my machine.
* Fix whitespace.
* Simplify nested usings.
* Fix availability correctly.
* [XHarness] Remove the old style mscorlib tests.
Remove the old style test and replace it with the xunit equivalent which
has more tests and is provided by the mono package.
* Only skip the mscorlib tests on watchOS devices with 32b. Run them anywhere else.
The tests in the mini project are the same ones found in
monotouch_Mono.Runtime.Tests_test.dll which is already ran via the
autogenerated bcl tests.
Mini tests therefore are redundant and require the mono sources which
will be removed.
* [xharness] Timestamp all the things.
I find myself wanting timestamps if I don't have them much more than wanting
them gone when I do.
Worst case scenario: timestamps can still be disabled for specific logs if
such a desire ever becomes overwhelming.
* [xharness] Don't timestamp xml files.
* [xharness] Refactor a bit to use named types for a few unnamed types with numerous fields.
Anonymous types becomes quite unwieldy the more fields they have.
* [xharness] Remove unnecessary field assignments.
* [xharness] Don't use a dash in the bundle identifer for watchOS projects.
It causes problems with the mscorlib test project, which can't be launched properly.
I'm not sure what's the underlying cause, but here are some of the symptoms:
* The watch app actually shows up fine on the device, but:
* mlaunch isn't notified about the new process, so it thinks the app didn't
launch.
* The new process doesn't receive any environment variables we try to give it,
which for instance means that it won't auto-start the tests upon launch.
* If we ask mlaunch to attach with lldb, mlaunch will ask watchOS to launch
the process in a suspended state while lldb attaches. Yet the watch app
shows up on the device as if not asked to be suspended upon launch.
It seems that the dash (I assume, because I haven't investigated this very
deeply, I just happened to find a solution that worked) makes watchOS launch
the app as if tapped, instead of launched from an IDE.
The strangest part is that this only happens with the mscorlib test project,
not any of the other test projects we run on the watch, and they all have
dashes in their bundle identifiers... yet replacing the dash with another
character (underscore, letter, removing it altogether) all made things work as
expected.
* [monotouch-test] Adjust expected value for watchOS bundle id.
The watchOS bundle ID changed in fc5067ee67, and
the test failure wasn't caught properly.
Trying to find a simulator will mark the test as a failure if the simulator
couldn't be found, and we don't want that to happen to ignored tests.
This should fix an issue where xharness seems to try to run the 32-bit
simulator tests when asked to run only device tests.
If a report already exists, it's probably because a previous attempt failed
for some reason. This is no reason to not try again, overwriting the previous
attempt.
It causes problems with the mscorlib test project, which can't be launched properly.
I'm not sure what's the underlying cause, but here are some of the symptoms:
* The watch app actually shows up fine on the device, but:
* mlaunch isn't notified about the new process, so it thinks the app didn't
launch.
* The new process doesn't receive any environment variables we try to give it,
which for instance means that it won't auto-start the tests upon launch.
* If we ask mlaunch to attach with lldb, mlaunch will ask watchOS to launch
the process in a suspended state while lldb attaches. Yet the watch app
shows up on the device as if not asked to be suspended upon launch.
It seems that the dash (I assume, because I haven't investigated this very
deeply, I just happened to find a solution that worked) makes watchOS launch
the app as if tapped, instead of launched from an IDE.
The strangest part is that this only happens with the mscorlib test project,
not any of the other test projects we run on the watch, and they all have
dashes in their bundle identifiers... yet replacing the dash with another
character (underscore, letter, removing it altogether) all made things work as
expected.
The entire Xml parsing code has been changed to make the following
improvements:
1. Simplify the logic.
2. Ensure that the tests are not added more than once in the human text
log.
3. Do not use the APIs that load the entire tests into memory. XmlReader
is used to read the file and parse the results.
Fixes: https://github.com/xamarin/xamarin-macios/issues/6072
Fixes: https://github.com/xamarin/maccore/issues/1632
Allow to add extra mtouch arguments to the bcl test applications to configure them. This will allow to pass required specific settings that some tests have, for example, for the linker.
This updates the project generation. We cannot yet fully remove the submodule because:
* We are missing the xunit dlls which should be added in the SDK.
* We have not yet remove all the old style tests. Would make the PR huge, better to deal with it in a diff PR.
* The xunit CoreLib tests have issues loading all the tests, needs some extra work and again, the PR is already large.
Fixes: xamarin/maccore#1199Fixes: xamarin/maccore#1204Fixes: xamarin/maccore#1209Fixes: xamarin/maccore#1510
To simplify the life of the monitorer, if we get a crash with a HE0038
we will propagate the result to the html and will provide a link to the
issue so that it is easy to report it.
The change allows to state the tests that have to be ran. ATM with these
changes, the vsts pipeline must add the following to the env vars:
* tvOS device pipelines: Must add run-tvos-tests to the labels.
* iOS device pipelines: Must add run-ios-tests to the labels.
This will ensure that only the tests for the devices are ran and if the
tests pass we get a green build with no unexpected skips.
We had issues in the code that adds a type found in an assembly to
ensure that it was not removed by the linker. This resulted in some
assemblies having 0 tests.
Added the needed ignore for the corlib tests and system ones.
* [xharness] Change url for server mode.
* [xharness] Show links to previous test runs in html report in server mode.
This makes it much easier to see what failed in a previous test run.
This makes it possible to run tests that are marked as 'BuildOnly' to see if
they've been fixed or not.
There's already a 'build' button if only a build is desired.
We automatically create simulators when needed, but it won't work if the
simulator runtime isn't installed. So handle the case where a test might not
have a simulator to execute in correctly.
Add support for running tests with the earliest possible simulator, and use it for introspection tests.
* Add support for running tests with the earliest possible simulator
(currently iOS 8.1, tvOS 9.1 and watchOS 2.0).
* Make the introspection tests run with the earliest possible simulator.
* Fix several binding issues (mostly missing availability attributes), and a
couple of issues in the introspection tests themselves.
Reference: https://github.com/xamarin/xamarin-macios/issues/3668.
The change allows to state the tests that have to be ran. ATM with these
changes, the vsts pipeline must add the following to the env vars:
* tvOS device pipelines: Must add run-tvos-tests to the labels.
* iOS device pipelines: Must add run-ios-tests to the labels.
This will ensure that only the tests for the devices are ran and if the
tests pass we get a green build with no unexpected skips.
We automatically create simulators when needed, but it won't work if the
simulator runtime isn't installed. So handle the case where a test might not
have a simulator to execute in correctly.
* [linker] Mark protocol interfaces when using the dynamic registrar.
Fixes this monotouch-test failure when using the dynamic registrar and the
linker at the same time:
[FAIL] RegistrarTest.TestProtocolRegistration : UIApplicationDelegate/17669
Expected: True
But was: False
* [tests] Adjust test after linker change.
All Xamarin.iOS apps will now link with QuickLook when using the dynamic
registrar, because NSUrl implements a QuickLook protocol:
fcac64ad6e/src/foundation.cs (L5445).
Adjust LinkAll_Frameworks accordingly, and add a new test that verifies that
the old behavior (not linking with QuickLook when linking all assemblies) is
still correct.
* Share much more marshalling code between the dynamic and static registrar
(by refactoring code into separate functions used by both).
* Fix an issue with multiple out/ref parameters in the dynamic registrar.
* Throw an error instead of silently ignoring out/ref parameter types we don't
completely understand in the dynamic registrar.
* Fix returning Class/SEL out/ref parameters in the static registrar.
* Fix returning NSString out/ref parameters in the dynamic registrar.
* Add/fix support for out/ref arrays everywhere.
* Fix support for the various supported out/ref parameter types in the
generator.
* Add lots of tests.
Fixes https://github.com/xamarin/xamarin-macios/issues/5171.
* [tests] Add sample tester.
Add a unit project that looks for iOS/macOS/tvOS sample projects in several
repositories, and builds them all.
* [tests][sampletester] Remove known issue which has now been fixed.
* [tests] Only run sample tests on CI in Azure Devops.
* Remove the possibility of automatically running the sample tests with
xharness (so the sample tests won't run on PR bots or internal bots when the
'run-all-tests' label is added). It's still possible to run the sample tests
manually from the xharness web UI.
* Automatically trigger the sample test run in Azure Devops if the
'run-sample-tests' label is applied to a PR (and that PR is executed on
internal Jenkins).
* Fix typo.
* Fix path.
* Verbose output to track down scheduling failure.
* Bump maccore to get improved debug spew.
Diff: f527c9c526..f89d74b165
* [tests][sampletester] Fix build for TodoWCF.
https://github.com/xamarin/xamarin-macios/pull/4884 introduced the logic of only building certain `RunTestTask`.
This was meant to disable iOS Extensions as part of a fix to https://github.com/xamarin/maccore/issues/1008.
However this didn't quite work and iOS extensions were still running (and failing).
The reason being that `BuildOnly` was set to a `RunDeviceTask` that's added to a list which is then given to `CreateTestVariations` which creates new instances of `RunDeviceTask`.
We now propagate `BuildOnly` to the new variation instance.
This commit improves the state of the BCL testing in the following ways:
1. Improve the device tets running. Less apps, faster results.
2. WatchOS apps are left as they were to ensure that we do not have deplouyment/run issues.
We now support the ignore files per assembly name to simplify the
tracking of the ignored tests. All
* [XHarness] Ensure we do a nuget restore on bcl imported tests. Fixes#5383
We need to ensure that a nuget restore is done, otherwise we might have
build issues with missing libs.
Fixes https://github.com/xamarin/xamarin-macios/issues/5383
Most of the tests are comming from the downloaded SDK, but some of them
have to reman because they are running/passing more tests. This
differences are due to how the tests that come from mono are written
(missing resources etc..)
Please take a look at
https://devdiv.visualstudio.com/DevDiv/_workitems/edit/794210/ for the
list of bcl tests and test number differences.
Manuel de la Pena
monojenkins
Rolf Bjarne Kvinge
Pramit Mallick
Sebastien Pouliot
Alex Soto
Vincent Dondain
Ankit Jain
Sebastien Pouliot
Alexander Köplinger