As per the documentation of the VSTS test uploader (https://docs.microsoft.com/en-us/azure/devops/pipelines/tasks/test/publish-test-results?view=azure-devops&tabs=yaml#attachments-support)
the tests support the addition of attachments. We want to be able to
access the logs of the different tests runs, this is achieved the
following way:
1. Move TouchUnit to use NUnit V3 which allows to add attachments.
2. Once the tests are ran, add the attachments to the following nodes:
1. The very first test-suite. This will allow to have the logs for
succesul tests.
2. Add logs to failing tests. Reduces the number of clicks to be done
to access to the logs when a test case fails.
3. Modify the assembly name of the test-suit to match the name of the
application. This ensures two things.
1. We have a consistent name for the file column in VSTS, that can be
used to see recurrent failing tests.
2. The name is more readable, since if not, it will contain the UUID
of the device.
Logs are not added to succesful tests because it will have the following
problems:
* Larger data storage usage.
* Longer upload time. The addtion of the logs per tests (succesful or
failed) was tested and resulted in an upload time LONGER than 6 hours
for all TouchUnit, NUnit blc tests and xUnit bcl tests.
In order for this to be useful, the task in the pipeline SHOULD NOT
merge test runs. We should have a test run PER application so that we do
not mix the logs.
Co-Authored-By: Rolf Bjarne Kvinge <rolf@xamarin.com>
Fixes:
> /Library/Frameworks/Mono.framework/External/xbuild/Xamarin/iOS/Xamarin.iOS.Common.targets(1726,3): warning : The Watch App 'MyWatchApp2' has a CFBundleVersion (1.33) that does not match the main app bundle's CFBundleVersion (1.0) [MyWatch2Container/MyWatch2Container.csproj]
> /Library/Frameworks/Mono.framework/External/xbuild/Xamarin/iOS/Xamarin.iOS.Common.targets(1726,3): warning : The Watch Extension 'MyWatchKit2Extension' has a CFBundleVersion (1.0) that does not match the main app bundle's CFBundleVersion (1.33) [MyWatch2Container/MyWatch2Container.csproj]
Fixes:
> MyWatchKit2IntentsExtension/IntentHandler.cs(36): warning MT4174: Unable to locate the block to delegate conversion method for the method MyWatchKit2IntentsExtension.IntentHandler.ResolveRecipients's parameter #2. [MyWatchKit2Extension/MyWatchKit2Extension.csproj]
The method signature probably changed from a beta release to the corresponding
stable release, and this method was never updated.
One of the reason why we lost the TestReport was because the xml added
from the new code did not use the same name for the log as the expected
one (https://github.com/xamarin/xamarin-macios/pull/7835/files#diff-03773861bcef485bf343300f31b60b0eR374).
If we are going to be using the description as a way to decide what is
going to be done with it, lets use static vars so that we do not have
logs added with the wrong name.
Add code to write the TestReport when we have xunit results. Also
refactored code for the NUnit format to not use LINQ, it will use less
memory and in the future we can move to async (not now since it might
raise other problems).
fixes: https://github.com/xamarin/xamarin-macios/issues/7826
Only one to start... it's been discussed before but we generally
found other ways to do them. Let's continue to pick the best place
but we now have more options :)
Use the xml parsing helper methods to decide if the xml can be parsed
and if we will be able to generate the report. That way we avoid an
exception that makes the CI noise.
* [master] Bump mono to pickup needed nunitlite changes.
Commits are:
* [interp] context can be uninitialized for get_resume_state callback (#18535) mono/mono@455cf7d
* [2019-12] [debugger] Native thread not executing managed code considered as terminated (#18504) mono/mono@a979811
* [jit] Compute has_references correctly for gshared types whose constraint is a generic valuetype. Also emit write barriers correctly for these types. (#18562) mono/mono@f9e5a6d
* [2019-12] Bump msbuild to track mono-2019-10 (#18577) mono/mono@01be275
* [runtime] Disable lldb backtrace display on osx, it hangs on attaching in lldb. (#18591) mono/mono@ef8188a
* configure.ac: remove AC_SEARCH_LIBS for libintl (#18595) mono/mono@e55302c
* Bump bockbuild for Pango patch 1e2d68b
* [corlib] Split corlib xunit tests even more for iOS (#18620) mono/mono@8b72dbb
* [2019-12] [merp] MONO_DEBUG=no-gdb-stacktrace shouldn't disable MERP (#18611) mono/mono@4c93e38
* [aot] Avoid inflating gparams with byreflike types during generic sharing. (#18682) mono/mono@0011444
* Update deprecated query parameter to header (#18705) mono/mono@e65846b
* [NUnitLite] Bump nunitlite submodule. (#18733) mono/mono@36073a0
Diff: 2edccc52a7...36073a0c74
Other commits need for this to land"
* Add a MONO define so that we get the extensions written for mono.
* Update the sources on iphone to remove the TextUI which is not used.
* BONUS: Remove an annoying warning when compiling NUnitLite
* Update the templates too which are use by xharness.
```
warning CS0659: 'ResultState' overrides Object.Equals(object o) but does not override Object.GetHashCode()
```
Don't separate command-line arguments to mmp with newlines, in dotnet those
newlines are passed to mmp instead of being treated as whitespace, and that
confuses mmp.
Also remove additional and unneeded whitespace in the command line arguments,
the CommandLineArgumentBuilder class already adds whitespace when needed.
* [msbuild] Fix the MyWatchApp2 test project to include resources already present.
Fixes several warnings:
> MyWatchApp2/obj/iPhoneSimulator/Debug/actool/cloned-assets/Resources/Images.xcassets : actool warning : The file "Icon-27.5x27.5@2x.png" for the image set "AppIcons" does not exist. [MyWatchApp2/MyWatchApp2.csproj]
> MyWatchApp2/obj/iPhoneSimulator/Debug/actool/cloned-assets/Resources/Images.xcassets : actool warning : The file "Icon-86x86@2x.png" for the image set "AppIcons" does not exist. [MyWatchApp2/MyWatchApp2.csproj]
> MyWatchApp2/obj/iPhoneSimulator/Debug/actool/cloned-assets/Resources/Images.xcassets : actool warning : The file "Icon-40x40@2x.png" for the image set "AppIcons" does not exist. [MyWatchApp2/MyWatchApp2.csproj]
> MyWatchApp2/obj/iPhoneSimulator/Debug/actool/cloned-assets/Resources/Images.xcassets : actool warning : The file "Icon-44x44@2x.png" for the image set "AppIcons" does not exist. [MyWatchApp2/MyWatchApp2.csproj]
> MyWatchApp2/obj/iPhoneSimulator/Debug/actool/cloned-assets/Resources/Images.xcassets : actool warning : The file "Icon-24x24@2x.png" for the image set "AppIcons" does not exist. [MyWatchApp2/MyWatchApp2.csproj]
> MyWatchApp2/obj/iPhoneSimulator/Debug/actool/cloned-assets/Resources/Images.xcassets : actool warning : The file "Icon-98x98@2x.png" for the image set "AppIcons" does not exist. [MyWatchApp2/MyWatchApp2.csproj]
> MyWatchApp2/obj/iPhoneSimulator/Debug/actool/cloned-assets/Resources/Images.xcassets : actool warning : The file "Icon-29x29@2x.png" for the image set "AppIcons" does not exist. [MyWatchApp2/MyWatchApp2.csproj]
> MyWatchApp2/obj/iPhoneSimulator/Debug/actool/cloned-assets/Resources/Images.xcassets : actool warning : The file "Icon-29x29@3x.png" for the image set "AppIcons" does not exist. [MyWatchApp2/MyWatchApp2.csproj]
* [msbuild] Add a 1024x1024 icon to the MyWatchApp2 test project.
Fixes this warning:
> MyWatchApp2/obj/iPhoneSimulator/Debug/actool/cloned-assets/Resources/Images.xcassets : actool warning : A 1024x1024 app store icon is required for watchOS apps [MyWatchApp2/MyWatchApp2.csproj]
* [msbuild] Remove the longLook icon entry for MyWatchApp2.
It doesn't seem to be a valid entry, there's no such entry when creating a
watchOS app in Xcode. Additionally it fixes this warning:
> MyWatchApp2/obj/iPhoneSimulator/Debug/actool/cloned-assets/Resources/Images.xcassets : actool warning : The app icon set "AppIcons" has an unassigned child. [MyWatchApp2/MyWatchApp2.csproj]
We ping the tcp listener to know that we have a tcp connection, that is
written in the xml logs, which means that parsing will not work. Ignore
the ping, parse xml, and make sure that the xml that will be consume by
vsts is valid.
This PR also fixes https://github.com/xamarin/maccore/issues/827 which does not longer happen.
When building extensions, we first store all the mtouch arguments in a file
when msbuild builds the extension, and then when msbuild builds the main
project, we load those arguments again and actually build the extension at the
same time as we build the main app.
As such, it's important to make sure that when we reload the extension
arguments we end up with the exact same build configuration as the first time.
Unfortunately that was not the case regarding the interpreter: we
automatically set the 'EnableRepl' value in Main according to whether the
interpreter was enabled or not, but Main is not called after re-loading the
arguments when building extensions.
Fix this by moving the logic that automatically sets 'EnableRepl' to somewhere
that is executed when re-loading arguments when building extensions.
Fixes https://github.com/xamarin/xamarin-macios/issues/7780.
* Acquire permission for the microphone (the microphone permission dialog is
not a blocking dialog, which is why things have worked so far).
* Set the last_modified field in the TCC database to now instead of 1970.
Apparently some permissions time out (kTCCServiceMediaLibrary), which means
that the last_modified field is important to get right.
Most false positives are (so far) because `nuget restore` fails - both
due to network issues and a mono issue [1].
This PR allows to filter/remove some projects from a solution before
nuget is invoked. This can reduce the number of packages to download
which, beside reducing false positives, is good performance wise.
It also allows some project (e.g. Xappy.iOS) to build even if some other
projects (e.g. Xappy.UWP) in the solution can't be built.
[1] https://github.com/mono/mono/issues/15418
Looks like we have issues with the internet sharing in the VSTS bots,
this means that now that we always try to parse the XML on CI, we get an
exception, catch it and do not show the results.
The workaround simply tries to read the xml, if possible, we will parse
it, else deal with the text only log. The fix will show the results, but
it is a workaround for a configuration issue in the CI.