Move to use interfaces, that will let us later add tests that will
verify that all the correct logging is performed. As an example, added a
test for XmlResultParser that ensures that the failures are correctly
generated. The test uses Moq to pass the different paths to be used and
later be able to verify the wirtten xml.
We support different outputs, lets add the avility for the caller to
decide which one to use. We default to NUnit V3 due to or dependency to
it in VSTS.
Xml should be:
```
<failure>
<message>Foo</message>
<stack-trace>Bar</stack-trace>
</failure>
```
But we generate:
```
<failure>
<message>Foo
<stack-trace>Bar</stack-trace>
</message>
</failure>
```
Makes the parsing of the failures impossible.
Because the PublishTest task is taking a regular expression, it is
importing the test results more than one, which gives wrong stats. Add a
prefix to better filter those files we are interested in (thos with
attachments).
As a bonus, refactored the xml failure code for less copy pasting and to
have a single place where we had to add the prefix.
As per the documentation of the VSTS test uploader (https://docs.microsoft.com/en-us/azure/devops/pipelines/tasks/test/publish-test-results?view=azure-devops&tabs=yaml#attachments-support)
the tests support the addition of attachments. We want to be able to
access the logs of the different tests runs, this is achieved the
following way:
1. Move TouchUnit to use NUnit V3 which allows to add attachments.
2. Once the tests are ran, add the attachments to the following nodes:
1. The very first test-suite. This will allow to have the logs for
succesul tests.
2. Add logs to failing tests. Reduces the number of clicks to be done
to access to the logs when a test case fails.
3. Modify the assembly name of the test-suit to match the name of the
application. This ensures two things.
1. We have a consistent name for the file column in VSTS, that can be
used to see recurrent failing tests.
2. The name is more readable, since if not, it will contain the UUID
of the device.
Logs are not added to succesful tests because it will have the following
problems:
* Larger data storage usage.
* Longer upload time. The addtion of the logs per tests (succesful or
failed) was tested and resulted in an upload time LONGER than 6 hours
for all TouchUnit, NUnit blc tests and xUnit bcl tests.
In order for this to be useful, the task in the pipeline SHOULD NOT
merge test runs. We should have a test run PER application so that we do
not mix the logs.
Co-Authored-By: Rolf Bjarne Kvinge <rolf@xamarin.com>
Add code to write the TestReport when we have xunit results. Also
refactored code for the NUnit format to not use LINQ, it will use less
memory and in the future we can move to async (not now since it might
raise other problems).
fixes: https://github.com/xamarin/xamarin-macios/issues/7826
Manuel de la Pena