[Xamarin.Android.NUnitLite] Improve test name formatting on Jenkins (#917)
We've been running on-device NUnit tests for quite some time, and [Jenkins is nice enough to provide a test report][xa668]. [xa668]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android/668/testReport/ There's just one small annoyance: the Package name for on-device tests is `(root)`: # All Tests +-----------------------------+-----------+-------+-------+-------+-------+ | Package | Duration | Fail | Skip | Pass | Total | +-----------------------------+-----------+-------+-------+-------+-------+ | (root) | 14 min | 0 | 814 | 20260 | 21074 | | Java.Interop.DynamicTests | 15 sec | 0 | 0 | 14 | 14 | | ... The reason the for the `(root)` package -- which constitutes all on-device tests -- is due to three interractions: 1. [junit-plugin renders `(root)`][junit-root] when the `//testcase/@classname` or `//testcase/@name` attribute does *not* contain a `.` (period). 2. [xunit-plugin's XSLT to convert NUnit2 XML to JUnit XML][xunit-xslt] uses `test-suite//results//test-case[1]/@name` to determine the assembly name (!). (Half of this XSLT is inexplicable to me. Why is line 39 checking to see if `test-suite//results//test-case[1]//@name` ends with `)` (close parenthesis)?) 3. `Xamarin.Android.NUnitLite`'s NUnit2 XML output never contains any `.`s within `//test-case/@name` values. ```xml <!-- fragment from TestResult-Mono.Android_Tests.xml --> <test-case name="ApplicationContextIsApp" executed="True" result="Success" success="True" time="0.003" asserts="2" /> ``` [junit-root]: https://github.com/jenkinsci/junit-plugin/blob/2c1655c/src/main/java/hudson/tasks/junit/CaseResult.java#L332-L336 [xunit-xslt]: https://github.com/jenkinsci/xunit-plugin/blob/f122ddd/src/main/resources/org/jenkinsci/plugins/xunit/types/nunit-1.0-to-junit-2.xsl Compare to e.g. the NUnit2 XML output that NUnit 3.7's `nunit3-console.exe` + `format=nunit2` generates -- as produced by `make run-nunit-tests`: ```xml <!-- fragment from TestResult-Java.Interop-Tests.xml --> <test-case name="Java.InteropTests.InvokeVirtualFromConstructorTests.ActivationConstructor" executed="True" result="Success" success="True" time="0.139" asserts="2" /> ``` the result of passing `Xamrin.Android.NUnitLite` NUnit2 XML through xunit-plugin's XSLT is: ```xml <!-- Fragment from: `xsltproc nunit-1.0-to-junit-2.xsl TestResult-Mono.Android_Tests.xml` --> <testcase classname="ApplicationTest" name="ApplicationContextIsApp" time="0.003"/> ``` Compare to NUnit 3.7 output: ```xml <!-- Fragment from: `xsltproc nunit-1.0-to-junit-2.xsl TestResult-Java.Interop-Tests.xml.txt --> <testcase classname="Java.InteropTests.InvokeVirtualFromConstructorTests" name="ActivationConstructor" time="0.139"/ ``` To get rid of the `(root)` package, I see only one simple way to do so: Update `Xamarin.Android.NUnitLite`s `NUnit2XmlOutputWriter` so that the `//test-case/@name` value *always* contains a `.`. Specifically, instead of using `ITestResult.Name`, use `ITestResult.FullName`. *However*, some of our existing tests are in "funny" namespaces. For example, `Mono.Android-Tests.csproj` contains an `Android.AppTests.ApplicationTest` fixture. Jenkins Test output also doesn't "care" about assembly or process boundaries; all "packages", regardless of where they're from, are merged. Try to improve on that by not just using `ITestResult.FullName` for the `//test-case/@name` value, but *also* use the value of the currently executing Android Package, as per `Android.App.Application.PackageName`, as an "assembly-qualified name" -like construct. This will hopefully result in a more useful table: # All Tests +---------------------------------------+-----------+-------+-------+-------+-------+ | Package | Duration | Fail | Skip | Pass | Total | +---------------------------------------+-----------+-------+-------+-------+-------+ | Java.Interop.DynamicTests | 15 sec | 0 | 0 | 14 | 14 | | Mono.Android_Tests, Android.AppTests | 0.18 sec | 0 | 0 | 4 | 4 | | ... Alternatively, maybe this is a sign we should "re-namespace" our existing NUnit tests to start with the assembly name...
This commit is contained in:
Родитель
a296784fcc
Коммит
23b2642ed0
|
@ -351,7 +351,7 @@ namespace Xamarin.Android.NUnitLite
|
|||
if (suite != null)
|
||||
return suite;
|
||||
else {
|
||||
var ts = new TestSuite ("");
|
||||
var ts = new TestSuite (global::Android.App.Application.Context.PackageName);
|
||||
Console.Error.WriteLine (ts.FullName);
|
||||
foreach (var i in AssemblyLevel.Cast<NUnitTest> ())
|
||||
ts.Add (i);
|
||||
|
|
|
@ -216,7 +216,7 @@ namespace NUnitLite.Runner
|
|||
else
|
||||
{
|
||||
xmlWriter.WriteStartElement("test-case");
|
||||
xmlWriter.WriteAttributeString("name", result.Name);
|
||||
xmlWriter.WriteAttributeString("name", global::Android.App.Application.Context.PackageName + ", " + result.FullName);
|
||||
}
|
||||
|
||||
if (test.Properties.ContainsKey(PropertyNames.Description))
|
||||
|
|
Загрузка…
Ссылка в новой задаче