Add RFC spec for Hierarchy property (#261)

* Add initial Hierarchy spec

* Some updates and clarifications.

* Clarify some terms and scenarios

* Update 0033-Hierarchy-TestCase-Property.md

* Clarify the wording around open/closed type example

Co-authored-by: Peter Waldschmidt <pewaldsc@microsoft.com>
Co-authored-by: Medeni Baykal <433724+Haplois@users.noreply.github.com>
This commit is contained in:
Peter Waldschmidt 2022-07-19 13:37:05 -04:00 коммит произвёл GitHub
Родитель c0eca6950c
Коммит e373b6d07b
Не найден ключ, соответствующий данной подписи
Идентификатор ключа GPG: 4AEE18F83AFDEB23
1 изменённых файлов: 29 добавлений и 0 удалений

Просмотреть файл

@ -0,0 +1,29 @@
# 0033 Hierarchy TestCase Property
## Summary
This document standardizes an additional `Hierarchy` property on TestCase to support more flexible visual display of tests in the Visual Studio Test Explorer.
## Overview
The `ManagedType` and `ManagedMethod` properties [have been defined](0017-Managed-TestCase-Properties.md) as a way of deterministically identifying the type and method to which a test case belongs (in managed code). However, the legacy `FullyQualifiedName` property is still required to deduce a `Namespace`, `Classname` and `TestGroup` for display in the Test Explorer hierarchy. When the Test Explorer added a hierarchy for easier grouping and navigation of tests, there was no specified way for a TestCase to provide those values so they were heuristically derived from `FullyQualifiedName`.
This document specifies a way for the TestCase to provide these display values directly to the test explorer, granting flexibility and control back to the test adapter.
## Specification
TestCases for managed code may include a string array (string[]) valued property named `Hierarchy`. The specification below outlines the requirements for the contents of the property.
### `Hierarchy` Property
The `Hierarchy` test case property represents the text strings used to display the test in the Visual Studio Test Explorer window.
* The property must be an array of strings with 4 elements. If the property is null, missing or doesn't have 4 elements then it is is ignored as invalid.
* The element values are arranged as `[Container, Namespace, Class, TestGroup]`, Each TestCase is coalesced into a tree using the following order from the root. `Container`, `Namespace`, `Class`, `TestGroup`, `DisplayName`.
* `Container` usually represents the project name or the assembly name. For C# tests, if this property is set as a `null` or empty string ("") then the project name will be used by VS.
* `Namespace` represents the namespace containing the test suite (or a suitable semantic equivalent). If there aren't any equivalent, or the class doesn't belong to a namespace this can be `null` or empty string ("").
* `Class` normally represents the display name of the test suite. For example, in C# this is typically the name of class containing the tests.
* `TestGroup` typically represents the method that implements a particular test or set of tests (if data-driven). If a test is not data-driven, then the TestGroup could be the same as the `DisplayName`. By data-driven, we mean tests that are run repeatedly with different sets of data (i.e. Theory tests in XUnit/NUnit). A test case should be generated for each test/data combination, where the `TestGroup` is set to the name of the test method (or similar) and the `DisplayName` includes a representation of the data used to execute the test in order to distinguish them in the test explorer UI.
* `TestGroup` can be `null`, or empty string ("").
* If `TestGroup` is the same for all tests under a particular `Class` or empty, then the `TestGroup` level is omitted from the hierarchy.
* `DisplayName` is the name of a particular single test case. For data-driven tests, the `DisplayName` should include a representation of the parameters in order to distinguish the unique test case to the end user. If the test case is not data-driven, then there is no need to include the parameters as the method name would likely be sufficient.
* If the display name for a test is not unique under a given hierarchy, tests will not be executed or discovered correctly, and depending on the test framework test discovery could fail. For the purposes of uniqueness `Namespace` and `TestGroup` elements with `null` and empty string ("") values are considered equal.
* In some cases `ClassName`, `TestGroup` and `Method` could represent generic methods or types (for instance, tests written in C# or C++). In that case, if possible, the text should reflect the "closed" or instantiated typename. For example when displaying the generic type `List<>`, the closed form (e.g. `List<String>, List<MyType>`) should be used. The generic type `List<>` is referenced in metadata as ``List`1`` and in the code as `List<T>`; both of these are "open" forms of the type because they do not specify the type of `T` and could cause confusion.