Rename references that we own to main (#250)
This commit is contained in:
Jakub Jareš
GitHub
Родитель
2e7becf3da
Коммит
0150a40724
20
README.md
20
README.md
|
|
@ -3,16 +3,16 @@ You've found the GitHub repository which contains the content for the Visual Stu
|
|||
If you are looking for the Visual Studio Test Platform product GitHub repository, you can find it [here](https://github.com/Microsoft/vstest).
|
||||
|
||||
### Documentation
|
||||
- [Test Platform Architecture](https://github.com/Microsoft/vstest-docs/blob/master/RFCs/0001-Test-Platform-Architecture.md)
|
||||
- [Test Discovery Protocol](https://github.com/Microsoft/vstest-docs/blob/master/RFCs/0002-Test-Discovery-Protocol.md)
|
||||
- [Test Execution Protocol](https://github.com/Microsoft/vstest-docs/blob/master/RFCs/0003-Test-Execution-Protocol.md)
|
||||
- [Adapter Extensibility](https://github.com/Microsoft/vstest-docs/blob/master/RFCs/0004-Adapter-Extensibility.md)
|
||||
- [Test Platform SDK](https://github.com/Microsoft/vstest-docs/blob/master/RFCs/0005-Test-Platform-SDK.md)
|
||||
- [Editors API Specification](https://github.com/Microsoft/vstest-docs/blob/master/RFCs/0007-Editors-API-Specification.md)
|
||||
- [Data collection Protocol](https://github.com/Microsoft/vstest-docs/blob/master/RFCs/0006-DataCollection-Protocol.md)
|
||||
- [Translation Layer](https://github.com/Microsoft/vstest-docs/blob/master/RFCs/0008-TranslationLayer.md)
|
||||
- [Editors API Revision Update](https://github.com/Microsoft/vstest-docs/blob/master/RFCs/0009-Editors-API-RevisionUpdate.md)
|
||||
- [Roadmap](https://github.com/Microsoft/vstest-docs/blob/master/docs/releases.md)
|
||||
- [Test Platform Architecture](https://github.com/Microsoft/vstest-docs/blob/main/RFCs/0001-Test-Platform-Architecture.md)
|
||||
- [Test Discovery Protocol](https://github.com/Microsoft/vstest-docs/blob/main/RFCs/0002-Test-Discovery-Protocol.md)
|
||||
- [Test Execution Protocol](https://github.com/Microsoft/vstest-docs/blob/main/RFCs/0003-Test-Execution-Protocol.md)
|
||||
- [Adapter Extensibility](https://github.com/Microsoft/vstest-docs/blob/main/RFCs/0004-Adapter-Extensibility.md)
|
||||
- [Test Platform SDK](https://github.com/Microsoft/vstest-docs/blob/main/RFCs/0005-Test-Platform-SDK.md)
|
||||
- [Editors API Specification](https://github.com/Microsoft/vstest-docs/blob/main/RFCs/0007-Editors-API-Specification.md)
|
||||
- [Data collection Protocol](https://github.com/Microsoft/vstest-docs/blob/main/RFCs/0006-DataCollection-Protocol.md)
|
||||
- [Translation Layer](https://github.com/Microsoft/vstest-docs/blob/main/RFCs/0008-TranslationLayer.md)
|
||||
- [Editors API Revision Update](https://github.com/Microsoft/vstest-docs/blob/main/RFCs/0009-Editors-API-RevisionUpdate.md)
|
||||
- [Roadmap](https://github.com/Microsoft/vstest-docs/blob/main/docs/releases.md)
|
||||
|
||||
### Contributing
|
||||
There are many ways to contribute to VSTest
|
||||
|
|
|
|||
|
|
@ -225,9 +225,9 @@ is detailed in a different document.
|
|||
communication with a TestSession.Terminate.
|
||||
9. On receiving a terminate message the test host process cleanly exits.
|
||||
|
||||
[DiscoveryPayload]: https://github.com/Microsoft/vstest/blob/master/src/Microsoft.TestPlatform.ObjectModel/Client/DiscoveryCriteria.cs
|
||||
[TestMessagePayload]: https://github.com/Microsoft/vstest/blob/master/src/Microsoft.TestPlatform.CommunicationUtilities/Messages/TestMessagePayload.cs
|
||||
[DiscoveryCompletePayload]: https://github.com/Microsoft/vstest/blob/master/src/Microsoft.TestPlatform.CommunicationUtilities/Messages/DiscoveryCompletePayload.cs
|
||||
[DiscoveryPayload]: https://github.com/Microsoft/vstest/blob/main/src/Microsoft.TestPlatform.ObjectModel/Client/DiscoveryCriteria.cs
|
||||
[TestMessagePayload]: https://github.com/Microsoft/vstest/blob/main/src/Microsoft.TestPlatform.CommunicationUtilities/Messages/TestMessagePayload.cs
|
||||
[DiscoveryCompletePayload]: https://github.com/Microsoft/vstest/blob/main/src/Microsoft.TestPlatform.CommunicationUtilities/Messages/DiscoveryCompletePayload.cs
|
||||
|
||||
#### Execution:
|
||||
|
||||
|
|
@ -259,10 +259,10 @@ is detailed in a different document.
|
|||
communication with a TestSession.Terminate.
|
||||
9. On receiving a terminate message the test host process cleanly exits.
|
||||
|
||||
[TestRunCriteriaWithSources]: https://github.com/Microsoft/vstest/blob/master/src/Microsoft.TestPlatform.CommunicationUtilities/ObjectModel/TestRunCriteriaWithSources.cs
|
||||
[TestRunStatsPayload]: https://github.com/Microsoft/vstest/blob/master/src/Microsoft.TestPlatform.CommunicationUtilities/Messages/TestRunStatsPayload.cs
|
||||
[TestMessagePayload]: https://github.com/Microsoft/vstest/blob/master/src/Microsoft.TestPlatform.CommunicationUtilities/Messages/TestMessagePayload.cs
|
||||
[TestRunCompletePayload]: https://github.com/Microsoft/vstest/blob/master/src/Microsoft.TestPlatform.CommunicationUtilities/Messages/TestRunCompletePayload.cs
|
||||
[TestRunCriteriaWithSources]: https://github.com/Microsoft/vstest/blob/main/src/Microsoft.TestPlatform.CommunicationUtilities/ObjectModel/TestRunCriteriaWithSources.cs
|
||||
[TestRunStatsPayload]: https://github.com/Microsoft/vstest/blob/main/src/Microsoft.TestPlatform.CommunicationUtilities/Messages/TestRunStatsPayload.cs
|
||||
[TestMessagePayload]: https://github.com/Microsoft/vstest/blob/main/src/Microsoft.TestPlatform.CommunicationUtilities/Messages/TestMessagePayload.cs
|
||||
[TestRunCompletePayload]: https://github.com/Microsoft/vstest/blob/main/src/Microsoft.TestPlatform.CommunicationUtilities/Messages/TestRunCompletePayload.cs
|
||||
|
||||
<!--The exact set of switches the vstest.console runner supports is listed here([Todo] add link here.))-->
|
||||
|
||||
|
|
|
|||
|
|
@ -28,23 +28,23 @@ Here is what the changed wire protocol looks like after TPV2 integrates with dot
|
|||
|
||||
|
||||
### Breaking changes for Adapter(IDE)
|
||||
1. The object model used via the wire protocol will change from using [Microsoft.Extensions.Testing.Abstractions.Test](https://github.com/dotnet/cli/blob/rel/1.0.0/src/Microsoft.Extensions.Testing.Abstractions/Test.cs) to [Microsoft.VisualStudio.TestPlatform.ObjectModel.TestCase](https://github.com/Microsoft/vstest/blob/master/src/Microsoft.TestPlatform.ObjectModel/TestCase.cs).
|
||||
1. The object model used via the wire protocol will change from using [Microsoft.Extensions.Testing.Abstractions.Test](https://github.com/dotnet/cli/blob/rel/1.0.0/src/Microsoft.Extensions.Testing.Abstractions/Test.cs) to [Microsoft.VisualStudio.TestPlatform.ObjectModel.TestCase](https://github.com/Microsoft/vstest/blob/main/src/Microsoft.TestPlatform.ObjectModel/TestCase.cs).
|
||||
2. The IDE adapter would now be sending the test Containers along with test platform settings to dotnet-test.
|
||||
3. The IDE adapter would now receive test cases in a batch as opposed to a single test case earlier to improve performance.
|
||||
4. The IDE adapter would now receive a [DiscoveryCompletePayload](https://github.com/Microsoft/vstest/blob/master/src/Microsoft.TestPlatform.CommunicationUtilities/Messages/DiscoveryCompletePayload.cs) message with the TestDiscovery.Completed message. This would have the run stats and the last set of test cases found.
|
||||
4. The IDE adapter would now receive a [DiscoveryCompletePayload](https://github.com/Microsoft/vstest/blob/main/src/Microsoft.TestPlatform.CommunicationUtilities/Messages/DiscoveryCompletePayload.cs) message with the TestDiscovery.Completed message. This would have the run stats and the last set of test cases found.
|
||||
|
||||
### Breaking Changes for Frameworks
|
||||
1. TestRunner would cease to exist. Framework writers would have to port their Rocksteady adapters to Core CLR to support all .NET Core scenarios.
|
||||
|
||||
### New Flow
|
||||
1. After the optional version check, the IDE adapter can choose to optionally send paths to extensions it wants to initialize the test platform with.
|
||||
2. It then sends a TestDiscovery.Start message with a [DiscoveryRequestPayload](https://github.com/Microsoft/vstest/blob/master/src/Microsoft.TestPlatform.VsTestConsole.TranslationLayer/Payloads/DiscoveryRequestPayload.cs) which contains the sources and settings. Dotnet-test will start the test host process and pass on the containers and settings to it.
|
||||
2. It then sends a TestDiscovery.Start message with a [DiscoveryRequestPayload](https://github.com/Microsoft/vstest/blob/main/src/Microsoft.TestPlatform.VsTestConsole.TranslationLayer/Payloads/DiscoveryRequestPayload.cs) which contains the sources and settings. Dotnet-test will start the test host process and pass on the containers and settings to it.
|
||||
3. The testhost then invokes the rocksteady adapter with the containers to discover tests. It uses the ITestDiscoverer.DiscoverTests as its entry point to the Rocksteady Adapter.
|
||||
4. The Rocksteady adapter then sends each test case discovered to the test host which caches this information.
|
||||
5. The test host passes on the cached test cases to dotnet-test in batches when a cache size is hit which then forwards it over to the IDE adapter.
|
||||
6. Optionally the IDE adapter may also receive a TestSession.Message with a [TestMessagePayload](https://github.com/Microsoft/vstest/blob/master/src/Microsoft.TestPlatform.CommunicationUtilities/Messages/TestMessagePayload.cs) notifying of any issues during discovery.
|
||||
7. After all tests are discovered, the test host sends a TestDiscovery.Completed message with a [DiscoveryCompletePayload](https://github.com/Microsoft/vstest/blob/master/src/Microsoft.TestPlatform.CommunicationUtilities/Messages/DiscoveryCompletePayload.cs), which has the stats and the last set of test cases found, to dotnet-test.
|
||||
8. dotnet-test then sends a TestDiscovery.Completed with the [DiscoveryCompletePayload](https://github.com/Microsoft/vstest/blob/master/src/Microsoft.TestPlatform.CommunicationUtilities/Messages/DiscoveryCompletePayload.cs) to the IDE adapter.
|
||||
6. Optionally the IDE adapter may also receive a TestSession.Message with a [TestMessagePayload](https://github.com/Microsoft/vstest/blob/main/src/Microsoft.TestPlatform.CommunicationUtilities/Messages/TestMessagePayload.cs) notifying of any issues during discovery.
|
||||
7. After all tests are discovered, the test host sends a TestDiscovery.Completed message with a [DiscoveryCompletePayload](https://github.com/Microsoft/vstest/blob/main/src/Microsoft.TestPlatform.CommunicationUtilities/Messages/DiscoveryCompletePayload.cs), which has the stats and the last set of test cases found, to dotnet-test.
|
||||
8. dotnet-test then sends a TestDiscovery.Completed with the [DiscoveryCompletePayload](https://github.com/Microsoft/vstest/blob/main/src/Microsoft.TestPlatform.CommunicationUtilities/Messages/DiscoveryCompletePayload.cs) to the IDE adapter.
|
||||
9. Once the IDE adapter is done, it sends dotnet-test a TestSession.Terminate which will cause dotnet test to shutdown.
|
||||
|
||||
### Notes
|
||||
|
|
|
|||
|
|
@ -32,33 +32,33 @@ And this protocol helps one launch a custom host or perform debugging:
|
|||
|
||||
|
||||
### Breaking changes for Adapter(IDE)
|
||||
1. The object model used via the wire protocol will change from using [Microsoft.Extensions.Testing.Abstractions.Test](https://github.com/dotnet/cli/blob/rel/1.0.0/src/Microsoft.Extensions.Testing.Abstractions/Test.cs) and [Microsoft.Extensions.Testing.Abstractions.TestResult](https://github.com/dotnet/cli/blob/rel/1.0.0/src/Microsoft.Extensions.Testing.Abstractions/TestResult.cs) to [Microsoft.VisualStudio.TestPlatform.ObjectModel.TestCase](https://github.com/Microsoft/vstest/blob/master/src/Microsoft.TestPlatform.ObjectModel/TestCase.cs) and [Microsoft.VisualStudio.TestPlatform.ObjectModel.TestResult](https://github.com/Microsoft/vstest/blob/master/src/Microsoft.TestPlatform.ObjectModel/TestCase.cs) appropriately.
|
||||
1. The object model used via the wire protocol will change from using [Microsoft.Extensions.Testing.Abstractions.Test](https://github.com/dotnet/cli/blob/rel/1.0.0/src/Microsoft.Extensions.Testing.Abstractions/Test.cs) and [Microsoft.Extensions.Testing.Abstractions.TestResult](https://github.com/dotnet/cli/blob/rel/1.0.0/src/Microsoft.Extensions.Testing.Abstractions/TestResult.cs) to [Microsoft.VisualStudio.TestPlatform.ObjectModel.TestCase](https://github.com/Microsoft/vstest/blob/main/src/Microsoft.TestPlatform.ObjectModel/TestCase.cs) and [Microsoft.VisualStudio.TestPlatform.ObjectModel.TestResult](https://github.com/Microsoft/vstest/blob/main/src/Microsoft.TestPlatform.ObjectModel/TestCase.cs) appropriately.
|
||||
2. The IDE adapter would now be sending the test Containers along with test platform settings to dotnet-test.
|
||||
3. The IDE adapter would now receive test results in a batch in the [TestRunStatsPayload](https://github.com/Microsoft/vstest/blob/master/src/Microsoft.TestPlatform.CommunicationUtilities/Messages/TestRunStatsPayload.cs) as opposed to a test start, test result and a test end earlier. The TestRunStatsPayload will also have the statistics of passed/failed/skipped tests so far along with list of tests currently being run.
|
||||
4. The IDE Adapter would now receive a [TestRunCompletePayload](https://github.com/Microsoft/vstest/blob/master/src/Microsoft.TestPlatform.CommunicationUtilities/Messages/TestRunCompletePayload.cs) with TestExecution.Completed which would have the run statistics along with the results for last set of tests run.
|
||||
3. The IDE adapter would now receive test results in a batch in the [TestRunStatsPayload](https://github.com/Microsoft/vstest/blob/main/src/Microsoft.TestPlatform.CommunicationUtilities/Messages/TestRunStatsPayload.cs) as opposed to a test start, test result and a test end earlier. The TestRunStatsPayload will also have the statistics of passed/failed/skipped tests so far along with list of tests currently being run.
|
||||
4. The IDE Adapter would now receive a [TestRunCompletePayload](https://github.com/Microsoft/vstest/blob/main/src/Microsoft.TestPlatform.CommunicationUtilities/Messages/TestRunCompletePayload.cs) with TestExecution.Completed which would have the run statistics along with the results for last set of tests run.
|
||||
|
||||
### Breaking Changes for Frameworks
|
||||
1. TestRunner would cease to exist. Framework writers would have to port their Rocksteady adapters to Core CLR to support all .NET Core scenarios.
|
||||
|
||||
### New Flow for Run Tests:
|
||||
1. After the optional version check, the IDE adapter can choose to optionally send paths to extensions it wants to initialize the test platform with.
|
||||
2. The IDE adapter then sends either a TestExecution.RunTestsWithContainersOnDefaultHost (or) TestExecution.RunTestsWithTestsOnDefaultHost with a [TestRunRequestPayload](https://github.com/Microsoft/vstest/blob/master/src/Microsoft.TestPlatform.VsTestConsole.TranslationLayer/Payloads/TestRunRequestPayload.cs) that contains containers (or) tests respectively.
|
||||
2. The IDE adapter then sends either a TestExecution.RunTestsWithContainersOnDefaultHost (or) TestExecution.RunTestsWithTestsOnDefaultHost with a [TestRunRequestPayload](https://github.com/Microsoft/vstest/blob/main/src/Microsoft.TestPlatform.VsTestConsole.TranslationLayer/Payloads/TestRunRequestPayload.cs) that contains containers (or) tests respectively.
|
||||
3. dotnet-test would then spawn an instance of the default test host process for the configuration specified.
|
||||
4. dotnet-test then sends across the extensions if the IDE adapter has specified any, followed by the containers/test cases to run tests on.
|
||||
5. The test host then invokes the Rocksteady adapter with the containers/tests to run via the ITestExeuctor.RunTests entry point.
|
||||
6. The Rocksteady adapter runs each test and calls back to the test host when a test starts, when a test result is received and when a test ends.
|
||||
7. The test host process maintains a cache of the test results received above.
|
||||
8. When the cache size is met the test host process sends a TestExecution.TestResults with a [TestRunStatsPayload](https://github.com/Microsoft/vstest/blob/master/src/Microsoft.TestPlatform.CommunicationUtilities/Messages/TestRunStatsPayload.cs), which contains the cached results in a batch, to dotnet-test which then forwards it to the IDE adapater.
|
||||
9. Optionally the IDE adapter may also receive a TestSession.Message with a [TestMessagePayload](https://github.com/Microsoft/vstest/blob/master/src/Microsoft.TestPlatform.CommunicationUtilities/Messages/TestMessagePayload.cs) notifying of any issues during execution.
|
||||
10. When execution is complete, the test host process sends a TestExecution.Completed message to dotnet-test with a [TestRunCompletePayload](https://github.com/Microsoft/vstest/blob/master/src/Microsoft.TestPlatform.CommunicationUtilities/Messages/TestRunCompletePayload.cs) which contains the run statistics along with the results for the last set of tests run.
|
||||
8. When the cache size is met the test host process sends a TestExecution.TestResults with a [TestRunStatsPayload](https://github.com/Microsoft/vstest/blob/main/src/Microsoft.TestPlatform.CommunicationUtilities/Messages/TestRunStatsPayload.cs), which contains the cached results in a batch, to dotnet-test which then forwards it to the IDE adapater.
|
||||
9. Optionally the IDE adapter may also receive a TestSession.Message with a [TestMessagePayload](https://github.com/Microsoft/vstest/blob/main/src/Microsoft.TestPlatform.CommunicationUtilities/Messages/TestMessagePayload.cs) notifying of any issues during execution.
|
||||
10. When execution is complete, the test host process sends a TestExecution.Completed message to dotnet-test with a [TestRunCompletePayload](https://github.com/Microsoft/vstest/blob/main/src/Microsoft.TestPlatform.CommunicationUtilities/Messages/TestRunCompletePayload.cs) which contains the run statistics along with the results for the last set of tests run.
|
||||
11. dotnet-test then forwards this message to the IDE adapter.
|
||||
12. Once the IDE adapter is done, it sends dotnet-test a TestSession.Terminate which will cause dotnet-test to shutdown.
|
||||
|
||||
### New Flow for Debug Tests/Custom test host process:
|
||||
This flow would have the same protocol to bootstrap the test platform which is essentially #1 in the default flow above. Here is the sequence of steps after the initialization phase:
|
||||
|
||||
1. The IDE adapter sends a TestExecution.GetProcessStartInfo with a [TestRunRequestPayload](https://github.com/Microsoft/vstest/blob/master/src/Microsoft.TestPlatform.VsTestConsole.TranslationLayer/Payloads/TestRunRequestPayload.cs) that contains the container/tests it wants to execute.
|
||||
2. dotnet-test then reverts back to the IDE adapter with a TestExecution.CustomTestHostLaunch with a [ProcessStartInfo](https://github.com/Microsoft/vstest/blob/master/src/Microsoft.TestPlatform.ObjectModel/TestProcessStartInfo.cs) payload which has the process start info of the default test host. As part of the arguments, we send a port that the test host should connect to communicate with dotnet-test.
|
||||
1. The IDE adapter sends a TestExecution.GetProcessStartInfo with a [TestRunRequestPayload](https://github.com/Microsoft/vstest/blob/main/src/Microsoft.TestPlatform.VsTestConsole.TranslationLayer/Payloads/TestRunRequestPayload.cs) that contains the container/tests it wants to execute.
|
||||
2. dotnet-test then reverts back to the IDE adapter with a TestExecution.CustomTestHostLaunch with a [ProcessStartInfo](https://github.com/Microsoft/vstest/blob/main/src/Microsoft.TestPlatform.ObjectModel/TestProcessStartInfo.cs) payload which has the process start info of the default test host. As part of the arguments, we send a port that the test host should connect to communicate with dotnet-test.
|
||||
3. At this point, the adapter can launch the test host process (and attach to it for debugging if it chooses to).
|
||||
4. Once the test host starts, it sends dotnet-test a handshake TestHost.Connected message that indicates it is ready to receive commands.
|
||||
|
||||
|
|
|
|||
|
|
@ -153,7 +153,7 @@ An attachment here, could be any file that the adapter generates during the run
|
|||
Data driven test cases can have multiple test results, one against each data value. In that case adapters would post a RecordStart, RecordEnd and RecordResult for each data value. If the TestResult objects posted for all the data values are associated with the same TestCase object, then the VS IDE client would create a single entry in the Test Explorer with the aggregated outcome along with a summary for each test case in a separate view. If the TestResult objects posted for all the data values are associated with their own TestCase objects with different ID's, then each test result would be different entries in the VS IDE Test Explorer.
|
||||
|
||||
#### Filtering
|
||||
The user might optionally also pass in a test case filter to run a subset of tests. In such cases along with invoking the adapters with the container set, the test platform also provides a callback in the run context instance in the form of [GetTestCaseFilter](https://github.com/Microsoft/vstest/blob/master/src/Microsoft.TestPlatform.ObjectModel/Adapter/Interfaces/IRunContext.cs#L38) that enables adapters to filter a [TestCase](https://github.com/Microsoft/vstest/blob/master/src/Microsoft.TestPlatform.ObjectModel/TestCase.cs) object. The adapters can query this API with the set of properties it supports filtering on, along with a provider of TestProperty instances for each property. From the filter expression returned by this API, the adapter would then just have to perform a [ITestCaseFilterExpression.MatchTestCase](https://github.com/Microsoft/vstest/blob/master/src/Microsoft.TestPlatform.ObjectModel/Adapter/Interfaces/ITestCaseFilterExpression.cs#L20) on each test case which returns false if the test case has been filtered out. Below is sample code that demonstrates filtering:
|
||||
The user might optionally also pass in a test case filter to run a subset of tests. In such cases along with invoking the adapters with the container set, the test platform also provides a callback in the run context instance in the form of [GetTestCaseFilter](https://github.com/Microsoft/vstest/blob/main/src/Microsoft.TestPlatform.ObjectModel/Adapter/Interfaces/IRunContext.cs#L38) that enables adapters to filter a [TestCase](https://github.com/Microsoft/vstest/blob/main/src/Microsoft.TestPlatform.ObjectModel/TestCase.cs) object. The adapters can query this API with the set of properties it supports filtering on, along with a provider of TestProperty instances for each property. From the filter expression returned by this API, the adapter would then just have to perform a [ITestCaseFilterExpression.MatchTestCase](https://github.com/Microsoft/vstest/blob/main/src/Microsoft.TestPlatform.ObjectModel/Adapter/Interfaces/ITestCaseFilterExpression.cs#L20) on each test case which returns false if the test case has been filtered out. Below is sample code that demonstrates filtering:
|
||||
|
||||
```csharp
|
||||
static readonly TestProperty PriorityProperty = TestProperty.Register(
|
||||
|
|
@ -187,7 +187,7 @@ if (filterExpression != null &&
|
|||
```
|
||||
|
||||
#### TestCase Extensibility
|
||||
Adapters can choose to fill in custom data into the [TestCase](https://github.com/Microsoft/vstest/blob/master/src/Microsoft.TestPlatform.ObjectModel/TestCase.cs) object through its [Property](https://github.com/Microsoft/vstest/blob/master/src/Microsoft.TestPlatform.ObjectModel/TestObject.cs#L112) bag. This data can be:
|
||||
Adapters can choose to fill in custom data into the [TestCase](https://github.com/Microsoft/vstest/blob/main/src/Microsoft.TestPlatform.ObjectModel/TestCase.cs) object through its [Property](https://github.com/Microsoft/vstest/blob/main/src/Microsoft.TestPlatform.ObjectModel/TestObject.cs#L112) bag. This data can be:
|
||||
|
||||
1. If the test case is adorned by a "Do Not Run" attribute.
|
||||
2. If the test case is data driven, the values of the data to drive it with.
|
||||
|
|
|
|||
|
|
@ -55,7 +55,7 @@ Test platform allows integration with an editor using a JSON wire protocol. The
|
|||
#### SDK requirements
|
||||
There is no strict requirement for a authoring an editor integration in a particular language since the communication can be driven with JSON over the wire. At the moment, a `.NET` implementation is available. It is used by Visual Studio to drive test execution from the Test Explorer.
|
||||
|
||||
Developers can use `Microsoft.TestPlatform.TranslationLayer` nuget package. It is a *redistributable* package. It is available from vstest myget feed: [https://dotnet.myget.org/F/vstest/api/v3/index.json](https://dotnet.myget.org/F/vstest/api/v3/index.json). Sample application is [here](https://github.com/Microsoft/vstest/tree/master/samples/Microsoft.TestPlatform.TranslationLayer.E2ETest)
|
||||
Developers can use `Microsoft.TestPlatform.TranslationLayer` nuget package. It is a *redistributable* package. It is available from vstest myget feed: [https://dotnet.myget.org/F/vstest/api/v3/index.json](https://dotnet.myget.org/F/vstest/api/v3/index.json). Sample application is [here](https://github.com/Microsoft/vstest/tree/main/samples/Microsoft.TestPlatform.TranslationLayer.E2ETest)
|
||||
|
||||
| Version | .NET runtime |
|
||||
|-----------|-----------------------------------|
|
||||
|
|
|
|||
|
|
@ -1587,4 +1587,4 @@ of the target framework. For example, ".NETFramework,Version=v4.5.2". Other supp
|
|||
|
||||
[discovery]: 0002-Test-Discovery-Protocol.md
|
||||
[execution]: 0003-Test-Execution-Protocol.md
|
||||
[sample]: https://github.com/Microsoft/vstest/tree/master/samples/Microsoft.TestPlatform.Protocol
|
||||
[sample]: https://github.com/Microsoft/vstest/tree/main/samples/Microsoft.TestPlatform.Protocol
|
||||
|
|
|
|||
|
|
@ -14,7 +14,7 @@ TranslationLayer provides simple APIs that can be invoked to get the desired res
|
|||
|
||||
## Sample
|
||||
There is a sample project available for reference. Please find the relevant links below.
|
||||
* Sample project : [https://github.com/Microsoft/vstest/tree/master/samples/Microsoft.TestPlatform.TranslationLayer.E2ETest](https://github.com/Microsoft/vstest/tree/master/samples/Microsoft.TestPlatform.TranslationLayer.E2ETest)
|
||||
* Sample project : [https://github.com/Microsoft/vstest/tree/main/samples/Microsoft.TestPlatform.TranslationLayer.E2ETest](https://github.com/Microsoft/vstest/tree/main/samples/Microsoft.TestPlatform.TranslationLayer.E2ETest)
|
||||
* Nuget package : [https://www.nuget.org/packages/Microsoft.TestPlatform.TranslationLayer](https://www.nuget.org/packages/Microsoft.TestPlatform.TranslationLayer)
|
||||
|
||||
## Details
|
||||
|
|
|
|||
|
|
@ -5,7 +5,7 @@ This note outlines the proposed changes for the JSON protocol between an Editor/
|
|||
and the test platform.
|
||||
|
||||
|
||||
Here is the link to v1 specs: [Editors-API-Specification](https://github.com/Microsoft/vstest-docs/blob/master/RFCs/0007-Editors-API-Specification.md)
|
||||
Here is the link to v1 specs: [Editors-API-Specification](https://github.com/Microsoft/vstest-docs/blob/main/RFCs/0007-Editors-API-Specification.md)
|
||||
|
||||
## Motivation
|
||||
Here are the key factors:
|
||||
|
|
|
|||
|
|
@ -7,8 +7,8 @@ and the test platform to support test case filtering.
|
|||
## Motivation
|
||||
The console runner has an ability to run a specific set of tests via the `/testcasefilter` argument. Editors require this functionality as well to run a subset of tests which have not yet been discovered. This document builds on top of the v2 protocol between the Editor/IDE and the test platform.
|
||||
|
||||
Here is the link to v1 specs: [Editors-API-Specification](https://github.com/Microsoft/vstest-docs/blob/master/RFCs/0007-Editors-API-Specification.md)
|
||||
and here is the link to the v2 specs: [Editors-API-Specification-V2](https://github.com/Microsoft/vstest-docs/blob/master/RFCs/0009-Editors-API-RevisionUpdate.md)
|
||||
Here is the link to v1 specs: [Editors-API-Specification](https://github.com/Microsoft/vstest-docs/blob/main/RFCs/0007-Editors-API-Specification.md)
|
||||
and here is the link to the v2 specs: [Editors-API-Specification-V2](https://github.com/Microsoft/vstest-docs/blob/main/RFCs/0009-Editors-API-RevisionUpdate.md)
|
||||
|
||||
## Design
|
||||
Filtering in the test platform is only supported when a test run request with sources is requested. This is unsupported when a run request with a set of TestCases is passed in. To that effect the following messages would change to accommodate filtering.
|
||||
|
|
|
|||
|
|
@ -17,7 +17,7 @@ Following telemetry data points will be collected by vstest and also enabled for
|
|||
For more details please refer to the **Detailed List of Data Points** section at the end
|
||||
|
||||
## Conceptual Flow
|
||||
|
||||
|
||||
|
||||
## Scope
|
||||
* vstest and vstest.console.exe will emit telemetry data points that any vstest consuming platform can listen to
|
||||
|
|
@ -77,9 +77,9 @@ The Microsoft distribution of vstest is licensed with the [Link](https://www.vis
|
|||
## Sending Consent for collecting Telemetry Metrics in Test Platform in Design Mode
|
||||
**Collection in Test Platform will only happen if TestPlatform receives consent from the consumers.**
|
||||
|
||||
The "CollectMetrics" field has been added in [TestPlatformOptions](https://github.com/Microsoft/vstest/blob/master/src/Microsoft.TestPlatform.ObjectModel/Client/TestPlatformOptions.cs) to send consent from Design Mode Scenarios(VS, VSCode etc) to TestPlatform. The users have to set "CollectMetrics" to "true" so that TestPlatform can collect Metrics and send it back to the client.
|
||||
The "CollectMetrics" field has been added in [TestPlatformOptions](https://github.com/Microsoft/vstest/blob/main/src/Microsoft.TestPlatform.ObjectModel/Client/TestPlatformOptions.cs) to send consent from Design Mode Scenarios(VS, VSCode etc) to TestPlatform. The users have to set "CollectMetrics" to "true" so that TestPlatform can collect Metrics and send it back to the client.
|
||||
|
||||
The [IVsTestConsoleWrapper](https://github.com/Microsoft/vstest/blob/master/src/Microsoft.TestPlatform.VsTestConsole.TranslationLayer/Interfaces/IVsTestConsoleWrapper.cs) and [IVsTestConsoleWrapperAsync](https://github.com/Microsoft/vstest/blob/master/src/Microsoft.TestPlatform.VsTestConsole.TranslationLayer/Interfaces/IVsTestConsoleWrapperAsync.cs) contains API's that support TestPlatformOptions. Users must use these API's to send consent to the TestPlatform for collecting Metrics.
|
||||
The [IVsTestConsoleWrapper](https://github.com/Microsoft/vstest/blob/main/src/Microsoft.TestPlatform.VsTestConsole.TranslationLayer/Interfaces/IVsTestConsoleWrapper.cs) and [IVsTestConsoleWrapperAsync](https://github.com/Microsoft/vstest/blob/main/src/Microsoft.TestPlatform.VsTestConsole.TranslationLayer/Interfaces/IVsTestConsoleWrapperAsync.cs) contains API's that support TestPlatformOptions. Users must use these API's to send consent to the TestPlatform for collecting Metrics.
|
||||
|
||||
**For Instance**
|
||||
For Discovery : Users have to use this API to pass TestPlatform Options.
|
||||
|
|
@ -107,11 +107,11 @@ For Execution using sources: Users have to use this API to pass TestPlatform Opt
|
|||
Similarly, while running tests using TestCases, there are API's available to send TestPlatformOptions along with them.
|
||||
|
||||
## Consuming Metrics in Design Mode Scenarios from Test Platform
|
||||
The whole aggregated metrics will be appended in vstest.console process in the final Execution/Disocvery Complete message and it will be send back to the consumers. When users use API's that are available in [IVSTestConsoleWrapper](https://github.com/Microsoft/vstest/blob/master/src/Microsoft.TestPlatform.VsTestConsole.TranslationLayer/Interfaces/IVsTestConsoleWrapper.cs), they have to pass the event handler which will contain metrics from TestPlatform.
|
||||
The whole aggregated metrics will be appended in vstest.console process in the final Execution/Disocvery Complete message and it will be send back to the consumers. When users use API's that are available in [IVSTestConsoleWrapper](https://github.com/Microsoft/vstest/blob/main/src/Microsoft.TestPlatform.VsTestConsole.TranslationLayer/Interfaces/IVsTestConsoleWrapper.cs), they have to pass the event handler which will contain metrics from TestPlatform.
|
||||
|
||||
In Case of Execution, users will pass [ITestRunEventsHandler](https://github.com/Microsoft/vstest/blob/master/src/Microsoft.TestPlatform.ObjectModel/Client/Interfaces/ITestRunEventsHandler.cs), which contains [TestRunCompleteEventArgs](https://github.com/Microsoft/vstest/blob/master/src/Microsoft.TestPlatform.ObjectModel/Client/Events/TestRunCompleteEventArgs.cs) which will have Metrics in it which users can consume.
|
||||
In Case of Execution, users will pass [ITestRunEventsHandler](https://github.com/Microsoft/vstest/blob/main/src/Microsoft.TestPlatform.ObjectModel/Client/Interfaces/ITestRunEventsHandler.cs), which contains [TestRunCompleteEventArgs](https://github.com/Microsoft/vstest/blob/main/src/Microsoft.TestPlatform.ObjectModel/Client/Events/TestRunCompleteEventArgs.cs) which will have Metrics in it which users can consume.
|
||||
|
||||
In Case of Discovery, users have to pass new interface [ITestDiscoveryEventsHandler2](https://github.com/Microsoft/vstest/blob/master/src/Microsoft.TestPlatform.ObjectModel/Client/Interfaces/ITestDiscoveryEventsHandler2.cs) which contains [DiscoveryCompleteEventArgs](https://github.com/Microsoft/vstest/blob/master/src/Microsoft.TestPlatform.ObjectModel/Client/Events/DiscoveryCompleteEventArgs.cs) which will have metrics in it which users can consume.
|
||||
In Case of Discovery, users have to pass new interface [ITestDiscoveryEventsHandler2](https://github.com/Microsoft/vstest/blob/main/src/Microsoft.TestPlatform.ObjectModel/Client/Interfaces/ITestDiscoveryEventsHandler2.cs) which contains [DiscoveryCompleteEventArgs](https://github.com/Microsoft/vstest/blob/main/src/Microsoft.TestPlatform.ObjectModel/Client/Events/DiscoveryCompleteEventArgs.cs) which will have metrics in it which users can consume.
|
||||
|
||||
## Telemetry Design
|
||||
### Collecting Data
|
||||
|
|
@ -129,10 +129,10 @@ We have to collect Total Discovery Time taken, Total Tests Run in case of parall
|
|||
* It may happen that TestHost is on a newer version whereas vstest.console is on older version. So, TestHost process should not collect Metrics by default. So, it should only collect Metrics when users give consent to collect Metrics.
|
||||
So, for sending consent from Vstest.console process to Test Host, Command line argument i.e. boolean **TelemetryOptedIn** is sent to TestHost Process from Vstest.console process.
|
||||
|
||||
The interface [IRequestData](https://github.com/Microsoft/vstest/blob/master/src/Microsoft.TestPlatform.ObjectModel/Client/Interfaces/IRequestData.cs) has been has been exposed to TestHost process which contains [IMetricCollection](https://github.com/Microsoft/vstest/blob/master/src/Microsoft.TestPlatform.ObjectModel/Client/Interfaces/IMetricsCollection.cs) which will collect the Metrics in a dictionary.
|
||||
The interface [IRequestData](https://github.com/Microsoft/vstest/blob/main/src/Microsoft.TestPlatform.ObjectModel/Client/Interfaces/IRequestData.cs) has been has been exposed to TestHost process which contains [IMetricCollection](https://github.com/Microsoft/vstest/blob/main/src/Microsoft.TestPlatform.ObjectModel/Client/Interfaces/IMetricsCollection.cs) which will collect the Metrics in a dictionary.
|
||||
|
||||
#### Sending Metrics from TestHost process to vstest.console process
|
||||
Currently, At the end of Discovery Complete, we are sending **TestDiscovery.Complete** message event along with DiscoveryCompletePayload, so we will add our collected Metrics along with [DiscoveryCompletePayload](https://github.com/Microsoft/vstest/blob/master/src/Microsoft.TestPlatform.CommunicationUtilities/Messages/DiscoveryCompletePayload.cs). Similar will be done at end of **TestExecution.Complete** event where we will add the Metrics in [TestRunCompleteEventArgs](https://github.com/Microsoft/vstest/blob/master/src/Microsoft.TestPlatform.ObjectModel/Client/Events/TestRunCompleteEventArgs.cs). This helps us in sending whole metrics in one go and helps us to decrease performance overhead of sending messages from test host process to vstest.console process.
|
||||
Currently, At the end of Discovery Complete, we are sending **TestDiscovery.Complete** message event along with DiscoveryCompletePayload, so we will add our collected Metrics along with [DiscoveryCompletePayload](https://github.com/Microsoft/vstest/blob/main/src/Microsoft.TestPlatform.CommunicationUtilities/Messages/DiscoveryCompletePayload.cs). Similar will be done at end of **TestExecution.Complete** event where we will add the Metrics in [TestRunCompleteEventArgs](https://github.com/Microsoft/vstest/blob/main/src/Microsoft.TestPlatform.ObjectModel/Client/Events/TestRunCompleteEventArgs.cs). This helps us in sending whole metrics in one go and helps us to decrease performance overhead of sending messages from test host process to vstest.console process.
|
||||
|
||||
In VsTestConsole process, the metrics which will be received from TestHost process will be aggregated with the VsTestConsole own metrics.
|
||||
|
||||
|
|
|
|||
|
|
@ -32,7 +32,7 @@ Below is one of the analysis done for one of the crash dump while calling Appdom
|
|||
Proposed guidelines are for customers and test adapters who wants to avoid these issues.
|
||||
|
||||
**Adapters**
|
||||
1. Test Adapter should honour ```<DisableAppDomain>``` setting inside RunConfiguration node of runsettings. Check https://github.com/Microsoft/vstest-docs/blob/master/docs/configure.md for information on this setting. This will ensure that adapters dont create AppDomain at all to run tests
|
||||
1. Test Adapter should honour ```<DisableAppDomain>``` setting inside RunConfiguration node of runsettings. Check https://github.com/Microsoft/vstest-docs/blob/main/docs/configure.md for information on this setting. This will ensure that adapters dont create AppDomain at all to run tests
|
||||
|
||||
**Test platform**
|
||||
|
||||
|
|
|
|||
|
|
@ -15,11 +15,11 @@ This design will be detailed through the following sections:
|
|||
3. Choosing appropriate TestHost
|
||||
|
||||
### RunTime Provider Contract
|
||||
A Test RunTime provider will implement [ITestRunTimeProvider](https://github.com/Microsoft/vstest/blob/master/src/Microsoft.TestPlatform.ObjectModel/Host/ITestRunTimeProvider.cs#L18)
|
||||
A Test RunTime provider will implement [ITestRunTimeProvider](https://github.com/Microsoft/vstest/blob/main/src/Microsoft.TestPlatform.ObjectModel/Host/ITestRunTimeProvider.cs#L18)
|
||||
|
||||
ITestRunTimeProvider provides set of API's needed for any Test RunTime provider to be able to deploy, & launch the Test RunTime.
|
||||
|
||||
Interface [ITestHostLauncher](https://github.com/Microsoft/vstest/blob/master/src/Microsoft.TestPlatform.ObjectModel/Client/Interfaces/ITestHostLauncher.cs) provides an Extension point for IDE's to start TestHost process themselves. For example, in case of debugging/profiling test, the VS IDE would need to launch the TestHost itself.
|
||||
Interface [ITestHostLauncher](https://github.com/Microsoft/vstest/blob/main/src/Microsoft.TestPlatform.ObjectModel/Client/Interfaces/ITestHostLauncher.cs) provides an Extension point for IDE's to start TestHost process themselves. For example, in case of debugging/profiling test, the VS IDE would need to launch the TestHost itself.
|
||||
The interfaces would be part of Object Model.
|
||||
|
||||
### Discovering TestHost
|
||||
|
|
|
|||
|
|
@ -6,7 +6,7 @@ Introduce APIs to improve debugging support in Visual Studio for tests that run
|
|||
# Motivation
|
||||
Some test frameworks (examples include [TAEF](https://docs.microsoft.com/en-us/windows-hardware/drivers/taef/) and [Python](https://docs.microsoft.com/en-us/visualstudio/python/unit-testing-python-in-visual-studio)) need to execute tests in a external process other than `testhost*.exe`. When it comes to debugging such tests in Visual Studio today, there are a couple of problems.
|
||||
|
||||
1. A test adapter can request to launch a child process with debugger attached by calling [`IFrameworkHandle.LaunchProcessWithDebuggerAttached()`](https://github.com/microsoft/vstest/blob/master/src/Microsoft.TestPlatform.ObjectModel/Adapter/Interfaces/IFrameworkHandle.cs#L29) within adapter's implementation of [`ITestExecutor.RunTests()`](https://github.com/microsoft/vstest/blob/master/src/Microsoft.TestPlatform.ObjectModel/Adapter/Interfaces/ITestExecutor.cs#L23) (after checking that [`IRunContext.IsBeingDebugged`](https://github.com/microsoft/vstest/blob/master/src/Microsoft.TestPlatform.ObjectModel/Adapter/Interfaces/IRunContext.cs#L32) is `true`). However, there is no supported way for a test adapter to request that debugger should be attached to an **already running process**.
|
||||
1. A test adapter can request to launch a child process with debugger attached by calling [`IFrameworkHandle.LaunchProcessWithDebuggerAttached()`](https://github.com/microsoft/vstest/blob/main/src/Microsoft.TestPlatform.ObjectModel/Adapter/Interfaces/IFrameworkHandle.cs#L29) within adapter's implementation of [`ITestExecutor.RunTests()`](https://github.com/microsoft/vstest/blob/main/src/Microsoft.TestPlatform.ObjectModel/Adapter/Interfaces/ITestExecutor.cs#L23) (after checking that [`IRunContext.IsBeingDebugged`](https://github.com/microsoft/vstest/blob/main/src/Microsoft.TestPlatform.ObjectModel/Adapter/Interfaces/IRunContext.cs#L32) is `true`). However, there is no supported way for a test adapter to request that debugger should be attached to an **already running process**.
|
||||
|
||||
2. Even though a test adapter can launch a child process with debugger attached as described above, today the debugger is always attached to the `testhost*.exe` process as well. This means that **Visual Studio ends up debugging two processes** instead of just the single process where tests are running.
|
||||
|
||||
|
|
@ -15,7 +15,7 @@ Debugging of Python tests is supported in VS today. However, the Python adapter
|
|||
While this works for Python, not all test frameworks that need to support debugging of tests running in external processes would want their users to also install a VS extension. For this reason, debugging of TAEF tests is currently not supported in VS.
|
||||
|
||||
# Proposed Changes
|
||||
1. Introduce a new `IFrameworkHandle2` interface that inherits [`IFrameworkHandle`](https://github.com/microsoft/vstest/blob/master/src/Microsoft.TestPlatform.ObjectModel/Adapter/Interfaces/IFrameworkHandle.cs#L12) and adds the following `AttachDebuggerToProcess()` API that an adapter can invoke from within [`ITestExecutor.RunTests()`](https://github.com/microsoft/vstest/blob/master/src/Microsoft.TestPlatform.ObjectModel/Adapter/Interfaces/ITestExecutor.cs#L23) to attach debugger to an already running process.
|
||||
1. Introduce a new `IFrameworkHandle2` interface that inherits [`IFrameworkHandle`](https://github.com/microsoft/vstest/blob/main/src/Microsoft.TestPlatform.ObjectModel/Adapter/Interfaces/IFrameworkHandle.cs#L12) and adds the following `AttachDebuggerToProcess()` API that an adapter can invoke from within [`ITestExecutor.RunTests()`](https://github.com/microsoft/vstest/blob/main/src/Microsoft.TestPlatform.ObjectModel/Adapter/Interfaces/ITestExecutor.cs#L23) to attach debugger to an already running process.
|
||||
|
||||
```
|
||||
namespace Microsoft.VisualStudio.TestPlatform.ObjectModel.Adapter
|
||||
|
|
@ -50,7 +50,7 @@ void ITestExecutor.RunTests(IEnumerable<TestCase> tests, IRunContext runContext,
|
|||
}
|
||||
```
|
||||
|
||||
2. Introduce a new `ITestExecutor2` interface that inherits [`ITestExecutor`](https://github.com/microsoft/vstest/blob/master/src/Microsoft.TestPlatform.ObjectModel/Adapter/Interfaces/ITestExecutor.cs#L15) and adds the following `ShouldAttachToTestHost()` API. Newer adapters can choose to implement `ITestExecutor2` instead of `ITestExecutor`. If implemented, `ITestExecutor.ShouldAttachToTestHost()` will be invoked before `ITestExecutor.RunTests()` is invoked for any test execution. `ITestExecutor2.ShouldAttachToTestHost()` will also be supplied the same set of inputs as `ITestExecutor.RunTests()`. This would allow adapters to control whether or not the debugger should be attached to `testhost*.exe` for the subsequent invocation of `ITestExecutor.RunTests()`.
|
||||
2. Introduce a new `ITestExecutor2` interface that inherits [`ITestExecutor`](https://github.com/microsoft/vstest/blob/main/src/Microsoft.TestPlatform.ObjectModel/Adapter/Interfaces/ITestExecutor.cs#L15) and adds the following `ShouldAttachToTestHost()` API. Newer adapters can choose to implement `ITestExecutor2` instead of `ITestExecutor`. If implemented, `ITestExecutor.ShouldAttachToTestHost()` will be invoked before `ITestExecutor.RunTests()` is invoked for any test execution. `ITestExecutor2.ShouldAttachToTestHost()` will also be supplied the same set of inputs as `ITestExecutor.RunTests()`. This would allow adapters to control whether or not the debugger should be attached to `testhost*.exe` for the subsequent invocation of `ITestExecutor.RunTests()`.
|
||||
|
||||
```
|
||||
namespace Microsoft.VisualStudio.TestPlatform.ObjectModel.Adapter
|
||||
|
|
@ -83,7 +83,7 @@ namespace Microsoft.VisualStudio.TestPlatform.ObjectModel.Adapter
|
|||
}
|
||||
```
|
||||
|
||||
3. Introduce a new `ITestHostLauncher2` interface that inherits [`ITestHostLauncher`](https://github.com/microsoft/vstest/blob/master/src/Microsoft.TestPlatform.ObjectModel/Client/Interfaces/ITestHostLauncher.cs#L11) and adds the following `AttachToProcess()` API. Visual Studio's Test Explorer will supply an implementation of this interface via [`IVsTestConsoleWrapper.RunTestsWithCustomTestHost()`](https://github.com/microsoft/vstest/blob/master/src/Microsoft.TestPlatform.VsTestConsole.TranslationLayer/Interfaces/IVsTestConsoleWrapper.cs#L120) and `ITestHostLauncher2.AttachToProcess()` will be called when `IFrameworkHandle2.AttachDebuggerToProcess()` is called within an adapter.
|
||||
3. Introduce a new `ITestHostLauncher2` interface that inherits [`ITestHostLauncher`](https://github.com/microsoft/vstest/blob/main/src/Microsoft.TestPlatform.ObjectModel/Client/Interfaces/ITestHostLauncher.cs#L11) and adds the following `AttachToProcess()` API. Visual Studio's Test Explorer will supply an implementation of this interface via [`IVsTestConsoleWrapper.RunTestsWithCustomTestHost()`](https://github.com/microsoft/vstest/blob/main/src/Microsoft.TestPlatform.VsTestConsole.TranslationLayer/Interfaces/IVsTestConsoleWrapper.cs#L120) and `ITestHostLauncher2.AttachToProcess()` will be called when `IFrameworkHandle2.AttachDebuggerToProcess()` is called within an adapter.
|
||||
|
||||
```
|
||||
namespace Microsoft.VisualStudio.TestPlatform.ObjectModel.Client.Interfaces
|
||||
|
|
|
|||
|
|
@ -61,7 +61,7 @@ By default `SupportsIncrementalProcessing` should be `False`, unless processing
|
|||
|
||||
|
||||
|
||||
2. Introduce a new `ProcessTestRunAttachmentsAsync` method in [IVsTestConsoleWrapper](https://github.com/microsoft/vstest/blob/master/src/Microsoft.TestPlatform.VsTestConsole.TranslationLayer/Interfaces/IVsTestConsoleWrapper.cs) interface:
|
||||
2. Introduce a new `ProcessTestRunAttachmentsAsync` method in [IVsTestConsoleWrapper](https://github.com/microsoft/vstest/blob/main/src/Microsoft.TestPlatform.VsTestConsole.TranslationLayer/Interfaces/IVsTestConsoleWrapper.cs) interface:
|
||||
|
||||
```
|
||||
/// <summary>
|
||||
|
|
|
|||
|
|
@ -21,7 +21,7 @@ A DataCollector is a test platform extension to monitor test run. It can be exte
|
|||
|
||||
You can author a DataCollector to collect code coverage data for a test run, to collect logs when a test case or test run fails, etc. These additional files are called Attachments and they can be attached to test result report(trx).
|
||||
|
||||
Please refer [here](https://github.com/Microsoft/vstest-docs/blob/master/docs/extensions/datacollector.md) for instructions on creating a DataCollector and [here](https://github.com/Microsoft/vstest-docs/blob/master/RFCs/0006-DataCollection-Protocol.md)
|
||||
Please refer [here](https://github.com/Microsoft/vstest-docs/blob/main/docs/extensions/datacollector.md) for instructions on creating a DataCollector and [here](https://github.com/Microsoft/vstest-docs/blob/main/RFCs/0006-DataCollection-Protocol.md)
|
||||
if you're interested in the architecture of data collection.
|
||||
|
||||
## Configure DataCollectors
|
||||
|
|
|
|||
|
|
@ -372,7 +372,7 @@ This section allows users to configure settings that were earlier required to be
|
|||
set using *testsettings file.
|
||||
These settings can be used for Ordered tests and MSTest v1 based tests when running in legacy mode.
|
||||
|
||||
Users can now migrate testsettings to runsettings using [SettingsMigrator](https://github.com/Microsoft/vstest-docs/blob/master/RFCs/0023-TestSettings-Deprecation.md#migration)
|
||||
Users can now migrate testsettings to runsettings using [SettingsMigrator](https://github.com/Microsoft/vstest-docs/blob/main/RFCs/0023-TestSettings-Deprecation.md#migration)
|
||||
|
||||
Here is an example:
|
||||
```xml
|
||||
|
|
|
|||
|
|
@ -7,7 +7,7 @@ Previously, `DataCollector` abstract class was present in `Microsoft.VisualStudi
|
|||
|
||||
Now, `DataCollector`abstract class is present in Object Model. Add reference to [`Microsoft.TestPlatform.ObjectModel`](https://www.nuget.org/packages/Microsoft.TestPlatform.ObjectModel/15.5.0-preview-20170810-02) (preview) nuget package. DataCollector APIs are present under namespace `Microsoft.VisualStudio.TestPlatform.ObjectModel.DataCollection`.
|
||||
It is recommended to target your DataCollector to netstandard, so that it can also run cross-platform, i.e. on non-Windows operating systems.
|
||||
For more info, refer to this [guide](https://github.com/Microsoft/vstest-docs/blob/master/docs/extensions/datacollector.md).
|
||||
For more info, refer to this [guide](https://github.com/Microsoft/vstest-docs/blob/main/docs/extensions/datacollector.md).
|
||||
|
||||
## Deprecated Attributes
|
||||
Currently `DataCollectorFriendlyNameAttribute` and `DataCollectorTypeUriAttribute` are the only attributes that are supported.
|
||||
|
|
@ -43,4 +43,4 @@ Following ten events that were exposed to DataCollectors through `DataCollection
|
|||
These events are no longer supported in TestPlatform and hence, these have been removed from DataCollection infrastructure as well.
|
||||
|
||||
## DataCollector RunSettings
|
||||
DataCollector RunSettings are highly compatible in all the versions of TestPlatform and old settings should continue to work with TestPlatform. For more info on runsettings, refer to [Configure DataCollectors](https://github.com/Microsoft/vstest-docs/blob/master/docs/analyze.md#configure-datacollectors).
|
||||
DataCollector RunSettings are highly compatible in all the versions of TestPlatform and old settings should continue to work with TestPlatform. For more info on runsettings, refer to [Configure DataCollectors](https://github.com/Microsoft/vstest-docs/blob/main/docs/analyze.md#configure-datacollectors).
|
||||
|
|
|
|||
|
|
@ -175,4 +175,4 @@ Attachments sent by DataCollectors will be part of Test Results and can also be
|
|||
Logs sent by DataCollectors will be displayed in console logger and can also be viewed in .trx report, if specified.
|
||||
|
||||
## Samples
|
||||
1. We have implemented a sample data collector [here](https://github.com/Microsoft/vstest/tree/master/test/TestAssets/OutOfProcDataCollector).
|
||||
1. We have implemented a sample data collector [here](https://github.com/Microsoft/vstest/tree/main/test/TestAssets/OutOfProcDataCollector).
|
||||
|
|
|
|||
|
|
@ -13,7 +13,7 @@ You can author a test logger to print messages on the console, generate result
|
|||
files of a specific reporting format, or even report results to various CI/CD
|
||||
services. Default inputs to a test logger can be provided in the command line.
|
||||
|
||||
Please refer to [this section](https://github.com/Microsoft/vstest-docs/blob/master/docs/report.md#create-a-test-logger) for instructions on creating a test logger and [todo]()
|
||||
Please refer to [this section](https://github.com/Microsoft/vstest-docs/blob/main/docs/report.md#create-a-test-logger) for instructions on creating a test logger and [todo]()
|
||||
if you're interested in the architecture of a test logger.
|
||||
|
||||
### Available test loggers
|
||||
|
|
@ -29,9 +29,9 @@ if you're interested in the architecture of a test logger.
|
|||
| Azure Pipelines | [AzurePipelines.TestLogger][azurepipelines.nuget] | [Azure Pipelines Logger][] |
|
||||
| TeamCity | [TeamCity.VSTest.TestAdapter][teamcity.nuget] | [Teamcity Logger][] |
|
||||
|
||||
[Trx Logger]: https://github.com/Microsoft/vstest/tree/master/src/Microsoft.TestPlatform.Extensions.TrxLogger
|
||||
[Html Logger]: https://github.com/Microsoft/vstest/tree/master/src/Microsoft.TestPlatform.Extensions.HtmlLogger
|
||||
[Console Logger]: https://github.com/Microsoft/vstest/blob/master/src/vstest.console/Internal/ConsoleLogger.cs
|
||||
[Trx Logger]: https://github.com/Microsoft/vstest/tree/main/src/Microsoft.TestPlatform.Extensions.TrxLogger
|
||||
[Html Logger]: https://github.com/Microsoft/vstest/tree/main/src/Microsoft.TestPlatform.Extensions.HtmlLogger
|
||||
[Console Logger]: https://github.com/Microsoft/vstest/blob/main/src/vstest.console/Internal/ConsoleLogger.cs
|
||||
[Xunit Logger]: https://github.com/spekt/xunit.testlogger
|
||||
[Nunit Logger]: https://github.com/spekt/nunit.testlogger
|
||||
[Junit Logger]: https://github.com/spekt/junit.testlogger
|
||||
|
|
@ -79,7 +79,7 @@ to load test loggers.
|
|||
Go through the following steps to create your own logger
|
||||
1) Add a nuget reference of package `Microsoft.TestPlatform.ObjectModel`.
|
||||
2) Implement ITestLoggerWithParameters (or ITestLogger, if your logger is not expecting any parameter). [Logger Example](https://github.com/spekt/xunit.testlogger/blob/master/src/Xunit.Xml.TestLogger/XunitXmlTestLogger.cs#L19)
|
||||
3) Name your logger assemlby `*.testlogger.dll`. [Detailed](https://github.com/Microsoft/vstest-docs/blob/master/docs/report.md#naming)
|
||||
3) Name your logger assemlby `*.testlogger.dll`. [Detailed](https://github.com/Microsoft/vstest-docs/blob/main/docs/report.md#naming)
|
||||
|
||||
## Enable a test logger
|
||||
A test logger must be explicitly enabled using the command line. E.g.
|
||||
|
|
|
|||
|
|
@ -1,5 +1,5 @@
|
|||
# Issue tracking
|
||||
The vstest project tracks issues and feature requests using the [issue template](https://github.com/Microsoft/vstest/blob/master/.github/ISSUE_TEMPLATE.md) for the vstest repository.
|
||||
The vstest project tracks issues and feature requests using the [issue template](https://github.com/Microsoft/vstest/blob/main/.github/ISSUE_TEMPLATE.md) for the vstest repository.
|
||||
|
||||
## Submitting an Issue
|
||||
First, please do a search in [issues](https://github.com/Microsoft/vstest/issues) to see if the issue or feature request has already been filed. Use this [query](https://github.com/Microsoft/vstest/issues?q=is%3Aopen+is%3Aissue+sort%3Areactions-%2B1-desc) to search for the most popular feature requests.
|
||||
|
|
@ -7,7 +7,7 @@ If you find your issue already exists, make relevant comments and add your [reac
|
|||
- 👍 - upvote
|
||||
- 👎 - downvote
|
||||
|
||||
If you cannot find an existing issue that describes your issue, submit it using the [issue template](https://github.com/Microsoft/vstest/blob/master/.github/ISSUE_TEMPLATE.md). Remember to follow the instruction mentioned therein carefully.
|
||||
If you cannot find an existing issue that describes your issue, submit it using the [issue template](https://github.com/Microsoft/vstest/blob/main/.github/ISSUE_TEMPLATE.md). Remember to follow the instruction mentioned therein carefully.
|
||||
|
||||
## Issue triage
|
||||
Please follow the guidelines in the issue template when filing an issue or a pull request.
|
||||
|
|
|
|||
|
|
@ -2,7 +2,7 @@ This repo is the modern, OSS, cross-plat testing engine that has been powering t
|
|||
|
||||
We aim to continuously deliver improvements that will ship with Visual Studio and with the .NET Tools SDK. These improvements are directly informed by your feedback filed as [issues](https://github.com/Microsoft/vstest/issues). If you do not see your issue addressed already, we will get to it soon! If you would like to help out, let us know!
|
||||
|
||||
Over the past several quarters, we have made many enhancements - from introducing support for Mono, to refactoring the platform to make it ready to support device testing, to performance improvements, to enabling robust C++ support, to improved documentation, and more. For a complete list see here: [Release Notes](https://github.com/Microsoft/vstest-docs/blob/master/docs/releases.md)
|
||||
Over the past several quarters, we have made many enhancements - from introducing support for Mono, to refactoring the platform to make it ready to support device testing, to performance improvements, to enabling robust C++ support, to improved documentation, and more. For a complete list see here: [Release Notes](https://github.com/Microsoft/vstest-docs/blob/main/docs/releases.md)
|
||||
|
||||
## Roadmap
|
||||
We typically plan for a quarter, and establish a set of themes we want to work towards. Here are the themes we will work on this quarter.
|
||||
|
|
|
|||
Загрузка…
Ссылка в новой задаче