Integration test harness for Visual Studio extension development
Перейти к файлу
Sam Harwell 0b36151638
Merge pull request #171 from sharwell/api-scan
Switch APIScan to use a managed identity
2024-03-20 07:47:24 -05:00
.config Update template to match current repository 2023-11-14 21:28:23 -06:00
.devcontainer Apply template from AArnott/Library.Template@faaca53c 2023-11-14 19:14:06 -06:00
.github Update template to match current repository 2023-11-14 21:28:23 -06:00
.vscode Apply template from AArnott/Library.Template@faaca53c 2023-11-14 19:14:06 -06:00
azure-pipelines Switch APIScan to use a managed identity 2024-03-19 15:42:59 -05:00
src Dispose ExtensionManagerService between uninstall and install 2023-12-06 14:16:22 -06:00
test Improve test script error handling 2023-11-14 21:28:23 -06:00
tools Apply template from AArnott/Library.Template@faaca53c 2023-11-14 19:14:06 -06:00
.editorconfig Update template to match current repository 2023-11-14 21:28:23 -06:00
.gitattributes Apply template from AArnott/Library.Template@faaca53c 2023-11-14 19:14:06 -06:00
.gitignore Apply template from AArnott/Library.Template@faaca53c 2023-11-14 19:14:06 -06:00
CODE_OF_CONDUCT.md Copy configuration from AArnott/Library.Template@5459e34d 2021-07-13 19:57:15 -07:00
CONTRIBUTING.md Update build instructions following CI results 2021-07-15 08:51:36 -07:00
Directory.Build.props Update template to match current repository 2023-11-14 21:28:23 -06:00
Directory.Build.rsp Copy configuration from AArnott/Library.Template@5459e34d 2021-07-13 19:57:15 -07:00
Directory.Build.targets Apply template from AArnott/Library.Template@faaca53c 2023-11-14 19:14:06 -06:00
ExtensionTesting.sln Fix source generator package name 2021-12-29 10:17:31 -08:00
LICENSE Copy configuration from AArnott/Library.Template@5459e34d 2021-07-13 19:57:15 -07:00
NuGet.config Update NuGet feed to include archived feeds 2023-04-12 18:06:20 -05:00
README.md Generate test framework attribute 2021-08-03 09:10:34 -07:00
SECURITY.md Copy configuration from AArnott/Library.Template@5459e34d 2021-07-13 19:57:15 -07:00
SUPPORT.md Remove support template header 2021-07-15 11:02:34 -07:00
THIRD-PARTY-NOTICES.TXT Add IdeInstanceTestCase for easy launching and/or debugging 2021-12-15 08:33:24 -08:00
azure-pipelines.yml Update template to match current repository 2023-11-14 21:28:23 -06:00
global.json Update template to match current repository 2023-11-14 21:28:23 -06:00
init.cmd Copy configuration from AArnott/Library.Template@5459e34d 2021-07-13 19:57:15 -07:00
init.ps1 Apply template from AArnott/Library.Template@faaca53c 2023-11-14 19:14:06 -06:00
stylecop.json Store expected test outputs on disk for easy review 2021-12-31 07:28:31 -08:00
version.json Copy configuration from AArnott/Library.Template@5459e34d 2021-07-13 19:57:15 -07:00

README.md

Visual Studio Extension Testing

This project allows Visual Studio extension developers to write integration tests that run inside an experimental instance of Visual Studio.

License NuGet package

Installation and Use

Requirements

  • Extension development requires Visual Studio 2017 or newer. Version 15.7 or newer is recommended for the best Test Explorer experience.
  • Extensions themselves must target one or more versions of Visual Studio from the following list:
    • Visual Studio 2012
    • Visual Studio 2013
    • Visual Studio 2015
    • Visual Studio 2017
    • Visual Studio 2019
    • Visual Studio 2022
  • Extensions must be deployed via one or more VSIX packages.
  • Test execution and debugging is only supported for versions of Visual Studio available on the same machine as the development IDE.

Install the test harness

Install the test package for the applicable version(s) of Visual Studio

Visual Studio Version Integration Testing Package
2022 Microsoft.VisualStudio.Extensibility.Testing.Xunit
2012 - 2019 Microsoft.VisualStudio.Extensibility.Testing.Xunit.Legacy

Configure the test framework

Classic projects

Add the following to AssemblyInfo.cs to enable the test framework:

using Xunit;

[assembly: TestFramework("Xunit.Harness.IdeTestFramework", "Microsoft.VisualStudio.Extensibility.Testing.Xunit")]

SDK projects

💡 By default, SDK projects automatically generate the required assembly attribute. Manual customization is only required if the default assembly attributes support has been disabled, or in cases where the automatic application of TestFrameworkAttribute is not desired.

To disable generation of TestFrameworkAttribute (which will require manual addition similar to classic projects), add the following to the project file:

<PropertyGroup>
  <GenerateTestFrameworkAttribute>false</GenerateTestFrameworkAttribute>
</PropertyGroup>

Configure extensions for deployment

Add the following to AssemblyInfo.cs to deploy extensions required for testing.

using Xunit.Harness;

[assembly: RequireExtension("Extension.File.Name.vsix")]

Ensure test discovery is enabled

Test projects using a customized xUnit test framework cannot currently be discovered while tests are being written. The test discovery process that runs after a build completes will detect the required tests. Ensure this feature is enabled by the following steps:

  1. Open ToolsOptions...
  2. Select the Test page on the left
  3. Ensure Additionally discover tests from built assemblies after builds is checked

Tests will be automatically discovered and Test Explorer updated after each successful build.

Write tests

Apply the [IdeFact] attribute to tests that need to run in the IDE. After building the project, the tests will appear in Test Explorer where they can be launched for running and/or debugging directly.

Contributing

Please see CONTRIBUTING.md for information about our Code of Conduct and contributing guidelines.