microsoft-ui-xaml/tools
Keith Mahoney 4616a30b74
Enable WinUI tests in Helix (#83)
This is the initial check-in of work that enables running WinUI tests in Helix.

There are some things that are missing that will come in a later PR. I wanted to get this work merged to master sooner so that we can start getting things up and running.

Not in this PR:
* Scheduling tests on RS1 - RS5 machines is not yet enabled. Currently all tests are running on the Windows.10.Amd64.Open Helix queue which is a Windows Server 2016 RS1 queue.
* Release configuration and/or x64 target platform. Currently only x86/debug is enabled.
* Not all tests are enabled yet such as IXMP tests, WPF hosting tests, NuGet package tests, Framework Package tests.

All of the above will come soon after this PR is completed.

**Details:**

Helix is a cloud hosted test execution environment which is accessed via the Arcade SDK.
More details:
* [Arcade](https://github.com/dotnet/arcade)
* [Helix](https://github.com/dotnet/arcade/tree/master/src/Microsoft.DotNet.Helix/Sdk)

WinUI tests are scheduled in Helix by the Azure DevOps Pipeline: RunHelixTests.yml.

The workflow is as follows:
1. NuGet Restore is called on the packages.config in this directory. This downloads any runtime dependencies that are needed to run tests.
2. PrepareHelixPayload.ps1 is called. This copies the necessary files from various locations into a Helix payload directory. This directory is what will get sent to the Helix machines.
3. RunTestsInHelix.proj is executed. This proj has a dependency on [Microsoft.DotNet.Helix.Sdk](https://github.com/dotnet/arcade/tree/master/src/Microsoft.DotNet.Helix/Sdk) 
which it uses to publish the Helix payload directory and to schedule the Helix Work Items. The WinUI tests are parallelized into multiple Helix Work Items.
4. Each Helix Work Item calls runtests.cmd with a specific query to pass to [TAEF](https://docs.microsoft.com/en-us/windows-hardware/drivers/taef/) which runs the tests.
5. TAEF produces logs in WTT format. Helix is able to process logs in XUnit format. We run ConvertWttLogToXUnit.ps1 to convert the logs into the necessary format.
6. RunTestsInHelix.proj has EnableAzurePipelinesReporter set to true. This allows the XUnit formatted test results to be reported back to the Azure DevOps Pipeline.

Example run: https://dev.azure.com/ms/microsoft-ui-xaml/_build/results?buildId=730

Full Atlas test pass: https://microsoft.visualstudio.com/WinUI/_build/results?buildId=13389551&view=ms.vss-test-web.test-result-details
2018-12-14 17:14:48 -08:00
..
CodeCoverage Initial commit 2018-12-03 17:20:35 -08:00
CustomTasks Enable WinUI tests in Helix (#83) 2018-12-14 17:14:48 -08:00
CustomTasksTool Initial commit 2018-12-03 17:20:35 -08:00
GenerateNewControlProjectFiles Initial commit 2018-12-03 17:20:35 -08:00
PublishPackageToArcade Initial commit 2018-12-03 17:20:35 -08:00
PublishSymbols Initial commit 2018-12-03 17:20:35 -08:00
WindowsPorting Back-porting external changes from the OS repo. (#55) 2018-12-10 09:24:53 -08:00
AdalAuth.ps1 Initial commit 2018-12-03 17:20:35 -08:00
BuildMachineUtils.psm1 Initial commit 2018-12-03 17:20:35 -08:00
CheckCompatibility.ps1 Initial commit 2018-12-03 17:20:35 -08:00
ExtractControls.ps1 Initial commit 2018-12-03 17:20:35 -08:00
ExtractPackageDependencies.ps1 Initial commit 2018-12-03 17:20:35 -08:00
GenerateAppxDependenciesXML.cmd Initial commit 2018-12-03 17:20:35 -08:00
GenerateAppxDependenciesXML.ps1 Initial commit 2018-12-03 17:20:35 -08:00
GenerateMergedXaml.cmd Initial commit 2018-12-03 17:20:35 -08:00
GenerateMergedXaml.ps1 Initial commit 2018-12-03 17:20:35 -08:00
GenerateNewControlProject.cmd Initial commit 2018-12-03 17:20:35 -08:00
GenerateNewControlProject.ps1 Initial commit 2018-12-03 17:20:35 -08:00
GenerateNewProperty.ps1 Initial commit 2018-12-03 17:20:35 -08:00
GenerateSystemDllImageResources.ps1 Initial commit 2018-12-03 17:20:35 -08:00
GenerateSystemDllStringResources.ps1 Initial commit 2018-12-03 17:20:35 -08:00
InstallTestApp.Desktop.cmd Initial commit 2018-12-03 17:20:35 -08:00
InstallTestApp.Phone.cmd Initial commit 2018-12-03 17:20:35 -08:00
LaunchXde.cmd Initial commit 2018-12-03 17:20:35 -08:00
LaunchXde.ps1 Initial commit 2018-12-03 17:20:35 -08:00
MakeAppxHelper.cmd Update version to 2.1 (#75) 2018-12-13 15:09:26 -08:00
NugetWrapper.cmd Initial commit 2018-12-03 17:20:35 -08:00
PSProfile.ps1 Initial commit 2018-12-03 17:20:35 -08:00
PerformFullOSBuilds.ps1 Initial commit 2018-12-03 17:20:35 -08:00
PkgGen.cmd Update to the officially released version of Microsoft.Windows.Apps.Test. 2018-12-04 09:54:40 -08:00
PostBuild.cmd Initial commit 2018-12-03 17:20:35 -08:00
PostBuildWrapper.cmd Initial commit 2018-12-03 17:20:35 -08:00
PostTestsWrapper.cmd Initial commit 2018-12-03 17:20:35 -08:00
PowershellWrapper.cmd Initial commit 2018-12-03 17:20:35 -08:00
ProcessTextTemplate.ps1 Initial commit 2018-12-03 17:20:35 -08:00
PublishChunk.cmd Initial commit 2018-12-03 17:20:35 -08:00
PublishPostBuild.cmd Initial commit 2018-12-03 17:20:35 -08:00
PublishPostBuildWrapper.cmd Initial commit 2018-12-03 17:20:35 -08:00
RebootBuildMachine.ps1 Initial commit 2018-12-03 17:20:35 -08:00
ReleaseHelper.ps1 Initial commit 2018-12-03 17:20:35 -08:00
StripConditionalXaml.ps1 Initial commit 2018-12-03 17:20:35 -08:00
UpdateBuildMachineSDKs.ps1 Initial commit 2018-12-03 17:20:35 -08:00
UpdateBuildMachineVSInstalls.ps1 Initial commit 2018-12-03 17:20:35 -08:00
Utils.psm1 Initial commit 2018-12-03 17:20:35 -08:00
VSTSRunLocWorkflow.cmd Initial commit 2018-12-03 17:20:35 -08:00
addaliases.cmd Initial commit 2018-12-03 17:20:35 -08:00
packages.pkggen.config Initial commit 2018-12-03 17:20:35 -08:00
ps.bat Initial commit 2018-12-03 17:20:35 -08:00
sign.cmd Initial commit 2018-12-03 17:20:35 -08:00
sources Initial commit 2018-12-03 17:20:35 -08:00