[main] Update Docker images, queues, etc. (#38427)

- part of dotnet/aspnetcore-internal#3950
  - also touches on #36032
- update Helix queues from Alpine 3.12 to 3.14, OSX 10.14 to 10.15, and (for Arm64) Debian 9 to 11
  - use OSX 11.00 when testing PRs and rolling builds; reduce 10.15 usage to scheduled runs
  - remove overlap (all 3 queues) between PRs / rolling builds and scheduled runs
  - remove quarantined-pr / quarantined-tests overlap
- build source-index on `windows-latest` (not `vs2017-win2016`)
- update build and Helix Docker images to latest tags

- skip a test class on macOS 10.15
  - #38542

nits:
- don't skip unused Helix queues
- remove versions from pipeline job display names
  - some were already outdated; rest will be confusing in the future
- remove most comments about unused Helix queues
This commit is contained in:
Doug Bunting 2021-11-19 22:08:45 -08:00 коммит произвёл GitHub
Родитель 30a18721e2
Коммит 64c37113aa
Не найден ключ, соответствующий данной подписи
Идентификатор ключа GPG: 4AEE18F83AFDEB23
11 изменённых файлов: 52 добавлений и 65 удалений

Просмотреть файл

@ -10,7 +10,7 @@ jobs:
- template: jobs/default-build.yml
parameters:
jobName: Windows_Build
jobDisplayName: "Build only : Windows"
jobDisplayName: "Build: Windows"
agentOs: Windows
buildArgs: -all -pack
artifacts:

Просмотреть файл

@ -574,7 +574,7 @@ stages:
jobDisplayName: "Build: Linux Musl ARM"
agentOs: Linux
useHostedUbuntu: false
container: mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-18.04-cross-arm-alpine-20210409142425-044d5b9
container: mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-18.04-cross-arm-alpine-20211022152824-78f7860
buildArgs:
--arch arm
--os-name linux-musl
@ -608,7 +608,7 @@ stages:
jobDisplayName: "Build: Linux Musl ARM64"
agentOs: Linux
useHostedUbuntu: false
container: mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-18.04-cross-arm64-alpine-20210409142425-b2c2436
container: mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-18.04-cross-arm64-alpine-20211022152824-538077f
buildArgs:
--arch arm64
--os-name linux-musl
@ -640,7 +640,7 @@ stages:
- template: jobs/default-build.yml
parameters:
jobName: Windows_Test
jobDisplayName: "Test: Windows Server 2016 x64"
jobDisplayName: "Test: Windows Server x64"
agentOs: Windows
isAzDOTestingJob: true
# Just uploading artifacts/logs/ files can take 15 minutes. Doubling the cancel timeout for this job.
@ -664,7 +664,7 @@ stages:
- template: jobs/default-build.yml
parameters:
jobName: MacOS_Test
jobDisplayName: "Test: macOS 10.15"
jobDisplayName: "Test: macOS"
agentOs: macOS
timeoutInMinutes: 240
isAzDOTestingJob: true
@ -685,7 +685,7 @@ stages:
- template: jobs/default-build.yml
parameters:
jobName: Linux_Test
jobDisplayName: "Test: Ubuntu 18.04 x64"
jobDisplayName: "Test: Ubuntu x64"
agentOs: Linux
isAzDOTestingJob: true
useHostedUbuntu: false
@ -739,7 +739,7 @@ stages:
parameters:
platform:
name: 'Managed'
container: 'mcr.microsoft.com/dotnet-buildtools/prereqs:centos-7-f39df28-20191023143754'
container: 'mcr.microsoft.com/dotnet-buildtools/prereqs:centos-7-20210714125435-9b5bbc2'
buildScript: './eng/build.sh $(_PublishArgs) --no-build-repo-tasks'
skipPublishValidation: true
@ -779,7 +779,7 @@ stages:
- Helix_x64
- Source_Build_Managed
pool:
vmImage: vs2017-win2016
vmImage: windows-latest
publishUsingPipelines: ${{ variables._PublishUsingPipelines }}
enablePublishBuildArtifacts: true # publish artifacts/log files
- ${{ if and(ne(variables['System.TeamProject'], 'public'), or(eq(parameters.testSourceIndexing, 'true'), eq(variables['Build.SourceBranch'], 'refs/heads/main'))) }}:

Просмотреть файл

@ -113,7 +113,7 @@ jobs:
- template: jobs/default-build.yml
parameters:
jobName: MacOS_Quarantined_Test
jobDisplayName: "Tests: macOS 10.14"
jobDisplayName: "Tests: macOS"
agentOs: macOS
timeoutInMinutes: 120
isAzDOTestingJob: true
@ -146,7 +146,7 @@ jobs:
- template: jobs/default-build.yml
parameters:
jobName: Linux_Quarantined_Test
jobDisplayName: "Tests: Ubuntu 18.04 x64"
jobDisplayName: "Tests: Ubuntu x64"
agentOs: Linux
timeoutInMinutes: 60
isAzDOTestingJob: true

Просмотреть файл

@ -2,7 +2,7 @@
Helix is the distributed test platform that we use to run tests. We build a helix payload that contains the publish directory of every test project that we want to test
send a job with with this payload to a set of queues for the various combinations of OS that we want to test
for example: `Windows.10.Amd64.ClientRS4.VS2017.Open`, `OSX.1012.Amd64.Open`, `Ubuntu.1804.Amd64.Open`. Helix takes care of unzipping, running the job, and reporting results.
for example: `Windows.10.Amd64.ClientRS4.VS2017.Open`, `OSX.1100.Amd64.Open`, `Ubuntu.1804.Amd64.Open`. Helix takes care of unzipping, running the job, and reporting results.
For more info about helix see: [SDK](https://github.com/dotnet/arcade/blob/main/src/Microsoft.DotNet.Helix/Sdk/Readme.md), [JobSender](https://github.com/dotnet/arcade/blob/main/src/Microsoft.DotNet.Helix/Sdk/Readme.md)
@ -19,7 +19,7 @@ This will restore, and then publish all the test project including some bootstra
## Overview of the helix usage in our pipelines
- Required queues: Windows10, OSX, Ubuntu1804
- Full queue matrix: Windows[7, 81, 10], Ubuntu[1804, 2004], Debian9, Redhat7, Arm64 (Win10, Debian9)
- Full queue matrix: Windows[10, 11], Ubuntu[1804, 2004], Debian11, Redhat7, Arm64 (Win10, Debian11)
- The queues are defined in [Helix.Common.props](https://github.com/dotnet/aspnetcore/blob/main/eng/targets/Helix.Common.props)
[aspnetcore-ci](https://dev.azure.com/dnceng/public/_build?definitionId=278) runs non quarantined tests against the required helix queues as a required PR check and all builds on all branches.
@ -55,31 +55,25 @@ An example of how to get the helix payload to inspect the contents of a test job
There's also a link embedded in the build.cmd log of the Tests: Helix x64 job on Azure Pipelines, near the bottom right that will look something like this:
``` text
Uploading payloads for Job on Ubuntu.1604.Amd64.Open...
Finished uploading payloads for Job on Ubuntu.1604.Amd64.Open...
Sending Job to Ubuntu.1604.Amd64.Open...
Sent Helix Job a5cbf405-1363-452f-af4b-de5b2a61c8cf
Uploading payloads for Job on Windows.10.Amd64.Open...
Finished uploading payloads for Job on Windows.10.Amd64.Open...
Sending Job to Windows.10.Amd64.Open...
Sent Helix Job cbec3697-c298-412a-953a-e375e49d1fe0
Uploading payloads for Job on OSX.1014.Amd64.Open...
Finished uploading payloads for Job on OSX.1014.Amd64.Open...
Sending Job to Ubuntu.1804.Amd64.Open...
Sent Helix Job; see work items at https://helix.dot.net/api/jobs/c1b425c8-0fef-4cba-9dee-29344d7a61b8/workitems?api-version=2019-06-17
Sending Job to Windows.11.Amd64.ClientPre.Open...
Sent Helix Job; see work items at https://helix.dot.net/api/jobs/1fc117ce-d52a-4ea4-8896-3c289fdf8e17/workitems?api-version=2019-06-17
Sending Job to OSX.1014.Amd64.Open...
Sent Helix Job a54359cf-f74d-4d02-9faf-07e0a8380995
Waiting for completion of job cbec3697-c298-412a-953a-e375e49d1fe0
Waiting for completion of job a54359cf-f74d-4d02-9faf-07e0a8380995
Waiting for completion of job a5cbf405-1363-452f-af4b-de5b2a61c8cf
Job a54359cf-f74d-4d02-9faf-07e0a8380995 is completed with 136 finished work items.
Job cbec3697-c298-412a-953a-e375e49d1fe0 is completed with 156 finished work items.
Job a5cbf405-1363-452f-af4b-de5b2a61c8cf is completed with 136 finished work items.
Stopping Azure Pipelines Test Run Ubuntu.1604.Amd64.Open
Stopping Azure Pipelines Test Run Windows.10.Amd64.Open
Sent Helix Job; see work items at https://helix.dot.net/api/jobs/53e2ca23-9efd-4299-8a8f-d9271265aeaa/workitems?api-version=2019-06-17
Waiting for completion of job 1fc117ce-d52a-4ea4-8896-3c289fdf8e17 on Windows.11.Amd64.ClientPre.Open
Waiting for completion of job c1b425c8-0fef-4cba-9dee-29344d7a61b8 on Ubuntu.1804.Amd64.Open
Waiting for completion of job 53e2ca23-9efd-4299-8a8f-d9271265aeaa on OSX.1014.Amd64.Open
Job 53e2ca23-9efd-4299-8a8f-d9271265aeaa on OSX.1014.Amd64.Open is completed with 139 finished work items.
Job c1b425c8-0fef-4cba-9dee-29344d7a61b8 on Ubuntu.1804.Amd64.Open is completed with 138 finished work items.
Job 1fc117ce-d52a-4ea4-8896-3c289fdf8e17 on Windows.11.Amd64.ClientPre.Open is completed with 170 finished work items.
Stopping Azure Pipelines Test Run Ubuntu.1804.Amd64.Open
Stopping Azure Pipelines Test Run Windows.11.Amd64.ClientPre.Open
Stopping Azure Pipelines Test Run OSX.1014.Amd64.Open
F:\workspace\_work\1\s\.packages\microsoft.dotnet.helix.sdk\5.0.0-beta.20280.1\tools\Microsoft.DotNet.Helix.Sdk.MultiQueue.targets(76,5): error : Work item a5cbf405-1363-452f-af4b-de5b2a61c8cf/Microsoft.AspNetCore.Authentication.Test--net5.0 in job a5cbf405-1363-452f-af4b-de5b2a61c8cf has failed. [F:\workspace\_work\1\s\eng\helix\helix.proj]
F:\workspace\_work\1\s\.packages\microsoft.dotnet.helix.sdk\5.0.0-beta.20280.1\tools\Microsoft.DotNet.Helix.Sdk.MultiQueue.targets(76,5): error : Failure log: https://helix.dot.net/api/2019-06-17/jobs/a5cbf405-1363-452f-af4b-de5b2a61c8cf/workitems/Microsoft.AspNetCore.Authentication.Test--net5.0/console [F:\workspace\_work\1\s\eng\helix\helix.proj]
##[error].packages\microsoft.dotnet.helix.sdk\5.0.0-beta.20280.1\tools\Microsoft.DotNet.Helix.Sdk.MultiQueue.targets(76,5): error : (NETCORE_ENGINEERING_TELEMETRY=Test) Work item a5cbf405-1363-452f-af4b-de5b2a61c8cf/Microsoft.AspNetCore.Authentication.Test--net5.0 in job a5cbf405-1363-452f-af4b-de5b2a61c8cf has failed.
Failure log: https://helix.dot.net/api/2019-06-17/jobs/a5cbf405-1363-452f-af4b-de5b2a61c8cf/workitems/Microsoft.AspNetCore.Authentication.Test--net5.0/console
D:\a\_work\1\s\.packages\microsoft.dotnet.helix.sdk\7.0.0-beta.21559.3\tools\Microsoft.DotNet.Helix.Sdk.MultiQueue.targets(78,5): error : Work item Microsoft.AspNetCore.Identity.Test--net7.0 in job 53e2ca23-9efd-4299-8a8f-d9271265aeaa has failed. [D:\a\_work\1\s\eng\helix\helix.proj]
D:\a\_work\1\s\.packages\microsoft.dotnet.helix.sdk\7.0.0-beta.21559.3\tools\Microsoft.DotNet.Helix.Sdk.MultiQueue.targets(78,5): error : Failure log: https://helix.dot.net/api/2019-06-17/jobs/53e2ca23-9efd-4299-8a8f-d9271265aeaa/workitems/Microsoft.AspNetCore.Identity.Test--net7.0/console [D:\a\_work\1\s\eng\helix\helix.proj]
##[error].packages\microsoft.dotnet.helix.sdk\7.0.0-beta.21559.3\tools\Microsoft.DotNet.Helix.Sdk.MultiQueue.targets(78,5): error : (NETCORE_ENGINEERING_TELEMETRY=Test) Work item Microsoft.AspNetCore.Identity.Test--net7.0 in job 53e2ca23-9efd-4299-8a8f-d9271265aeaa has failed.
Failure log: https://helix.dot.net/api/2019-06-17/jobs/53e2ca23-9efd-4299-8a8f-d9271265aeaa/workitems/Microsoft.AspNetCore.Identity.Test--net7.0/console
```
The https://helix.dot.net/ home page displays information about the available public queues (nothing about the related BYOC pools and queues or the internal Helix queues)

Просмотреть файл

@ -8,14 +8,16 @@
.PARAMETER HelixQueues
Set the Helix queues to use. The list is '+' or ';'-separated.
Some supported queues:
Debian.11.Amd64.Open
Mariner
Redhat.7.Amd64.Open
Ubuntu.1804.Amd64.Open
Ubuntu.2004.Amd64.Open
OSX.1015.Amd64.Open
OSX.1100.Amd64.Open
Windows.10.Amd64.Server20H2.Open
Windows.81.Amd64.Open
Windows.7.Amd64.Open
OSX.1014.Amd64.Open
Debian.9.Amd64.Open
Redhat.7.Amd64.Open
Windows.11.Amd64.ClientPre.Open
Windows.Amd64.Server2022.Open
.PARAMETER RunQuarantinedTests
By default quarantined tests are not run. Set this to $true to run only the quarantined tests.
.PARAMETER TargetArchitecture

Просмотреть файл

@ -1,13 +1,13 @@
<Project>
<!-- this file is shared between Helix.proj and .csproj files -->
<PropertyGroup>
<HelixQueueAlpine312>(Alpine.312.Amd64.Open)Ubuntu.1804.Amd64.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:alpine-3.12-helix-20200908125345-56c6673</HelixQueueAlpine312>
<HelixQueueDebian11>(Debian.11.Amd64.Open)ubuntu.1804.amd64.open@mcr.microsoft.com/dotnet-buildtools/prereqs:debian-11-helix-amd64-20210304164428-5a7c380</HelixQueueDebian11>
<HelixQueueFedora34>(Fedora.34.Amd64.Open)ubuntu.1804.amd64.open@mcr.microsoft.com/dotnet-buildtools/prereqs:fedora-34-helix-20210728124700-4f64125</HelixQueueFedora34>
<HelixQueueAlpine314>(Alpine.314.Amd64.Open)Ubuntu.1804.Amd64.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:alpine-3.14-helix-amd64-20210910135833-1848e19</HelixQueueAlpine314>
<HelixQueueDebian11>(Debian.11.Amd64.Open)Ubuntu.1804.Amd64.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:debian-11-helix-amd64-20211001171307-0ece9b3</HelixQueueDebian11>
<HelixQueueFedora34>(Fedora.34.Amd64.Open)Ubuntu.1804.Amd64.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:fedora-34-helix-20210924174119-4f64125</HelixQueueFedora34>
<HelixQueueMariner>(Mariner)Ubuntu.1804.Amd64.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:cbl-mariner-1.0-helix-20210528192219-92bf620</HelixQueueMariner>
<HelixQueueArmDebian9>(Debian.9.Arm64.Open)Ubuntu.1804.Armarch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:debian-9-helix-arm64v8-a12566d-20190807161036</HelixQueueArmDebian9>
<HelixQueueArmDebian11>(Debian.11.Arm64.Open)Ubuntu.1804.Armarch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:debian-11-helix-arm64v8-20211001171229-97d8652</HelixQueueArmDebian11>
</PropertyGroup>
<ItemGroup Condition="'$(IsWindowsOnlyTest)' != 'true'">
<HelixAvailablePlatform Include="Windows" />
<HelixAvailablePlatform Include="OSX" />
@ -22,36 +22,28 @@
<ItemGroup Condition="'$(IsRequiredCheck)' == 'true' AND '$(TargetArchitecture)' == 'x64' AND '$(IsHelixDaily)' != 'true'">
<HelixAvailableTargetQueue Include="Ubuntu.1804.Amd64.Open" Platform="Linux" />
<HelixAvailableTargetQueue Include="Windows.11.Amd64.ClientPre.Open" Platform="Windows" />
<HelixAvailableTargetQueue Include="OSX.1014.Amd64.Open" Platform="OSX" />
</ItemGroup>
<!-- x64 Quarantined-only (quarantined-pr.yml and quarantined-tests.yml) test queues -->
<ItemGroup Condition="'$(IsRequiredCheck)' == 'true' AND '$(TargetArchitecture)' == 'x64' AND '$(_UseHelixOpenQueues)' == 'true' AND '$(RunQuarantinedTests)' == 'true'">
<HelixAvailableTargetQueue Include="$(HelixQueueFedora34)" Platform="Linux" />
<HelixAvailableTargetQueue Include="OSX.1100.Amd64.Open" Platform="OSX" />
</ItemGroup>
<!-- x64 Queues for public helix-matrix.yml and quarantine pipelines, except in windows-only cases -->
<ItemGroup Condition="'$(TargetArchitecture)' == 'x64' AND '$(IsHelixDaily)' == 'true' AND '$(_UseHelixOpenQueues)' == 'true' AND '$(IsWindowsOnlyTest)' != 'true'">
<!-- Linux -->
<HelixAvailableTargetQueue Include="$(HelixQueueDebian11)" Platform="Linux" />
<HelixAvailableTargetQueue Include="Redhat.7.Amd64.Open" Platform="Linux" />
<HelixAvailableTargetQueue Include="Ubuntu.2004.Amd64.Open" Platform="Linux" />
<HelixAvailableTargetQueue Include="Ubuntu.1804.Amd64.Open" Platform="Linux" />
<!-- Mac -->
<HelixAvailableTargetQueue Include="OSX.1014.Amd64.Open" Platform="OSX" />
<HelixAvailableTargetQueue Include="OSX.1100.Amd64.Open" Platform="OSX" />
<HelixAvailableTargetQueue Include="OSX.1015.Amd64.Open" Platform="OSX" />
<!-- Containers -->
<HelixAvailableTargetQueue Include="$(HelixQueueAlpine314)" Platform="Linux" />
<HelixAvailableTargetQueue Include="$(HelixQueueDebian11)" Platform="Linux" />
<HelixAvailableTargetQueue Include="$(HelixQueueFedora34)" Platform="Linux" />
<HelixAvailableTargetQueue Include="$(HelixQueueAlpine312)" Platform="Linux" />
<HelixAvailableTargetQueue Include="$(HelixQueueMariner)" Platform="Linux" />
<HelixAvailableTargetQueue Include="$(HelixQueueMariner)" Platform="Linux" />
</ItemGroup>
<!-- x64 Queues for public helix-matrix.yml and quarantine pipelines, Windows cases-->
<ItemGroup Condition="'$(TargetArchitecture)' == 'x64' AND '$(IsHelixDaily)' == 'true' AND '$(_UseHelixOpenQueues)' == 'true'">
<HelixAvailableTargetQueue Include="Windows.10.Amd64.Server20H2.Open" Platform="Windows" />
<HelixAvailableTargetQueue Include="Windows.11.Amd64.ClientPre.Open" Platform="Windows" />
<HelixAvailableTargetQueue Include="Windows.Amd64.Server2022.Open" Platform="Windows" />
</ItemGroup>
@ -62,7 +54,7 @@
<!-- arm64 queues for helix-matrix.yml and quarantine pipeline -->
<ItemGroup Condition="'$(TargetArchitecture)' == 'arm64' AND '$(IsHelixDaily)' == 'true' AND '$(_UseHelixOpenQueues)' == 'true' AND '$(IsWindowsOnlyTest)' != 'true'">
<HelixAvailableTargetQueue Include="$(HelixQueueArmDebian9)" Platform="Linux" />
<HelixAvailableTargetQueue Include="$(HelixQueueArmDebian11)" Platform="Linux" />
</ItemGroup>
<!-- IIS Express isn't supported on arm64 and most of the IsWindowsOnlyTests depend on it's setup scripts. -->

Просмотреть файл

@ -10,14 +10,12 @@
<PropertyGroup Condition="'$(TestDependsOnPlaywright)' == 'true'">
<SkipHelixQueues>
$(HelixQueueAlpine312);
$(HelixQueueAlpine314);
$(HelixQueueDebian11);
$(HelixQueueFedora34);
$(HelixQueueMariner);
Redhat.7.Amd64.Open;
Ubuntu.2004.Amd64.Open;
Windows.7.Amd64.Open;
Windows.81.Amd64.Open;
</SkipHelixQueues>
<SkipHelixArm>true</SkipHelixArm>
</PropertyGroup>

Просмотреть файл

@ -17,6 +17,7 @@ using Xunit.Abstractions;
namespace Microsoft.AspNetCore.Identity.Test;
[SkipOnHelix("https://github.com/dotnet/aspnetcore/issues/38542", Queues="OSX.1015.Amd64.Open;OSX.1015.Amd64")] //slow
public class IdentityUIScriptsTest : IDisposable
{
private readonly ITestOutputHelper _output;

Просмотреть файл

@ -35,7 +35,7 @@ public class GrpcTemplateTest : LoggedTest
}
[ConditionalFact]
[SkipOnHelix("Not supported queues", Queues = "Windows.7.Amd64;Windows.7.Amd64.Open;Windows.81.Amd64.Open;All.OSX;" + HelixConstants.Windows10Arm64 + HelixConstants.DebianArm64)]
[SkipOnHelix("Not supported queues", Queues = "All.OSX;" + HelixConstants.Windows10Arm64 + HelixConstants.DebianArm64)]
[SkipOnAlpine("https://github.com/grpc/grpc/issues/18338")]
public async Task GrpcTemplate()
{

Просмотреть файл

@ -6,6 +6,6 @@ namespace Microsoft.AspNetCore.Testing;
public static class HelixConstants
{
public const string Windows10Arm64 = "Windows.10.Arm64v8.Open;";
public const string DebianArm64 = "Debian.9.Arm64.Open;";
public const string DebianArm64 = "Debian.11.Arm64.Open;";
public const string RedhatAmd64 = "Redhat.7.Amd64.Open;";
}

Просмотреть файл

@ -32,7 +32,7 @@ public class SkipOnHelixAttribute : Attribute, ITestCondition
}
}
// Queues that should be skipped on, i.e. "Windows.10.Amd64.ClientRS4.VS2017.Open;OSX.1012.Amd64.Open"
// Queues that should be skipped on, i.e. "Windows.Amd64.Server2022.Open;OSX.1015.Amd64.Open"
public string Queues { get; set; }
public string SkipReason