История

Kirill Ivlev b2f4f4484c Update master with dotnet 6 (#4254 ) * Draft: Dotnet6 migration (#3879) * Agent on .NET 6 (draft) * Suppress object dispose warnings * Fixed test code analysis * disable RHEL6 * Disable RHEL6, install dotnet 6.0.2 * Disable RHEL6 * fix Winx86 pipeline * Install dotnet x86 * fix dotnet 6 migrations warnings * bump git verstion to 2.36.1 (#3839) Co-authored-by: Andrey Ivanov <v-andivanov@microsoft.com> * Removed NPM module and executable which comes with NodeJS fox non windows distribution (#3833) * Removem NPM module and executable which comes with NodeJS fox non windows distribution * Changed the logic from \|\| and && when removing NPM Co-authored-by: Denis Rumyantsev <mr.denis.rumyantsev@gmail.com> Co-authored-by: Alexander Smolyakov <v-alsmo@microsoft.com> * Added message - in case when account is not managed, but WindowsLogon… (#3845) * Added message - in case when account is not managed, but WindowsLogonPassword was not specified * Added message - in case when account is not managed, but WindowsLogonPassword was not specified * Fixed typo with condition * Fixed indents Co-authored-by: AndreyIvanov42 <93121155+AndreyIvanov42@users.noreply.github.com> * update scripts / rebase * add RHEL6 warning * Agent on .NET 6 (draft) * Suppress object dispose warnings * disable RHEL6 * Disable RHEL6, install dotnet 6.0.2 * fix Winx86 pipeline * Install dotnet x86 * fix dotnet 6 migrations warnings * update azure pipeline to use dotnet 6.0.200 * fix yml indentation * fix windows-x32 pipeline * Update build-job.yml * disable docker detector for legacy containers * disable security checks * disable security checks * Update build-job.yml * Update build-job.yml * Update build-job.yml * update config.sh for net6 support * add native support of the node16 on mac m1 * add osx-arm64 pipeline * fix job name for macos-arm64 * update pipelines for osx-arm64 support * Update build-job.yml * Update build-job.yml * update macos image & disable unit test for osx-arm64 * Update pipeline.yml * switch back to macos-10.15 * set macos image to macos-12 * add osx-arm64 as allowed package * set image to 10.15 * disable falling test for macos * update .NET SDK to 6.0.302 * node16 knob & skip tests on macos * add rhel7 * update pipeline for RHEL7 * update pipeline for rhel7.2 * update pipelines (RHEL7.2) * Update pipeline.yml * update image reference * update runtime framework to 6.0.7 * Update pipeline.yml Co-authored-by: Denis Rumyantsev <v-denisr@microsoft.com> Co-authored-by: Andrey Ivanov <v-andivanov@microsoft.com> Co-authored-by: AndreyIvanov42 <93121155+AndreyIvanov42@users.noreply.github.com> Co-authored-by: Max Podriezov <maxpodriezov@microsoft.com> Co-authored-by: Denis Rumyantsev <mr.denis.rumyantsev@gmail.com> Co-authored-by: Alexander Smolyakov <v-alsmo@microsoft.com> Co-authored-by: Anatoly Bolshakov <v-anbols@microsoft.com> Co-authored-by: Liliia Sabitova <107196662+LiliaSabitova@users.noreply.github.com> * added logic to skip downgrade if agent have version 3.* and server version is 2.* (#3996) * fix issues after conflict resolving (#4001) * Fix GitHub only release condition (#4002) * BUG 1972388: vsbuild task in YAML build pipeline hangs forever in ADO even though the task has already logged completion (#3979) * moved logic to drain queue in separate method "DrainQueues" * Implemented a call of method "DrainQueues" once step is completed * Simplified syntax of property 'JobServerQueue' * Fix condition for onlyGitHubRelease Co-authored-by: Sergey Koryshev <sergey.koryshev@akvelon.com> * Remove condition from Blob store step * remove branch check on createBranch step * fix pipeline / update agent version * Fixed NullReferenceException in v3.212.0 preview version Agent (#4018) Fix for PublishTelemetry: Added check for taskDefinition.Data.Execution, because it turned out that not all tasks have Execution information. * Update dotnet6-agent from master (#4068) Update dotnet6-agent branch from master * Update releaseNote.md * Fix CA2000 error with secrets masker * update System.IO.FileSystem.AccessControl * Update Common.props (#4109) * Update dotnet6-agent branch from master (#4128) * Catch updating exception (#4082) * Node16 version updated to 16.17.1" (#4084) * Bump ubuntu image version to 20.04 (#4079) * Prevent creating host container network (#3788) * Prevent creating host container network * Remove redundant empty lines * Fix invalid knob value fetching * Update ContainerOperationProvider.cs Co-authored-by: Denis Rumyantsev <mr.denis.rumyantsev@gmail.com> Co-authored-by: Kirill Ivlev <102740624+kirill-ivlev@users.noreply.github.com> Co-authored-by: Konstantin Tyukalov <52399739+KonstantinTyukalov@users.noreply.github.com> * Allow env.sh to accept additional env var names (#3339) The hard-coded list is not necessarily enough. This allows the caller to specify additional environment variables to write to .env. Co-authored-by: Konstantin Tyukalov <52399739+KonstantinTyukalov@users.noreply.github.com> Co-authored-by: Kirill Ivlev <102740624+kirill-ivlev@users.noreply.github.com> * Revert "Fix execution of scripts in variables (#4016)" (#4088) This reverts commit `af5328b265`. * Fix CA1711 for ServiceBootFlag * Fix CA2000 for JobRunner * Resolve CA2000 in StepHost * Resolve CA2000 in Windows service * Resolve CA2000 for WorkerCommandManager * Resove CA2000 for CodeCoverageCommands * Add fixture for SecretMasker tests * Add fixture for SecretMasker tests * Fail the installation of agent for unsupported OS - RHEL6/CentOS6 * Change Fixtures -> Disposable * git update-index --chmod=+x .\src\dev.sh * Use dispose pattern to fix errors Co-authored-by: Roman-Shchukin <111063382+Roman-Shchukin@users.noreply.github.com> Co-authored-by: Konstantin Tyukalov <52399739+KonstantinTyukalov@users.noreply.github.com> Co-authored-by: Steve <hez2010@outlook.com> Co-authored-by: Denis Rumyantsev <mr.denis.rumyantsev@gmail.com> Co-authored-by: Kirill Ivlev <102740624+kirill-ivlev@users.noreply.github.com> Co-authored-by: Rick Brown <ricksbrown73@gmail.com> Co-authored-by: Konstantin Tyukalov <v-ktyukalov@microsoft.com> * Merge master to dotnet6-agent branch (#4129) * Merge master to dotnet6-agent branch * git update-index --chmod=+x .\src\dev.sh * Update step name * Change step name * Revert "Allow env.sh to accept additional env var names" (#4144) * Upd dotnet6 agent (#4150) * Catch updating exception (#4082) * Node16 version updated to 16.17.1" (#4084) * Bump ubuntu image version to 20.04 (#4079) * Prevent creating host container network (#3788) * Prevent creating host container network * Remove redundant empty lines * Fix invalid knob value fetching * Update ContainerOperationProvider.cs Co-authored-by: Denis Rumyantsev <mr.denis.rumyantsev@gmail.com> Co-authored-by: Kirill Ivlev <102740624+kirill-ivlev@users.noreply.github.com> Co-authored-by: Konstantin Tyukalov <52399739+KonstantinTyukalov@users.noreply.github.com> * Allow env.sh to accept additional env var names (#3339) The hard-coded list is not necessarily enough. This allows the caller to specify additional environment variables to write to .env. Co-authored-by: Konstantin Tyukalov <52399739+KonstantinTyukalov@users.noreply.github.com> Co-authored-by: Kirill Ivlev <102740624+kirill-ivlev@users.noreply.github.com> * Revert "Fix execution of scripts in variables (#4016)" (#4088) This reverts commit `af5328b265`. * Add Ubuntu 18.04 (#3943) Co-authored-by: Kirill Ivlev <102740624+kirill-ivlev@users.noreply.github.com> Co-authored-by: Konstantin Tyukalov <52399739+KonstantinTyukalov@users.noreply.github.com> * Make agent update list of systems supporting .NET 6 periodically (#4081) * implemented logic to fetch net6.json from server every hour * added await keyword * Moved logic to set warning from ExecutionContext to JobRunner since InitializeJob method is not async * fixed issue with condition in Equals method of OperatingSystem class * small refactoring * net6.json must be read only if it was not fetched from server * refactored code of method "GetNet6SupportedSystems" * restored field net6SupportedSystems to reduce IO oprations and cover case when agent started but net6.json is not older than 1 hour Co-authored-by: Kirill Ivlev <102740624+kirill-ivlev@users.noreply.github.com> * Retries to lock Services database (#4085) * retries to lock database * Decrease timeout + Add retry message Co-authored-by: Konstantin Tyukalov <v-ktyukalov@microsoft.com> Co-authored-by: Konstantin Tyukalov <52399739+KonstantinTyukalov@users.noreply.github.com> * Fixed name of DockerAdditionalNetworkOptions Knob (#3888) Co-authored-by: Konstantin Tyukalov <52399739+KonstantinTyukalov@users.noreply.github.com> * Format agent source code (#4101) * Run dotnet format * Remove extra semicolons * Fix SonarQube's "'Any()' should be used to test for emptiness" / Code Smell (#3404) * Add script to predict v2->v3 upgrade compatibility (#4092) * Add script to predict v2->v3 upgrade compatibility * Fix typo * Fixed another typo: frpom Co-authored-by: Kirill Ivlev <102740624+kirill-ivlev@users.noreply.github.com> * Added strict boolean knob values (#4103) * Add ConvertToBooleanStrict * Add AsBooleanStrict for agent knobs * update build-job (#4110) * update build-job - added UseDotNet task for linux os * Switch EsrpCodeSigning from V1 to V2 returned back UseDotNet task condition * Set UseDotNet version to 6 on osx * added dotnet 3.1.x before unit tests on win x86 * fix dotnet host search (#4124) * fix dotnet host search - added performMultiLevelLookup * Added runtime to unitTests step * Changed patch version in semver of runtime * remove sdk and runtime from unitTests script * Fixed problem with .NET Core 3.1 installation * Added UseDotNet task in functionalTests Co-authored-by: Roman Shchukin <roman.shchukin@akvelon.com> * Add option to skip rhel6 in pipeline and ci (#4121) * skipRhelRelease parameter added * skipRhelRelease parameter fixed * releaseRhelOnly parameter added * Logic of removing not filling lines from releaseNote * Fix pipeline with test releaseNote job * Fix pipeline with test releaseNote job * Fix pipeline with test releaseNote job * Fix pipeline with test releaseNote job * Fix pipeline with test releaseNote job * Fix pipeline with test releaseNote job * Fix pipeline with test releaseNote job * Fix pipeline with test releaseNote job * Fix pipeline with test releaseNote job * Fix pipeline with test releaseNote job * Removed test logs * Added clear empty lines logic * Commented code for test. Test changes in pipeline * Removed test changes * skipRhelRelease parametr added to ci pipeline * Update dotnet to 3.1.32 (#4112) * DockerLogin retries (#4100) * DockerLogin retries - added changes from PR microsoft/azure-pipelines-agent#3840 - fixed incorrect variables * DockerLogin retries - added step UseDotNet@2 * DockerLogin retries - added linux to UseDotNet task * Docker Login retries Implement retry into DockerLogin method * Implement feature flag for docker login retries * revert build-job back * Added retry for docker start * fix delay * update knob description * Update agent Docker methods - reworked Docker methods with retries behavior - changed InvokeWithRetryIfNonZero on to incapsulated logic method * - renamed local variable as is in ADO * Added docker retry behavior to docker version method Co-authored-by: Your Name <you@example.com> Co-authored-by: Kirill Ivlev <102740624+kirill-ivlev@users.noreply.github.com> Co-authored-by: Merlyn Oppenheim <merlynop@microsoft.com> * Add the exact operation system version and environment to the telemetry (#4104) * Update TaskRunner.cs * Update build-job.yml * Added new variables to telemetry AgentName, MachineName, IsSelfHosted Co-authored-by: Kirill Ivlev <102740624+kirill-ivlev@users.noreply.github.com> * Mingit version upgraded to 2.39.1 (#4126) * Fix agent lint errors (#4117) * Fix CA2000 error with secrets masker * Fix CA1711 for ServiceBootFlag * Fix CA2000 for JobRunner * Resolve CA2000 in StepHost * Resolve CA2000 in Windows service * Resolve CA2000 for WorkerCommandManager * Resove CA2000 for CodeCoverageCommands * Remove unused namespaces * Add fixture for SecretMasker tests * Change Fixtures -> Disposable * Use dispose pattern to fix errors * Formatting * Revert "Allow env.sh to accept additional env var names (#3339)" (#4143) This reverts commit `27e3d4e177`. --------- Co-authored-by: Roman-Shchukin <111063382+Roman-Shchukin@users.noreply.github.com> Co-authored-by: Konstantin Tyukalov <52399739+KonstantinTyukalov@users.noreply.github.com> Co-authored-by: Steve <hez2010@outlook.com> Co-authored-by: Denis Rumyantsev <mr.denis.rumyantsev@gmail.com> Co-authored-by: Rick Brown <ricksbrown73@gmail.com> Co-authored-by: Liliia Sabitova <107196662+LiliaSabitova@users.noreply.github.com> Co-authored-by: Eric van Wijk <eric@van-wijk.com> Co-authored-by: Sergey Koryshev <sergey.koryshev@akvelon.com> Co-authored-by: Konstantin Tyukalov <v-ktyukalov@microsoft.com> Co-authored-by: Rami <icnocop@users.noreply.github.com> Co-authored-by: Martin <modermatt@tuta.io> Co-authored-by: ivanduplenskikh <115665590+ivanduplenskikh@users.noreply.github.com> Co-authored-by: Roman Shchukin <roman.shchukin@akvelon.com> Co-authored-by: Your Name <you@example.com> Co-authored-by: Merlyn Oppenheim <merlynop@microsoft.com> Co-authored-by: İsmayıl İsmayılov <110806089+ismayilov-ismayil@users.noreply.github.com> * Update minor .net6 version (#4151) * Bump azure pipelines task lib to 4.2.0 (#4156) * Upd dotnet6 agent mar2023 (#4191) * Catch updating exception (#4082) * Node16 version updated to 16.17.1" (#4084) * Bump ubuntu image version to 20.04 (#4079) * Prevent creating host container network (#3788) * Prevent creating host container network * Remove redundant empty lines * Fix invalid knob value fetching * Update ContainerOperationProvider.cs Co-authored-by: Denis Rumyantsev <mr.denis.rumyantsev@gmail.com> Co-authored-by: Kirill Ivlev <102740624+kirill-ivlev@users.noreply.github.com> Co-authored-by: Konstantin Tyukalov <52399739+KonstantinTyukalov@users.noreply.github.com> * Allow env.sh to accept additional env var names (#3339) The hard-coded list is not necessarily enough. This allows the caller to specify additional environment variables to write to .env. Co-authored-by: Konstantin Tyukalov <52399739+KonstantinTyukalov@users.noreply.github.com> Co-authored-by: Kirill Ivlev <102740624+kirill-ivlev@users.noreply.github.com> * Revert "Fix execution of scripts in variables (#4016)" (#4088) This reverts commit `af5328b265`. * Add Ubuntu 18.04 (#3943) Co-authored-by: Kirill Ivlev <102740624+kirill-ivlev@users.noreply.github.com> Co-authored-by: Konstantin Tyukalov <52399739+KonstantinTyukalov@users.noreply.github.com> * Make agent update list of systems supporting .NET 6 periodically (#4081) * implemented logic to fetch net6.json from server every hour * added await keyword * Moved logic to set warning from ExecutionContext to JobRunner since InitializeJob method is not async * fixed issue with condition in Equals method of OperatingSystem class * small refactoring * net6.json must be read only if it was not fetched from server * refactored code of method "GetNet6SupportedSystems" * restored field net6SupportedSystems to reduce IO oprations and cover case when agent started but net6.json is not older than 1 hour Co-authored-by: Kirill Ivlev <102740624+kirill-ivlev@users.noreply.github.com> * Retries to lock Services database (#4085) * retries to lock database * Decrease timeout + Add retry message Co-authored-by: Konstantin Tyukalov <v-ktyukalov@microsoft.com> Co-authored-by: Konstantin Tyukalov <52399739+KonstantinTyukalov@users.noreply.github.com> * Fixed name of DockerAdditionalNetworkOptions Knob (#3888) Co-authored-by: Konstantin Tyukalov <52399739+KonstantinTyukalov@users.noreply.github.com> * Format agent source code (#4101) * Run dotnet format * Remove extra semicolons * Fix SonarQube's "'Any()' should be used to test for emptiness" / Code Smell (#3404) * Add script to predict v2->v3 upgrade compatibility (#4092) * Add script to predict v2->v3 upgrade compatibility * Fix typo * Fixed another typo: frpom Co-authored-by: Kirill Ivlev <102740624+kirill-ivlev@users.noreply.github.com> * Added strict boolean knob values (#4103) * Add ConvertToBooleanStrict * Add AsBooleanStrict for agent knobs * update build-job (#4110) * update build-job - added UseDotNet task for linux os * Switch EsrpCodeSigning from V1 to V2 returned back UseDotNet task condition * Set UseDotNet version to 6 on osx * added dotnet 3.1.x before unit tests on win x86 * fix dotnet host search (#4124) * fix dotnet host search - added performMultiLevelLookup * Added runtime to unitTests step * Changed patch version in semver of runtime * remove sdk and runtime from unitTests script * Fixed problem with .NET Core 3.1 installation * Added UseDotNet task in functionalTests Co-authored-by: Roman Shchukin <roman.shchukin@akvelon.com> * Add option to skip rhel6 in pipeline and ci (#4121) * skipRhelRelease parameter added * skipRhelRelease parameter fixed * releaseRhelOnly parameter added * Logic of removing not filling lines from releaseNote * Fix pipeline with test releaseNote job * Fix pipeline with test releaseNote job * Fix pipeline with test releaseNote job * Fix pipeline with test releaseNote job * Fix pipeline with test releaseNote job * Fix pipeline with test releaseNote job * Fix pipeline with test releaseNote job * Fix pipeline with test releaseNote job * Fix pipeline with test releaseNote job * Fix pipeline with test releaseNote job * Removed test logs * Added clear empty lines logic * Commented code for test. Test changes in pipeline * Removed test changes * skipRhelRelease parametr added to ci pipeline * Update dotnet to 3.1.32 (#4112) * DockerLogin retries (#4100) * DockerLogin retries - added changes from PR microsoft/azure-pipelines-agent#3840 - fixed incorrect variables * DockerLogin retries - added step UseDotNet@2 * DockerLogin retries - added linux to UseDotNet task * Docker Login retries Implement retry into DockerLogin method * Implement feature flag for docker login retries * revert build-job back * Added retry for docker start * fix delay * update knob description * Update agent Docker methods - reworked Docker methods with retries behavior - changed InvokeWithRetryIfNonZero on to incapsulated logic method * - renamed local variable as is in ADO * Added docker retry behavior to docker version method Co-authored-by: Your Name <you@example.com> Co-authored-by: Kirill Ivlev <102740624+kirill-ivlev@users.noreply.github.com> Co-authored-by: Merlyn Oppenheim <merlynop@microsoft.com> * Add the exact operation system version and environment to the telemetry (#4104) * Update TaskRunner.cs * Update build-job.yml * Added new variables to telemetry AgentName, MachineName, IsSelfHosted Co-authored-by: Kirill Ivlev <102740624+kirill-ivlev@users.noreply.github.com> * Mingit version upgraded to 2.39.1 (#4126) * Fix agent lint errors (#4117) * Fix CA2000 error with secrets masker * Fix CA1711 for ServiceBootFlag * Fix CA2000 for JobRunner * Resolve CA2000 in StepHost * Resolve CA2000 in Windows service * Resolve CA2000 for WorkerCommandManager * Resove CA2000 for CodeCoverageCommands * Remove unused namespaces * Add fixture for SecretMasker tests * Change Fixtures -> Disposable * Use dispose pattern to fix errors * Formatting * Revert "Allow env.sh to accept additional env var names (#3339)" (#4143) This reverts commit `27e3d4e177`. * Bump azure pipelines task lib to 4.2.0 (#4155) * Add warning for deprecated runners (#4162) * Add warning for deprecated runners * Update strings.json * fix typo * Update NodeHandler.cs * FEATURE 2019014 - Gather telemetry on Agent Azure & Docker Container usage (#4166) * FEATURE 2019014 - Gather telemetry on Agent Azure & Docker Container usage * Re-write try-catch * Update TaskRunner.cs * cleanup * Distinguish AzureInstanceMetadata detection to a separate class --------- Co-authored-by: Kirill Ivlev <102740624+kirill-ivlev@users.noreply.github.com> * Added legacy lib source for support on Ubuntu 22.04 (#4161) * Moved logic to drain queues after each task under knob "AGENT_DRAIN_QUEUES_AFTER_TASK" (#4176) * Added knob "DrainQueuesAfterTask" with default "false" * move logic to drain queues behind knob DrainQueuesAfterTask * added mocking of GetScopedEnvironment method in step execution context * added mocking of GetScopedEnvironment method in step execution context * changed default value for knob "AGENT_DRAIN_QUEUES_AFTER_TASK" to true * implementd AsBooleanStrict and check in JobRunner * Revert "implementd AsBooleanStrict and check in JobRunner" This reverts commit `a1c0c67889`. * Reversed the knob * Add issue templates (#4154) * Add issue templates * Update config.yml * update templates * Update config.yml * Added knob to break pipeline unless user opt-in (#4165) * Added knob to force agent fail if user doesn't opt-in updates on unsupported OS * Resolved PR comments --------- Co-authored-by: Kirill Ivlev <102740624+kirill-ivlev@users.noreply.github.com> * Merge remote-tracking branch 'origin/master' into upd_dotnet6_agent_mar2023 # Conflicts: # .azure-pipelines/build-job.yml # .azure-pipelines/pipeline.yml # .vsts.ci.yml # src/Agent.Sdk/Util/PlatformUtil.cs # src/Agent.Worker/JobRunner.cs # src/Agent.Worker/StepsRunner.cs # src/Agent.Worker/TaskRunner.cs # src/Common.props # src/Misc/layoutbin/installdependencies.sh # src/dev.sh * Localization update (#4187) * Added files for onelocbuild * Update localize-pipeline.yml for Azure Pipelines * Fixed typo in resource.resx path * Removed en-US * Update localize-pipeline.yml for Azure Pipelines * LEGO: check in for Localization to temporary branch. (#3246) * LEGO: check in for Localization to temporary branch. (#3268) * LEGO: check in for Localization to temporary branch. (#3265) * LEGO: check in for Localization to temporary branch. (#3279) * Temporary renamed localization folders * Renamed localization folders * Set up schedule and notifications for the localization pipeline (#3269) * [localization] Fixed localization pipeline issue with already localized strings replaced (#3332) * Localized file check-in by OneLocBuild Task: Build definition ID 10944: Build ID 14646614 Localized file check-in by OneLocBuild Task * LEGO: check in for Localization to temporary branch. (#3363) Co-authored-by: csigs <csigs@outlook.com> * LEGO: check in for Localization to temporary branch. (#3364) Co-authored-by: csigs <csigs@outlook.com> * Localized file check-in by OneLocBuild Task: Build definition ID 10944: Build ID 14919763 Localized file check-in by OneLocBuild Task * Create PR in OneLocBuild task only on third week of sprint (#3374) * Fix localization pipeline * Add missed change * Added option to disable PR creation * update localize pipeline (#3637) Co-authored-by: Ilya Kuleshov <v-ikuleshov@microsoft.com> * Localization update (#3612) * Removing Localize folder * RHEL/CentOS 7 Fix for Lets Encrypt Change (#3601) * Revert "RHEL/CentOS 7 Fix for Lets Encrypt Change (#3601)" This reverts commit `629921e0fa`. * Revert "Removing Localize folder" This reverts commit `33cb0ae7c1`. Co-authored-by: AndreyIvanov42 <93121155+AndreyIvanov42@users.noreply.github.com> Co-authored-by: kuleshovilya <87485027+kuleshovilya@users.noreply.github.com> Co-authored-by: Ilya Kuleshov <v-ikuleshov@microsoft.com> * Removed OneLocBuild folder * Move notifications about Agent Localization PR from Slack to MS Teams - Part 1 (#3744) * Include strings from Misc/src/layoutbin to LocProject file (#4021) * Update LocProject file * Update CopyOption for LocProject * Localized file check-in by OneLocBuild Task: Build definition ID 10944: Build ID 18665127 (#4022) * Revert "Include strings from Misc/src/layoutbin to LocProject file (#4021)" This reverts commit `8f5c0c05b3`. * Revert "Localized file check-in by OneLocBuild Task: Build definition ID 10944: Build ID 18665127 (#4022)" (#4023) This reverts commit `b9c714bdba`. * Include layoutbin into locproject * Localized file check-in by OneLocBuild Task: Build definition ID 10944: Build ID 18680899 (#4028) * Juno: check in to lego/hb_5733438c-38c5-41ae-9557-a9d9754c4535_20221104175414554. (#4030) * Juno: check in to lego/hb_5733438c-38c5-41ae-9557-a9d9754c4535_20221105085259861. (#4031) * Juno: check in to lego/hb_5733438c-38c5-41ae-9557-a9d9754c4535_20221106085521421. (#4032) * Localized file check-in by OneLocBuild Task: Build definition ID 10944: Build ID 18697046 (#4033) * Juno: check in to lego/hb_5733438c-38c5-41ae-9557-a9d9754c4535_20221107143113520. (#4039) * Localized file check-in by OneLocBuild Task: Build definition ID 10944: Build ID 18706620 (#4044) * Localized file check-in by OneLocBuild Task: Build definition ID 10944: Build ID 19415018 (#4163) * Juno: check in to lego/hb_5733438c-38c5-41ae-9557-a9d9754c4535_20230223095051399. (#4168) * Juno: check in to lego/hb_5733438c-38c5-41ae-9557-a9d9754c4535_20230224095040576. (#4170) * Localized file check-in by OneLocBuild Task: Build definition ID 10944: Build ID 19476421 (#4179) * Update loc strings * Localized file check-in by OneLocBuild Task: Build definition ID 10944: Build ID 19476736 (#4181) * Juno: check in to lego/hb_5733438c-38c5-41ae-9557-a9d9754c4535_20230305095015478. (#4183) * Juno: check in to lego/hb_5733438c-38c5-41ae-9557-a9d9754c4535_20230306095225210. (#4184) * Removing Localize and OneLocBuild folder --------- Co-authored-by: Anatolii Bolshakov (Akvelon INC) <v-anbols@microsoft.com> Co-authored-by: csigs <csigs@users.noreply.github.com> Co-authored-by: Egor Bryzgalov <v-egbryz@microsoft.com> Co-authored-by: Anatoly Bolshakov <anatoly.bolshakov@akvelon.com> Co-authored-by: csigs <csigs@outlook.com> Co-authored-by: Nikita Ezzhev <v-niezz@microsoft.com> Co-authored-by: kuleshovilya <87485027+kuleshovilya@users.noreply.github.com> Co-authored-by: Ilya Kuleshov <v-ikuleshov@microsoft.com> Co-authored-by: AndreyIvanov42 <93121155+AndreyIvanov42@users.noreply.github.com> Co-authored-by: Denis Tikhomirov <90906678+denis-tikhomirov@users.noreply.github.com> Co-authored-by: KonstantinTyukalov <v-ktyukalov@microsoft.com> * Add "Mariner" to list of .NET 6 supporting systems (#4197) * Cover unsupported OS failure with FF (#4196) * Cover unsupported OS failure with FF * Update strings.json * Update JobRunner.cs * Update JobRunner.cs * Add knob to enable fetching net6.json file from GitHub (#4200) * Added knob "AGENT_ENABLE_FETCHING_NET6_LIST" * implemented knob EnableFetchingNet6List in function GetNet6SupportedSystems * Removed RuntimeKnobSource for EnableFetchingNet6List * removed logic which fails pipeline if there is some error occurred during checking if system supports .NET 6 * fixed typo * Update autoAssignABTT.yml (#4201) * Update AzureInstanceMetadataProvider.cs --------- Co-authored-by: Roman-Shchukin <111063382+Roman-Shchukin@users.noreply.github.com> Co-authored-by: Konstantin Tyukalov <52399739+KonstantinTyukalov@users.noreply.github.com> Co-authored-by: Steve <hez2010@outlook.com> Co-authored-by: Denis Rumyantsev <mr.denis.rumyantsev@gmail.com> Co-authored-by: Rick Brown <ricksbrown73@gmail.com> Co-authored-by: Liliia Sabitova <107196662+LiliaSabitova@users.noreply.github.com> Co-authored-by: Eric van Wijk <eric@van-wijk.com> Co-authored-by: Sergey Koryshev <sergey.koryshev@akvelon.com> Co-authored-by: Konstantin Tyukalov <v-ktyukalov@microsoft.com> Co-authored-by: Rami <icnocop@users.noreply.github.com> Co-authored-by: Martin <modermatt@tuta.io> Co-authored-by: ivanduplenskikh <115665590+ivanduplenskikh@users.noreply.github.com> Co-authored-by: Roman Shchukin <roman.shchukin@akvelon.com> Co-authored-by: Your Name <you@example.com> Co-authored-by: Merlyn Oppenheim <merlynop@microsoft.com> Co-authored-by: İsmayıl İsmayılov <110806089+ismayilov-ismayil@users.noreply.github.com> Co-authored-by: Maksim Petrov <47208721+vmapetr@users.noreply.github.com> Co-authored-by: Anatolii Bolshakov (Akvelon INC) <v-anbols@microsoft.com> Co-authored-by: csigs <csigs@users.noreply.github.com> Co-authored-by: Egor Bryzgalov <v-egbryz@microsoft.com> Co-authored-by: Anatoly Bolshakov <anatoly.bolshakov@akvelon.com> Co-authored-by: csigs <csigs@outlook.com> Co-authored-by: Nikita Ezzhev <v-niezz@microsoft.com> Co-authored-by: kuleshovilya <87485027+kuleshovilya@users.noreply.github.com> Co-authored-by: Ilya Kuleshov <v-ikuleshov@microsoft.com> Co-authored-by: AndreyIvanov42 <93121155+AndreyIvanov42@users.noreply.github.com> Co-authored-by: Denis Tikhomirov <90906678+denis-tikhomirov@users.noreply.github.com> Co-authored-by: Maxim Zaytsev <v-mazayt@microsoft.com> * Dotnet6 agent merge master (#4243) * Catch updating exception (#4082) * Node16 version updated to 16.17.1" (#4084) * Bump ubuntu image version to 20.04 (#4079) * Prevent creating host container network (#3788) * Prevent creating host container network * Remove redundant empty lines * Fix invalid knob value fetching * Update ContainerOperationProvider.cs Co-authored-by: Denis Rumyantsev <mr.denis.rumyantsev@gmail.com> Co-authored-by: Kirill Ivlev <102740624+kirill-ivlev@users.noreply.github.com> Co-authored-by: Konstantin Tyukalov <52399739+KonstantinTyukalov@users.noreply.github.com> * Allow env.sh to accept additional env var names (#3339) The hard-coded list is not necessarily enough. This allows the caller to specify additional environment variables to write to .env. Co-authored-by: Konstantin Tyukalov <52399739+KonstantinTyukalov@users.noreply.github.com> Co-authored-by: Kirill Ivlev <102740624+kirill-ivlev@users.noreply.github.com> * Revert "Fix execution of scripts in variables (#4016)" (#4088) This reverts commit `af5328b265`. * Add Ubuntu 18.04 (#3943) Co-authored-by: Kirill Ivlev <102740624+kirill-ivlev@users.noreply.github.com> Co-authored-by: Konstantin Tyukalov <52399739+KonstantinTyukalov@users.noreply.github.com> * Make agent update list of systems supporting .NET 6 periodically (#4081) * implemented logic to fetch net6.json from server every hour * added await keyword * Moved logic to set warning from ExecutionContext to JobRunner since InitializeJob method is not async * fixed issue with condition in Equals method of OperatingSystem class * small refactoring * net6.json must be read only if it was not fetched from server * refactored code of method "GetNet6SupportedSystems" * restored field net6SupportedSystems to reduce IO oprations and cover case when agent started but net6.json is not older than 1 hour Co-authored-by: Kirill Ivlev <102740624+kirill-ivlev@users.noreply.github.com> * Retries to lock Services database (#4085) * retries to lock database * Decrease timeout + Add retry message Co-authored-by: Konstantin Tyukalov <v-ktyukalov@microsoft.com> Co-authored-by: Konstantin Tyukalov <52399739+KonstantinTyukalov@users.noreply.github.com> * Fixed name of DockerAdditionalNetworkOptions Knob (#3888) Co-authored-by: Konstantin Tyukalov <52399739+KonstantinTyukalov@users.noreply.github.com> * Format agent source code (#4101) * Run dotnet format * Remove extra semicolons * Fix SonarQube's "'Any()' should be used to test for emptiness" / Code Smell (#3404) * Add script to predict v2->v3 upgrade compatibility (#4092) * Add script to predict v2->v3 upgrade compatibility * Fix typo * Fixed another typo: frpom Co-authored-by: Kirill Ivlev <102740624+kirill-ivlev@users.noreply.github.com> * Added strict boolean knob values (#4103) * Add ConvertToBooleanStrict * Add AsBooleanStrict for agent knobs * update build-job (#4110) * update build-job - added UseDotNet task for linux os * Switch EsrpCodeSigning from V1 to V2 returned back UseDotNet task condition * Set UseDotNet version to 6 on osx * added dotnet 3.1.x before unit tests on win x86 * fix dotnet host search (#4124) * fix dotnet host search - added performMultiLevelLookup * Added runtime to unitTests step * Changed patch version in semver of runtime * remove sdk and runtime from unitTests script * Fixed problem with .NET Core 3.1 installation * Added UseDotNet task in functionalTests Co-authored-by: Roman Shchukin <roman.shchukin@akvelon.com> * Add option to skip rhel6 in pipeline and ci (#4121) * skipRhelRelease parameter added * skipRhelRelease parameter fixed * releaseRhelOnly parameter added * Logic of removing not filling lines from releaseNote * Fix pipeline with test releaseNote job * Fix pipeline with test releaseNote job * Fix pipeline with test releaseNote job * Fix pipeline with test releaseNote job * Fix pipeline with test releaseNote job * Fix pipeline with test releaseNote job * Fix pipeline with test releaseNote job * Fix pipeline with test releaseNote job * Fix pipeline with test releaseNote job * Fix pipeline with test releaseNote job * Removed test logs * Added clear empty lines logic * Commented code for test. Test changes in pipeline * Removed test changes * skipRhelRelease parametr added to ci pipeline * Update dotnet to 3.1.32 (#4112) * DockerLogin retries (#4100) * DockerLogin retries - added changes from PR microsoft/azure-pipelines-agent#3840 - fixed incorrect variables * DockerLogin retries - added step UseDotNet@2 * DockerLogin retries - added linux to UseDotNet task * Docker Login retries Implement retry into DockerLogin method * Implement feature flag for docker login retries * revert build-job back * Added retry for docker start * fix delay * update knob description * Update agent Docker methods - reworked Docker methods with retries behavior - changed InvokeWithRetryIfNonZero on to incapsulated logic method * - renamed local variable as is in ADO * Added docker retry behavior to docker version method Co-authored-by: Your Name <you@example.com> Co-authored-by: Kirill Ivlev <102740624+kirill-ivlev@users.noreply.github.com> Co-authored-by: Merlyn Oppenheim <merlynop@microsoft.com> * Add the exact operation system version and environment to the telemetry (#4104) * Update TaskRunner.cs * Update build-job.yml * Added new variables to telemetry AgentName, MachineName, IsSelfHosted Co-authored-by: Kirill Ivlev <102740624+kirill-ivlev@users.noreply.github.com> * Mingit version upgraded to 2.39.1 (#4126) * Fix agent lint errors (#4117) * Fix CA2000 error with secrets masker * Fix CA1711 for ServiceBootFlag * Fix CA2000 for JobRunner * Resolve CA2000 in StepHost * Resolve CA2000 in Windows service * Resolve CA2000 for WorkerCommandManager * Resove CA2000 for CodeCoverageCommands * Remove unused namespaces * Add fixture for SecretMasker tests * Change Fixtures -> Disposable * Use dispose pattern to fix errors * Formatting * Revert "Allow env.sh to accept additional env var names (#3339)" (#4143) This reverts commit `27e3d4e177`. * Bump azure pipelines task lib to 4.2.0 (#4155) * Add warning for deprecated runners (#4162) * Add warning for deprecated runners * Update strings.json * fix typo * Update NodeHandler.cs * FEATURE 2019014 - Gather telemetry on Agent Azure & Docker Container usage (#4166) * FEATURE 2019014 - Gather telemetry on Agent Azure & Docker Container usage * Re-write try-catch * Update TaskRunner.cs * cleanup * Distinguish AzureInstanceMetadata detection to a separate class --------- Co-authored-by: Kirill Ivlev <102740624+kirill-ivlev@users.noreply.github.com> * Added legacy lib source for support on Ubuntu 22.04 (#4161) * Moved logic to drain queues after each task under knob "AGENT_DRAIN_QUEUES_AFTER_TASK" (#4176) * Added knob "DrainQueuesAfterTask" with default "false" * move logic to drain queues behind knob DrainQueuesAfterTask * added mocking of GetScopedEnvironment method in step execution context * added mocking of GetScopedEnvironment method in step execution context * changed default value for knob "AGENT_DRAIN_QUEUES_AFTER_TASK" to true * implementd AsBooleanStrict and check in JobRunner * Revert "implementd AsBooleanStrict and check in JobRunner" This reverts commit `a1c0c67889`. * Reversed the knob * Add issue templates (#4154) * Add issue templates * Update config.yml * update templates * Update config.yml * Added knob to break pipeline unless user opt-in (#4165) * Added knob to force agent fail if user doesn't opt-in updates on unsupported OS * Resolved PR comments --------- Co-authored-by: Kirill Ivlev <102740624+kirill-ivlev@users.noreply.github.com> * Localization update (#4187) * Added files for onelocbuild * Update localize-pipeline.yml for Azure Pipelines * Fixed typo in resource.resx path * Removed en-US * Update localize-pipeline.yml for Azure Pipelines * LEGO: check in for Localization to temporary branch. (#3246) * LEGO: check in for Localization to temporary branch. (#3268) * LEGO: check in for Localization to temporary branch. (#3265) * LEGO: check in for Localization to temporary branch. (#3279) * Temporary renamed localization folders * Renamed localization folders * Set up schedule and notifications for the localization pipeline (#3269) * [localization] Fixed localization pipeline issue with already localized strings replaced (#3332) * Localized file check-in by OneLocBuild Task: Build definition ID 10944: Build ID 14646614 Localized file check-in by OneLocBuild Task * LEGO: check in for Localization to temporary branch. (#3363) Co-authored-by: csigs <csigs@outlook.com> * LEGO: check in for Localization to temporary branch. (#3364) Co-authored-by: csigs <csigs@outlook.com> * Localized file check-in by OneLocBuild Task: Build definition ID 10944: Build ID 14919763 Localized file check-in by OneLocBuild Task * Create PR in OneLocBuild task only on third week of sprint (#3374) * Fix localization pipeline * Add missed change * Added option to disable PR creation * update localize pipeline (#3637) Co-authored-by: Ilya Kuleshov <v-ikuleshov@microsoft.com> * Localization update (#3612) * Removing Localize folder * RHEL/CentOS 7 Fix for Lets Encrypt Change (#3601) * Revert "RHEL/CentOS 7 Fix for Lets Encrypt Change (#3601)" This reverts commit `629921e0fa`. * Revert "Removing Localize folder" This reverts commit `33cb0ae7c1`. Co-authored-by: AndreyIvanov42 <93121155+AndreyIvanov42@users.noreply.github.com> Co-authored-by: kuleshovilya <87485027+kuleshovilya@users.noreply.github.com> Co-authored-by: Ilya Kuleshov <v-ikuleshov@microsoft.com> * Removed OneLocBuild folder * Move notifications about Agent Localization PR from Slack to MS Teams - Part 1 (#3744) * Include strings from Misc/src/layoutbin to LocProject file (#4021) * Update LocProject file * Update CopyOption for LocProject * Localized file check-in by OneLocBuild Task: Build definition ID 10944: Build ID 18665127 (#4022) * Revert "Include strings from Misc/src/layoutbin to LocProject file (#4021)" This reverts commit `8f5c0c05b3`. * Revert "Localized file check-in by OneLocBuild Task: Build definition ID 10944: Build ID 18665127 (#4022)" (#4023) This reverts commit `b9c714bdba`. * Include layoutbin into locproject * Localized file check-in by OneLocBuild Task: Build definition ID 10944: Build ID 18680899 (#4028) * Juno: check in to lego/hb_5733438c-38c5-41ae-9557-a9d9754c4535_20221104175414554. (#4030) * Juno: check in to lego/hb_5733438c-38c5-41ae-9557-a9d9754c4535_20221105085259861. (#4031) * Juno: check in to lego/hb_5733438c-38c5-41ae-9557-a9d9754c4535_20221106085521421. (#4032) * Localized file check-in by OneLocBuild Task: Build definition ID 10944: Build ID 18697046 (#4033) * Juno: check in to lego/hb_5733438c-38c5-41ae-9557-a9d9754c4535_20221107143113520. (#4039) * Localized file check-in by OneLocBuild Task: Build definition ID 10944: Build ID 18706620 (#4044) * Localized file check-in by OneLocBuild Task: Build definition ID 10944: Build ID 19415018 (#4163) * Juno: check in to lego/hb_5733438c-38c5-41ae-9557-a9d9754c4535_20230223095051399. (#4168) * Juno: check in to lego/hb_5733438c-38c5-41ae-9557-a9d9754c4535_20230224095040576. (#4170) * Localized file check-in by OneLocBuild Task: Build definition ID 10944: Build ID 19476421 (#4179) * Update loc strings * Localized file check-in by OneLocBuild Task: Build definition ID 10944: Build ID 19476736 (#4181) * Juno: check in to lego/hb_5733438c-38c5-41ae-9557-a9d9754c4535_20230305095015478. (#4183) * Juno: check in to lego/hb_5733438c-38c5-41ae-9557-a9d9754c4535_20230306095225210. (#4184) * Removing Localize and OneLocBuild folder --------- Co-authored-by: Anatolii Bolshakov (Akvelon INC) <v-anbols@microsoft.com> Co-authored-by: csigs <csigs@users.noreply.github.com> Co-authored-by: Egor Bryzgalov <v-egbryz@microsoft.com> Co-authored-by: Anatoly Bolshakov <anatoly.bolshakov@akvelon.com> Co-authored-by: csigs <csigs@outlook.com> Co-authored-by: Nikita Ezzhev <v-niezz@microsoft.com> Co-authored-by: kuleshovilya <87485027+kuleshovilya@users.noreply.github.com> Co-authored-by: Ilya Kuleshov <v-ikuleshov@microsoft.com> Co-authored-by: AndreyIvanov42 <93121155+AndreyIvanov42@users.noreply.github.com> Co-authored-by: Denis Tikhomirov <90906678+denis-tikhomirov@users.noreply.github.com> Co-authored-by: KonstantinTyukalov <v-ktyukalov@microsoft.com> * Add "Mariner" to list of .NET 6 supporting systems (#4197) * Cover unsupported OS failure with FF (#4196) * Cover unsupported OS failure with FF * Update strings.json * Update JobRunner.cs * Update JobRunner.cs * Add knob to enable fetching net6.json file from GitHub (#4200) * Added knob "AGENT_ENABLE_FETCHING_NET6_LIST" * implemented knob EnableFetchingNet6List in function GetNet6SupportedSystems * Removed RuntimeKnobSource for EnableFetchingNet6List * removed logic which fails pipeline if there is some error occurred during checking if system supports .NET 6 * fixed typo * Update autoAssignABTT.yml (#4201) * Fixed typos in strings.json (#3523) * Added trimmed user secret to masker dictionary (#4195) * Add trimmed secret value to masker dictionary * [Refactor] Update MinSecretLength property. (#4137) * [Refactor] Update MinSecretLength property. Removed exception throwing, Added MinSecretLengthLimit public property * Update test MinValue test * Update test * add ability to disable publishing tests metadata (#4209) * moved logic of checking if system supports .NET 6 to JobExtension (#4207) * Increase limit for AZP_IGNORE_SECRETS_SHORTER_THAN knob to 6 (#4214) * Update secret masker length limit * Make property static * Decrease limit to 6 * Update tests * Update tests * Update comment --------- Co-authored-by: Kirill Ivlev <102740624+kirill-ivlev@users.noreply.github.com> * Refactor logic to drain queues after each task (#4213) * move draining queues before calling method "Complete" * Revert "BUG 1972388: vsbuild task in YAML build pipeline hangs forever in ADO even though the task has already logged completion (#3979)" This reverts commit `b3cf2c0e31`. * implemented logic to drain web console and timeline queues after each task * corrected typo Co-authored-by: Konstantin Tyukalov <52399739+KonstantinTyukalov@users.noreply.github.com> * renamed variable "drain" to "shouldDrain" --------- Co-authored-by: Konstantin Tyukalov <52399739+KonstantinTyukalov@users.noreply.github.com> Co-authored-by: Kirill Ivlev <102740624+kirill-ivlev@users.noreply.github.com> * git lfs version upgraded (#4218) * Add noderunner.md (#4202) * Add noderunner.md * Minor fixes * Update noderunner.md * Update noderunner.md * Resolve comments * Minor fixes in noderunner doc * minor fixes * Bug 2030875 - azure-pipelines-task-lib repo logs passwords in plain text (#4223) * added logic to unescape vso command in case there is encoded symbols * Added "TaskCommandHelper" class with method "AddSecret" which checks if decoding percents disabled and adds secret with decoded percents as well * fixed issue with test SetEndpointAuthParameter * moved logic to add decoded secret under check if value is empty * Update AzureInstanceMetadataProvider.cs * Update agentversion --------- Co-authored-by: Roman-Shchukin <111063382+Roman-Shchukin@users.noreply.github.com> Co-authored-by: Konstantin Tyukalov <52399739+KonstantinTyukalov@users.noreply.github.com> Co-authored-by: Steve <hez2010@outlook.com> Co-authored-by: Denis Rumyantsev <mr.denis.rumyantsev@gmail.com> Co-authored-by: Rick Brown <ricksbrown73@gmail.com> Co-authored-by: Liliia Sabitova <107196662+LiliaSabitova@users.noreply.github.com> Co-authored-by: Eric van Wijk <eric@van-wijk.com> Co-authored-by: Sergey Koryshev <sergey.koryshev@akvelon.com> Co-authored-by: Konstantin Tyukalov <v-ktyukalov@microsoft.com> Co-authored-by: Rami <icnocop@users.noreply.github.com> Co-authored-by: Martin <modermatt@tuta.io> Co-authored-by: ivanduplenskikh <115665590+ivanduplenskikh@users.noreply.github.com> Co-authored-by: Roman Shchukin <roman.shchukin@akvelon.com> Co-authored-by: Your Name <you@example.com> Co-authored-by: Merlyn Oppenheim <merlynop@microsoft.com> Co-authored-by: İsmayıl İsmayılov <110806089+ismayilov-ismayil@users.noreply.github.com> Co-authored-by: Maksim Petrov <47208721+vmapetr@users.noreply.github.com> Co-authored-by: Anatolii Bolshakov (Akvelon INC) <v-anbols@microsoft.com> Co-authored-by: csigs <csigs@users.noreply.github.com> Co-authored-by: Egor Bryzgalov <v-egbryz@microsoft.com> Co-authored-by: Anatoly Bolshakov <anatoly.bolshakov@akvelon.com> Co-authored-by: csigs <csigs@outlook.com> Co-authored-by: Nikita Ezzhev <v-niezz@microsoft.com> Co-authored-by: kuleshovilya <87485027+kuleshovilya@users.noreply.github.com> Co-authored-by: Ilya Kuleshov <v-ikuleshov@microsoft.com> Co-authored-by: AndreyIvanov42 <93121155+AndreyIvanov42@users.noreply.github.com> Co-authored-by: Denis Tikhomirov <90906678+denis-tikhomirov@users.noreply.github.com> Co-authored-by: Maxim Zaytsev <v-mazayt@microsoft.com> Co-authored-by: Rob Hensley <hensleyrob@gmail.com> * Upd dotnet6 agent before merge with master (#4252) * Catch updating exception (#4082) * Node16 version updated to 16.17.1" (#4084) * Bump ubuntu image version to 20.04 (#4079) * Prevent creating host container network (#3788) * Prevent creating host container network * Remove redundant empty lines * Fix invalid knob value fetching * Update ContainerOperationProvider.cs Co-authored-by: Denis Rumyantsev <mr.denis.rumyantsev@gmail.com> Co-authored-by: Kirill Ivlev <102740624+kirill-ivlev@users.noreply.github.com> Co-authored-by: Konstantin Tyukalov <52399739+KonstantinTyukalov@users.noreply.github.com> * Allow env.sh to accept additional env var names (#3339) The hard-coded list is not necessarily enough. This allows the caller to specify additional environment variables to write to .env. Co-authored-by: Konstantin Tyukalov <52399739+KonstantinTyukalov@users.noreply.github.com> Co-authored-by: Kirill Ivlev <102740624+kirill-ivlev@users.noreply.github.com> * Revert "Fix execution of scripts in variables (#4016)" (#4088) This reverts commit `af5328b265`. * Add Ubuntu 18.04 (#3943) Co-authored-by: Kirill Ivlev <102740624+kirill-ivlev@users.noreply.github.com> Co-authored-by: Konstantin Tyukalov <52399739+KonstantinTyukalov@users.noreply.github.com> * Make agent update list of systems supporting .NET 6 periodically (#4081) * implemented logic to fetch net6.json from server every hour * added await keyword * Moved logic to set warning from ExecutionContext to JobRunner since InitializeJob method is not async * fixed issue with condition in Equals method of OperatingSystem class * small refactoring * net6.json must be read only if it was not fetched from server * refactored code of method "GetNet6SupportedSystems" * restored field net6SupportedSystems to reduce IO oprations and cover case when agent started but net6.json is not older than 1 hour Co-authored-by: Kirill Ivlev <102740624+kirill-ivlev@users.noreply.github.com> * Retries to lock Services database (#4085) * retries to lock database * Decrease timeout + Add retry message Co-authored-by: Konstantin Tyukalov <v-ktyukalov@microsoft.com> Co-authored-by: Konstantin Tyukalov <52399739+KonstantinTyukalov@users.noreply.github.com> * Fixed name of DockerAdditionalNetworkOptions Knob (#3888) Co-authored-by: Konstantin Tyukalov <52399739+KonstantinTyukalov@users.noreply.github.com> * Format agent source code (#4101) * Run dotnet format * Remove extra semicolons * Fix SonarQube's "'Any()' should be used to test for emptiness" / Code Smell (#3404) * Add script to predict v2->v3 upgrade compatibility (#4092) * Add script to predict v2->v3 upgrade compatibility * Fix typo * Fixed another typo: frpom Co-authored-by: Kirill Ivlev <102740624+kirill-ivlev@users.noreply.github.com> * Added strict boolean knob values (#4103) * Add ConvertToBooleanStrict * Add AsBooleanStrict for agent knobs * update build-job (#4110) * update build-job - added UseDotNet task for linux os * Switch EsrpCodeSigning from V1 to V2 returned back UseDotNet task condition * Set UseDotNet version to 6 on osx * added dotnet 3.1.x before unit tests on win x86 * fix dotnet host search (#4124) * fix dotnet host search - added performMultiLevelLookup * Added runtime to unitTests step * Changed patch version in semver of runtime * remove sdk and runtime from unitTests script * Fixed problem with .NET Core 3.1 installation * Added UseDotNet task in functionalTests Co-authored-by: Roman Shchukin <roman.shchukin@akvelon.com> * Add option to skip rhel6 in pipeline and ci (#4121) * skipRhelRelease parameter added * skipRhelRelease parameter fixed * releaseRhelOnly parameter added * Logic of removing not filling lines from releaseNote * Fix pipeline with test releaseNote job * Fix pipeline with test releaseNote job * Fix pipeline with test releaseNote job * Fix pipeline with test releaseNote job * Fix pipeline with test releaseNote job * Fix pipeline with test releaseNote job * Fix pipeline with test releaseNote job * Fix pipeline with test releaseNote job * Fix pipeline with test releaseNote job * Fix pipeline with test releaseNote job * Removed test logs * Added clear empty lines logic * Commented code for test. Test changes in pipeline * Removed test changes * skipRhelRelease parametr added to ci pipeline * Update dotnet to 3.1.32 (#4112) * DockerLogin retries (#4100) * DockerLogin retries - added changes from PR microsoft/azure-pipelines-agent#3840 - fixed incorrect variables * DockerLogin retries - added step UseDotNet@2 * DockerLogin retries - added linux to UseDotNet task * Docker Login retries Implement retry into DockerLogin method * Implement feature flag for docker login retries * revert build-job back * Added retry for docker start * fix delay * update knob description * Update agent Docker methods - reworked Docker methods with retries behavior - changed InvokeWithRetryIfNonZero on to incapsulated logic method * - renamed local variable as is in ADO * Added docker retry behavior to docker version method Co-authored-by: Your Name <you@example.com> Co-authored-by: Kirill Ivlev <102740624+kirill-ivlev@users.noreply.github.com> Co-authored-by: Merlyn Oppenheim <merlynop@microsoft.com> * Add the exact operation system version and environment to the telemetry (#4104) * Update TaskRunner.cs * Update build-job.yml * Added new variables to telemetry AgentName, MachineName, IsSelfHosted Co-authored-by: Kirill Ivlev <102740624+kirill-ivlev@users.noreply.github.com> * Mingit version upgraded to 2.39.1 (#4126) * Fix agent lint errors (#4117) * Fix CA2000 error with secrets masker * Fix CA1711 for ServiceBootFlag * Fix CA2000 for JobRunner * Resolve CA2000 in StepHost * Resolve CA2000 in Windows service * Resolve CA2000 for WorkerCommandManager * Resove CA2000 for CodeCoverageCommands * Remove unused namespaces * Add fixture for SecretMasker tests * Change Fixtures -> Disposable * Use dispose pattern to fix errors * Formatting * Revert "Allow env.sh to accept additional env var names (#3339)" (#4143) This reverts commit `27e3d4e177`. * Bump azure pipelines task lib to 4.2.0 (#4155) * Add warning for deprecated runners (#4162) * Add warning for deprecated runners * Update strings.json * fix typo * Update NodeHandler.cs * FEATURE 2019014 - Gather telemetry on Agent Azure & Docker Container usage (#4166) * FEATURE 2019014 - Gather telemetry on Agent Azure & Docker Container usage * Re-write try-catch * Update TaskRunner.cs * cleanup * Distinguish AzureInstanceMetadata detection to a separate class --------- Co-authored-by: Kirill Ivlev <102740624+kirill-ivlev@users.noreply.github.com> * Added legacy lib source for support on Ubuntu 22.04 (#4161) * Moved logic to drain queues after each task under knob "AGENT_DRAIN_QUEUES_AFTER_TASK" (#4176) * Added knob "DrainQueuesAfterTask" with default "false" * move logic to drain queues behind knob DrainQueuesAfterTask * added mocking of GetScopedEnvironment method in step execution context * added mocking of GetScopedEnvironment method in step execution context * changed default value for knob "AGENT_DRAIN_QUEUES_AFTER_TASK" to true * implementd AsBooleanStrict and check in JobRunner * Revert "implementd AsBooleanStrict and check in JobRunner" This reverts commit `a1c0c67889`. * Reversed the knob * Add issue templates (#4154) * Add issue templates * Update config.yml * update templates * Update config.yml * Added knob to break pipeline unless user opt-in (#4165) * Added knob to force agent fail if user doesn't opt-in updates on unsupported OS * Resolved PR comments --------- Co-authored-by: Kirill Ivlev <102740624+kirill-ivlev@users.noreply.github.com> * Localization update (#4187) * Added files for onelocbuild * Update localize-pipeline.yml for Azure Pipelines * Fixed typo in resource.resx path * Removed en-US * Update localize-pipeline.yml for Azure Pipelines * LEGO: check in for Localization to temporary branch. (#3246) * LEGO: check in for Localization to temporary branch. (#3268) * LEGO: check in for Localization to temporary branch. (#3265) * LEGO: check in for Localization to temporary branch. (#3279) * Temporary renamed localization folders * Renamed localization folders * Set up schedule and notifications for the localization pipeline (#3269) * [localization] Fixed localization pipeline issue with already localized strings replaced (#3332) * Localized file check-in by OneLocBuild Task: Build definition ID 10944: Build ID 14646614 Localized file check-in by OneLocBuild Task * LEGO: check in for Localization to temporary branch. (#3363) Co-authored-by: csigs <csigs@outlook.com> * LEGO: check in for Localization to temporary branch. (#3364) Co-authored-by: csigs <csigs@outlook.com> * Localized file check-in by OneLocBuild Task: Build definition ID 10944: Build ID 14919763 Localized file check-in by OneLocBuild Task * Create PR in OneLocBuild task only on third week of sprint (#3374) * Fix localization pipeline * Add missed change * Added option to disable PR creation * update localize pipeline (#3637) Co-authored-by: Ilya Kuleshov <v-ikuleshov@microsoft.com> * Localization update (#3612) * Removing Localize folder * RHEL/CentOS 7 Fix for Lets Encrypt Change (#3601) * Revert "RHEL/CentOS 7 Fix for Lets Encrypt Change (#3601)" This reverts commit `629921e0fa`. * Revert "Removing Localize folder" This reverts commit `33cb0ae7c1`. Co-authored-by: AndreyIvanov42 <93121155+AndreyIvanov42@users.noreply.github.com> Co-authored-by: kuleshovilya <87485027+kuleshovilya@users.noreply.github.com> Co-authored-by: Ilya Kuleshov <v-ikuleshov@microsoft.com> * Removed OneLocBuild folder * Move notifications about Agent Localization PR from Slack to MS Teams - Part 1 (#3744) * Include strings from Misc/src/layoutbin to LocProject file (#4021) * Update LocProject file * Update CopyOption for LocProject * Localized file check-in by OneLocBuild Task: Build definition ID 10944: Build ID 18665127 (#4022) * Revert "Include strings from Misc/src/layoutbin to LocProject file (#4021)" This reverts commit `8f5c0c05b3`. * Revert "Localized file check-in by OneLocBuild Task: Build definition ID 10944: Build ID 18665127 (#4022)" (#4023) This reverts commit `b9c714bdba`. * Include layoutbin into locproject * Localized file check-in by OneLocBuild Task: Build definition ID 10944: Build ID 18680899 (#4028) * Juno: check in to lego/hb_5733438c-38c5-41ae-9557-a9d9754c4535_20221104175414554. (#4030) * Juno: check in to lego/hb_5733438c-38c5-41ae-9557-a9d9754c4535_20221105085259861. (#4031) * Juno: check in to lego/hb_5733438c-38c5-41ae-9557-a9d9754c4535_20221106085521421. (#4032) * Localized file check-in by OneLocBuild Task: Build definition ID 10944: Build ID 18697046 (#4033) * Juno: check in to lego/hb_5733438c-38c5-41ae-9557-a9d9754c4535_20221107143113520. (#4039) * Localized file check-in by OneLocBuild Task: Build definition ID 10944: Build ID 18706620 (#4044) * Localized file check-in by OneLocBuild Task: Build definition ID 10944: Build ID 19415018 (#4163) * Juno: check in to lego/hb_5733438c-38c5-41ae-9557-a9d9754c4535_20230223095051399. (#4168) * Juno: check in to lego/hb_5733438c-38c5-41ae-9557-a9d9754c4535_20230224095040576. (#4170) * Localized file check-in by OneLocBuild Task: Build definition ID 10944: Build ID 19476421 (#4179) * Update loc strings * Localized file check-in by OneLocBuild Task: Build definition ID 10944: Build ID 19476736 (#4181) * Juno: check in to lego/hb_5733438c-38c5-41ae-9557-a9d9754c4535_20230305095015478. (#4183) * Juno: check in to lego/hb_5733438c-38c5-41ae-9557-a9d9754c4535_20230306095225210. (#4184) * Removing Localize and OneLocBuild folder --------- Co-authored-by: Anatolii Bolshakov (Akvelon INC) <v-anbols@microsoft.com> Co-authored-by: csigs <csigs@users.noreply.github.com> Co-authored-by: Egor Bryzgalov <v-egbryz@microsoft.com> Co-authored-by: Anatoly Bolshakov <anatoly.bolshakov@akvelon.com> Co-authored-by: csigs <csigs@outlook.com> Co-authored-by: Nikita Ezzhev <v-niezz@microsoft.com> Co-authored-by: kuleshovilya <87485027+kuleshovilya@users.noreply.github.com> Co-authored-by: Ilya Kuleshov <v-ikuleshov@microsoft.com> Co-authored-by: AndreyIvanov42 <93121155+AndreyIvanov42@users.noreply.github.com> Co-authored-by: Denis Tikhomirov <90906678+denis-tikhomirov@users.noreply.github.com> Co-authored-by: KonstantinTyukalov <v-ktyukalov@microsoft.com> * Add "Mariner" to list of .NET 6 supporting systems (#4197) * Cover unsupported OS failure with FF (#4196) * Cover unsupported OS failure with FF * Update strings.json * Update JobRunner.cs * Update JobRunner.cs * Add knob to enable fetching net6.json file from GitHub (#4200) * Added knob "AGENT_ENABLE_FETCHING_NET6_LIST" * implemented knob EnableFetchingNet6List in function GetNet6SupportedSystems * Removed RuntimeKnobSource for EnableFetchingNet6List * removed logic which fails pipeline if there is some error occurred during checking if system supports .NET 6 * fixed typo * Update autoAssignABTT.yml (#4201) * Fixed typos in strings.json (#3523) * Added trimmed user secret to masker dictionary (#4195) * Add trimmed secret value to masker dictionary * [Refactor] Update MinSecretLength property. (#4137) * [Refactor] Update MinSecretLength property. Removed exception throwing, Added MinSecretLengthLimit public property * Update test MinValue test * Update test * add ability to disable publishing tests metadata (#4209) * moved logic of checking if system supports .NET 6 to JobExtension (#4207) * Increase limit for AZP_IGNORE_SECRETS_SHORTER_THAN knob to 6 (#4214) * Update secret masker length limit * Make property static * Decrease limit to 6 * Update tests * Update tests * Update comment --------- Co-authored-by: Kirill Ivlev <102740624+kirill-ivlev@users.noreply.github.com> * Refactor logic to drain queues after each task (#4213) * move draining queues before calling method "Complete" * Revert "BUG 1972388: vsbuild task in YAML build pipeline hangs forever in ADO even though the task has already logged completion (#3979)" This reverts commit `b3cf2c0e31`. * implemented logic to drain web console and timeline queues after each task * corrected typo Co-authored-by: Konstantin Tyukalov <52399739+KonstantinTyukalov@users.noreply.github.com> * renamed variable "drain" to "shouldDrain" --------- Co-authored-by: Konstantin Tyukalov <52399739+KonstantinTyukalov@users.noreply.github.com> Co-authored-by: Kirill Ivlev <102740624+kirill-ivlev@users.noreply.github.com> * git lfs version upgraded (#4218) * Add noderunner.md (#4202) * Add noderunner.md * Minor fixes * Update noderunner.md * Update noderunner.md * Resolve comments * Minor fixes in noderunner doc * minor fixes * Bug 2030875 - azure-pipelines-task-lib repo logs passwords in plain text (#4223) * added logic to unescape vso command in case there is encoded symbols * Added "TaskCommandHelper" class with method "AddSecret" which checks if decoding percents disabled and adds secret with decoded percents as well * fixed issue with test SetEndpointAuthParameter * moved logic to add decoded secret under check if value is empty * Fix: Remove invalid chars from filename (#3831) Co-authored-by: Konstantin Tyukalov <52399739+KonstantinTyukalov@users.noreply.github.com> * Add force of _tasks creation by knob (#4236) * Upgrade vss api netcore version, has BuildXL changes (#4220) * Upgrade vss api netcore version, has BuildXL changes * Adding Nuget authenticate task * Nuget auth task before the build * Try v0 * Remove nuget auth task * Nuget auth * Remove auth task * Remove redundant security package declaration * Adding System.Security.Cryptography.ProtectedData back * Adding cred provider * Removing redundant system.buffers reference * Supress warnings * Remove unused variable * Removing HashType.64k hardcoding * update packages * Update agentversion --------- Co-authored-by: Roman-Shchukin <111063382+Roman-Shchukin@users.noreply.github.com> Co-authored-by: Konstantin Tyukalov <52399739+KonstantinTyukalov@users.noreply.github.com> Co-authored-by: Steve <hez2010@outlook.com> Co-authored-by: Denis Rumyantsev <mr.denis.rumyantsev@gmail.com> Co-authored-by: Rick Brown <ricksbrown73@gmail.com> Co-authored-by: Liliia Sabitova <107196662+LiliaSabitova@users.noreply.github.com> Co-authored-by: Eric van Wijk <eric@van-wijk.com> Co-authored-by: Sergey Koryshev <sergey.koryshev@akvelon.com> Co-authored-by: Konstantin Tyukalov <v-ktyukalov@microsoft.com> Co-authored-by: Rami <icnocop@users.noreply.github.com> Co-authored-by: Martin <modermatt@tuta.io> Co-authored-by: ivanduplenskikh <115665590+ivanduplenskikh@users.noreply.github.com> Co-authored-by: Roman Shchukin <roman.shchukin@akvelon.com> Co-authored-by: Your Name <you@example.com> Co-authored-by: Merlyn Oppenheim <merlynop@microsoft.com> Co-authored-by: İsmayıl İsmayılov <110806089+ismayilov-ismayil@users.noreply.github.com> Co-authored-by: Maksim Petrov <47208721+vmapetr@users.noreply.github.com> Co-authored-by: Anatolii Bolshakov (Akvelon INC) <v-anbols@microsoft.com> Co-authored-by: csigs <csigs@users.noreply.github.com> Co-authored-by: Egor Bryzgalov <v-egbryz@microsoft.com> Co-authored-by: Anatoly Bolshakov <anatoly.bolshakov@akvelon.com> Co-authored-by: csigs <csigs@outlook.com> Co-authored-by: Nikita Ezzhev <v-niezz@microsoft.com> Co-authored-by: kuleshovilya <87485027+kuleshovilya@users.noreply.github.com> Co-authored-by: Ilya Kuleshov <v-ikuleshov@microsoft.com> Co-authored-by: AndreyIvanov42 <93121155+AndreyIvanov42@users.noreply.github.com> Co-authored-by: Denis Tikhomirov <90906678+denis-tikhomirov@users.noreply.github.com> Co-authored-by: Maxim Zaytsev <v-mazayt@microsoft.com> Co-authored-by: Rob Hensley <hensleyrob@gmail.com> Co-authored-by: Mihai Ratoiu <89085590+mihaiandreiratoiu@users.noreply.github.com> Co-authored-by: fadnavistanmay <43892780+fadnavistanmay@users.noreply.github.com> * +x src/dev.sh --------- Co-authored-by: Denis Rumyantsev <v-denisr@microsoft.com> Co-authored-by: Andrey Ivanov <v-andivanov@microsoft.com> Co-authored-by: AndreyIvanov42 <93121155+AndreyIvanov42@users.noreply.github.com> Co-authored-by: Max Podriezov <maxpodriezov@microsoft.com> Co-authored-by: Denis Rumyantsev <mr.denis.rumyantsev@gmail.com> Co-authored-by: Alexander Smolyakov <v-alsmo@microsoft.com> Co-authored-by: Anatoly Bolshakov <v-anbols@microsoft.com> Co-authored-by: Liliia Sabitova <107196662+LiliaSabitova@users.noreply.github.com> Co-authored-by: Sergey Koryshev <sergey.koryshev@akvelon.com> Co-authored-by: Konstantin Tyukalov <52399739+KonstantinTyukalov@users.noreply.github.com> Co-authored-by: Konstantin Tyukalov <v-ktyukalov@microsoft.com> Co-authored-by: Liliia Sabitova <v-lisabitova@microsoft.com> Co-authored-by: Roman-Shchukin <111063382+Roman-Shchukin@users.noreply.github.com> Co-authored-by: Steve <hez2010@outlook.com> Co-authored-by: Rick Brown <ricksbrown73@gmail.com> Co-authored-by: Eric van Wijk <eric@van-wijk.com> Co-authored-by: Rami <icnocop@users.noreply.github.com> Co-authored-by: Martin <modermatt@tuta.io> Co-authored-by: ivanduplenskikh <115665590+ivanduplenskikh@users.noreply.github.com> Co-authored-by: Roman Shchukin <roman.shchukin@akvelon.com> Co-authored-by: Your Name <you@example.com> Co-authored-by: Merlyn Oppenheim <merlynop@microsoft.com> Co-authored-by: İsmayıl İsmayılov <110806089+ismayilov-ismayil@users.noreply.github.com> Co-authored-by: Maksim Petrov <47208721+vmapetr@users.noreply.github.com> Co-authored-by: csigs <csigs@users.noreply.github.com> Co-authored-by: Egor Bryzgalov <v-egbryz@microsoft.com> Co-authored-by: Anatoly Bolshakov <anatoly.bolshakov@akvelon.com> Co-authored-by: csigs <csigs@outlook.com> Co-authored-by: Nikita Ezzhev <v-niezz@microsoft.com> Co-authored-by: kuleshovilya <87485027+kuleshovilya@users.noreply.github.com> Co-authored-by: Ilya Kuleshov <v-ikuleshov@microsoft.com> Co-authored-by: Denis Tikhomirov <90906678+denis-tikhomirov@users.noreply.github.com> Co-authored-by: Maxim Zaytsev <v-mazayt@microsoft.com> Co-authored-by: Rob Hensley <hensleyrob@gmail.com> Co-authored-by: Mihai Ratoiu <89085590+mihaiandreiratoiu@users.noreply.github.com> Co-authored-by: fadnavistanmay <43892780+fadnavistanmay@users.noreply.github.com>		2023-04-27 15:50:22 +04:00
..
CredScanPatternExtractor.cs	Added some additional, automated secrets maskers (#2959 )	2020-05-11 13:58:48 -04:00
CredScanRegexes.csproj	Added some additional, automated secrets maskers (#2959 )	2020-05-11 13:58:48 -04:00
PatternProcessor.cs	Added some additional, automated secrets maskers (#2959 )	2020-05-11 13:58:48 -04:00
Program.cs	Added some additional, automated secrets maskers (#2959 )	2020-05-11 13:58:48 -04:00
nuget.config	Resolve Security Analysis warnings (#3393 )	2021-05-07 11:35:55 -04:00
readme.md	Update master with dotnet 6 (#4254 )	2023-04-27 15:50:22 +04:00

readme.md

CredScan Regex Extractor

CredScan, an internal Microsoft tool, detects credentials using a variety of mechanisms. Its goal is to keep those credentials from ending up in the wrong place: a Git repository, test logs, etc. It would be perfect for helping scrub build & release logs.

CredScan is not available publicly, so we cannot ship it with the Pipelines agent. We have secured permission to ship a small subset of CredScan -- its set of battle-tested regexes for detecting common credential formats. This tool extracts the regexes from CredScan's knowledge base and outputs them in a format suitable for shipping with the agent.

Requirements:

.NET 6.0 or higher
The Azure Artifacts credential provider
Access to the "msazure" org's CredScanSDK feed

To run this tool:

Extract the credential provider to the right place on your system
dotnet build --interactive (so you get prompted to auth to the feed)
dotnet run > ../../src/Microsoft.VisualStudio.Services.Agent/AdditionalMaskingRegexes.CredScan.cs

You must check in the generated code after running this tool. At the time of writing, the target file is src/Microsoft.VisualStudio.Services.Agent/AdditionalMaskingRegexes.CredScan.cs.