azure-sdk-for-js/eng/pipelines/docindex.yml

178 строки
6.9 KiB
YAML
Исходник Обычный вид История

trigger: none
variables:
- template: /eng/pipelines/templates/variables/globals.yml
jobs:
- template: /eng/common/pipelines/templates/jobs/docindex.yml
Validate docs using the REX tool (#27935) Fixes #25684 * Removes docker container * Updates relevant scripts and removes old code paths * Adds job to upgrade the REX tool if tests pass Example runs: * Successful run of daily pipeline: https://dev.azure.com/azure-sdk/internal/_build/results?buildId=3298420&view=logs&j=f8e040b3-c0ff-5789-0eda-99daaaa3fc1b&t=7702a9c8-4b61-5cd3-c28d-9b1cea81bf7e * Successful SDK package release: https://dev.azure.com/azure-sdk/internal/_build/results?buildId=3302248&view=logs&j=19b87903-f8bc-5cbb-22d9-4f6eb3469d8e&t=39d7f548-7a77-5d57-b2f3-bf0cc5f8dda2&l=150 * Successful docindex run: * pipeline run: https://dev.azure.com/azure-sdk/internal/_build/results?buildId=3298899&view=logs&j=dc056dfc-c0cf-5958-c8c4-8da4f91a3739&t=dc056dfc-c0cf-5958-c8c4-8da4f91a3739 * diff showing package upgrades did not change (may show drift as packages release, focus on `ci-configs/packages-*.json`): https://github.com/MicrosoftDocs/azure-docs-sdk-node/compare/djurek/rex-tests?expand=1 * Tool upgrade: * No new version: https://dev.azure.com/azure-sdk/internal/_build/results?buildId=3298685&view=logs&j=f0b9ce84-b5fb-5cd2-d7e4-8e2b722448a2&t=95e002a9-090a-5230-9ea9-b23402ed153e * New version test and upgrade: https://dev.azure.com/azure-sdk/internal/_build/results?buildId=3298739&view=logs&j=f0b9ce84-b5fb-5cd2-d7e4-8e2b722448a2&t=95e002a9-090a-5230-9ea9-b23402ed153e * PR: https://github.com/Azure/azure-sdk-for-js/pull/27917 * New version test fails: https://dev.azure.com/azure-sdk/internal/_build/results?buildId=3298727&view=logs&j=f0b9ce84-b5fb-5cd2-d7e4-8e2b722448a2&t=95e002a9-090a-5230-9ea9-b23402ed153e
2023-12-01 00:40:53 +03:00
- job: UpgradeRexTool
timeoutInMinutes: 10
pool:
vmImage: ubuntu-22.04
steps:
- template: /eng/common/pipelines/templates/steps/sparse-checkout.yml
parameters:
SkipCheckoutNone: true
- task: Powershell@2
inputs:
pwsh: true
filePath: eng/scripts/docs/Update-RexValidationTool.ps1
displayName: Update JS REX Validation Tool
- template: /eng/common/pipelines/templates/steps/create-pull-request.yml
parameters:
PRBranchName: docs/rex-tool-upgrade
# Force push as this task updates a single file and history is not
# necessary for changes in this branch.
PushArgs: -f
PRTitle: Upgrade JS REX Validation Tool
PRBody: |
This PR upgrades the JS REX Validation Tool to the latest version.
Testing was done in $(System.CollectionUri)$(System.TeamProject)/_build/results?buildId=$(Build.BuildId)
- job: UpdateDocsMsBuildConfig
timeoutInMinutes: 90
pool:
vmImage: ubuntu-20.04
variables:
DocRepoLocation: $(Pipeline.Workspace)/docs
DocRepoOwner: MicrosoftDocs
DocRepoName: azure-docs-sdk-node
steps:
# Sync docs repo (this can be sparse)
- template: /eng/common/pipelines/templates/steps/sparse-checkout.yml
parameters:
SkipCheckoutNone: true
Paths:
- ci-configs/
Daily reference docs for JS (#15749) * Add dev version to Save-Package-Properties.ps1 and includd in build process * Now with fewer pipelines * Add daily docs.ms work to docindex.yml * Update-DocsMsMetadata and required functions * Common * Docs meatadata release and daily docs build * eq * Loop over artifacts in paramaters.Artifacts * Add PackageInfo to path * download: current * Output artifacts * Pool * Use relative paths in package properties json file and use dev version as Version (if available) when setting metadata * Skip checking out docs repo if it's already checked out * Don't overcomplicate docs-metadata-release.yml * With parameters properly defined and redundant logic removed * Use TargetBranchName * Retain original version to differentiate where metadata and readme docs end up * Add docs repo support for GetDocsMetadata: first cut * Sparse checkout needs to include metadata * Add metadata from packages that aren't tracked in the CSV file yet * Use "dev" if the package has a "dev" version in JS. This will not work in other languages and shouldn't be put into eng/common. That will need to be refactored into logic that could be in Language-Settings.ps1 * Refactor and use sparse checkout everywhere * Refactor long path support * Correct path * Parameter names * Remove unnecessary parameter * Close quote * Add pipeline starting * Keep the single quotes * Output link to CI build * Refactor: * Rename docs-metadata-release2.yml" to "update-docsms-metadata.yml" * Documentation for Update-DocsMsMetadata.ps1 * Documentation for Update-DocsMsPackages.ps1 * Add function invocation to Update-DocsMsMetadata.ps1 to override the package metadata from PackageInfo before it's written into the metadata folder * Refactor logic for generating daily branch name * Add ability to override daily docs branch name at queue time * Apply suggestions from code review Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com> * Review feedback: Save-Package-Properties.ps1 * Update comments for path normalization * Change path normalization logic to not switch current location * Update docs * Add ability to specify variable name for default branch * Try convertToJson * Tab scope * Do doc artifact info loop in Update-DocsMsMetadata.ps1 instead of yaml template * Single quotes * Remove template loop * Remove extra template references * Move convertToJson * Try different formatting given convertToJson * Reverse Update-DocsMsMetadata.ps1 to operate on a single object. Looping can happen in yaml * Loop in yaml, Rename Update-DocsMsMetadata parameter * Try re-formatting yaml * Spell parameters properly * Missing backtick * Remove extra logging * First cut at default implementation * Move business logic inside Update-DocsMsMetadata.ps1, PowerShell can handle an array or a single item in the cmdlet parameters * Use proper function name * PackageProps has a DevVersion * No DefaultImplementation in this change. This should be part of a wider effort. * Straggling master -> main rename * Use existing devops starting logic * Remove extra condition * Fix backslashes * Correct number of backslashes * Use variable for branch name * Update eng/scripts/Language-Settings.ps1 Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com> * Review feedback * Do not replace the default branch with the tag in links when doing a Dev build (pointing to the default branch is preferable and the tag does not exist) * Conditions on docs version for template project * Do not run PublishDocsToNightlyBranch if TestPipeline is true * Performance improvements, switch to Linux to take advantage of paths that don't look like escape sequences to ConvertFrom-Json * Add markdown files to sparse checkout * Reproduce failure for template publishing * Remove eng, it's already there by default * Fix instances where metadata might not be available because of filtering Hide == true (e.g. in template packages). The only information consumed from CSV metadata at this time is the service name which gets normalized and placed in the package overview metadata. Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com>
2021-07-01 21:41:25 +03:00
- metadata/
- docs-ref-mapping/
- docs-ref-services/
Repositories:
- Name: $(DocRepoOwner)/$(DocRepoName)
WorkingDirectory: $(DocRepoLocation)
Validate docs using the REX tool (#27935) Fixes #25684 * Removes docker container * Updates relevant scripts and removes old code paths * Adds job to upgrade the REX tool if tests pass Example runs: * Successful run of daily pipeline: https://dev.azure.com/azure-sdk/internal/_build/results?buildId=3298420&view=logs&j=f8e040b3-c0ff-5789-0eda-99daaaa3fc1b&t=7702a9c8-4b61-5cd3-c28d-9b1cea81bf7e * Successful SDK package release: https://dev.azure.com/azure-sdk/internal/_build/results?buildId=3302248&view=logs&j=19b87903-f8bc-5cbb-22d9-4f6eb3469d8e&t=39d7f548-7a77-5d57-b2f3-bf0cc5f8dda2&l=150 * Successful docindex run: * pipeline run: https://dev.azure.com/azure-sdk/internal/_build/results?buildId=3298899&view=logs&j=dc056dfc-c0cf-5958-c8c4-8da4f91a3739&t=dc056dfc-c0cf-5958-c8c4-8da4f91a3739 * diff showing package upgrades did not change (may show drift as packages release, focus on `ci-configs/packages-*.json`): https://github.com/MicrosoftDocs/azure-docs-sdk-node/compare/djurek/rex-tests?expand=1 * Tool upgrade: * No new version: https://dev.azure.com/azure-sdk/internal/_build/results?buildId=3298685&view=logs&j=f0b9ce84-b5fb-5cd2-d7e4-8e2b722448a2&t=95e002a9-090a-5230-9ea9-b23402ed153e * New version test and upgrade: https://dev.azure.com/azure-sdk/internal/_build/results?buildId=3298739&view=logs&j=f0b9ce84-b5fb-5cd2-d7e4-8e2b722448a2&t=95e002a9-090a-5230-9ea9-b23402ed153e * PR: https://github.com/Azure/azure-sdk-for-js/pull/27917 * New version test fails: https://dev.azure.com/azure-sdk/internal/_build/results?buildId=3298727&view=logs&j=f0b9ce84-b5fb-5cd2-d7e4-8e2b722448a2&t=95e002a9-090a-5230-9ea9-b23402ed153e
2023-12-01 00:40:53 +03:00
- template: /eng/pipelines/templates/steps/install-rex-validation-tool.yml
- task: Powershell@2
inputs:
pwsh: true
filePath: eng/common/scripts/Update-DocsMsPackageMonikers.ps1
arguments: -DocRepoLocation $(DocRepoLocation)
displayName: Move deprecated packages to legacy moniker
condition: and(succeeded(), or(eq(variables['Build.Reason'], 'Schedule'), eq(variables['Force.MainUpdate'], 'true')))
- task: Powershell@2
inputs:
pwsh: true
filePath: eng/common/scripts/Update-DocsMsPackages.ps1
Validate docs using the REX tool (#27935) Fixes #25684 * Removes docker container * Updates relevant scripts and removes old code paths * Adds job to upgrade the REX tool if tests pass Example runs: * Successful run of daily pipeline: https://dev.azure.com/azure-sdk/internal/_build/results?buildId=3298420&view=logs&j=f8e040b3-c0ff-5789-0eda-99daaaa3fc1b&t=7702a9c8-4b61-5cd3-c28d-9b1cea81bf7e * Successful SDK package release: https://dev.azure.com/azure-sdk/internal/_build/results?buildId=3302248&view=logs&j=19b87903-f8bc-5cbb-22d9-4f6eb3469d8e&t=39d7f548-7a77-5d57-b2f3-bf0cc5f8dda2&l=150 * Successful docindex run: * pipeline run: https://dev.azure.com/azure-sdk/internal/_build/results?buildId=3298899&view=logs&j=dc056dfc-c0cf-5958-c8c4-8da4f91a3739&t=dc056dfc-c0cf-5958-c8c4-8da4f91a3739 * diff showing package upgrades did not change (may show drift as packages release, focus on `ci-configs/packages-*.json`): https://github.com/MicrosoftDocs/azure-docs-sdk-node/compare/djurek/rex-tests?expand=1 * Tool upgrade: * No new version: https://dev.azure.com/azure-sdk/internal/_build/results?buildId=3298685&view=logs&j=f0b9ce84-b5fb-5cd2-d7e4-8e2b722448a2&t=95e002a9-090a-5230-9ea9-b23402ed153e * New version test and upgrade: https://dev.azure.com/azure-sdk/internal/_build/results?buildId=3298739&view=logs&j=f0b9ce84-b5fb-5cd2-d7e4-8e2b722448a2&t=95e002a9-090a-5230-9ea9-b23402ed153e * PR: https://github.com/Azure/azure-sdk-for-js/pull/27917 * New version test fails: https://dev.azure.com/azure-sdk/internal/_build/results?buildId=3298727&view=logs&j=f0b9ce84-b5fb-5cd2-d7e4-8e2b722448a2&t=95e002a9-090a-5230-9ea9-b23402ed153e
2023-12-01 00:40:53 +03:00
arguments: -DocRepoLocation $(DocRepoLocation)
displayName: Update Docs Onboarding for main branch
condition: and(succeeded(), or(eq(variables['Build.Reason'], 'Schedule'), eq(variables['Force.MainUpdate'], 'true')))
2022-07-20 20:23:10 +03:00
- task: Powershell@2
inputs:
pwsh: true
filePath: eng/common/scripts/Service-Level-Readme-Automation.ps1
arguments: -DocRepoLocation $(DocRepoLocation)
displayName: Generate Service Level Readme for main branch
condition: and(succeeded(), or(eq(variables['Build.Reason'], 'Schedule'), eq(variables['Force.MainUpdate'], 'true')))
2022-07-20 20:23:10 +03:00
- task: Powershell@2
inputs:
pwsh: true
filePath: eng/common/scripts/Update-DocsMsToc.ps1
arguments: >-
-DocRepoLocation $(DocRepoLocation)
-OutputLocation $(DocRepoLocation)/docs-ref-mapping/reference-unified.yml
displayName: Generate ToC for main branch
condition: and(succeeded(), or(eq(variables['Build.Reason'], 'Schedule'), eq(variables['Force.MainUpdate'], 'true')))
- task: Powershell@2
inputs:
pwsh: true
filePath: eng/common/scripts/Verify-RequiredDocsJsonMembers.ps1
arguments: >-
-DocRepoLocation $(DocRepoLocation)
Validate docs using the REX tool (#27935) Fixes #25684 * Removes docker container * Updates relevant scripts and removes old code paths * Adds job to upgrade the REX tool if tests pass Example runs: * Successful run of daily pipeline: https://dev.azure.com/azure-sdk/internal/_build/results?buildId=3298420&view=logs&j=f8e040b3-c0ff-5789-0eda-99daaaa3fc1b&t=7702a9c8-4b61-5cd3-c28d-9b1cea81bf7e * Successful SDK package release: https://dev.azure.com/azure-sdk/internal/_build/results?buildId=3302248&view=logs&j=19b87903-f8bc-5cbb-22d9-4f6eb3469d8e&t=39d7f548-7a77-5d57-b2f3-bf0cc5f8dda2&l=150 * Successful docindex run: * pipeline run: https://dev.azure.com/azure-sdk/internal/_build/results?buildId=3298899&view=logs&j=dc056dfc-c0cf-5958-c8c4-8da4f91a3739&t=dc056dfc-c0cf-5958-c8c4-8da4f91a3739 * diff showing package upgrades did not change (may show drift as packages release, focus on `ci-configs/packages-*.json`): https://github.com/MicrosoftDocs/azure-docs-sdk-node/compare/djurek/rex-tests?expand=1 * Tool upgrade: * No new version: https://dev.azure.com/azure-sdk/internal/_build/results?buildId=3298685&view=logs&j=f0b9ce84-b5fb-5cd2-d7e4-8e2b722448a2&t=95e002a9-090a-5230-9ea9-b23402ed153e * New version test and upgrade: https://dev.azure.com/azure-sdk/internal/_build/results?buildId=3298739&view=logs&j=f0b9ce84-b5fb-5cd2-d7e4-8e2b722448a2&t=95e002a9-090a-5230-9ea9-b23402ed153e * PR: https://github.com/Azure/azure-sdk-for-js/pull/27917 * New version test fails: https://dev.azure.com/azure-sdk/internal/_build/results?buildId=3298727&view=logs&j=f0b9ce84-b5fb-5cd2-d7e4-8e2b722448a2&t=95e002a9-090a-5230-9ea9-b23402ed153e
2023-12-01 00:40:53 +03:00
displayName: Verify Required Docs Json Members
# Push changes to docs repo
- template: /eng/common/pipelines/templates/steps/set-default-branch.yml
parameters:
WorkingDirectory: $(DocRepoLocation)
- template: /eng/common/pipelines/templates/steps/git-push-changes.yml
parameters:
BaseRepoBranch: $(DefaultBranch)
BaseRepoOwner: $(DocRepoOwner)
Validate docs using the REX tool (#27935) Fixes #25684 * Removes docker container * Updates relevant scripts and removes old code paths * Adds job to upgrade the REX tool if tests pass Example runs: * Successful run of daily pipeline: https://dev.azure.com/azure-sdk/internal/_build/results?buildId=3298420&view=logs&j=f8e040b3-c0ff-5789-0eda-99daaaa3fc1b&t=7702a9c8-4b61-5cd3-c28d-9b1cea81bf7e * Successful SDK package release: https://dev.azure.com/azure-sdk/internal/_build/results?buildId=3302248&view=logs&j=19b87903-f8bc-5cbb-22d9-4f6eb3469d8e&t=39d7f548-7a77-5d57-b2f3-bf0cc5f8dda2&l=150 * Successful docindex run: * pipeline run: https://dev.azure.com/azure-sdk/internal/_build/results?buildId=3298899&view=logs&j=dc056dfc-c0cf-5958-c8c4-8da4f91a3739&t=dc056dfc-c0cf-5958-c8c4-8da4f91a3739 * diff showing package upgrades did not change (may show drift as packages release, focus on `ci-configs/packages-*.json`): https://github.com/MicrosoftDocs/azure-docs-sdk-node/compare/djurek/rex-tests?expand=1 * Tool upgrade: * No new version: https://dev.azure.com/azure-sdk/internal/_build/results?buildId=3298685&view=logs&j=f0b9ce84-b5fb-5cd2-d7e4-8e2b722448a2&t=95e002a9-090a-5230-9ea9-b23402ed153e * New version test and upgrade: https://dev.azure.com/azure-sdk/internal/_build/results?buildId=3298739&view=logs&j=f0b9ce84-b5fb-5cd2-d7e4-8e2b722448a2&t=95e002a9-090a-5230-9ea9-b23402ed153e * PR: https://github.com/Azure/azure-sdk-for-js/pull/27917 * New version test fails: https://dev.azure.com/azure-sdk/internal/_build/results?buildId=3298727&view=logs&j=f0b9ce84-b5fb-5cd2-d7e4-8e2b722448a2&t=95e002a9-090a-5230-9ea9-b23402ed153e
2023-12-01 00:40:53 +03:00
CommitMsg: "Update docs CI configuration Build: $(System.CollectionUri)$(System.TeamProject)/_build/results?buildId=$(Build.BuildId)"
TargetRepoName: $(DocRepoName)
TargetRepoOwner: $(DocRepoOwner)
WorkingDirectory: $(DocRepoLocation)
Daily reference docs for JS (#15749) * Add dev version to Save-Package-Properties.ps1 and includd in build process * Now with fewer pipelines * Add daily docs.ms work to docindex.yml * Update-DocsMsMetadata and required functions * Common * Docs meatadata release and daily docs build * eq * Loop over artifacts in paramaters.Artifacts * Add PackageInfo to path * download: current * Output artifacts * Pool * Use relative paths in package properties json file and use dev version as Version (if available) when setting metadata * Skip checking out docs repo if it's already checked out * Don't overcomplicate docs-metadata-release.yml * With parameters properly defined and redundant logic removed * Use TargetBranchName * Retain original version to differentiate where metadata and readme docs end up * Add docs repo support for GetDocsMetadata: first cut * Sparse checkout needs to include metadata * Add metadata from packages that aren't tracked in the CSV file yet * Use "dev" if the package has a "dev" version in JS. This will not work in other languages and shouldn't be put into eng/common. That will need to be refactored into logic that could be in Language-Settings.ps1 * Refactor and use sparse checkout everywhere * Refactor long path support * Correct path * Parameter names * Remove unnecessary parameter * Close quote * Add pipeline starting * Keep the single quotes * Output link to CI build * Refactor: * Rename docs-metadata-release2.yml" to "update-docsms-metadata.yml" * Documentation for Update-DocsMsMetadata.ps1 * Documentation for Update-DocsMsPackages.ps1 * Add function invocation to Update-DocsMsMetadata.ps1 to override the package metadata from PackageInfo before it's written into the metadata folder * Refactor logic for generating daily branch name * Add ability to override daily docs branch name at queue time * Apply suggestions from code review Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com> * Review feedback: Save-Package-Properties.ps1 * Update comments for path normalization * Change path normalization logic to not switch current location * Update docs * Add ability to specify variable name for default branch * Try convertToJson * Tab scope * Do doc artifact info loop in Update-DocsMsMetadata.ps1 instead of yaml template * Single quotes * Remove template loop * Remove extra template references * Move convertToJson * Try different formatting given convertToJson * Reverse Update-DocsMsMetadata.ps1 to operate on a single object. Looping can happen in yaml * Loop in yaml, Rename Update-DocsMsMetadata parameter * Try re-formatting yaml * Spell parameters properly * Missing backtick * Remove extra logging * First cut at default implementation * Move business logic inside Update-DocsMsMetadata.ps1, PowerShell can handle an array or a single item in the cmdlet parameters * Use proper function name * PackageProps has a DevVersion * No DefaultImplementation in this change. This should be part of a wider effort. * Straggling master -> main rename * Use existing devops starting logic * Remove extra condition * Fix backslashes * Correct number of backslashes * Use variable for branch name * Update eng/scripts/Language-Settings.ps1 Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com> * Review feedback * Do not replace the default branch with the tag in links when doing a Dev build (pointing to the default branch is preferable and the tag does not exist) * Conditions on docs version for template project * Do not run PublishDocsToNightlyBranch if TestPipeline is true * Performance improvements, switch to Linux to take advantage of paths that don't look like escape sequences to ConvertFrom-Json * Add markdown files to sparse checkout * Reproduce failure for template publishing * Remove eng, it's already there by default * Fix instances where metadata might not be available because of filtering Hide == true (e.g. in template packages). The only information consumed from CSV metadata at this time is the service name which gets normalized and placed in the package overview metadata. Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com>
2021-07-01 21:41:25 +03:00
# Prepare daily docs CI
- template: /eng/common/pipelines/templates/steps/set-daily-docs-branch-name.yml
parameters:
DailyBranchVariableName: DailyDocsBranchName
- pwsh: |
$ErrorActionPreference = "Continue"
git checkout "origin/$(DailyDocsBranchName)" 2>&1 | Out-Null
$LASTEXITCODE = 0 # This ignores any error from git checkout
git status
displayName: Checkout daily branch if it exists
workingDirectory: $(DocRepoLocation)
- task: Powershell@2
inputs:
pwsh: true
filePath: eng/common/scripts/Update-DocsMsPackages.ps1
2021-12-10 06:19:31 +03:00
arguments: -DocRepoLocation $(DocRepoLocation)
displayName: Update Docs Onboarding for Daily docs
- task: Powershell@2
inputs:
pwsh: true
filePath: eng/common/scripts/Service-Level-Readme-Automation.ps1
arguments: -DocRepoLocation $(DocRepoLocation)
displayName: Generate Service Level Readme for Daily docs
- task: Powershell@2
inputs:
pwsh: true
filePath: eng/common/scripts/Update-DocsMsToc.ps1
arguments: >-
-DocRepoLocation $(DocRepoLocation)
-OutputLocation $(DocRepoLocation)/docs-ref-mapping/reference-unified.yml
displayName: Generate ToC for Daily docs
- task: Powershell@2
inputs:
pwsh: true
filePath: eng/common/scripts/Verify-RequiredDocsJsonMembers.ps1
arguments: >-
-DocRepoLocation $(DocRepoLocation)
displayName: Verify Required Docs Json Members
Daily reference docs for JS (#15749) * Add dev version to Save-Package-Properties.ps1 and includd in build process * Now with fewer pipelines * Add daily docs.ms work to docindex.yml * Update-DocsMsMetadata and required functions * Common * Docs meatadata release and daily docs build * eq * Loop over artifacts in paramaters.Artifacts * Add PackageInfo to path * download: current * Output artifacts * Pool * Use relative paths in package properties json file and use dev version as Version (if available) when setting metadata * Skip checking out docs repo if it's already checked out * Don't overcomplicate docs-metadata-release.yml * With parameters properly defined and redundant logic removed * Use TargetBranchName * Retain original version to differentiate where metadata and readme docs end up * Add docs repo support for GetDocsMetadata: first cut * Sparse checkout needs to include metadata * Add metadata from packages that aren't tracked in the CSV file yet * Use "dev" if the package has a "dev" version in JS. This will not work in other languages and shouldn't be put into eng/common. That will need to be refactored into logic that could be in Language-Settings.ps1 * Refactor and use sparse checkout everywhere * Refactor long path support * Correct path * Parameter names * Remove unnecessary parameter * Close quote * Add pipeline starting * Keep the single quotes * Output link to CI build * Refactor: * Rename docs-metadata-release2.yml" to "update-docsms-metadata.yml" * Documentation for Update-DocsMsMetadata.ps1 * Documentation for Update-DocsMsPackages.ps1 * Add function invocation to Update-DocsMsMetadata.ps1 to override the package metadata from PackageInfo before it's written into the metadata folder * Refactor logic for generating daily branch name * Add ability to override daily docs branch name at queue time * Apply suggestions from code review Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com> * Review feedback: Save-Package-Properties.ps1 * Update comments for path normalization * Change path normalization logic to not switch current location * Update docs * Add ability to specify variable name for default branch * Try convertToJson * Tab scope * Do doc artifact info loop in Update-DocsMsMetadata.ps1 instead of yaml template * Single quotes * Remove template loop * Remove extra template references * Move convertToJson * Try different formatting given convertToJson * Reverse Update-DocsMsMetadata.ps1 to operate on a single object. Looping can happen in yaml * Loop in yaml, Rename Update-DocsMsMetadata parameter * Try re-formatting yaml * Spell parameters properly * Missing backtick * Remove extra logging * First cut at default implementation * Move business logic inside Update-DocsMsMetadata.ps1, PowerShell can handle an array or a single item in the cmdlet parameters * Use proper function name * PackageProps has a DevVersion * No DefaultImplementation in this change. This should be part of a wider effort. * Straggling master -> main rename * Use existing devops starting logic * Remove extra condition * Fix backslashes * Correct number of backslashes * Use variable for branch name * Update eng/scripts/Language-Settings.ps1 Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com> * Review feedback * Do not replace the default branch with the tag in links when doing a Dev build (pointing to the default branch is preferable and the tag does not exist) * Conditions on docs version for template project * Do not run PublishDocsToNightlyBranch if TestPipeline is true * Performance improvements, switch to Linux to take advantage of paths that don't look like escape sequences to ConvertFrom-Json * Add markdown files to sparse checkout * Reproduce failure for template publishing * Remove eng, it's already there by default * Fix instances where metadata might not be available because of filtering Hide == true (e.g. in template packages). The only information consumed from CSV metadata at this time is the service name which gets normalized and placed in the package overview metadata. Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com>
2021-07-01 21:41:25 +03:00
- template: /eng/common/pipelines/templates/steps/git-push-changes.yml
parameters:
BaseRepoBranch: $(DailyDocsBranchName)
BaseRepoOwner: $(DocRepoOwner)
CommitMsg: "Update targeting packages based on release metadata. (Daily docs)"
TargetRepoName: $(DocRepoName)
TargetRepoOwner: $(DocRepoOwner)
WorkingDirectory: $(DocRepoLocation)
ScriptDirectory: $(Build.SourcesDirectory)/eng/common/scripts
PushArgs: -f
Daily reference docs for JS (#15749) * Add dev version to Save-Package-Properties.ps1 and includd in build process * Now with fewer pipelines * Add daily docs.ms work to docindex.yml * Update-DocsMsMetadata and required functions * Common * Docs meatadata release and daily docs build * eq * Loop over artifacts in paramaters.Artifacts * Add PackageInfo to path * download: current * Output artifacts * Pool * Use relative paths in package properties json file and use dev version as Version (if available) when setting metadata * Skip checking out docs repo if it's already checked out * Don't overcomplicate docs-metadata-release.yml * With parameters properly defined and redundant logic removed * Use TargetBranchName * Retain original version to differentiate where metadata and readme docs end up * Add docs repo support for GetDocsMetadata: first cut * Sparse checkout needs to include metadata * Add metadata from packages that aren't tracked in the CSV file yet * Use "dev" if the package has a "dev" version in JS. This will not work in other languages and shouldn't be put into eng/common. That will need to be refactored into logic that could be in Language-Settings.ps1 * Refactor and use sparse checkout everywhere * Refactor long path support * Correct path * Parameter names * Remove unnecessary parameter * Close quote * Add pipeline starting * Keep the single quotes * Output link to CI build * Refactor: * Rename docs-metadata-release2.yml" to "update-docsms-metadata.yml" * Documentation for Update-DocsMsMetadata.ps1 * Documentation for Update-DocsMsPackages.ps1 * Add function invocation to Update-DocsMsMetadata.ps1 to override the package metadata from PackageInfo before it's written into the metadata folder * Refactor logic for generating daily branch name * Add ability to override daily docs branch name at queue time * Apply suggestions from code review Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com> * Review feedback: Save-Package-Properties.ps1 * Update comments for path normalization * Change path normalization logic to not switch current location * Update docs * Add ability to specify variable name for default branch * Try convertToJson * Tab scope * Do doc artifact info loop in Update-DocsMsMetadata.ps1 instead of yaml template * Single quotes * Remove template loop * Remove extra template references * Move convertToJson * Try different formatting given convertToJson * Reverse Update-DocsMsMetadata.ps1 to operate on a single object. Looping can happen in yaml * Loop in yaml, Rename Update-DocsMsMetadata parameter * Try re-formatting yaml * Spell parameters properly * Missing backtick * Remove extra logging * First cut at default implementation * Move business logic inside Update-DocsMsMetadata.ps1, PowerShell can handle an array or a single item in the cmdlet parameters * Use proper function name * PackageProps has a DevVersion * No DefaultImplementation in this change. This should be part of a wider effort. * Straggling master -> main rename * Use existing devops starting logic * Remove extra condition * Fix backslashes * Correct number of backslashes * Use variable for branch name * Update eng/scripts/Language-Settings.ps1 Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com> * Review feedback * Do not replace the default branch with the tag in links when doing a Dev build (pointing to the default branch is preferable and the tag does not exist) * Conditions on docs version for template project * Do not run PublishDocsToNightlyBranch if TestPipeline is true * Performance improvements, switch to Linux to take advantage of paths that don't look like escape sequences to ConvertFrom-Json * Add markdown files to sparse checkout * Reproduce failure for template publishing * Remove eng, it's already there by default * Fix instances where metadata might not be available because of filtering Hide == true (e.g. in template packages). The only information consumed from CSV metadata at this time is the service name which gets normalized and placed in the package overview metadata. Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com>
2021-07-01 21:41:25 +03:00
- task: PowerShell@2
displayName: Queue Docs CI build
inputs:
pwsh: true
filePath: eng/common/scripts/Queue-Pipeline.ps1
arguments: >
-Organization "apidrop"
-Project "Content%20CI"
-DefinitionId 3452
-AuthToken "$(azuresdk-apidrop-devops-queue-build-pat)"
Daily reference docs for JS (#15749) * Add dev version to Save-Package-Properties.ps1 and includd in build process * Now with fewer pipelines * Add daily docs.ms work to docindex.yml * Update-DocsMsMetadata and required functions * Common * Docs meatadata release and daily docs build * eq * Loop over artifacts in paramaters.Artifacts * Add PackageInfo to path * download: current * Output artifacts * Pool * Use relative paths in package properties json file and use dev version as Version (if available) when setting metadata * Skip checking out docs repo if it's already checked out * Don't overcomplicate docs-metadata-release.yml * With parameters properly defined and redundant logic removed * Use TargetBranchName * Retain original version to differentiate where metadata and readme docs end up * Add docs repo support for GetDocsMetadata: first cut * Sparse checkout needs to include metadata * Add metadata from packages that aren't tracked in the CSV file yet * Use "dev" if the package has a "dev" version in JS. This will not work in other languages and shouldn't be put into eng/common. That will need to be refactored into logic that could be in Language-Settings.ps1 * Refactor and use sparse checkout everywhere * Refactor long path support * Correct path * Parameter names * Remove unnecessary parameter * Close quote * Add pipeline starting * Keep the single quotes * Output link to CI build * Refactor: * Rename docs-metadata-release2.yml" to "update-docsms-metadata.yml" * Documentation for Update-DocsMsMetadata.ps1 * Documentation for Update-DocsMsPackages.ps1 * Add function invocation to Update-DocsMsMetadata.ps1 to override the package metadata from PackageInfo before it's written into the metadata folder * Refactor logic for generating daily branch name * Add ability to override daily docs branch name at queue time * Apply suggestions from code review Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com> * Review feedback: Save-Package-Properties.ps1 * Update comments for path normalization * Change path normalization logic to not switch current location * Update docs * Add ability to specify variable name for default branch * Try convertToJson * Tab scope * Do doc artifact info loop in Update-DocsMsMetadata.ps1 instead of yaml template * Single quotes * Remove template loop * Remove extra template references * Move convertToJson * Try different formatting given convertToJson * Reverse Update-DocsMsMetadata.ps1 to operate on a single object. Looping can happen in yaml * Loop in yaml, Rename Update-DocsMsMetadata parameter * Try re-formatting yaml * Spell parameters properly * Missing backtick * Remove extra logging * First cut at default implementation * Move business logic inside Update-DocsMsMetadata.ps1, PowerShell can handle an array or a single item in the cmdlet parameters * Use proper function name * PackageProps has a DevVersion * No DefaultImplementation in this change. This should be part of a wider effort. * Straggling master -> main rename * Use existing devops starting logic * Remove extra condition * Fix backslashes * Correct number of backslashes * Use variable for branch name * Update eng/scripts/Language-Settings.ps1 Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com> * Review feedback * Do not replace the default branch with the tag in links when doing a Dev build (pointing to the default branch is preferable and the tag does not exist) * Conditions on docs version for template project * Do not run PublishDocsToNightlyBranch if TestPipeline is true * Performance improvements, switch to Linux to take advantage of paths that don't look like escape sequences to ConvertFrom-Json * Add markdown files to sparse checkout * Reproduce failure for template publishing * Remove eng, it's already there by default * Fix instances where metadata might not be available because of filtering Hide == true (e.g. in template packages). The only information consumed from CSV metadata at this time is the service name which gets normalized and placed in the package overview metadata. Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com>
2021-07-01 21:41:25 +03:00
-BuildParametersJson '{"params":"{ \"target_repo\": { \"url\": \"https://github.com/MicrosoftDocs/azure-docs-sdk-node\", \"branch\": \"$(DailyDocsBranchName)\", \"folder\": \"./\" }, \"source_repos\": [] }"}'