azure-sdk-for-js/eng/pipelines/docindex.yml

156 строки
6.2 KiB
YAML
Исходник Обычный вид История

trigger: none
variables:
- template: /eng/pipelines/templates/variables/globals.yml
jobs:
- template: /eng/common/pipelines/templates/jobs/docindex.yml
- job: UpdateDocsMsBuildConfig
timeoutInMinutes: 90
pool:
vmImage: ubuntu-20.04
variables:
DocRepoLocation: $(Pipeline.Workspace)/docs
DocRepoOwner: MicrosoftDocs
DocRepoName: azure-docs-sdk-node
DocValidationImageId: azuresdkimages.azurecr.io/jsrefautocr:latest
steps:
# Sync docs repo (this can be sparse)
- template: /eng/common/pipelines/templates/steps/sparse-checkout.yml
parameters:
SkipCheckoutNone: true
Paths:
- ci-configs/
Daily reference docs for JS (#15749) * Add dev version to Save-Package-Properties.ps1 and includd in build process * Now with fewer pipelines * Add daily docs.ms work to docindex.yml * Update-DocsMsMetadata and required functions * Common * Docs meatadata release and daily docs build * eq * Loop over artifacts in paramaters.Artifacts * Add PackageInfo to path * download: current * Output artifacts * Pool * Use relative paths in package properties json file and use dev version as Version (if available) when setting metadata * Skip checking out docs repo if it's already checked out * Don't overcomplicate docs-metadata-release.yml * With parameters properly defined and redundant logic removed * Use TargetBranchName * Retain original version to differentiate where metadata and readme docs end up * Add docs repo support for GetDocsMetadata: first cut * Sparse checkout needs to include metadata * Add metadata from packages that aren't tracked in the CSV file yet * Use "dev" if the package has a "dev" version in JS. This will not work in other languages and shouldn't be put into eng/common. That will need to be refactored into logic that could be in Language-Settings.ps1 * Refactor and use sparse checkout everywhere * Refactor long path support * Correct path * Parameter names * Remove unnecessary parameter * Close quote * Add pipeline starting * Keep the single quotes * Output link to CI build * Refactor: * Rename docs-metadata-release2.yml" to "update-docsms-metadata.yml" * Documentation for Update-DocsMsMetadata.ps1 * Documentation for Update-DocsMsPackages.ps1 * Add function invocation to Update-DocsMsMetadata.ps1 to override the package metadata from PackageInfo before it's written into the metadata folder * Refactor logic for generating daily branch name * Add ability to override daily docs branch name at queue time * Apply suggestions from code review Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com> * Review feedback: Save-Package-Properties.ps1 * Update comments for path normalization * Change path normalization logic to not switch current location * Update docs * Add ability to specify variable name for default branch * Try convertToJson * Tab scope * Do doc artifact info loop in Update-DocsMsMetadata.ps1 instead of yaml template * Single quotes * Remove template loop * Remove extra template references * Move convertToJson * Try different formatting given convertToJson * Reverse Update-DocsMsMetadata.ps1 to operate on a single object. Looping can happen in yaml * Loop in yaml, Rename Update-DocsMsMetadata parameter * Try re-formatting yaml * Spell parameters properly * Missing backtick * Remove extra logging * First cut at default implementation * Move business logic inside Update-DocsMsMetadata.ps1, PowerShell can handle an array or a single item in the cmdlet parameters * Use proper function name * PackageProps has a DevVersion * No DefaultImplementation in this change. This should be part of a wider effort. * Straggling master -> main rename * Use existing devops starting logic * Remove extra condition * Fix backslashes * Correct number of backslashes * Use variable for branch name * Update eng/scripts/Language-Settings.ps1 Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com> * Review feedback * Do not replace the default branch with the tag in links when doing a Dev build (pointing to the default branch is preferable and the tag does not exist) * Conditions on docs version for template project * Do not run PublishDocsToNightlyBranch if TestPipeline is true * Performance improvements, switch to Linux to take advantage of paths that don't look like escape sequences to ConvertFrom-Json * Add markdown files to sparse checkout * Reproduce failure for template publishing * Remove eng, it's already there by default * Fix instances where metadata might not be available because of filtering Hide == true (e.g. in template packages). The only information consumed from CSV metadata at this time is the service name which gets normalized and placed in the package overview metadata. Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com>
2021-07-01 21:41:25 +03:00
- metadata/
- docs-ref-mapping/
- docs-ref-services/
Repositories:
- Name: $(DocRepoOwner)/$(DocRepoName)
WorkingDirectory: $(DocRepoLocation)
# Pull and build the docker image.
- template: /eng/common/pipelines/templates/steps/docker-pull-image.yml
parameters:
ContainerRegistryClientId: $(azuresdkimages-cr-clientid)
ContainerRegistryClientSecret: $(azuresdkimages-cr-clientsecret)
ImageId: "$(DocValidationImageId)"
- task: Powershell@2
inputs:
pwsh: true
filePath: eng/common/scripts/Update-DocsMsPackageMonikers.ps1
arguments: -DocRepoLocation $(DocRepoLocation)
displayName: Move deprecated packages to legacy moniker
condition: and(succeeded(), or(eq(variables['Build.Reason'], 'Schedule'), eq(variables['Force.MainUpdate'], 'true')))
- task: Powershell@2
inputs:
pwsh: true
filePath: eng/common/scripts/Update-DocsMsPackages.ps1
arguments: -DocRepoLocation $(DocRepoLocation) -ImageId '$(DocValidationImageId)'
displayName: Update Docs Onboarding for main branch
condition: and(succeeded(), or(eq(variables['Build.Reason'], 'Schedule'), eq(variables['Force.MainUpdate'], 'true')))
2022-07-20 20:23:10 +03:00
- task: Powershell@2
inputs:
pwsh: true
filePath: eng/common/scripts/Service-Level-Readme-Automation.ps1
arguments: -DocRepoLocation $(DocRepoLocation)
displayName: Generate Service Level Readme for main branch
condition: and(succeeded(), or(eq(variables['Build.Reason'], 'Schedule'), eq(variables['Force.MainUpdate'], 'true')))
2022-07-20 20:23:10 +03:00
- task: Powershell@2
inputs:
pwsh: true
filePath: eng/common/scripts/Update-DocsMsToc.ps1
arguments: >-
-DocRepoLocation $(DocRepoLocation)
-OutputLocation $(DocRepoLocation)/docs-ref-mapping/reference-unified.yml
displayName: Generate ToC for main branch
condition: and(succeeded(), or(eq(variables['Build.Reason'], 'Schedule'), eq(variables['Force.MainUpdate'], 'true')))
- task: Powershell@2
inputs:
pwsh: true
filePath: eng/common/scripts/Verify-RequiredDocsJsonMembers.ps1
arguments: >-
-DocRepoLocation $(DocRepoLocation)
displayName: Verify Required Docs Json Members
# Push changes to docs repo
- template: /eng/common/pipelines/templates/steps/set-default-branch.yml
parameters:
WorkingDirectory: $(DocRepoLocation)
- template: /eng/common/pipelines/templates/steps/git-push-changes.yml
parameters:
BaseRepoBranch: $(DefaultBranch)
BaseRepoOwner: $(DocRepoOwner)
CommitMsg: "Update docs CI configuration"
TargetRepoName: $(DocRepoName)
TargetRepoOwner: $(DocRepoOwner)
WorkingDirectory: $(DocRepoLocation)
Daily reference docs for JS (#15749) * Add dev version to Save-Package-Properties.ps1 and includd in build process * Now with fewer pipelines * Add daily docs.ms work to docindex.yml * Update-DocsMsMetadata and required functions * Common * Docs meatadata release and daily docs build * eq * Loop over artifacts in paramaters.Artifacts * Add PackageInfo to path * download: current * Output artifacts * Pool * Use relative paths in package properties json file and use dev version as Version (if available) when setting metadata * Skip checking out docs repo if it's already checked out * Don't overcomplicate docs-metadata-release.yml * With parameters properly defined and redundant logic removed * Use TargetBranchName * Retain original version to differentiate where metadata and readme docs end up * Add docs repo support for GetDocsMetadata: first cut * Sparse checkout needs to include metadata * Add metadata from packages that aren't tracked in the CSV file yet * Use "dev" if the package has a "dev" version in JS. This will not work in other languages and shouldn't be put into eng/common. That will need to be refactored into logic that could be in Language-Settings.ps1 * Refactor and use sparse checkout everywhere * Refactor long path support * Correct path * Parameter names * Remove unnecessary parameter * Close quote * Add pipeline starting * Keep the single quotes * Output link to CI build * Refactor: * Rename docs-metadata-release2.yml" to "update-docsms-metadata.yml" * Documentation for Update-DocsMsMetadata.ps1 * Documentation for Update-DocsMsPackages.ps1 * Add function invocation to Update-DocsMsMetadata.ps1 to override the package metadata from PackageInfo before it's written into the metadata folder * Refactor logic for generating daily branch name * Add ability to override daily docs branch name at queue time * Apply suggestions from code review Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com> * Review feedback: Save-Package-Properties.ps1 * Update comments for path normalization * Change path normalization logic to not switch current location * Update docs * Add ability to specify variable name for default branch * Try convertToJson * Tab scope * Do doc artifact info loop in Update-DocsMsMetadata.ps1 instead of yaml template * Single quotes * Remove template loop * Remove extra template references * Move convertToJson * Try different formatting given convertToJson * Reverse Update-DocsMsMetadata.ps1 to operate on a single object. Looping can happen in yaml * Loop in yaml, Rename Update-DocsMsMetadata parameter * Try re-formatting yaml * Spell parameters properly * Missing backtick * Remove extra logging * First cut at default implementation * Move business logic inside Update-DocsMsMetadata.ps1, PowerShell can handle an array or a single item in the cmdlet parameters * Use proper function name * PackageProps has a DevVersion * No DefaultImplementation in this change. This should be part of a wider effort. * Straggling master -> main rename * Use existing devops starting logic * Remove extra condition * Fix backslashes * Correct number of backslashes * Use variable for branch name * Update eng/scripts/Language-Settings.ps1 Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com> * Review feedback * Do not replace the default branch with the tag in links when doing a Dev build (pointing to the default branch is preferable and the tag does not exist) * Conditions on docs version for template project * Do not run PublishDocsToNightlyBranch if TestPipeline is true * Performance improvements, switch to Linux to take advantage of paths that don't look like escape sequences to ConvertFrom-Json * Add markdown files to sparse checkout * Reproduce failure for template publishing * Remove eng, it's already there by default * Fix instances where metadata might not be available because of filtering Hide == true (e.g. in template packages). The only information consumed from CSV metadata at this time is the service name which gets normalized and placed in the package overview metadata. Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com>
2021-07-01 21:41:25 +03:00
# Prepare daily docs CI
- template: /eng/common/pipelines/templates/steps/set-daily-docs-branch-name.yml
parameters:
DailyBranchVariableName: DailyDocsBranchName
- pwsh: |
$ErrorActionPreference = "Continue"
git checkout "origin/$(DailyDocsBranchName)" 2>&1 | Out-Null
$LASTEXITCODE = 0 # This ignores any error from git checkout
git status
displayName: Checkout daily branch if it exists
workingDirectory: $(DocRepoLocation)
- task: Powershell@2
inputs:
pwsh: true
filePath: eng/common/scripts/Update-DocsMsPackages.ps1
2021-12-10 06:19:31 +03:00
arguments: -DocRepoLocation $(DocRepoLocation)
displayName: Update Docs Onboarding for Daily docs
- task: Powershell@2
inputs:
pwsh: true
filePath: eng/common/scripts/Service-Level-Readme-Automation.ps1
arguments: -DocRepoLocation $(DocRepoLocation)
displayName: Generate Service Level Readme for Daily docs
- task: Powershell@2
inputs:
pwsh: true
filePath: eng/common/scripts/Update-DocsMsToc.ps1
arguments: >-
-DocRepoLocation $(DocRepoLocation)
-OutputLocation $(DocRepoLocation)/docs-ref-mapping/reference-unified.yml
displayName: Generate ToC for Daily docs
- task: Powershell@2
inputs:
pwsh: true
filePath: eng/common/scripts/Verify-RequiredDocsJsonMembers.ps1
arguments: >-
-DocRepoLocation $(DocRepoLocation)
displayName: Verify Required Docs Json Members
Daily reference docs for JS (#15749) * Add dev version to Save-Package-Properties.ps1 and includd in build process * Now with fewer pipelines * Add daily docs.ms work to docindex.yml * Update-DocsMsMetadata and required functions * Common * Docs meatadata release and daily docs build * eq * Loop over artifacts in paramaters.Artifacts * Add PackageInfo to path * download: current * Output artifacts * Pool * Use relative paths in package properties json file and use dev version as Version (if available) when setting metadata * Skip checking out docs repo if it's already checked out * Don't overcomplicate docs-metadata-release.yml * With parameters properly defined and redundant logic removed * Use TargetBranchName * Retain original version to differentiate where metadata and readme docs end up * Add docs repo support for GetDocsMetadata: first cut * Sparse checkout needs to include metadata * Add metadata from packages that aren't tracked in the CSV file yet * Use "dev" if the package has a "dev" version in JS. This will not work in other languages and shouldn't be put into eng/common. That will need to be refactored into logic that could be in Language-Settings.ps1 * Refactor and use sparse checkout everywhere * Refactor long path support * Correct path * Parameter names * Remove unnecessary parameter * Close quote * Add pipeline starting * Keep the single quotes * Output link to CI build * Refactor: * Rename docs-metadata-release2.yml" to "update-docsms-metadata.yml" * Documentation for Update-DocsMsMetadata.ps1 * Documentation for Update-DocsMsPackages.ps1 * Add function invocation to Update-DocsMsMetadata.ps1 to override the package metadata from PackageInfo before it's written into the metadata folder * Refactor logic for generating daily branch name * Add ability to override daily docs branch name at queue time * Apply suggestions from code review Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com> * Review feedback: Save-Package-Properties.ps1 * Update comments for path normalization * Change path normalization logic to not switch current location * Update docs * Add ability to specify variable name for default branch * Try convertToJson * Tab scope * Do doc artifact info loop in Update-DocsMsMetadata.ps1 instead of yaml template * Single quotes * Remove template loop * Remove extra template references * Move convertToJson * Try different formatting given convertToJson * Reverse Update-DocsMsMetadata.ps1 to operate on a single object. Looping can happen in yaml * Loop in yaml, Rename Update-DocsMsMetadata parameter * Try re-formatting yaml * Spell parameters properly * Missing backtick * Remove extra logging * First cut at default implementation * Move business logic inside Update-DocsMsMetadata.ps1, PowerShell can handle an array or a single item in the cmdlet parameters * Use proper function name * PackageProps has a DevVersion * No DefaultImplementation in this change. This should be part of a wider effort. * Straggling master -> main rename * Use existing devops starting logic * Remove extra condition * Fix backslashes * Correct number of backslashes * Use variable for branch name * Update eng/scripts/Language-Settings.ps1 Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com> * Review feedback * Do not replace the default branch with the tag in links when doing a Dev build (pointing to the default branch is preferable and the tag does not exist) * Conditions on docs version for template project * Do not run PublishDocsToNightlyBranch if TestPipeline is true * Performance improvements, switch to Linux to take advantage of paths that don't look like escape sequences to ConvertFrom-Json * Add markdown files to sparse checkout * Reproduce failure for template publishing * Remove eng, it's already there by default * Fix instances where metadata might not be available because of filtering Hide == true (e.g. in template packages). The only information consumed from CSV metadata at this time is the service name which gets normalized and placed in the package overview metadata. Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com>
2021-07-01 21:41:25 +03:00
- template: /eng/common/pipelines/templates/steps/git-push-changes.yml
parameters:
BaseRepoBranch: $(DailyDocsBranchName)
BaseRepoOwner: $(DocRepoOwner)
CommitMsg: "Update targeting packages based on release metadata. (Daily docs)"
TargetRepoName: $(DocRepoName)
TargetRepoOwner: $(DocRepoOwner)
WorkingDirectory: $(DocRepoLocation)
ScriptDirectory: $(Build.SourcesDirectory)/eng/common/scripts
PushArgs: -f
Daily reference docs for JS (#15749) * Add dev version to Save-Package-Properties.ps1 and includd in build process * Now with fewer pipelines * Add daily docs.ms work to docindex.yml * Update-DocsMsMetadata and required functions * Common * Docs meatadata release and daily docs build * eq * Loop over artifacts in paramaters.Artifacts * Add PackageInfo to path * download: current * Output artifacts * Pool * Use relative paths in package properties json file and use dev version as Version (if available) when setting metadata * Skip checking out docs repo if it's already checked out * Don't overcomplicate docs-metadata-release.yml * With parameters properly defined and redundant logic removed * Use TargetBranchName * Retain original version to differentiate where metadata and readme docs end up * Add docs repo support for GetDocsMetadata: first cut * Sparse checkout needs to include metadata * Add metadata from packages that aren't tracked in the CSV file yet * Use "dev" if the package has a "dev" version in JS. This will not work in other languages and shouldn't be put into eng/common. That will need to be refactored into logic that could be in Language-Settings.ps1 * Refactor and use sparse checkout everywhere * Refactor long path support * Correct path * Parameter names * Remove unnecessary parameter * Close quote * Add pipeline starting * Keep the single quotes * Output link to CI build * Refactor: * Rename docs-metadata-release2.yml" to "update-docsms-metadata.yml" * Documentation for Update-DocsMsMetadata.ps1 * Documentation for Update-DocsMsPackages.ps1 * Add function invocation to Update-DocsMsMetadata.ps1 to override the package metadata from PackageInfo before it's written into the metadata folder * Refactor logic for generating daily branch name * Add ability to override daily docs branch name at queue time * Apply suggestions from code review Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com> * Review feedback: Save-Package-Properties.ps1 * Update comments for path normalization * Change path normalization logic to not switch current location * Update docs * Add ability to specify variable name for default branch * Try convertToJson * Tab scope * Do doc artifact info loop in Update-DocsMsMetadata.ps1 instead of yaml template * Single quotes * Remove template loop * Remove extra template references * Move convertToJson * Try different formatting given convertToJson * Reverse Update-DocsMsMetadata.ps1 to operate on a single object. Looping can happen in yaml * Loop in yaml, Rename Update-DocsMsMetadata parameter * Try re-formatting yaml * Spell parameters properly * Missing backtick * Remove extra logging * First cut at default implementation * Move business logic inside Update-DocsMsMetadata.ps1, PowerShell can handle an array or a single item in the cmdlet parameters * Use proper function name * PackageProps has a DevVersion * No DefaultImplementation in this change. This should be part of a wider effort. * Straggling master -> main rename * Use existing devops starting logic * Remove extra condition * Fix backslashes * Correct number of backslashes * Use variable for branch name * Update eng/scripts/Language-Settings.ps1 Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com> * Review feedback * Do not replace the default branch with the tag in links when doing a Dev build (pointing to the default branch is preferable and the tag does not exist) * Conditions on docs version for template project * Do not run PublishDocsToNightlyBranch if TestPipeline is true * Performance improvements, switch to Linux to take advantage of paths that don't look like escape sequences to ConvertFrom-Json * Add markdown files to sparse checkout * Reproduce failure for template publishing * Remove eng, it's already there by default * Fix instances where metadata might not be available because of filtering Hide == true (e.g. in template packages). The only information consumed from CSV metadata at this time is the service name which gets normalized and placed in the package overview metadata. Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com>
2021-07-01 21:41:25 +03:00
- task: PowerShell@2
displayName: Queue Docs CI build
inputs:
pwsh: true
filePath: eng/common/scripts/Queue-Pipeline.ps1
arguments: >
-Organization "apidrop"
-Project "Content%20CI"
-DefinitionId 3452
-AuthToken "$(azuresdk-apidrop-devops-queue-build-pat)"
Daily reference docs for JS (#15749) * Add dev version to Save-Package-Properties.ps1 and includd in build process * Now with fewer pipelines * Add daily docs.ms work to docindex.yml * Update-DocsMsMetadata and required functions * Common * Docs meatadata release and daily docs build * eq * Loop over artifacts in paramaters.Artifacts * Add PackageInfo to path * download: current * Output artifacts * Pool * Use relative paths in package properties json file and use dev version as Version (if available) when setting metadata * Skip checking out docs repo if it's already checked out * Don't overcomplicate docs-metadata-release.yml * With parameters properly defined and redundant logic removed * Use TargetBranchName * Retain original version to differentiate where metadata and readme docs end up * Add docs repo support for GetDocsMetadata: first cut * Sparse checkout needs to include metadata * Add metadata from packages that aren't tracked in the CSV file yet * Use "dev" if the package has a "dev" version in JS. This will not work in other languages and shouldn't be put into eng/common. That will need to be refactored into logic that could be in Language-Settings.ps1 * Refactor and use sparse checkout everywhere * Refactor long path support * Correct path * Parameter names * Remove unnecessary parameter * Close quote * Add pipeline starting * Keep the single quotes * Output link to CI build * Refactor: * Rename docs-metadata-release2.yml" to "update-docsms-metadata.yml" * Documentation for Update-DocsMsMetadata.ps1 * Documentation for Update-DocsMsPackages.ps1 * Add function invocation to Update-DocsMsMetadata.ps1 to override the package metadata from PackageInfo before it's written into the metadata folder * Refactor logic for generating daily branch name * Add ability to override daily docs branch name at queue time * Apply suggestions from code review Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com> * Review feedback: Save-Package-Properties.ps1 * Update comments for path normalization * Change path normalization logic to not switch current location * Update docs * Add ability to specify variable name for default branch * Try convertToJson * Tab scope * Do doc artifact info loop in Update-DocsMsMetadata.ps1 instead of yaml template * Single quotes * Remove template loop * Remove extra template references * Move convertToJson * Try different formatting given convertToJson * Reverse Update-DocsMsMetadata.ps1 to operate on a single object. Looping can happen in yaml * Loop in yaml, Rename Update-DocsMsMetadata parameter * Try re-formatting yaml * Spell parameters properly * Missing backtick * Remove extra logging * First cut at default implementation * Move business logic inside Update-DocsMsMetadata.ps1, PowerShell can handle an array or a single item in the cmdlet parameters * Use proper function name * PackageProps has a DevVersion * No DefaultImplementation in this change. This should be part of a wider effort. * Straggling master -> main rename * Use existing devops starting logic * Remove extra condition * Fix backslashes * Correct number of backslashes * Use variable for branch name * Update eng/scripts/Language-Settings.ps1 Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com> * Review feedback * Do not replace the default branch with the tag in links when doing a Dev build (pointing to the default branch is preferable and the tag does not exist) * Conditions on docs version for template project * Do not run PublishDocsToNightlyBranch if TestPipeline is true * Performance improvements, switch to Linux to take advantage of paths that don't look like escape sequences to ConvertFrom-Json * Add markdown files to sparse checkout * Reproduce failure for template publishing * Remove eng, it's already there by default * Fix instances where metadata might not be available because of filtering Hide == true (e.g. in template packages). The only information consumed from CSV metadata at this time is the service name which gets normalized and placed in the package overview metadata. Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com>
2021-07-01 21:41:25 +03:00
-BuildParametersJson '{"params":"{ \"target_repo\": { \"url\": \"https://github.com/MicrosoftDocs/azure-docs-sdk-node\", \"branch\": \"$(DailyDocsBranchName)\", \"folder\": \"./\" }, \"source_repos\": [] }"}'