mlops-v2/sparse_checkout.sh

69 строки
2.2 KiB
Bash
Исходник Постоянная ссылка Обычный вид История

Dev (#13) * Update QUICKSTART.md added shell script for sparse checkout * Specify data type in inner loop * Initial draft of NLP classification architecture * Update QUICKSTART.md * removed project-specific folders and files * removed .github folder * added sparse checkout bash script * Update QUICKSTART.md added a step to fork the mlops-templates repo * Update QUICKSTART.md added instructions for authenticating via ssh key * Update QUICKSTART.md updated instructions to suggest unique service principal names * removed ado vs github actions for now * Update sparse_checkout.sh * Update sparse_checkout.sh * Add files via upload * Add files via upload * Update QUICKSTART.md updated pre-requisites * Highlight accelerator features still in dev * Add files via upload * Fix highlight text * Update QUICKSTART.md updated service connection details * Add files via upload * Update QUICKSTART.md updated instructions for the outer loop * principle => principal * spelling and format edits * Add files via upload * edits * Create QUICKSTART.md updated training pipeline * Add files via upload * Update QUICKSTART.md updated file paths * Add files via upload * Delete ADO-run6.png * Add files via upload * Add files via upload * Update QUICKSTART.md * Add files via upload * Add files via upload * Add files via upload * Update QUICKSTART.md updates to batch endpoint pipeline * Add files via upload * Add files via upload * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update sparse_checkout.sh * Update QUICKSTART.md * changed steps 2.x * added create empty sparse iimage * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md Co-authored-by: manu-kanwarpal <manu.kanwarpal@gmail.com> Co-authored-by: Scott Donohoo <sdonohoo@microsoft.com> Co-authored-by: Cindy Weng <8880364+cindyweng@users.noreply.github.com> Co-authored-by: cindyweng <weng.cindy@gmail.com> Co-authored-by: lostmygithubaccount <cody.dkdc2@gmail.com> Co-authored-by: Setu Chokshi <setuc@users.noreply.github.com> Co-authored-by: msteller-Ai <56898142+msteller-Ai@users.noreply.github.com>
2022-06-04 02:04:20 +03:00
infrastructure_version=terraform #options: terraform / bicep
project_type=classical #options: classical / cv / nlp
mlops_version=aml-cli-v2 #options: aml-cli-v2 / python-sdk-v1 / python-sdk-v2 / rai-aml-cli-v2
orchestration=azure-devops #options: github-actions / azure-devops
git_folder_location='<local path>' #replace with the local root folder location where you want to create the project folder
project_name=Mlops-Test #replace with your project name
github_org_name=orgname #replace with your github org name
project_template_github_url=https://github.com/azure/mlops-project-template #replace with the url for the project template for your organization created in step 2.2, or leave for demo purposes
Dev (#13) * Update QUICKSTART.md added shell script for sparse checkout * Specify data type in inner loop * Initial draft of NLP classification architecture * Update QUICKSTART.md * removed project-specific folders and files * removed .github folder * added sparse checkout bash script * Update QUICKSTART.md added a step to fork the mlops-templates repo * Update QUICKSTART.md added instructions for authenticating via ssh key * Update QUICKSTART.md updated instructions to suggest unique service principal names * removed ado vs github actions for now * Update sparse_checkout.sh * Update sparse_checkout.sh * Add files via upload * Add files via upload * Update QUICKSTART.md updated pre-requisites * Highlight accelerator features still in dev * Add files via upload * Fix highlight text * Update QUICKSTART.md updated service connection details * Add files via upload * Update QUICKSTART.md updated instructions for the outer loop * principle => principal * spelling and format edits * Add files via upload * edits * Create QUICKSTART.md updated training pipeline * Add files via upload * Update QUICKSTART.md updated file paths * Add files via upload * Delete ADO-run6.png * Add files via upload * Add files via upload * Update QUICKSTART.md * Add files via upload * Add files via upload * Add files via upload * Update QUICKSTART.md updates to batch endpoint pipeline * Add files via upload * Add files via upload * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update sparse_checkout.sh * Update QUICKSTART.md * changed steps 2.x * added create empty sparse iimage * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md Co-authored-by: manu-kanwarpal <manu.kanwarpal@gmail.com> Co-authored-by: Scott Donohoo <sdonohoo@microsoft.com> Co-authored-by: Cindy Weng <8880364+cindyweng@users.noreply.github.com> Co-authored-by: cindyweng <weng.cindy@gmail.com> Co-authored-by: lostmygithubaccount <cody.dkdc2@gmail.com> Co-authored-by: Setu Chokshi <setuc@users.noreply.github.com> Co-authored-by: msteller-Ai <56898142+msteller-Ai@users.noreply.github.com>
2022-06-04 02:04:20 +03:00
cd $git_folder_location
# Clone MLOps Project repo
Dev (#13) * Update QUICKSTART.md added shell script for sparse checkout * Specify data type in inner loop * Initial draft of NLP classification architecture * Update QUICKSTART.md * removed project-specific folders and files * removed .github folder * added sparse checkout bash script * Update QUICKSTART.md added a step to fork the mlops-templates repo * Update QUICKSTART.md added instructions for authenticating via ssh key * Update QUICKSTART.md updated instructions to suggest unique service principal names * removed ado vs github actions for now * Update sparse_checkout.sh * Update sparse_checkout.sh * Add files via upload * Add files via upload * Update QUICKSTART.md updated pre-requisites * Highlight accelerator features still in dev * Add files via upload * Fix highlight text * Update QUICKSTART.md updated service connection details * Add files via upload * Update QUICKSTART.md updated instructions for the outer loop * principle => principal * spelling and format edits * Add files via upload * edits * Create QUICKSTART.md updated training pipeline * Add files via upload * Update QUICKSTART.md updated file paths * Add files via upload * Delete ADO-run6.png * Add files via upload * Add files via upload * Update QUICKSTART.md * Add files via upload * Add files via upload * Add files via upload * Update QUICKSTART.md updates to batch endpoint pipeline * Add files via upload * Add files via upload * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update sparse_checkout.sh * Update QUICKSTART.md * changed steps 2.x * added create empty sparse iimage * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md Co-authored-by: manu-kanwarpal <manu.kanwarpal@gmail.com> Co-authored-by: Scott Donohoo <sdonohoo@microsoft.com> Co-authored-by: Cindy Weng <8880364+cindyweng@users.noreply.github.com> Co-authored-by: cindyweng <weng.cindy@gmail.com> Co-authored-by: lostmygithubaccount <cody.dkdc2@gmail.com> Co-authored-by: Setu Chokshi <setuc@users.noreply.github.com> Co-authored-by: msteller-Ai <56898142+msteller-Ai@users.noreply.github.com>
2022-06-04 02:04:20 +03:00
git clone \
--branch 'main' \
Dev (#13) * Update QUICKSTART.md added shell script for sparse checkout * Specify data type in inner loop * Initial draft of NLP classification architecture * Update QUICKSTART.md * removed project-specific folders and files * removed .github folder * added sparse checkout bash script * Update QUICKSTART.md added a step to fork the mlops-templates repo * Update QUICKSTART.md added instructions for authenticating via ssh key * Update QUICKSTART.md updated instructions to suggest unique service principal names * removed ado vs github actions for now * Update sparse_checkout.sh * Update sparse_checkout.sh * Add files via upload * Add files via upload * Update QUICKSTART.md updated pre-requisites * Highlight accelerator features still in dev * Add files via upload * Fix highlight text * Update QUICKSTART.md updated service connection details * Add files via upload * Update QUICKSTART.md updated instructions for the outer loop * principle => principal * spelling and format edits * Add files via upload * edits * Create QUICKSTART.md updated training pipeline * Add files via upload * Update QUICKSTART.md updated file paths * Add files via upload * Delete ADO-run6.png * Add files via upload * Add files via upload * Update QUICKSTART.md * Add files via upload * Add files via upload * Add files via upload * Update QUICKSTART.md updates to batch endpoint pipeline * Add files via upload * Add files via upload * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update sparse_checkout.sh * Update QUICKSTART.md * changed steps 2.x * added create empty sparse iimage * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md Co-authored-by: manu-kanwarpal <manu.kanwarpal@gmail.com> Co-authored-by: Scott Donohoo <sdonohoo@microsoft.com> Co-authored-by: Cindy Weng <8880364+cindyweng@users.noreply.github.com> Co-authored-by: cindyweng <weng.cindy@gmail.com> Co-authored-by: lostmygithubaccount <cody.dkdc2@gmail.com> Co-authored-by: Setu Chokshi <setuc@users.noreply.github.com> Co-authored-by: msteller-Ai <56898142+msteller-Ai@users.noreply.github.com>
2022-06-04 02:04:20 +03:00
--depth 1 \
--filter=blob:none \
--sparse \
$project_template_github_url \
$project_name
cd $project_name
git sparse-checkout init --cone
git sparse-checkout set infrastructure/$infrastructure_version $project_type/$mlops_version
# Move files to appropiate level
Dev (#13) * Update QUICKSTART.md added shell script for sparse checkout * Specify data type in inner loop * Initial draft of NLP classification architecture * Update QUICKSTART.md * removed project-specific folders and files * removed .github folder * added sparse checkout bash script * Update QUICKSTART.md added a step to fork the mlops-templates repo * Update QUICKSTART.md added instructions for authenticating via ssh key * Update QUICKSTART.md updated instructions to suggest unique service principal names * removed ado vs github actions for now * Update sparse_checkout.sh * Update sparse_checkout.sh * Add files via upload * Add files via upload * Update QUICKSTART.md updated pre-requisites * Highlight accelerator features still in dev * Add files via upload * Fix highlight text * Update QUICKSTART.md updated service connection details * Add files via upload * Update QUICKSTART.md updated instructions for the outer loop * principle => principal * spelling and format edits * Add files via upload * edits * Create QUICKSTART.md updated training pipeline * Add files via upload * Update QUICKSTART.md updated file paths * Add files via upload * Delete ADO-run6.png * Add files via upload * Add files via upload * Update QUICKSTART.md * Add files via upload * Add files via upload * Add files via upload * Update QUICKSTART.md updates to batch endpoint pipeline * Add files via upload * Add files via upload * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update sparse_checkout.sh * Update QUICKSTART.md * changed steps 2.x * added create empty sparse iimage * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md Co-authored-by: manu-kanwarpal <manu.kanwarpal@gmail.com> Co-authored-by: Scott Donohoo <sdonohoo@microsoft.com> Co-authored-by: Cindy Weng <8880364+cindyweng@users.noreply.github.com> Co-authored-by: cindyweng <weng.cindy@gmail.com> Co-authored-by: lostmygithubaccount <cody.dkdc2@gmail.com> Co-authored-by: Setu Chokshi <setuc@users.noreply.github.com> Co-authored-by: msteller-Ai <56898142+msteller-Ai@users.noreply.github.com>
2022-06-04 02:04:20 +03:00
mv $project_type/$mlops_version/data-science data-science
mv $project_type/$mlops_version/mlops mlops
mv $project_type/$mlops_version/data data
if [[ "$mlops_version" == "python-sdk-v1" ]]
Dev (#13) * Update QUICKSTART.md added shell script for sparse checkout * Specify data type in inner loop * Initial draft of NLP classification architecture * Update QUICKSTART.md * removed project-specific folders and files * removed .github folder * added sparse checkout bash script * Update QUICKSTART.md added a step to fork the mlops-templates repo * Update QUICKSTART.md added instructions for authenticating via ssh key * Update QUICKSTART.md updated instructions to suggest unique service principal names * removed ado vs github actions for now * Update sparse_checkout.sh * Update sparse_checkout.sh * Add files via upload * Add files via upload * Update QUICKSTART.md updated pre-requisites * Highlight accelerator features still in dev * Add files via upload * Fix highlight text * Update QUICKSTART.md updated service connection details * Add files via upload * Update QUICKSTART.md updated instructions for the outer loop * principle => principal * spelling and format edits * Add files via upload * edits * Create QUICKSTART.md updated training pipeline * Add files via upload * Update QUICKSTART.md updated file paths * Add files via upload * Delete ADO-run6.png * Add files via upload * Add files via upload * Update QUICKSTART.md * Add files via upload * Add files via upload * Add files via upload * Update QUICKSTART.md updates to batch endpoint pipeline * Add files via upload * Add files via upload * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update sparse_checkout.sh * Update QUICKSTART.md * changed steps 2.x * added create empty sparse iimage * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md Co-authored-by: manu-kanwarpal <manu.kanwarpal@gmail.com> Co-authored-by: Scott Donohoo <sdonohoo@microsoft.com> Co-authored-by: Cindy Weng <8880364+cindyweng@users.noreply.github.com> Co-authored-by: cindyweng <weng.cindy@gmail.com> Co-authored-by: lostmygithubaccount <cody.dkdc2@gmail.com> Co-authored-by: Setu Chokshi <setuc@users.noreply.github.com> Co-authored-by: msteller-Ai <56898142+msteller-Ai@users.noreply.github.com>
2022-06-04 02:04:20 +03:00
then
echo "mlops_version=python-sdk-v1"
Dev (#13) * Update QUICKSTART.md added shell script for sparse checkout * Specify data type in inner loop * Initial draft of NLP classification architecture * Update QUICKSTART.md * removed project-specific folders and files * removed .github folder * added sparse checkout bash script * Update QUICKSTART.md added a step to fork the mlops-templates repo * Update QUICKSTART.md added instructions for authenticating via ssh key * Update QUICKSTART.md updated instructions to suggest unique service principal names * removed ado vs github actions for now * Update sparse_checkout.sh * Update sparse_checkout.sh * Add files via upload * Add files via upload * Update QUICKSTART.md updated pre-requisites * Highlight accelerator features still in dev * Add files via upload * Fix highlight text * Update QUICKSTART.md updated service connection details * Add files via upload * Update QUICKSTART.md updated instructions for the outer loop * principle => principal * spelling and format edits * Add files via upload * edits * Create QUICKSTART.md updated training pipeline * Add files via upload * Update QUICKSTART.md updated file paths * Add files via upload * Delete ADO-run6.png * Add files via upload * Add files via upload * Update QUICKSTART.md * Add files via upload * Add files via upload * Add files via upload * Update QUICKSTART.md updates to batch endpoint pipeline * Add files via upload * Add files via upload * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update sparse_checkout.sh * Update QUICKSTART.md * changed steps 2.x * added create empty sparse iimage * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md Co-authored-by: manu-kanwarpal <manu.kanwarpal@gmail.com> Co-authored-by: Scott Donohoo <sdonohoo@microsoft.com> Co-authored-by: Cindy Weng <8880364+cindyweng@users.noreply.github.com> Co-authored-by: cindyweng <weng.cindy@gmail.com> Co-authored-by: lostmygithubaccount <cody.dkdc2@gmail.com> Co-authored-by: Setu Chokshi <setuc@users.noreply.github.com> Co-authored-by: msteller-Ai <56898142+msteller-Ai@users.noreply.github.com>
2022-06-04 02:04:20 +03:00
mv $project_type/$mlops_version/config-aml.yml config-aml.yml
fi
rm -rf $project_type
mv infrastructure/$infrastructure_version $infrastructure_version
rm -rf infrastructure
mv $infrastructure_version infrastructure
2022-09-05 12:47:03 +03:00
if [[ "$orchestration" == "github-actions" ]]
then
echo "github-actions"
rm -rf mlops/devops-pipelines
mkdir -p .github/workflows/
mv mlops/github-actions/* .github/workflows/
rm -rf mlops/github-actions
mv infrastructure/github-actions/* .github/workflows/
rm -rf infrastructure/devops-pipelines
rm -rf infrastructure/github-actions
2022-09-05 12:47:03 +03:00
fi
if [[ "$orchestration" == "azure-devops" ]]
then
echo "azure-devops"
rm -rf mlops/github-actions
rm -rf infrastructure/github-actions
2022-09-05 12:47:03 +03:00
fi
# Upload to custom repo in Github
Dev (#13) * Update QUICKSTART.md added shell script for sparse checkout * Specify data type in inner loop * Initial draft of NLP classification architecture * Update QUICKSTART.md * removed project-specific folders and files * removed .github folder * added sparse checkout bash script * Update QUICKSTART.md added a step to fork the mlops-templates repo * Update QUICKSTART.md added instructions for authenticating via ssh key * Update QUICKSTART.md updated instructions to suggest unique service principal names * removed ado vs github actions for now * Update sparse_checkout.sh * Update sparse_checkout.sh * Add files via upload * Add files via upload * Update QUICKSTART.md updated pre-requisites * Highlight accelerator features still in dev * Add files via upload * Fix highlight text * Update QUICKSTART.md updated service connection details * Add files via upload * Update QUICKSTART.md updated instructions for the outer loop * principle => principal * spelling and format edits * Add files via upload * edits * Create QUICKSTART.md updated training pipeline * Add files via upload * Update QUICKSTART.md updated file paths * Add files via upload * Delete ADO-run6.png * Add files via upload * Add files via upload * Update QUICKSTART.md * Add files via upload * Add files via upload * Add files via upload * Update QUICKSTART.md updates to batch endpoint pipeline * Add files via upload * Add files via upload * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update sparse_checkout.sh * Update QUICKSTART.md * changed steps 2.x * added create empty sparse iimage * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md Co-authored-by: manu-kanwarpal <manu.kanwarpal@gmail.com> Co-authored-by: Scott Donohoo <sdonohoo@microsoft.com> Co-authored-by: Cindy Weng <8880364+cindyweng@users.noreply.github.com> Co-authored-by: cindyweng <weng.cindy@gmail.com> Co-authored-by: lostmygithubaccount <cody.dkdc2@gmail.com> Co-authored-by: Setu Chokshi <setuc@users.noreply.github.com> Co-authored-by: msteller-Ai <56898142+msteller-Ai@users.noreply.github.com>
2022-06-04 02:04:20 +03:00
rm -rf .git
git init -b main
gh repo create $project_name --private
Dev (#13) * Update QUICKSTART.md added shell script for sparse checkout * Specify data type in inner loop * Initial draft of NLP classification architecture * Update QUICKSTART.md * removed project-specific folders and files * removed .github folder * added sparse checkout bash script * Update QUICKSTART.md added a step to fork the mlops-templates repo * Update QUICKSTART.md added instructions for authenticating via ssh key * Update QUICKSTART.md updated instructions to suggest unique service principal names * removed ado vs github actions for now * Update sparse_checkout.sh * Update sparse_checkout.sh * Add files via upload * Add files via upload * Update QUICKSTART.md updated pre-requisites * Highlight accelerator features still in dev * Add files via upload * Fix highlight text * Update QUICKSTART.md updated service connection details * Add files via upload * Update QUICKSTART.md updated instructions for the outer loop * principle => principal * spelling and format edits * Add files via upload * edits * Create QUICKSTART.md updated training pipeline * Add files via upload * Update QUICKSTART.md updated file paths * Add files via upload * Delete ADO-run6.png * Add files via upload * Add files via upload * Update QUICKSTART.md * Add files via upload * Add files via upload * Add files via upload * Update QUICKSTART.md updates to batch endpoint pipeline * Add files via upload * Add files via upload * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update sparse_checkout.sh * Update QUICKSTART.md * changed steps 2.x * added create empty sparse iimage * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md * Update QUICKSTART.md Co-authored-by: manu-kanwarpal <manu.kanwarpal@gmail.com> Co-authored-by: Scott Donohoo <sdonohoo@microsoft.com> Co-authored-by: Cindy Weng <8880364+cindyweng@users.noreply.github.com> Co-authored-by: cindyweng <weng.cindy@gmail.com> Co-authored-by: lostmygithubaccount <cody.dkdc2@gmail.com> Co-authored-by: Setu Chokshi <setuc@users.noreply.github.com> Co-authored-by: msteller-Ai <56898142+msteller-Ai@users.noreply.github.com>
2022-06-04 02:04:20 +03:00
git remote add origin git@github.com:$github_org_name/$project_name.git
git add . && git commit -m 'initial commit'
git push --set-upstream origin main