diff --git a/.gitignore b/.gitignore index 8c60b1f..a216c99 100644 --- a/.gitignore +++ b/.gitignore @@ -1,3 +1,3 @@ quickstart/configs/cloud-setup/hol.json quickstart/outputs/hol.json -.DS_Store \ No newline at end of file +.DS_Store diff --git a/lab-files/Before the HOL - DataOps.md b/hands-on-lab/Before the HOL - DataOps.md similarity index 100% rename from lab-files/Before the HOL - DataOps.md rename to hands-on-lab/Before the HOL - DataOps.md diff --git a/lab-files/HOL Exercise 2 only.md b/hands-on-lab/HOL Exercise 2 only.md similarity index 100% rename from lab-files/HOL Exercise 2 only.md rename to hands-on-lab/HOL Exercise 2 only.md diff --git a/lab-files/HOL Exercise 3.md b/hands-on-lab/HOL Exercise 3.md similarity index 100% rename from lab-files/HOL Exercise 3.md rename to hands-on-lab/HOL Exercise 3.md diff --git a/lab-files/HOL Guidelines.md b/hands-on-lab/HOL Guidelines.md similarity index 100% rename from lab-files/HOL Guidelines.md rename to hands-on-lab/HOL Guidelines.md diff --git a/lab-files/HOL step-by-step - DataOps.md b/hands-on-lab/HOL step-by-step - DataOps.md similarity index 88% rename from lab-files/HOL step-by-step - DataOps.md rename to hands-on-lab/HOL step-by-step - DataOps.md index 474d6b3..7eb9d44 100644 --- a/lab-files/HOL step-by-step - DataOps.md +++ b/hands-on-lab/HOL step-by-step - DataOps.md @@ -1,28 +1,6 @@ -![Microsoft Cloud Workshops](https://github.com/Microsoft/MCW-Template-Cloud-Workshop/raw/main/Media/ms-cloud-workshop.png "Microsoft Cloud Workshops") +# DevSquad In a Day -
-DataOps -
- -
-Hands-on lab step-by-step -
- -
-May 2021 -
- -Information in this document, including URL and other Internet Web site references, is subject to change without notice. Unless otherwise noted, the example companies, organizations, products, domain names, e-mail addresses, logos, people, places, and events depicted herein are fictitious, and no association with any real company, organization, product, domain name, e-mail address, logo, person, place or event is intended or should be inferred. Complying with all applicable copyright laws is the responsibility of the user. Without limiting the rights under copyright, no part of this document may be reproduced, stored in or introduced into a retrieval system, or transmitted in any form or by any means (electronic, mechanical, photocopying, recording, or otherwise), or for any purpose, without the express written permission of Microsoft Corporation. - -Microsoft may have patents, patent applications, trademarks, copyrights, or other intellectual property rights covering subject matter in this document. Except as expressly provided in any written license agreement from Microsoft, the furnishing of this document does not give you any license to these patents, trademarks, copyrights, or other intellectual property. - -The names of manufacturers, products, or URLs are provided for informational purposes only and Microsoft makes no representations and warranties, either expressed, implied, or statutory, regarding these manufacturers or the use of the products with any Microsoft technologies. The inclusion of a manufacturer or product does not imply endorsement of Microsoft of the manufacturer or product. Links may be provided to third party sites. Such sites are not under the control of Microsoft and Microsoft is not responsible for the contents of any linked site or any link contained in a linked site, or any changes or updates to such sites. Microsoft is not responsible for webcasting or any other form of transmission received from any linked site. Microsoft is providing these links to you only as a convenience, and the inclusion of any link does not imply endorsement of Microsoft of the site or the products contained therein. - -© 2021 Microsoft Corporation. All rights reserved. - -Microsoft and the trademarks listed at are trademarks of the Microsoft group of companies. All other trademarks are property of their respective owners. - -**Contents** +### Index - [DataOps hands-on lab step-by-step](#leveraging-azure-digital-twins-in-a-supply-chain-hands-on-lab-step-by-step) @@ -30,16 +8,16 @@ Microsoft and the trademarks listed at -# Leveraging Azure Digital Twins in a supply chain hands-on lab step-by-step +# Hands-on lab step-by-step ## Abstract and learning objectives diff --git a/lab-files/lab-files/my-lab-file.json b/hands-on-lab/lab-files/my-lab-file.json similarity index 100% rename from lab-files/lab-files/my-lab-file.json rename to hands-on-lab/lab-files/my-lab-file.json diff --git a/lab-files/media/02-One-Notebook-to-Rule-Them-All-1.PNG b/hands-on-lab/media/02-One-Notebook-to-Rule-Them-All-1.PNG similarity index 100% rename from lab-files/media/02-One-Notebook-to-Rule-Them-All-1.PNG rename to hands-on-lab/media/02-One-Notebook-to-Rule-Them-All-1.PNG diff --git a/lab-files/media/02-One-Notebook-to-Rule-Them-All-2.PNG b/hands-on-lab/media/02-One-Notebook-to-Rule-Them-All-2.PNG similarity index 100% rename from lab-files/media/02-One-Notebook-to-Rule-Them-All-2.PNG rename to hands-on-lab/media/02-One-Notebook-to-Rule-Them-All-2.PNG diff --git a/lab-files/media/02-One-Notebook-to-Rule-Them-All-3.PNG b/hands-on-lab/media/02-One-Notebook-to-Rule-Them-All-3.PNG similarity index 100% rename from lab-files/media/02-One-Notebook-to-Rule-Them-All-3.PNG rename to hands-on-lab/media/02-One-Notebook-to-Rule-Them-All-3.PNG diff --git a/lab-files/media/02-One-Notebook-to-Rule-Them-All-4.PNG b/hands-on-lab/media/02-One-Notebook-to-Rule-Them-All-4.PNG similarity index 100% rename from lab-files/media/02-One-Notebook-to-Rule-Them-All-4.PNG rename to hands-on-lab/media/02-One-Notebook-to-Rule-Them-All-4.PNG diff --git a/lab-files/media/02-One-Notebook-to-Rule-Them-All-5.PNG b/hands-on-lab/media/02-One-Notebook-to-Rule-Them-All-5.PNG similarity index 100% rename from lab-files/media/02-One-Notebook-to-Rule-Them-All-5.PNG rename to hands-on-lab/media/02-One-Notebook-to-Rule-Them-All-5.PNG diff --git a/lab-files/media/89-git-repositories.png b/hands-on-lab/media/89-git-repositories.png similarity index 100% rename from lab-files/media/89-git-repositories.png rename to hands-on-lab/media/89-git-repositories.png diff --git a/lab-files/media/91-git-workflow.png b/hands-on-lab/media/91-git-workflow.png similarity index 100% rename from lab-files/media/91-git-workflow.png rename to hands-on-lab/media/91-git-workflow.png diff --git a/lab-files/media/92-git-workflow-databricks-notebooks.png b/hands-on-lab/media/92-git-workflow-databricks-notebooks.png similarity index 100% rename from lab-files/media/92-git-workflow-databricks-notebooks.png rename to hands-on-lab/media/92-git-workflow-databricks-notebooks.png diff --git a/lab-files/media/92-git-workflow-library.png b/hands-on-lab/media/92-git-workflow-library.png similarity index 100% rename from lab-files/media/92-git-workflow-library.png rename to hands-on-lab/media/92-git-workflow-library.png diff --git a/lab-files/media/93-naming-conventions.png b/hands-on-lab/media/93-naming-conventions.png similarity index 100% rename from lab-files/media/93-naming-conventions.png rename to hands-on-lab/media/93-naming-conventions.png diff --git a/lab-files/media/94-release-lifecycle.png b/hands-on-lab/media/94-release-lifecycle.png similarity index 100% rename from lab-files/media/94-release-lifecycle.png rename to hands-on-lab/media/94-release-lifecycle.png diff --git a/lab-files/media/ADFPipelineRunning.PNG b/hands-on-lab/media/ADFPipelineRunning.PNG similarity index 100% rename from lab-files/media/ADFPipelineRunning.PNG rename to hands-on-lab/media/ADFPipelineRunning.PNG diff --git a/lab-files/media/CI-Iac.PNG b/hands-on-lab/media/CI-Iac.PNG similarity index 100% rename from lab-files/media/CI-Iac.PNG rename to hands-on-lab/media/CI-Iac.PNG diff --git a/lab-files/media/PRDEV2QA-1.PNG b/hands-on-lab/media/PRDEV2QA-1.PNG similarity index 100% rename from lab-files/media/PRDEV2QA-1.PNG rename to hands-on-lab/media/PRDEV2QA-1.PNG diff --git a/lab-files/media/PRDEV2QA-2.PNG b/hands-on-lab/media/PRDEV2QA-2.PNG similarity index 100% rename from lab-files/media/PRDEV2QA-2.PNG rename to hands-on-lab/media/PRDEV2QA-2.PNG diff --git a/lab-files/media/PRDEV2QA.PNG b/hands-on-lab/media/PRDEV2QA.PNG similarity index 100% rename from lab-files/media/PRDEV2QA.PNG rename to hands-on-lab/media/PRDEV2QA.PNG diff --git a/lab-files/media/Pipelines-ADF.PNG b/hands-on-lab/media/Pipelines-ADF.PNG similarity index 100% rename from lab-files/media/Pipelines-ADF.PNG rename to hands-on-lab/media/Pipelines-ADF.PNG diff --git a/lab-files/media/Pipelines-Databricks.PNG b/hands-on-lab/media/Pipelines-Databricks.PNG similarity index 100% rename from lab-files/media/Pipelines-Databricks.PNG rename to hands-on-lab/media/Pipelines-Databricks.PNG diff --git a/lab-files/media/Pipelines-IaC.PNG b/hands-on-lab/media/Pipelines-IaC.PNG similarity index 100% rename from lab-files/media/Pipelines-IaC.PNG rename to hands-on-lab/media/Pipelines-IaC.PNG diff --git a/lab-files/media/Pipelines-lib.PNG b/hands-on-lab/media/Pipelines-lib.PNG similarity index 100% rename from lab-files/media/Pipelines-lib.PNG rename to hands-on-lab/media/Pipelines-lib.PNG diff --git a/lab-files/media/Pipelines.PNG b/hands-on-lab/media/Pipelines.PNG similarity index 100% rename from lab-files/media/Pipelines.PNG rename to hands-on-lab/media/Pipelines.PNG diff --git a/lab-files/media/RGComputeDev.PNG b/hands-on-lab/media/RGComputeDev.PNG similarity index 100% rename from lab-files/media/RGComputeDev.PNG rename to hands-on-lab/media/RGComputeDev.PNG diff --git a/lab-files/media/RGDataDev.PNG b/hands-on-lab/media/RGDataDev.PNG similarity index 100% rename from lab-files/media/RGDataDev.PNG rename to hands-on-lab/media/RGDataDev.PNG diff --git a/lab-files/media/Run-CDPipeline-ADF.PNG b/hands-on-lab/media/Run-CDPipeline-ADF.PNG similarity index 100% rename from lab-files/media/Run-CDPipeline-ADF.PNG rename to hands-on-lab/media/Run-CDPipeline-ADF.PNG diff --git a/lab-files/media/Run-CDPipeline-ADFGood.PNG b/hands-on-lab/media/Run-CDPipeline-ADFGood.PNG similarity index 100% rename from lab-files/media/Run-CDPipeline-ADFGood.PNG rename to hands-on-lab/media/Run-CDPipeline-ADFGood.PNG diff --git a/lab-files/media/Run-CDPipeline-Databricks-Lib.PNG b/hands-on-lab/media/Run-CDPipeline-Databricks-Lib.PNG similarity index 100% rename from lab-files/media/Run-CDPipeline-Databricks-Lib.PNG rename to hands-on-lab/media/Run-CDPipeline-Databricks-Lib.PNG diff --git a/lab-files/media/Run-CDPipeline-Databricks-Notebooks.PNG b/hands-on-lab/media/Run-CDPipeline-Databricks-Notebooks.PNG similarity index 100% rename from lab-files/media/Run-CDPipeline-Databricks-Notebooks.PNG rename to hands-on-lab/media/Run-CDPipeline-Databricks-Notebooks.PNG diff --git a/lab-files/media/Run-CDPipeline-Iac.PNG b/hands-on-lab/media/Run-CDPipeline-Iac.PNG similarity index 100% rename from lab-files/media/Run-CDPipeline-Iac.PNG rename to hands-on-lab/media/Run-CDPipeline-Iac.PNG diff --git a/lab-files/media/Run-CDPipeline-lib.PNG b/hands-on-lab/media/Run-CDPipeline-lib.PNG similarity index 100% rename from lab-files/media/Run-CDPipeline-lib.PNG rename to hands-on-lab/media/Run-CDPipeline-lib.PNG diff --git a/lab-files/media/Run-CIPipeline-Databricks.PNG b/hands-on-lab/media/Run-CIPipeline-Databricks.PNG similarity index 100% rename from lab-files/media/Run-CIPipeline-Databricks.PNG rename to hands-on-lab/media/Run-CIPipeline-Databricks.PNG diff --git a/lab-files/media/Run-CIPipeline-Iac.PNG b/hands-on-lab/media/Run-CIPipeline-Iac.PNG similarity index 100% rename from lab-files/media/Run-CIPipeline-Iac.PNG rename to hands-on-lab/media/Run-CIPipeline-Iac.PNG diff --git a/lab-files/media/Run-CIPipeline-lib.PNG b/hands-on-lab/media/Run-CIPipeline-lib.PNG similarity index 100% rename from lab-files/media/Run-CIPipeline-lib.PNG rename to hands-on-lab/media/Run-CIPipeline-lib.PNG diff --git a/lab-files/media/SP-secret.PNG b/hands-on-lab/media/SP-secret.PNG similarity index 100% rename from lab-files/media/SP-secret.PNG rename to hands-on-lab/media/SP-secret.PNG diff --git a/lab-files/media/Versionlib.PNG b/hands-on-lab/media/Versionlib.PNG similarity index 100% rename from lab-files/media/Versionlib.PNG rename to hands-on-lab/media/Versionlib.PNG diff --git a/lab-files/media/adf-copy-data-blob-storage.png b/hands-on-lab/media/adf-copy-data-blob-storage.png similarity index 100% rename from lab-files/media/adf-copy-data-blob-storage.png rename to hands-on-lab/media/adf-copy-data-blob-storage.png diff --git a/lab-files/media/adf-dataops-eastus2-dev-author.PNG b/hands-on-lab/media/adf-dataops-eastus2-dev-author.PNG similarity index 100% rename from lab-files/media/adf-dataops-eastus2-dev-author.PNG rename to hands-on-lab/media/adf-dataops-eastus2-dev-author.PNG diff --git a/lab-files/media/adf-dataops-eastus2-dev-overview.PNG b/hands-on-lab/media/adf-dataops-eastus2-dev-overview.PNG similarity index 100% rename from lab-files/media/adf-dataops-eastus2-dev-overview.PNG rename to hands-on-lab/media/adf-dataops-eastus2-dev-overview.PNG diff --git a/lab-files/media/adf-dataops-eastus2-dev-process-data.PNG b/hands-on-lab/media/adf-dataops-eastus2-dev-process-data.PNG similarity index 100% rename from lab-files/media/adf-dataops-eastus2-dev-process-data.PNG rename to hands-on-lab/media/adf-dataops-eastus2-dev-process-data.PNG diff --git a/lab-files/media/adf-dataops-eastus2-dev-workspace1.PNG b/hands-on-lab/media/adf-dataops-eastus2-dev-workspace1.PNG similarity index 100% rename from lab-files/media/adf-dataops-eastus2-dev-workspace1.PNG rename to hands-on-lab/media/adf-dataops-eastus2-dev-workspace1.PNG diff --git a/lab-files/media/airport-codes-source-csv.PNG b/hands-on-lab/media/airport-codes-source-csv.PNG similarity index 100% rename from lab-files/media/airport-codes-source-csv.PNG rename to hands-on-lab/media/airport-codes-source-csv.PNG diff --git a/lab-files/media/airport-codes-sync.PNG b/hands-on-lab/media/airport-codes-sync.PNG similarity index 100% rename from lab-files/media/airport-codes-sync.PNG rename to hands-on-lab/media/airport-codes-sync.PNG diff --git a/lab-files/media/alpbaVersionlib.PNG b/hands-on-lab/media/alpbaVersionlib.PNG similarity index 100% rename from lab-files/media/alpbaVersionlib.PNG rename to hands-on-lab/media/alpbaVersionlib.PNG diff --git a/lab-files/media/behave-results.png b/hands-on-lab/media/behave-results.png similarity index 100% rename from lab-files/media/behave-results.png rename to hands-on-lab/media/behave-results.png diff --git a/lab-files/media/behave-script.png b/hands-on-lab/media/behave-script.png similarity index 100% rename from lab-files/media/behave-script.png rename to hands-on-lab/media/behave-script.png diff --git a/lab-files/media/betaVersionlib.PNG b/hands-on-lab/media/betaVersionlib.PNG similarity index 100% rename from lab-files/media/betaVersionlib.PNG rename to hands-on-lab/media/betaVersionlib.PNG diff --git a/lab-files/media/branch-policies-builder.PNG b/hands-on-lab/media/branch-policies-builder.PNG similarity index 100% rename from lab-files/media/branch-policies-builder.PNG rename to hands-on-lab/media/branch-policies-builder.PNG diff --git a/lab-files/media/branch-policies-own-owner.PNG b/hands-on-lab/media/branch-policies-own-owner.PNG similarity index 100% rename from lab-files/media/branch-policies-own-owner.PNG rename to hands-on-lab/media/branch-policies-own-owner.PNG diff --git a/lab-files/media/compute-template-json.PNG b/hands-on-lab/media/compute-template-json.PNG similarity index 100% rename from lab-files/media/compute-template-json.PNG rename to hands-on-lab/media/compute-template-json.PNG diff --git a/lab-files/media/copy-airport-codes-sink.PNG b/hands-on-lab/media/copy-airport-codes-sink.PNG similarity index 100% rename from lab-files/media/copy-airport-codes-sink.PNG rename to hands-on-lab/media/copy-airport-codes-sink.PNG diff --git a/lab-files/media/copy-airport-codes.PNG b/hands-on-lab/media/copy-airport-codes.PNG similarity index 100% rename from lab-files/media/copy-airport-codes.PNG rename to hands-on-lab/media/copy-airport-codes.PNG diff --git a/lab-files/media/copy-value-clientsecret.PNG b/hands-on-lab/media/copy-value-clientsecret.PNG similarity index 100% rename from lab-files/media/copy-value-clientsecret.PNG rename to hands-on-lab/media/copy-value-clientsecret.PNG diff --git a/lab-files/media/dbw-dataops-attaching-cluster.PNG b/hands-on-lab/media/dbw-dataops-attaching-cluster.PNG similarity index 100% rename from lab-files/media/dbw-dataops-attaching-cluster.PNG rename to hands-on-lab/media/dbw-dataops-attaching-cluster.PNG diff --git a/lab-files/media/dbw-dataops-eastus2-dev-overview.PNG b/hands-on-lab/media/dbw-dataops-eastus2-dev-overview.PNG similarity index 100% rename from lab-files/media/dbw-dataops-eastus2-dev-overview.PNG rename to hands-on-lab/media/dbw-dataops-eastus2-dev-overview.PNG diff --git a/lab-files/media/dbw-dataops-eastus2-dev-ws.PNG b/hands-on-lab/media/dbw-dataops-eastus2-dev-ws.PNG similarity index 100% rename from lab-files/media/dbw-dataops-eastus2-dev-ws.PNG rename to hands-on-lab/media/dbw-dataops-eastus2-dev-ws.PNG diff --git a/lab-files/media/dbw-dataops-new-cluster.PNG b/hands-on-lab/media/dbw-dataops-new-cluster.PNG similarity index 100% rename from lab-files/media/dbw-dataops-new-cluster.PNG rename to hands-on-lab/media/dbw-dataops-new-cluster.PNG diff --git a/lab-files/media/environments-DEV-Databricks-Notebooks.PNG b/hands-on-lab/media/environments-DEV-Databricks-Notebooks.PNG similarity index 100% rename from lab-files/media/environments-DEV-Databricks-Notebooks.PNG rename to hands-on-lab/media/environments-DEV-Databricks-Notebooks.PNG diff --git a/lab-files/media/environments-DEV-Databricks.PNG b/hands-on-lab/media/environments-DEV-Databricks.PNG similarity index 100% rename from lab-files/media/environments-DEV-Databricks.PNG rename to hands-on-lab/media/environments-DEV-Databricks.PNG diff --git a/lab-files/media/environments-qa-prod.PNG b/hands-on-lab/media/environments-qa-prod.PNG similarity index 100% rename from lab-files/media/environments-qa-prod.PNG rename to hands-on-lab/media/environments-qa-prod.PNG diff --git a/lab-files/media/environments.PNG b/hands-on-lab/media/environments.PNG similarity index 100% rename from lab-files/media/environments.PNG rename to hands-on-lab/media/environments.PNG diff --git a/lab-files/media/globaltable-flight_delays_view.PNG b/hands-on-lab/media/globaltable-flight_delays_view.PNG similarity index 100% rename from lab-files/media/globaltable-flight_delays_view.PNG rename to hands-on-lab/media/globaltable-flight_delays_view.PNG diff --git a/hands-on-lab/media/high-level-overview-dataops.png b/hands-on-lab/media/high-level-overview-dataops.png new file mode 100644 index 0000000..b967343 Binary files /dev/null and b/hands-on-lab/media/high-level-overview-dataops.png differ diff --git a/lab-files/media/iac-ci.PNG b/hands-on-lab/media/iac-ci.PNG similarity index 100% rename from lab-files/media/iac-ci.PNG rename to hands-on-lab/media/iac-ci.PNG diff --git a/lab-files/media/iac-file-corejson-databricks.PNG b/hands-on-lab/media/iac-file-corejson-databricks.PNG similarity index 100% rename from lab-files/media/iac-file-corejson-databricks.PNG rename to hands-on-lab/media/iac-file-corejson-databricks.PNG diff --git a/lab-files/media/iac-folder-databricks.png b/hands-on-lab/media/iac-folder-databricks.png similarity index 100% rename from lab-files/media/iac-folder-databricks.png rename to hands-on-lab/media/iac-folder-databricks.png diff --git a/lab-files/media/iac-folder-infrastructure.PNG b/hands-on-lab/media/iac-folder-infrastructure.PNG similarity index 100% rename from lab-files/media/iac-folder-infrastructure.PNG rename to hands-on-lab/media/iac-folder-infrastructure.PNG diff --git a/lab-files/media/iac-folder-linkedtemplates-subfolders.PNG b/hands-on-lab/media/iac-folder-linkedtemplates-subfolders.PNG similarity index 100% rename from lab-files/media/iac-folder-linkedtemplates-subfolders.PNG rename to hands-on-lab/media/iac-folder-linkedtemplates-subfolders.PNG diff --git a/lab-files/media/iac-folder-linkedtemplates.png b/hands-on-lab/media/iac-folder-linkedtemplates.png similarity index 100% rename from lab-files/media/iac-folder-linkedtemplates.png rename to hands-on-lab/media/iac-folder-linkedtemplates.png diff --git a/lab-files/media/iac-folder-parameters.png b/hands-on-lab/media/iac-folder-parameters.png similarity index 100% rename from lab-files/media/iac-folder-parameters.png rename to hands-on-lab/media/iac-folder-parameters.png diff --git a/lab-files/media/iac-folder-subfolder-tests.png b/hands-on-lab/media/iac-folder-subfolder-tests.png similarity index 100% rename from lab-files/media/iac-folder-subfolder-tests.png rename to hands-on-lab/media/iac-folder-subfolder-tests.png diff --git a/lab-files/media/iac-linkedtemplates-template-compute.PNG b/hands-on-lab/media/iac-linkedtemplates-template-compute.PNG similarity index 100% rename from lab-files/media/iac-linkedtemplates-template-compute.PNG rename to hands-on-lab/media/iac-linkedtemplates-template-compute.PNG diff --git a/lab-files/media/iac-scripts.PNG b/hands-on-lab/media/iac-scripts.PNG similarity index 100% rename from lab-files/media/iac-scripts.PNG rename to hands-on-lab/media/iac-scripts.PNG diff --git a/lab-files/media/iac-service-principal.PNG b/hands-on-lab/media/iac-service-principal.PNG similarity index 100% rename from lab-files/media/iac-service-principal.PNG rename to hands-on-lab/media/iac-service-principal.PNG diff --git a/lab-files/media/infrastructure-as-code-folder.PNG b/hands-on-lab/media/infrastructure-as-code-folder.PNG similarity index 100% rename from lab-files/media/infrastructure-as-code-folder.PNG rename to hands-on-lab/media/infrastructure-as-code-folder.PNG diff --git a/lab-files/media/lakedataopseastus2dev-airport-metadata.PNG b/hands-on-lab/media/lakedataopseastus2dev-airport-metadata.PNG similarity index 100% rename from lab-files/media/lakedataopseastus2dev-airport-metadata.PNG rename to hands-on-lab/media/lakedataopseastus2dev-airport-metadata.PNG diff --git a/lab-files/media/lakedataopseastus2dev-layer-landing.PNG b/hands-on-lab/media/lakedataopseastus2dev-layer-landing.PNG similarity index 100% rename from lab-files/media/lakedataopseastus2dev-layer-landing.PNG rename to hands-on-lab/media/lakedataopseastus2dev-layer-landing.PNG diff --git a/lab-files/media/lakedataopseastus2dev-layers.PNG b/hands-on-lab/media/lakedataopseastus2dev-layers.PNG similarity index 100% rename from lab-files/media/lakedataopseastus2dev-layers.PNG rename to hands-on-lab/media/lakedataopseastus2dev-layers.PNG diff --git a/lab-files/media/lakedataopseastus2dev-overview.PNG b/hands-on-lab/media/lakedataopseastus2dev-overview.PNG similarity index 100% rename from lab-files/media/lakedataopseastus2dev-overview.PNG rename to hands-on-lab/media/lakedataopseastus2dev-overview.PNG diff --git a/lab-files/media/last-pipeline-run.png b/hands-on-lab/media/last-pipeline-run.png similarity index 100% rename from lab-files/media/last-pipeline-run.png rename to hands-on-lab/media/last-pipeline-run.png diff --git a/lab-files/media/mount-adls-1.PNG b/hands-on-lab/media/mount-adls-1.PNG similarity index 100% rename from lab-files/media/mount-adls-1.PNG rename to hands-on-lab/media/mount-adls-1.PNG diff --git a/lab-files/media/mount-adls-2.PNG b/hands-on-lab/media/mount-adls-2.PNG similarity index 100% rename from lab-files/media/mount-adls-2.PNG rename to hands-on-lab/media/mount-adls-2.PNG diff --git a/lab-files/media/notebook-01-adls-mount.PNG b/hands-on-lab/media/notebook-01-adls-mount.PNG similarity index 100% rename from lab-files/media/notebook-01-adls-mount.PNG rename to hands-on-lab/media/notebook-01-adls-mount.PNG diff --git a/lab-files/media/notebook-01-adls-runcell.PNG b/hands-on-lab/media/notebook-01-adls-runcell.PNG similarity index 100% rename from lab-files/media/notebook-01-adls-runcell.PNG rename to hands-on-lab/media/notebook-01-adls-runcell.PNG diff --git a/lab-files/media/parameters-dev-json.PNG b/hands-on-lab/media/parameters-dev-json.PNG similarity index 100% rename from lab-files/media/parameters-dev-json.PNG rename to hands-on-lab/media/parameters-dev-json.PNG diff --git a/lab-files/media/pipeline-run-results.png b/hands-on-lab/media/pipeline-run-results.png similarity index 100% rename from lab-files/media/pipeline-run-results.png rename to hands-on-lab/media/pipeline-run-results.png diff --git a/lab-files/media/pipeline-stages-run.png b/hands-on-lab/media/pipeline-stages-run.png similarity index 100% rename from lab-files/media/pipeline-stages-run.png rename to hands-on-lab/media/pipeline-stages-run.png diff --git a/lab-files/media/pipeline-trigger.PNG b/hands-on-lab/media/pipeline-trigger.PNG similarity index 100% rename from lab-files/media/pipeline-trigger.PNG rename to hands-on-lab/media/pipeline-trigger.PNG diff --git a/lab-files/media/rcVersionlib.PNG b/hands-on-lab/media/rcVersionlib.PNG similarity index 100% rename from lab-files/media/rcVersionlib.PNG rename to hands-on-lab/media/rcVersionlib.PNG diff --git a/lab-files/media/resource-groups.PNG b/hands-on-lab/media/resource-groups.PNG similarity index 100% rename from lab-files/media/resource-groups.PNG rename to hands-on-lab/media/resource-groups.PNG diff --git a/lab-files/media/rg-dataops-compute-dev.PNG b/hands-on-lab/media/rg-dataops-compute-dev.PNG similarity index 100% rename from lab-files/media/rg-dataops-compute-dev.PNG rename to hands-on-lab/media/rg-dataops-compute-dev.PNG diff --git a/lab-files/media/rg-dataops-data-dev.png b/hands-on-lab/media/rg-dataops-data-dev.png similarity index 100% rename from lab-files/media/rg-dataops-data-dev.png rename to hands-on-lab/media/rg-dataops-data-dev.png diff --git a/lab-files/media/scope-dataops.PNG b/hands-on-lab/media/scope-dataops.PNG similarity index 100% rename from lab-files/media/scope-dataops.PNG rename to hands-on-lab/media/scope-dataops.PNG diff --git a/lab-files/media/select-test-yml.png b/hands-on-lab/media/select-test-yml.png similarity index 100% rename from lab-files/media/select-test-yml.png rename to hands-on-lab/media/select-test-yml.png diff --git a/lab-files/media/stgdataopseastus2dev-airport-metadata.PNG b/hands-on-lab/media/stgdataopseastus2dev-airport-metadata.PNG similarity index 100% rename from lab-files/media/stgdataopseastus2dev-airport-metadata.PNG rename to hands-on-lab/media/stgdataopseastus2dev-airport-metadata.PNG diff --git a/lab-files/media/stgdataopseastus2dev-containers.PNG b/hands-on-lab/media/stgdataopseastus2dev-containers.PNG similarity index 100% rename from lab-files/media/stgdataopseastus2dev-containers.PNG rename to hands-on-lab/media/stgdataopseastus2dev-containers.PNG diff --git a/lab-files/media/stgdataopseastus2dev.PNG b/hands-on-lab/media/stgdataopseastus2dev.PNG similarity index 100% rename from lab-files/media/stgdataopseastus2dev.PNG rename to hands-on-lab/media/stgdataopseastus2dev.PNG diff --git a/lab-files/media/templates-folder.png b/hands-on-lab/media/templates-folder.png similarity index 100% rename from lab-files/media/templates-folder.png rename to hands-on-lab/media/templates-folder.png diff --git a/lab-files/media/high-level-overview-dataops.png b/lab-files/media/high-level-overview-dataops.png deleted file mode 100644 index 1b4736e..0000000 Binary files a/lab-files/media/high-level-overview-dataops.png and /dev/null differ diff --git a/quickstart/README.md b/quickstart/README.md index 8a29709..859dda1 100644 --- a/quickstart/README.md +++ b/quickstart/README.md @@ -1,12 +1,157 @@ -# Before Hands-On Lab Quickstart +# Setup the Hands-On Lab -## Prerequisites +Execute all these steps below to setup your evironment for running the Hands-On Lab. -* Follow the instructions on [Prerequisites](./docs/0-prerequisites.md) to set up your Azure DevOps project and the required tools. +## Step 1: Azure DevOps Setup -## Getting Started +1. Create a new [Azure Azure DevOps Project](https://docs.microsoft.com/en-us/azure/devops/organizations/projects/create-project?view=azure-devops&tabs=preview-page). -* [Step 1: Create a Cloud Setup configuration file](./docs/1a-create-hol-setup-file-basic.md) -* [Step 2: Create the pre-required Azure resources](./docs/2-create-prereqs-azure.md) -* [Step 3: Prepare your Azure DevOps project](./docs/3a-azdo-setup-basic.md) -* [Step 4: Create Azure Databricks secrets scope for all environments](./docs/4-create-databricks-secrets-scope.md) +2. [Install](https://docs.microsoft.com/en-us/azure/devops/marketplace/install-extension?view=azure-devops&tabs=browser) the [GitTools extension](https://marketplace.visualstudio.com/items?itemName=gittools.gittools&targetId=0d8e54d4-e229-47bd-9dc5-9be0f116a5c0&utm_source=vstsproduct&utm_medium=ExtHubManageList) in the Organization level of the new Azure DevOps project. + +3. Authorize the project **Build Service** to be an Administrator of Variable Groups: + + - Select **Library** under **Pipelines**: + + ![Azure DevOps Library](./docs/images/quickstart-buildservice-1.png) + + - Click the **Security** button: + + ![Azure DevOps Security](./docs/images/quickstart-buildservice-2.png) + + - Make sure the **Build Service** has the **Administrator** role: + + ![Azure DevOps Build Service](./docs/images/quickstart-buildservice-3.png) + + > In case the Build Service user is not present on this list, click on `+ Add` and search for ` Build Service`, assigning the `Administrator` role. + +4. Make sure the Organization where the project is created in the Azure DevOps is [connected with the Azure Active Directory](https://docs.microsoft.com/en-us/azure/devops/organizations/accounts/connect-organization-to-azure-ad?view=azure-devops) of the Azure Subscription that will be used in the lab. + + +## Step 2: Setup the Hands-On Lab configuration file + +1. Open the [Azure Cloud Shell](https://shell.azure.com) and select the **PowerShell** option. + + - When using the Azure Cloud Shell, be sure you are logged in Azure with the subscription you want to use for the hands-on lab. + + +2. **On the Azure PowerShell terminal:** clone the source code of the lab and go to the `hol` directory. + + ```powershell + git clone https://github.com/microsoft/devsquad-in-a-day.git + + cd devsquad-in-a-day + ``` + +3. **On the Azure PowerShell terminal**: execute the following command to create a new config file based on an existing template. Provide arguments to this setup based on the table below: + + ```powershell + # Be sure to execute this script under the hol directory + ./quickstart/scripts/cloud-setup/Replace-TemplateArgs.ps1 + ``` + + |Argument|Description|Example| + |-----|-----------|-------| + |<_orgName_>|Azure DevOps organization name where you will execute the Hands-On Lab|_org-dataops_| + |<_projectName_>|Name of the existing project inside Azure DevOps where you will execute the Hands-On Lab|_project-dataops_| + |<_projectAlias_>|An unique string with less than 8 characteres that will be used as part of your resource group names|_dataops_| + |<_subscriptionId_>|Azure Subscription ID where the resources will be deployed|_f7e5bb9e-0f98-4c5d-a5c1-a9154bf3cd61_| + +- (_OPTIONAL_) If you are using this project as a Hands-On Lab, feel free to proceed to the next step of the lab setup. If you are using this project as a template for dataops, check [this additional documentation](./1b-create-prereqs-azure-advanced.md) that explains advanced configuration options. + + +## Step 3: Deploy Azure Resources in your subscription + +1. Generate a Personal Access Token (PAT) on Azure DevOps. + + - An environment variable called `AZURE_DEVOPS_EXT_PAT` that stores a [PAT (Personal Access Token)](https://docs.microsoft.com/en-us/azure/devops/organizations/accounts/use-personal-access-tokens-to-authenticate?view=azure-devops&tabs=preview-page) with **Full Access** is required to allow the next script to connect to the new Azure DevOps project and deploy all the resources. + + - To do so, create the PAT on your new Azure DevOps project then run the following command **on the Azure PowerShell terminal**: + + ```powershell + $env:AZURE_DEVOPS_EXT_PAT="" + ``` + +2. Connect and authenticate to your Azure account. + + - **On the Azure PowerShell terminal**: connect to Azure with your authenticated account. + + ```powershell + Connect-AzAccount -UseDeviceAuthentication + ``` + +3. Run the deployment script + + - **On the Azure PowerShell terminal**: run the `Deploy-AzurePreReqs.ps1` script to deploy the pre-required Azure resources: + + ```powershell + ./quickstart/scripts/cloud-setup/Deploy-AzurePreReqs.ps1 -ConfigurationFile "quickstart/configs/cloud-setup/hol.json" + ``` + + - The diagram below shows what Azure resources will be created after the execution of the script. + + ![Azure resources](./docs/images/azure-prereqs-script.png) + +## Step 4: Prepare your Azure DevOps project + +1. Create an Artifact Feed + + - An artifact feed is required on the lab for publishing Python libraries with versioning. + + - On your Azure DevOps project, go to the `Artifacts` section -> `Create Feed`, then set the name as `lib-packages`: + + ![Artifact feed](./docs/images/create-artifact-feed.png) + +2. Azure DevOps project setup + + - **On the Azure PowerShell terminal**: run the command below. + + ```powershell + # You don't need to change any of the following values below + git config --global user.email "hol@microsoft.com" + git config --global user.name "HOL Setup User" + $env:AZURE_DEVOPS_EXT_PAT_TEMPLATE="2je7narfoc2rusvewdjpfnlcn3pyponyrpsko3w5b6z26zj4wpoa" + ``` + + > This command configures git and defines an environment variable that will be used to setup your Azure DevOps project. + + - **On the Azure PowerShell terminal**: run the following script to clone the `hol` repo, create the pipelines and service connections inside your new Azure DevOps. + + ```powershell + ./quickstart/scripts/dataops/Deploy-AzureDevOps.ps1 -ConfigurationFile "./quickstart/outputs/hol.json" -UsePAT $true + ``` + + > Note the file name is the one inside the output directory and the name is the same name of the _projectName_ that was replaced in the first config file. + + +3. (Optional) If you are using this project as a Hands-On Lab, feel free to proceed to the next step of the lab setup. If you are using this project as a template for dataops, check [this documentation](./3b-azdo-setup-advanced.md) for understanding more details about the `AZURE_DEVOPS_EXT_PAT_TEMPLATE` environment variable. + + +## Step 5: Create the Hands-On Lab VM + +1. **On the Azure PowerShell terminal**: run `Deploy-LabVM.ps1` to create a VM inside your Azure subscription that is pre-configured with all the tools that will be needed for executing the Hands-On Lab. + + ```powershell + ./quickstart/scripts/labvm/Deploy-LabVM.ps1 + ``` + + - When prompted for the `sourceSas` parameter, provide the following value: `https://stlabvm.blob.core.windows.net/vhd/labvm-001.vhd?sp=r&st=2021-09-21T19:04:44Z&se=2021-09-25T03:04:44Z&spr=https&sv=2020-08-04&sr=b&sig=qJATNoRXYE34IBybuAMTaMzWyPTr8mZSp3EiYFoRnLk%3D` + + +2. After the VM is created, assign a new password to the user `labuser`. + + ![Reset VM password](./docs/images/vm-lab-reset-password.png) + + - Feel free to create any password of your preference, or to use the suggestion below: + + - **Username:** `labuser` + + - **Password:** `Password.1!!` + +3. Connect to the VM using a RDP client: + + ![RDP Access to the Lab VM](./docs/images/vm-lab-rdp-connection.png) + + > Note that the RDP port of the VM is already configured to allow inbound access. + + +You should follow all these steps provided *before* starting the Hands-on lab. diff --git a/quickstart/docs/Setup-Instructions.md b/quickstart/docs/Setup-Instructions.md deleted file mode 100644 index e086110..0000000 --- a/quickstart/docs/Setup-Instructions.md +++ /dev/null @@ -1,158 +0,0 @@ -# Setup the Hands-On Lab - -Execute all these steps below to setup your evironment for running the Hands-On Lab. - -## Step 1: Azure DevOps Setup - -1. Create a new [Azure Azure DevOps Project](https://docs.microsoft.com/en-us/azure/devops/organizations/projects/create-project?view=azure-devops&tabs=preview-page). - -2. [Install](https://docs.microsoft.com/en-us/azure/devops/marketplace/install-extension?view=azure-devops&tabs=browser) the [GitTools extension](https://marketplace.visualstudio.com/items?itemName=gittools.gittools&targetId=0d8e54d4-e229-47bd-9dc5-9be0f116a5c0&utm_source=vstsproduct&utm_medium=ExtHubManageList) in the Organization level of the new Azure DevOps project. - -3. Authorize the project **Build Service** to be an Administrator of Variable Groups: - - - Select **Library** under **Pipelines**: - - ![Azure DevOps Library](./images/quickstart-buildservice-1.png) - - - Click the **Security** button: - - ![Azure DevOps Security](./images/quickstart-buildservice-2.png) - - - Make sure the **Build Service** has the **Administrator** role: - - ![Azure DevOps Build Service](./images/quickstart-buildservice-3.png) - - > In case the Build Service user is not present on this list, click on `+ Add` and search for ` Build Service`, assigning the `Administrator` role. - -4. Make sure the Organization where the project is created in the Azure DevOps is [connected with the Azure Active Directory](https://docs.microsoft.com/en-us/azure/devops/organizations/accounts/connect-organization-to-azure-ad?view=azure-devops) of the Azure Subscription that will be used in the lab. - - -## Step 2: Setup the Hands-On Lab configuration file - -1. Open the [Azure Cloud Shell](https://shell.azure.com) and select the **PowerShell** option. - - - When using the Azure Cloud Shell, be sure you are logged in Azure with the subscription you want to use for the hands-on lab. - - -2. **On the Azure PowerShell terminal:** clone the source code of the lab and go to the `hol` directory. - - ```powershell - git clone https://github.com/microsoft/devsquad-in-a-day.git - - cd devsquad-in-a-day - ``` - -3. **On the Azure PowerShell terminal**: execute the following command to create a new config file based on an existing template. Provide arguments to this setup based on the table below: - - ```powershell - # Be sure to execute this script under the hol directory - ./quickstart/scripts/cloud-setup/Replace-TemplateArgs.ps1 - ``` - - |Argument|Description|Example| - |-----|-----------|-------| - |<_orgName_>|Azure DevOps organization name where you will execute the Hands-On Lab|_org-dataops_| - |<_projectName_>|Name of the existing project inside Azure DevOps where you will execute the Hands-On Lab|_project-dataops_| - |<_projectAlias_>|An unique string with less than 8 characteres that will be used as part of your resource group names|_dataops_| - |<_subscriptionId_>|Azure Subscription ID where the resources will be deployed|_f7e5bb9e-0f98-4c5d-a5c1-a9154bf3cd61_| - -- (_OPTIONAL_) If you are using this project as a Hands-On Lab, feel free to proceed to the next step of the lab setup. If you are using this project as a template for dataops, check [this additional documentation](./1b-create-prereqs-azure-advanced.md) that explains advanced configuration options. - - -## Step 3: Deploy Azure Resources in your subscription - -1. Generate a Personal Access Token (PAT) on Azure DevOps. - - - An environment variable called `AZURE_DEVOPS_EXT_PAT` that stores a [PAT (Personal Access Token)](https://docs.microsoft.com/en-us/azure/devops/organizations/accounts/use-personal-access-tokens-to-authenticate?view=azure-devops&tabs=preview-page) with **Full Access** is required to allow the next script to connect to the new Azure DevOps project and deploy all the resources. - - - To do so, create the PAT on your new Azure DevOps project then run the following command **on the Azure PowerShell terminal**: - - ```powershell - $env:AZURE_DEVOPS_EXT_PAT="" - ``` - -2. Connect and authenticate to your Azure account. - - - **On the Azure PowerShell terminal**: connect to Azure with your authenticated account. - - ```powershell - Connect-AzAccount -UseDeviceAuthentication - ``` - -3. Run the deployment script - - - **On the Azure PowerShell terminal**: run the `Deploy-AzurePreReqs.ps1` script to deploy the pre-required Azure resources: - - ```powershell - ./quickstart/scripts/cloud-setup/Deploy-AzurePreReqs.ps1 -ConfigurationFile "quickstart/configs/cloud-setup/hol.json" - ``` - - - The diagram below shows what Azure resources will be created after the execution of the script. - - ![Azure resources](./images/azure-prereqs-script.png) - -## Step 4: Prepare your Azure DevOps project - -1. Create an Artifact Feed - - - An artifact feed is required on the lab for publishing Python libraries with versioning. - - - On your Azure DevOps project, go to the `Artifacts` section -> `Create Feed`, then set the name as `lib-packages`: - - ![Artifact feed](./images/create-artifact-feed.png) - -2. Azure DevOps project setup - - - **On the Azure PowerShell terminal**: run the command below. - - ```powershell - # You don't need to change any of the following values below - git config --global user.email "hol@microsoft.com" - git config --global user.name "HOL Setup User" - $env:AZURE_DEVOPS_EXT_PAT_TEMPLATE="2je7narfoc2rusvewdjpfnlcn3pyponyrpsko3w5b6z26zj4wpoa" - ``` - - > This command configures git and defines an environment variable that will be used to setup your Azure DevOps project. - - - **On the Azure PowerShell terminal**: run the following script to clone the `hol` repo, create the pipelines and service connections inside your new Azure DevOps. - - ```powershell - ./quickstart/scripts/dataops/Deploy-AzureDevOps.ps1 -ConfigurationFile "./quickstart/outputs/hol.json" -UsePAT $true - ``` - - > Note the file name is the one inside the output directory and the name is the same name of the _projectName_ that was replaced in the first config file. - - -3. (Optional) If you are using this project as a Hands-On Lab, feel free to proceed to the next step of the lab setup. If you are using this project as a template for dataops, check [this documentation](./3b-azdo-setup-advanced.md) for understanding more details about the `AZURE_DEVOPS_EXT_PAT_TEMPLATE` environment variable. - - -## Step 5: Create the Hands-On Lab VM - -1. **On the Azure PowerShell terminal**: run `Deploy-LabVM.ps1` to create a VM inside your Azure subscription that is pre-configured with all the tools that will be needed for executing the Hands-On Lab. - - ```powershell - ./quickstart/scripts/labvm/Deploy-LabVM.ps1 - ``` - - - When prompted for the `sourceSas` parameter, provide the following value: `https://stlabvm.blob.core.windows.net/vhd/labvm-001.vhd?sp=r&st=2021-09-21T19:04:44Z&se=2021-09-25T03:04:44Z&spr=https&sv=2020-08-04&sr=b&sig=qJATNoRXYE34IBybuAMTaMzWyPTr8mZSp3EiYFoRnLk%3D` - - -2. After the VM is created, assign a new password to the user `labuser`. - - ![Reset VM password](./images/vm-lab-reset-password.png) - - - Feel free to create any password of your preference, or to use the suggestion below: - - - **Username:** `labuser` - - - **Password:** `Password.1!!` - -3. Connect to the VM using a RDP client: - - ![RDP Access to the Lab VM](./images/vm-lab-rdp-connection.png) - - > Note that the RDP port of the VM is already configured to allow inbound access. - - -You should follow all these steps provided *before* starting the Hands-on lab. -