Update GH actions to deploy shared services & tear down (#169)

* 5

* 6

* 7

* aa

* jj

* Update

* ll

* ll

* mm

* vv

* cv

* df

* Added logic for the NSG flow logs com vs gov

* changes to merge conflicts

* fixed conflict merge

* ee

* bnm

* yh

* vv

* sd

* bn

* xx

* vb

* tt

* ss

* zz

* remove sub ids

* aa

* updates

* ff

* updates

* tt

* updates

* mm

* rr

* Added info Azure cli to remove legal hold & other misc updates

* Fix typos

* Moved env variables for toolkit & subscription in the code

* ss

* kk

* Adding Az.Accounts to dockerfile

* cc

* ii

* ll

* yy

* vv

* cc

* ee

* Added all azure regions to AzureBastion module

* nn

* gg

* tt

* dd

* Adding install module in the code itself

* jk

* Added condition to connect to azure & install modules for dev ops

* qaz

* wsx

* bb

* Commented env variables in debug

* ff

* HUB vnet module

* changed MSVDI to connect to shrd svcs hub

* dummy values for config files

* changed para for msvdi with shrd svcs

* do not need to lowercase regions so commented out

* added variables to file so don't need to input

* new prereq script. Not necessary to run

* readme for shared services

* updated readme

* Update

* edc

* Topological path for DevOps pipeline

* test

* Update

* Running individual modules

* Updates

* updated comments

* new modules

* Create dockflow.yml

* Updates to SharedServices & MS-VDI readme

* qq

* Added more info on password restrictions

* Update

* 56

* 985

* 12

* 67

* 45

* 12

* 678

* 12

* 456

* tt

* 12

* 12

* 1q23

* 125

* 343

* 25

* 345

* 2134

* 12

* 2

* 454

* 124

* 312

* 12

* 23

* 34

* mylife

* q3

* 12

* 24

* q1234

* 696

* qw23

* q12e4

* w5

* 213

* 2198

* qw

* 255

* 89876

* 447

* 3242

* 89

* 43234

* 2342342

* q4eq3214

* 87

* 323

* 2345

* 123456

* New version of code for github action

* updates to files

* updated av set infoo

* 789234

* 234143

* 24223412342

* Teardown test

* Copied workflow from Jack's branch

* new changes

* update to readme in shrdsvcs

* new document for github actions

* 234

* adding changes to script for cleanup

* update readme

* update readme

* sdf

* 235

* 123

* 2345

* new changes to readme

* new changes to readme

* readme

* readme

* readmeupdate

* readme

* red

* read

* readme

* 1234

* readme

* 7897894

* update readme shrd svcs

* 345

* new changes to readme

* removed the cleanup and added to different script

* new change to clean up script

* Updates to shared services readme

* update

* 234

* Added passing parameters for subscription & tenant to parameters.json for shared services

* update for networkwatcher

* removed statement in av sets

* Test GH Actions

* Test GH Actions

* Update

* Update

* Cleared values

* Update

* changes to dockerfile version.

* Update

* Update readme

* Update README.md

* Updates to docs - added SPN info

* All documentation updates - removed personal GH repo reference & referencing shared services deployment in quickstart

* Added release notes

* Update

* Merge

* Readding docs updates after merge conflict

* Update GH actions workflow file

* Update

* Removed duplicated folder

* Clean up

* Remove ms-vdi for GH action wf & added teardown

* no change

* added password randomization

* no change

* added sentinel changes

* formatting

* sentinel change and secret changes to kv

* secret changes to kv

* sentinel changes

* dublicate code correction... No code change

* added sentinel env var

* Test Gov Deployment

* updated SS readme

* Merge changes for Azure Sentinel addition & auto-generate password

* naming convention changes

* Test

Co-authored-by: jvalley19 <52843322+jvalley19@users.noreply.github.com>

.github/workflows/dockerimage.yml

@@ -16,15 +16,17 @@ jobs:
         ADMIN_USER_NAME: ${{ secrets.ADMIN_USER_NAME }}
         ADMIN_USER_PWD: ${{ secrets.ADMIN_USER_PWD }}
         DOMAIN_ADMIN_USERNAME: ${{ secrets.DOMAIN_ADMIN_USERNAME }}
-        DOMAIN_ADMIN_USER_PWD: ${{ secrets.DOMAIN_ADMIN_USER_PWD }}
+        DOMAIN_ADMIN_USER_PWD: "Random"
-        ORGANIZATION_NAME : "MSSK"
+        ORGANIZATION_NAME : "jvgovm"
-        AZURE_LOCATION : "USGov Arizona"
+        AZURE_LOCATION : "USGov Virginia"
         AZURE_ENVIRONMENT_NAME : "AzureUSGovernment"
         TENANT_ID : ${{ secrets.TENANT_ID }}
         SUBSCRIPTION_ID : ${{ secrets.SUBSCRIPTION_ID }}
         KEYVAULT_MANAGEMENT_USER_ID  : ${{ secrets.KEYVAULT_MANAGEMENT_USER_ID }}
         AZURE_DISCOVERY_URL : "https://management.azure.com/metadata/endpoints?api-version=2019-05-01"
         ADMIN_USER_SSH : ${{ secrets.ADMIN_USER_SSH }}
+        AZURE_SENTINEL : "true"
+        test:  "true"
     runs-on: ubuntu-latest
     
     steps:

Config/toolkit.subscription.json

@@ -1,6 +1,6 @@
 {
-  "Comments": "ToolKit for creating a new Virtual Data Center",
+  "Comments": "Cleaned up from deployment",
-  "TenantId": "000000-000-0000-0000",
+  "TenantId": "00000-0000000-000000-0000-0",
   "SubscriptionId": "000000-000-0000-0000",
-  "Location": "USGov Arizona"
+  "Location": "DUMMYVALUE"
 }

Environments/MS-VDI/parameters.json

@@ -222,11 +222,11 @@
                 "Comments": "Creating an object so we can use a secretsobject parameter type in our ARM template",
                 "Secrets": [
                     {
-                        "secretName": "admin-user",
+                        "secretName": "vm-admin-user",
                         "secretValue": "env(ADMIN_USER_NAME)"
                     },
                     {
-                        "secretName": "admin-user-pswd",
+                        "secretName": "vm-admin-password",
                         "secretValue": "env(ADMIN_USER_PWD)"
                     }
                 ]

Environments/SharedServices/orchestration.json

@@ -60,6 +60,9 @@
                     },
                     "location": {
                         "value": "${Parameters.ModuleConfigurationParameters.LogAnalytics.Location}"
+                    },
+                    "azureSentinel": {
+                        "value": "${Parameters.AzureSentinel}"
                     }
                 }
             }
@@ -834,7 +837,7 @@
                             "keyVault": {
                                 "id": "reference(KeyVault.keyVaultResourceId)"
                             },
-                            "secretName": "${Parameters.ModuleConfigurationParameters.KeyVault.SecretsObject.Secrets[0].secretName}"
+                            "secretName": "${Parameters.ModuleConfigurationParameters.KeyVault.SecretsObject.Secrets[1].secretName}"
                         }
                     },
                     "storageBlobUrl": {
@@ -947,7 +950,7 @@
                             "keyVault": {
                                 "id": "reference(KeyVault.keyVaultResourceId)"
                             },
-                            "secretName": "${Parameters.ModuleConfigurationParameters.KeyVault.SecretsObject.Secrets[2].secretName}"
+                            "secretName": "${Parameters.ModuleConfigurationParameters.KeyVault.SecretsObject.Secrets[4].secretName}"
                         }
                     },
                     "storageBlobUrl": {

Environments/SharedServices/parameters.json

@@ -8,6 +8,7 @@
     "Location": "env(AZURE_LOCATION)",
     "EnvironmentName": "env(AZURE_ENVIRONMENT_NAME)",
     "StorageBlobUrl": "env(AZURE_STORAGE_BLOB_URL)",
+    "AzureSentinel": "env(AZURE_SENTINEL)",
     "ModuleConfigurationParameters": {
         "OnPremisesInformation": {
             "InstanceName": "${Parameters.InstanceName}",
@@ -741,11 +742,19 @@
                 "Comments": "Creating an object so we can use a secretsobject parameter type in our ARM template",
                 "Secrets": [
                     {
-                        "secretName": "admin-user",
+                        "secretName": "vm-admin-user",
+                        "secretValue": "env(ADMIN_USER_NAME)"
+                    },
+                    {
+                        "secretName": "vm-admin-password",
                         "secretValue": "env(ADMIN_USER_PWD)"
                     },
                     {
-                        "secretName": "env(DOMAIN_ADMIN_USERNAME)",
+                        "secretName": "domain-admin-user",
+                        "secretValue": "env(DOMAIN_ADMIN_USERNAME)"
+                    },
+                    {
+                        "secretName": "domain-admin-password",
                         "secretValue": "env(DOMAIN_ADMIN_USER_PWD)"
                     },
                     {
@@ -768,7 +777,7 @@
         "ArtifactsStorageAccount": "file(../_Common/artifactsStorageAccount.json)",
         "Jumpbox": {
             "ResourceGroup": "${Parameters.InstanceName}-jumpbox-rg",
-            "AdminUsername": "${Parameters.ModuleConfigurationParameters.KeyVault.SecretsObject.Secrets[0].secretName}",
+            "AdminUsername": "env(ADMIN_USER_NAME)",
             "SubnetName": "${Parameters.ModuleConfigurationParameters.VirtualNetwork.Subnets[0].name}",
             "StorageBlobUrl": "${Parameters.StorageBlobUrl}",
             "Windows": {
@@ -821,7 +830,7 @@
                 "keyVault": {
                     "id": "reference(KeyVault.keyVaultResourceId)"
                 },
-                "secretName": "${Parameters.ModuleConfigurationParameters.KeyVault.SecretsObject.Secrets[1].secretName}"
+                "secretName": "${Parameters.ModuleConfigurationParameters.KeyVault.SecretsObject.Secrets[3].secretName}"
             },
             "VMSize": "Standard_DS3_v2",
             "OSImage": {
@@ -842,12 +851,12 @@
             "ResourceGroup": "${Parameters.ModuleConfigurationParameters.ActiveDirectory.ResourceGroup}",
             "Comments": "Windows VM name cannot exceed 13 characters. Additionally, Make sure that AddsIPAddressStart and ActiveDirectory.PrimaryDomainControllerIP are in the same subnet address prefix and they don't overlap",
             "StorageBlobUrl": "${Parameters.StorageBlobUrl}",
-            "AdminUsername": "${Parameters.ModuleConfigurationParameters.KeyVault.SecretsObject.Secrets[0].secretName}",
+            "AdminUsername": "env(ADMIN_USER_NAME)",
             "AdminPassword": {
                 "keyVault": {
                     "id": "reference(KeyVault.keyVaultResourceId)"
                 },
-                "secretName": "${Parameters.ModuleConfigurationParameters.KeyVault.SecretsObject.Secrets[0].secretName}"
+                "secretName": "${Parameters.ModuleConfigurationParameters.KeyVault.SecretsObject.Secrets[1].secretName}"
             },
             "Kek": {
                 "Name": "AddsKey",

Environments/SharedServices/readme.md

@@ -85,6 +85,7 @@ $ENV:ADMIN_USER_NAME = "[VM_ADMIN_USER_NAME]"
 $ENV:ADMIN_USER_PWD = "[VM_ADMIN_USER_PASSWORD]"
 $ENV:AZURE_DISCOVERY_URL = "https://management.azure.com/metadata/endpoints?api-version=2019-05-01"
 $ENV:ADMIN_USER_SSH = "[SSH_KEY]"
+$ENV:AZURE_SENTINEL = "[BOOLEAN]"
 ```
 
 **NOTE:** Examples to setting the env variables
@@ -114,12 +115,20 @@ $ENV:ADMIN_USER_SSH = "[SSH_KEY]"
   - Domain user name - will be used for AD deployment and not yet included in current deployment
 - "[DOMAIN_ADMIN_USER_PASSWORD]"
   - Domain user password - will be used for AD deployment and not yet included in current deployment. Follow the [guidelines](https://docs.microsoft.com/en-us/azure/virtual-machines/windows/faq#what-are-the-password-requirements-when-creating-a-vm) for setting the password.
+  - UPDATE: If the deployment admin wants a random password for the Domain Admin account please set the value to "" 
+    - Ex. $ENV:DOMAIN_ADMIN_USER_PWD=""
 - "[VM_ADMIN_USER_NAME]"
   - VM log in username
 - "[VM_ADMIN_USER_PASSWORD]"
   - VM user password. Follow the [guidelines](https://docs.microsoft.com/en-us/azure/virtual-machines/windows/faq#what-are-the-password-requirements-when-creating-a-vm) for setting the password.
+  - UPDATE: If the deployment admin wants a random password for the VM Admin account please set the value to "" 
+    - Ex. $ENV:ADMIN_USER_PWD=""
 - "[SSH_KEY]"
   - Needs to be a valid public ssh rsa key for SSH to linux box
+- "[BOOLEAN]
+  - This value needs to be "True" or "False"
+    - "True" will deploy Azure Sentinel to the Shared Services Environment
+    - "False" will NOT deploy Azure Sentinel 
   
 To use the above script:
 
@@ -131,6 +140,7 @@ To use the above script:
 
 #### Pre-req script
 ##### This script will ensure that the configuration files are updated with your environment variables.
+##### This script has the functionality for creating random passwords for the VM's
 
   ``` PowerShell
   ./Orchestration/OrchestrationService/Pre_req_script.ps1

Environments/_Common/subscriptions.json

@@ -1,38 +1,38 @@
 {
-  "Comments": "ToolKit for Jack",
+    "Comments":  "ToolKit for VDC Deployment",
-  "VDCVDI": {
+    "VDCVDI":  {
-    "Comments": "Microsoft VDC with VDI environment subscription and tenant information",
+        "Comments":  "Microsoft VDC with VDI environment subscription and tenant information",
-    "TenantId": "000000-000-0000-0000",
+        "TenantId":  "000000-000-0000-0000",
-    "SubscriptionId": "000000-000-0000-0000"
+        "SubscriptionId":  "000000-000-0000-0000"
-  },
+    },
-  "OnPremises": {
+    "OnPremises":  {
-    "Comments": "Simulated On-Premises subscription and tenant information",
+        "Comments":  "Simulated On-Premises subscription and tenant information",
-    "TenantId": "000000-000-0000-0000",
+        "TenantId":  "000000-000-0000-0000",
-    "SubscriptionId": "000000-000-0000-0000"
+        "SubscriptionId":  "000000-000-0000-0000"
-  },
+    },
-  "SharedServices": {
+    "SharedServices":  {
-    "Comments": "Shared services subscription and tenant information",
+        "Comments":  "Shared services subscription and tenant information",
-    "TenantId": "000000-000-0000-0000",
+        "TenantId":  "000000-000-0000-0000",
-    "SubscriptionId": "000000-000-0000-0000"
+        "SubscriptionId":  "000000-000-0000-0000"
-  },
+    },
-  "AKS": {
+    "AKS":  {
-    "Comments": "Shared services subscription and tenant information",
+        "Comments":  "Shared services subscription and tenant information",
-    "TenantId": "000000-000-0000-0000",
+        "TenantId":  "000000-000-0000-0000",
-    "SubscriptionId": "000000-000-0000-0000"
+        "SubscriptionId":  "000000-000-0000-0000"
-  },
+    },
-  "ASE_SQLDB": {
+    "ASE_SQLDB":  {
-    "Comments": "Workload subscription and tenant information",
+        "Comments":  "Workload subscription and tenant information",
-    "TenantId": "000000-000-0000-0000",
+        "TenantId":  "000000-000-0000-0000",
-    "SubscriptionId": "00000000-0000-0000-0000-000000000000"
+        "SubscriptionId":  "00000000-0000-0000-0000-000000000000"
-  },
+    },
-  "NTier_IaaS": {
+    "NTier_IaaS":  {
-    "Comments": "Workload subscription and tenant information",
+        "Comments":  "Workload subscription and tenant information",
-    "TenantId": "000000-000-0000-0000",
+        "TenantId":  "000000-000-0000-0000",
-    "SubscriptionId": "000000-000-0000-0000"
+        "SubscriptionId":  "000000-000-0000-0000"
-  },
+    },
-  "Artifacts": {
+    "Artifacts":  {
-    "Comments": "Subscription and tenant information where the Artifacts Storage Account will reside",
+        "Comments":  "Subscription and tenant information where the Artifacts Storage Account will reside",
-    "TenantId": "000000-000-0000-0000",
+        "TenantId":  "000000-000-0000-0000",
-    "SubscriptionId": "000000-000-0000-0000"
+        "SubscriptionId":  "000000-000-0000-0000"
-  }
+    }
 }

Modules/LogAnalytics/deploy.json

@@ -87,10 +87,21 @@
             "metadata": {
                 "description": "Optional. Automation Account resource identifier, value used to create a LinkedService between Log Analytics and an Automation Account."
             }
+        },
+        "azureSentinel": {
+            "type": "string",
+            "defaultValue": "false",
+            "metadata": {
+                "description": "Install Azure Sentinel as part of the Log Analytics Workspace."
+            }
+
         }
     },
     "variables": {
         "logAnalyticsSearchVersion": 1,
+        "azureSentinelSolutionName": "[concat('SecurityInsights', '(', parameters('logAnalyticsWorkspaceName'), ')')]",
+        "product": "OMSGallery/SecurityInsights",
+        "publisher": "Microsoft",
         "solutions": [
             {
                 "name": "[concat('Updates', '(', parameters('logAnalyticsWorkspaceName'), ')')]",
@@ -1095,6 +1106,25 @@
             "properties": {
                 "level": "CannotDelete"
             }
+        },
+        {
+            "type": "Microsoft.OperationsManagement/solutions",
+            "apiVersion": "2015-11-01-preview",
+            "name": "[variables('azureSentinelSolutionName')]",
+            "condition": "[bool(parameters('azureSentinel'))]",
+            "location": "[parameters('location')]",
+            "plan": {
+                "name": "[variables('azureSentinelSolutionName')]",
+                "promotionCode": "",
+                "product": "[variables('product')]",
+                "publisher": "[variables('publisher')]"
+            },
+            "dependsOn": [
+                "[resourceId('Microsoft.OperationalInsights/workspaces/', parameters('logAnalyticsWorkspaceName'))]"
+            ],
+            "properties": {
+                "workspaceResourceId": "[resourceId('Microsoft.OperationalInsights/workspaces/', parameters('logAnalyticsWorkspaceName'))]"
+            }
         }
     ],
     "outputs": {

Modules/NetworkSecurityGroups/Scripts/enable.flow.logs.ps1

@@ -39,7 +39,7 @@ try {
     else {
         Write-Host "No subscription switching is required."
     }
-
+    
     $NetworkWatcherRegion = $NetworkWatcherRegion.Replace(' ', '').ToLower()
 
     $registered = Get-AzResourceProvider -ProviderNamespace Microsoft.Insights

Orchestration/OrchestrationService/ModuleConfigurationDeployment.ps1

@@ -66,13 +66,6 @@ $ENV:VDC_TOOLKIT_SUBSCRIPTION = (Get-Content .\Config\toolkit.subscription.json
 Write-Debug "AZURE_STORAGE_BLOB_URL: $ENV:AZURE_STORAGE_BLOB_URL"
 Write-Debug "AzureManagementUrl: $AzureManagementUrl"
 
-
-# Get the config files
-$ENV:VDC_SUBSCRIPTIONS = (Get-Content ./Environments/_Common/subscriptions.json -Raw)
-$ENV:VDC_TOOLKIT_SUBSCRIPTION = (Get-Content ./Config/toolkit.subscription.json -Raw)
-#Write-Debug "ToolkitJSON: $ENV:VDC_SUBSCRIPTIONS"
-#Write-Debug "SubscriptionJson: $ENV:VDC_TOOLKIT_SUBSCRIPTION"
-
 Function Start-Deployment {
     [CmdletBinding()]
     param (

Orchestration/OrchestrationService/Pre_req_script.ps1

@@ -26,3 +26,34 @@ $onprem = (Get-Content -Path .\Environments\_Common\subscriptions.json) | Conver
 $onprem.OnPremises.SubscriptionId = $ENV:SUBSCRIPTION_ID
 $onprem.OnPremises.TenantId = $ENV:TENANT_ID
 $onprem | ConvertTo-Json | Set-Content -Path .\Environments\_Common\subscriptions.json
+
+
+#### Check if random passwords are needed or if passwords are provided for the VM admin accounts and the Active Directory Account
+
+# Random Password Function
+function Get-RandomPassword {
+    $Alphabets = 'a,b,c,d,e,f,g,h,i,j,k,l,m,n,o,p,q,r,s,t,u,v,w,x,y,z'
+    $numbers = 0..9
+    $specialCharacters = '~,!,@,#,$,%,^,&,*,(,),?,\,/,_,-,=,+'
+    $array = @()
+    $counter= Get-Random -Minimum 5 -Maximum 7
+    $array += $Alphabets.Split(',') | Get-Random -Count $counter
+    $array[0] = $array[0].ToUpper()
+    $array[-1] = $array[-1].ToUpper()
+    $array += $numbers | Get-Random -Count $counter
+    $array += $specialCharacters.Split(',') | Get-Random -Count $counter
+    $password = ($array | Get-Random -Count $array.Count) -join "" 
+    
+    return $password #| ConvertTo-SecureString -AsPlainText -Force
+}
+    
+### Check the VM password 
+if (($null -eq $ENV:ADMIN_USER_PWD) -or ("" -eq $ENV:ADMIN_USER_PWD) -or ("Random" -eq $ENV:ADMIN_USER_PWD) ) {
+    $ENV:ADMIN_USER_PWD = Get-RandomPassword
+}
+
+### Check the Active Directory (Domain Password)
+if (($null -eq $ENV:DOMAIN_ADMIN_USER_PWD) -or ("" -eq $ENV:DOMAIN_ADMIN_USER_PWD) -or ("Random" -eq $ENV:DOMAIN_ADMIN_USER_PWD) ) {
+    $ENV:DOMAIN_ADMIN_USER_PWD = Get-RandomPassword
+}
+

entrypoint1.ps1

@@ -24,12 +24,13 @@ Write-Host "Starting the script for deploying your Shared Services"
 
 Write-Host "The deployment was succesfull if: Exit code $LASTEXITCODE == 0" -Verbose
 
-Write-Host "Starting the script for deploying MS-VDI"
+## Enter the main script for teardown shared services
-./Orchestration/OrchestrationService/ModuleConfigurationDeployment.ps1 -DefinitionPath ./Environments/MS-VDI/definition.json
+Write-Host "Starting the script for tearing down Shared Services"
-
+./Orchestration/OrchestrationService/ModuleConfigurationDeployment.ps1 -TearDownEnvironment -DefinitionPath ./Environments/SharedServices/definition.json
-Write-Host "The deployment was succesfull if: Exit code $LASTEXITCODE == 0" -Verbose
 
 ## Run the cleanup script so that no values are retained in code for the config files
 Write-Host "Executing the cleanup script"
 
-./Orchestration/OrchestrationService/Cleanup_Script.ps1
+./Orchestration/OrchestrationService/Cleanup_Script.ps1
+
+Write-Host "The deployment was succesfull if: Exit code $LASTEXITCODE == 0" -Verbose

vdc

@@ -1 +0,0 @@
-Subproject commit 8b8ecd33efc8364fd8c4d0629b28cb867e985ae7