Changes innereye template to use identity (#1022)

Changes innereye template to use identity

docs/tre-admins/setup-instructions/configuring-shared-services.md

@@ -8,16 +8,18 @@ Complete the configuration of the shared services (Nexus and Gitea) from inside
 2. Download Git for Windows from [https://git-scm.com/download/win](https://git-scm.com/download/win) and install
 3. Download Azure CLI from [https://aka.ms/installazurecliwindows](https://aka.ms/installazurecliwindows) and install
 4. Open Git Bash
-5. Login to Azure ```az login```
+5. Login to Azure ```az login``` and set the default subscription if needed: ```az account set --subscription <subscription_id>```
 6. Git clone the TRE repository: ```git clone https://github.com/microsoft/AzureTRE.git```
 7. Download jq ```curl -L -o /usr/bin/jq.exe https://github.com/stedolan/jq/releases/latest/download/jq-win64.exe```
 
 ## Configure Nexus repository
 
 1. Run the Nexus configuration script to reset the password and setup a PyPI proxy on Nexus:
-```./scripts/config_nexus.sh -t <tre_id>```
+```./scripts/configure_nexus.sh -t <tre_id>```
 
 ## Configure Gitea repository
 
 1. Migrate the required repositories to Gitea by running:
 ```./scripts/gitea_migrate_repo.sh -t <tre_id> -g <URL_of_github_repo_to_migrate>```
+1. If you have issues with token or token doesn't work, you can reset the token by setting it's value to null in Key Vault:
+```az keyvault secret set --name gitea-<tre-id>-admin-token --vault-name kv-<tre-id> --value null```

docs/tre-templates/workspace-services/inner-eye.md

@@ -46,36 +46,44 @@ URLs:
 ### Preparation steps performed by the TRE Admin
 
 1. Ensure that you have completed ["Configuring Shared Services"](../tre-admins/setup-instructions/configuring-shared-services.md)
-2. Log onto a TREAdmin Jumpbox and mirror Github repos needed by InnerEye Helloworld:
+1. Log onto a TREAdmin Jumpbox and mirror Github repos needed by InnerEye Helloworld:
 
-  ```cmd
-  ./scripts/gitea_migrate_repo.sh -t <tre_id> -g https://github.com/microsoft/InnerEye-DeepLearning
-  ./scripts/gitea_migrate_repo.sh -t <tre_id> -g https://github.com/analysiscenter/radio
-  ```
+    ```cmd
+    ./scripts/gitea_migrate_repo.sh -t <tre_id> -g https://github.com/microsoft/InnerEye-DeepLearning
+    ./scripts/gitea_migrate_repo.sh -t <tre_id> -g https://github.com/analysiscenter/radio
+    ```
 
 ### Setup the InnerEye run from AML Compute Instance
 
 1. Log onto a VM in the workspace, open Edge and navigate to [ml.azure.com](https://ml.azure.com)
-2. Select the Notebooks tab and then click Terminal. This should open a terminal on a running compute instance
-3. Pull the InnerEye-DeepLearning git repo from Gitea mirror and configure:
+1. Select the Notebooks tab and then click Terminal. This should open a terminal on a running compute instance
+1. Pull the InnerEye-DeepLearning git repo from Gitea mirror and configure:
 
-  ```cmd
-  git clone https://gitea-<TRE_ID>.azurewebsites.net/giteaadmin/InnerEye-DeepLearning
-  cd InnerEye-DeepLearning
-  curl -s https://packagecloud.io/install/repositories/github/git-lfs/script.deb.sh | sudo bash
-  sudo apt-get install git-lfs
-  git lfs install
-  git lfs pull
-  export PIP_INDEX_URL=https://nexus-<TRE_ID>.azurewebsites.net/repository/pypi-proxy-repo/simple
-  conda init
-  conda env create --file environment.yml
-  conda activate InnerEye
-  ```
+    ```cmd
+    git clone https://gitea-<TRE_ID>.azurewebsites.net/giteaadmin/InnerEye-DeepLearning
+    cd InnerEye-DeepLearning
+    curl -s https://packagecloud.io/install/repositories/github/git-lfs/script.deb.sh | sudo bash
+    sudo apt-get install git-lfs
+    git lfs install
+    git lfs pull
+    export PIP_INDEX_URL=https://nexus-<TRE_ID>.azurewebsites.net/repository/pypi-proxy-repo/simple
+    conda init
+    conda env create --file environment.yml
+    conda activate InnerEye
+    ```
+1. Get storage keys for your storage:
 
-1. Open Azure Storage Explorer and connect to your Storage Account using name and access key
-1. On the storage account create a container with name ```datasets``` and a folder named ```hello_world```
-1. Copy `dataset.csv` file from `Tests/ML/test_data/dataset.csv` to the `hello_world` folder
-1. Copy the whole `train_and_test_data` folder from `Test/ML/test_data/train_and_test_data` to the `hello_world` folder
+    ```az storage account keys list --account-name stgws<workspace_id>```
+1. Create a "datasets" container
+
+    ```az storage container create --name datasets --account-name stgws<workspace_id>```
+1. Copy `dataset.csv` file from `Tests/ML/test_data/dataset.csv` to the `hello_world` folder:
+
+    ```az storage blob upload --account-name stgws<workspace_id> --container-name datasets --file ./Tests/ML/test_data/dataset.csv --name /hello_world/dataset.csv```
+1. Copy the whole `train_and_test_data` folder from `Test/ML/test_data/train_and_test_data` to the `hello_world` folder:
+
+    ```az storage blob directory upload -c datasets --account-name stgws<workspace_id> -s "./Test/ML/test_data/train_and_test_data" -d hello_world --recursive```
+  
 1. Update the following variables in `InnerEye/settings.yml`: subscription_id, resource_group, workspace_name, cluster (see [AML setup](https://github.com/microsoft/InnerEye-DeepLearning/blob/main/docs/setting_up_aml.md) for more details).
 1. Open your browser to ml.azure.com, login, select the right Subscription and AML workspace and then navigate to `Data stores`. Create a New datastore named `innereyedatasets` and link it to your storage account and datasets container.
 1. Back from PowerShell run

scripts/configure_nexus.sh

@@ -32,8 +32,6 @@ export NEXUS_ADMIN_PASSWORD_NAME="nexus-${tre_id,,}-admin-password"
 export KEYVAULT_NAME="kv-${tre_id}"
 export STORAGE_ACCOUNT_NAME="stg${tre_id//-/}"
 
-echo ${KEYVAULT_NAME}
-
 export NEXUS_PASS=$(az keyvault secret show --name ${NEXUS_ADMIN_PASSWORD_NAME} --vault-name ${KEYVAULT_NAME} -o json | jq -r '.value')
 
 if [ -z "$NEXUS_PASS" ]; then
@@ -70,5 +68,5 @@ if [[ ${STATUS_CODE} == 404 ]]
     $NEXUS_URL/service/rest/v1/repositories/apt/proxy \
     -H 'accept: application/json' \
     -H 'Content-Type: application/json' \
-    -d '@pypi_proxy_conf.json'
+    -d '@./scripts/pypi_proxy_conf.json'
 fi

templates/core/terraform/firewall/firewall.tf

@@ -112,7 +112,7 @@ resource "azurerm_firewall_application_rule_collection" "shared_subnet" {
       "*github.com",
       "*powershellgallery.com",
       "git-scm.com",
-      "github-cloud.githubusercontent.com",
+      "*githubusercontent.com",
       "*core.windows.net",
       "aka.ms",
       "management.azure.com",

templates/core/terraform/firewall/variables.tf

@@ -12,5 +12,5 @@ variable "nexus_allowed_fqdns" {
 variable "gitea_allowed_fqdns" {
   type        = string
   description = "comma seperated string of allowed FQDNs for Gitea"
-  default     = "github.com, www.github.com, api.github.com, git-lfs.github.com, github-cloud.githubusercontent.com"
+  default     = "github.com, www.github.com, api.github.com, git-lfs.github.com, *githubusercontent.com"
 }

templates/workspace_services/innereye/porter.yaml

@@ -1,5 +1,5 @@
 name: tre-service-innereye
-version: 0.1.4
+version: 0.1.5
 description: "An Azure TRE service for InnerEye Deep Learning"
 registry: azuretre
 dockerfile: Dockerfile.tmpl
@@ -58,17 +58,8 @@ install:
       arguments:
         - login
       flags:
-        service-principal:
+        identity:
         username: "{{ bundle.credentials.azure_client_id}}"
-        password: "{{ bundle.credentials.azure_client_secret}}"
-        tenant: "{{ bundle.credentials.azure_tenant_id}}"
-  # - az:
-  #     description: "az login"
-  #     arguments:
-  #       - login
-  #     flags:
-  #       identity:
-  #       username: "{{ bundle.credentials.azure_client_id}}"
   - az:
       description: "acr login"
       arguments:

templates/workspaces/innereye/install_service_innereye.sh

@@ -1,7 +1,7 @@
 #!/bin/bash
 set -e
 
-porter install tre-service-innereye --reference "${MGMT_ACR_NAME}.azurecr.io/tre-service-innereye:v0.1.4" \
+porter install tre-service-innereye --reference "${MGMT_ACR_NAME}.azurecr.io/tre-service-innereye:v0.1.5" \
     --cred ./azure.json \
     --parameter-set ./parameters_service_innereye.json \
     --allow-docker-host-access \

templates/workspaces/innereye/porter.yaml

@@ -1,6 +1,6 @@
 name: tre-workspace-innereye
 
-version: 0.1.9
+version: 0.1.10
 description: "An Azure TRE workspace with Azure Machine Learning, Dev Test Labs and InnerEye deep learning"
 registry: azuretre
 dockerfile: Dockerfile.tmpl