Merge pull request #64 from nightism/enhancement/azure-deploy-tutorial

[Enhancement] Update deployment tutorial
2022-06-27 14:09:13 +01:00 · 2022-06-27 14:09:13 +01:00 · 264b31ee34
--- a/.github/workflows/test.yaml
+++ b/.github/workflows/test.yaml
@ -37,6 +37,7 @@ jobs:
      - name: Install dependencies
        run: |
          python -m pip install --upgrade pip
+          python -m pip install 'protobuf>=3.10,<3.20'
          python -m pip install db-dtypes 
          python -m pip install -e feast/sdk/python[ci]
      - name: Run Feast unit tests
--- a/README.md
+++ b/README.md
@ -30,6 +30,8 @@ We recommend the [getting started tutorial](provider/tutorial/) that walks you t

 You can deploy the infrastructure for feast using:

+> You may want to first make sure your subscription has registered `Microsoft.Synapse`, `Microsoft.SQL` and `Microsoft.Network` providers before running the template below, as some of them may require explicit registration.
+
 [![Deploy to Azure](https://aka.ms/deploytoazurebutton)](https://portal.azure.com/#create/Microsoft.Template/uri/https%3A%2F%2Fraw.githubusercontent.com%2FAzure%2Ffeast-azure%2Fmain%2Fprovider%2Fcloud%2Ffs_synapse_azuredeploy.json)

 The only 2 required parameters during the set-up are:
@ -38,8 +40,13 @@ The only 2 required parameters during the set-up are:
 - **Principal ID** this is to set the storage permissions for the feast registry store. You can find the value for this by opening **Cloud Shell** and run the following command:

 ```bash
+# If you are using Azure portal CLI or Azure CLI 2.37.0 or above
+az ad signed-in-user show --query id -o tsv
+
+# If you are using Azure CLI below 2.37.0
 az ad signed-in-user show --query objectId -o tsv
 ```
+
 ## Feast on Azure Kubernetes Service (AKS)
 The [Feast on AKS](./cluster/README.md) is a secure enterprise deployment on a customer provisioned AKS cluster in an Azure VNET. This deployment does not expose any public endpoints and leverages Azure Private Link over the Feast core and serving so that the traffic flows over the Microsoft Network. This deployment also supports secure connectivity from Azure Synapse Analytics and Azure Machine Learning via private endpoints.
 ### 📐 Architecture
--- a/provider/README.md
+++ b/provider/README.md
@ -24,6 +24,8 @@ We recommend the [getting started tutorial](./tutorial) that walks you through a

 You can deploy the infrastructure for feast using:

+> You may want to first make sure your subscription has registered `Microsoft.Synapse`, `Microsoft.SQL` and `Microsoft.Network` providers before running the template below, as some of them may require explicit registration.
+
 [![Deploy to Azure](https://aka.ms/deploytoazurebutton)](https://portal.azure.com/#create/Microsoft.Template/uri/https%3A%2F%2Fraw.githubusercontent.com%2FAzure%2Ffeast-azure%2Fmain%2Fprovider%2Fcloud%2Ffs_synapse_azuredeploy.json)

 The only 2 required parameters during the set-up are:
@ -32,6 +34,10 @@ The only 2 required parameters during the set-up are:
 - **Principal ID** this is to set the storage permissions for the feast registry store. You can find the value for this by opening **Cloud Shell** and run the following command:

 ```bash
+# If you are using Azure portal CLI or Azure CLI 2.37.0 or above
+az ad signed-in-user show --query id -o tsv
+
+# If you are using Azure CLI below 2.37.0
 az ad signed-in-user show --query objectId -o tsv
 ```

--- a/provider/cloud/fs_sqldb_azuredeploy.json
+++ b/provider/cloud/fs_sqldb_azuredeploy.json
@ -11,7 +11,7 @@
        "principalId": {
            "type": "string",
            "metadata": {
-                "description": "Specifies the principal ID assigned to the role. You can find in cloud shell using 'az ad signed-in-user show --query objectId -o tsv'"
+                "description": "Specifies the principal ID assigned to the role. You can find in cloud shell using 'az ad signed-in-user show --query id -o tsv'"
            }
        },
        "administratorLogin": {
@ -287,8 +287,7 @@
                    "dependsOn": [
                        "[resourceId('Microsoft.Sql/servers', concat(parameters('sqlServerName')))]"
                    ],
-                    "properties": {
-                    }
+                    "properties": {}
                }
            ]
        },
@ -337,7 +336,5 @@
            ]
        }
    ],
-    "outputs": {
-
-    }
+    "outputs": {}
 }
--- a/provider/cloud/fs_synapse_azuredeploy.json
+++ b/provider/cloud/fs_synapse_azuredeploy.json
@ -11,7 +11,7 @@
        "principalId": {
            "type": "string",
            "metadata": {
-                "description": "Specifies the principal ID assigned to the role. You can find in cloud shell using 'az ad signed-in-user show --query objectId -o tsv'"
+                "description": "Specifies the principal ID assigned to the role. You can find in cloud shell using 'az ad signed-in-user show --query id -o tsv'"
            }
        },
        "sku": {
@ -355,7 +355,9 @@
                    "apiVersion": "2019-06-01-preview",
                    "name": "allowAll",
                    "location": "[variables('location')]",
-                    "dependsOn": [ "[variables('workspaceName')]" ],
+                    "dependsOn": [
+                        "[variables('workspaceName')]"
+                    ],
                    "properties": {
                        "startIpAddress": "0.0.0.0",
                        "endIpAddress": "255.255.255.255"
@ -366,7 +368,9 @@
                    "apiVersion": "2019-06-01-preview",
                    "name": "AllowAllWindowsAzureIps",
                    "location": "[variables('location')]",
-                    "dependsOn": [ "[variables('workspaceName')]" ],
+                    "dependsOn": [
+                        "[variables('workspaceName')]"
+                    ],
                    "properties": {
                        "startIpAddress": "0.0.0.0",
                        "endIpAddress": "0.0.0.0"
@ -377,7 +381,9 @@
                    "apiVersion": "2019-06-01-preview",
                    "name": "default",
                    "location": "[variables('location')]",
-                    "dependsOn": [ "[variables('workspaceName')]" ],
+                    "dependsOn": [
+                        "[variables('workspaceName')]"
+                    ],
                    "properties": {
                        "grantSqlControlToManagedIdentity": {
                            "desiredState": "Enabled"
@ -403,7 +409,5 @@
            }
        }
    ],
-    "outputs": {
-
-    }
+    "outputs": {}
 }
--- a/provider/tutorial/README.md
+++ b/provider/tutorial/README.md
@ -24,8 +24,15 @@ The only 2 required parameters during the set-up are:
 - **Principal ID** this is to set the storage permissions for the feast registry store. You can find the value for this by opening **Cloud Shell** and run the following command:

 ```bash
+# If you are using Azure portal CLI or Azure CLI 2.37.0 or above
+az ad signed-in-user show --query id -o tsv
+
+# If you are using Azure CLI below 2.37.0
 az ad signed-in-user show --query objectId -o tsv
 ```
+
+> You may want to first make sure your subscription has registered `Microsoft.Synapse`, `Microsoft.SQL` and `Microsoft.Network` providers before running the template below, as some of them may require explicit registration.
+
 [![Deploy to Azure](https://aka.ms/deploytoazurebutton)](https://portal.azure.com/#create/Microsoft.Template/uri/https%3A%2F%2Fraw.githubusercontent.com%2FAzure%2Ffeast-azure%2Fmain%2Fprovider%2Fcloud%2Ffs_synapse_azuredeploy.json)

 ![feast architecture](media/arch.png)
@ -69,3 +76,5 @@ In the Azure ML Studio, select *Notebooks* from the left-hand menu and then open
 In the Azure ML Studio, select *Notebooks* from the left-hand menu and then open the [train and deploy a model using feast notebook](notebooks/part3-train-and-deploy-with-feast.ipynb). Work through this notebook.

 > __💁Ensure the Jupyter kernel is set to Python 3.8 - AzureML__
+>
+> If problems are encountered during model training stage, create a new cell and rexecute `!pip install scikit-learn==0.22.1`. Upon completion, restart the Kernel and start over.
--- a/provider/tutorial/notebooks/inference.dockerfile
+++ b/provider/tutorial/notebooks/inference.dockerfile
@ -10,8 +10,10 @@ ENV PATH $AZUREML_CONDA_ENVIRONMENT_PATH/bin:$PATH
 RUN apt-get install -y gcc
 RUN apt-get install -y unixodbc-dev

-RUN pip install 'azureml-defaults==1.35.0' \
-                'feast-azure-provider==0.3.0' \
-                'scikit-learn==0.22.2.post1' \
-		        'joblib===1.1.0' \
-                'itsdangerous==2.0.1'
+RUN pip install \
+    'protobuf>=3.10,<3.20' \
+    'azureml-defaults==1.42.0' \
+    'feast-azure-provider==0.3.0' \
+    'scikit-learn==0.22.2.post1' \
+    'joblib===1.1.0' \
+    'itsdangerous==2.0.1'
--- a/provider/tutorial/notebooks/part3-train-and-deploy-with-feast.ipynb
+++ b/provider/tutorial/notebooks/part3-train-and-deploy-with-feast.ipynb
@ -347,6 +347,22 @@
        "\n",
        "service.run(input_data=input_payload)"
      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "## Clean up service"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {},
+      "outputs": [],
+      "source": [
+        "service.delete()"
+      ]
    }
  ],
  "metadata": {