зеркало из https://github.com/microsoft/SynapseML.git
doc: update find_secret on Fabric and doc (#2132)
* add find secret doc for synapse and adb * Apply suggestions from code review * update find_secret function * minor fix * fix style * update find_secret for fabric * style * fix find_secret e2e test * Update Platform.py * fix find_secret ---------
This commit is contained in:
JessicaXYWang
GitHub
Родитель
23222c0840
Коммит
2a01c8e68a
|
|
@ -47,10 +47,15 @@ def running_on_databricks():
|
|||
|
||||
|
||||
def find_secret(secret_name, keyvault):
|
||||
if running_on_synapse() or running_on_synapse_internal():
|
||||
if running_on_synapse():
|
||||
from notebookutils.mssparkutils.credentials import getSecret
|
||||
|
||||
return getSecret(keyvault, secret_name)
|
||||
elif running_on_synapse_internal():
|
||||
from notebookutils.mssparkutils.credentials import getSecret
|
||||
|
||||
keyVaultURL = f"https://{keyvault}.vault.azure.net/"
|
||||
return getSecret(keyVaultURL, secret_name)
|
||||
elif running_on_databricks():
|
||||
from pyspark.sql import SparkSession
|
||||
from pyspark.dbutils import DBUtils
|
||||
|
|
|
|||
|
|
@ -64,6 +64,160 @@
|
|||
"* [Create Azure AI service (Decision, Language, Speech, Vision) resource](https://ms.portal.azure.com/#create/Microsoft.CognitiveServicesAllInOne). You can follow the steps at [Create a multi-service resource for Azure AI services](https://learn.microsoft.com/en-us/azure/ai-services/multi-service-resource?tabs=windows&pivots=azportal#create-a-new-azure-cognitive-services-resource). \n",
|
||||
"* Get your Azure AI Services resource's key. After your resource is successfully deployed, select **Next Steps** > **Go to resource**. Once at the resource, you can get the key from **Resource Management** > **Keys and Endpoint**. Copy the key and paste it into the notebook. Store keys securely and do not share them. "
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Manage secrets with Azure Key Vault and access the secrets with find_secret"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"After you create an Azure AI resource, you will obtain a resource key. You can use this resource key directly in our notebooks as a string, but we recommend to manage secrets with Azure Key Vault. Azure Key Vault is a cloud-based service that allows you to store and manage cryptographic keys, certificates, and secrets used by cloud applications and services.\n",
|
||||
"\n",
|
||||
"You can skip the following content if you want to use a secret string instead of Azure Key Vault. This is not recommended for production workloads."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Create Azure Key Vault\n",
|
||||
"\n",
|
||||
"Refer to [this article](https://learn.microsoft.com/azure/key-vault/general/quick-create-portal) to create a key vault using Azure Portal"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Save secret to Azure Key Vault\n",
|
||||
"\n",
|
||||
"- Go to Access control (IAM) and assign **Key Vault Administrator Role** to yourself.\n",
|
||||
"- Go to Secrets and choose **+ Generate/Import**, create a key with the secret value obtained from Azure AI service.\n",
|
||||
"- Choose **Create**."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Use find_secret on Microsoft Fabric / Power BI\n",
|
||||
"\n",
|
||||
"- Go to Azure Key Vault, Select Access control (IAM), Select **+ Add**, Add role assignment.\n",
|
||||
"- Granted the Fabric or Power BI Service Account Get permissions in the Azure Key Vault."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Use find_secret on Azure Databricks\n",
|
||||
"\n",
|
||||
"- In the Azure Portal, find and select the Azure Key Vault Instance. Select the **Access configuration** tab under Settings. Set Permission model to Vault access policy.\n",
|
||||
"- On Databricks, go to `https://<databricks-instance>#secrets/createScope`. This URL is case sensitive.\n",
|
||||
" \n",
|
||||
"\n",
|
||||
"<img src=\"https://mmlspark.blob.core.windows.net/graphics/adb_create_secret_scope.png\" width=\"600\" style=\"float: center;\"/>\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"- Enter the name of the secret scope and choose desired Manage Principal.\n",
|
||||
"- Go to your Azure Key Vault -> Properties and find the **DNS Name** (Vault URI on Azure Key Vault) and **Resource ID**. Enter the DNS Name and Resource ID on Databricks createScope page.\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"<img src=\"https://mmlspark.blob.core.windows.net/graphics/adb_find_resource_id.png\" width=\"600\" style=\"float: center;\"/>\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"- Click the **Create** Button.\n",
|
||||
" \n",
|
||||
"\n",
|
||||
"Refer to [this article](https://learn.microsoft.com/azure/databricks/security/secrets/secret-scopes#--create-an-azure-key-vault-backed-secret-scope) for more details.\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Use find_secret on Synapse\n",
|
||||
"\n",
|
||||
"- Open the Synapse Studio and go to the Manage tab.\n",
|
||||
"- Under External connections, select Linked services.\n",
|
||||
"- To add a linked service, select New.\n",
|
||||
"- Select the Azure Key Vault tile from the list and select Continue.\n",
|
||||
"- Enter a linked service name and choose the key vault you want to connect to.\n",
|
||||
"- Select Create\n",
|
||||
"- Click Publish\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"- Go to Azure Key Vault, Select Access control (IAM), Select **+ Add**, Add role assignment.\n",
|
||||
"- Choose **Key Vault Administrator** in Role blade, select Next.\n",
|
||||
"- In Members blade, choose Assign access to **Managed identity**. Select members, choose the subscription your Synapse Workspace in. For Managed identity, select Synapse workspace, choose your workspace.\n",
|
||||
"- Select **Review + assign**."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Quick Test"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"vscode": {
|
||||
"languageId": "plaintext"
|
||||
}
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from synapse.ml.services.language import AnalyzeText\n",
|
||||
"from synapse.ml.core.platform import find_secret\n",
|
||||
"\n",
|
||||
"ai_service_key = find_secret(\n",
|
||||
" secret_name=\"ai-services-api-key\", keyvault=\"mmlspark-build-keys\"\n",
|
||||
") # use your own key vault name and api name\n",
|
||||
"ai_service_location = \"eastus\" # use your own AI service location\n",
|
||||
"\n",
|
||||
"df = spark.createDataFrame(\n",
|
||||
" data=[\n",
|
||||
" [\"en\", \"Dr. Smith has a very modern medical office, and she has great staff.\"],\n",
|
||||
" [\"en\", \"I had a wonderful trip to Seattle last week.\"],\n",
|
||||
" ],\n",
|
||||
" schema=[\"language\", \"text\"],\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"entity_recognition = (\n",
|
||||
" AnalyzeText()\n",
|
||||
" .setKind(\"EntityRecognition\")\n",
|
||||
" .setLocation(ai_service_location)\n",
|
||||
" .setSubscriptionKey(ai_service_key)\n",
|
||||
" .setTextCol(\"text\")\n",
|
||||
" .setOutputCol(\"entities\")\n",
|
||||
" .setErrorCol(\"error\")\n",
|
||||
" .setLanguageCol(\"language\")\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"df_results = entity_recognition.transform(df)\n",
|
||||
"display(df_results.select(\"language\", \"text\", \"entities.documents.entities\"))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Your result should looks like\n",
|
||||
"\n",
|
||||
"| language | text | entities |\n",
|
||||
"|--------------|-----------|------------|\n",
|
||||
"| en | Dr. Smith has a very modern medical office, and she has great staff. | [{\"category\": \"Person\", \"confidenceScore\": 0.98, \"length\": 5, \"offset\": 4, \"subcategory\": null, \"text\": \"Smith\"}, {\"category\": \"Location\", \"confidenceScore\": 0.79, \"length\": 14, \"offset\": 28, \"subcategory\": \"Structural\", \"text\": \"medical office\"}, {\"category\": \"PersonType\", \"confidenceScore\": 0.85, \"length\": 5, \"offset\": 62, \"subcategory\": null, \"text\": \"staff\"}] |\n",
|
||||
"| en | I had a wonderful trip to Seattle last week. | [{\"category\": \"Event\", \"confidenceScore\": 0.74, \"length\": 4, \"offset\": 18, \"subcategory\": null, \"text\": \"trip\"}, {\"category\": \"Location\", \"confidenceScore\": 1, \"length\": 7, \"offset\": 26, \"subcategory\": \"GPE\", \"text\": \"Seattle\"}, {\"category\": \"DateTime\", \"confidenceScore\": 0.8, \"length\": 9, \"offset\": 34, \"subcategory\": \"DateRange\", \"text\": \"last week\"}] |"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
|
|
|
|||
Загрузка…
Ссылка в новой задаче