Added ALL AML Snippets
This commit is contained in:
Abraham Omorogbe
Родитель
ba1fd779b2
Коммит
79c4da3fd8
|
|
@ -0,0 +1,19 @@
|
|||
---
|
||||
name: Request AzureML Snippet
|
||||
about: Issue to request a new snippet for AzureML Snippets
|
||||
title: "[Snippet]"
|
||||
labels: snippets-request
|
||||
assignees: ''
|
||||
|
||||
---
|
||||
|
||||
## Proposal for the new snippet:
|
||||
[Describe snippet you want to be added and what it will do]
|
||||
|
||||
## Example of the snippet:
|
||||
```
|
||||
[Example of Snippet Code]
|
||||
```
|
||||
|
||||
NOTE:
|
||||
All snippets are added to [this folder](https://github.com/Azure/azureml-web/tree/main/website/docs/vs-code-snippets). Feel free to submit a PR directly to the repo to fast track this snippet request
|
||||
|
|
@ -0,0 +1,13 @@
|
|||
# PR into Azure/azureml-web
|
||||
|
||||
## Checklist
|
||||
|
||||
I have:
|
||||
|
||||
- [ ] read and followed the contributing guidelines
|
||||
|
||||
## Changes
|
||||
|
||||
-
|
||||
|
||||
fixes #
|
||||
|
|
@ -0,0 +1,65 @@
|
|||
name: deploy-website
|
||||
on:
|
||||
pull_request:
|
||||
branches: [main]
|
||||
paths: [website/**]
|
||||
push:
|
||||
branches: [main]
|
||||
paths: [website/**]
|
||||
jobs:
|
||||
checks:
|
||||
if: github.event_name != 'push'
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v1
|
||||
- uses: actions/setup-node@v1
|
||||
with:
|
||||
node-version: '12.x'
|
||||
- name: Test Build
|
||||
run: |
|
||||
cd website
|
||||
if [ -e yarn.lock ]; then
|
||||
yarn install --frozen-lockfile
|
||||
elif [ -e package-lock.json ]; then
|
||||
npm ci
|
||||
else
|
||||
npm i
|
||||
fi
|
||||
yarn build
|
||||
gh-release:
|
||||
if: github.event_name != 'pull_request'
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v1
|
||||
- uses: actions/setup-node@v1
|
||||
with:
|
||||
node-version: '12.x'
|
||||
- name: Add key to allow access to repository
|
||||
env:
|
||||
SSH_AUTH_SOCK: /tmp/ssh_agent.sock
|
||||
run: |
|
||||
mkdir -p ~/.ssh
|
||||
ssh-keyscan github.com >> ~/.ssh/known_hosts
|
||||
echo "${{ secrets.GH_PAGES_DEPLOY }}" > ~/.ssh/id_rsa
|
||||
chmod 600 ~/.ssh/id_rsa
|
||||
cat <<EOT >> ~/.ssh/config
|
||||
Host github.com
|
||||
HostName github.com
|
||||
IdentityFile ~/.ssh/id_rsa
|
||||
EOT
|
||||
- name: Release to GitHub Pages
|
||||
env:
|
||||
USE_SSH: true
|
||||
GIT_USER: git
|
||||
run: |
|
||||
git config --global user.email "actions@gihub.com"
|
||||
git config --global user.name "gh-actions"
|
||||
cd website
|
||||
if [ -e yarn.lock ]; then
|
||||
yarn install --frozen-lockfile
|
||||
elif [ -e package-lock.json ]; then
|
||||
npm ci
|
||||
else
|
||||
npm i
|
||||
fi
|
||||
yarn deploy
|
||||
|
|
@ -0,0 +1 @@
|
|||
website
|
||||
15
README.md
15
README.md
|
|
@ -1,14 +1,13 @@
|
|||
# Project
|
||||
# VS Code Snippets
|
||||
|
||||
> This repo has been populated by an initial template to help get you started. Please
|
||||
> make sure to update the content to build a great experience for community-building.
|
||||
Notes for contributing Azure ML Snippets.
|
||||
|
||||
As the maintainer of this project, please make a few updates:
|
||||
For using snippets see https://azure.github.io/azureml-web/docs/vs-code-snippets/snippets.
|
||||
|
||||
1. Add snippets to `python.json`. For more details on VS Code snippets: [vs-code-docs](https://code.visualstudio.com/docs/editor/userdefinedsnippets)
|
||||
2. Run `python snippets-parser.py` to automatically update the `snippets.md` (which will document your changes)
|
||||
3. Make a PR to the `main` branch and request a review.
|
||||
|
||||
- Improving this README.MD file to provide a great experience
|
||||
- Updating SUPPORT.MD with content about this project's support experience
|
||||
- Understanding the security reporting process in SECURITY.MD
|
||||
- Remove this section from the README
|
||||
|
||||
## Contributing
|
||||
|
||||
|
|
|
|||
|
|
@ -0,0 +1,481 @@
|
|||
{
|
||||
"Imports Group: Basic": {
|
||||
"prefix": ["import-basic"],
|
||||
"body": [
|
||||
"from azureml.core import Workspace # connect to workspace",
|
||||
"from azureml.core import Experiment # connect/create experiments",
|
||||
"from azureml.core import ComputeTarget # connect to compute",
|
||||
"from azureml.core import Environment # manage e.g. Python environments",
|
||||
"from azureml.core import Datastore, Dataset # work with data",
|
||||
"$0"
|
||||
],
|
||||
"description": "Import collection of basic Azure ML classes"
|
||||
},
|
||||
|
||||
"Import Workspace": {
|
||||
"prefix": ["import-workspace"],
|
||||
"body": [
|
||||
"from azureml.core import Workspace",
|
||||
"$0"
|
||||
],
|
||||
"description": "Import Workspace class"
|
||||
},
|
||||
|
||||
"Import Compute Target": {
|
||||
"prefix": ["import-compute-target"],
|
||||
"body": [
|
||||
"from azureml.core import ComputeTarget",
|
||||
"$0"
|
||||
],
|
||||
"description": "Import ComputeTarget class"
|
||||
},
|
||||
|
||||
"Import Environment": {
|
||||
"prefix": ["import-environment"],
|
||||
"body": [
|
||||
"from azureml.core import Environment",
|
||||
"$0"
|
||||
],
|
||||
"description": "Import Environment class"
|
||||
},
|
||||
|
||||
"Import ScriptRunConfig": {
|
||||
"prefix": ["import-script-run-config", "import-src"],
|
||||
"body": [
|
||||
"from azureml.core import ScriptRunConfig",
|
||||
"$0"
|
||||
],
|
||||
"description": "Import ScriptRunConfig class"
|
||||
},
|
||||
|
||||
"Import Dataset": {
|
||||
"prefix": ["import-dataset"],
|
||||
"body": [
|
||||
"from azureml.core import Dataset",
|
||||
"$0"
|
||||
],
|
||||
"description": "Import Dataset class"
|
||||
},
|
||||
|
||||
"Import Datastore": {
|
||||
"prefix": ["import-datastore"],
|
||||
"body": [
|
||||
"from azureml.core import Datastore",
|
||||
"$0"
|
||||
],
|
||||
"description": "Import Datastore class"
|
||||
},
|
||||
|
||||
"Import Run": {
|
||||
"prefix": ["import-run"],
|
||||
"body": [
|
||||
"from azureml.core import Run",
|
||||
"$0"
|
||||
],
|
||||
"description": "Import Run class"
|
||||
},
|
||||
|
||||
"Import Conda Dependencies": {
|
||||
"prefix": ["import-conda-dependencies"],
|
||||
"body": [
|
||||
"from azureml.core.conda_dependencies import CondaDependencies",
|
||||
"$0"
|
||||
],
|
||||
"description": "Import CondaDependencies class"
|
||||
},
|
||||
|
||||
"Get Workspace From Config": {
|
||||
"prefix": ["get-workspace-config", "ws-config"],
|
||||
"body": [
|
||||
"from azureml.core import Workspace",
|
||||
"ws = Workspace.from_config()",
|
||||
"$0"
|
||||
],
|
||||
"description": "Get Azure ML Workspace from config"
|
||||
},
|
||||
|
||||
"Get Workspace": {
|
||||
"prefix": ["get-workspace", "get-ws"],
|
||||
"body": [
|
||||
"from azureml.core import Workspace",
|
||||
"ws = Workspace.get(",
|
||||
" name='${1:name}',",
|
||||
" subscription_id='${2:subscription_id}',",
|
||||
" resource_group='${3:resource_group}',",
|
||||
")",
|
||||
"$0"
|
||||
],
|
||||
"description": "Get Azure ML Workspace"
|
||||
},
|
||||
|
||||
"Get Compute": {
|
||||
"prefix": ["get-compute"],
|
||||
"body": [
|
||||
"from azureml.core import ComputeTarget",
|
||||
"target = ComputeTarget(${2:ws}, '${1:<compute_target_name>}')",
|
||||
"$0"
|
||||
],
|
||||
"description": "Get Azure ML Compute Target"
|
||||
},
|
||||
|
||||
"Get Compute with SSH": {
|
||||
"prefix": ["get-compute-ssh"],
|
||||
"body": [
|
||||
"from azureml.core.compute import AmlCompute",
|
||||
"from azureml.core.compute_target import ComputeTargetException",
|
||||
"",
|
||||
"ssh_public_key = 'public-key-here'",
|
||||
"compute_config = AmlCompute.provisioning_configuration(",
|
||||
" vm_size='$2',",
|
||||
" min_nodes=$3,",
|
||||
" max_nodes=$4,",
|
||||
" admin_username='$5',",
|
||||
" admin_user_ssh_key=ssh_public_key,",
|
||||
" vm_priority='${6|lowpriority,dedicated|}',",
|
||||
" remote_login_port_public_access='Enabled',",
|
||||
" )",
|
||||
"",
|
||||
"cluster = ComputeTarget.create(",
|
||||
" workspace=${7:workspace_name},",
|
||||
" name='${8:target_name}',",
|
||||
" compute_config=compute_config,",
|
||||
")",
|
||||
"$0"
|
||||
],
|
||||
"description": "Get Azure ML Compute Target with SSH"
|
||||
},
|
||||
|
||||
"Get Environment": {
|
||||
"prefix": ["get-environment"],
|
||||
"body": [
|
||||
"from azureml.core import Environment",
|
||||
"${2:env} = Environment('${1:<env-name>}')",
|
||||
"$0"
|
||||
],
|
||||
"description": "Get Azure ML Environment"
|
||||
},
|
||||
|
||||
"Get Environment From Pip": {
|
||||
"prefix": ["get-environment-pip", "env-pip"],
|
||||
"body": [
|
||||
"from azureml.core import Environment",
|
||||
"env = Environment.from_pip_requirements(",
|
||||
" name='${1:env_name}',",
|
||||
" file_path='${2:requirements.txt}',",
|
||||
")",
|
||||
"$0"
|
||||
],
|
||||
"description": "Create environment from pip requirements.txt"
|
||||
},
|
||||
|
||||
"Get Environment From Conda": {
|
||||
"prefix": ["get-environment-conda", "env-conda"],
|
||||
"body": [
|
||||
"from azureml.core import Environment",
|
||||
"env = Environment.from_conda_specification(",
|
||||
" name='${1:env_name}',",
|
||||
" file_path='${2:env.yml}',",
|
||||
")",
|
||||
"$0"
|
||||
],
|
||||
"description": "Create environment from Conda env.yml file"
|
||||
},
|
||||
|
||||
"Get Environment From SDK": {
|
||||
"prefix": ["get-environment-sdk", "env-sdk"],
|
||||
"body": [
|
||||
"from azureml.core import Environment",
|
||||
"from azureml.core.conda_dependencies import CondaDependencies",
|
||||
"env = Environment('${1:my-env}')",
|
||||
"",
|
||||
"conda = CondaDependencies()",
|
||||
"",
|
||||
"# add channels",
|
||||
"conda.add_channel('$2')",
|
||||
"",
|
||||
"# add conda packages",
|
||||
"conda.add_conda_package('$3')",
|
||||
"",
|
||||
"# add pip packages",
|
||||
"conda.add_pip_package('$4')",
|
||||
"",
|
||||
"# add conda dependencies to environment",
|
||||
"env.python.conda_dependencies = conda",
|
||||
"$0"
|
||||
],
|
||||
"description": "Create environment using CondaDependencies class"
|
||||
},
|
||||
|
||||
"Get Environment From Custom image": {
|
||||
"prefix": ["get-environment-custom-image", "env-image"],
|
||||
"body": [
|
||||
"from azureml.core import Environment",
|
||||
"env = Environment('${1:my-env}')",
|
||||
"",
|
||||
"env.docker.enabled = True",
|
||||
"",
|
||||
"# base image for DockerHub",
|
||||
"env.docker.base_image = '${2}'",
|
||||
"",
|
||||
"# if you are using base image from a Dockerfile",
|
||||
"# env.docker.base_image = None",
|
||||
"# env.docker.base_dockerfile = './Dockerfile'",
|
||||
"",
|
||||
"# The user_managed_dependencies flag to True will use your custom image's built-in Python environment. ",
|
||||
"env.python.user_managed_dependencies = True",
|
||||
"",
|
||||
"$0"
|
||||
],
|
||||
"description": "Create environment using Custom image"
|
||||
},
|
||||
|
||||
"Workspace Compute Targets": {
|
||||
"prefix": ["ws-compute-target"],
|
||||
"body": [
|
||||
"target = ws.compute_targets['${1:target-name}']",
|
||||
"$0"
|
||||
],
|
||||
"description": "Get compute target from workspace"
|
||||
},
|
||||
|
||||
"Workspace Environments": {
|
||||
"prefix": ["ws-environment"],
|
||||
"body": [
|
||||
"env = ws.environments['${1:env-name}']",
|
||||
"$0"
|
||||
],
|
||||
"description": "Get environment from workspace"
|
||||
},
|
||||
|
||||
"Workspace Datastores": {
|
||||
"prefix": ["ws-datastore"],
|
||||
"body": [
|
||||
"datastore = ws.datastores['${1:datastore-name}']",
|
||||
"$0"
|
||||
],
|
||||
"description": "Get datastore from workspace"
|
||||
},
|
||||
|
||||
"Workspace Datasets": {
|
||||
"prefix": ["ws-dataset"],
|
||||
"body": [
|
||||
"dataset = ws.datasets['${1:dataset-name}']",
|
||||
"$0"
|
||||
],
|
||||
"description": "Get dataset from workspace"
|
||||
},
|
||||
|
||||
"Workspace Experiment": {
|
||||
"prefix": ["ws-experiment"],
|
||||
"body": [
|
||||
"exp = ws.experiments['${1:experiment-name}']",
|
||||
"$0"
|
||||
],
|
||||
"description": "Get (existing) experiment from workspace"
|
||||
},
|
||||
|
||||
"Workspace Models": {
|
||||
"prefix": ["ws-model"],
|
||||
"body": [
|
||||
"model = ws.models['${1:model-name}']",
|
||||
"$0"
|
||||
],
|
||||
"description": "Get model from workspace"
|
||||
},
|
||||
|
||||
"Script Run Config": {
|
||||
"prefix": ["script-run-config", "src"],
|
||||
"body": [
|
||||
"from azureml.core import Workspace, Experiment, ScriptRunConfig",
|
||||
"",
|
||||
"# get workspace",
|
||||
"ws = Workspace.from_config()",
|
||||
"",
|
||||
"# get compute target",
|
||||
"target = ws.compute_targets['${1:target-name}']",
|
||||
"",
|
||||
"# get registered environment",
|
||||
"env = ws.environments['${2:env-name}']",
|
||||
"",
|
||||
"# get/create experiment",
|
||||
"exp = Experiment(ws, '${3:experiment_name}')",
|
||||
"",
|
||||
"# set up script run configuration",
|
||||
"config = ScriptRunConfig(",
|
||||
" source_directory='${4:.}',",
|
||||
" script='${5:script.py}',",
|
||||
" compute_target=target,",
|
||||
" environment=env,",
|
||||
" arguments=[${6:'--meaning', 42}],",
|
||||
")",
|
||||
"",
|
||||
"# submit script to AML",
|
||||
"run = exp.submit(config)",
|
||||
"print(run.get_portal_url()) # link to ml.azure.com",
|
||||
"run.wait_for_completion(show_output=True)",
|
||||
"$0"
|
||||
],
|
||||
"description": "Set up ScriptRunConfig including compute target, environment and experiment"
|
||||
},
|
||||
|
||||
"Script Run Config with Command": {
|
||||
"prefix": ["script-run-config-command", "command-src", "src-command"],
|
||||
"body": [
|
||||
"from azureml.core import Workspace, Experiment, ScriptRunConfig",
|
||||
"",
|
||||
"# get workspace",
|
||||
"ws = Workspace.from_config()",
|
||||
"",
|
||||
"# get compute target",
|
||||
"target = ws.compute_targets['${1:target-name}']",
|
||||
"",
|
||||
"# get registered environment",
|
||||
"env = ws.environments['${2:env-name}']",
|
||||
"",
|
||||
"# get/create experiment",
|
||||
"exp = Experiment(ws, '${3:experiment_name}')",
|
||||
"",
|
||||
"# create command",
|
||||
"command = 'python ${4:script.py} ${5:--argument value}'.split()",
|
||||
"",
|
||||
"# set up script run configuration",
|
||||
"config = ScriptRunConfig(",
|
||||
" source_directory='${6:.}',",
|
||||
" command=command,",
|
||||
" compute_target=target,",
|
||||
" environment=env,",
|
||||
")",
|
||||
"",
|
||||
"# submit script to AML",
|
||||
"run = exp.submit(config)",
|
||||
"print(run.get_portal_url()) # link to ml.azure.com",
|
||||
"run.wait_for_completion(show_output=True)",
|
||||
"$0"
|
||||
],
|
||||
"description": "Set up ScriptRunConfig using command argument"
|
||||
},
|
||||
"Script Run Config with Distributed Config": {
|
||||
"prefix": ["script-run-config-distributed", "distributed-src", "src-distributed"],
|
||||
"body": [
|
||||
"from azureml.core import Workspace, ScriptRunConfig, Environment, Experiment",
|
||||
"from azureml.core.runconfig import MpiConfiguration",
|
||||
"",
|
||||
"# get workspace",
|
||||
"ws = Workspace.from_config()",
|
||||
"",
|
||||
"# get compute target",
|
||||
"target = ws.compute_targets['${1:target-name}']",
|
||||
"",
|
||||
"# get curated environment",
|
||||
"curated_env_name = '${2:AzureML-PyTorch-1.6-GPU}'",
|
||||
"env = Environment.get(workspace=ws, name=curated_env_name)",
|
||||
"",
|
||||
"# get/create experiment",
|
||||
"exp = Experiment(ws, '${3:experiment_name}')",
|
||||
"",
|
||||
"# distributed job configuration",
|
||||
"distributed_job_config = MpiConfiguration(process_count_per_node=4, node_count=2)",
|
||||
"",
|
||||
"# set up script run configuration",
|
||||
"config = ScriptRunConfig(",
|
||||
" source_directory='${4:.}',",
|
||||
" script='${5:script.py}',",
|
||||
" compute_target=target,",
|
||||
" environment=env,",
|
||||
" distributed_job_config=distributed_job_config,",
|
||||
")",
|
||||
"",
|
||||
"# submit script to AML",
|
||||
"run = exp.submit(config)",
|
||||
"print(run.get_portal_url()) # link to ml.azure.com",
|
||||
"run.wait_for_completion(show_output=True)",
|
||||
"$0"
|
||||
],
|
||||
"description": "Set up ScriptRunConfig for distributed training."
|
||||
},
|
||||
|
||||
"Run Details Widget": {
|
||||
"prefix": ["run-details-widget"],
|
||||
"body": [
|
||||
"from azureml.core import Workspace,Experiment,Run",
|
||||
"from azureml.widgets import RunDetails",
|
||||
"",
|
||||
"# get workspace",
|
||||
"ws = Workspace.from_config()",
|
||||
"",
|
||||
"# get/create experiment",
|
||||
"exp = Experiment(ws, '${1:experiment_name}')",
|
||||
"",
|
||||
"# get run",
|
||||
"run = Run(exp,'${2:run_id}')",
|
||||
"",
|
||||
"# submit script to AML",
|
||||
"RunDetails(run).show()",
|
||||
"$0"
|
||||
],
|
||||
"description": "Represents a Jupyter notebook widget used to view the progress of model training."
|
||||
},
|
||||
|
||||
"Consume Dataset": {
|
||||
"prefix": ["consume-dataset"],
|
||||
"body": [
|
||||
"#azureml-core of version 1.0.72 or higher is required",
|
||||
"from azureml.core import Workspace, Dataset",
|
||||
"",
|
||||
"# get/create experiment",
|
||||
"ws = Workspace.from_config()",
|
||||
"",
|
||||
"# get dataset",
|
||||
"dataset = Dataset.get_by_name(ws, name='${1:dataset_name}')",
|
||||
"dataset.download(target_path='.', overwrite=False)",
|
||||
"$0"
|
||||
],
|
||||
"description": "Download Azure ML dataset to current working directory"
|
||||
},
|
||||
|
||||
"Create Tabular Dataset": {
|
||||
"prefix": ["create-tabular-dataset"],
|
||||
"body": [
|
||||
"from azureml.core import Workspace, Datastore, Dataset",
|
||||
"",
|
||||
"datastore_name = '${1:datastore_name}'",
|
||||
"",
|
||||
"# get workspace",
|
||||
"ws = Workspace.from_config()",
|
||||
"",
|
||||
"# retrieve an existing datastore in the workspace by name",
|
||||
"datastore = Datastore.get(ws, datastore_name)",
|
||||
"",
|
||||
"# create a TabularDataset from 1 file paths in datastore",
|
||||
"datastore_paths = [(datastore, ${2:file_path})]",
|
||||
"",
|
||||
"custom_ds = Dataset.Tabular.from_delimited_files(path=datastore_paths)",
|
||||
"$0"
|
||||
],
|
||||
"description": "Create Azure ML tabular dataset."
|
||||
},
|
||||
|
||||
"Create File Dataset": {
|
||||
"prefix": ["create-file-dataset"],
|
||||
"body": [
|
||||
"# create a FileDataset pointing to files in 'animals' folder and its subfolders recursively",
|
||||
"from azureml.core import Workspace, Datastore, Dataset",
|
||||
"",
|
||||
"datastore_name = '${1:datastore_name}'",
|
||||
"",
|
||||
"# get workspace",
|
||||
"ws = Workspace.from_config()",
|
||||
"",
|
||||
"# retrieve an existing datastore in the workspace by name",
|
||||
"datastore = Datastore.get(ws, datastore_name)",
|
||||
"",
|
||||
"# create a FileDataset pointing to files in your folder and its subfolders recursively, you can also use public web urls paths",
|
||||
"datastore_paths = [(datastore, ${2:file_path})]",
|
||||
"",
|
||||
"custom_ds = Dataset.File.from_files(path=datastore_paths)",
|
||||
"$0"
|
||||
],
|
||||
"description": "Create Azure ML file dataset."
|
||||
}
|
||||
|
||||
}
|
||||
|
|
@ -0,0 +1,112 @@
|
|||
import json
|
||||
from typing import List
|
||||
|
||||
|
||||
class Snippet:
|
||||
"""Handle json snippets
|
||||
|
||||
Parse json (VS Code) snippets file and generate markdown summary.
|
||||
"""
|
||||
|
||||
def __init__(self, name, snippet_json):
|
||||
self.name = name
|
||||
self.description = snippet_json.get("description")
|
||||
self.prefix = self._read_prefix(snippet_json.get("prefix"))
|
||||
self.body = snippet_json.get("body")
|
||||
|
||||
def __repr__(self):
|
||||
return f"Snippet({self.name})"
|
||||
|
||||
@staticmethod
|
||||
def _read_prefix(prefix):
|
||||
"""Guarentee prefix is of type List."""
|
||||
if type(prefix) == list:
|
||||
return prefix
|
||||
else:
|
||||
assert type(prefix) == str
|
||||
return [prefix]
|
||||
|
||||
def to_markdown(self) -> List[str]:
|
||||
"""Convert snippet to markdown (as list of lines)."""
|
||||
lines = []
|
||||
|
||||
# add heading
|
||||
heading = f"### {self.name}"
|
||||
lines.append(heading)
|
||||
lines.append("")
|
||||
|
||||
# add description
|
||||
description = f"Description: {self.description}"
|
||||
lines.append(description)
|
||||
lines.append("")
|
||||
|
||||
# add prefix(es)
|
||||
if len(self.prefix) > 1:
|
||||
prefix = f"Prefixes: "
|
||||
else:
|
||||
prefix = f"Prefix: "
|
||||
for p in self.prefix:
|
||||
prefix += f"`{p}`, "
|
||||
prefix = prefix[:-2] # remove trailing comma and whitespace
|
||||
lines.append(prefix)
|
||||
lines.append("")
|
||||
|
||||
# add python snippet
|
||||
lines.append("```python")
|
||||
for line in self.body:
|
||||
if line == "$0":
|
||||
continue
|
||||
lines.append(line)
|
||||
lines.append("```")
|
||||
|
||||
return lines
|
||||
|
||||
@staticmethod
|
||||
def _convert_to_json(body):
|
||||
json_body = []
|
||||
for line in body[:-1]:
|
||||
line = '"' + line + '",'
|
||||
json_body.append(line)
|
||||
line = '"' + body[-1] + '"'
|
||||
json_body.append(line)
|
||||
return json_body
|
||||
|
||||
|
||||
frontmatter = """---
|
||||
title: VS Code Snippets
|
||||
description: A collection of VS Code Snippets for working with Azure ML.
|
||||
---
|
||||
|
||||
We have compiled a collection of useful templates in the form of
|
||||
[VS code snippets](https://code.visualstudio.com/docs/editor/userdefinedsnippets).
|
||||
|
||||
|
||||
|
||||
To add these snippets to your VS Code: `ctrl+shift+p` > Type 'Configure user
|
||||
snippets' > Select `python.json`. All of these snippets are available here:
|
||||
[python.json](https://github.com/Azure/azureml-snippets/python.json)
|
||||
|
||||
"""
|
||||
|
||||
if __name__ == "__main__":
|
||||
|
||||
# parse snippets
|
||||
with open("python.json") as f:
|
||||
snippets_file = json.load(f)
|
||||
|
||||
snippets = []
|
||||
for name, snippet_json in snippets_file.items():
|
||||
snippet = Snippet(name, snippet_json)
|
||||
snippets.append(snippet)
|
||||
|
||||
# create file and write frontmatter
|
||||
md_filename = "snippets.md"
|
||||
with open(md_filename, "w") as f:
|
||||
# write frontmatter
|
||||
f.writelines(frontmatter)
|
||||
|
||||
# write each snippet
|
||||
for snippet in snippets:
|
||||
lines = snippet.to_markdown()
|
||||
for line in lines:
|
||||
f.write(line + "\n")
|
||||
|
|
@ -0,0 +1,494 @@
|
|||
---
|
||||
title: VS Code Snippets
|
||||
description: A collection of VS Code Snippets for working with Azure ML.
|
||||
---
|
||||
|
||||
We have compiled a collection of useful templates in the form of
|
||||
[VS code snippets](https://code.visualstudio.com/docs/editor/userdefinedsnippets).
|
||||
|
||||
|
||||
|
||||
To add these snippets to your VS Code: `ctrl+shift+p` > Type 'Configure user
|
||||
snippets' > Select `python.json`. All of these snippets are available here:
|
||||
[python.json](https://github.com/Azure/azureml-web/blob/main/website/docs/vs-code-snippets/python.json)
|
||||
|
||||
### Imports Group: Basic
|
||||
|
||||
Description: Import collection of basic Azure ML classes
|
||||
|
||||
Prefix: `import-basic`
|
||||
|
||||
```python
|
||||
from azureml.core import Workspace # connect to workspace
|
||||
from azureml.core import Experiment # connect/create experiments
|
||||
from azureml.core import ComputeTarget # connect to compute
|
||||
from azureml.core import Environment # manage e.g. Python environments
|
||||
from azureml.core import Datastore, Dataset # work with data
|
||||
```
|
||||
### Import Workspace
|
||||
|
||||
Description: Import Workspace class
|
||||
|
||||
Prefix: `import-workspace`
|
||||
|
||||
```python
|
||||
from azureml.core import Workspace
|
||||
```
|
||||
### Import Compute Target
|
||||
|
||||
Description: Import ComputeTarget class
|
||||
|
||||
Prefix: `import-compute-target`
|
||||
|
||||
```python
|
||||
from azureml.core import ComputeTarget
|
||||
```
|
||||
### Import Environment
|
||||
|
||||
Description: Import Environment class
|
||||
|
||||
Prefix: `import-environment`
|
||||
|
||||
```python
|
||||
from azureml.core import Environment
|
||||
```
|
||||
### Import ScriptRunConfig
|
||||
|
||||
Description: Import ScriptRunConfig class
|
||||
|
||||
Prefixes: `import-script-run-config`, `import-src`
|
||||
|
||||
```python
|
||||
from azureml.core import ScriptRunConfig
|
||||
```
|
||||
### Import Dataset
|
||||
|
||||
Description: Import Dataset class
|
||||
|
||||
Prefix: `import-dataset`
|
||||
|
||||
```python
|
||||
from azureml.core import Dataset
|
||||
```
|
||||
### Import Datastore
|
||||
|
||||
Description: Import Datastore class
|
||||
|
||||
Prefix: `import-datastore`
|
||||
|
||||
```python
|
||||
from azureml.core import Datastore
|
||||
```
|
||||
### Import Run
|
||||
|
||||
Description: Import Run class
|
||||
|
||||
Prefix: `import-run`
|
||||
|
||||
```python
|
||||
from azureml.core import Run
|
||||
```
|
||||
### Import Conda Dependencies
|
||||
|
||||
Description: Import CondaDependencies class
|
||||
|
||||
Prefix: `import-conda-dependencies`
|
||||
|
||||
```python
|
||||
from azureml.core.conda_dependencies import CondaDependencies
|
||||
```
|
||||
### Get Workspace From Config
|
||||
|
||||
Description: Get Azure ML Workspace from config
|
||||
|
||||
Prefixes: `get-workspace-config`, `ws-config`
|
||||
|
||||
```python
|
||||
from azureml.core import Workspace
|
||||
ws = Workspace.from_config()
|
||||
```
|
||||
### Get Workspace
|
||||
|
||||
Description: Get Azure ML Workspace
|
||||
|
||||
Prefixes: `get-workspace`, `get-ws`
|
||||
|
||||
```python
|
||||
from azureml.core import Workspace
|
||||
ws = Workspace.get(
|
||||
name='${1:name}',
|
||||
subscription_id='${2:subscription_id}',
|
||||
resource_group='${3:resource_group}',
|
||||
)
|
||||
```
|
||||
### Get Compute
|
||||
|
||||
Description: Get Azure ML Compute Target
|
||||
|
||||
Prefix: `get-compute`
|
||||
|
||||
```python
|
||||
from azureml.core import ComputeTarget
|
||||
target = ComputeTarget(${2:ws}, '${1:<compute_target_name>}')
|
||||
```
|
||||
### Get Compute with SSH
|
||||
|
||||
Description: Get Azure ML Compute Target with SSH
|
||||
|
||||
Prefix: `get-compute-ssh`
|
||||
|
||||
```python
|
||||
from azureml.core.compute import AmlCompute
|
||||
from azureml.core.compute_target import ComputeTargetException
|
||||
|
||||
ssh_public_key = 'public-key-here'
|
||||
compute_config = AmlCompute.provisioning_configuration(
|
||||
vm_size='$2',
|
||||
min_nodes=$3,
|
||||
max_nodes=$4,
|
||||
admin_username='$5',
|
||||
admin_user_ssh_key=ssh_public_key,
|
||||
vm_priority='${6|lowpriority,dedicated|}',
|
||||
remote_login_port_public_access='Enabled',
|
||||
)
|
||||
|
||||
cluster = ComputeTarget.create(
|
||||
workspace=${7:workspace_name},
|
||||
name='${8:target_name}',
|
||||
compute_config=compute_config,
|
||||
)
|
||||
```
|
||||
### Get Environment
|
||||
|
||||
Description: Get Azure ML Environment
|
||||
|
||||
Prefix: `get-environment`
|
||||
|
||||
```python
|
||||
from azureml.core import Environment
|
||||
${2:env} = Environment('${1:<env-name>}')
|
||||
```
|
||||
### Get Environment From Pip
|
||||
|
||||
Description: Create environment from pip requirements.txt
|
||||
|
||||
Prefixes: `get-environment-pip`, `env-pip`
|
||||
|
||||
```python
|
||||
from azureml.core import Environment
|
||||
env = Environment.from_pip_requirements(
|
||||
name='${1:env_name}',
|
||||
file_path='${2:requirements.txt}',
|
||||
)
|
||||
```
|
||||
### Get Environment From Conda
|
||||
|
||||
Description: Create environment from Conda env.yml file
|
||||
|
||||
Prefixes: `get-environment-conda`, `env-conda`
|
||||
|
||||
```python
|
||||
from azureml.core import Environment
|
||||
env = Environment.from_conda_specification(
|
||||
name='${1:env_name}',
|
||||
file_path='${2:env.yml}',
|
||||
)
|
||||
```
|
||||
### Get Environment From SDK
|
||||
|
||||
Description: Create environment using CondaDependencies class
|
||||
|
||||
Prefixes: `get-environment-sdk`, `env-sdk`
|
||||
|
||||
```python
|
||||
from azureml.core import Environment
|
||||
from azureml.core.conda_dependencies import CondaDependencies
|
||||
env = Environment('${1:my-env}')
|
||||
|
||||
conda = CondaDependencies()
|
||||
|
||||
# add channels
|
||||
conda.add_channel('$2')
|
||||
|
||||
# add conda packages
|
||||
conda.add_conda_package('$3')
|
||||
|
||||
# add pip packages
|
||||
conda.add_pip_package('$4')
|
||||
|
||||
# add conda dependencies to environment
|
||||
env.python.conda_dependencies = conda
|
||||
```
|
||||
### Get Environment From Custom image
|
||||
|
||||
Description: Create environment using Custom image
|
||||
|
||||
Prefixes: `get-environment-custom-image`, `env-image`
|
||||
|
||||
```python
|
||||
from azureml.core import Environment
|
||||
env = Environment('${1:my-env}')
|
||||
|
||||
env.docker.enabled = True
|
||||
|
||||
# base image for DockerHub
|
||||
env.docker.base_image = '${2}'
|
||||
|
||||
# if you are using base image from a Dockerfile
|
||||
# env.docker.base_image = None
|
||||
# env.docker.base_dockerfile = './Dockerfile'
|
||||
|
||||
# The user_managed_dependencies flag to True will use your custom image's built-in Python environment.
|
||||
env.python.user_managed_dependencies = True
|
||||
|
||||
```
|
||||
### Workspace Compute Targets
|
||||
|
||||
Description: Get compute target from workspace
|
||||
|
||||
Prefix: `ws-compute-target`
|
||||
|
||||
```python
|
||||
target = ws.compute_targets['${1:target-name}']
|
||||
```
|
||||
### Workspace Environments
|
||||
|
||||
Description: Get environment from workspace
|
||||
|
||||
Prefix: `ws-environment`
|
||||
|
||||
```python
|
||||
env = ws.environments['${1:env-name}']
|
||||
```
|
||||
### Workspace Datastores
|
||||
|
||||
Description: Get datastore from workspace
|
||||
|
||||
Prefix: `ws-datastore`
|
||||
|
||||
```python
|
||||
datastore = ws.datastores['${1:datastore-name}']
|
||||
```
|
||||
### Workspace Datasets
|
||||
|
||||
Description: Get dataset from workspace
|
||||
|
||||
Prefix: `ws-dataset`
|
||||
|
||||
```python
|
||||
dataset = ws.datasets['${1:dataset-name}']
|
||||
```
|
||||
### Workspace Experiment
|
||||
|
||||
Description: Get (existing) experiment from workspace
|
||||
|
||||
Prefix: `ws-experiment`
|
||||
|
||||
```python
|
||||
exp = ws.experiments['${1:experiment-name}']
|
||||
```
|
||||
### Workspace Models
|
||||
|
||||
Description: Get model from workspace
|
||||
|
||||
Prefix: `ws-model`
|
||||
|
||||
```python
|
||||
model = ws.models['${1:model-name}']
|
||||
```
|
||||
### Script Run Config
|
||||
|
||||
Description: Set up ScriptRunConfig including compute target, environment and experiment
|
||||
|
||||
Prefixes: `script-run-config`, `src`
|
||||
|
||||
```python
|
||||
from azureml.core import Workspace, Experiment, ScriptRunConfig
|
||||
|
||||
# get workspace
|
||||
ws = Workspace.from_config()
|
||||
|
||||
# get compute target
|
||||
target = ws.compute_targets['${1:target-name}']
|
||||
|
||||
# get registered environment
|
||||
env = ws.environments['${2:env-name}']
|
||||
|
||||
# get/create experiment
|
||||
exp = Experiment(ws, '${3:experiment_name}')
|
||||
|
||||
# set up script run configuration
|
||||
config = ScriptRunConfig(
|
||||
source_directory='${4:.}',
|
||||
script='${5:script.py}',
|
||||
compute_target=target,
|
||||
environment=env,
|
||||
arguments=[${6:'--meaning', 42}],
|
||||
)
|
||||
|
||||
# submit script to AML
|
||||
run = exp.submit(config)
|
||||
print(run.get_portal_url()) # link to ml.azure.com
|
||||
run.wait_for_completion(show_output=True)
|
||||
```
|
||||
### Script Run Config with Command
|
||||
|
||||
Description: Set up ScriptRunConfig using command argument
|
||||
|
||||
Prefixes: `script-run-config-command`, `command-src`, `src-command`
|
||||
|
||||
```python
|
||||
from azureml.core import Workspace, Experiment, ScriptRunConfig
|
||||
|
||||
# get workspace
|
||||
ws = Workspace.from_config()
|
||||
|
||||
# get compute target
|
||||
target = ws.compute_targets['${1:target-name}']
|
||||
|
||||
# get registered environment
|
||||
env = ws.environments['${2:env-name}']
|
||||
|
||||
# get/create experiment
|
||||
exp = Experiment(ws, '${3:experiment_name}')
|
||||
|
||||
# create command
|
||||
command = 'python ${4:script.py} ${5:--argument value}'.split()
|
||||
|
||||
# set up script run configuration
|
||||
config = ScriptRunConfig(
|
||||
source_directory='${6:.}',
|
||||
command=command,
|
||||
compute_target=target,
|
||||
environment=env,
|
||||
)
|
||||
|
||||
# submit script to AML
|
||||
run = exp.submit(config)
|
||||
print(run.get_portal_url()) # link to ml.azure.com
|
||||
run.wait_for_completion(show_output=True)
|
||||
```
|
||||
### Script Run Config with Distributed Config
|
||||
|
||||
Description: Set up ScriptRunConfig for distributed training.
|
||||
|
||||
Prefixes: `script-run-config-distributed`, `distributed-src`, `src-distributed`
|
||||
|
||||
```python
|
||||
from azureml.core import Workspace, ScriptRunConfig, Environment, Experiment
|
||||
from azureml.core.runconfig import MpiConfiguration
|
||||
|
||||
# get workspace
|
||||
ws = Workspace.from_config()
|
||||
|
||||
# get compute target
|
||||
target = ws.compute_targets['${1:target-name}']
|
||||
|
||||
# get curated environment
|
||||
curated_env_name = '${2:AzureML-PyTorch-1.6-GPU}'
|
||||
env = Environment.get(workspace=ws, name=curated_env_name)
|
||||
|
||||
# get/create experiment
|
||||
exp = Experiment(ws, '${3:experiment_name}')
|
||||
|
||||
# distributed job configuration
|
||||
distributed_job_config = MpiConfiguration(process_count_per_node=4, node_count=2)
|
||||
|
||||
# set up script run configuration
|
||||
config = ScriptRunConfig(
|
||||
source_directory='${4:.}',
|
||||
script='${5:script.py}',
|
||||
compute_target=target,
|
||||
environment=env,
|
||||
distributed_job_config=distributed_job_config,
|
||||
)
|
||||
|
||||
# submit script to AML
|
||||
run = exp.submit(config)
|
||||
print(run.get_portal_url()) # link to ml.azure.com
|
||||
run.wait_for_completion(show_output=True)
|
||||
```
|
||||
### Run Details Widget
|
||||
|
||||
Description: Represents a Jupyter notebook widget used to view the progress of model training.
|
||||
|
||||
Prefix: `run-details-widget`
|
||||
|
||||
```python
|
||||
from azureml.core import Workspace,Experiment,Run
|
||||
from azureml.widgets import RunDetails
|
||||
|
||||
# get workspace
|
||||
ws = Workspace.from_config()
|
||||
|
||||
# get/create experiment
|
||||
exp = Experiment(ws, '${1:experiment_name}')
|
||||
|
||||
# get run
|
||||
run = Run(exp,'${2:run_id}')
|
||||
|
||||
# submit script to AML
|
||||
RunDetails(run).show()
|
||||
```
|
||||
### Consume Dataset
|
||||
|
||||
Description: Download Azure ML dataset to current working directory
|
||||
|
||||
Prefix: `consume-dataset`
|
||||
|
||||
```python
|
||||
#azureml-core of version 1.0.72 or higher is required
|
||||
from azureml.core import Workspace, Dataset
|
||||
|
||||
# get/create experiment
|
||||
ws = Workspace.from_config()
|
||||
|
||||
# get dataset
|
||||
dataset = Dataset.get_by_name(ws, name='${1:dataset_name}')
|
||||
dataset.download(target_path='.', overwrite=False)
|
||||
```
|
||||
### Create Tabular Dataset
|
||||
|
||||
Description: Create Azure ML tabular dataset.
|
||||
|
||||
Prefix: `create-tabular-dataset`
|
||||
|
||||
```python
|
||||
from azureml.core import Workspace, Datastore, Dataset
|
||||
|
||||
datastore_name = '${1:datastore_name}'
|
||||
|
||||
# get workspace
|
||||
ws = Workspace.from_config()
|
||||
|
||||
# retrieve an existing datastore in the workspace by name
|
||||
datastore = Datastore.get(ws, datastore_name)
|
||||
|
||||
# create a TabularDataset from 1 file paths in datastore
|
||||
datastore_paths = [(datastore, ${2:file_path})]
|
||||
|
||||
custom_ds = Dataset.Tabular.from_delimited_files(path=datastore_paths)
|
||||
```
|
||||
### Create File Dataset
|
||||
|
||||
Description: Create Azure ML file dataset.
|
||||
|
||||
Prefix: `create-file-dataset`
|
||||
|
||||
```python
|
||||
# create a FileDataset pointing to files in 'animals' folder and its subfolders recursively
|
||||
from azureml.core import Workspace, Datastore, Dataset
|
||||
|
||||
datastore_name = '${1:datastore_name}'
|
||||
|
||||
# get workspace
|
||||
ws = Workspace.from_config()
|
||||
|
||||
# retrieve an existing datastore in the workspace by name
|
||||
datastore = Datastore.get(ws, datastore_name)
|
||||
|
||||
# create a FileDataset pointing to files in your folder and its subfolders recursively, you can also use public web urls paths
|
||||
datastore_paths = [(datastore, ${2:file_path})]
|
||||
|
||||
custom_ds = Dataset.File.from_files(path=datastore_paths)
|
||||
```
|
||||
Двоичный файл не отображается.
|
После Ширина: | Высота: | Размер: 416 KiB |
Загрузка…
Ссылка в новой задаче