edit github workflow naming conventions (#364)
* edit github workflow naming conventions * edits to contributing * a few more minor edits while we're here * prevent bug
This commit is contained in:
Родитель
088ce2c552
Коммит
ba95da855d
|
@ -1,4 +1,4 @@
|
||||||
name: notebook-tll
|
name: notebook-train-lightgbm-local
|
||||||
on:
|
on:
|
||||||
schedule:
|
schedule:
|
||||||
- cron: "0 0/2 * * *"
|
- cron: "0 0/2 * * *"
|
||||||
|
@ -7,7 +7,7 @@ on:
|
||||||
- main
|
- main
|
||||||
paths:
|
paths:
|
||||||
- notebooks/train-lightgbm-local.ipynb
|
- notebooks/train-lightgbm-local.ipynb
|
||||||
- .github/workflows/notebook-tll.yml
|
- .github/workflows/notebook-train-lightgbm-local.yml
|
||||||
jobs:
|
jobs:
|
||||||
build:
|
build:
|
||||||
runs-on: ubuntu-latest
|
runs-on: ubuntu-latest
|
|
@ -1,4 +1,4 @@
|
||||||
name: tutorial-ai
|
name: tutorial-an-introduction
|
||||||
on:
|
on:
|
||||||
schedule:
|
schedule:
|
||||||
- cron: "0 0/1 * * *"
|
- cron: "0 0/1 * * *"
|
||||||
|
@ -7,7 +7,7 @@ on:
|
||||||
- main
|
- main
|
||||||
paths:
|
paths:
|
||||||
- tutorials/an-introduction/**
|
- tutorials/an-introduction/**
|
||||||
- .github/workflows/tutorial-ai.yml
|
- .github/workflows/tutorial-an-introduction.yml
|
||||||
jobs:
|
jobs:
|
||||||
build:
|
build:
|
||||||
runs-on: ubuntu-latest
|
runs-on: ubuntu-latest
|
|
@ -1,4 +1,4 @@
|
||||||
name: tutorial-awp
|
name: tutorial-automl-with-pycaret
|
||||||
on:
|
on:
|
||||||
schedule:
|
schedule:
|
||||||
- cron: "0 0/2 * * *"
|
- cron: "0 0/2 * * *"
|
||||||
|
@ -7,7 +7,7 @@ on:
|
||||||
- main
|
- main
|
||||||
paths:
|
paths:
|
||||||
- tutorials/automl-with-pycaret/**
|
- tutorials/automl-with-pycaret/**
|
||||||
- .github/workflows/tutorial-awp.yml
|
- .github/workflows/tutorial-automl-with-pycaret.yml
|
||||||
jobs:
|
jobs:
|
||||||
build:
|
build:
|
||||||
runs-on: ubuntu-latest
|
runs-on: ubuntu-latest
|
|
@ -1,4 +1,4 @@
|
||||||
name: tutorial-de
|
name: tutorial-deploy-edge
|
||||||
on:
|
on:
|
||||||
schedule:
|
schedule:
|
||||||
- cron: "0 0/2 * * *"
|
- cron: "0 0/2 * * *"
|
||||||
|
@ -7,7 +7,7 @@ on:
|
||||||
- main
|
- main
|
||||||
paths:
|
paths:
|
||||||
- tutorials/deploy-edge/**
|
- tutorials/deploy-edge/**
|
||||||
- .github/workflows/tutorial-de.yml
|
- .github/workflows/tutorial-deploy-edge.yml
|
||||||
jobs:
|
jobs:
|
||||||
build:
|
build:
|
||||||
runs-on: ubuntu-latest
|
runs-on: ubuntu-latest
|
|
@ -1,4 +1,4 @@
|
||||||
name: tutorial-dt
|
name: tutorial-deploy-triton
|
||||||
on:
|
on:
|
||||||
schedule:
|
schedule:
|
||||||
- cron: "0 */2 * * *"
|
- cron: "0 */2 * * *"
|
||||||
|
@ -7,7 +7,7 @@ on:
|
||||||
- main
|
- main
|
||||||
paths:
|
paths:
|
||||||
- tutorials/deploy-triton/**
|
- tutorials/deploy-triton/**
|
||||||
- .github/workflows/tutorial-dt.yml
|
- .github/workflows/tutorial-deploy-triton.yml
|
||||||
jobs:
|
jobs:
|
||||||
build:
|
build:
|
||||||
runs-on: ubuntu-latest
|
runs-on: ubuntu-latest
|
|
@ -1,4 +1,4 @@
|
||||||
name: tutorial-ud
|
name: tutorial-using-dask
|
||||||
on:
|
on:
|
||||||
schedule:
|
schedule:
|
||||||
- cron: "0 */2 * * *"
|
- cron: "0 */2 * * *"
|
||||||
|
@ -7,7 +7,7 @@ on:
|
||||||
- main
|
- main
|
||||||
paths:
|
paths:
|
||||||
- tutorials/using-dask/**
|
- tutorials/using-dask/**
|
||||||
- .github/workflows/tutorial-ud.yml
|
- .github/workflows/tutorial-using-dask.yml
|
||||||
jobs:
|
jobs:
|
||||||
build:
|
build:
|
||||||
runs-on: ubuntu-latest
|
runs-on: ubuntu-latest
|
|
@ -1,4 +1,4 @@
|
||||||
name: tutorial-upl
|
name: tutorial-using-pytorch-lightning
|
||||||
on:
|
on:
|
||||||
schedule:
|
schedule:
|
||||||
- cron: "0 */2 * * *"
|
- cron: "0 */2 * * *"
|
||||||
|
@ -7,7 +7,7 @@ on:
|
||||||
- main
|
- main
|
||||||
paths:
|
paths:
|
||||||
- tutorials/using-pytorch-lightning/**
|
- tutorials/using-pytorch-lightning/**
|
||||||
- .github/workflows/tutorial-upl.yml
|
- .github/workflows/tutorial-using-pytorch-lightning.yml
|
||||||
jobs:
|
jobs:
|
||||||
build:
|
build:
|
||||||
runs-on: ubuntu-latest
|
runs-on: ubuntu-latest
|
|
@ -1,4 +1,4 @@
|
||||||
name: tutorial-ur
|
name: tutorial-using-rapids
|
||||||
on:
|
on:
|
||||||
schedule:
|
schedule:
|
||||||
- cron: "0 */2 * * *"
|
- cron: "0 */2 * * *"
|
||||||
|
@ -7,7 +7,7 @@ on:
|
||||||
- main
|
- main
|
||||||
paths:
|
paths:
|
||||||
- tutorials/using-rapids/**
|
- tutorials/using-rapids/**
|
||||||
- .github/workflows/tutorial-ur.yml
|
- .github/workflows/tutorial-using-rapids.yml
|
||||||
jobs:
|
jobs:
|
||||||
build:
|
build:
|
||||||
runs-on: ubuntu-latest
|
runs-on: ubuntu-latest
|
|
@ -1,4 +1,4 @@
|
||||||
name: tutorial-ux
|
name: tutorial-using-xgboost
|
||||||
on:
|
on:
|
||||||
schedule:
|
schedule:
|
||||||
- cron: "0 */2 * * *"
|
- cron: "0 */2 * * *"
|
||||||
|
@ -7,7 +7,7 @@ on:
|
||||||
- main
|
- main
|
||||||
paths:
|
paths:
|
||||||
- tutorials/using-xgboost/**
|
- tutorials/using-xgboost/**
|
||||||
- .github/workflows/tutorial-ux.yml
|
- .github/workflows/tutorial-using-xgboost.yml
|
||||||
jobs:
|
jobs:
|
||||||
build:
|
build:
|
||||||
runs-on: ubuntu-latest
|
runs-on: ubuntu-latest
|
|
@ -26,14 +26,15 @@ This repo is an opinionated set of examples using a subset of Azure Machine Lear
|
||||||
|
|
||||||
## Issues
|
## Issues
|
||||||
|
|
||||||
All forms of feedback are welcome through issues - please follow the pre-defined templates where applicable.
|
All forms of feedback are welcome through [issues](https://github.com/Azure/azureml-examples/issues) - please follow the pre-defined templates where applicable.
|
||||||
|
|
||||||
|
*Note:* [Discussions](https://github.com/Azure/azureml-examples/discussions) are new to GitHub, feel free to start one!
|
||||||
|
|
||||||
## Pull Requests
|
## Pull Requests
|
||||||
|
|
||||||
Pull requests (PRs) to this repo require review and approval by the Azure Machine Learning team to merge. Please follow the pre-defined template and read all relevant sections below.
|
Pull requests (PRs) to this repo require review and approval by the Azure Machine Learning team to merge. Please follow the pre-defined template and read all relevant sections below.
|
||||||
|
|
||||||
> **Important:**
|
**Important:** PRs from forks of this repository are likely to fail automated workflows due to access to secrets. PRs from forks will be considered but may experience additional delay for testing.
|
||||||
> PRs from forks of this repository are likely to fail automated workflows due to access to secrets. PRs from forks will be considered but may experience additional delay for testing.
|
|
||||||
|
|
||||||
### General rules
|
### General rules
|
||||||
|
|
||||||
|
@ -44,13 +45,14 @@ Pull requests (PRs) to this repo require review and approval by the Azure Machin
|
||||||
|
|
||||||
### Miscellaneous
|
### Miscellaneous
|
||||||
|
|
||||||
- to modify `README.md`, you need to modify `readme.py` and accompanying markdown files
|
- to modify `README.md`, you need to modify `readme.py` and accompanying markdown files other files (`prefix.md` and `suffix.md`)
|
||||||
- the tables in the `README.md` are auto-generated, including description, via other files
|
- you probably should not modify `README.md` or accompany markdown files
|
||||||
- develop on a branch, not a fork, for workflows to run properly
|
- develop on a branch, not a fork, for workflows to run properly
|
||||||
- use an existing environment where possible
|
- use an existing environment where possible
|
||||||
- use an existing dataset where possible
|
- use an existing dataset where possible
|
||||||
- don't register environments
|
- don't register environments
|
||||||
- don't create compute targets
|
- don't create compute targets
|
||||||
|
- don't register datasets
|
||||||
- don't modify `requirements.txt`
|
- don't modify `requirements.txt`
|
||||||
- you probably shouldn't modify any files in the root of the repo
|
- you probably shouldn't modify any files in the root of the repo
|
||||||
- you can `!pip install --upgrade packages` as needed in notebooks
|
- you can `!pip install --upgrade packages` as needed in notebooks
|
||||||
|
@ -62,7 +64,7 @@ If modifying existing examples, before a PR:
|
||||||
|
|
||||||
- run `python readme.py` from the root of the repo
|
- run `python readme.py` from the root of the repo
|
||||||
- this will generate the `README.md` file
|
- this will generate the `README.md` file
|
||||||
- this will generate github workflow files
|
- this will generate github workflow files (for workflows and notebooks)
|
||||||
- this will format Python code and notebooks
|
- this will format Python code and notebooks
|
||||||
|
|
||||||
### Enforced naming
|
### Enforced naming
|
||||||
|
@ -73,7 +75,7 @@ Enforced naming includes:
|
||||||
- directories under `tutorials` must be words separated by hyphens
|
- directories under `tutorials` must be words separated by hyphens
|
||||||
- directories under `workflows` must be one of [`train`, `deploy`, `score`, `dataprep`] - directories under are organized by ML tool
|
- directories under `workflows` must be one of [`train`, `deploy`, `score`, `dataprep`] - directories under are organized by ML tool
|
||||||
- job definition file(s) under `workflows` must contain `job` in the name
|
- job definition file(s) under `workflows` must contain `job` in the name
|
||||||
- tutorial workflows (and workflow files) use the naming convention `tutorial-*initials*`, where *initials* is the initials of the words
|
- tutorial workflows (and workflow files) use the naming convention `tutorial-*name*`, where *name* is the directory name
|
||||||
- `experiment_name` = "logical-words-example|tutorial" e.g. "hello-world-tutorial"
|
- `experiment_name` = "logical-words-example|tutorial" e.g. "hello-world-tutorial"
|
||||||
- `compute_name` = "compute-defined-in-setup-workspace.py" e.g. "gpu-K80-2"
|
- `compute_name` = "compute-defined-in-setup-workspace.py" e.g. "gpu-K80-2"
|
||||||
|
|
||||||
|
@ -146,7 +148,6 @@ Checklist:
|
||||||
- [ ] use an existing directory or add a new scenario and/or ML tool directory
|
- [ ] use an existing directory or add a new scenario and/or ML tool directory
|
||||||
- [ ] add job definition file(s) under this directory with `job` in the name
|
- [ ] add job definition file(s) under this directory with `job` in the name
|
||||||
- [ ] add user code, preserving any licensing information, under a `src` dir specific to the workflow
|
- [ ] add user code, preserving any licensing information, under a `src` dir specific to the workflow
|
||||||
- [ ] add any required environment files under a `envs` dir, shared commonly across workflows
|
|
||||||
- [ ] run `python readme.py`
|
- [ ] run `python readme.py`
|
||||||
- [ ] test
|
- [ ] test
|
||||||
- [ ] submit PR, which will run the relevant workflow(s)
|
- [ ] submit PR, which will run the relevant workflow(s)
|
||||||
|
@ -171,7 +172,7 @@ Checklist:
|
||||||
- [ ] add the tutorial directory under `tutorials/`, following naming conventions
|
- [ ] add the tutorial directory under `tutorials/`, following naming conventions
|
||||||
- [ ] add tutorial files, which are usually notebooks and may be ordered
|
- [ ] add tutorial files, which are usually notebooks and may be ordered
|
||||||
- [ ] add `README.md` in the tutorial directory with a description (see other tutorials for format)
|
- [ ] add `README.md` in the tutorial directory with a description (see other tutorials for format)
|
||||||
- [ ] add `run-tutorial-initials`, where *initials* are the initials of the description directory (see other tutorial workflows)
|
- [ ] add `tutorial-*name*`, where *name* is the name of the directory (see other tutorial workflows)
|
||||||
- [ ] run `python readme.py`
|
- [ ] run `python readme.py`
|
||||||
- [ ] test
|
- [ ] test
|
||||||
- [ ] submit PR, which will run your tutorial if setup properly
|
- [ ] submit PR, which will run your tutorial if setup properly
|
||||||
|
|
22
README.md
22
README.md
|
@ -48,7 +48,7 @@ A lightweight template repository for automating the ML lifecycle can be found [
|
||||||
|`.cloud`|cloud templates|
|
|`.cloud`|cloud templates|
|
||||||
|`.github`|GitHub specific files like Actions workflow yaml definitions and issue templates|
|
|`.github`|GitHub specific files like Actions workflow yaml definitions and issue templates|
|
||||||
|`notebooks`|interactive jupyter notebooks for iterative ML development|
|
|`notebooks`|interactive jupyter notebooks for iterative ML development|
|
||||||
|`tutorials`|self-contained directories of end-to-end tutorials|
|
|`tutorials`|self-contained directories of tutorials|
|
||||||
|`workflows`|self-contained directories of job to be run, organized by scenario then tool then project|
|
|`workflows`|self-contained directories of job to be run, organized by scenario then tool then project|
|
||||||
|
|
||||||
## Examples
|
## Examples
|
||||||
|
@ -57,20 +57,20 @@ A lightweight template repository for automating the ML lifecycle can be found [
|
||||||
|
|
||||||
path|status|notebooks|description
|
path|status|notebooks|description
|
||||||
-|-|-|-
|
-|-|-|-
|
||||||
[an-introduction](tutorials/an-introduction)|[![an-introduction](https://github.com/Azure/azureml-examples/workflows/tutorial-ai/badge.svg)](https://github.com/Azure/azureml-examples/actions?query=workflow%3Atutorial-ai)|[1.hello-world.ipynb](tutorials/an-introduction/1.hello-world.ipynb)<br>[2.pytorch-model.ipynb](tutorials/an-introduction/2.pytorch-model.ipynb)<br>[3.pytorch-model-cloud-data.ipynb](tutorials/an-introduction/3.pytorch-model-cloud-data.ipynb)|learn the basics of Azure Machine Learning
|
[an-introduction](tutorials/an-introduction)|[![an-introduction](https://github.com/Azure/azureml-examples/workflows/tutorial-an-introduction/badge.svg)](https://github.com/Azure/azureml-examples/actions?query=workflow%3Atutorial-an-introduction)|[1.hello-world.ipynb](tutorials/an-introduction/1.hello-world.ipynb)<br>[2.pytorch-model.ipynb](tutorials/an-introduction/2.pytorch-model.ipynb)<br>[3.pytorch-model-cloud-data.ipynb](tutorials/an-introduction/3.pytorch-model-cloud-data.ipynb)|learn the basics of Azure Machine Learning
|
||||||
[automl-with-pycaret](tutorials/automl-with-pycaret)|[![automl-with-pycaret](https://github.com/Azure/azureml-examples/workflows/tutorial-awp/badge.svg)](https://github.com/Azure/azureml-examples/actions?query=workflow%3Atutorial-awp)|[1.classification.ipynb](tutorials/automl-with-pycaret/1.classification.ipynb)|learn how to automate ML with [PyCaret](https://github.com/pycaret/pycaret)
|
[automl-with-pycaret](tutorials/automl-with-pycaret)|[![automl-with-pycaret](https://github.com/Azure/azureml-examples/workflows/tutorial-automl-with-pycaret/badge.svg)](https://github.com/Azure/azureml-examples/actions?query=workflow%3Atutorial-automl-with-pycaret)|[1.classification.ipynb](tutorials/automl-with-pycaret/1.classification.ipynb)|learn how to automate ML with [PyCaret](https://github.com/pycaret/pycaret)
|
||||||
[deploy-edge](tutorials/deploy-edge)|[![deploy-edge](https://github.com/Azure/azureml-examples/workflows/tutorial-de/badge.svg)](https://github.com/Azure/azureml-examples/actions?query=workflow%3Atutorial-de)|[ase-gpu.ipynb](tutorials/deploy-edge/ase-gpu.ipynb)|learn how to use Edge device for model deployment and scoring
|
[deploy-edge](tutorials/deploy-edge)|[![deploy-edge](https://github.com/Azure/azureml-examples/workflows/tutorial-deploy-edge/badge.svg)](https://github.com/Azure/azureml-examples/actions?query=workflow%3Atutorial-deploy-edge)|[ase-gpu.ipynb](tutorials/deploy-edge/ase-gpu.ipynb)|learn how to use Edge device for model deployment and scoring
|
||||||
[deploy-triton](tutorials/deploy-triton)|[![deploy-triton](https://github.com/Azure/azureml-examples/workflows/tutorial-dt/badge.svg)](https://github.com/Azure/azureml-examples/actions?query=workflow%3Atutorial-dt)|[1.densenet-local.ipynb](tutorials/deploy-triton/1.densenet-local.ipynb)<br>[2.bidaf-aks-v100.ipynb](tutorials/deploy-triton/2.bidaf-aks-v100.ipynb)|learn how to efficiently deploy to GPUs using [triton inference server](https://github.com/triton-inference-server/server)
|
[deploy-triton](tutorials/deploy-triton)|[![deploy-triton](https://github.com/Azure/azureml-examples/workflows/tutorial-deploy-triton/badge.svg)](https://github.com/Azure/azureml-examples/actions?query=workflow%3Atutorial-deploy-triton)|[1.densenet-local.ipynb](tutorials/deploy-triton/1.densenet-local.ipynb)<br>[2.bidaf-aks-v100.ipynb](tutorials/deploy-triton/2.bidaf-aks-v100.ipynb)|learn how to efficiently deploy to GPUs using [triton inference server](https://github.com/triton-inference-server/server)
|
||||||
[using-dask](tutorials/using-dask)|[![using-dask](https://github.com/Azure/azureml-examples/workflows/tutorial-ud/badge.svg)](https://github.com/Azure/azureml-examples/actions?query=workflow%3Atutorial-ud)|[1.intro-to-dask.ipynb](tutorials/using-dask/1.intro-to-dask.ipynb)<br>[2.dask-cloudprovider.ipynb](tutorials/using-dask/2.dask-cloudprovider.ipynb)|learn how to read from cloud data and scale PyData tools (Numpy, Pandas, Scikit-Learn, etc.) with [Dask](https://dask.org)
|
[using-dask](tutorials/using-dask)|[![using-dask](https://github.com/Azure/azureml-examples/workflows/tutorial-using-dask/badge.svg)](https://github.com/Azure/azureml-examples/actions?query=workflow%3Atutorial-using-dask)|[1.intro-to-dask.ipynb](tutorials/using-dask/1.intro-to-dask.ipynb)<br>[2.dask-cloudprovider.ipynb](tutorials/using-dask/2.dask-cloudprovider.ipynb)|learn how to read from cloud data and scale PyData tools (Numpy, Pandas, Scikit-Learn, etc.) with [Dask](https://dask.org)
|
||||||
[using-pytorch-lightning](tutorials/using-pytorch-lightning)|[![using-pytorch-lightning](https://github.com/Azure/azureml-examples/workflows/tutorial-upl/badge.svg)](https://github.com/Azure/azureml-examples/actions?query=workflow%3Atutorial-upl)|[1.train-single-node.ipynb](tutorials/using-pytorch-lightning/1.train-single-node.ipynb)<br>[2.log-with-tensorboard.ipynb](tutorials/using-pytorch-lightning/2.log-with-tensorboard.ipynb)<br>[3.log-with-mlflow.ipynb](tutorials/using-pytorch-lightning/3.log-with-mlflow.ipynb)<br>[4.train-multi-node-ddp.ipynb](tutorials/using-pytorch-lightning/4.train-multi-node-ddp.ipynb)|learn how to train and log metrics with [PyTorch Lightning](https://github.com/PyTorchLightning/pytorch-lightning)
|
[using-pytorch-lightning](tutorials/using-pytorch-lightning)|[![using-pytorch-lightning](https://github.com/Azure/azureml-examples/workflows/tutorial-using-pytorch-lightning/badge.svg)](https://github.com/Azure/azureml-examples/actions?query=workflow%3Atutorial-using-pytorch-lightning)|[1.train-single-node.ipynb](tutorials/using-pytorch-lightning/1.train-single-node.ipynb)<br>[2.log-with-tensorboard.ipynb](tutorials/using-pytorch-lightning/2.log-with-tensorboard.ipynb)<br>[3.log-with-mlflow.ipynb](tutorials/using-pytorch-lightning/3.log-with-mlflow.ipynb)<br>[4.train-multi-node-ddp.ipynb](tutorials/using-pytorch-lightning/4.train-multi-node-ddp.ipynb)|learn how to train and log metrics with [PyTorch Lightning](https://github.com/PyTorchLightning/pytorch-lightning)
|
||||||
[using-rapids](tutorials/using-rapids)|[![using-rapids](https://github.com/Azure/azureml-examples/workflows/tutorial-ur/badge.svg)](https://github.com/Azure/azureml-examples/actions?query=workflow%3Atutorial-ur)|[1.train-and-hpo.ipynb](tutorials/using-rapids/1.train-and-hpo.ipynb)<br>[2.train-multi-gpu.ipynb](tutorials/using-rapids/2.train-multi-gpu.ipynb)|learn how to accelerate PyData tools (numpy, pandas, scikit-learn, etc) on NVIDIA GPUs with [rapids](https://github.com/rapidsai)
|
[using-rapids](tutorials/using-rapids)|[![using-rapids](https://github.com/Azure/azureml-examples/workflows/tutorial-using-rapids/badge.svg)](https://github.com/Azure/azureml-examples/actions?query=workflow%3Atutorial-using-rapids)|[1.train-and-hpo.ipynb](tutorials/using-rapids/1.train-and-hpo.ipynb)<br>[2.train-multi-gpu.ipynb](tutorials/using-rapids/2.train-multi-gpu.ipynb)|learn how to accelerate PyData tools (numpy, pandas, scikit-learn, etc) on NVIDIA GPUs with [rapids](https://github.com/rapidsai)
|
||||||
[using-xgboost](tutorials/using-xgboost)|[![using-xgboost](https://github.com/Azure/azureml-examples/workflows/tutorial-ux/badge.svg)](https://github.com/Azure/azureml-examples/actions?query=workflow%3Atutorial-ux)|[1.local-eda.ipynb](tutorials/using-xgboost/1.local-eda.ipynb)<br>[2.distributed-cpu.ipynb](tutorials/using-xgboost/2.distributed-cpu.ipynb)|learn how to use [XGBoost](https://github.com/dmlc/xgboost) on Azure
|
[using-xgboost](tutorials/using-xgboost)|[![using-xgboost](https://github.com/Azure/azureml-examples/workflows/tutorial-using-xgboost/badge.svg)](https://github.com/Azure/azureml-examples/actions?query=workflow%3Atutorial-using-xgboost)|[1.local-eda.ipynb](tutorials/using-xgboost/1.local-eda.ipynb)<br>[2.distributed-cpu.ipynb](tutorials/using-xgboost/2.distributed-cpu.ipynb)|learn how to use [XGBoost](https://github.com/dmlc/xgboost) on Azure
|
||||||
|
|
||||||
**Notebooks** ([notebooks](notebooks))
|
**Notebooks** ([notebooks](notebooks))
|
||||||
|
|
||||||
path|status|description
|
path|status|description
|
||||||
-|-|-
|
-|-|-
|
||||||
[notebooks/train-lightgbm-local.ipynb](notebooks/train-lightgbm-local.ipynb)|[![train-lightgbm-local](https://github.com/Azure/azureml-examples/workflows/notebook-tll/badge.svg)](https://github.com/Azure/azureml-examples/actions?query=workflow%3Anotebook-tll)|use mlflow for tracking local notebook experimentation in the cloud
|
[notebooks/train-lightgbm-local.ipynb](notebooks/train-lightgbm-local.ipynb)|[![train-lightgbm-local](https://github.com/Azure/azureml-examples/workflows/notebook-train-lightgbm-local/badge.svg)](https://github.com/Azure/azureml-examples/actions?query=workflow%3Anotebook-train-lightgbm-local)|use mlflow for tracking local notebook experimentation in the cloud
|
||||||
|
|
||||||
**Train** ([workflows/train](workflows/train))
|
**Train** ([workflows/train](workflows/train))
|
||||||
|
|
||||||
|
@ -111,5 +111,5 @@ This project has adopted the [Microsoft Open Source Code of Conduct](https://ope
|
||||||
## Reference
|
## Reference
|
||||||
|
|
||||||
- [GitHub Template](https://github.com/Azure/azureml-template)
|
- [GitHub Template](https://github.com/Azure/azureml-template)
|
||||||
- [Cheat Sheet, VSCode Snippets, and Templates](https://azure.github.io/azureml-web)
|
- [Open Source Azure ML Cheat Sheet](https://azure.github.io/azureml-web)
|
||||||
- [Azure Machine Learning Documentation](https://docs.microsoft.com/azure/machine-learning)
|
- [Azure Machine Learning Documentation](https://docs.microsoft.com/azure/machine-learning)
|
||||||
|
|
|
@ -48,7 +48,7 @@ A lightweight template repository for automating the ML lifecycle can be found [
|
||||||
|`.cloud`|cloud templates|
|
|`.cloud`|cloud templates|
|
||||||
|`.github`|GitHub specific files like Actions workflow yaml definitions and issue templates|
|
|`.github`|GitHub specific files like Actions workflow yaml definitions and issue templates|
|
||||||
|`notebooks`|interactive jupyter notebooks for iterative ML development|
|
|`notebooks`|interactive jupyter notebooks for iterative ML development|
|
||||||
|`tutorials`|self-contained directories of end-to-end tutorials|
|
|`tutorials`|self-contained directories of tutorials|
|
||||||
|`workflows`|self-contained directories of job to be run, organized by scenario then tool then project|
|
|`workflows`|self-contained directories of job to be run, organized by scenario then tool then project|
|
||||||
|
|
||||||
## Examples
|
## Examples
|
||||||
|
|
23
readme.py
23
readme.py
|
@ -67,12 +67,11 @@ def write_readme(tutorials, notebooks, workflows):
|
||||||
nbs = [f"[{nb}]({tutorial}/{nb})" for nb in nbs]
|
nbs = [f"[{nb}]({tutorial}/{nb})" for nb in nbs]
|
||||||
nbs = "<br>".join(nbs)
|
nbs = "<br>".join(nbs)
|
||||||
|
|
||||||
# get tutorial name and initials
|
# get tutorial name
|
||||||
name = tutorial.split("/")[-1]
|
name = tutorial.split("/")[-1]
|
||||||
initials = "".join(word[0][0] for word in name.split("-"))
|
|
||||||
|
|
||||||
# build entries for tutorial table
|
# build entries for tutorial table
|
||||||
status = f"[![{name}](https://github.com/Azure/azureml-examples/workflows/tutorial-{initials}/badge.svg)](https://github.com/Azure/azureml-examples/actions?query=workflow%3Atutorial-{initials})"
|
status = f"[![{name}](https://github.com/Azure/azureml-examples/workflows/tutorial-{name}/badge.svg)](https://github.com/Azure/azureml-examples/actions?query=workflow%3Atutorial-{name})"
|
||||||
description = "*no description*"
|
description = "*no description*"
|
||||||
try:
|
try:
|
||||||
with open(f"{tutorial}/README.md", "r") as f:
|
with open(f"{tutorial}/README.md", "r") as f:
|
||||||
|
@ -89,16 +88,15 @@ def write_readme(tutorials, notebooks, workflows):
|
||||||
|
|
||||||
# process notebooks
|
# process notebooks
|
||||||
for notebook in notebooks:
|
for notebook in notebooks:
|
||||||
# get notebook name and initials
|
# get notebook name
|
||||||
name = notebook.split("/")[-1].replace(".ipynb", "")
|
name = notebook.split("/")[-1].replace(".ipynb", "")
|
||||||
initials = "".join(word[0][0] for word in name.split("-"))
|
|
||||||
|
|
||||||
# read in notebook
|
# read in notebook
|
||||||
with open(notebook, "r") as f:
|
with open(notebook, "r") as f:
|
||||||
data = json.load(f)
|
data = json.load(f)
|
||||||
|
|
||||||
# build entries for notebook table
|
# build entries for notebook table
|
||||||
status = f"[![{name}](https://github.com/Azure/azureml-examples/workflows/notebook-{initials}/badge.svg)](https://github.com/Azure/azureml-examples/actions?query=workflow%3Anotebook-{initials})"
|
status = f"[![{name}](https://github.com/Azure/azureml-examples/workflows/notebook-{name}/badge.svg)](https://github.com/Azure/azureml-examples/actions?query=workflow%3Anotebook-{name})"
|
||||||
description = "*no description*"
|
description = "*no description*"
|
||||||
try:
|
try:
|
||||||
if "description: " in str(data["cells"][0]["source"]):
|
if "description: " in str(data["cells"][0]["source"]):
|
||||||
|
@ -161,12 +159,11 @@ def write_readme(tutorials, notebooks, workflows):
|
||||||
def write_workflows(notebooks, workflows):
|
def write_workflows(notebooks, workflows):
|
||||||
# process notebooks
|
# process notebooks
|
||||||
for notebook in notebooks:
|
for notebook in notebooks:
|
||||||
# get notebook name and initials
|
# get notebook name
|
||||||
name = notebook.split("/")[-1].replace(".ipynb", "")
|
name = notebook.split("/")[-1].replace(".ipynb", "")
|
||||||
initials = "".join(word[0][0] for word in name.split("-"))
|
|
||||||
|
|
||||||
# write workflow file
|
# write workflow file
|
||||||
write_notebook_workflow(notebook, name, initials)
|
write_notebook_workflow(notebook, name)
|
||||||
|
|
||||||
# process workflows
|
# process workflows
|
||||||
for workflow in workflows:
|
for workflow in workflows:
|
||||||
|
@ -215,9 +212,9 @@ def format_code():
|
||||||
os.system("black-nb --clear-output .")
|
os.system("black-nb --clear-output .")
|
||||||
|
|
||||||
|
|
||||||
def write_notebook_workflow(notebook, name, initials):
|
def write_notebook_workflow(notebook, name):
|
||||||
creds = "${{secrets.AZ_AE_CREDS}}"
|
creds = "${{secrets.AZ_AE_CREDS}}"
|
||||||
workflow_yaml = f"""name: notebook-{initials}
|
workflow_yaml = f"""name: notebook-{name}
|
||||||
on:
|
on:
|
||||||
schedule:
|
schedule:
|
||||||
- cron: "0 0/2 * * *"
|
- cron: "0 0/2 * * *"
|
||||||
|
@ -226,7 +223,7 @@ on:
|
||||||
- main
|
- main
|
||||||
paths:
|
paths:
|
||||||
- {notebook}
|
- {notebook}
|
||||||
- .github/workflows/notebook-{initials}.yml
|
- .github/workflows/notebook-{name}.yml
|
||||||
jobs:
|
jobs:
|
||||||
build:
|
build:
|
||||||
runs-on: ubuntu-latest
|
runs-on: ubuntu-latest
|
||||||
|
@ -251,7 +248,7 @@ jobs:
|
||||||
run: papermill {notebook} out.ipynb -k python\n"""
|
run: papermill {notebook} out.ipynb -k python\n"""
|
||||||
|
|
||||||
# write workflow
|
# write workflow
|
||||||
with open(f".github/workflows/notebook-{initials}.yml", "w") as f:
|
with open(f".github/workflows/notebook-{name}.yml", "w") as f:
|
||||||
f.write(workflow_yaml)
|
f.write(workflow_yaml)
|
||||||
|
|
||||||
|
|
||||||
|
|
|
@ -10,5 +10,5 @@ This project has adopted the [Microsoft Open Source Code of Conduct](https://ope
|
||||||
## Reference
|
## Reference
|
||||||
|
|
||||||
- [GitHub Template](https://github.com/Azure/azureml-template)
|
- [GitHub Template](https://github.com/Azure/azureml-template)
|
||||||
- [Cheat Sheet, VSCode Snippets, and Templates](https://azure.github.io/azureml-web)
|
- [Open Source Azure ML Cheat Sheet](https://azure.github.io/azureml-web)
|
||||||
- [Azure Machine Learning Documentation](https://docs.microsoft.com/azure/machine-learning)
|
- [Azure Machine Learning Documentation](https://docs.microsoft.com/azure/machine-learning)
|
||||||
|
|
Загрузка…
Ссылка в новой задаче