Updates readme with cookiecutter readme
This commit is contained in:
Родитель
1d11d8d7a3
Коммит
8a1dd22d6c
35
README.md
35
README.md
|
@ -22,7 +22,7 @@ Each of the notebooks trains a simple Convolution Neural Network on the CIFAR10
|
|||
This project was developed and tested on an [Azure Ubuntu DSVM](https://azuremarketplace.microsoft.com/en-us/marketplace/apps/microsoft-ads.linux-data-science-vm-ubuntu) but should be compatible with any Linux distribution. The prerequisites for this are:
|
||||
* [Azure account](https://azure.microsoft.com/en-gb/free/)
|
||||
* [Register for Batch AI](https://docs.microsoft.com/en-gb/azure/batch-ai/quickstart-python)
|
||||
* [anaconda-project installed](https://github.com/Anaconda-Platform/anaconda-project) on VM or local machine
|
||||
* [Cookiecutter installed](https://github.com/audreyr/cookiecutter) on VM or local machine
|
||||
* Docker installed (only required for local testing and creating docker images)
|
||||
* Install [jq](https://github.com/stedolan/jq) if not already installed
|
||||
```bash
|
||||
|
@ -39,24 +39,22 @@ sudo usermod -aG docker $USER
|
|||
You may need to log out and log back in again for changes to take effect. Instructions from https://docs.docker.com/engine/installation/linux/linux-postinstall/#manage-docker-as-a-non-root-user
|
||||
|
||||
|
||||
##### *** Tip ***
|
||||
Anaconda project sets a number of environment variables and when executing sudo commands these won't be available in the sudo environment. To make them available execute your sudo commands with the -E switch.
|
||||
|
||||
## Setting up project
|
||||
Once you have docker installed and anaconda-project you can start setting up the project.
|
||||
Many of the interactions happen through anaconda-project and to list the available commands simply run:
|
||||
|
||||
```bash
|
||||
anaconda-project list-commands
|
||||
cookiecutter https://github.com/Microsoft/deep_bait.git --checkout cookiecutter
|
||||
```
|
||||
|
||||
When you first set the project you will need to set a number of things up. These are handled through anaconda-project. It will create the appropriate environment, install the appropriate packages, download and prepare the data etc. Run the following to prepare the environment:
|
||||
When you first set the project you will need to set a number of things up. These are handled through cookiecutter, for options that you want to leave the same simply press enter.
|
||||
It will create the appropriate anaconda environment locally inside the envs folder.
|
||||
Once that is done activate the newly created conda env and run the initial-setup by running the commands below.
|
||||
```bash
|
||||
anaconda-project prepare
|
||||
conda activate envs
|
||||
make initial-setup
|
||||
```
|
||||
It will ask you for a number of variables as well as where to store the data locally. If nothing is specified it will use the local folder data.
|
||||
|
||||
Once the environment is set up we need to install some further packages as well as create some Azure resources. The following command will:
|
||||
The above command will do the following:
|
||||
* Install the Azure CLI
|
||||
* Install Blobxfer
|
||||
* Log you in to Azure and select the appropriate subscription
|
||||
|
@ -65,34 +63,25 @@ Once the environment is set up we need to install some further packages as well
|
|||
* Create a storage account and fileshare
|
||||
* Download the CIFAR data and upload it to the fileshare
|
||||
|
||||
```bash
|
||||
anaconda-project run setup
|
||||
```
|
||||
The command can take a while. If you want more information on what is going on you can use the --verbose flag. Pay attention since it will ask you to log in to your Azure account. If you want a better understanding of what is going on have a look at the [Makefile]({{cookiecutter.repo_name}}/Makefile)
|
||||
|
||||
The command can take a while. Pay attention since it will ask you to log in to your Azure account. If you want a better understanding of what is going on have a look at the [Makefile]({{cookiecutter.repo_name}}/Makefile)
|
||||
|
||||
## Run Batch AI
|
||||
Instructions on how to setup the cluster and start submitting jobs are detailed in [ExploringBatchAI]({{cookiecutter.repo_name}}/ExploringBatchAI.ipynb) notebook. To start it run the following command:
|
||||
|
||||
```bash
|
||||
anaconda-project run notebook --no-browser
|
||||
make notebook
|
||||
```
|
||||
|
||||
We are assuming you are executing on a VM and that is the reason for the --no-browser switch.
|
||||
This will start a Jupyter notebook server that should be reachable the same way you would reach your standard Jupyter notebook server.
|
||||
We are assuming you are executing on a VM so it will not try to open up a browser. This will start a Jupyter notebook server that should be reachable the same way you would reach your standard Jupyter notebook server.
|
||||
The above commands assumes that the appropriate ports are open on the VM and that the server has been set up to accept connections from any ip.
|
||||
Other switches and arguments will also work so if you want to specify ip or port then you can simply add them on to the end.
|
||||
|
||||
You can also interact with Batch AI by running:
|
||||
```bash
|
||||
anaconda-project run ipython -r setup_bait.py
|
||||
```
|
||||
|
||||
## Local Development
|
||||
When executing jobs on services such as Batch AI it is important to iron out as many of the bugs before executing on the cluster. That is why with this project there is a folder called [local_test]({{cookiecutter.repo_name}}/local_test) that includes a Makefile that allows you to run notebook servers inside the containers as well as test the execution of the containers.
|
||||
|
||||
To run any of the notebooks:
|
||||
```bash
|
||||
anaconda-project --verbose run bash
|
||||
cd local_test
|
||||
make cntk-nb-server
|
||||
```
|
||||
|
|
Загрузка…
Ссылка в новой задаче