Updated metrics for latest retrained models
In the last update, we updated model retraining instructions to CNTK 2.1 and TF 1.2. This notebook's model performance metrics now reflect the models retrained in the latest DL framework versions.
This commit is contained in:
Mary Wahl
Родитель
5f5074c6b0
Коммит
72e394f99b
|
|
@ -68,20 +68,20 @@
|
|||
"#### Azure Data Lake Store\n",
|
||||
"We have provided directions for provisioning and setting up the ADLS through the [Azure CLI 2.0](https://docs.microsoft.com/en-us/cli/azure/install-azure-cli?view=azure-cli-latest), which comes preinstalled on the Data Science Virtual Machine.\n",
|
||||
"\n",
|
||||
"1. From a command prompt, login to the Azure CLI by running the following command:\n",
|
||||
"1. From a command prompt, log into the Azure CLI by running the following command:\n",
|
||||
"\n",
|
||||
" ```\n",
|
||||
" az login\n",
|
||||
" ```\n",
|
||||
"\n",
|
||||
" You will be asked to visit a website and type in a temporary code. You will then be asked to provide your Azure account credentials.\n",
|
||||
" You will be asked to visit a website and type in a temporary code. The website may ask you to provide your Azure account credentials.\n",
|
||||
"1. When login is complete, return to the CLI and run the following command to determine which subscriptions are available in your account:\n",
|
||||
"\n",
|
||||
" ```\n",
|
||||
" az account list\n",
|
||||
" ```\n",
|
||||
"\n",
|
||||
" Note the \"id\" of the subscription you would like to use when creating resources, then execute the command below to set it as the active subscription:\n",
|
||||
" Copy the \"id\" of the subscription you would like to use when creating resources, then execute the command below to set it as the active subscription:\n",
|
||||
"\n",
|
||||
" ```\n",
|
||||
" az account set --subscription [subscription id]\n",
|
||||
|
|
@ -222,7 +222,7 @@
|
|||
" `/usr/bin/anaconda/envs/cntk-py35/bin/python`<br/><br/>\n",
|
||||
" \n",
|
||||
"1. Click on the green \"Save\" button that appears at upper right.\n",
|
||||
" - You may receive an error regarding a setting that you did not change. This value was set by default during HDInsight cluster deployment; the error can be safely disregarded.\n",
|
||||
" - You may receive a warning regarding a setting that you did not change. This value was set by default during HDInsight cluster deployment; the warning can be safely disregarded.\n",
|
||||
"1. When prompted, click the orange \"Restart\" button and select \"Restart all affected\".\n",
|
||||
"1. When the restart concludes, close the window. This will return you to a pane of dashboard options."
|
||||
]
|
||||
|
|
@ -280,7 +280,7 @@
|
|||
"data": {
|
||||
"text/html": [
|
||||
"<table>\n",
|
||||
"<tr><th>ID</th><th>YARN Application ID</th><th>Kind</th><th>State</th><th>Spark UI</th><th>Driver log</th><th>Current session?</th></tr><tr><td>4</td><td>application_1496866958141_0014</td><td>pyspark3</td><td>idle</td><td><a target=\"_blank\" href=\"http://hn1-mawahd.1moayawcuddupndti3u4gwc3nf.gx.internal.cloudapp.net:8088/proxy/application_1496866958141_0014/\">Link</a></td><td><a target=\"_blank\" href=\"http://10.0.0.16:30060/node/containerlogs/container_1496866958141_0014_01_000001/livy\">Link</a></td><td>✔</td></tr></table>"
|
||||
"<tr><th>ID</th><th>YARN Application ID</th><th>Kind</th><th>State</th><th>Spark UI</th><th>Driver log</th><th>Current session?</th></tr><tr><td>0</td><td>application_1504722417824_0006</td><td>pyspark3</td><td>idle</td><td><a target=\"_blank\" href=\"http://hn1-mawahd.w0jjvdca0zye3nndw25ggycjfa.cx.internal.cloudapp.net:8088/proxy/application_1504722417824_0006/\">Link</a></td><td><a target=\"_blank\" href=\"http://10.0.0.13:30060/node/containerlogs/container_1504722417824_0006_01_000001/livy\">Link</a></td><td>✔</td></tr></table>"
|
||||
],
|
||||
"text/plain": [
|
||||
"<IPython.core.display.HTML object>"
|
||||
|
|
@ -317,8 +317,8 @@
|
|||
" folder, _ = os.path.split(my_filename)\n",
|
||||
" return(label_to_number_dict[os.path.basename(folder)])\n",
|
||||
"\n",
|
||||
"adls_name = ''\n",
|
||||
"adls_folder = 'balancedtest'\n",
|
||||
"adls_name = 'mawahdemo2'\n",
|
||||
"adls_folder = 'balancedvalidationset'\n",
|
||||
"\n",
|
||||
"n_workers = 10\n",
|
||||
"local_tmp_dir = '/tmp/models'\n",
|
||||
|
|
@ -352,7 +352,7 @@
|
|||
"source": [
|
||||
"from cntk import load_model\n",
|
||||
"\n",
|
||||
"cntk_model_filepath = '{}/cntkv2rc1_50.dnn'.format(local_tmp_dir)\n",
|
||||
"cntk_model_filepath = '{}/retrained.model'.format(local_tmp_dir)\n",
|
||||
"cntk_model_filepath_bc = sc.broadcast(cntk_model_filepath)\n",
|
||||
"sc.addFile(cntk_model_filepath)"
|
||||
]
|
||||
|
|
@ -419,7 +419,7 @@
|
|||
"output_type": "stream",
|
||||
"text": [
|
||||
"Scored 11760 images\n",
|
||||
"0:06:47.688211"
|
||||
"Time elapsed: 0:07:49.978144"
|
||||
]
|
||||
}
|
||||
],
|
||||
|
|
@ -430,7 +430,7 @@
|
|||
"cntk_results = labeled_images.collect()\n",
|
||||
"print('Scored {} images'.format(len(cntk_results)))\n",
|
||||
"stop = pd.datetime.now()\n",
|
||||
"print(stop - start)"
|
||||
"print('Time elapsed: {}'.format(stop - start))"
|
||||
]
|
||||
},
|
||||
{
|
||||
|
|
@ -440,11 +440,12 @@
|
|||
"editable": true
|
||||
},
|
||||
"source": [
|
||||
"Note that this step may take 5-10 minutes to complete.\n",
|
||||
"Note that this step may take up to minutes to complete.\n",
|
||||
"\n",
|
||||
"<a name=\"cntkevaluate\"></a>\n",
|
||||
"#### Evaluate the model's performance\n",
|
||||
"\n",
|
||||
"We first report the model's raw overall accuracy. We then calculate the overall accuracy when all undeveloped land types are grouped under the same label. (We will use the latter grouping in a subsequent notebook to simplify result interpretation.)"
|
||||
"We first report the model's raw overall accuracy. We then calculate the overall accuracy when all undeveloped land types are grouped under the same label. (This is done to illustrate that the majority of errors confuse different types of undeveloped land.)"
|
||||
]
|
||||
},
|
||||
{
|
||||
|
|
@ -460,8 +461,8 @@
|
|||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"When using all six categories, correctly predicted 9212 of 11760 images (78.33%)\n",
|
||||
"After regrouping land use categories, correctly predicted 10818 of 11760 images (91.99%)"
|
||||
"When using all six categories, correctly predicted 9528 of 11760 images (81.02%)\n",
|
||||
"After regrouping land use categories, correctly predicted 10861 of 11760 images (92.36%)"
|
||||
]
|
||||
}
|
||||
],
|
||||
|
|
@ -639,7 +640,7 @@
|
|||
"output_type": "stream",
|
||||
"text": [
|
||||
"Scored 11760 images\n",
|
||||
"0:05:36.727280"
|
||||
"0:09:13.936763"
|
||||
]
|
||||
}
|
||||
],
|
||||
|
|
@ -660,11 +661,12 @@
|
|||
"editable": true
|
||||
},
|
||||
"source": [
|
||||
"Note that this step may take 5-10 minutes to complete.\n",
|
||||
"Note that this step may take up to 10 minutes to complete.\n",
|
||||
"\n",
|
||||
"<a name=\"tfevaluate\"></a>\n",
|
||||
"#### Evaluate the model's performance\n",
|
||||
"\n",
|
||||
"We first report the model's raw overall accuracy. We also report the overall accuracy when all undeveloped land types are grouped under the same label. (We will use the latter grouping in a subsequent notebook to simplify result interpretation.)"
|
||||
"We first report the model's raw overall accuracy. We then calculate the overall accuracy when all undeveloped land types are grouped under the same label. (This is done to illustrate that the majority of errors confuse different types of undeveloped land.)"
|
||||
]
|
||||
},
|
||||
{
|
||||
|
|
@ -680,8 +682,8 @@
|
|||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"When using all six categories, correctly predicted 9611 of 11760 images (81.73%)\n",
|
||||
"After regrouping land use categories, correctly predicted 10788 of 11760 images (91.73%)"
|
||||
"When using all six categories, correctly predicted 8844 of 11760 images (75.20%)\n",
|
||||
"After regrouping land use categories, correctly predicted 10931 of 11760 images (92.95%)"
|
||||
]
|
||||
}
|
||||
],
|
||||
|
|
|
|||
Загрузка…
Ссылка в новой задаче