microsoft
/
CNTK
зеркало из
1
0
Форкнуть 0
Код Задачи Пакеты Проекты Релизы Вики Активность

addressed comments from CR

This commit is contained in:
REDMOND\sayanpa 2017-01-18 19:36:30 -08:00
Родитель e8099671a4
Коммит cf6f50cc26
4 изменённых файлов: 40 добавлений и 53 удалений
Показать статистику Скачать Patch файл Скачать Diff файл Показать все файлы Свернуть все файлы

12
Tests/EndToEndTests/CNTKv2Python/Examples/CNTK_206A_IOT_Example_with_LSTM_test.py → Tests/EndToEndTests/CNTKv2Python/Examples/CNTK_106A_LSTM_Timeseries_with_Simulated_Data_test.py
Просмотреть файл

@ -9,22 +9,18 @@ import re
import numpy as np
abs_path = os.path.dirname(os.path.abspath(__file__))
notebook = os.path.join(abs_path, "..", "..", "..", "..", "Tutorials", "CNTK_206A_IOT_Example_with_LSTM.ipynb")
notebook = os.path.join(abs_path, "..", "..", "..", "..", "Tutorials", "CNTK_106A_LSTM_Timeseries_with_Simulated_Data.ipynb")
def test_cntk_206A_iot_example_with_LSTM_noErrors(nb):
def test_cntk_106A_lstm_timeseries_with_simulated_data_noErrors(nb):
errors = [output for cell in nb.cells if 'outputs' in cell
for output in cell['outputs'] if output.output_type == "error"]
assert errors == []
expectedEvalErrorByDeviceId = { -1: 0.0001, 0: 0.0001 }
def test_cntk_206A_iot_example_with_LSTM_evalCorrect(nb, device_id):
def test_cntk_106A_lstm_timeseries_with_simulated_data_evalCorrect(nb, device_id):
testCell = [cell for cell in nb.cells
if cell.cell_type == 'code' and re.search('# Print validate and test error', cell.source)]
assert len(testCell) == 1
print(testCell[0].outputs[0]['text'])
m = re.match(r"msre for test: (?P<actualEvalError>\d+\.\d+)\r?$", testCell[0].outputs[0]['text'])
print(m)
print(float(m.group('actualEvalError')))
# TODO tighten tolerances
m = re.match(r"mse for test: (?P<actualEvalError>\d+\.\d+)\r?$", testCell[0].outputs[0]['text'])
assert np.isclose(float(m.group('actualEvalError')), expectedEvalErrorByDeviceId[device_id], atol=0.000001)

4
Tools/samples.json
Просмотреть файл

@ -201,8 +201,8 @@
},
{
"category": ["Numeric"],
"name": "Timeseries with LSTM",
"url": "https://github.com/Microsoft/CNTK/blob/master/Tutorials/CNTK_206A_IOT_Example_with_LSTM.ipynb",
"name": "LSTM based Timeseries with Simulated Data",
"url": "https://github.com/Microsoft/CNTK/blob/master/Tutorials/CNTK_106A_LSTM_Timeseries_with_Simulated_Data.ipynb",
"description": "Timeseries modeling with LSTMs using simulated data.",
"language": ["Python"],
"type": ["Tutorial"]

68
Tutorials/CNTK_206A_IOT_Example_with_LSTM.ipynb → Tutorials/CNTK_106A_LSTM_Timeseries_with_Simulated_Data.ipynb
Просмотреть файл

@ -6,7 +6,7 @@
"source": [
"# CNTK 206: Part A - Time series prediction with LSTM (Basics)\n",
"\n",
"This tutorial build on top of the CNTK_104 tutorial where the first introduced use of the Cognitive Toolkit for Time Series analysis. In that tutorial we used dense feedforward network for directional prediction (up and down) of movements in stock market data. In this tutorial demonstrates how to use CNTK to predict future values in a time series.\n",
"This tutorial demonstrates how to use CNTK to predict future values in a time series using LSTMs.\n",
"\n",
"**Goal**\n",
"\n",
@ -111,27 +111,21 @@
"\n",
"- **`generate_data()`**\n",
"\n",
"> Assuming N=5, M=5, generate_data() will generate the desired sin wave and returns numpy arrrays in the following shape:\n",
"> Assuming `N` = 5, `M` = 5, k = batch size, generate_data() will generate the desired sin wave and returns numpy arrrays in the following shape:\n",
"\n",
"```\n",
"# the input to the lstm\n",
"X = [\n",
" [[y0],[y1],[y2],[y3],[y4], ...],\n",
" [[y1],[y2],[y3],[y4],[y5], ...],\n",
" ...\n",
"]\n",
"> The input set ($X$) to the lstm: $$ X = [\\{y_{11}, y_{12}, \\cdots , y_{1N}\\},\n",
" \\{y_{21}, y_{22}, \\cdots, y_{2N}\\}, \\cdots,\n",
" \\{y_{k1}, y_{k2}, \\cdots, y_{kN}\\}]\n",
"$$\n",
"\n",
"# the desired output, M(=5) steps in the future\n",
"Y = [\n",
" [y4+5],\n",
" [y5+5]\n",
" ...\n",
"]\n",
"```\n",
"> The desired output ($L$) with `M`(=5) steps in the future: $$ L = [ \\{y_{1,N+5}\\},\n",
" \\{y_{2,N+5}\\}, \\cdots, \\{y_{k,N+5}\\}]$$\n",
"\n",
"> Note: `k` is a function of the length of the time series and the number of windows of size `N` one can have for the time series.\n",
"\n",
"- **`split_data()`**\n",
"split_data() will split the data into training, validation and test sets.\n",
"\n"
"\n",
"> As the name suggests, `split_data` function will split the data into training, validation and test sets."
]
},
{
@ -151,9 +145,7 @@
"\n",
" train, val, test = data[:pos_val], data[pos_val:pos_test], data[pos_test:]\n",
"\n",
" return {\"train\": train, \"val\": val, \"test\": test}\n",
"\n",
"\n"
" return {\"train\": train, \"val\": val, \"test\": test}"
]
},
{
@ -170,8 +162,8 @@
" \"\"\"\n",
" data = fct(x)\n",
" if not isinstance(data, pd.DataFrame):\n",
" data = pd.DataFrame(dict(a=data[0:len(data) - time_shift],\n",
" b=data[time_shift:]))\n",
" data = pd.DataFrame(dict(a = data[0:len(data) - time_shift],\n",
" b = data[time_shift:]))\n",
" rnn_x = []\n",
" for i in range(len(data) - time_steps):\n",
" rnn_x.append(data['a'].iloc[i: i + time_steps].as_matrix())\n",
@ -205,7 +197,7 @@
"f, a = plt.subplots(3, 1, figsize=(12, 8))\n",
"for j, ds in enumerate([\"train\", \"val\", \"test\"]):\n",
" a[j].plot(Y[ds], label=ds + ' raw');\n",
"[i.legend() for i in a];\n"
"[i.legend() for i in a];"
]
},
{
@ -229,7 +221,7 @@
"source": [
"def lstm_model(x):\n",
" \"\"\"Create the model for time series prediction\"\"\"\n",
" with C.layers.default_options(initial_state=0.1):\n",
" with C.layers.default_options(initial_state = 0.1):\n",
" m = C.layers.Recurrence(C.layers.LSTM(N))(x)\n",
" m = C.ops.sequence.last(m)\n",
" m = C.layers.Dropout(0.2)(m)\n",
@ -248,8 +240,8 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"We define the next_batch() iterator that produces batches we can feed to the training function. \n",
"Note that because CNTK supports variable sequence length, we must feed the batches as list of sequences. This is a convenience function to generate small batches of data often referred to as minibacth."
"We define the `next_batch()` iterator that produces batches we can feed to the training function. \n",
"Note that because CNTK supports variable sequence length, we must feed the batches as list of sequences. This is a convenience function to generate small batches of data often referred to as minibatch."
]
},
{
@ -334,12 +326,12 @@
"# loss function\n",
"loss = C.ops.squared_error(z, y)\n",
"\n",
"# use msre to determin error for now\n",
"# use squared error to determine error for now\n",
"error = C.ops.squared_error(z, y)\n",
"\n",
"# use adam optimizer\n",
"momentum_time_constant = C.learner.momentum_as_time_constant_schedule(BATCH_SIZE / -math.log(0.9)) \n",
"learner = C.learner.adam_sgd(z.parameters, lr=lr_schedule, momentum=momentum_time_constant, unit_gain = True)\n",
"learner = C.learner.adam_sgd(z.parameters, lr = lr_schedule, momentum = momentum_time_constant, unit_gain = True)\n",
"trainer = C.Trainer(z, loss, error, [learner])"
]
},
@ -400,7 +392,7 @@
"outputs": [],
"source": [
"# validate\n",
"def get_msre(X,Y,label):\n",
"def get_mse(X,Y,label):\n",
" result = 0.0\n",
" for x1, y1 in next_batch(X, Y, label):\n",
" eval_error = trainer.test_minibatch({x: x1, y: y1})\n",
@ -418,7 +410,7 @@
"source": [
"# Print the train and validation errors\n",
"for label in [\"train\", \"val\"]:\n",
" print(\"msre for {}: {:.6f}\".format(label, get_msre(X, Y, label)))"
" print(\"mse for {}: {:.6f}\".format(label, get_mse(X, Y, label)))"
]
},
{
@ -431,7 +423,7 @@
"source": [
"# Print validate and test error\n",
"label = \"test\"\n",
"print(\"msre for {}: {:.6f}\".format(label, get_msre(X, Y, label)))"
"print(\"mse for {}: {:.6f}\".format(label, get_mse(X, Y, label)))"
]
},
{
@ -450,14 +442,14 @@
"outputs": [],
"source": [
"# predict\n",
"f, a = plt.subplots(3, 1, figsize=(12, 8))\n",
"f, a = plt.subplots(3, 1, figsize = (12, 8))\n",
"for j, ds in enumerate([\"train\", \"val\", \"test\"]):\n",
" results = []\n",
" for x1, y1 in next_batch(X, Y, ds):\n",
" pred = z.eval({x: x1})\n",
" results.extend(pred[:, 0])\n",
" a[j].plot(Y[ds], label=ds + ' raw');\n",
" a[j].plot(results, label=ds + ' predicted');\n",
" a[j].plot(Y[ds], label = ds + ' raw');\n",
" a[j].plot(results, label = ds + ' predicted');\n",
"[i.legend() for i in a];"
]
},
@ -469,11 +461,9 @@
"source": [
"Not perfect but close enough, considering the simplicity of the model.\n",
"\n",
"Here we used a simple sin wave but you can tinker yourself and try other time series data. generate_data() allows you to pass in a dataframe with 'a' and 'b' columns instead of a function.\n",
"Here we used a simple sin wave but you can tinker yourself and try other time series data. `generate_data()` allows you to pass in a dataframe with 'a' and 'b' columns instead of a function.\n",
"\n",
"To improve results, we could train with more data points, let the model train for more epochs or improve on the model itself.\n",
"\n",
"In future [part B of this tutorial](CNTK_206B_IOT_Example_with_LSTM.ipynb) we will use some real world data."
"To improve results, we could train with more data points, let the model train for more epochs or improve on the model itself."
]
},
{

9
bindings/python/doc/tutorials.rst
Просмотреть файл

@ -13,6 +13,10 @@ Tutorials
* Part A: `MNIST data preparation`_
* Part B: `Feed Forward autoencoder`_
#. CNTK 106: LSTM based Time Series
* Part A: `Basic LSTMs using simulated data`_
#. CNTK 201: Image classifiers with CIFAR-10 data
* Part A: `CIFAR-10 Data preparation`_
@ -26,9 +30,6 @@ Tutorials
#. CNTK 205: `Artistic Style Transfer`_
#. CNTK 206: Time Series with LSTMs
* Part A: `Basic LSTMs using simulated data`_
For our Japanese users, you can find some of the `tutorials in Japanese`_.
@ -38,6 +39,7 @@ For our Japanese users, you can find some of the `tutorials in Japanese`_.
.. _`Feed Forward classifier`: https://github.com/Microsoft/CNTK/tree/v2.0.beta8.0/Tutorials/CNTK_103B_MNIST_FeedForwardNetwork.ipynb
.. _`Time Series basics`: https://github.com/Microsoft/CNTK/tree/v2.0.beta8.0/Tutorials/CNTK_104_Finance_Timeseries_Basic_with_Pandas_Numpy.ipynb
.. _`Feed Forward autoencoder`: https://github.com/Microsoft/CNTK/tree/v2.0.beta8.0/Tutorials/CNTK_105_Basic_Autoencoder_for_Dimensionality_Reduction.ipynb
.. _`Basic LSTMs using simulated data`: https://github.com/Microsoft/CNTK/blob/v2.0.beta8.0/Tutorials/CNTK_106A_LSTM_Timeseries_with_Simulated_Data.ipynb
.. _`CIFAR-10 Data preparation`: https://github.com/Microsoft/CNTK/tree/v2.0.beta8.0/Tutorials/CNTK_201A_CIFAR-10_DataLoader.ipynb
.. _`VGG and ResNet classifiers`: https://github.com/Microsoft/CNTK/tree/v2.0.beta8.0/Tutorials/CNTK_201B_CIFAR-10_ImageHandsOn.ipynb
@ -45,6 +47,5 @@ For our Japanese users, you can find some of the `tutorials in Japanese`_.
.. _`Reinforcement learning basics`: https://github.com/Microsoft/CNTK/blob/v2.0.beta8.0/Tutorials/CNTK_203_Reinforcement_Learning_Basics.ipynb
.. _`Sequence to sequence basics`: https://github.com/Microsoft/CNTK/blob/v2.0.beta8.0/Tutorials/CNTK_204_Sequence_To_Sequence.ipynb
.. _`Artistic Style Transfer`: https://github.com/Microsoft/CNTK/blob/v2.0.beta8.0/Tutorials/CNTK_205_Artistic_Style_Transfer.ipynb
.. _`Basic LSTMs using simulated data`: https://github.com/Microsoft/CNTK/blob/v2.0.beta8.0/Tutorials/CNTK_206A_IOT_Example_with_LSTM.ipynb
.. _`tutorials in Japanese`: https://notebooks.azure.com/library/cntkbeta2_ja