1361 строка
223 KiB
Plaintext
1361 строка
223 KiB
Plaintext
{
|
|
"cells": [
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 1,
|
|
"metadata": {
|
|
"collapsed": true
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"from IPython.display import Image"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"# CNTK 201B: Hands On Labs Image Recognition"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"This hands-on lab shows how to implement image recognition task using [convolution network][] with CNTK v2 Python API. You will start with a basic feedforward CNN architecture to classify [CIFAR dataset](https://www.cs.toronto.edu/~kriz/cifar.html), then you will keep adding advanced features to your network. Finally, you will implement a VGG net and residual net like the one that won ImageNet competition but smaller in size.\n",
|
|
"\n",
|
|
"[convolution network]:https://en.wikipedia.org/wiki/Convolutional_neural_network\n",
|
|
"\n",
|
|
"## Introduction\n",
|
|
"\n",
|
|
"In this hands-on, you will practice the following:\n",
|
|
"\n",
|
|
"* Understanding subset of CNTK python API needed for image classification task.\n",
|
|
"* Write a custom convolution network to classify CIFAR dataset.\n",
|
|
"* Modifying the network structure by adding:\n",
|
|
" * [Dropout][] layer.\n",
|
|
" * Batchnormalization layer.\n",
|
|
"* Implement a [VGG][] style network.\n",
|
|
"* Introduction to Residual Nets (RESNET).\n",
|
|
"* Implement and train [RESNET network][].\n",
|
|
"\n",
|
|
"[RESNET network]:https://docs.microsoft.com/en-us/cognitive-toolkit/Hands-On-Labs-Image-Recognition \n",
|
|
"[VGG]:http://www.robots.ox.ac.uk/~vgg/research/very_deep/\n",
|
|
"[Dropout]:https://en.wikipedia.org/wiki/Dropout_(neural_networks)\n",
|
|
"\n",
|
|
"## Prerequisites\n",
|
|
"\n",
|
|
"CNTK 201A hands-on lab, in which you will download and prepare CIFAR dataset is a prerequisites for this lab. This tutorial depends on CNTK v2, so before starting this lab you will need to install CNTK v2. Furthermore, all the tutorials in this lab are done in python, therefore, you will need a basic knowledge of Python. \n",
|
|
"\n",
|
|
"CNTK 102 lab is recommended but not a prerequisite for this tutorial. However, a basic understanding of Deep Learning is needed. Familiarity with basic convolution operations is highly desirable (Refer to CNTK tutorial 103D).\n",
|
|
"\n",
|
|
"## Dataset\n",
|
|
"\n",
|
|
"You will use CIFAR 10 dataset, from https://www.cs.toronto.edu/~kriz/cifar.html, during this tutorial. The dataset contains 50000 training images and 10000 test images, all images are 32 x 32 x 3. Each image is classified as one of 10 classes as shown below:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 2,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/html": [
|
|
"<img src=\"https://cntk.ai/jup/201/cifar-10.png\" width=\"500\" height=\"500\"/>"
|
|
],
|
|
"text/plain": [
|
|
"<IPython.core.display.Image object>"
|
|
]
|
|
},
|
|
"execution_count": 2,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"# Figure 1\n",
|
|
"Image(url=\"https://cntk.ai/jup/201/cifar-10.png\", width=500, height=500)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"The above image is from: https://www.cs.toronto.edu/~kriz/cifar.html\n",
|
|
"\n",
|
|
"## Convolution Neural Network (CNN)\n",
|
|
"\n",
|
|
"We recommend completing CNTK 103D tutorial before proceeding. Here is a brief recap of Convolution Neural Network (CNN). CNN is a feedforward network comprise of a bunch of layers in such a way that the output of one layer is fed to the next layer (There are more complex architecture that skip layers, we will discuss one of those at the end of this lab). Usually, CNN start with alternating between convolution layer and pooling layer (downsample), then end up with fully connected layer for the classification part.\n",
|
|
"\n",
|
|
"### Convolution layer\n",
|
|
"\n",
|
|
"Convolution layer consist of multiple 2D convolution kernels applied on the input image or the previous layer, each convolution kernel outputs a feature map."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 3,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/html": [
|
|
"<img src=\"https://cntk.ai/jup/201/Conv2D.png\"/>"
|
|
],
|
|
"text/plain": [
|
|
"<IPython.core.display.Image object>"
|
|
]
|
|
},
|
|
"execution_count": 3,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"# Figure 2\n",
|
|
"Image(url=\"https://cntk.ai/jup/201/Conv2D.png\")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"The stack of feature maps output are the input to the next layer."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 4,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/html": [
|
|
"<img src=\"https://cntk.ai/jup/201/Conv2DFeatures.png\"/>"
|
|
],
|
|
"text/plain": [
|
|
"<IPython.core.display.Image object>"
|
|
]
|
|
},
|
|
"execution_count": 4,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"# Figure 3\n",
|
|
"Image(url=\"https://cntk.ai/jup/201/Conv2DFeatures.png\")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"> Gradient-Based Learning Applied to Document Recognition, Proceedings of the IEEE, 86(11):2278-2324, November 1998\n",
|
|
"> Y. LeCun, L. Bottou, Y. Bengio and P. Haffner\n",
|
|
"\n",
|
|
"#### In CNTK:\n",
|
|
"\n",
|
|
"Here the [convolution][] layer in Python:\n",
|
|
"\n",
|
|
"```python\n",
|
|
"def Convolution(filter_shape, # e.g. (3,3)\n",
|
|
" num_filters, # e.g. 64\n",
|
|
" activation, # relu or None...etc.\n",
|
|
" init, # Random initialization\n",
|
|
" pad, # True or False\n",
|
|
" strides) # strides e.g. (1,1)\n",
|
|
"```\n",
|
|
"\n",
|
|
"[convolution]:https://www.cntk.ai/pythondocs/layerref.html#convolution\n",
|
|
"\n",
|
|
"### Pooling layer\n",
|
|
"\n",
|
|
"In most CNN vision architecture, each convolution layer is succeeded by a pooling layer, so they keep alternating until the fully connected layer. \n",
|
|
"\n",
|
|
"The purpose of the pooling layer is as follow:\n",
|
|
"\n",
|
|
"* Reduce the dimensionality of the previous layer, which speed up the network.\n",
|
|
"* Provide a limited translation invariant.\n",
|
|
"\n",
|
|
"Here an example of max pooling with a stride of 2:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 5,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/html": [
|
|
"<img src=\"https://cntk.ai/jup/201/MaxPooling.png\" width=\"400\" height=\"400\"/>"
|
|
],
|
|
"text/plain": [
|
|
"<IPython.core.display.Image object>"
|
|
]
|
|
},
|
|
"execution_count": 5,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"# Figure 4\n",
|
|
"Image(url=\"https://cntk.ai/jup/201/MaxPooling.png\", width=400, height=400)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"#### In CNTK:\n",
|
|
"\n",
|
|
"Here the [pooling][] layer in Python:\n",
|
|
"\n",
|
|
"```python\n",
|
|
"\n",
|
|
"# Max pooling\n",
|
|
"def MaxPooling(filter_shape, # e.g. (3,3)\n",
|
|
" strides, # (2,2)\n",
|
|
" pad) # True or False\n",
|
|
"\n",
|
|
"# Average pooling\n",
|
|
"def AveragePooling(filter_shape, # e.g. (3,3)\n",
|
|
" strides, # (2,2)\n",
|
|
" pad) # True or False\n",
|
|
"```\n",
|
|
"\n",
|
|
"[pooling]:https://www.cntk.ai/pythondocs/layerref.html#maxpooling-averagepooling\n",
|
|
"\n",
|
|
"### Dropout layer\n",
|
|
"\n",
|
|
"Dropout layer takes a probability value as an input, the value is called the dropout rate. Let us say the dropout rate is 0.5, what this layer does it pick at random 50% of the nodes from the previous layer and drop them out of the network. This behavior help regularize the network.\n",
|
|
"\n",
|
|
"> Dropout: A Simple Way to Prevent Neural Networks from Overfitting\n",
|
|
"> Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, Ruslan Salakhutdinov\n",
|
|
"\n",
|
|
"\n",
|
|
"#### In CNTK:\n",
|
|
"\n",
|
|
"Dropout layer in Python:\n",
|
|
"\n",
|
|
"```python\n",
|
|
"\n",
|
|
"# Dropout\n",
|
|
"def Dropout(prob) # dropout rate e.g. 0.5\n",
|
|
"```\n",
|
|
"\n",
|
|
"### Batch normalization (BN)\n",
|
|
"\n",
|
|
"Batch normalization is a way to make the input to each layer has zero mean and unit variance. BN help the network converge faster and keep the input of each layer around zero. BN has two learnable parameters called gamma and beta, the purpose of those parameters is for the network to decide for itself if the normalized input is what is best or the raw input.\n",
|
|
"\n",
|
|
"> Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift\n",
|
|
"> Sergey Ioffe, Christian Szegedy\n",
|
|
"\n",
|
|
"#### In CNTK:\n",
|
|
"\n",
|
|
"[Batch normalization][] layer in Python:\n",
|
|
"\n",
|
|
"```python\n",
|
|
"\n",
|
|
"# Batch normalization\n",
|
|
"def BatchNormalization(map_rank) # For image map_rank=1\n",
|
|
"```\n",
|
|
"\n",
|
|
"[Batch normalization]:https://www.cntk.ai/pythondocs/layerref.html#batchnormalization-layernormalization-stabilizer\n",
|
|
"\n",
|
|
"## Microsoft Cognitive Network Toolkit (CNTK)\n",
|
|
"\n",
|
|
"CNTK is a highly flexible computation graphs, each node take inputs as tensors and produce tensors as the result of the computation. Each node is exposed in Python API, which give you the flexibility of creating any custom graphs, you can also define your own node in Python or C++ using CPU, GPU or both.\n",
|
|
"\n",
|
|
"For Deep learning, you can use the low level API directly or you can use CNTK layered API. We will start with the low level API, then switch to the layered API in this lab.\n",
|
|
"\n",
|
|
"So let's first import the needed modules for this lab."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 6,
|
|
"metadata": {
|
|
"collapsed": true
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"from __future__ import print_function # Use a function definition from future version (say 3.x from 2.7 interpreter)\n",
|
|
"\n",
|
|
"import matplotlib.pyplot as plt\n",
|
|
"import math\n",
|
|
"import numpy as np\n",
|
|
"import os\n",
|
|
"import PIL\n",
|
|
"import sys\n",
|
|
"try: \n",
|
|
" from urllib.request import urlopen \n",
|
|
"except ImportError: \n",
|
|
" from urllib import urlopen\n",
|
|
"\n",
|
|
"import cntk as C"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"In the block below, we check if we are running this notebook in the CNTK internal test machines by looking for environment variables defined there. We then select the right target device (GPU vs CPU) to test this notebook. In other cases, we use CNTK's default policy to use the best available device (GPU, if available, else CPU)."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 7,
|
|
"metadata": {
|
|
"collapsed": true
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"if 'TEST_DEVICE' in os.environ:\n",
|
|
" if os.environ['TEST_DEVICE'] == 'cpu':\n",
|
|
" C.device.try_set_default_device(C.device.cpu())\n",
|
|
" else:\n",
|
|
" C.device.try_set_default_device(C.device.gpu(0))"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 8,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/html": [
|
|
"<img src=\"https://cntk.ai/jup/201/CNN.png\"/>"
|
|
],
|
|
"text/plain": [
|
|
"<IPython.core.display.Image object>"
|
|
]
|
|
},
|
|
"execution_count": 8,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"# Figure 5\n",
|
|
"Image(url=\"https://cntk.ai/jup/201/CNN.png\")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"Now that we imported the needed modules, let's implement our first CNN, as shown in Figure 5 above.\n",
|
|
"\n",
|
|
"Let's implement the above network using CNTK layer API:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 9,
|
|
"metadata": {
|
|
"collapsed": true
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"def create_basic_model(input, out_dims):\n",
|
|
" with C.layers.default_options(init=C.glorot_uniform(), activation=C.relu):\n",
|
|
" net = C.layers.Convolution((5,5), 32, pad=True)(input)\n",
|
|
" net = C.layers.MaxPooling((3,3), strides=(2,2))(net)\n",
|
|
"\n",
|
|
" net = C.layers.Convolution((5,5), 32, pad=True)(net)\n",
|
|
" net = C.layers.MaxPooling((3,3), strides=(2,2))(net)\n",
|
|
"\n",
|
|
" net = C.layers.Convolution((5,5), 64, pad=True)(net)\n",
|
|
" net = C.layers.MaxPooling((3,3), strides=(2,2))(net)\n",
|
|
" \n",
|
|
" net = C.layers.Dense(64)(net)\n",
|
|
" net = C.layers.Dense(out_dims, activation=None)(net)\n",
|
|
" \n",
|
|
" return net"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"To train the above model we need two things:\n",
|
|
"* Read the training images and their corresponding labels.\n",
|
|
"* Define a cost function, compute the cost for each mini-batch and update the model weights according to the cost value.\n",
|
|
"\n",
|
|
"To read the data in CNTK, we will use CNTK readers which handle data augmentation and can fetch data in parallel.\n",
|
|
"\n",
|
|
"Example of a map text file:\n",
|
|
"\n",
|
|
" S:\\data\\CIFAR-10\\train\\00001.png\t9\n",
|
|
" S:\\data\\CIFAR-10\\train\\00002.png\t9\n",
|
|
" S:\\data\\CIFAR-10\\train\\00003.png\t4\n",
|
|
" S:\\data\\CIFAR-10\\train\\00004.png\t1\n",
|
|
" S:\\data\\CIFAR-10\\train\\00005.png\t1\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 10,
|
|
"metadata": {
|
|
"collapsed": true
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"# Determine the data path for testing\n",
|
|
"# Check for an environment variable defined in CNTK's test infrastructure\n",
|
|
"envvar = 'CNTK_EXTERNAL_TESTDATA_SOURCE_DIRECTORY'\n",
|
|
"def is_test(): return envvar in os.environ\n",
|
|
"\n",
|
|
"if is_test():\n",
|
|
" data_path = os.path.join(os.environ[envvar],'Image','CIFAR','v0','tutorial201')\n",
|
|
" data_path = os.path.normpath(data_path)\n",
|
|
"else:\n",
|
|
" data_path = os.path.join('data', 'CIFAR-10')\n",
|
|
"\n",
|
|
"# model dimensions\n",
|
|
"image_height = 32\n",
|
|
"image_width = 32\n",
|
|
"num_channels = 3\n",
|
|
"num_classes = 10\n",
|
|
"\n",
|
|
"import cntk.io.transforms as xforms \n",
|
|
"#\n",
|
|
"# Define the reader for both training and evaluation action.\n",
|
|
"#\n",
|
|
"def create_reader(map_file, mean_file, train):\n",
|
|
" print(\"Reading map file:\", map_file)\n",
|
|
" print(\"Reading mean file:\", mean_file)\n",
|
|
" \n",
|
|
" if not os.path.exists(map_file) or not os.path.exists(mean_file):\n",
|
|
" raise RuntimeError(\"This tutorials depends 201A tutorials, please run 201A first.\")\n",
|
|
"\n",
|
|
" # transformation pipeline for the features has jitter/crop only when training\n",
|
|
" transforms = []\n",
|
|
" # train uses data augmentation (translation only)\n",
|
|
" if train:\n",
|
|
" transforms += [\n",
|
|
" xforms.crop(crop_type='randomside', side_ratio=0.8) \n",
|
|
" ]\n",
|
|
" transforms += [\n",
|
|
" xforms.scale(width=image_width, height=image_height, channels=num_channels, interpolations='linear'),\n",
|
|
" xforms.mean(mean_file)\n",
|
|
" ]\n",
|
|
" # deserializer\n",
|
|
" return C.io.MinibatchSource(C.io.ImageDeserializer(map_file, C.io.StreamDefs(\n",
|
|
" features = C.io.StreamDef(field='image', transforms=transforms), # first column in map file is referred to as 'image'\n",
|
|
" labels = C.io.StreamDef(field='label', shape=num_classes) # and second as 'label'\n",
|
|
" )))"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 11,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Reading map file: c:\\Data\\CNTKTestData\\Image\\CIFAR\\v0\\tutorial201\\train_map.txt\n",
|
|
"Reading mean file: c:\\Data\\CNTKTestData\\Image\\CIFAR\\v0\\tutorial201\\CIFAR-10_mean.xml\n",
|
|
"Reading map file: c:\\Data\\CNTKTestData\\Image\\CIFAR\\v0\\tutorial201\\test_map.txt\n",
|
|
"Reading mean file: c:\\Data\\CNTKTestData\\Image\\CIFAR\\v0\\tutorial201\\CIFAR-10_mean.xml\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"# Create the train and test readers\n",
|
|
"reader_train = create_reader(os.path.join(data_path, 'train_map.txt'), \n",
|
|
" os.path.join(data_path, 'CIFAR-10_mean.xml'), True)\n",
|
|
"reader_test = create_reader(os.path.join(data_path, 'test_map.txt'), \n",
|
|
" os.path.join(data_path, 'CIFAR-10_mean.xml'), False) "
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"Now let us write the training and validation loop."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 12,
|
|
"metadata": {
|
|
"collapsed": true
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"#\n",
|
|
"# Train and evaluate the network.\n",
|
|
"#\n",
|
|
"def train_and_evaluate(reader_train, reader_test, max_epochs, model_func):\n",
|
|
" # Input variables denoting the features and label data\n",
|
|
" input_var = C.input_variable((num_channels, image_height, image_width))\n",
|
|
" label_var = C.input_variable((num_classes))\n",
|
|
"\n",
|
|
" # Normalize the input\n",
|
|
" feature_scale = 1.0 / 256.0\n",
|
|
" input_var_norm = C.element_times(feature_scale, input_var)\n",
|
|
" \n",
|
|
" # apply model to input\n",
|
|
" z = model_func(input_var_norm, out_dims=10)\n",
|
|
"\n",
|
|
" #\n",
|
|
" # Training action\n",
|
|
" #\n",
|
|
"\n",
|
|
" # loss and metric\n",
|
|
" ce = C.cross_entropy_with_softmax(z, label_var)\n",
|
|
" pe = C.classification_error(z, label_var)\n",
|
|
"\n",
|
|
" # training config\n",
|
|
" epoch_size = 50000\n",
|
|
" minibatch_size = 64\n",
|
|
"\n",
|
|
" # Set training parameters\n",
|
|
" lr_per_minibatch = C.learning_rate_schedule([0.01]*10 + [0.003]*10 + [0.001], \n",
|
|
" C.UnitType.minibatch, epoch_size)\n",
|
|
" momentum_time_constant = C.momentum_as_time_constant_schedule(-minibatch_size/np.log(0.9))\n",
|
|
" l2_reg_weight = 0.001\n",
|
|
" \n",
|
|
" # trainer object\n",
|
|
" learner = C.momentum_sgd(z.parameters, \n",
|
|
" lr = lr_per_minibatch, \n",
|
|
" momentum = momentum_time_constant, \n",
|
|
" l2_regularization_weight=l2_reg_weight)\n",
|
|
" progress_printer = C.logging.ProgressPrinter(tag='Training', num_epochs=max_epochs)\n",
|
|
" trainer = C.Trainer(z, (ce, pe), [learner], [progress_printer])\n",
|
|
"\n",
|
|
" # define mapping from reader streams to network inputs\n",
|
|
" input_map = {\n",
|
|
" input_var: reader_train.streams.features,\n",
|
|
" label_var: reader_train.streams.labels\n",
|
|
" }\n",
|
|
"\n",
|
|
" C.logging.log_number_of_parameters(z) ; print()\n",
|
|
"\n",
|
|
" # perform model training\n",
|
|
" batch_index = 0\n",
|
|
" plot_data = {'batchindex':[], 'loss':[], 'error':[]}\n",
|
|
" for epoch in range(max_epochs): # loop over epochs\n",
|
|
" sample_count = 0\n",
|
|
" while sample_count < epoch_size: # loop over minibatches in the epoch\n",
|
|
" data = reader_train.next_minibatch(min(minibatch_size, epoch_size - sample_count), \n",
|
|
" input_map=input_map) # fetch minibatch.\n",
|
|
" trainer.train_minibatch(data) # update model with it\n",
|
|
"\n",
|
|
" sample_count += data[label_var].num_samples # count samples processed so far\n",
|
|
" \n",
|
|
" # For visualization... \n",
|
|
" plot_data['batchindex'].append(batch_index)\n",
|
|
" plot_data['loss'].append(trainer.previous_minibatch_loss_average)\n",
|
|
" plot_data['error'].append(trainer.previous_minibatch_evaluation_average)\n",
|
|
" \n",
|
|
" batch_index += 1\n",
|
|
" trainer.summarize_training_progress()\n",
|
|
" \n",
|
|
" #\n",
|
|
" # Evaluation action\n",
|
|
" #\n",
|
|
" epoch_size = 10000\n",
|
|
" minibatch_size = 16\n",
|
|
"\n",
|
|
" # process minibatches and evaluate the model\n",
|
|
" metric_numer = 0\n",
|
|
" metric_denom = 0\n",
|
|
" sample_count = 0\n",
|
|
" minibatch_index = 0\n",
|
|
"\n",
|
|
" while sample_count < epoch_size:\n",
|
|
" current_minibatch = min(minibatch_size, epoch_size - sample_count)\n",
|
|
"\n",
|
|
" # Fetch next test min batch.\n",
|
|
" data = reader_test.next_minibatch(current_minibatch, input_map=input_map)\n",
|
|
"\n",
|
|
" # minibatch data to be trained with\n",
|
|
" metric_numer += trainer.test_minibatch(data) * current_minibatch\n",
|
|
" metric_denom += current_minibatch\n",
|
|
"\n",
|
|
" # Keep track of the number of samples processed so far.\n",
|
|
" sample_count += data[label_var].num_samples\n",
|
|
" minibatch_index += 1\n",
|
|
"\n",
|
|
" print(\"\")\n",
|
|
" print(\"Final Results: Minibatch[1-{}]: errs = {:0.1f}% * {}\".format(minibatch_index+1, (metric_numer*100.0)/metric_denom, metric_denom))\n",
|
|
" print(\"\")\n",
|
|
" \n",
|
|
" # Visualize training result:\n",
|
|
" window_width = 32\n",
|
|
" loss_cumsum = np.cumsum(np.insert(plot_data['loss'], 0, 0)) \n",
|
|
" error_cumsum = np.cumsum(np.insert(plot_data['error'], 0, 0)) \n",
|
|
"\n",
|
|
" # Moving average.\n",
|
|
" plot_data['batchindex'] = np.insert(plot_data['batchindex'], 0, 0)[window_width:]\n",
|
|
" plot_data['avg_loss'] = (loss_cumsum[window_width:] - loss_cumsum[:-window_width]) / window_width\n",
|
|
" plot_data['avg_error'] = (error_cumsum[window_width:] - error_cumsum[:-window_width]) / window_width\n",
|
|
" \n",
|
|
" plt.figure(1)\n",
|
|
" plt.subplot(211)\n",
|
|
" plt.plot(plot_data[\"batchindex\"], plot_data[\"avg_loss\"], 'b--')\n",
|
|
" plt.xlabel('Minibatch number')\n",
|
|
" plt.ylabel('Loss')\n",
|
|
" plt.title('Minibatch run vs. Training loss ')\n",
|
|
"\n",
|
|
" plt.show()\n",
|
|
"\n",
|
|
" plt.subplot(212)\n",
|
|
" plt.plot(plot_data[\"batchindex\"], plot_data[\"avg_error\"], 'r--')\n",
|
|
" plt.xlabel('Minibatch number')\n",
|
|
" plt.ylabel('Label Prediction Error')\n",
|
|
" plt.title('Minibatch run vs. Label Prediction Error ')\n",
|
|
" plt.show()\n",
|
|
" \n",
|
|
" return C.softmax(z)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 13,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Training 116906 parameters in 10 parameter tensors.\n",
|
|
"\n",
|
|
"Learning rate per minibatch: 0.01\n",
|
|
"Momentum per sample: 0.9983550962823424\n",
|
|
"Finished Epoch[1 of 5]: [Training] loss = 2.127933 * 50000, metric = 77.96% * 50000 15.851s (3154.4 samples/s);\n",
|
|
"Finished Epoch[2 of 5]: [Training] loss = 1.768975 * 50000, metric = 64.90% * 50000 11.970s (4177.1 samples/s);\n",
|
|
"Finished Epoch[3 of 5]: [Training] loss = 1.589650 * 50000, metric = 58.28% * 50000 11.958s (4181.3 samples/s);\n",
|
|
"Finished Epoch[4 of 5]: [Training] loss = 1.495402 * 50000, metric = 54.32% * 50000 11.960s (4180.6 samples/s);\n",
|
|
"Finished Epoch[5 of 5]: [Training] loss = 1.421285 * 50000, metric = 51.41% * 50000 11.965s (4178.9 samples/s);\n",
|
|
"\n",
|
|
"Final Results: Minibatch[1-626]: errs = 46.4% * 10000\n",
|
|
"\n"
|
|
]
|
|
},
|
|
{
|
|
"data": {
|
|
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYwAAACfCAYAAADqDO7LAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztnXmYVNW1t98fIIgREIegOKPJNY7g54DRKEJMFCMiRhNn\njVOMRqM316i50cQhMdcYBzRGcUjEgag4ICpKEJUhOAGCAgYFEVAmZZ4b1vfHOpWqbqq6q5uqrqp2\nvc9TzzlnT2ed3V1n1d5r7b1kZgRBEARBXTQrtQBBEARBZRAKIwiCIMiLUBhBEARBXoTCCIIgCPIi\nFEYQBEGQF6EwgiAIgrwIhfEVRdI9kn5d37KSjpA0s7jS/ee+0yV1b4x7NQWSv83EQpdtgBwjJJ1Z\njLaD0tKi1AIEhUXSJ8C2QEcz+zIjfRywH7CLmX1qZhfl22aWsg1avCNpZ2A60MLM1jekjaaCpMOA\nl/C+bAZsBiwDlKTtaWaz6tOmmb0O7FPoskGQIkYYTQ/DX8qnpBIk7Q20poEv+gKSehmq6DeSmhf7\nHhuDmY00szZm1hbYC++Xdqm0mspCCSURNggSQmE0TfoDZ2VcnwX8PbOApIckXZ+cHyFppqQrJM2V\nNFvS2dnKppN0taT5kqZJOjUjo6eksZIWS5oh6bqMeq8nx0WSlkg6OKlzvqRJSdr7kjpn1Oki6T1J\nCyU9LqlltgeWdJakkZL+LGk+cJ2k6yT1zyizs6T1kpol18MlXZ/UWyJpiKQtc7Q/SVLPjOvmkuZJ\n6iyplaT+khYkcr4paZts7dRBNYWQTO1cL2k0PvrYUdK5GX01VdK5GeV7SJqecT1T0uWSJiRyPSpp\nk/qWTfKvlvR5Uu68pB93qvOBnGslfSJpjqQHJbVJ8lon90n125hU/yfPOT15zo8kndyA/gwKTCiM\npskYoI2k/0pejj8CHqH2X/bbAm2AjsB5wN2S2tVSdsuk7NnAfZK+keQtA84ws3bAscBPJfVK8g5P\njm2TX9FvSjoJuBY4Pfm13Qv4IuNeJwHfA3bFp9TOruUZDgY+AjoANyVpNUdVNa9PwRXqNkAr4Jc5\n2n4MODXj+mhgvpmNT+q3BbbH++WnwMpa5KwPp+PP3BaYDcwBjkn66nygbzKCTFHz+U4CegCdgAOA\nM+pbVtIPgIuBI4BvAt2z1M3F+Xi/HQ7shvfP7UneOfjIt2OS/jNgVaJQbgV6JM95KDAhz/sFRSQU\nRtMlNco4CpgMfFZH+TXADWa2zsxewl/8/5WjrAG/MbO1ZvYG8AJwMoCZvWFmHyTn7wMD8BdNJpmK\n61zg/8xsbFJnmpllGtXvMLO5ZrYIeB7IHH3UZLaZ/cXM1pvZ6jqeN8VDZvZxUv6JWtp/HOgladPk\n+pQkDWAtsBXwTXPGmdmyPO9fFw+a2b+Tv8s6M3vBzGYAmNlrwDDgO7XUv83M5pvZQmAwtfdfrrIn\nAQ8kcqwEflcP+U8F/pTYzZYD15BWvGuBrUn321gzW5HkrQf2kdQq+ftPqcc9gyIRCqPp8gj+xTwb\neDiP8l/UMESvADbPUXahma3KuJ6B/0pE0sGSXk2maxYBF+IvhVzsCHxcS/7cPGUCaIj31px82jez\nj4FJwHGSWuMjoceS7P7Ay8AASbMk3azC2VCqPZOkHyRTN19IWoj/IKitf+vTf7nKdqwhx0zyt0N1\nxP8/UswAWiVTdn8D/gk8kUx1/V5SMzNbiivkS4A5kgZljGCDEhIKo4liZp/ixu9jgKcL3Hz75KWZ\nYifSI5hHgWeB7c1sC+Be0i+XbNMYM/GpikJQs/3luPdRiu02sv0BuBI+HvjAzKYBmFmVmd1gZnsB\n3waOAwrlVvqfZ0pGN0/i023bmFl7YCjFdyL4HNgh43on8p+S+gzYOeN6Z2B1MpJZa2bXm9mewGFA\nH+A0ADN72cyOwqc/P8b/j4ISEwqjafMToHsyjVBIBPxO0iaSvoPbKp5I8jbHRyBrJR1E9Xn/+fhU\nQ6aCuB/4paT9ASTtJmnHAsk5Hjhc0o6JPeaqjWxvAG5PuYj06AJJ3STtndiLluFTLfV1G87npd8K\n2ARYAFhiW+hRz/s0hCeAcyV9U9JmwP/Wo+7jwBWJw0Eb4EaSvpN0pKS9JImMfpO0bTKSag1U4Yp/\nXSEfKGgYoTCaHv/55Wdm01O2gZp59WknC58DC/Ffj/2BC81sapL3M+AGSYvxF8s/MuRZif86HiXp\nS0kHmdlTSdpjkpYAz+AG0PrKu+EDmP0zuf8E4G3cBlKtSD3bmwP8C+hKxnPhv4KfAhYDHwDD8X5J\nLXr8Sz7N15VmZouBy/ER3Bf4L/Kaz1RXm/Uua2aDgXuAN4APgZFJVi47UWZb/fC+GoE7JCwGfpHk\ndcRHv4uBicAruDJpDvwP/v81HzgEN7oHJUbFDKAkaQd8/rwD/ourn5ndmaPsgcBo4EdmVugplCAI\nCkTilfWumbUqtSxB41LsEUYVcEUyt3sIcLGkPWoWSobyN+OGwyAIygxJvZMpyC3x7+qzpZYpaHyK\nqjDMbE7ip07iZjgZ91Wvyc/xIf28YsoTBEGDuRi3nfwb96C6pLTiBKWg0faSkrQL7tf9Zo30jkBv\nMzsyMZIGQVBmJB5LwVecRlEYkjbHRxCXZVnQdDvwq8ziOdoo9T5IQRAEFYmZFcT1uuheUpJa4Mqi\nv5k9l6XIAfiCp+nAD/EtKXplKYeZlf3nuuuuK7kMIWfIWakyhpyF/xSSxhhhPAhMMrM7smWaWafU\nuaSHgOfNbFAjyBUEQRDUg6IqDEmH4is3J8rjMRi+l8zOgJnZfTWqxLRTEARBmVJUhWFmo/BFOPmW\n/0kRxWkUunXrVmoR8iLkLCyVIGclyAghZzlT1IV7hUSSVYqsQRAE5YIkrFKM3kEQBEHTIBRGEARB\nkBeNtnCvULzzDvTqBV26wLe+BbvvDnvsAXvtBds0JChmEARBkBcVZ8NYvx5mzYJ334V//xumToXJ\nk2HnneGxxzasV1XlxxYVpxqDIAg2nkLaMCpOYdSX++6Dyy+H006DE06A73wHNq8t5lgQBEETIoze\n9eCCC+DDD6FTJ7jhBthuO/je92DSpFJLFgRBUFk0+RFGTVasgIEDoUcP6NixAIIFQRCUMRUzJZVP\nACVJp5LefHApcJGZTczSVqzDCIIgqCeVNCWVTwClacDhZrYfHu+3X5FlysqaNfDjH8Pbb5fi7kEQ\nBOVPyQMomdkY81jFAGNq5jcWLVvCwQdDz54wenQpJAiCIChvSh5AqQbnAS81hjzZuPxyX9dxxhlu\nKA9X3CAIgjTlEEApVeZI4BzgsFzt/Pa3v/3Pebdu3Yqy+ddxx8Ett8BDD8H55xe8+SAIgqLy2muv\n8dprrxWl7aJ7SSUBlAYDL+WKiSFpX2AgcLSZfZyjTKMZvd98E7p2dY+q1q0b5ZZBEARFoWK8pAAk\nPQwsMLMrcuTvBAwDzjCzMbW006heUqNGwaGHNtrtgiAIikLFKIwkgNIbwEQ8ONIGAZQk9QP6ADPw\neN5rzeygLG2FW20QBEE9qRiFUUhKqTDeftu9qPbbryS3D4IgaDCVtA6jSfCvf8Gf/1xqKYIgCEpL\njDDy4OOP3d120CD3ogqCIKgUYkqqkTGD00+H996DCROgWYzLgiCoEGJKqpGR4OGHYc4cuPtuT+vU\nydPnzCmtbEEQBI1FKIw8ad7cY2scdpiPOGbP9vQirB0MgiAoS2JKaiMYNcoVyMqVsOmmpZYmCIJg\nQ8KGUUasWeMut0EQBOVI2DDKiJSymDIFHn/cFUiKTz4piUhBEARFoagKQ9IOkl6V9IGkiZIuzVHu\nTklTJY2X1LmYMhWLkSPh1FOhbVuforr1Vth1V3ipZHvvBkEQFJaSB1CSdAywm5l9A7gQ+GuRZSoK\nP/mJK4zVq+HKK2H//T29Z09YsKC0sgVBEBSCkgdQAo7Hw7hiZm8C7SR1KKZcxaBZM3j0UfjgA7jx\nRjjySFi71vNGj4ZVq3zR39q17mUVBEFQaZRDAKXtgZkZ17OTtLmNIliB2XPP9HmLFmnlsH49DB6c\ntnmsXh3G8iAIKouyCaCUD40RQKlYNGsGzz+f3lqkZUsYONCVyvHHl1a2IAiaDk06gJKkvwLDzewf\nyfUU4Agzm1ujXFm61W4Md9wBTz0FnTtD374+GlFBnN+CIAicSnOrfRCYlCvaHjAIOBNAUldgUU1l\n0VTZZx/3rnrgAZg1y0chp50GS5d6fsr2EQRBUA6UPIBSUu4u4GhgOXCOmY3N0laTG2HMnw9f/7or\nhZYt05sa9ugBQ4f69R//6F5XQRAEDSFWejch1q6FTTbx86oqH3G0bw977+3pZnDVVfCHP5RWziAI\nKpNQGF8hWrSAAw6AMTmjnQdBEOSm0mwYwUZQVZVWFpMnw7p18OSTtddZt674cgVB8NUjFEYF0bu3\njzhOPhmWL/e0Zcvg6afTZS67zMv88Y+lkTEIgqZLKIwK4pZb/HjRRbDZZm7vaNMGTjwRxo9376o7\n7/QynTqVTs4gCJomedkwJO0GzDKz1ZK6AfsCD5vZoiLLlynDV9KGURtVVW4Y79DBQ8jeeCPccw9c\nfnmpJQuCoFxodKO3pPHAAcAuwIvAc8BeZtazEELkQyiM7Jj5luorVrh3VSbXXguXXOKuu0EQfDUp\nhcIYa2b7S/ofYJWZ9ZU0zsy6FEKIfAiFUX9Sq8a//nWYmyyFXLwY2rUrnUxBEDQupfCSWivpFOAs\nfJsPgE0KIUBQPF55xY/z5vlx7lzYYgsfjUybll5RHgRBkA/5Koxz8HgWN5nZdEm7Av3rqiTpAUlz\nJU3Ikd9W0qAkcNJESWfnLXlQJ0cd5VNWZvDRR7Dttp6+2WZuKL80azirIAiC7OSlMMxskpldamaP\nS2oPtDGzfBw3HwK+X0v+xcAHZtYZOBK4NdmsMCgwbdr48bbb/Hj11fC3v7mdIwiCIB/yUhiSXktG\nA1sCY4F+kv5cVz0zGwksrK0IkLzKaAN8YWZV+cgU1I8OHXyk8Ytf+PUxx/jxhhv8uHQpPPhgBHcK\ngiA3+U5JtTOzJUAf3J32YOC7Bbj/XcCekj4D3gMuK0CbQR60aeMrwp96ypXEhRfCuefC3XfDnDlu\nMJ8/v9RSBkFQTuQ7/dNC0nbAycCvC3j/7wPjzKx7stZjqKR9cwVZquQASuVIs2Zuy1i/Hh5/3NN+\n8xvYfXc/HzrU45QHQVA5lDyAkqSTgN8Ao8zsIkmdgFvM7MQ86u4MPG9m+2bJGwz8wcxGJdfDgF+Z\n2TtZyoZbbRGp6W57zz2+vciUKbDppvm1cdNNcPHF7okVBEF50OhutWb2pJnta2YXJdfT8lEWCUo+\n2ZhBMrUlqQPwTWBanu0GBaTm2oxjj4UZM2DJkrrrTp/uiwcHD4a//7048gVBUHryNXrvIOkZSfOS\nz0BJO+RR7zFgNPBNSZ9KOkfShZIuSIrcCHw7cbsdClxpZl829GGCwrHTTvDFF9lXia9ZA3/5i5+b\n+b5V3/42nHcevPVW48oZBEHjke+U1FDgMdJrL04HTjOzo4ooW00ZYkqqRIwc6Tvgdu3q10OGuJeV\nGXz+OXTs6C663bu7ohk3zuOUB0FQekqx0nsbM3vIzKqSz9+AbQohQFD+PPIIHH00nHmmrxD/frKy\n5vTT4ZlnfBRy1lmw446e/uMfw4IFvrtuTR0/YABMyLqMMwiCciffEcYwfBFe4kvDKXjs7R5FlK2m\nDDHCKBFLl0Lbtn6+bBl87Wvpfar++7/hjTfSU1HjxsHWW/tIA2DRorR9ZNQoOOww2GMPDwaVL1Om\neJ0gCOpPIUcY+brV/gToC9yGL7YbDZxdCAGC8qdNG7jrLjj0UFcW4HaM5s1dcWTq8S5dfBsSgIED\n08pi1Sp4910/r09EwAUL4Fvf8v2wtokxbRCUlAbH9Jb0CzO7vcDy1Ha/GGFUIK+/DqnlMrNnu73D\nDFavhhdfhFatPK9jR1c2NbnpJvjf/3VPrF12aSypg6DpUC4xva8ohABB0+bTT/3YrVt680MJWrf2\nRYO33w7vvedGc3BlMmKEKxczVxZ33VVdWfTpA9dd50okCILGY2NGGDPNbMcCy1Pb/WKE0YQ44ww3\npq9f7wqjSxe44gr44Q/dRRc8hvmzz7oNZfPNfVqrVStfSLhmjdtKYvuSIKidchlhxNs7aDD9+/sI\nQnIX3C23hL/+1Y3kV14JJ58Mv/2tK4nNN/dRR+vW8MADMHGiu/VOmQLXXw+zZmW/x5o1jfpIQdDk\nqXWEIWkp2RWDgNZm1mhbkccIo2mzaJG77T73XNoDK5PJk2HPPf089W/w7LNwwgl+XlXlRvgUL77o\nq9VHjHDPrHx57z1XWC++WL29IKhUGm2EYWZtzKxtlk+bfJRFXQGUkjLdJI2T9L6k4Q15iKDy2WIL\nGDQou7KAtKfUgw+m03r1cvsGwNixbueQ4LHHXFmA78C7YAGsXFn7/ceO9fUjnTt7pMJJkzb6kYKg\nybExU1L5UGsAJUntgLuBH5jZ3sBJRZYnqGC22QbOOSd93ayZb3a4ZInbQGbM8PR99vFFg++/74rg\n5ps9ymBtTJ7sU1hffgkHHQQff5y/XPPm+VqUIGjqFHVKycxGJrvV5uJUYKCZzU7KLyimPEHTJBVN\ncMECVyLt27vSSPHDH8Ktt/ro45134IAD3LA+YgQMH+4v/NNP9z2x2reH734Xli/P//4dOvix5rRY\nEDQ1Sh0O9ZvAJslU1ObAnWZWZ6zwIMjGVltlT+/aFY4/3u0jqRXjo0f7KOSqq9yYDnBBsiXmTTdt\n2IaZG+DN0osX337b14+kWLcuFEbQtCm1wmgB7A90B74G/EvSv8zso2yFI4BS0FCefTZ9Pm+e2ym6\ndHGF0amTT1/loqoKNtnEzz/5JJ0+cCD07OmjlVGjqteZMAGefBLOPz+9TUo+rFkDLVvmV3b6dFde\n2XYUDr66lDyA0kbdoPYASr8CNjWz3yXX9wMvmdnALGXDSyooCin33kxWrICHH3a7yX77eeTBt9+G\n227zuOhjxrjNpFcvV0A77+ztfPGFrw/ZcktYuBAuuwwuucSVTl37YZn5lNrNN7u3Vt++3t7DD8ON\nN25YPiVzfC2C2iiklxRmVtQPsAswMUfeHngcjObAZsBEYM8cZS0IGou1a838VWzWr5+nvfWW2eef\n567zzjtme+3l5+++63U//dRs66393Mxs/vzc9WfO9HKnnZa+92ef+XHmzA3LX3ON533yScOeMfhq\nkLw7C/I+L+oIIwmg1A3YCpgLXAe0TB7gvqTML4FzgHVAPzPrm6MtK6asQVCTtm19lXm+xuzMqavM\nf9XUSGDSJF9Lktrxd8ECt5+kbCIvvuhbpbzyCvz5z74TcMpID75AcfvtN7zf+PE+CgqCbBRyhFH0\nKalCEQojqASyTRNNmLDhC/3dd31B4R13uJ0D4Oc/961S7r7bFzL27w8XXujH886Dzz6D7bZrmFwz\nZvheXqnNHmsybZqH123VCu680wNj5VoTE1QWoTCCoExZtcrtH1tuWT196VIfDQwZ4m7AQ4bAn/7k\n8UOuvtoVzHPPedlcL3VwhbJ6td8jl1fYkiXw6qu+Cn70aLepHHmke4H16eMjp5qG+AsugH790tf/\n+Idvz1IbVVVu1znkkNrLBaWlXPaSCoKgBptuuqGyAFcSm27qGyr26JHe/2qffVx5vPyyG7xrUxbg\niwo328yVwOrVnrZ8uRvGFy1yRdGunbcFvktwv36+++9228G117qBPpM5c7xMz56ukAB+9CNfp1Ib\n553nHmL12QDSDKZOzb98UGYUyhhS7A9h9A6aEPPnm40c6edTp7rxujaDeIqUYbx583Raqn7mZ8cd\n/diunR932skN+RMm+HVVVfV2wezNN/189WqzhQs9bdEiT5swwWzo0Op1nnzSyxx8sBv5d9vN7NVX\nzT74ILf8w4Z5nddfr/tZg8JApRi9C0lMSQWBs3atG+GbZcwPpOwNzZq5obxzZzeqr1/v6zXatfP9\nuMBtGaNH+/qT2ujRAy6/HH7wg+q2mZkzfertG9/wLVG23x5237163dWrs68nee45H2WBx4E/5BC3\n07z1lttodt/dRywDBsC992aX68Yb4Te/CXfifAkbRhAE1Vi/vroCqQ3J15U8+qgfe/fObq+48UZf\nS5LaVuWBB3zF/NZbe/6XX/pWKpkx3+fP97UrP/2pG++bNfMV8C1apOtMm+aeYql1t5axDuaaa+D3\nv/fzOXNcOaW8xFJlU8+5fv3GG+Yz791Uqah1GIX6EFNSQVAQJkwwmz7d15fUnN7K5JVXPD819bRy\npdmMGelpr/Xrs9c79NB0md69zZ55Jn29cmW63Pz5nvbAA358/nlfUwJmH39stscefj5xYrrOrbdu\nOH23enW6/UzWrjW7//7ccs6ala6zbp3ZnDnpvFWr/LmbAhRwSqrkiiBvQUNhBEHBWL/ev/2bbJK7\nTFWVl3niCbOTTqpe97PPctdbtiy94BD8xT18ePYX989/bnb44WZ7771h3rx5Xv/ww81+9jM/HzDA\nbOnS6uVefz27wvjudz1tyRKzrl39mEnfvmmFeMMNZj16pPOefnpDZVUXqT695x7LaicqFaEwgiDY\naF54YcOXb03WrWt4+8uWmc2eXXe5tWvN1qzJnnfvvWbnnptWCH36pPOeesrTzj/f669ebXbCCWYv\nv+yr8sHsiCO8bKr+l1+m66ecBV591Y8dO3r6F1+kldC992aXq6pqw75Jjb6WLvXV/cOG1f3sjUHF\nKAzgAXyF94Q6yh0IrAX61FKmgF0YBEElsGaN/3KvqnLFksnkyf4G69IlnXbIIZ528sl+nD7d01MK\nI+Wd9f776TpXXeV5Kc+w1Ev/kkvMttoqu1ydOpl16GA2ZIh7h5n5yOLYY/38xBNtgxFPig8/9FFb\nQ5k7N+29lg+VpDAOAzrXpjDwtSDDgMGhMIIgqA9gdsst6etevTxt3rzq5ZYt8329Vq40a9XKy6xY\n4XmrVvmIJNUemL3xhts0RoxwJTBgQPUptVS5Ll38OHq0Hx95xPOHD7ecdp4WLXIrk3yfuT71C6kw\nSrpbbZJ/GbAGH2UMNrOnc5SzYssaBEFls3y5e21tu232/MWLPRwwZHfLNXP33vbtfYGkZXhlZe4p\nNnEiHH647w/WsSPsuKN7dE2enN6V+LTTfGfjbt3cRbh7d3dBfv11OOII3zvsmGOq3z8Viz6XWzLU\nf5fiJrPSW1JHoLeZ3QM0cee2IAiKzde+lltZgK9HGTbM3XuzIflLPRXSN/Vybt/elcUVV/gK+H32\ncZfj7bbztt57z1/gmVvYP/qorzFZsQJ+9zv/AOy2mx979kyXXbfO18ksW+bXqRX/Q4a4G3KKNWs8\nb9Wq/Pqj0JQ6gNLtwK8yrmtVGhFAKQiCjaV79/qVz/wlf9tt/slMa98+d90RI1zJ9O7tCuLgg10h\nffklbL65L6r86CMfqUyZ4jHot90W9trLF03efru38/TTvufYSSf5OpVWrbztpUurKx5o2gGUUrpT\nwNbAcuACMxuUpWxMSQVBUDLmzPERxf33w7nn1r/+4sWuJDK3yk+NYI49Fl54oboi6tsXLr0UXnst\nvcixRw/45z/9fKutXPHU9VqstCkpkWPkYGadks+uwFPAz7IpiyAIglLz6ae+lUlDlAX4dFjNuCoH\nHujH88/3nX8zufhi3zTyiCPgmWd8Z+PBg9P5KcUxZkzD5GkIJQ+glFH2QcLoHQRBkDeST4nlssl4\nmdhLKgiC4CvP8uXuTZWK9JiNUBhBEARBXlSaDSMIgiBoAoTCCIIgCPIiFEYQBEGQF6EwgiAIgrwI\nhREEQRDkRSiMIAiCIC9CYQRBEAR5UVSFIekBSXMlTciRf6qk95LPSEn7FFOexqBYm34VmpCzsFSC\nnJUgI4Sc5UyxRxgPAd+vJX8acLiZ7QfcCPQrsjxFp1L+iULOwlIJclaCjBByljNF3d7czEYmu9Xm\nys/cNmsMsH0x5QmCIAgaTjnZMM4DXiq1EEEQBEF2Sh6iNSlzJHAXcJiZLcxRJjaSCoIgaACF2kuq\n1BH3kLQvcB9wdC5lAYV74CAIgqBhlDSAkqSdgIHAGWb2cSPIEgRBEDSQkgZQktQP6APMwJXKWjM7\nqGgCBUEQBA2mYuJhBEEQBKWlnLykciLpaElTJP1b0q9KLMsnyULDcZLeStLaS3pF0oeSXpbULqP8\n1ZKmSpos6XtFlGuDRZINkUvS/pImJH19eyPJeZ2kWZLGJp+jy0DOHSS9KukDSRMlXZqkl02fZpHx\n50l6WfWnpFaS3ky+MxMlXZekl01f1iFnWfVnxj2aJfIMSq6L359mVtYfXKl9BOwMbAKMB/YooTzT\ngPY10v4IXJmc/wq4OTnfExiHOxfskjyHiiTXYUBnYMLGyAW8CRyYnL8IfL8R5LwOuCJL2W+VUM5t\ngc7J+ebAh8Ae5dSntchYjv25WXJsjq+5Oqic+rIOOcuuP5N2LwceAQYl10Xvz0oYYRwETDWzGWa2\nFhgAHF9CecSGI7Pjgb8n538HeifnvYABZlZlZp8AU/HnKThmNhKo6WVWL7kkbQu0MbO3k3IPZ9Qp\nppyQ3THi+BLKOcfMxifny4DJwA6UUZ/mkDG1+LXc+nNFctoKf3EZZdSXdcgJZdafknYAegL315Cn\nqP1ZCQpje2BmxvUsSrsi3IChkt6WdF6S1sHM5oJ/iYGvJ+k1ZZ9N48r+9XrKtT3evykas68vkTRe\n0v0ZQ+mykFPSLvioaAz1/1s3iqwZMr6ZJJVVfybTJ+OAOcDQ5CVVdn2ZQ04os/4EbgP+h7RCg0bo\nz0pQGOXGoWa2P67dL5b0Har/0chyXS6Uq1x/ATqZWWf8i3prieX5D5I2B54CLkt+xZfd3zqLjGXX\nn2a23sy64KO0gyTtRRn2ZRY596TM+lPSscDcZHRZ2/q0gvdnJSiM2cBOGdc7JGklwcw+T47zgWfx\nKaa5kjoAJMO8eUnx2cCOGdUbW/b6ylUSec1sviWTqPgGlKlpu5LKKakF/iLub2bPJcll1afZZCzX\n/kxkWwKv/UJAAAAEtklEQVS8BhxNmfVlLjnLsD8PBXpJmgY8DnSX1B+YU+z+rASF8Tawu6SdJbUE\nfgwMKoUgkjZLfs0h6WvA94CJiTxnJ8XOAlIvl0HAjyW1lLQrsDvwVjFFpPovjnrJlQxjF0s6SJKA\nMzPqFE3O5J87RR/g/TKR80FgkpndkZFWbn26gYzl1p+Stk5N40hqDRyF21vKqi9zyDml3PrTzK4x\ns53MrBP+PnzVzM4AnqfY/Vloy30xPvivkQ9xY81VJZRjV9xLaxyuKK5K0rcE/pnI+AqwRUadq3Gv\nhMnA94oo22PAZ8Bq4FPgHKB9feUC/l/ybFOBOxpJzoeBCUnfPovPxZZazkOBdRl/77HJ/2G9/9bF\nkrUWGcuqP4F9EtnGJ3L9uqHfmxLJWVb9WUPmI0h7SRW9P2PhXhAEQZAXlTAlFQRBEJQBoTCCIAiC\nvAiFEQRBEORFKIwgCIIgL0JhBEEQBHkRCiMIgiDIi1AYQdkhab2khzOum0uan7GN83GSrqyjje0k\nPZGcnyWpbz1luDqPMg9J6lOfdguJpOGS9i/V/YOvHqEwgnJkObC3pFbJ9VFkbJ5mZs+b2f/V1oCZ\nfW5mJ2cm1VOGa+pZvqKQ1LzUMgSVRyiMoFx5ETg2OT8F3zMHqD5iSH7l3yFplKSPUr/4k61kJma0\nt1Pyi/xDSddmtPVMsvPwxNTuw5L+ALSWB6fpn6SdqXTgrL9ntHtEzXtnksgxSdJ9kt6XNCSlCDNH\nCJK2kjQ94/mekQfDmSbpYkmXJ/KMlrRFxi3OTGSaIOnApP5m8kBVYyS9K+m4jHafkzQMXxEcBPUi\nFEZQjhge9+SU5OW6L+ltuzPLpNjWzA4FjsODyGQrcyBwArAfcFLGVM45ZnZgkn+ZpPZmdjWwwsz2\nN7Mz5DuWXgN0M9/J9LI87p3J7kBfM9sbWAycWMtzp9gLj01wEHATsMx8l+Qx+J4/KVonMl2M7ysF\n8GtgmJl1BboDf0r2RgLoAvQxsyNzyBAEOQmFEZQlZvY+Hh3sFOAFat/G+dmkzmTSMQBqMtTMFpnZ\nKuBpPPIfwC8kjcdfxDsA30jSM+/XHXjSzBYm91lUz3tPN7PUaOfd5LnqYriZrTCzBcAiYHCSPrFG\n/ceT+48A2khqi2+KeZU8rsNrQEvSOz4PNbPFedw/CDagRakFCIJaGATcAnQDtq6l3OqM81yKZYPY\nC5KOwJXBwWa2WtJwYNN6ypjPvTPLrMu4RxXpH20175tZxzKu11P9e5stpoSAE81samaGpK64fSgI\nGkSMMIJyJPXifRD4nZl90IC6NTlK0hbJ1ExvYBTQDliYKIs9gK4Z5ddkGIZfxaextgSQ1L6e986V\n/glwQHJ+Uo4ydfGjRKbDgMVmthR4Gbj0PzeXOjew7SCoRiiMoBwxADObbWZ35VO2lusUb+FTUePx\n6aWxwBBgE0kfAL8H/pVR/j5goqT+ZjYpyX89meZJRVzL99650v8EXCTpXXxr6lzU1u4qSWPxqHA/\nSdJvwJ9rgqT3getraTsI8ia2Nw+CIAjyIkYYQRAEQV6EwgiCIAjyIhRGEARBkBehMIIgCIK8CIUR\nBEEQ5EUojCAIgiAvQmEEQRAEefH/AaGWf7My2dlRAAAAAElFTkSuQmCC\n",
|
|
"text/plain": [
|
|
"<matplotlib.figure.Figure at 0x2ec6cb8f908>"
|
|
]
|
|
},
|
|
"metadata": {},
|
|
"output_type": "display_data"
|
|
},
|
|
{
|
|
"data": {
|
|
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYwAAACfCAYAAADqDO7LAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztnXecFeX1/98fERELYC/YEmNDYyyxJDbEht1oLGgsmHzt\nMbaomIKx/NRYsWtiwd4bauyK2AsoSBNFUBBRlKYgbc/vjzOXO3v37t3Z5e7du3Ler9e8ZuaZZ57n\nzLN358x5yjkyM4IgCIKgIRZpaQGCIAiC1kEojCAIgiAToTCCIAiCTITCCIIgCDIRCiMIgiDIRCiM\nIAiCIBOhMKoMSTdI+ltj80raQdIXzSvd/Ho/k9StEnW1NiTVSPp5pe9dUNJ1N+Y3WKSc6ZLWKqds\nQfUQCqNCSBoj6UdJyxakD0r+WdcAMLPjzezCLGUWydukRTWS1kxkiN8DC9weC7Kwqd57Jb0iaaak\naZK+lvSwpJUWoK566876G5T0sqSjaxVitrSZjSmjXLm6xkiakTz/9GR/dbnrCUoTL4jKYcBnQI9c\ngqSNgPYs2EumHCiRQc1ekdSmuesoE039myxIG5a614ATzKwDsC7QCbiyaCFNU3TN/rdfQAzY08w6\nJEqpg5mdXCxjsd9YY393reh3WlFCYVSWO4EjU+dHAn3TGSTdJum85HgHSV9IOk3SREnjJR1VLG8+\nSb0kfSNptKRDUxf2kDRQ0lRJYyX1Tt3XP9lPSb7ctkru+T9Jw5K0jyRtkrpnU0kfSpos6V5JixV7\nYElHSnpN0hWSvgF6S+ot6c5Unlpf9MmX63nJfdMkPVNomaXuHSZpj9R5m+QLfBNJ7STdKWlSIufb\nklYoVk5WJG0h6Y2kvPGSrpG0aEG2PSV9msjx74L7j05k/lbS/3KWZdbqAcxsCvAwsFFS5m2Srpf0\nlKTpQFdJi0m6LPlbT0iut0vJ8VdJX0oaJ6knKQVZ+LuStG9iCU+VNErSrpIuALYDrk1/7at211YH\nSXck7fCZUt1cye9igKRLJX2XtFf3LM9fJ7H2b2wS/hsrliZJf5dbK19Jul1Sh6SM3G/waEljgRez\n/lEWJkJhVJa3gKUlrZe8HA8G7qL0193KwNLAqsCfgOskdSyRd9kk71HAzZLWSa59DxxuZh2BPYHj\nJO2TXNs+2XdIvtzelnQg8E/gD8lX7T7At6m6DgR2BX4G/Cqprz62Aj4BVgJyXR2FX/CF5z1whboC\n0A44o56y7wEOTZ13B74xsw+S+zsAnfF2OQ6YWULOLMwDTknK+w3QDTihIM9+wGbJtq+SbhtJ+wJn\nJ9dXAAYA9zZWAEnLAwcAA1PJPYDzzWxp4HXgEuAXwMbJvjP+9yR5MZ8G7ASsA+xcoq4t8Y+a05Pf\nzvbAGDP7eyL/SQVf++m/47X4b3ctoCtwRKKccmwJDAeWAy4FbmlMOxSQ+42tSP43VpjWEzgC2AH4\neSLbtQXlbA+sD+y2ALL8ZAmFUXlyVsYu+D/Llw3kn42/COaZ2f/wF/969eQ14B9mNsfMXgWeAg4C\nMLNXzWxocvwRcB/+j5Mmrbj+CPzbzAYm94w2s/Sgeh8zm5h87fYD0tZHIePN7HozqzGzWQ08b47b\nzOzTJP8DJcq/F9hH0uLJeQ/yL+E5+MtoXXMGmdn3GesvipkNNLN3kvI+B26mbjtebGZTzWwccBX5\nbshjgYvM7GMzqwEuBjaRtHrG6q+R9B0wCP/dnJ669riZvZXIOAv4P+DURI4fkrpychyIt+9wM5sJ\nnFuizqOBW8zspaTsCWb2cYn8gvndYgcDZ5vZDDMbC1wOHJ7KO9bMbjV3aNcXWFnSiiXKfiyxRiYn\n+z+mrhX7jRWmHQpcYWZjzWwG0As4RPkuPAN6m9nMRvxOFyoKTemg+bkLeBX/Mr8jQ/5vk5dLjhnA\nUvXknWxmP6bOx+LWBvJupovwbozFku3BEvWuDnxa4vrEAplWKZG3KbO3vioov+gzm9mnkoYBe0t6\nEreE/plcvhNYDbgvscruAv5mZvOaIA8AicV2BfBrfPxpUeD9gmzjUsfz/wbAmkAfSZfnisNfUp3J\n1kZ/NrNb67k2//6k220J4H1p/jfAIuQ/CFYF3iuQsT4rd3X8w6OxLI+3zecF9XROnc//G5vZTLmw\nSwFf11Pmvmb2cj3XirVfYdqqiQxpeRbFLd8c4wjqJSyMCpN8lX4G7A48Uubil5HUPnW+BnkL5m7g\nMaCzmXUCbiL/kig2wPsFsHaZ5Cos/wf8hZajlLLJwn341+O+wFAzGw1gZnPN7Hwz2xD4LbA33iWx\nINyAW4ZrJ+34N+q+bNMWw5rk/wZfAMea2bLJtoyZLZWzDBaQdBtPwpXshqm6OiVdSgATishY3yB/\nqd9BqYkBk3ALb82CesaXuKchGpoU0FDal0XkmUPtj5+WnoBS1YTCaBmOBrol3QHlRMC/JLWVtB0+\nVvFAcm0p3AKZk/RLp/v9vwFqqP1i+C9whqTNACSt3Yiuk4b4ANhe0urJl//ZC1jeffh4yvH4mAYA\nkrpK2ijpcvgefznUFC+iDgIWTwbOc5vwfu9pZjZD0vpJnYX8VVKnpL1OTuQDuBE4R1KXRL6Okn7f\n+MctTdLF8x/gqsTaQFJnSbsmWR4AjpK0gaQlyFtkxbgF6Clpx2TQeFVJuS7RifhYQDEZapJ6LpS0\nlKQ1gVNxq6+luBc4VdJakpbCxzXuS1nw1T5TrMUJhVE50vPcP8uNDRRea0w5RZgATMa/pO7Ev2ZH\nJddOAM6XNBX4O3B/Sp6Z+D/P60nf8JZm9lCSdo+kacCj+EBvY+Wt+wBmLyT1DwbexcdAamVpZHlf\nAW8CW5N6LnwSwEPAVGAo8DLJC0u+OO36UsUC0/Ev9ZnJfkd83OCwpE1uIq8M0vc9jndTDUye7dZE\nzsfwsYT7JE3Bn797wb2l5GnMtbPwAd+3krqew6fjYmbP4GMrLwEfU2JGkJm9iw8WX4W34yu45QrQ\nBzhQPuPrqiKynIy322i8G/YuM7utkc+Rpl8yIyu3PdxA/kJuxf/+r+LdrTMSGbPWv9Cj5gygJOkW\nYC9gopltXE+eq/HumR+Ao5LZLUEQBEGV0dwWxm2UmJ4maXe8L3gdfAbJjc0sTxAEQdBEmlVhmNlr\neBdJfexLMlPIzN4GOqq87g6CIAiCMlFSYchXzdY3ja0cFE4nHE/taXdBEARBlVByHYaZzUuWy3c0\ns6mVEqoYkmJAKgiCoAmYWVlmgGXpkvoeGCLpFklX57ZyVI5bFOmpmqtRYp62mVX91rt37xaXIeQM\nOVurjCFn+bdykmWl9yMs2AIzUf/85ieAE4H7JW0NTDGzifXkDYIgCFqQBhWGmfWVeyJdN0kaaWZz\nshQu6R7c6dhykj4HeuMuKczMbjazp+VeVD/Bp9X2rL+0IAiCoCVpUGFI6oo7BhuDWwqrSzrS3Lld\nSczs0Ax5TmpYzNZD165dW1qETISc5aU1yNkaZISQs5ppcOGepPeBQ81sZHK+LnCvmW1eAfnScli5\n++OCIAh+6kjCKjjo3TanLADMXRu3LUflQRAEQeshy6D3e5L+i7uGBjiM2q6RgyAIgoWALF1S7fCZ\nTNsmSQOA663CAUaiSyoIgqDxlLNLqqTCkAdCv8PMDitHZQtCKIwgCILGU7ExDPPIZGsm02qDIAiC\nhZgsYxij8TgJT+BrJQAwsyuaTaogCIKg6siiMD5NtkXwaGNBEATBQkhJhZGMYSxtZmc0tQJJ3fFo\nXYsAt5jZJQXXO+GRsNbGI5sdbWbDmlpfEARB0DxkGcPYpqmFJ7GUr8WDKG0I9EjiIKc5BxhkZr8C\njgTK5dgwCIIgKCNZFu59IOkJSYdL2j+3ZSx/S2CUmY1N/E/dhwdNStMFjy1MskBwrVzg+qIMHw6n\nnZax+iAIgqBcZFEYiwPfAt2AvZNtr4zlFwZIGkfdAEkfAvsDSNoSDzC/Wr0ljh4NV14JQ4dmFCEI\ngiAoB1m81Ta3B9mLgT6SBgJDgEHAvGIZzz33XD/YaCO67rgjXb/6ChZp7rDkQRAErYdXXnmFV155\npVnKrnfhnqQHzOyg5PgSMzsrde05M9u1wcI9xsW5ZtY9OT8bd21+SYl7PgN+aWbfF6TnF+5NnQqd\nOsH220P//g2JEQRBsNBSzoV7pSyMdVLHuwBnpc7rH2OozbvALyStCUwADgF6pDNI6gjMMLM5kv4P\n6F+oLOrQsSMMGQJTpmQUIwiCIFhQSimMUn44MvnoMI8JfhLwHPlptcMlHUsSRAnYAOgrqQYYCvwx\nk+QbbZQpWxAEQVAeSimMJSRtir/o2yfHuXCr7bNWYGbPAOsVpN2UOn6r8HqjueEGOOooaJ9ZrCAI\ngqCRlBrDeLnUjWa2Y7NIVA8lnQ927gwHHABXxxKOIAiCNBXzVltNlFQYzz0Hu+0G06fDUkvBhx/C\nCy/A6adXVsggCIIqo9IR96qfXZMJW/sn6wnPPhvOOAMuu6zlZAqCIPiJ8dOwMADuvRcOPRRqaryL\nasIET28lzxcEQdAcRJdUKebOhcceg/328/NFszjkDYIg+GlScYUhqTOwJqlZVWb2ajkEyEpE3AuC\nIGg8lVq4l6vsEuBgYBh5lx0GVFRhBEEQBC1LgxaGpJHAxmY2qzIi1StHeSyME0+E66939yIdOix4\neUEQBFVMpWdJjQbaNrUCSd0ljZD0saSzilzvkLhP/0DSEElHNbWuogwcCN9958fjxrmyAHg1MZDu\nuAPOO6+sVQZBEPwUyWJhPAz8CngRmG9lmNnJDRbuAZQ+BnYCvsR9Sx1iZiNSeXoBHcysl6TlgZHA\nSmY2t6CsplkYShTr3LkwZw688QYMGACrrQa77AJrrunXw+IIguAnSKUtjCeA84E3gPdTWxayBFAy\n8rHClwa+LVQWC8SNN/q+Z09o1w66dYPeveGPf/QZVDvvDBtuCE89Bdde6wrmjTfKVn0QBMFPhSzx\nMPpKWgxYN0kambz8s1AsgNKWBXmuBZ6Q9CWwFD7AXj6OPRZGjIDJk/PWRo5VV4Xnn3frY9FF4fXX\nPf2KK+C3vy2rGEEQBK2dLLOkugJ9gTG448HVJR1Zxmm1u+ExvbtJWht4XtLGxVyczw+gBHTt2pWu\nXbtmq+HKK0tfz63V2GYb+OYbWGst775q2+ShmyAIghahRQIozc8gvQ8cmsTbRtK6wL1mtnmDhWcI\noCTpSeAiM3s9OX8ROMvM3isoq3LrMDbYAO6/HzbeuDL1BUEQNBOVHsNom1MWAGb2MdlnTc0PoJR0\nax2Cj4mkGQvsDCBpJbzra3TG8puHQw91CyMIgiCYTxYL41agBrgrSToMaGNmR2eqQOoO9CEfQOni\ndAAlSasAtwOrJLdcZGb3Fimn8iu9f/wRzjrLB8jD2giCoBVSUdcgktoBJwLbJkkDgOsrvZCvRRRG\nbpD8L3+Bq66qbN1BEARlIJwPVoqHHoL//AceeQSOOML3c+dCmzb5PB99BDNnwhZbVFa2IAiCDFRE\nYUh6wMwOkjSEIjG8zayifTQt7nwwZ23cdZePceTOl18evv3WZ1cts0xtZRIEQdDCVMr54F+S/V7l\nqKjVM2yYL+779ltYZBEYORLWXdfPAfbc012q9+rVsnIGQRA0E1nGMC4xs7MaSmtuWtzCyPHjj9C+\nvR8//7y7F5k1y1eJ9+0LTz/tAZyKUVMD48f79UV+GsEOgyCobio9rXaXImm7l6PyVsnii8NNN/nx\niivCDTfAYovB7rvD4MGwV8ogO+MMXzUOPh7Spg2ssYavPB80qPKyB0EQLAClxjCOB04A1gY+SV1a\nGnjDzA5rfvFqyVMdFga4pfDCC/lY4gDz5vmK8Wefdd9UY8bAtsnEMjO47jo46SRYYQW47TZXLHfd\nBYdVtBmDIFjIqNSgd0dgGeAi4OzUpelm9l05Km8MVaUwGmLjjWHIEJg+HVZayd2rt2uXvz5zJiyx\nhB+/8w48/DBcfHHLyBoEwU+ainRJmdlUMxuDL7r7LvE4OxaYK2mrclT+k2TaNFcWa6wBSy0FM2Z4\nN1aa9u09NseOO7q1cskldR0jBkEQVBlZxjBuANKOAL9P0jKRIYDSGZIGSRqYBFCaK6lT1vKrjgkT\nfD9qlO9ffBHOOaduvs6d4aWXYKuU7v3uO/eq+0Xi4PeGG1yRDBjQvDIHQRBkIMssqQ/MbJOCtMFZ\n1mFkCaBUkH8v4BQz27nItdbTJTVtWuOCMQ0b5gGdOnTIWxqXX+4K45NP4LLL4PTTm0fWIAh+0lQ8\nRKukkyW1Tba/kN05YJYASml6AHX8SLU6Ghu5r0uX/D1nJUbYrFnw2GOw3XZw0EF17xk/3pVJIXPm\nQI8eMGlS42QIgiBogCwWxorA1UA3fMX3i7gV8HWDhUsHALuZ2THJ+R+ALYuFd5XUHg+wtLaZTSly\nvfVYGM3NE0/AvvvC2mvnlcasWfDee9CxI/zylz4Lq1+/lpUzCIIWp1IrvQFIFMMh5aisAfYGXium\nLHI0OYBSa8fMLZDvv/fjfRMj7Z578tdXXNG7wiZO9EiCTz7ZcvIGQdBitEgAJUlnmtm/JV1DcV9S\ndayEImU0GEAplfcR4AEzu6+eshZeC2P27Py03JoaeO01jwa49dbw1lvwm9/k8373HTz3nFsXd91V\nvLwgCBYaKrUOY28z6yfpyGLXzaxvg4VLbYCR+KD3BOAdoIeZDS/I1xEfF1nNzGbWU9bCqzAADj8c\n9t8ffve72um5QfLvv/cV57mwsjU17qZkrwJXYF9+Cddf7y5OLrus+eUOgqBFaVXuzRsKoJTkORIf\n6zi0RDkLt8KoD8mVyW231faUO3487LSTuyEBWGUVz7PCCvDrX8P220P//i0jcxAEFaNSFkY/inRF\n5TCzfcohQFZCYTQSM3e9PnQodOrkiwV/8xvvzmrTxp0l7refz6b61a9aWtogCJqJSimMHZLD/YGV\nyYdo7QFMNLNTyyFAVkJhNIFcd5UZPP64K4ixYz19tdXyHnO//NItkBxz5nhY2iuvhOWWq7zcQRCU\njUq5BulvZv2BbczsYDPrl2yHAtuVo/KgmUm7Wd9nH++GGjYMVl/dlcaVV/q1wYPd+pBcWbz0Etx5\nJ3z+efa6pk2Da64pr/xBEFQVWdZhDAf2NLPRyfnPgKfNbIMKyJeWIyyMBWXmTB8UX7TIbOqcNbLj\njvDyy3DKKbDRRvCnP7mF0hAffwzrrQevvuqLDYMgqAoqvdL7VOAVSa9I6g+8DJxSjsqDCtO+fXFl\nAT5IfuONcPPNfn7ccW6RAHz1le+feQZWXtnHRV5/PX/vSSd5bHOAbt2aR/YgCFqcTLOkJLUD1k9O\nR5jZrGaVqrgMYWG0BLvs4tbGLrvk14L07g3/+pdbHjU1Pog+aJBP7d1uO5+ue+qpfn3OnLreehvD\ne++5u5QXXyzP8wTBQkZFLQxJSwB/BU4ysw+BNRIngcHCQJcuPgD+hz/4+ZgxcOKJsOyyfp7zpLvJ\nJr6Q8IwzfHv/fQ8OtdVWvriwFPfcU//K9Fde8djpzzzj0QuHD/cZXkcfXY6nC4KgEWTpkroNmA3k\nlhOPBy5oNomC6mLvvX1B4AEHuJv2NdeEZZbxFeWffALHHJPPu+iicOmlvsDws8/cr9XgwfDggx4j\nZNIkn5E1d27tOq64Ag6px/vMiBE+7XfKFPfY26WLH992m08RDoKgYmRRGGub2b+BOQBmNgOIaD8L\nCzvv7LOlDj4YLrzQ03LjII895gPdl19e+5411oDbb3fniOBdVBtvDL//vc/catvWQ9zutJMPtt98\nM/zwA1xQ5Dtk6FBXEnvv7ef77gt//7sfn3lm3fyzZ8Mee/h+xgy3UIIgKAsNOh8EZieeZA1A0tpA\n5jGMZKX3VeRXehfzI9UVuBJoC3xjZjtmLT9oId59F372M1hrLfj5z2tf+/RT+N//vJvpuOPy6f37\n+8yrjz5yX1cvveTpuTUg771Xt4633oLNN4cll/Spu+3b+7V99/W1JYU8+qjXPWqU1wXZZnkFQdAg\nWabV7gL8HegCPAdsAxxlZq80WHiGAEqJH6k3gF3NbLyk5c2sTjCHGPRuRYwdCx9+6Gs/cowf74sF\np03zF/pyy7n1MneuD5pPmOCD5uusk7/njjvcinnkkWz1fvihj6XcfrvHTD/oIB+Ef/XVhu+dPt09\nAh94IDzwQKMeNwiqmYr5kpIkYDVgBrA13hX1VrEXej33bw30NrPdk/M63molHQ+sYmb/bKCsUBit\nnXQkwmnTfBxkrbXq5hsxwqMN9unTuPJXXBG++cZf/kst5XUsvXTteOkjR7pFlHPSmOP9993H1oYb\nwnXXwaabNj4QVhBUIRWbJZW8oZ82s2/N7CkzezKrskjoDHyROh+XpKVZF1hW0suS3pV0eCPKD1oT\n6Rdwhw7FlcUxx8AGG8DVV2frSho71gfYX33VB9i/+caVRa6OXEz0gQPdill/fR/ET3PWWXD88dC9\nu3eXnX66K5ZS7LNP3vIx80WRQfATJ8sYxkBJW5jZu80ow2Z4RL8lgTclvWlmdeKPLrQBlBYmvv3W\n90OH1rYM6mOTTXzWFPiaj2ILE/v394H5yZN93OOYY/wl//33bo38+9+e77//9f0qq7hyyfH119Cz\np4/J5GTq18+3L7/0gFXQfGMlZtnaIgho3gBKmFnJDRgBzAM+BQYDQ4DBDd2X3Ls18Ezq/GzgrII8\nZ+HdVrnz/wIHFCnLgoWAmprG5X/xRTMwu+ee+vO8/rrnufFGsxkz/Pjee32f2z7/PJ//uOPMrrkm\nf77GGp7nzTfzMq63nqe9+26+jMbK3hBz55qttVbzlB0sNCTvzgbf11m2LNNqdwN+jlsAewN7Jfss\nvAv8QtKakhbDQ70+UZDncWBbSW2SRYJbAcMJFk4a+yXdrZvHM+/Ro/48uYiE773ns6xOPBF22MG7\nrP7xD7+WdtS4zjoetTBHzgnj1lvDP/8JU6f6OIuZj3vMmlVeKyC3gn7SJF8oucQSYWEEVUG9CkPS\n4pJOwVd5dwfGm9nY3JalcDObB5yEz64aCtxnZsMlHSvpmCTPCOBZ3Hp5C7jZzIYt0FMFCxeFYxKF\nSN71dN11fn7ttfDFFz4ofuKJ/oJeJPWvsPnm3t1UU+PnL74Ib77p+c8/v278kFz9Dz8Mf/2rHw8Y\nAM8/78dmMG+eH+f2pcY8HnzQZ47l/HXde69PMe7evXa+kSPhoYdKP3uOmhpvh8MOy5Y/CIpRn+kB\n3I/HwDgWeAzoUy6zpikb0SUVlJP//Kd0V8/555sdcIDZtGn5tAED/J5rry1+T79+Zh06mP34o5nk\nea+7zuyEE/x47FjfX3qp7995p24Zp51mduut+W6uOXM8fb/9/PzHH/N5c3lmzWr4eR9/3PPus0/D\neefONfvgg4bzBa0CytglVeoFPSR1vCgwsFyVNknQUBhBpcm9kH/4wc+HDjU74oj6848Z4/m7d8+/\n9O+/32yJJfz4zDN9f/bZ+bKnTvVxEDNXXmD29NNmt99udv31+bJzYy6DBtWW7/TTs41v/PrXZuus\n48qgPj76yGzVVfPjMvPmlS7zgQeSV0hQzVRKYQwsdV7pLRRGUHFyL/U0EyfWn7+mxmz99fP3XHut\nv6BXWsnTTjzR9999Z/bss2aHH272j3942qhRboGsvHL95e+3n9mDD/rxrFk232opF+nnXXttt8Jy\nzJtnNnt27fwXXOD5J03y8w03rK3QgqqgnAqjVIjWecAPuVOgPb6AT4kAFV3VFAv3gorz1Vc+jnDA\nAQtWzrBh8ItfFB9rmTXL3b936eL5oP7pubfe6n66dt65dvo118BTT7lH38KyL7jAB/YbGueZMcPd\nrxx6KNx9t4+h1NT4WMvii7v8n37qsk2f7tOX27f3cZGePeGWW3wcaPJkjyFfivHjvezVV8+nXX65\n+/3q16/0vUGjKefCvRazGBq7ERZG8FOlTx/vCrrhBt9KMWNG7a4qs7xlMGWKWwIDB3r6888Xt5Ju\nuCE/RTjH7NlmG2xg9u23fj5mjHenpcvPjd/kjqdPN/vnP82uuMLs66/NllnG7MsvzXr2NJs50++9\n/36zwYPz9cybl78/3ZWWlnP8eLNjjindDkFmqESXVLVtoTCCnyxDhpgddVS2vJ9/7v+2Z5yRT/vu\nO7O2bc3uuiv/4p0zx/eLLmr2r3/VLmO55cyOPz5bfbkyX3jB5o+h5Or45pt8vkMP9fUqkyb5tSef\nNPvTn/x4jz3MPv7Y7NFHze67L3//hAl+74gRfn7NNWbPPZe/3quXzR/nyULfvsUnAMye7RMPHnvM\n7L33spXVVIYNq7o1M6EwgmBhppjVMG6cv4Bz1y691Kx/f7c6Crnqquxf8MXqqqkx+/77uvkWX9yP\n//Y337bf3uZbErlydtmlbh1vvJGXJ219XH11/vyzz3x/882uDPv2zSscs3zefv1ql33EEWYbb2zW\nrp3Z1lubrbhi/c96663ZFWlNTV3FMHduXUVaBZRTYWRZuBcEQbWx3Xa1zzt39njrZu7ifeedPSZ7\nx4517918c49BkhvzOP10X/tRjFGj4OOPa6dJPt6RZsIEj4YIvsDxwgt9keO553r+q67ya7mY8eBr\nYT791BdW3nSTp82d6/7AJPjzn92nWJcu7koffJxnySXhyCN9Xc24cT7ukYv8+PTTvn/vPTjvPB+D\nGTwYdtwRttnG3bzkeOaZ2uthllnGnV5moUsX2Gsvj+Py7ru+vibnlmb4cDjhhGzlNIWamuZzQ9MQ\n5dI8zb0RFkYQOLNn59dnNIUff7T5XUK5rqs+fcon38SJVmfNiFntNS1m/vUPZpdd1nCZAwaY3X23\nWzYbbOD3jRtntuuuxa2t444zO+cctwJy3VBTpnjeyZPNXnvNj++4w62x99/3ds1ZNHPn+jTqkSO9\nO66Q9LgOmF18se/fftvLArfwinHjjWZXXtnwM9dH164+4y4jVGha7XRgWrJNT51PB6ZlrsBXiY/A\n42KcVeT6DsAUYGCy/b2echrZqkEQ1Euur78xi/8aw+jRDecZN87r7tat6fUUrpVJpz31VPH8Bx9s\ndsstvsg6h45cAAAM1klEQVTy7bfz+adP97GY3FqY3NqZjTcuXs7+++fHeN5+2+yRR+rKMHGiTwTI\nke6emzvXbLvtvLstff3GG80++aT+Z5Z8SnbmJiqfwqjXW62ZLb2g1ksSQOlaUgGUJD1uqQBKCa+a\n2T51CgiCoHnITbN99NHa5+Ui14VUis6d3S9Xzl1KU7j4Yvc6nIvECN4Vdt55sO22dfP/8IM/a9u2\nsOeeHqM+x5JL+pTiu+/O++7q1Mn9eU2aBMsvn887ebJ390lw110uw+9+l78+c6bLtNJKfm5JF1JO\n1pkzPR7MgAHeVq+/7ukzZuSjVM6eXTduy9y5XtYVVzSpuRaUBiPuAUjaFljHzG6TtDywtJl9luG+\nLAGUdgDOMLOSDg1jHUYQBGVjxAhYdlkPulWMpZeGk0/2sZiddvJxnj328Bf2Rx+5W/2GmD0b2rXz\n41wwr0Ikr6dzZ3jtNfdHduSR7j/sxx/z9+d46CGPCpnzDZaBigVQSirrjbsg75UkLYb7mMpClgBK\nAL+R9IGkpyR1yVh2EARB01h//fqVBfjixAsv9ONOndwamTsX+vaF3/42Wx2LLeYxWh55xAfnx4yp\nm8fMI0vOnOmD98OGwT33eHpaWWy+ucdy2XNPX8CZi80yeHDmRy4HWQIo/Q7YFB9fwMy+lLTA3VUp\n3gfWMLMZknbHHR2uWyxjBFAKgqDi3HSTR2xs08bjzB90UPZ7F13Uu6q6d4dnn3VX+ekV7jk22MD3\n661X95qZ17/ffm5Z9OzpQcHAvRCMGlUre3MGUGqwS0rSO2a2paSBZraZpCWBN81s4wYL9y6pc82s\ne3Jep0uqyD2fAZub2XcF6dElFQRBy1FT41NvP/kEVlihcfdedBGccw5cdpl3b2Vh5EjvDtt5Z3jg\nAbdCRo6Eddd1JXLOOT41eNCgksWUs0sqi4XxgKSbgE6S/g84GvhPxvLnB1ACJuABlGpFupG0kplN\nTI63xJXYd3VKCoIgaEkGDvSxiMYqC4BevTz+/IEHZr9n/fV9/8MPPhg+ejSstpqnSe4nrFev+u9v\nBrIOeu8C7JqcPmdmz2euQOoO9MHHS24xs4slHYtbGjdLOhE4HpgDzARONbO3i5QTFkYQBC3HFVe4\ndVCp99DDD8PBB/vYyQJQTgsjq8JYGdgSMOBdM/uqHJU3hlAYQRC0KDNn+sr0jTZqaUkaRaVnSf0J\neAfYH/g98Jako8tReRAEQauhfftWpyzKTZZB75HAb83s2+R8OeANMysynN98hIURBEHQeCpqYQDf\n4u5AckxP0oIgCIKFiHpnSUk6LTn8BHhb0uP4GMa+QGVXiwRBEAQtTqlptbnFeZ8mW47Hm0+cIAiC\noFrJNEuqGogxjCAIgsZT0YV7klYAzgQ2BBbPpZtZt3IIEARBELQOsgx6343Hs/gZ8C9gDL6COwiC\nIFiIyKIwljOzW4A5ZtbfzI4GMlsXkrpLGiHpY0lnlci3haQ5kvbPWnY10lxOv8pNyFleWoOcrUFG\nCDmrmSwKY06ynyBpT0mbAstmKTwVQGk3vEurh6T168l3MfBsJqmrmNbyIwo5y0trkLM1yAghZzWT\nxfngBZI6AqcD1wAdgFMylr8lMMrMxgJIug+fllsYce/PwEPAFhnLDYIgCCpMgxaGmT1pZlPN7CMz\n29HMNgfWzlh+gwGUJK0K7GdmNwBlGckPgiAIyk+TptVK+tzM1siQ7wBgNzM7Jjn/A7ClmZ2cyvMA\ncJmZvSPpNuBJM3u4SFkxpzYIgqAJVDIeRjGyVj4eSCuW1ZK0NL8G7pMkYHlgd0lzzOyJdKZyPXAQ\nBEHQNJqqMLJ+7TcYQMnMfp47TiyMfoXKIgiCIGh5SvmSmk5xxSCgfZbCzWyepJOA58gHUBqeDqBU\neEs2sYMgCIJK02pcgwRBEAQtS5Z1GC1O1sV/FZJljKQPJQ2S9E6Stoyk5ySNlPRsMg05l7+XpFGS\nhkvatf6SF1iuWyRNlDQ4ldZouSRtJmlw0tZXVUjO3pLGSRqYbN2rQM7VJL0kaaikIZJOTtKrpk2L\nyPjnJL2q2lNSO0lvJ/8zQyT1TtKrpi0bkLOq2jNVxyKJPE8k583fnmZW1Ruu1D4B1gTaAh8A67eg\nPKOBZQrSLgHOTI7PAi5OjrsAg/Cuv7WS51AzybUtsAkweEHkAt4GtkiOn8ZnuTW3nL2B04rk3aAF\n5VwZ2CQ5XgoYCaxfTW1aQsZqbM8lkn0b4C18jVbVtGUDclZdeyblngrcBTyRnDd7e7YGC2P+4j8z\nmwPkFv+1FKKuZbYv0Dc57gvslxzvA9xnZnPNbAwwCn+esmNmrwGTF0Queez2pc0s5yvsjtQ9zSkn\nFJ95t28LyvmVmX2QHH8PDMdn+VVNm9YjY26dU7W154zksB3+4srF1qmKtmxATqiy9pS0GrAH8N8C\neZq1PVuDwmhw8V+FMeB5Se/K450DrGRmE8H/iYEVk/RC2cdTWdlXbKRcnfH2zVHJtj5J0geS/psy\npatCTklr4VbRWzT+b10RWVMyvp0kVVV7Jt0ng4CvgOeTl1TVtWU9ckKVtSdwJfBXak8Uavb2bA0K\no9rYxsw2w7X7iZK2o+7srmqdSVCtcl0P/NzMNsH/US9vYXnmI2kp3G3NX5Kv+Kr7WxeRsera08xq\nzGxT3ErbUtKGVGFbFpGzC1XWnpL2BCYm1mWp9Wllb8/WoDCyLP6rGGY2Idl/AzyGdzFNlLQSQGLm\nfZ1kHw+snrq90rI3Vq4WkdfMvrGkExX4D/luuxaVU9Ki+Iv4TjPLRZqsqjYtJmO1tmci2zTgFaA7\nVdaW9clZhe25DbCPpNHAvUA3SXcCXzV3e7YGhTF/8Z+kxfDFfy2ysE/SEsnXHJKWBHYFhiTyHJVk\nO5J8GNsngEMkLSbpZ8AvgHeaU0Rqf3E0Sq7EjJ0qaUtJAo6geULy1pIz+XHn2B/4qErkvBUYZmZ9\nUmnV1qZ1ZKy29pS0fK4bR1J7YBd8vKWq2rIeOUdUW3ua2Tlmtob5oudDgJfM7HCgH83dnuUeuW+O\nDf8aGYkP1pzdgnL8DJ+lNQhXFGcn6csCLyQyPgd0St3TC5+VMBzYtRlluwf4EpgFfA70BJZprFzA\n5smzjQL6VEjOO4DBSds+hvfFtrSc2wDzUn/vgcnvsNF/6+aStYSMVdWewC8T2T5I5PpbU/9vWkjO\nqmrPApl3ID9LqtnbMxbuBUEQBJloDV1SQRAEQRUQCiMIgiDIRCiMIAiCIBOhMIIgCIJMhMIIgiAI\nMhEKIwiCIMhEKIyg6pBUI+mO1HkbSd+k3DjvLenMBspYRR4vHklHSrqmkTL0ypDnNkn7N6bcciLp\nZUmbtVT9wcJHKIygGvkB2EhSu+R8F1LO08ysn5n9u1QBZjbBzA5KJzVShnMamb9VIalNS8sQtD5C\nYQTVytPAnslxD9xnDlDbYki+8vtIel3SJ7kv/sSVzJBUeWskX+QjJf0zVdajiefhITnvw5IuAtrL\ng9PcmaQdoXzgrL6pcncorDtNIscwSTdL+kjSMzlFmLYQJC0n6bPU8z0qD4YzWtKJkk5N5HlDUqdU\nFUckMg2WtEVy/xLyQFVvSXpf0t6pch+X9CK+IjgIGkUojKAaMTzuSY/k5boxebfd6Tw5VjazbYC9\n8SAyxfJsAfwO+BVwYKorp6eZbZFc/4ukZcysFzDDzDYzs8PlHkvPAbqaezL9S4a60/wCuMbMNgKm\nAgeUeO4cG+KxCbYELgS+N/eS/Bbu8ydH+0SmE3G/UgB/A140s62BbsBliW8kgE2B/c1sx3pkCIJ6\nCYURVCVm9hEeHawH8BSl3Tg/ltwznHwMgEKeN7MpZvYj8Age+Q/gFEkf4C/i1YB1kvR0fd2AB81s\nclLPlEbW/ZmZ5ayd95PnaoiXzWyGmU0CpgBPJulDCu6/N6l/ALC0pA64U8yz5XEdXgEWI+/x+Xkz\nm5qh/iCow6ItLUAQlOAJ4FKgK7B8iXyzUsf1KZY6sRck7YArg63MbJakl4HFGyljlrrTeeal6phL\n/qOtsN70PZY6r6H2/22xmBICDjCzUekLkrbGx4eCoEmEhRFUI7kX763Av8xsaBPuLWQXSZ2Srpn9\ngNeBjsDkRFmsD2ydyj87NTD8Et6NtSyApGUaWXd96WOAXyfHB9aTpyEOTmTaFphqZtOBZ4GT51cu\nbdLEsoOgFqEwgmrEAMxsvJldmyVvifMc7+BdUR/g3UsDgWeAtpKGAv8PeDOV/2ZgiKQ7zWxYcr1/\n0s2Ti7iWte760i8Djpf0Pu6auj5KlfujpIF4VLijk/Tz8ecaLOkj4LwSZQdBZsK9eRAEQZCJsDCC\nIAiCTITCCIIgCDIRCiMIgiDIRCiMIAiCIBOhMIIgCIJMhMIIgiAIMhEKIwiCIMjE/we4YxytAU4D\nAQAAAABJRU5ErkJggg==\n",
|
|
"text/plain": [
|
|
"<matplotlib.figure.Figure at 0x2ec09a99e10>"
|
|
]
|
|
},
|
|
"metadata": {},
|
|
"output_type": "display_data"
|
|
}
|
|
],
|
|
"source": [
|
|
"pred = train_and_evaluate(reader_train, \n",
|
|
" reader_test, \n",
|
|
" max_epochs=5, \n",
|
|
" model_func=create_basic_model)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"Although, this model is very simple, it still has too much code, we can do better. Here the same model in more terse format:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 14,
|
|
"metadata": {
|
|
"collapsed": true
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"def create_basic_model_terse(input, out_dims):\n",
|
|
"\n",
|
|
" with C.layers.default_options(init=C.glorot_uniform(), activation=C.relu):\n",
|
|
" model = C.layers.Sequential([\n",
|
|
" C.layers.For(range(3), lambda i: [\n",
|
|
" C.layers.Convolution((5,5), [32,32,64][i], pad=True),\n",
|
|
" C.layers.MaxPooling((3,3), strides=(2,2))\n",
|
|
" ]),\n",
|
|
" C.layers.Dense(64),\n",
|
|
" C.layers.Dense(out_dims, activation=None)\n",
|
|
" ])\n",
|
|
"\n",
|
|
" return model(input)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 15,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Training 116906 parameters in 10 parameter tensors.\n",
|
|
"\n",
|
|
"Learning rate per minibatch: 0.01\n",
|
|
"Momentum per sample: 0.9983550962823424\n",
|
|
"Finished Epoch[1 of 10]: [Training] loss = 2.064481 * 50000, metric = 75.90% * 50000 12.260s (4078.3 samples/s);\n",
|
|
"Finished Epoch[2 of 10]: [Training] loss = 1.703777 * 50000, metric = 63.17% * 50000 12.127s (4123.0 samples/s);\n",
|
|
"Finished Epoch[3 of 10]: [Training] loss = 1.561847 * 50000, metric = 57.11% * 50000 12.116s (4126.8 samples/s);\n",
|
|
"Finished Epoch[4 of 10]: [Training] loss = 1.463862 * 50000, metric = 53.17% * 50000 12.078s (4139.8 samples/s);\n",
|
|
"Finished Epoch[5 of 10]: [Training] loss = 1.374724 * 50000, metric = 49.49% * 50000 12.117s (4126.4 samples/s);\n",
|
|
"Finished Epoch[6 of 10]: [Training] loss = 1.294328 * 50000, metric = 46.14% * 50000 12.158s (4112.5 samples/s);\n",
|
|
"Finished Epoch[7 of 10]: [Training] loss = 1.231594 * 50000, metric = 43.62% * 50000 12.084s (4137.7 samples/s);\n",
|
|
"Finished Epoch[8 of 10]: [Training] loss = 1.179700 * 50000, metric = 41.84% * 50000 12.156s (4113.2 samples/s);\n",
|
|
"Finished Epoch[9 of 10]: [Training] loss = 1.136541 * 50000, metric = 39.93% * 50000 12.065s (4144.2 samples/s);\n",
|
|
"Finished Epoch[10 of 10]: [Training] loss = 1.096253 * 50000, metric = 38.56% * 50000 12.148s (4115.9 samples/s);\n",
|
|
"\n",
|
|
"Final Results: Minibatch[1-626]: errs = 34.4% * 10000\n",
|
|
"\n"
|
|
]
|
|
},
|
|
{
|
|
"data": {
|
|
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYwAAACfCAYAAADqDO7LAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztnXm4FNXR/z9fRBHcUDQgIKgQ5XVBwD0SRYyKGtHoqwbj\nrvmp0WjM5vZGoiYuCcYYjSbGJUZFxC2Ca1DRqLixI7iDC6uIAgLKWr8/qsfpO8xc5sLMvTOX+jzP\nPN3n9OnT1Q23q+vUqToyM4IgCIJgVTRpaAGCIAiC6iAURhAEQVAUoTCCIAiCogiFEQRBEBRFKIwg\nCIKgKEJhBEEQBEURCmMtRdItki6ta1tJ+0n6pLzSfXPdKZJ618e1GgPJv82EUrddDTlelHRSOfoO\nGpamDS1AUFokfQi0Adqa2eep+jHALsDWZvaxmZ1dbJ952q5W8I6kjsAUoKmZrVidPhoLknoCT+LP\nsgnQAlgAKKnbwcym1qVPM3sB2LnUbYMgQ1gYjQ/DX8r9MhWSdgKas5ov+hKSeRmq7BeS1in3NdYE\nM3vJzDYys42BHfHnskmmLldZKKFBhA2ChFAYjZO7gZNT5ZOBu9INJN0p6Ypkfz9Jn0j6uaRZkqZJ\nOiVf22yVLpY0W9JkScenDhwqabSkeZI+ktQ/dd4LyXaupPmS9kzO+bGkSUndm5K6pc7pLmmcpC8k\n3SdpvXw3LOlkSS9J+pOk2UB/Sf0l3Z1q01HSCklNkvJwSVck582X9JSkzQr0P0nSoanyOpI+ldRN\nUjNJd0v6LJHzNUlb5OtnFdRQCMnQzhWSRuDWx1aSTk89q/cknZ5qf4CkKanyJ5IukDQ+keteSevW\ntW1y/GJJM5J2ZyTPscMqb8i5TNKHkmZKukPSRsmx5sl1Ms/t1czzT+5zSnKf70s6djWeZ1BiQmE0\nTl4FNpK0ffJyPA64h9q/7NsAGwFtgTOAv0rapJa2myVtTwFulfTt5NgC4EQz2wQ4DDhLUt/k2L7J\nduPkK/o1SccAlwEnJF/bfYE5qWsdAxwEbIMPqZ1Syz3sCbwPtAZ+n9TlWlW55X64Qt0CaAb8skDf\nA4HjU+U+wGwzG5ucvzHQDn8uZwFf1SJnXTgBv+eNgWnATOCQ5Fn9GLgxsSAz5N7fMcABwLbAbsCJ\ndW0r6fvAOcB+wHZA7zznFuLH+HPbF+iEP58/J8dOxS3ftkn9T4CvE4VyHXBAcp/7AOOLvF5QRkJh\nNF4yVsaBwFvA9FW0XwJcaWbLzexJ/MW/fYG2BvzGzJaa2X+Bx4FjAczsv2Y2Mdl/ExiEv2jSpBXX\n6cAfzGx0cs5kM0s71W8ws1lmNhcYCqStj1ymmdnNZrbCzBav4n4z3GlmHyTtB9fS/31AX0nrJ+V+\nSR3AUqAVsJ05Y8xsQZHXXxV3mNm7yb/LcjN73Mw+AjCz54Fnge/Wcv71ZjbbzL4AHqP251eo7THA\n7YkcXwGX10H+44EBid9sIXAJWcW7FNic7HMbbWaLkmMrgJ0lNUv+/d+uwzWDMhEKo/FyD/6HeQrw\nryLaz8lxRC8CNizQ9gsz+zpV/gj/SkTSnpKeS4Zr5gJn4i+FQmwFfFDL8VlFygSwOrO3ZhbTv5l9\nAEwCDpfUHLeEBiaH7waeBgZJmirpGpXOh1LjniR9Pxm6mSPpC/yDoLbnW5fnV6ht2xw5PqF4P1Rb\n/P9Hho+AZsmQ3T+BZ4DByVDXVZKamNmXuEI+F5gpaUjKgg0akFAYjRQz+xh3fh8CPFzi7jdNXpoZ\nOpC1YO4F/g20M7OWwN/JvlzyDWN8gg9VlILc/hfis48ybLmG/Q/ClfARwEQzmwxgZsvM7Eoz2xH4\nDnA4UKpppd/cU2LdPIAPt21hZpsCwyj/JIIZQPtUuQPFD0lNBzqmyh2BxYkls9TMrjCzHYCewFHA\njwDM7GkzOxAf/vwA/38UNDChMBo3pwG9k2GEUiLgcknrSvou7qsYnBzbELdAlkrag5rj/rPxoYa0\ngrgN+KWkHgCSOknaqkRyjgX2lbRV4o+5aA37G4T7U84ma10gqZeknRJ/0QJ8qKWu04aLeek3A9YF\nPgMs8S0cUMfrrA6DgdMlbSepBfB/dTj3PuDnyYSDjYDfkTw7SftL2lGSSD03SW0SS6o5sAxX/MtL\neUPB6hEKo/HxzZefmU3J+AZyj9WlnzzMAL7Avx7vBs40s/eSYz8BrpQ0D3+x3J+S5yv86/hlSZ9L\n2sPMHkzqBkqaDzyCO0DrKu/KN2D2THL98cAbuA+kRpM69jcTeAXYi9R94V/BDwLzgInAcPy5ZIIe\nby6m+1XVmdk84ALcgpuDf5Hn3tOq+qxzWzN7DLgF+C/wDvBScqiQnyjd1z/wZ/UiPiFhHvCz5Fhb\n3PqdB0wA/oMrk3WAX+H/v2YDe+NO96CBUTkXUJLUHh8/b41/cf3DzP5SoO3uwAjgODMr9RBKEAQl\nIpmVNcrMmjW0LEH9Um4LYxnw82Rsd2/gHEldchslpvw1uOMwCIIKQ9KRyRDkZvjf6r8bWqag/imr\nwjCzmck8dZJphm/hc9Vz+Slu0n9aTnmCIFhtzsF9J+/iM6jObVhxgoag3nJJSdoan9f9Wk59W+BI\nM9s/cZIGQVBhJDOWgrWcelEYkjbELYjz8wQ0/Rm4MN28QB8NnQcpCIKgKjGzkky9LvssKUlNcWVx\nt5k9mqfJbnjA0xTgf/GUFH3ztMPMKv7Xv3//Bpch5Aw5q1XGkLP0v1JSHxbGHcAkM7sh30Ez2zaz\nL+lOYKiZDakHuYIgCII6UFaFIWkfPHJzgnw9BsNzyXQEzMxuzTklhp2CIAgqlLIqDDN7GQ/CKbb9\naWUUp17o1atXQ4tQFCFnaakGOatBRgg5K5myBu6VEklWLbIGQRBUCpKwanF6B0EQBI2DqlrT+x//\ngI8+gr33ht69oXnzVZ8TBEEQlIaqsjB69YIFC2DAAGjfHs47DyZPbmipgiAI1g6q1ofxySfw17/C\n7bfDmDGuQIIgCIKalNKHUbUKI8PixdAscmYGQRDkpWqc3pLaJ8t1TpQ0QdJ5edocL2lc8ntJ0s51\nuUYoiyAIgvqh3OthtAHamNnYJJ/UKOAISy3oLmkv4C0zmyepD/BbM9srT18xrTYIgqCOVI2FYUWk\nNzezV81XEgN4Nfd4XVm6FJ5/fk16CIIgCPJRb7OkCqU3z+EM4Mk1uc6ECbD//jB//pr0EgRBEORS\nCenNM232B04Fehbq57e//e03+7169cobmt+jB5xwAlx/PfTvv2ZyB0EQVBvPP/88z5dpmKXss6SS\n9OaPAU8WylgrqSvwENDHzD4o0KZoH8aUKbD77jBxIrRuvZqCB0EQNAKqxoeRUGt6c0kdcGVxYiFl\nUVe22QaOOcZjNIIgCILSUAnpzX8DbAbcLEnAUjNb46Va990XHnhgTXsJgiAIMlR94F4hZs+GqVOh\ne/cyChUEQVDhRKR3EARBUBTV5sMIgiAIGgGhMIIgCIKiaPQKY8kSuOoqiNGsIAiCNaPRK4z58+HS\nS/0XBEEQrD4Nnq02afcXSe9JGiupWyll2HxzuPJKuPrqUvYaBEGw9lEJ2WoPAc41s8Mk7QncUOps\ntQsXwoYb+lTbzTdfvXsJgiCoRqpmllQx2WqBI4B/JW1eAzaRVNKEHhtsAC1bwuWXe3ny5PBpBEEQ\n1JV6ST4ItWarbQd8kipPS+pmlfL6c+ZAkybQtCksXw5vvAG77VbKKwRBEDRuKiZbbTEUk622EE0S\nW2r5ct9uu+3qShEEQVC5NOpstZL+Bgw3s/uT8tvAfmY2K6ddySK9zUAlGdELgiCobKrGh5FQa7Za\nYAhwEnyzXOvcXGVRajLKwix8GUEQBMVS7llS+wD/BSbgmWrzZatF0k1AH2AhcKqZjc7TV0lzSZll\nh6lCaQRB0FiJ5IMlYPJk6NTJ9597Dm65BQYPLln3QRAEFUEojBKwaBH84x9wzjmw7rpet2JF+DaC\nIGhchMIoMbvtBqNGwWuvwR5rvHRTEARB5RAKowy8/TZsv33WwhgxAjp0gPbty3bJIAiCshMKo4z0\n6we9esFZZ3m5Sh5PEARBXqptWm3VcPbZMGgQvPwyXHCB102fDo8+2rByBUEQVAJhYaS4/3744Q/d\nqli2DPr0gYkTYeZMuP12aNUK+vYNx3gQBNVD1VgYkm6XNEvS+ALHN5Y0JElrPkHSKeWUZ1Ucd1x2\nCKppU3jmGejf38tmcOSR7tsIgiBYGyl34F5PYAHwLzPrmuf4xcDGZnaxpM2Bd4DWZrYsT9t68WEU\n4osvYLPNfL9KjLIgCIKSWhhlTT5oZi9J6lhbE2CjZH8jYE4+ZVEJbLopHHIITJuWrYucVEEQrE0U\nNSQlqZOkZsl+L0nnSWpZguvfBOwgaTowDji/BH2WjSeegHHjsuU+feDhhxtOniAIgvqkWAvjIWA3\nSZ2BW4FHgYHAoWt4/YOBMWbWW1InYJikroVSoK9JevNSMmSIO8V33hkmTICjjmoQMYIgCFaiwdOb\nSxptZj0k/Qr42sxulDTGzLoXcW5HYGgBH8ZjwNVm9nJSfha40MxG5mnboD6MNJlhqD/8AX79a/jw\nQ+hY28BbEARBA9EQs6SWSuoHnIyvbQGwbpHnKvnl4yPgewDJsqzbAZOL7LfBOOEEGDrUI8HB1wyf\nM8eD/a68smFlC4IgKBfFWhg7AGcBr5jZfZK2AY41s2tXcd5AoBfQCl9ytT+wHklqc0lbAv8EtkxO\nudrM7ivQV8VYGGkWLoRdd/Xst0uXel1GzHffhblzIz9VEAQNR4OmBpG0KbCVmeWNrSgXlaowAL71\nLZg9G9q186Vfd9nF62+6ybe5Yr/7riuYPn3qV84gCNY+6l1hSHoe6Is7yUcBnwIvm9nPSyFEMVSy\nwpg+3ZXEp5+64mjduubxESPgvvugRw+PFG/VCg4+GJ56qmHkDYJg7aEh4jA2MbP5ks7Ag/D6F4re\nXhtp29YVBbi1keHaa93f0a7dyuccfnj9yBYEQVAqilUYTRN/w7HApWWUp1GQNoTyGUX77QdnnhmB\nf0EQVBfFzpK6Anga+MDM3pC0LfBe+cRqPEg+VLVsmSsIM3j+eR+matIE/vtfX4vDDJYv9/aPPQZf\nf+3lRYuyfZl5pHmFjswFQdDIiWy1DcTMmbDlltnyXXfBsGFwzz3Zuk6d4IMPYMkSX0Z2yBA44gj4\n9799GwRBsCrqPQ5DUntJj0j6NPk9JGmVa9GtKltt0qaXpDGS3pQ0vC7CVzNt2sCLL8Lvfuflk092\nXwjApcmgX2ZNjslJZMq8eb49+OD6kzMIgiBDsUNSdwJDgLbJb2hSV8x5BV9vkjYB/gp838x2Ao4p\nUp5GQc+erhyuusrLV1/tw02/+pWXt9zSV//7+GMvv/EGDBgA669fs5+rr4axY+tN7CAI1lKKnVY7\n1sy6raquwLm1pQY5G9jSzC4rop9GNSS1KoYMge7dffZVhw6uJB56yIME778fDjvMVwY85xxo3tzP\n+cEP4NhjfRGoIAgCaJg4jGdxayEThd0PONXMDiji3NoUxvV4ipEdgQ2Bv5jZ3QX6WasURprnn4f9\n94fPP/c06+mZVe+/78vK/t//Zevq+phmzvTYkZixFQSNj4aIwzgNuBG4Hl/DYgRwSomu3wPoDWwA\nvCLpFTN7P1/jSslWW98cfbRvP/zQFUaazp1dQVx6KVxyiQ9PSZ6ypEWL4vrfckt47jlXSkEQVDcN\nnq0274nSz8zsz0W0q83CuBBY38wuT8q3AU+a2UN52q61FsbEifDOO9k06jNn+uypr75y5XBAYueZ\nwS9/CX/6k5dfeMGHsmrLZTVpEuy4I9x7Lxx/vNetWOFpTc47r3z3FARB/dCguaRSQnxsZh2KaLc1\nrjB2znOsC2659AGaAa8Bx5nZpDxt11qFURdmz4ZTT4XHH8/W/eIX7ixfvtxnWr33nqcn6dwZBg/2\ntcxHj3afCfgaH127+nannRrmPoIgKA2VskTrKgVIZ6uV9DE52WrN7G1JTwPjgeXArfmURVA8W2zh\ngX+ZFz64s7x3b7jsMhg1Ktv2P/+Byy9366J7amWTN9/07ZIl9Sd3EASVT9ktjFIRFkbdGT8eNtwQ\n2reHZs1ciYzPExGTeaxPPw1dusCTT8Jrr3l0+jPPeHLFQg7xUaNgt93cemlS7CTtIAjqjXoL3JP0\npaT5eX5f4vEYQQXTtaunWz/nHC8PGOAv/xUrstZD586+nT7d060ffTTssAP8/e8edT5zpq/pkWbp\nUp/Ku2KFKwuAm2+u2cbM+474kCBoPERqkLWAF1/0oL+f5ySjX7AAmjZ1x/gWW8Bnn2WPmcG4ca4I\nli6FMWPglVfc2d6mDWy+eTY31sUXe6T6l19mHexnneVKB3zo68ADV5Zr+nQ4/3x44IHy3HcQBBXi\n9K5vQmGUl88/9xlT3/2ul9OPOnc46tRTPWjw/vvdGllvvWybwYPhoIPgwgtd4bz6qtcdkyeGPzMN\neOrU/CnggyBYcxpiTe+gkbPZZp6qZOhQXxEwTa9e8JvfZMt33ultund33wjA9ddnj7VsCR07ukMd\nPPr8yy9hq62yy9jec0927ZAZM8p2W0EQlJBQGEENvv99+Pa3a9YNHw5XXJEdgrrrLpgypWabn/0M\nPvoIjjzSy0cc4ZbGL37h5WHD3JIYPNitkRNPzCZXzMzcknx46oknoH//bN9LlmSTNOaycCHMmbNm\n9xwEQXHEkFSw2nTuDN26wYMP1qyXPFXJlVdm6y66yB3lqWB9XnoJ9tnH95ct8xTu22/vQYqZunXW\n8aGvH/4wf/T6sce6fyZXgQVB4FTNkFQx6c2TdrtLWirpqHLKE5SWiRN9rfJcZsxwBZHh6699udod\nd/Ty+ed7AGFGWYD7TyCrLABuuMG3p57q27ZtPa8W+AwtM0+VksnuGwRBeSn3kFSt6c0BJDUBrsFX\n9AuqiGbN3CrIpU0b2GCDbHnJEujbN5th99prYeONs8cvucR9HD/+sSuH9dd35/vIkX78O9/xtCjz\n5nm+qy++8Ky9J5zgyRc7dcovX6dOvqLh44/D66+X7r6DYG2l7ENSteWSSo6fDywBdgceM7OHC7SL\nIalGSmaG1YoVvs0sVws1FdIuu6wceNixozvab7kF/vY3T3nyz3/CKafUbHf11W71jBvnCmvgwHLc\nSRBUHlUzJLUqJLUFjjSzWygi1UjQONlkE99K/mvSxBXFhAlevusuPz52rK8Tkuajj/z8Bx5wqwOy\nymKXXbLt/ud/fG31bt18GG3x4vyyvPOOx6cEQbAya5JLqhT8GbgwVa5Vaayt6c0bO599lv8lnbEy\n7rjDAwMlOPxwr3vhBdh9d589lVEMPXp4CnjwYa7f/97L3/2uR6u3bg233w6nnw633ZaNgAcfEluw\nwFOjvPIK7LVXue42CMpLOdObY2Zl/QEdgfEFjk1OflOAL4GZQN8CbS1Y+xg0yGzZslW369/f7LTT\nfH/XXc0GD/b9a67xycBTp2YmBZutWGH22WdmvXqZvf662b33mm26afb4gAFm48b5+Z9/bnb77fmv\nWYxcQdDQJO/OkrzP68OHsTUF0pvntLszaRc+jKDOLFzoVkKbNjXr33rLh5mOPNJndS1c6OlL7rsv\nu/5HPjbd1GNPHnvMkzK+/bZP+U1z4IEeZ9KnT+nvJwhKRdWkBkmnNwdmkZPePKftHYTTO6gn0ulO\njj7aU8DPnu05sq691ofJBgyoeU76v99zz2UXrtplF2/frp1n+U0zZYorsq5da/ZTKPvvddf5tOOm\nDT1YHDQaqkZhlJJQGEEpkeDMM92vccklHkSYUQDnnJPNvjtwoE/L7dLF06N8+qknasy88LfZxheZ\nGjrUy3/8o5/fvDksWgQ77wyTJ6+cm+vFFz0VSy5bbOGBiBtu6MorCNaUSllAKQiqliVL/Cs+8+LP\nKAtwJ/nNN/taIAccAP36+ZK14PmvBgzwnFhPPeXTeH/yEz+2664eRDhlCvz1r/DII64swGNKpk3z\nPjt3zubRSjN+vK+vfttt7rCfN69mvEoQNDSRSypYK1l33cLDQkcd5Wnc00rk3HPd2jj3XF83ff58\nz7u1/fY+A8sM/pyscP90EoL6zjuw776+P2eO+1FuusmDDYcMySqTWbPg4Yd9aGvWrGy0e9u2PkyW\nQfKkjWnee89TqARBfRAKIwhyWG89X3gql379fKgJsmt4bLZZNo9Vz54ehf700/4Sv+MO+Phjtypy\nHeO/+pVHoku+nvpJJ3n9p5+6c374cHfQb7edWzO//GX2eJrttsuuO1KI0aPhqquKv/8gKEQojCCo\nAwcf7AkPW7fOf7xlS/j1rz1AcNo0T8CY9lXMmuVL4KYZO9bzbWXYbz9PKQ9uvay3nlsd66/vi2B9\n9JEfO/po32644cpyLFjgqyWee66nVrn0Ur92Pj78sKa19dZbNX0uGRYudOWXZsoUv89gLaFU83PL\n/SPiMIIqYd99szEdb7/tdSNGmC1Zkm0zapTZpEnZdmA2Z47Z0qXZNh984DEgmeNmHkMCZosWeaxJ\noT+LzDktW2b3r7kmf9snnvDjL7yQPXfoUN+//Xazdu18/4gjVr6eVFiGoDKghHEYDZqtVtLxksYl\nv5ck1RqrEQTVwKBBvh02LBu7sffeNfNi9ejh6UrOP9+3d9/tw1vp6bTbbgunnQb33utxIOA5s8DT\nvH/ve/ktgTRz52aHuwpFr2f6PumkrP/lq698e/rpWQvi0Ud9m8n5tWyZzwIL1iJKpXny/YCeQDcK\nR3rvBWyS7PcBXq2lr1Iq3SCoSrbdNmsx9OzpdWB26KE12y1aZPb1135s/HizDz80mzbNo9PBrHNn\nb7dkidmLL5o9+2xNa+eJJ7J9//znZmPGeCT9H//o9fvv78cuucS3d95pdtxxZjNnZmWYMSPbT9Bw\nUEILoz6GkgqmBslp1xL4pJbjJXuAQVCtHHBA9qX++edel37RjxlTs/3Uqb5durRmu06dzI4+2vff\nesvb/OhHXr7rruz5O+xg9sorXn/VVV73+edmfft63bhxZhtvXLPvL7/0dhdfnK2rjVatzK64Ys2e\nS1CYxqowfgncWsvxkj3AIKhWZs1yayHNtGlmW2+dfTk//vjK52V8H2C2++41X/CZP60PP8xaKsOG\nebsRI8z22MPbLFhQU/FkFJaZl7fayreTJmXrMr+MbyaToysNmB15ZLZ83XVmixev3O74481uu624\n55TLm29m/UlrG6VUGA2+HkbSZn/gJqCnmX1RoI31Ty30HNlqg6Am6ZlOI0dm071nmD/ffRodOvia\n6Vdc4fWXXQaXX56/r6++8kDCn/7UX/2ffebR6Mcf776VXGbPdl/Mn/7ks8UGDfLldadO9fOaNfPZ\nW5lZZpMmZVdiXLzYMxS3aOErK+63n9dPmODXzmQlXtUr66WXfMGss8/OToM+7jj3FaWXCG6s5Gar\nvfzyy7ESRXo3uIUBdAXeAzqtop/SqNsgaKTMnev+CjC7557izlm0yK2PXDbf3GzPPX1/+XK3bMy8\n7THH1JzxlUvGz7L++l5euDB7bOedzUaPzpZzLZ3Mr0+frLXzhz/4bKzvfz9rDaXJtUbatKlpOZmZ\nHXus2cCBvj9tWmHZGyNUyyypBFFgnQtJHYCHgBPN7IN6kCUIGi2bbOKzlq6/3qPQi6F58/wR7++/\n7/muwBe0yqQykWDw4PxL82bIRLB/kPxFt2jhqyJecYVbH5de6v1kluCdNCk7Uwvg0EPd2njiCbeK\n2rf31/+ZZ/p68ZJbLyNHet/NmmXPXbHCLZhcZszw6Pp77vEkkbkrNy5Y4DKC91soC0A+Hn20cIxL\no6NUmiffDxgITAcWAx8DpwJnAv8vOf4PYA4wGhgDvF5LX6VWvEEQlIGFC7PO9gyZL/7u3bP7Dz7o\n21wLZ/jwrN/k29/Otl+wwH+ZctOm2f0tt6xpobzyivtUFi/2/nItmMceq3nNG2/MWiS51sk995hd\ncIHZc8+5rGec4VZXhiZNfA2VQkydarb33nV9ijXp18/st79dvXOpJh9GqYhstUFQvWS+2A87zLP4\nDh/ukeuZeI9C7TOk//Qvvhiuucb3f/Qjj01JJ2k84QRPy9K8uV9j4UKvP/NM94dssw1stZWnkl+y\nxNO3dOzobaZNcwskfc2MLM2a+ZrwXbp4JP/w4X7+lVd6Qsnzz6/9Xlas8P0FC/JH56fZYQd4+WVf\nlyXdx+q8AkuZrbasFkYpf4SFEQRVy9y5NX0HkyaZTZxYuP2KFdmZXRddtPLxWbP82LRpZtOn+/4D\nD/jXfiZivV27rLUwaZJbFX36mF14YbZ+4sTsfvPmvp0xw2zKFI+0N6tpmVxwgVmHDjXr2rbN7s+d\na/bSSzVlPftsP5axSsBs5Ei3fsBswoSa7W+5xevTvp9Bg2paPWY+/bmY1yJhYQRBsDbwz3/CiSfC\nOusUbrNkCYwYkc2/laZVK/dXtGsHo0bBbrv5qz3Xghkxwv0h3bp5Xq8OHTzR49Chvi5J166wwQb5\nr3/bbXDGGb7frp1bKaec4pbMZpt5Pq8Mc+Z4f2PG+PGWLbPHLrrIrYpmzTxn1557+myx9dd3mU86\nyf0s8+f7fWVY1WsxLIwgCIIieOaZmuXp03378cdmn3ziX+j+LeqA5+jKfL2D2U47ZY9fcolbH23a\neOR7ZrbYc8952+23r2l9pH00K1aYvfyy2Xbb1bzeIYfUbL/vvmZ//3vWAnr33WwMSdeu7svIxNOc\nddaqnwFVNksqCIKgQUivaQK+QBW4D6N9e/ehpDMFQ/ZrvksXL2+xRfbY738PW2/tS/RedFF2ttj+\n+3turd694S9/8bpM2Ngee3ja+yZNfP2S7t2z/b37rlsxBx6YTVP/1Vcec9Kjh+cY+9e/svnJxo+H\n3/3O95s2Lew3KRcxJBUEQZCHn/7Ukz926ZINAFwVPXrABRf4UNNBB/lw0sSJPgzVujWcd54PbfXt\n62vHP/JIzfP/939dcbRqVXPY7Kc/dUWUqRs50tdC2WijVcsUS7QGQRCUmRtvrPs5Bx7oFkUmS3Hu\nN26LFm4gHQQEAAAI5klEQVSlNCkwtvPgg9n9BQs8vqNTJ1dE4PEx48dno/jHjvX+Mgt6lZsGTW+e\ntPmLpPckjZXUrZzy1AfpkPxKJuQsLdUgZzXICNUt57XXZpVFPq65pvigwA02yK78mFmxsVMn+MEP\nsm0+/bSmkik35fZh3AkcXOigpEPwlCDfxgP6/lZmecpONf9nr0RCztJRDTJC45Vz5Mjscr6/+EXN\nF39tmEGbNvmP9e7tPpD6oqxDUmb2UpJ8sBBHAP9K2r4maRNJrc1sbQm0D4JgLSGdDHLAgNL02bRp\n8WlgSkFDz5JqB3ySKk9L6oIgCIIKo0HTm0saClxtZiOS8jPAr81sdJ62MUUqCIJgNWgss6SmAVul\nyu2TupUo1Q0HQRAEq0eDpjcHhgAnAUjaC5gb/osgCILKpKwWhqSBQC+glaSPgf7Aenio+q1m9oSk\nQyW9DyzE058HQRAEFUjVRHoHQRAEDUtDz5IqCkl9JL0t6V1JF9bztVcKPpS0qaT/SHpH0tOSNkkd\nuzgJRHxL0kGp+h6Sxif38OcyyNle0nOSJkqaIOm8SpRVUjNJr0kak8jZvxLlTPpvImm0pCGVKmNy\njQ8ljUue6euVKGsyZf6B5JoTJe1ZgTJulzzD0cl2nqTzKk3OpP8LJL2ZXONeSevVi5ylymJYrh+u\n1N7H1wZfFxgLdKnH6/cEupFalxy4Fp/NBXAhcE2yvwO+cmBTYOtE7owV9xqwe7L/BHBwieVsA3RL\n9jcE3gG6VKisLZLtOsCrwB4VKucFwD3AkEr9d0/6nQxsmlNXUbIC/wROTfabAptUmow58jbBVwvd\nqtLkBNom/+brJeX7gZPrQ86SP+gy/MPtBTyZKl8EXFjPMnSkpsJ4G2id7LcB3s4nG/AksGfSZlKq\n/ofALWWW+d/A9ypZVqAFMBLYvdLkxGfsDcN9cBmFUVEypvqdArTKqasYWYGNgQ/y1FeMjHlkOwh4\nsRLlxBXGR8CmuBIYUl9/69UwJJUb3DeVhg/u+5Yls7nMbCbwraS+UCBiO1zuDGW9B0lb41bRq/h/\noIqSNRnqGQPMBIaZ2RsVKOf1wK+AtJOv0mTMYMAwSW9IOqMCZd0G+EzSnclwz62SWlSYjLkcBwxM\n9itKTjObDlwHfJxcc56ZPVMfclaDwqgGKmbmgKQNgQeB881sASvL1uCymtkKM+uOf8XvIWlHKkhO\nSYcBs8xsLIWnhEMFPMuEfcysB3AocI6k71JBzxP/Cu4B/DWRcyH+1VtJMn6DpHWBvkAmB2xFySmp\nJZ5WqSNubWwg6Ud55Cq5nNWgMKYBHVLlgsF99cgsSa0BJLUBPk3qCwUiFh2guCZIaoori7vN7NFK\nlhXAzOYDzwN9KkzOfYC+kiYD9wG9Jd0NzKwgGb/BzGYk29n4UOQeVNbznAp8YmYjk/JDuAKpJBnT\nHAKMMrPPknKlyfk9YLKZfW5my4FHgO/Uh5zVoDDeADpL6ihpPXycbUg9y5AbfDgEOCXZPxl4NFX/\nw2TGwjZAZ+D1xDycJ2kPScKDFR+l9NyBj0neUKmySto8M3tDUnPgQOCtSpLTzC4xsw5mti3+/+05\nMzsRGFopMmaQ1CKxKpG0AT72PoHKep6zgE8kbZdUHQBMrCQZc+iHfyhkqDQ5Pwb2krR+0v8BwKR6\nkbMcDqMyOKD64LN+3gMuqudrD8RnSyxO/qFOxZ1NzyQy/QdomWp/MT4L4S3goFT9rvgf8nvADWWQ\ncx9gOT6LbAwwOnlum1WSrMDOiWxjgfHApUl9RcmZusZ+ZJ3eFScj7h/I/JtPyPx9VJqswC74x99Y\n4GF8llRFyZj03wKYDWyUqqtEOfsn1xwP3IXPIC27nBG4FwRBEBRFNQxJBUEQBBVAKIwgCIKgKEJh\nBEEQBEURCiMIgiAoilAYQRAEQVGEwgiCIAiKIhRGUHFIWiHpX6nyOpJmK5tm/HBJv15FH1tKGpzs\nnyzpxjrKcHERbe6UdFRd+i0lkoZL6tFQ1w/WPkJhBJXIQmAnSc2S8oGkkqeZ2VAz+0NtHZjZDDM7\nNl1VRxkuqWP7qkLSOg0tQ1B9hMIIKpUngMOS/RqpGtIWQ/KVf4OklyW9n/niT1LJTEj11yH5In9H\n0mWpvh5JsrxOyGR6lXQ10DzJrHp3UneSsosU3ZXqd7/ca6dJ5JiUZGh9U9JTGUWYthAktZI0JXV/\nj8gXw5ks6Rz5gjmjJY2QJ5/LcFIi03hJuyfnt5Av/PWqpFGSDk/1+6ikZ/GI4CCoE6EwgkrEgEFA\nv+Tl2hVf6CW3TYY2ZrYPcDi+iEy+NrsDP8BTVByTGso51cx2T46fL2lTM7sYWGRmPczsREk74BZH\nL/Msu+cXce00nYEbzWwnYB5wdC33nWFH4Eg8keDvgQXmmV5fxXP+ZGieyHQOnksM4FLgWTPbC+gN\nDEjydgF0B44ys/0LyBAEBQmFEVQkZvYmvjpYP+Bxak8z/u/knLfIrgGQyzAzm2tmX+O5jHom9T+T\nNBZ/EbcHvp3Up6/XG3jAzL5IrjO3jteeYmYZa2dUcl+rYriZLTLPmDoXeCypn5Bz/n3J9V8ENpK0\nMZ6A8CL5miPPA+uRzfg8zMzmFXH9IFiJpg0tQBDUwhDgj/iqd5vX0m5xar+QYllprQBJ++HKYE8z\nWyxpOLB+HWUs5trpNstT11hG9qMt97rpcyxVXkHNv9t8ayAIONrM3ksfkLQX7h8KgtUiLIygEsm8\neO8ALjeziatxbi4HSmqZDM0cCbyMZ0z9IlEWXfDlgDMsSTmGn8OHsTYDkLRpHa9dqP5DYLdk/5gC\nbVbFcYlMPfGV174EngbO++biUrfV7DsIahAKI6hEDMDMppnZTcW0raWc4XV8KGosPrw0GngKWFfS\nROAq4JVU+1uBCZLuNrNJyfEXkmGe6+p47UL1A4CzJY3CU1MXorZ+v5Y0GrgZOC2pvxK/r/GS3gSu\nqKXvICiaSG8eBEEQFEVYGEEQBEFRhMIIgiAIiiIURhAEQVAUoTCCIAiCogiFEQRBEBRFKIwgCIKg\nKEJhBEEQBEXx/wHqoxxPkjxZ7wAAAABJRU5ErkJggg==\n",
|
|
"text/plain": [
|
|
"<matplotlib.figure.Figure at 0x2ec09b867b8>"
|
|
]
|
|
},
|
|
"metadata": {},
|
|
"output_type": "display_data"
|
|
},
|
|
{
|
|
"data": {
|
|
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYwAAACfCAYAAADqDO7LAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztnXecFEX2wL8PBAEJZkwkUcwJFT0Dcnh6eCqcZ0D0zPDz\nDOeZ03mip57hzGLCiHoGjIAeigGUMyKgoCIiWUUEyVFg3++PV830zM7M9i6zs7Pyvp9Pf7q6urrq\nde9sv656Ve+JquI4juM4FVGnpgVwHMdxageuMBzHcZxEuMJwHMdxEuEKw3Ecx0mEKwzHcRwnEa4w\nHMdxnES4wigxROR+Efl7ZcuKyEEiMr16pVvd7mQR6VyMtmobIlImIlsX+9o1Jd52ZX6DWepZKCKt\nCymbUzq4wigSIjJFRJaJyIYZ+aPDP2tLAFU9S1VvSFJnlrJVWlQjIq2CDP57YI2fx5osbMp5rYgM\nE5GlIrJARH4SkRdFpPkatJWz7aS/QREZKiKnp1Wi2kRVpxRQrqitKSKyJNz/wrC/u9DtOPnxF0Tx\nUGAy0CPKEJGdgYas2UumEEiQQaq9IZG61d1Ggajq32RNnmG+axU4W1WbAu2A9YE7slZSNUVX7X/7\nNUSBw1W1aVBKTVX1vGwFs/3GKvu7q0W/06LiCqO4PAmcEjs+BegXLyAij4nIP0P6IBGZLiIXishM\nEfleRE7NVjaVJVeIyCwRmSQiJ8RO/EFERonIfBGZKiK9Y9e9G/bzwpfbPuGaXiLyVcj7QkR2j12z\nh4h8LiJzReQZEamf7YZF5BQR+Z+I3C4is4DeItJbRJ6MlUn7og9frv8M1y0Qkdcze2axa78SkT/E\njuuGL/DdRWRdEXlSRGYHOT8WkU2y1ZMUEdlbRD4I9X0vIveIyDoZxQ4XkYlBjlsyrj89yPyziAyO\nepZJmwdQ1XnAi8DOoc7HROQ+EXlNRBYCnUSkvojcGv7WM8L5dWNyXCIiP4jIdyJyGjEFmfm7EpFu\noSc8X0QmiMihInI9cCDQJ/61L+lDW01F5InwHCZLbJgr/C6Gi8i/RWROeF5dktx/ucz039hs7DeW\nLU9E5Cqx3sqPIvK4iDQNdUS/wdNFZCrwdtI/ytqEK4zi8hHQRES2Cy/H7sBT5P+62wxoAmwB9ATu\nFZFmecpuGMqeCvQVkW3DuUXASaraDDgc+IuIdA3nOoZ90/Dl9rGIHAtcDfw5fNV2BX6OtXUscCjQ\nBtgttJeLfYBvgeZANNSR+QWfedwDU6ibAOsCF+eo+2nghNhxF2CWqn4Wrm8KbIk9l78AS/PImYRV\nwPmhvt8AnYGzM8r8EWgftm4Shm1EpBtweTi/CTAceKayAojIxsDRwKhYdg/gOlVtArwP3AxsA+wa\n9ltif0/Ci/lC4GBgW+B3edrqgH3UXBR+Ox2BKap6VZD/3Iyv/fjfsQ/2220NdAJODsopogMwDtgI\n+DfwSGWeQwbRb2xTUr+xzLzTgJOBg4Ctg2x9MurpCGwP/H4NZPnV4gqj+ES9jEOwf5YfKij/C/Yi\nWKWqg7EX/3Y5yirwD1VdoarvAa8BxwGo6nuq+mVIfwE8i/3jxIkrrjOAW1R1VLhmkqrGjep3qerM\n8LU7CIj3PjL5XlXvU9UyVV1ewf1GPKaqE0P5/nnqfwboKiINwnEPUi/hFdjLqJ0ao1V1UcL2s6Kq\no1T1k1DfNKAv5Z/jTao6X1W/A+4kNQx5JnCjqn6jqmXATcDuItIiYfP3iMgcYDT2u7kodm6Aqn4U\nZFwO9AIuCHIsDm1FchyLPd9xqroUuCZPm6cDj6jqO6HuGar6TZ7yAquHxboDl6vqElWdCtwGnBQr\nO1VVH1VzaNcP2ExENs1T9yuhNzI37M+Incv2G8vMOwG4XVWnquoS4ArgeEkN4SnQW1WXVuJ3ulaR\n2ZV2qp+ngPewL/MnEpT/ObxcIpYAjXOUnauqy2LHU7HeBmLDTDdiwxj1w/Z8nnZbABPznJ+ZIdPm\necpWZfbWjxn1Z71nVZ0oIl8BR4rIq1hP6Opw+klgK+DZ0Ct7Cvi7qq6qgjwAhB7b7cBemP1pHWBk\nRrHvYunVfwOgFXCXiNwWVYe9pLYk2TP6q6o+muPc6uvDsFsjYKTI6m+AOqQ+CLYAPs2QMVcvtwX2\n4VFZNsaezbSMdraMHa/+G6vqUjFhGwM/5aizm6oOzXEu2/PLzNsiyBCXZx2s5xvxHU5OvIdRZMJX\n6WTgMOClAle/gYg0jB23JNWD+Q/wCrClqq4PPEjqJZHNwDsdaFsguTLrX4y90CLyKZskPIt9PXYD\nvlTVSQCqulJVr1PVnYD9gCOxIYk14X6sZ9g2PMe/U/5lG+8xtCL1N5gOnKmqG4ZtA1VtHPUM1pD4\nM56NKdmdYm2tH4aUAGZkkTGXkT/f7yDfxIDZWA+vVUY73+e5piIqmhRQUd4PWeRZQfrHT01PQClp\nXGHUDKcDncNwQCER4FoRqSciB2K2iv7hXGOsB7IijEvHx/1nAWWkvxgeBi4WkfYAItK2EkMnFfEZ\n0FFEWoQv/8vXsL5nMXvKWZhNAwAR6SQiO4chh0XYy6EsexXlEKBBMJxHm2Dj3gtUdYmIbB/azOQS\nEVk/PK/zgnwADwBXisiOQb5mInJM5W83P2GI5yHgztDbQES2FJFDQ5H+wKkisoOINCLVI8vGI8Bp\nIvLbYDTeQkSiIdGZmC0gmwxloZ0bRKSxiLQCLsB6fTXFM8AFItJaRBpjdo1nYz34Up8pVuO4wige\n8XnukyPbQOa5ytSThRnAXOxL6knsa3ZCOHc2cJ2IzAeuAp6LybMU++d5P4wNd1DVF0Le0yKyAHgZ\nM/RWVt7yN6D6Vmh/DDACs4GkFalkfT8CHwL7ErsvbBLAC8B84EtgKOGFJbY47b581QILsS/1pWH/\nW8xucGJ4Jg+SUgbx6wZgw1Sjwr09GuR8BbMlPCsi87D775JxbT55KnPuMszg+1Foawg2HRdVfR2z\nrbwDfEOeGUGqOgIzFt+JPcdhWM8V4C7gWLEZX3dmkeU87LlNwoZhn1LVxyp5H3EGhRlZ0fZiBeUz\neRT7+7+HDbcuCTImbX+tR6o7gFKYkXEnppweUdWbM86vj/0h22L/mKer6lfVKpTjOI5Taaq1hxGG\nAvpgU9R2AnqEbnycK4HRqrobNnvIV286juOUINU9JNUBmBCmsa3Auu/dMsrsiHWNUdXxQGtZw8VV\njuM4TuHJqzDEVs3mmsaWhMzpgt+RPq0O4HPgT6G9Dtj46FZr0KbjOI5TDeRdh6Gqq8Jy+WaqOr+a\nZLgJm5s+ChiLLUoqN09eRNwg5TiOUwVUtSAzwJIMSS0CxorIIyJyd7QlrP97UjMqwHoOafOwVXWh\nqp6uqu1V9RRsGf+kbJVpt27ovHmoasluvXv3rnEZXE6Xs7bK6HIWfiskSRTGS8A/sKloI2NbEkYA\n2wTHXvWB44GB8QJhLnq9kO4FvKu53DcMGAD335+wacdxHKeQVOgaRFX7hZd9u5A1Xs2AXSFqQ1rn\nYnPAo2m140TkTDutfYEdgH4iUobNlT8jZ4UnnwxLliRp2nEcxykwFSoMEemEOQabgq2EbCEip6g5\nt6sQtUVC22XkPRhLf5R5PicdO8Lw4YmK1hSdOnWqaRES4XIWltogZ22QEVzOUqbChXsiMhI4QW3K\nKyLSDnhGVfcsgnxxOVSHDoV//KPklYbjOE6pICJoEY3e9SJlAaDm2rheIRqvNG3bgri7F8dxnJog\nSQ/jUcxh21Mh60SgrqqenvuqwiMiWmiLv+M4zq+dQvYwkiiMdYFzgANC1nDgPi1ygBFXGI7jOJWn\naApDLBD6E6p6YiEaWxNcYTiO41Seotkw1CKTRWsoSgNV+M6DYjmO4xSbJCFaJ2FxEgZikdIAUNXb\nq02qfJSVQYsWMGoUtG9vx24IdxzHqXaSzJKaCLwayjaJbTVD3bq2339/2w8ZUmOiOI7jrE0ksWHc\nrKoXV7mBigMoNcVmYLUE6gK3qerjWepJ2TAyexTey3Acx8lKIW0YSbzV7l/VymMBlA7GwoaOEJEB\nqvp1rNg5wJeq2lVENgbGi8hTqroyZ8WPPWa2jF12gfXWg2bNYP58VxqO4zjVSBIbxmfBfvE86TaM\nlxJcuzqAEoCIRAGU4gpDSQ1xNQF+zqssAE49NZX+9FNYuBBmzIAttkggkuM4jlMVkiiMBsDPQOdY\nnmJebCsiWwClDhll+gADReQHoDHQPUG9Kfbay/YPPgiNG0ObNnDssTBzJtSrBxtsUKnqHMdxnOwk\n8VZ7WjXL8HsspndnEWkLvCkiu2ZzcX7NNdesTnfq1Cnd+dc//2n799+3ffPmsOuu8Pnn1Sa44zhO\nqTFs2DCGDRtWLXXnNHqLSH9VPS6kb1bVy2LnhqjqoRVWLrIvcI2qdgnHl2NuzW+OlXkVuFFV3w/H\nbwOXqeqnGXXlX7i3cqX1KBo3hkUxXeMGccdx1mKKZfTeNpY+BLgsdrxJwvpXB1ACZmABlHpklJkK\n/A5b69Eci7uRNeJeXtZZB665Bho0gDp1YORIeO45VxaO4zgFIl8PY5Sqts9MZzvO24BNq72L1LTa\nm+IBlERkc+BxYPNwyY2q+kyWeirnGqSsDKZNg9atk1/jOI7zK6MovqRE5GusN1AHWydxAhZASYCn\nVHWHQgiQFPcl5TiOU3mKpTCG5rtQVX9bCAGSUiWFsXIlLF5s6zQcx3HWQorq3rxUqJLC6N3bZk89\n9BD06uUGcMdx1jqKHXGv9hJNu+3Vy/azZtn+22+hR6bt3XEcx8lHkoV7tZcdd7T9BhtAt27QsGF6\nD+OZcrZ1x3EcJwe/7h5G8+Zw770wZYr5n2ra1PIfe8z2CxbUmGiO4zi1jUQ2DBHZEmhFrEeiqu9V\no1zZZFjzWVKDBsHbb8Odd0K7dvD887DbboUR0HEcpwQpmrfa0NjNmH+nr4BVIVuBoiqMgnDkkbaB\nDVfNn1+z8jiO49QiKuxhiMh4YFdVXV4ckXLK4eswHMdxKkmxZ0lNAupVtQER6SIiX4vINyJyWZbz\nF4vIaBEZJSJjRWSliKxf1fYqxbXX+jRbx3GchCTpYbwI7Aa8DazuZajqeRVWbgGUviEWQAk4PiOA\nUrz8EcD5qvq7LOcK38PYbz/48ENo2xYmTrSgTBE//QSNGpkzQ8dxnFpKsXsYA4HrgA+AkbEtCasD\nKKnqCiAKoJSLHkDx5rp+8IHtJ060/QMPwPKgE5s3h44diyaK4zhOqZN0llR9zIsswPjw8k9y3dHA\n71X1/8Lxn4EO2XonItIQC7DUVlXnZTlfPTaMc86x9Rm33WbHzZpZ9L7jjoNly+DNNwvfpuM4TpEo\n9iypTkA/YArmeLCFiJxSDdNqjwT+l01ZROQNoFRV7r3X9hdeCFtuaTOnGjVKL7NkifmlitZxOI7j\nlCg1EkBpdQGRkcAJqjo+HLcDnlHVPSusPEEApVjZl4D+qvpsjrqqf5bU4sVw9tnwxBNw8cVw662W\n37at9Tb22gv+/W/46itbOe44jlPiFNX5oIiMUdVdK8rLcW1dYDxm9J4BfAL0UNVxGeWaYbOxtlLV\npTnqKv602lWrLDBTNnyKr+M4tYCiDkkBn4rIw1hMDIATgU/zlF+Nqq4SkXOBIaQCKI2LB1AKRf8I\nvJFLWdQYP/xQ0xI4juOUDEl6GOsC5wAHhKzhwH3FXshXYwv3rrjCPNvusAPUr2/pI46AE04oviyO\n4ziVxONh1BSvvmo+qFq3hunT4cEHzRPulVeajWPddWtWPsdxnAyKsg5DRPqH/VgRGZO5FaLxWscR\nR5jC6NQJttkGRoyA+++3c1tvXb58795mKM/khhtg++2rVVTHcZxCky9E6+aqOkNEWmU7r6pTq1Wy\n8vLUfA8j4t13TWmcey706QM77wxdu9rMqg02SJXbbTcYM8YM5LNnw8KF0KYNbLQRzJnjhnPHcaqd\novQwVHVGSJ4dVmqv3oCzC9F4reWgg+xl3y6sZRw7Fv71L9hww1SZpUth330tXVZmvZOoFzJnDmy7\nbXFldhzHWUOSuAY5JEveYYUWpFZyxhnlV4JPnw79+9vivwcftMWA06aZwTzimGOgb18cx3FqE/mG\npM7CehJtgW9jp5oAH6jqidUvXpo8pTMklY3582HAADjlFPNB9d571rOYOdOUxahRcMgh8N13pkQc\nx3GKQLGcDz6NuesYEPbRtmexlUWtoFkzOP54S0dDT/36wWabwcknw2GHmZPDrbYyZRLx0UdmE3Ec\nxylx8tkw5qvqFOAuYE7MfrFSRPYploC1imjY6fHHzaFhu3Zm63jtNTjpJGjZ0s5fcknqmoEDoXt3\nuOMOeP1164Hk44EHYEUi34+O4zgFJYkN435gUex4UchLREUBlEKZTiGI0hciMjRp3SXJ1KmmIJYs\nsXgbP/5o+TvsYG5G/vpXePllmDfPgje1bw/bbWfODw87DFq0gE9zLKRftQrOOgtmzSre/TiO4wSS\nuAZJMx6oapmIJLkuCqDUh1gAJREZEA+gFPxI3Qscqqrfi8jGlbqDUqNly1RPAmxI6pln4Oij7fju\nu20/bZrtx4xJH6IC2Hvv7AsBI/vHFltUj+yO4zh5SBSiVUTOE5F6Yfsb5igwCUkCKJ0AvKiq3wOo\n6uykwtcKRMy2US8jyu1WW9n+ySdTeX//OzwVXHbde6/1JCLFAqYwousi3ngDnn46FfjJcRynmkii\nMP4C7Ad8jwU42gf4v4T1bwlMjx1/F/LitAM2FJGhIjJCRE5KWHftpk4dG2KaPBm++cbyrrvO3Ka3\namUrwbfc0tI//ACffALjxtnCvzg33wwnnggNGpRvY/FiCwblOI5TACocWlLVn4Djq1mG9kBnYD3g\nQxH5UFW/zSxYLQGUapI6QV9vuy18/731Rho3hilTLB0Rn4bbu3d6HQsWpNI//QSbbpo67tkTnn3W\nV5Q7zlpEjQRQEpFLVfUWEbkHKFcoW5jVLHVUGEApGMIbqOq14fhhYLCqvphRV2mvwygky5bZLKs4\nRx1l03VPPx122slWkjdoYKvLzzrLDOlffmmKZvx4C/rUsSN8+CHMnQvrr185GVTTlZbjOLWSYq3D\niIIcfQqMzLIlYQSwjYi0CnHBjwcGZpQZABwgInVFpBE25DWOtZkGDWCXXUwJDB5seRMmWNzxyGlh\nw4ZmTF+1Cq6/3qIAgtk5tt8e7rsPNg7zB+L+reIsXw5HHlk+f8QI6/2UlRX2vhzHqdXkHJJS1UFh\n36+qlScJoKSqX4vIG8AYYBXQV1W/qmqbvxrGBIfAK1eabaNxY7jggtQwFlhMjr32sp6Aqp279lo7\nt/nmcNddMGiQHY8cCUOGmI1kxx0tb/p0c9m+556233xzy3/8cdsfdZStXnccxyH/kNQgsgxFRahq\n1+oSKhtr1ZBURbz0Umqa7rBh5gwRTHGccYb1LqLQsvPn27DVXnvZ+o4HHoC//MXOvfMOdO5s6X79\nbEU6WCzzF1+0Vei5nrmIzeBq0aJabtFxnMJQlABKIhLeQvwJ2IxUiNYewExVvaAQAiTFFUYGU6fC\n+++nR/4TsaGqJUss3batDWVFvZLttzffVnPn2vHuu5td5KWX7Dh6vlddZdOAr7nGpvmemMUTjAi8\n/XZK4TiOU5IUy735u6r6LrC/qnZX1UFhOwE4sBCNO2tAq1blw8SOHAmff546njjRXuyRK5Hddksp\niyOOgM8+s57EBUH3f/SR9Uh+/3u46CLLe/nl7O1362ar1cEUzPz5poyWZoRlnzLFZ2k5zq+EJOsw\n1hOR1eHkRKQNNv3VKTXat0/F2Rg82PxUgQ1PvfuuLRLs2tWUxtixqetuv90M6L/5jc2m6tjRbCav\nvmrG98zZUg0b2vqRAQPMMH7DDXbdZpvBQw+ll91jDxhaSW8vAwemKz7HcUqCJC4+LgCGicgkQIBW\nwJnVKpWz5nTpkn7csaPtIyP2/ffD6NGp80OGlK/j8MOtJwKmND791Ia1li2zeB+ffgrPP59+TTxU\n7axZ5k9rzpzsMi5darO8GjdOz+/WzZTXBx/kv0fHcYpKkoV7r4vItkAUhPprVXU/FLWdww6zLSKa\nIfXJJ9C8eSp/u+2gbl2bttuzp82cArjpJnOYePzxsGgR/PKLGdu//RaGD4cDD0wtIvzvf22/887p\nPZsePcxWkql0wGOeO04JktPovbqArY24EGilqr2C8thOVV8thoAxOdzoXRNEz7xOnfL5xxxjNpAJ\nE2CbbSx/993N1cmjj5pC6N4dnnvOznXpYkNlN95oimbrrU0xjBuXqnPaNGjdGn7+OT3k7eLFsF6O\nkdDJky0eSby84zhA8RbuRTwG/AL8Jhx/D1xfiMadWoBIavV4FKM8UiJ77WU9jUhZAFx9tQ01RTOr\nnngC/vc/q+OQQ8zwfuWVqaGrr79OrWrv1cuUxXvvWc9jenBDtmKFDVtFa1My2Xpr2Gijgt624zjl\nSaIw2qrqLcAKAFVdgtkynLWJdu3sqz/O5ZfDZRkhTv7wB9tvvrm94OvXTy0s7N/f8qIhrz32sP2y\nZbaPVqRvtBE0bZpyEz9/vu27d0+1c+aZZmyfPdu8/GZj0SJfre44BSSJwvhFRBoSFvGJSFsgsQ2j\nogBKInKQiMwTkVFhuyqx9E5xGTo03QaRjQYNrFcxZIjNsAJ7+ffsaSvPwVy3g8U5f+cdS4uYApo9\nO7USHUzRbLKJpSPD/cyZ0Lev1XfRRSmlMG9eqlcC0KRJdi++juNUDVXNuwGHAO8Cs4D/AFOAThVd\nF66tA3yLzayqB3wGbJ9R5iBgYIK61PkVMHmy7X/+2fZLl6qaWrAtYto01ZUrLQ2ql1+eOhcvv9FG\nqoMHp+fNmqW6xRap47Iy1WOOUZ0xw+pcsCC3fLNmqa5YUdBbdpyaJLw7K3xfJ9ny9jBERICvsdXe\npwLPAHup6rCE+ihJACXwIa61h9atbR8ZqBs0sNd6z57p5Vq0gLfeSk3rveYa27rGPNI88ogNk+28\nc/riwD59LIbIK6+Y+/e5c+GFF2yY7Oijrcfzyy/Z5dt00/Q2Tjih4l6V46wl5J1Wq6oqIv9V1V2A\n16pQf7YASh2ylPuNiHyGGdQvUXc+uPax887l8xo1Sq3FWHfdlGNFMLftw4dbOopCePPNNrR16aV2\nPHaseeO94YbUddE6lMWLzb6SScuWKQ/BYAsaX365/Ap2x1kLSbJwb5SI7K2qI6pJhpFAS1VdIiKH\nAa9gUfjK8asLoOSkOO882+LUq2e9gylT0vOHD4cOHcxXVtz54aWXmjuUSy81O8lZZ9kK9AULrI6o\nd/Pii+ku36PeybJl5syxTRubGtyuXSo/G88+a/HVI9tKNmbPTrmZd5wiUJ0BlJLYIb7G3I5PxFyQ\njwXGJBnvAvYFXo8dXw5cVsE1k4ENs+QXakjPqS0sX242iMGDk1+zcqXq1VenjuO2kZYtU+lXXzWb\nRrNmlnfllba/8UbV9u0tfcABtt9tt+xtZdpdcpX56KPk8jtOgaGANowkC/da5VA0UytSRiJSFxgP\nHAzMAD4BeqjquFiZ5qo6M6Q7AP1VtXWWurQiWZ1fITNnml2hqtH/Fi+2WVRNmqTyVMsvRIwYO9am\n/153XSpv5Ur4979tCnFcjig9erT1SurVs2G0OB07Wl2RC/p8rFhha1omTbLV9Y5TAIrl3rwB8Bdg\nG6xX8Yiqrqx0AyJdgLtIBVC6KR5ASUTOAc7C1nksBS5Q1Y+z1OMKwykcuRTQvHm2kHDdde34lVcs\nVvr//Z8dH3FEKijVH/8Ir78Of/0r3HmnTf+dMcPis2+xhZWJ6oo8++ZjyhRzqRKfGuw4a0ixFMZz\n2Et8OHAYMFVV/1aIRquCKwynoIwda+tKRo+21enLl5f3X3XjjXD22fa1H++hLFxoMUTGjIFdd4VT\nTkm/7sMPbVX8N9+YL644Bx9stpobbzR38vfcA+eea+eGDjVZDj7Y7CbXX58KhAUWfjcy8DtOQgqp\nMPLZEsbG0usAowo1DlaVDbdhODXJhAkpm0XcvvHBB6m8+++3/RFH2DXduqn27696yinp12ZuEXff\nnbKfgOqZZ6bLAKqXXlr99zptWvW34RQNirQOY0VMqVR6KMpxflVss429xs85x45btrRwt1H8EVUL\nffvWW3DHHZY3YEAqAiJk93d1663mT+uhh6znMXFi6tyDD6bSkR+tmTPTrxcxNymFZMcdkw2hOWsd\n+RTGbiKyIGwLgV2jtIgsKJaAjlNS9Olj+5EjbXho443TFw0efLAZud9+244PPdTC3M6YYVNsFy40\nV/H9+pmR+5hjYIcdzEZywQU25ffjj1NTjLt0sfrbtYPTTzfF8tZb6W0emCUA5i235HbWWBEtWxbG\njvLdd+buxfnVkC9Ea11VbRq2Jqq6TizdtJhCOk5JceGF5r23WbPs55cvN5fvTZrY4sD69S0aIZjX\n3R12gJNPNvtE5EsLLGTuwQfbGpObb7a8N96wa487zla2T5tmXn/r1DE5AE46yfxyxbnsMvjXvyz9\n9NNmMwHrzYik917AFNj771t6q63sZR+xcqXdk4jNIJs922afLVtmvrxmzMj+HEaPNntOZRgwwJSq\nU5oUamyrujfchuGUEsccY+tEshHZIPbdt+J6ctkzsp1fuVL17bcrtoXEr4unH3ssvfzixdnL77ij\n6k47WTrTTxeoXnxx+byePVXPPz9dhptuMpvMJ59U/Bzicvi6lYJCAW0YNa4IEgvqCsOpLWy8sa52\nelgRCxaoTp+uOnas6jfflD//ySeqbdqotm1rx19+qVq3ruXNnq26cKFqnTqpF3nEXnupHnSQ6tCh\nuRXMo4+qrrOOle/a1fLmzlW98MKU8vjhh/RrOnfOXV+0DR1q1+66qx3/4Q/JntuSJVY+ckxZVcrK\nzImko6quMByntCkrs1XkxWzvkkvs3/nNN20/darqb3+r2qSJvYi//dbyjz7aFM2BB6q2aqXaqJHl\nT59u9YC96IcNU73tNtV+/VQ320x10CBr6+GHU4ph9GjVxx8vrzAuu8zKgurWW9t+xQpTPhdemC77\nkiWqL7+EctH5AAAPnElEQVRs6csvT65o8/HssymF57jCcBwng4kT01/aixaZUoj/33z9teqyZZZe\nZ5308kuXpnoTY8ao3nJL6lycVatU999f04a0pk5V/ewze/m3bJkagjr6aNVRo6zs8OGq99xj6Tlz\nUvVFbbRpk2pz+HDVd95RbdAgve1XXlEdMsSG5pYssd6VqskdUVam+sIL5RXPQw+pPvBA1Z9vLaYo\nCgNYCCwI28LY8UJgQeIGoAvmj+ob8viRAvbGpvL+Kcf56niWjvPrYexY+5c+9lg77t4995f2/Pnp\nCkNV9frrU+lffrH0lClrLlevXqaEXnnF6hw8WPWaa9LbB9UNNjCFBqq/+Y2u7plERP6+ol4RWDyU\n+D2+9lrq3O23p/Lj99mnj+oXX+SWd9y4VK8nCatWJS9bA9SaHgYJAijFyr0NvOoKw3HWgJtusi9+\nVdV582zBYT7+8Q/Vq66y9IwZZhhfU8rKzNYSBcBSNdvIqlWqI0emehpgw1ALF+rq4TLVdCXyzjup\nOnr2LK9k4oqgrEz1sMNSeePHp64Fs/2cf77qJpvkVwiZPatJk1TvuCP1XLOVf+qpyj2jIlJ0hQEc\nAJwW0hsDbRJety8wOHac1Vst8DfMn9SjrjAcp5Zz6632ajnttFRe/CV8ySWqW22laT2IceNsmElV\n9cgjVc85x4bJVM3m0r696jPPqN53n9lYDj7Yrt9pJ1uZPmWK6nrrpbfzwgup4S8whbjddqq77KJ6\n7rkpBZVJpsJ45BE7/sc/cpfPXJUfZ/Hi1FBgVbn77iob8ouqMIDewCDgm3C8BfB+osrhaKBv7PjP\nwN0ZZbYAhob0Y64wHKeWc8YZmtNWoap63HGq//lP7utFUmVbtUpde955lrdkib2A+/a1WWaRnSTz\nRR/1WiK7iKpq48bpZSdMsOvjzJ1rQ3JjxqTL3rOnHS9dqnrWWentnHCC9Z7i9O1rMmezBVWWeA+s\n0pcWTmEkCaB0FLAHMCq8tX8QkSb5L6kUdwKXxY5zOsnyAEqOUwt46CHo2zfdhfz06bZQcfFiW/yX\nGSwrzqpV9ooFmBqLohD9/zdsaPtevWw/IsR2u+SS1OLDWbNs/+KLqWuefx4WLYLHHzeHj4sWwZdf\nmtfhc8+1xZL16tmCy4cfNlcvAOutZwsXH37YnElOm2bpO+4wT8QDBsCTT8Kee6YcWR5+uHkxXr48\n932WlaXc2C9ebIsrr78+de/x5xHdy6hR0L597jqp+QBKn4T9qLBfjwIGUAImhW0yZlD/Eeiapa4q\naVfHcUqMLl1smCkJkL83omoLKDPLlJWZYT1aCzJjhuobb6S+9I8/PjWsFd+aN7fhpRNPTM+/7rry\nZR95xGwXM2faYkdQbdGifLn771f9179Uv/rK2r7zzvL19++f3pvq1SvVY4mmCR9yiOoTT1he586q\nI0aY0b/CR1i4HkaSl/7FwIPhpd4L+BD4a6LKoS4po3d9zOi9Q57yPiTlOE7hKCtLn14bpRcsMGP7\naaep7rNP+sv74ottOC063m8/M9hHx9Gal2h77jnVzz+3faZn4ttvT0WBBDO2x8/vu2/5xZHxbf58\nk/XBB1M2GtX0MtFQXQ4KqTAqHJJS1VtF5BBsSm074GpVfTNh72WViJwLDCEVQGlcPIBS5iVJ6nUc\nx0lEZqCs6DiKb/Loo6lzH39scUz+/GeL+T5njg1trb++Da9p7PU0Z4759gKLK7/rrrZ1726OJDt1\ngm7dUhEYo/1RR6XqqF8fPvggXcYFC+C118yz8aGHQu/eVu7mm81j8kUX2VBWnFy+vKqBCkO0AojI\nZkAH7IU+QlV/rG7BssigSWR1HMepEitXmrPEbN5/czF3rimXilixwl78YPtffoEvvoCddrK8n382\nxRQPzXvOOXDffZYuKzPFMmgQdO1qsv70E1x6qTmOzBVymCJF3Is11hO4GngHM0gfBPxTVR/Ne2GB\ncYXhOE6tRhV+/BGaNjVDehKi3kf07ps6FVq3tkkECaMvFlthjAf2U9Wfw/FGwAequl3eCwuMKwzH\ncdY6xo61WPGRe3wwJXLRRRZ8KwGFVBhJptX+jM1eilgY8hzHcZzqZJddyuc9+aTZSGqAnD0MEQnR\nWdgd2AUYgNkwumHTak8thoAxebyH4TiOU0mK1cOIFudNDFvEgEI07DiO49QuEs2SKgW8h+E4jlN5\nimrDEJFNgEuBnYAGUb6qdi6EAI7jOE7tIPfk3RT/weJZtAGuBaYAI6pRJsdxHKcESaIwNlLVR4AV\nqvquqp4OJO5diEgXEflaRL4RkcuynO8qIp+LyGgR+URE9q+E/CVHtTn9KjAuZ2GpDXLWBhnB5Sxl\nkiiMFWE/Q0QOF5E9gA2TVC4idYA+wO+xIa0eIrJ9RrG3VHU3Vd0DOAN4OJnopUlt+RG5nIWlNshZ\nG2QEl7OUSbIO43oRaQZcBNwDNAXOT1h/B2CCqk4FEJFnsWm5X0cFVHVJrHxjIMNRiuM4jlMKJHE+\n+GpIzgd+CyAiSRXGlsD02PF3mBJJQ0T+CNwIbAIcnrBux3Ecp4hUaVqtiExT1ZYJyh0N/F5V/y8c\n/xnooKpZo6eIyAFAb1U9JMs5n1PrOI5TBYrpGiQbSRv/Hogrlq1CXlZU9X8isrWIbKiqczLOFeSG\nHcdxnKqRxOidjaRf+yOAbUSklYjUB44HBsYLiEjbWLo9UD9TWTiO4zg1T84ehogsJLtiEKBhksoT\nBlA6WkROBn4BlgLHVfIeHMdxnCJQa1yDOI7jODVLVYekikpFi/+que1HRGSmiIyJ5W0gIkNEZLyI\nvBGmHUfnrhCRCSIyTkQOjeW3F5Ex4R7urAY5txKRd0TkSxEZKyLnlaKsIrKuiHwcFmqOFZHepShn\nqL+OiIwSkYGlKmNoY0p88WspyioizUTk+dDmlyKyTwnK2C48w1FhP19Ezis1OUP9F4jIF6GN/4hI\n/aLIWajg4NW1YUrtW6AVUA/4DNi+iO0fgLl4HxPLuxm4NKQvA24K6R2B0dhQX+sgd9SL+xjYO6T/\ni80eK6ScmwG7h3RjYDywfYnK2ijs6wIfYVOtS1HOC4CngIGl+ncP9U4CNsjIKylZgceB00J6HaBZ\nqcmYIW8d4AegRanJCWwR/ub1w/FzwCnFkLPgD7oa/nD7AoNjx5cDlxVZhlakK4yvgeYhvRnwdTbZ\ngMHAPqHMV7H844H7q1nmV4DflbKsQCPgU2DvUpMTm9H3JtCJlMIoKRlj9U7GXPjE80pGVmyx78Qs\n+SUjYxbZDgWGl6KcmMKYCmyAKYGBxfpfrw1DUtkW/21ZQ7JEbKqqMwFU9Udg05CfKev3IW9LTO6I\nar0HEWmN9Yo+wn5AJSVrGOoZDfwIvKmqI0pQzjuAS0if+FFqMkYo8KaIjBCRniUoaxtgtog8FoZ7\n+opIoxKTMZPuwNMhXVJyquoPwG3AtNDmfFV9qxhy1gaFURsomZkDItIYeAH4m6ouorxsNS6rqpap\n+Q7bCuggIjtRQnKKyOHATFX9jPxrjmr8WQb2V9X2wB+Ac0TkQEroeWJfwe2Be4Oci7Gv3lKScTUi\nUg/oCjwfskpKThFZH3Ox1ArrbawnIidmkavgctYGhVGpxX9FYqaINAcQkc2An0L+99iYZ0Qka678\ngiIi62DK4klVjSIjlqSsAKq6ABgGdCkxOfcHuorIJOAZoLOIPAn8WEIyrkZVZ4T9LGwosgOl9Ty/\nA6ar6qfh+EVMgZSSjHEOA0aq6uxwXGpy/g6YpKpzVHUV8DKwXzHkrA0Ko8LFf0VASP/SHAicGtKn\nkApbOxA4PsxYaANsA3wSuofzRaSDiAhwMtUT6vZRbEzyrlKVVUQ2jmZviEhD4BBgXCnJqapXqmpL\nVd0a+729o6onAYNKRcYIEWkUepWIyHrY2PtYSut5zgSmi0i7kHUw8GUpyZhBD+xDIaLU5JwG7Csi\nDUL9BwNfFUXO6jAYVYMBqgs262cCcHmR234amy2xPPyhTsOMTW8FmYYA68fKX4HNQhgHHBrL3xP7\nR54A3FUNcu4PrMJmkY0GRoXntmEpyQrsEmT7DBgD/D3kl5ScsTYOImX0LjkZMftA9DcfG/1/lJqs\nwG7Yx99nwEvYLKmSkjHU3wiYBTSJ5ZWinL1Dm2OAftgM0mqX0xfuOY7jOImoDUNSjuM4TgngCsNx\nHMdJhCsMx3EcJxGuMBzHcZxEuMJwHMdxEuEKw3Ecx0mEKwyn5BCRMhF5InZcV0RmScrN+JEicmkF\ndWwuIv1D+hQRuaeSMlyRoMxjIvKnytRbSERkqFiUSscpCq4wnFJkMbCziKwbjg8h5jxNVQep6i35\nKlDVGaoaj95Y2QVHV1ayfK1CROrWtAxO7cMVhlOq/Bc4PKTTXDXEewzhK/8uEXlfRL6NvviDK5mx\nsfpahi/y8SJydayul4OX17GRp1cRuRFoGDyrPhnyTpZUkKJ+sXoPymw7TpDjq+Ch9QsReT1ShPEe\ngohsJCKTY/f3slgwnEkico5YwJxRIvKBmPO5iJODTGNEZO9wfSOxwF8fichIETkyVu8AEXkbWxHs\nOJXCFYZTiijwLNAjvFx3xQK9ZJaJ2ExV9weOxILIZCuzN3AU5qLi2NhQzmmqunc4/zcR2UBVrwCW\nqGp7VT1JRHbEehyd1Lzs/i1B23G2Ae5R1Z2B+cDRee47Yifgj5gjwRuARWqeXj/CfP5ENAwynYP5\nEgP4O/C2qu4LdAZuDX67APYA/qSqv80hg+PkxBWGU5Ko6hdYdLAewGvkdzP+SrhmHKkYAJm8qarz\nVHUZ5svogJB/voh8hr2ItwK2Dfnx9joDz6vq3NDOvEq2PVlVo97OyHBfFTFUVZeoeUydB7wa8sdm\nXP9MaH840EREmmIOCC8XizkyDKhPyuPzm6o6P0H7jlOOdWpaAMfJw0Dg31jUu43zlFseS+dSLOVi\nBYjIQZgy2EdVl4vIUKBBJWVM0na8zKpYGytJfbRlthu/RmPHZaT/32aLgSDA0ao6IX5CRPbF7EOO\nUyW8h+GUItGL91HgWlX9sgrXZnKIiKwfhmb+CLyPeUydG5TF9lg44IhfYobhd7BhrA0BRGSDSrad\nK38KsFdIH5ujTEV0DzIdgEVeWwi8AZy3unGR3atYt+Ok4QrDKUUUQFW/V9U+ScrmOY74BBuK+gwb\nXhoFvA7UE5EvgX8BH8bK9wXGisiTqvpVOP9uGOa5rZJt58q/FThLREZirqlzka/eZSIyCrgPOD3k\nX4fd1xgR+QL4Z566HScx7t7ccRzHSYT3MBzHcZxEuMJwHMdxEuEKw3Ecx0mEKwzHcRwnEa4wHMdx\nnES4wnAcx3ES4QrDcRzHScT/A8pPhRhTHDZLAAAAAElFTkSuQmCC\n",
|
|
"text/plain": [
|
|
"<matplotlib.figure.Figure at 0x2ec10332128>"
|
|
]
|
|
},
|
|
"metadata": {},
|
|
"output_type": "display_data"
|
|
}
|
|
],
|
|
"source": [
|
|
"pred_basic_model = train_and_evaluate(reader_train, \n",
|
|
" reader_test, \n",
|
|
" max_epochs=10, \n",
|
|
" model_func=create_basic_model_terse)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"Now that we have a trained model, let us classify the following image of a truck. We use PIL to read the image."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 16,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/html": [
|
|
"<img src=\"https://cntk.ai/jup/201/00014.png\" width=\"64\" height=\"64\"/>"
|
|
],
|
|
"text/plain": [
|
|
"<IPython.core.display.Image object>"
|
|
]
|
|
},
|
|
"execution_count": 16,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"# Figure 6\n",
|
|
"Image(url=\"https://cntk.ai/jup/201/00014.png\", width=64, height=64)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 17,
|
|
"metadata": {
|
|
"collapsed": true
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"# Download a sample image \n",
|
|
"# (this is 00014.png from test dataset)\n",
|
|
"# Any image of size 32,32 can be evaluated\n",
|
|
"\n",
|
|
"url = \"https://cntk.ai/jup/201/00014.png\"\n",
|
|
"myimg = np.array(PIL.Image.open(urlopen(url)), dtype=np.float32)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"During training we have subtracted the mean from the input images. Here we take an approximate value of the mean and subtract it from the image."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 18,
|
|
"metadata": {
|
|
"collapsed": true
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"def eval(pred_op, image_data):\n",
|
|
" label_lookup = [\"airplane\", \"automobile\", \"bird\", \"cat\", \"deer\", \"dog\", \"frog\", \"horse\", \"ship\", \"truck\"]\n",
|
|
" image_mean = 133.0\n",
|
|
" image_data -= image_mean\n",
|
|
" image_data = np.ascontiguousarray(np.transpose(image_data, (2, 0, 1)))\n",
|
|
" \n",
|
|
" result = np.squeeze(pred_op.eval({pred_op.arguments[0]:[image_data]}))\n",
|
|
" \n",
|
|
" # Return top 3 results:\n",
|
|
" top_count = 3\n",
|
|
" result_indices = (-np.array(result)).argsort()[:top_count]\n",
|
|
"\n",
|
|
" print(\"Top 3 predictions:\")\n",
|
|
" for i in range(top_count):\n",
|
|
" print(\"\\tLabel: {:10s}, confidence: {:.2f}%\".format(label_lookup[result_indices[i]], result[result_indices[i]] * 100))"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 19,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Top 3 predictions:\n",
|
|
"\tLabel: truck , confidence: 96.59%\n",
|
|
"\tLabel: ship , confidence: 2.31%\n",
|
|
"\tLabel: cat , confidence: 0.43%\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"# Run the evaluation on the downloaded image\n",
|
|
"eval(pred_basic_model, myimg)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"Adding dropout layer, with drop rate of 0.25, before the last dense layer:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 20,
|
|
"metadata": {
|
|
"collapsed": true
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"def create_basic_model_with_dropout(input, out_dims):\n",
|
|
"\n",
|
|
" with C.layers.default_options(activation=C.relu, init=C.glorot_uniform()):\n",
|
|
" model = C.layers.Sequential([\n",
|
|
" C.layers.For(range(3), lambda i: [\n",
|
|
" C.layers.Convolution((5,5), [32,32,64][i], pad=True),\n",
|
|
" C.layers.MaxPooling((3,3), strides=(2,2))\n",
|
|
" ]),\n",
|
|
" C.layers.Dense(64),\n",
|
|
" C.layers.Dropout(0.25),\n",
|
|
" C.layers.Dense(out_dims, activation=None)\n",
|
|
" ])\n",
|
|
"\n",
|
|
" return model(input)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 21,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Training 116906 parameters in 10 parameter tensors.\n",
|
|
"\n",
|
|
"Learning rate per minibatch: 0.01\n",
|
|
"Momentum per sample: 0.9983550962823424\n",
|
|
"Finished Epoch[1 of 5]: [Training] loss = 2.107245 * 50000, metric = 79.08% * 50000 12.977s (3853.0 samples/s);\n",
|
|
"Finished Epoch[2 of 5]: [Training] loss = 1.795581 * 50000, metric = 67.10% * 50000 12.244s (4083.6 samples/s);\n",
|
|
"Finished Epoch[3 of 5]: [Training] loss = 1.657041 * 50000, metric = 61.52% * 50000 12.265s (4076.6 samples/s);\n",
|
|
"Finished Epoch[4 of 5]: [Training] loss = 1.567592 * 50000, metric = 57.72% * 50000 12.251s (4081.3 samples/s);\n",
|
|
"Finished Epoch[5 of 5]: [Training] loss = 1.500142 * 50000, metric = 54.97% * 50000 12.228s (4089.0 samples/s);\n",
|
|
"\n",
|
|
"Final Results: Minibatch[1-626]: errs = 47.1% * 10000\n",
|
|
"\n"
|
|
]
|
|
},
|
|
{
|
|
"data": {
|
|
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYwAAACfCAYAAADqDO7LAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztnXeYVOX1xz9fQAgqoNhQELA3VEBFY2NFjb38NJhgiT1q\nbIkldkFNsEWNNREjqChixYJdBDs2QBBRsVBEUUSqgFLO749zxxmW2d3ZZWZndjmf55nn3vve9773\n3Hd37pnznnPeV2ZGEARBEFRFg2ILEARBENQNQmEEQRAEOREKIwiCIMiJUBhBEARBToTCCIIgCHIi\nFEYQBEGQE6EwVlAk/UfSJdWtK6mrpMmFle7X+34lqVtt3Ks+kPxtxuS7bg3keF3SnwrRdlBcGhVb\ngCC/SJoAtALWM7MfM8pHAtsC7c1skpmdlmubWerWKHlHUjvgK6CRmS2pSRv1BUm7As/hfdkAWBmY\nCygp29LMvq5Om2b2KrB1vusGQYqwMOofhr+Ue6QKJHUAmlLDF30eSb0MVfAbSQ0LfY/lwczeMLNm\nZtYc2ArvlxapsvLKQglFETYIEkJh1E/6A8dmHB8L3JtZQVI/SVcm+10lTZZ0jqTvJE2RdFy2uuki\nXSRpmqQvJR2ZcWJ/SSMkzZI0UVLPjOteTbYzJc2WtGNyzcmSPk7KPpLUMeOaTpI+lDRD0oOSGmd7\nYEnHSnpD0o2SpgE9JfWU1D+jTjtJSyQ1SI6HSroyuW62pOcltayg/Y8l7Z9x3FDS95I6Smoiqb+k\nHxI535G0VrZ2qmAphZAM7Vwp6S3c+lhf0okZfTVe0okZ9feU9FXG8WRJf5M0OpHrAUkrVbducv4i\nSd8m9U5K+rFtlQ/kXC5pgqSpkvpKapaca5rcJ9Vvw1P9nzznV8lzfi7piBr0Z5BnQmHUT4YDzSRt\nlrwc/wDcT+W/7FsBzYD1gJOA2yW1qKRuy6TucUAfSZsk5+YCx5hZC+AA4FRJByfndk+2zZNf0e9I\n6g5cDhyd/No+GJieca/uwO+ADfAhteMqeYYdgc+BdYB/JmXlraryxz1whboW0AQ4r4K2BwBHZhzv\nC0wzs1HJ9c2B1ni/nArMr0TO6nA0/szNgSnAVGC/pK9OBm5NLMgU5Z+vO7AnsCGwPXBMdetKOhA4\nHegKbAp0y3JtRZyM99vuwEZ4//w7OXc8bvmul5T/BViQKJQbgD2T59wFGJ3j/YICEgqj/pKyMvYG\nxgHfVFH/F+AqM1tsZs/hL/7NKqhrwGVmttDMXgOeAY4AMLPXzGxssv8RMBB/0WSSqbhOBK4zsxHJ\nNV+aWaZT/WYz+87MZgJPA5nWR3mmmNkdZrbEzH6u4nlT9DOzL5L6D1fS/oPAwZJ+kxz3SMoAFgJr\nAJuaM9LM5uZ4/6roa2afJX+XxWb2jJlNBDCzYcAQYLdKrr/JzKaZ2QxgMJX3X0V1uwN3J3LMB66o\nhvxHAv9K/GY/AReTVrwLgTVJ99sIM5uXnFsCbC2pSfL3/6Qa9wwKRCiM+sv9+BfzOOC+HOpPL+eI\nngesWkHdGWa2ION4Iv4rEUk7SnolGa6ZCZyCvxQqYn3gi0rOf5ejTAA1id6amkv7ZvYF8DFwkKSm\nuCU0IDndH3gBGCjpa0nXKH8+lKWeSdKBydDNdEkz8B8ElfVvdfqvorrrlZNjMrn7odbD/z9STASa\nJEN29wAvAw8nQ129JTUwszm4Qj4DmCrpqQwLNigioTDqKWY2CXd+7wc8nufmV09eminakrZgHgCe\nAFqb2WrAnaRfLtmGMSbjQxX5oHz7P+HRRynWXc72B+JK+BBgrJl9CWBmi8zsKjPbCtgZOAjIV1jp\nr8+UWDeP4MNta5nZ6sBLFD6I4FugTcZxW3IfkvoGaJdx3A74ObFkFprZlWa2JbArcBhwFICZvWBm\ne+PDn1/g/0dBkQmFUb85AeiWDCPkEwFXSFpJ0m64r+Lh5NyquAWyUFIXlh73n4YPNWQqiP8B50nq\nDCBpI0nr50nOUcDuktZP/DEXLmd7A3F/ymmkrQsklUnqkPiL5uJDLdUNG87lpd8EWAn4AbDEt7Bn\nNe9TEx4GTpS0qaSVgUurce2DwDlJwEEz4B8kfSdpD0lbSRIZ/SapVWJJNQUW4Yp/cT4fKKgZoTDq\nH7/+8jOzr1K+gfLnqtNOFr4FZuC/HvsDp5jZ+OTcX4CrJM3CXywPZcgzH/91/KakHyV1MbNHk7IB\nkmYDg3AHaHXlXfYBzF5O7j8aeA/3gSxVpZrtTQXeBnYi47nwX8GPArOAscBQvF9SSY935NJ8VWVm\nNgv4G27BTcd/kZd/pqrarHZdMxsM/Ad4DfgUeCM5VZGfKLOtu/C+eh0PSJgF/DU5tx5u/c4CxgAv\n4sqkIXA+/v81Dfgt7nQPioxiAaUgCKpDEpX1gZk1KbYsQe0SFkYQBFUi6dBkCLIlcA1u5QQrGKEw\ngiDIhdNx38lneATVGcUVJygGMSQVBEEQ5ESdmXxQUmi2IAiCGmBmeQm9rlNDUmbG3LnG9dcbnToZ\nW21lnHqqce+9xtixhlnxPz179iy6DCFnyFlXZQw58//JJ3VKYQCssgqcdx588AFccQWsuSY8/jj8\n9FOxJQuCIKjf1JkhqfJIcPjh/qkIM68XBEEQLD91zsLIlQULoEEDePZZmJ/vPOdKKCsrq72bLQch\nZ36pC3LWBRkh5Cxl6kyUlCSrjqxmcNppcOed0K0bvPACNKqz9lQQBEHNkITlyeldbxVGijlzoGtX\naNIEXnkFmjat+pogCIL6Qj4VRkGHpCS1Saa6HitpjKSzstQ5Ur6i2ofylc/yus5ws2buIN9sM/jw\nw3y2HARBsGJRUAtDUiuglZmNkrQq8AFwiGUshiJpJ2Ccmc2StC/Qy8x2ytJWjSyMIAiCFZl8WhgF\nHdU3n91zarI/V9I4fBnLTzLqDM+4ZHhyPgiCICgxai1KSlJ7fMnHdyqpdhLwXG3I849/wN/+VrsR\nVEEQBHWZWlEYyXDUo8DZVsFax5L2wBeFv6DQ8ixZAiNHwr//DW3bFvpuQRAE9YOCB5pKaoQri/5m\n9mQFdbYB+gD7mi9An5VevXr9ul9WVlbjOOgGDeCxx2DuXHeKSx6GGwRBUNcZNmwYw4YNK0jbBQ+r\nlXQf8IOZnVPB+bbAEOCYcv6M8vUK4vR+/XWYMAGOOSbvTQdBEBSdOpOHIWkXfFnHMfiyjQZcjC8E\nb2bWR9Jd+FKTE/F1jReaWZcsbdVKlNRll8GsWXDVVdCiRcFvFwRBUFDqjMLIJ7WlMNq3h4kTfX/m\nzFAaQRDUbepM4l5dZMIEGDcOzjjDs8SDIAgCJyyMIAiCekxYGLXML78UW4IgCILiEwqjCj79FDp1\nKrYUQRAExScURhVstBF8/DFst12s6hcEwYpNKIwqaNQI7roLRoyAVVdNl4c7JQiCFY1QGDlw0knw\nzTdw5JF+PHWqZ4s/9FBx5QqCIKhNQmHkyLrrwgMP+P7VV/v29tuLJ08QBEFtEwqjBpx/Pgwd6tOK\nXHddsaUJgiCoHWKV6xrQpo1/XnstfBlBEKw4ROJeEARBPSYS90qMvn1hlVViKpEgCOo3oTDywEcf\nwbx50Lx5DFEFQVB/CYWRB2680Rdk6tzZF2MKgiCojxRUYUhqI+kVSWMljZF0VgX1bpE0XtIoSR0L\nKVOhOOww+OADWLTIw23D0giCoL5RaAtjEXCOmW0F/BY4XdLmmRUk7QdsZGabAKcA/y2wTAXl3Xd9\navQCrZAYBEFQNAqqMMxsqpmNSvbnAuOA1uWqHQLcl9R5B2ghaZ1CylVIdt4ZdtwRunWLWW6DIKhf\n1JoPQ1J7oCPwTrlTrYHJGcdTWFap1Cnuu8+3Awemt+PGFU+eIAiCfFAriXuSVgUeBc5OLI0a0atX\nr1/3y8rKKCsrW27ZCsGmm/rKfW3auC+jRw8vnzcPmjZ1x/gXX8CGGxZVzCAI6iHDhg1jWIHGxAue\nuCepETAYeM7Mbs5y/r/AUDN7KDn+BOhqZt+Vq1cnE/e+/NKnSAc4+WRYffX0dCJ18HGCIKhj5DNx\nrzYsjL7Ax9mURcJTwOnAQ5J2AmaWVxZ1mbZt4auvfIbbTTeFNdeENdZwCyMIgqAuUVALQ9IuwGvA\nGMCSz8VAO8DMrE9S7zZgX+An4HgzG5GlrTppYZRn4kRo3drX2Vi0CMaOhY039kzxIAiCfJNPCyPm\nkioi//wnXHqp78+Zs/QCTQsXwtprw4svwg47FEe+IAjqPjGXVD3hzDM9omqLLeDtt71sxgx46y24\n9lqYOROefrq4MgZBEKQIhVFEmjeHY46Bfff1Ff3MoGVL2GUXd44DHH10cWUMgiBIEUNSJcRZZ8Gt\nt8LIkdCxoysQyZXGhRdChw7FljAIgrpG+DDqKdttByNGLBtuK7lTfPLktOURBEGQC7Xuw5C0kaQm\nyX6ZpLMkrZYPAYI0778PixcvW/7aa/DTT3DEEX5+yZLaly0IgiBXH8ZjwGJJGwN9gPWBAQWTagVF\nggZZ/iKpoaizz4ZttoFOnZZWLJMnwyuv1I6MQRCsuOQ0JCVphJl1lnQ+sMDMbpU00sw6FV7EX2Wo\n90NSuZBab2PqVFgnmaJx++19avUlS2I9jiAIlqYYYbULJfUAjsWn+QBYKR8CBNXjnntgs81cWcyd\nCw88AP36+bmhQ4sqWhAE9ZxcLYwtgVOBt83sQUkbAEeY2bWFFjBDhrAwynH66XDHHW5ZNGgAjRvD\nbrvBc8/BSok6f/552G8/+Pzz9JxWQRCsONT6XFJm9jFwVnLz1YFmtaksgmV56SVXFr//vQ9DDR8O\nLVp4EmDjxunhqUce8fobbxyTHQZBsHzkpDAkDQMOTup/AHwv6U0zO6eAsgWVkHJ69+nj2x13XPr8\n/Pmw8spw/PEwe7bndQRBECwPuQ5JjTSzTpJOAtY3s56SRpvZNoUX8VcZYkiqHKnEvkx++cUtjLlz\noW9fTwbMZMEC+M1v0tc3aAAnngj/+1/tyBwEQe1SDKd3I0nrAkeQdnoHRSZbRFTjxr5t1szDcL/6\nKn3uscd8AaeU3l2wIF0eBEFQFbkqjCuBF4AvzOw9SRsC4wsnVrC8TJni2xtv9O3s2e7v2HRTGDQI\n2rVz5dGrl09yWJ5x42B8/IWDIMig0Oth3A0cCHyXbfhKUnPgfqAt0BC4wczuqaCtGJKqJpnDTwsX\nuvXRty88/jgMHuwWyI03ws8/+6y4W27pSYKpoSoIR3kQ1HWKMTVIG0mDJH2ffB6T1CaHS/sB+1Ry\n/nRgrJl1BPYAbkiWdA3yQEpZgIfZLl7sTvBHH/WyJ590xdC0KdxwA2y9tZffcotv99oLRo/2JMEg\nCIJch6T64Uuprpd8nk7KKsXM3gBmVFYFaJbsNwOmm9miHGUKqknKamjSxC2HTP/Gtdemtw0awDXX\neL1tt4V1183eXlgfQbBikavCWMvM+pnZouRzD7BWHu5/G7ClpG+AD4Gz89BmUAO6dvXtZpv5wk4X\nXJD2gwwuF+YwYAA0bJh93qsgCOovuQ7/TJd0NPBgctwDmJ6H++8DjDSzbpI2Al6StI2Zzc1WuVev\nXr/ul5WVUVZWlgcRAvCIq1mzfFGnFO+842uPm8F113ny3+GH+7kOHeCjj3x1wJ13Tl/z9tvw8stw\n+eXuG0lFbQVBUDsMGzaMYcOGFaTtXPMw2gG3Ar/Fh5HeAs40s8k5Xvt0BU7vwcDVZvZmcjwEuMDM\n3s9SN5zeRaR8CO+iRa5Mjj/eHekA8+b5uh2HHgpPPOFlZu5wHzcONt88FEgQ1Da17vQ2s4lmdrCZ\nrWVma5vZocDhOd5DyScbE4G9ACStA2wKfJlju0Ets+eePjy1cKEPSd14I5x8cvr8OUne//33w6mn\nutMcoHt394Wst17tyxwEQf6ocVitpElm1raKOgOAMmAN4DugJ9AYMDPrkyQD3gOk3KpXm9mDWZoK\nC6PIPPwwbLWVfzL55Ref0+qSS9zCaNsWJk5Mn58715MIwRXJUUfVnsxBEJTIEq2SJpvZ+vkQIsf7\nhcIoQRYv9qEpcD/HUUcta0lMmwZrLWeIRLZpUIIgqJpiTA2SjXh7BzRsCP/5Dzz7LJx2WvZhp4qU\nxY8/uhL4/HP45pt02Q8/LFu3QQO/TxAExaNShSFpjqTZWT5z8HyMIODUU33NjVVXrbru4sU+Fcmz\nz8Jdd3nZl19C69auPNZYwxXMa6+lr/nxR9++++6y7X3wAfzrX1Xfd9Ag+OMfq64XBEHFVKowzKyZ\nmTXP8mlmZpGRHVSbPfeE1Vf3dTpSM+mmpmgHuOwy3/72t+myUaN826qV54ikrBGASZPg/POrvu8/\n/wkPPbR8sgfBik6kXgW1SsOGvt12W5+SZNw4+Pvf/aU/YQJccYXnb6y0EuyzD4wdC2VlnkR4zDFw\n221ujUyf7rkeqUiskSN9+8wzMGfO0vf89lu3RG680a2h6fnIIAqCFZCCTj6YT8LpXT9YssRf7tts\nk15GtiJSTu4330wnB260kQ9hpdhwQ+jUyUN3v/jCo7XOP9/rH3qo13n9ddh9d88d6dQJ+vd3hVVT\nXn/d79u6dc3bCILaoiSipGqbUBgrHr17uwLIljEu+frld9wBN93kw1W9e/u5hg3dVzJwoA9fHXOM\nX9+8uV+3yioe7psLQ4Z426lw4muvhQsvdIWTGioLglImFEYQZDBqlE9jcuqpngvy8cew/fauKPr3\nh/XXd18HwCGHwFNP5T5xYsrKSWWqT5kCbdrACSfA3XcX5nmCIJ+EwgiCcpj50NWuu6bX/hgyxJ3s\n778P223n9ebMcUtj4cJ0/khFLFmS9rl89hlssonvf/BBur18yt+gAVx/PZx3Xn7bDlZsQmEEQRWk\nLII+feCUU5Y9bwZ33unnMhMCe/f2NUAGDlz2mg8/hJYt3WLJN9Omwdprp2ULgnwRCiMIlpNJk3yZ\n2qlTYZ11PELrxx/TlkP5f7VZs2C11eDf//aVCufO9eirlNWRD/7xD3fMZ0zKHATLTalkegdBnSWV\nBLjnnj4MtMEG6TyNO+7wX/yvvpqun0oOXLzYt6+9Bj16pM8vXuyWypAhNZfp0ktDWQSlTSiMYIWk\nWzcfsvrrX2H4cC/bfnvPPu/e3YeHMpdbueoqzxvp2NGPN94YZiRrSd5wA+y/v+/vtZevE/Ltt+6A\nz4W5c9NtffGFKx7Jw4cXxfqTQSlhZnXi46IGQf5ZssTsnXfMfvnFjydMMAOzo46q+Jr5871Onz6+\nBbNLLvHtzTf7tqzMbNGiitsYPtxs1qz09SlSx+3bmw0YULX848f7MwRBNpJ3Z17ewwX1YUi6GzgQ\n+M6yLKCU1CkDbgJWAqaZ2R4V1LNCyhoEmbRo4Wt/7LZbxXUyneUdO3o01uTJbiXstZdbGdts44l+\nm2229LWpSK6VV3ZL5KijfPp3gHPP9SGyM8/048x/ezN48sl0UuL48bDpph4d9vrry//cQf2jzji9\nJe0KzAXuy6YwJLXAV+/7nZlNkbSmmWWZqzQURlB69O3r63+kpidJMXEitG/vfo1UWG75f90ff/SJ\nFlMsWpSum2LrrX14K/PaX36BJk1g9mxXOs2bpzPm4+sRZKPOOL3N7A1gRiVVjgQeM7MpSf2syiII\nSpETTlhWWYBHX6XyKnr29LKUszxFy5a+jG3fvh7KW15ZADz9tOeWpFiyBK65xvcHDnSFs9JKbsk8\n9xy8+OLS15vB0KE1f77qsmCBBwsE9ZdiO703BVpKGirpPUnHFFmeIMgrhx2W3v/887RT+4wzPOv8\n+OPhoouyX9u+vc+JNWGCTwl/112ugA48EF5+OV2vVSv49FM48kg/njYNHnjArZNu3eCTT+DRR/2F\nXkj+7//SuSRB/aTYU5Q3AjoD3YBVgLclvW1mn2er3Csj5rCsrIyyzDCWIChBtt7a/QyprPLrr/dt\n6uWeCxts4Mqja1e3KGbPdv8KpMOD11nHZ+GdPx922cXved99fm6LLXw7eDAccEDV95szx9c2Kb/C\n4dZbexvt2mW/LrUU78MPw957+zT2Qe0zbNgwhg0bVpjG8+U9r+gDtANGV3DuAqBnxvH/gMMrqFvj\nKIEgKDapyKcePcy6dKnetVtskb6+UyezmTPNzjtv6TpLlvj5a6/17VZbed0ePcyGDPGyhx6q+l5P\nPOF1n346Xfbuu2affebl++5r9v776YiyFIsX+/kHH/TtddeZvfaa2XvvVXyvNddcOjosKAzkMUqq\nNoaklHyy8SSwq6SGklYGdgTG1YJMQVCrfP019OvnEVHHHlu9a1NL0959t/96b9EibamkkKBDB89I\nb9fOh6DefdcnX+zWzX0cW265bNvz58MLL/j+p5+mh7pGjEjX6dLFI7E6dPBIsEMOcb/JwIGwxx6u\nyhYtghNPhIMP9mt69/Yp5XfYoeLnSi2SVdlQmZk7+jNZsMD9R5nT3Ae1RL40T7YPMAD4BvgZmAQc\nD5wC/DmjznnAWGA0cGYlbeVb8QZBrfP882ZTplTvmsWLPU8kFz7/3H+1L1687LmZM80+/XTpst13\n9/oLF/p27Fi3En7/ez/+8MO0xdCli+9vuKHZEUekrZ4ZM5ZuM5WPsscevs2WI9K7t9m8eX7+4osr\nfp6UxdK3r9mkSWavv27WubOXHXSQ2auv5tYvKzLk0cIo+JBU3gQNhREEOVFREl/37v6N/+yzdFnq\npf/nP/v2xx+9fOJEP7799rRCmTTJh6NS14DZK694omLnzuk2Fywwa9LEbOpUV3TXXed1f/556fvO\nmuXJi9Ompcvvusvs3HPTx+eea7bPPkvfs3dvs7/8JX381ltmm2/uMgbLkk+FEZMPBsEKwvz5nijY\nurUPkYEnAO63n68lAkvncqSc3uW/drfd5k7199/3sN+11oIffli63syZPlljZjvz5vn0KtOn+4qF\nM2f6uR9+8BDhSZPcuQ8eQjx1Kqy3nueazJ6dbvvSS92xP2CAD7mlGD/ep2ypLe64w4MZ/vzn3Op/\n/bUPE2ZGztUG+czDKHaUVBAEtUTTpr6u+ZNPeuhtq1bp/BAzOOKIpeu/9156jqtMzjhj6eMNNvCX\n/ty5Hl0FaWUBvkrhnDm+0NUnn/gcXpttllYka63l/pWPPvLM+O228/rrreeRWY89Bm+95ddefXW6\n3X33hR139PVJ+vVzJXL++R4JlooMKySnn+7bihTG1Knub2ra1I/PPhsef7xuJ1iGhREEKxAzZ/ov\n+dtv97VCUiG0y0NqMaryYbjlOeEEf7GfeSa88Ubasf7UU+5Iv/pqX/4WKrZuKqJXL5+WpW/f6l0H\n8Mwznki51lruvN988/S5xYs9AbP8s1mSmNm9uwciZEOC3/8eHnnEZWvbtvqy5YM6k+kdBEFpsdpq\n/iu+Tx9PslteZQGeG1KVsgC4+GK3Rr7/Ho47Ll2eiqzKZs3kSq9ebj2lkFwZVsbMmZ7geOCBniD5\nwAPpiDTwYbFGjeCkk5a9VvK+O+ec7G2nhvx+SOauaNnSt2PGwLrr1l0rIxRGEKyAnHuu+yBqk3XW\nga++8lDc1HTwKSZPXnq4afRoz4qvDi1auEUwfrwfv/FGxXXnzPHEwqOPTpftsAPccotbDFJ6upYh\nQ3yY6623lm1jp52ytz92rA+7XXmlZ8A3auSWWIcOPlQ1alT1nq1UiCGpIAhqhdQwTp8+cPLJhb2X\n5L6QlFL8+WeftDFF48b+Ar/5ZrcgnnjCX+wrr5yus/febkUMGpQumzXLLZHFi92XM2KEK7dMi+mA\nA9yiOP102HbbdJup15fkuTGpxbYGDfJcl9at0228/LIrl1at8tEX+RuSKnq4bK4fIqw2COo8kybV\nzn3efDOdozF5soff3n+/2f77mz3yiNkPP3gmevlQ3P32M3v7bV8TxcxszhxvJxXCm8q6P/xwP//c\nc3783HPpNipa3yTzHqnjGTN8f/DgpXNnyl9TEd9/b3bTTZXXIfIwgiAIcuPWW/1N9+KLub+Iy/Pz\nz57f8t//2q8JjmZe1rq12SabmL3xhlnjxmaNGplddln62unTfZGrFE8+aXb99b6/887e3nbbmT37\nrCuwe+9Ny5lSaG+8Yfb118vK9dZbVU81k0+FEUNSQRDUaxo2dMf89OnuO7nlFh8qyheZC1qBD3U1\nbJhbIMC223qIc1mZ56YceKBPmfLee7Dmmj7ZY9euHi7cpInfK5OHHvIorEcfrfgeESUVBEFQBd9+\nCxdcAHfe6eG2q6wCr76aX2UBcNBBnsQHHjacS4jx5MnuZL/3Xs8/adsW3n7bV04Ez0tp395DkLfZ\nxn0wTz3lU92//LJHcA0f7rkp5VdzLCShMIIgqJcMGgTXXeeRUNWZTr66NGgAp53miY9/+ENu18yZ\n4w75//7XHeQ77ACvvOLO9CuuSDvKTznFt/fc49tnn3Vn/ODBbom0bw9/+lO+n6hiQmEEQVAvSb10\nBw6snfs99FA6q7sqUlbBKqv49tBDffXGJUvg8svT9fbeGzp1Sq9B0q2bbw85xLeNGoWFEQRBsNyk\nQl2/+aaoYmSlYUO3Ejp3Th+/9NKyQ1mHH57OiB861P0cKY44Ao46qnbkTRFO7yAI6i2pV0YuDuj6\nSp1xeku6W9J3kkZXUW8HSQsl1fI8jvmnYEsj5pmQM7/UBTnrgoyQXzmlwimLutKf+aTQQ1L9gH0q\nqyCpAXAN8EKBZakV6so/UciZX+qCnHVBRgg5S5mCKgwzewOoakqxM4FHge8LKUsQBEGwfBTV6S1p\nPeBQM/sPFa/7HQRBEJQABXd6S2oHPG1m22Q59zDwLzN7V1I/YLCZPVZBO+HxDoIgqAH5cnoXe8W9\n7YGBkgSsCewnaaGZPVW+Yr4eOAiCIKgZtaEwRAXDTWb2a1RxYmE8nU1ZBEEQBMWnoApD0gCgDFhD\n0iSgJ9AYnz2xT7nqMeQUBEFQwtSZxL0gCIKguNSJqUEk7SvpE0mfSbqgyLJMkPShpJGS3k3KVpf0\noqRPJb0gqUVG/YskjZc0TtLvCijXMkmSNZFLUmdJo5O+/nctydlT0teSRiSffUtAzjaSXpE0VtIY\nSWcl5SXu1zN8AAAG+klEQVTTp1lkPDMpL6n+lNRE0jvJd2aMpJ5Jecn0ZRVyllR/ZtyjQSLPU8lx\n4fszXwtrFOqDK7XPgXbASsAoYPMiyvMlsHq5smuBvyf7FwDXJPtbAiPxob/2yXOoQHLtCnQERi+P\nXMA7wA7J/rPAPrUgZ0/gnCx1tyiinK2Ajsn+qsCnwOal1KeVyFiK/blysm0IDAe6lFJfViFnyfVn\n0u7fgPuBp5LjgvdnXbAwugDjzWyimS0EBgKHFFEesaxldghwb7J/L5BaTuVgYKCZLTKzCcB4/Hny\njmVPkqyWXJJaAc3M7L2k3n0Z1xRSTsgeGHFIEeWcamajkv25wDigDSXUpxXImFoZutT6c16y2wR/\ncRkl1JdVyAkl1p+S2gD7A/8rJ09B+7MuKIzWwOSM469JfymKgQEvSXpP0klJ2Tpm9h34lxhYOykv\nL/sUalf2taspV2u8f1PUZl+fIWmUpP9lmNIlIaek9rhVNJzq/61rRdYMGd9JikqqP5Phk5HAVOCl\n5CVVcn1ZgZxQYv0J3AScz9LBQgXvz7qgMEqNXcysM67dT5e0G8tGeJVqJEGpynUHsKGZdcS/qDcU\nWZ5fkbQqPnXN2cmv+JL7W2eRseT608yWmFkn3ErrImkrSrAvs8i5JSXWn5IOAL5LrMvK8tPy3p91\nQWFMAdpmHLdJyoqCmX2bbKcBT+BDTN9JWgcgMfNS82JNAdbPuLy2Za+uXEWR18ymWTKICtxFetiu\nqHJKaoS/iPubWWo15ZLq02wylmp/JrLNBoYB+1JifVmRnCXYn7sAB0v6EngQ6CapPzC10P1ZFxTG\ne8DGktpJagz8EShKcp+klZNfc0haBfgdMCaR57ik2rFA6uXyFPBHSY0lbQBsDLxbSBFZ+hdHteRK\nzNhZkrpIEvCnjGsKJmfyz53iMOCjEpGzL/Cxmd2cUVZqfbqMjKXWn5LWTA3jSGoK7I37W0qqLyuQ\n85NS608zu9jM2ponPv8ReMXMjgGeptD9mW/PfSE++K+RT3FnzYVFlGMDPEprJK4oLkzKWwIvJzK+\nCKyWcc1FeFTCOOB3BZRtAPAN8DMwCTgeWL26cgHbJc82Hri5luS8Dxid9O0T+FhsseXcBVic8fce\nkfwfVvtvXShZK5GxpPoT2DqRbVQi1yU1/d4USc6S6s9yMnclHSVV8P6MxL0gCIIgJ+rCkFQQBEFQ\nAoTCCIIgCHIiFEYQBEGQE6EwgiAIgpwIhREEQRDkRCiMIAiCICdCYQQlh6Qlku7LOG4oaVrGNM4H\nSfp7FW2sK18zHknHSrq1mjJclEOdfpIOq067+UTSUEmdi3X/YMUjFEZQivwEdJDUJDnem4zJ08zs\naTO7rrIGzOxbMzsis6iaMlxczfp1CkkNiy1DUPcIhRGUKs8CByT7PfA5c4ClLYbkV/7Nkt6U9Hnq\nF38ylcyYjPbaJr/IP5V0eUZbg5KZh8ekZh+WdDXQVL44Tf+k7E9KL5x1b0a7XcvfO5NEjo8l9ZH0\nkaTnU4ow00KQtIakrzKeb5B8MZwvJZ0u6W+JPG9JWi3jFn9KZBotaYfk+pXlC1UNl/SBpIMy2n1S\n0hA8IzgIqkUojKAUMXzdkx7Jy3Ub0tN2Z9ZJ0crMdgEOwheRyVZnB+D/gG2B7hlDOceb2Q7J+bMl\nrW5mFwHzzKyzmR0jn7H0YqDMfCbTs3O4dyYbA7eaWQdgFnB4Jc+dYit8bYIuwD+BueazJA/H5/xJ\n0TSR6XR8XimAS4AhZrYT0A34VzI3EkAn4DAz26MCGYKgQkJhBCWJmX2Erw7WA3iGyqdxfiK5Zhzp\nNQDK85KZzTSzBcDj+Mp/AH+VNAp/EbcBNknKM+/XDXjEzGYk95lZzXt/ZWYpa+eD5LmqYqiZzTOz\nH4CZwOCkfEy56x9M7v860ExSc3xSzAvl6zoMAxqTnvH5JTOblcP9g2AZGhVbgCCohKeA64EyYM1K\n6v2csV+RYllm7QVJXXFlsKOZ/SxpKPCbasqYy70z6yzOuMci0j/ayt838xrLOF7C0t/bbGtKCDjc\nzMZnnpC0E+4fCoIaERZGUIqkXrx9gSvMbGwNri3P3pJWS4ZmDgXeBFoAMxJlsTmwU0b9XzIcw6/g\nw1gtASStXs17V1Q+Adg+2e9eQZ2q+EMi067ALDObA7wAnPXrzaWONWw7CJYiFEZQihiAmU0xs9ty\nqVvJcYp38aGoUfjw0gjgeWAlSWOB3sDbGfX7AGMk9Tezj5PzrybDPKkV13K9d0Xl/wJOk/QBPjV1\nRVTW7gJJI/BV4U5Iyq/Cn2u0pI+AKytpOwhyJqY3D4IgCHIiLIwgCIIgJ0JhBEEQBDkRCiMIgiDI\niVAYQRAEQU6EwgiCIAhyIhRGEARBkBOhMIIgCIKc+H8egeVPyVBwPwAAAABJRU5ErkJggg==\n",
|
|
"text/plain": [
|
|
"<matplotlib.figure.Figure at 0x2ec1f9b5630>"
|
|
]
|
|
},
|
|
"metadata": {},
|
|
"output_type": "display_data"
|
|
},
|
|
{
|
|
"data": {
|
|
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAZMAAACfCAYAAAA8qTSuAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztnXecFdX1wL9HQAQpdhQLKIpEsKGisSIRxVijwYCx19h7\nIfEnscRuLDEaVFQswRgbNhSNHRFQQDqIFBWQJiCdBc7vj3OHN+/t27ez+97uvoXz/XzmMzN3bjkz\n+3bO3HvPPUdUFcdxHMfJhw1qWgDHcRyn9uPKxHEcx8kbVyaO4zhO3rgycRzHcfLGlYnjOI6TN65M\nHMdxnLxxZVJLEJHHROQvFc0rIoeJyA9VK93adqeISKfqaKu2ISJrRGSn6i6bL/G2K/IbzFLPIhFp\nWUjZnOLClUkNIyJTRWS5iGyWkT48/CPvAKCqF6nq35LUmSVvpRYTiUiLIIP/Tsj7eeSzoKvMsiLy\nsYgsE5FfRGS2iLwiIs3yaKvMtpP+BkXkIxE5J60S1caqOrWAckVtTRWRpeH+F4X9w4Vuxykff0nU\nPApMAbpHCSLSDmhAfi+gQiBBBqnyhkTqVHUbBaKyf5N8nmGusgpcrKpNgNbAJsADWSupnBKs8r99\nnihwjKo2CQqriapeni1jtt9YRX93teh3Wu24MikOngPOjJ2fCfSJZxCRp0Xk1nB8mIj8ICJXi8gs\nEZkuImdly5tKkh4iMkdEJovIqbELvxWRYSKyUESmiUjPWLlPwn5B+OLbP5Q5X0TGhrTRIrJXrMze\nIvKNiMwXkb4ismG2GxaRM0XkcxH5u4jMAXqKSE8ReS6WJ60nEL54bw3lfhGRdzN7dLGyY0Xkt7Hz\nOuHLfS8RqS8iz4nI3CDnYBHZMls9SRGR/UTki1DfdBH5h4jUzch2jIh8F+S4J6P8OUHmeSLSP+qR\nJm0eQFUXAK8A7UKdT4vIoyLytogsAjqKyIYicl/4W88M1+vH5LhORGaIyI8icjYx5Zn5uxKRE0IP\neqGIfCsiR4rI7cAhwCPxXoKkD5c1EZFnw3OYIrGhs/C7+ExE7hWRn8Pz6pLk/kslpv/G5mK/sWxp\nIiI3ifVyfhKRZ0SkSagj+g2eIyLTgP8l/aOsb7gyKQ6+BBqLyK7hxfkH4HlyfxVuDTQGmgPnAf8U\nkaY58m4W8p4FPC4iu4Rri4HTVbUpcAzwJxE5Plw7NOybhC++wSLSFbgZOC18DR8PzIu11RU4EtgR\n2DO0Vxb7A5OAZkA0fJL55Z953h1TtlsC9YFry6j738CpsfMuwBxVHRHKNwG2xZ7Ln4BlOeRMwmrg\nylDfr4FOwMUZeU4E2oftBAlDQSJyAnBjuL4l8BnQt6ICiMgWwMnAsFhyd+A2VW0MDATuBnYG9gj7\nbbG/J+GlfTXwG2AX4IgcbXXAPniuCb+dQ4GpqnpTkP/SjF5C/O/4CPbbbQl0BM4IiiuiAzAO2By4\nF+hdkeeQQfQb24rUbywz7WzgDOAwYKcg2yMZ9RwKtAGOykOWdRpXJsVD1DvpjP0jzSgn/0rsJbFa\nVftjSmHXMvIq8H+qWqKqnwJvA6cAqOqnqjomHI8GXsT+qeLEldq5wD2qOiyUmayq8Qn+h1R1VvhK\nfhOI91oyma6qj6rqGlVdUc79Rjytqt+F/C/lqL8vcLyIbBTOu5N6QZdgL6rWagxX1cUJ28+Kqg5T\n1SGhvu+Bxyn9HO9S1YWq+iPwIKmhzQuBO1V1oqquAe4C9hKR7RM2/w8R+RkYjv1urold66eqXwYZ\nVwDnA1cFOZaEtiI5umLPd5yqLgP+mqPNc4DeqvphqHumqk7MkV9g7VDbH4AbVXWpqk4D7gdOj+Wd\npqpPqTkO7ANsLSJb5aj79dCLmR/258auZfuNZaadCvxdVaep6lKgB9BNUsOCCvRU1WUV+J2ud2R2\nw52a43ngU+yL/tkE+eeFF0/EUqBRGXnnq+ry2Pk0rJeC2NDVndjQyIZh+2+OdrcHvstxfVaGTNvk\nyFsZK7OfMurPes+q+p2IjAWOE5G3sB7UzeHyc8B2wIuhN/c88BdVXV0JeQAIPb2/A/ti8111ga8z\nsv0YO177NwBaAA+JyP1RddgLbFuSPaPLVPWpMq6tLR+G8hoCX4us/T7YgNTHQnPgqwwZy+odb499\nlFSULbBn831GO9vGztf+jVV1mZiwjYDZZdR5gqp+VMa1bM8vM615kCEuT12sxxzxI05OvGdSJISv\n2SnA0cCrBa5+UxFpEDvfgVTP5wXgdWBbVd0E6EXqBZJtsvkHoFWB5Mqsfwn2sovIpYiS8CL21XkC\nMEZVJwOo6ipVvU1V2wIHAsdhwxz58BjWo2wVnuNfKP0ijvc0WpD6G/wAXKiqm4VtU1VtFPUo8iT+\njOdiCrhtrK1NwjAVwMwsMpZlcJDrd5DLSGEu1jNskdHO9BxlyqM8A4Xy0mZkkaeE9A+jmjaGKXpc\nmRQX5wCdwhBDIRHgFhGpJyKHYHMjL4VrjbCeS0kYB4/PM8wB1pD+0ngSuFZE2gOISKsKDMeUxwjg\nUBHZPvQYbsyzvhex+ZuLsDkUAESko4i0C8MYi7EXx5rsVZRCgI3CJH60CTbO/ouqLhWRNqHNTK4T\nkU3C87o8yAfwL+DPIrJbkK+piPy+4rebmzBs9ATwYOilICLbisiRIctLwFki8isRaUiqJ5eN3sDZ\nInJ4mMBuLiLRMOssbO4hmwxrQjt/E5FGItICuArrLdYUfYGrRKSliDTC5lFejPX8i92irShwZVLz\nxO34p0RzEZnXKlJPFmYC87EvsOewr+Bvw7WLgdtEZCFwE/CfmDzLsH+sgWEsuoOqvhzS/i0ivwCv\nYZPOFZW39A2ofhDaHwkMxeZc0rJUsL6fgEHAAcTuCzNIeBlYCIwBPiK8zMQW5j2aq1pgEfaFvyzs\nD8fmKf4YnkkvUooiXq4fNvQ1LNzbU0HO17G5ixdFZAF2/10yyuaSpyLXbsAmn78MbQ3ATIpR1Xex\nuZwPgYnksFxS1aHYxPWD2HP8GOvxAjwEdBWzTHswiyyXY89tMja0+7yqPl3B+4jzZrAci7ZXysmf\nyVPY3/9TbAh3aZAxafsOIFUdHCtYiDyIKa7eqnp3xvVNsD9mK+yf8xxVHZukrOM4jlMcVKkyCcMI\nEzFTwxnY12Y3VR0fy3MPsEhVbwvd5H+q6hFJyjqO4zjFQVUPc3UAvg0mdyVY1/+EjDy7Yd1qVHUC\n0DKM5yYp6ziO4xQBOZWJ2KrhskzukpBp2vgj6SaAAN8AJ4X2OmDjrtslLOs4juMUATnXmajq6uBK\noKmqLqwiGe7CbOyHAaOwhVcVsvcXEZ8gcxzHqSCqWjBLtSTDXIuBUSLSW0QejraE9U8nZeEB1uNI\nsydX1UWqeo6qtlfVMzEXB5OTlM2op6i3nj171rgMLqfL6XK6nNFWaJKsgH+Vyi+iGwrsHGzJZwLd\niHnHBbOpB5aqrXM4H/hEVReLSLllHcdxnOKgXGWiqn3EPL+2DkkT1CbEy0VtmOxSzJY9Mu8dJyIX\n2mV9HPgV0EdE1mA2/+fmKltmY6tWwbRp0KpQi7Mdx3GcpJSrTESkI+ZsbSq2EnR7ETlTzWFguagt\nhNo1I61X7PjLzOu5ypbJgAFwzDGwejVsUHxrMTt27FjTIiTC5SwsLmdhcTmLl3LXmYjI18Cpama7\niEhroK+q7lMN8iVCRFRXroQNN4RXX4Xf/a6mRXIcxylqRAQt4AR8EmUyUlX3KC+tJhERVVWIPKFW\nweSS4zjOukShlUmS8aCvROTJ4Byvo4g8Qbqb6uLhX/+CbX0piuM4TnWTpGdSH7gEODgkfQY8qkUU\nJGZtzwSsVxINdRXh3InjOE4xUK3DXCJSB3hWVf9Y6QbKd/TYBAtOtANQB7hfVZ8J16ZiHknXACWq\n2qGMNlLKZOlS2HhjO16zJjX0FbFkSeq64zjOekq1DnOpRZ5rEUyDK0xw1vgIFje5LdA9xHqIcwkW\nuGgvzJX3/SISWZmtATqq6t5lKZJSNIzFVrr4YmjUCGbPhscegz32gK228jkVx3GcApNk0eJkLJ7F\nG1gkPABU9e8Jyq511gggIpGzxrjnX8UCCxH281R1VTgXKuOM8qyzYPFiaNvWeiIPPwzDh0OHDjBj\nBsyZY0rFcRzHKQhJXtTfAW+FvI1jWxKSOGt8BNhNRGZgTh+viF1T4H0RGRpWxyfj6afhv/+1nslG\nG0HjxvDOO3DaadC0KXzxReKqHMdxnPLJ2TMJcyaNVfXaKpThKGC4qnYSkVaY8thDVRcDB6nqzOCS\n/n0RGaeqn2er5K9//eva444dO9qioQ02gGXL4OOP7UKHDnD00bZSPs6qVfDVV3DAAYW/O8dxnCLg\n448/5uPoXVgFJPEafFAe9Sdx1ng2cGdo7zsRmQK0Ab5S1ZkhfY6IvIYNm5WrTEqx/fZwzjk2n9Kq\nFXz3naUvXw4NGsDuu8OoUdkn7B3HcdYB1n5kB2655ZaC1p9kmGuEiLwhIqeLyEnRlrD+tc4awyR+\nN+CNjDzTgCMARKQZ5gNssog0FJFGIX1j4EhgdMJ202nVCnr3tuPttrP5E7AV82CKpGVLGDs2vdzs\n2ZVqznEcZ30jyQT8RsA8oFMsTUngSTiho8fbgWdEZGQodr2q/iwiOwKvhVgldYEXVHVA4jsri9//\nHtq3h1NOgSuvhHPPteGvN9+Eb7+1SXswE+NmzWDwYLvuOI7jlEmVxoCvLtLWmSThvfegSxc7jsqd\nfroNca1aBS+8AMcea5P2v/xiE/iO4zjrENW2zkREXoodZy40zL+HUJMccACceCLcemsqrX59eO45\n6NvXJu6//NLmU1yROI7jlEuZPRMRGa6qe4fjYaraPtu1YqDCPZNsrFoFixbBZpvZeby+yK19p07w\n+utmXuw4jlOLKXTPJNecSa63c+0fG8ukbl3YdFPrnUyYkEpXtWvvv28mxptsAuPGQZvMhfyO4zjr\nL7mUSUMR2RsbCmsQjiVsDapDuBrhtNPSz885x/Y33QS33AI9e1ovxnEcx1lLLmUyE4hcpvwUO47O\nE5Gno8ecZauFrbYyC6/+/a3ncuCBKYsvx3EcB6hia67g6HEi8BtgBrbupJuqjo/l6QE0UdUeIrIF\nMAFohjl5zFk2Vkf+cyYVYd482GILWLkS6tWrvnYdx3EKRE0Ex8qHtY4eVbUEiBw9xinL0WOSsjXD\nu+/aft68mpXDcRynSKhqZZKPo8ckZWuGk04yq6+5c9PTV660CfuJE2tGLsdxnBoiyQr4qiaro8eK\nVpLV0WNV0aABdO4MAwdCu3aWdtNN8Le/waOPwrXXmgt89/PlOE6RUKOOHiNEZFugRTy/qn6aoGg+\njh6TlF1LTkePVcH116efL1+e2jdqBD/9BNtsU70yOY7jlEFVO3osV5mE1e9/AMYCq0OyAkmUyVpH\nj5h1WDege0aeyNHjwLijRyxcb3lla4727W1I66OPYN994d57LZLjzjubg8j//Md8fzmO46wHlGvN\nJSITgD1UdUWlGjDz3odImffeFXf0KCLbAM8A0Wf8narat6yyZbRRvdZcEWvWQJ06dhxvf7fdbGHj\nOuD3zHGcdZNCW3MlUSb9ga4hWFVRUmPKBGxIa8mSlMsVgLffhu7dzUlkxMSJ0Lp1zcjoOI6TQU0o\nk1eAPYH/AWt7J6p6eaGEyJcaVSarVplV19Zb584nAsOGwd5F49LMcZz1mOr0zRXxBqUDWjkRdeuW\nrUgmToQdd4SSEjsfPNiVieM46ySJVsCHKInRGM2EsIiwaKjRnkkuItPgl1+2oFzHHmtBuBzHcWqY\nmhjm6gj0AaZiTh63B85MaBqcxDfXtcAfMQuxesCvgC1UdYGITMWsutYAJaqaNeRh0SuTRYvsePZs\n66k4juPUMDWhTL4GTlXVCeG8NdBXVfcpt/IEvrky8h8LXKmqUUz4ycA+qjq/nHaKU5mAWXT54kXH\ncYqMmvDNVS9SJACqOhHrQSShov61ugN9Y+eSUMbiJa5IfvzRzotV8TmO41SSJC/qr0TkSRHpGLYn\ngK8S1p/Yv5aINAC6AK/EkhVzrzJURM5P2GbxEkVozOa7a9IkeOyx6pXHcRynQCSx5roIuASITIE/\nAx6tAlmOAz5X1QWxtINUdaaIbIkplXGq+nm2wtXqm6uyRPHkzz8fPs2YctplF9v/6U8+LOY4TsGp\nat9cVR3P5ADgr6raJZzfiK18LxXkSkReBV5S1RfLqKsnsEhV/57lWvHOmWTSq5eFAP75Z3MS+fDD\nlj55MrRqBV9+CRtuaIG4dtwR5syx2CkRn3wCJ59c2mOx4zhOBai2ORMReSnsR4nIyMwtYf1rfXMF\n8+JuZFmzIiJNgcOAfrG0hiLSKBxvDBwJjE56Y0XLEUeYD6+PPoIXg97s0wemBx+WS5fC3XdbZEew\nMMFxfvMbi6PyxRcwdmz1ye04jpODMnsmIrJNGGJqke26qk5L1EA5vrlCnjOBo1T11Fi5HYHXsHmT\nusALReebKx969bIhrRkzoHlzePVV+N3vbJX8PvvAoEHw1Ve2PfNMqlw0BLbzzjBzprm6dxzHqSDV\ntgJeVWeGw4tV9YYMIe4GbihdKms97wK7ZqT1yjjvg61liadNAfZK0katZLvtbN+8ue2bNLF9NOy1\n887mzv6ll9LLLV9uw2ADBsB991WPrI7jOOWQxJqrc5a0owstyHrHMcfAihWwYIG5qz/sMEv/xz/s\nePPNzfvwscemynz4ofVMRCxA1wcf5G6jCifbHMdx4uQa5roIuBhoBUyKXWoMfKGqf6x68ZJRK4e5\nKoOIuWM59lgLEVy/PixbBhttVDrvvHmpiftJk2xy33EcJ1Cdixb/jZnr9gv7aNunmBTJekPkzr5z\n6ChuuCF062Z+v+Ku7iPmzIGttrLjceOqR0bHcdZbcs2ZLAQWishDwM+qughARJqIyP6qOri6hFzv\nWbMGLr3UjuvXT6X37Wu9lf32gyFD0svMmmXxUzp3djNix3GqnCRzJo8BcZOhxSEtESLSRUTGi8hE\nESk1aS8i14rIcBEZFsyQV4nIJknKrhdE0Ry//z779eOOM4uwTGbPhmbNbK3KtGkwciT88IMpnzlz\nqlZmx3HWO5Iok7QJCVVdQ7KV85Gjx0eAo4C2QHcRaRPPo6r3qereqtoe6AF8HDwGl1t2vWD2bNu3\naZPdp9dll9kaleeeS61VAbjtNltlf9JJ1js5/HDo1MmufZXUG47jOE4ykiiTySJyuYjUC9sVwOSE\n9efj6LGiZddNosBbK1dmv965M7zyCpxxhpkbz5tn6X36wGuvWTCun3+2bVKwozg6gTHeKafAAw/k\nL7/jOOsFSZTJn4ADgemYo8b9gQsS1p+Po8fEZdd5vv8eHs3hDu2kk+D55+144EDb7703HHSQHUcT\n8C+/nLpeHv/9L7z7buXkdRxnvaPc4SpVnY25Qalqsjl6TEytcPRYWbbfvvw83brBaaelFkPGue46\n2yJmzrTFkp98YnMohxySujZnDrwRPN4sW5af3I7jFA015uhRRK5X1XtE5B+YS5M0VPXyLMUy66i0\no8cKll0/1pmUx+LFNhz27LNw6qkp0+BMNtsM5sfijcWfXdOmVrZTJ7MSa9mySkV2HKdmqM51JtHi\nhK+Ar7NsSai0o8ekZZ0YjRqZG5arrjJLrrLYYw/bn3aa7ZcuTV3bbTdL79rV5lk++SS97F132Up8\nx3GcGFXqgh4q7+ixrLJltOE9k4h58+DCC+Hqq+HAA7PniWLSb7wxbLCBrapv2RK23dZ6LT/8YMNl\n7drBmDHWc/nuOxg8GP4Y1qv683acWk21xYAXkTfJMrwVoarHF0qIfHFlkic//AA77JA6j57l1Klw\n8MEWbviuu6BHj1SeTz+1eRd30+I4tZJq8xoMRC5pTwK2BoK5EN2BWYUSwCkCIqutP/0Jttwyld6i\nha1dGTnSFEmLFrZGZcst4dBDLU+keAYOtPgq59f+6MqO41ScXO5UPgEQkftVdd/YpTdFxFe9rUtM\nnAht25aOQR/FTrkhOB/o2jU96mO9euYSf6ONoEsX+PWvy1Ymq1fbUFnLluZXzHGcdYok60w2FpGd\nopMQtGrjqhPJqXbuvRdGlxHEcvx4C9xVUgL33GNpkybB44/bhP9331na4sUWjnjoUFi1qnQ9114L\nu+4Kl5drBOg4Ti2k3An4MAn+OLbqXYAWwIWq+l7Vi5cMnzOpITp0gJtvth5NSYmtrL/6ajjzTLj/\nfovJAjB8OLRvbz2YjTYyp5RjxsCJJ9as/I6zHlOdpsHA2kiJuwBXAJcDu1ZEkSRx1igiHYOzx9Ei\n8lEsfaqIfBOuDclW1qlB9tzT5lTeeQeOP95W3QOcd54Nh/3wgwX76hUCaz7zjAUDa93aQhRfcUVh\n5Pjzn2HKlMLU5ThOpShXmYhIQ+A64FJV/QbYQUSOLadYVLZcZ41hjck/gWNVtR3QNXZ5DdAxOILs\nkKRNpxrp3NkiPoK5xj/4YHjiCZvIB5u0v/xy81xcrx6cfLKlv/WW7aP1Lvnwyy9w553w4ouVryPb\nsJzjOBUiyZzJ08BK4NfhfDpwe8L6kzhrPBV4RVWnA6hqPPiGJJTRqQlOOSW1Qv7886FuXeuVjBlj\naaNG2f6SS2xlft26cNFFcMQRZgXWtatN8p93nrnaX73a3OmL2JqWiNWr7YW/erVtcSKvypVF1RTd\nkiX51eM46zlJXtStVPUeoARAVZdiL/kkJHHW2BrYTEQ+EpGhInJ67JoC74d0tzktRtq2hddfT0+b\nOtUm6a+5Bu64wybqIx59NBXgK1IYvXtbzJYttkitW4lbfHXqBPvvb8qobswAccUKM0l+9NF0k+bo\n2oMPli9/5EDz4YfLz+s4TpkkiUuyMnj0VQARaQWsKLAM7YFOmJXYIBEZpKqTgINUdaaIbIkplXGq\n+nm2StZpR4/FzOabwwkZnc0WLWxf3oLGAw+Ec8+FP/wBjjzS5lP69bO5ldatrYfyr3/ZAsk4IraQ\nculSuPXWlEXZ0KHmTwzgnHPg3/+2XlG9evD552YkcPjhqXpUUxEsX3klfVGm46xj1Jijx7UZRDoD\nNwG7AQOAg4CzVLVcqZI4awyT8hup6i3h/Emgv6q+klFXT2CRqv49SztuzVXbefddswZbudJe/h99\nZD2Srl3NHf5999kQWI8e1gtp3docUj78sLmQmT/fXMEsWGDbZ5/B6afD3/9uvsqiNTPx38lPP9m8\nzUUXWU+mXj0LKhbRr5+tn4mHSo6YMcOG3JJ4dHacIqTQ1lyoapkbNpy1PbA5cAxwLLBFrjIZ5esA\nkzBz4g2BEcCvMvK0Ad4PeRsCozDF1RBoFPJsDAwEjiyjHXVqOSUlqi+9pLpmjZ2vWaN6552qJ59c\nOu+TT6qecooqqO64o6V9842dX3yx7a+9VvWpp1R/+MGumxpJr+eLL1T339+OH3oo/fqCBXY+aZKd\nL1mSXrZx49L1OU4tIrw3E73Lk2w550xCg++o6jxVfVtV39L0CfLyFNVq4FKsRzMGeFFVx4nIhSJy\nQcgzHngPGAl8CTyuqmOBZsDnIjI8pL+pqgOStu3UMurWTU3Ig+27dzcT4kx23x2GDYOGDeHbby1t\nl11sH82BrFwJZ59t1l69e9vczk8/WVyXTz+1+nv0MM/IAL/6le2jnsvXwTH2oEHmGHPjjc1q7Pvv\n0/M1bZqyBvvLX6wH5TjrI+VpG6APsF8hNVihN/wLcf1iwQLV3/1OtUOH9PQ5c6wH8fHHqoMGWVrU\nI/nlF+vtgJWL0q+6KlUeVAcPtuNHHlE96ihL+9e/UvkffNCu77VXKm3MGNVly+y4e/eqv3/HKQAU\nuGeSZAJ+f+A0EZkKLMGGvlRVC7BIwHEqQdOm5uIlk8hv2GGHlb7WuLHt99zTVuCDWZ1FxgIAAwbY\nXAzYIshOnaznMmKEpXXrZibMYMYF/ftbj2jx4lSwsf79bZ7m6qvzusU0dtzR5oCiKJpRT2kDt5p3\niockE/AtsqWr6rQqkagS+AS8UyY//ghbb50yKRaBnXZKWYCVxfnnm4XZ//5nQ1377WdDa8OHp0/i\njxlj7vsbN7Y6+/c3a7SvYr5Qo6G7k04yZ5jXXptc/nh4gF694IILrL4bb7RhN8epJNXmTkVENhKR\nK7HV712A6WqLD6cVkyJxnJxst1362pT69aFJk/LLPfGEzeEsXmw9gSuusHUzLVrYy3zOHMvXtm2q\n19OqlSmMqVNLL64E603dnnS9byCag2nePDU/BL5q3yk6cvWT+wD7YtZVRwP3V6aBPH1zlVvWcSrE\npEnwwQfJ899xhy3K3H13izwZDWfVqZPKI5LqfTRvbqbKM2faeZT/lltsn8SFzJQpqfwLF5r584EH\nmmK5/vr0+hynWChrMgUYFTuuCwyr6IQMpqwi0+B6mGlwm4w8TTFLr23D+RZJy8bqKMyMlOOoqvbr\np7piRfZru+9uE+2RCbNqaiI+ft6vn+r48arDh6suWmTpb7yh2ratmS+vXKn63nvZ23j44VSdTzyh\nOm2a6ty5dn7DDaqtW6vedpvqlCkFuV1n/YRqNA0uiSmcyvap8/HNlaSs4xSWrl1tRf+8edmvP/SQ\nzV1IbKj5hhvSJ/IvusiGx9q1sxX6kTuZ446Djh3NfHnePDjqqJQfsziLF6eOTz/d5kw23xzmzrXw\nyRMmwP/9n03MO06RkEuZ7Ckiv4RtEbBHdCwivySsPx/fXEnKOk5hiYalttkm+/XDD7dJ8Dh33WXz\nJBGPPmqr81etsjgucaJJ88ifWbt2pdt44AGbpO/ZM331fRQfBlLrYhynSMgVtrdOWdeqQIZSvrkq\nWon75nIKQqG8B3/yie0zI0tGk/VzQwd8q63Sr8+ZY9v//V9uQ4G334a+fcu+PneuuZ2ZNat0G856\nSY375sqr8jx8c2Gu7nOWjdWhVXkfznrE9Ok26b3bbvnVM3u2RaDs2bNi5caMsfUts2aVn3fhQnjy\nSfM9lrnmZOBAiy/zwAMWX2ajjWDZMpvcf+01s06Le3MuD1XrrTVrBt98U7F7qigLFpgHgt/8pmrb\nWc+pVt8uyD5KAAAS5klEQVRc+W7k55ur3LKxOvKZh3Kc4mDECNWzz06e/+23bVL+sstSvslmzrRr\nr7+emsSP/j8if2ag+tFHtn/ppbLrHzMmZYgwY0aqbPv25cs2d64ZIFSGv/0t3aChLM44I+U7zakw\nVKdvrgIoqkr75iqrbFXK6zjVzurV5rEY0gOCJeGoo1J17LmnHUfhALbeGh5/PN0wYPny1HHkiv/m\nm8uuv21b6/mAzd38/vfwyCO2eLMsVOGpp2yYr02b7OttyuPSS83vWuRtIBuzZ8Ozz5rJdibHH29+\n3ZzqpZCaqaY2vGfi1FYuuki1eXM7bt1a9aabKlZ+333TeyDR9vrrdn3SpNRX/mWXqd59t+p++1na\nBReozpql2rt3Kn+ca65Rveuu9LRVq6zs/Plm3jxkiPWKttzS/Jndfbfqxhurvvyy5Rs+vGL3E7H1\n1imPz9m4/HKr/847VV94wdKeeir9GbjpdE6orp5JZLUVs+BaVAlrLsdxctG5s8VGef99mDgx3Sos\nCUOH2rwImKuX886z46ieyHx4yhSb72jfHj780EyTe/WyHsS558KJJ5rxQbw3MGOGuW0ZPBjGj7e0\naLHmIYfYgs4OHWxuY84c672MH2/1RO5k3n0XzjzTLN5ETNann4Y//hGmBUcaffrYfYD5P+ve3Xyk\nTZiQkuX2220+a9EiOz/hBAt+9u9/W13z5kFkhNOrl80h1atXsWfp5EchNVNNbXjPxKmtzJ+f+pI+\n8sjUAseKsmyZ7cePt7pOOy11bckSXestORtxD8qPP55KHzw4/Us/YtAgO7/0Utt36WL7I45I5T3p\nJNUPPlC98UY733tv219/fXqdM2faQs5nn7X5mQ03VL3vPtXzz1d97LHUvUXP5+qrVUeOVJ03z64d\neaRda9nS9r/9rXmPLiQ//qj6v/8Vts4igAL3TJK+rA8Gzg7HWwA7FlKIvG/ClYlTmxkwQHX27MLV\nN3du6YnpbMHBIsaNU/3uO9U77lC97rrUZL6q6ujRdhxXMqoWtAxUFy40BQiqXbum2rnjDsv39dd2\nftddtp8+PV2ZRNvHH6eUXr9+qkuXprwMPPWU6nbbqd5/fyr/McfYtaVLVXfdNff9qao+95zqYYdV\n6DGu5ZFHrO6SksqVz2T1antuNUy1KxOgJ/AmMDGcNwcGFlKIvG/ClYnj5ObVV1Nf82XRt6/q0Uen\nXsyrVql+/70dz5qVnjeyFovqXLlSdfly61WUlKRHzPz++1S5SFFNmGA9l5497XzxYrsONg8Tp00b\n61WNHJmSbeed0/PsvbdZq2WyYoX1erIpm//8R3WffXI/k3PPtXsDm5u55hrr+cydm7tcLiJZVq+u\nfB0FoCaUyQgshsnwWNrIxA2Yx+HxwETghizXDwMWAMPCdlPs2lTgG2A4MCRHG4V9yo6zPjJmTOpF\nF/+fGjEie/5Zs9J9lCXhv/9VPfHE0unRi3Xq1FSdrVvby/vEE21Cf+VK1bp1syuG1atLv5zXrFEd\nOjT9nuIKtXt3LdXjWLNG9ZJLzBAiGjK88sr03s8OO5RuvyL06mXlv/mm8nUUgJpQJkPCfljYb5xU\nmZDM0eNhwBtllJ8MbJqgnQI+YsdZjzn4YOvFVNVX8+zZqmPHlk7PpiCitH79UpZZoDp5simW8ojK\nn366av/+dvzAA6nr111Xus2+fdOVT7Q9/LBqu3bZle38+aWHwMpTsmecofrVV+lpL71UugeYDxMn\n5pyDqwllci3QK7zYzwcGAZclqhwOAPrHzm/M7J0EZfJmGeWnAJsnaKcyj9pxnEwGDrTeQXUTvaCX\nLEmlffWVpY0bl0obPTp5byjzpb98uWrTplaHqi3cHDDAjn/+2fazZ6crjKeeSh92AzPlvuee9LSO\nHVPnJSW6dpgwzg8/WPqUKRZGesQI6zk1bWpKDlQvvDDZvUWsWaN67LHZletBB6l+9lmZRQutTMoN\n26uq94lIZ+AXzCnjzar6fnnlAtmcNXbIku/XIjICc6FynaqOjZoH3heR1dhixicStus4TmU48MCa\naXflSnMh07BhKq19e3NJs+uuqbS2bZPXOXGiuZyJqF/fztu1g7feMvPmJk2gRw8zXR461OLdjBlj\nJtK77w5nnZXuITryh/bb39qizObN7Tzu8+qzz2wvYos269Qx9zBRdM+DD7YIoB98YPuFC80lDpTv\n4qZZM3OPE8WzmTXL7uXPf4Z7703Pu3y5udGpJpLEgAdzcdIAe7mPKrAMXwM7qOpSETkaeB1TWgAH\nqepMEdkSUyrjVPXzbJW4o0fHqcXUq5eKcR8hYi/OyrLLLqXTGjaEpUvh2GNNqYwalYqaOXq0rcHp\n1g1KSuD559MVCdg1sH7LhAmptTCffgojR5ojz969Le244+Cdd0wx7buvpT37bMrn2NZb23qcBg1g\nr71Med5/fyq65rJlptiisiUltvL/1ltNAT3wAGyyiXk5aN0avvjClEqfPpa2ZInVHahqR49JhqrO\nA74HnsGiL04FzknS7cGGud6NnZca5spSZgqwWZb0nsDVZZTJ2RN0HMdRVRsOitbPHHKIpV11la41\nZ77++uR1DRmipSzRQPU3v0kfKnvmmdJDbqqqH35oMsyYYWtp+vdPXwv03ntmhNCtm+prr1nannum\n6vrkExuyO/NM1SZNUumvvmp5d9opp8k5NTBnMoHYvAWwOTAhUeXJHD02ix13AKaG44ZAo3C8MTAQ\nOLKMdsp8YI7jOKW45RbVAw6w4wULzO1Lhw7pcyEVJVIiJSU293PxxWZGDKrbbJOaq4mYMMGuLV9u\n5yUlNoczaJCdP/mk6qGHWp4WLVLlJk9OV1ZRFM5oixZ7lkOhlUkSR4/zgEWx80UhrVw0gaNH4Pch\n9vtw4EHgDyG9GfB5SP8Sm6QfkKRdx3GcnNx8sw0LATRtasNMQ4bA2LG5y+XiwQetnjp1bDjtn/80\nNzYAb7xRer5n2xDrLwrIVrcuzJwJDz9s5zNmpIb+7rzThtxWrDAXOcuX2/zWHntY0LT69S2K54kn\n2lBd5vBcNVBmPBMRuToc7gXsDvTD5kxOwEyDz6oOAZPg8Uwcx8mbRx6Bo4+GVq0KV+e8efaCjybl\nM5k6FVq2TJ0PGmSxZoYMMeVzwQV2HimH+HuuY0eLl3P44Tbv07ChKaTXXjP/Zj//DJtuWqZohY5n\nkkuZ9MxVUEMwq2LAlYnjOOsEc+ZYZMwZM8xS7N13LdTACy/At9+mnFmCKb033ywdyG31alMqo0fn\ntH6rNmVSm3Bl4jjOOoFqKmrmoYeaMolZZKXx44+lLeAihg41K7Acw13VrkyCWe71QFtgrdGyqnYq\nlBD54srEcZx1hv79bR1LFGq5iii0MkkyAf8C5ltrR+AWzDR4aKEEcBzHcWIcfbTtp0+vWTkqSBJl\nsrmq9gZKVPUTVT0HSNwrEZEuIjJeRCaKyA1Zrh8mIgtEZFjYbkpatjZRpYuFCojLWVhczsKyXsi5\ndKntc0yeFyNJlElJ2M8UkWNEZG9gsySVi8gGwCPAUdgwWXcRaZMl66eq2j5st1ewbK1gvfgnqEZc\nzsLichaWvORs0MCGumrAvDcfkrhTuV1EmgLXAP8AmgBXJqy/A/Ctqk4DEJEXMdPi8Rn5sj21pGUd\nx3HWHUSgS5ealqLClNszUdW3VHWhqo5W1cNVdR8gqSF2NkeP22bJ92sRGSEib4tIZOeWtKzjOI5T\nw1TKNFhEvlfVHRLkOxk4SlUvCOenAR1U9fJYnkbAGk05enxIVVsnKRurw025HMdxKkghrbmSeg3O\nJKkA04G40tkupK1FVRfHjvuLyKMislmSsrFytWtw0XEcZx0jyQR8NpL2BIYCO4tICxHZEOgGvBHP\nICLNYscdsN7Sz0nKOo7jOMVBmT0TEVlEdqUhWGyTclHV1SISOXrcAOgdOXq0y/o45ujxIsxqbBnB\n0WNZZZPfmuM4jlNdrBPuVBzHcZyapbLDXEVBsS1qFJGpIvKNiAwXkSEhbVMRGSAiE0TkvWBmHeXv\nISLfisg4ETmyCuXqLSKzRGRkLK3CcolIexEZGZ73g9UgY08R+TG2oLVL7Fq1yxjq305EPhSRMSIy\nSkQuD+nF9jwz5bwspBfVMxWR+iIyOPzPjJLgYLYIn2dZchbV8wz1bxBkeSOcV8+zLGRwlOrcMEUY\nBd6qhwXealPDMk0GNs1Iuxu4PhzfANwVjncDhmNDjS3DvUgVyXUwFkpgZD5yAYOB/cLxO5i1XVXK\nmDW6JvCrmpAx1Lk1sFc4boQFj2tThM+zLDmL8Zk2DPs6WOyiDsX2PHPIWYzP8yrgeeCNcF4tz7I2\n90zWLmpU1RIgWtRYkwile3snYOGOCfsTw/HxWLCwVao6FfgWu6eCo6qfA/PzkUtEtgYaq2rkl+3Z\nWJmqkhGyWw6eUBMyBjl/UtUR4XgxMA6zNCy255lNzmidVrE90+A/hPrYiy2Km1Q0zzOHnFBEz1NE\ntgN+CzyZIUuVP8varEyKcVGjAu+LyFAROS+kNVPVWWD/4MBWIT1T/ulUr/xbVVCubbFnHFFdz/tS\nsQWtT8a650Uho4i0xHpTX1Lxv3O1yRqTc3BIKqpnGoZlhgM/Ae+Hl1jRPc8y5ITiep4PANeRbjxV\nLc+yNiuTYuQgVW2PfRlcIiKHUNoirlgtHopRrkeBnVR1L+wf+P4almctYottXwauCF/+Rfl3ziJn\n0T1TVV2jqntjPbwOItKWInyeWeTcjSJ6niJyDDAr9Ehzrb2rkmdZm5VJ4kWN1YWqzgz7OcDr2LDV\nLAlraUL3cXbIPh3YPla8uuWvqFzVLq+qztEwaAs8QWoYsEZlFJG62Av6OVXtF5KL7nlmk7NYn2mQ\n7RfgY6ALRfg8s8lZZM/zIOB4EZkM9AU6ichzwE/V8SxrszIpqkWNItIwfAUiIhsDRwKjgkxnhWxn\nAtHL5w2gm4hsKCI7AjsDQ6pSRNK/ViokV+geLxSRDiIiwBmxMlUiY/jhR5wEjC4CGQGeAsaq6kOx\ntGJ8nqXkLLZnKiJbRENDItIA6IzN7xTV8yxDzvHF9DxV9c+quoOq7oS9Dz9U1dOBN6mOZ1lIK4Lq\n3rAvmAnYxNGNNSzLjphF2XBMidwY0jcDPghyDgA2iZXpgVlQjAOOrELZ/g3MAFYA3wNnA5tWVC5g\nn3Bv32I+1KpaxmeBkeG5vo6N/daYjKH+g4DVsb/1sPA7rPDfuYqfZ1lyFtUzBXYPso0Icv2lsv83\nNSRnUT3PWBuHkbLmqpZn6YsWHcdxnLypzcNcjuM4TpHgysRxHMfJG1cmjuM4Tt64MnEcx3HyxpWJ\n4ziOkzeuTBzHcZy8cWXi1CpEZI2IPBs7ryMic2Luto8TkevLqWMbEXkpHJ8pIv+ooAw9EuR5WkRO\nqki9hUREPhKR9jXVvrP+4crEqW0sAdqJSP1w3pmYszpVfVNV78lVgarOVNVT4kkVlOHPFcxfqxCR\nOjUtg1P7cGXi1EbeAY4Jx90xP0RAek8j9A4eEpGBIjIp6ikEFzyjYvXtEL7kJ4jIzbG6XgseoEdF\nXqBF5E6ggVjwoedC2hmSCorWJ1bvYZltxwlyjBWRx0VktIi8GynJeM9CRDYXkSmx+3tNLNjRZBG5\nRESuCvJ8ISKbxJo4I8g0UkT2C+UbigUi+1JEvhaR42L19hOR/2GrpR2nQrgycWobisWu6R5evHuQ\ncq0ezxOxtaoeBByHBQnKlmc/4HfAnkDX2PDQ2aq6X7h+hYhsqqo9gKWq2l5VTxfzHPtnoKOaR9kr\nErQdZ2fgH6raDlgInJzjviPaYvElOgB/Axareav+EvOjFNEgyHQJ5qcL4C/A/1T1AKATcF/wNQWw\nN3CSqh5ehgyOUyauTJxah6qOxiLDdQfeJre77ddDmXGk4jhk8r6qLlDV5cCrWNRHgCtFZAT2kt4O\n2CWkx9vrBPxXVeeHdhZUsO0pqhr1kr4O91UeH6nqUlWdCywA3grpozLK9w3tfwY0FpEmmAPSG8Xi\ncnwMbEjK+/b7qrowQfuOU4q6NS2A41SSN4B7gY7AFjnyrYgdl6V0SsXOEJHDMEWxv6quEJGPgI0q\nKGOStuN5VsfaWEXqYy+z3XgZjZ2vIf1/OltMEAFOVtVv4xdE5ABsPspxKoX3TJzaRvRSfgq4RVXH\nVKJsJp1FZJMw3HMiMBBoCswPiqQNcEAs/8rYJPWH2NDYZgAismkF2y4rfSqwbzjuWkae8vhDkOlg\nYKGqLgLeAy5f27jIXpWs23HScGXi1DYUQFWnq+ojSfLmOI8Ygg1vjcCGrIYB7wL1RGQMcAcwKJb/\ncWCUiDynqmPD9U/C0FEUaS9p22Wl3wdcJCJfYy7EyyJXvctFZBgWDfCckH4bdl8jRWQ0cGuOuh0n\nMe6C3nEcx8kb75k4juM4eePKxHEcx8kbVyaO4zhO3rgycRzHcfLGlYnjOI6TN65MHMdxnLxxZeI4\njuPkzf8DWLE0Mi4j5sAAAAAASUVORK5CYII=\n",
|
|
"text/plain": [
|
|
"<matplotlib.figure.Figure at 0x2ec1f9ccb38>"
|
|
]
|
|
},
|
|
"metadata": {},
|
|
"output_type": "display_data"
|
|
}
|
|
],
|
|
"source": [
|
|
"pred_basic_model_dropout = train_and_evaluate(reader_train, \n",
|
|
" reader_test, \n",
|
|
" max_epochs=5, \n",
|
|
" model_func=create_basic_model_with_dropout)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"Add batch normalization after each convolution and before the last dense layer:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 22,
|
|
"metadata": {
|
|
"collapsed": true
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"def create_basic_model_with_batch_normalization(input, out_dims):\n",
|
|
"\n",
|
|
" with C.layers.default_options(activation=C.relu, init=C.glorot_uniform()):\n",
|
|
" model = C.layers.Sequential([\n",
|
|
" C.layers.For(range(3), lambda i: [\n",
|
|
" C.layers.Convolution((5,5), [32,32,64][i], pad=True),\n",
|
|
" C.layers.BatchNormalization(map_rank=1),\n",
|
|
" C.layers.MaxPooling((3,3), strides=(2,2))\n",
|
|
" ]),\n",
|
|
" C.layers.Dense(64),\n",
|
|
" C.layers.BatchNormalization(map_rank=1),\n",
|
|
" C.layers.Dense(out_dims, activation=None)\n",
|
|
" ])\n",
|
|
"\n",
|
|
" return model(input)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 23,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Training 117290 parameters in 18 parameter tensors.\n",
|
|
"\n",
|
|
"Learning rate per minibatch: 0.01\n",
|
|
"Momentum per sample: 0.9983550962823424\n",
|
|
"Finished Epoch[1 of 5]: [Training] loss = 1.536584 * 50000, metric = 55.22% * 50000 12.978s (3852.7 samples/s);\n",
|
|
"Finished Epoch[2 of 5]: [Training] loss = 1.215455 * 50000, metric = 43.35% * 50000 12.196s (4099.7 samples/s);\n",
|
|
"Finished Epoch[3 of 5]: [Training] loss = 1.092067 * 50000, metric = 38.66% * 50000 12.260s (4078.3 samples/s);\n",
|
|
"Finished Epoch[4 of 5]: [Training] loss = 1.011021 * 50000, metric = 35.57% * 50000 12.330s (4055.2 samples/s);\n",
|
|
"Finished Epoch[5 of 5]: [Training] loss = 0.952613 * 50000, metric = 33.38% * 50000 12.286s (4069.7 samples/s);\n",
|
|
"\n",
|
|
"Final Results: Minibatch[1-626]: errs = 30.1% * 10000\n",
|
|
"\n"
|
|
]
|
|
},
|
|
{
|
|
"data": {
|
|
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYwAAACfCAYAAADqDO7LAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztnXmYVNW1t9+fqIgBBYMTouAQTBSNccIZJc4aNSbG6Cdx\niMYYFaNXY9BcUHON93rVkGgSr3PEoHEWNYojKhpwAASZkRYQBUERZRTo9f2xTlnV3VXVVU1VV3Wz\n3uepp87ZZ52919nQZ9Xaa++1ZWYEQRAEQWOsU2kFgiAIgpZBGIwgCIKgIMJgBEEQBAURBiMIgiAo\niDAYQRAEQUGEwQiCIAgKIgzGWoqkv0m6slhZSb0lzS6vdl+3WyOpT3O01RpI/m3Gl1q2CXq8Juln\n5ag7qCzrVlqBoLRI+gDYAuhiZp9llI8Bvgt0N7NZZnZeoXVmkW3S4h1J3YAaYF0zq21KHa0FSQcA\nz+B9uQ6wIbAYUFK2k5l9WEydZvYKsEupZYMgRXgYrQ/DX8qnpAok9QTa0cQXfQlJvQxV9oakNuVu\nY00wsxFm1sHMNgJ2xvtl41RZfWOhhIooGwQJYTBaJ4OB0zPOTwf+nikg6W5J1yTHvSXNlnSJpHmS\n5kg6I5tsukj9Jc2XNEPSqRkXjpY0WtIiSTMlDcy475Xk+3NJX0jqldxzjqSJSdl7knbLuOd7kt6V\ntFDS/ZLWz/bAkk6XNELSTZLmAwMlDZQ0OEOmm6RaSesk5y9Luia57wtJz0raJEf9EyUdnXHeRtIn\nknaT1FbSYEkLEj1HSdo0Wz2NUMcgJEM710h6A/c+tpb084y+mibp5xny35dUk3E+W9LFksYlev1D\n0nrFyibX+0v6OJE7O+nHbRp9IGeApA8kzZV0l6QOybV2STupfhuZ6v/kOWuS55wu6SdN6M+gxITB\naJ2MBDpI2jF5OZ4M3Ef+X/ZbAB2ALsDZwF8kbZxHdpNE9gzgNknfSq4tBvqa2cbAMcAvJR2XXDso\n+d4o+RU9StJJwADgtOTX9nHApxltnQQcDmyLD6mdkecZegHTgc2Ba5Oy+l5V/fNTcIO6KdAWuDRH\n3UOAUzPOjwTmm9nY5P6NgK3wfvklsCyPnsVwGv7MGwFzgLnAUUlfnQPcnHiQKeo/30nA94HtgD2B\nvsXKSjoWOB/oDfQA+mS5Nxfn4P12ELA93j+Dkmtn4p5vl6T8V8DyxKDcCHw/ec79gXEFtheUkTAY\nrZeUl3EYMAn4qBH5r4Dfm9lqM3sGf/HvmEPWgP80s5Vm9irwNPATADN71cwmJMfvAQ/gL5pMMg3X\nz4HrzWx0cs8MM8sMqv/JzOaZ2efAk0Cm91GfOWb2VzOrNbMVjTxvirvN7P1E/sE89d8PHCdpg+T8\nlKQMYCXwTaCHOWPMbHGB7TfGXWY2Nfl3WW1mT5vZTAAzGw68CByY5/4/mtl8M1sIPEX+/sslexJw\nZ6LHMuDqIvQ/FbghiZstAa4gbXhXAp1J99toM1uaXKsFdpHUNvn3n1xEm0GZCIPRerkP/8M8A7i3\nAPlP6wWilwLtc8guNLPlGecz8V+JSOol6aVkuOZz4Fz8pZCLrYH381yfV6BOAE2ZvTW3kPrN7H1g\nIvADSe1wT2hIcnkwMAx4QNKHkv5bpYuh1HkmSccmQzefSlqI/yDI17/F9F8u2S719JhN4XGoLvj/\njxQzgbbJkN09wAvAg8lQ1x8krWNmX+IG+QJgrqShGR5sUEHCYLRSzGwWHvw+Cni0xNV3Sl6aKbYh\n7cH8A3gc2MrMOgL/R/rlkm0YYzY+VFEK6te/BJ99lGLLNaz/AdwIHw9MMLMZAGa2ysx+b2Y7A/sB\nPwBKNa3062dKvJuH8OG2Tc2sE/A85Z9E8DHQNeN8GwofkvoI6JZx3g1YkXgyK83sGjPbCTgAOBH4\nfwBmNszMDsOHP9/H/x8FFSYMRuvmLKBPMoxQSgRcLWk9SQfisYoHk2vtcQ9kpaS9qTvuPx8fasg0\nEHcAl0raHUDS9pK2LpGeY4GDJG2dxGN+u4b1PYDHU84j7V0g6WBJPZN40WJ8qKXYacOFvPTbAusB\nCwBLYgvfL7KdpvAg8HNJPSRtCPyuiHvvBy5JJhx0AP6LpO8kHSJpZ0kio98kbZF4Uu2AVbjhX13K\nBwqaRhiM1sfXv/zMrCYVG6h/rZh6svAxsBD/9TgYONfMpiXXfgX8XtIi/MXyzwx9luG/jl+X9Jmk\nvc3s4aRsiKQvgMfwAGix+jZ8ALMXkvbHAW/hMZA6IkXWNxf4N7APGc+F/wp+GFgETABexvsltejx\nr4VU31iZmS0CLsY9uE/xX+T1n6mxOouWNbOngL8BrwJTgBHJpVxxosy6bsf76jV8QsIi4NfJtS64\n97sIGA88hxuTNsBl+P+v+cC+eNA9qDAq5wZKkrri4+eb47+4bjezP+eQ3Qt4AzjZzEo9hBIEQYlI\nZmW9Y2ZtK61L0LyU28NYBVySjO3uC5wv6dv1hRJX/r/xwGEQBFWGpBOSIchN8L/VxyutU9D8lNVg\nmNncZJ46yTTDSfhc9fpciLv0n5RTnyAImsz5eOxkKj6D6oLKqhNUgmbLJSWpOz6ve1S98i7ACWZ2\nSBIkDYKgykhmLAVrOc1iMCS1xz2Ii7IsaBoEXJ4pnqOOSudBCoIgaJGYWUmmXpd9lpSkdXFjMdjM\nnsgisie+4KkG+DGekuK4LHKYWdV/Bg4cWHEdQs/Qs6XqGHqW/lNKmsPDuAuYaGZ/ynbRzLZLHUu6\nG3jSzIY2g15BEARBEZTVYEjaH1+5OV6+H4PhuWS6AWZmt9W7JYadgiAIqpSyGgwzex1fhFOo/Fll\nVKdZOPjggyutQkGEnqWlJejZEnSE0LOaKevCvVIiyVqKrkEQBNWCJKylBL2DIAiC1kGLMxhTp8Kn\nnzYuFwRBEJSWFmcwfvc7ePHFSmsRBEGw9lFWgyGpa7KZzgRJ4yX1yyJzqnzP5nfleyvvkq/Ojh1h\n4cLy6RwEQRBkp9zrMFLJB8cmq73fkfSc1d1ucQZwkJktknQkng55n1wVduoEn39eXqWDIAiChpR7\nWu1cki0wzWyxpFTywckZMiMzbhlJ9uSEX9OxYxiMIAiCStBsMYxcyQfrcTbwTL56wmAEQRBUhmpI\nPpiSOQQ4E9/bNytXXXUV06fDnDkwfPjBa+XCmSAIgnwMHz6c4cOHl6Xusi/cS5IPPgU8kyuflKRd\ngUeAI83s/RwysXAvCIKgSFrawr28yQclbYMbi765jEUQBEFQecq9p/f++Mbx4/HEgg2SD0q6Hd/M\nfia+F8ZKM2uwkVJ4GEEQBMVTSg8jckkFQRC0YlrakFQQBEHQCmiRBuONN2DVqkprEQRBsHbRIg3G\nySf71NogCIKg+WiRBmOrrcJgBEEQNDcVTz6YyP1Z0jRJYyXt1li9XbqEwQiCIGhuKp58UNJRwPZm\n9i1JvYBbyZN8EMLDCIIgqARl9TDMbK6ZjU2OFwOp5IOZHA/cm8iMAjaWtHm+erffHqZPL4PCQRAE\nQU6qIfngVsDsjPM5NJKxdq+9oHPnUmoXBEEQNEbVJB8shKuuuurrY088ePAaahYEQdC6aNXJByXd\nCrxsZv9MzicDvc1sXj25WOkdBEFQJC1tpXfe5IPAUOBnAJL2AT6vbyyCIAiCylPx5IOJ3C3AkcAS\n4EwzG52lrvAwgiAIiiSSDwZBEAQF0dKGpMrCypVwxx2V1iIIgmDtocV6GKtXw0Ybwdy50KFDBRUL\ngiCoYsLDANq0gaVL4Z57Kq1JEATB2kG5c0ndKWmepHE5rm8kaWiSQ2q8pDOKqf/oo+H++0uiahAE\nQdAI5Z4ldQCwGLjXzHbNcr0/sJGZ9ZfUGZgCbG5mDXa7yBb0njkTuneH+fNj5XcQBEE2WsyQlJmN\nABbmEwFSEYgOwKfZjEUuunWDc87x4akuXaBv3zVQNgiCIMhLc6z07gY8mcPDaI8v3Ps20B442cye\nyVFP3mm1ffrAyy9DbS2oJLY0CIKg5VNKD6NZcknl4QhgjJn1kbQ98LykXXPlm6qfS8rzSTkvvgib\nbZYepgqCIFgbaem5pPJ5GE8B15nZ68n5i8DlZvZ2FtlGF+6deqqvz3joodLoHgRB0NJpMTGMBCWf\nbMwEDgVI9sDoAcxoakNnnAH779/Uu4MgCIJ8lHuW1BA8B/k3gXnAQGB9kjxSkrYE7gG2TG65zsyy\nTpSN1CBBEATFE7mkgiAIgoJo9iEpSdtLapscHyypn6SOpVCgHKRmSkkQNiYIgqA0FBrDeARYLWkH\n4DZga2BI2bRaQzKn1Y7LusY8CIIgKJZCDUZtsqDuh8DNZnYZ6bhD1SG5l3H99bD55vD66+FpBEEQ\nrCkFxTAkjQIGAVcCPzCzGknvmVnPciuYoUOTYxjt2sHbb8POO5dYqSAIgiqnEtNqzwT2Ba5NjMW2\nwODGbmos+WAic7CkMZLek/RygfoUxX77wYcf+vHll8OoUeVoJQiCoHVT9CwpSZ2Arc2s0ehAAckH\nNwbeAA43szmSOpvZghx1NdnDSMU05s+HTTf145qaWBEeBEHrpxKzpIYnqcg3AUYDt0u6qbH7Ckg+\neCrwiJnNSeSzGos1ZcQI/549Ox3L2HbbcrQUBEHQeik0l9TGZvaFpLNxb2FgvmGmIugBrJcMRbUH\n/mxmjQ51Fcv++9cNek+eDI89VupWgiAIWjeFGox1k1XZP8ED36Vsf3egD/AN4N+S/m1m07MJ50s+\nWAw77gi//S0sXw5bbw1HHQX33tukqoIgCKqKiicflHQS8J/A62Z2nqTtgP81sx8VcG++5IOXAxuY\n2dXJ+R3AM2b2SBbZsqz07tsX7rvP9wbffPOSVx8EQVBRmj2GYWYPmdmuZnZecj6jEGORkC/54BPA\nAZLaSNoQ6AVMKrDeknD33dCjR8ycCoIgaIxCg95dJT0m6ZPk84ikrgXcNwSfBdVD0ixJZ0o6V9Iv\nAMxsMjAMGAeMBG4zs4lNf5ziWXddOOIIePXV5mw1CIKg5VHokNTzeCqQVED6NOD/mdlhZdStvg5l\nSz64bBmsWgUdOsDHH8OWVbuGPQiCoDgqsXBvUzO728xWJZ97gE1LoUA10K6dG4s//9n3Bo/1GUEQ\nBA0p1GB8Kum0JNbQRtJpwKflVKwSnHgi7LGHb/P6aZanW7IEpk1rfr2CIAiqgUINxln4lNq5wMfA\nj4EzyqRTxeja1XNOXXstLFjgazdmzoQvv/Tr48Z5gHxilijLa6/BBx80q7pBEATNSpM3UJL0azMb\nVGJ98rXX7Bsode/uBgPgD3+Aiy/24SvwvcPXTVaxrFoF663nx5EVNwiCaqJa9vS+pDGBQpIPJnJ7\nSVop6cQ10KfkDBkCP/0pXHUV/PznsMEG8Oijfm3KFPjnP+FHP/JU6n37evk771RM3SAIgrKyJh7G\nbDPbuhGZvMkHE5l1gOeBZcBdZvZoDrmq2aJ16lTYbru0V/Hhh7DVVnDSSXDFFfC971VWvyAIghSl\n9DAKTQ2SjUbf3mY2IlnpnY8LgYeBvdZAl2alRw//vvlm2GwzNxYADz2UXX7JEvjGN5pHtyAIgnKR\n12BI+pLshkFAuzVtXFIX4AQzO0TS3mtaX3NzwQWNy7z9NuyVmMK33oI99yyvTkEQBOUir8Ewsw5l\nbn8QcHnGeV63qVTJB5uTPfeEQw+FF15w47HHHnX3HM/GjBmwySbQsaPHR9bJiDR9/rmXB0EQZKPi\nyQfXqIH8yQdnpA6BzsAS4BdmNjSLbNXEMBrjvPOgWzf4j//wOMfixR4k33VX2GYbT3T42mtwwAF1\n71u1Cp57Do45Bg4+GK65xgPuL77o15cvT283u8cezf1UQRC0RKplllSh5Ew+aGbbJZ9t8TjGr7IZ\ni5bGnDnQvz+svz4sXQrt2/sLfr314IQTXObAA31qbm1t+r4FC9xYgMdH9twTXnrJjQjAbbf59+67\nN9+zBEEQpCirwWgs+WA9Wob7UAAPP5w+3nDDutf+9jdfRX7//b6Oo00bOPZYX/S3xRbQpw/cfjv0\n7OneRPfu8O9/+70XXeRGR4KFC2PNRxAEzUvZh6RKRUsakgJPIbLJJvDNb+aXS8UzTjnF133U57XX\n4KCD3Bvp3BmmT4d589ygHHssPPlk6XUPgqD1UMohqTAYFWbaNB+K2nff3DK77gpDh7rsoYd6DGPv\nZE5ZK+ySIAhKSBiMgI8+gn79fPjrvfdgl108mD5tGnz3u559F2D1ap9l1djMrCAIWictLegdlIEu\nXdxYmLmxAF9EePTRsNFGfr5qlcdJCtmvfPhwuOWWhuWPPOKB+0yWLfO8WqkAfhAEawflDnrnzSUl\n6VRJ7yafEZJ2Kac+rZGRI/17wAD3Im6/3c/bt3dj0a5d9uy6U6d6PGTpUlixAg45BC68sK7MokXw\n4x+7J5PJG2/AoEHwxBOlf54gCKqXsg5JNZZLStI+wCQzWyTpSOAqM9snR10xJJWDxx+Hww5Lpx/p\n2dPjHkOGeKr23/0uPTQFPpW3TRs//uUvYcIED64vWpT2TsAXCXbq5Meprl+82I3R/vt73S1g7WQQ\nrNW0qBhGvoV79eQ6AuNzJTQMg9E0li51Q/LKKz6MtHq1T+3929989fljj/msq0cegb//veH9M2f6\n1N7aWrjrLjj77MID7cuX+/cGG5TscYIgKJJqST5Yas4Gnqm0Eq2NDTf0QPiWW8Lo0V628cYN4xrH\nHeffL7wA55zje5tPmOBZeXv18oD6TTd5ffXTlWRj2TKX3WADP85k/HgPytffCverr/x7/fWb9KhB\nEJSZqjAYkg4BzgQOyCfXEnNJVQM77ODfDz7oL++ddsot27FjeufArbf2uMjIkT5kNXGiG51MY/F/\n/wf/+lc6nnHuuXD99WlvZfnyhvmvBgzwYbTMTajApwp36OBtgU8fXm89HypbtQpGjPAV9Km08oVS\nW+tttW1b3H1B0BIpZy4pzKysH6AbMC7P9V2BacD2jdRjQfPw+ONmo0fXLVu61Oyoo8yWL69b/oc/\nmIHZzJlmq1b5MZgtXmz2zDNmr71mVltb957bbnOZv/wlXVZbm763psbLUucXX5w+Xriw+Oc54QS/\nNwjWRpJ3Z0ne5xXNJSVpG+ARoK+Zvd8MugQFcPzxDTeBatfOPYn6v9L79/fvHXaAN9/04yee8LjJ\nkUd6gkXJZ2IdfbR7D+ec45tNrV7tZuA734FPPkl7No8+Cjfe6MczZsANN/jxoYfmz9R7//3pBY2Z\nPP64f7/1VnpYLgiC4inrkFSSS+pg4JuSZgEDgfVxi3cb8J/AJsBfJQlYaWYtbl+MtZ1bb/Vhn802\n8xlZqXhIihtugMsu8+NttvHEi1dc4XGR1PBW584+cysVUP/jH30BYvfubnAKCbQ//7wbhUyWLPFY\nSk2NG45Bg3z22G67+fV//AO++MIzDLc0Pv7YY1NB0GyUylUp94cYU2ixzJ/vQ0LXXVe3fOVKLz/w\nwMLrmjjRTDL74ouG14480ut77bV0WU2N2SWX+PHYsemhLbP0MNgee+Rv89NPG+peab780nWfOLHS\nmgTVDi1sSCpYy+nc2QPfl15at3yddXyF+auvFl5Xx47+yk8lXZw+3T0VM9hnH/cyMvcK6d49Pby1\n667u/fTt6+epfUbuvddnaL36qnszl1/u6VZqarz8wQd96G3lShgzxu9ZutSTQg4enN37ufxyrys1\n86vULF7s36edVp76gyArpbI85f4QHkaQcM01ZpddZvb552mPIZvH8f77ZitW5K5HMttgAz+eNi1d\n16mnpo8HDkwfP/982juZOjVdPm1a3XqHD09fW7So8Odavtzs0UfNPvwwv9xf/mK2ZIlPKjjooMLr\nD9ZOCA8jWJvZd1/fI2ThwnRZtrUbffumA97ZmDPHkziCB+1vvx2OOMLjGt/5jpcffbR/v/GG71UC\n7jl06QJnneUp7HfYwb2PqVN96u9++8Gvf+2LIlMr5+fMcX2//NI9rZqauptnAfTuDSeeCMOG5dZ5\n3jw4/3xfaNmrl3tFqQWSQVB2SmV5yv0hPIwgYdGiur/s60+1ra01u+ces512Mnv33aa1UVtrNneu\nH3/2Wbo85TksX2721Vc+fXj69LreSX1+9Sv7OlaydGlatlcvn5a8YIHLpWIwK1d62ylP45NPvI2a\nGrMHHjBr29bs7bf92gsv+HcqpnHVVU173qD1Qgk9jHK/5O8E5pF/Hcaf8XUYY4Hd8siVsg+DFs6k\nSbmv1daadepkX68HKTX115W8/nraCHz8cUP51LXUS15yIzJ/vtkhh5gNGJCWXb3av7/7Xb/nxhvN\ntt/ej08/3YPvv/lN3foz1788/bQPxc2bl74+a5bZDjt4e1ddZbbxxn5c7DPXX4OTi8WLzR5+OL/M\nmDFmo0YVp0PQNFqSwTgA2C2XwQCOAp5OjnsBI/PUVco+DFo5P/2p2Y47VloLZ9IksylT0ucpo2Dm\nL1bwxYWZDBrk5X36mL36qh//4hdm/fub3XRTwzY22SQ9G6xrV5efNcvPMxdEpo5XrPDvQl/aM2ak\n722Me+7JL5fpkb3zjntSF12UP95UDr76ymzEiOZtsxK0GIPhuuZe6Q3cCpyccT4J2DyHbAm7MGjt\n1NY29ASqkSVL/K/wllsaXvvkEx9+W77cbN11/aU9bJjZBx/krzPT4xk92r+ffNKv9erlgfJJk9Iy\nkyenj5ctMxsypGGdy5alZUaMSMukDE/mn+f555tdeWVu/RYvNrv6arPOnc0uvDB9fzbjlTKcxXLJ\nJWYvv5xfZtgws/btm99QNTetyWA8CeyXcf4CsHsO2RJ2YRC0bu680z2shQvNrr8+u0xNjf+yHzYs\n/dJOHX/yicssXGh2661+/OWXHl+54QaX2XNPL0/dO2BAevbY+PE+mwvShru2tm48KMW8eS631151\ny1evTtedbahv2TKzCy4we++9htfAva4VK9yjycaAAS730kvZr+diwAAfUiyEyZM91U4lKaXBqGh6\nc0lPAteZ2RvJ+QvAb8ysQQIHSTZw4MCvzyP5YBCUBjNfNd6lSzqd/T77+Ey01Na+y5en08KsXJme\nlVZb65+bbvJV/O+/D1de6XWOG+er9UeN8qSSqaSX2V45b77piTF793YdttzSv3v0gKef9v1e6ied\n7NcPbr45e53HHedrdYYO9eNhw+Dww+vKXHopPPusz7pLbTyWr49SfdG2ra+vaezV+dJLnrHgqadg\n9mzo2jW/fKmon3zw6quvxkqU3rzSHkb9IanJxJBUEFQUMOvd26xfP6sznJXJBx9kH/9fscJXxqdo\n187ryIxbLF3aePs9ezYsT80wO+YYP7/uOvs6DpIKyl9zjXseS5f6jLJUfam4TX2mTMnu9WQyZky6\njlWrzM49148ffdSvz5zpn0zefttlnn3Wv//nfxrWO2GC69oUBg3ySRCFQAsbkuqOb4yU7drRpIPe\n+xBB7yCoGt56y2zo0DWr44QTzDp08OPGXsxmnuok9XKePbvutW99K31t6tS61yZMSF+rH7tKLfB8\n4QWzAw7w48YWR5qZXXutt5OaRJD5CnrsMbPbb/c+qn8t1V63bn7er5/PXttsM7NXXknLgdmllzau\nRzZSbQ4eXIhsCzEYwBDgI2AFMAvf8+Jc4BcZMrcA04F3yRG/sDAYQbBWsGCB2YMP+ou4PpMmeeB/\nypSGRmHWLH+bnXhi9nqffNI9kdSLtl+/hjIrV5rdd59nDRgyxL6O0yxZ4jGUk05qeE+qvqeeSs94\nS3k+mdkHMrMSjBqV9rwy4zup4y++8GnhqThSJl995d8PPNCwjVy0GINRyk8YjCAI8lHIrLjaWk/d\n8uWXdcs/+yz9Qr/lFjcUkB7WysWCBek1L6n7hw/PLjt+vH3tOYHZySd7+aWX+iLTM8+s67HMn++z\nyFI89ZSXP/hg4c/repXOYJQ96F0qYk/vIAjKxcSJsPPO8MMfwh13+DbG4Cn3C2XcOE+o2bNncW33\n7OnbIb/5Juy4o7d91lm+b8y++3pKmfbt00H/sWN9MkGhlHJP7zAYQRAEFeTpp+HYY31DsXXWcYPQ\no4fv43LUUTBrlu/zUlPjM68uvLC4+sNgBEEQtBJWr4bJk93Dqc+ZZ8I99/gU4222aVr9YTCCIAjW\nAszcs9huu6bXUUqDUfb05pKOlDRZ0lRJl2e5vpGkoZLGShov6Yxy61ROMhfMVDOhZ2lpCXq2BB0h\n9MxEWjNjUWrKajAkrYNPmz0C2Bk4RdK364mdD0wws92AQ4AbJZV1r/FyEv/ZS0voWTpago4QelYz\n5fYw9gammdlMM1sJPAAcX0/GgA7JcQfgUzNbVWa9giAIgiIpt8HYCpidcf5hUpbJLcBOkj7CF+9d\nVGadgiAIgiZQ1qC3pB8BR5jZL5Lz04C9zaxfPZn9zOw/JG0PPA/samaL69UVEe8gCIImUKqgd7lj\nBXOAzMlgXZOyTM4ErgMws/cl1QDfBt7OFCrVAwdBEARNo9xDUm8BO0jqJml94KfA0HoyM4FDASRt\nDvQAZpRZryAIgqBIyuphmNlqSRcAz+HG6U4zmyTpXL9stwH/BdwjaVxy22/M7LNy6hUEQRAUT4tZ\nuBcEQRBUlrIv3CsFjS3+a2ZdPpD0rqQxkt5MyjpJek7SFEnDJG2cId9f0jRJkyQdnrvmNdbrTknz\nMjy1JuklaXdJ45K+HtRMeg6U9KGk0cnnyCrQs6uklyRNSBaU9kvKq6ZPs+h4YVJeVf0pqa2kUcnf\nzHhJA5PyqunLRvSsqv7MaGOdRJ+hyXn5+7NUaW/L9cGN2nR85771gLHAtyuozwygU72y/8GH0gAu\nB/47Od4JGIMP/XVPnkNl0usAYDcydjdsil7AKGCv5Phf+Cy3cus5ELgki+x3KqjnFsBuyXF7YAo+\nGaNq+jSPjtXYnxsm322Akfgararpy0b0rLr+TOq9GLgPGJqcl70/W4KHUcjiv+ZENPTMjgf+nhz/\nHTghOT4OeMDMVpnZB8A0/HlKjpmNABauiV6StgA6mNlbidy9GfeUU0/wfq3P8RXUc66ZjU2OFwOT\n8Fl+VdPMib1nAAAF5ElEQVSnOXRMrXOqtv5cmhy2xV9cRhX1ZSN6QpX1p6Su+I6ld9TTp6z92RIM\nRiGL/5oTA56X9Jaks5Oyzc1sHvgfMbBZUl5f9zk0r+6bFanXVnj/pmjOvr5Ank/sjgxXuir0lNQd\n94pGUvy/dbPomqHjqKSoqvozGT4ZA8wFnk9eUlXXlzn0hCrrT+CPwGWkDRo0Q3+2BINRbexvZrvj\n1v18SQdS9x+NLOfVQrXq9VdgO/N8YnOBGyusz9dIag88DFyU/Iqvun/rLDpWXX+aWa2ZfQ/30vaW\ntDNV2JdZ9NyJKutPSccA8xLvMt/6tJL3Z0swGIUs/ms2zOzj5Hs+8Dg+xDRPvoaExM37JBGfA2yd\ncXtz616sXhXR18zmWzKICtxOetiuonrKk2A+DAw2syeS4qrq02w6Vmt/Jrp9AQwHjqTK+jKXnlXY\nn/sDx0maAdwP9JE0GJhb7v5sCQajkMV/zYKkDZNfc0j6BnA4MD7R54xE7HQg9XIZCvxU0vqStgV2\nAN4sp4rU/cVRlF6JG7tI0t6SBPws456y6Zn8505xIvBeleh5FzDRzP6UUVZtfdpAx2rrT0mdU8M4\nktoBh+Hxlqrqyxx6Tq62/jSzK8xsGzPbDn8fvmRmfYEnKXd/ljpyX44P/mtkCh6s+W0F9dgWn6U1\nBjcUv03KNwFeSHR8DuiYcU9/fFbCJODwMuo2BPgIWAHMwlOudCpWL2CP5NmmAX9qJj3vBcYlffs4\nPhZbaT33B1Zn/HuPTv4fFv1vXS5d8+hYVf0J7JLoNjbR68qm/t1USM+q6s96OvcmPUuq7P0ZC/eC\nIAiCgmgJQ1JBEARBFRAGIwiCICiIMBhBEARBQYTBCIIgCAoiDEYQBEFQEGEwgiAIgoIIgxFUHZJq\nJd2bcd5G0vyMNM4/kPSbRurYUtKDyfHpkm4uUof+BcjcLenEYuotJZJelrR7pdoP1j7CYATVyBKg\np6S2yflhZCRPM7Mnzez6fBWY2cdm9pPMoiJ1uKJI+RaFpDaV1iFoeYTBCKqVfwHHJMen4DlzgLoe\nQ/Ir/0+SXpc0PfWLP0klMz6jvm2SX+RTJA3IqOuxJPPw+FT2YUnXAe3km9MMTsp+pvTGWX/PqLd3\n/bYzSfSYKOk2Se9JejZlCDM9BEnflFST8XyPyTfDmSHpfEkXJ/q8IaljRhM/S3QaJ2mv5P4N5RtV\njZT0jqQfZNT7hKQX8RXBQVAUYTCCasTwfU9OSV6uu5JO250pk2ILM9sf+AG+iUw2mb2AHwLfBU7K\nGMo508z2Sq5fJKmTmfUHlprZ7mbWV56x9ArgYPNMphcV0HYmOwA3m1lPYBHwozzPnWJnfG+CvYFr\ngcXmWZJH4jl/UrRLdDofzysFcCXwopntA/QBbkhyIwF8DzjRzA7JoUMQ5CQMRlCVmNl7+O5gpwBP\nkz+N8+PJPZNI7wFQn+fN7HMzWw48iu/8B/BrSWPxF3FX4FtJeWZ7fYCHzGxh0s7nRbZdY2Ypb+ed\n5Lka42UzW2pmC4DPgaeS8vH17r8/af81oIOkjfCkmL+V7+swHFifdMbn581sUQHtB0ED1q20AkGQ\nh6HA/wIHA53zyK3IOM5lWBrsvSCpN24MepnZCkkvAxsUqWMhbWfKrM5oYxXpH2312828xzLOa6n7\nd5ttTwkBPzKzaZkXJO2Dx4eCoEmEhxFUI6kX713A1WY2oQn31ucwSR2ToZkTgNeBjYGFibH4NrBP\nhvxXGYHhl/BhrE0AJHUqsu1c5R8AeybHJ+WQaYyTE50OABaZ2ZfAMKDf141LuzWx7iCoQxiMoBox\nADObY2a3FCKb5zzFm/hQ1Fh8eGk08CywnqQJwB+Af2fI3waMlzTYzCYm119JhnlSO64V2nau8huA\n8yS9g6emzkW+epdLGo3vCndWUv57/LnGSXoPuCZP3UFQMJHePAiCICiI8DCCIAiCggiDEQRBEBRE\nGIwgCIKgIMJgBEEQBAURBiMIgiAoiDAYQRAEQUGEwQiCIAgK4v8DHuL7yDM7MD4AAAAASUVORK5C\nYII=\n",
|
|
"text/plain": [
|
|
"<matplotlib.figure.Figure at 0x2ec25a34e80>"
|
|
]
|
|
},
|
|
"metadata": {},
|
|
"output_type": "display_data"
|
|
},
|
|
{
|
|
"data": {
|
|
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYwAAACfCAYAAADqDO7LAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztnXfYFNX1xz9fCygKNlSiYm9Rf/aWaASNNcaaWDCxRmM3\nGmPsYjSJmsRujLFhS+y9Y0ONBRVEUUFUQKWoiIgoiMh7fn+cGXbeZXffWdh3333hfJ5nnpm5c+fe\ns3d358y995x7ZGYEQRAEQUvM09YCBEEQBO2DUBhBEARBLkJhBEEQBLkIhREEQRDkIhRGEARBkItQ\nGEEQBEEuQmE0GJL+Jen0avNK6iHp49aVbka9IyRtU4+62huSmiStXO97Z5ds3dX8BkuUM0nSirWU\nLWgcQmHUCUkjJX0rafGi9NeTP+vyAGZ2pJn9JU+ZJfLOklONpBUSGeL3wGy3x+w4NpW9V1I/SVMk\nfSXpM0l3S1p6NuoqW3fe36CkZyQd0qwQs85mNrKGcqV1jZQ0Ofn8k5L9ZbWuJ6hMPCDqhwEjgF5p\ngqR1gAWZvYdMLVAig1q9Imne1q6jRszqdzI7bVjpXgOOMrMuwOrAosDFJQuZNUXX6t/9bGLAzmbW\nJVFKXczsuFIZS/3Gqv3dtaPfaV0JhVFfbgYOzJwfCNyYzSCpj6RzkuMekj6W9HtJn0oaLemgUnkL\nSTpV0jhJwyXtl7nwM0kDJU2U9KGk3pn7nk32XyZvbpsl9xwm6Z0k7S1J62fu2UDSG5ImSLpVUodS\nH1jSgZL+J+kiSeOA3pJ6S7o5k6fZG33y5npOct9Xkh4r7pll7n1H0s8y5/Mmb+DrS+oo6WZJnydy\n9pe0ZKly8iJpE0kvJuWNlnS5pPmKsu0s6YNEjr8V3X9IIvN4SY+mPcu81QOY2ZfA3cA6SZl9JF0p\n6WFJk4CekjpI+kfyXY9NrnfMyHGSpDGSRkk6mIyCLP5dSdot6QlPlPSepO0l/Rn4CXBF9m1fzYe2\nuki6KWmHEcoMcyW/i+cl/V3SF0l77Zjn88+U2Pw39jn+GyuVJklnyHsrn0i6QVKXpIz0N3iIpA+B\np/J+KXMToTDqy8tAZ0lrJA/HfYBbqPx21w3oDCwDHAr8U9IiFfIunuQ9CLha0mrJta+B/c1sEWBn\n4AhJuybXtkr2XZI3t/6S9gLOAn6dvNXuCozP1LUXsD2wErBeUl85NgPeB5YG0qGO4jf44vNeuEJd\nEugI/KFM2f8F9suc7wiMM7NByf1dgGXxdjkCmFJBzjxMB45PyvsRsA1wVFGe3YENk203JcM2knYD\nTkmuLwk8D9xarQCSugK/AAZmknsB55pZZ+AF4AJgVWDdZL8s/n2SPJh/D/wUWA3YtkJdm+IvNScm\nv52tgJFmdkYi/zFFb/vZ7/EK/Le7ItATOCBRTimbAkOAJYC/A9dV0w5FpL+xpSj8xorTDgYOAHoA\nKyeyXVFUzlbAmsAOsyHLHEsojPqT9jK2w/8sY1rI/x3+IJhuZo/iD/41yuQ14Ewzm2ZmzwEPA3sD\nmNlzZvZ2cvwWcBv+x8mSVVy/Af5mZgOTe4abWXZS/VIz+zR5230QyPY+ihltZleaWZOZTW3h86b0\nMbMPkvx3VCj/VmBXSQsk570oPISn4Q+j1c153cy+zll/ScxsoJm9kpT3EXA1M7fj+WY20cxGAZdQ\nGIY8HDjPzIaZWRNwPrC+pO45q79c0hfA6/jv5sTMtfvN7OVExqnAYcAJiRzfJHWlcuyFt+8QM5sC\nnF2hzkOA68zs6aTssWY2rEJ+wYxhsX2AU8xsspl9CFwI7J/J+6GZXW++oN2NQDdJS1Uo+76kNzIh\n2f8mc63Ub6w4bT/gIjP70MwmA6cC+6owhGdAbzObUsXvdK6iuCsdtD63AM/hb+Y35cg/Pnm4pEwG\nFi6Td4KZfZs5/xDvbSAfZjoPH8bokGx3Vqi3O/BBheufFsn0gwp5Z8V665Oi8kt+ZjP7QNI7wC6S\nHsJ7Qmcll28GlgNuS3pltwCnm9n0WZAHgKTHdhGwMT7/NB8woCjbqMzxjO8AWAG4VNKFaXH4Q2pZ\n8rXRsWZ2fZlrM+5Pht06AQOkGe8A81B4IVgGeK1IxnK93O74i0e1dMXb5qOiepbNnM/4js1silzY\nhYHPypS5m5k9U+ZaqfYrTlsmkSErz3x4zzdlFEFZoodRZ5K30hHATsA9NS5+MUkLZs6Xp9CD+Q9w\nH7CsmS0K/JvCQ6LUBO/HwCo1kqu4/G/wB1pKJWWTh9vwt8fdgLfNbDiAmX1vZuea2drAj4Fd8CGJ\n2eFfeM9wlaQdT2fmh222x7AChe/gY+BwM1s82RYzs4XTnsFskm3jz3Elu3amrkWTISWAsSVkLDfJ\nX+l3UMkw4HO8h7dCUT2jK9zTEi0ZBbSUNqaEPNNo/vLT1gYoDU0ojLbhEGCbZDiglgj4k6T5Jf0E\nn6u4I7m2MN4DmZaMS2fH/ccBTTR/MFwL/EHShgCSVqli6KQlBgFbSeqevPmfMpvl3YbPpxyJz2kA\nIKmnpHWSIYev8YdDU+kiZkLAAsnEeboJH/f+yswmS1ozqbOYkyQtmrTXcYl8AFcBp0laK5FvEUm/\nrP7jViYZ4rkGuCTpbSBpWUnbJ1nuAA6S9ENJnSj0yEpxHXCwpK2TSeNlJKVDop/icwGlZGhK6vmL\npIUlrQCcgPf62opbgRMkrShpYXxe47ZMD77RLcXanFAY9SNr5z4inRsovlZNOSUYC0zA36Ruxt9m\n30uuHQWcK2kicAZwe0aeKfif54VkbHhTM7srSfuvpK+Ae/GJ3mrlnfkDmD2Z1P8m8Co+B9IsS5Xl\nfQK8BGxO5nPhRgB3AROBt4FnSB5Ycue0KysVC0zC39SnJPut8XmDXyVt8m8KyiB73/34MNXA5LNd\nn8h5Hz6XcJukL/HPv2PRvZXkqebayfiE78tJXX1xc1zM7DF8buVpYBgVLILM7FV8svgSvB374T1X\ngEuBveQWX5eUkOU4vN2G48Owt5hZnyo/R5YHE4usdLu7hfzFXI9//8/hw62TExnz1j/Xo9YOoJRY\nZFyCK6frzOyCouuL4l/kKvgf8xAze6dVhQqCIAiqplV7GMlQwBW4idraQK+kG5/lNOB1M1sPtx4K\n780gCIIGpLWHpDYF3kvM2Kbh3ffdivKshXeNMbN3gRU1m85VQRAEQe2pqDDkXrPlzNjyUGwuOIrm\nZnUAbwB7JvVtio+PLjcbdQZBEAStQEU/DDObnrjLL2JmE1tJhvNx2/SBwGDcKWkmO3lJMSEVBEEw\nC5hZTSzA8gxJfQ0MlnSdpMvSLWf5oylYVID3HJrZYZvZJDM7xMw2NLMDcTf+4aUKM7OG33r37t3m\nMoScIWd7lTHkrP1WS/J4et/DrDuYvQqsmthgjwX2JbNaK7gtOjDZ3D/gMOBZm83lG4IgCILa06LC\nMLMb5SuRrp4kvWs+gd0i5kNax+A24KlZ7RBJh/tluxr4IXCjpCbcVv435UsMgiAI2ooWFYaknvjC\nYCNxT8jukg40X9yuRcydhNYoSvt35vjl4uvtmZ49e7a1CLkIOWtLe5CzPcgIIWcj06LjnqQBwH7m\nJq9IWh241cw2qoN8WTms1uNxQRAEczqSsDpOes+fKgsA86WN569F5UEQBEH7Ic+k92uSrsWXhgb4\nFc2XRg6CIAjmAvIMSXUEjga2TJKeB660OgcYiSGpIAiC6qnlkFRFhSEPhH6Tmf2qFpXNDqEwgiAI\nqqducxjmkclWSMxqgyAIgrmYPHMYw/E4CQ/gkdIAMLOLWk2qIAiCoOHIozA+SLZ58GhjQRAEwVxI\nRYWRzGF0NrM/zGoFOQIodcEtsJYH5gUuNLMbZrW+IAiCoHXIYyX1kpn9aJYK9wBKw4Cf4mFDXwX2\nNbOhmTynAl3M7FRJXYF3gaXN7PuismLSOwiCoEpqOemdZ0hqUDJ/cSfN5zDyLEg4I4ASgKQ0gNLQ\nTB6jMNTVGRhfrCyCIAiCtiePwlgAGA9sk0kz8q1gWyqA0qZFea4AHpA0BlgY2KdiifffD2uuCWvM\nMctPBUEQtAvyrFZ7cCvLsAMe03sbSasAT0hat9QS52effTbcey+stBI9jz9+rlz8KwiCoBL9+vWj\nX79+rVJ22TkMSXeY2d7J8QVmdnLmWl8z277FwqXNgbPNbMfk/BR8WfMLMnkeAs4zsxeS86eAk83s\ntaKyfA7jxBOhWzc46aSqP2wQBMHcRr0c91bLHG9XdG3JnOXPCKCUOP/tCzxQlOdDYFsASUvjcTdK\nRtzzmpeEceNyVh8EQRDUikpDUpVMknKZK+UMoPRn4AZJbya3/dHMvihb6JJLwrvvlr0cBEEQtA6V\nFEYnSRvgD/oFk2Ml24J5K8gRQGksPo+Rj6WWih5GEARBG1BJYYwF0uU/Pskcp+dtw9prw267tVn1\nQRAEcystOu41CuG4FwRBUD31jrgXBEEQBKEwgiAIgnyEwgiCIAhykWdpECQtC6yQzW9mz7WWUEEQ\nBEHj0aLCkHQBvr7TO8D0JNmAtlMYL74In34Ke+zRZiIEQRDMbeTpYewOrGFmU1tbmNyMGwdXXx0K\nIwiCoI7kmcMYDsw/qxVI2lHSUEnDJJ1c4vofJL0uaaCkwZK+l7RoxUKXXx7GjJlVkYIgCIJZIE8A\npbuB9YCngBm9DDM7rsXCcwRQKsr/c+B4M9u2xLWCH8ann8Jaa8H48S2JEARBMFdT7wBKDzDzgoF5\nyRNAKUsv4NYWS11qKZgyBSZNgs4RZjwIgqAe5ImHcWOy0uzqSdK7ZjYtZ/l5AigBIGlBYEfg6BZL\nlWDFFWHkSPi//8spShAEQTA75LGS6gncCIzEFx7sLunAVjCr3QX4n5l9WS7D2WefPeO457770rNr\n1xqLEARB0L5pkwBKMzJIA4D9zOzd5Hx14FYz26jFwnMEUMrkvQe4w8xuK1NWrCUVBEFQJfVeS2r+\nVFkAmNkw8ltN5QmghKRFgB7A/TnLDYIgCOpMnknv1yRdC9ySnP8KeK1C/hnkDKAE7uvxuJlNqU58\nYOxYmGceWHrpqm8NgiAI8pNnSKojPhG9ZZL0PHBlvR35Sg5JnXcenHaaH8dwVRAEwUzUckiqfcfD\neOYZ2GYb712MHevWU0EQBMEM6qIwJN1hZntLGkyJGN5mtm4tBMhL2Unva6+F/faDTp3qKU4QBEG7\noF4K4wdmNlbSCqWup8549SKspIIgCKqnLlZSZjY2OTzKzD7MbsBRtag8CIIgaD/kMavdrkTaTrUW\nZLY56ijo1q2tpQiCIJhjqTQkdSTek1gFeD9zqTPwopn9qvXFayZP5SGpIUNg/fVh6lT4/nvfFlig\nfgIGQRA0IPWaw1gEWAw4Dzglc2mSmX1Ri8qroUWFYeb+GJKvZPv22/DJJ+GfEQTBXE295jAmmtlI\n4FLgi8z8xfeSNqtF5TUlNant3h1WT9ZJvPjitpMnCIJgDiOP497rwIbp630S4+I1M9swVwXSjsAl\nFDy9S60j1RO4GF9yZJyZbV0iT3VWUn37wrbbwgEHwKuvwuDB0KFD/vuDIAjmAOrquCdpkJmtX5T2\nZh4/jDwBlJKhrxeB7c1stKSuZvZ5ibJmzaw27XkMGAAb5tJxQRAEcwz1XnxwuKTjJM2fbL/Dw7bm\nYUYApSSGRhpAKct+wN1mNhqglLKYLYYOhXPOgWWXrWmxQRAEcxt5FMYRwI+B0XgApM2A3+Ysv1QA\npeIn9+rA4pKekfSqpP1zlp2PNdaAM88sTH7/4x8wLW/8pyAIgiAlT8S9z/BlyVtThg2BbYCFgJck\nvWRm7xdnbBZAqWdPevbsWV1N6fDUiSfOoqhBEASNTZsEUJL0RzP7m6TLKb2W1HEtFp4jgJKkk4EF\nzOxPyfm1wKNmdndRWbO3NMj06TDffKnw8M03vv5ULFgYBMEcTL3mMIYk+9eAASW2POQJoHQ/sKWk\neSV1woe8hlBrXn4ZFlsMJk+GF16AhRd2v40gCIIgF2WHpMzswWR/46wWnieAkpkNlfQ48CYwHbja\nzN6Z1TrLsu668NxzsOCCsELReorTp8O889a8yiAIgjmJSkNSD1JiKCrFzHZtLaFK0Wqr1X7zjfc2\n+veHTTetfflBEARtSL2WBumRHO4JdKMQorUX8KmZnVALAfLSqsubp/MY06fHMFUQBHMU9Voa5Fkz\nexbYwsz2MbMHk20/4Ce1qLxhuOIK3z/xRNvKEQRB0MDk8fQeAuxsZsOT85WAR8zsh3WQLytH6wdQ\nmjrVV7idNq1gURUEQdCOqWUPI89T8QSgn6ThgIAVgMNrUXnDkTr0zT+/m94GQRAEM2ixhwEgqSOw\nZnI61MymtqpUpWWoT4jWiy+G3/9+ZoXx1Vfw9dewzDKtL0MQBEGNqGsPI/GN+D2wgpkdJmk1SWuY\n2UO1EKDhOPRQmDjRj818QvzLL92HI2XNNT1gUxAEwVxEHpOgPsB3wI+S89HAn1tNoramc2c4+2w4\n+mi3mBowwJdKB08DX9Dw5pvbTMQgCIK2II/CWMXM/gZMAzCzyfhcxpzNLrv4fuONfQmRCy+Eyy+H\nKVPg17+GLbeECy6A8ePbVs4gCII6kcdK6kU8nsULZrahpFWAW80sl5dbSwGUEn+P+yksmX6Pmc3U\ng6nbHEbzSuH55105FJOGhAX405/gySe9B7LPPvWVMQiCoAL1jofRG3gM6C7pP8BTwB/zFJ4EULoC\n2AFYG+glac0SWZ8zsw2TrXGGu8xKKwtwZXLIIX58112uWN54o36yBUEQ1JmKCkOSgKG4t/dBwK3A\nxmbWL2f5eQIoQXsd4rruOjj5ZPjvf/382GNL57vsMlcwY8dWV/7gwTB69OzJGARBUCMqKoxkDOgR\nMxtvZg+b2UNVRsTLE0AJ4EeSBkl6WNJaVZTf9px/PqyzjvdGfvCD0nn+/nff339//nLHjfMFE3ff\nvXn6lCmzJmcQBMFsksdxb6CkTczs1VaSYQCwvJlNlrQTcB8ehW8mZjuAUmszfrz3Ok44wZ3/Uj7+\nGC66CN56K185ffr4Qog/+hG8/Xbza506Qe/ebskVBEFQRJsEUJqRQRoKrAaMBL7Bh4/MzNZtsfAc\nAZRK3DMC2MjMvihKr/+kd7Xcf7/3CPr2he22a37twQddEdx7r/dI3nzTV8jdfPPm+b77Djp2hKee\ncgutRRaBb7/1tAkTYPHFPT75qFH1+1xBELRb6j3pvQOwMh5CdRfg58k+Dy0GUJK0dOZ4U1yJfUF7\n5Oc/dwe/7bf38/79vbeRXrvnHo/F8dZbcOml3oP4YdGSXO++6/stt4QuXWCLLVxRAHzyiZv7jh4N\nH31Un88UBEGQUGl58wWAI4BVgcG4Sez3VVfgZrWXUjCrPT8bQEnS0cCRuJ/HFOAEM+tfopzG72EA\nPPSQP9T79IGDD/alRLIT16k57lZbeUAngMcfhxdfhF69/J6XXqq8ltXnn0PXrs3TmppiafYgCGai\nXvEwbscf4s8DOwEfmtnvalHprNBuFEZKGmPjoYdg551nvp5+lqamwsq4N9wABx3kPZTHHy/kfekl\n72mcdhr8uYTV8ZAhsNZapYfCgiCYq6nXWlJrmdn/JRVeB7xSiwrnGqZO9Ylvlfme0nQzVypXXgk/\n+YnPV3To0Dzvj3/s+3JLrqc9lYcegsMPh/fei5CzQRDUnEo9jIFmtmG583rT7noYteSqq3y46be/\nLaQ99BA88ogrmlNOgYUW8h7IMsvAoEHlTXyDIJirqNek93qSvkq2ScC66bGkr2pReZCTI45orizA\n45D/61/uDDhuHCy3nPcqllvOLahGjPDeRiVGjCishbXHHu6xnpeJE+GDD6r7HEEQtGvKDkmZWYxp\nNDLrrOP7ZZbxeZCmJj9//3334TjsMLjmGvjmG7jllsJ9xx/vPhwLLggrr+xWW7fdBvfdB/vtV7qu\npiYfKuvUqZC2+ea+au/c2usLgrmQMKtpr3TtCtdfD7fe6vMh6ZzFGWf4fo89fP+f/xTu6dvXzXmv\nucZD0QI8/XRh2fZddy3k7d8fllwSXnjBlc9CC8Evf1mYe7n0Ut9Xu9xJOb74ohBbPQiCxsTM2sXm\nogZVMWyY2ZNP+vE++5jdfruZ9wnM9tjD7IAD/Nobb5jdf3/ze6+5xvMddljhnhEjfP/ww55n773N\nrr9+5nrHjate1iuu8LJ32636e4MgKEvy7KzJczh6GHMyq60GP/2pD0Hdfrt7ob/xhvuA3HMP3Hij\n51t3Xe9dTJ/uPYjzz4cDD/RrXbr4/tFHCxPpqZnwTjt5NMIse+7pPZPHH3fnw0cfbX59rbVg1VVn\nljWtLx1ie/BBP69VDyYIgtkmV0zvRmCutpKaXbImvNXknTDBh6KyZr6PPeYP/FVXhWnT3NQ3XYl3\n0iRYYw3P98tfFibR77gDunVzs2HJfUluuMGH0zbeuFD222/D8svDZ595+cOGweqr+xzL3nvDmWe6\nn0mPHrPdJEEwt1AXK6nUGipjGTVpVqykJO0oaaikYZJOrpBvE0nTJO1Z7YcIWmCZZcovvV7M88/7\n3syXOSn2Cdlxx0IPIfUzMfM61lvPV9OdMMGXfH/sMc/3wAPw7LPuoQ7ws5/55Hy6QNqTT/q1tdf2\nELnpnMq0ab7fd1+//pe/QM+eseR7ELQRZRWGmXU2sy7J1jlz3tnMuuQpPG8ApSTf+cDjxdeCGjBy\nJFxySb68myaBFKdOzV/+oYf6fvJkn0xfdFFXJjvs4IpjrbXg3/92RQGwwQauBE46yc+PO6750NOa\na/raW6usAkss4WlLLuke7+AT8aUYMgTOOiu/3EEQVEWuOQxJW0o6ODnuKmmlnOXnDaB0LHAX8FnO\ncoNqmH/+/OtMdejgcwipFVUerr/e96W82nv18hV2R42Cf/6zsLjiXnv5/oor/EG/yiqFe444wudc\nOnb0YakRIzx98829nF/+srQcEyfCuecWTIxTJN+ammaOJzJ5cvPzb7/19bzyDn9+Fj/ZYO6hxaeI\npN7AycCpSVIH4JbydzSjxQBKkpYBdjezf9FeI+/NaZRbzqQcX31VObDTwQf7vls3n5MAnxw3KwyV\nZX081lyzcL744rDiioVryy5bXvmlS8VfeGEhLR3WAjcA6NTJTYnBl5JfaCHvgaWMGePzK+lczBtv\n+AT+NdfA/vsX8pnBa6/B0ksXnB+zDB0Kf/1raTlrQVOTf08TJ7ZeHUFQRJ4ASnsAGwADAcxsjKTO\nNZThElwhpZR9WjV8AKW5lc4t/Bw6dCj/xn7ffdUrqCxmhcUXATbayK29Uuaf3+dUbrnF50j69XPF\nYeYT7OD7Z5/1yfSVV/Yhsv7Jgsnrr1+YpwG46SY/X331gif9GWe41z3A99+7IUC6bH1Tk5fx85+7\nRdkee7g/TDbA1qzw3nu+HzYMNtlk9soK5ihaM4BSHv+HV5L9wGS/EPBmHptdYHPgscz5KcDJRXmG\nJ9sIYBLwCbBribJqYJEczBE88ohZr15mV11l9uqrBT+R8ePN7rzT7LzzPN9VV5ndckvzey+6yOzl\nl80+/tjv2Xhjs3feMVtoIbMddnBfk7fe8munn+77vn3NJk3y49tuMzv8cLN55jFravLy9t7by77u\nOs/zzTe+X3hhsz//2ezAA/36oYd6+sSJnmd26NPHbOedXYYgqAA19MPI89D/A/Dv5KF+GPAScGyu\nwmFe4H1gBXwoaxDwwwr5+wB7lrnWCk0ZtEvef7+gJMaONevaNfkpF7H33mbXXlu+nKuv9gduU1Oh\nvAkT/NqYMWb9+pmtuGIh/9JLu7NjmtfMbOBAs/XXNzv++EL65Mlml11m9u23Znfd5Wkvv+z377ST\nO1SmeR9/3JXR+PFm06aZnXmmK7FivvuuuUPkRhsVFGNb8913Zq+80tZSBGWoq8Lw+tgO+HuybVdV\nBbAj8C7wHnBKknY48NsSea8PhRG0yPTp/tM99VQ/v/vu5g/2lHXXNXvttXxlHnOM2dNP5897660F\nWaZPd2958GvZt/7hwz29Tx/fDx5sNmWKWY8efn7SSWaLLOLHhx5q1ru3H3/3nSuc1LO+e3c/bmpy\nJbnBBv6QbmoqKDkz7y2VYvz4fJ9t3DhXftWQ9pyChqQtFEY3YFc8NGu3WlVelaDxgwyyvPyyP1TL\ncffdZh07mn39df1kKkffvmYjR5o9+KArl5SxY/0znHGG/xWff97TwWyzzVzZgdnNN/u+Qwe/B8ym\nTvW8xx7r5x99ZPbFF35cPAz3yiuenqcXcPLJnnfSpObp995rdtRRpe9Jh+LS7+O++7xnFTQE9R6S\nOhT4CLgBuBEYCRxSKwFyCxoKI6iGpZbyeYn2wPTpBQVgZrbFFma77GIzhq2++caHqe691+y005q/\nzX/6aSFfdmgtpV+/Qi/nrLOa1/v1154+bVohbeJETzvxxELaTTfNXK6Z2W9/60NsZt7De/NNPwaz\ns89u+XNPmuSytQZZxdyWfPaZ9yjbkFoqjDzG+ScBG5jZQWZ2ILARza2agqDxeOIJeKWdBImcZ57m\nHvX/+597x6+yin+OTp3g1Vd9LbAOHeD00wt5l1rKrbW22sqtt4YN8/SxY906rGdPX8L+sMPgnHOa\nW5ClDpDXXAObbeb3p3Jkrbj++U/f9+8PvXu7Q+ZLL8HVV3u94PFZbr/d6wT3v5Hguutm/rx33un3\nbrONW6WBW5Ntsgl88omfDx/uMWCGDoUjj3RrulJ8/rnXPWZMIW3sWF+9+c47S98DbhL95JNu9daa\nLLVU8++rvdOSRgFeBDpkzjsAL9ZKY+XdiB5GEJitvrrZOedUzvP44z5EBWarreZpEyb4ysIXXWS2\n5ZY+vLTYYp7n0UcLPYi0t/H116VXIk7zpRZkL7zg6V995ff+7neFnsi8887cK0kt0K66qjBU9vDD\n3jsBsx139OHGtJ5023bb0p81vb7//qXTy73dp9ePO65yW5bi4otb/g6K62lDqEcPQ9LvJf0et3Lq\nL+nsxInvZWBYK+uxIAhK8cUXhbfycmy/va/tBYU3/kUXdQfK99/3HsyVV7rz5CWX+Ppgzz3nPYg0\nbvw558DJpZs8AAAMPklEQVQhhxRWK0554w3v9Zx7rr+hb7aZp3fu7PeOGAHHHONpHyc+u48+6r4o\nN91UeKPfcsuC/8i++8KHH/rxY48VeoZ33+2f9dBD4bLLSvcYN94YfvGLwmoDKaNG+f6bbzxo2Hff\nNb/++uu+r8aXq6nJe2jdu8NTT1XO+9VX3iuEwvI2eZkwwXtr5Tj11NZ1Cq1EOU0C9K601Upj5d2I\nHkYQVMdjj5WOTXLmmWZDhlS+97vvCm/H1fp6ZPNfcIH7zICbFZ9xhlt4pXMMY8e6jJ99ZjZ6tMdv\n+f57lz3L1Klexj//2bye1ErNzK3KimVNjQT69p1ZznPPNfvyS+/hVDKgSDnrLJvh7wNmb7/t6c89\n51ZwWa6+2ntFX37p5+V6Gtn5o5QTTvC8o0eXliMt6/bbW5bZatvDqOtDf7YEDYURBPUl9VGZXf7x\nD/dVGTRo9srp0cP9T8wKvjhZ+bK+LSkDB7oD5fTpZhde6Nf33LNwPTUU6NixdJ0ffeQWX6kp93zz\nFepabTWzP/2ptGKdZx6zrbeeWTYz99MxKwwFfvRRId+773par16llYlZQQE/9phZ//5lm6tQdR0V\nBrAk7n/xCPB0utVKgNyChsIIgrmbCRP8kZW+gWefCd9/X0gbNaqQvtdehXyLL+7HRxzRvNxbbvH0\nrbd2E+Httis8/Lt182tpRMgRIzz9kUcKvjRg9sEHnv7OOwVP/6wZctr7OeUUv3b33Waff+7Hf/1r\nc3nOPNNNpFti1Ci/f+LEitnqrTD6Ar8BhgA9cOe6C2olQG5BQ2EEQZBO+qeKI8uwYe74mOU//zH7\nzW/8+L77vLdTih//2Ms88kjfH3KI2fnn+3GfPv4Av/TSmXtcn3xSGBJLwxqnWylnyS+/LFyfMsXN\nl1Pz7zvvbO5o+tRTzZXiI4/4kjZZ1l23RY//eiuMAcn+zUzaq7krcE/vofhE+cklru8KvAG8DrwC\nbFGmnIqN0ig888wzbS1CLkLO2tIe5GwPMprllPOyy5r7rtSKKVPMjj7anSQHDjRbYgm3ACvBTHIO\nGuSP1O7dK9fx7LNmL73kx/36+T2pt/xttxXypXMl06cXhs6WWKJ5Wffc01yplKCWCiOPH0a6PvRY\nSTtL2gBYPM+Ees4ASk+a2XpmtkHSk7k2T9mNSqutElljQs7a0h7kbA8yQk45jz125miQtWCBBeDy\ny91KbN113c+jzGrMM8m53npuhZZafJVjq60KS/H36OGhAa5NHnvbblvIt/jibu320UdumbbQQs39\nTcBXP/YX6rqQZ3nzP0taBDgRuBzoAhyfs/wZAZQAJKUBlIamGcwsG8FmYaAo+k0QBEEdkeDkWfRN\nzgYCy8sCC7hiGjt2ZhPcMWNgpUy8utZQklXQosIws4eSw4nA1gCS8iqMUgGUNi3OJGl34Dx8gn3n\nnGUHQRDMGSyxRGl/jQED4MUXPZZL1671l6sI2Sx0ZyR9ZGbL58j3C2AHM/ttcv5rYFMzO65M/i1x\nH4/tSlyrX78rCIJgDsLMahLNNM+QVCnyVj4ayCqW5ZK0kpjZ/yStLGlxM/ui6FqEbw2CIGhD8kx6\nlyLv2/6rwKqSVpDUAdgXeCCbQdIqmeMN8XWrviAIgiBoKMr2MCRNorRiELBgnsLNbLqkY3BfjnmA\n68xsiKTD/bJdDfxC0gHAd8AUYO8qP0MQBEFQB2ZpDiMIgiCY+5jVIam6ImlHSUMlDZPUprE4JI2U\n9Iak1yW9kqQtJqmvpHclPZ6YIaf5T5X0nqQhkrZvRbmuk/SppDczaVXLJWlDSW8mbX1JneTsLWmU\npIHJtmMDyLmcpKclvS1psKTjkvSGadMSMh6bpDdUe0rqKKl/8p8ZnKx63VBt2YKcDdWemTrmSeR5\nIDlv/faslQdga224UnsfWAGYHxgErNmG8gwHFitKuwD4Y3J8MnB+crwW7sE+H7Bi8jnUSnJtCaxP\nc4/8quUC+gObJMeP4FZurS1nb+D3JfL+sA3l7AasnxwvjMelX7OR2rSCjI3Ynp2S/bx4iIRNG6kt\nW5Cz4dozKfcE4BbggeS81duzPfQwZjj/mdk0IHX+ayvEzD2z3fDwtST73ZPjXYHbzOx7MxsJvEcJ\nP5RaYGb/AybMjlySugGdzSxZyJ+bMve0ppxQ2vJutzaU8xMzG5Qcf42vpbYcDdSmZWRcNrncaO2Z\nOuh2xB9cRgO1ZQtyQoO1p6TlgJ/RfGWMVm/P9qAwSjn/LVsmbz0w4AlJr0o6NElb2sw+Bf8TA0sl\n6cWyj6a+si9VpVzL4u2bUs+2PkbSIEnXZrrSDSGnpBXxXtHLVP9d10XWjIz9k6SGas9k+OR14BPg\nieQh1XBtWUZOaLD2BC7Gw2dnJ6FbvT3bg8JoNLYwsw1x7X60pJ8wszVZo1oSNKpcVwIrm9n6+B/1\nwjaWZwaSFgbuAn6XvMU33HddQsaGa08zazJfL245/O12bRqwLUvIuRYN1p6SdgY+TXqXlfzTat6e\n7UFhVOX819qY2dhkPw64Dx9i+lTS0gBJN++zJPtooHvm9nrLXq1cbSKvmY2zZBAVuIbCsF2byilp\nPvxBfLOZ3Z8kN1SblpKxUdszke0roB++inVDtWU5ORuwPbcAdpU0HLgV2EbSzcAnrd2e7UFhtOj8\nVy8kdUre5pC0ELA9MDiR56Ak24FA+nB5ANhXUgdJKwGr4ku4t5qINH/jqEqupBs7UdKmkgQckLmn\n1eRMftwpewJvNYic1wPvmNmlmbRGa9OZZGy09pTUNR3GkbQgsB0+39JQbVlGzqGN1p5mdpqZLW9m\nK+PPw6fNbH/gQVq7PWs9c98aG/428i4+WXNKG8qxEm6l9TquKE5J0hcHnkxk7AssmrnnVNwqYQiw\nfSvK9l9gDDAV+Ag4GFisWrmAjZLP9h5waZ3kvAl4M2nb+/Cx2LaWcwtgeub7Hpj8Dqv+rltL1goy\nNlR7Av+XyDYokev0Wf3ftJGcDdWeRTL3oGAl1ertGY57QRAEQS7aw5BUEARB0ACEwgiCIAhyEQoj\nCIIgyEUojCAIgiAXoTCCIAiCXITCCIIgCHIRCiNoOCQ1Sbopcz6vpHGZZZx3kfTHFsr4gaQ7kuMD\nJV1epQyn5sjTR9Ke1ZRbSyQ9I49SGQR1IRRG0Ih8A6wjqWNyvh2ZxdPM7EEz+1ulAsxsrJllozdW\n63B0WpX52xWS5m1rGYL2RyiMoFF5BNg5Oe6Fr5kDNO8xJG/5l0p6QdL76Rt/spTM4Ex5yydv5O9K\nOitT1r3JysOD09WHJZ0HLCgPTnNzknaACoGzbsyU26O47iyJHO9IulrSW5IeSxVhtocgaQlJIzKf\n7155MJzhko6WdEIiz4uSFs1UcUAi05uSNknu7yQPVPWypAGSdsmUe7+kp3CP4CCoilAYQSNieNyT\nXsnDdV0Ky3Zn86R0M7MtgF3wIDKl8mwC7AGsB+yVGco52Mw2Sa7/TtJiZnYqMNnMNjSz/eUrlp4G\n9DRfyfR3OerOsipwuZmtA0wEflHhc6esjccm2BT4C/C1+SrJL+Nr/qQsmMh0NL6uFMDpwFNmtjmw\nDfCPZG0kgA2APc1s6zIyBEFZQmEEDYmZvYVHB+sFPEzlZZzvS+4ZQiEGQDFPmNmXZvYtcA8e+Q/g\neEmD8AfxcsBqSXq2vm2AO81sQlLPl1XWPcLM0t7OgORztcQzZjbZzD4HvgQeStIHF91/a1L/80Bn\nSV3wRTFPkcd16Ad0oLDi8xNmNjFH/UEwE/O1tQBBUIEHgL8DPYGuFfJNzRyXUywzxV6Q1ANXBpuZ\n2VRJzwALVCljnrqzeaZn6viewktbcb3Zeyxz3kTz/22pmBICfmFm72UvSNocnx8KglkiehhBI5I+\neK8H/mRmb8/CvcVsJ2nRZGhmd+AFYBFgQqIs1gQ2z+T/LjMx/DQ+jLU4gKTFqqy7XPpIYOPkeK8y\neVpin0SmLYGJZjYJeBw4bkbl0vqzWHYQNCMURtCIGICZjTazK/LkrXCe8go+FDUIH14aCDwGzC/p\nbeCvwEuZ/FcDgyXdbGbvJNefTYZ50ohreesul/4P4EhJA/ClqctRqdxvJQ3Eo8IdkqSfi3+uNyW9\nBZxToewgyE0sbx4EQRDkInoYQRAEQS5CYQRBEAS5CIURBEEQ5CIURhAEQZCLUBhBEARBLkJhBEEQ\nBLkIhREEQRDk4v8BMiqYKRHjm+IAAAAASUVORK5CYII=\n",
|
|
"text/plain": [
|
|
"<matplotlib.figure.Figure at 0x2ec09a89860>"
|
|
]
|
|
},
|
|
"metadata": {},
|
|
"output_type": "display_data"
|
|
}
|
|
],
|
|
"source": [
|
|
"pred_basic_model_bn = train_and_evaluate(reader_train, \n",
|
|
" reader_test, \n",
|
|
" max_epochs=5, \n",
|
|
" model_func=create_basic_model_with_batch_normalization)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"Let's implement an inspired VGG style network, using layer API, here the architecture:\n",
|
|
"\n",
|
|
"| VGG9 |\n",
|
|
"| ------------- |\n",
|
|
"| conv3-64 |\n",
|
|
"| conv3-64 |\n",
|
|
"| max3 |\n",
|
|
"| |\n",
|
|
"| conv3-96 |\n",
|
|
"| conv3-96 |\n",
|
|
"| max3 |\n",
|
|
"| |\n",
|
|
"| conv3-128 |\n",
|
|
"| conv3-128 |\n",
|
|
"| max3 |\n",
|
|
"| |\n",
|
|
"| FC-1024 |\n",
|
|
"| FC-1024 |\n",
|
|
"| |\n",
|
|
"| FC-10 |\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 24,
|
|
"metadata": {
|
|
"collapsed": true
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"def create_vgg9_model(input, out_dims):\n",
|
|
" with C.layers.default_options(activation=C.relu, init=C.glorot_uniform()):\n",
|
|
" model = C.layers.Sequential([\n",
|
|
" C.layers.For(range(3), lambda i: [\n",
|
|
" C.layers.Convolution((3,3), [64,96,128][i], pad=True),\n",
|
|
" C.layers.Convolution((3,3), [64,96,128][i], pad=True),\n",
|
|
" C.layers.MaxPooling((3,3), strides=(2,2))\n",
|
|
" ]),\n",
|
|
" C.layers.For(range(2), lambda : [\n",
|
|
" C.layers.Dense(1024)\n",
|
|
" ]),\n",
|
|
" C.layers.Dense(out_dims, activation=None)\n",
|
|
" ])\n",
|
|
" \n",
|
|
" return model(input)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 25,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Training 2675978 parameters in 18 parameter tensors.\n",
|
|
"\n",
|
|
"Learning rate per minibatch: 0.01\n",
|
|
"Momentum per sample: 0.9983550962823424\n",
|
|
"Finished Epoch[1 of 5]: [Training] loss = 2.267064 * 50000, metric = 84.67% * 50000 18.672s (2677.8 samples/s);\n",
|
|
"Finished Epoch[2 of 5]: [Training] loss = 1.877782 * 50000, metric = 69.81% * 50000 12.578s (3975.2 samples/s);\n",
|
|
"Finished Epoch[3 of 5]: [Training] loss = 1.689757 * 50000, metric = 63.07% * 50000 12.729s (3928.0 samples/s);\n",
|
|
"Finished Epoch[4 of 5]: [Training] loss = 1.564912 * 50000, metric = 57.57% * 50000 12.536s (3988.5 samples/s);\n",
|
|
"Finished Epoch[5 of 5]: [Training] loss = 1.475126 * 50000, metric = 53.79% * 50000 13.171s (3796.2 samples/s);\n",
|
|
"\n",
|
|
"Final Results: Minibatch[1-626]: errs = 50.1% * 10000\n",
|
|
"\n"
|
|
]
|
|
},
|
|
{
|
|
"data": {
|
|
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYwAAACfCAYAAADqDO7LAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztnXmYVNW1t99fgwyGQXBGGRyDM2JEDEYBL05xuhqNc5wS\noxiNJnGIX0Ickhg196pcNcEBFa8ah6jo1RhnBRWNgIziwKgCggqGUaDX98c6ZVUP1V3dVHVVda/3\neeo55+yzzz7r7O46q9Zee68lMyMIgiAI6qOi2AIEQRAE5UEojCAIgiAnQmEEQRAEOREKIwiCIMiJ\nUBhBEARBToTCCIIgCHIiFEYLRdJtkq5oaF1JB0iaV1jpvrnvLEmDm+JezYHkbzM533UbIcdrkk4r\nRNtBcWldbAGC/CJpNrAF0M3MvsgonwDsAfQys7lmdm6ubdZSt1GLdyT1BGYBrc2ssjFtNBck7Qc8\ng/dlBbAhsAxQUrazmX3ckDbN7BVgt3zXDYIUYWE0Pwx/KZ+YKpC0K9CeRr7o80jqZaiC30hqVeh7\nrA9mNsbMOppZJ2AXvF86p8qqKwslFEXYIEgIhdE8GQX8KOP4R8A9mRUkjZR0VbJ/gKR5ki6WtFDS\nJ5JOr61uukiXS1okaaakkzJOHCZpvKSlkuZIGpZx3SvJdomkryTtk1zzY0nTkrIpkvpkXLOnpHcl\nfSnpAUltantgST+SNEbSf0laBAyTNEzSqIw6PSVVSqpIjl+SdFVy3VeS/iGpa5b2p0k6LOO4laTP\nJPWR1FbSKEmLEznHSdq0tnbqoYpCSIZ2rpL0Om59dJd0VkZffSDprIz6B0qalXE8T9JFkiYlcv2v\npA0aWjc5f7mk+Um9s5N+7FHvAzm/lTRb0gJJd0nqmJxrn9wn1W9vpvo/ec5ZyXN+KOn4RvRnkGdC\nYTRP3gQ6Svp28nL8IXAfdf+y3wLoCHQDzgZukdS5jrpdk7qnAyMk7ZCcWwacamadge8DP5V0ZHJu\n/2TbKfkVPU7SccBvgVOSX9tHAp9n3Os44CBgG3xI7fQ6nmEf4ENgc+D3SVl1q6r68Ym4Qt0UaAv8\nMkvb9wMnZRwfAiwys4nJ9Z2ArfB++Smwsg45G8Ip+DN3Aj4BFgCHJn31Y2B4YkGmqP58xwEHAtsC\n3wFObWhdSYcDQ4EDgB2BwbVcm40f4/22P7Ad3j83JufOwC3fbkn5ecCqRKH8GTgwec4BwKQc7xcU\nkFAYzZeUlTEEmA58Wk/9r4GrzWydmT2Dv/i/naWuAb8xszVm9irwf8DxAGb2qplNTfanAA/iL5pM\nMhXXWcB1ZjY+uWammWU61W8ys4VmtgR4Esi0PqrziZndamaVZra6nudNMdLMPkrqP1RH+w8AR0pq\nlxyfmJQBrAE2BnY0Z4KZLcvx/vVxl5m9n/xd1pnZ/5nZHAAzexl4AfheHdf/t5ktMrMvgaeou/+y\n1T0OuDORYyVwZQPkPwm4IfGbLQd+TVrxrgE2Id1v481sRXKuEthNUtvk7/9eA+4ZFIhQGM2X+/Av\n5unAvTnU/7yaI3oF0CFL3S/NbFXG8Rz8VyKS9pH0YjJcswQ4B38pZKM78FEd5xfmKBNAY2ZvLcil\nfTP7CJgGHCGpPW4J3Z+cHgU8Czwo6WNJ1yp/PpQqzyTp8GTo5nNJX+I/COrq34b0X7a63arJMY/c\n/VDd8P+PFHOAtsmQ3d3A88BDyVDXHyRVmNm/cYV8PrBA0ugMCzYoIqEwmilmNhd3fh8K/D3PzXdJ\nXpopepC2YP4XeBzYysw2Av5K+uVS2zDGPHyoIh9Ub385PvsoxZbr2f6DuBI+CphqZjMBzGytmV1t\nZrsA3wWOAPI1rfSbZ0qsm4fx4bZNzawL8ByFn0QwH9g647gHuQ9JfQr0zDjuCaxOLJk1ZnaVme0M\n7AccA5wMYGbPmtkQfPjzI/z/KCgyoTCaN2cCg5NhhHwi4EpJG0j6Hu6reCg51wG3QNZI6kfVcf9F\n+FBDpoK4A/ilpL4AkraT1D1Pck4E9pfUPfHHXLae7T2I+1POJW1dIGmgpF0Tf9EyfKilodOGc3np\ntwU2ABYDlvgWDmzgfRrDQ8BZknaUtCHw/xpw7QPAxcmEg47ANSR9J2mQpF0kiYx+k7RFYkm1B9bi\nin9dPh8oaByhMJof3/zyM7NZKd9A9XMNaacW5gNf4r8eRwHnmNkHybnzgKslLcVfLH/LkGcl/ut4\nrKQvJPUzs0eSsvslfQU8hjtAGypvzQcwez65/yTgbdwHUqVKA9tbALwB9CfjufBfwY8AS4GpwEt4\nv6QWPd6aS/P1lZnZUuAi3IL7HP9FXv2Z6muzwXXN7CngNuBVYAYwJjmVzU+U2dbteF+9hk9IWAr8\nPDnXDbd+lwKTgX/iyqQV8Cv8/2sRsC/udA+KjAqZQEnS1vj4+eb4L67bzezmLHX3Bl4Hfmhm+R5C\nCYIgTySzst4xs7bFliVoWgptYawFLk7GdvcFhkrqXb1SYspfizsOgyAoMSQdnQxBdsW/q48XW6ag\n6SmowjCzBck8dZJphtPxuerV+Rlu0n9WSHmCIGg0Q3Hfyfv4DKrziytOUAyaLJaUpF74vO5x1cq7\nAUeb2aDESRoEQYmRzFgKWjhNojAkdcAtiAtrWdB0I3BpZvUsbRQ7DlIQBEFZYmZ5mXpd8FlSklrj\nymKUmT1RS5Xv4AueZgE/wENSHFlLPcys5D/Dhg0rugwhZ8hZrjKGnPn/5JOmsDDuAqaZ2U21nTSz\nbVP7kkYCT5rZ6CaQKwiCIGgABVUYkgbgKzcny/MxGB5LpidgZjai2iUx7BQEQVCiFFRhmNlYfBFO\nrvXPLKA4TcLAgQOLLUJOhJz5pRzkLAcZIeQsZQq6cC+fSLJykTUIgqBUkISVi9M7CIIgaB6EwgiC\nIAhyoskW7uWLqVPh1FNBgooKaNsW2rWDXXeFG2+sWf/jj+G226B9e2jTBjbYwD9bbgnHHluz/sqV\n8OmnsNFG0KWLl1WEWg2CICg/hdGrF9x+O5hBZSWsXg2rVsGGG9Zev6LClcXKlbB0KaxZ459//7v2\n+jNmwDHHwJIl/jFzpXTAAfBsLZGuZs+Gl16C3XZzBTN3Lmy/PXTPV4DuIAiCEiGc3nWQut3q1a5k\nOnasWWfKFLjmGpg2zRVMjx6udM46C669tknFDYIgqEE+nd6hMArAunVuwWy0Uc1zo0bBHnvA7rs3\nvVxBELQ8YpZUidOqVe3KAmDmTDjwQDj3XLdOgiAIyoWCKgxJW0t6UdJUSZMlXVBLnZMkvZt8xkja\nrZAyFZthw+Ctt2DjjWHwYDj5ZHj33WJLFQRBUD+lkEBpJrC/me2B5/u9vcAyFZ1ttnG/x9tvQ+fO\noTCCICgPmtSHIelxYLiZvZDl/EbAZDOrMceonHwYQRAEpUJZ+jCyJVCqxtnAM00hTxAEQdAwSiGB\nUqrOIOAMYL9s7fzud7/7Zn/gwIHNMviXGRx3HAwf7osLgyAIGsLLL7/Myy+/XJC2Cz4klSRQegp4\nJltODEm7A48Ch5jZR1nqtJghqfPO83UdTz+dfUFiEARBLpTbkFSdCZQk9cCVxanZlEVLY/hwX8Xe\nqZMvAgyCICgFCmphJAmUXgUm48mRaiRQknQ7cAwwB8/nvcbM+tXSVouxMMAVxoAB8OabsHAhbLZZ\nsSUKgqAciZXeLQQzGDkSjjrK120EQRA0lFAYQRAEQU6Umw8jCIIgaAaEwigTKivhX/8qthRBELRk\nQmGUCWvXwmGHwfvvF1uSIAhaKqEwyoQ2beCnP4Xrriu2JEEQtFTC6V1GLFzomf2eeQb22qvY0gRB\nUA6E07uFsvnmcPHFvj4jCIKgqQkLo8yYOxd69vSUsa3LLiN7EARNTdlYGLkkUErq3SzpA0kTJfUp\npEzlTo8evqAvlEUQBE1N0RMoSToU2M7MdgDOAf5SYJmaDWvWwJlnwiuvFFuSIAhaAkVPoCTpL8BL\nZva35Hg6MNDMFla7NoakqqEMI3PVKmjbFj77DLp2DQskCAInn0NSTfZaqSOB0lbAvIzjT5KyhQR1\n8uyzMHUqtG/v1ka7dl5+wAFQoHD4QRC0YEomgVIutIQESg3hoIP8k0nPnvDEEx7ldt993d8RBEHL\noVknUKplSOo94IAYklo/nn8ehgyB2bNdiQRB0DIpm1lSCXUmUAJGA6cBSOoPLKmuLIKG8x//4dvD\nDy+uHEEQNB8KPa12AHAyMFjSBEnjJR0i6RxJPwEws6eBWZI+BP4KnFdImVoSP/kJTJnigQu7doU7\n7yy2REEQlDOxcK8ZU1npwQq32SbtEM/WhfPnw6abxuyqIGhulNuQVFAkKiqgd2+fbrt2rZcNGQLz\n5sGrr6brrVoF3brB2WcXR84gCMqDsDBaEJ07u2J47z0/njcPunSBDh38ePp0VzBBEDQfynIdRlB8\nvvjCh6Q++QR69YLu3f34hhs8Em7v3vDpp3DvvTBwIPTvX2yJgyAoJcLCaKHMmwcHH+wL/zJXjL/4\nIhx4oO9//rk7y4MgKF+a3IchaTtJbZP9gZIukLRRPgQIikP37jBtWlVlATBokIcXARg+vOnlCoKg\ndMnJwpA0EfgO0At4GngC2MXMDiuodFVlCAujCVm50rfdu7ulUVlZU7msD7/9rftMHn44f20GQVCT\nYsySqjSztcB/4sEDfwVsmQ8BgtKkfXv/3HqrH198sSuNNm1g8eL1a3vtWrj6anjkEc/vEQRBeZCr\nwlgj6UTgR3iYD4ANCiNSUEocf7xvb7wRPvzQgxzed5+Xff21Wx1t2jSszSuuSO+//35+5AyCoPDk\nqjDOwPNZ/N7MZknaBhhV30WS7pS0UNKkLOc7SRqdJE6aLOn0nCUPmow1a2DFCk8R+4c/wEUXwbvv\nphXFmjWwfDmcdporkFmz6m7v2mthyRKfoZUKYRIEQemTk8Iws2lmdoGZPSCpC9DRzP6Uw6UjgYPr\nOD8UmGpmfYBBwJ+TYIVBCdG6tQ9Pde4Ml10G3/oWXJDkTqyshN1396m6HTt6WSoESWVlesEgwKmn\nwi23uFLp3NnLHnvMh6ZSvPZa2n8SBEFpkessqZcTa6ArMB64XdJ/1XedmY0BvqyrCpC8ZugIfJ74\nSoISRYJly9JZ/iS3NnbcEW6+Gb73PfhT8lPikENggw1g3TrPz3HffXD++VXbW7wYnn7a95csgf33\nhw03rFpnzBi/z4cfFvTRgiCoh1yHpDqb2VfAMcC9ZrYPkI/BhP8Bdpb0KfAucGEe2gyKRKtWrhhW\nrPBFgs89B3vu6SFKBg3yECXVrYedd/bpvQDXXZcuT02IW7fOlQjAVlvB2LE+cysIgqYn1+Gf1pK2\nBI4HrqivcgM4GJhgZoMlbQc8J2n3bEmWIoFS6VNR4Z9rr/XjlHUwc6bn5aio9hNlp51g3DiYMcMd\n7Cec4ENcme2Zwe23w+rVsN9+Xn755X6PFSt8uCwIAqfoCZQkHQf8BhhrZudK2ha43syOzeHansCT\nZrZ7LeeeAv5oZmOT4xeAS83sX7XUjXUYZcTq1fDMM3D00fXXlTyabqb1MW6cK4r+/T2S7mabuT+k\nXTv3n6xZA7fd5v6PY+v9L/S1JJ06+RBZELQk8rkOoyky7vXCFcZutZy7BfjMzK6UtDnwL2APM/ui\nlrqhMJopZj70lBlafdAgd4CvzeLRWrfO41399a8+rFUfqUWH8S8UtDSaXGFI2hoYDgxIil7D83N/\nXM919wMDgY2BhcAwoA1gZjYiGea6m/QiwD+a2QNZ2gqF0YL44AN3pI8aBaec0rBr//lPH8rKnLLb\nuTN89VX+V6wXgwULYMst3cqK/CVBfRRDYTwH3E967cUpwMlmNiQfQuRCKIyWhZm/9L/7XXd018X5\n5/v6kE6d/FiCM89MT+/99FN3mC9b5nGyZs2CwYPrv//ixZ5UqtRo1coV3/z5sMUWxZYmKHWKERpk\nUzMbaWZrk8/dQAl+lYLmguQWwYsv1l/3llvgrLN8/+9/9+3NN6fPz5/v2299y4evDjzQU9dC1TAn\nH37oU3tXrvS8IZtt5sqm1KishOuvD2URND25KozPJZ0iqVXyOQX4vJCCBUHHjj4Vtz6GDk3HpTr2\nWNh2W1cOKfbay1+ykE4etdtusHSpWxC/+hVMmAA77OAJpR5/3Id9NtnEV7fnyrp1udddH666Cs47\nr2nuFQSZ5KowzsSn1C4A5gM/AE4vkExB0CB++UvfmrkTPLXA78knYdEi30/5LXr29FS08+enV5vf\ncAM8+GC6vT339GnAixb58E8urFzp/oRsTvp88pvf1FzcGARNQaNnSUn6uZndmGd56rpf+DCCrOy0\nk+fvSDm6v/46bZ28+qqvQK+Nl1+GkSPhjjvqn3L71luuQPbaq+a5P//Z/SifV7O7zeD55/3+7do1\n6JHqJbW2JVelFrRMiuHDqI2L8yFAEOSDZ59NrwgHD4zYt6/vb7JJ9usGDoR77smuLB56CI46yi2U\np5+G73zHnefgCuTEE2H8eLdyUrnRUzz/vDvuDzrIV7OnhsXyxXbbpSMHB0FTsD4Ko8wnJwbNiR49\naoZZf+MNH1baaafGtzttGowe7fvduvn2mmtg111hn318KCvlOB8/vuq1qRAmxx0H3/62WwIffdR4\nWaqz/fbuhwmCpmJ9hqTmmlmPPMtT1/1iSCpocubPdwvhqad8au4ee7gC6tvX83osWZL2hQC8845b\nIa+84hbPmjVp66WuxYODB/usp/vvzy5LZaXnD+ndu2Z7Emy9tedqz8aKFa60cplIEDQfmmxIStK/\nJX1Vy+ffQLd8CBAEpcyWW8Lkye4raN0apk71GVm//rW/qDOVBfjqdIAhyQqlzKGu6dN9e9dd/oKX\n3En+r3/BSy/5Cx287IQTas66mjfPpwSnmDDBt6lov4sW1b2S/YQT3FczcSIsXJh7HwRBijoVhpl1\nNLNOtXw6mlm9a0zrS6CU1BkoaYKkKZJeasxDBEGp8POf+wv8669r+ix694Zf/MKVToq5c2HvvaFX\nL19DMnu2Twn+29/SlkNqLcpZZ1VdF9Knj7/8U+2tWpV9FbuZzxrbfHOfBbbFFk0zoytoXqyPDyMX\n6kygJKkzcAtwuJntChxXYHmCoOAceih8/HHNyLzgU3ivv97Dv5u57+XSSz1ab0WF//L/+muvm5r9\nNHGibzfdFM49t2p7e+zh60gyLYvU9ZncmMxnPPZYV0obbpi2hoIgV5oi+GBd0WrPBbY0s9/m0E74\nMIIWwXXX+cK8Dh1gxAg45xxfSLjFFj6slfJh1MZf/uJKpfpXZbPNqg5ZXXutD3Hdckvdsnz1lS+g\nrCv+1nvv1S1TUFxKZVptPtgR6CrpJUlvSzq1yPIEQdG55JL0FN299/bt4sXQr5/PtqqLI4/07d13\nVy3/4IOqYVCOPRbmzKm7reeecx9N9dlfmcycWXMWmll66nHQvCi2hTEc2AsYDHwLeAM4zMxqJOOU\nZMOGDfvmOBIoBS0BM0+B26dP7tc0NJT73Lk+BTjTihg92tefQM0kVaNG+XTjK6/0GVcDBniirBQ3\n3+xpej/5JHeZg/xRPYHSlVdeWVb5MOpSGJcC7czsyuT4DuAZM3u0lroxJBUEObBqlQ8/5ZLKduVK\n92d07uxThN99130py5Z5pOAZMzzMfIpUFGFwhTFsmFtEw4alw5WksiS+8IK3ke8V7kHDKLchKZF9\nkd8TwH5JQMMNgX2A6U0gUxA0W9q1c2WxcqX7Lq65JnvdlFWRWgDYp4+nyN13X3/p77ijZ0/88Y/T\n9S9OYjxcfLH7VP70J5+ym1pDcuSR8Oij8P3vp6MHV+dvf4O3317/Zw2aloIqjCSB0uvAjpLmSjpD\n0jmSfgJgZu8BzwKTgDeBEWY2rZAyBUFL4eij3dL4zW+y12nXLj2rauxYVxDHH1+1TkWFr9+QXAnd\ncIMvUOzQIe3s3mwzXxdSWenrVnbYwaMIZ3Oqn3CCh1M5/HBvd/hwL2/MIMLKlT7rLCg8BVUYZnaS\nmXUzs7Zm1iPJqfFXMxuRUecGM9vFzHY3s+GFlCcIWhK33urDRqmQ7tnYYANfy9G3r68kv/76mudT\ncbrWrfMXfCpOV4r+/V2RvPaav/R32cWnF7/+eno9ypw50LWrJ7ECX2/ycZKzs1Wr9HBYQ1i82IfC\nNt64YdeVEhJcdFGxpciNgvsw8kX4MIKgsHz0kQdjnDu35jTatWs9dPzQobVfu3ChT/udM8dnTqXm\no/Ts6b6M7bdPt7l6teds328/uPpqd5xPmOB1OnaE5cs9pIoZbLRR3TLPmuX5T8Drf/qpW0y9ejWy\nE4qA5BEFCpWsK58+jMgIHAQB4NFvs8Wiat06u7IAX0HevbtHBu6REWFuwABfeJgKofLIIx4kcuxY\ntzwqKmrmWc9MfpVaB5Ja9X7WWR5a5eGHPa/JhRd63K7Jk32tylZbuQ/m3XfTbSxf7rO8Gmq9NBVP\nPglHHFFsKXKjRLswCIJyY+7cmomd7r4bfvAD93n06+frP1KkXuCZyuKxxzxkfCpC8KOPun+iosLj\nbd11l5evXeuO+quu8uGyoUPhgAP83PXX+zTfn/7UFVOHDjVzhlxzDVx+ue8vXuwxupYv91D3y5fD\nbbf5vTOZMaMwMbhS+eW//DL/beebGJIKgqAkueMOH9I66CA/HjvWLZa1a93i2Gkn/3Xeu7cPc7Vr\n5/lLfvCDtDL66CO3nPr18xf+0qVpawXcutl2Ww+Xss8+MG5cVRnuucf9LgcfnA6fv3atD5lVVHgC\nrhNPdKUjuY+nMQmtHn7YrYxCTEHO55AUZlYWHxc1CIKWxuGHm510ktnq1WazZ9dep7LSbPr09P4x\nx5ilXhmrVvnWVYXZ9debtWtntummZmPGeNlJJ5m9957ZeeeZPf202S23mO27r9mKFenr+vf37f33\nm3Xv7vsjR/p2zBizSy7x/QULzGbMSMv2xRe1y/zOO2ZTpqSPJ0xYr27KSvLuzMt7OCyMIAhaBMuW\neQytK65wn8a8eT78dMYZbinUlidk6lRPlgXuT7n7bo/z1bp1uqxTJ58l9uabXrZypbe/dKkvotx8\nc0/d27Vr1bYld87PmuUWS5s2vtDxqqt86Oukk2rK8847PkOtrthe1Sm3hXtBEARFp0MH912kwpx0\n7+7DWW+8kT2p1M47w6RJbmN07Ag/+5krBPAhso4dPXTKq6962Y03poeVOnd2ZzzATTdVbXf1at/+\n/ve+TeVNef11j/t18snwj39UvebLLz0511dfNe7580EojCAIgixIsNtuVcs6dHAF8t3v+nH79v7C\n//hjOP98L7vsMt/uvbe/5K+6qmpAxksu8e0Pf5gumz3bt6mJA7/4ha+LSSXeeustb6tDB19N//zz\neXvMnCnokJSkO4HDgYVWSyypjHp74yvCf2hmtQYTiCGpIAjKkc8/9+nG48b51N/334cLLvBEW0cf\nXbVuyiE/fryvlu/UKV1+xBGugHr08GG0Qw+FP/7RraDMzI7VyeeQVKEVxn7AMuDebApDUgXwHLAS\nuCsURhAEzY2ZMz0BVqYCyIXLLvNYXRMnepyvESN8CG3kSPdn7LUX3HknnHlm9jbKxodhZmOA+mYX\n/wx4BPiskLIEQRAUi223dd8E+HqVXPnDH9wXsskm8MQTcMop7if54gtPtQtNu36jqCu9JXUDjjaz\nQZL6FVOWIAiCQtK3b8ODK1ZUQJcuHg8slRwrMzfJyJHwzDP5k7E+ih0a5Ebg0ozjOs2m3/3ud9/s\nRwKlIAhaArNmZT83ZEhNJVQ9gVI+KXYCpZmpXWATYDnwEzMbXUvd8GEEQRA0kHILPpg1gZKZbftN\nJWkkrlhqKIsgCIKg+BRUYSQJlAYCG0uaCwwD2uBL1UdUqx7mQxAEQQkToUGCIAiaMWUzrTYIgiBo\nPoTCCIIgCHIiFEYQBEGQE6EwgiAIgpwIhREEQRDkRCiMIAiCICdCYQRBEAQ5EQojCIIgyImCKgxJ\nd0paKGlSlvMnSXo3+YyRtFtt9cqJQgX9yjchZ34pBznLQUYIOUuZQlsYI4GD6zg/E9jfzPYArgFu\nL7A8Badc/olCzvxSDnKWg4wQcpYyBY0lZWZjkmi12c6/mXH4JrBVIeUJgiAIGk8p+TDOBpowFUgQ\nBEHQEIqaDyOjziDgf4D9zKzWhIOSIvJgEARBIyinfBh1Iml3YARwSDZlAfl74CAIgqBxNMWQVNYE\nSpJ6AI8Cp5rZR00gSxAEQdBICjoklZlACVhItQRKkm4HjgHm4EpljZn1K5hAQRAEQaMpmwRKQRAE\nQXEppVlSWZF0iKT3JL0v6dIiyzI7WWg4QdJbSVkXSf+UNEPSs5I6Z9S/XNIHkqZLOqiActVYJNkY\nuST1lTQp6esbm0jOYZI+ljQ++RxSAnJuLelFSVMlTZZ0QVJeMn1ai4w/S8pLqj8ltZU0LvnOTJY0\nLCkvmb6sR86S6s+Me1Qk8oxOjgvfn2ZW0h9cqX0I9AQ2ACYCvYsoz0ygS7WyPwGXJPuXAtcm+zsD\nE/DJBb2S51CB5NoP6ANMWh+5gHHA3sn+08DBTSDnMODiWuruVEQ5twD6JPsdgBlA71Lq0zpkLMX+\n3DDZtsLXXPUrpb6sR86S68+k3YuA+4DRyXHB+7McLIx+wAdmNsfM1gAPAkcVUR5R0zI7Crgn2b8H\nODrZPxJ40MzWmtls4AP8efKOmY0Bqs8ya5BckrYAOprZ20m9ezOuKaScUPvEiKOKKOcCM5uY7C8D\npgNbU0J9mkXG1OLXUuvPFcluW/zFZZRQX9YjJ5RYf0raGjgMuKOaPAXtz3JQGFsB8zKOP6a4K8IN\neE7S25LOTso2N7OF4F9iYLOkvLrsn9C0sm/WQLm2wvs3RVP29fmSJkq6I8OULgk5JfXCraI3afjf\nuklkzZBxXFJUUv2ZDJ9MABYAzyUvqZLryyxyQon1J/DfwK9IKzRogv4sB4VRagwws764dh8q6XtU\n/aNRy3GpUKpy3Qpsa2Z98C/qn4sszzdI6gA8AlyY/Iovub91LTKWXH+aWaWZ7Ylbaf0k7UIJ9mUt\ncu5MifU2Dw6bAAAFFElEQVSnpO8DCxPrsq71aXnvz3JQGJ8APTKOt07KioKZzU+2i4DH8SGmhZI2\nB0jMvM+S6p8A3TMub2rZGypXUeQ1s0WWDKLiAShTw3ZFlVNSa/xFPMrMnkiKS6pPa5OxVPszke0r\n4GXgEEqsL7PJWYL9OQA4UtJM4AFgsKRRwIJC92c5KIy3ge0l9ZTUBjgBGF0MQSRtmPyaQ9K3gIOA\nyYk8pyfVfgSkXi6jgRMktZG0DbA98FYhRaTqL44GyZWYsUsl9ZMk4LSMawomZ/LPneIYYEqJyHkX\nMM3MbsooK7U+rSFjqfWnpE1SwziS2gNDcH9LSfVlFjnfK7X+NLNfm1kPM9sWfx++aGanAk9S6P7M\nt+e+EB/818gM3FlzWRHl2AafpTUBVxSXJeVdgecTGf8JbJRxzeX4rITpwEEFlO1+4FNgNTAXOAPo\n0lC5gL2SZ/sAuKmJ5LwXmJT07eP4WGyx5RwArMv4e49P/g8b/LculKx1yFhS/Qnslsg2MZHrisZ+\nb4okZ0n1ZzWZDyA9S6rg/RkL94IgCIKcKIchqSAIgqAECIURBEEQ5EQojCAIgiAnQmEEQRAEOREK\nIwiCIMiJUBhBEARBToTCCEoOSZWS7s04biVpUUYY5yMkXVJPG1tKeijZ/5Gk4Q2U4fIc6oyUdExD\n2s0nkl6S1LdY9w9aHqEwglJkObCrpLbJ8RAygqeZ2ZNmdl1dDZjZfDM7PrOogTL8uoH1ywpJrYot\nQ1B+hMIISpWnge8n+yfiMXOAqhZD8iv/JkljJX2Y+sWfhJKZnNFej+QX+QxJv81o67Ek8vDkVPRh\nSX8E2suT04xKyk5TOnHWPRntHlD93pkkckyTNELSFEn/SCnCTAtB0saSZmU832PyZDgzJQ2VdFEi\nz+uSNsq4xWmJTJMk7Z1cv6E8UdWbkt6RdERGu09IegFfERwEDSIURlCKGJ735MTk5bo76bDdmXVS\nbGFmA4Aj8CQytdXZG/hPYA/guIyhnDPMbO/k/IWSupjZ5cAKM+trZqfKI5b+GhhoHsn0whzuncn2\nwHAz2xVYChxbx3On2AXPTdAP+D2wzDxK8pt4zJ8U7ROZhuJxpQCuAF4ws/7AYOCGJDYSwJ7AMWY2\nKIsMQZCVUBhBSWJmU/DsYCcC/0fdYZwfT66ZTjoHQHWeM7MlZrYK+Due+Q/g55Im4i/irYEdkvLM\n+w0GHjazL5P7LGngvWeZWcraeSd5rvp4ycxWmNliYAnwVFI+udr1DyT3fw3oKKkTHhTzMnleh5eB\nNqQjPj9nZktzuH8Q1KB1sQUIgjoYDVwPDAQ2qaPe6oz9bIqlRu4FSQfgymAfM1st6SWgXQNlzOXe\nmXXWZdxjLekfbdXvm3mNZRxXUvV7W1tOCQHHmtkHmSck9cf9Q0HQKMLCCEqR1Iv3LuBKM5vaiGur\nM0TSRsnQzNHAWKAz8GWiLHoD/TPqf53hGH4RH8bqCiCpSwPvna18NvCdZP+4LHXq44eJTPsBS83s\n38CzwAXf3Fzq08i2g6AKoTCCUsQAzOwTM/ufXOrWcZziLXwoaiI+vDQe+AewgaSpwB+ANzLqjwAm\nSxplZtOS868kwzypjGu53jtb+Q3AuZLewUNTZ6OudldJGo9nhTszKb8af65JkqYAV9XRdhDkTIQ3\nD4IgCHIiLIwgCIIgJ0JhBEEQBDkRCiMIgiDIiVAYQRAEQU6EwgiCIAhyIhRGEARBkBOhMIIgCIKc\n+P9QXNyrPLKEDwAAAABJRU5ErkJggg==\n",
|
|
"text/plain": [
|
|
"<matplotlib.figure.Figure at 0x2ec0f89f588>"
|
|
]
|
|
},
|
|
"metadata": {},
|
|
"output_type": "display_data"
|
|
},
|
|
{
|
|
"data": {
|
|
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYwAAACfCAYAAADqDO7LAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztnXecFdX5/98fARWlKLFFFBQR6xcVFYkaWUmMJsYSicbe\nEo0tltijRmLvPcbuzxJFDRaMJRojdmyAgFRFVJoFqSL9+f3xzOXevdy9O7vcvXsXnvfrNa+Zc+bM\nOc/M3p1nznnOeR6ZGUEQBEFQGys1tgBBEARB0yAURhAEQZCKUBhBEARBKkJhBEEQBKkIhREEQRCk\nIhRGEARBkIpQGBWGpH9IuqCuZSX1lPRlw0q3pN3PJPUqR1tNDUmLJXUq97XLSm7bdfkNFqhnlqSN\nSilbUDmEwigTksZLmiupXV7+4OSftQOAmZ1oZpenqbNA2XotqpHUMZEhfg8s8/NYloVNNV4raYCk\nHyTNlPS1pH6S1l2GtmpsO+1vUNKrko6tVolZazMbX0K5Mm2NlzQnuf9Zyf6WUrcTFCdeEOXDgM+A\nQzIZkrYGWrJsL5lSoEQGNXhDUrOGbqNE1PdvsizPsNi1BpxkZm2ALsAawI0FK6mfomvwv/0yYsDe\nZtYmUUptzOzUQgUL/cbq+rtrQr/TshIKo7w8BByVkz4KeCC3gKT7JV2SHPeU9KWkP0v6StJESUcX\nKpvN0vmSvpE0TtKhOSd+JWmQpBmSPpd0cc51ryX76cmX207JNcdJGpHkDZe0bc4120n6SNI0SY9K\nWrnQDUs6StKbkm6Q9A1wsaSLJT2UU6baF33y5XpJct1MSS/m98xyrh0h6Vc56WbJF/i2klaR9JCk\nbxM535W0dqF60iJpR0lvJ/VNlHSrpOZ5xfaW9GkixzV51x+byDxV0guZnmXa5gHMbDrQD9g6qfN+\nSbdLek7SLKBK0sqSrkv+1pOT86vkyHG2pEmSJkg6hhwFmf+7krRf0hOeIWmspF9Iugz4KXBb7te+\nqg9ttZH0YPIcPlPOMFfyu3hD0rWSvkue115p7n+pzOq/sW/x31ihPEm6UN5bmSLp/0lqk9SR+Q0e\nK+lz4JW0f5QViVAY5WUg0FrSZsnL8XfAwxT/ulsPaA2sD/wB+LuktkXKtkvKHg3cJWnT5Nxs4Agz\nawvsDZwgad/k3G7Jvk3y5faupAOBvwKHJ1+1+wJTc9o6EPgFsDGwTdJeTewEfAKsC2SGOvK/4PPT\nh+AKdW1gFeCsGup+BDg0J70X8I2ZDUmubwO0x5/LCcAPReRMwyLg9KS+nwC9gJPyyuwPdEu2/ZQM\n20jaDzgvOb828AbwaF0FkLQW0BsYlJN9CHCpmbUG3gKuBjoDXZN9e/zvSfJi/jPwM2BT4OdF2uqO\nf9Scmfx2dgPGm9mFifyn5H3t5/4db8N/uxsBVcCRiXLK0B0YCfwIuBa4ty7PIY/Mb2wdsr+x/Lxj\ngCOBnkCnRLbb8urZDdgc2HMZZFluCYVRfjK9jD3wf5ZJtZSfj78IFpnZC/iLf7MayhpwkZktMLPX\ngeeAgwDM7HUz+zg5Hg70xf9xcslVXL8HrjGzQck148ws16h+s5l9lXztPgvk9j7ymWhmt5vZYjOb\nV8v9ZrjfzD5Nyj9epP5HgX0lrZqkDyH7El6Av4y6mDPYzGanbL8gZjbIzN5L6vsCuIuln+NVZjbD\nzCYAN5EdhvwjcKWZjTGzxcBVwLaSNkzZ/K2SvgMG47+bM3POPWNmAxMZ5wHHAWckcnyftJWR40D8\n+Y40sx+APkXaPBa418z+l9Q92czGFCkvWDIs9jvgPDObY2afA9cDR+SU/dzM7jN3aPcAsJ6kdYrU\n/XTSG5mW7H+fc67Qbyw/71DgBjP73MzmAOcDBys7hGfAxWb2Qx1+pysU+V3poOF5GHgd/zJ/MEX5\nqcnLJcMcoFUNZaeZ2dyc9Od4bwP5MNOV+DDGysn2RJF2NwQ+LXL+qzyZflykbH1mb03Jq7/gPZvZ\np5JGAPtI+jfeE/prcvohYAOgb9Irexi4wMwW1UMeAJIe2w3ADrj9qTnwYV6xCTnHS/4GQEfgZknX\nZ6rDX1LtSfeM/mRm99Vwbsn1ybDbasCH0pJvgJXIfhCsD3yQJ2NNvdwN8Q+PurIW/my+yGunfU56\nyd/YzH6QC9sK+LqGOvczs1drOFfo+eXnrZ/IkCtPc7znm2ECQY1ED6PMJF+lnwG/BJ4scfVrSmqZ\nk+5AtgfzT+BpoL2ZrQHcSfYlUcjA+yWwSYnkyq//e/yFlqGYsklDX/zrcT/gYzMbB2BmC83sUjPb\nCtgZ2AcfklgW/oH3DDdJnuMFLP2yze0xdCT7N/gS+KOZtUu2Nc2sVaZnsIzkPuNvcSW7VU5bayRD\nSgCTC8hYk5G/2O+g2MSAb/EeXse8diYWuaY2apsUUFvepALyLKD6x09jT0CpaEJhNA7HAr2S4YBS\nIuBvklpI+iluq3g8OdcK74EsSMalc8f9vwEWU/3FcA9wlqRuAJI2qcPQSW0MAXaTtGHy5X/eMtbX\nF7ennIjbNACQVCVp62TIYTb+clhcuIqlELBqYjjPbMLHvWea2RxJmydt5nO2pDWS53VqIh/AHcBf\nJG2ZyNdW0m/rfrvFSYZ47gZuSnobSGov6RdJkceBoyVtIWk1sj2yQtwLHCNp98RovL6kzJDoV7gt\noJAMi5N2LpfUSlJH4Ay819dYPAqcIWkjSa1wu0bfnB58pc8Ua3RCYZSP3Hnun2VsA/nn6lJPASYD\n0/AvqYfwr9mxybmTgEslzQAuBB7LkecH/J/nrWRsuLuZ/SvJe0TSTOAp3NBbV3mXvgGz/ybtDwXe\nx20g1YrUsb4pwDtAD3LuC58E8C9gBvAx8CrJC0u+OO32YtUCs/Av9R+S/e643eCw5JncSVYZ5F73\nDD5MNSi5t/sSOZ/GbQl9JU3H73+vvGuLyVOXc+fiBt+BSVsv4dNxMbMXcdvK/4AxFJkRZGbv48bi\nm/DnOADvuQLcDBwon/F1UwFZTsWf2zh8GPZhM7u/jveRy7PJjKzM1q+W8vnch//9X8eHW+ckMqZt\nf4VHEUApCIIgSEP0MIIgCIJUhMIIgiAIUlFUYchXzdY0jS0IgiBYgSi6DsPMFiXL5dua2YxyCVUI\nSWFsCYIgqAdmVpIZYGmGpGYDwyTdK+mWzFaKxuuKmVX8dvHFFze6DCFnyNlUZQw5S7+VkjQrvZ+k\n9AvMgiAIgiZGrQrDzB6QeyLtkmSNNrMFDStWEARBUGnUqjAkVeGOwcbjKyE3lHSUuXO7II+qqqrG\nFiEVIWdpaQpyNgUZIeSsZGpduCfpQ+BQMxudpLsAj5rZ9mWQL1cOK/V4XBAEwfKOJKyMRu8WGWUB\nYO7auEUpGg+CIAiaDmkUxgeS7kkcuVVJupvqrpGLImkvSaMkjZF0boHza0h6Uh69bWDGMVsQBEFQ\nWaQZkloFOBnYNcl6A7jdUgQYSbyEjsEje03CHc0dbGajcspcA8wys0sTL5h/N7OlIoDFkFQQBEHd\nKduQlDwQ+n1mdoOZHZBsN6ZRFgndgbHmEa4W4J4998srsyXuNZNk6GsjpY27fMEFoPBIHARBUA6K\nKgzzyGQdk2m19SE/ktgEqkfcAvgIOACWxA/ugEdJq53hwzOC1lO8IAiCIC1pFu6Nw+Mk9McjpQFg\nZjeUSIar8LCVg4BheLzidCE0OybBsyZNgvb5eqgO/PADrLYafPQRdO1a/3qCIAiWY9IojE+TbSU8\n2lhdmEg22Ap4z6FaiEYzm4VHoANA0me4klqKPn36QN++0KIFVbfeStUtt8Ann8CcOXUUK4977/V9\nu3bFywVBEFQ4AwYMYMCAAQ1Sd1Gjd2LDuNrMzqpX5X79aNzoPRl4DzjEzEbmlGkLzDEPHXocsIuZ\nHV2gLjd6n3EGPP44TFyW0MB5XHstTJ4MN9wAbdrACy/ALruUrv4gCIJGomxG78SGUe83Z3L9KXh4\nyI/x+LkjJf1R0vFJsS2A4ZJGAnsCpxWt9OyzfQhq0qT6irU0b7yRHd6aNQt23bV4+SAIghWQNENS\nQxL7xRNUt2GkckhoHj94s7y8O3OOB+afL8r66/v+ySfhlFP8eMECmD0b1lwzdTXVWGUV6NHDjydO\ndHtI375w8MFwyy3w/vvwUGPGrg+CIGh80qzDKBS03czs2AL5DUa1dRiPPw577glt23q6f3+48054\n7rnSNHb44fDPf7p9pHNnzyv0nD79FFq0gA4dlj4XBEFQAZRySKpWhVEpFF24N3Ik7LsvjB3r6d69\n4dRToWfP+je4cCF89x2suy5cd50rqE02gZYtc4XyfRN5hkEQrHiUxYYh6fGc46vzzr1UisZLRufO\nMGGCT4+dONGHqwp5kpw2Db7/fun8QjRvDuusA2+/DSecAEccASNGZM8vSmb+3lCq2cVBEASVTTGj\n96Y5x3vknUu3ErtctGgBc+fCY495byMzVDVnjiuJDO3aQatWdav7Jz+B1VeHIUNghx2y+WPGQKdO\nPmurNj75BM5dyo1WEARBk6KYwig2zlKZYzB9+sDPfuZDSYsWwYMP+kt+/nw3ioO7E8nl9de9d1Ib\nr77q+5kzYfFiH556+eXarzvgALjxRrjmmjrdShAEQaVRTGGsJmk7SdsDLZPjbpl0meRLz5w5MGqU\n2xVWWsm3XXeFceNccRx0kJe77LLq1116afWhppqoqoJevWDvvaFZM1dInToVLvvrX3tZgKee8llc\nANdfX69bC4IgqARqNHpLerXYhWa2e4NIVAP18lZr5sNQxx0Hv/+99zK6dPFZUM88A19+6TOc0roE\nmTjReyxTprhRvFmzpctMmAAbbujHc+f68Njs2T5slpEpCIKgTJTF6G1muxfbStF4gyP5FNwpU2Cz\nzWD77X14qn9/N2ZnpsPW1FPIp317OP10P84oi8GDs70JgKlTYccd4dln4a67YN48N6DPnAkfpA4j\nEgRBUHEsH9Nq68raa3sv4/PPYY894MQT61/X5Mmw7bbw1Vdw222+ePCww/zcOee4orjjjtLIHQRB\nUEdiHcay8uijvpL7mWeWva7Fi7238fOfew9kt93g2GRN48KFbr9oWXkmnyAIVgzKHdN7+aNLFx+W\nKgUrJY/wv/91A3uum/XmzUNZBEGw3JDGlxSS2gMdc8ub2esNJVSDs/32pXVeaOaG7datfWV4MRYt\nKmwsD4IgqHDS+JK6GvgdMIJsYCMzs30bWLZ8OSo7pvfXX7uymD8/OyOqUJmtt/Z9EARBGSirDUPS\naKBrHeJ4NwgVrzDSsGCBrxqfNcs95IKvH+nTJxb2BUHQIJTbhjEOqOGTuXYk7SVplKQxkpbyjyGp\njaT+koZIGibp6Pq2VfG0aOFxN4YMyeZ16+YBnJ580qfsNnWlGATBckuaHkY/YBvgFWBJL8PMTq21\ncmklYAwecW8S8D5wsJmNyilzPtDGzM6XtBYeoW9dM1uYV1fT72HA0h5uN98cRo/Onn/uOfjVr8ov\nVxAEyyWl7GGkMXr3T7b60B0Ya2afA0jqC+wHjMopY2RjhbcGpuYri+WK22/PumEHdzXy4ovw7ru+\nbmOrrfx8585Z5RIEQVABpFqHIWlloEuSHG1mC1JVLvUG9jSz45P04UD33N6JpFa4QtocaAX8zsxe\nKFDX8tHDSIMEAwfCTjs1tiRBEDRxytrDkFQFPACMBwRsKOmoEk6r3RMYbGa9JG0CvCypq5nNzi/Y\np0+fJcdVVVVUFYp50VSYORNuvhkuuqh6/sKkc9WjR9gzgiCoMwMGDGDAgAENUncaG8aHwKFmNjpJ\ndwEeNbPta61c6gH0MbO9kvR5+JTcq3PK/Bu40szeStKvAOea2Qd5dS1fPYwTTvCwsoWcGPbq5e7U\nBw6Eb75x77dBEAT1oNzTaoeaWdfa8mq4thluxP4ZMBl4DzjEzEbmlPk78LWZ/U3SusAHwDZm9l1e\nXcuXwigW3nXBAvd6m3GKuDzddxAEZaXc02o/kHSPpKpkuxt/qdeKmS0CTgFeAj4G+prZSEl/lHR8\nUuwyYGdJQ4GXgXPylcVyx8SJvs+dHZVLixaw8cbuzBA89GwQBEEjk6aHsQpwMrBrkvUGcHu5F/It\nVz2MhQs90l+uW/Sa6NIFrrsO9i3rwvogCJYTwlvtisR//uMBmbbcsrElCYKgCVIWhSHpcTM7SNIw\nCsTwTmPDKCUrrMIIgiBYBsqlMH5sZpMldSx0PrMYr1ys0Apj1ix4+OFlC/QUBMEKSblCtE5ODk8y\ns89zN+CkUjQepOT99+Gkk+APf4AXXoAZMxpboiAIVkDSGL0HmVm3vLxU02pLyQrdwzDLBmrq3h2O\nPx5+//uay7/8MlRV1exmPQiCFYay9DAknZjYLzaXNDRn+wwYVorGg5Rk1mz89a++iO/++12JFFKg\nc+fCL34BDz1UXhmDIFjuKWbDaAusCVwJnJdzalZjrJNYoXsY4K5EVl/d3YmceSYccQQ8/bTn5zJg\nAOy+ux9/9BF0LWtHMAiCCqPcK717AB+b2awk3QbYwszeLYUAaVnhFUaG6dNhzTWz6UGD4IEH4Kab\nPH3hhfDEEzBmjKfjmQXBCk25FcZgoFvmbZ3EuPgg367R0ITCyEOCAw+EKVPgjTdcMTz+OOyxh0fz\nO/xweOqpUBhBsIJTbtcg1d7UZraYdHE0gobk88/hscfg+ec9PW8e/O53bt9YbTXo2xcOOAAWL25c\nOYMgWG5IFaJV0qmSWiTbaXjY1qAx6dDBexmtWkH79t7LAFccACuvDP36ZWdXBUEQLCNphqTWAW4B\neuErvl8BTjezrxtevGpyxJBUTey+O+y2G3zwgYd4DYIgSAhfUsHSfPopTJoEP/1p9fzZs91d+nXX\nwamnQrt20Lu3z7CK3kcQLPeUyzXIOWZ2jaRbKexL6tQClzUYoTDqwYgRHiN8/fVdmay1lts3vvjC\nZ1YdeWT18h9+CM8+C7/5DWyzTePIHARBSSlXiNZMkKNUsS9qQtJewE24veTe3Gh7yfmzgMNwpdQC\n2AJYy8ymL0u7AXD55b5/6y345z99yu3XX8NGG8HIkdlyyvstdeoUCiMIgqVo0CGpZAruGDzi3iTg\nfeBgMxtVQ/lf4/aRnxc4Fz2MujJ2rC/0u+02WLQImjf3qbYdOsC33/qK8AUL3ECeKb/ppvDee+5O\nffZsn2XVrp1P1Q2CoMlRlh6GpGcpMBSVwczSRPTpDozNeLaV1BfYDyioMIBDgEdT1BukYdNNs1H7\nmjVzp4Vt2nh69Gjo0cMDOV10kTs23GADOPpoH8backufugtuF3n99Ua5hSAIKodiNoyeyeEBwHrA\nw0n6EOArMzuj1sql3sCeZnZ8kj4c6F7I/iGpJTAB2KTQcFT0MErMxImuIA480Bf85ZM/TJV59mPG\neBTASZO8J7LBBrDDDunaXLDAw822aeM9F2npdoIgKCll6WGY2WtJY9ebWe4b4VlJy2TXqIF9gDeL\n2S769Omz5LiqqoqqqqoGEGMF4cc/9v0TTxQ+378/7LKL2zy22CKbv9lmMHw4PPmkO0Ps2tV9VhVj\n8mT473/h2mth2DBXPhtt5P6wMnaWIAhKwoABAxgwYEDDVG5mRTfc+N0pJ70xMLK265KyPYAXc9Ln\nAefWUPZJ3L5RU10WlJh//MPsww9rL7dokfvG/eKLjI9cs0su8f3XX9d+fceOXvb++33/3Xe+f+KJ\n6uUWLDA78USzOXPqczdBEBQgeXfW+r5Os6VZuLcXcBe+ultAR+CPZvaf2pSRpGbAaNzoPRl4DzjE\nzEbmlWub1L+Bmf1QQ11Wm6xBAyLB/vt7z+Qf//BY48cdB//7H3Tu7ENVm25a+NoTT4Q77nDHiWus\n4eU/+QQWLnTbSm4bkB2uymX6dPfWGzE+gqBOlH3hnqRVgM2T5Cgzm5e6AVc4N5OdVnuVpD/iWu+u\npMxRuK3j0CL1hMJoTDIvcDN3P5KZNbVwob/Ed9zRbRqF+OEHuOsuOO00OP10aNkSrrrK63rtNXdt\n0rkz/OQn8O671f1fDRvmZf70JzjlFLj11oa9zyBYziir80FJqwFnA6eY2UdAh2T6ayrM7EUz28zM\nNjWzq5K8OzPKIkk/UExZBBXAuHHw2Wd+nDvFtnlzOOMMDyMLPptq9uzs+fbt/dxpp3n6ppvg0kth\nyBB4+22PDHjmmX6uUycYONAVUubj4J134F//8uPvv/f9DTfAdts1yG0GQVAzaXxD3A/MB36SpCcC\nlzWYREFlsvHGbqguxBVXQOvW/pLv2dPXfgwf7ivGJ01aeqiqeXNfGJhRCvvs44bxf/7TQ9CuuqpP\n7wUYPz47C+vee2HoULj+elc4QRCUlTQKYxMzuwZYAGBmc3BbRhA4q67qNobRoz39/vs+Xffppz2d\nmZGVT+fOvj/uOHdfMn589tyDD/p+/HjYdltXLpIrmosu8nMLFmTz11qr1HcVBEEeaRTG/GSNRCaA\n0iZAahtGsIKw0kpZ20PbtjBqlIeP/fTTmq9Zd11/4bdr5+l11vH9Nde4cRzchtGhQ7ZegF69fP3H\npElZ28rUqdl6Fy1yu0kuc+fW3CuZPTsCTQVBCtIojIuBF4ENJf0Td29+ToNKFTRNttzSjeCXXOLp\ne+5xu0RtTJ3qL+zVVvP0kUe6kRx8aGu99fz4oYc8omCXLj5j67rrXDE88ww8muMg4KKLsnVluPLK\nmu0erVvHepAgSEHRWVKSBGwAzMHXVAgYaGbflke8arLELKmmhOThY9ddt37XT5jgBvFHH/UZWDW1\n8eqrXi4/H6r3GjJ5uVN5p051A3v79r5I8aijoFs32H77+skcBBVI2WZJJW/o581sqpk9Z2b/bgxl\nETRBzOqvLABeeMGHs2pa2wEekvaii7Kzp045xXs1GRYsyB5vvrmvSM8oi3nz3O7Rvr2n58+H4493\nA/sZZ4TLkiAoQJohqUGSavjEC4IGonVr32fsFoWYPh3efDPrbffvf3cD+tNPu/0kd5Hf9OlZ5bB4\ncda2csUV7v5k443hxhs975FHSnsvQbCckEZh7AQMlPSppKGShkka2tCCBSs4W2/thvRiX/qnnurD\nUS1aZIefzj0X9tvPFc7FF/v1M2a4gbxdO48J0qwZTJvma0POPx/WXhsee8wXB4LbRqD6epKa+P57\nj2BY7uHS1q3h/vvL22awwpPGNUjHQvmWuCwvF2HDCGpl5sys+3Zwu8Tbb3sv4vzzPW///d1IDoVd\nkHz2ma83eeopVwQHHQQPP+y9lp/8xGdn5fLKK/Dzn3tdb7zhruAzdT75pEcxvOKK0iqUuXN9tTzE\n7K6gVsoVonVV4ASgMzAMd+uxsBSN1odQGEG9OOYY6Ns3O832m298Btavf117VMFzzvFZWj17wnnn\nwb77ZpUN+DDXmmu6Uf7GG2HXXT0sbsa7b64yKtVvd8oU7x1NneqKau7c0tQbLLeUy+j9ALADrix+\nCVxfigaDoKzcfrsPR2VYe2244IJ0IWgPP9yN75nrV189u2jwrbeyCxX33htOPtmPx471vVm2FwDp\nhrfS0L+/r3TffHNo1ao0dQZBSorF9N7SzP4PQNK9uKfZIGhatGxZ/cVdFzI9hUMPdWWx1lru9+rI\nI703Admew0knZcPfgvc+Vl7ZjesffOCr4TOr0gvx1lvu8feYY4rLNHGiG+9/9CP37xUEZaRYD2PJ\nnMTGHIoKgkYjY0zfaSe3Q2Sm726wgS9IzBjHwXsuxx/vbkzA3be3bOluUfbZxxXMB0ncscWLfVrv\nm2/CCSd43tFHw7HH1i7TxInuRkVy470EZ59dslsOgmIUUxjbSJqZbLOArpljSTPTNiBpL0mjJI2R\ndG4NZaokDZY0XNKrdb2JICgLp5/uhu2PP/av+/32q7lsv35ub8iw0koweLArht139x7HvHlw553w\n//6fKxhwBfDQQ9nrhg+v3pPI9DAg6yH4uuvcMWPv3qW4yyComVJFYiq04QrpEzzoUgtgCLB5Xpm2\nwMdA+yS9Vg11FY0qFQRl49NPPWLgwoWFz99+u9nAgWbz5mXzWrTIRivMbM884/vjj/f9pZf6/rDD\nfN+vX7Zshq5dzQYN8uPZs81uu83sgAOWLleMjz82++qr+t170OSghBH30qzDWBa6A2PN7HMzWwD0\nBfI/yw4F+pnZxEQrxEryoLLp1Mlfz7nRAjP07u32jNdeyy4oBD9ee+3qZauqfC3FHXd4fRdc4LaS\n+fOzdWXo0sX3W24JG27ox6uv7m317et177lnOvkvuMDjlgRBHWlohdEe+DInPSHJy6UL0E7Sq5Le\nl3REA8sUBA3HHnv4fp99qud/9JHbMN56C+bM8dC2bdr4EFXGEC75tNw//cmHm9ZZx8uDz77q3999\na+W6cpfc1vL11/DcczXLlQl+lalr0009EFVMyw3qQLFZUuWiOdAN6AWsDrwj6R0z+yS/YJ8+fZYc\nV1VVUZXvdC4IGpsTTvDFfhmX7Rk22cT3mVlUu+9e+PoOHXybP99Xn++8sztMbN4cDjjAj2vi1Vdd\nYf3lL9W9786b572iL790I3zGR9e227r7+FgxvlwxYMAABgwY0CB1p4rpXe/KpR5AHzPbK0mfh4+n\nXZ1T5lxgVTP7W5K+B3jBzPrl1WUNKWsQVDQjR/pU2kzMkEK8+aavNIfqCwXnz8+G1f3gA1/wN22a\nr3ofP37Zohe+/rqvgM/12xVUFGVZuJeZDZUzM2pWPWZJvQ90ltRR0srAwUD/vDLPALtKapbED98J\nGFmfmwmC5ZYttiiuLMCn7k6a5MNd48f7NOB+/bxHIXmMkAsv9DUi4L60cl2p5PPcc76SfdCgmsv0\n7FndVhMs19Q4JGVmrZe1cjNbJOkU4CVcOd1rZiMl/dFP211mNkrSf4ChwCLgLjMbsaxtB8EKyY9/\n7DaSjTf26bcTJ7ovrJEjvaex9dbucwt8mGrcOPjXv9xVyqqrev6oUXDEEd4LmT7dIxxec42Xe+ml\n2mWYP9+veeONwgsVM5EZV2poE2pQalINSUnaFdjUzO6XtBbQ2sw+q+26UhJDUkGQkk039XUdv/xl\nNq5IocildN9OAAAQRklEQVSHixa5vWXKFPfyu+++PpSVeZHPmOGG9n32yYbMHTzY47VnXKD8/e+u\nWL791o3omWvbtnVDe74tB9zT7uzZ8NVXS/eaFiyI4a0SU8ohqTRrKS4GngXGJOn1gbdKNa837Uas\nwwiCdMyfb7b//r6//XazxYuLl89fwwFmRx2VTX/xhef97W9m117rx7fdZnbPPWY//OBl2rXLlhs1\nyve//nXh9o47zs8feaTvO3Xy/fDh6deSZBgypPb7W8GhhOsw0rg3HwJsBwwys+2SvKFm1rUkGisl\n0cMIggbiqqvc/fu228Itt7hTw65dq68zmTPHbSAZD70Z5s/3HsHWW8Oll/pMrmOPhfvu8/OF/mcz\n7tl33tndz2fYYAMPzVvI7XwhFi92GXfbzde9BAUpW4jWhPkZLZU0vnopGg6CoELIeNqdNMlfvNtt\nt/SixNVW8/0aa7gS2HdfT2eGj7bYwl2uS64s3nvP3Z5ceGH1ep580oezJk50ZdGqlde59tpZr8BP\nP51OYXybrPF9/fXSuI+fNq34tOWGYuZMmDWr/O3WgzQK43FJdwJrSDoO+C9wd8OKFQRB2Wjd2m0Z\n++yTfQnXRr9+7ucqw667uuv3xYv95b3jjh6I6vLLq9fZuzc8/7w7ULzySreBTJ3qPYvTT3e7yahR\nXvaNN3z/4ovVeyIA//63TzPOLGx85hmvZ1lo125pR47vv9/wQap22MFnmzUF0oxbAXsA1ybbHqUa\nD6vLRtgwgqBhuf9+tyHk+sBKy6uvFrY/gNmtt/rxuHGenjOneF0zZ3q53r2zdeTWffHFnr7iimze\nzjubvfaa2YwZZt99l80vZN+YNcts7lyzu++unn/OOV7v3Lme/vprTw8eXFzeXL791u+zLoDZnnvW\n7Zo6VV86G0baeW3DgDeA15PjIAiWN3r18n19ZilVVWV9YOVz2WXeQ/j97z1dW3yS1q19ttRbb/nQ\n1FNPQY8ebmcxg9/+1sv95S/Zazp1gscf99lZmTC6U6f6ce7w1nnnef0rrwzHHVd9KCjjr+u553yY\nqJBPr3yk7PAcuD2l0Iy0mshMcc6N5FjB1OoaRNIfgL8C/wME3CrpEjO7r6GFC4KgjHTosGzDL4UU\nzezZriCaNXM7SI8e6epq3jzrHn7NNf34qqugWzef1vvNN9VlXXttn/ILsO66MHmyu2mZNMn9dY0Z\n42tRHnnEy3z2ma9VGTXKh8/AjfUbbujuVX7zGzjsMG/riSeKy/rss76fOdN9gdWFoUP9njIr8Sud\n2rogwGjgRznpHwGjS9XFSbsRQ1JB0DQZONCHXfr2rdt133/vU27ffNOv79rVp+pmhoxyufRSH1Ia\nMcLbO+wws8028+smTMgOa62+utmxx7pr+kze+PFmPXtm65o82fO//tqnDc+atXR7c+b4uX/8w8ve\nf7/fX79+Zq1aeV6vXrXf49ChZjfdZPbYY35Npq1PPzW75JK6Pa8aoIRDUmle1G8DK+ekVwbeLpUA\nqQUNhREETZObb7Y6r6/IkLFBjB1rdvbZXs8nnyxdbt99s2189JEfn3OOpzO2k3z7ybXXmrVubXbe\neWbt22fz77qrdnm33NLsZz/zmCTvvJOtf8YMlzmTnjDBFV8hvv8+u47llFO8fGZdCXj9rVubLVpU\n+3MqQikVRjFfUn+W9Gc8ANK7kvpIuhgYCIxpmP5OEATLHaeemvVfVVcy9ofOnT1M7scfZz3/5vLI\nIz4MBb7SHXwWFfhsrRdf9CmzufaTs87yazJDXbn5m21WXK4RI+CVVzwmSY8e2XUgmTUkH33k6e23\nh222KVzHb38LJ57oxx07+v6dd7Kr5WfN8u3hh7PXzEwd7LRBKGb0bp1snwJPk6zDwJ0FltUtSBAE\nTZy2bZe9jubNPYBUIVZfHdZbz49btvQX7VlneVry4FKFAl6tniwry7y4Aa6/Hq69NpseP96DTgHc\neKMrh5Yt4T//yZbZbjs48sisC5WMwvnqK3fTUmh9xwsvZO0np58OX3xR3S7z1FO+P+oo348b556B\nG5Fizgf/Vk5BgiAISkarVunLjh+fjVMC8Ic/VD9/+eVwzz2+Ev7Pf4arr/YYI5lgWeAzrx54IJte\nZRV/+b/9Nuyyiy9W3GKL7PlvvvH9d9/5vnlzN7jvtpunFy3ynsaIEVnZpkzxHobkCqiQAmxgap1W\nK2ltSddKel7S/zJbOYQLgiBocDp2LL6y/OabfX/QQb4/8khXAGlWo++8M9x9t08TluCvf/X93Xf7\nUFZuzwZ8au8tt2SHpbbYItsL+vbb7NTfu+/2eso8RJXGl9RLwGPAWcAJwFHAN2Z2bsOLV00Oq03W\nIAiCBuGEEzxi4fPP12/q8YAB1aMsXn+991bScvLJbm+ZNs2nBM+c6cfdusGHHxa9tNy+pH5kZvcC\nC8zsNTM7Fg+nmgpJe0kaJWlMEl0v/3xPSdMlDUq2CwvV01RoqNCIpSbkLC1NQc6mICNUqJx33OGu\n3ocOXZJVJzmrqtyO86c/ebouymLCBLj9drdpLFrkgbEykwiksvqhSqMwFiT7yZL2lrQdUMDJ/dJI\nWgm4DdgT2Ao4RNLmBYq+bmbdku2yNHVXKhX5Yy9AyFlamoKcTUFGqGA5mzWD//u/Jck6yzltmg83\n1bWHsuGGvh850pXWe+/5sRkcfrivgC8Tta70Bi6T1BY4E7gVaAOcnrL+7sBYM/scQFJfYD9gVF65\n0gT3CIIgqFTS2DwKceaZbuTePPnWzszEAp9inCYKYomoVWGY2b+TwxnA7gCS0iqM9sCXOekJuBLJ\n5ydJ3I2JwNkWIVqDIAic666r+dxWW/kalTKRKkTrUhdJX5hZhxTlegN7mtnxSfpwoLuZnZpTphWw\n2MzmSPolcLOZdSlQV1i8gyAI6kGpjN5phqQKkbbxiUCuYtkgyVuCmc3OOX5B0u2S2pnZd3nlYtgq\nCIKgEUnr3jyftF/77wOdJXWUtDJwMNA/t4CkdXOOu+O9nu8IgiAIKooaexiSZlFYMQioxaG9Y2aL\nJJ0CvIQrp3vNbKSkP/ppuwv4raQT8dlYPwC/q+M9BEEQBGWgXjaMIAiCYMWjvkNSZaW2xX9llmW8\npI8kDZb0XpK3pqSXJI2W9J9kGnKm/PmSxkoaKekXDSjXvZK+kjQ0J6/OcknqJmlo8qxvKpOcF0ua\nkLN4c68KkHODxA3Ox5KGSTo1ya+YZ1pAxj8l+RX1PCWtIund5H9mmNzrdUU9y1rkrKjnmdPGSok8\n/ZN0wz/PUvlJb6gNV2qfAB2BFsAQYPNGlGccsGZe3tXAOcnxucBVyfGWwGB86G+j5D7UQHLtCmwL\nDF0WuYB3gR2T4+fxWW4NLefFwJ8LlN2iEeVcD9g2OW6FBxLbvJKeaREZK/F5rpbsm+EhErpX0rOs\nRc6Ke55JvWcADwP9k3SDP8+m0MNYsvjPzBYAmcV/jYVYume2H5BxVfkAsH9yvC/Q18wWmtl4YCyF\n16EsM2b2JjBtWeSStB7Q2szeT8o9mHNNQ8oJhWfe7deIck4xsyHJ8WxgJD7Lr2KeaQ0ytk9OV9rz\nnJMcroK/uIwKepa1yAkV9jwlbQD8CrgnT54GfZ5NQWEUWvzXvoay5cCAlyW9L493DrCumX0F/k8M\nrJPk58s+kfLKvk4d5WqPP98M5XzWp0gaIumenK50RcgpaSO8VzSQuv+tyyJrjozvJlkV9TyT4ZPB\nwBTg5eQlVXHPsgY5ocKeJ3AjcDbVJyY1+PNsCgqj0tjFzLrh2v1kST9l6dlklTqToFLluh3oZGbb\n4v+o1zeyPEuQLyz9F3Ba8hVfcX/rAjJW3PM0s8Vmth3eS+suaSsq8FkWkHNLKux5Stob+CrpXRZb\nn1by59kUFEati//KiZlNTvbf4JEIuwNfKVlPknTzvk6KTwQ2zLm83LLXVa5GkdfMvrFkEBW4m+yw\nXaPKKak5/iJ+yMyeSbIr6pkWkrFSn2ci20xgALAXFfYsa5KzAp/nLsC+ksYBjwK9JD0ETGno59kU\nFEati//KhaTVkq85JK0O/AIYlshzdFLsKDyMLUn+wZJWlrQx0Bl4ryFFpPoXR53kSrqxMyR1lyTg\nyJxrGkzO5Med4QBgeIXIeR8wwsxuzsmrtGe6lIyV9jwlrZUZxpHUEtgDt7dU1LOsQc5RlfY8zewv\nZtbBzDrh78P/mdkRwLM09PMsteW+ITb8a2Q0bqw5rxHl2BifpTUYVxTnJfntgP8mMr4ErJFzzfn4\nrISRwC8aULZHgEnAPOAL4BhgzbrKBWyf3NtY3K9XOeR8EBiaPNun8bHYxpZzF2BRzt97UPI7rPPf\nuqFkLSJjRT1P4P8S2YYkcl1Q3/+bRpKzop5nnsw9yc6SavDnGQv3giAIglQ0hSGpIAiCoAIIhREE\nQRCkIhRGEARBkIpQGEEQBEEqQmEEQRAEqQiFEQRBEKQiFEZQcUhaLOnBnHQzSd/kuHHeR9I5tdTx\nY0mPJ8dHSbq1jjKcn6LM/ZIOqEu9pUTSq5K6NVb7wYpHKIygEvke2FrSKkl6D3Kcp5nZs2Z2TbEK\nzGyymR2Um1VHGf5Sx/JNCknNGluGoOkRCiOoVJ4H9k6OD8F95gDVewzJV/7Nkt6S9Enmiz9xJTMs\np74OyRf5aEl/zanrqcTz8LCM92FJVwIt5cFpHkryjlQ2cNYDOfX2zG87l0SOEZLukjRc0osZRZjb\nQ5D0I0mf5dzfU/JgOOMknSzpjESetyWtkdPEkYlMQyXtmFy/mjxQ1UBJH0raJ6feZyS9gq8IDoI6\nEQojqEQMj3tySPJy7UrWbXdumQzrmdkuwD54EJlCZXYEfgNsAxyYM5RzjJntmJw/TdKaZnY+MMfM\nupnZEXKPpX8Bqsw9mZ6Wou1cOgO3mtnWwAygd5H7zrAVHpugO3A5MNvcS/JA3OdPhpaJTCfjfqUA\nLgBeMbMeQC/gusQ3EsB2wAFmtnsNMgRBjYTCCCoSMxuORwc7BHiO4m6cn06uGUk2BkA+L5vZdDOb\nCzyJR/4DOF3SEPxFvAGwaZKf214v4Akzm5a0M72ObX9mZpnezofJfdXGq2Y2x8y+BaYD/07yh+Vd\n/2jS/htAa0ltcKeY58njOgwAVibr8fllM5uRov0gWIrmjS1AEBShP3AtUAWsVaTcvJzjmhTLUrEX\nJPXElcFOZjZP0qvAqnWUMU3buWUW5bSxkOxHW367uddYTnox1f9vC8WUENDbzMbmnpDUA7cPBUG9\niB5GUIlkXrz3AX8zs4/rcW0+e0haIxma2R94C2gLTEuUxeZAj5zy83MMw//Dh7HaAUhas45t15Q/\nHtghOT6whjK18btEpl2BGWY2C/gPcOqSxqVt61l3EFQjFEZQiRiAmU00s9vSlC2SzvAePhQ1BB9e\nGgS8CLSQ9DFwBfBOTvm7gGGSHjKzEcn515JhnkzEtbRt15R/HXCipA9x19Q1UazeuZIG4VHhjk3y\nL8Xva6ik4cAlReoOgtSEe/MgCIIgFdHDCIIgCFIRCiMIgiBIRSiMIAiCIBWhMIIgCIJUhMIIgiAI\nUhEKIwiCIEhFKIwgCIIgFf8f4QHNJgnY6BMAAAAASUVORK5CYII=\n",
|
|
"text/plain": [
|
|
"<matplotlib.figure.Figure at 0x2ec10398ef0>"
|
|
]
|
|
},
|
|
"metadata": {},
|
|
"output_type": "display_data"
|
|
}
|
|
],
|
|
"source": [
|
|
"pred_vgg = train_and_evaluate(reader_train, \n",
|
|
" reader_test, \n",
|
|
" max_epochs=5, \n",
|
|
" model_func=create_vgg9_model)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Residual Network (ResNet)\n",
|
|
"\n",
|
|
"One of the main problem of a Deep Neural Network is how to propagate the error all the way to the first layer. For a deep network, the gradients keep getting smaller until it has no effect on the network weights. [ResNet](https://arxiv.org/abs/1512.03385) was designed to overcome such problem, by defining a block with identity path, as shown below:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 26,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/html": [
|
|
"<img src=\"https://cntk.ai/jup/201/ResNetBlock2.png\"/>"
|
|
],
|
|
"text/plain": [
|
|
"<IPython.core.display.Image object>"
|
|
]
|
|
},
|
|
"execution_count": 26,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"# Figure 7\n",
|
|
"Image(url=\"https://cntk.ai/jup/201/ResNetBlock2.png\")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"The idea of the above block is 2 folds:\n",
|
|
"\n",
|
|
"* During back propagation the gradients have a path that does not affect its magnitude.\n",
|
|
"* The network need to learn residual mapping (delta to x).\n",
|
|
"\n",
|
|
"So let's implements ResNet blocks using CNTK:\n",
|
|
"\n",
|
|
" ResNetNode ResNetNodeInc\n",
|
|
" | |\n",
|
|
" +------+------+ +---------+----------+\n",
|
|
" | | | |\n",
|
|
" V | V V\n",
|
|
" +----------+ | +--------------+ +----------------+\n",
|
|
" | Conv, BN | | | Conv x 2, BN | | SubSample, BN |\n",
|
|
" +----------+ | +--------------+ +----------------+\n",
|
|
" | | | |\n",
|
|
" V | V |\n",
|
|
" +-------+ | +-------+ |\n",
|
|
" | ReLU | | | ReLU | |\n",
|
|
" +-------+ | +-------+ |\n",
|
|
" | | | |\n",
|
|
" V | V |\n",
|
|
" +----------+ | +----------+ |\n",
|
|
" | Conv, BN | | | Conv, BN | |\n",
|
|
" +----------+ | +----------+ |\n",
|
|
" | | | |\n",
|
|
" | +---+ | | +---+ |\n",
|
|
" +--->| + |<---+ +------>+ + +<-------+\n",
|
|
" +---+ +---+\n",
|
|
" | |\n",
|
|
" V V\n",
|
|
" +-------+ +-------+\n",
|
|
" | ReLU | | ReLU |\n",
|
|
" +-------+ +-------+\n",
|
|
" | |\n",
|
|
" V V\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 27,
|
|
"metadata": {
|
|
"collapsed": true
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"def convolution_bn(input, filter_size, num_filters, strides=(1,1), init=C.he_normal(), activation=C.relu):\n",
|
|
" if activation is None:\n",
|
|
" activation = lambda x: x\n",
|
|
" \n",
|
|
" r = C.layers.Convolution(filter_size, \n",
|
|
" num_filters, \n",
|
|
" strides=strides, \n",
|
|
" init=init, \n",
|
|
" activation=None, \n",
|
|
" pad=True, bias=False)(input)\n",
|
|
" r = C.layers.BatchNormalization(map_rank=1)(r)\n",
|
|
" r = activation(r)\n",
|
|
" \n",
|
|
" return r\n",
|
|
"\n",
|
|
"def resnet_basic(input, num_filters):\n",
|
|
" c1 = convolution_bn(input, (3,3), num_filters)\n",
|
|
" c2 = convolution_bn(c1, (3,3), num_filters, activation=None)\n",
|
|
" p = c2 + input\n",
|
|
" return C.relu(p)\n",
|
|
"\n",
|
|
"def resnet_basic_inc(input, num_filters):\n",
|
|
" c1 = convolution_bn(input, (3,3), num_filters, strides=(2,2))\n",
|
|
" c2 = convolution_bn(c1, (3,3), num_filters, activation=None)\n",
|
|
"\n",
|
|
" s = convolution_bn(input, (1,1), num_filters, strides=(2,2), activation=None)\n",
|
|
" \n",
|
|
" p = c2 + s\n",
|
|
" return C.relu(p)\n",
|
|
"\n",
|
|
"def resnet_basic_stack(input, num_filters, num_stack):\n",
|
|
" assert (num_stack > 0)\n",
|
|
" \n",
|
|
" r = input\n",
|
|
" for _ in range(num_stack):\n",
|
|
" r = resnet_basic(r, num_filters)\n",
|
|
" return r"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"Let's write the full model:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 28,
|
|
"metadata": {
|
|
"collapsed": true
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"def create_resnet_model(input, out_dims):\n",
|
|
" conv = convolution_bn(input, (3,3), 16)\n",
|
|
" r1_1 = resnet_basic_stack(conv, 16, 3)\n",
|
|
"\n",
|
|
" r2_1 = resnet_basic_inc(r1_1, 32)\n",
|
|
" r2_2 = resnet_basic_stack(r2_1, 32, 2)\n",
|
|
"\n",
|
|
" r3_1 = resnet_basic_inc(r2_2, 64)\n",
|
|
" r3_2 = resnet_basic_stack(r3_1, 64, 2)\n",
|
|
"\n",
|
|
" # Global average pooling\n",
|
|
" pool = C.layers.AveragePooling(filter_shape=(8,8), strides=(1,1))(r3_2) \n",
|
|
" net = C.layers.Dense(out_dims, init=C.he_normal(), activation=None)(pool)\n",
|
|
" \n",
|
|
" return net"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 29,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Training 272474 parameters in 65 parameter tensors.\n",
|
|
"\n",
|
|
"Learning rate per minibatch: 0.01\n",
|
|
"Momentum per sample: 0.9983550962823424\n",
|
|
"Finished Epoch[1 of 5]: [Training] loss = 1.895607 * 50000, metric = 70.00% * 50000 24.547s (2036.9 samples/s);\n",
|
|
"Finished Epoch[2 of 5]: [Training] loss = 1.594962 * 50000, metric = 59.18% * 50000 21.075s (2372.5 samples/s);\n",
|
|
"Finished Epoch[3 of 5]: [Training] loss = 1.456406 * 50000, metric = 53.31% * 50000 21.631s (2311.5 samples/s);\n",
|
|
"Finished Epoch[4 of 5]: [Training] loss = 1.354717 * 50000, metric = 49.36% * 50000 20.848s (2398.3 samples/s);\n",
|
|
"Finished Epoch[5 of 5]: [Training] loss = 1.275108 * 50000, metric = 45.98% * 50000 21.164s (2362.5 samples/s);\n",
|
|
"\n",
|
|
"Final Results: Minibatch[1-626]: errs = 43.9% * 10000\n",
|
|
"\n"
|
|
]
|
|
},
|
|
{
|
|
"data": {
|
|
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYwAAACfCAYAAADqDO7LAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztnXe4lNW1/z9fkapYCHZs0VgQVPSKDRUxWKKAMTEGjaKJ\n0URjfRIN5v7EknajKbbkCooFry0aFSwxKmAXCyAIFhQFBKlSVepZvz/WO85wmJkzAzNnZg7r8zzz\nnF3Wu9/17nPOu2bvtffaMjOCIAiCoCE2qLQCQRAEQW0QBiMIgiAoiDAYQRAEQUGEwQiCIAgKIgxG\nEARBUBBhMIIgCIKCCIOxniLpH5J+U6yspCMkTSuvdl/f92NJPRrjXk2B5HczvtSya6HHi5LOKEfb\nQWXZsNIKBKVF0ifA1sC2ZvZ5RvkYYB9gJzObamY/L7TNLLJrtXlH0o7Ax8CGZla3Nm00FSR1A57C\n+3IDoA2wBFBS1tHMPi2mTTN7HuhcatkgSBEjjKaH4S/lvqkCSZ2A1qzli76EpF6GKvuNpGblvse6\nYGYvmVlbM9sE2Avvl01TZfWNhRIqomwQJITBaJoMAfpl5PsBd2UKSLpD0jVJ+ghJ0yRdKmmWpOmS\nzswmmy5Sf0lzJE2WdGpGxXckjZa0UNIUSQMyrns++blA0iJJBybX/FTSxKTsHUn7ZlzTRdLbkuZL\nuk9Si2wPLKmfpJck/UXSHGCApAGShmTI7CipTtIGSX6EpGuS6xZJ+rekdjnanyjpOxn5ZpJmS9pX\nUktJQyTNTfQcJWmLbO00wGoGIZnauUbSK/joY3tJP8noq0mSfpIhf5SkjzPy0yRdImlcotf/SWpe\nrGxS31/SZ4nc2Uk/7tDgAzlXSvpE0kxJgyW1TepaJ/dJ9dtrqf5PnvPj5Dk/lPSDtejPoMSEwWia\nvAa0lbR78nI8BbiH/N/stwbaAtsCZwO3SNo0j2y7RPZMYKCkbyV1S4DTzWxT4HjgZ5J6J3WHJz83\nSb5Fj5J0MnAl8KPk23ZvYF7GvU4GjgZ2xqfUzszzDAcCHwJbAb9LyuqPqurn++IGdQugJfDLHG3f\nC5yakT8WmGNmY5PrNwG2w/vlZ8BXefQshh/hz7wJMB2YCRyX9NVPgZuSEWSK+s93MnAU8E3gv4DT\ni5WVdAJwPnAEsBvQI8u1ufgp3m+HA7vg/fO3pO4sfOS7bVJ+HrA0MSh/Bo5KnvNQYFyB9wvKSBiM\npktqlNETeBeY0YD8cuBaM1tlZk/hL/7dc8ga8P/MbIWZvQA8AfwAwMxeMLMJSfod4H78RZNJpuH6\nCfAnMxudXDPZzDKd6jeY2SwzWwAMAzJHH/WZbmZ/N7M6M1vWwPOmuMPMPkrkH8zT/n1Ab0mtknzf\npAxgBfANYDdzxpjZkgLv3xCDzeyD5PeyysyeMLMpAGY2EngOOCzP9X81szlmNh94nPz9l0v2ZOD2\nRI+vgKuL0P9U4PrEb/YFcAVpw7sCaE+630ab2ZdJXR3QWVLL5Pf/XhH3DMpEGIymyz34P+aZwN0F\nyM+r54j+Etg4h+x8M1uakZ+Cf0tE0oGShifTNQuAc/GXQi62Bz7KUz+rQJ0A1mb11sxC2jezj4CJ\nQC9JrfGR0L1J9RDgaeB+SZ9K+qNK50NZ7ZkknZBM3cyTNB//QpCvf4vpv1yy29bTYxqF+6G2xf8+\nUkwBWiZTdncCzwIPJlNdv5e0gZktxg3yL4CZkoZmjGCDChIGo4liZlNx5/dxwL9K3PzmyUszxQ6k\nRzD/BzwKbGdmmwG3kn65ZJvGmIZPVZSC+u1/ga8+SrHNOrZ/P26E+wATzGwygJmtNLNrzWwv4BCg\nF1CqZaVfP1MyuvknPt22hZltDjxD+RcRfAZ0yMjvQOFTUjOAHTPyOwLLkpHMCjO7xsw6At2Ak4DT\nAMzsaTPriU9/foT/HQUVJgxG0+bHQI9kGqGUCLhaUnNJh+G+igeTuo3xEcgKSV1Zfd5/Dj7VkGkg\nbgN+KWk/AEm7SNq+RHqOBQ6XtH3ij/n1OrZ3P+5P+Tnp0QWSukvqlPiLluBTLcUuGy7kpd8SaA7M\nBSzxLRxV5H3WhgeBn0jaTVIb4L+LuPY+4NJkwUFb4LckfSfpSEl7SRIZ/SZp62Qk1RpYiRv+VaV8\noGDtCIPR9Pj6m5+ZfZzyDdSvK6adLHwGzMe/PQ4BzjWzSUndecC1khbiL5YHMvT5Cv92/LKkzyV1\nNbOHkrJ7JS0CHsEdoMXqu+YDmD2b3H8c8AbuA1lNpMj2ZgKvAgeR8Vz4t+CHgIXABGAE3i+pTY9/\nL6T5hsrMbCFwCT6Cm4d/I6//TA21WbSsmT0O/AN4AXgfeCmpyuUnymxrEN5XL+ILEhYCFyd12+Kj\n34XAeOA/uDFpBvwK//uaAxyMO92DCqNyHqAkqSX+R9YC3yT4kJmt4TCTdCM+dfIFcGay8iQIgiok\nWZX1lpm1rLQuQeNS1hFGsvLkSDPrgq+4OC6ZpvgaSccBu5jZt3AH6f+WU6cgCIpH0onJFGQ74I/4\nKCdYzyj7lFTGMrmW+Cij/pCmD8kqHjMbBWwqaaty6xUEQVGcj/tOPsBXUP2isuoElaDssaQSR+Bb\nuKPzFjN7o57Idqy+ZG96UjaLIAiqgmTFUrCeU3aDkazt7yJpE+BRSR3NbGKx7UiqdBykIAiCmsTM\nSrL0utFWSZnZInz1yLH1qqbjm7dSdEjKsrVBnz7GI48YZtX5GTBgQMV1CD1Dz1rVMfQs/aeUlNVg\nSGqfikeUrKnuCdTf4j+UZJOTpIOABWaWczqqVStYujRXbRAEQVAuyj0ltQ1wV+LH2AB4wMyelHQu\nYGY2MMl/R9KH+LLas/I1GAYjCIKgMpTVYJjZeGC/LOW31ssXvOKi2g1G9+7dK61CQYSepaUW9KwF\nHSH0rGbKunGvlEgyM+Oii2DnneHiixu+JgiCYH1HElZrTu9SseWW0DL2lwZBEDQ65Q4N0gHflLcV\nHoxtkJndWE9mEzwU9w54DJk/m9mdWdqyWhkNBUEQVAulHGGU22BsDWxtZmMlbYxv4OtjGYehSOqP\nn8DWX1J7PLjZVma2sl5bYTCCIAiKpGampMxspiWBBM1PIHsX38W9mhh+NCjJz3n1jUUQBEFQecq+\n0zuFpJ3wAISj6lXdDAyVNAM/S+GUxtIpCIIgKJxGcXon01EPARfZmmcdHwOMMbNtgS7ALYl8EARB\nUEU0RvDBDXFjMcTMHssichbwB/BzkyV9DOwBvFlf8KqrrmLFCli+HI4/vvt6uQ46CIIgHyNHjmTk\nyJFlabvs+zAk3Q3MNbNLc9TfAsw2s6uTsOZvAvuY2ef15MzMGDYMbr0VHn+8rGoHQRA0CUrp9C7r\nCEPSofih7uMljcEd3FfgB8GbmQ3Ez/i9U9K45LLL6huLTNq0gS+/zFUbBEEQlItyhwZ5Gd9bkU/m\nM9yPURBhMIIgCCpDze30DoMRBEFQGcJgBEEQBAVRcwajbVto167SWgRBEKx/VDyWVCLXHfgr0ByY\nY2ZHZpGJ0CBBEARF0tRiSW0KvAIcbWbTJbU3s7lZ2gqDEQRBUCRNLZbUqcDDZjY9kVvDWARBEASV\np9F8GHliSe0GtJM0QtIbkk5vLJ2CIAiCwmmU4IMNxJLaED/GtQewEfCqpFfN7MP67Vx11VVfp7t3\nj9AgQRAE9an10CAbAo8DT5nZDVnqLwdamdnVSf62RPbhenJf+zBmzoRNNvEltkEQBEFuasaHkTAY\nmJjNWCQ8BnST1ExSG+BA3NeRk3794IUXSqxlEARBkJeKx5Iys/ckPQ2MA1YBA81sYr5227WDz3NG\nmwqCIAjKQcVjSSVy1wPXF9puGIwgCILGp+Z2ekMYjCAIgkoQBiMIgiAoiLIaDEkdJA2XNEHSeEkX\n5pE9QNIKSSc11G6HDtCqVWl1DYIgCPJT8dAgidwGwDPAV8BgM/tXlrYiNEgQBEGR1Myy2gJDgwBc\ngG/sm11OfYIgCIK1p+KhQSRtC5xoZv8ASmIFgyAIgtJTDaFB/gZcnimeq50IDRIEQZCfph4aZHIq\nCbQHvgDOMbOh9eTChxEEQVAkNXMeBoCku4G5ZnZpAbJ3AMMKcXp/+CFsvz20bFlSdYMgCJoUNeP0\nzggN0kPSGEmjJR0r6VxJ52S5pGDrddJJMDFvAJEgCIKglFRFaJAM+R8XKrvjjjB1KnTpslaqBUEQ\nBEVSkzu9waejpk2rtBZBEATrDzVtMKZMqbQWQRAE6w8VDw0i6VRJbyeflyR1LqTtPfcMH0YQBEFj\nUu59GCuBSzNDg0j6T73QIJOBw81soaRjgUHAQQ01vO++0Lp1eZQOgiAI1qSgZbWSdgE+NbNlkroD\newN3m9mCom4mPQrcZGbP5ajfDBhvZttnqYt9GEEQBEVSiWW1DwOrJO0KDAS2B+4t5ka5QoPU42zg\nqWLaDYIgCBqHQqek6sxspaTv4iOEm5IjVwuigdAgKZkjgbOAbrnaidAgQRAE+al4aBBJo/CYT78B\nepnZx5LeMbNOBVybNzRIIrM3Poo51sw+yiETU1JBEARFUokpqbOAg4HfJcZiZ2BIgdcOBibmMRY7\n4Mbi9FzGoiFWrFibq4IgCIJiKDqWlKTNge3NbFwBsocCLwDj8bAfBlwB7AiYmQ2UNAg4CZiCByBc\nYWZds7SVdYTx73/DccfBypXQrOA95UEQBOsHjR58UNJIoDfu83gLP+jo5UICCpaKXAbDDDp2hEMO\ngdtvbyxtgiAIaoNKTEltamaL8JHA3WZ2IPDtUiiwrkjw4IMweDA8/3yltQmCIGi6FGowNpS0DfAD\n3IFdVXTuDO3aQc+eUFdXaW2CIAiaJoUajGuAp4GPzOwNSd8EJpVPreKZNw/eew82qNnoWEEQBNVN\nWQ9QktQBuBvYCqgDBpnZjVnkbgSOw0/bO9PMxmaRKXpZ7YIFsNFG0Lz52mgfBEFQ+zS6DyMJIviI\npNnJ5+HEGDREKpbUXviy3PMl7VGv7eOAXczsW8C5wP8W+QxZOeQQ2HxzaNECHnusFC0GQRCs3xS6\n0/sOPBTIyUn+R0lZz3wXmdlMYGaSXiLpXWA7IDP4YB98FIKZjZK0qaStzGxWwU+Rhe9+1w9Z+sY3\nYLPN1qWlIAiCAAo3GFuY2R0Z+TslXVzMjfLEktoOyDwKaXpStk4G41e/WpergyAIgvoUajDmSfoR\ncF+S7wvMK/QmhcSSKoSIJRUEQZCfaogltSNwE+6HMOAV4AIza/CQ1IZiSUn6X2CEmT2Q5N8Djqg/\nJVWqWFKff+6O8C+/dB9HEARBU6bRnd5mNsXMepvZFma2pZmdCHyvwHvkjSUFDAXOAJB0ELBgXf0X\nuRg50n0arVrBH/5QjjsEQRA0XdZl10KDYUGSWFKnAT0kjZE0WtKxks6VdA6AmT0JfCzpQ+BW4Lx1\n0CkvI0b4WeAA++/vP6+5BmbOLNcdgyAImg5rvQ9D0rRsJ+OVi1KGN1+1Kh2o8PDD4cUX4a23YL/9\nStJ8EARB1VCJWFLZqNnDKTKj2l52mf/cf3+YM8fTU6bArLJMigVBENQueQ2GpMWSFmX5LAa2bSQd\ny8oJJ3jE2xNPhOXLveyaa+D++yurVxAEQbVR7tAgtwMnALPMbO8s9ZsA9wA7AM2AP5vZnTnaarQT\n94YPh6OO8thUu+/eKLcMgiAoC9UyJVUIdwDH5Kk/H5hgZvsCRwJ/TpbhVpQePTy0yB57wJiCTy4P\ngiBo2pTVYJjZS8D8fCJA2yTdFphnZivLqVOhPP007LNP2q8Bq6eDIAjWNyodDPxmoKOkGcDbwEUV\n1udrNt4Yxo6Fo4/2/AcfwJZbwq23wpJ6e9Vnzkw7yefOXbM+CIKgKVDp6Z9jgDFm1kPSLsAzkvbO\nFT6kkqFBdtvNp6l+9jP/mMFtt8FPf+r1vXrBww/DFlt4fv78CHoYBEHjU/HQIOt0Aw8rMiyH0/tx\n4A9m9nKSfw643MzezCLbaE7vXNTVwbBhPlW1007wxz9C//5et3KlH970wAPQty/85S9wySUVVTcI\ngqCkTu/GMBg74Qajc5a6W4DZZna1pK2AN4F9zOzzLLIVNxiF8ve/wz33wCuvFHfdK6/AgQeuvk8k\nCIJgXagZgyHpXqA78A08XPkAoAVgZjYwOSf8TmCb5JI/mNl9WZqqKYPx5ZcweTJ06pS9fsYM3+sx\neTL8z/9Aly7uL+nSxfeDPPJI4+obBEHTpWYMRimpJYORyYoVHoqkVat02csvQ7du6fzSpXDvvfDj\nH3t+0iTYddfG1TMIgqZJLe3DWO858URo3Rpeey19qNOhh7oReewx+O//hpYt4ayz3HBccgm0aeOj\nlL59XfaGjDi/NWgzgyBoIsQIo8y8+qqvrgI4/nh4/PHCrlu1CjbMWMNWVwfXXgsDBnhayfcFM7jg\nAjdGG2/s4dsBLr4YLrwQvvlNz69cCe+/D3vtVZrnCoKgNqiZKamGQoMkMt2BvwLNgTlmdmQOuZo0\nGOAjh+eeg2OPLc6hbQaLFvmIo3lz3++xzTarr8DKNEipa15/3Z3nW28NEyf6CGajjbz+kkv8+qVL\noUULX9kVBEHTpZampPKGBpG0KXALcIKZdQJOLrM+FaFVKx9dFLv6SYJNN3VjAW4A7rwTLr0UzktO\nDTn4YDcqe+4Js2d72e67w0UXuYE55RQ3OIcc4gYiNcJp3Rq+V+8IrGefhaee8vRRR6VHMUEQBFD5\nfRg/B7YxsysLaKdmRxil5tprYd99fbNgPubOhXbt0qOIVav806JF2hgcdhg8/7znU2Vz58Ibb8Bx\nx8HixT7VBb5hsUsXOPfc8jxXEASlp2ampKBBg5GaitoL2Bi40cyG5GgnDEaJkaBjR3jnHRg1ykcr\nAAsW+MjmiCP83PNHH/WlwNtt5/V//av7SHJRVxdTXUFQLZTSYFQ6NMiGwH5AD2Aj4FVJr5rZh9mE\nKxkapCmSaX+/9S2f7urXL132t7/5KYSvvw7bJqefPPOMG5lc3H47nH02jB7tU2S77+5xuHr2jCmu\nIGgMmnJokMuBVmZ2dZK/DXjKzB7OIhsjjArw0ktw0EGrr9hKsXQpXHcdXHmljyoWLUrHz8o8Bhd8\nmfDs2R5SZerU9NnqQRCUl1pyegMo+WTjMaCbpGaS2gAHAu82gk5BgXTrlt1YrFrljvMrr/SpK7O0\nsVi+3KekPv3U/R9vv+2yLVp4/Y9+5MYGfI/Jaae5EQF31u+zz+r3WrnSDVIQBJWloqFBEplfAmcB\nq4BBZnZTjrZihFFlfPKJT2XNnes+j8mTvTy19yMbgwbBOef4dR98sPo0lZlvcDz4YBg/Ph1apXt3\nd8yb+cqvSy6Bu+5KG6AgCHJTU07vUhEGo+nwwAO+wfDb3/ZprNGjfenwoYd6/R57+CZDM/jiCx+l\n7Labl116qTvdwY3LgQcWds/UFNncuekQ9B99lN+45WP6dF8undooGQTVSq1NSQXBapxyihsLgE02\n8RFEyliAx9UCX5k1caKnx43zn127wqmn+tLe1Kqt007zkUr79rBsmcfhuvnmdHtLl/q0WpcuPhJK\n8ac/Zdevf//0MuNly7zsL3+B//zH02a+h+WWW9a6C4KgJqn0KqkgWIP99vOXfMuW8OST8P3vexrg\nhz/0Tyb3JfGNe/XykcpWW3m4lGXLfINjmzZef/TRXp85UD3qKB8tXH01HHOM+2FGjUrXf/AB7J2x\nXMPMRxbLl7tu9Vm82P0x9UOwTJ0KHTrEcuOgxjGzsn2A23HfxbgG5A4AVgAn5ZGxIMjF0qXpdF2d\nmb/azZYsMTvjDLPXX89+3csvp2V/+1svu+sus4ULPT13rtcdcojZ8uVmq1Z5/uc/z97e2LFmu+yy\nZnn37maTJzf8HJ9/ntY7CEpB8u4syTu93E7vbsAS4G7LHUtqA+AZ4CtgsJn9K4eclVPXoGmxapWv\nrEqFVcnHhx/6aOGww3zaKh9z5vhU1nXXeX7ZsnTo+u9/30c7zZvDP//pefClyYcd5v6Y1GgnF+ee\nCwMHehTja69tWPcgaIiacnrn24eR1F8ELMdHGY+HwQhqjbZtYckSuOwyPxCrUyeYMMGXAy9bBgcc\n4NNkw4f7dNeMGb4UuV07v/6JJzwMywYbeDDJHj18Si7Xn/t118F3vuPTXqnIxbEpMshFk3F6S9oW\nONHM/kHuvRpBUNUsXgxvveVnvEPaQf+vf3mU4IkT0/6O738fTjrJV1etXOnLhU84wQM/QjqY5Isv\npg3GZ5/56i6A225zw9SpkxuLZs3c0Pzyl433vMH6S6VdcH8DLs/Ih9EIapL99kt/y99gA5gyBfr0\ngXffhREj0tNWU6f60mDwkxdTzvyHM2IbNG/uGyYluP56D8uyxRbw1Ve+uuy733W5RYvS16Q2Oy5f\n7mfKH3ywG5xHH4U33yzfc5eDoUN9yXNQfVQ6NMjkVBJoD3wBnGNmQ7PI2oABA77ORyypoJaZPduN\ngORTVUcdlX0KavvtPR7XhRdC797p8lSAx1mzfLoL/PrUKqyWLX2aLOXDMfMgk+ee64aqUO65x/fF\njBzpS5NTQSd794Zhw1Y/zGtdST3T2Wd7TLKU3ilGjfIQ/zvuWJr7NVXqx5K6+uqrSzYlVdZVUokx\n2gkYX4DcHcQqqWA9JLXy6ssv16ybMsXrC2XQILNTTjH77DPP33abt/3tb5vdeaenP/7Y6+bNMzvv\nPLO99jJ7+GEv++ors0WLPN2nj8u/8IJZp06e/uQTs/ffT68s69/fbOJETw8fvvoKta++KlzvN97w\naz74wGzwYE/fcEO6fuFCLzv/fLNly8xWriy87fUdSrhKqtzG4l5gBrAMmIqHADkXH0XUlx0cBiMI\nSs+IEf4yNzPr0cP/6997z+ztt9Mv9zPPNHvkEbOdd/Z85ot/4cL0cl8wmznT7P77fUnyjBnpcnDj\ntvXWnu7dO63Dm2+arVjh6U8+MdtzT7PttjN79VWzJ59MX19X522MGpW+9rzz0vVffGHWurXZRRet\n/oyTJ/t1N99s1r692fTpZe3SmqJmDEYpP2EwgmDdmT/fbO+9/Vu6mb+8Z870dK9e/kbYbTfPP/WU\nG4oUt9/uRqY+Q4aYPf202TnneH75crN77zWbNcvzv/3t6sbn8MPT+RkzzH7/e0/nesm/9ppZu3Y+\nIjIze/zxtHGZNi3d1vDhqxueujqz++5LG6pqpq6ufG2HwQiCoOTMmGHWtavZ7NmlbXfXXc223NLs\nd7/z/LRpZs8+azZp0tq1V1dnttFG6Sk2MPvFL3wU07Gj2dSpLvfcc+n61MbOyy5z45MymGZmzzzj\nhrRQ5s/3zZ2LFxen9zvvuDHN9jxgdsUVxbVXKGEwgiBYr+nc2d9eV13lPpVsrFxp9s9/utyjj5pN\nmJA2IGec4TKLFnk+5cNJMW2aG5fu3T2fmiozM3v++XQ7ZmbXXWdfj5ZysWCBWYsWHjmgb18fkZl5\nm/36+fVPPJH92qVLzXr2NBs92uzXv/aRXjGEwQiCYL1m9uy0c74hMr/VL17sYWJOO83zmb6XTPlU\necpXksrvvbcbj8ce8/yECR42JuWQTzFr1urTTKnrx4xJp++7zxc6gNm112bX/dZb0/KLF5tttpmn\n77/f6x94wH1P+agZg0EDsaSAU4G3k89LQOc8beXvlSphxIgRlVahIELP0lILetaCjmaNp+fKlWZb\nbOGOcjP3xaxY4S/mzp3NfvaztGzqpZ1pFI48coTdeKOnjzkmbXguu8zTvXp53QcfeP7NN92IXHCB\nLy5IrVabMyfd5hdfrJ5PxTp78EHPz5rl+YceMrvxRk8ffnj+5yylwSj3xr07gGPy1E8GDjezfYDf\nAoPKrE/ZKddZuqUm9CwttaBnLegIjadns2a+H+b88z2Ey+mn+76V1q19t/4//pGWnT7d94lkhs0/\n/PCRXHCBp594Au680/ek7Lablw0b5me/zJzp+f339/obb/TDxnbaycvbt0+3effdHl4/xSGHuKk6\n+WTPb7klTJrk4fUHJW/LTJ3KTVkNhpm9BMzPU/+amS1Msq8B25VTnyAIgmy0bJneGLl8+Zr1226b\nf4Nis2bQr5/L/OQn/pKfMMFf9N26eeDJQujY0cO/SDBvXnaZXXf1n+PGua6dOxfWdimopvMwzgae\nqrQSQRCsnzzyiMf0at26NO117JhONxSlOMUBB6TTm2/esHwh0ZhLScWj1SYyRwI3A93MLOuIRFKE\nqg2CIFgLrEShQSo+wpC0NzAQODaXsYDSPXAQBEGwdjRGtFqRIwqtpB2Ah4HTzSziUwZBEFQx5T5x\n716gO/ANfHntAKAFvsxroKRBwEnAFNyorDCzrmVTKAiCIFhryu7DCIIgCJoGlT5AqSAkHSvpPUkf\nSLq84SvKqssnkt6WNEbS60nZ5pL+I+l9SU9L2jRDvr+kSZLelXR0GfW6XdIsSeMyyorWS9J+ksYl\nff23RtJzgKRPJY1OPsdWgZ4dJA2XNEHSeEkXJuVV06dZdLwgKa+q/pTUUtKo5H9mvKQBSXnV9GUD\nelZVf2bcY4NEn6FJvvz9WaodgOX64EbtQ2BHoDkwFtijgvpMBjavV/Y/wGVJ+nLgj0m6IzAGX1yw\nU/IcKpNe3YB9ydhVvzZ6AaOAA5L0k8AxjaDnAODSLLJ7VlDPrYF9k/TGwPvAHtXUp3l0rMb+bJP8\nbIbvuepaTX3ZgJ5V159Ju5cA9wBDk3zZ+7MWRhhdgUlmNsXMVgD3A30qqI9Yc2TWB7grSd8FnJik\newP3m9lKM/sEmIQ/T8mx7Jski9JL0tZAWzN7I5G7O+OacuoJ2RdG9KmgnjPNbGySXgK8C3Sgivo0\nh46pza/V1p9fJsmW+IvLqKK+bEBPqLL+lNQB+A5wWz19ytqftWAwtgOmZeQ/pbI7wg14RtIbks5O\nyrYys1ng/8TAlkl5fd2n07i6b1mkXtvh/ZuiMfv6F5LGSrotYyhdFXpK2gkfFb1G8b/rRtE1Q8dR\nSVFV9WcyfTIGmAk8k7ykqq4vc+gJVdafwF+BX5E2aNAI/VkLBqPaONTM9sOt+/mSDmP1XxpZ8tVC\nter1d+BgZ1FSAAAFRklEQVSbZrYv/o/65wrr8zWSNgYeAi5KvsVX3e86i45V159mVmdmXfBRWldJ\ne1GFfZlFz45UWX9KOh6YlYwu8+1PK3l/1oLBmA7skJHvkJRVBDP7LPk5B3gUn2KaJWkrgGSYNzsR\nnw5sn3F5Y+terF4V0dfM5lgyiYoHoExN21VUT0kb4i/iIWb2WFJcVX2aTcdq7c9Et0XASOBYqqwv\nc+lZhf15KNBb0mTgPqCHpCHAzHL3Zy0YjDeAXSXtKKkF8ENgaCUUkdQm+TaHpI2Ao4HxiT5nJmL9\ngNTLZSjwQ0ktJO0M7Aq8Xk4VWf0bR1F6JcPYhZK6ShJwRsY1ZdMz+eNOcRLwTpXoORiYaGY3ZJRV\nW5+uoWO19aek9qlpHEmtgZ64v6Wq+jKHnu9VW3+a2RVmtoOZfRN/Hw43s9OBYZS7P0vtuS/HB/82\n8j7urPl1BfXYGV+lNQY3FL9OytsBzyY6/gfYLOOa/viqhHeBo8uo273ADGAZMBU4C9i8WL2A/ZNn\nmwTc0Eh63g2MS/r2UXwuttJ6Hgqsyvh9j07+Dov+XZdL1zw6VlV/Ap0T3cYmev1mbf9vKqRnVfVn\nPZ2PIL1Kquz9GRv3giAIgoKohSmpIAiCoAoIgxEEQRAURBiMIAiCoCDCYARBEAQFEQYjCIIgKIgw\nGEEQBEFBhMEIqg5JdZLuzsg3kzQnI4xzL0mXNdDGNpIeTNL9JN1UpA79C5C5Q9JJxbRbSiSNkLRf\npe4frH+EwQiqkS+ATpJaJvmeZARPM7NhZvanfA2Y2Wdm9oPMoiJ1uKJI+ZpCUrNK6xDUHmEwgmrl\nSeD4JN0Xj5kDrD5iSL7l3yDpZUkfpr7xJ6Fkxme0t0Pyjfx9SVdmtPVIEnl4fCr6sKQ/AK3lh9MM\nScrOUPrgrLsy2j2i/r0zSfSYKGmgpHck/TtlCDNHCJK+IenjjOd7RH4YzmRJ50u6JNHnFUmbZdzi\njESncZIOSK5vIz+o6jVJb0nqldHuY5Kew3cEB0FRhMEIqhHDzz3pm7xc9yYdtjtTJsXWZnYo0As/\nRCabzAHAd4F9gJMzpnLOMrMDkvqLJG1uZv2BL81sPzM7XR6x9Aqgu3kk04sKuHcmuwI3mVknYCHw\nvTzPnWIv/GyCrsDvgCXmUZJfw2P+pGid6HQ+HlcK4DfAc2Z2ENADuD6JjQTQBTjJzI7MoUMQ5CQM\nRlCVmNk7+OlgfYEnyB/G+dHkmndJnwFQn2fMbIGZLQX+hZ/8B3CxpLH4i7gD8K2kPPN+PYB/mtn8\n5D4Lirz3x2aWGu28lTxXQ4wwsy/NbC6wAHg8KR9f7/r7kvu/CLSVtAkeFPPX8nMdRgItSEd8fsbM\nFhZw/yBYgw0rrUAQ5GEocB3QHWifR25ZRjqXYVnj7AVJR+DG4EAzWyZpBNCqSB0LuXemzKqMe6wk\n/aWt/n0zr7GMfB2r/99mO1NCwPfMbFJmhaSDcP9QEKwVMcIIqpHUi3cwcLWZTViLa+vTU9JmydTM\nicDLwKbA/MRY7AEclCG/PMMxPByfxmoHIGnzIu+dq/wT4L+S9Mk5ZBrilESnbsBCM1sMPA1c+PXN\npX3Xsu0gWI0wGEE1YgBmNt3Mbi5ENk8+xev4VNRYfHppNPBvoLmkCcDvgVcz5AcC4yUNMbOJSf3z\nyTRP6sS1Qu+dq/x64OeS3sJDU+ciX7tLJY3GT4X7cVJ+Lf5c4yS9A1yTp+0gKJgIbx4EQRAURIww\ngiAIgoIIgxEEQRAURBiMIAiCoCDCYARBEAQFEQYjCIIgKIgwGEEQBEFBhMEIgiAICuL/A3DgAqGi\nDRpVAAAAAElFTkSuQmCC\n",
|
|
"text/plain": [
|
|
"<matplotlib.figure.Figure at 0x2ec0f8674e0>"
|
|
]
|
|
},
|
|
"metadata": {},
|
|
"output_type": "display_data"
|
|
},
|
|
{
|
|
"data": {
|
|
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYwAAACfCAYAAADqDO7LAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztnXe4FdXVh98fCiKiYO+9i7GLGpOI5LPEEqPG2BJrbDGx\npKhoEvQzRjTG8lmiRCyxEWKLaCyJiC0qShGUIkoTRERUBBG9wPr+WHM4c88959y5l3PPPRfW+zzz\nzMyePXuv2ffcWbP3XmttmRlBEARB0BjtWluAIAiCoG0QCiMIgiDIRCiMIAiCIBOhMIIgCIJMhMII\ngiAIMhEKIwiCIMhEKIwaQ9JfJF3S1LyS9pH0fstKt7jeiZJ6VqOutoakRZI2q/a9S0q67qb8BouU\nM0fSJpWULagdQmFUCUmTJM2XtFpB+vDkn3UjADM7y8yuyFJmkbzNcqqRtHEiQ/weWOL2WBLHppL3\nShos6UtJn0v6SNJDktZegrpK1p31NyjpOUmn1CvEbGUzm1RBuXJ1TZI0L3n+Ocn+/ypdT1CeeEFU\nDwMmAsfmEiRtD6zIkr1kKoESGdTiFUnLtXQdFaK5f5MlacNy9xrwMzNbBdgK6ApcV7SQ5im6Fv/b\nLyEGHGxmqyRKaRUzO6dYxmK/sab+7trQ77SqhMKoLvcAJ6bOTwTuTmeQdKek/02O95H0vqRfSpoh\naZqkk4rlzSepl6SZkiZIOi514SBJwyTNljRZUu/Ufc8n+8+SL7c9kntOkzQ6SXtL0k6pe3aW9Kak\nTyU9IKlDsQeWdKKklyRdK2km0FtSb0n3pPLU+6JPvlz/N7nvc0lPFfbMUveOlnRQ6ny55At8J0kr\nSLpH0seJnK9JWrNYOVmRtLuk/yblTZN0o6TlC7IdLOm9RI6rC+4/JZF5lqQncz3LrNUDmNlnwEPA\n9kmZd0q6RdITkuYAPSR1kHRN8reenlxfISXHbyR9IGmqpJNJKcjC35Wkw5Ke8GxJ4yXtL+kPwLeB\nm9Jf+6o/tLWKpL8l7TBRqWGu5HfxoqQ/Sfokaa8Dszx/g8T6v7GP8d9YsTRJ+q28t/KhpLskrZKU\nkfsNniJpMvBs1j/KskQojOryKrCypK2Tl+PRwL2U/7pbB1gZWA/4KXCzpC5l8q6W5D0J6Ctpy+Ta\nXOAnZtYFOBg4U9L3k2vfSfarJF9ur0k6Cvg98OPkq/b7wKxUXUcB+wObAjsm9ZViD+BdYG0gN9RR\n+AVfeH4srlDXBFYAfl2i7PuB41LnBwIzzWxEcv8qwPp4u5wJfFlGziwsBM5LytsL6An8rCDPD4Bd\nku0wJcM2kg4DLkqurwm8CDzQVAEkrQEcCQxLJR8LXG5mKwMvA1cBWwA7JPv18b8nyYv5l8B3gS2B\n/ylTV3f8o+ZXyW/nO8AkM/ttIv/PC77203/Hm/Df7iZAD+CERDnl6A6MAVYH/gT0a0o7FJD7ja1F\n/jdWmHYycAKwD7BZIttNBeV8B9gGOGAJZFlqCYVRfXK9jP3wf5YPGsn/Nf4iWGhmT+Iv/q1L5DXg\nd2ZWZ2YvAE8APwIwsxfM7O3k+C2gP/6PkyatuE4FrjazYck9E8wsPal+g5nNSL52BwLp3kch08zs\nFjNbZGZfNfK8Oe40s/eS/APKlP8A8H1JHZPzY8m/hOvwl9FW5gw3s7kZ6y+KmQ0zsyFJeVOAvjRs\nxz5mNtvMpgLXkx+GPAO40szeMbNFQB9gJ0kbZqz+RkmfAMPx382vUtf+aWavJjJ+BZwGnJ/I8UVS\nV06Oo/D2HWNmXwKXlqnzFKCfmQ1Kyp5uZu+UyS9YPCx2NHCRmc0zs8nAn4GfpPJONrM7zAPa3Q2s\nI2mtMmU/mvRGPk32p6auFfuNFaYdB1xrZpPNbB7QCzhG+SE8A3qb2ZdN+J0uUxR2pYOW517gBfzL\n/G8Z8s9KXi455gGdS+T91Mzmp84n470N5MNMV+LDGB2S7R9l6t0QeK/M9RkFMq1bJm9zrLc+LCi/\n6DOb2XuSRgOHSnoc7wn9Prl8D7AB0D/pld0LXGJmC5shDwBJj+1aYDd8/ml5YGhBtqmp48V/A2Bj\n4AZJf84Vh7+k1idbG/3CzO4ocW3x/cmwWydgqLT4G6Ad+Q+C9YA3CmQs1cvdEP/waCpr4G0zpaCe\n9VPni//GZvalXNjOwEclyjzMzJ4rca1Y+xWmrZfIkJZnebznm2MqQUmih1Flkq/SicD3gIcrXPyq\nklZMnW9EvgdzH/AosL6ZdQVuI/+SKDbB+z6weYXkKiz/C/yFlqOcsslCf/zr8TDgbTObAGBmC8zs\ncjPrBnwTOBQfklgS/oL3DDdP2vESGr5s0z2Gjcn/Dd4HzjCz1ZJtVTPrnOsZLCHpNv4YV7LdUnV1\nTYaUAKYXkbHUJH+530E5w4CP8R7exgX1TCtzT2M0ZhTQWNoHReSpo/7HT2sboNQ0oTBah1OAnslw\nQCURcJmk9pK+jc9VDEiudcZ7IHXJuHR63H8msIj6L4bbgV9L2gVA0uZNGDppjBHAdyRtmHz5X7SE\n5fXH51POwuc0AJDUQ9L2yZDDXPzlsKh4EQ0Q0DGZOM9twse9PzezeZK2Seos5DeSuibtdU4iH8Ct\nwMWStkvk6yLph01/3PIkQzx/Ba5PehtIWl/S/kmWAcBJkraV1Il8j6wY/YCTJe2bTBqvJyk3JDoD\nnwsoJsOipJ4rJHWWtDFwPt7ray0eAM6XtImkzvi8Rv9UD77WLcVanVAY1SNt5z4xNzdQeK0p5RRh\nOvAp/iV1D/41Oz659jPgckmzgd8Cf0/J8yX+z/NyMjbc3cweTNLul/Q58Ag+0dtUeRs+gNl/kvpH\nAq/jcyD1sjSxvA+BV4A9ST0XbgTwIDAbeBt4juSFJXdOu6VcscAc/Ev9y2S/Lz5vcHzSJreRVwbp\n+/6JD1MNS57tjkTOR/G5hP6SPsOf/8CCe8vJ05RrF+ITvq8mdT2Dm+NiZk/hcyuDgHcoYxFkZq/j\nk8XX4+04GO+5AtwAHCW3+Lq+iCzn4O02AR+GvdfM7mzic6QZmFhk5baHGslfyB343/8FfLh1XiJj\n1vqXedTSCyglFhnX48qpn5ldVXC9K/6H3Bz/xzzFzEa3qFBBEARBk2nRHkYyFHATbqLWDTg26can\nuRgYbmY74tZD4b0ZBEFQg7T0kFR3YHxixlaHd98PK8izHd41xszGAZtoCZ2rgiAIgspTVmHIvWZL\nmbFlodBccCr1zeoA3gSOSOrrjo+PbrAEdQZBEAQtQFk/DDNbmLjLdzGz2S0kQx/cNn0YMAp3Smpg\nJy8pJqSCIAiagZlVxAIsy5DUXGCUpH6S/i+3ZSx/GnmLCvCeQz07bDObY2anmNkuZnYi7sY/oVhh\nZlbzW+/evVtdhpAz5GyrMoacld8qSRZP74dpvoPZ68AWiQ32dOAYUtFawW3RgXnm/gGnAc/bEoZv\nCIIgCCpPowrDzO6WRyLdKkkaZz6B3SjmQ1o/x23Ac2a1YySd4ZetL7AtcLekRbit/KmlSwQuuwzW\nXRdOPz2LCEEQBEGFaFRhSOqBBwabhHtCbijpRPPgdo1i7iS0dUHabanjVwuvl6VLFxg1KnP2atOj\nR4/WFiETIWdlaQtytgUZIeSsZRp13JM0FDjO3OQVSVsBD5jZrlWQLy2HmRk8+ij06wcDC52DgyAI\ngkIkYVWc9G6fUxYA5qGN21ei8max0Ubw+OOwsNkBR4MgCIJmkEVhvCHp9iSQWw9Jf6V+aOTqsk3i\nKD46oocEQRBUkyxWUmcBZ5MP0vUiUC5oW8vSqRP06QPLx1IeQRAE1aTsHIZ8IfS/mdnx1ROppCxW\naZviIAiCpZ2qzWGYr0y2cWJWGwRBECzDZBnXmYCvk/AYvlIaAGZ2bYtJFQRBENQcWRTGe8nWDl9t\nLAiCIFgGKaswkjmMlc3s11WSJwiCIKhRssxh7F0lWZrGU0+F814QBEEVyeLp/Rd8DYt/UH8Oo7kB\nCZtFAyupww93r++vv4b2redHGARBUMtU29O7IzAL6AkcmmyHZK1A0oGSxkp6R9KFRa6vIukxSSMk\njZJ0UqaCt07CT916a1ZRgiAIgiWg0R7GEhXua3q/A3wX+AAPd36MmY1N5ekFrGJmvSStAYwD1jaz\nBQVl1e9hmMGAAXD00S0mfxAEQVunKj0MSQNSx1cVXHsmY/lZ1vQ28tZXKwOzCpVFCQFDWQRBEFSR\nckNSW6aO9yu4tmbG8rOs6X0TsJ2kD/D1vc/NWLbzyiuuPN59t0m3BUEQBE2jnMIoN1ZVyXGsA4Dh\nZrYesDNws6TOme/OTXhvuWX5fEEQBMESUc4Po5OknXGlsmJyrGRbMWP5ja7pDZwMXAlgZu9Jmghs\nQ5GIuJdeeuni4x49evgCJrvtBtdeC6uvnlGkIAiCpZfBgwczePDgFim75KS3pOfK3Whm+zZauDv+\njcMnvacDQ4BjzWxMKs/NwEdmdpmktXFFsaOZfVJQVuPBB818eCoIgiAAKjvp3aJWUuBmtcAN5Nf0\n7pNe01vSusBdwLrJLVea2QNFyimvMMaPh622cqURBEEQAG1MYVSKRhWGGbRrB3PmQOfsUyBBEARL\nM9V23GsbSPCNb8Brr7W2JEEQBEslS4/CADjlFLj8cj8eN879NP76Vze9DYIgCJaITENSktYHNiZl\nVWVmL7SgXMVkaHzSu64OVlsNxo6FDTbwtHPP9eNfR8DdIAiWPSo5JNXoehiJl/fRwGhgYZJsQFUV\nRibat4crr4R33vHzYcO8d/HMM6EwgiAIlpAs0WrHATuY2VfVEamkHNnX9P7gA/j3v+HEE+E//4H9\n9ouotkEQLJNU1UpK0pPAUWY2txIVNpcmKYxCVl8dBg/2SfEgCIJliKoOSQHzgBGSngUW9zLM7JxK\nCFAVrr0WVszqnB4EQRAUI0sP48Ri6WZ2d4tIVFqO5vcwgiAIllGq7rgnqQOwVXI6LglVXlUqrjAi\njEgQBMsAVXXck9QDGA/cDNwCvCPpO5WovOqMHQtPPgmbb+5e4XUFeu/zzyO0SBAEQQmyDEkNBY4z\ns3HJ+VbAA2a2axXkS8uxZD2M++6DH/+4ftozz8DMmXDccfnQIi+9BHvvvWTCBkEQ1AjVnvRun1MW\nAGb2jqS2Z5+6bhLbsG9fOP106NMH9t/f00aMgB49/HivvVpFvCAIglonSw/jDmARcG+SdDywnJmd\nkqkCj1Z7PflotYXLvf46KdOA9sC2wBpm9llBviXrYSxYAM8/D9/9bj6trg46dIBOnWDePE+75hp3\n8ps1y73GgyAI2jDVDj54Fu7lfU6yjU7SGkVSO3wJ1gOAbsCxkrZJ5zGza8xsZzPbBegFDC5UFhVh\n+eXrKwtwR765c93Rr1Mnn8PIKYnnn6+4CEEQBG2ZRoekEg/va5OtqXQHxpvZZABJ/YHDgLEl8h8L\nNFgLo0VZaSXfz5oFHTvCySfDG2/A1KlVFSMIgqDWKdnDkDQg2Y+SNLJwy1j++sD7qfOpSVqx+lYE\nDgQeylh2ZenYMX88ZQqc03b8EoMgCKpBuR7Gucn+kGoIAhwKvFRuOKromt4twV13+VBVEARBG6NV\n1vRenEG6yswubCytxL17Apea2YHJ+UX40qxXFcn7MDDAzPqXKKv6nt4zZ/qcxnLLNbz22WcwezZs\nvHF1ZQqCIGgC1Q4+OCyZkE6njTSzHRotXFoOGAd8F5gODAGONbMxBfm6ABOADczsyxJlVVdhLFrk\niuLSS+HII2HSJA9emFMQOS/xcPQLgqCGqYqVlKSzJI0CtimYv5gIjMpSuJktBH4OPAO8DfQ3szGS\nzpB0eirrD4CnSymLVqFfP9/PmOGK4tBDffW+CRMaKom6Og+fHgRBsBRTsoeRfPWvClwJXJS6NMfM\nPqmCbIXyVLeHsXAh/OUv0KsXjBoFm24KQ4fCromD+9ChrijWXtt9N/bdF444Iu8xHgRBUANUe0hq\nT+BtM5uTnK8CbGtmr1VCgKy0yhzGvHnwr3/BD38I8+fDCiu4MmjfPt+jyA1NDRoETz8NV12VvzdC\nqgdB0MpU23HvL0DaZGhukrb006mTKwtws1sJLrgAbrkln+fMM32/xx6w225+vMUWoSyCIFjqyNLD\nGGFmOxWkZZr0riQ1ux7G/Pm+bvi++/r59OmwzjpNC51eVxfLxwZB0CJUu4cxQdI5kton27m4RVMA\n3vPIKQvwIIcSXH+9z4E0hpnPg3z+Obzwgs+dBEEQ1CBZFMaZwDeBabin9h7A6WXvCNwk99lnG893\n1lnuzzF5MuyzDzzySMvLFgRB0AyyxJL6CDimCrIsXay5JjxUIsrJ++97DKuVVoLbbvO0++/3/csv\n5+dNgiAIaohyZrUXmNnVkm7EQ4/Xw8yqGmypZucwSpEzr/3kE1h1VU+7/HKf8xg0CF59Ff7+dx+C\n6tjR5z3++le4447i5c2bB8cfDw8/HEvLBkGQmaqY1Uo61MwGSjqx2HUzu7sSAmSlzSkMgJ139h5E\n9+5+nnvR3347/PSnflz4TIsWwZAhsOee+bSpU2H99V3xTJjQcJ2OM8+Eww+HAw5omecIgqDNUlU/\njFqhTSqMV1+FbbaBrl39PKcwFi709TYWLWq4RkeuZ3LGGXDrrfn71lnHFcaAAbD99vn8Cxf6Wh+n\nneYT7Z06tfxzBUHQZqhWD2MgRYaicpjZ9yshQFbapMIoZPZs33fpUj5fTrFMmeI9i1zww3328Uny\no4/O5739djj7bM83cWK+x2LmTocHHRRDWEGwDFMts9prgD8DE4Evgb8m21zgvUpUvszw/PNw332u\nKBpTFuC+HAAbbeTK4uKLfSJ83XVh2rT6eU87zb3O997bzx95BDbZBP70JzjkEPjwQw9dIrm/R5pS\n8a+uv95DoRRj4cL8cFoQBMsUWRz33jCz3RpLK3N/2TW9kzw9gOvwNb1nmtm+RfK0zR7G++/7ix+a\nHtm2MCLueefBxx/7vEjHjjBsmK8U2L07rLyyr09+552+amCanj19ov2hhzzeFcBvfuOKZPLkvHw5\n1l3XFU2hvJ9+6pP4W2zhfiMrr9y05wmCoOpU23FvJUmbpSrfFFgpS+FZ1vROghzeDBxiZtsDR2WU\nvW2w4Ya+P70Zritm9Rdy+vWvYfXV4fzzYeBAVxQHHuiT4O3be/5NNoGttoIHH4THHvOex/rJIoeH\nHupKSMo7CN50U8N6c46IH3xQP3211XyIC+CZZ5r+PEEQtGka9cMAzgcGS5oACNgYOCNj+VnW9D4O\neMjMpgGY2ccZy247LEnPaKWUbt5gAzj4YPjjH90CC+DLL+vHrerRA8aNy58feih89BH88pf1w4+c\nf75bYm2yifdUunSB115z099Bg3zdj1VWyedfsMD322/v96Qn3oMgWCbIZCUlaQUg1zMYa2ZfZSpc\nOhI4wMxOT85/DHRP+3BIyg1FdQM6A/9nZvcUKattDklVmvQQFzRdGX34oVtt5dYw/9OfPKBi166w\n337wj3+48+A3v+lDWD/8IXz1lUfqBVdQ6fXPgyCoaSo5JNVoD0NSJ+CXwMZmdpqkLSVtbWaPV0KA\nRIZdgJ74UNcrkl4xs3cLM1ZtTe9aJjfEdc01sP/+Tb9/nXXqn+d6Efvu68roqadcWYAvHAVw4YXw\n5puw3nrZlMXcufDWWz5kFmuDBEFVae01vf8ODAVOMLPtEwXy38IItiXubXRNb0kXAh3N7LLk/Hbg\nSTN7qKCs6GHkeOUVfxkXW2u8qZjBk0+601+XLm6Flbbkyk28L1pU3zx30SKfAF9jjYZldu3qJsSX\nXOI+Ir16uUXWSiuFAgmCKlPtSe/NzexqoA7AzObhcxlZeB3YQtLGkjrgMakeK8jzT+BbkpZLlNEe\nwBiC0uy1V2WUBbgSOOggN7k99dSGZr/DhrmCKvTlmDmzdA/n8st9f8UVcNllbrm1yio+j/Lggx7e\nZOzY+tF8333Xra/mz6/McwVBUHGyTHp/LWlFEic+SZsDmeYwzGyhpNya3jmz2jGSzvDL1tfMxkp6\nGhgJLAT6mtno5jxMsAR07Ag33NAwPTe5Xsgaa/jk+pw58Nln+aEyM/jFL3zbYQdf3jZnpbXeenBU\ngRHcKaf4/Mgf/gDvvQfjx+eHwoIgqCmy9DB6A08BG0q6D3gWuCBrBWb2lJltbWZbmlmfJO02M+ub\nynONmXUzsx3M7MYmPkPQGiy3nPth3HmnT8Iff7z3Qtq1y3u0v/66D3G98473in7zG7fiWmMN73mA\n+46AD2/deGMoiyCoYcrOYUgSsAEwD9gTH4p6tTVMX2MOowYpFXKk8O80bx588YWHfC9k9mwfrtp5\nZw9zstlmPj9SbG6kMRYtgqFDYffdm35vECylVM1KysxM0r/M7BvAE5WoMFiK+O9/PXru3nu78rjl\nFncaLKRTp+JBEc3ygRlvvhm23hr69HF/kZRFXGaeftrnY+LDIghahCxWUncDN5nZ69URqaQc0cNY\nGikMf3LttfCrX9V/6c+c6UNXuQn5N97w4y23rF/WXXd5WJRcr6UYkyb5nMruu3tvpJTxwDnneBiV\nZdF0O1iqqLaV1B7Aq5LekzRS0ihJIytReRAwcaJ7luc49dR8mJMc11zjIUwuvNAXmNp99/o9mbFj\nYa218gETJ03KX/vwQ5+YB/dW33RTj8n15pvw+9+7OfHkyQ3levNNuPrq/PmCBR4V+O23l/iRg6Ct\nksVKKlblCVqOTTbxLUeXLm7iO3WqW17NmeMv7vvu84n1n/zE86WHrHbYwe/Zbjv3RK+r84i/667r\n2/77e1TfXAiVt97yfb9+MGMGjBnjoVDSXHst7LZb/vq++8KRR8KIEdCtWws1RhDUNiV7GJI6SjoP\n+A1wIDDNzCbntqpJGCx7dO7sgRPBv/QhPwl+8cXw6KMeC+vKK907PRe2fdVV3Tz4/vvdhHfWLJ9o\n32MPN+c95BCfd3nhBQ978sorft/nnzeUYafEL3XDDfPBGLfe2uV69lk3Jf7+9936K81HH8UcSrDU\nUm5I6m5gN2AU8D18bYwgaHkmTHD/DPC4Vnvt5aHVwV/ahx3mcxQXX+wmvUOG+DzH8kmHuWdP3z/9\ntF9PD1/ttJOvWnjEET48deaZfu8tt+SHpv75T1c2d9/t8bXAeyV1dX7v//yPK6eBA31VxQED3Bz4\n6adh7bXza7gHwdKGmRXdgFGp4+WBYaXyVmNzUYNlhnPOMQOzyy83++ILs7o6s3ffrZ/nD3/wPIsW\nNbz/F78wO/xwvz5rltmCBWbjxjXM16eP2RlneL4LLjAbPdqP//tfvz59utmdd/rx8OF+Dcy6djU7\n6yw/vukm3++2m+8vuaSiTREES0Ly7qzIe7jcEq3DzGyXUufVJqykljEGDcqvd96cv/uf/+zhRrp1\ng5//vHS+KVM8/MkNN0A6YNv8+fkIvWnq6nySPedg+PHHvkZJu3beK9pxR7ewKlbPF1/Atts2/VlK\nMWNGfi2UIChBtdb0Xgh8kTsFVsQd+IRrrBJ2iy1DKIxljK+/9hf22LE+DNXS3HSThzO5/XZfHOrv\nf2/a/ddd52utr7eeK5uttnILsJzZ7pprunIBNwveddfi5Sxa5PdMnZoPqVIKyYflrriifvr06e5V\n/4MfwG9/620YLLNURWHUGqEwgjbD3LkeNmXIkLzXeadObsEFPgfzyCOlPeUlV0DnnVe6DjPv1Zx4\nos+1/P73+XAr99wDJ5xQP2+wzFJtP4wlQtKBksZKeicJZV54fR9Jn0kalmy/bWmZgqBF6dzZ93fe\nmU/bbDM3yd15Z/fn6NzZFUNdnfekvkrF82zf3q3A0tx8M7z4Yv78l7/0fW59kpypMLihQC5i8Kmn\nVuaZqs3XX3tvL6gpWlRhZFnTO+EFM9sl2f7QkjIFQVXo29djaOX48EP3CRk2zC2vctc6dPCX41NP\n5fN+5zv1h6xGjvR5mPnzvbcwZYp7tN91lyuSAQN8OOy663ye5Lbb3DfFzAM6/uc/VXnkJeLll91X\nJsfYsb6GfVBbVGr2vNiGByx8MnV+EXBhQZ59gIEZymqukUAQVJ/hw8123dWPFy0yO+oot9TK8Y1v\nmB17rNnFF+etsj7/3OyFF8y+/trsq6/yea+6yvPMn2/24IN+XMg22+Qtxvbd12zOHE+fM8esY0ez\njz9uuWddUkaOzFuf5cidT59udsMNrSfbUgAVtJJq6SGp9YH3U+dTk7RC9pI0QtITkrZrYZmCoOXZ\nYQd47TV44gkP8T5gQP24Va+/7nMNV1zhE9177eULSx13nA9JTZrkZQBsvrnvO3TwNdaLMWKE77fa\nyi3McsNinTt7z2SNNXwI7N0GKx83j/nz4cc/rkxZubmdQw5xr/qcccCZZ8K//gXnnhvzMDVCOU/v\nOZI+T7Y5qfM5koq4xjabocBG5ku+3gQ8WsGyg6B1aNfOX6qHHJJfXCrNCivkFUhu8vvii91THFxp\njBrlZro9e/qw1axZfu3uu4uXd9JJjQ/jpIennnvOQ67keP11V17FMMu/yC+8ED74wO+V3Bz5oIPK\n11uKF16A4cO9/IEDPbzLmmu6crzuuryXfaElWNAqlIwlZWYrV6D8acBGqfMNkrR0PXNTx09KukXS\namb2SWFhl6biB/Xo0YMeEUk0qGVyE9m5r/1yfPqpv4RzL/RNN3UF8sc/umJp3957CR9+6N7kxUhP\nsqe5+mr45jfh8cdh9Gh/Offq5dF+zznHY3TNmePrxN96q1t39ezp8yynnea9kuOP9/TDD3cLr3Rg\nxtxLfeJEn9zv39/nVLIwbJh79oOv4JjjjTd8Qn/TTV3p5sK4BI0yePBgBqd9iipJlnEr4FvAycnx\nGsCmGe9bDngX2BjoAIwAti3Is3bquDswqURZFR3XC4Kq8MorZtde23i+zz/3MfuZM/NpEyaYXXhh\n5WS59VazTp3M7r3X68rtZ840++Y3zTp0MHviifz8QW5OJH2e3ubN83mZ1Vd3L/fNNvP0k082GzjQ\n6xw82OdkcvcMGlRfpm9/26x3bz+eOtXzLFxYP89VV5n17OnzOsW8+oOyUME5jCwv/d7AQOCd5Hw9\n4OXMFXgo4NdsAAAOAElEQVTgwnHAeOCiJO0M4PTk+GzgLWA48F9gjxLltEhjBkHNUCx0SaVZsMDs\nssvqK4MNN8y/0KdM8f20ab7faivfn3ii2fjxrlw++cSvF/Ktb5ltsonZ1lv7PS++6Pvf/S5fRy7M\nSg4wO+mkbLKD2fbb+75v33x6795mc+c2s0GayPDhbqjQhqi2whiBe3cPT6WNrJQAmQUNhREEleHU\nU81++EM//t3vzB5+2GNvPfusp+WsucDs4IN9P2xY4+XW1fm+Xz+/56678oro/ffNDjnEr3/5pdnE\niW7BBd67aozCns6PfuQ9kbfe8vNTT7XFVlbPPuu9n3//O3OTZOYnP8nXY2Z2/vl+Pny42WuvNb/c\n/ff3mGktQLUVxpBkPyzZrxQKIwjaMA8/bPbOO43n+/RTf1GnzYGz0q+f2Ztvem/kuuvqXzv5ZFs8\npJWVQYOsXq/oj3/MD4GB2Uor+f6AA3z/ve/5Pt3zOO+8vKlzc+nd28v96CMfIttxx/qKrByjR3sv\nrZC6Or939OjG63/88SYPy1VbYfwauA2YAJwGvAL8olICZBY0FEYQtH2GDs2/XCdOzH7fggVmb7zh\nx1Om+DGYde9u9uc/m730Un64Csx69fJ9165mb7/t9227rfesBg3K1qsxM1ttNbP77vOXek755Hxe\n2rUzu/9+n2caMsTTyvUScrLNmFE/3/Tpnn722cXvmzvXe2M5n52mtJtVWWF4fewH/CnZ9qtU5U0S\nNBRGELR9Jk/2107OsXBJALNHHqmf9vHH+aG13r3diTH3Ik/Pq1x2Wf37Jk1qWP7cuZ43F74+9w4a\nMCB//skn+fxbb23Wv39pedMK7eST8+mvvmq2ww7Few45Y4j01sQeXyUVRlbHvVHAi8ALyXEQBEHT\n2Wgjf+1lMTVujGefbej/sfrq+QW0Lr3Ul9oFN1EGd4JcaSXo3Tt//tOf+jLBffvmy5Hyvh8/+1n9\nOo46CmbP9vD7n32WTz/mGHj++dLy5laP3GUXN5WeP9/r7NbNzYuLBaPs39/3Z58N3/uexxM76igP\nNPnJJ75yZDVpTKMAPwWmAHfhq/BNAk6plMbKuhE9jCAImsrChf5Vvs8+PqdiZvbyy1bPIixnwfWr\nX9W/5+ijzTbf3NPWWsvs+eez1VlXl5/72XZbszFjzCSzhx7y67lQL926+f6EE0qX9fHHnmfChHxa\nrqdx6KG+f+65suJQ5TmMccDqqfPVgXGVEiCzoKEwgiBoDrNmuWVWGnD/EfDJ66lT83lmzPBrdXX5\n4Z+sPiBpa66nnvL96afbYqsxMx+O69zZh87AbOxYT58502y99RoOORXWm1shsmfPvPxlqKTCKOnp\nnWIWMCd1PidJC4IgqH1WW61hmpkvNPXZZx6GZP31PX7XjBk+rDRrVn6NePA8WfCPW2fIEN/X1fk+\nt0ZJ587uWV+Yf9VV3dv/hBM84vCCBdC1a8Ohqltvhcce81Ulv/GN+jHKWpiSCkNSEnCfd4HXJP0T\nMOAwYGQVZAuCIGg51l3Xtxw9esDkyR56Zccdm1dmu3auhF56CdZaC/bc04NP3nln6QWzciy3HPTp\n46FSVk4iMz33nMtVyMSJxZcQbmHK9TBysaTeS7Yc/2w5cYIgCFqJFVf0/aRJ+YWpmkOXLnDwwfnz\n//wH9tkn273du3vMrxybblo8XysoC4glWoMgCJyPPoJtt81HBW4NZszwJXz79vUeR7duS1xkVdf0\nlrQmcAG+Yt5itWtmPSshQFZCYQRBEDSdaq/pfR8wFtgUuAw3q329EpUHQRAEbYcsCmN1M+sH1JnZ\n82Z2CpC5dyHpQEljJb0j6cIy+XaXVCfpiKxl1yItFoe+woSclaUtyNkWZISQs5bJojASmzCmSzpY\n0s5AETu1hkhqh6+idwA+pHWspG1K5OsDPJ1J6hqmrfyIQs7K0hbkbAsyQshZy2Txw/iDpC7Ar4Ab\ngVWA8zKW3x0Yb2aTAST1x81yxxbk+wXwILB7xnKDIAiCKtNoD8PMHjez2Wb2lpnta2a7AptnLH99\n4P3U+dQkbTGS1gN+YGZ/wdfdCIIgCGqQZpnVSppiZhtlyHckcICZnZ6c/xjobmbnpPIMAK4xsyGS\n7gQeN7MGEbUkhYlUEARBM6iUlVSWIaliZK18GpBWLBskaWl2A/pLEr5e+Pck1ZnZY+lMlXrgIAiC\noHk0V2Fk/dp/HdhC0sbAdOAY4Nh6BZltljtOehgDC5VFEARB0PqUiyU1h+KKQcCKWQo3s4WSfg48\ng8+X9DOzMZLO8MvWt/CWbGIHQRAE1abNhAYJgiAIWpesK+61Klmd/6okyyRJb0oaLmlIkraqpGck\njZP0dGKGnMvfS9J4SWMk7d+CcvWTNEPSyFRak+WStIukkUlbX18lOXtLmippWLIdWANybiBpkKS3\nJY2SdE6SXjNtWkTGXyTpNdWeklaQ9FryPzNKUu8kvWbashE5a6o9U3W0S+R5LDlv+fas1MIaLbXh\nSu1dYGOgPTAC2KYV5ZkArFqQdhVwQXJ8IdAnOd4OGI4P/W2SPIdaSK5vATsBI5dELuA1YPfk+F+4\nlVtLy9kb+GWRvNu2opzrADslx53xhcS2qaU2LSNjLbZnp2S/HPAq7qNVM23ZiJw1155JuecD9wKP\nJect3p5toYex2PnPzOqAnPNfayEa9swOw5evJdn/IDn+PtDfzBaY2SRgPP48FcfMXgI+XRK5JK0D\nrGxmuVhhf0vd05JyQnHLu8NaUc4PzWxEcjwXGINb+dVMm5aQMefnVGvtOS85XAF/ceXW1qmJtmxE\nTqix9pS0AXAQcHuBPC3anm1BYTTq/FdlDPi3pNcl/TRJW9vMZoD/EwNrJemFsk+jurKv1US51sfb\nN0c12/rnkkZIuj3Vla4JOSVtgveKXqXpf+uqyJqS8bUkqabaMxk+GQ58CPw7eUnVXFuWkBNqrD2B\n64DfUN9QqMXbsy0ojFpjbzPbBdfuZ0v6Ng2tu2rVkqBW5boF2MzMdsL/Uf/cyvIsRlJnPGzNuclX\nfM39rYvIWHPtaWaLzGxnvJfWXVI3arAti8i5HTXWnpIOBmYkvcty/mkVb8+2oDCyOP9VDTObnuxn\nAo/iQ0wzJK0NkHTzPkqyTwM2TN1ebdmbKleryGtmMy0ZRAX+Sn7YrlXllLQ8/iK+x8xyK03WVJsW\nk7FW2zOR7XNgMHAgNdaWpeSswfbcG/i+pAnAA0BPSfcAH7Z0e7YFhbHY+U9SB9z5r1Uc+yR1Sr7m\nkLQSsD8wKpHnpCTbieSXsX0MOEZSB0mbAlsAQ1pSROp/cTRJrqQbO1tSd0kCTqBlluStJ2fy485x\nBPBWjch5BzDazG5IpdVamzaQsdbaU9IauWEcSSsC++HzLTXVliXkHFtr7WlmF5vZRuZOz8cAg8zs\nJ8BAWro9Kz1z3xIb/jUyDp+suagV5dgUt9IajiuKi5L01YD/JDI+A3RN3dMLt0oYA+zfgrLdD3wA\nfAVMAU4GVm2qXMCuybONB26okpx/A0YmbfsoPhbb2nLuDSxM/b2HJb/DJv+tW0rWMjLWVHsC30hk\nG5HIdUlz/29aSc6aas8CmfchbyXV4u0ZjntBEARBJtrCkFQQBEFQA4TCCIIgCDIRCiMIgiDIRCiM\nIAiCIBOhMIIgCIJMhMIIgiAIMhEKI6g5JC2S9LfU+XKSZqbCOB8q6YJGylhXvl48kk6UdGMTZeiV\nIc+dko5oSrmVRNJzknZprfqDZY9QGEEt8gWwvaQVkvP9SAVPM7OBZnZ1uQLMbLqZ/Sid1EQZLm5i\n/jaFpOVaW4ag7REKI6hV/gUcnBwfi8fMAer3GJKv/BskvSzp3dwXfxJKZlSqvI2SL/Jxkn6fKuuR\nJPLwqFz0YUlXAivKF6e5J0k7QfmFs+5OlbtPYd1pEjlGS+or6S1JT+UUYbqHIGl1SRNTz/eIfDGc\nCZLOlnR+Is9/JXVNVXFCItNISbsn93eSL1T1qqShkg5NlftPSc/iHsFB0CRCYQS1iOHrnhybvFx3\nIB+2O50nxzpmtjdwKL6ITLE8uwOHAzsCR6WGck42s92T6+dKWtXMegHzzGwXM/uJPGLpxUAP80im\n52aoO80WwI1mtj0wGziyzHPn6IavTdAduAKYax4l+VU85k+OFROZzsbjSgFcAjxrZnsCPYFrkthI\nADsDR5jZviVkCIKShMIIahIzewtfHexY4AnKh3F+NLlnDPk1AAr5t5l9ZmbzgYfxlf8AzpM0An8R\nbwBsmaSn6+sJ/MPMPk3q+ayJdU80s1xvZ2jyXI3xnJnNM7OPgc+Ax5P0UQX3P5DU/yKwsqRV8KCY\nF8nXdRgMdCAf8fnfZjY7Q/1B0IDlW1uAICjDY8CfgB7AGmXyfZU6LqVYGqy9IGkfXBnsYWZfSXoO\n6NhEGbPUnc6zMFXHAvIfbYX1pu+x1Pki6v/fFltTQsCRZjY+fUHSnvj8UBA0i+hhBLVI7sV7B3CZ\nmb3djHsL2U9S12Ro5gfAy0AX4NNEWWwD7JnK/3VqYngQPoy1GoCkVZtYd6n0ScBuyfFRJfI0xtGJ\nTN8CZpvZHOBp4JzFlUs7NbPsIKhHKIygFjEAM5tmZjdlyVvmPMcQfChqBD68NAx4Cmgv6W3gj8Ar\nqfx9gVGS7jGz0cn155NhntyKa1nrLpV+DXCWpKF4aOpSlCt3vqRh+KpwpyTpl+PPNVLSW8D/lik7\nCDIT4c2DIAiCTEQPIwiCIMhEKIwgCIIgE6EwgiAIgkyEwgiCIAgyEQojCIIgyEQojCAIgiAToTCC\nIAiCTPw/3Dt3R1gXoRgAAAAASUVORK5CYII=\n",
|
|
"text/plain": [
|
|
"<matplotlib.figure.Figure at 0x2ec2b341b00>"
|
|
]
|
|
},
|
|
"metadata": {},
|
|
"output_type": "display_data"
|
|
}
|
|
],
|
|
"source": [
|
|
"pred_resnet = train_and_evaluate(reader_train, reader_test, max_epochs=5, model_func=create_resnet_model)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"collapsed": true
|
|
},
|
|
"outputs": [],
|
|
"source": []
|
|
}
|
|
],
|
|
"metadata": {
|
|
"anaconda-cloud": {},
|
|
"kernelspec": {
|
|
"display_name": "Python 3",
|
|
"language": "python",
|
|
"name": "python3"
|
|
},
|
|
"language_info": {
|
|
"codemirror_mode": {
|
|
"name": "ipython",
|
|
"version": 3
|
|
},
|
|
"file_extension": ".py",
|
|
"mimetype": "text/x-python",
|
|
"name": "python",
|
|
"nbconvert_exporter": "python",
|
|
"pygments_lexer": "ipython3",
|
|
"version": "3.5.2"
|
|
}
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 1
|
|
}
|