704 строки
74 KiB
Plaintext
704 строки
74 KiB
Plaintext
{
|
|
"cells": [
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 1,
|
|
"metadata": {
|
|
"collapsed": true
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"from IPython.display import Image"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"# CNTK 206: Part A - Basic GAN with MNIST data\n",
|
|
"\n",
|
|
"**Prerequisites**: We assume that you have successfully downloaded the MNIST data by completing the tutorial titled CNTK_103A_MNIST_DataLoader.ipynb.\n",
|
|
"\n",
|
|
"## Introduction\n",
|
|
"\n",
|
|
"[Generative models](https://en.wikipedia.org/wiki/Generative_model) have gained a [lot of attention](https://openai.com/blog/generative-models/) in deep learning community which has traditionally leveraged [discriminative models](https://en.wikipedia.org/wiki/Discriminative_model) for (semi-supervised) and unsupervised learning. In generative modeling, the idea is to collect a huge amount of data in a domain of interest (e.g., pictures, audio, words) and come up with a trained model that generates such real world data sets. This is an active area of research needing mechanisms to scale up training and having large datasets. As stated in the [OpenAI blog](https://openai.com/blog/generative-models/), such approaches may be used to perform computer aided art generation, or morph images to some word descriptions such as \"make my smile wider\". This approach has found use in image denoising, inpainting, super-resolution, structured prediction, exploration in reinforcement learning, and neural network pretraining in cases where labeled data is expensive. \n",
|
|
"\n",
|
|
"Generating models that can produce realistic content (images, sounds etc.) mimicking real world observations is challenging. Generative Adversarial Network (GAN) is one of the approaches that holds promise. A [quote](https://www.quora.com/What-are-some-recent-and-potentially-upcoming-breakthroughs-in-deep-learning) from Yann LeCun summarizes GAN and its variations as the most important idea in the last 10 years. The original idea was proposed by [Goodfellow et al](https://arxiv.org/pdf/1406.2661v1.pdf) at NIPS 2014. In this tutorial, we show how to use the Cognitive Toolkit to create a basic GAN network for generating synthetic MNIST digits."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 2,
|
|
"metadata": {
|
|
"collapsed": true
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"import matplotlib as mpl\n",
|
|
"import matplotlib.pyplot as plt\n",
|
|
"import numpy as np\n",
|
|
"import os\n",
|
|
"\n",
|
|
"import cntk as C\n",
|
|
"import cntk.tests.test_utils\n",
|
|
"cntk.tests.test_utils.set_device_from_pytest_env() # (only needed for our build system)\n",
|
|
"C.cntk_py.set_fixed_random_seed(1) # fix a random seed for CNTK components\n",
|
|
"\n",
|
|
"%matplotlib inline"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"There are two run modes:\n",
|
|
"- *Fast mode*: `isFast` is set to `True`. This is the default mode for the notebooks, which means we train for fewer iterations or train / test on limited data. This ensures functional correctness of the notebook though the models produced are far from what a completed training would produce.\n",
|
|
"\n",
|
|
"- *Slow mode*: We recommend the user to set this flag to `False` once the user has gained familiarity with the notebook content and wants to gain insight from running the notebooks for a longer period with different parameters for training. \n",
|
|
"\n",
|
|
"**Note**\n",
|
|
"If the `isFlag` is set to `False` the notebook will take a few hours on a GPU enabled machine. You can try fewer iterations by setting the `num_minibatches` to a smaller number say `20,000` which comes at the expense of quality of the generated images."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 3,
|
|
"metadata": {
|
|
"collapsed": true
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"isFast = True "
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Data Reading\n",
|
|
"The input to the GAN will be a vector of random numbers. At the end of the traning, the GAN \"learns\" to generate images of hand written digits drawn from the [MNIST database](https://en.wikipedia.org/wiki/MNIST_database). We will be using the same MNIST data generated in tutorial 103A. A more in-depth discussion of the data format and reading methods can be seen in previous tutorials. For our purposes, just know that the following function returns an object that will be used to generate images from the MNIST dataset. Since we are building an unsupervised model, we only need to read in `features` and ignore the `labels`."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 4,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Data directory is ..\\Examples\\Image\\DataSets\\MNIST\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"# Ensure the training data is generated and available for this tutorial\n",
|
|
"# We search in two locations in the toolkit for the cached MNIST data set.\n",
|
|
"\n",
|
|
"data_found = False\n",
|
|
"for data_dir in [os.path.join(\"..\", \"Examples\", \"Image\", \"DataSets\", \"MNIST\"),\n",
|
|
" os.path.join(\"data\", \"MNIST\")]:\n",
|
|
" train_file = os.path.join(data_dir, \"Train-28x28_cntk_text.txt\")\n",
|
|
" if os.path.isfile(train_file):\n",
|
|
" data_found = True\n",
|
|
" break\n",
|
|
" \n",
|
|
"if not data_found:\n",
|
|
" raise ValueError(\"Please generate the data by completing CNTK 103 Part A\")\n",
|
|
" \n",
|
|
"print(\"Data directory is {0}\".format(data_dir))"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 5,
|
|
"metadata": {
|
|
"collapsed": true
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"def create_reader(path, is_training, input_dim, label_dim):\n",
|
|
" deserializer = C.io.CTFDeserializer(\n",
|
|
" filename = path,\n",
|
|
" streams = C.io.StreamDefs(\n",
|
|
" labels_unused = C.io.StreamDef(field = 'labels', shape = label_dim, is_sparse = False),\n",
|
|
" features = C.io.StreamDef(field = 'features', shape = input_dim, is_sparse = False\n",
|
|
" )\n",
|
|
" )\n",
|
|
" )\n",
|
|
" return C.io.MinibatchSource(\n",
|
|
" deserializers = deserializer,\n",
|
|
" randomize = is_training,\n",
|
|
" max_sweeps = C.io.INFINITELY_REPEAT if is_training else 1\n",
|
|
" )"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"The random noise we will use to train the GAN is provided by the `noise_sample` function to generate random noise samples from a uniform distribution within the interval [-1, 1]."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 6,
|
|
"metadata": {
|
|
"collapsed": true
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"np.random.seed(123)\n",
|
|
"def noise_sample(num_samples):\n",
|
|
" return np.random.uniform(\n",
|
|
" low = -1.0,\n",
|
|
" high = 1.0,\n",
|
|
" size = [num_samples, g_input_dim] \n",
|
|
" ).astype(np.float32)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Model Creation\n",
|
|
"\n",
|
|
"A GAN network is composed of two sub-networks, one called the Generator ($G$) and the other Discriminator ($D$). \n",
|
|
"- The **Generator** takes random noise vector ($z$) as input and strives to output synthetic (fake) image ($x^*$) that is indistinguishable from the real image ($x$) from the MNIST dataset. \n",
|
|
"- The **Discriminator** strives to differentiate between the real image ($x$) and the fake ($x^*$) image."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 7,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/html": [
|
|
"<img src=\"https://www.cntk.ai/jup/GAN_basic_flow.png\"/>"
|
|
],
|
|
"text/plain": [
|
|
"<IPython.core.display.Image object>"
|
|
]
|
|
},
|
|
"execution_count": 7,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"# Figure 1\n",
|
|
"Image(url=\"https://www.cntk.ai/jup/GAN_basic_flow.png\")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"In each training iteration, the Generator produces more realistic fake images (in other words *minimizes* the difference between the real and generated counterpart) and also the Discriminator *maximizes* the probability of assigning the correct label (real vs. fake) to both real examples (from training set) and the generated fake ones. The two conflicting objectives between the sub-networks ($G$ and $D$) leads to the GAN network (when trained) converge to an equilibrium, where the Generator produces realistic looking fake MNIST images and the Discriminator can at best randomly guess whether images are real or fake. The resulting Generator model once trained produces realistic MNIST image with the input being a random number. "
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Model config\n",
|
|
"\n",
|
|
"First, we establish some of the architectural and training hyper-parameters for our model. \n",
|
|
"\n",
|
|
"- The generator network is a fully-connected network with a single hidden layer. The input will be a 100-dimensional random vector and the output will be a 784 dimensional vector, corresponding to a flattened version of a 28 x 28 fake image. The discriminator is also a single layer dense network. It takes as input the 784 dimensional output of the generator or a real MNIST image and outputs a single scalar - the estimated probability that the input image is a real MNIST image."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Model components\n",
|
|
"We build a computational graph for our model, one each for the generator and the discriminator. First, we establish some of the architectural parameters of our model. \n",
|
|
"\n",
|
|
"- The generator takes a 100-dimensional random vector (for starters) as input ($z$) and the outputs a 784 dimensional vector, corresponding to a flattened version of a 28 x 28 fake (synthetic) image ($x^*$). In this tutorial we simply model the generator with two dense layers. We use a tanh activation on the last layer to make sure that the output of the generator function is confined to the interval [-1, 1]. This is necessary because we also scale the MNIST images to this interval, and the outputs of the generator must be able to emulate the actual images as closely as possible.\n",
|
|
"\n",
|
|
"\n",
|
|
"- The discriminator takes as input ($x^*$) the 784 dimensional output of the generator or a real MNIST image and outputs the estimated probability that the input image is a real MNIST image. We also model this with two dense layers with a sigmoid activation in the last layer ensuring that the discriminator produces a valid probability."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 8,
|
|
"metadata": {
|
|
"collapsed": true
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"# architectural parameters\n",
|
|
"g_input_dim = 100\n",
|
|
"g_hidden_dim = 128\n",
|
|
"g_output_dim = d_input_dim = 784\n",
|
|
"d_hidden_dim = 128\n",
|
|
"d_output_dim = 1"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 9,
|
|
"metadata": {
|
|
"collapsed": true
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"def generator(z):\n",
|
|
" with C.layers.default_options(init = C.xavier()):\n",
|
|
" h1 = C.layers.Dense(g_hidden_dim, activation = C.relu)(z)\n",
|
|
" return C.layers.Dense(g_output_dim, activation = C.tanh)(h1)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 10,
|
|
"metadata": {
|
|
"collapsed": true
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"def discriminator(x):\n",
|
|
" with C.layers.default_options(init = C.xavier()):\n",
|
|
" h1 = C.layers.Dense(d_hidden_dim, activation = C.relu)(x)\n",
|
|
" return C.layers.Dense(d_output_dim, activation = C.sigmoid)(h1)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"We use a minibatch size of 1024 and a fixed learning rate of 0.00005 for training. In the fast mode (`isFast = True`) we verify only functional correctness with 300 iterations. \n",
|
|
"\n",
|
|
"**Note**: In the slow mode, the results look a lot better but it requires patient waiting (few hours) depending on your hardware. In general, the more number of minibatches one trains, the better is the fidelity of the generated images."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 11,
|
|
"metadata": {
|
|
"collapsed": true
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"# training config\n",
|
|
"minibatch_size = 1024\n",
|
|
"num_minibatches = 300 if isFast else 40000\n",
|
|
"lr = 0.00005"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Build the graph\n",
|
|
"\n",
|
|
"The rest of the computational graph is mostly responsible for coordinating the training algorithms and parameter updates, which is particularly tricky with GANs for couple reasons. \n",
|
|
"\n",
|
|
"- First, the discriminator must be used on both the real MNIST images and fake images generated by the generator function. One way to represent this in the computational graph is to create a clone of the output of the discriminator function, but with substituted inputs. Setting `method=share` in the `clone` function ensures that both paths through the discriminator model use the same set of parameters.\n",
|
|
"\n",
|
|
"\n",
|
|
"- Second, we need to update the parameters for the generator and discriminator model separately using the gradients from different loss functions. We can get the parameters for a `Function` in the graph with the `parameters` attribute. However, when updating the model parameters, update only the parameters of the respective models while keeping the other parameters unchanged. In other words, when updating the generator we will update only the parameters of the $G$ function while keeping the parameters of the $D$ function fixed and vice versa.\n",
|
|
"\n",
|
|
"### Training the Model\n",
|
|
"The code for training the GAN very closely follows the algorithm as presented in the [original NIPS 2014 paper](https://arxiv.org/pdf/1406.2661v1.pdf). In this implementation, we train $D$ to maximize the probability of assigning the correct label (fake vs. real) to both training examples and the samples from $G$. In other words, $D$ and $G$ play the following two-player minimax game with the value function $V(G,D)$:\n",
|
|
"\n",
|
|
"$$\n",
|
|
" \\min_G \\max_D V(D,G)= \\mathbb{E}_{x}[ log D(x) ] + \\mathbb{E}_{z}[ log(1 - D(G(z))) ]\n",
|
|
"$$\n",
|
|
"\n",
|
|
"At the optimal point of this game the generator will produce realistic looking data while the discriminator will predict that the generated image is indeed fake with a probability of 0.5. The [algorithm referred below](https://arxiv.org/pdf/1406.2661v1.pdf) is implemented in this tutorial."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 12,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/html": [
|
|
"<img src=\"https://www.cntk.ai/jup/GAN_goodfellow_NIPS2014.png\" width=\"500\"/>"
|
|
],
|
|
"text/plain": [
|
|
"<IPython.core.display.Image object>"
|
|
]
|
|
},
|
|
"execution_count": 12,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"# Figure 2\n",
|
|
"Image(url=\"https://www.cntk.ai/jup/GAN_goodfellow_NIPS2014.png\", width = 500)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 13,
|
|
"metadata": {
|
|
"collapsed": true
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"def build_graph(noise_shape, image_shape,\n",
|
|
" G_progress_printer, D_progress_printer):\n",
|
|
" input_dynamic_axes = [C.Axis.default_batch_axis()]\n",
|
|
" Z = C.input_variable(noise_shape, dynamic_axes=input_dynamic_axes)\n",
|
|
" X_real = C.input_variable(image_shape, dynamic_axes=input_dynamic_axes)\n",
|
|
" X_real_scaled = 2*(X_real / 255.0) - 1.0\n",
|
|
"\n",
|
|
" # Create the model function for the generator and discriminator models\n",
|
|
" X_fake = generator(Z)\n",
|
|
" D_real = discriminator(X_real_scaled)\n",
|
|
" D_fake = D_real.clone(\n",
|
|
" method = 'share',\n",
|
|
" substitutions = {X_real_scaled.output: X_fake.output}\n",
|
|
" )\n",
|
|
"\n",
|
|
" # Create loss functions and configure optimazation algorithms\n",
|
|
" G_loss = 1.0 - C.log(D_fake)\n",
|
|
" D_loss = -(C.log(D_real) + C.log(1.0 - D_fake))\n",
|
|
"\n",
|
|
" G_learner = C.fsadagrad(\n",
|
|
" parameters = X_fake.parameters,\n",
|
|
" lr = C.learning_parameter_schedule_per_sample(lr),\n",
|
|
" momentum = C.momentum_schedule_per_sample(0.9985724484938566)\n",
|
|
" )\n",
|
|
" D_learner = C.fsadagrad(\n",
|
|
" parameters = D_real.parameters,\n",
|
|
" lr = C.learning_parameter_schedule_per_sample(lr),\n",
|
|
" momentum = C.momentum_schedule_per_sample(0.9985724484938566)\n",
|
|
" )\n",
|
|
"\n",
|
|
" # Instantiate the trainers\n",
|
|
" G_trainer = C.Trainer(\n",
|
|
" X_fake,\n",
|
|
" (G_loss, None),\n",
|
|
" G_learner,\n",
|
|
" G_progress_printer\n",
|
|
" )\n",
|
|
" D_trainer = C.Trainer(\n",
|
|
" D_real,\n",
|
|
" (D_loss, None),\n",
|
|
" D_learner,\n",
|
|
" D_progress_printer\n",
|
|
" )\n",
|
|
"\n",
|
|
" return X_real, X_fake, Z, G_trainer, D_trainer"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"With the value functions defined we proceed to iteratively train the GAN model. The training of the model can take significantly long depending on the hardware especially if `isFast` flag is turned off."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 14,
|
|
"metadata": {
|
|
"collapsed": true
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"def train(reader_train):\n",
|
|
" k = 2\n",
|
|
" \n",
|
|
" # print out loss for each model for upto 50 times\n",
|
|
" print_frequency_mbsize = num_minibatches // 50\n",
|
|
" pp_G = C.logging.ProgressPrinter(print_frequency_mbsize)\n",
|
|
" pp_D = C.logging.ProgressPrinter(print_frequency_mbsize * k)\n",
|
|
"\n",
|
|
" X_real, X_fake, Z, G_trainer, D_trainer = \\\n",
|
|
" build_graph(g_input_dim, d_input_dim, pp_G, pp_D)\n",
|
|
" \n",
|
|
" input_map = {X_real: reader_train.streams.features}\n",
|
|
" for train_step in range(num_minibatches):\n",
|
|
"\n",
|
|
" # train the discriminator model for k steps\n",
|
|
" for gen_train_step in range(k):\n",
|
|
" Z_data = noise_sample(minibatch_size)\n",
|
|
" X_data = reader_train.next_minibatch(minibatch_size, input_map)\n",
|
|
" if X_data[X_real].num_samples == Z_data.shape[0]:\n",
|
|
" batch_inputs = {X_real: X_data[X_real].data, \n",
|
|
" Z: Z_data}\n",
|
|
" D_trainer.train_minibatch(batch_inputs)\n",
|
|
"\n",
|
|
" # train the generator model for a single step\n",
|
|
" Z_data = noise_sample(minibatch_size)\n",
|
|
" batch_inputs = {Z: Z_data}\n",
|
|
" G_trainer.train_minibatch(batch_inputs)\n",
|
|
"\n",
|
|
" G_trainer_loss = G_trainer.previous_minibatch_loss_average\n",
|
|
"\n",
|
|
" return Z, X_fake, G_trainer_loss"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 15,
|
|
"metadata": {
|
|
"scrolled": false
|
|
},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Learning rate per sample: 5e-05\n",
|
|
"Learning rate per sample: 5e-05\n",
|
|
" Minibatch[ 1- 12]: loss = 0.460688 * 12288, metric = 0.00% * 12288;\n",
|
|
" Minibatch[ 1- 6]: loss = 2.744962 * 6144, metric = 0.00% * 6144;\n",
|
|
" Minibatch[ 13- 24]: loss = 0.496159 * 12288, metric = 0.00% * 12288;\n",
|
|
" Minibatch[ 7- 12]: loss = 2.408652 * 6144, metric = 0.00% * 6144;\n",
|
|
" Minibatch[ 25- 36]: loss = 0.468794 * 12288, metric = 0.00% * 12288;\n",
|
|
" Minibatch[ 13- 18]: loss = 2.497859 * 6144, metric = 0.00% * 6144;\n",
|
|
" Minibatch[ 37- 48]: loss = 0.549526 * 12288, metric = 0.00% * 12288;\n",
|
|
" Minibatch[ 19- 24]: loss = 2.715347 * 6144, metric = 0.00% * 6144;\n",
|
|
" Minibatch[ 49- 60]: loss = 1.273582 * 12288, metric = 0.00% * 12288;\n",
|
|
" Minibatch[ 25- 30]: loss = 1.957965 * 6144, metric = 0.00% * 6144;\n",
|
|
" Minibatch[ 61- 72]: loss = 1.263255 * 12288, metric = 0.00% * 12288;\n",
|
|
" Minibatch[ 31- 36]: loss = 2.055775 * 6144, metric = 0.00% * 6144;\n",
|
|
" Minibatch[ 73- 84]: loss = 1.041039 * 12288, metric = 0.00% * 12288;\n",
|
|
" Minibatch[ 37- 42]: loss = 1.913864 * 6144, metric = 0.00% * 6144;\n",
|
|
" Minibatch[ 85- 96]: loss = 1.190252 * 12288, metric = 0.00% * 12288;\n",
|
|
" Minibatch[ 43- 48]: loss = 1.834975 * 6144, metric = 0.00% * 6144;\n",
|
|
" Minibatch[ 97- 108]: loss = 0.916644 * 12288, metric = 0.00% * 12288;\n",
|
|
" Minibatch[ 49- 54]: loss = 1.856111 * 6144, metric = 0.00% * 6144;\n",
|
|
" Minibatch[ 109- 120]: loss = 0.949138 * 12288, metric = 0.00% * 12288;\n",
|
|
" Minibatch[ 55- 60]: loss = 1.612386 * 6144, metric = 0.00% * 6144;\n",
|
|
" Minibatch[ 121- 132]: loss = 1.159921 * 12288, metric = 0.00% * 12288;\n",
|
|
" Minibatch[ 61- 66]: loss = 1.720942 * 6144, metric = 0.00% * 6144;\n",
|
|
" Minibatch[ 133- 144]: loss = 1.069388 * 12288, metric = 0.00% * 12288;\n",
|
|
" Minibatch[ 67- 72]: loss = 1.742269 * 6144, metric = 0.00% * 6144;\n",
|
|
" Minibatch[ 145- 156]: loss = 1.160007 * 12288, metric = 0.00% * 12288;\n",
|
|
" Minibatch[ 73- 78]: loss = 1.752263 * 6144, metric = 0.00% * 6144;\n",
|
|
" Minibatch[ 157- 168]: loss = 1.109970 * 12288, metric = 0.00% * 12288;\n",
|
|
" Minibatch[ 79- 84]: loss = 2.054680 * 6144, metric = 0.00% * 6144;\n",
|
|
" Minibatch[ 169- 180]: loss = 1.092135 * 12288, metric = 0.00% * 12288;\n",
|
|
" Minibatch[ 85- 90]: loss = 1.904574 * 6144, metric = 0.00% * 6144;\n",
|
|
" Minibatch[ 181- 192]: loss = 1.127416 * 12288, metric = 0.00% * 12288;\n",
|
|
" Minibatch[ 91- 96]: loss = 1.740344 * 6144, metric = 0.00% * 6144;\n",
|
|
" Minibatch[ 193- 204]: loss = 1.096188 * 12288, metric = 0.00% * 12288;\n",
|
|
" Minibatch[ 97- 102]: loss = 1.672640 * 6144, metric = 0.00% * 6144;\n",
|
|
" Minibatch[ 205- 216]: loss = 1.076912 * 12288, metric = 0.00% * 12288;\n",
|
|
" Minibatch[ 103- 108]: loss = 1.567401 * 6144, metric = 0.00% * 6144;\n",
|
|
" Minibatch[ 217- 228]: loss = 1.157102 * 12288, metric = 0.00% * 12288;\n",
|
|
" Minibatch[ 109- 114]: loss = 1.651072 * 6144, metric = 0.00% * 6144;\n",
|
|
" Minibatch[ 229- 240]: loss = 1.028144 * 12288, metric = 0.00% * 12288;\n",
|
|
" Minibatch[ 115- 120]: loss = 1.683286 * 6144, metric = 0.00% * 6144;\n",
|
|
" Minibatch[ 241- 252]: loss = 0.904863 * 12288, metric = 0.00% * 12288;\n",
|
|
" Minibatch[ 121- 126]: loss = 1.785782 * 6144, metric = 0.00% * 6144;\n",
|
|
" Minibatch[ 253- 264]: loss = 0.852108 * 12288, metric = 0.00% * 12288;\n",
|
|
" Minibatch[ 127- 132]: loss = 1.827977 * 6144, metric = 0.00% * 6144;\n",
|
|
" Minibatch[ 265- 276]: loss = 0.696002 * 12288, metric = 0.00% * 12288;\n",
|
|
" Minibatch[ 133- 138]: loss = 2.021851 * 6144, metric = 0.00% * 6144;\n",
|
|
" Minibatch[ 277- 288]: loss = 0.584447 * 12288, metric = 0.00% * 12288;\n",
|
|
" Minibatch[ 139- 144]: loss = 2.205963 * 6144, metric = 0.00% * 6144;\n",
|
|
" Minibatch[ 289- 300]: loss = 0.560834 * 12288, metric = 0.00% * 12288;\n",
|
|
" Minibatch[ 145- 150]: loss = 2.256393 * 6144, metric = 0.00% * 6144;\n",
|
|
" Minibatch[ 301- 312]: loss = 0.571917 * 12288, metric = 0.00% * 12288;\n",
|
|
" Minibatch[ 151- 156]: loss = 2.246409 * 6144, metric = 0.00% * 6144;\n",
|
|
" Minibatch[ 313- 324]: loss = 0.603770 * 12288, metric = 0.00% * 12288;\n",
|
|
" Minibatch[ 157- 162]: loss = 2.246180 * 6144, metric = 0.00% * 6144;\n",
|
|
" Minibatch[ 325- 336]: loss = 0.646258 * 12288, metric = 0.00% * 12288;\n",
|
|
" Minibatch[ 163- 168]: loss = 2.244751 * 6144, metric = 0.00% * 6144;\n",
|
|
" Minibatch[ 337- 348]: loss = 0.743782 * 12288, metric = 0.00% * 12288;\n",
|
|
" Minibatch[ 169- 174]: loss = 2.081807 * 6144, metric = 0.00% * 6144;\n",
|
|
" Minibatch[ 349- 360]: loss = 0.766006 * 12288, metric = 0.00% * 12288;\n",
|
|
" Minibatch[ 175- 180]: loss = 2.005488 * 6144, metric = 0.00% * 6144;\n",
|
|
" Minibatch[ 361- 372]: loss = 0.751595 * 12288, metric = 0.00% * 12288;\n",
|
|
" Minibatch[ 181- 186]: loss = 2.073293 * 6144, metric = 0.00% * 6144;\n",
|
|
" Minibatch[ 373- 384]: loss = 0.741732 * 12288, metric = 0.00% * 12288;\n",
|
|
" Minibatch[ 187- 192]: loss = 2.031459 * 6144, metric = 0.00% * 6144;\n",
|
|
" Minibatch[ 385- 396]: loss = 0.800578 * 12288, metric = 0.00% * 12288;\n",
|
|
" Minibatch[ 193- 198]: loss = 2.146479 * 6144, metric = 0.00% * 6144;\n",
|
|
" Minibatch[ 397- 408]: loss = 0.703397 * 12288, metric = 0.00% * 12288;\n",
|
|
" Minibatch[ 199- 204]: loss = 2.148305 * 6144, metric = 0.00% * 6144;\n",
|
|
" Minibatch[ 409- 420]: loss = 0.808739 * 12288, metric = 0.00% * 12288;\n",
|
|
" Minibatch[ 205- 210]: loss = 1.946966 * 6144, metric = 0.00% * 6144;\n",
|
|
" Minibatch[ 421- 432]: loss = 0.878929 * 12288, metric = 0.00% * 12288;\n",
|
|
" Minibatch[ 211- 216]: loss = 1.935109 * 6144, metric = 0.00% * 6144;\n",
|
|
" Minibatch[ 433- 444]: loss = 0.838549 * 12288, metric = 0.00% * 12288;\n",
|
|
" Minibatch[ 217- 222]: loss = 1.906769 * 6144, metric = 0.00% * 6144;\n",
|
|
" Minibatch[ 445- 456]: loss = 0.946877 * 12288, metric = 0.00% * 12288;\n",
|
|
" Minibatch[ 223- 228]: loss = 1.882014 * 6144, metric = 0.00% * 6144;\n",
|
|
" Minibatch[ 457- 468]: loss = 0.883135 * 12288, metric = 0.00% * 12288;\n",
|
|
" Minibatch[ 229- 234]: loss = 1.900080 * 6144, metric = 0.00% * 6144;\n",
|
|
" Minibatch[ 469- 480]: loss = 0.911140 * 12288, metric = 0.00% * 12288;\n",
|
|
" Minibatch[ 235- 240]: loss = 1.947830 * 6144, metric = 0.00% * 6144;\n",
|
|
" Minibatch[ 481- 492]: loss = 0.814674 * 12288, metric = 0.00% * 12288;\n",
|
|
" Minibatch[ 241- 246]: loss = 2.027527 * 6144, metric = 0.00% * 6144;\n",
|
|
" Minibatch[ 493- 504]: loss = 0.846405 * 12288, metric = 0.00% * 12288;\n",
|
|
" Minibatch[ 247- 252]: loss = 1.890244 * 6144, metric = 0.00% * 6144;\n",
|
|
" Minibatch[ 505- 516]: loss = 0.807406 * 12288, metric = 0.00% * 12288;\n",
|
|
" Minibatch[ 253- 258]: loss = 2.031275 * 6144, metric = 0.00% * 6144;\n",
|
|
" Minibatch[ 517- 528]: loss = 0.795118 * 12288, metric = 0.00% * 12288;\n",
|
|
" Minibatch[ 259- 264]: loss = 2.050018 * 6144, metric = 0.00% * 6144;\n",
|
|
" Minibatch[ 529- 540]: loss = 0.828629 * 12288, metric = 0.00% * 12288;\n",
|
|
" Minibatch[ 265- 270]: loss = 1.943776 * 6144, metric = 0.00% * 6144;\n",
|
|
" Minibatch[ 541- 552]: loss = 0.851117 * 12288, metric = 0.00% * 12288;\n",
|
|
" Minibatch[ 271- 276]: loss = 1.986959 * 6144, metric = 0.00% * 6144;\n",
|
|
" Minibatch[ 553- 564]: loss = 0.760867 * 12288, metric = 0.00% * 12288;\n",
|
|
" Minibatch[ 277- 282]: loss = 2.125000 * 6144, metric = 0.00% * 6144;\n",
|
|
" Minibatch[ 565- 576]: loss = 0.727046 * 12288, metric = 0.00% * 12288;\n",
|
|
" Minibatch[ 283- 288]: loss = 2.122426 * 6144, metric = 0.00% * 6144;\n",
|
|
" Minibatch[ 577- 588]: loss = 0.753314 * 12288, metric = 0.00% * 12288;\n",
|
|
" Minibatch[ 289- 294]: loss = 2.060394 * 6144, metric = 0.00% * 6144;\n",
|
|
" Minibatch[ 589- 600]: loss = 0.794642 * 12288, metric = 0.00% * 12288;\n",
|
|
" Minibatch[ 295- 300]: loss = 1.989756 * 6144, metric = 0.00% * 6144;\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"reader_train = create_reader(train_file, True, d_input_dim, label_dim=10)\n",
|
|
"\n",
|
|
"G_input, G_output, G_trainer_loss = train(reader_train)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 16,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Training loss of the generator is: 2.07\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"# Print the generator loss \n",
|
|
"print(\"Training loss of the generator is: {0:.2f}\".format(G_trainer_loss))"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Generating Fake (Synthetic) Images\n",
|
|
"\n",
|
|
"Now that we have trained the model, we can create fake images simply by feeding random noise into the generator and displaying the outputs. Below are a few images generated from random samples. To get a new set of samples, you can re-run the last cell."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 17,
|
|
"metadata": {
|
|
"scrolled": true
|
|
},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAWUAAAEECAYAAADwLSVEAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzsvXlwXNeVmP+91/veaDT2HcROiDsJbpIpipItSqYk27Rc\nkjzyWE5VnNiZqsRTSVwTl10j/Vx2jcfKOOWykylZznhkeeSytdiydhISVwhcBJLYN2LtBtAAutH7\n+vuD1S8ESZAgCBCvxff9RaJfd5/T993z7j3n3HOEVCqVQkFBQUFBFoirLYCCgoKCwv9DMcoKCgoK\nMkIxygoKCgoyQjHKCgoKCjJCMcoKCgoKMkIxygoKCgoyQjHKCgoKCjJCMcoKCgoKMkIxygoKCgoy\nQjHKCgoKCjJCMcoKCgoKMkJ9u79QEITb/ZVXsdLlPu4EHeHO0PN26qhSqUgmk1fpdCeM5Z2gIyxO\nT2WlrKAgE65lkBXkhSiKqNUru5ZVjHKGoNVqUalUqy2GwgqiGGT5k0qlVnycMsooq1QqRDGjRL7t\niKKIxWJZbTEUMhSLxYJer19tMWRLKpUikUis6Hfcdp/yUtm8eTMulwu/34/NZiMnJweHw4HL5cLr\n9aJWq1GpVMzNzeFyuVZb3CXR1NREIpFgdHSUqakpYrGY9Fo0Gl3UZ4iiiE6nWykRFa5Begdzq5NV\nr9cTi8VWfNLfSAaAcDi87J9tNBqJRCKrqt+toNFoEEWRSCSyot8je6OsVqtxOp187Wtf449//CPj\n4+Ns3bqVpqYm7rvvPo4fP05vby9arZZAIEBLS0vGGWVBEDCZTHz1q18lGAzyzjvvcPLkSckoi6JI\nMplc1GfF43GmpqZWUtwVRa/XIwiCpLter8fv96+yVNdHo9EA1zfKgiCgVqvnPWivxGaz4fV6V9Vo\nTU5OLttnqVQqtFotoVAIgKysLGZmZggGg8v2HbcTg8GAxWIhGo3O+53Su/fFztEbIdzuziOLiYCm\nrxFFkdzcXL7+9a/T3NzMzMwMdrsdtVqNRqPhP/2n/0R1dTVarZaZmRleeOEFfv7zn9/w8+UUsRdF\nEYPBwLZt22htbZUM0OUyCoJw0zJnQjRbFEXpgZO+oRsaGjAYDIyPjxOPx2loaODw4cMLfoacxvJK\nRFFEr9ejUqlIpVJYrVbcbrfkhksmk8Tj8RtOZrmP5bXuT7VaTUFBAcXFxRw/fhyVSnXdh43cdYRL\nD99Nmzaxbds2fvazn0l/N5vNAAQCgRvqsRg9ZblSTgtusVgoLS3l+eefJxKJkEqlpB82JyeHI0eO\nUF1dzd///d9TUFDA3Nzcaop901itVsrLyykoKODw4cNEIhFyc3OJRqPMzs4Cl26k4uJiJicnV2RL\nuVpoNBpqa2vJy8tjcHCQvr4+ADo7O9m3bx8Wi4Xm5maOHDmyypIunbVr13LPPfdQWVlJX18fv/jF\nL9BqtTz55JM0NDTQ09PD+++/z+joaMauHjUaDU6nE5fLNW9+3nvvvfz1X/81s7OzfPzxx+zcuZPT\np0/LftdzPR577DHq6+t56aWX5v1dFEWsVismkwm32y39XaPRkEwmb3rnI0ujnGZubo7z588TCASu\nes3r9dLc3Mzo6Cg7d+7EZrMxODh4+4VcInV1dVgsFhKJBCUlJdLATU9Pz3uaplIp3G73dbe9mYbD\n4WDXrl3s2rULq9XKiRMnJKP8jW98g4GBAVpbW0mlUsTj8VWW9uawWq185zvfQaVSMTU1hcViQavV\nUlVVxfe+9z1OnjzJvn37OHr0KH/+85+ZmJiQ/diKokhhYeG8+3Dfvn186UtfYnBwkJ/85CfSPZtK\npXj66af59//+31NYWEh7ezv/8T/+R37zm9/I5sFzzz33EAwGaW1tve51BoOBz3/+86jVasrKyigo\nKCAUCnHfffexceNG3njjDbKysgiFQrjdbuk3SO8c4vH4knYAsk5lSCQS1zTIcCkQ0dfXx86dOxEE\ngd7eXsbHx2+zhDePXq9HFEXGx8fp7++nv7+f999/XzLK8Xj8qidrNBr9VKRL6fV6Pv/5z/Pkk09i\nt9uJRCI4nU4qKyulaz788EPOnj2L0WikqqpqFaW9ebRaLRaLhddeew2LxcLAwABjY2MIgkB/fz9H\njx7lwoULdHd3MzAwQGVlJU8//TQNDQ2rLfp1SSaTzMzMSPfl3XffTWNjI263G51ORyqVYs+ePdI2\nvrKykurqakwmE+Xl5Tz55JMUFRWteH7vYunu7r7hAs5qtbJ582bsdjuPPPIIarWa4eFhpqam8Hq9\nbN26FafTyb59+ygvLycWi0kLiMsfUEtBHr/SEjCZTGzdupWSkhJOnjyJ1WqlsLBwtcW6IYlEglQq\nhc/nAy4NXNpVcS3UajUPPfQQx48fx+PxSJ+RaQiCgF6vp7CwkAcffJDR0VFaW1tpaWnB6/VK13V2\ndmIymbBarbJ3R10egDWZTOTm5uJwODAYDDgcDmZnZ2lvb6evr4+xsTFGR0cJBAK0tbVRVlZGTk7O\nPFeVnEkvjnbs2MGuXbtYs2YNs7OzvP322ySTSdxut2SUTp8+zalTp9i+fTu5ubkYDAYCgcCCvvPb\nneZ6o0QAm83G+vXrueeee+jo6MButzM+Ps6ZM2eIx+N4vV4mJyeZm5sjHA6TTCaXFPdZiIw0yllZ\nWWzbto0vfOEL5ObmkpOTw8TExDx/jlyJxWLo9XoqKirQarVcvHjxqklpMplwOBxoNBrGx8cRRZHy\n8nLJP6XRaJiYmFglDZaGIAjodDoqKiqIRqPMzc3R2dnJsWPHpC2xRqMhHo8jCAI+n2/BXZKc0Wq1\n1NXV0drayvj4ON3d3XR3d0uvq1QqRkZGyM/PJx6P09HRwcjIyCpKvHhMJhP5+fmEw2EGBweZmJig\ntbWVZDJJR0cHACUlJTidTiKRCIFAgGg0yv/5P/+HkZGRjHBFZWVlUVFRQVVVFSUlJVI21JkzZ+jp\n6SESiRCJRKSVdktLy7IvHjLOKBuNRjZu3MhTTz3FXXfdRXl5OUNDQ7zyyiscPXoUrVaL2Wxmenp6\ntUVdkKKiIqqqqojH40xOTl5llLVaLfn5+VIedldXF06nk7q6OsLhMDMzMxlllK1WKw6HA7PZTElJ\nCc3NzRw+fJiLFy+iUqmIxWIIgkBpaSljY2Oo1eqMcNdcufKLx+NMTEyg1+v5+OOPrwpqiaJITU0N\nMzMzfPTRR+j1+nm7BDmSzg4KBALodDq6urpwu93MzMwwMDBwVfA5KyuL3bt3U1ZWxvDwMIlEgn/+\n539GpVItuJpcrlSy5cBoNGIymYjFYkxMTKDT6WhpaWFwcJBYLIZarSYSiSAIAgUFBdIYpoOcy3Hf\nyt4oC4IgpcH5fD6qqqrYv38/e/fuBSAUCtHc3MzAwABw6UctKyuTtVHOzc1lbGyMgYGBa25dZ2Zm\n6OvrIxAIEI/HeeSRR/jDH/7AvffeSzwe5ze/+c0qSL00LBYLjY2NrFu3jnA4TFdXF2fPnmV4eJjZ\n2Vlp62c0GqmsrCQYDErberkbrDQWi4Xy8nJsNhvnzp3D7XZfFdTS6/WsWbOG3bt3c+jQITo6OmQf\n4INL7jOHw0EwGESv10vppy6X65rZQAMDAxw5cgRRFMnKypJ2Dn19fbhcLtmvlqenp3E4HCQSCfr7\n+/nTn/7E17/+daanpzlz5ox0nVarZceOHRQVFdHc3IzX60UQhGVxLco60CcIAjabjccee4xvfOMb\n1NfXc++99/LAAw8wMTGBz+fj7NmznDhxQnJdzM7Ozvvx5Ehrayu1tbUcOHAAs9ksBf/g0vZWp9Ph\n9/tpb28nmUxy+vRpzGYz8XgctVo9LzAmd5qammhqasJisXD//ffz8MMP43A4EAQBjUaD0WgkKyuL\nyspKDh06hFqtZmhoiM7OztUWfdFs376doqIijhw5wtzcHAaDQRpPURRRqVRUVVXx4osv0tzcTG5u\nLgUFBass9eKIRqMMDw+j0+n41re+xY9//GMaGxuZmZm56lpRFLn33ntxu910d3ej0+n47W9/S3Nz\nc8a4L+rr63n88cc5ePAgubm57N+/n9/85jecO3cO+H9uuLy8PAKBAP/6r//KJ598sqTUt4WQ9Uo5\nPz+fxx57jIcffpiysjIaGhqwWq2YzWY0Gg1zc3P89Kc/ZevWrSSTSdkb4zRWq5W33noLv99PRUUF\nTz/9NC+88AJDQ0Ns2rSJ3bt3Mz09zUsvvUQikaC0tJQHHngAlUrFoUOH6OrqWm0VFs3hw4dJJpM8\n88wz1NfXMzg4yIEDB7Db7dhsNjZs2EAikaCvr4/BwUFGR0czwnVxOYcOHZLyc5PJJFNTU5IOaR+r\nyWRienqaYDBIS0uLrA1UunZK2ldqMBj4b//tv0luiIXcDRUVFTz55JNUV1fT3d3Nyy+/zG9/+9vb\nJvdykDawsViMjRs38vOf/5wf/vCHvPDCC7S0tGA2m2loaOAf//EfOXLkCD09PdKObrnGVNZG2WQy\nUVNTw9zcHCqViqKiInQ6HclkkuHhYU6dOsXevXv5/e9/nxGGShAEGhsb0Wg0DA8PU19fzxe+8AWK\niopobGzkM5/5DDU1NdTV1eH3+zGbzRQWFrJ7924AXn75ZcrKyvjBD37A3/3d362yNosjHo9z7tw5\n/uVf/oW9e/eyceNGNBoN9913Hz09PRw9epTc3Fw2b97MCy+8wPe//30uXLiw2mLfFFdOxlQqhVqt\nJpFI4Pf7qa2tZd26dfzkJz9BrVYjCIKs/KhXcmWANRwO88tf/pKmpiaGh4clV6EgCKhUKkn/kZER\nPvzwQ8rLy3G73bz66quLrtkiF9JpuKdPn+aNN97gwQcfpLCwEKvVSjKZZHp6mp6eHv7lX/4FURSJ\nRqNSRtVyIVuj7HA42LhxI01NTQSDQUZGRnjnnXfo6OggHA4TCoXwer3s3r0bl8slm8T065FKpRgZ\nGcFqtRKNRlmzZg2f+9znSCaTWCwW6uvr8fv9/O53v2NgYIBIJMKGDRvYvn074+PjhEIhBgYGMsbX\nejkej4e3336biYkJzp49SyAQoL6+njVr1hAKhfj4448xGAwZMY6LIT1Ry8vLycvLo7W1lVOnTkmr\nMDlz5QNDrVazY8cOJicnKS8vp6qqivHxcZLJJOvWrcPj8fDJJ59QUFBAKpWivb2dtra2a7o4MoGx\nsTFmZ2fxer1EIhGam5vp6+ujqKgIh8NBLBZDpVLxl7/8BZVKhclkWtaTirI1yrm5uZSUlNDb2ytV\nhPN4PJw4cYLJyUl0Oh1lZWVMTExgMpkwGAxS4RM5MzMzQygUIpFIMD4+zunTpyksLKSoqIgtW7Zw\n7Ngxzp8/T1tbGxqNhpmZGe666y7WrFlDPB5HFEXWr1+PVqvl448/lvU2OE0oFGJ0dJTR0VFqa2vJ\nzs5mamqKtrY2YrEYdXV1UvEag8FwwzoJmUAqlaKyshK73c7MzAx+v1/2edcLkUwmGR0dpa+vj2Aw\nSCwWIxQKUV1dzcGDB3n//ffp7e2lpqaGrKwsTpw4wbFjx1Zb7CUTCASk3cLlu7YNGzaQl5fH7Ows\nWq2WvLw8TCYT58+fl/Lwl6Ogk+wCfbm5uTidTuLxOL29vbS0tBAKhejs7GRkZGSe4RUEQSr+nkm1\nlo1Go1TV7vz587zxxhts2rSJtrY2Tp8+Pe+QyPT0NK+//jqiKBKPx4nFYmRlZcmitQ1cWkXd6Hf3\n+/2Mj48zPT1NX18f0WgUQRDo6emho6ODoqIidu7cec3TjJmMIAiYzWbq6urYsWPHaouzZBKJBC0t\nLUxOTtLa2sonn3zC7OwsRqMRh8PByMgIGzZsoKSkBFEUuXjxIqOjo6st9rITCoUIh8PYbDbKy8ul\njLDlLuUpu5Vyuh6EWq1Gr9fjdDqZmJjgxIkTXLhwQdompA1yfn4+J06ckHIHMwGTyUQqlcJoNKLX\n6+no6MDlcvHjH/943kECi8XCrl27mJmZoaOjQzrl9tJLLzE9Pc2mTZs4d+7cqu4QRFG8oT/NZDJh\nNBqZnJzk7bfflv6em5tLQ0MDRUVFJBIJOjo6mJ6e/tQY5kgkQnV1NTt27OD48eOrLc4tkXZppCve\nJZNJxsbGeOONNzh16hR/8zd/w/T0NP39/Xg8nowL1t4Io9EopT6mi5+dPHlynitxuXZCsjPK6cI0\nBw4c4Fvf+haiKPLqq69SVFSEzWaTjJbNZuPBBx+UztvL3U93OcPDw8ClnE673U51dTX/5b/8l3k6\nqFQqiouLefrppxEEgaNHj2K1WqmpqWFwcBCLxcITTzzBD3/4w1U1yosJ5DidTgoKCvB4POTl5TE1\nNYVWq2XPnj3s37+fubk5/vSnP6FSqWRTH2E52LJlC3V1dZw5c4bnn39+tcVZFrRarXSqdGpqil//\n+tdSCmdHRweDg4NMTU19KioaCoKAxWLB7/dTV1fHQw89JAU7v/nNb65YsFaW9ZQ1Gg1qtZq1a9fy\nve99D6fTyS9+8QvefvttKR/ZYDBQW1uLwWCgo6PjpuoHyLkGb5rs7Gw2b97M7t27MZvNzMzM8MYb\nb2A2m6Wc1+shx/q06QpqL774IsXFxTzyyCPU1tZy4sQJnn322SXJIMexTHd/KSws5ODBg2g0Gp59\n9tklyyrHsdy5cydr1qzhlVdeWRYDLEcd9Xo9+/bt46OPPuLb3/42jz76KCdOnOB73/seoigyPT19\n04Z5MXrK0ig/8cQTfOYzn0EURerq6igrK6O5uZmf/vSnnD59GoA1a9bwox/9CI1Gw//8n/+TkydP\nLrpWghwn8rU+w2w2c9ddd/Hzn/+cwcFBnn/+eem01I1WqHK8ydPBkD179tDV1UVdXR21tbU0NzdL\n43qzyG0s02Uet23bJp3wa29vl0p5LgU5jmVpaSn3338/VVVV/Pf//t+BS+62cDi8pF2rHHVMN9mY\nmpriwQcfpLKykkAgIC0Y/+7v/u6mM6Ey1iina+2Wl5dTVFTE8ePHmZmZoaenh0AggNfrZWBggOLi\nYmw2m5Qmttinltwm8kKUlpbyt3/7twSDQV599VU6OzuZm5tbVMaFHG/y9GnE2dlZ6dh1Z2cnfr9/\nyastuYylIAhkZ2ezY8cO1q9fT319PVarlbfffpvDhw+jVqs5e/bskmSQ41hu2rSJ2tpajh07RiAQ\nwOPxSLUfliKvHHWES/dsPB5n+/btUiC6oaGBDz/8kLGxsZuOf2Rs55Hz58/j8XhwOBwYjUa+/OUv\n87//9/9maGiIVCpFNBolEonQ19cnVRb7tAUWcnNzKS8v56WXXmLz5s1oNBoqKyuZmpri4sWLqy3e\nkkgkErjdbvx+P+fPnycajcq6RsnNkC5IY7Va2bJlC0VFRVJaY2FhIR9//PE135cJBbQ0Gg0PP/ww\nH3zwAV6vF71eT0lJCTk5OQwNDaHVapdsjOXEww8/zNDQEG1tbdLf0gugzs5OKisrMZlMHD58+Kay\nS9Lt6xYb+5GlUfZ6vQQCASnNzWw209vbK/mN1Wo1RqNRypn8tGEwGFCr1QQCASorK7lw4QLj4+PE\nYrGMbqeTSqWk7V6mNbddDPF4HI/Hw5kzZ3jzzTe5ePGiVExroYMUyWRS9qfe0idoL29mOzw8zNjY\nGKlUasW7O98uRkdHF4xNzc7OMjY2RmVlJWvWrLkpo5xKpW5qRS1L98WNUKvVaLXaJZ/+ksuWdyHS\nBW20Wi333Xcfhw4dYnZ29qYGVq7bweVGLmOZrnSXm5uL0Wikr68PQRCk3N1bKbAkt7F0OBzSv0tL\nS5fslrkcuemYRqvVEovFJPnSZXebm5uXJEPG+pRXGrlM5JVErjf5ciOXsUxXg1uJnZvcxjKdhupw\nOFi/fj1vvPHGLX+/3HRMX6/X6wmHw8smn2KUF0AuE3klkeNNvhIoY7k8rLaenzYdFyp4vxg9M+Nc\nsoKCgkIGka7hshRu+0pZQUFBQWFhlJWygoKCgoxQjLKCgoKCjFCMsoKCgoKMUIyygoKCgoxQjLKC\ngoKCjFCMsoKCgoKMuO21L1Y7SR2UAwfLxZ2g52J01Ol0ACtWA+JOGMs7QUfI4CpxCplF+ma/U1Pe\nV6oDhcKdSca4L1QqlSyedAoKVxKPxzOiq7jC8rOYxsE3i+yNsiiK5Ofno9VqV1sUhQVYbC3ddCeH\n5ejDp1KpyMrKuuXPWQ4+DbWEFZbGSiwWV8V9ke6AvJgbWaPR0NjYSHt7O5OTk2g0GnQ6HdFolFQq\nRSgUUiZEhqDVatmyZQt9fX2Mj48TCAQQBAGDwYBWq8Xv90vjeiN0Oh3FxcW3QWqFxeJwOBAEgXg8\nTjgcvsrHni72HovFMnZnYTAYcDgcJBIJPB7PisQRVqVKXFZWFqFQaFEtgNJF7vfu3csnn3xCWVkZ\na9eupbe3l2g0yrFjx276h5FDcGilkWPgxGAw8LnPfY4vfvGLvP7667z33nsYjUa2b99OWVkZ7733\nHj09PTdVJ1sZy+VhOfT8m7/5GzQaDW63m7a2NqmDR1r+kpISCgoKGB0dvapIfKbouHPnTv7Df/gP\neDwe/umf/omhoaGryrWmW0hdC9mW7lzMSjknJ4eCggISiQTf+MY3aGlpweVy4XA42LhxI1lZWfzo\nRz9idHSU0tJSvF7volvqyH0i19TU4HQ66erqwuPxAEi97eSiI9ycniqVCqfTyYEDBzhy5AjDw8PU\n1dVRUlJCR0cHLpeLYDDIunXrCAaDDA8PMzc3d8PPlftYLgdyG8uFsNlsPPDAA9TU1BAIBJicnMTr\n9fKnP/0JuGSsVCqV1OvucjJBR71ez1133cX69et59dVX8Xq9N10/W7bZF4uJVvt8PqLRKDqdjsHB\nQU6fPo3H40Gj0XD+/Hny8vLYsGGD1LRxOdqcrzaPPvooubm55OTkEAgE8Pl8klF2u93E4/EF67TK\nEa1WSyKRYO3atWzevJmuri7OnDnD4OAgBoOBiYkJxsbG8Pl8AMRiMfr6+kgkEovuZyZ3srKy0Gg0\nTExMXPe6662u5IxGo6G4uJiKigrKyspwuVx8/PHHGAwGiouLWbduHYcOHSIQCGR0QPTuu+9m48aN\n5OTkMDk5ueTO5ItBlilxJpMJtVqNTqdjzZo1vPPOOwwODkpuivHxcTweD/v37yeZTOL3+zM2Lclu\nt7N161buvvtuVCoVs7OzDA4O0t/fj9frpbi4mFQqxa5du5iZmaGtrQ23273aYi+KRCJBXl4eOp2O\nmZkZ1q1bx9DQEF1dXUQiEYLBIOFwGLvdzu7du3nvvfeYm5sjmUxm1HhqNBrg0kNFo9Gwbt061q5d\ny9GjRwkGg1Iec5q6ujqmp6fnGepM0vdyjEYjGzduRKfTYTQaGRkZAaCiooJ4PM7Q0BAmk4lgMJgx\ni4krKS0tZdu2bWzevBmfz8f58+dX9PtkaZTT/c6sVivJZJKurq55N63dbqe2thatVovFYskIoyyK\nIlarFZ/PN0/W7Oxs7r//fux2Ox0dHXzyySdcuHABt9uNxWJh27ZtOJ1OvvjFL3LkyBH8fj95eXnz\nOu7KlfQWdWxsjMHBQbZv3y5NzkAgIF2XzqSwWCzEYjFCoVDGNuPUaDTU1NRQW1vLxYsXicfj5Obm\nYjKZuHDhAnDpXrDb7dhsNgB6enpkf/9eC7PZTGlpKTMzM/h8PpxOJxMTE5KrKplMMjIyQllZGR6P\n56Z6TMqJdLdujUaDXq/HaDRyzz33cOLEiRVpeitLo+z3+9FqtUSjUU6ePHnV61arlaqqKnJycsjO\nzmZqakr2Xa1VKhVWq5VgMCgNZF5eHlu2bCE3N5cf/vCH5Obm0t3dPW8lLIoiTU1NFBQUkJWVxbp1\n60gmkxlhlOGSQU6zUC+3WCzG5OQkdXV1DA0NZZwrKr1Cttls2O128vLy6O7uJplMkp+fT2lpKclk\nUjLK7e3tbNy4kU2bNuHz+ejt7cVgMGRcJpHT6aSuro533nkHr9dLTU0NoVAIn89HKpUiLy8Pn89H\nZWWlLHzzN8vli8N0vz6tVovD4cDhcHD69OkVMcqyzVMOh8MLtvseHR3l6NGjZGVlMTQ0lBH+x1gs\nxtDQEKIoSk029+7dy7e//W18Ph8ajeaqbfvc3BxHjx4llUphsVh45pln2LVr14pvn1aKK5PsBUFA\nrVbjcDjYsmULjzzyCBqNRjLKmTSRs7KyaGxsZM+ePTz66KPU1NQQjUZ56623+OlPf8prr70mXavT\n6aivr8fhcDA6Ooparaa4uBiz2bzsBxFWCq1Wi8FgQKPRUFtbi0qloru7W4oPwKVdYH19PcePH8/I\nw19arZZNmzbxzW9+k6997WusW7eO8fFxfvWrX/GTn/wEv9+/4HtvRVdZ3gEajYZ4PM7MzMw1Xy8s\nLGTXrl0IgsDatWul7rqZQDgcJplMUlhYyMzMDD/60Y/47ne/Szgc5syZM/MCCBqNhtLSUg4cOIDd\nbmd0dJTjx4/T0tKyihosDZ1OR0FBgXSzqtVqzGYztbW11NXV8c///M8MDQ1JKw+9Xo/JZFpNkReN\nXq9Hp9ORTCYJBALMzMwwNTXF5OTkPDdNmt27d1NYWEhHRwcnT54kFovR29vLpk2byMnJWQUNbp6m\npiaampp45ZVXOHXq1DzXhMFgwGq14vF4OH36NN/4xjfYu3dvRs1TuOQm/ad/+id++9vf8tFHH+F2\nuwkGg+Tl5d3wvVarVYo13CyydF9cL0Lb0NDA3XffjcPh4N/+7d/o7u6+5o0vZx544AEmJiY4duwY\nWVlZbNq0idbW1mtuX+PxOFqtFo1GQygUYm5uLiN9c9FolImJCVKpFMXFxXi9XgKBgHRic3Z2dt4u\nIRKJrMjWcLkQRZHvfve7/P73vyeVSklpmsPDwzz77LOMjY1JmTNX8vHHH1NeXs6aNWuoqamho6OD\nZDLJ6dOnM2LXB3DmzBlGR0cpLy+np6dH+rvFYqGpqYnCwkI6OzuZnJzk5ZdfJhwOZ8w8NZlMGAwG\n4vE43d3dfOtb30IURUZGRtBoNNTX19PR0XHV+9RqNQaDgbm5uVuKc8nCKGs0GgoLCxkdHSUej1/X\nr5ZIJIiJ/tixAAAgAElEQVTH44RCIdrb268b1ZVr+lhnZydVVVUUFRWRSqUoKChgenqanp4eKcBV\nWFjI/fffz2OPPQZc8kP+/ve/56OPPlpN0W+ayzMT1Go1+/fv59SpU4TDYaxWKw6HA5PJxF133UUs\nFsPpdDI9PU00GkWtVt/UQZLbSSqVknypd999N9u2bWNkZITBwUH8fj8ej2fBh0pBQQFdXV3Mzc3N\nix8sJi9bLmRlZZGVlUVfXx9qtZotW7Zw7tw5cnJymJiY4OLFi0xPTxOPxzMmWyhNJBJBp9NRVVWF\n3W4nlUrR19fH5OQk/f39nD59+prvSyQSkuvtVhZOsjDKqVSKSCSy6PoJfr8ft9udEU9eq9WKVqud\n55Zwu92YzWaqq6spLS0lNzeXubk5du7cydmzZ3E4HOzdu5ft27ezc+dOZmZmuHjxIqdOneLixYur\nqM3Nc/mYplfF6SDQpk2baGhoIJFIUFhYSDwep7a2ltnZWYaGhmS9I0ilUrS0tFBUVITH4+Hs2bOI\nokhFRQUnT568SnZBEKSDE9FolN7eXnw+X0bm7apUKimXPB0jqaioYHR0lIaGBi5evDhv9ZxppI+J\nq9VqtFotH330ER0dHdjtdrKyssjNzWVwcPCq96VSqWVJOJCFUY7H47hcrkVdazQaiUQinDt37obX\nymGVnK7VcTkqlYrBwUEEQaCgoABRFCkqKuLgwYO8++67lJaW8sgjj6DT6aSTj729vSuasL5SpI2O\nTqcjPz8fl8tFXl4earWaiooK8vPz6enpIZFIMDc3R1VVFX19fczNzWGxWOjv719lDa6PKIoMDw8z\nNjaGw+EgPz+f6enpaz5QNBoNu3fvpr29HZ/Ph8ViQaVSLfrelwsqlQq1Wo3dbic/Px+z2UxVVRWh\nUAij0ZiR6X1XEovFcLlcnDx5krfffhtBEMjPzyc7O1uq8ZGTk8Ps7Oyyu9lkGehbCK1WSzweZ3x8\nPGNWjB6P56pz/ukteXrVPz09TXZ2thS4LC8vJ5FIkEqlJH2PHDkiJeZnGoIgkJ2dzc6dO4nFYuTl\n5REIBDh9+jQfffQRbW1teDwe6ViuwWBg7dq1rF+/frVFvyaCIKDT6RAEgenpaRwOBxUVFXi9Xt58\n881rrn7TC4RvfvOblJaWIooiRqMRi8Vyu8W/ZeLxOKWlpTQ1NVFWVsYzzzzD3Xffzb333sv58+fp\n7OxcbRFvmWQyicfj4Ve/+hWtra1SOlxbWxtvvfUWgiBQVlaGXq/HYrEsaxBTFivlxSCKopSEfqWR\nuxK5+pLTCIJAMBhEpVLR2NjIxo0bGRgYYGRkhNLSUiwWCx6PR6qeNjk5id/vz8itLiClvaVXU4cO\nHUKn0zE+Pi75yFUqFTt37uS3v/0tGzduJB6P89prr8lyLDUaDeXl5QwNDbF+/XpcLhdarZbHHnuM\n119/nXPnzl21WkzXe3nzzTelVMBMWVhcSTKZ5MiRIxw9ehRRFHnttdf4xS9+QX9/PzMzM7J2Oy2W\nRCKB1+vF6/UC8N577yGKIk6nE7j0G5w5cwZBENizZw+RSIQPP/xwWb57VQoS3QiVSgXMd5Zv2LCB\nzZs3S1uKhbbyFouFwsJCurq6Fvz81S5iU1RURCAQIBAIYDQapR3A66+/Tnl5Of/6r/+Ky+XiW9/6\nFl6vlyeffJKSkhL6+/vp6+tblAxyK/Ci0WgwGAzEYjEeeeQRzpw5c9VJNq1WS05ODtFoFIPBgNPp\nxO/3093dveDnrtZYarVavvzlL3PixAkGBgaksgChUOgqv2J2djbV1dVoNBrMZjNFRUV88sknnDt3\nblEHZeQ2lnApDTA3N5e6ujo+97nP0dfXR21tLb/61a84e/bsTcssRx2vRK/XI4oiwWAQtVrNF7/4\nRfLz86XUv/b29ht+hmwLEt2Ia/mkduzYgcvl4sKFC/MS1K8kGAzO2+YbjUYSiYSsju16PB6pOEs0\nGiUvLw+z2cx3v/tdUqkUtbW1PPjgg+j1eiYmJvB4PExMTGTcSbft27czMTGB1+vFbrczNTXF97//\nfV555RWGh4evGudoNMqWLVsoKSmhs7OTlpYW2a66otEohw4dko4PL3SPmUwmdu7cyb59+/j+978v\n1Y9O147OJNauXcvo6KjkR52YmCCRSKDRaPjkk0946qmnaGpqIhAIXPdBmqmEw2EEQcBms1FfX8+J\nEyfQarWUlJQsS+OGNKtilJ1OJ4FAYMGczGs9TVpaWqRTfte6mdNBsWg0SjAYxGAwEA6HicVistv+\nXm5c04dkgsEgbrebWCxGfX09NTU1TE5O8vrrrxMKha57ekiujI+P4/f7sVqt5OfnMz4+zpkzZ+jr\n67sq1U2j0fD4449jNBql4vdms3neMW25cSM3GlzKvonH43R0dHDXXXdx7tw5XC6XbB8218Pj8bBh\nwwaGhoYYGRlBEARcLhetra00NDTQ3t7O1NTUgoe+Mh21Ws26deukuhfp+erz+ZZ1PFfFKN+MkVSp\nVJSVlXHx4kWSyeSCK97NmzcTj8el027p75B7TYz0oDY1NTE7O0tFRQVms5ljx45x7tw53nnnnYw5\nUHAlaZ+pKIpMTk6SSqUYHR29akzS5R/379/Pa6+9RltbW8btChbCarXi9/t5//33F3USTI4IgoDV\namVycpLy8nIEQaC6uprNmzfj8XgYHBxk/fr1vP322/T398tqV3orpAtHmUwmQqEQTqeTDRs24HQ6\nmZyc5J577qG1tXXZ87BXxSgvdNLpWqhUKikVpa+v7yoDlQ4ErV+/nkgkQktLC6lUKqMmtVqtpq6u\njp6eHoqKihgeHubw4cN0dXWh1WozOsVIp9MxNzfH1NQUFouFkpIS2tvb5638TSaTVHQp7X7KpIMU\nC5E+OCMIAoIgcOTIkVWWaOmkU1HPnDlDPB7n85//PF/60pcYGxvj9ddfZ2RkhLGxsU+NQQak3ZrD\n4ZDqeJSUlDA2NibN2c7OzkU3nlgssvQpX040GqW1tZVNmzZhMpmkhHuVSoXZbCYQCGAymRgYGJAc\n8JmWpRAIBHjhhRcwm81cuHCBYDAolXzMycmhra1N9iv+hbDb7QiCgNvtxu/38+677xIIBNBoNFLa\nn81mY8uWLeTn57NmzRrOnj37qTDKNptNyrdev359xvpZU6kUbrcbk8lEPB7HarUSi8VobW1lYGCA\noaEh2tvbM3rxcC0SiQQjIyMMDQ3xX//rf6WxsRG9Xs/27dvp6uril7/85Yo8hDIiTzldwjOVSkkd\njJ1OJ3/1V3+FTqfjqaeewu/3097eTkFBwSpLe/OoVCpKSkoIhUJ87Wtf4/Of/zzhcJihoSFOnz6d\nsQYZLp1edLlcpFIpkskkk5OThMNh8vLyyM7Oxmg0kp2djcfjkUpbpg/bZFpVsSux2+1s2bIFm802\nr0pcpiEIgnTcuLi4mP/8n/8zBw4coKOjg6GhIT772c8uufiOnBEEQTrgY7PZOHLkCM899xwvv/wy\nH3zwwYrtCmSXEmez2YhGo9f0o6rValKpFIlEArVajc1mY2ZmBrvdLnXPFUXxhkZstVPirsRgMPDg\ngw/ywQcfoFaricViUn7kUpF7ilF6LNVqNQUFBTQ2NtLT04PH45FagS0GuY3l5ZjNZrRaLZFI5JZK\nAqz2WGo0Gu6//36OHTtGYWEhu3fvZtu2bUSjUV577TV8Ph+1tbW8/vrrS97Kr7aOCyGKIgUFBTzz\nzDPk5eVx4sQJmpubGRoaWpIMsm2cej3SdYVXMjott4msUqkoLCzE5XIRi8UoKiqitLRU2iIuBbne\n5FciiiI6nQ6z2czMzMxNu57kNpaXI4oi2dnZaDSaW8oiWe2xFASBwsJCJiYm0Ov1ZGVlsXXrVnJy\ncvjjH/9IKBQiNzeX4eHhJa8eV1vHa5Huvn7y5Enp1N709DTDw8NLdpFmZJ5yLBZDpVJJRU/uBBKJ\nBMPDw2i1WkRRJB6P4/f7Mzbr4mZIJpOEQqFPpa7pbKFMv4/TWTNwaX7Ozc0RiUSwWCxS5sH1zg5k\nKolEgvHxcbxe721NzZTdShmQuhSsVMBOLqsrg8GATqeTOqzo9XpisRiiKCIIwi0dLpDjymMlkMtY\n3gwWi4VoNLroVeWdMJZ3go6wOD1lGehL10z+tGMwGKTAJVw6VJJIJIjFYhl32kth8VitVvR6/WqL\noSBTZLlSXmkycXV1sygrj+XhTtARVl/PO0FHkGmgT0FBQUFhYWTpvlBQUFC4U1GMsoKCgoKMUIyy\ngoKCgoxQjLKCgoKCjFCMsoKCgoKMuO0n+lYjLSX9nelEEyWNanm4E/S8E3SE1dfzTtARMvjwyHKT\nSqVk131E4c4k3ectExAEIWNkXS7Sta9XkzvrF88w1Go1ubm5V00Ms9lMVVXVKkmlcCsoiwOFG6EY\nZRmjUqkwmUxXPbm1Wu2849mZSm5uLsXFxdL/7Xb7sjaglCORSCRjisGna2AvlnRBLaPRiNFoXNR7\nBEGQ1ZFzOeyqM/aYdboN1FK4XX7Iy+s/r9T3LKTL7fLRqVQqksnkVd+n0WiIx+NotVoEQZDacwmC\nQHFxMRqNht27d+N0OnnzzTcBKCoq4vTp0/Mab95onDPFp6zRaMjPz0ev10tNR1Op1KKq42WKv7W4\nuJhoNEp+fj7hcJju7m7J6IZCIanbzOWGXq1W43A4lr3P3bVYqo5arZZUKoVOp0MQhFvqipORpTuv\nxZWBOo1Gg1arvaXC4beDvLw8YrEYExMTS3r/9QxS+mYPh8Or+mS32WwEg8GreiIWFBQwNTVFdXU1\noihy5swZBEHAYDDws5/9jKqqKkRRxGaz8dxzzzEzM8OLL75Ib2+vZJTVajU6nU7247wYysvLef75\n52lsbOTJJ5+USrQeO3ZstUVbNtavX09BQQFGo5G2tja6u7sxmUysXbuW1tZWSktLmZiYmGfU4vH4\nkufHSpO2O+Xl5cTjcRoaGlCr1bz66qsr+r2yNsoqlQqn08kjjzyC1Wrlz3/+Mx0dHcRisXlV5HQ6\nHTt27CAcDnPixIlVlHg+Lpdrye+tqqpi3bp1/OEPf7jm6+nmsKu91Zqdnb2mDGNjYyQSCfr6+uY9\nVEOhECqVipmZGfR6PRaLhenpaXw+Hy+++CJer5f169cTCoXo7u7O+FrEAPX19Tz++OP4/X6i0Sj/\n43/8D1QqFd3d3Wg0Gpqbm4FLK83p6WmCweAqS7w0Dh06xP79+4nFYiQSCbZu3YrdbufDDz8kkUgw\nNDSUMdUfKyoq+MpXvkJvby9HjhwhHo+Tk5NDTk4OVVVV5OTkcOrUKaLRKCqVCmDZ7lVZG+W8vDwe\neughAoEAra2tjIyMUFVVhclk4pNPPpGuq6qqoqqqipGRkVWU9mqWMkgWiwWLxYJWq6Wvr++61662\nQQYW9DmmJ18gEEAQBMrKyjh48CBut5v+/n7y8/OpqKjA6/XS2dlJT08PLpeLYDDIxYsXpd9ODjre\nCnV1dZSXl9PT08PDDz/M66+/jtFoJCsrC7/fjyiKiKJIMpnE4/FkZMlWlUrF2rVrqampQaPR0NXV\nxcDAACaTCafTSXV1NdPT00xMTGTMeMbjcUKhEOXl5Xz00UfMzs7y3nvvodPpSKVSPPTQQzQ1NfH7\n3/9eagCwXMjaKEciEaampti4cSOtra1EIhF8Pp9UHFytVrNv3z527txJNBolGAyyfv36eQY707Db\n7Wzfvp2ioiKmp6fZt28fv/zlL/H7/ast2pJIB078fj/nz5/H5/PR29tLVVUVVquVmZkZ/H4/RUVF\nUh3tdNH/TCc7O1vScWRkhM7OTo4fP051dTUtLS20t7fj9Xqpqamhs7MzI7uvGAwGGhoacDqduFwu\nqcdiMBgkGo0yNjbGzMwMwWAwYwKcgiDg9Xo5fPgwjY2NRKNRYrEYw8PDwKV62B9++CGpVIpAILDs\nDxrZGmW73Y7T6aSvr4/i4mJp5XS5/0kURTZu3Mi9996L2+0mNzcXu92esUY53Ztv8+bNlJeXc/78\nedauXcsLL7xw1bW3EuhcDTweD2+99RZwKevC7XYzOzvL9PQ0eXl5HDx4cJUlXF5sNhs6nY6cnBwc\nDgfDw8OMjY3R3d3N7Owsvb29DA4OYrVaKSoqWm1xb5p0kDedd+33++nr68Pj8Uh/i0ajDA8PMzc3\nl3Ed2QOBAOfPnycUCl0V0wiFQnR0dFBVVUVDQwM9PT3L6heXlVG+PJJfUFDA9u3b8Xg8vPTSSyQS\niWsaoXA4jEajoampiampKTwezypIfuuo1Wp27dqFVqslmUwyMjLCiRMnaG9vv+YKKtOM8uVs3bqV\n/v5+6WaPx+N4PJ6MWUldj3Rb+h07djAyMkJ+fj5r166VYgADAwO0tbVJ1/t8vozsb6dSqcjNzaWo\nqIhz586RlZWFRqNBrVYTDocxGo1oNBpmZ2elNmeZwuUZU11dXVe9bjab2bVrFw8//DDhcJhf//rX\nn16jbDKZCAaD6HQ67HY7hYWF3HXXXRw7dow1a9Zw8eJFpqampOsFQSA7O5vZ2Vny8/MJBALzWn+r\n1eoFjbmcEAQBo9FIY2Mj3d3dnDx5Uto+LYRcDNhiHw6iKKLX6zEYDOzbt4+BgQE++OADLl68CEBz\nczNqtVryry6GdIBFTmi1WjZv3swTTzxBX18fH3/8Mc3NzbhcLhwOR8YG8a5Ep9Nx11138dnPfha3\n243b7SYWi5FKpaT72WQyMTs7m3E6L3RPq9VqBEHAbDazc+dOTCYTRqMRg8GwrN8vK6OcXjE888wz\nrF27ljfffJOjR49SXV1NV1fXVdsIlUrF9u3bmZ6e5vz587hcrnn5jnV1dYyMjMjeR5lKpfD5fDz3\n3HPs27eP/fv3MzIycl2jLBfSq6AbRdWdTicPP/wwX/jCF6ipqUGr1dLU1ERPTw8qlYq///u/54kn\nnuDdd99dVOBEp9NRUFCwXGosC+kJ+9nPfpYf/OAHfPnLX2br1q0A5OTkcOLECdkvEBZLSUkJiUSC\nf/zHf6SyspKSkhL6+/ulfORgMHhbco+Xm7SBvdbutLq6GpPJRCQSYf/+/fT19fGHP/zhmqvpW0E2\nRlkQBHbs2MHmzZsZGBjgL3/5C9FolIKCArq7uwkGg1fd0PF4nHfffZfa2loMBgOBQGBevmxfX5/s\no9lOp5OtW7ficrno7Oyks7MTr9c77wDF9bhWQv7tZLEn1ILBIK2trZw6dYpnn32WhoYGGhsbqamp\nYXx8nGQyidfrXfQ2Nx1EkhNZWVl85jOfoba2lq9+9as4nU48Hg9qtZrp6elPhUFOn9qbnZ3FYDCw\nfv16tm/fzvPPP08ymSQcDhONRkkmkySTyYxzs12Zb5+moaEBURTJzc3l4MGD/Nu//Rsvv/wyo6Oj\ny+5+ko1RTqVSDAwMEAwGCQQCOBwOCgsLGR0dRafTLRiZjkQiZGVlMTs7S39//7xVcSZEs9MBkoqK\nCvr6+hgdHWViYmLRxmk1DTIs3o2i1WqprKzkgQceoLS0FJ1Oh8FgIBwOS7nMVqsVjUazqM9LpVKy\ne+BGo1HGx8c5d+4ca9asIRgMYrfb2bp1K3q9nv7+/ozyrV6LeDxOSUkJGzZsIC8vD5/PhyiKlJeX\nMzk5SSgUmmfYMskgw8Lyzs3NsXXrVvbs2YPFYuHVV1+lu7t7QSN+K8jGKAOMj48zPj6O0WikqamJ\nTZs2YbVaGRoawmw2EwwGpZvaZrNx7733snXrVnQ6HRcuXKC7uzvjAn3hcJj+/n4EQSASiRCJRMjN\nzUWv19Pd3X3D98vFt3wj7HY7u3btYsuWLdJEzsnJweVy8e6771JVVUVxcTF5eXl4PJ4VudlXmkQi\ngc/nY3h4GLvdjsvlIjs7m/r6enQ6HWfOnGFgYCBjDlBci2QyiV6vx2QyIYoiPp+P999/n9zcXAoL\nCwE4f/48AwMDqyzp8lJWVkZpaSlqtZpYLEZ2dvaKVdCTlVFOk765JycnSaVSGI1GotHovInqcDh4\n/PHH2b59O+3t7YyOjuJ2uzNidXwl8Xh8nl/KYDBgNptXUaLlJxaLMTo6yiuvvEJlZSUWiwWA1tZW\nPvroI/bs2cP4+DjxeJyCggIcDgepVIrTp0+vsuSLRxRFdDodyWQSn88nZSKMjY3h9Xql2gmZjlar\nxefzEQgEsNvtjI2N4XQ6KSwsJBAIEAqFCIVCt3SiVW44nU6ysrIQRRG1Wo3Var2zjLLBYCAWi3Hq\n1Cna2trYsGED58+flw6NwCUftFarlbbv4XB43uuZTE9Pz6di8l7O8PAwzz//PADf/va38Xq9HDt2\njN7eXgoLC3E6nfzsZz9Dq9Vy3333sXPnTkZHRzPKKCcSCQKBAFNTU+Tm5lJZWUl/fz/vv/8+Y2Nj\ndHZ2rraI12Wh4lJXotPppCyZ0tJSmpqacLlcTE9P85e//IXi4mIqKiqua5TTtZrlfIxepVKRl5dH\nIpGgq6uLxsZGysrKOHfuHMePH7+mvRFFEavVis/nW/IuVrZV4j73uc+xc+dOvve9713zdYfDwYED\nB1i3bh1ms5nW1lY++OADent7b/jZt6OymCiKSy4DaLPZUKlUTE9PL1mGTKksdvlnqdVq/t2/+3c8\n+uijHD58mB/96Ec3nLRyrRJntVoJh8M4nU50Ot0tbedv11hmZWURCAQW5avPycnBYrEwODiIRqPh\n7rvv5uTJk4uuoGYwGNBqtXi9XkCe96vNZuM73/kOwWCQ/v5+BgYGGBoauu7DxmQycf/99/P+++9f\n87fI6M4jhw8f5h/+4R8WfN1oNLJhwwb27dvHli1bmJ2dZXBw8PYJeAMWs+K4Fg6HA0D2aXzLSbrw\n1DPPPMNXvvIVEokEOp2OPXv2rLZoS8bpdKLX63G5XPNy5+XM7OzsooOn6ePU27ZtY9euXRw/fnxe\nKYB0samFCIfDsj804/P5+Id/+AcKCwvZuHEjarX6hodEgsEgb7/99i2VRZCl+wIuDdr1gj3T09P8\n+te/5tChQ9TU1NDf35/RAZQ0c3NzN11cPJNRq9WUlJTw0EMP8f777zM+Ps5DDz1EKBTi7Nmzqy3e\nTZHOINm7dy/l5eW89dZbdHR0SK/JPRPhRvJZLBbC4bBkbGdnZ+ns7ESlUhEMBnE6nWzevJnBwUG6\nu7uvm2ki99/CZDJRW1vLrl27+N3vfofb7WZiYuKG83KxNbKvh2yN8vUoKiqitraWtrY2vF4vXV1d\nuFwudDodGo2GZDLJ3XffzZEjRzKuFm96ha1SqVCr1RnrJ8/JyZlXPGohkskkgUAAt9vN/fffz/j4\nOM3NzYyNjWWM7iaTifz8fHQ6HaOjoxQXF9Pe3s7k5KR0jdyN0PUQBIGcnBwCgQC5ubmo1WrUajVm\ns5mhoSFEUaSmpoZIJEIwGMyo7ioLkUwmiUQiRKNROjs7550kvhyz2Uw0Gl3W9EzZui8WQq/XYzQa\nSSQSqFQq9u7di0qlklbVaT9uJt4U6SI2NyLtf5U7VVVV5ObmSv9PuyQuzyyx2WzU1NTgdDr5yle+\nwoYNG+jp6cm4VXIaURRpbGzE6XQuOuc6E0ilUkQiEeLxuJR1UV9fL7V/ysrKkgr1uN1uqUFBphKJ\nRJicnJRasi0031KplHTcermQ/8y+AoPBgCiKzM3NUVtby4EDBxgdHWVoaIi5uTlpdfXuu++usqSL\nJx3lVavVTE1NSQ+UhYJc6QpdcnbXTE5OUltbi1arlU5jNjY28uCDDzI9PU04HEalUlFVVcVjjz1G\naWkpdrtdymDwer0Z82ANBoP4fD7KysrYtWsXmzZtoqioiMHBQcbHx1dbvFsmlUoxOTmJKIp4vV5E\nUcThcFBTU0N7ezvxeJyxsTGpolo6JqDRaDJmt3M5arVaWuhlZWVhMpmumm+iKFJbW8vQ0JBUiGm5\nanzI1iin8wFTqRQajYZQKITVasVsNtPQ0MCjjz7K7Owsw8PD9PX13VLfrNVGp9Nx33338ec//xmV\nSoXBYLiuXyq9tZI7R44cobq6moqKCgwGA3/1V3/FiRMnWLdunXQwKH3M3O1209bWxpEjRxgZGckY\ng5z2rzY2NvL1r3+ddevWSf5HrVZ7w7FciHSrezn9DhqNBofDgcVioaCggMLCQjZt2kR3dzddXV0U\nFBQQj8clF0Ym3KPXIr0bn56e5o9//CP9/f1X6aLRaDh48CBvvfUWg4ODy1p0SbZGOSsri8LCQikX\n8ujRozzzzDNSrYTDhw/zf//v/5Uqy2UyaXeEIAh86UtfwuVy8Ze//GW1xVoWenp6aGpq4ktf+hLx\neJxXXnmFv/3bv2X79u1kZ2cTjUaZnJxEEASee+45uru7M2oyP/744wwODjI6Osq5c+fYtWsXdrud\n73znO2zevBmdTifVkb4Z0sfQF1sD5XYQiUQYGxvjvvvuY+3atYyNjXH8+HEOHDgglQj4NOD3+6Xs\nifb29mteE4lEePbZZ/n//r//jw8//FBq/rscyDZPOR3oUqlUFBYWMjw8LLVJ0ul02Gw2PB4PLpfr\npusJ3K7cVqvVSjKZvCo95socZkEQsNlszM3NYbVaicVit9xpRE55nwaDAZvNBoDb7eaJJ57AbDZj\ns9koLi7GaDTicrn4yU9+smDPv4VY7Tzlr33ta6RSKUZHR6muruapp56irKyMp556iq6uLubm5pY0\nlpcfrpDTWMKlwGZ9fT1Wq5WWlhapbvKtyCkHHY1GI/F4/KaCdnl5eQQCgUWP8WL0lK1Rvvz6y7s2\na7VaSktLKSkpoaWl5ZrV427E7ZrIGo2GVCp123y/Go1GKpt4u2/ydK+5xehaVFSETqdDpVJRV1fH\n+vXr+d3vfrekuhCrbZRLS0sRRZFQKITBYKC8vJzy8nLeeecd3G73spxYk4PBuhKTyYROp8Pr9X5q\ndFSpVCueJPCpMMpXolarcTgc2Gw2enp6lvQZqz2RV4rL0+hWwyjD4gskpVMXnU4nZWVltLS0LEkG\nuWKaVtMAACAASURBVI1lOlfZ7/cv2xFiORislSaTdFSpVNhsNmZmZlZkQZhxRnk5kNNEFgQBjUaz\nbHmOer2e0tLSZS+8fS2Usbw9ZJLBWiqZpKNarSYrK2tJLcwy+pj1nUK6sthyYTQaaWhoWLbPkwur\nbTQUFNLE43EmJydXzM2hrJRXgDtBR7i9emq1WuLx+FUTQRnL5WG19bwTdASZui8UFBQUFBZGcV8o\nKCgoyAjFKCsoKCjICMUoKygoKMgIxSgrKCgoyAjFKCsoKCjIiNtekOh2p6Vc6+ikkka1PNwJet4J\nOsLq63kn6AiL01O2VeKWCzl3y1VQUFC4EsV9oaCgoHATrPSKWzHKCgoKCjLiU2uU01XLMgmtVoso\nilitVhwOx2qLs6Lo9XpJX4PBgMPhkIXPT0HhRlzpF87KypLqhS8HmWe5FkHasGVCc9HLqa6uxm63\n8+CDD/L0009Lhc4/bWg0GtauXUt5eTk2m43Nmzfz1a9+dVmbTyoorDQqlQpRFHn88cf5whe+ACyP\na+NTMeNFUZz3YySTSXw+n6wbi16LgwcPUlVVRTgcpqqqih//+Mf8r//1v7Db7ast2pLJzs7G6XQC\nl3YCO3bs4LnnnsNut7Nu3Tr++q//Gq/Xy4svvkgwGGTbtm1UVlZm3ANV4c4jkUjw1a9+lZKSEmKx\nGJ/97Gf5wQ9+cMuGOeONcrqNe25uLjk5OdLf5dRwcrHE43Huv/9+Hn30Uerr66UmlYIg8OSTT7Jn\nzx6sVutV77vyoSQn5ubm8Pl8wP/bwUxOTjI7O4tKpaKkpIR169ZhMBgQBOH/b+88g+O6rjv+27e9\nL7C76L2ziQVsEiUWkLa6LVuKm2yNbTn2jO2M45Sxk7h8SD7EntgTjzUe27Iiy7KtKFFvVCHNIkqs\nIlhAEgRRF21RtmCxveYDZ18AFhAgQeIt+X6fONx2L+57/3fuOeeeg9/vp7Kykvr6+ou+q6ioSNIP\nKK1WS2lpqbgWNpsNk8m0wKO6vuj1epqbm3nsscdYtWrVTT3fmpoali5dOu3/IpEIRUVFrFq1Cr1e\nz7vvvsu99957TX+HnDVHbDYbmzZtYseOHYRCISKRyA3JdZxvSkpKePDBBykvL2fNmjUYDAbsdjuh\nUIhQKEQ4HOYrX/kKLS0t7Nu3j/7+flHkskh53lOL96vVaioqKsR+i2NjY7z99tuMjo4SiURQKpVY\nLBZsNhuRSASDwTCtKW4kEplzP8YbQbYediqVIhwOo1arSSQSxOPxmzYl02g0Ulpait1uJxAIUFtb\nS2trK8XFxXg8Hrxe70IPcV6prKzEaDReNK/KykoSiQSxWAylUslHH31EXV3dNV2nOSXKKpUKi8WC\n3W6noKAAlUolXvSBQABBEDAajZhMJkZGRhZ4tLPDZrPR0tLCxo0b0Wg07Ny5kyNHjhAKhRgYGKCu\nro6WlhaqqqrYt28f1dXVlJeXs3v3bvE7pCzKU0mlUgwPD1NdXc3ExARutxuv10tfXx9w3urKNpU1\nGo04HA5cLpf4+YmJiYUa+oxkLeNkMonP50Oj0QBcsst6Xl6e+Foude2+kGy8I51O43K5cLlclJeX\nMz4+jt/vX+jhzQtKpZLKykqqq6vJz88nlUrR09OD2+0mlUohCAKTk5N88MEHYjeSSCTCyZMnr+l3\nc0KUNRqNuG3Nz8+nqqoKlUrFiy++OO19CoUCg8GAw+HICVHOy8ujqKiIoaEhgsEgOp2OP/7xj+zc\nuROTyYTNZkMQBLxer9gQdeXKlReJci6gVqsxmUy4XC4+//nPMzIywsjICGazGfj/3nb9/f04nU5s\nNpvompI60WgUpVI5zUK+FIIgUF5eTiQSwe1257QoB4NB2tvbxYa57e3trFmzhp6eHsbHx8X3KZVK\njEbjRbs7qaPX66mrq+Puu+9m48aNnD59momJCXQ6HW1tbaIoHzx4EK1WS01NDXq9fl5+W/KinG1S\nuGnTJvr6+hgbG6O1tfWSoqtUKgkGg4yNjV32+6SUzbB8+XIeeeQRotEoExMTjI2NiW6LUCjEyMgI\nXV1d4u5g06ZN+P1+Dh48uNBDnxOCIGC329m8eTMPPvggpaWlrF+/HpfLxdGjR0Wrq6GhgZ6eHrRa\nLcFgkK6uroUe+qzI7tB0Oh0+n++yW9es0TA8PHxJK1rqZHcx2Z1Btr9kIpEgkUjw29/+9qJ5abVa\nGhoaaG1tzSlXTklJCV/96lfZvHkzSqVSdLVNNYaywn3//fczMDDAs88+Oy+/LR2FugRKpRKdTsf4\n+LjYCPSTn/wkn/3sZy/5/lWrVvHQQw9d9vvUajV2u/26jPVqOH36NAcPHqSlpYWRkRF+9rOfcfz4\ncfF1jUZDQ0MDTz75JKWlpXzwwQc88cQTPP300ws46rnjcDhoamoiLy8Ps9lMW1sb77zzDmfPnkWt\nVmM0GlGpVLS0tPCtb32LcDh81d2tF4KioiLy8vLIZDIsXrz4sg/+TCbD5OQky5Yto6Sk5AaP8top\nKChAp9ORTqdJp9OkUimi0SipVIoTJ04QiUQu+kw4HObo0aM5JcgAHo+HXbt2MTY2Rnt7Ox988AE9\nPT3i61qtlr/7u7+jsLCQ4eFhOjs76e/vn5fflpSlXFxcjM/nIxqNAud9kJFIBEEQOHfuHAqFQvQ/\nXojZbKa3t5f+/n6Kiopwu90XvSfr85MCTU1NrFixgrKyMg4ePMjw8DAfffTRtPElEgncbjdlZWVk\nMhlefvllzpw5I8lg1+VQqVQIgiDudo4fP47b7RatS7/fTyqVQqfTsWrVKjQaDeFwmJGREcms1UwI\ngoDH48FsNqPX6+no6Lgo86e4uJjKykqOHDlCWVkZ7e3tlJaWkpeXx4kTJxZo5HPH4/FQUFBAKBS6\nyG8ci8XE2EbW8PF4PEBuZUI1NzezevVqSkpKKCoqEuNTHR0dDA0Nie/LZDKEw2EqKipYs2YNcD74\nqVarrzn2ISlRnpiYIJFIoFarKSoqIj8/n8HBQdLpNBMTE2J0+1LEYjGi0SjpdBqr1cr69es5fvz4\ntKd3JpORTO6yWq1Gr9czPj7O3r17+cpXvkImkxEtCrPZzPLly/nc5z6HRqNh+/bttLW1iQ+sXMHp\ndLJhwwY2btwInN/NvPDCC/h8PuLxODabjYqKCvFkn8ViYWRkZN6sjuuNQqEgkUgQCAQIh8MXWYvZ\nQ0y9vb1kMhlisRiTk5OEw+GcCdBmMRqN1NTUMDg4eJEoT51LKBS67HdcqmqjlEgkEkQiETQaDbfd\ndhsajYbe3l7GxsamGUOZTIbR0VECgQCRSIS2tjYmJyfnZV6SEuWs4CoUCmKxGEajkY997GP85S9/\nueJnVSoVKpWKaDRKMBictz/Q9cLtdovbuu7ubioqKpicnATOWxrl5eU0Njby6U9/GpfLxZ49e+jt\n7V3YQV8F2YeIw+HA4XAgCAJ33HEHfr8fk8lEWVkZDoeD0tJS+vv7KS8vR6lUolQqF3jksyN7jQmC\ngFarJZ1OT3tw1tfXU1tby+joKC0tLaI/9vTp0zPGPqRIKpViZGREvE6zZOMBGo0Gq9Uq7lgvRSaT\nkfTDaGBggMrKSoqKishkMrzwwgtUV1fT3NwsinJdXR0mkwmTyUQ6neaDDz7A7/dTW1vLqVOnrnkM\nkhLlLFk3w9jYGI2NjYTDYdLpNDabjWQySTAYRBAEMUNhdHQUjUZDQUEB6XSazs7OefnjXE/Gxsam\n3ZR/+tOfxEi1xWJBoVDg9XqZnJxkfHycwcHBGS0QqeLz+Th27Bg1NTU88sgj9Pf3s2zZMrq6unC7\n3ZjNZpRKJS0tLXz00UcEAgH0ej1WqzUnMmiyAmMymXA6nbjdbmpqauju7qawsJCtW7eyfv16ces/\nMDDAhx9+mDNBzKlMTk5eJMgKhQKr1cr999+PVqtlbGyM0dHRy36HlA0llUpFLBZDp9NRXV0NnHfB\nfPKTn0Sr1WKz2RgfHyc/P5877rgDgP3799PZ2YkgCNMOr10Lkgj0XSowYjKZiMfjPP3000xOTpLJ\nZCgoKBDzPFUqFQ6Hg2XLluF0OhEEAZvNRnl5+Y0e/ryhUqmw2WxkMhn6+/s5e/YsJ0+epLW1lWAw\nuNDDu2rcbjeHDh2io6ODZDLJgQMH8Pl8HDp0iF/+8pe89NJLjIyMsHTpUjGD4cKbX+pkRdlgMHDP\nPfdQWlpKS0sLH/vYx1i9ejXLli1jZGQEm81Ga2trzrhnroRWq6Wuro577rmHFStW0NraSkdHx0IP\n66qorKxkyZIlonFXXl7Oj370IyYnJ8XSBwaDgR07dhCNRlm6dCmdnZ0MDQ3R2dnJ/v3752UcC24p\nq1QqtFrtRVZg9iLv6+tDoVCQyWSmLXY8Hqenp4dwOExlZSUmk4mBgQEOHz58o6cwb8RiMQYHB9m6\ndSv19fX09PQwOTnJz3/+85wIel0Oh8NBSUkJp06doqOjgx07dojBEIVCQSqVoq+vj/7+fiorK6mr\nq6OoqIjh4eEFHvmVUalUpNNpzGYztbW1qFQqXC4Xa9eupaysDL/fz8DAAAMDA/zhD38gHA7fNIIs\nCAL5+fls27aN0tJSDh48iNfrlbR7YiYeffRRLBaLGNvwer1Eo1F+97vfsXPnTsbHx7HZbDQ1NXHy\n5En6+vpYvnw5brf7mg+MTGXBRTmZTF4yXUahUKBQKFAqlZSWlhIIBAgGg9MCdYIgUFdXR1VVFceP\nH2d0dJTCwkL8fn/OBcSmsm/fPtra2qiqqqK9vT2nsi0uRX9/P2+88QZWqxWv1zstGOZ0Ornjjjto\nbm5meHhYPKWZKylU2etxYGCAYDCI1+tly5Yt/MM//AMqlYrdu3fz6quvEg6HOXv2LEajEUA8DZdL\nmEwmotGoOOfGxka++c1vcvfdd5NOpzl58mROGw//+Z//iSAINDc309nZyeuvv86XvvQltmzZQk9P\nD5lMhhUrVtDS0sKdd96JVqvl2LFjvP/++/M6jgUXZbj4mHBJSQlarZbOzk7S6bQY+bzwRk2lUpw6\ndYqenh4mJiaIRCKoVKqcF7FYLEZDQwOPPvoof/jDHy57QixXSKVSYsbBhdkvkUiEQCCA0+nkmWee\noba2lpGRkZx7qGarEprNZj788EP+9m//FgCr1YrZbCYcDrNkyRKOHj0qxkhyjUgkMm3c4+PjHDp0\niPXr11NYWMiWLVtwuVwcO3bskjnLUsNms1FUVER7ezuAeOrw8OHD9Pb2kkqlePrppzEYDAwODhIO\nh/H7/ej1eux2OwqFgvfee0/8/HwhCVG+kKwrI3uCaOoCZ6PXWq2W8vJyenp6pj2dc8XCmonNmzfz\n8MMPiw+dXH/ITM2MuZDCwkIaGhr48MMPqa6upqOjgyNHjlwyz3whcTgcpFKpy1qC2Zx6hUJBIBCg\nsLCQoaEhrFYrdXV1KJVKksmkeNgiF5k6bkEQiEajDAwMAIiZF+l0WjJpp1ciGo2KBYayO/NMJkNV\nVRWLFi1i9+7dlJSU0N7eTiQSwWKxUFhYSFlZGfF4nFOnTonHr+cTSYryxMQE8XhcrL4F5+tEhMNh\n0WrMptbkqv9qJlasWMGiRYt47rnnJFuEZ65cap2yNU2MRiPDw8MsXbqUEydOMDAwILlaCbO51rKZ\nQTqdjry8PAwGg1jbOxAI4PP5pgmWRqNBpVLl5JHr6upqVq1aRWVlJVqtlnA4zL59++jr68sZIyIa\njYqGQvb4uNPpFB/AarWaj33sY0SjUbq7u6cVyurv7+eFF164LsaDJEVZrVaTTCanWchms1kskQfn\nt/idnZ0LNcTrQnFxsRjpDQaDaLVaFi1axLlz53LG+riQbE2IVColHhyIRqNiipHFYkGn09HU1MTA\nwABer1eS7prs6bQrkclk0Gg0DA0NUVpailKppL+/n3PnziEIAo2NjbhcLkKhEGq1Go1GQzQaxWQy\nEQwGc8at4XA4WLNmDcXFxRw6dIhYLMYrr7wyo0jlgh+9qqqKUCjE/v37aWxspLGxkY8++oihoSEm\nJiZEI6mvr4/du3dfF6NJkqJsNBpRKBTTtooDAwM3pVUsCAI6nQ6j0cjDDz/M2bNnSSQSCILAl770\nJWpqavjXf/3XnLWYCwsLxZxPt9tNMBhkaGiIiooKHA4Hy5cv56GHHiIej/PLX/6S999/X3JW8lzI\nZDIEAgFRjKdWglOr1fzVX/0Vf/rTn+jq6hILT2k0GioqKjh37lzOVI47ePAgY2Nj3Hnnnbz33nu4\n3e4r3p8qlYpkMikpYc42XggEAmg0GtRqNel0GrVajVqtpq+vj+HhYdGijsViuN1uRkdHr9s8FJkb\nrHRS6JBxvac8lznabDbuuOMOHn74YbZt20ZeXh5KpRKFQoHL5eLf//3f+Z//+Z85b3FvxLLOdp5r\n165l27Zt9Pb28sILLxCPx/nWt75FQ0MDk5OT6PV6otEoP/3pT+f88JHSWs6Gy1mLWX/mpZDSWl6I\nyWRi0aJF11x0aKHm6HQ6+cY3vsETTzyB3+8X31NZWcmnP/1p3nnnHbq6ukRRzlY0vNqj4rOZpyQO\nj9yKZINf8Xgcv99PU1MTBoMBpVJJKpWira2NJ598kr17915kPZWUlPCJT3xCEg+42XDy5En+/Oc/\n09PTQ01NDVqtlueee44zZ87Q1NSEIAjs2LEjJyL2V0O2ljRc/kRbru4CQ6GQWF84F/F6vfzmN78h\nEAhQV1eH0+kkk8ng9/s5cOAA3d3d0+6/bH2a62ntS9J9cSuQvYhtNhurVq2ivb2dX//61zz00EMs\nXrwYt9stHjy48IL3+/0cO3YsZ27kSCTCwMAAExMTqNVqUqkUXq+Xt956i4GBAUwmE62trTkTIJor\n2YNQuXrSbSYuzI7KNVKplFjuYHR0lHg8LmaWtLW1LcjcZFFeILKCGg6HOX36NF6vl+3btzM4OEhF\nRQV+v5/e3t5LBr3C4fC0Nkm5QLaeSVaUAVwuFz6fD5vNdtO0ELoUsVgsZ2MCM5FNI5tqNWZPOErJ\nbzxbsrGMbOW/aDQq+sFvJLJP+Tow1zkqlUrKysoYHh6et8wDKfsh5xOpreX1QKprmfWvTt3JZQNl\nc3VnSHWO2Uyw+RrfbL5HFuXrwK0wR7g15nkrzBEWfp63whxBDvTJyMjI5Bw33FKWkZGRkbk8sqUs\nIyMjIyFkUZaRkZGRELIoy8jIyEgIWZRlZGRkJIQsyjIyMjISQhZlGRkZGQlxw49Z50oC97VwK8wR\nbo153gpzhIWf560wR5APj8jIyFwF2ZoWMguDLMoyMjIyEiKnRDkvLw+z2XzJ1woLCykoKLjBI7p6\ndDodOp1uoYchSXLdUtPpdDk9/pu19+X1wmw2X1aXroacKd2p0WioqqoSW7nb7XaMRiOCIODz+Vi/\nfj3BYJAdO3agUCjQ6/VEIhFJXlwKhQKr1QowrcOzzWZDoVCgVqtRqVRiX7tEIoHf77+o3vBClBW8\nXgiCgEKhIJVKiYImxbW7Emq1GqvVisfjIZ1Oz0rgFAoFOp2OaDSak3O+VVCr1QiCcFHTiaKiIlKp\nFJOTk/PyOzkjylVVVfT09OD3+1m/fj3f+9732LZtGzqdjt/85je0tbVx9OhR4Lylsm7dOg4ePCjJ\nTsFGoxG/33/R4j7yyCOo1WrKy8spKyvD7/djtVoZGBjgqaeeoqenR7xpVSoVRUVFuN3unBdmpVKJ\nxWJBpVIRCASIx+M5KU4qlQqj0cjo6CgKhQKTyUQqlSIcDs84H41Gw7Jlyzh58mROF4y/2SkrK8Ng\nMHDq1Klp/3/u3Ll5/Z0FK905V2soWxx97dq1fPnLX+buu+8mFotRXFwsdn/+7W9/y09+8pMrWh4L\nHbGfOner1cqyZcvQaDRs3bqV7du3o9fruf3229Hr9eTl5VFSUsLk5CRHjhxhx44dnDx5EpjZUs6F\naPbKlSuxWq3s3r0bg8HAj3/8YwwGA16vl927d7Nr164rfsdCr+VUnE4nmzZtQqFQcNddd/Hcc8/h\ndDpxOp089dRTM/6GRqO57MNIamtZV1cHMGM3+UWLFlFYWMju3btn9Z1SmyOcN+42btzIgQMHCAQC\nqFQqBEG4pprns5nnglnKc12E7NY9Ho+zZ88ePvroI3784x+jVCrRaDSMjY0RDoex2+14PB5JWxxT\n557dsp8+fZqhoSGGh4fFRqL19fU0Nzej1WoZHx8Hzj+csuSqhVxcXMzExARjY2NMTk6Sn5/P/fff\nTyQSIZFIoNFoMBqNCz3MObFixQo+85nPsHjxYt58802Ki4v55Cc/yYEDB674cMlkMjnTxRqYFgtR\nKpXYbDa8Xi8NDQ34fD5Wr16NTqfj/fffX8BRXhvZXc+JEycIh8OUlpYSi8XE+3AqFouFTCZz67kv\nsvT19eHxeMjPz2f//v3ceeedeL1e9u3bx7Fjx3Lq4obzjTSzFtLw8LDYdj4rTvfeey9Hjhzh4MGD\n1NTUsGbNGvx+P93d3Qs99KsmHA6TTCYZGxtDEAQEQaCrq4tgMEhNTQ2Tk5MMDAws9DBnxZ133kl5\neTlLlixh69atlJeXo9FoOHDgAKdOneL48eP09PRgNptZsWIF7e3tbN68mTNnztDW1rbQw58zFRUV\nJBIJUZwymYy4I52YmKC5uRmTycSZM2fE3ne5SLZbdfZ6DIfDl+0hmUgk5tXSz6nsC0AMoCxfvlxs\ndNjT08PBgwc5deoUwWBwoYc4a4xGIxaLhbGxMTHjQKFQkEwmicfjJJNJEomE6HMNBoP4/X4EIeeW\nbRoTExPE43FisRiCIGC1Wjl06BBKpZKRkRFcLpeke/apVCpKS0tRKpXk5eVx++23c9ddd+FwOAiF\nQvT09LB9+3befvttenp6sNvtLFu2DK1Wy7Jly9iyZQtlZWULPY2rQhAEPB4PHo8HOG9UhEIh4HyP\nu6KiIkZHR2lvb1/IYV4z2ViAUqlEoVDg8/mmaYtSqWTLli2iK2dqwP5aycm722QyUV9fLwZH9u7d\ny9DQ0EIPa87o9Xp0Oh0ul0vsXp19GqtUKjKZDCdOnKChoYEHHniA7u5unn/++Rl9ebmGwWCgoKAA\nQRBoaGjA7XajUqkoLCyUbFpZdnwOh4Pjx48zOjpKfn4+iUSCl156ie9+97scPnxYFCubzUZhYSF7\n9+5l2bJluN1uYrEYZrNZsnPMko3PZOnt7b3kFh7OX88ej2dadlEuE4vFeP/99y8y9DQaDTU1NXzt\na19jyZIlmEwmlErlvP1uTomyUqlEqVTS1dXFm2++SV5eHt///vd54oknMJlM3HbbbQs9xDkxPj5+\nWYENhUL09/czMDDAu+++y9tvv01/f/8NHuH8olAosFgs0yz9sbExTpw4gcFgIBQKUV9fz4oVKygt\nLZ3mP5cS0WiU1tZW6urqCIVChMNhtFotnZ2d/OhHP7ooENTb28vbb7+NwWAAYPfu3QiCQF1dneR3\nPWq1moKCArFJ6kx4vV7C4TA1NTWiBZlLCIKAVqvFZDJhMpku+768vDw+9alPYbFY8Hq96PV6nE7n\nvAlzTvmUa2pqiEQilJeX893vfpdkMslbb73F5z73OXbu3JmTaVSXY926dXz7299mcnKSkpISfvCD\nH9DV1bXQw7omLBYL//RP/8Qvf/lLBgcHxf/Pz8/nkUce4dOf/jTt7e288cYbHDhwYN46e18PMpkM\nBw4cEHORXS4Xhw4dumRMo7S0lA0bNqBWq3nmmWeYnJzEbDZfVdfnG008Hqe/v59MJkNhYSETExOX\nDaLbbDb0ej1ut3vazlWhUOTEvVlQUMCiRYtYs2YN6XSa//iP/7jk+5LJJOPj48RiMb7//e8TCoXY\nv38/r7/++rzsYnNGlMvLy7nvvvvQaDTYbDaWL1+O1+sllUqRyWRyNhPhUixevJjNmzdTWFiI3+/n\nz3/+M/39/aTT6YUe2lVjMpkoLy/nV7/6FaOjo9NeSyaTeDweOjs78fl81NfX4/V6JR8oygpqX18f\nhw8fvuwN6ff7OXLkCH6/H6PRSCQSIRAI3MihXhNZQfV4PDPeZ5s2bWL9+vVi8M/lck37vNRZvXo1\nNpuN/fv3Xzb3OOtuCoVCqFQq7HY7DQ0NjI+Pz5srKmdE2e/3k06nWbduHVVVVSiVSo4ePcru3bvp\n6OhY6OHNK4lEAp/Px4kTJ3j22Wfp7++XdOBrJvR6PQqFAqPRSG1tLa2treJrlZWVrF69msbGRlQq\nFYcOHSKVStHT05NTu4IDBw5w9uzZywptKBRifHyckpISBgYGSCaTlJSUkE6ncyYWotPpiMVilxRY\nrVbLl7/8ZVwuF3/5y18IhUL09vbe+EFeIydPnmTLli08+OCDDA8P8+KLL4q7hCyZTEb0matUKiKR\nCGNjY7hcLoaHh+dlHJIXZbVazdKlS6mvr6e2tpbKykoqKirwer1oNBrefffdnBWsyzEyMsLevXs5\nduwYhw8fXujhXBPpdBqz2czixYu5//77RR9lIBDA6XSybt06GhsbGRwc5MUXX2RwcJAzZ87gdrsv\n+52CIIjHz6XA0NDQjOKajeRnDzmlUini8XhO7XxmGqtareYTn/gER48e5fnnn+fs2bOSWZu5oFAo\nGBgYwGKxUFVVxaOPPsobb7yB2WzG7/djs9mwWq2kUikaGhooKSkhkUjQ0dExr5lfkhdlQRCw2Ww8\n/PDDWCwW1Go1oVCI7u5uenp6JO+TuxoCgQAnTpxY6GHMC7FYDIfDwZIlS9i0aRNqtZp169aJrqeC\nggIsFgsajYYNGzbw1FNP4ff7Zzw9NZugk9SIxWL09vaK1+uFLhypczn/fvb+rKqqwm63884778xa\nkLOHhHw+33wO9apxOp1EIhFcLheLFi3iC1/4Ag6HA6vVSl9fHyUlJZSXlzM+Pi4aiCMjI5w9e5Zo\nNMqKFSs4duzYNY9D8qIci8XYtWsXX/ziF0kmk4TDYVKpFNu3b+fpp59e6OFdF3K5IM9UjEYj0NS/\nuAAAHWVJREFUarWaoqIi7HY7oVCITZs2odPpEASBTCYj5iYPDw9z11138eyzzxKPx9FoNOLR4wtJ\npVI5+TDOxTHPhCAI5Ofn09zcTDqdJhKJzGmOer2ewsLCBRdlQRBIp9MoFArWrVtHS0sLa9asQa1W\n85WvfIVwOMzY2BixWAyj0UhzczMGgwGNRsPk5CRnzpzBYDCwZcuWW0OUswLl9/tRKpUcOHCA/fv3\n09fXN+vP55q4zaaqWC7MaeXKlRQVFVFbW0tNTQ379+9n3bp15OfnIwgC/f39hEIhJiYm+Ju/+Rt+\n+MMfYrPZyMvLE4/Ny0gThUJBfn4+GzZs4PHHH+eVV16hrKxsThkzExMTTExMXMdRXplsHnYkEuGv\n//qvKSoqQqvVolarSafTeDweDAYDxcXF/P73v+fs2bN85zvfoaCggFQqhcvlYmRkhA8//HDedm+S\nFmWtVovT6SSZTKLX6xkeHqajo2NOR3DtdjvxeDynot16vR6lUnlJH5XNZsNgMOREgGhoaEjc/qnV\napLJJDabDYvFwr59+3jttdcIh8PcfffdPPfcc3R2dmK32zGbzZLPvJgLSqUStVo9r6e+Fprq6mo+\n//nP8/jjj2M0Gunv7+enP/1pzh1symQyYonfjo4O3nrrLSKRCA8//DB2u52KigqcTiednZ1UV1ez\nZMkS9Ho9oVCIwcFBFi9eTFlZGbfffjt1dXU888wz1zwmSYuyIAg4HA4ee+wxbr/9dn72s5/R0dEx\npy1SIBDIqYAKnHfZXC69JhgMSrrY0lSGhoaIx+OcO3cOk8lEXV0dTU1NVFRU8Prrr/P222+jUCgI\nh8P84z/+I62trXzwwQfzFsWWAjqdjvz8fAwGwzTBulQRm5KSEnw+X06sr8lkEkvQms1m2tracLvd\nORngy+46T58+jcvlEk8sLl26lKamJgYGBnjyySdpbW1Fo9FQW1vLz372MzKZDL/4xS/Yt2/fvGac\nSFaUV61axdq1azlz5gzRaJR9+/Zx5swZTCYTdrt91u4LKR9AyJ5QvHCMMz1EksnkjLmiarUau90+\nb2O8FqLRKG63m5KSEjZt2kQ8Huejjz7ipZdeYufOncTjcfLy8ggEAgQCAfR6PYFAICdEabbodDpU\nKtVFftNLHTIJBoOSz7dXqVSk02ny8/MpLi7G7Xazf/9+du3aJR4rz1WOHj1KOBymoaGBjRs3kp+f\nz3vvvUcgEGD//v2cOXMGhUJBKBQinU5jsVhoa2tjaGiIdDotrrFGo7mmIkWSE+WysjJCoRBGoxG7\n3Y5SqaSsrIyhoSExTzIX/KmzJRvsKisrY3h4eMaHiNlsJpPJXDH1Rkp/n2xxpWwXlbKyMvbv38/k\n5KSYYTEyMiKmw91MW3w474IDxAI+cP6mTSaTF+34csnFlu20MTExwZtvvklnZ6fkHyhXIrtDi0aj\njIyM4Pf7KSsr49ixY4yNjYldcbL55d3d3YyNjZFOp8UsopvyRF9+fj7V1dVYrVaGh4dZunQpDzzw\nAGfPnsXlcvHmm2/O2kqWMgaDAZvNhiAIeL1e7rjjDt59913R3ZK9ANRqtSjUer3+iqKcSCQYGRm5\nUdO4IhaLhZ6eHp566ik2bNjAXXfdxdq1a+ns7KS3t1csVdrV1SXpXc1cmRrRVygUKJVKrFYrfr9f\nbPeVrQSYS2THOzo6yqlTpzCbzbS3t8/KpWgymYhGo5Kf88mTJzl58iR5eXn85Cc/YWRkRNwFKBQK\nMTd5165deL1eAKxWKwUFBXR2dl7zdSw5UT5x4gR///d/z8MPP4xSqaSyspJwOIxareab3/wmgiDw\nxBNPLPQwr5nS0lJuv/12jEYjO3bsYPHixXR0dBCJRJicnMTj8aBSqbBarQwNDZFKpXIutxXOn9rT\narUMDAwwPDzMt7/9bTZv3kwymRTdFCqVCrPZjM/nywn/f1ZwZ8JkMqHRaMSGmhaLhfr6eo4dO0Yq\nlcJgMKBUKuetMPqNxufzMT4+jtPpZGho6KLdmUqlQqlUTnPTlJSU4Ha7c2ZH4PP5+PrXvz7t/9Lp\nNBMTE5w6dYq8vDw0Gg1wviXUfLWFWrB2UDPx1a9+lccff5z6+nosFgsKhYLt27djMBh4/vnnZ2yt\nMxuk0EIoW/hk2bJlNDY2cuzYMZYsWYLZbObQoUMcOnRI/K533nlnztt6qbTXUSgU4onMt99+m0gk\nIm4B54MbvZazPfBw++23c/fdd2O32zl79ix79uyhvb2dRCLBhg0bGBwcnHVgSCpreanPZGvRjI6O\nThtnTU0Ndrt91idSpTrHy5FNf5vrdSzpdlCXY/PmzZSVldHR0cH4+DiNjY1UVFQwOjqKxWK5Kba4\ngiDQ09PD8PAwvb29okArFApUKhUKhUJM4zt58mTOdVPJ4nA4iEajdHR04PP5aGxs5Pjx4zlhDV+O\nRCIxK0vvi1/8IsuWLSMSiVBWVkZ9fT3f+c53AGhtbc2pLAW1Ws29997Lnj17xLzibPfxWCyGx+O5\nSGyyO6OblXQ6LQqz1Wolk8nM2w5AcpZyUVER+fn5aDQatFqtWAC9r68PlUrFyMjINdcVloKlnMVi\nsbBkyRKxMWy2s8P4+LhYmOhqkILlodVqSaVSJJNJ1Go1ZrNZ9MHNF1Jay6msXbsWk8lEIpHA4XDg\ncDh48skn5/QdSqUSk8l0Q2q7zDRPQRDEQPTUh8l8HmKSwvU6G7LdgqY+cLJ1v2fzoJ3NPCUnylMR\nBAG1Wj3vlqJUb+T5JFcu8mtFqmtptVqJRCLE43EsFgs1NTUUFBSwZ8+eWV/PgiCg0+luSKrZQq9l\nrlyvWq0Wg8FwXY0lSVd1SafTObt1l7m1yfZ2g/Opbn19fXNu/5ROp+Wj5hIjFotd91odkraU5/s3\ns1OVqnU1n+SK5XGtyGs5Pyz0PG+FOcJNYCnPF0qlEpVKcjFNGRkZmYu44ZayjIyMjMzluSUsZRkZ\nGZlcQRZlGRkZGQkhi7KMjIyMhJBFWUZGRkZCyKIsIyMjIyFueJ7YQuQKNjY2otFoOHnyJHBz5rZm\nU/6yZRHlvM/54VaYIyz8PBdqjgqFAq1WK9Zqv97kZEGi68H4+DhKpfKG/65WqyWTydyQIkq5XORH\nRmahyGQykmuscEuI8tSuDzeSG9klRRZlaZO1yKQmANeLbF/CuTQ5ljmP7FO+jsTj8Zwq0Shz/ch2\nkVloN8F8o1Ao0Ov1F81Lr9djsVjE98jMnpyqfaFQKDAajQiCQCAQuOrSgTeLH3Km+d8Kfki4edZy\nJqS8lmq1GofDwdjY2DW1eZLKHLNuzqntrYxGI7FYDKVSiSAIJJNJCgoKmJiYIJFIIAgCqVRqVm7K\nm6L2RbbHmSAIGI1GtmzZwn333SeWNZTCTbMQKBQKDAZDTs8/u67Zf2fnMvXfMtJBrVaL65UlkUgw\nPDw8TZAFQViQGM58YLPZsNls0/5v/fr1FBcX09DQwKpVq6isrOTnP/85LS0t1NfXs3z5cqqrq+dt\nDJL2KZeUlHD//fcTiUTYtGkTk5OTOBwOurq6SKfTon/OarUSi8VuOn/dTL3gMpkM4XBYUp2r50pJ\nSQnl5eVoNBqcTifxeJz29nYGBgbE/n03G9n285WVlaTTaVwul/haZWUl4+PjN6R+8tVwJVdcc3Mz\nY2NjjIyMYDabMRqNOdfk+FJlOf1+P2azGYPBgMVioaKigoKCAlpaWrjzzjuprKxk9+7dnD17dl7G\nIFlR1mg0FBQUsGzZMsxmM5OTk2Lr7+xCZwXprrvuoqurizNnzizkkOeNlStXYrVa+eCDD3A6nWIb\n82y7+myN6VwW5KamJurr6wmHw3R3d2O32zlx4gQej4fm5mZisdis+7tJlaamJlatWoXFYsHlctHf\n309HRweZTIaSkhLR6qyurmbPnj2MjY3lXLszo9HIbbfdxqFDh+ju7sbhcGAymQgEAjlZC/pSRpDP\n56OhoYHm5maxe/crr7xCVVUVp0+fpr+/n56ennkbg2RF2el0UlNTQzAYpK6ujvHxcc6ePYvP52Nw\ncBCLxcL69esZGxvD6/VSXFyMWq2mp6dH8h2CNRoNqVRK9FvpdDo+/vGPo1arOXjwIPn5+RQVFVFa\nWso999zDs88+S2lpKcFgMCc7Wl+KaDQqup+GhoZQKBQkk0k2btwo7oKqq6sZGhrKyUYHLS0tVFRU\nsHz5cmpqanC5XLz44otkMhkWL15MPB5nbGyMUCiE2+0GWFARczqdeL3eab7U2ZBMJvF4PFRUVDA4\nOEhlZaXYty/XHjAXolKpqK6uxul0YjQa6e7uxuv10t/fTzweZ+vWrXg8nnlpUTftd+ftm+YZpVKJ\nVqtFoVDg9XoJhUKcO3eOnp4esSOyVqtl1apVDAwM4HA4sFgsYvNCqQvzVJ9pNjJfXl7OqVOn8Hq9\nOBwOVq1aRWlpKYWFhRQWFhKNRjEYDOh0upwTZ5VKhc1mE3sujo2N4fP5yMvLw2g0itv6uro62tvb\ngfPb+VxrvqlUKiksLGTbtm2YTCaKi4spLy9HEAQGBwfFhpsDAwOiGC9UyuZUso1Q50osFsPlcvHA\nAw9gs9lIJpM5aSFfDo1GQyQSobu7m6GhIUZGRoDz69zW1saqVasIhULzmvq34KI8NYPAbrczOTlJ\nPB4nEokwOTmJSqVi7969jIyMiB2gs2zfvp1PfepT3HXXXfj9fs6cOSNGSqUsylkLQqPRiIv+2muv\n0dDQwOjoqJhlUl1dze7du9Hr9eJ20G63Y7fbc1KUnU4n27ZtQ6fTceLECWKxGCMjI9hsNqqqqti1\naxddXV3Y7XasVitjY2M5FyfQarVs27aNpqYmMpkMwWCQkydPMjAwwLlz5wBoa2vDbrdTXFxMIBCQ\nhA85KzZzQafTYbfb0el0GI1GKioqOHDgAOPj49dhhDcWpVKJXq9nYGAAm82Gx+MhGAyKr+Xl5RGJ\nRKipqSGRSNDV1TVvv73gopwVZIVCwapVqzh+/Dijo6MUFBRgs9l4+eWXOXDgwDRfTzZqr1AoeOml\nlygtLUWhUBCPxwkGgzljXdlsNoqLixkfHycSidDe3i5uH/ft28e+ffvE99bX16NWq0kmk5J+4FyO\nRCLBxMQENpsNs9nMZz/7Wd58801eeOEFALq7u4HzD6yCggJ8Pp9oSUqNbG5uJBK5yK9vMBj4zGc+\nQ0lJCYcPH+aPf/wj77///kXfsWrVKgoKCjh69GjOxUJUKhVKpZKKigq2bt1KOBzm9ddfF42pmZjP\nDtjXE51OR1VVFQqFgk2bNrFz505Onz6NQqHAYrGwevVqqqur8Xg8qFQqdDrdvP22pPKUlUqlGNBa\nsmQJJSUlvPXWWxf5uQwGA8XFxTgcDoaGhqitrRUDgJe6US5EKrmtCoWCxsZGHn30Uf7t3/5tRt+p\nIAhUVFSQyWTo7++/4gk+qeR9Zqmuruaxxx5Do9HQ3d3N3r176enpuSi3ValUYjQaAQiFQlf0cS7E\nWubl5fHQQw/xwgsvXPSAdDqd/Pa3v+WHP/whnZ2dJBKJS84hm/OaSqVybi0XL16MzWZDqVSycuVK\nPB4PZWVlpNNpXn75ZTo7O+f8+1KbI4DFYuGLX/wir7zyCm63m3Q6jdFopKioCLvdzq9+9StOnTrF\n7373O/bv3z+rPO2cq32RvXiLi4uJxWLs378flUp10UUdjUYZGBjA6/Xy8MMP09jYyOuvv057e7vk\nn8JT09waGhqor6/n17/+9RWDIul0mqGhIfHfuURBQQG1tbUEg0HuuOMOUqkUGzZsQKVSiVaiWq3m\nq1/9Kj6fD5VKRVdXFwcPHlzgkV+aQCDA66+/fpHboaamhq997WuEw2ExW+hyTA30ShmNRkNJSQku\nl4t0Ok1tbS2hUAiXy8WiRYuw2+289tprfOELX+CNN96Y14DXQmG32ykqKiIYDPLiiy/i8Xioqqoi\nEolQWlrKY489RjKZJBKJcODAAXp7e6/p4MyFSEqU4fzWyOfziT6cSz3dnE4na9euZfny5dhsNgRB\nIJFISF6QszuBLKOjo0QiEQYHB2f1+Vw9sm21WrHb7QwNDeH3+wkGg6hUKjQajfieTCaDz+fD6/Xi\n8XjEB5AUSaVS0/ymhYWFNDU14XQ62bVrF4899ph43Wq1WqxWKxqNBrfbPa83740gmUzi9XrF63Z8\nfByHw8HmzZtxOp3s2LGD2267jTfffJOurq6czJS5kFAoxPj4OCaTiUWLFnHkyBFisRg2m43KykqK\nioooLi7mvffeo7W1lYmJiXn9fcmJciaTIRAIzGgNZjIZDAYDy5cv56WXXiKdTjM2NnYDR3l1XFig\nyOfzXTJZfabP6/V61Gq1mGWSC0SjUfx+PyqVitdeew2Hw0Emk6GwsJCVK1eSyWQYGhpiYmKCeDw+\nLcqdCyQSCQKBAGq1mkgkwquvvsq2bdvo7+9HoVCIebtlZWUcPXo0p1LF0un0tGttYmICo9EoPnSy\nD6iJiQny8vLwer05L8zRaBSbzcbatWvp7Oxkw4YNZDIZqqqqaG5uxuFwUFJSQmtrK729vfN+0Ely\nojybLZ3f7+fYsWNUVlbS2dmJz+fLiZt4PtwOgiCItZNzBY/HQ3d3N1VVVQSDQSKRCAUFBRiNRhKJ\nBHq9nqqqKmpqajh27FjOuWe8Xi9erxe73c7KlSv58MMP+clPfkIwGBQziCYmJvD7/cD57ItsJD8X\nSSaTtLe3U1hYyNKlS/nLX/6CXq/P6ePVF2IwGCgrK+OFF17gn//5n0mn0zQ0NLB8+XJisRihUIiq\nqioOHz487ztYyde+uJBslkVfXx/bt2+nurqaycnJnMxIuBpCoRBer3ehhzEn4vE4Xq+XYDDIxz/+\ncdra2ujq6qKjo4Ndu3Zx9OhRli5dSlNTEx6Ph4mJiYtqLOQCHo+HvXv3UlFRIR50UigU2O12Fi9e\nTCgU4s477yQvL2+hh3pNZB+m2UMUS5YsobOzk+7u7mk5yhqNBqvVuoAjvXr8fj+HDh0inU7j8Xjw\n+XyEw2HGx8c5fPgw27dvZ+vWrTidznn/bUllX8xU6yH7WZVKRSaTQRAECgoKGB4eJpVKid87m+lI\nJfvieiKFaLZKpSKdTpOfn09xcTHhcFgsYDPVusimOFZVVWGz2ejv7591HrYU1zJ7GCiVSvGNb3yD\ne+65h1dffZXf//73VxXck8JazubzF46zsrKSFStW8Oqrr17x81Kbo9VqpaGhgY8++kj87Kc+9SkK\nCgr41a9+hVar5d577+XDDz+c05mB2cxTMqKsUChYuXIl3d3d4jZvJrJR4f7+flKpFA0NDaRSqVkl\ncUvxRp5vpHSRK5VKMYsmk8lME6ZskZexsTGWL1/O448/zhtvvMH27dtn9d1SXcsf/OAHHD16lFOn\nTuFwOKirq2NycpKdO3fO2ecqpbWcLSaTCYvFQiqVEoOFM81DanPMPljj8TgbNmxAq9UyOjrK4OAg\niUSCFStW0NHRgdfrnVPwNqdS4jKZDJ2dnYTDYbE49qWCWU1NTaxduxalUslzzz0n3uCBQOCiCWu1\nWhKJRM75KG82Zkr/isVi6HQ6vve973H06FH+8Ic/iAdJcpn//u//ZmJigoqKCh588EFWrFjB7t27\n2bt3b84HwmbCarWKh0jUajVms/mKgiwlso0IUqkUKpWKeDzO6dOnEQSBSCRCY2Mjn/vc50in07S1\ntV2XbBpJOe4CgQDJZJJYLIYgCJjN5oveo1QqxeOrU/NAfT7fRRZ21jLLNerq6li+fPlCD2PeKCoq\nYunSpZd8rbS0lLvuuot4PM5tt93G8PBwTmTSXInOzk6sVisf//jHufPOOwmHw7z//vs5L8gmk0ms\nVjjTe3Q6HcFgEL/fn1OpnOl0GovFQnl5uTjubIpuOBxmZGSEc+fOsWbNGgoLC1Gr1fM+BslYylPJ\nXrhT81izWCwWEonERQcLLnWx51pOaBan00leXh7Hjx8HZn80NRv9luJNMHX8ZrOZaDRKJpMhLy8P\nh8PB8PAwGzZswGAwLOAorx2tVsuGDRsIhUKsXr2aDRs2oFQq2bNnj+ifzGWy66hUKlEqlSSTyWk7\n0amGUiKRmPFaVCgU4ndIhay7JbuzW7p0KZ2dnWLVQr1ez5tvvondbge4LgFpSYoynBfZqUKbn5+P\nyWSisLCQVCqF3++npqYGq9XKuXPnxFzBXDgldSEKhQKdTodKpcJqtaJQKPD7/ZhMJvEAzWxFWaVS\nSU6U3W43Xq+XgoICxsbGyM/PF31xgiAQj8exWCx0dXVJojjPtaBWq1myZAmCIGC32/H5fLhcrpvC\nJVNbW8vo6CixWAy1Wi0GcqeK8oX37Uxkr1cpiTKcz7zIFhvasmULSqWSrq4uysvLcTgcvPPOO/zx\nj38kFApdl7FLSpQvJz4ajYbbb7+dZcuWcfbsWQ4ePEhtbS0bNmxg8eLF/OIXv2BgYEDsxpFLqNVq\n8vLyKCsrw2w2s3LlSoaGhjh58iQlJSV0dHTM2ieeTCYldYHrdDrRn+x0Olm9ejXvvPOO+FosFsNi\nsVBTU8PRo0f5/e9/n9MdRxQKBbFYjGeeeYaVK1fi9Xoxm81UV1fn/A4A4L777mPHjh2cOXNGtIKn\nFmeC84HbZDI5qwMyUjxqrlar0Wq1lJaW0tLSgkKhYP369Wg0GgKBAPF4nLq6Opqamnjttdeuy/gl\nk30B510T4XB4mrCo1Wqampp47LHH2Lp1K36/n7fffpudO3ficrmuymclpYh9XV0d3/zmN8X2OZWV\nlbS1tfGnP/2JvXv3XvUYpBDN3rhxI8PDw3g8HpqamqisrOT999/H5/Nxxx134PP5aG1tFcd7NQFZ\nKa2lTqejsrKSlpYWXnzxRRoaGggEArS1tV10mnMuSGEt1Wo16XT6IhEym82sWbOGvXv3kkwmue++\n+xgYGODEiRNz+n0pzBFg0aJFNDc3U1BQwMjICP/7v/9LXl4eX//611m/fj1ut5s9e/bQ09PDwYMH\n53w6M6dS4uD/85SzzvNsH6yXX36Zr3/962zcuJFAIEBHRwd9fX08//zzF/1RsrUGZsodlNKNrNFo\nqKmp4V/+5V84duwY69at4/Dhw/zXf/3XNRU/l8JFnrWaUqkUWq0Wh8PBt7/9bfbt28fIyAj5+fmE\nQiFJP3zmspZGo5HGxkaam5t56aWXgP8/gn0t3Mi1zMvLIx6PT3MjCYJAdXW1eFhkKoIgoNfrxfeb\nzWYSicSc62BL4XqF8/qh1+spKipiw4YNlJeXc+TIESYnJ/H7/VgsFqqqqnjxxRdnVZHyQnJOlKe+\nJ+tnLS4upru7m+rqakpLS4nH40xMTBCNRunr6yOTyUxzewiCgEajmfGikNKNrNPpKCkpEdu0Z/2Q\nvb2917Q1kspFPhW9Xs8DDzxAa2sr4+Pj6HQ6GhsbKSkp4bnnnruqMUhpLZVKJRaLhYKCArFy2NXu\nAKZyI9dSo9GQTqcvcoOZzWZCodB1Sy+V2vVqs9lobm6mqKiIgwcPigWKbDYb0Wj0qsqTQg6L8uXI\nnua7UKzmWjhbSjeywWCgqKiInp6eeR2X1C5yOP/AdDqdeDwe8aYvLS2ltrb2qq1lKa2lRqMRg5jz\nWXRIims530htjtm1VKvVuN1uEokEVqsVrVY75wMjU7npRHm+kNKNrFarMZlMc6oWNxukdpFfL6S0\nltn2SENDQzf9A3a+uRXmCDkqyhd2er4eSOlGvhBBEK4pKJRFvsjnh+wc51JbZb65FdbyVpgjzG6e\nkjrRB+cPTlzqJN+tQjqdzslTiDc78/GgvBxSEAsZ6XDDLWUZGRkZmcsjOUtZRkZG5lZGFmUZGRkZ\nCSGLsoyMjIyEkEVZRkZGRkLIoiwjIyMjIWRRlpGRkZEQsijLyMjISAhZlGVkZGQkhCzKMjIyMhJC\nFmUZGRkZCSGLsoyMjIyEkEVZRkZGRkLIoiwjIyMjIWRRlpGRkZEQsijLyMjISAhZlGVkZGQkhCzK\nMjIyMhJCFmUZGRkZCSGLsoyMjIyEkEVZRkZGRkLIoiwjIyMjIWRRlpGRkZEQsijLyMjISAhZlGVk\nZGQkxP8B4fQNaNfg3iwAAAAASUVORK5CYII=\n",
|
|
"text/plain": [
|
|
"<matplotlib.figure.Figure at 0x206925cf400>"
|
|
]
|
|
},
|
|
"metadata": {},
|
|
"output_type": "display_data"
|
|
}
|
|
],
|
|
"source": [
|
|
"def plot_images(images, subplot_shape):\n",
|
|
" plt.style.use('ggplot')\n",
|
|
" fig, axes = plt.subplots(*subplot_shape)\n",
|
|
" for image, ax in zip(images, axes.flatten()):\n",
|
|
" ax.imshow(image.reshape(28, 28), vmin = 0, vmax = 1.0, cmap = 'gray')\n",
|
|
" ax.axis('off')\n",
|
|
" plt.show()\n",
|
|
" \n",
|
|
"noise = noise_sample(36)\n",
|
|
"images = G_output.eval({G_input: noise})\n",
|
|
"plot_images(images, subplot_shape =[6, 6])"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"Larger number of iterations should generate more realistic looking MNIST images. A sampling of such generated images are shown below."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 18,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/html": [
|
|
"<img src=\"http://www.cntk.ai/jup/GAN_basic_slowmode.jpg\"/>"
|
|
],
|
|
"text/plain": [
|
|
"<IPython.core.display.Image object>"
|
|
]
|
|
},
|
|
"execution_count": 18,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"# Figure 3\n",
|
|
"Image(url=\"http://www.cntk.ai/jup/GAN_basic_slowmode.jpg\")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"**Note**: It takes a large number of iterations to capture a representation of the real world signal. Even simple dense networks can be quite effective in modelling data albeit MNIST is a relatively simple dataset as well."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"collapsed": true
|
|
},
|
|
"source": [
|
|
"**Suggested Task**\n",
|
|
"\n",
|
|
"- Explore the impact of changing the dimension of the input random noise (say from 100 to 10) in terms of computation time, loss and memory footprint for the same number of iterations.\n",
|
|
"\n",
|
|
"- Scale the image from 0 to 1. What other changes in the network are needed?\n",
|
|
"\n",
|
|
"- Performance is a key aspect to deep neural networks training. Study how the changing the minibatch sizes impact the performance both with regards to quality of the generated images and the time it takes to train a model.\n",
|
|
"\n",
|
|
"- Try generating fake images using the CIFAR-10 data set as the training data. How does the network above perform?\n"
|
|
]
|
|
}
|
|
],
|
|
"metadata": {
|
|
"anaconda-cloud": {},
|
|
"kernelspec": {
|
|
"display_name": "Python 3",
|
|
"language": "python",
|
|
"name": "python3"
|
|
},
|
|
"language_info": {
|
|
"codemirror_mode": {
|
|
"name": "ipython",
|
|
"version": 3
|
|
},
|
|
"file_extension": ".py",
|
|
"mimetype": "text/x-python",
|
|
"name": "python",
|
|
"nbconvert_exporter": "python",
|
|
"pygments_lexer": "ipython3",
|
|
"version": "3.5.4"
|
|
}
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 2
|
|
}
|