CNTK/Tutorials/CNTK_203_Reinforcement_Learning_Basics.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from IPython.display import Image"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# CNTK 203: Reinforcement Learning Basics\n",
    "\n",
    "\n",
    "Reinforcement learning (RL) is an area of machine learning inspired by behaviorist psychology, concerned with how [software agents](https://en.wikipedia.org/wiki/Software_agent) ought to take [actions](https://en.wikipedia.org/wiki/Action_selection) in an environment so as to maximize some notion of cumulative reward. In machine learning, the environment is typically formulated as a [Markov decision process](https://en.wikipedia.org/wiki/Markov_decision_process) (MDP) as many reinforcement learning algorithms for this context utilize [dynamic programming](https://en.wikipedia.org/wiki/Dynamic_programming) techniques.\n",
    "\n",
    "In some machine learning settings, we do not have immediate access to labels, so we cannot rely on supervised learning techniques. If, however, there is something we can interact with and thereby get some feedback that tells us occasionally, whether our previous behavior was good or not, we can use RL to learn how to improve our behavior.\n",
    "\n",
    "Unlike in supervised learning, in RL, labeled correct input/output pairs are never presented and sub-optimal actions are never explicitly corrected. This mimics many of the online learning paradigms which involves finding a balance between exploration (of conditions or actions never learnt before) and exploitation (of already learnt conditions or actions from previous encounters). Multi-arm bandit problems is one of the category of RL algorithms where exploration vs. exploitation trade-off have been thoroughly studied. See figure below for [reference](http://www.simongrant.org/pubs/thesis/3/2.html)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<img src=\"https://cntk.ai/jup/polecart.gif\" width=\"300\" height=\"300\"/>"
      ],
      "text/plain": [
       "<IPython.core.display.Image object>"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Figure 1\n",
    "Image(url=\"https://cntk.ai/jup/polecart.gif\", width=300, height=300)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Problem**\n",
    "\n",
    "We will use the [CartPole](https://gym.openai.com/envs/CartPole-v0) environment from OpenAI's [gym](https://github.com/openai/gym) simulator to teach a cart to balance a pole. As described in the link above, in the CartPole example, a pole is attached by an un-actuated joint to a cart, which moves along a frictionless track. The system is controlled by applying a force of +1 or -1 to the cart. A reward of +1 is provided for every timestep that the pole remains upright. The episode ends when the pole is more than 15 degrees from vertical, or the cart moves more than 2.4 units from the center. See figure below for reference.\n",
    "\n",
    "**Goal**\n",
    "\n",
    "Our goal is to prevent the pole from falling over as the cart moves with the pole in upright position (perpendicular to the cart) as the starting state. More specifically if the pole is less than 15 degrees from vertical while the cart is within 2.4 units of the center we will collect reward. In this tutorial, we will train till we learn a set of actions (policies) that lead to an average reward of 200 or more over last 50 batches.\n",
    "\n",
    "In, RL terminology, the goal is to find _policies_ $a$, that maximize the _reward_ $r$ (feedback) through interaction with some environment (in this case the pole being balanced on the cart). So given a series of experiences $$s \\xrightarrow{a} r, s'$$ we then can learn how to choose action $a$ in a given state $s$ to maximize the accumulated reward $r$ over time:\n",
    "\n",
    "$$\n",
    "Q(s,a) = r_0 + \\gamma r_1 + \\gamma^2 r_2 + \\ldots = r_0 + \\gamma \\max_a Q^*(s',a)\n",
    "$$\n",
    "\n",
    "where $\\gamma \\in [0,1)$ is the discount factor that controls how much we should value reward that is further away. This is called the [*Bellmann*-equation](https://en.wikipedia.org/wiki/Bellman_equation).\n",
    "\n",
    "In this tutorial we will show how to model the state space, how to use the received reward to figure out which action yields the highest future reward."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We present two different popular approaches here:\n",
    "\n",
    "**Deep Q-Networks**: DQNs have become famous in 2015 when they were successfully used to train how to play Atari just form raw pixels. We train neural network to learn the $Q(s,a)$ values (thus $Q$-Network ). From these $Q$ functions values we choose the best action.\n",
    "\n",
    "**Policy gradient**: This method directly estimates the policy (set of actions) in the network. The outcome is a learning of an ordered set of actions which leads to maximize reward by probabilistically choosing a subset of actions. In this tutorial, we learn the actions using a gradient descent approach to learn the policies."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this tutorial, we focus how to implement RL in CNTK. We choose a straight forward shallow network. One can extend the approaches by replacing our shallow model with deeper networks that are introduced in other CNTK tutorials.\n",
    "\n",
    "Additionally, this tutorial is in its early stages and will be evolving in future updates.\n",
    "\n",
    "## Prerequisites\n",
    "Please run the following cell from the menu above or select the cell below and hit `Shift + Enter` to ensure the environment is ready. Verify that the following imports work in your notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "from __future__ import print_function\n",
    "from __future__ import division\n",
    "import matplotlib.pyplot as plt\n",
    "from matplotlib import style\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import seaborn as sns\n",
    "\n",
    "style.use('ggplot')\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We use the following construct to install the OpenAI gym package if it is not installed. For users new to Jupyter environment, this construct can be used to install any python package. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting gym\n",
      "  Downloading gym-0.9.4.tar.gz (157kB)\n",
      "Requirement already satisfied: numpy>=1.10.4 in c:\\local\\anaconda3-4.1.1-windows-x86_64\\envs\\cntk-py35\\lib\\site-packages (from gym)\n",
      "Requirement already satisfied: requests>=2.0 in c:\\local\\anaconda3-4.1.1-windows-x86_64\\envs\\cntk-py35\\lib\\site-packages (from gym)\n",
      "Requirement already satisfied: six in c:\\local\\anaconda3-4.1.1-windows-x86_64\\envs\\cntk-py35\\lib\\site-packages (from gym)\n",
      "Collecting pyglet>=1.2.0 (from gym)\n",
      "  Using cached pyglet-1.2.4-py3-none-any.whl\n",
      "Requirement already satisfied: certifi>=2017.4.17 in c:\\local\\anaconda3-4.1.1-windows-x86_64\\envs\\cntk-py35\\lib\\site-packages (from requests>=2.0->gym)\n",
      "Requirement already satisfied: urllib3<1.23,>=1.21.1 in c:\\local\\anaconda3-4.1.1-windows-x86_64\\envs\\cntk-py35\\lib\\site-packages (from requests>=2.0->gym)\n",
      "Requirement already satisfied: idna<2.7,>=2.5 in c:\\local\\anaconda3-4.1.1-windows-x86_64\\envs\\cntk-py35\\lib\\site-packages (from requests>=2.0->gym)\n",
      "Requirement already satisfied: chardet<3.1.0,>=3.0.2 in c:\\local\\anaconda3-4.1.1-windows-x86_64\\envs\\cntk-py35\\lib\\site-packages (from requests>=2.0->gym)\n",
      "Building wheels for collected packages: gym\n",
      "  Running setup.py bdist_wheel for gym: started\n",
      "  Running setup.py bdist_wheel for gym: finished with status 'done'\n",
      "  Stored in directory: C:\\Users\\yuqtang\\AppData\\Local\\pip\\Cache\\wheels\\ad\\46\\5a\\1833d0ef8149a50e78fdb24a41f47e22a7f4a9dcd649b8db53\n",
      "Successfully built gym\n",
      "Installing collected packages: pyglet, gym\n",
      "Successfully installed gym-0.9.4 pyglet-1.2.4\n"
     ]
    }
   ],
   "source": [
    "try:\n",
    "    import gym\n",
    "except:\n",
    "    !pip install gym\n",
    "    import gym"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Select the notebook run mode**\n",
    "\n",
    "There are two run modes:\n",
    "- *Fast mode*: `isFast` is set to `True`. This is the default mode for the notebooks, which means we train for fewer iterations or train / test on limited data. This ensures functional correctness of the notebook though the models produced are far from what a completed training would produce.\n",
    "\n",
    "- *Slow mode*: We recommend the user to set this flag to `False` once the user has gained familiarity with the notebook content and wants to gain insight from running the notebooks for a longer period with different parameters for training. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "isFast = True"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## CartPole: Data and Environment"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will use the [CartPole](https://gym.openai.com/envs/CartPole-v0) environment from OpenAI's [gym](https://github.com/openai/gym) simulator to teach a cart to balance a pole. Please follow the links to get more details.\n",
    "\n",
    "In every time step, the agent\n",
    " * gets an observation $(x, \\dot{x}, \\theta, \\dot{\\theta})$, corresponding to *cart position*, *cart velocity*, *pole angle with the vertical*, *pole angular velocity*,\n",
    " * performs an action `LEFT` or `RIGHT`, and\n",
    " * receives\n",
    "  * a reward of +1 for having survived another time step, and\n",
    "  * a new state $(x', \\dot{x}', \\theta', \\dot{\\theta}')$\n",
    "\n",
    "The episode ends, if\n",
    " * the pole is more than 15 degrees from vertical and/or\n",
    " * the cart is moving more than 2.4 units from center.\n",
    "\n",
    "The task is considered done, if\n",
    " * the agent achieved and averaged reward of 200 over the last 50 episodes (if you manage to get a reward of 200 averaged over the last 100 episode you can consider submitting it to OpenAI).\n",
    "\n",
    "In fast mode these targets are relaxed."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 1: DQN\n",
    "\n",
    "After a transition $(s,a,r,s')$, we are trying to move our value function $Q(s,a)$ closer to our target $r+\\gamma \\max_{a'}Q(s',a')$, where $\\gamma$ is a discount factor for future rewards and ranges in value between 0 and 1.\n",
    "\n",
    "DQNs\n",
    " * learn the _Q-function_ that maps observation (state, action) to a `score`\n",
    " * use memory replay (previously recorded $Q$ values corresponding to different $(s,a)$ to decorrelate experiences (sequence state transitions)\n",
    " * use a second network to stabilize learning (*not* part of this tutorial)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Model: DQN\n",
    "\n",
    "$$\n",
    "l_1 = relu( x W_1 + b_1) \\\\\n",
    "Q(s,a) = l_1 W_2 + b_2 \\\\\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will start with a slightly modified version for Keras, https://github.com/jaara/AI-blog/blob/master/CartPole-basic.py, published by Jaromír Janisch in his [AI blog](https://jaromiru.com/2016/09/27/lets-make-a-dqn-theory/), and will then incrementally convert it to use CNTK.\n",
    "\n",
    "We use a simple two-layer densely connected network, for simpler illustrations. More advance networks can be substituted.\n",
    "\n",
    "**CNTK** concepts: The commented out code is meant to be an illustration of the similarity of concepts between CNTK API/abstractions against Keras. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import numpy\n",
    "import math \n",
    "import os \n",
    "import random\n",
    "\n",
    "import cntk as C"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the block below, we check if we are running this notebook in the CNTK internal test machines by looking for environment variables defined there. We then select the right target device (GPU vs CPU) to test this notebook. In other cases, we use CNTK's default policy to use the best available device (GPU, if available, else CPU)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Select the right target device when this notebook is being tested:\n",
    "if 'TEST_DEVICE' in os.environ:\n",
    "    if os.environ['TEST_DEVICE'] == 'cpu':\n",
    "        C.device.try_set_default_device(C.device.cpu())\n",
    "    else:\n",
    "        C.device.try_set_default_device(C.device.gpu(0))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "STATE_COUNT = 4 (corresponding to $(x, \\dot{x}, \\theta, \\dot{\\theta})$),\n",
    "\n",
    "ACTION_COUNT = 2 (corresponding to `LEFT` or `RIGHT`)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(4, 2)"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "env = gym.make('CartPole-v0')\n",
    "\n",
    "STATE_COUNT  = env.observation_space.shape[0]\n",
    "ACTION_COUNT = env.action_space.n\n",
    "\n",
    "STATE_COUNT, ACTION_COUNT"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note: in the cell below we highlight how one would do it in Keras. And a marked similarity with CNTK. While CNTK allows for more compact representation, we present a slightly verbose illustration for ease of learning.\n",
    "\n",
    "Additionally, you will note that, CNTK model doesn't need to be compiled explicitly and is implicitly done when data is processed during training.\n",
    "\n",
    "CNTK effectively uses available memory on the system between minibatch execution. Thus the learning rates are stated as **rates per sample** instead of **rates per minibatch** (as with other toolkits)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Targetted reward\n",
    "REWARD_TARGET = 30 if isFast else 200\n",
    "# Averaged over these these many episodes\n",
    "BATCH_SIZE_BASELINE = 20 if isFast else 50\n",
    "\n",
    "H = 64 # hidden layer size\n",
    "\n",
    "class Brain:\n",
    "    def __init__(self):\n",
    "        self.params = {}\n",
    "        self.model, self.trainer, self.loss = self._create()\n",
    "        # self.model.load_weights(\"cartpole-basic.h5\")\n",
    "\n",
    "    def _create(self):\n",
    "        observation = C.sequence.input_variable(STATE_COUNT, np.float32, name=\"s\")\n",
    "        q_target = C.sequence.input_variable(ACTION_COUNT, np.float32, name=\"q\")\n",
    "\n",
    "        # Following a style similar to Keras\n",
    "        l1 = C.layers.Dense(H, activation=C.relu)\n",
    "        l2 = C.layers.Dense(ACTION_COUNT)\n",
    "        unbound_model = C.layers.Sequential([l1, l2])\n",
    "        model = unbound_model(observation)\n",
    "\n",
    "        self.params = dict(W1=l1.W, b1=l1.b, W2=l2.W, b2=l2.b)\n",
    "\n",
    "        # loss='mse'\n",
    "        loss = C.reduce_mean(C.square(model - q_target), axis=0)\n",
    "        meas = C.reduce_mean(C.square(model - q_target), axis=0)\n",
    "\n",
    "        # optimizer\n",
    "        lr = 0.00025\n",
    "        lr_schedule = C.learning_parameter_schedule(lr)\n",
    "        learner = C.sgd(model.parameters, lr_schedule, gradient_clipping_threshold_per_sample=10)\n",
    "        trainer = C.Trainer(model, (loss, meas), learner)\n",
    "\n",
    "        # CNTK: return trainer and loss as well\n",
    "        return model, trainer, loss\n",
    "\n",
    "    def train(self, x, y, epoch=1, verbose=0):\n",
    "        #self.model.fit(x, y, batch_size=64, nb_epoch=epoch, verbose=verbose)\n",
    "        arguments = dict(zip(self.loss.arguments, [x,y]))\n",
    "        updated, results =self.trainer.train_minibatch(arguments, outputs=[self.loss.output])\n",
    "\n",
    "    def predict(self, s):\n",
    "        return self.model.eval([s])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `Memory` class stores the different states, actions and rewards."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "class Memory:   # stored as ( s, a, r, s_ )\n",
    "    samples = []\n",
    "\n",
    "    def __init__(self, capacity):\n",
    "        self.capacity = capacity\n",
    "\n",
    "    def add(self, sample):\n",
    "        self.samples.append(sample)\n",
    "\n",
    "        if len(self.samples) > self.capacity:\n",
    "            self.samples.pop(0)\n",
    "\n",
    "    def sample(self, n):\n",
    "        n = min(n, len(self.samples))\n",
    "        return random.sample(self.samples, n)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `Agent` uses the `Brain` and `Memory` to replay the past actions to choose optimal set of actions that maximize the rewards."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "MEMORY_CAPACITY = 100000\n",
    "BATCH_SIZE = 64\n",
    "\n",
    "GAMMA = 0.99 # discount factor\n",
    "\n",
    "MAX_EPSILON = 1\n",
    "MIN_EPSILON = 0.01 # stay a bit curious even when getting old\n",
    "LAMBDA = 0.0001    # speed of decay\n",
    "\n",
    "class Agent:\n",
    "    steps = 0\n",
    "    epsilon = MAX_EPSILON\n",
    "\n",
    "    def __init__(self):\n",
    "        self.brain = Brain()\n",
    "        self.memory = Memory(MEMORY_CAPACITY)\n",
    "\n",
    "    def act(self, s):\n",
    "        if random.random() < self.epsilon:\n",
    "            return random.randint(0, ACTION_COUNT-1)\n",
    "        else:\n",
    "            return numpy.argmax(self.brain.predict(s))\n",
    "\n",
    "    def observe(self, sample):  # in (s, a, r, s_) format\n",
    "        self.memory.add(sample)\n",
    "\n",
    "        # slowly decrease Epsilon based on our eperience\n",
    "        self.steps += 1\n",
    "        self.epsilon = MIN_EPSILON + (MAX_EPSILON - MIN_EPSILON) * math.exp(-LAMBDA * self.steps)\n",
    "\n",
    "    def replay(self):\n",
    "        batch = self.memory.sample(BATCH_SIZE)\n",
    "        batchLen = len(batch)\n",
    "\n",
    "        no_state = numpy.zeros(STATE_COUNT)\n",
    "\n",
    "\n",
    "        # CNTK: explicitly setting to float32\n",
    "        states = numpy.array([ o[0] for o in batch ], dtype=np.float32)\n",
    "        states_ = numpy.array([(no_state if o[3] is None else o[3]) for o in batch ], dtype=np.float32)\n",
    "\n",
    "        p = agent.brain.predict(states)\n",
    "        p_ = agent.brain.predict(states_)\n",
    "\n",
    "        # CNTK: explicitly setting to float32\n",
    "        x = numpy.zeros((batchLen, STATE_COUNT)).astype(np.float32)\n",
    "        y = numpy.zeros((batchLen, ACTION_COUNT)).astype(np.float32)\n",
    "\n",
    "        for i in range(batchLen):\n",
    "            s, a, r, s_ = batch[i]\n",
    "\n",
    "            # CNTK: [0] because of sequence dimension\n",
    "            t = p[0][i]\n",
    "            if s_ is None:\n",
    "                t[a] = r\n",
    "            else:\n",
    "                t[a] = r + GAMMA * numpy.amax(p_[0][i])\n",
    "\n",
    "            x[i] = s\n",
    "            y[i] = t\n",
    "\n",
    "        self.brain.train(x, y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Training\n",
    "\n",
    "As any learning experiences, we expect to see the initial state of actions to be wild exploratory and over the iterations the system learns the range of actions that yield longer runs and collect more rewards. The tutorial below implements the [epsilon-greedy](https://en.wikipedia.org/wiki/Reinforcement_learning) approach (a.k.a. $\\epsilon$-greedy). "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": true,
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "def plot_weights(weights, figsize=(7,5)):\n",
    "    '''Heat map of weights to see which neurons play which role'''\n",
    "    sns.set(style=\"white\")\n",
    "    f, ax = plt.subplots(len(weights), figsize=figsize)\n",
    "    cmap = sns.diverging_palette(220, 10, as_cmap=True)\n",
    "    \n",
    "    for i, data in enumerate(weights):\n",
    "        axi = ax if len(weights)==1 else ax[i]\n",
    "        if isinstance(data, tuple):\n",
    "            w, title = data\n",
    "            axi.set_title(title)\n",
    "        else:\n",
    "            w = data\n",
    "                \n",
    "        sns.heatmap(w.asarray(), cmap=cmap, square=True, center=True, #annot=True,\n",
    "                    linewidths=.5, cbar_kws={\"shrink\": .25}, ax=axi)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exploration - exploitation trade-off\n",
    "\n",
    "Note that the initial $\\epsilon$ is set to 1 which implies we are entirely exploring but as steps increase we reduce exploration and start leveraging the learnt space to collect rewards (a.k.a. exploitation) as well."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0,0.5,'$\\\\epsilon$')"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAENCAYAAAAVPvJNAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3Xl8VOXd///XLAkkJAGSSaiAIm3t\nfYtrbbVWbxQXUBSDCFwsKlAVNygKorhWRCmLCAYVkMWtIHCBC6gIClq9EbVo719V9GeLWiugZiGB\nhECSSeb7xww2pgMkITlnJvN+Ph55mHPmmpnPNQfznnNdZ/GEQiFERETq8rpdgIiIxCYFhIiIRKWA\nEBGRqBQQIiISlQJCRESiUkCIiEhUCggREYlKASEiIlEpIEREJCq/2wUcJp0GLiLSOJ5DNYj3gGDH\njh2Nel4gEKCwsLCJq4lt6nNiUJ8Tw+H0uWPHjvVqpyEmERGJSgEhIiJRKSBERCQqBYSIiESlgBAR\nkagcOYrJGPME0AfIt9YeH+VxD5AHXASUAyOstX91ojYREYnOqT2Ip4ALD/J4b+CYyM+1wFwHahIR\nkYNwJCCstW8DOw/SpC/wjLU2ZK19D2hnjDmiuerxffUVvrvvhpqa5noLEZG4FysnynUCvqm1vC2y\n7tu6DY0x1xLey8BaSyAQaPCbeZ95Bt+DD9Jh506qH30UvIkxFeP3+xv1ecUz9TkxqM/N9B7N+ur1\nF+2U76iX0bDWzgfm72/TqDMJr7ySDsXF+KZPZ19lJbumTAHPIc86j3s62zQxqM+JwYkzqWMlILYB\nR9Za7gw07hoa9eHxUD1pEuV79pD+2GPg9bJr8uSECAkRkfqKlYBYDYw2xiwDfgPsstb+x/BSk/J4\nKL3jDjzV1aTNm0fI52P3pEkKCRGRCKcOc10K9AACxphtwL1AEoC1dh6whvAhrlsJH+b6OyfqwuNh\n9913Q3U1aQsWgNfL7okTFRIiIjgUENbaIYd4PASMcqKW/+DxsPvee6GmhrSFC8Mh8Yc/KCREJOHF\nyhCTuzwedt93Xzgk5s8Hn4/dd92lkBCRhKaA2M/jYff994fnJObOJeTzUXr77QoJEUlYCojaPJ7w\n0UzV1aRHzo8ove02hYSIJCQFRF1eL7umToVQiPTZs8MhMX68QkJEEo4CIhqvl13TpkFNDekPPwzB\noIabRCThKCAOxOtl14MPgs9H+qOP4gkGw4fEKiREJEEoIA5m/3CT30/avHlQVRU+2kkhISIJQAFx\nKJHLcIT8ftIWLcJTVRWeyE6QC/yJSOJSQNTH/vMkkpNJmzsXgsHwHIVCQkRaMAVEfXk87L7rLkJ+\nP+mPPIInGKRkxgzw+dyuTESkWSggGsLjoXTCBELJyWQ89BAEg5TMmgV+fYwi0vLoL1tDeTyUjRsH\nPh8Z06fjCQYpnj0bkpLcrkxEpEkpIBqp7KabCCUn0/aBByAYpPixxyA52e2yRESajGZZD8OeG25g\n18SJpKxZQ+bIkbBvn9sliYg0GQXEYdozciQlf/wjrdevJ2vYMDx79rhdkohIk1BANIHy4cMpzssj\n+d13yRo8GE9JidsliYgcNgVEE9k7YADFjz9O0scfEzAGb4LdQF1EWh4FRBPad9FF7HzqKXxffEFW\n//54d+xwuyQRkUZTQDSxih492LlkCb7vviNw2WX4vv7a7ZJERBpFAdEMKk8/naLly/GWlhLo1w//\n3//udkkiIg2mgGgmVSefTOFzz0FNDVn9+5P08cdulyQi0iAKiGYU/O//pvD55wmlpJA1cCDJmze7\nXZKISL05dia1MeZCIA/wAQuttVPrPN4FeALIBnYCV1hrtzlVX3Op/ulPKXrhBbIGDSJzyBCKn3iC\nirPOcrssEZFDcmQPwhjjAx4DegPdgCHGmG51ms0AnrHWnghMAqY4UZsTqjt1ovCFF6g++mgyhw2j\n9erVbpckInJITg0xnQZstdZ+aa2tBJYBfeu06QZsiPz+ZpTH41pNdjaFK1dSefLJtL/xRlKfftrt\nkkREDsqpgOgEfFNreVtkXW1/A/pHfu8HpBtjshyozTGhdu3YuXQpFeeeS7s77yRt1iwIhdwuS0Qk\nKqfmIKLdxLnuX8bxwKPGmBHA28B2IFj3ScaYa4FrAay1BAKBRhXk9/sb/dzDtmoV1ddfT8aMGbTZ\ns4fqmTMduTudq312ifqcGNTnZnqPZn31f9sGHFlruTPwo9OMrbU7gMsAjDFpQH9r7a66L2StnQ/M\njyyGCht5SYtAIEBjn9skpkwho00b0ubOpeLbb8M3Hmrmy4W73mcXqM+JQX1umI4dO9arnVMBsRk4\nxhjTlfCewWBgaO0GxpgAsNNaWwPcQfiIppbL62X3PfdQk5VFxh//iLekhOIFCwilprpdmYgI4NAc\nhLU2CIwG1gGfhVfZLcaYScaY3EizHsDnxpi/Ax2AyU7U5iqPh7JRoyiZMYNWb79N1qBBeIqL3a5K\nRAQATyi+J0lDOxp5QbxY2yVt/eqrtB81imCXLhQtWUJNPXcBGyLW+uwE9TkxqM8NExliijY3/CM6\nkzpG7Ovdm6LFi/Ht2EHg0kvxbd3qdkkikuAUEDGk8owzKFq5Ek9FBdl9+5L0wQdulyQiCUwBEWOq\nTjiBwlWrqGnXjsCgQbReu9btkkQkQSkgYlD10UdTuHo1VcceS/uRI0l96im3SxKRBKSAiFE1WVkU\nWRs+6/quu0ifMkVnXYuIoxQQMSyUmsrORYvYc/nlpD/6KO1uugkqK90uS0QShGOX+5ZG8vvZNW0a\n1R07kvHgg/jy89m5YAGh9HS3KxORFk57EPHA46Hs5pspnjmT5E2bCPTvj/e779yuSkRaOAVEHNk7\naBA7n3kG31dfEcjNxf+Pf7hdkoi0YAqIOFPRowdFzz2Hp7KSwKWXkvyXv7hdkoi0UAqIOFR14okU\nrl5NTWYmWYMGkfLii26XJCItkAIiTlUfdRQFq1ZRecoptB81irSHH9ZhsCLSpBQQcSyUmUnRs89S\nftllZDz4IO3GjtVhsCLSZHSYa7xr1YqS2bMJdu1KxkMP4du2jZ0LFxJq187tykQkzmkPoiXweCgb\nN47i2bNJ/vBDArm5+L7+2u2qRCTOKSBakL39+1O0dCm+oiICffqQtHmz2yWJSBxTQLQwlaefTsHq\n1YQyMsJXg121yu2SRCROKSBaoOqf/YzCl16i8qSTyLzxRtJmz9YRTiLSYAqIFqomM5OiZcvCRzhN\nm0a7W27REU4i0iAKiJYscoRT6bhxpC5fjr93b7xFRW5XJSJxQgHR0nk8lN5yCzvnzMHzwQcELroI\n/2efuV2ViMQBBUSC2Ne3L8ENG/BUVRHo25dWr73mdkkiEuMUEAkk9OtfU/DKKwR//nMyr7qKtDlz\nNHktIgfk2JnUxpgLgTzAByy01k6t8/hRwNNAu0ib2621a5yqL1HUHHEERc89R7tx48iYPBn/559T\nMm0atG7tdmkiEmMc2YMwxviAx4DeQDdgiDGmW51mdwPWWvtLYDAwx4naElEoJYXiOXPYPX48qStX\nEjAGb36+22WJSIxxaojpNGCrtfZLa20lsAzoW6dNCMiI/N4W2OFQbYnJ46Fs7Fh2zp+P/9NPCVx8\nMf5PPnG7KhGJIU4NMXUCvqm1vA34TZ02E4HXjDG/B9oA50d7IWPMtcC1ANZaAoFAowry+/2Nfm68\nitrn4cOpPukk/AMGkN2vH8EnnyR06aXuFNgMtJ0Tg/rcTO/RrK/+b54o6+rOjg4BnrLWPmSM+S3w\nJ2PM8dbamtqNrLXzgfn7X6OwsLBRBQUCARr73Hh1wD537ox39Woyr76a5EGDKB03jtKxY8Eb/8cw\naDsnBvW5YTp27Fivdk79BdgGHFlruTP/OYR0NWABrLXvAq2BxPpK4KKanBwKV6yg3BjSZ86k/dVX\n4yktdbssEXGRUwGxGTjGGNPVGJNMeBJ6dZ02/wLOAzDGHEs4IAocqk8AWremZOZMSh54gNYbNhDo\n0wff1q1uVyUiLnEkIKy1QWA0sA74LLzKbjHGTDLG5Eaa3QKMNMb8DVgKjLDW6iB9p3k8lP/udxQt\nX463uJjsPn10Up1IgvKE4vtEqdCOHY072Eljlofm276d9tdcQ/JHH7F7/HjKbrop7uYltJ0Tg/rc\nMJE5iGhzwz8SX/+3i6OqO3Wi8PnnKe/fn4wZM2g/cqTmJUQSiAJCDi4lhZK8PHbddx+tX3+dwCWX\n4PviC7erEhEHKCDk0Dwe9lxzDUVLl+ItLAzPS6xf73ZVItLMFBBSb5Vnnknh2rUEjzqKzBEjSJs1\nC2pqDv1EEYlLCghpkOrOnSl68UX29utHxowZZA4fjqe42O2yRKQZKCCkwUIpKZTMnk3J5Mm0+t//\nJbt3b5I+/tjtskSkiSkgpHE8HspHjKDwuefwBIME+vYldelSt6sSkSakgJDDUvWrX1Gwbh2Vp51G\nu/HjaXvLLbB3r9tliUgTUEDIYavJyqJoyRJKx4yhzbJlBC69FN/XX7tdlogcJgWENA2fj9IJEyh6\n6in833xDdu/eOhRWJM4pIKRJVfTsScGrr1LduTNZw4eTPn06VFe7XZaINIICQppcdZcuFKxaRfmg\nQaTn5ZF5xRV4E+w6OSItgQJCmkdKCiUPPUTJ9Om0ev99snv1Ivndd92uSkQaQAEhzcfjofzyyylY\nvZpQaipZxpCWl6ezr0XihAJCml3w+OMpWLuWvbm5ZEyfTubll2vISSQOKCDEEaG0NEoeffTHQ06b\nNrldlogchAJCnLN/yOmllwi1aUPWoEHhC/7pKCeRmKSAEMcFjzuOgldfZW/fvmTMmEHW5ZfjLdDt\nx0VijQJCXBFKS6PkkUcoefBBkjdvDg85vfOO22WJSC0KCHGPx0P50KEUvPQSNenpZA0eTPpDD0Ew\n6HZlIoICQmJAsFs3Cl99lb2XXkr6zJlkDRyIb/t2t8sSSXgKCIkJoTZtKHnkEYrz8kjasoXsnj1p\n/fLLbpclktD8Tr2RMeZCIA/wAQuttVPrPD4LOCeymArkWGvbOVWfxIa9AwZQ+etf0370aDKvu449\nQ4ey+777CKWmul2aSMJxZA/CGOMDHgN6A92AIcaYbrXbWGvHWmtPttaeDDwCPO9EbRJ7qo8+msIX\nXqB09GhSly4l0Ls3/k8+cbsskYTj1BDTacBWa+2X1tpKYBnQ9yDthwC6PVkiS0qi9I47KFq6FG9p\nKdmXXEKbBQsgFHK7MpGE4VRAdAK+qbW8LbLuPxhjugBdgTccqEtiXGX37hSsX0/F2WfTduJEMocN\n02U6RBzi1ByEJ8q6A30VHAystNZGPb3WGHMtcC2AtZZAINCogvx+f6OfG6/its+BALz0EsF582g1\nYQIdevUiuGgRoZ49D/nUuO3zYVCfE4MTfXYqILYBR9Za7gzsOEDbwcCoA72QtXY+MD+yGCps5LfJ\nQCBAY58br+K+zwMH4j/+eNqPGkVSnz6UjRzJ7ttvh9atD/iUuO9zI6jPieFw+tyxY8d6tavXEJMx\n5q91J5WNMW2NMVn1rGczcIwxpqsxJplwCKyO8j7/BbQHdOMAiSp47LEUvPIKe0aMIG3BArIvvhj/\nli1ulyXSItV3DuLn1tpPAYwxIyPrMoBX6/Nka20QGA2sAz4Lr7JbjDGTjDG5tZoOAZZZazUTKQeW\nksKuyZMpWrwY786dZPfpQ5u5c3XRP5Em5gnV46gQY0wBcKS1dp8xpsxamxZZX2ytbd/cRR5EaMeO\nA41UHZx2SVsG786dtJ0wgZQ1a6g4/XRK8vKo7tz5h8dbYp8PRX1ODE0wxBRtbvhH6rsH8Tow3xhz\nD1BjjPmpMSa9Pm8g0pxqMjMpnj+f4lmzSPrkE7LPP5+UFSt0OKxIE6hvQFwP7AGygduBl4HnAF1+\nU9zn8bDXGArWr6eqWzfa33wz7a+7Ds/OnW5XJhLX6nUUk7V2N3DD/mVjzDbgOODJZqpLpMGqjzyS\nohUrSHv8cdKnTyfngw+oWbgQTjnF7dJE4lK95iBimOYgGiCR+uz/5BPajxlD0uefU/a731F6112E\nUlLcLssRibSd91OfG6ap5yBE4krw+OMpWLOG6jFjSHvySbJ79iRp82a3yxKJKwoIablat6b6wQcp\nXLECgkEC/fqRcf/9sHev25WJxAUFhLR4lWecQcH69ZRfcQVp8+aRfeGFJP31r26XJRLzFBCSEEJp\naeyaOpWipUvxlJcT6NuX9ClToKLC7dJEYpYCQhJKxVlnUbBhA+WDBpH+6KNk9+5N0kcfuV2WSExS\nQEjCCWVksGvGDIr+9Ce8u3YR6NOH9AcfhMpKt0sTiSkKCElYFeeeS/6GDezt14/0hx/Whf9E6lBA\nSEILtWtHSV4eRU8+ibeggOyLLiL9oYe0NyGCAkIEgIpevch/4w325uaSPnOmjnQSQQEh8oNQZiYl\njzxC0dNP4929m0BuLhkTJ+IpL3e7NBFXKCBE6qg4/3zy33wzfN7EggVkn38+yRs3ul2WiOMUECJR\nhNLT2TV1KoUrV4LHQ2DQINreeiueXbvcLk3EMQoIkYOo/O1vyV+/ntIbbyR12TJyzjmH1uvWuV2W\niCMUECKHkpJC6V13Ufjyy9RkZpJ51VW0v/56vAUFblcm0qwUECL1VHXSSRS8+iq7b7uN1uvWkdOj\nBynLl+vuddJiKSBEGiIpibKbbqLgtdcI/vzntB83jqyBA/Ft3ep2ZSJNTgEh0gjBY46h8IUXKJk6\nlaQtW8jp2ZO0WbN08T9pURQQIo3l9VJ+5ZXkv/UWe3v3JmPGDLJ79iT53XfdrkykSSggRA5TTU4O\nJXPmULR4MZ6qKgIDBtBu3Dg8O3e6XZrIYfE79UbGmAuBPMAHLLTWTo3SxgATgRDwN2vtUKfqEzlc\nFeecQ8Ebb5D28MOkzZtHq9dfZ/cf/sDeAQPAc8jb/4rEHEf2IIwxPuAxoDfQDRhijOlWp80xwB3A\nmdba44CbnahNpCmFUlIoveMOCtaupbprV9rffDNZgwbh++ILt0sTaTCnhphOA7Zaa7+01lYCy4C+\nddqMBB6z1hYDWGvzHapNpMkFjz2WwhdfpGTKFJI+/vjfk9j79rldmki9ORUQnYBvai1vi6yr7RfA\nL4wx7xhj3osMSYnEL6+X8mHDyH/rLfZdcAEZM2aQc955tHrjDbcrE6kXp+Ygog3A1j27yA8cA/QA\nOgP/a4w53lpbUruRMeZa4FoAay2BQKBRBfn9/kY/N16pzy4JBGDFCqreeAP/zTeTdeWV1OTmEpwx\nA7p0afK3i4k+O0x9bqb3aNZX/7dtwJG1ljsDO6K0ec9aWwV8ZYz5nHBgbK7dyFo7H5gfWQwVFhY2\nqqBAIEBjnxuv1GeXnXgirF1L2oIFpM2ahf/EEym76SbKrrsOWrVqsreJqT47RH1umI4dO9arnVND\nTJuBY4wxXY0xycBgYHWdNi8C5wAYYwKEh5y+dKg+EWckJ1M2ahT5b71FxbnnkjFtWnjY6a233K5M\n5D84EhDW2iAwGlgHfBZeZbcYYyYZY3IjzdYBRcaYT4E3gVuttUVO1CfitJpOnShesICiJUsAyBo6\nlPYjR+Ldvt3lykT+zROK7wuNhXbsqDtSVT/aJU0McdHnigrSHn+ctLw88HgoGzuWspEjITm5US8X\nF31uYupzw0SGmA55co7OpBZxW6tWlI0ZQ8Gf/0zF2WeT8cc/kt2zp4adxHUKCJEYUX3kkRQvWkTR\nM8/gCQbJGjqUzBEj8H31ldulSYJSQIjEmIrzziP/jTfYfdddJG/aRM4555A+eTKesjK3S5MEo4AQ\niUWtWlF2443kb9zI3ssuI33OHHK6dw/foKimxu3qJEEoIERiWE1ODiUzZ1LwyitUd+5M+3HjCFxy\nCUkffuh2aZIAFBAicaDq5JMpXLWK4tmz8X33Hdm5ubT7/e/xfved26VJC6aAEIkXXi97+/cn/+23\nKf3970l55RVyuncnbfZsXQRQmoUCQiTOhNq0ofT228l/883wYbHTppHTowetV62C+D6vSWKMAkIk\nTlV36ULxwoUULltGKC2NzBtvJJCbi+e999wuTVoIBYRInKvs3p2CdesonjkT3/btJJ19Nu2vuw7f\n11+7XZrEOQWESEvg87F30CDyN24keM89tNqwgZwePciYNAnPrl1uVydxSgEh0oKEUlOpuftu8jdu\npPyyy2gzfz4dzjiDNosWQVWV2+VJnFFAiLRANT/5CbseeoiCdeuoOuEE2v7hD+Sccw6t167VRLbU\nmwJCpAULHnccRUuXUvTMM4T8fjKvvpqsgQNJ+tvf3C5N4oACQqSl83ioOO88Ctavp2TKFPx//zvZ\nF10Unsj+UvfkkgNTQIgkCr+f8mHDyN+0idJx42j1xhvk9OhB29tvx/v9925XJzFIASGSYEJpaZTe\ncgv5mzaxZ9gwUpcuJefMM0mfNg3P7t1ulycxRAEhkqBqsrPZ/cAD5L/1FvsuuID02bPJOeMM2jz+\nuC7dIYACQiThVR99NCWPPUbB2rVUnXgibSdNIuess0ixFqqr3S5PXKSAEBEAqk44gZ3PPkvhsmXU\nZGXRfuxYsnv1otXrr+vQ2ASlgBCRH6ns3p3CV15h59y5ePbtI2vECLL69SN50ya3SxOHKSBE5D95\nvezLzSX/z38OHxr7zTcEBg4ka9Ag3awogSggROTAkpIoHzaM7zduZNfEifg/+4zs3Fwyhw3D/8kn\nblcnzczv1BsZYy4E8gAfsNBaO7XO4yOAB4HtkVWPWmsXOlWfiBxESgp7Ro6kfOhQ2jz5JGlz55Jz\nwQXsvfhiSsePJ/iLX7hdoTQDRwLCGOMDHgN6AtuAzcaY1dbaT+s0XW6tHe1ETSLScKE2bSgbPZo9\nw4aRNn8+bRYsoPWaNezt14/SceOo7trV7RKlCTk1xHQasNVa+6W1thJYBvR16L1FpImFMjIoHT+e\n/HffpeyGG2i9Zg05Z59N29tuw7d9+6FfQOKCU0NMnYBvai1vA34TpV1/Y8xZwN+Bsdbab+o2MMZc\nC1wLYK0lEAg0qiC/39/o58Yr9TkxONrnQABmzSI4YQK+6dNJXbCA1BUrqLnqKqpvvRU6d3akDG3n\nZnqPZn31f/NEWVf3wOqXgKXW2gpjzPXA08C5dZ9krZ0PzN//GoWFhY0qKBAI0Njnxiv1OTG40me/\nH+68E+/w4aTn5ZG6cCHeJ56gfPBgykaPprpTp2Z9e23nhunYsWO92jk1xLQNOLLWcmdgR+0G1toi\na21FZHEB8CuHahORJlLTqRO7pk8n/513KDfmh+s8tZ0wAd+2bW6XJw3kVEBsBo4xxnQ1xiQDg4HV\ntRsYY46otZgLfOZQbSLSxKo7d2bXtGnhoBgyhFRryfmf/wnPUfzrX26XJ/XkSEBYa4PAaGAd4T/8\n1lq7xRgzyRiTG2k2xhizxRjzN2AMMMKJ2kSk+VR36sSuKVP4/p13KL/8clJXrCCne3fa3nILvq+/\ndrs8OQRPKL6vsRLasWPHoVtFoTHLxKA+xxbvt9+SNmcObZYsgWCQvf37UzpmzGEfHhvLfW4uTTAH\nEW1u+Ed0JrWIOKbmiCPYff/9fL9pE3tGjCBl9Wpyzj6bdmPG4P/HP9wuT+pQQIiI42p+8hN2T5oU\nDoqrrqL1K6+Qfc45tB85UvfLjiEKCBFxTU2HDuyeOJH8v/yFsjFjaLVxI9kXXUTm0KEkv/uuLjPu\nMgWEiLiuJiuL0ttu4/u//IXdd95J0pYtBAYMIHDppbRav15B4RIFhIjEjFB6OmWjRvH9e+9RMnky\n3u++I2v4cLJ79qT1qlW6w53DFBAiEntSUigfMYL8jRspfvhhqKoi88YbyTnrLFKffRYqKg79GnLY\nFBAiEruSktg7cCAFb77JzoULqcnIoN2tt9LhjDNoM28entJStyts0RQQIhL7vF729e5N4Zo1FC1d\nSrBrV9refz8dTjuN9MmToZHnQ8nBKSBEJH54PFScdRZFK1dSsGYNFWefTdq8eST94he0GzsW/+ef\nu11hi6KAEJG4VHXSSRTPm0f+xo3UXHMNrV96iZxzzyXzyitJ3rRJRz41AQWEiMS16i5dqH744fAh\nsrfeStJHHxEYOJDAxRfTevVqCAbdLjFuKSBEpEUIZWZSdvPN4UNkp03DW1pK5g03kNO9O22eeAJP\nebnbJcYdBYSItCwpKZRfcQX5b73FzkWLqMnJoe0999Dh1FNJnzIF77fful1h3FBAiEjL5PWy78IL\nKVy1ioIXX6Tit78lbc4cOpx+Ou1Gjybp//7P7QpjngJCRFq8qlNPpXjhQvLfeSd8ccD168nu04dA\n3760fuklzVMcgAJCRBJG9VFHsfvee/n+gw/YNWkS3sJCMq+/npzf/pY2c+fiKSlxu8SYooAQkYQT\nSktjz9VXk//22xQ9+STVRx9N2wceoMOvf03bO+/Et3Wr2yXGBAWEiCQun4+KXr0oWrGC/NdeY19u\nLqlLl9Lh7LPJvPJKWr39dkKfT6GAEBEBgscdR8nMmeHzKcaPJ+njj8kaMoTsHj3Ch8km4HWfFBAi\nIrXUZGdTNnYs37//PsV5eYTS08OHyZ5yCm3vuCOhLuehgBARiaZVK/YOGEDhyy9TsGYN+/r0IXX5\ncnLOPZesAQNo/fLLUFXldpXNSgEhInIIVSedRMmsWXz/wQfsvusufNu2kXnddXQ4/XTSZs3Cm5/v\ndonNwu/UGxljLgTyAB+w0Fo79QDtBgArgFOttR84VZ+IyKHUZGZSduONlF13Ha02bKDN00+TMWMG\n6Xl57L3oIspHjKDy1FPB43G71CbhyB6EMcYHPAb0BroBQ4wx3aK0SwfGAO87UZeISKNEjn7auWQJ\n37/9NnuGD6f1m28S6NeP7F69SP3Tn/CUlbld5WFzaojpNGCrtfZLa20lsAzoG6Xd/cB0YJ9DdYmI\nHJbqn/2M3ffdx/cffkjJ9OkAtLv9djr88pe0ve02kj76yOUKG8+pgOgEfFNreVtk3Q+MMb8EjrTW\nvuxQTSIiTSaUmkr55ZdT8NprFLz0EvsuuYSU554ju3dvAr17k7p4cdztVTg1BxFtQO6Hs0+MMV5g\nFjDiUC9kjLkWuBbAWksgEGjAtOpEAAAJVElEQVRUQX6/v9HPjVfqc2JQn2NAr17QqxfBkhK8S5fi\nX7iQdhMm0Pb++6kZPJiakSMJnXzyYb2FE312KiC2AUfWWu4M1L6JbDpwPPBnYwzAT4DVxpjcuhPV\n1tr5wPzIYqiwsLBRBQUCARr73HilPicG9TnGDBwIAwaQ9OGHtFm8mJTFi/EtXEjlSSdRfsUV7O3b\nl1CbNg1+2cPpc8eOHevVzhNy4DRyY4wf+DtwHrAd2AwMtdZuOUD7PwPj63EUU2hHI29WHtP/oJqJ\n+pwY1OfY5ikpIfX550ldvJikzz+nJi2Nvf36seeKKwgef3y9X6cJAuKQh1o5MgdhrQ0Co4F1wGfh\nVXaLMWaSMSbXiRpERGJBqF079lx1FQUbNlD44ovsu+ACUq0l54ILCFxwAalPPRUzV5V1ZA+iGWkP\nogHU58SgPscfT3ExqS+8QOrSpSR9+imhVq3Y27s35YMHU3nmmeD9z+/yLWYPQkREDizUvn14r+L1\n1ylYu5byIUPC51UMHkzOGWeQNmsWvu3bHa9LASEiEkOqTjiBXZMn892HH1L86KNUd+lCxowZ5Pzm\nN2QOHUrr1auhosKRWhQQIiKxKCWFvf36UbR8Od+/+y5lN92E/x//IPOGG/jJKafgXb682UtQQIiI\nxLjqo46i9NZbyX/vPYqWLKGie3dCRx3V7O/r2MX6RETkMPl8VPToQUWPHuGT5Jp5Yl57ECIiEpUC\nQkREolJAiIhIVAoIERGJSgEhIiJRKSBERCQqBYSIiESlgBARkaji/mqubhcgIhKnWvzVXD2N/THG\nfHg4z4/HH/U5MX7U58T4aYI+H1K8B4SIiDQTBYSIiESVyAEx3+0CXKA+Jwb1OTE0e5/jfZJaRESa\nSSLvQYiIyEEk5P0gjDEXAnmAD1horZ3qckmNYow5EngG+AlQA8y31uYZYzKB5cDRwD8BY60tNsZ4\nCPf7IqAcGGGt/WvktYYDd0de+gFr7dNO9qWhjDE+4ANgu7W2jzGmK7AMyAT+Clxpra00xrQi/Bn9\nCigCBllr/xl5jTuAq4FqYIy1dp3zPakfY0w7YCFwPOHDu68CPqcFb2djzFjgGsL9/Rj4HXAELWg7\nG2OeAPoA+dba4yPrmuz/X2PMr4CngBRgDXCTtbbew0YJtwcR+cPyGNAb6AYMMcZ0c7eqRgsCt1hr\njwVOB0ZF+nI7sMFaewywIbIM4T4fE/m5FpgLP/yDvBf4DXAacK8xpr2THWmEm4DPai1PA2ZF+lxM\n+A8Ckf8WW2t/DsyKtCPyOQ0GjgMuBOZE/m3EqjxgrbX2v4GTCPe9xW5nY0wnYAzw68gfTh/h7dXS\ntvNThOuqrSm369xI2/3Pq/teB5VwAUH4A9xqrf3SWltJ+NtIX5drahRr7bf7v0FYa0sJ/9HoRLg/\n+78ZPg1cGvm9L/CMtTZkrX0PaGeMOQK4AHjdWrvTWlsMvE4D/yE5yRjTGbiY8DdqIt+szgVWRprU\n7fP+z2IlcF6kfV9gmbW2wlr7FbCV8L+NmGOMyQDOAhYBWGsrrbUltPDtTHiEI8UY4wdSgW9pYdvZ\nWvs2sLPO6ibZrpHHMqy170b2Gp6p9Vr1kogB0Qn4ptbytsi6uGaMORr4JfA+0MFa+y2EQwTIiTQ7\nUN/j7TN5GLiN8LAaQBZQYq0NRpZr1/9D3yKP74q0j6c+/xQoAJ40xvyfMWahMaYNLXg7W2u3AzOA\nfxEOhl3Ah7Ts7bxfU23XTpHf666vt0QMiGhnEMb1oVzGmDTgOeBma+3ugzQ9UN/j5jMxxuwfr/2w\n1uqD1R/3fSb8TfoUYK619pfAHv497BBN3Pc5MkTSF+gKdATaEB5iqaslbedDaWgfD7vviRgQ24Aj\nay13Bna4VMthM8YkEQ6HJdba5yOrv4/sXhL5b35k/YH6Hk+fyZlArjHmn4SHB88lvEfRLjIUAT+u\n/4e+RR5vS3iXPp76vA3YZq19P7K8knBgtOTtfD7wlbW2wFpbBTwPnEHL3s77NdV23Rb5ve76ekvE\ngNgMHGOM6WqMSSY8gbXa5ZoaJTLGugj4zFo7s9ZDq4Hhkd+HA6tqrR9mjPEYY04HdkV2YdcBvYwx\n7SPf3HpF1sUca+0d1trO1tqjCW+7N6y1lwNvAgMizer2ef9nMSDSPhRZP9gY0ypyBNQxwF8c6kaD\nWGu/A74xxvxXZNV5wKe04O1MeGjpdGNMauTf+f4+t9jtXEuTbNfIY6XGmNMjn+GwWq9VLwkXEJHx\nydGEP9TPwqvsFnerarQzgSuBc40x/1/k5yJgKtDTGPMPoGdkGcKHuX1JeKJuAXAjgLV2J3A/4fDc\nDEyKrIsnE4BxxpithMeeF0XWLwKyIuvHERmaiWxzS/iPzlpglLW22vGq6+/3wBJjzEfAycAfacHb\nObK3tJLwoawfE/5bNZ8Wtp2NMUuBd4H/MsZsM8ZcTdNu1xsIH8yxFfgCeLUh9elMahERiSrh9iBE\nRKR+FBAiIhKVAkJERKJSQIiISFQKCBERiUoBISIiUSkgRA6DMWaiMWax23WINAcFhIiIRKUT5UTq\nyRgzgfA9CjIIX9NmHPAC4YuiVQBfWGtPMsa0BWYSvrFLDfAkcK+1ttoYMwIYSfgM4WGEr1Q6ylq7\nweHuiByS9iBE6iFyHaTRwKnW2nTC1+D//wlf8mK5tTbNWntSpPnThG/m9HPCl2DvRfjOaPv9hvAl\nEwKEb/TyfOSmLyIxRQEhUj/VQCugmzEmyVr7T2vtF3UbGWM6EL4s9c3W2j3W2nzCdzgbXKtZPvCw\ntbbKWruc8K1DL27+Log0TELek1qkoay1W40xNwMTgeOMMesIDzHV1QVIAr41xuxf5+XHN3TZXue+\nwF8TvueBSExRQIjUk7X2WeDZyC1AHyd83+OtdZp9Q3g+IlDrzmd1dTLGeGqFxFHE6SXnpWXTJLVI\nPUTmIDoB7xC+K9c8wnsG7wFXAGdZa2sibVcB/wTuAcoI3xWts7X2rcgk9ULgFmAO4XsELwK6WmuL\nHOySyCFpDkKkfloRvi5/IfAd4fsE3wmsiDxeZIz5a+T3YUAy4XsQFBO+r8ERtV7rfcI3rikEJgMD\nFA4Si7QHIeKgyB7ENdba/3G7FpFD0R6EiIhEpYAQEZGoNMQkIiJRaQ9CRESiUkCIiEhUCggREYlK\nASEiIlEpIEREJCoFhIiIRPX/ACUx34CYedb8AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x23270c60668>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "def epsilon(steps):\n",
    "    return MIN_EPSILON + (MAX_EPSILON - MIN_EPSILON) * np.exp(-LAMBDA * steps)\n",
    "plt.plot(range(10000), [epsilon(x) for x in range(10000)], 'r')\n",
    "plt.xlabel('step');plt.ylabel('$\\epsilon$')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We are now ready to train our agent using **DQN**. Note this would take anywhere between 2-10 min and we stop whenever the learner hits the average reward of 200 over past 50 batches. One would get better results if they could train the learner until say one hits a reward of 200 or higher for say larger number of runs. This is left as an exercise."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Episode: 20, Average reward for episode 21.400000.\n",
      "Episode: 40, Average reward for episode 24.450000.\n",
      "Episode: 60, Average reward for episode 20.250000.\n",
      "Episode: 80, Average reward for episode 20.900000.\n",
      "Episode: 100, Average reward for episode 21.750000.\n",
      "Episode: 120, Average reward for episode 16.400000.\n",
      "Episode: 140, Average reward for episode 21.300000.\n",
      "Episode: 160, Average reward for episode 20.150000.\n",
      "Episode: 180, Average reward for episode 18.250000.\n",
      "Episode: 200, Average reward for episode 16.250000.\n",
      "Episode: 220, Average reward for episode 16.200000.\n",
      "Episode: 240, Average reward for episode 14.000000.\n",
      "Episode: 260, Average reward for episode 14.150000.\n",
      "Episode: 280, Average reward for episode 16.000000.\n",
      "Episode: 300, Average reward for episode 13.800000.\n",
      "Episode: 320, Average reward for episode 13.650000.\n",
      "Episode: 340, Average reward for episode 12.400000.\n",
      "Episode: 360, Average reward for episode 12.900000.\n",
      "Episode: 380, Average reward for episode 13.000000.\n",
      "Episode: 400, Average reward for episode 11.550000.\n",
      "Episode: 420, Average reward for episode 11.400000.\n",
      "Episode: 440, Average reward for episode 12.400000.\n",
      "Episode: 460, Average reward for episode 12.250000.\n",
      "Episode: 480, Average reward for episode 11.650000.\n",
      "Episode: 500, Average reward for episode 11.700000.\n",
      "Episode: 520, Average reward for episode 12.100000.\n",
      "Episode: 540, Average reward for episode 12.650000.\n",
      "Episode: 560, Average reward for episode 11.300000.\n",
      "Episode: 580, Average reward for episode 11.250000.\n",
      "Episode: 600, Average reward for episode 11.000000.\n",
      "Episode: 620, Average reward for episode 11.450000.\n",
      "Episode: 640, Average reward for episode 10.850000.\n",
      "Episode: 660, Average reward for episode 11.850000.\n",
      "Episode: 680, Average reward for episode 12.200000.\n",
      "Episode: 700, Average reward for episode 10.900000.\n",
      "Episode: 720, Average reward for episode 11.700000.\n",
      "Episode: 740, Average reward for episode 11.050000.\n",
      "Episode: 760, Average reward for episode 11.400000.\n",
      "Episode: 780, Average reward for episode 11.650000.\n",
      "Episode: 800, Average reward for episode 11.200000.\n",
      "Episode: 820, Average reward for episode 11.150000.\n",
      "Episode: 840, Average reward for episode 11.900000.\n",
      "Episode: 860, Average reward for episode 11.300000.\n",
      "Episode: 880, Average reward for episode 11.300000.\n",
      "Episode: 900, Average reward for episode 10.900000.\n",
      "Episode: 920, Average reward for episode 11.050000.\n",
      "Episode: 940, Average reward for episode 11.000000.\n",
      "Episode: 960, Average reward for episode 11.350000.\n",
      "Episode: 980, Average reward for episode 9.700000.\n",
      "Episode: 1000, Average reward for episode 12.250000.\n",
      "Episode: 1020, Average reward for episode 10.700000.\n",
      "Episode: 1040, Average reward for episode 10.900000.\n",
      "Episode: 1060, Average reward for episode 11.750000.\n",
      "Episode: 1080, Average reward for episode 10.900000.\n",
      "Episode: 1100, Average reward for episode 10.900000.\n",
      "Episode: 1120, Average reward for episode 12.050000.\n",
      "Episode: 1140, Average reward for episode 11.550000.\n",
      "Episode: 1160, Average reward for episode 12.000000.\n",
      "Episode: 1180, Average reward for episode 12.750000.\n",
      "Episode: 1200, Average reward for episode 13.400000.\n",
      "Episode: 1220, Average reward for episode 14.100000.\n",
      "Episode: 1240, Average reward for episode 25.950000.\n",
      "Episode: 1260, Average reward for episode 22.700000.\n",
      "Episode: 1280, Average reward for episode 13.850000.\n",
      "Episode: 1300, Average reward for episode 10.350000.\n",
      "Episode: 1320, Average reward for episode 11.600000.\n",
      "Episode: 1340, Average reward for episode 114.150000.\n",
      "Task solved in 1340 episodes\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAtoAAABWCAYAAADxAzG5AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3XlcVOX+B/DPMIgLi6TmTa+5h1rm\ngl1vFpKoaHIVETcwESPFBcUFiUVUBAQhl7JyQU0FFzAt0fSKGCq53jSpUDFLsy7dTEWvDKYs8/z+\nmOvkjMPMmXNcoN/n/XrN6+XMeeZ7npnvPGcezzycr0oIIUBERERERA+VzZPuABERERHRnxEn2kRE\nREREjwAn2kREREREjwAn2kREREREjwAn2kREREREjwAn2kREREREjwAn2kREREREjwAn2kRENYC3\ntzf27dunv19YWIh27dphx44d+scuXLiArl274tatW0+ii0REZMT2SXeAiOh+7dq1g4uLC2xsDM8D\nfPjhh2jWrJnJ53z77bdYvXo1li1b9lD6sHfvXmzatAnp6emSn5OVlYW1a9dCpVKhbt26mD17Nl58\n8UUAwMGDB7F48WKUlZWhXbt2SExMhIODg8Vt93NyckJJSYn+/oYNG+Do6GjwWHp6Onx8fODk5CT3\npRMRVXsf7jticD+k36tVttVqtYiNjcX58+dhZ2eHhIQEtGjRQr9906ZN+OSTT6BSqRASEgIPDw/c\nuXMH4eHhuH79Ouzt7ZGcnIwGDRrI6ivPaBNRtbNhwwZkZWUZ3KqaZAPAiy+++NAm2XJcvHgR77zz\nDtasWYOsrCxMmjQJU6dOBQAUFxcjKioK77//PrKzs/Hss89i0aJFFrcZc3BwgEaj0T/vwIED8PX1\n1U+0b926hV27dmH06NGP4RUTEdUM+/fvR1lZGTIzMxEWFoaFCxfqtxUXF2Pz5s3IyMjA+vXrERsb\nCyEEtmzZAhcXF2zevBk+Pj5Yvny57P1zok1ENcaJEycwfPhwTJs2DYMGDcLw4cPxww8/4MSJExg4\ncCAAoLS0FKGhoRg8eDCGDBmCmJgYaLVaAEBmZiYGDhwIb29vBAUF4dKlS/rY7733Hvr27Ythw4Yh\nJydH/3hubi6GDx8OHx8f+Pn54fTp0w/0695ZksaNGwMAOnbsiGvXrqGsrAyHDx/Giy++iJYtWwIA\n/P39sWvXLgghzG4z5ujoqJ9oZ2RkwMvLC02bNtU/tn37dri6uqJNmzYK32UioupNCGFwM+fUqVPo\n2bMnAKBLly4oKCjQb2vQoAGysrJQq1YtXLt2DU5OTlCpVAbPcXd3x7Fjx2T3lRNtIqp2AgMDMXjw\nYP0tJCREv62goAABAQHYtWsXfH19ER4ebvDcnJwclJaWIisrC9u2bQMA/Pzzzzh27BjWrFmDtLQ0\n7Ny5EwMHDkRISAiEENi/fz/27duHHTt2ICMjQz95/fHHH7F06VKkpqZix44diI+Px9SpU3H79m2D\nfTZr1gy9evUCoPsCSEpKQu/evWFnZ4dff/0VzzzzjL7tM888A41Gg9LSUrPbjN2baJeXlyMjIwOj\nR4+Gvb09NBoNtFotNm3ahDFjxqCkpATDhg1D165d8d133ylLBBFRNSSE4c0cjUZjsBxPrVajoqJC\nf9/W1hYbN27EyJEj0b9/f/1zHB0dAQD29vYGS/SsxTXaRFTtbNiwocr1cO3bt8dLL70EABg6dCji\n4uJw8+ZN/fZu3bph6dKlCAgIwCuvvILAwEC0aNECmZmZ8PLy0sf19fXFggUL8O9//xvHjh2Dp6en\n/mA8dOhQpKen48iRI/jtt98wduxYfXyVSoWffvoJ7du3f6Bvt2/fRmRkJH799VesWbMGgG59oEql\neqCtjY2N2W3GHB0dcePGDWRnZ6N9+/Zo3bo1zp8/j5KSEhw8eBBqtRru7u6oqKhAamoqUlJSqnp7\niYhqNK2l2fV9HBwcDE5eaLVa2NoaTn9Hjx6NESNGYPz48Th+/LjBc0pLSxX93QvPaBNRjaJWq80+\n9uyzzyInJwfBwcHQaDR48803kZubq18+cj8hhP7Mxv0/P96Lp9Vq0aNHD4O14lu3bsVzzz33QKxf\nfvkFfn5+UKvVSEtL0x+YmzRpgt9++03f7sqVK6hfvz7q1atndpuxe2u009LSMGbMGIPHNm7ciICA\nAKhUKtSqVUv2H+0QEdUE1iwdcXV1RV5eHgAgPz8fLi4u+m0XL17ElClTIIRArVq1YGdnBxsbG7i6\nuuLQoUMAgLy8PHTr1k12XznRJqIapbCwEIWFhQB0a667du2q/4kPADZv3oyoqCi4ubkhPDwcbm5u\nOHv2LHr27Ik9e/aguLgYgG5Ns7OzM1q0aAF3d3fs3bsXt27dglarRVZWFgCgR48eOHLkCH744QcA\nwKFDh+Dt7Y07d+4Y9Emj0SAgIAD9+vXD0qVLUadOHf02Nzc3fP311/jxxx8B6NZX9+nTx+I2Y46O\njjh58iQ0Gg3c3NwA6Cba586dw7fffoshQ4YoeVuJiGoMaybanp6esLOzg5+fH5KSkhAVFYV169bh\n888/R+vWrdG+fXuMHDkSfn5+6Ny5M7p37w5/f39cuHAB/v7+yMzMxJQpU2T3lUtHiKjaCQwMfGD5\nxMyZM1GnTh00atQI7777LoqKitCgQQOkpKSgqKhI387Hxwf/+te/4OXlhbp166JJkyYICAhA/fr1\nMXbsWAQGBkKr1aJBgwZYtWoVbGxs8Nprr+H8+fMYOnQonJyc0L59e9y4cQNt27ZFXFwcZs6cCSEE\nbG1tsWLFCtjb2xv0bdOmTfjll1+Qk5Nj8IeU69evR8OGDZGUlITQ0FCUl5ejefPmSE5OBgCz24w5\nOjriypUrmDx5sv4xBwcHXL16FYGBgQ/0iYjoz8qapSM2NjaIi4szeOz+PxqfMmXKAxPpunXrPrQr\nWamEpf8KEBFVEydOnEB8fDw+++yzJ92Vai8yMhJBQUEGP5MSEf0ZLP7soMH9sIG9nkg/pOAZbSKi\nP5nx48fj3LlzuHTpEkaOHAlfX98n3SUiooemJp0h5hltIiIiIqoxUnYdMLj/9iCPJ9QTy3hGm4iI\niIhqjJp0jpgTbSIiIiKqMbRaTrSJiIiIiB46ntEmIiIiInoErLm835P2yCfaA5NXm93+WcR4AMD7\new+bbTf1dTd4zP/Q4v4OzAvBzLQsi+2WjBmMKR99YrbNB0G6v9TvG7/cbLv9c3TXtQ1L32m23eIA\nbwStyLDYt48m+SHx0/1m20QP6QsAWGR0iRtjs/53yRtL+/1okh/e3rTLYt9S3hj0wGV1jN27zM4b\ny9LNttsUGgAAGLZkndl222a+iXErLb9vayb6IW77PrNt5g7tBwCI2rLbbLsk/38AAELXmf+MLHvT\nFyPf3WCxb5nTA/GPhakW2+2ODMa09Z+abfPe2CGS+wYAXknm97snKlhy33rFfmC2zcHYKVb1LWKz\n+cv0JY8aiMDlmy32bcPkUZI/l1KPSd4pa8y22/n2OASnbrXYt9TgEZI+4wDw3j+/MNtu2oCeAKQd\nL6XmVOq4n/fxXrNt5g9/HQCQ8EmO2XYxvp4ApOXB0mcX0H1+/d5NM9smY7qukmbyzlyz7SK8ewMA\n3ly+xWy7dZP9Lba5127y2m1m2yx/axgAYOLqj822Wzl+OADg9cRVZtvtjZ6AOVv/abFv8SMGSDo2\nAMCQRR+ZbffprCAAgM87a8222xH+FmZs2GGxb0sDfSx+BwK678GkHZ+bbRPloysAJTUPUuYFlmLd\niyclVwCwdPchs+1m/OM1AJD0/Sb1+0jqMbq6qjnTbJ7RJiIiIqIaxJqlI1qtFrGxsTh//jzs7OyQ\nkJCAFi1aGLQpLi6Gn58fdu3ahdq1a0MIAXd3d7Rs2RIA0KVLF4SFhcnqq+SJtlarfaBSGxERERFR\ndbV//36UlZUhMzMT+fn5WLhwIVasWKHf/sUXX2Dx4sW4du2a/rGffvoJL7zwAlauXKl4/2Znzj//\n/DMmT54Md3d39O3bF7169UJwcDAuXbqkeMdERERERNbSCmFwM+fUqVPo2VO37K5Lly4oKCgw2G5j\nY4N169bB2dlZ/9iZM2dw5coVBAQEYPz48bh48aLsvpo9oz179myEhYWhc+fO+sfy8/MRFRWFjAzL\na2aJiIiIiB4ma5aOaDQaODg46O+r1WpUVFTA1lY3BX711VcfeM7TTz+N4OBgDBgwACdPnkR4eDi2\nb98uq69mJ9plZWUGk2xA978BIiIiIqInwZqrjjg4OKC0tPSP52q1+kl2VTp27Ai1Wg0AeOmll3Dl\nyhUIIaBSqazuq9k9tWvXDlFRUejZsyccHR1RWlqKQ4cOoV27dlbviIiIiIhIKWuu7ufq6ooDBw7A\ny8sL+fn5cHFxsficDz74AM7Ozhg/fjwKCwvRtGlTWZNswMJEOzY2Fvv378epU6f0p949PDzg6ekp\na2dEREREREpYs3TE09MTR44cgZ+fH4QQSExMxLp169C8eXP06dPH5HOCg4MRHh6OQ4cOQa1WIykp\nSXZfzU60VSoVPD09ObEmIiIiomrBmqUjNjY2iIuLM3isTZs2D7TLzf3jOvv169dHaqrl6/hLoRI1\nqY4lEREREf2/ZlzsLHnUwCfUE8seecGaMR9uMrs9LeQNAJBUcc4zYYXZNgCQEzPJYhU2QFeJTWqV\nO0tVCddM9AMgrerYw65wODtjj9l2C/y8AEirbim18qbUio+WKs6lvDEIgLSqWFKrXfkuNp/7T8J0\nFfjiLVSvm/O/6nVSqmdZqm4I6A4ClqpRArqKlFIrF0qtbCqlyp3U3IdvNF/99J3R3gCkV96UUhlS\naqVXKdVPASBl1wGz7d4e5AEAiN2WbbZd7LD+FscpoBurUseC1IqwUl6D1Iqall4noHutlir6LQ30\nASC9kq6UMWhp/AG6MThpjflKfSvG6ar+ST2Wx2Sar6yYMHKAxeMgoDsW9l9g/jq82bMnAoDk91dK\nVVBL322A7vtNasVSKVWPAWD8qkyz7VZPGCk5p1KPq1IrlkqtHCvlu9dSFU9AV8lTaqVXqZ8RKVVX\npVartfSdmjk90GKcJ6kmnSNmBRoiIiIiokeAJdiJiIiIqMaoSWe0OdEmIiIiohpDW3Pm2ZxoExER\nEVHN8ac5ox0QEIDy8nKDx+5VxmEJdiIiIiJ63P40E+1Zs2YhJiYGH374ob4UJRERERHRk2LNdbSf\nNLMT7c6dO2Pw4ME4f/48i9YQERER0RP3pzmjDQDjxo17HP0gIiIiIrLImj+G1Gq1iI2Nxfnz52Fn\nZ4eEhAS0aNFCv33r1q3IyMiAra0tJk2aBA8PDxQXF2PWrFm4c+cOGjdujKSkJNStW1dWX1kZkoiI\niIhqjMlrDYtULX9rWJVt9+3bh9zcXCxcuBD5+flYtWoVVqzQFUC8evUqgoKCsH37dty9exejRo3C\n9u3bkZKSgueffx6+vr5ITU2FnZ0dxo4dK6uvj/yqI1IrSo1+f6PZdhunjpZcwcxSZTJAV51MajWx\nPnHm430+V1fpTEoFPksV4gBdlTipVcIsVahaOX44AMDv3TSz7TKmj5FcictSFcED80IASKuqCEir\nhmepchagq54ltRKp1CqC7+7JM9tuupe7xWpogK4imtRKfVKqqwHSx5aU9/dhV1OV+hosVX1cMmaw\nxRwAujy8nrjKbJu90RMAQHJlUynxpFaI83/P/D63TNPt0ztljdl2O9/W/cIopUKn1MqFlqrLAroK\nsz7vrDXbZkf4WwDwUKvhSX0NUqrLAtJyCkg7rs7Zar56JADEjxiAaes/NdvmvbFDAEivWillDEr9\nnpFSAROQVo0SgKTXailXgC5flirfArrqt1LH1iILlSFn/a8y5PCl6822+3jGWIvjD9CNQUufkfgR\nuvdX6riXUknXUsVVQFd11Ssp1WybPVHBFuM8SdacIj516hR69tQdd7p06YKCggL9tm+++QZdu3aF\nnZ0d7Ozs0Lx5cxQWFuLUqVOYMEF3PHB3d8eSJUtkT7RZGZKIiIiIagwhhMHNHI1GAwcHB/19tVqN\niooK/TZHR0f9Nnt7e2g0GoPH7e3tUVJSIruvVk+0y8rKZO+MiIiIiEgJrRAGN3McHBxQWlr6x3O1\nWtja2prcVlpaCkdHR4PHS0tL4eTkJLuvVU60c3Nz4eHhAU9PT+zZ88dPi/zjSCIiIiKqCVxdXZGX\np1uCmJ+fDxcXF/22Tp064dSpU7h79y5KSkrwww8/wMXFBa6urjh0SLfMKS8vD926dZO9/yrXaK9c\nuRKffvophBCYNm0a7t69iyFDhtSoS6oQERER0Z+LNXNRT09PHDlyBH5+fhBCIDExEevWrUPz5s3R\np08fBAQEYNSoURBCYMaMGahduzYmTZqEiIgIbN26FU899RQWL14su69VTrRr1aoFZ2dnAMDy5csR\nGBiIJk2aQKVSyd4ZEREREZES1ky0bWxsEBcXZ/BYmzZt9P8eMWIERowYYbC9UaNGWLvW/B+AS95/\nVRv++te/IikpCbdv34aDgwM++OADxMXF4eLFiw9lx0RERERE1tIKw1t1VuVEOzExEe3atdOfwW7S\npAnS0tIwYMCAx9Y5IiIiIqL7WXPVkSeNBWuIiIiIqMYwrkuxYfKoJ9QTyx55wRoiIiIiooelJp0j\nfuQTbanVF6VUHbNUkRDQVSW0VDEP0FXNk1q1KTh1q9l2qcG6RfSWKlfGDutv8f0AdO+J1CphUvYJ\nSHt/LVW2BHTVLaXmVErVP+DB/5ka2zB5lORqYlJiAXioFdEsVaMEdBUpLVX/AnQVwKRUJgOAxRYq\nnYX9r9KZlIpolmLdi/dBtvn3Y0p/XYU4S9UGF/h5AZBWDc9SlVRAVyk1bvs+s23mDu0HAPjHQvMV\n0XZH6iqiSamcZun9AHTvSdSW3WbbJPn/A4D046WUaniWqvQB1lXq+3DfEbNtQvq9CkB6tVpLFezm\n+HpK7pvUqn9Sx4ylyopSqvcCus+l1Ap8Ur+PpLSTmnupVRWl5tRShdltM9+0mCtAly+p3+NSj5dS\nv1OlvL9Sj5dSKxBLHffJO3PNtovw7i35e1xKrqozbXVfmH0fntEmIiIiohqjJp3Rtqoy5J07d1gZ\nkoiIiIhIArMT7Z9//hmTJ0/G3LlzcfToUXh5ecHLywsHDhx4XP0jIiIiItKrSZf3M7t0JDo6GlOn\nTkVRURFCQ0ORnZ2N2rVrY9y4cfDw8HhcfSQiIiIiAqB86cidO3cQHh6O69evw97eHsnJyWjQoMED\n7S5fvoyQkBB89tlnAICbN2+if//++jLuffv2RWBgoNl9mZ1oV1RUoHv37gCAEydOoGHDhron2XJp\nNxERERE9fgLKJtpbtmyBi4sLpk6dit27d2P58uWIiYkxaLNjxw6kpaXhxo0b+sfOnj2LgQMHYs6c\nOZL3ZXbpSKtWrTB79mxotVosXLgQAJCamopGjRpZ83qIiIiIiB4KrRAGN2udOnUKPXv2BAC4u7vj\n2LFjD7SpX78+Nm7caPBYQUEBzpw5g9GjRyM0NBS//fabxX2ZPTWdkJCA3Nxc2Nj8MR//y1/+goCA\nAEkvhIiIiIjoYbJm6cjHH3+MDRs2GDzWsGFDODo6AgDs7e1RUlLywPNMLZFu3bo1OnbsiFdeeQU7\nd+5EQkICli1bZnb/rAxJRERERDWG72LD64B/Embddb+nTJmC4OBgdOrUCSUlJfD399evwzb26quv\n4sgRXR0BjUaDunXrQq1W4/fff8egQYOwf/9+s/uy6vJ+RERERERPkhDC4GYtV1dXHDqkK4iVl5eH\nbt26SXpeTEwMsrN1hQKPHTuGF154weJzONEmIiIiohpD6UTb398fFy5cgL+/PzIzMzFlyhQAQEpK\nCr755psqnxcWFoYtW7YgICAAGRkZmD17tsV9cekIEREREdUY3ilrDO7vfHvcE+qJZTyjTURERET0\nCPCC2ERERERUY8i5pN+Twok2EREREdUYNWnVMyfaRERERFRj1KB59uObaGu1WsTGxuL8+fOws7ND\nQkICWrRoIStWeXk5oqOjUVRUhLKyMkyaNAl9+vRR1L/r16/D19cXH330Edq0aaMo1qpVq5Cbm4vy\n8nL4+/tj+PDhsmOVl5cjMjISRUVFsLGxQXx8vKz+ff3111i0aBHS09Nx+fJlREZGQqVS4bnnnsO8\nefMMihJZG+/cuXOIj4+HWq2GnZ0dkpOTraoeen+se3bt2oWNGzciMzPTqn4Zx7t+/TpiYmJw69Yt\nVFZWIiUlBc2bN5cV69y5c5g3bx7UajVatmyJBQsWSH7fTH1m27ZtKzsPpuI1bdpUVh7MjSc5eTAV\nr0uXLrLyUNXrlJuHyspKxMTE4NKlS1Cr1UhKSoIQQlYeTMUqLS2VPRZMxbv3HlmbB1Ox7O3tZY+F\nql6r3DwAhsdcW1tbxcek++OVlZUpOiaZ+j5Qcky6P56zs7OiY5Lx61SSAx8fH33RjmbNmmHkyJFY\nsGAB1Go13Nzc9FdikBvP29sb7777LmxtbdGwYUMkJyejbt26smIlJSUBAFasWIHvvvsOS5cuVdS3\niRMnYt68eSgvL4ednR2WLFmCp556SlasQYMGYdGiRbC1tUWPHj0wY8YMq/pmPGfo3r277PFgHKtj\nx46KxkJ1VZOWjkA8JtnZ2SIiIkIIIcTp06fFxIkTZcfatm2bSEhIEEIIUVxcLF577TVFfSsrKxOT\nJ08W/fr1E99//72iWMePHxcTJkwQlZWVQqPRiGXLlimKl5OTI0JDQ4UQQhw+fFhMmTLF6hipqali\n4MCBYvjw4UIIISZMmCCOHz8uhBBizpw5Yt++fYrivfHGG+Ls2bNCCCG2bNkiEhMTZccSQoizZ8+K\nMWPGGDwmN15ERITYvXu3EEKIY8eOiQMHDsiONXnyZHHw4EEhhBAzZ84Un3/+ueRYpj6zSvJgKp7c\nPFQ1nuTmwVQ8uXkwFUtJHnJyckRkZKQQQjdWJ06cKDsPpmIpGQum4gkhLw+mYikZC6biKcmD8TFX\n6THJOJ6SPJj6PlByTDKOpyQPxrGU5ODOnTti8ODBBo95e3uLy5cvC61WK8aNGycKCgoUxevXr5+4\nevWqEEKIRYsWiQ0bNsiOJYQQBw8eFH5+fmL69OmS+1VVvICAAHH69GkhhBB79+4VX331lexYgwcP\nFhcuXBBarVb4+fmJwsJCyX0zNWeQOx5MxVIyFqqzfgkrDG7V2WO76sj9deW7dOmCgoIC2bFef/11\nTJs2TX9frVYr6ltycjL8/PzQuHFjRXEA4PDhw3BxcUFISAgmTpyIXr16KYrXqlUrVFZWQqvVQqPR\nwNbW+h8hmjdvjvfff19//8yZM+jevTsAwN3dHUePHlUUb8mSJejQoQMA3Zmv2rVry45148YNLFq0\nCNHR0Vb1qap4X331Fa5cuYKxY8di165d+tctJ1aHDh1w8+ZNCCFQWlpqVS5MfWaV5MFUPLl5MBVL\nSR5MxZObB1OxlOShb9++iI+PBwD88ssvaNSokew8mIqlZCyYiic3D6ZiKRkLpuIpyYPxMVfpMck4\nnpI8GMdSekwyjqckD8axlOSgsLAQv//+O4KCgjBmzBh8+eWXKCsrQ/PmzaFSqeDm5oZjx47Jjpef\nn4/09HT92dOKigrJeTAV6/Lly8jMzMTUqVMl96mqeKdPn0ZxcTEOHDiAgIAA5Ofno1OnTrL7di8P\n5eXluHv3rlVzElNzBrnjwVQsJWOhOtMKYXCrzh7bRFuj0cDBwUF/X61Wo6KiQlYse3t7ODg4QKPR\nIDQ0FNOnT5fdr08++QQNGjTQ/ydAqRs3bqCgoADvvfce5s+fj1mzZilatF+vXj0UFRVhwIABmDNn\nDgICAqyO0b9/f4MDsBACKpUKgO69LCkpURTv/i+QjRs3YuzYsbJiVVZWYvbs2YiOjoa9vb1Vfaqq\nb0VFRXBycsL69evRpEkTrF69Wnasez/NDhgwANevX8ff//53ybFMfWaV5MFUPLl5MI41bdo0RXkw\n1Te5eTAVS0keAMDW1hYRERGIj49H//79FeXBOJaSsWAcr1+/foryYNw3JWPBVDy5eTB1zFWSA1Px\n5ObBOJZWq1WUA1N9k5sHU7GUjIU6dergrbfewtq1azF//nxERUUZLOuwNg/G8WbNmoUGDRoAAHJy\ncnDixAn4+PjIjjV37lzExcXJOrFmHC8sLAwXLlxAjx49kJaWhv/+97/49NNPZfetTZs2mDhxIry8\nvNCkSRO0bt1act+qmjPIGQ+mYj399NMA5B+TqiuhsGDN4/TYJtoODg4oLS3V39dqtbLOzt7zn//8\nB2PGjMHgwYMxaNAg2XG2b9+Oo0ePIiAgAOfOnUNERASuXr0qO56zszPc3NxgZ2eH1q1bo3bt2igu\nLpYdb/369XBzc0N2djaysrIQGRmJu3fvyo4HwGCtV2lpKZycnBTFA4A9e/Zg3rx5SE1N1R9crXXm\nzBlcvnwZsbGxmDlzJr7//nssWLBAUb+cnZ3Ru3dvAEDv3r0V/ZKyYMECbNq0CXv37oWPjw8WLlxo\n1fONP7NK82BqDMjNw/2xWrZsqTgPxn1TkgfjWErzAOjODGZnZ2POnDkG40lOHu6Pdfv2bcVj4V68\nkJAQFBYWKsrD/X1zdHRUPBbuj5eQkCArD6aOufcfI63NQVXHcDl5MI41aNAgfPfdd7JzYKpvNjY2\nsvJgKlZkZKTssdCqVSt4e3tDpVKhVatWcHR0xM2bN/Xbrc2DcTxnZ2dcvXoV69evx9q1a7FmzRrJ\nZ1ONY6nVahQVFWHGjBlITEzE8ePHkZqaKrtv99Ziv/zyy1CpVPDw8JCcB1N9e+edd7B7927s378f\nLVq0wEcffSS5b6bmDPdPrK3JQ1Xzj4fx/VzdCGF4q84e20Tb1dUVeXl5AID8/Hy4uLjIjnXt2jUE\nBQUhPDwcw4YNU9SvTZs2YePGjUhPT0eHDh2QnJys/x+gHN26dcMXX3wBIQSuXLmC33//Hc7OzrLj\nOTk56f/oon79+qioqEBlZaXseADw/PPP48SJEwCAvLw8vPTSS4riZWVl6d/DZ599VnacTp06Yffu\n3UhPT8eSJUvQtm1bSeVNzenTOj2MAAADGUlEQVTWrRsOHToEAPjyyy/Rtm1b2bHq16+v/1WmcePG\nuHXrluTnmvrMKsmDqXhy82AcS2keTPVNbh5MxVKShx07dmDVqlUAgLp160KlUqFjx46y8mAqVk5O\njuyxYByvUaNG+Oc//ykrD6b61r17d9ljwVQ8Z2dnWXkwdcx1d3eXPRZMxTt69KisPBjHujd5kjsW\nTPXNw8NDVh5MxWrWrJnssbBt2zb9xPzed1W9evXw008/QQiBw4cPW5UH43gajQbbt2/HyZMnsX79\neqsmeMaxVCoV9u7di/T0dERHR+Pll19GcHCw7HilpaV44YUXcPLkSQC6PDz33HOyYpWXl6NZs2ao\nV68eAOvzYGrO0KNHD1njwVSsvLy8h/L9XN3UpKUjj+2qI56enjhy5Aj8/PwghEBiYqLsWCtXrsSt\nW7ewfPlyLF++HACwevVq1KlT52F1VzYPDw98+eWXGDZsGIQQmDt3rqI15GPHjkV0dDRGjRqF8vJy\nzJgxQz+g5YqIiMCcOXOwZMkStG7dGv3795cdq7KyEgsWLECTJk30a+f+9re/ITQ0VFEfH5aIiAjE\nxMQgIyMDDg4OWLx4sexYCQkJmDFjBmxtbVGrVi39mlUpTH1mZ8+ejYSEBFl5MI5XWVmJCxcuoGnT\nplbn4WGPJ1PxFi5cKCsPpmIpyUO/fv0QFRWFN954AxUVFYiOjkabNm1kjQdTsaKjo2WPBVPx5K6n\nNBWrQ4cOsseCqXjOzs6y82DsYR6TtFotj0kSDBs2DFFRUfD394dKpUJiYiJsbGwwa9YsVFZWws3N\nDZ07d5YdLy4uDkFBQXj++ecxfvx4AMCAAQMwatQoWX1T8gu4qXj16tXD/PnzUVlZiWbNmmHWrFmy\nYi1atAjXr19HUFAQateuDUdHR6t+WTA1Z2jWrJms8WAqVlhYWLUdC0pU9+Ui91OJmtRbIiIiIqIa\n4rEtHSEiIiIi+v+EE20iIiIiokeAE20iIiIiokeAE20iIiIiokeAE20iIiIiokeAE20iIiIiokeA\nE20iIiIiokeAE20iIiIiokfg/wBi49/1fqyZiQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x2326d0f0b70>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAtoAAABYCAYAAADLCVDJAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3XlcU1f6P/BPIIJKQOrWah1ci9Va\nq9ix9Vukg2ulKKigQI1aqtYNXBkWcQVxqbiLiktVXKDVKnWpFiqKe9VKW1TUbtZhOlZFK0SRJef3\nR35kTIzJzY1U6XzefeX1au49eXKSJ+fmeDm5j0IIIUBERERERE+U3dPuABERERHRXxEn2kRERERE\nlYATbSIiIiKiSsCJNhERERFRJeBEm4iIiIioEnCiTURERERUCTjRJiIiIiKqBJxoExFVAX369MGX\nX36pv5+Xl4eWLVti165d+m1XrlxB+/btcffu3afRRSIiMqJ82h0gInpYy5Yt4e7uDjs7w/MAK1as\nQKNGjUw+5vvvv8eaNWuwdOnSJ9KH/fv3Y8uWLUhJSbH6sZmZmYiIiMC5c+f02w4dOoTExESUlJSg\nZcuWSEhIgEqlsrjvYS4uLigsLNTf37hxI5ydnQ22paSkwN/fHy4uLlb3m4ioqlh78JTB/WFd3nhK\nPbGME20ieuZs3LgRtWvXltz+1VdffWKTbFv88ssvmDdvnsG2goICREdHY9u2bWjSpAk++ugjLFiw\nADNmzDC7z5hKpUJRUZE+ZlZWFvr166efaN+9exe7d+/G9u3bK/11EhE9TdoqVNScS0eIqMo4deoU\nAgMDMW7cOPTu3RuBgYH48ccfcerUKfj6+gIANBoNwsPD4efnh759+yI2NhZarRYAkJaWBl9fX/Tp\n0wehoaH4+eef9bGXLFmCbt26ISAgABkZGfrtBw8eRGBgIPz9/REUFGRwpvph9+/fR0REBKKiogy2\nHz16FK+++iqaNGkCAAgODsbu3bshhDC7z5izs7N+op2amgofHx80bNhQv23Hjh3w8PBA8+bNZbyz\nRERVhxDC4GaOVqvFtGnTMHDgQKjValy9etVg/5YtW9C/f38EBAQgKysLAFBcXIywsDCEhIRg+PDh\nKCgokN1XTrSJ6JkzZMgQ+Pn56W9jxozR78vNzYVarcbu3bvRr18/REREGDw2IyMDGo0G6enp+rO7\n165dw4kTJ7B27Vps2rQJn3/+OXx9fTFmzBgIIZCZmYkvv/wSu3btQmpqqn7y+ssvv2DRokVITk7G\nrl27EBcXh7CwMNy7d++RPlccyFu2bGmw/T//+Q9eeOEF/f0XXngBRUVF0Gg0ZvcZq5hol5aWIjU1\nFYMGDYKTkxOKioqg1WqxZcsWDB48GIWFhQgICED79u1x+fJlGe8+EdGzTSuEwc2czMxMlJSUIC0t\nDZMmTcLcuXP1+woKCrB161akpqZiw4YNmDFjBoQQ2LZtG9zd3bF161b4+/sjKSlJdl850SaiZ87G\njRuRnp6uv61YsUK/7+WXX8brr78OAOjfvz8uXryIO3fu6Pd36NABP/zwA9RqNZKTkzFkyBA0btwY\nR44cgY+Pj35JSr9+/XD9+nX861//wokTJ9C9e3eoVCoolUr0798fAHDs2DH8/vvvGDp0KPz8/DB5\n8mQoFAr8+uuvBv3dsmULlEolAgICHnktWq0WCoXike12dnZm9xlzdnaGRqPBgQMH8PLLL6NZs2ZQ\nqVQoLCzEoUOHYG9vDy8vL1SvXh3Jycno2bOnlLeaiKjKsWaiffbsWXTu3BkA0K5dO+Tm5ur31a5d\nG+np6ahWrRpu3rwJFxcXKBQKg8d4eXnhxIkTsvvKNdpEVKXY29ub3fa3v/0NGRkZOHXqFE6ePIn3\n338fs2bN0i8feZgQAmVlZfr/N46n1WrRqVMnLF68WL/vt99+Q/369Q3i7Ny5E8XFxfDz80Npaan+\n/5OTk9GgQQN8++23+rbXr19HrVq1ULNmTbP7jKlUKly7dg2bNm1CeHi4fltRURE2b94MtVoNhUKB\natWqWbW+nYioqrG0XORhRUVFBj8wt7e3R1lZGZRK3RRYqVRi8+bNWLZsGdRqtf4xzs7OAAAnJyeD\nH51bi2e0iahKycvLQ15eHgDdmuv27dvrD4gAsHXrVkRHR8PT0xMRERHw9PTEhQsX0LlzZ+zbt0+/\n1m7Hjh1wdXVF48aN4eXlhf379+Pu3bvQarVIT08HAHTq1AnHjh3Djz/+CAA4fPgw+vTpg+LiYoM+\nbd++HXv27EF6ejqSk5NRvXp1pKen4/nnn4enpye+/fZb/PLLLwB066u7du0KAGb3GXN2dsaZM2dQ\nVFQET09PALqJ9sWLF/H999+jb9++T+DdJSJ69glheDNHpVIZLMfTarX6SXaFQYMG4ciRIzh9+jRO\nnjxp8BiNRmPTlZx4RpuInjlDhgx5ZPnExIkTUb16ddStWxeLFy9Gfn4+ateujfnz5yM/P1/fzt/f\nH19//TV8fHxQo0YNNGjQAGq1GrVq1cLQoUMxZMgQaLVa1K5dG6tXr4adnR3efvttXLp0Cf3794eL\niwtefvll3L59Gy1atMCsWbMwceJECCGgVCqxcuVKODk5SX4tderUwZw5cxAeHo7S0lK4ubnpr0xi\nbp8xZ2dnXL9+HaNHj9ZvU6lUuHHjBoYMGWJVn4iIqjJrrjri4eGBrKws+Pj4ICcnB+7u7vp9P/30\nExYuXIhly5ahWrVqcHBwgJ2dHTw8PHD48GG0bdsW2dnZ6NChg+y+KoQ159+JiJ6iU6dOIS4uDnv2\n7HnaXXnmRUVFITQ01OBLhYjor2DxvmyD++N9vB7bVqvVYsaMGbh8+TKEEEhISEB2djbc3NzQtWtX\nLF++HNnZ2VAoFOjcuTPGjh2L+/fvIzIyEjdu3EC1atWQmJiIevXqyeorJ9pEVGVwoi3N8OHDcfHi\nRbz44osYOHAg+vXr97S7RET0xCzae9jg/oR3335KPbGME20iIiIiqjIS9xwyuD/J9x9PpR9ScI02\nEREREVUZVekMMSfaRERERFRlVKUS7JxoExEREVGVUZVWPXOiTURERERVhlbLibae/0frzO7fFfEB\nAGD5gaNm243t6Ynu8SstPl9G7ChMSvncYrtEdR+Ef/yZ2TZL39f9Ur/n7FVm2x2YMhIA8M8tu822\nm/9ebwxfnWaxb2s+HIg5u74y2ybaX1fUwvgHAcYqfiBg6XnXfDgQkVstX8lhXojvI7/2NVbx69/B\nK7aYbbdpzHsAgIGLN5ptlzZ+iOT3Le6zDLNtpvbrDgCYkrrPbLvZQT4AgHEbdpptt2RoXwQt3mSx\nb6njB6PP/LUW233+z2GYuCndbJuFg/0k9w0AfOetMdtuT+RwyX3rOivJbJuvpo22qm9S8hC6MtVi\n39aPCpL8uZR6TOq7YL3Zdjsnh2Lkmk8t9m3V8EBJn3EAWLbf/HEw7B1doRopx0upOY3ettdiuznB\n72LG9gNm28wI0JV8T9iZabZdTN9uAKTl4d25yRb7tjdqBN5bmmK2zZZwXbW3+buzzLb7Z29vALD4\nmVs/KgjDVln+XK4dGYSx681/zywP1X3PjF633Wy7pA8CAMDie7I3aoTFXAG6fEk5NgBAv8SPzbb7\nbNL7AKSNGanfz5a+AwHd96DUnEr9vpdy7LIUqyKelFwBj16qzljFpeukfL9J/T6Seox+VvGMNhER\nERFRJfhLrtHWarWPVGojIiIiInpWVRSsuXTpEhwcHBAfH4/GjRsbtCkoKEBQUBB2794NR0dHCCHg\n5eWFJk2aAADatWuHSZMmyXp+sxPta9euYc6cOcjNzYVSqYRWq4W7uzuio6PRtGlTWU9IRERERCSX\nNeezMzMzUVJSgrS0NOTk5GDu3LlYufK/S5GPHDmCxMRE3Lx5U7/t119/xSuvvIJVq8wvHZbC7ER7\nypQpmDRpEl577TX9tpycHERHRyM11fL6NCIiIiKiJ8maNdpnz55F586dAejOTOfm5hrst7Ozw8cf\nf4z+/fvrt50/fx7Xr1+HWq1G9erVER0djWbNmsnqq9mJdklJicEku6KTRERERERPgzVrtIuKiqBS\nqfT37e3tUVZWBqVSNwV+6623HnlMvXr1MGLECPTq1QtnzpxBREQEduzYIauvZifaLVu2RHR0NDp3\n7gxnZ2doNBocPnwYLVu2lPVkRERERES2sOaMtkqlgkaj0d/XarX6SfbjtGnTBvb29gCA119/Hdev\nX4cQAgqFwuq+mn2mGTNmIDMzE2fPntX/i8Db2xvdu3e3+omIiIiIiGxlzRltDw8PZGVlwcfHBzk5\nOXB3d7f4mOXLl8PV1RXDhw9HXl4eGjZsKGuSDQAKUZUuRkhERERE/9Ni074wuB8/sNdj21ZcdeTy\n5csQQiAhIQHZ2dlwc3ND165d9e26dOmCL774Ao6Ojvjjjz8QERGBe/fuwd7eHtOmTUPz5s1l9ZUT\nbSIiIiKqMoyLnVUUmXsWVXrBGikVtgCgW5z5inOZU0fDZ47lKmH7okdIroz0jxnLzbY5NGMsAGBE\n8idm2yWPGABAWtUxS7Eq4sVbqAAV+/8rHE795Auz7eIG6P6V907CarPt9sd8aDEHgC4PUis+Wqo0\nOS/EF4C0KneWKusBuup6gYs2mG3z6YShACD5/ZVS1U1qRU1LuQJ0+ZJauVBqJbkhSVvNtts4OgTe\nM1dY7FvW9DEWq7olqvsAePRsg7GKsw9SPiMRmy1XkvtoUB+LVRpXDQ/Uxfz8oNl2kX26AACmf7rf\nbLuZge9IrsBnqfrinOB3AUivXLjkiyNm243r1RnvJ22z2LePRwdLfg0TNu4y22bREH8AsFjBNyN2\nFABpY1BqBT6pubdUzXHtSN33kZTj6qBlmy32bXPYIIvfW/uiddUBpY6tFV8eM9tuTI+3LH63Abrv\nN6kVSy2NwY8G6fomJQ+WKhICuqqEUiuWSj2WS62iLKX6otSKsFJyBUj7fgakVV2V2rfgJearqW4b\np7YY52n6SxasISIiIiJ62jjRJiIiIiKqBFVp1TMn2kRERERUZXCiTURERERUCbRVZ55tfqKtVqtR\nWlpqsK3igt0swU5EREREf7a/zBntyZMnIzY2FitWrNBXyCEiIiIielr+MhPt1157DX5+frh06RKr\nQRIRERHRU1eVrjrCgjVEREREVGUYX2N/6fv9Htu2ojLkpUuX4ODggPj4eDRu3Fi//5NPPkFqaiqU\nSiVGjRoFb29vFBQUYPLkySguLkb9+vUxZ84c1KhRQ1ZfK/3HkFIvdC+lCMqsHV9afL5p/Xug5+xV\nFtsdmDJScsEPS/EOTBkJQFpxHqkX4ZdavGDUWvOvYeUw3Wt4b6n5i9NvCVc/UmnJlNlBPpKKCwGw\nWJhgydC+AIA5u74y2y7av6vkYkWWinR8PDoYgPTCFVIKg0gtECC1gIjU4hv/3LLbbLv57/UGIO39\nlVoQyFJhm6zpYwBIL/Ik5XNuKQeALg/vzjX/GdkbpSsMIrXgkpR4lo4hgO44Yqm4yeawQQAguViR\nlNxLLahiqbgQoCsw1C/xY7NtPpv0PgBg2X7zBajC3vEEIK1Ih6VcAbp8Ld6XbbbNeB8vANJyCkg7\nrj7pQj9Sx4yUMfikimhVFDyTUlQMACZuSjfbbuFgP4u5AnT5klrYRsp3GyC9YI2lgnep4wdLfn+l\nFL0CpI97S9/Rs4N8LBbwAXRFfHznrTHbZk/kcItxqorMzEyUlJQgLS0NOTk5mDt3Llau1BXWunHj\nBlJSUrBjxw48ePAAISEheOutt5CUlARfX1/069cPycnJSEtLw9ChQ2U9v90TfC1ERERERJVKKwxv\n5pw9exadO3cGALRr1w65ubn6fd999x3at28PBwcHODs7w83NDXl5eQaP8fLywvHjx2X3lZf3IyIi\nIqIqQyu0ktsWFRVBpVLp79vb26OsrAxKpRJFRUVwdnbW73NyckJRUZHBdicnJxQWFsruq9VntEtK\nSmQ/GRERERGRLbRaYXAzR6VSQaPRPPRYLZRKpcl9Go0Gzs7OBts1Gg1cXFxk9/WxE+2DBw/C29sb\n3bt3x759/10XNGzYMNlPRkRERERkC60QBjdzPDw8kJ2t+21ATk4O3N3d9fvatm2Ls2fP4sGDBygs\nLMSPP/4Id3d3eHh44PDhwwCA7OxsdOjQQXZfH7t0ZNWqVdi5cyeEEBg3bhwePHiAvn37VqlrFxIR\nERHR/67u3bvj2LFjCAoKghACCQkJ+Pjjj+Hm5oauXbtCrVYjJCQEQghMmDABjo6OGDVqFCIjI/HJ\nJ5/gueeeQ2Jiouznf+xEu1q1anB1dQUAJCUlYciQIWjQoAEUCoXsJyMiIiIisoU1J33t7Owwa9Ys\ng23NmzfX//+AAQMwYMAAg/1169bFunXmrwYj+fkft+PFF1/EnDlzcO/ePahUKixfvhyzZs3CTz/9\n9ESemIiIiIjIWkIIg9uz7LET7YSEBLRs2VJ/BrtBgwbYtGkTevXq9ad1joiIiIjoYdZc3u9pY2VI\nIiIiIqoyjAvAVRSjexZV+nW0pVZflFJ1zFJFLEBXFctStTlAV3HOUqXJaf17AIDkSn1S4kmtJCe1\ncqHU1yDl/bVU2RLQVbeUmlOpVUGlVPayVJkM0FUnk1KdE4DkimirM0+Ybfdht04WcwXo8mWpEheg\nq8ZlqUpj2vghAKRX4JPSTmr1xeSvTpptM6LrmwBgsdpg/EDd+ytlbFmqNgfoKs5ZqtQ3I6AnACBg\nofmxsH2irsKhlGptlj4fgO4zIqWiGyC9OqCU1yD1fZOa+7UHT5ltM6zLGwCAses/M9tueaiuVLKl\nCnax/bpb/OwCus+v1MqQUiq9ArBY+W9eiK/F4xugO8ZJrUYptYqglLEltW/BS8wfe7eN01VVlHrM\nl/K5XLT3sMW+TXj3bcmvoe+C9Wbb7JwcCkD697iUPEgdM1IrFUsdM/N3Z5lt98/e3pK/jwIXbTDb\n5tMJQy3GeZq0qDrniFmwhoiIiIiqjKq0GIMTbSIiIiKqMiwVqXmWWDXRLi4uhp2dHRwcHCqrP0RE\nREREj1WVzmibLcF+7do1jB49GtOmTcPx48fh4+MDHx8fZGWZXydERERERFQZbL3qSHFxMcLCwhAS\nEoLhw4ejoKDAZLurV6/C19dXf//OnTt44403oFaroVarsXGj+d9UARbOaMfExCAsLAz5+fkIDw/H\ngQMH4OjoiGHDhsHb29vKl0VEREREZBtbz2hv27YN7u7uCAsLw969e5GUlITY2FiDNrt27cKmTZtw\n+/Zt/bYLFy7A19cXU6dOlfxcZs9ol5WVoWPHjujbty+6deuGOnXqQKVSQank0m4iIiIi+vMJo/+s\ndfbsWXTurLvKkJeXF06cePQKUrVq1cLmzZsNtuXm5uL8+fMYNGgQwsPD8fvvv1t8LrMz5qZNm2LK\nlCmIi4vD3LlzAQDJycmoW7eu5BdDRERERPSkaK04o/3pp58+ssSjTp06cHZ2BgA4OTmhsLDwkceZ\nWrnRrFkztGnTBv/3f/+Hzz//HPHx8Vi6dKnZ5zc70Y6Pj8fBgwdhZ/ffE9/PP/881Gq12aBERERE\nRJXBmqUjgYGBCAwMNNg2duxYaDQaAIBGo4GLi4ukWG+++SZq1KgBAOjevbvFSTbAypBEREREVIUY\nF+H7bNL7Vj1+/fr10Gg0+jXaX3/9NWbOnGmy7VtvvYVjx44BAMaPH48ePXrAx8cHBw8eRHp6OpYs\nWWL2ucyu0SYiIiIiepYIIQxu1goODsaVK1cQHByMtLQ0jB07FgAwf/58fPfdd4993KRJk7Bt2zao\n1WqkpqZiypQpFp+LZ7SJiIiIqMrw/2idwf1dER88pZ5YxjPaRERERESVgNfpIyIiIqIqw5qrjjxt\nnGgTERERUZXBiTYRERERUSWoSj8v5ESbiIiIiKqMKjTP/vMm2lqtFjNmzMClS5fg4OCA+Ph4NG7c\nWFas0tJSxMTEID8/HyUlJRg1ahS6du1qU/9u3bqFfv36Yf369WjevLlNsVavXo2DBw+itLQUwcHB\nj1wo3RqlpaWIiopCfn4+7OzsEBcXJ6t/3377LRYsWICUlBRcvXoVUVFRUCgUeOmllzB9+nSDokTW\nxrt48SLi4uJgb28PBwcHzJs3z6rqoQ/HqrB7925s3rwZaWlpVvXLON6tW7cQGxuLu3fvory8HPPn\nz4ebm5usWBcvXsT06dNhb2+PJk2aYPbs2ZLfN1Of2RYtWsjOg6l4DRs2lJUHc+NJTh5MxWvXrp2s\nPDzudcrNQ3l5OWJjY/Hzzz/D3t4ec+bMgRBCVh5MxdJoNLLHgql4Fe+RtXkwFcvJyUn2WHjca5Wb\nB8DwmKtUKm0+Jj0cr6SkxKZjkqnvA1uOSQ/Hc3V1temYZPw6bcmBv7+/vjpeo0aNMHDgQMyePRv2\n9vbw9PTUX/JMbrw+ffpg8eLFUCqVqFOnDubNm6cv9mFtrDlz5gAAVq5cicuXL2PRokU29W3kyJGY\nPn06SktL4eDggIULF+K5556TFat3795YsGABlEolOnXqhAkTJljVN+M5Q8eOHWWPB+NYbdq0sWks\nPKuq0tIRiD/JgQMHRGRkpBBCiHPnzomRI0fKjrV9+3YRHx8vhBCioKBAvP322zb1raSkRIwePVr0\n6NFD/PDDDzbFOnnypPjwww9FeXm5KCoqEkuXLrUpXkZGhggPDxdCCHH06FExduxYq2MkJycLX19f\nERgYKIQQ4sMPPxQnT54UQggxdepU8eWXX9oU77333hMXLlwQQgixbds2kZCQIDuWEEJcuHBBDB48\n2GCb3HiRkZFi7969QgghTpw4IbKysmTHGj16tDh06JAQQoiJEyeKr776SnIsU59ZW/JgKp7cPDxu\nPMnNg6l4cvNgKpYtecjIyBBRUVFCCN1YHTlypOw8mIply1gwFU8IeXkwFcuWsWAqni15MD7m2npM\nMo5nSx5MfR/YckwyjmdLHoxj2ZKD4uJi4efnZ7CtT58+4urVq0Kr1Yphw4aJ3Nxcm+L16NFD3Lhx\nQwghxIIFC8TGjRtlxxJCiEOHDomgoCAxfvx4yf16XDy1Wi3OnTsnhBBi//794ptvvpEdy8/PT1y5\nckVotVoRFBQk8vLyJPfN1JxB7ngwFcuWsfAs6xG/0uD2LPvTLu939uxZdO7cGQDQrl075Obmyo71\nzjvvYNy4cfr79vb2NvVt3rx5CAoKQv369W2KAwBHjx6Fu7s7xowZg5EjR+If//iHTfGaNm2K8vJy\naLVaFBUVQam0/o8Qbm5uWLZsmf7++fPn0bFjRwCAl5cXjh8/blO8hQsXolWrVgB0Z74cHR1lx7p9\n+zYWLFiAmJgYq/r0uHjffPMNrl+/jqFDh2L37t361y0nVqtWrXDnzh0IIaDRaKzKhanPrC15MBVP\nbh5MxbIlD6biyc2DqVi25KFbt26Ii4sDAPz73/9G3bp1ZefBVCxbxoKpeHLzYCqWLWPBVDxb8mB8\nzLX1mGQcz5Y8GMey9ZhkHM+WPBjHsiUHeXl5uH//PkJDQzF48GCcPn0aJSUlcHNzg0KhgKenJ06c\nOCE7Xk5ODlJSUvRnT8vKyiTnwVSsq1evIi0tDWFhYZL79Lh4586dQ0FBAbKysqBWq5GTk4O2bdvK\n7ltFHkpLS/HgwQOr5iSm5gxyx4OpWLaMhWeZVgiD27PsT5toFxUVQaVS6e/b29ujrKxMViwnJyeo\nVCoUFRUhPDwc48ePl92vzz77DLVr19b/I8BWt2/fRm5uLpYsWYKZM2di8uTJNi3ar1mzJvLz89Gr\nVy9MnToVarXa6hg9e/Y0OAALIaBQKADo3svCwkKb4j38BbJ582YMHTpUVqzy8nJMmTIFMTExcHJy\nsqpPj+tbfn4+XFxcsGHDBjRo0ABr1qyRHaviT7O9evXCrVu38MYbb0iOZeoza0seTMWTmwfjWOPG\njbMpD6b6JjcPpmLZkgcAUCqViIyMRFxcHHr27GlTHoxj2TIWjOP16NHDpjwY982WsWAqntw8mDrm\n2pIDU/Hk5sE4llartSkHpvomNw+mYtkyFqpXr44PPvgA69atw8yZMxEdHW2wrMPaPBjHmzx5MmrX\nrg0AyMjIwKlTp+Dv7y871rRp0zBr1ixZJ9aM402aNAlXrlxBp06dsGnTJvzxxx/YuXOn7L41b94c\nI0eOhI+PDxo0aIBmzZpJ7tvj5gxyxoOpWPXq1QMg/5j0rBI2VoYsLi5GWFgYQkJCMHz4cBQUFDzS\nZtGiRQgMDMSAAQP01SILCgoQGhqKkJAQjB8/Hvfv37f4XH/aRFulUkGj0ejva7VaWWdnK/z2228Y\nPHgw/Pz80Lt3b9lxduzYgePHj0OtVuPixYuIjIzEjRs3ZMdzdXWFp6cnHBwc0KxZMzg6OppMoFQb\nNmyAp6cnDhw4gPT0dERFReHBgwey4wEwWOul0Wjg4uJiUzwA2LdvH6ZPn47k5GT9wdVa58+fx9Wr\nVzFjxgxMnDgRP/zwA2bPnm1Tv1xdXdGlSxcAQJcuXWz6S8rs2bOxZcsW7N+/H/7+/pg7d65Vjzf+\nzNqaB1NjQG4eHo7VpEkTm/Ng3Ddb8mAcy9Y8ALozgwcOHMDUqVMNxpOcPDwc6969ezaPhYp4Y8aM\nQV5enk15eLhvzs7ONo+Fh+PFx8fLyoOpY+7Dx0hrc/C4Y7icPBjH6t27Ny5fviw7B6b6ZmdnJysP\npmJFRUXJHgtNmzZFnz59oFAo0LRpUzg7O+POnTv6/dbmwTieq6srbty4gQ0bNmDdunVYu3at5LOp\nxrHs7e2Rn5+PCRMmICEhASdPnkRycrLsvlWsxX7zzTehUCjg7e0tOQ+m+vbRRx9h7969yMzMROPG\njbF+/XrJfTM1Z3h4Ym1NHh43/3gS38/PGiEMb9batm0b3N3dsXXrVvj7+yMpKclg/4ULF5CTk4NP\nPvkECxcuRGxsLAAgKSkJvr6+2Lp1K1q3bi3pNxt/2kTbw8MD2dnZAICcnBy4u7vLjnXz5k2EhoYi\nIiICAQEBNvVry5Yt2Lx5M1JSUtCqVSvMmzdP/y9AOTp06IAjR45ACIHr16/j/v37cHV1lR3PxcVF\n/6OLWrVqoaysDOXl5bLjAUBjRj9sAAAEQ0lEQVTr1q1x6tQpAEB2djZef/11m+Klp6fr38O//e1v\nsuO0bdsWe/fuRUpKChYuXIgWLVpgypQpNvWtQ4cOOHz4MADg9OnTaNGihexYtWrV0v9Vpn79+rh7\n967kx5r6zNqSB1Px5ObBOJateTDVN7l5MBXLljzs2rULq1evBgDUqFEDCoUCbdq0kZUHU7EyMjJk\njwXjeHXr1sUXX3whKw+m+taxY0fZY8FUPFdXV1l5MHXM9fLykj0WTMU7fvy4rDwYx6qYPMkdC6b6\n5u3tLSsPpmI1atRI9ljYvn27fmJe8V1Vs2ZN/PrrrxBC4OjRo1blwTheUVERduzYgTNnzmDDhg1W\nTfCMYykUCuzfvx8pKSmIiYnBm2++iREjRsiOp9Fo8Morr+DMmTMAdHl46aWXZMUqLS1Fo0aNULNm\nTQDW58HUnKFTp06yxoOpWNnZ2U/k+/lZY+vSkYeXM3t5eT2yTKp169ZYt24dFAqFfrmcqcdJWdbz\np111pHv37jh27BiCgoIghEBCQoLsWKtWrcLdu3eRlJSk/1fImjVrUL169SfVXdm8vb1x+vRpBAQE\nQAiBadOm2bSGfOjQoYiJiUFISAhKS0sxYcIE/YCWKzIyElOnTsXChQvRrFkz9OzZU3as8vJyzJ49\nGw0aNNCvnfv73/+O8PBwm/r4pERGRiI2NhapqalQqVRITEyUHSs+Ph4TJkyAUqlEtWrV9GtWpTD1\nmZ0yZQri4+Nl5cE4Xnl5Oa5cuYKGDRtanYcnPZ5MxZs7d66sPJiKZUseevTogejoaLz33nsoKytD\nTEwMmjdvLms8mIoVExMjeyyYiid3PaWpWK1atZI9FkzFc3V1lZ0HY0/ymKTVanlMkiAgIADR0dEI\nDg6GQqFAQkIC7OzsMHnyZJSXl8PT0xOvvfaa7HizZs1CaGgoWrdujeHDhwMAevXqhZCQEFl9s+Uv\n4Kbi1axZEzNnzkR5eTkaNWqEyZMny4q1YMEC3Lp1C6GhoXB0dISzs7NVf1kwNWdo1KiRrPFgKtak\nSZOe2bFgC2uWi3z66afYuHGjwbY6deroT2I+bnmOUqnEokWLsGnTJkydOhWAbhm0pccZUwhbFhAT\nEREREVUhY8eOxYgRI9C2bVsUFhYiODgYe/bsMdm2qKgIAwcOxMqVKzFu3DisXbsWderUQV5eHhYt\nWqT/a9/j/GlLR4iIiIiInjYPDw/9Eq7s7Gx06NDBYP+JEycwc+ZMAICjoyOUSiUUCoXFx5nCM9pE\nRERE9D/j/v37+h9OV6tWDYmJiahXrx7mz5+Pd955B6+88gpmzZqFS5cuQavVIiAgAAMGDMDNmzcR\nGRkJjUaD5557DomJiRaX83KiTURERERUCbh0hIiIiIioEnCiTURERERUCTjRJiIiIiKqBJxoExER\nERFVAk60iYiIiIgqASfaRERERESVgBNtIiIiIqJK8P8AH8R5fjyuSU4AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x2326d20c198>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAtoAAABXCAYAAAA6X+IcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3Xl8U1X6+PFP2tIiXagIzsAgCGhx\nQWVxUMZSRShIB1lKgRYJm5R9p0xp2dtSlilFZC87ZWkRFESUArKvKmOdqSzi6KDDzCBQHdqwdMn5\n/ZEvGRJCcnNDgfp73q9XXpDckyeneXLuPTm9vY9BKaUQQgghhBBC3FVe97sDQgghhBBC/BrJRFsI\nIYQQQogyIBNtIYQQQgghyoBMtIUQQgghhCgDMtEWQgghhBCiDMhEWwghhBBCiDIgE20hhBBCCCHK\ngEy0hRCiHGjfvj07d+603j99+jT169dny5Yt1sfOnj1Lo0aNuHLlyv3oohBCCDs+97sDQghxq/r1\n6xMSEoKXl+06wIIFC6hZs6bD5/ztb39j6dKlvPvuu3elDzt27GDdunVkZmZqfs6ZM2dISUmhoKAA\nLy8vkpKSaNCgAQD79u1j9uzZFBUVUb9+fVJTUwkICHC57VZBQUEUFBRY769evZrAwECbxzIzM+nY\nsSNBQUF6f3QhhHjgrdj3mc39vq81vU89cU1WtIUQD5zVq1ezdetWm9udJtkAzz333F2bZOtx7do1\n3n77bfr168eWLVsYPHgwcXFxAOTn55OQkMC8efPIycnhscceIy0tzeU2ewEBARQWFlqft3fvXiIj\nI60T7StXrrBt2zZ69OhxD35iIYQQWsiKthCi3Dh+/DhpaWnUqFGD7777jooVKzJjxgwuXbpEcnIy\nH330ESaTiYSEBM6dO4eXlxfPPvssSUlJeHl5kZ2dTWZmJl5eXlStWpWJEydSp04dAObOncu2bdsI\nDg6mdu3a1tfcs2cPixYtori4mIoVKxIfH0+jRo1s+nX48GEee+wxXn31VQBatmxp/WJw6NAhnnvu\nOR5//HEAYmJi6NChA5MnT3a6zWAw2LxGYGCgdaKdlZVFREQENWrU4KeffgJg8+bNNG7cmHr16t3d\nN10IIR4wSt3vHmgnK9pCiAdOr1696NChg/U2ZMgQ67a8vDyMRiPbtm0jMjKSsWPH2jx3165dmEwm\ntm7dyqZNmwD48ccfOXr0KMuWLWPNmjV8+OGHtGvXjiFDhqCUYvfu3ezcuZMtW7aQlZVlndD+4x//\nYM6cOWRkZLBlyxaSk5MZNmwYV69etXnN77//nmrVqpGYmEhkZCR9+vShtLQUgP/85z/89re/tbb9\n7W9/S2FhISaTyek2ezcn2sXFxWRlZdGjRw/8/f0pLCzEbDazbt06evbsSUFBAVFRUTRq1IhvvvnG\nw0wIIcSDRyllc3PGbDYzadIkunXrhtFo5Ny5czbb161bR+fOnYmKimLv3r0AXL9+nWHDhtG9e3di\nY2PJz8/X3VeZaAshHjj2p44sWLDAuu2pp57ixRdfBKBz586cOnWKX375xbq9SZMmfPvttxiNRjIy\nMujVqxe1a9fm4MGDREREUKVKFQAiIyO5cOEC//znPzl69Cjh4eEEBATg4+ND586dActK9U8//UTv\n3r3p0KEDcXFxGAwGfvjhB5v+lpSUsH//frp168b7779Pjx496N+/P0VFRZjN5ttWpwG8vLycbrMX\nGBiIyWQiJyeHp556irp16xIQEEBBQQH79u3D29ubsLAwKlasSEZGBm3atNHxzgshxIPPrJTNzZnd\nu3dTVFREdnY2Y8aMYcaMGdZt+fn5rF+/nqysLFatWsWUKVNQSrFhwwZCQkJYv349HTt2ZOHChbr7\nKhNtIUS54u3t7fSxxx57jF27dtG/f38KCwvp06cPe/bswWw23/Y8pRQlJSXW/9vHM5vNNGvWzGbS\nv3HjRp588kmbOI8++ij16tXjhRdeAKBVq1aUlpby448/Ur16devpHQAXLlygcuXKVKpUyek2ezfP\n0V6zZg09e/a0eWzt2rUYjUYMBgMVKlSwfpkQQohfI3dWtE+cOEHz5s0BaNiwIXl5edZtVapUYevW\nrVSoUIFLly4RFBSEwWCweU5YWBhHjx7V3VeZaAshypXTp09z+vRpALKzs2nUqBGBgYHW7evXrych\nIYHQ0FDGjh1LaGgoJ0+epHnz5nz88cfWXwFu3rzZej52WFgYO3bs4MqVK5jNZrZu3QpAs2bNOHz4\nMH//+98B2L9/P+3bt+f69es2fQoLC+Of//yndQf++eefYzAYqFmzJqGhoXz11Vf84x//ACznV7ds\n2RLA6TZ7gYGBfPHFFxQWFhIaGgpYJtqnTp3ib3/7G506dfL0rRVCiHLBnRXtwsJCmys5eXt7WxdY\nAHx8fFi7di3dunWz/iawsLDQelzx9/e3ubqTu+SPIYUQD5xevXrddvrE6NGjqVixIlWrVuWdd97h\n/PnzVKlShVmzZnH+/Hlru44dO/LZZ58RERHBQw89RPXq1TEajVSuXJnevXvTq1cvzGYzVapUYcmS\nJXh5efHqq69y5swZOnfuTFBQEE899RQ///wzTzzxBElJSYwePRqlFD4+PixatAh/f3+bvlWrVo0F\nCxYwdepUrl27hq+vL/PmzcPPzw8/Pz+mT5/O8OHDKS4uplatWsycOROARx555I7b7AUGBnLhwgUG\nDx5sfSwgIICLFy/Sq1ev2/okhBC/Vq4m17cKCAiw+bsXs9mMj4/t9LdHjx507dqV2NhYjh07ZvMc\nk8nk0SVTDcrVmrsQQjwgjh8/br26iHBu3Lhx9O3bl5CQkPvdFSGEuKvm5xyyuT+0Tegd2+bk5LB3\n715mzJhBbm4u8+fPZ9myZQB89913pKenM2/ePAD69+9PbGwseXl5mEwmhg0bxvbt2/nss8+YOnWq\nrr7KirYQQvzKxMbGcurUKb7//nu6detGZGTk/e6SEELcNe4sEYeHh3P48GGio6NRSpGamsrKlSup\nVasWLVu25KmnnqJbt24YDAaaN29O06ZNee6554iPjycmJoYKFSowe/Zs3X2VFW0hhBBCCFFuzP3k\noM39EW2b36eeuCYr2kIIIYQQotwoT2vEMtEWQgghhBDlhky0hRBCCCGEKAPuXHXkfpOJthBCCCGE\nKDfKzzT7Hky0O6WtcLr9g7i+ACzYedhpuyGtX6HNtMUuXy9n/EDGrv3QZbs/92jPiFUfOG0zt7el\nAMQbqUucttuROACA+PXOLzk2s3s7+mdsdNm3jP5dmfnhHqdt4tu/DsCc7fudthv1x1cBXL5uRv+u\nJGzY7rJv02P+yDsfH3DaZmREGAC9Fq532m714O4AxMzNdNpuwwgjA5e+57Jvi2O7kPL+LqdtJkSG\nW/7N/sRpu5RubQEYtXqL03ZzenXkrXed9x9g3XAjHf+83GW7LWPfZkym88/vbGN7AEav2eq0XXrP\nDgC0n7XMabsP/9RPc99aJTsvQ7t7ouUaz1reN9CWh36Ls1z2bdnAaM2fS637pMjZK522e39MHwYv\n3+SybwvfjiL6nTVO22SNtFR61LIf1NrO1c8Jlp91fNbHLttNi44gafNOp20mdW4NwPQtnzptl9DR\nUpBHSx5cfXbB8vntMW+t0zZrh/UAIO2jfU7bxbV7DcDlZ27ZwGhil2S77NvSAd0YvvJ9p23e7WO5\nIszQFc7bze9raddu5lKn7T6Kj3WZK7Dky9W43zL2bQC6zFnltN17o3oDEJXufMxsGt1H8/HZ1TEQ\nLMdBrTnVerzX0s5Vm5vttOQKbv/DPns3/9BPy/FN6/FI6/HjQSUr2kIIIYQQQpSBX+U52maz+bZK\nbUIIIYQQQtxLZvOvZKL9448/Mn36dPLy8vDx8cFsNhMSEkJCQgJ16tS5V30UQgghhBACcG9F22w2\nM2XKFM6cOYOvry8pKSnUrl3bpk1+fj7R0dFs27YNPz8/lFKEhYXx+OOPA9CwYUPGjBmjq69OJ9rj\nx49nzJgxvPDCC9bHcnNzSUhIICvL9XmTQgghhBBC3E3unKO9e/duioqKyM7OJjc3lxkzZrBo0SLr\n9oMHDzJ79mwuXbpkfeyHH37g2WefZfFi138b6IrTc0GKiopsJtlgmdULIYQQQghxPyi7mzMnTpyg\neXPLH5Q2bNiQvLw8m+1eXl6sXLmS4OBg62Nff/01Fy5cwGg0Ehsby3fffae7r05XtOvXr09CQgLN\nmzcnMDAQk8nE/v37qV+/vu4XFEIIIYQQQi93Th0pLCwkICDAet/b25uSkhJ8fCxT4FdeeeW251Sr\nVo3+/fvTtm1bvvjiC8aOHcvmzZt19dXpRHvKlCns3r2bEydOWDvaokULwsPDdb2YEEIIIYQQnnDn\n1JGAgABMJtP/nms2WyfZd9KgQQO8vb0BePHFF7lw4QJKKQwGg9t9dfpKBoOB8PBwmVgLIYQQQogH\ngjsr2o0bN2bv3r1ERESQm5tLSEiIy+fMnz+f4OBgYmNjOX36NDVq1NA1yQYwqPJ0MUIhhBBCCPH/\ntYkbbYudJXdte8e2N6868s0336CUIjU1lQMHDlCrVi1atmxpbff666/zySef4Ofnx3//+1/Gjh3L\n1atX8fb2ZtKkSdSrV09XX8t8oq2lwhbgsupjzviBmquEaa2M9NqU+U7b7JsyFIBBy5xXf1vULwrQ\nVnXMVayb8bRWhpz83g6n7aZ2eQOAP87IcNpu+7j+mitv9lm4wWmblYNjAFxWmpwe80cAluw+6rTd\ngFbN6PbOapd9yx7Zy2W77JG9AEj9YLfTdomdWgHaqrW5qggKlqqgUzbluGw3JaqNpoqEWvsG2ip0\ntpi6wGXf9k4eorlqpf1O0N7NnaKWz4jWSnKuqjQufNsyTrVWLtQytrRW4HNVfXFadAQAs13sQ8b8\nX5W7RbuOOG03KPwP9F3k+spQKwZFa/5caq2sF56yyGm7XRMGAdry4KqqIlgqK2rNvatqjksHdAO0\n5d5VNUqwVKTUWh3Q1ef8zz0sYyvj02NO2/Vv+TItk5xXcAX4dNJgl8fKdcONAPxp3Tan7Wa99SaA\npjxoraqotWLprG17nbb505stAO1jS0v1Ra3HcS3HNoCI6c6Pzx8n9Ae0jRmtfdNaTfVBZV9V+GY1\n5weRVIYUQgghhBDlRnk6GUMm2kIIIYQQotxw548h7zeZaAshhBBCiHJDJtpCCCGEEEKUgV/NqSNG\no5Hi4mKbx25eR1BKsAshhBBCiHvtVzPRjouLY8KECSxYsMB64W4hhBBCCCHuF3P5mWc7n2i/8MIL\ndOjQgTNnzkjRGiGEEEIIcd/9ala0Afr163cv+iGEEEIIIYRL7ky0bxasOXPmDL6+vqSkpFC7dm3r\n9o0bN5KVlYWPjw+DBg2iRYsW5OfnExcXx/Xr13n00UeZPn06Dz30kK6+SmVIIYQQQghRbtgXPrpZ\nMMuRnTt3smfPHmbMmEFubi5Llixh0SJLYa2LFy/St29fNm/ezI0bN+jevTubN29m1qxZPPPMM0RG\nRpKRkYGvry+9e/fW1dcyv+qI1opSWqrXJbuo2AQwMTKcN1KXuGy3I3GA5sp6ruLtSBwAaKs6prXa\nldYqYVorommpAmVfacmRlG5tNVXxBBi9ZqvTduk9OwBoqoLpqroaWCqsuaqGt2KQpRJp/4yNTttl\n9O8KwLwdh5y2G/ZGqOZKXK6qpIKlUqrWSqRax5aWamLR76xx2beskT1pley84tzuiYMBGLj0Paft\nFsd2AbRVw3OVA7DkQWsFPi37GkBTPFf7ELDsR3ouWOe0zZohbwHQKW2F03YfxPUFtOXe1WvefF1X\nVTzBUskzKt15xdJNoy0VS+fnOM/X0DahgLZqta5yBZZ8zf3koNM2I9o2B3BZXfjDP1l+g6tlv6q1\nKqjW/aDWMeOq6uOnkwZrrlartbKw1pxqqRz7zscHXPZtZEQYo1ZvcdluTq+Omiscztm+32m7UX98\nFYCYuc6rZW4YYXRZ0RYsVW21vr9aqwG7OkandGvrsuoxWCofax0LDyp31ohPnDhB8+aWfUDDhg3J\ny8uzbvvrX/9Ko0aN8PX1xdfXl1q1anH69GlOnDjBgAGWuV1YWBjp6em6J9peup4lhBBCCCHEfWBW\ntjdnCgsLCQgIsN739vampKTEui0wMNC6zd/fn8LCQpvH/f39KSgo0N1XtyfaRUVFul9MCCGEEEII\nT5iV2ebmTEBAACaT6X/PNZvx8fFxuM1kMhEYGGjzuMlkIigoSHdf7zjR3rNnDy1atCA8PJyPP/7f\n6Q7yx5FCCCGEEOJ+Ucr25kzjxo05cMByylJubi4hISHWbc8//zwnTpzgxo0bFBQU8Pe//52QkBAa\nN27M/v2W040OHDhAkyZNdPf1judoL168mA8++AClFCNGjODGjRt06tSpXF1SRQghhBBC/Lq4MxcN\nDw/n8OHDREdHo5QiNTWVlStXUqtWLVq2bInRaKR79+4opRg1ahR+fn4MGjSI+Ph4Nm7cyMMPP8zs\n2bN19/WOE+0KFSoQHBwMwMKFC+nVqxfVq1fHYDDofjEhhBBCCCE8YXZjou3l5UVSUpLNY/Xq1bP+\nv2vXrnTt2tVme9WqVVm+fLlnnbz5+nfa8Lvf/Y7p06dz9epVAgICmD9/PklJSXz33Xd35YWFEEII\nIYRwl1LK5vYgu+NEOzU1lfr161tXsKtXr86aNWto27btPeucEEIIIYQQtypPE20pWCOEEEIIIcoN\n+5oZN2tkPIjKvGCNEEIIIYQQd0t5WiMu84m21uqLWqqOvTZlvsvX2zdlqMvqVGCpUOWqglJip1aA\n9uqLKS4qV06IDNdcSa7Pwg1O26wcHAPgsjrZpM6tAW2Vp7RWLtT6fmitXOiqKmHWyJ4s2X3UZd8G\ntGrm8mdYNtDyrVdrxa5le447bdfv9ZdcVpkES6VJV5XwwFINT8v7AdqrtS3YedhpuyGtX2HRriMu\n+zYo/A8s3/uZ0zZvt2gK4LIC6rToCEsfNewfXFWPBEsFSa051VL5DbTtk7R+Ll1VX0zuajklT0t1\nWdBW4VDrftDV5wgsn6UV+5znvu9rltzbl0a2d7NUsqtKvxMjw11+dsHy+dVa9U/rmNHy+dVafVFr\nxdIpm3KctpsS1UZz37TmXuu+ZvhK5+P03T7aj+NaK726On6A5Rii9f3VOhfRkgetY0ZrRVitYybt\no31O28W1e03z8ajbO6udtske2ctlnPvJjEy0hRBCCCGEuOt+tSva169fx8vLC19f37LqjxBCCCGE\nEHdkdlV3/QHitAT7jz/+yODBg5k0aRJHjhwhIiKCiIgI9u7de6/6J4QQQgghhJWnVx25fv06w4YN\no3v37sTGxpKfn++w3blz52jXrp31/i+//MJLL72E0WjEaDSyerXzU3DAxYp2YmIiw4YN4/z58wwf\nPpycnBz8/Pzo168fLVq0cPPHEkIIIYQQwjOeLmhv2LCBkJAQhg0bxvbt21m4cCETJkywabNlyxbW\nrFnDzz//bH3s5MmTtGvXjokTJ2p+Lacr2iUlJTRt2pROnTrRqlUrHnnkEQICAvDxkVO7hRBCCCHE\nvefpivaJEydo3rw5AGFhYRw9evsftleuXJm1a9faPJaXl8fXX39Njx49GD58OD/99JPL13I6Y65T\npw7jx48nOTmZGTNmAJCRkUHVqlU1/zBCCCGEEELcLcqNq4689957t53i8cgjjxAYGAiAv78/BQUF\ntz3P0ZkbdevWpUGDBvzhD3/gww8/JCUlhXfffdfp6zudaKekpLBnzx68vP638P2b3/wGo9HoNKgQ\nQgghhBBlwezGKnaXLl3o0qWLzWNDhw7FZDIBYDKZCAoK0hTr5Zdf5qGHHgIgPDzc5SQbpDKkEEII\nIYQoR+yv2b5pdB+3nr9ixQpMJpP1HO3PPvuMqVOnOmz7yiuvcPiw5Zr+I0eOpHXr1kRERLBnzx62\nbt3K3Llznb6W03O0hRBCCCGEeJCYle3NXTExMZw9e5aYmBiys7MZOnQoALNmzeKvf/3rHZ83ZswY\nNmzYgNFoJCsri/Hjx7t8LVnRFkIIIYQQ5UantBU29z+I63ufeuKarGgLIYQQQghRBuQ6fUIIIYQQ\notwoNZvvdxc0k4m2EEIIIYQoN9y56sj9JhNtIYQQQghRbpSnPy+Uc7SFEEIIIYQoA/dsRdtsNjNl\nyhTOnDmDr68vKSkp1K5dW1es4uJiEhMTOX/+PEVFRQwaNIiWLVt61L/Lly8TGRnJihUrqFevnkex\nlixZwp49eyguLiYmJua2C6W7o7i4mHHjxnH+/Hm8vLxITk7W1b+vvvqKtLQ0MjMzOXfuHOPGjcNg\nMPDkk08yefJkm6JE7sY7deoUycnJeHt74+vry8yZM92qHnprrJu2bdvG2rVryc7Odqtf9vEuX77M\nhAkTuHLlCqWlpcyaNYtatWrpinXq1CkmT56Mt7c3jz/+ONOmTdP8vjn6zD7xxBO68+AoXo0aNXTl\nwdl40pMHR/EaNmyoKw93+jn15qG0tJQJEybw/fff4+3tzfTp01FK6cqDo1gmk0n3WHAU7+Z75G4e\nHMXy9/fXPRbu9LPqzQPY7nN9fHw83ifdGq+oqMijfZKj44En+6Rb4wUHB3u0T7L/OT3JQceOHa3V\n8WrWrEm3bt2YNm0a3t7ehIaGWi95pjde+/bteeedd/Dx8eGRRx5h5syZ1mIf7saaPn06AIsWLeKb\nb75hzpw5HvVt4MCBTJ48meLiYnx9fUlPT+fhhx/WFevNN98kLS0NHx8fmjVrxqhRo9zqm/2coWnT\nprrHg32sBg0aeDQWHlTlaEEb1D2Sk5Oj4uPjlVJKffnll2rgwIG6Y23atEmlpKQopZTKz89Xr776\nqkd9KyoqUoMHD1atW7dW3377rUexjh07pgYMGKBKS0tVYWGhevfddz2Kt2vXLjV8+HCllFKHDh1S\nQ4cOdTtGRkaGateunerSpYtSSqkBAwaoY8eOKaWUmjhxotq5c6dH8d566y118uRJpZRSGzZsUKmp\nqbpjKaXUyZMnVc+ePW0e0xsvPj5ebd++XSml1NGjR9XevXt1xxo8eLDat2+fUkqp0aNHq08//VRz\nLEefWU/y4Cie3jzcaTzpzYOjeHrz4CiWJ3nYtWuXGjdunFLKMlYHDhyoOw+OYnkyFhzFU0pfHhzF\n8mQsOIrnSR7s97me7pPs43mSB0fHA0/2SfbxPMmDfSxPcnD9+nXVoUMHm8fat2+vzp07p8xms+rX\nr5/Ky8vzKF7r1q3VxYsXlVJKpaWlqdWrV+uOpZRS+/btU9HR0WrkyJGa+3WneEajUX355ZdKKaV2\n7Nih/vKXv+iO1aFDB3X27FllNptVdHS0On36tOa+OZoz6B0PjmJ5MhYeZG2mLba5uevatWtq6NCh\nKiYmRvXr109dvnz5tjbp6ekqKipKdenSRX311VdKKaUuX76s+vTpo2JiYtSIESPU1atXXb7WPTt1\n5MSJEzRv3hyAhg0bkpeXpzvWG2+8wYgRI6z3vb29PerbzJkziY6O5tFHH/UoDsChQ4cICQlhyJAh\nDBw4kNdee82jeHXq1KG0tBSz2UxhYSE+Pu7/EqJWrVrMmzfPev/rr7+madOmAISFhXHkyBGP4qWn\np/P0008DlpUvPz8/3bF+/vln0tLSSExMdKtPd4r3l7/8hQsXLtC7d2+2bdtm/bn1xHr66af55Zdf\nUEphMpncyoWjz6wneXAUT28eHMXyJA+O4unNg6NYnuShVatWJCcnA/Cvf/2LqlWr6s6Do1iejAVH\n8fTmwVEsT8aCo3ie5MF+n+vpPsk+nid5sI/l6T7JPp4nebCP5UkOTp8+zbVr1+jbty89e/bk888/\np6ioiFq1amEwGAgNDeXo0aO64+Xm5pKZmWldPS0pKdGcB0exzp07R3Z2NsOGDdPcpzvF+/LLL8nP\nz2fv3r0YjUZyc3N5/vnndfftZh6Ki4u5ceOGW3MSR3MGvePBUSxPxsKDrNRstrm5a8OGDYSEhLB+\n/Xo6duzIwoULbbafPHmS3NxcNm7cSHp6OhMmTABg4cKFtGvXjvXr1/PMM89o+g3XPZtoFxYWEhAQ\nYL3v7e1NSUmJrlj+/v4EBARQWFjI8OHDGTlypO5+vf/++1SpUsX6JcBTP//8M3l5ecydO5epU6cS\nFxfn0Un7lSpV4vz587Rt25aJEydiNBrdjtGmTRubHbBSCoPBAFjey4KCAo/i3XoAWbt2Lb1799YV\nq7S0lPHjx5OYmIi/v79bfbpT386fP09QUBCrVq2ievXqLF26VHesm7+abdu2LZcvX+all17SHMvR\nZ9aTPDiKpzcP9rFGjBjhUR4c9U1vHhzF8iQPAD4+PsTHx5OcnEybNm08yoN9LE/Ggn281q1be5QH\n+755MhYcxdObB0f7XE9y4Cie3jzYxzKbzR7lwFHf9ObBUSxPxkLFihV5++23Wb58OVOnTiUhIcHm\ntA5382AfLy4ujipVqgCwa9cujh8/TseOHXXHmjRpEklJSboW1uzjjRkzhrNnz9KsWTPWrFnDf//7\nXz744APdfatXrx4DBw4kIiKC6tWrU7duXc19u9OcQc94cBSrWrVqgP590oNKKWVzc9eti79hYWG3\nfal85plnWL58OQaDwbq44Oh5Wr4E3bOJdkBAACaTyXrfbDbrWp296d///jc9e/akQ4cOvPnmm7rj\nbN68mSNHjmA0Gjl16hTx8fFcvHhRd7zg4GBCQ0Px9fWlbt26+Pn5kZ+frzveqlWrCA0NJScnh61b\ntzJu3Dhu3LihOx5gc66XyWQiKCjIo3gAH3/8MZMnTyYjI8O6c3XX119/zblz55gyZQqjR4/m22+/\nZdq0aR71Kzg4mNdffx2A119/3aPfpEybNo1169axY8cOOnbsyIwZM9x6vv1n1tM8OBoDevNwa6zH\nH3/c4zzY982TPNjH8jQPYFkZzMnJYeLEiTbjSU8ebo119epVj8fCzXhDhgzh9OnTHuXh1r4FBgZ6\nPBZujZeSkqIrD472ubfuI93NwZ324XryYB/rzTff5JtvvtGdA0d98/Ly0pUHR7HGjRuneyzUqVOH\n9u3bYzAYqFOnDoGBgfzyyy9eAgwHAAAE7ElEQVTW7e7mwT5ecHAwFy9eZNWqVSxfvpxly5ZpXk21\nj+Xt7c358+cZNWoUqampHDt2jIyMDN19u3ku9ssvv4zBYKBFixaa8+Cob3/+85/Zvn07u3fvpnbt\n2qxYscJ1oP/jaM5w68TanTzcaf5xN47P5dl7771Hu3btbG4FBQXW8+zv9GXGx8eHOXPmMGDAANq1\nawdYFo1dPc/ePZtoN27cmAMHDgCQm5tLSEiI7liXLl2ib9++jB07lqioKI/6tW7dOtauXUtmZiZP\nP/00M2fOtH4D1KNJkyYcPHgQpRQXLlzg2rVrBAcH644XFBRkTWrlypUpKSmhtLRUdzywfFM7fvw4\nAAcOHODFF1/0KN7WrVut7+Fjjz2mO87zzz/P9u3byczMJD09nSeeeILx48d71LcmTZqwf/9+AD7/\n/HOeeOIJ3bEqV65s/a3Mo48+ypUrVzQ/19Fn1pM8OIqnNw/2sTzNg6O+6c2Do1ie5GHLli0sWbIE\ngIceegiDwUCDBg105cFRrF27dukeC/bxqlatyieffKIrD4761rRpU91jwVG84OBgXXlwtM8NCwvT\nPRYcxTty5IiuPNjHujl50jsWHPWtRYsWuvLgKFbNmjV1j4VNmzZZJ+Y3j1WVKlXihx9+QCnFoUOH\n3MqDfbzCwkI2b97MF198wapVq9ya4NnHMhgM7Nixg8zMTBITE3n55Zfp37+/7ngmk4lnn32WL774\nArDk4cknn9QVq7i4mJo1a1KpUiXA/Tw4mjM0a9ZM13hwFOvAgQN35fj8oFHK9uZMly5d+Oijj2xu\ngYGB1sVfZ19mRo0axcGDB1m+fDk//PCDzaKx1i9B9+yqI+Hh4Rw+fJjo6GiUUqSmpuqOtXjxYq5c\nucLChQut59UsXbqUihUr3q3u6taiRQs+//xzoqKiUEoxadIkj84h7927N4mJiXTv3p3i4mJGjRpl\nHdB6xcfHM3HiRNLT06lbty5t2rTRHau0tJRp06ZRvXp167lzv//97xk+fLhHfbxb4uPjmTBhAllZ\nWQQEBDB79mzdsVJSUhg1ahQ+Pj5UqFDBes6qFo4+s+PHjyclJUVXHuzjlZaWcvbsWWrUqOF2Hu72\neHIUb8aMGbry4CiWJ3lo3bo1CQkJvPXWW5SUlJCYmEi9evV0jQdHsRITE3WPBUfx9J5P6SjW008/\nrXssOIoXHBysOw/27uY+yWw2yz5Jg6ioKBISEoiJicFgMJCamoqXlxdxcXGUlpYSGhrKCy+8oDte\nUlISffv25ZlnniE2NhaAtm3b0r17d1198+Q34I7iVapUialTp1JaWkrNmjWJi4vTFSstLY3Lly/T\nt29f/Pz8CAwMdOs3C47mDDVr1tQ1HhzFGjNmzAM7FjzhacGaxo0bs3//fp5//nkOHDhAkyZNbLYf\nPXqUnTt3MnnyZPz8/PDx8cFgMFifFxkZ6fB5jhiUJycQCyGEEEIIUY5cu3bNeppZhQoVmD17NtWq\nVWPWrFm88cYbPPvssyQlJXHmzBnMZjNRUVF07dqVS5cuER8fj8lk4uGHH2b27NkuFz9loi2EEEII\nIUQZkMqQQgghhBBClAGZaAshhBBCCFEGZKIthBBCCCFEGZCJthBCCCGEEGVAJtpCCCGEEEKUAZlo\nCyGEEEIIUQZkoi2EEEIIIUQZkIm2EEIIIYQQZeD/AVcHa6LwOmfwAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x232781cb400>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAtoAAABWCAYAAADxAzG5AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3XlcU1f6P/BPWFVWcZnqUFyLa62i\nY2uLtKhYdVDRisUlirbu+zaIQkWJLFa07hXrCi5otVJ3sS7UpUyl2ikqaG2r1s7XougIcWHJ+f2R\nHxkT483NTV3ofN595fVqcs99cpIn5+Z4ObmPSgghQEREREREfyi7590BIiIiIqI/I060iYiIiIie\nAk60iYiIiIieAk60iYiIiIieAk60iYiIiIieAk60iYiIiIieAk60iYiIiIieAk60iYgqgB49euDg\nwYOG+7m5uWjUqBF27txpeOzSpUto1aoV7t69+zy6SEREJhyedweIiB7VqFEj+Pr6ws7O+DzAsmXL\n4O3tbXafH374AatWrcLixYv/kD7s378fGzduREpKiux9MjIysHjxYtjZ2cHDwwMajQY+Pj4AgKNH\njyIpKQnFxcVo1KgR4uLi4OrqanHbo9zd3VFYWGi4v379eri5uRk9lpKSgpCQELi7uyt96UREL7x1\nx741uh/+9t+eU08s4xltInrhrF+/Hunp6Ua3J02yAeDVV1/9wybZSjx48ADTpk3D0qVLkZ6ejg4d\nOkCj0QAACgoKEBkZiSVLluDAgQN4+eWXMX/+fIvbTLm6uqKoqMiw35EjR9C7d2/DRPvu3bvYtWsX\nBg4c+AxeMRHR86MTwuj2IuNEm4gqjKysLISGhmLChAno3r07QkNDcfnyZWRlZSE4OBgAoNVqMX78\nePTs2RO9evVCVFQUdDodACAtLQ3BwcHo0aMHhg4dip9//tkQe9GiRejUqRP69OmDjIwMw+OHDx9G\naGgoQkJCEBYWhjNnzjzWr7KyMgghDJNerVYLZ2dnAMDx48fx6quvom7dugCAfv36YdeuXRBCSG4z\n5ebmZphob9myBd26dUPt2rUNj23fvh1+fn5o0KCBLW8xEdELTwjj24uMS0eI6IUzePBgo6Uj3t7e\nWLZsGQAgJycHERERaNOmDTZv3oxp06YhIiLC0DYjIwNarRbp6ekoKyvDrFmzcO3aNfz222/47LPP\nkJaWBi8vL+zYsQNjxozBnj178NVXX+HgwYPYuXMnKlWqhDFjxgAAfvnlFyxcuBAbNmxA1apVcenS\nJQwZMgQHDx5ElSpVDM/p4uKC2bNnIywsDJ6entDpdNi8eTMA4P/+7//w0ksvGdq+9NJLKCoqglar\nldxmunykfKJdUlKCLVu2YN26dcjOzkZRURF0Oh02btyI6OhoFBYWYsiQIbh8+TLS0tLg6+v7B2aG\niOj5M3cy4kXFiTYRvXDWr18PLy8vs9saN26MNm3aAADee+89zJkzB3fu3DFsb926NRYuXAi1Wo03\n33wTgwcPRp06dZCWloZu3boZ4vbu3Rtz587Fr7/+ilOnTiEoKMgwuX3vvfeQkpKCEydO4Pfff0d4\neLghvkqlwtWrV9G4cWPDY3l5eVi2bBn27t0LHx8fbNiwAePGjUN6ejp0Oh1UKtVjr8POzk5ymyk3\nNzfcvn0bBw4cQOPGjVG/fn3k5eWhsLAQR48ehb29PQICAlBaWork5GTMmzdPxjtNRFTxWLNcRKfT\nISYmBnl5eXBycoJGo0GdOnUM2zdu3IgdO3ZApVJhzJgxCAwMNCwHvHXrFlxcXJCYmPjE7yRLuHSE\niCoUe3t7ycdefvllZGRkYPjw4SgqKsKQIUNw+PBhw/KRRwkhUFpaavh/03g6nQ7t2rUzWiu+detW\nvPLKK0Zxjh8/Dj8/P8OPHwcMGIBLly7h9u3bqFWrFn7//XdD2xs3bsDDwwNVqlSR3GaqfI32hg0b\nMGjQIKPHUlNToVaroVKp4OjoqPgLgYioIhBCGN2kHDp0CMXFxUhLS8OUKVOQkJBg2FZQUIBNmzYZ\n/koYExMDIQQ2b94MX19fbNq0CSEhIVi+fLnivnKiTUQVSm5uLnJzcwHo11y3atUKbm5uhu2bNm1C\nZGQk/P39MW3aNPj7++P8+fNo37499u7di4KCAgD6Nc2enp6oU6cOAgICsH//fty9exc6nQ7p6ekA\ngHbt2uHEiRO4fPkyAODYsWPo0aMHHjx4YNSnpk2b4ttvv8XNmzcB6A/s3t7e8PLygr+/P77//nv8\n8ssvAPTrqzt27AgAkttMubm54fTp0ygqKoK/vz8A/UT7woUL+OGHH9CrVy9b31oiogrBmh9DZmdn\no3379gCAli1bIicnx7DNy8sL6enpcHR0xM2bN+Hu7g6VSmW0T0BAAE6dOqW4r1w6QkQvHNM12gAw\nefJkVKpUCdWrV8cnn3yC69evw8vLC/PmzcP169cN7UJCQvDPf/4T3bp1Q+XKlVGrVi2o1Wp4eHgg\nPDwcgwcPhk6ng5eXF1auXAk7Ozu8/fbbyMvLw3vvvQd3d3c0btwYt2/fRsOGDTFnzhxMnjwZQgg4\nODhgxYoVcHFxMepbu3bt8MEHH0CtVsPR0REeHh6GMyDVqlVDfHw8xo8fj5KSEvj4+CAxMdHiNlNu\nbm64ceMGRo8ebXjM1dUV+fn5GDx48GN9IiL6s7Jm6UhRUZHRb17s7e1RWloKBwf9FNjBwQGpqalY\nsmQJ1Gq1YZ/yEzguLi5Gl1G1lkpUpBXlRPQ/LSsrC7Gxsdi9e/fz7soLb/r06Rg6dCh/DElEfzpL\nDxw3uj/2Xf8nto2Pj8drr72Gbt26AdCfoc7MzHysXXFxMYYNG4ZRo0YhNTUVw4cPR4sWLVBYWIh+\n/fop/t7h0hEioj+ZYcOG4fjx44iOjsaOHTued3eIiP5Q1lzez8/PzzCxPnv2rNHJh59++gljx46F\nEAKOjo5wcnKCnZ0d/Pz8cOzYMQBAZmYmWrdurbivPKNNRERERBXGon1fG92f0LX9E9uWX3Xk4sWL\nEEIgLi4OmZmZ8PHxQceOHbF06VJkZmZCpVKhffv2GDt2LO7fv4+IiAjk5+fD0dERSUlJqFGjhqK+\ncqJNRERERBXGJ3uNl35M7BbwnHpiGX8MSUREREQVRkU6R8yJNhERERFVGNZcdeR540SbiIiIiCqM\nijPNfgYT7d5JayW375gyBACw7OAJyXZjOr+FLnErLT7f/hkj8I+Nuyy2mzegOyat3ynZZuHgEABA\nt/hkyXZ7I4cDACI375FsF9/v7xi5apvFvn06LBTzdh2RbPOP7oH6Pu45Jtlu0t/fBgCLz/vpsFDM\n3LLXYt/mhnV7bG2UqfK1UkOWb5Zst3Z0PwBAv0Upku02T1DLft/ivjgk2WZGr04AgOit+yTbxfbt\nCgCYvCFdst2CQT0xYLF0/wFg43g1es1fY7HdF1OHYlrql5JtPh7YQ3bfACDk49WS7XZO+0B234I0\nKyTbZESNsqpvUWnSedC83xXDVqZZ7NuqEe8/9uMYU+U/lpF7TOqzQLrd55OHYPTqzy32bfkHfWR9\nxgF5x0G57eTmVO64j92RIdkmuncQACDxy8OS7SJ6dAAgLw+WPruA/vM7aNlGyTYbxgwAACTtPirZ\nbkrwOwBg8TO3asT7GJ681WLfkof3xYR1X0i2WRSuLzQ0fq301WEWD+kNAOgx7zPJdl/+40PM2X7Q\nYt8+eq+zrGMDAIQuXCfZbtukcNntLB3fAP0xztLnCNB/luZbyOnU/59TuXmQMy+w1Ka8nZxcAcCS\n/ccl243ror90nZzvN7nfR3KP0S8qntEmIiIiInoK/pRrtHU63WOV2oiIiIiIniWd7k8y0b527Rri\n4+ORk5MDBwcH6HQ6+Pr6IjIyEvXq1XtWfSQiIiIiAmDdGe3y62jn5eXByckJGo0GderUMWpTUFCA\nsLAw7Nq1C87OzhBCICAgAHXr1gUAtGzZElOmTFHUV8mJ9syZMzFlyhS89tprhsfOnj2LyMhIbNmy\nRdETEhEREREpZc0a7UOHDqG4uBhpaWk4e/YsEhISsGLFf39v9PXXXyMpKQk3b940PHb16lU0a9YM\nn376qc19lVwLUlxcbDTJBvSzeiIiIiKi50GY3KRkZ2ejfXv9j+FbtmyJnJwco+12dnZYu3YtPD09\nDY+dO3cON27cgFqtxrBhw/DTTz8p7qvkGe1GjRohMjIS7du3h5ubG7RaLY4dO4ZGjRopfkIiIiIi\nIqWsWTpSVFQEV1dXw317e3uUlpbCwUE/BX7rrbce26dGjRoYPnw4unbtitOnT2PatGnYvn27or5K\nTrRjYmJw6NAhZGdnGzoaGBiIoKAgRU9GRERERGQLa5aOuLq6QqvV/ndfnc4wyX6S5s2bw97eHgDQ\npk0b3LhxA0IIqFQqq/sq+UwqlQpBQUGcWBMRERHRC8GaM9p+fn44cuQIunXrhrNnz8LX19fiPkuX\nLoWnpyeGDRuG3Nxc1K5dW9EkGwBUoiJdjJCIiIiI/qeZFp0rLzJnTvlVRy5evAghBOLi4pCZmQkf\nHx907NjR0K5Dhw7Yt28fnJ2d8Z///AfTpk3DvXv3YG9vj48++ggNGjRQ1NenPtGWU2ELkFd9UW6l\ns4FLUi22Sx03EO/ELJVsczRmLABg1GfS1d9WfNgHgLyqY5ZilceTW+0q5vMDku1i+rwLAAhOXCXZ\nbnfEMNmVN4eukL7izJpRYQDkVcoEgM8OZ0m2+7DD6xYrjgH6qmNhn2yQbLNl4iAA8ipsAcDYNdLV\n2pYO7W3xdQL612qpsh6gr64ntwqb3MqmH34qna/PRoZZHAuAfjxMSZGu6pak1letlPu5tFSVcG5Y\nN9mV5OTkCgDid34l2S4yRH/gnbVtv2S72aFdoJGR06jeQbJeJyC/0mvyV99Ithve8Q2LeQf0uZdb\nRVBuZb1350r/Sv/AzJEA5FWQtFQtEdBXTJTbN7nfR3JyP3j5Jot9Wz+6v6xjLwCLFY3nDegOQN7x\n0lIOAH0eLL2G9aP7W9U3OdUtLeUK0OdLbsVSudU+5bazNKajegfJrlQsJ1cA8PcE6fnPnun66tNy\njl1yq9XKrab6ojKtKqx5/8kT7eeNlSGJiIiIqMKoSIsxONEmIiIiogrDmh9DPm+caBMRERFRhcGJ\nNhERERHRU/CnWTqiVqtRUlJi9Fj5dQRZgp2IiIiInrU/zUR76tSpiIqKwrJlywwX7iYiIiIiel50\nFWeeLT3Rfu2119CzZ0/k5eWxaA0RERERPXcV6Yw2C9YQERERUYVhWkeivG6EOeUFa/Ly8uDk5ASN\nRoM6deoYtm/duhVbtmyBg4MDRo0ahcDAQBQUFGDq1Kl48OABatasifj4eFSuXFlRX+0U7UVERERE\n9BzohDC6STl06BCKi4uRlpaGKVOmICEhwbAtPz8fKSkp2LJlC1avXo0FCxaguLgYy5cvR3BwMDZt\n2oSmTZsiLU262JWUp37VkYhNuyW3J/YPBgAMWb5Zst3a0f1kV2GzVGUS0FealFPFCpBXtRKQV3VM\nbrUrS9XwPh6or8AntxqenCpQppWWzNG839ViBcn9M0YAACZvSJdst2BQTwDAvF1HJNv9o3ugxepq\ngL7CmpwqiAAwPHmrZLvk4X0BAEv2H5dsN66Lv+xqn3Ir9cmtRCp3bMmpwNdvUYrFvm2eoEan2OWS\nbQ5FjwYgv5qqnIpzSw9I5wAAxr7rjx7zPpNs8+U/PgQg71gDQFY8S+MP0I9BuRX4eietlWy3Y8oQ\nAPJyL7dyoaUqiIC+EqLciqWW8jX2XX8A8qrhWcoVoM/Xon1fS7aZ0LU9AHk5BeQdV+VW1JRbTVXu\nmJEzBuVWq5VTARMAlh08IdluTOe3AEDWa7WUK0CfL0uVbwH9WUy5FQ7lVl0dsFj6WLhxvFr2+yu3\nQm6fBdLj/vPJ+nFv6Tta835Xi9UjAX0FyZCPV0u22TntA4txnidrFmNkZ2ejfXv9MaBly5bIyckx\nbPvXv/6FVq1awcnJCU5OTvDx8UFubi6ys7MxYoR+LhMQEIAFCxYgPDxcUV95RpuIiIiIKgydML5J\nKSoqgqurq+G+vb09SktLDdvc3NwM21xcXFBUVGT0uIuLCwoLCxX31eqJdnFxseInIyIiIiKyhU7o\njG5SXF1dodVq/7uvTgcHBwez27RaLdzc3Iwe12q1cHd3V9zXJ060Dx8+jMDAQAQFBWHv3v8ud/jw\nww8VPxkRERERkS2EML5J8fPzQ2ZmJgDg7Nmz8PX1NWxr0aIFsrOz8fDhQxQWFuLy5cvw9fWFn58f\njh3TLzfKzMxE69atFff1iWu0P/30U3zxxRcQQmDChAl4+PAhevXqVaEuqUJEREREfy7WzEWDgoJw\n4sQJhIWFQQiBuLg4rF27Fj4+PujYsSPUajX69+8PIQQmTZoEZ2dnjBo1ChEREdi6dSuqVq2KpKQk\nxX194kTb0dERnp6eAIDly5dj8ODBqFWrFlQqleInIyIiIiKyhaUrjTzKzs4Oc+bMMXqsQYMGhv/v\n27cv+vbta7S9evXqWL1a+gejsp//SRv++te/Ij4+Hvfu3YOrqyuWLl2KOXPm4KeffvpDnpiIiIiI\nyFpCCKPbi+yJE+24uDg0atTIcAa7Vq1a2LBhA7p27frMOkdERERE9KiKNNFmZUgiIiIiqjCGrjCu\nS7FmVNhz6ollT71gDRERERHRH6UinSN+6hNtudUXe81fI9nui6lD8U7MUovPdzRmrMXqVIC+QpWc\ninkAMHq1dMWu5R/oK3ZZqlwZ1TvI4vsB6N8TudXrYi08Z3TvIADyKs5ZqmwJ6Ktbyq1GKbdyoZyK\nc8lffWOxb8M7viGrOicA2RW7Vh/5p2S7DwLbYuSqbRb79umwUNkVS9//ZL1km7SJgwHIr3Qmp93K\nQ6cs9m1Ep3ay3g8AFiugzg3rBkBeBT5L1SMBfQVJuTkduCRVsl3quIEAICsPcqtWRm+VrugW21e/\nJE9uZVNLlVJ3RwyzWF0W0FeYtVT9FNBXQF137FvJNuFv/w0ALFb0Wzg4BIC8Mbgi46TFvo0KelP2\nsVxuhUM5Ffjkvr9yq1HK/fzKOa5OWPeFxb4tCu8lu9qnpXiLwnsBkFfhUO7nzdLrBPSvVW7lZrnf\nW3K+Uy19jgD9Z0nusUbu+5u0+6hkuynB71isegzoKx/Lzf2LSgdOtImIiIiI/nAV6Yy2VZUhHzx4\nwMqQRERERPTc6HTC6GatBw8eYNy4cejfvz+GDRuGgoICs+2uXLmC4OBgw/07d+7g9ddfh1qthlqt\nxvr10n/5BCyc0b527Rri4+NRvXp1dOnSBVFRUbCzs8PMmTMRGBho5csiIiIiIrKNrWe0N2/eDF9f\nX4wbNw579uzB8uXLERUVZdRm586d2LBhA27fvm147Pz58wgODkZ0dLTs55I8oz1jxgyEh4ejVatW\nGD9+PLZt24adO3di5cqVVr4kIiIiIiLb6YTxzVrZ2dlo3749ACAgIACnTj3+OyUPDw+kphqvs8/J\nycG5c+cwcOBAjB8/Hr///rvF55I8o11aWoq2bfU/bsrKykK1atX0OzlwaTcRERERPXvWnNHetm3b\nY0s8qlWrBjc3NwCAi4sLCgsLH9vP3MqN+vXro3nz5njzzTfx5ZdfQqPRYPHixZLPL3lGu169epg5\ncyZ0Oh0SEhIAAMnJyahevbr0qyIiIiIiegqEyX9SQkNDsXv3bqObm5sbtFotAECr1cLd3V3W877x\nxht4/fXXAQBBQUE4f/68xX0kJ9oajQaBgYGws/tvs7/85S+Ij4+X1SEiIiIioj+STgijm7X8/Pxw\n7Jj+0reZmZlo3bq1rP2ioqJw4ID+UpynTp1Cs2bNLO7DypBEREREVGGYXrP988lDrNr//v37iIiI\nQH5+PhwdHZGUlIQaNWpg3rx56NKlC1q0aGFo+9Zbb+HECf21069du4YZM2YAACpXrgyNRoOaNWtK\nPhcn2kRERERUYZgW4dsxxbqJ9rPEXzUSERERUYVRkc4Rc6JNRERERBUGJ9pERERERE+Bkh9APi+c\naBMRERFRhVGRJtqSl/cjIiIiIiJleEabiIiIiCoMnZK668/JM5to63Q6xMTEIC8vD05OTtBoNKhT\np46iWCUlJZgxYwauX7+O4uJijBo1Ch07drSpf7du3ULv3r2xZs0aNGjQwKZYK1euxOHDh1FSUoJ+\n/fohNDRUcaySkhJMnz4d169fh52dHWJjYxX17/vvv8f8+fORkpKCK1euYPr06VCpVHjllVcwa9Ys\no6JE1sa7cOECYmNjYW9vDycnJyQmJlpVPfTRWOV27dqF1NRUpKWlWdUv03i3bt1CVFQU7t69i7Ky\nMsybNw8+Pj6KYl24cAGzZs2Cvb096tati7lz58p+38x9Zhs2bKg4D+bi1a5dW1EepMaTkjyYi9ey\nZUtFeXjS61Sah7KyMkRFReHnn3+Gvb094uPjIYRQlAdzsbRareKxYC5e+XtkbR7MxXJxcVE8Fp70\nWpXmATA+5jo4ONh8THo0XnFxsU3HJHPfB7Yckx6N5+npadMxyfR12pKDkJAQQxlqb29vvP/++5g7\ndy7s7e3h7++PsWPHWvU6TeP16NEDn3zyCRwcHFCtWjUkJiaicuXKimKVF8pbsWIFLl68iIULF9rU\nt5EjR2LWrFkoKSmBk5MTFixYgKpVqyqK1b17d8yfPx8ODg5o164dJk2aZFXfTOcMbdu2VTweTGM1\nb97cprHworJ16ciDBw8wbdo03Lp1Cy4uLkhMTISXl5dRm4ULF+LkyZNQqVSIiopCixYtUFBQgKlT\np+LBgweoWbMm4uPjLX+mxTNy4MABERERIYQQ4syZM2LkyJGKY33++edCo9EIIYQoKCgQb7/9tk19\nKy4uFqNHjxadO3cWP/74o02xvvnmGzFixAhRVlYmioqKxOLFi22Kl5GRIcaPHy+EEOL48eNi7Nix\nVsdITk4WwcHBIjQ0VAghxIgRI8Q333wjhBAiOjpaHDx40KZ4AwYMEOfPnxdCCLF582YRFxenOJYQ\nQpw/f14MGjTI6DGl8SIiIsSePXuEEEKcOnVKHDlyRHGs0aNHi6NHjwohhJg8ebL46quvZMcy95m1\nJQ/m4inNw5PGk9I8mIunNA/mYtmSh4yMDDF9+nQhhH6sjhw5UnEezMWyZSyYiyeEsjyYi2XLWDAX\nz5Y8mB5zbT0mmcazJQ/mvg9sOSaZxrMlD6axbMnBgwcPRM+ePY0e69Gjh7hy5YrQ6XTiww8/FDk5\nOTbF69y5s8jPzxdCCDF//nyxfv16xbGEEOLo0aMiLCxMTJw4UXa/nhRPrVaLM2fOCCGE2L9/v/ju\nu+8Ux+rZs6e4dOmS0Ol0IiwsTOTm5srum7k5g9LxYC6WLWPhRdZZs8LoZq01a9YY5me7d+8WsbGx\nRtvPnTsnBg0aJHQ6nbh27Zro3r27EEKI2NhYsX37diGEECtXrhRr1661+FzPbI12dnY22rdvDwBo\n2bIlcnJyFMfq0qULJkyYYLhvb29vU98SExMRFhZmsbqPHMePH4evry/GjBmDkSNH4p133rEpXr16\n9VBWVgadToeioiI4OFj/RwgfHx8sWbLEcP/cuXNo27YtACAgIAAnT560Kd6CBQvQpEkTAPozX87O\nzopj3b59G/PnzzdUXrKWabzvvvsON27cQHh4OHbt2mV43UpiNWnSBHfu3IEQAlqt1qpcmPvM2pIH\nc/GU5sFcLFvyYC6e0jyYi2VLHjp16oTY2FgAwG+//Ybq1asrzoO5WLaMBXPxlObBXCxbxoK5eLbk\nwfSYa+sxyTSeLXkwjWXrMck0ni15MI1lSw5yc3Nx//59DB06FIMGDcK3336L4uJi+Pj4QKVSwd/f\nH6dOnVIc7+zZs0hJSTGcPS0tLZWdB3Oxrly5grS0NIwbN052n54U78yZMygoKMCRI0egVqtx9uxZ\no0qA1vatPA8lJSV4+PChVXMSc3MGpePBXCxbxsKLzNYS7I/OSQMCAh77rDdt2hSrV6+GSqUyHPPM\n7ScnN89sol1UVARXV1fDfXt7e5SWliqK5eLiAldXVxQVFWH8+PGYOHGi4n7t2LEDXl5ehjfOVrdv\n30ZOTg4WLVqE2bNnY+rUqTZd77FKlSq4fv06unbtiujoaKjVaqtjvPvuu0YHYCEEVCoVAP17WVhY\naFO8R79AUlNTER4erihWWVkZZs6ciRkzZsDFxcWqPj2pb9evX4e7uzvWrVuHWrVqYdWqVYpjlf9p\ntmvXrrh16xZef/112bHMfWZtyYO5eErzYBprwoQJNuXBXN+U5sFcLFvyAAAODg6IiIhAbGws3n33\nXZvyYBrLlrFgGq9z58425cG0b7aMBXPxlObB3DHXlhyYi6c0D6axdDqdTTkw1zeleTAXy5axUKlS\nJXzwwQdYvXo1Zs+ejcjISKM/gVubB9N4U6dONfwpPiMjA1lZWQgJCVEc66OPPsKcOXMUnVgzjTdl\nyhRcunQJ7dq1w4YNG/Cf//wHX3zxheK+NWjQACNHjkS3bt1Qq1Yt1K9fX3bfnjRnUDIezMWqUaMG\nAOXHpBeVEMLoJmXbtm0IDg42uhUWFhqW/zzpPXZwcMDChQsxYsQIBAcHA9DPZS3tZ+qZTbRdXV2h\n1WoN93U6naKzs+X+/e9/Y9CgQejZsye6d++uOM727dtx8uRJqNVqXLhwAREREcjPz1ccz9PTE/7+\n/nByckL9+vXh7OyMgoICxfHWrVsHf39/HDhwAOnp6Zg+fToePnyoOB4Ao7VeWq0W7u7uNsUDgL17\n92LWrFlITk5+bJ2TXOfOncOVK1cQExODyZMn48cff8TcuXNt6penpyc6dOgAAOjQoYNNf0mZO3cu\nNm7ciP379yMkJAQJCQlW7W/6mbU1D+bGgNI8PBqrbt26NufBtG+25ME0lq15APRnBg8cOIDo6Gij\n8aQkD4/Gunfvns1joTzemDFjkJuba1MeHu2bm5ubzWPh0XgajUZRHswdcx89Rlqbgycdw5XkwTRW\n9+7dcfHiRcU5MNc3Ozs7RXkwF2v69OmKx0K9evXQo0cPqFQq1KtXD25ubrhz545hu7V5MI3n6emJ\n/Px8rFu3DqtXr8Znn30m+2xTdqDVAAAE6ElEQVSqaSx7e3tcv34dkyZNQlxcHL755hskJycr7lv5\nWuw33ngDKpUKgYGBsvNgrm8ff/wx9uzZg0OHDqFOnTpYs2aN7L6ZmzM8OnmzJg9Pmn/8Ed/PLxoh\njG9SQkNDsXv3bqObm5ubYU4q9R5PmjQJX3/9NVavXo2rV68azWXl5uaZTbT9/PyQmZkJADh79ix8\nfX0Vx7p58yaGDh2KadOmoU+fPjb1a+PGjUhNTUVKSgqaNGmCxMREw78AlWjdujW+/vprCCFw48YN\n3L9/H56enorjubu7G/715OHhgdLSUpSVlSmOB+j/JJKVlQUAyMzMRJs2bWyKl56ebngPX375ZcVx\nWrRogT179iAlJQULFixAw4YNMXPmTJv61rp1axw7dgwA8O2336Jhw4aKY3l4eBj+KlOzZk3cvXtX\n9r7mPrO25MFcPKV5MI1lax7M9U1pHszFsiUPO3fuxMqVKwEAlStXhkqlQvPmzRXlwVysjIwMxWPB\nNF716tWxb98+RXkw17e2bdsqHgvm4nl6eirKg7ljbkBAgOKxYC7eyZMnFeXBNFb55EnpWDDXt8DA\nQEV5MBfL29tb8Vj4/PPPDRPz8u+qKlWq4OrVqxBC4Pjx41blwTReUVERtm/fjtOnT2PdunVWTfBM\nY6lUKuzfvx8pKSmYMWMG3njjDQwfPlxxPK1Wi2bNmuH06dMA9Hl45ZVXFMUqKSmBt7c3qlSpAsD6\nPJibM7Rr107ReDAXKzMz8w/5fn7R2Lp0xM/PzzAOMzMz0bp1a6Ptp06dwuzZswEAzs7OcHBwgEql\nsrifOSphy7oGK5RfdeTixYsQQiAuLk7x1T00Gg327dtn9OeZVatWoVKlSjb1Ua1WIyYmxuarjsyb\nNw9ZWVkQQmDSpEk2LUvRarWYMWMG8vPzUVJSgkGDBik6g//rr79i8uTJ2Lp1K37++WdER0ejpKQE\n9evXh0ajsfrPceXxNm/ejHbt2qFWrVqGf9n97W9/w/jx4xX1TeoxJfGuX7+OqKgo3L9/H66urkhK\nSoKHh4eiWKdPnzb8stzR0RGxsbHw9vaWFcfcZ3bmzJnQaDSK8mAar6ysDJcuXULt2rWtzoPUeFKS\nB3PxEhISFOXBXKwJEyYozsO9e/cQGRmJmzdvorS0FMOGDUODBg0UjQdzsWbMmKF4LJiL16lTJwDW\njwdzsZo0aaJ4LJiL5+npqTgP5cqPuXZ2djYfk8rjffTRRxgwYIBNx6RH+1b+fWDLMenReJUqVbLp\nmPRorPL140pyUFxcjMjISPz2229QqVSYOnUq7OzsEBcXh7KyMvj7+1t19QzTeBMnTsTQoUPRtGlT\nw5nsrl27on///or65ufnBwDIysrCli1brLrqiLl4VapUwezZs1FWVgZvb28kJCTAyclJUaxbt24h\nOTkZzs7OcHNzQ0JCglU5NZ0zeHt7Kx4PprGmTJli81h4Eb0Ts9To/tEY666Qc//+fcNfvxwdHZGU\nlIQaNWpg3rx56NKlC5o1a4Y5c+YgLy8POp0Offr0Qd++fXHz5k1ERERAq9WiatWqSEpKMvwj60me\n2USbiIiIiOh/CStDEhERERE9BZxoExERERE9BZxoExERERE9BZxoExERERE9BZxoExERERE9BZxo\nExERERE9BZxoExERERE9BZxoExERERE9Bf8P56726GnDYFMAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x23279b8bd30>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAtoAAABXCAYAAAA6X+IcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3XlcVPX+P/DXwIALi+TSTTP3MJdM\nsWt5A4oMF0IlRUUTUXPfF7wI4pIiqAGaGipqLriAS2kuiTuooVdNeoSKWt20673XVPTKjAvLfH5/\nzJfJGYdzzpwRhX6vZ4955Mz5nPd8Zt7zOefDmTPnrRFCCBARERER0VPl8Lw7QERERET0Z8SJNhER\nERFRGeBEm4iIiIioDHCiTURERERUBjjRJiIiIiIqA5xoExERERGVAU60iYiIiIjKACfaREQVTLdu\n3bB//37T/dzcXDRt2hQ7duwwPXblyhW0adMG9+7dex5dJCIiANrn3QEioqZNm8LT0xMODuZ/+3/x\nxReoW7eu1XV+/PFHrFy5EosXL34qfdi3bx82btyIlJQUm9YTQmDq1Knw9PTEJ598Ynr86NGjSEhI\nQEFBAZo2bYrY2Fi4urpKLpNa53Hu7u7Iz8833V+3bh3c3NzMHktJSUFQUBDc3d1tfSuIiMq11Uf+\nYXb/E792z6kn8nhEm4jKhXXr1mHnzp1mt9Im2QDw+uuvP7VJtlo///wzwsLCkJ6ebvZ4Xl4eIiMj\nsWTJEqSnp+OVV15BfHy85DKpdSy5urpCp9OZ4h05cgQ9evQwTbTv3buHXbt2oX///mX46omISA6P\naBNRuXbq1CnEx8ejTp06+OWXX1C5cmXMmzcPt27dwpw5c7B7927o9XpERkbi6tWrcHBwQIsWLTB7\n9mw4ODggLS0NKSkpcHBwQM2aNTF9+nQ0bNgQAPD5559j165d8PDwQP369U3PefjwYSxbtgyFhYWo\nXLkyIiIi0KZNmyf6tnHjRvTq1Qt16tQxe/z48eN4/fXX0aBBAwBA37590b17d8ycObPUZV5eXqWu\no9FozOK7ubmZJtqpqakICAhAnTp18PvvvwMAtm/fDi8vLzRu3Nju95+IqLwRQihuazAYMGvWLFy6\ndAnOzs6IiYkx295v3LgRX331FTQaDUaPHg0/Pz88fPgQU6ZMwe3bt+Hi4oL58+ejevXqqvrKI9pE\nVC6EhYWhe/fuptvo0aNNy3JychAaGopdu3ahR48emDJlitm6Bw4cgF6vx86dO7Ft2zYAwG+//Yas\nrCysWrUK69evxzfffIPAwECMHj0aQggcPHgQ+/fvx44dO5CammqauP76669YuHAhkpOTsWPHDsyZ\nMwdjx47F/fv3n+jzjBkz0LVr1yce/+9//4uXXnrJdP+ll16CTqeDXq8vddm1a9dKXcdSyUS7sLAQ\nqamp6N+/P1xcXKDT6WAwGLBx40YMGDAAAJCfn4/g4GC0adMGly9fVpQLIqLyzCCE2U3KwYMHUVBQ\ngLS0NEyePBnz5s0zLcvLy8OmTZuQmpqKtWvXYtasWRBCYPPmzfD09MSmTZsQFBSEpKQk1X3lRJuI\nygXLU0e++OIL07LXXnsNb775JgCgZ8+euHjxIu7evWta3rZtW/z0008IDQ1FcnIywsLCUL9+fRw7\ndgwBAQGmIxE9evTAjRs38K9//QtZWVnw9/eHq6srtFotevbsCQA4ceIEfv/9dwwcOBDdu3dHeHg4\nNBoNrl27pvi1GAyGJ45CA4CDg0OpyxwdHUtdx5Kbmxv0ej3S09Px2muvoVGjRnB1dUV+fj6OHj0K\nR0dH+Pr6AgAqV66M5ORkdOrUSXH/iYjKMyGE2U3K2bNn4ePjAwBo3bo1cnJyTMuqV6+OnTt3wsnJ\nCbdu3YK7uzs0Go3ZOr6+vsjKylLdV060iajcc3R0lHzslVdewYEDBzBs2DDodDoMGjQIhw8fhsFg\neGI9IQSKiopM/7aMZzAY0L59e7NJ/5YtW/Dqq68q7m/t2rVNp3EAwI0bN1CtWjVUrVq11GWPn/ph\nuY6lknO0169fbzpyXfLYhg0bEBoaapq0Ozk5qf7Kk4ioPLLliLZOpzP7Ubmjo6NpHwAAWq0WGzZs\nQJ8+fUwHJHQ6Hdzc3AAALi4uZj80txUn2kRU7uXm5iI3NxcAkJaWhjZt2pg2ggCwadMmREZGwtvb\nG1OmTIG3tzcuXLgAHx8f7N27F3l5eQCM5y6XnI/t6+uLffv24d69ezAYDNi5cycAoH379jhx4gR+\n/vlnAEBGRga6deuGhw8fKu6vt7c3fvjhB/z6668AjOdRd+jQQXKZ1DqW3NzccObMGeh0Onh7ewMw\nTrQvXryIH3/8ER999JHivhIRVTS2TLRdXV3NTsEzGAzQas1/oti/f38cO3YMp0+fxsmTJ83W0ev1\ndl29iT+GJKJyISws7InTJCZNmoTKlSujZs2aWLRoEa5fv47q1atjwYIFuH79uqldUFAQ/vGPfyAg\nIABVqlRB7dq1ERoaimrVqmHgwIEICwuDwWBA9erVsWLFCjg4OODdd9/FpUuX0LNnT7i7u+O1117D\nnTt30KRJE8yePRuTJk2CEAJarRbLli2Di4uL4tdSo0YNxMXFYdy4cSgsLES9evUwf/58yWUeHh6l\nrmPJzc0NN27cwKhRo0yPubq64ubNmwgLC7Opr0REFY0tP4b08vLCkSNHEBAQgOzsbHh6epqW/fLL\nL0hMTMSSJUvg5OQEZ2dnODg4wMvLCxkZGWjVqhUyMzPRtm1b1X3VCFt6S0T0jJ06dcp0dRFSb+rU\nqRg8eLDZToaIqCJasu+42f2xnb1LbVty1ZHLly9DCIHY2FhkZmaiXr166NChA5YuXYrMzExoNBr4\n+PhgzJgxePDgASIiInDz5k04OTkhISEBtWrVUtVXTrSJqFzjRNt+Q4cOxcWLF/Hyyy+jT58+6NGj\nx/PuEhGRap9/e8zs/vguPs+pJ/I40SYiIiKiCmPR3kyz+xMCfJ9TT+TxHG0iIiIiqjAq0jFiTrSJ\niIiIqMKQu9JIecKJNhERERFVGBVnms2JNhERERFVIDyiTURERERUBniO9mM+iv9ScvnX4YMBAF/s\nPyHZbnTHd9Bp7nLZ50ufNgJTNnwj2+6z/t0wfu3Xkm0+H2isrtY5doVku31RwwEAEZukLz82v18g\nhiVvke1b8rDemP/NYck2Ed3eBwAs3JMh2W7ih+8CgOzzJg/rjcjNe2T7Ftf3wyd+7Wup5Ne/YUmb\nJNutG9UPABCyaL1ku9QJAzBi5VbZvi0f2gsxXx2QbBPdw9/4/7RvJdvF9OkCAJi0fqdku8QB3fHx\n4hTZvm0cF4qgz1bLttsx5RNMTpH+/CaEdgMATFy3Q7LdwrAgAEC3Bask233z9yGK++Yfs0yyzYHo\nkTb1TUkehixPle3bqhEhij+XSrdJPRLWSLb7avIgjFy1TbZvy4YEo+/n0p+RzeNDAQBL049LthvT\nyVtxO7nXCRhf67TUvbLt5oYEYI7M2Jr+f2MrbschyXaRQcZql0ryIPfZBYyf3/5LNki22TC2PwAg\nfvdRyXbhge8BgOxnbtWIEAxdkSbbt5XD+2Dcmq8k2yweZLzU4pgvpdstHWxsFzh/pWS73RFDMWtb\numzfZgV3UrRtAIBeC9dKtts6cSAAIDhResxsmzRI8f5Zbh8IGPeDC3YdkWzz965+AKB4f6+knVyb\nknZKcgU8eak6SyWXrlOyf1O6P1KybyvPDIY/4UTbYDA8UbWNiIiIiOhZsuWIdknBmkuXLsHZ2Rkx\nMTGoX7++WZu8vDyEhIRg165dqFSpEoQQ8PX1RYMGDQAArVu3xuTJk1X1VXKi/dtvvyEuLg45OTnQ\narUwGAzw9PREZGQkGjZsqOoJiYiIiIjUsuUc7YMHD6KgoABpaWnIzs7GvHnzsGzZH9/OHjt2DAkJ\nCbh165bpsWvXrqFFixZYvlz+TAo5khPtadOmYfLkyXjjjTdMj2VnZyMyMhKpqfJf5xIRERERPU22\nnDhy9uxZ+PgYT79p3bo1cnJyzJY7ODhgzZo16Nmzp+mx8+fP48aNGwgNDUXlypURGRmJRo0aqeqr\n5ES7oKDAbJJd0kkiIiIioufBllNHdDodXF1dTfcdHR1RVFQErdY4BX7nnXeeWKdWrVoYNmwYunTp\ngjNnzmDKlCnYvn27qr5KTrSbNm2KyMhI+Pj4wM3NDXq9HhkZGWjatKmqJyMiIiIisoctp464urpC\nr9f/sa7BYJpkl6Zly5ZwdHQEALz55pu4ceMGhBDQaDQ291XymWbNmoWDBw/i7Nmzpr8I/Pz84O/v\nb/MTERERERHZy5Yj2l5eXjhy5AgCAgKQnZ0NT09P2XWWLl0KDw8PDB06FLm5uahTp46qSTYgM9HW\naDTw9/fnxJqIiIiIygVbjmj7+/vjxIkTCAkJgRACsbGxWLNmDerVq4cOHTpYXWfYsGGYMmUKMjIy\n4OjoiLi4ONV91YiKdNVvIiIiIvr/mmUNhpLaF+VRmResUXLhf0DZRfjl2pS0U3rB9vdmLZVsc3TW\nGACQLUqxbEgwACBBphjC5MD3FBdeUVqwRq4wwazgTgCAgLhkyXZ7I4cpLgg0KGmzZJs1o/oa+6ig\ngA8ArDiYJdlu+AftZQsmAMaiCX0WrZNskzYhDICyC/8DyopIKC30I1fwAzAW/VD6GpQWwpArrLFy\neB/ZsQAYx8PfN+6SbLPg464AgOlbpAvRzOlt3CjKvXdxfT+ULeADGIv4jFotPU6TPjGOU6UFVWZu\n3SfZ7tNenRXnVK4ozNyQAADKC6ooKVgzeJn8laG+HBkiOxYA43hQWvBDbjuSPm0EAGV5kBt/gHEM\nKi32onR/pOTzK1eQCzAW5VJatESukMtn/Y2FqpRsLz+YkyTbt4PTR8nuKzeOMxZSUjruleRBbrsF\nGLddSgspKS1Yo7S4m5LCTEr348mHTkq2GdbhbQDK9s8AEPv1Qcl2UR99ILsdBIzbQqVFnsqrinSM\nmCXYiYiIiKjCsOXUkeeNE20iIiIiqjA40SYiIiIiKgMV6dQRh+fdASIiIiKiPyPJI9qhoaEoLCw0\ne6zkgt0swU5EREREz1pFOqItOdEODw9HdHQ0vvjiC1OFHCIiIiKi58VQcebZ0hPtN954A927d8el\nS5dYtIaIiIiInjtbjmgbDAbMmjULly5dgrOzM2JiYlC/fn3T8i1btiA1NRVarRYjR46En58f8vLy\nEB4ejocPH+LFF19EXFwcqlSpoqqvsudoDxkyhJNsIiIiIioXhBBmNykHDx5EQUEB0tLSMHnyZMyb\nN8+07ObNm0hJSUFqaipWr16NxMREFBQUICkpCYGBgdi0aROaN2+OtDTpehRSWBmSiIiIiCoMywJa\nJQWzrImLi0OrVq3w4YcfAgB8fHxw7NgxAMChQ4eQkZGB2bNnAwBGjx6N4cOHY+bMmUhOTkatWrWQ\nm5uLxMREJCdLFxYqTZlf3k9pRSm5KlvrRvVTXMGsc+wK2Xb7ooYrriYmF29f1HAAyirwKa12pbRK\nmNJqeEqqQFmWNLUmpk8XxZXfJq3fKdkucUB3AFBU2evDefIf8D1Th8lWw/typLHy27DkLZLtkof1\nBgAs2SddgW9sZ2/ZyqGAsXqoXFU6wFiZTq7q2PKhvQAor7yppAJfyKL1sn1LnTAAHWZLV5w7NGMU\nAOXVVJV8zuWqIALGSojdFqySbPPN34cAULatAZRVq1VauXDAFxsl26wf/TEAoEfCGsl2X00eBEBZ\n7uWes+R55aogAsZKiMGJ0n3bNsnYNyVVKwHIjuk9U4cprr74+bfHJNuM7+IDAIo/I0r2DbO375ft\n24yeHRVvB5WOe7mqjwenj1JcrVZJ9VNAeU7lqrgmhHbDor2Zsn2bEOCLiet2yLZbGBakeGwprQyp\npFqm0vdXaeVmpeNebh8d06eLbPVIwFhBUulYKK9sOUas0+ng6upquu/o6IiioiJotVrodDq4ubmZ\nlrm4uECn05k97uLigvz8fNV95eX9iIiIiKjCMAjzmxRXV1fo9fo/1jUYoNVqrS7T6/Vwc3Mze1yv\n18Pd3V11X22eaBcUFKh+MiIiIiIiexiEwewmxcvLC5mZxm9SsrOz4enpaVrWqlUrnD17Fo8ePUJ+\nfj5+/vlneHp6wsvLCxkZxm9BMjMz0bZtW9V9LXWiffjwYfj5+cHf3x979/5xusOQIeX76wQiIiIi\n+vMSwvwmxd/fH87OzggJCUFcXBwiIyOxZs0aHDp0CLVq1UJoaCj69euHsLAwTJw4EZUqVcLIkSOx\nZ88ehISE4Ny5c+jfv7/qvpZ6jvby5cvx9ddfQwiB8ePH49GjR/joo48q1EXCiYiIiOjPxZa5qIOD\ng+nHjiUaN25s+nfv3r3Ru3dvs+U1a9bE6tWr7evk/yl1ou3k5AQPDw8AQFJSEsLCwlC7dm1oNJqn\n8sRERERERLYyVKCDvqWeOvLyyy8jLi4O9+/fh6urK5YuXYrZs2fjl19+eZb9IyIiIiIyseU62s9b\nqRPt2NhYNG3a1HQEu3bt2li/fj26dOnyzDpHRERERPS4ijTRZsEaIiIiIqowLGtmlNTIKI/KvGAN\nEREREdHTUpGOEZf5RNuyTKalkrKZSqqEyVXEAoxVseSqUwHGClVylSaje/gDUF7lbo5MvOk9/BVX\nklNa4VCuOtmMnh0BQFFVN6WVC5VWo5SrnhXX11gOte/n0pW4No8PxbID38n2baT/32Rfw6oRtr1v\nKw5mSbYb/kF7xe+b0oqlSquEyVVYmxDgCwCI331Usl144HuK31+5diP9/wZAWQUzQNnYkqsuCxgr\nzCqtwtZn0TrJdmkTwgAAvRaulWy3deJAxVUrlb4fSiuWKhnPSreDchXzAGPVvNVH/iHZ5hO/dgCU\nVVUElG0vlX4uE2Q+45MD3zM+t8IKh3LVMuf07qL4/VVSYRSA4s+vks+S0qqKSra9ADBujXROFw8y\n5lTJtkuuiidgrOSpdNwHfSZ9ZYgdUz4B8HT340rHvdKqlUrnSUo+50r3R0orvZZXBnCiTURERET0\n1FWkI9o2VYZ8+PAhK0MSERER0XNjMAizm60ePnyIsWPHol+/fhg6dCjy8vKstrt69SoCAwNN9+/e\nvYu33noLoaGhCA0Nxbp10t+QAjJHtH/77TfExcWhZs2a6Ny5M6Kjo+Hg4IBp06bBz8/PxpdFRERE\nRGQfe49ob968GZ6enhg7diz27NmDpKQkREdHm7XZsWMH1q9fjzt37pgeu3DhAgIDAzF9+nTFzyV5\nRDsqKgoDBw5EmzZtMG7cOGzduhU7duzAihXy55oSERERET1tBmF+s9XZs2fh4+MDAPD19UVW1pO/\nx6pWrRo2bNhg9lhOTg7Onz+P/v37Y9y4cfj9999ln0vyiHZRURHatTP+wOXUqVOoUaOGcSUtT+0m\nIiIiomfPliPaW7dufeIUjxo1asDNzQ0A4OLigvz8/CfWs3bmRqNGjdCyZUv87W9/wzfffIOYmBgs\nXrxY8vklj2g3bNgQ06ZNg8FgwLx58wAAycnJqFmzpvSrIiIiIiIqA8LiPym9evXC7t27zW5ubm7Q\n6/UAAL1eD3d3d0XP+/bbb+Ott94CAPj7++PChQuy60hOtGNiYuDn5wcHhz+a/eUvf0FcXJyiDhER\nERERPU0GIcxutvLy8kJGhvHSppmZmWjbtq2i9aKjo5GebrwUZ1ZWFlq0aCG7DitDEhEREVGFYXkd\ncFuv+/3gwQNERETg5s2bcHJyQkJCAmrVqoUFCxagc+fOaNWqlantO++8gxMnTgAwXiQkKioKAFCl\nShXExMTgxRdflHwuTrSJiIiIqMKwLI5UUsitPOKvGomIiIiowqhIx4g50SYiIiKiCoMTbSIiIiKi\nMqDmB5DPCyfaRERERFRhVKSJtuTl/YiIiIiISB0e0SYiIiKiCsOgpu76c/LMJtoGgwGzZs3CpUuX\n4OzsjJiYGNSvX19VrMLCQkRFReH69esoKCjAyJEj0aFDB7v6d/v2bfTo0QNffvklGjdubFesFStW\n4PDhwygsLETfvn3Rq1cv1bEKCwsxdepUXL9+HQ4ODpgzZ46q/v3www+Ij49HSkoKrl69iqlTp0Kj\n0eDVV1/FzJkzzYoS2Rrv4sWLmDNnDhwdHeHs7Iz58+fbVD308Vgldu3ahQ0bNiAtLc2mflnGu337\nNqKjo3Hv3j0UFxdjwYIFqFevnqpYFy9exMyZM+Ho6IgGDRpg7ty5it83a5/ZJk2aqM6DtXh16tRR\nlQep8aQmD9bitW7dWlUeSnudavNQXFyM6Oho/POf/4SjoyPi4uIghFCVB2ux9Hq96rFgLV7Je2Rr\nHqzFcnFxUT0WSnutavMAmG9ztVqt3dukx+MVFBTYtU2ytj+wZ5v0eDwPDw+7tkmWr9OeHAQFBZnK\nUNetWxd9+vTB3Llz4ejoCG9vb4wZM8am12kZr1u3bli0aBG0Wi1q1KiB+fPno0qVKqpilRTKW7Zs\nGS5fvoyFCxfa1bcRI0Zg5syZKCwshLOzMxITE/HCCy+oitW1a1fEx8dDq9Wiffv2mDhxok19s5wz\ntGvXTvV4sIzVsmVLu8ZCeVWRTh2BeEbS09NFRESEEEKIc+fOiREjRqiOtW3bNhETEyOEECIvL0+8\n++67dvWtoKBAjBo1SnTs2FH89NNPdsU6efKkGD58uCguLhY6nU4sXrzYrngHDhwQ48aNE0IIcfz4\ncTFmzBibYyQnJ4vAwEDRq1cvIYQQw4cPFydPnhRCCDF9+nSxf/9+u+J9/PHH4sKFC0IIITZv3ixi\nY2NVxxJCiAsXLogBAwaYPaY2XkREhNizZ48QQoisrCxx5MgR1bFGjRoljh49KoQQYtKkSeLQoUOK\nY1n7zNqTB2vx1OahtPGkNg/W4qnNg7VY9uThwIEDYurUqUII41gdMWKE6jxYi2XPWLAWTwh1ebAW\ny56xYC2ePXmw3Obau02yjGdPHqztD+zZJlnGsycPlrHsycHDhw9F9+7dzR7r1q2buHr1qjAYDGLI\nkCEiJyfHrngdO3YUN2/eFEIIER8fL9atW6c6lhBCHD16VISEhIgJEyYo7ldp8UJDQ8W5c+eEEELs\n27dPfP/996pjde/eXVy5ckUYDAYREhIicnNzFffN2pxB7XiwFsuesVCedYxZZnaz1YMHD8SYMWNE\n3759xZAhQ8Tt27efaJOYmCiCg4NFr169xA8//CCEEOL27dti0KBBom/fvmL8+PHi/v37ss/1zM7R\nPnv2LHx8fAAArVu3Rk5OjupYnTt3xvjx4033HR0d7erb/PnzERISIlvdR4njx4/D09MTo0ePxogR\nI/Dee+/ZFa9hw4YoLi6GwWCATqeDVmv7lxD16tXDkiVLTPfPnz+Pdu3aAQB8fX3x3Xff2RUvMTER\nzZo1A2A88lWpUiXVse7cuYP4+HhT5SVbWcb7/vvvcePGDQwcOBC7du0yvW41sZo1a4a7d+9CCAG9\nXm9TLqx9Zu3Jg7V4avNgLZY9ebAWT20erMWyJw8ffPAB5syZAwD497//jZo1a6rOg7VY9owFa/HU\n5sFaLHvGgrV49uTBcptr7zbJMp49ebCMZe82yTKePXmwjGVPDnJzc/HgwQMMHjwYAwYMwOnTp1FQ\nUIB69epBo9HA29sbWVlZquNlZ2cjJSXFdPS0qKhIcR6sxbp69SrS0tIwduxYxX0qLd65c+eQl5eH\nI0eOIDQ0FNnZ2WaVAG3tW0keCgsL8ejRI5vmJNbmDGrHg7VY9oyFP7PNmzfD09MTmzZtQlBQEJKS\nksyWX7hwAdnZ2diyZQsSExMRHR0NAEhKSkJgYCA2bdqE5s2bK/qG65lNtHU6HVxdXU33HR0dUVRU\npCqWi4sLXF1dodPpMG7cOEyYMEF1v7766itUr17d9EeAve7cuYOcnBx8/vnn+PTTTxEeHm7X9R6r\nVq2K69evo0uXLpg+fTpCQ0NtjtGpUyezDbAQAhqNBoDxvczPz7cr3uM7kA0bNmDgwIGqYhUXF2Pa\ntGmIioqCi4uLTX0qrW/Xr1+Hu7s71q5di9q1a2PlypWqY5V8NdulSxfcvn0bb731luJY1j6z9uTB\nWjy1ebCMNX78eLvyYK1vavNgLZY9eQAArVaLiIgIzJkzB506dbIrD5ax7BkLlvE6duxoVx4s+2bP\nWLAWT20erG1z7cmBtXhq82AZy2Aw2JUDa31TmwdrsewZC5UrV8Ynn3yC1atX49NPP0VkZKTZaR22\n5sEyXnh4OKpXrw4AOHDgAE6dOoWgoCDVsWbMmIHZs2erOrBmGW/y5Mm4cuUK2rdvj/Xr1+N///sf\nvv76a9V9a9y4MUaMGIGAgADUrl0bjRo1Uty30uYMasaDtVi1atUCoH6bVF4ZhDC72erxg7++vr5P\n/FHZvHlzrF69GhqNxnRwwdp6Sv4IemYTbVdXV+j1etN9g8Gg6uhsif/85z8YMGAAunfvjq5du6qO\ns337dnz33XcIDQ3FxYsXERERgZs3b6qO5+HhAW9vbzg7O6NRo0aoVKkS8vLyVMdbu3YtvL29kZ6e\njp07d2Lq1Kl49OiR6ngAzM710uv1cHd3tyseAOzduxczZ85EcnKyaeNqq/Pnz+Pq1auYNWsWJk2a\nhJ9++glz5861q18eHh54//33AQDvv/++Xd+kzJ07Fxs3bsS+ffsQFBSEefPm2bS+5WfW3jxYGwNq\n8/B4rAYNGtidB8u+2ZMHy1j25gEwHhlMT0/H9OnTzcaTmjw8Huv+/ft2j4WSeKNHj0Zubq5deXi8\nb25ubnaPhcfjxcTEqMqDtW3u49tIW3NQ2jZcTR4sY3Xt2hWXL19WnQNrfXNwcFCVB2uxpk6dqnos\nNGzYEN26dYNGo0HDhg3h5uaGu3fvmpbbmgfLeB4eHrh58ybWrl2L1atXY9WqVYqPplrGcnR0xPXr\n1zFx4kTExsbi5MmTSE5OVt23knOx3377bWg0Gvj5+SnOg7W+ffbZZ9izZw8OHjyI+vXr48svv1Tc\nN2tzhscn1rbkobT5x9PYP5c3Qgizm5StW7ciMDDQ7Jafn286z760P2a0Wi0WLlyI4cOHIzAwEIDx\noLHcepae2UTby8sLmZmZAICrYO19AAAERElEQVTs7Gx4enqqjnXr1i0MHjwYU6ZMQXBwsF392rhx\nIzZs2ICUlBQ0a9YM8+fPN/0FqEbbtm1x7NgxCCFw48YNPHjwAB4eHqrjubu7m5JarVo1FBUVobi4\nWHU8wPiX2qlTpwAAmZmZePPNN+2Kt3PnTtN7+Morr6iO06pVK+zZswcpKSlITExEkyZNMG3aNLv6\n1rZtW2RkZAAATp8+jSZNmqiOVa1aNdO3Mi+++CLu3buneF1rn1l78mAtnto8WMayNw/W+qY2D9Zi\n2ZOHHTt2YMWKFQCAKlWqQKPRoGXLlqryYC3WgQMHVI8Fy3g1a9bEt99+qyoP1vrWrl071WPBWjwP\nDw9VebC2zfX19VU9FqzF++6771TlwTJWyeRJ7Viw1jc/Pz9VebAWq27duqrHwrZt20wT85J9VdWq\nVXHt2jUIIXD8+HGb8mAZT6fTYfv27Thz5gzWrl1r0wTPMpZGo8G+ffuQkpKCqKgovP322xg2bJjq\neHq9Hi1atMCZM2cAGPPw6quvqopVWFiIunXromrVqgBsz4O1OUP79u1VjQdrsTIzM5/K/rm8EcL8\nJqVXr17YvXu32c3Nzc108Ffqj5mJEyfi2LFjWL16Na5du2Z20FjpH0HP7Koj/v7+OHHiBEJCQiCE\nQGxsrOpYy5cvx71795CUlGQ6r2blypWoXLny0+quan5+fjh9+jSCg4MhhMCMGTPsOod84MCBiIqK\nQr9+/VBYWIiJEyeaBrRaERERmD59OhITE9GoUSN06tRJdazi4mLMnTsXtWvXNp0799e//hXjxo2z\nq49PS0REBKKjo5GamgpXV1ckJCSojhUTE4OJEydCq9XCycnJdM6qEtY+s9OmTUNMTIyqPFjGKy4u\nxpUrV1CnTh2b8/C0x5O1ePPmzVOVB2ux7MlDx44dERkZiY8//hhFRUWIiopC48aNVY0Ha7GioqJU\njwVr8dSeT2ktVrNmzVSPBWvxPDw8VOfB0tPcJhkMBm6TFAgODkZkZCT69u0LjUaD2NhYODg4IDw8\nHMXFxfD29sYbb7yhOt7s2bMxePBgNG/eHEOHDgUAdOnSBf369VPVN3u+AbcWr2rVqvj0009RXFyM\nunXrIjw8XFWs+Ph43L59G4MHD0alSpXg5uZm0zcL1uYMdevWVTUerMWaPHlyuR0L9rD3qiNeXl7I\nyMhAq1atkJmZibZt25otz8rKwv79+zFz5kxUqlQJWq0WGo3GtF6PHj2srmeNRthzAjERERERUQXy\n4MED02lmTk5OSEhIQK1atbBgwQJ07twZLVq0wOzZs3Hp0iUYDAYEBwejd+/euHXrFiIiIqDX6/HC\nCy8gISFB9uAnJ9pERERERGWAJdiJiIiIiMoAJ9pERERERGWAE20iIiIiojLAiTYRERERURngRJuI\niIiIqAxwok1EREREVAY40SYiIiIiKgOcaBMRERERlYH/B0L+ckgQkQqfAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x23279b6c748>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAtoAAABXCAYAAAA6X+IcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3Xl8jOf6P/DPJJNYskgJpxy1N9Yq\n0aPVRio0tkakaRBqBCX2qO1EFhURIkS02gYptURIbI2iB7ElaDio9NSSVKtF9VQRjsxYssz9/WN+\nmZoxZp55Rkj6+7y95vUyz9xzzT1zzf3MnXueeS6FEEKAiIiIiIieKLtn3QEiIiIior8iTrSJiIiI\niCoAJ9pERERERBWAE20iIiIiogrAiTYRERERUQXgRJuIiIiIqAJwok1EREREVAE40SYiqmL8/f2x\nd+9e/fX8/Hy0bNkSmZmZ+m0XLlxAx44dcefOnWfRRSIiAifaRFQJtGzZEv369UP//v0NLr/++utj\n7/P9998jLCzsifVh9+7dUKlUVt9PCIHw8HCsWrXKYPv27dvh7++P/v37Izg4GN9//73+tkOHDqFf\nv37o1asXwsLCoFarzW435urqiqKiIv31tWvXwsXFxWBbamoqAgIC4OrqavVzIiKiJ0P5rDtARATo\nJou1a9eW3P6ll17C0qVLK7BHlv3000+YM2cO/vOf/8DDw0O//eLFi1i0aBG2bduGevXqITs7G5Mm\nTcKhQ4dQWFiIiIgIbNy4EU2aNMGiRYuQmJiIsLAwk9tjYmIeeVxnZ2f9JLywsBAHDx5EYGCgfqJ9\n584d7NixA1u2bHkqrwMR0dO08sBxg+ujur/62LZarRYxMTEoKCiAo6Mj4uLi0LhxY/3taWlp2LZt\nGxQKBSZMmAAfHx/cv38fM2bMwM2bN+Hk5ISEhASrPp8exok2EVVqx48fR2JiIho0aICLFy+ievXq\nWLBgAW7cuIG5c+di586d0Gg0iIiIwKVLl2BnZ4e2bdsiNjYWdnZ2yMjIQGpqKuzs7ODu7o5Zs2ah\nadOmAICPP/4YO3bsgJubm8GO98CBA1i2bBlKSkpQvXp1hIeHo2PHjo/0LS0tDQMGDECDBg0Mtpfv\nzOvVqwcAaNeuHW7cuIHi4mIcOXIEL730Epo0aQIAGDx4MPr37w9PT0+T22fPng2FQmEQ38XFRT/R\nTk9PR9++fdGgQQP88ccfAICtW7fC09MTzZs3tz0BRESVjFYIyW337duH4uJiZGRkIC8vDwsWLMCy\nZcsA6BYqNmzYgMzMTDx48ABvv/02unXrho0bN8LDwwOTJk3Crl27kJycjOjoaFl95aEjRFQphISE\nGBw2MmHCBP1tZ86cgUqlwo4dOxAYGIgZM2YY3DcrKwsajQbbt2/Xr+JeuXIFubm5WLlyJdatW4ev\nvvoKfn5+mDBhAoQQ2LdvH/bu3YvMzEykp6frJ66//PILlixZgpSUFGRmZmLu3LmYNGkS7t69+0if\nP/zwQ/Tr1++R7Q0bNkS3bt0A6A4tiY+PR/fu3eHo6Ijff/8dzz//vL7t888/D7VajcuXL5vcrtFo\nHolfPtEuKSlBeno6hg4dCicnJ6jVami1WqSlpWHYsGEAgKKiIgQFBaFjx4744YcfpKaDiKjSEkIY\nXMw5deoUunbtCgDo0KEDzpw5o7+tdu3a2L59OxwcHHDjxg24urpCoVAY3Mfb2xu5ubmy+8oVbSKq\nFMwdOtKqVSu88sorAIB3330XsbGxuH37tv72Tp06YcmSJVCpVHj99dcREhKCxo0bIyMjA3379tXH\nDQwMxLx58/Drr78iNzcXvr6+cHZ21sdNTU3F0aNH8ccff2D48OH6+AqFApcvX0arVq2sek53797F\nzJkz8fvvv2PlypUAdF9jGq9QA4C9vb3J7XZ2j66HuLi44NatW9izZw9atWqFZs2aoaCgAEVFRTh0\n6BDs7e3h7e0NAKhevTpSUlKwcOFCq/pORFRZWbOirVar9ft5QLevLS0thVKpmwIrlUqsX78en3zy\nif53Omq1Gi4uLgAAJycng9+/WIsr2kRU6dnb25vd9sILLyArKwuhoaFQq9UYMWIEDhw4AK1W+8j9\nhBAoLS3V/984nlarRZcuXbB9+3b9ZdOmTXjxxRet6vNvv/2G4OBg2NvbY926dfofJdavX19/iAcA\nXLt2DbVq1TI49OPh7TVr1nwkdvkx2uvWrdOvXJdvW79+PVQqlX7S7uDgIPvYQiKiykgrhMHFHGdn\nZ4NvBrVarX6SXW7o0KE4fPgwTpw4gWPHjhncR6PR2PSjck60iajSy8/PR35+PgAgIyMDHTt21K82\nAMCGDRsQEREBLy8vzJgxA15eXjh37hy6du2Kr7/+GoWFhQB0xy6XH4/t7e2N3bt3486dO9Bqtdi+\nfTsAoEuXLjh69Ch++uknAEB2djb8/f1x//59yf1Vq9VQqVTo2bMnlixZgurVq+tv8/LywnfffYdf\nfvkFgO4Y6x49ejx2uykuLi44efIk1Go1vLy8AOg+TM6fP4/vv/8e77zzjuS+EhFVNdYcOuLp6Ymc\nnBwAQF5e3iM/XJ84cSKEEHBwcICjoyPs7Ozg6emJ7OxsAEBOTg46deoku688dISIKoWQkJBHDpOY\nOnUqqlevDnd3d3z00Ue4evUqateujYULF+Lq1av6dgEBAfj3v/+Nvn37okaNGqhfvz5UKhVq1aqF\n4cOHIyQkBFqtFrVr18aKFStgZ2eHN998EwUFBXj33Xfh6uqKVq1a4datW2jRogViY2MxdepUCCGg\nVCqxbNkyODk5SX4uaWlp+O2335CVlYWsrCz99jVr1qBOnTqIj49HWFgYSkpK0KhRIyQkJMDNzc3k\ndlNcXFxw7do1jB8/Xr/N2dkZ169fR0hIiFV9JSKqaqw4cgS+vr44evQogoODIYTA/PnzsXr1ajRq\n1Ag9evRAq1atMGjQICgUCnTt2hWdO3fGSy+9hPDwcAwePBgODg5YvHix7L4qhKU/BYiInqHjx4/r\nzy5C8s2cORMjR440WM0hIqqKPv7XYYPrk/t0fUY9sYwr2kREf3GjR4/G+fPn8fPPP2PQoEEIDAx8\n1l0iIpKtKq0Rc0WbiIiIiKqMJbuyDa5PefvNZ9QTy7iiTURERERVhjWn93vWONEmIiIioiqj6kyz\nOdEmIiIioiqEK9pERERERBWgKv28kBNtIiIiIqoytFpOtPXeSfzC7O1fTh8JAPhs71Gz7Sb0fAO+\nccssPl5W9DjMWP+VxXaLhvpjytpMs22WhAQAAHrPX2G23e7IMQCA8A3mz/ObMMQPoSmbLPYtJXQg\nEr46YLZNuH93XR+NfnlrrPyXuJYeNyV0ICI27rLYt/jBb+Ojr3PMtvmgrzcAICR5g9l2a8cPAQAE\nf7TObLv0D4Zh7OebLfZt+egBiNuWZbZNdKAvAGDWpn+ZbTd3YB8AwNR12822SxrWH+8tTbXYt7Qw\nFQIWrbLYLnPG+xbfv4uG+gOA5Pev/8KVZtt99c9RFscpoBurlsZgVvQ4q/oWnWE+D3GD+mDU8nSL\nfVs5Nljy+zJw8Wqz7bZNGyG53biVWyz2bdmoIAz+2Px7ZONkFQDgk91HzLab1NtLcjupOY1K/9pi\nu3nBfTHXwtia9f/GltR9l5TPBkvvXUD3/rU0BtPCdK9v4s5DZttN9+sGABbfcyvHBmP0igyLfft8\nzCCErd5mts3SEbpTLU78wny7T0fq2r29IMVsu10zQxGzZY/FvsUE9ZK0bwCAAUvWmG23ecpwAEBQ\nkvkxs2XqCMmfz5beR4DuvbRwx0Gzbf7ZzwcAMHnNl2bbfTz8HcntLLUpb+eX8LnZNjvDR+vaGp0T\n2lj5OaKlfL5J/TySuo/+K9BqtYiJiUFBQQEcHR0RFxeHxo0bG7QpLCxEcHAwduzYgWrVqkEIAW9v\nbzRp0gQA0KFDB0ybNk3W40ueaGu12keqthERERERPU3WHDqyb98+FBcXIyMjA3l5eViwYAGWLftz\n0ejw4cNYvHgxbty4od92+fJltG3bFsuXL7e5r2ZnzleuXMH48ePh7e2Nt956C926dUNoaCh+/vln\nmx+YiIiIiMhaWiEMLuacOnUKXbvqvhXo0KEDzpw5Y3C7nZ0dVq9eDTc3N/22s2fP4tq1a1CpVBg9\nejQuXrwou69mV7SjoqIwbdo0vPzyy/pteXl5iIiIQHq65a9ziYiIiIieJGuO0Far1XB2dtZft7e3\nR2lpKZRK3RT4jTfeeOQ+devWRWhoKPr06YOTJ09ixowZ2Lp1q6y+mp1oFxcXG0yyAd1fA0RERERE\nz4I1h444OztDo9Hor2u1Wv0k+3HatWsHe3t7AMArr7yCa9euQQgBhUJhdV/NPlLLli0RERGBrl27\nwsXFBRqNBtnZ2WjZsqXVD0REREREZCtrzqPt6emJgwcPom/fvsjLy4OHh4fF+3z66adwc3PD6NGj\nkZ+fjwYNGsiaZAMWJtoxMTHYt28fTp06pV969/Hxga+vr6wHIyIiIiKyhTUr2r6+vjh69CiCg4Mh\nhMD8+fOxevVqNGrUCD169DB5n9DQUMyYMQPZ2dmwt7dHfHy87L6anWgrFAr4+vpyYk1ERERElYI1\nK9p2dnaIjY012Na8efNH2h048OcpJWvVqoWUFPOn0pRKIapSeR0iIiIi+v+acQ2GuEF9nlFPLGNl\nSCIiIiKqMqrSGnGFT7RHLjN/GsAvxgUDkFYlzFKFRkBXpVFqZaQesclm2+z/cDwAWKz+tmxUEABg\nsYWqY9P8uj3xCoeWKoDFBPUCIK26Za95lk/MvidqLEYkbzTbZvX4wQCAf6btMNtu4Xv9AEircmep\nsh6gq6436KO1ZttkfBACQFqFLUBaZUhLVSYBXaVJqdXapFYRlFpJTkqVO585n1ns28HZEyxWEZwX\n3BcAJLeTUhnSUg4AXR6e5DgFgNite822+/DdnpJzKvX1mP/lPrPtIt95CwAkVcOTWlHT0mOWP+60\nVPMV/RardBVLpVQuBCCp0qSlqoqArrKipSqNn48ZBACS20l5X1qqfAvoqt9KfT2kVoSVUkWwW8yn\nFvt2KGaipEqOgPR9+fhV5sdg8vtBkqsqPumKpVIrx0oZ91Irwn66x/xn28ReukqvUt8j8Zn7zbaL\nCOghecxI/ZyprKw5dORZ44o2EREREVUZnGgTEREREVUAHjpCRERERFQB/jITbZVKhZKSEoNt5ZVx\nWIKdiIiIiJ42bdWZZ5ufaE+fPh3R0dH47LPP9KUoiYiIiIielb/MivbLL7+M/v37o6CggEVriIiI\niOiZs2airdVqERMTg4KCAjg6OiIuLg6NGzfW375p0yakp6dDqVRi3Lhx8PHxQWFhIaZPn4779++j\nXr16iI+PR40aNWT11c5Sg1GjRnGSTURERESVglYIg4s5+/btQ3FxMTIyMjBt2jQsWLBAf9v169eR\nmpqK9PR0rFq1CklJSSguLkZycjL8/PywYcMGtGnTBhkZ5k8Nag4rQxIRERFRlWF8vvClIwIf2zY+\nPh7t27fH22+/DQDo2rUrDh/WnY9+//79yM7O1pdonzBhAsaMGYPZs2cjJSUFdevWRX5+PpKSkmSX\nZK/ws46Eb9hp9vaEIX4AYPHk/2vHD5FcWKFvvOUX4+uIUMkFP6QUewGkFUOQehJ+qQUCpD6HYZ+l\nmW23bsJ7Fos0ALpCDZYK2+yJGgtAWrEXAEi0UEBkul83iyf0B3Qn9ZdaICk0ZZPZdimhAwFIKw4h\ntQiR1AIiluItHz0AABCxcZfZdvGDdTuVhK8OmG0X7t/dYqEfQFfsR2qRJymFKwBpRTosFX0AdIUf\n/BeuNNvmq3+OAiBtXwMAfgmfm223M3y05OIQUsYfAMkFRKQUwJFaUGX25t0W280Z0BuBi833bds0\nXd+kFumQ8vpKfQ5SH1Pqe0TKftVSYRNAV9xE6n5QasElKftfqZ8zUoqzANJzKqWokaXCMYCueMyU\ntZkW2y0JCZBcQE1qoSpLBe/SwlQW5zWAbm4j9fWVOrakFFKSOk8KWLTKbJvMGe9bjPMsWfNjSLVa\nDWdnZ/11e3t7lJaWQqlUQq1Ww8XFRX+bk5MT1Gq1wXYnJycUFRXJ7qvFQ0eIiIiIiCoLrdAaXMxx\ndnaGRqP5875aLZRKpcnbNBoNXFxcDLZrNBq4urrK7qvVE+3i4mLZD0ZERERE9LR4enoiJ0f3TUpe\nXh48PDz0t7Vv3x6nTp3CgwcPUFRUhJ9++gkeHh7w9PREdnY2ACAnJwedOnWS/fiPnWgfOHAAPj4+\n8PX1xddf//k11KhRo2Q/GBERERGRLYQwvJjj6+sLR0dHBAcHIz4+HhEREVi9ejX279+PunXrQqVS\nYciQIQgJCcGUKVNQrVo1jBs3Drt27UJwcDBOnz6NoUOHyu7rY4/RXr58Ob788ksIITB58mQ8ePAA\n77zzTpU6dyERERER/bVYMxe1s7PT/9ixXPPmzfX/HzhwIAYOHGhwu7u7O1atMn8cu1SPnWg7ODjA\nzc0NAJCcnIyQkBDUr18fCoXiiTwwEREREZG1LJ3SrzJ57KEjf//73xEfH4+7d+/C2dkZn376KWJj\nY3Hx4sWn2T8iIiIiIj0hhMGlMnvsRHv+/Plo2bKlfgW7fv36WLduHfr06fPUOkdERERE9LCqNNF+\n7KEjSqUSgYGGJwB3d3dHVFRUhXeKiIiIiMgUa86j/ayxMiQRERERVRnGhYrKCxNVRhVeGVJqVSzf\nuGVm22VFj7NYSQzQVROzVJ0K0FWoitmyx2ybmKBeAKRX7JJSBUpqJTmplfXmbssy225WoC8AWKz8\nl/FBiOTKhVL7JqWKFQCLj7tybLDFCo2ArkqjlFiA9NdNSmVIqdXrLFWlA3SV6SxVwdw1MxQA8Nne\no2bbTej5BgBpz+GT3ZarL07q7SUpFiBtLADSKnRaqoAJ6KpgSs2p1Ipo7yR+Ybbdl9NHYuWB4xb7\nNqr7q5LHgqVqeEtCAnQxJbzPLe17Ad3+11LFPEBXNW9Z1jdm24zzfR0AJFc2lVIRVmpV0JT9x8y2\nCe3xGgBYfJ9P6q2rcCjls8FS9V5AV8FXajVKqWNGSqVXqZUhpVbItfS5VV7+WsrnjNTKkFKrLw7+\n2Hwlx42TVQCkVwO2VFkx8p23JO8vn/TrK6W6paVKmYBuUir1daustKg6a8QVPtEmIiIiInpSqtLB\nGFZVhrx//z4rQxIRERHRM6PVCoOLte7fv49JkyZhyJAhGD16NAoLC022u3TpEvz8/PTXb9++jVdf\nfRUqlQoqlQpr15r/FgewsKJ95coVxMfHw93dHb1790Z0dDTs7OwQFRUFHx8fK58WEREREZFtbF3R\n3rhxIzw8PDBp0iTs2rULycnJiI6ONmiTmZmJdevW4datW/pt586dg5+fH2bNmiX5scyuaEdGRmL4\n8OHo2LEjwsLCsHnzZmRmZmLFihVWPiUiIiIiIttpheHFWqdOnULXrrrfFHl7eyM3N/eRNrVq1cL6\n9esNtp05cwZnz57F0KFDERYWhj/++MPiY5ld0S4tLUXnzp0BAMePH0edOnV0d1Ly0G4iIiIievqs\nWdHevHnzI4d41KlTBy4uLgAAJycnFBUVPXI/U0duNGvWDO3atcPrr7+Or776CnFxcVi6dKnZxze7\not20aVNERUVBq9ViwYIFAICUlBS4u7ubf1ZERERERBVAGP0zZ8CAAdi5c6fBxcXFBRqNBgCg0Wjg\n6uoq6XFfe+01vPrqqwAAX19fnDt3zuJ9zE604+Li4OPjAzu7P5v97W9/Q3x8vKQOERERERE9SVoh\nDC7W8vT0RHZ2NgAgJycHnTp1knS/6Oho7NmjO/1nbm4u2rZta/E+LFhDRERERFVGUNJqg+tbpo6w\n6v737t1DeHg4rl+/DgcHByxevBh169bFwoUL0bt3b7Rv317f9o033sDRo7q6FVeuXEFkZCQAoEaN\nGoiLi0O9evXMPhYn2kRERERUZQQuNpxob5tm3UT7aeKvGomIiIioyqhKa8ScaBMRERFRlcGJNhER\nERFRBZDzA8hnhRNtIiIiIqoyqtJE2+zp/YiIiIiISB6uaBMRERFRlcFjtE3QarWIiYlBQUEBHB0d\nERcXh8aNG8uKVVJSgsjISFy9ehXFxcUYN24cevToYVP/bt68icDAQHzxxRdo3ry5TbFWrFiBAwcO\noKSkBIMHD8aAAQNkxyopKcHMmTNx9epV2NnZYe7cubL699133yExMRGpqam4dOkSZs6cCYVCgRdf\nfBGzZ882KEpkbbzz589j7ty5sLe3h6OjIxISEqyqHvpwrHI7duzA+vXrkZGRYVW/jOPdvHkT0dHR\nuHPnDsrKyrBw4UI0atRIVqzz589j9uzZsLe3R5MmTTBv3jzJr5up92yLFi1k58FUvAYNGsjKg7nx\nJCcPpuJ16NBBVh4e9zzl5qGsrAzR0dH4+eefYW9vj/j4eAghZOXBVCyNRiN7LJiKV/4aWZsHU7Gc\nnJxkj4XHPVe5eQAM97lKpdLmfdLD8YqLi23aJ5n6PLBln/RwPDc3N5v2ScbP05YcBAQE6MtQN2zY\nEIMGDcK8efNgb28PLy8vTJw40arnaRzP398fH330EZRKJerUqYOEhATUqFFDVqzyQnnLli3DDz/8\ngCVLltjUt7Fjx2L27NkoKSmBo6MjkpKS8Nxzz8mK1a9fPyQmJkKpVKJLly6YMmWKVX0znjN07txZ\n9ngwjtWuXTubxkJlVYXm2YB4Svbs2SPCw8OFEEKcPn1ajB07VnasLVu2iLi4OCGEEIWFheLNN9+0\nqW/FxcVi/PjxomfPnuLHH3+0KdaxY8fEmDFjRFlZmVCr1WLp0qU2xcvKyhJhYWFCCCGOHDkiJk6c\naHWMlJQU4efnJwYMGCCEEGLMmDHi2LFjQgghZs2aJfbu3WtTvPfee0+cO3dOCCHExo0bxfz582XH\nEkKIc+fOiWHDhhlskxsvPDxc7Nq1SwghRG5urjh48KDsWOPHjxeHDh0SQggxdepUsX//fsmxTL1n\nbcmDqXhy8/C48SQ3D6biyc2DqVi25CErK0vMnDlTCKEbq2PHjpWdB1OxbBkLpuIJIS8PpmLZMhZM\nxbMlD8b7XFv3ScbxbMmDqc8DW/ZJxvFsyYNxLFtycP/+fdG/f3+Dbf7+/uLSpUtCq9WKUaNGiTNn\nztgUr2fPnuL69etCCCESExPF2rVrZccSQohDhw6J4OBg8cEHH0ju1+PiqVQqcfr0aSGEELt37xbf\nfvut7Fj9+/cXFy5cEFqtVgQHB4v8/HzJfTM1Z5A7HkzFsmUsVGa95i03uFjr3r17YuLEiWLw4MFi\n1KhR4ubNm4+0SUpKEkFBQWLAgAHiu+++E0IIcfPmTTFixAgxePBgMXnyZHH37l2Lj/XUjtE+deoU\nunbtCgDo0KEDzpw5IztW7969MXnyZP11e3t7m/qWkJCA4OBgi9V9pDhy5Ag8PDwwYcIEjB07Ft26\ndbMpXtOmTVFWVgatVgu1Wg2l0vovIRo1aoRPPvlEf/3s2bPo3LkzAMDb2xvffPONTfGSkpLQunVr\nALqVr2rVqsmOdevWLSQmJuorL1nLON63336La9euYfjw4dixY4f+ecuJ1bp1a9y+fRtCCGg0Gqty\nYeo9a0seTMWTmwdTsWzJg6l4cvNgKpYteXjrrbcwd+5cAMBvv/0Gd3d32XkwFcuWsWAqntw8mIpl\ny1gwFc+WPBjvc23dJxnHsyUPxrFs3ScZx7MlD8axbMlBfn4+7t27h5EjR2LYsGE4ceIEiouL0ahR\nIygUCnh5eSE3N1d2vLy8PKSmpupXT0tLSyXnwVSsS5cuISMjA5MmTZLcp8fFO336NAoLC3Hw4EGo\nVCrk5eUZVAK0tm/leSgpKcGDBw+smpOYmjPIHQ+mYtkyFiozIYTBxVobN26Eh4cHNmzYgICAACQn\nJxvcfu7cOeTl5WHTpk1ISkpCdHQ0ACA5ORl+fn7YsGED2rRpI+kbrqc20Var1XB2dtZft7e3R2lp\nqaxYTk5OcHZ2hlqtRlhYGD744APZ/dq2bRtq166t/yPAVrdu3cKZM2fw8ccfY86cOZg+fbpNxxLV\nrFkTV69eRZ8+fTBr1iyoVCqrY/Tq1ctgByyEgEKhAKB7LYuKimyK9/AHyPr16zF8+HBZscrKyhAV\nFYXIyEg4OTlZ1afH9e3q1atwdXXFmjVrUL9+fXz++eeyY5V/NdunTx/cvHkTr776quRYpt6ztuTB\nVDy5eTCONXnyZJvyYKpvcvNgKpYteQAApVKJ8PBwzJ07F7169bIpD8axbBkLxvF69uxpUx6M+2bL\nWDAVT24eTO1zbcmBqXhy82AcS6vV2pQDU32TmwdTsWwZC9WrV8f777+PVatWYc6cOYiIiDA4rMPa\nPBjHmz59OmrXrg0AyMrKwvHjxxEQECA71ocffojY2FhZC2vG8aZNm4YLFy6gS5cuWLduHf73v//h\nyy+/lN235s2bY+zYsejbty/q16+PZs2aSe7b4+YMcsaDqVh169YFIH+fVFlphTC4WOvhxV9vb+9H\n/qhs06YNVq1aBYVCoV9cMHU/KX8EPbWJtrOzMzQajf66VquVtTpb7r///S+GDRuG/v37o1+/frLj\nbN26Fd988w1UKhXOnz+P8PBwXL9+XXY8Nzc3eHl5wdHREc2aNUO1atVQWFgoO96aNWvg5eWFPXv2\nYPv27Zg5cyYePHggOx4Ag2O9NBoNXF1dbYoHAF9//TVmz56NlJQU/c7VWmfPnsWlS5cQExODqVOn\n4scff8S8efNs6pebmxu6d+8OAOjevbtN36TMmzcPaWlp2L17NwICArBgwQKr7m/8nrU1D6bGgNw8\nPByrSZMmNufBuG+25ME4lq15AHQrg3v27MGsWbMMxpOcPDwc6+7duzaPhfJ4EyZMQH5+vk15eLhv\nLi4uNo+Fh+PFxcXJyoOpfe7D+0hrc/C4fbicPBjH6tevH3744QfZOTDVNzs7O1l5MBVr5syZssdC\n06ZN4e/vD4VCgaZNm8LFxQW3b9/W325tHozjubm54fr161izZg1WrVqFlStXSl5NNY5lb2+Pq1ev\nYsqUKZg/fz6OHTuGlJQU2X0OC9i6AAAEvklEQVQrPxb7tddeg0KhgI+Pj+Q8mOrbokWLsGvXLuzb\ntw+NGzfGF198IblvpuYMD0+srcnD4+YfT+LzubKxZkV78+bN8PPzM7gUFRXpj7N/3B8zSqUSS5Ys\nwZgxY+Dn5wdAt2hs6X7GntpE29PTEzk5OQCAvLw8eHh4yI5148YNjBw5EjNmzEBQUJBN/UpLS8P6\n9euRmpqK1q1bIyEhQf8XoBydOnXC4cOHIYTAtWvXcO/ePbi5ucmO5+rqqk9qrVq1UFpairKyMtnx\nAN1fasePHwcA5OTk4JVXXrEp3vbt2/Wv4QsvvCA7Tvv27bFr1y6kpqYiKSkJLVq0QFRUlE1969Sp\nE7KzswEAJ06cQIsWLWTHqlWrlv5bmXr16uHOnTuS72vqPWtLHkzFk5sH41i25sFU3+TmwVQsW/KQ\nmZmJFStWAABq1KgBhUKBdu3aycqDqVhZWVmyx4JxPHd3d/zrX/+SlQdTfevcubPssWAqnpubm6w8\nmNrnent7yx4LpuJ98803svJgHKt88iR3LJjqm4+Pj6w8mIrVsGFD2WNhy5Yt+ol5+WdVzZo1cfny\nZQghcOTIEavyYBxPrVZj69atOHnyJNasWWPVBM84lkKhwO7du5GamorIyEi89tprCA0NlR1Po9Gg\nbdu2OHnyJABdHl588UVZsUpKStCwYUPUrFkTgPV5MDVn6NKli6zxYCpWTk7OE/l8rmyEMLyYM2DA\nAOzcudPg4uLiol/8NffHzJQpU3D48GGsWrUKly9fNlg0lvpH0FM764ivry+OHj2K4OBgCCEwf/58\n2bGWL1+OO3fuIDk5WX9czeeff47q1as/qe7K5uPjgxMnTiAoKAhCCHz44Yc2HUM+fPhwREZGYsiQ\nISgpKcGUKVP0A1qu8PBwzJo1C0lJSWjWrBl69eolO1ZZWRnmzZuH+vXr64+d+8c//oGwsDCb+vik\nhIeHIzo6Gunp6XB2dsbixYtlx4qLi8OUKVOgVCrh4OCgP2ZVClPv2aioKMTFxcnKg3G8srIyXLhw\nAQ0aNLA6D096PJmKt2DBAll5MBXLljz07NkTEREReO+991BaWorIyEg0b95c1ngwFSsyMlL2WDAV\nT+7xlKZitW7dWvZYMBXPzc1Ndh6MPcl9klar5T5JgqCgIERERGDw4MFQKBSYP38+7OzsMH36dJSV\nlcHLywsvv/yy7HixsbEYOXIk2rRpg9GjRwMA+vTpgyFDhsjqmy3fgJuKV7NmTcyZMwdlZWVo2LAh\npk+fLitWYmIibt68iZEjR6JatWpwcXGx6psFU3OGhg0byhoPpmJNmzat0o4FW9hasMbT0xPZ2dlo\n3749cnJy0KlTJ4Pbc3NzsXfvXsyePRvVqlWDUqmEQqHQ3y8wMNDk/UxRCFsOICYiIiIieoq6xXxq\ncP1QjHWnorx3757+MDMHBwcsXrwYdevWxcKFC9G7d2+0bdsWsbGxKCgogFarRVBQEAYOHIgbN24g\nPDwcGo0Gzz33HBYvXmxx8ZMTbSIiIiKiCsAS7EREREREFYATbSIiIiKiCsCJNhERERFRBeBEm4iI\niIioAnCiTURERERUATjRJiIiIiKqAJxoExERERFVAE60iYiIiIgqwP8B4bxivpemLwsAAAAASUVO\nRK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x23279da7cc0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAtoAAABWCAYAAADxAzG5AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3XtclGXe+PHPAOIBUFLXzdZTWlib\nlWHrkxtQpKSyKIiSeEDNs+IhD6yiqCgIYh7b0sQ8ogGmKXlIkzyf2HRlixSzMmvpecykVodCkLl+\nf/BjHgbHmXvuSYV9vu9e83o191z3dy7mO9c1l/fcc38NSimFEEIIIYQQ4jflcr87IIQQQgghxH8i\nWWgLIYQQQghxF8hCWwghhBBCiLtAFtpCCCGEEELcBbLQFkIIIYQQ4i6QhbYQQgghhBB3gSy0hRBC\nCCGEuAtkoS2EEDVMz549+eijj8z38/Pzadu2LTt27DBvu3jxIs888wzXr1+/H10UQggBuN3vDggh\nRNu2bfHx8cHFxfLf/m+99RbNmjWzus9nn33G6tWreeONN36TPuzdu5fNmzeTlpbm0H5KKaZPn46P\njw/Dhg0zb9+0aRPp6ekYDAaaN29OYmIijRo1stg3OzubmJgYzp49C8ChQ4dYvHgxJSUltG3blqSk\nJDw9PW97zvr163Pjxg3z/Q0bNuDl5WWxLS0tjbCwMOrXr+/Q3yOEENXdmoN/t7g/LLDjfeqJfbLQ\nFkJUCxs2bKBhw4aa2z/55JO/2SJbr6+++oq5c+fy6aef4uPjY96el5fH2rVrycrKwsvLi5SUFJYv\nX868efPMbb755htSUlLM9wsLC4mNjSU9PZ1WrVrx+uuvs2jRIuLj4297Xk9PT4xGo3m/gwcPEh4e\nbl5oX79+nZ07d7J169a79JcLIUTNYDKZiI+P58KFC7i7u5OYmEjLli3Nj2/evJn3338fg8FAdHQ0\ngYGBFBcXExMTw7Vr1/Dw8CAlJcWhz6fK5NQRIUS1lpOTQ0REBBMnTqRHjx5ERETw1VdfkZOTQ0hI\nCABFRUVMmDCB0NBQevXqRVxcHCaTCYDMzExCQkLo2bMnQ4cO5dKlS+bYy5cvp0uXLvTp04f9+/eb\ntx84cICIiAjCwsKIjIw0H3GuavPmzURERNCtWzeL7e3atWPfvn14eXlx8+ZNrly5gre3t/nxX3/9\nlZiYGKZPn27eduzYMZ588klatWoFQL9+/di5cydKqdue18vLy7zQzsjIIDg4mIceesi8bdu2bfj6\n+tKmTRvNr7MQQtQUSimLmy3Z2dmUlJSQmZnJlClTWLBggfmxwsJC3n33XTIyMli/fj3x8fEopUhP\nT8fHx4d3332XsLAwVqxYobuvstAWQlQLgwcPJjQ01HyLjo42P5aXl0dUVBQ7d+4kPDycmJgYi333\n799PUVERWVlZ5qO43333HSdPnuSdd95h48aNfPDBB4SEhBAdHY1SiuzsbD766CN27NhBRkaGeZH6\nzTffsHTpUlJTU9mxYwcJCQmMHz+eX3755bY+z549mx49elj9e2rVqkV2djYBAQF88sknhIeHW+zX\nt29f2rZta972P//zPzz44IPm+w8++CBGo5GioqLbYlcstEtLS8nIyGDgwIF4eHhgNBoxmUxs3ryZ\nQYMGAXDjxg369OnDM888wxdffGE3D0IIUd2ZlLK42XLmzBn8/f0BaN++PXl5eebHGjZsSFZWFrVq\n1eLHH3+kfv36GAwGi30CAgI4efKk7r7KQlsIUS1s2LCBrKws8+2tt94yP/bYY4/x7LPPAtC7d2/O\nnz/Pzz//bH68Q4cOfPnll0RFRZGamsrgwYNp2bIlR48eJTg42PyVX3h4OFeuXOFf//oXJ0+eJCgo\nCE9PT9zc3OjduzcAx48f54cffmDIkCGEhoYydepUDAYD3377rcN/U5cuXcjJyWH8+PEMGzbMvAh2\nc3OjT58+Fm1NJhMGg+G2GFXPW4fyhXZRURH79u3jscceo3Xr1nh6enLjxg0OHTqEq6srAQEBANSp\nU4fU1FS6du3qcP+FEKI6cuSIttFotPiti6urK7du3TLfd3NzY9OmTfTt29c8TxqNRry8vADw8PCw\n+P2Lo2ShLYSo9lxdXW1ua968Ofv372fkyJEYjUZeffVVDhw4YD59pDKllHmSrTxBV8QzmUx06tTJ\nYtG/ZcsWHn30Uc39vXz5MqdPnzbf7927N99//z3//ve/2b59O5999hmhoaGMHDmS4uJiQkNDefDB\nB/nhhx/M+1y5coUGDRpQr1692+JXnKO9ceNG85Hrim2bNm0iKirKvGivVauW7nMLhRCiOnLkiLan\np6fFN4Mmkwk3N8ufKA4cOJCjR4/yySefcOrUKYt9ioqKnPpRuSy0hRDVXn5+Pvn5+UD5OdfPPPOM\n+WgDwLvvvktsbCx+fn7ExMTg5+fHuXPn8Pf3Z8+ePRQWFgLl5y57e3vTsmVLAgIC2Lt3L9evX8dk\nMpGVlQVAp06dOH78OF999RUAhw8fpmfPnhQXF2vu79WrV5k8ebL5eXfu3Mmjjz7KAw88wNatW9m1\naxdZWVmkpqZSp04dsrKy8Pf355///CfffPMNUH7udefOna3G9/Ly4vTp0xiNRvz8/IDyD5Pz58/z\n2Wef0atXLwdeXSGEqFkcWWj7+vpy5MgRAHJzcy1+uP71118zbtw4lFLUqlULd3d3XFxc8PX15fDh\nwwAcOXKEDh066O6rXHVECFEtDB48+LbTJCZPnkydOnVo3Lgxy5Yto6CggIYNG7Jw4UIKCgrM7cLC\nwvj73/9OcHAwdevWpWnTpkRFRdGgQQOGDBnC4MGDMZlMNGzYkFWrVuHi4sILL7zAhQsX6N27N/Xr\n1+exxx7jp59+4pFHHmHevHlMnjwZpRRubm6sXLkSDw8PzX/Ls88+y+jRoxk0aBCurq40adLE4lQY\naxo1akRycjITJkygtLSUFi1aWFyVpDIvLy+uXLnC2LFjzds8PT25evUqgwcPdqivQghR09g7XaSy\noKAgjh8/TmRkJEopkpKSWLduHS1atKBz58489thj9O3bF4PBgL+/Px07duTJJ59k2rRp9OvXj1q1\narF48WLdfTUoR3orhBD3WE5ODgkJCezatet+d6VGmz59OkOHDrU4miOEEDXR3/Yes7g/vpvffeqJ\nfXJEWwgh/sONGDGC8+fPc+nSJfr27WtxBRQhhKhp7J0uUp3IEW0hhBBCCFFjLNtzxOL+a8EB96kn\n9skRbSGEEEIIUWPUpGPEstAWQgghhBA1Rk06dUQW2kIIIYQQosaoOctsWWgLIYQQQogaRI5oV9Jn\nyTqbj2+d/CoAq7Jt15Ef1aUT3ZJW2X2+vTNGMe1d+5cBS+kfwpS0D2y2WRzVE4CQlNU22+2aNgKA\nmRl7bLabHxnMmHe22u3byuF9SPnggM0203q+BNz+g4CqKn4gMHr1ezbbvT0igrjMD+32LbFvd5Z/\neNRmm4nd/QF4dUW6zXbrxvYDoN/yNJvt0idGMXaN/ddtxbA+JG3PttlmRq8uAMzaYvtvTXilOwAx\nm2y/R14f2JPBK96127cNY/vTa9Fau+22Tx1q9/2b0j8EgEkbdthst3RwGABhr6+x2W5HzDC74xTK\nx6q9Mbh3xigAJm/MstluyaBQAOa8t9dmu7kR3RixKtNu31aP6qv5fdl32Qab7TJfG6y5ndb35cC/\nbbLZZtP4gQC8ue+YzXbjupZfwqrqpa2qGt/NT3NOtY77xPf322wTFx4EwMKdB222+2uPQADCF9vu\n3/tTXtU8ZrS+vkt3H7bZbtJfXgBgZOoWm+1SR75id06F8nlV6zidsO59m+3eeLX8SjE9F75js90H\nfx1O/NZ9dvsW36erprkBIHLZRpvtMl4rr04asXS9zXbvTRqi+fPZ3vsIyt9LWt9vWuckLfmyF6si\nnpZcgfZxn7zjY5vtYsM6M+itzXb7tjF6gObXo7qSc7SFEEIIIYS4C0wm7Qttk8lEfHw8Fy5cwN3d\nncTERFq2bGnRprCwkMjISHbu3Ent2rVRShEQEECrVq0AaN++PVOmTNHVV80LbZPJdFvVNiGEEEII\nIe4lR45oZ2dnU1JSQmZmJrm5uSxYsICVK1eaHz969CiLFy/mxx9/NG/79ttveeKJJ3j77bed7qvN\nlfN3333H2LFjCQgIoEuXLrz44ouMHDmSS5cuOf3EQgghhBBCOMqklMXNljNnzuDvX37qYPv27cnL\ny7N43MXFhXXr1uHt7W3e9vnnn3PlyhWioqIYMWIEX3/9te6+2jyiPXPmTKZMmcLTTz9t3pabm0ts\nbCwZGRm6n1QIIYQQQgg9HDlD22g04unpab7v6urKrVu3cHMrXwI///zzt+3zu9/9jpEjR9K9e3dO\nnz5NTEwM27Zt09VXmwvtkpISi0U2lP9rQAghhBBCiPvBkVNHPD09KSoqMt83mUzmRfadtGvXDldX\nVwCeffZZrly5glIKg8HgcF9tPlPbtm2JjY3F398fLy8vioqKOHz4MG3btnX4iYQQQgghhHCWI5f3\n8/X15eDBgwQHB5Obm4uPj4/dfd588028vb0ZMWIE+fn5PPTQQ7oW2WBnoR0fH092djZnzpwxH3oP\nDAwkKChI15MJIYQQQgjhDEeOaAcFBXH8+HEiIyNRSpGUlMS6deto0aIFnTt3trrPyJEjiYmJ4fDh\nw7i6upKcnKy7rzYX2gaDgaCgIFlYCyGEEEKIasGRI9ouLi7MmzfPYlubNm1ua3fgwP/WL2nQoAGp\nqan6O1iJQdWkq34LIYQQQoj/06oW2krs2/0+9cS+u16wxl7VvA1j+wPaKk8FJ9v/18We2JF2q4RB\neaUwrVWbtFbs0lKlUWs1sQQ7Vdhm/f8qbPYqgMX36Qpg97XbEztS8+s7/G3bV5x5Z3QkgObKm1qq\nYtmr1Abl1doGvGG7yuTmCVEAml9fLdU+7cWqiGcvVkU8rRU1J67fbrPd8iG9AG1V7uxVP4XyCqha\nKjnC7ZNgVRWT4rxtH9lsN7v3y3b/Tij/W7VUPwXtVdi0VBnVmnutr5vWirCLdx2y2W5KyIua5xp7\n1eagvOKc1pxqrTaopdKkvcqsUF6d1V7F3ZXD+wAwbq3tufzNoeVzuZbqrENX2r/y1toxkZpfDy1V\naEFbVdDO81bY7dvHs8dqrpKqtVqtvUqpK4b14a+bd9rt28IBPTTPl1orlmod91rGqr3qkVBeQVJL\nrkD7mNEyP2jtm9a1WXVVk44RS2VIIYQQQghRYzhy6sj9JgttIYQQQghRY8hCWwghhBBCiLtATh0R\nQgghhBDiLviPWWhHRUVRWlpqsa2iMo6UYBdCCCGEENWZyWQiPj6eCxcu4O7uTmJiIi1btjQ/vmXL\nFjIyMnBzc2PMmDEEBgZSWFjI1KlTKS4upkmTJiQnJ1O3bl1dz29zoT116lTi4uJ46623zKUohRBC\nCCGEuF9MDhzQzs7OpqSkhMzMTHJzc1mwYAErV64E4OrVq6SlpbFt2zZu3rxJ//79ef7551mxYgUh\nISGEh4eTmppKZmYmQ4YM0dVXF1sPPv3004SGhnLhwgX+8Ic/WNyEEEIIIYS410wmk8XNljNnzuDv\n7w9A+/btycvLMz/26aef8swzz+Du7o6XlxctWrQgPz/fYp+AgABOnDihu692z9EePny47uBCCCGE\nEEL8lhy56ojRaMTT09N839XVlVu3buHm5obRaMTLy8v8mIeHB0aj0WK7h4cHN27c0N1XqQwphBBC\nCCFqjKqFzCoKtFmTnJzM008/TXBwMFB+hPrIkfICgx9//DFHjx4lPj4egOjoaEaPHs3s2bN55513\naNSoEfn5+SxdupRVq1bp6utdv+qIlsp6oK2CpL2qSFBeGUlrlTutFR//ssB2xcTd00cC2irwzdpi\nu7oaQMIr3YlN322zTXK/vwDaq1ZqqTaotW9aqkwCdiuALRzQA4Cluw/bbDfpLy9ozumIVZk226we\n1RdAcyU5LZW97MWqiKe1Up/WvtnLV8Ir5ZX6tFQRtFeZFcqrs3ad/7bNNvtmjga0vy+1vEdW7rf/\nld2YoD8TvnidzTbvT3kV0F6tVkvl2N+6CpuWCrmgrSKsvTEP5ePeXnVOKK/QGbF0vc02700aAsBb\nHx232S765ecB6LVorc1226cO1fw3rMo+abPNqC6dAO0V+OzldengMLsVCUFbdcuKio9aq1Zq+TzS\nOpdrqX4K2B2DY4L+DGgbz8s/PGq3bxO7+9utLAzl1YW1VirW8jkD2uYHe5/PUP4ZrbVqZZ8ltueu\nrZPL5y4tVSu1VnrVOl9WV44cI/b19eXgwYMEBweTm5uLj4+P+bGnnnqKZcuWcfPmTUpKSvjqq6/w\n8fHB19eXw4cPEx4ezpEjR+jQoYPuvsrl/YQQQgghRI3hyI8hg4KCOH78OJGRkSilSEpKYt26dbRo\n0YLOnTsTFRVF//79UUoxadIkateuzZgxY5g2bRpbtmzhgQceYPHixbr76vBCu6SkBHd3d91PKIQQ\nQgghhF4mZfsHkJW5uLgwb948i21t2rQx//8rr7zCK6+8YvF448aNWbPG9jdgmp//Tg8cOHCAwMBA\ngoKC2LPnf0//kB9HCiGEEEKI+0Upy1t1dscj2m+//Tbbt29HKcXEiRO5efMmvXr1qlHVeIQQQggh\nxH+WmrQWveNCu1atWnh7ewOwYsUKBg8eTNOmTTEYDPesc0IIIYQQQlTmyOX97rc7njryhz/8geTk\nZH755Rc8PT158803mTdvHl9//fW97J8QQgghhBBmSimLW3V2x4V2UlISbdu2NR/Bbtq0KRs3bqR7\n9+73rHNCCCGEEEJUVpMW2lKwRgghhBBC1BhDV1peP33tmMj71BP75DraQgghhBCixqhJx4jv+kI7\nLtN2harEvuWnorwY/6bNdofix9mtngTlFZSmvbvLbruU/iGaK3bZq+j39ogIABLsVIGaFR6kudrV\n5I1ZNtssGRQKoLmy18C/bbLZbtP4gXYrW0J5dUutFcy09k3L66u1CpuW6pyA3Sqj03q+pLldv+Vp\ndvuWPjHKbg6gPA9aqy+mfnzKZruRnZ8DtFW31FpNbNmeIzbbvBYcAGjPvZaqblpzv8hOBcypIS8C\naH59tVSGXHPw73b7Niywo+YKuVrnJC3VQ7VWrdRaqU9r7rXOl1oq9dlrU9HOXh6GBXYEtFVJ1dpO\na3VArdUo7Y3B2LDOgLbXTWtlSK1zuZZKmQAD3rA9F26eEKW5wrO9tQOUrx+0Vl3VWkFSy/vc3lwD\n5fNN1TLhVVWUDdeaBy19s/d6QPlrYu9zK31ilN0495MJ5xbaxcXFxMTEcO3aNTw8PEhJSaFhw4a3\ntbt8+TLR0dHs2lW+rvz555/p2rWrubpkly5dGDx4sM3nkiPaQgghhBCixnD2iHZ6ejo+Pj6MHz+e\n3bt3s2LFCuLi4iza7Nixg40bN/LTTz+Zt507d46QkBBmzZql+bnu+GNIa4qLiykpKXFkFyGEEEII\nIX4zJpOyuDnqzJkz+Pv7AxAQEMDJkydva9OgQQM2bbL8JjovL4/PP/+cgQMHMmHCBH744Qe7z2Xz\niPZ3331HcnIyjRs3plu3bsTFxeHi4sLMmTMJDAx05G8SQgghhBDCaY4c0X7vvffYsGGDxbZGjRrh\n5eUFgIeHBzdu3LhtP2vr3NatW9OuXTv+/Oc/88EHH5CYmMgbb7xh8/ltLrRnzJjB+PHjKSgoYMKE\nCezbt4/atWszfPhwWWgLIYQQQoh7zpGD2BEREURERFhsGzduHEVFRQAUFRVRv359TbGee+456tat\nC0BQUJDdRTbYOXXk1q1bdOzYkV69etGlSxcaNWqEp6cnbm5yarcQQgghhLj3nL2Otq+vL4cPl/+w\n+MiRI3To0EHTfnFxcezbtw+AkydP8sQTT9jdx+ZC++GHH2bmzJmYTCYWLFgAQGpqKo0bN9bUISGE\nEEIIIX5Lqsp/jurXrx8XL16kX79+ZGZmMm7cOAAWLlzIp59+esf9pkyZQnp6OlFRUWRkZDBz5ky7\nz2Xz0HRiYiIHDhzAxeV/1+O///3viYqq3pd9EUIIIYQQ/5lMTl51pG7dulZP+/jrX/9627bjx4+b\n/7958+akpdm/pG9lUhlSCCGEEELUGFXrqmyd/Op96ol9crK1EEIIIYSoMXRc0e++kYW2EEIIIYSo\nMWrSyRiy0BZCCCGEEDWGLLSFEEIIIYS4C5z9MeS9JAttIYQQQghRY8hCWwghhBBCiLtATh0RQggh\nhBDiLnB2nV1cXExMTAzXrl3Dw8ODlJQUGjZsaNFm6dKlnDhxAoPBQFxcHE899RSFhYVMnTqV4uJi\nmjRpQnJysrkk+53cs4W2yWQiPj6eCxcu4O7uTmJiIi1bttQVq7S0lBkzZlBQUEBJSQljxoyhc+fO\nTvXv2rVrhIeHs3btWtq0aeNUrFWrVnHgwAFKS0vp168fERERumOVlpYyffp0CgoKcHFxISEhQVf/\n/vnPf7Jo0SLS0tK4fPky06dPx2Aw8OijjzJnzhyLokSOxjt//jwJCQm4urri7u5OSkqKQ9VDK8eq\nsHPnTjZt2kRmZqZD/aoa79q1a8TFxXH9+nXKyspYuHAhLVq00BXr/PnzzJkzB1dXV1q1asX8+fM1\nv27W3rOPPPKI7jxYi/fQQw/pyoOt8aQnD9bitW/fXlce7vR36s1DWVkZcXFxXLp0CVdXV5KTk1FK\n6cqDtVhFRUW6x4K1eBWvkaN5sBbLw8ND91i409+qNw9gOee6ubk5PSdVjldSUuLUnGTt88CZOaly\nPG9vb6fmpKp/pzM5CAsLw8vLC4BmzZrRt29f5s+fj6urK35+fuZqeXrj9ezZk2XLluHm5kajRo1I\nSUmxuyi5U6zk5GQAVq5cyRdffMHSpUud6tvo0aOZM2cOpaWluLu7s2TJEh544AFdsXr06MGiRYtw\nc3OjU6dOTJo0yaG+VV0zdOzYUfd4qBqrXbt2To2F6srZU0fS09Px8fFh/Pjx7N69mxUrVhAXF2d+\n/Ny5c+Tm5rJlyxYKCgoYO3YsH3zwAStWrCAkJITw8HBSU1PJzMxkyJAhtp9M3SP79u1T06ZNU0op\ndfbsWTV69GjdsbZu3aoSExOVUkoVFhaqF154wam+lZSUqLFjx6qXX35Zffnll07FOnXqlBo1apQq\nKytTRqNRvfHGG07F279/v5owYYJSSqljx46pcePGORwjNTVVhYSEqIiICKWUUqNGjVKnTp1SSik1\na9Ys9dFHHzkVb8CAAercuXNKKaXS09NVUlKS7lhKKXXu3Dk1aNAgi216402bNk3t3r1bKaXUyZMn\n1cGDB3XHGjt2rDp06JBSSqnJkyerjz/+WHMsa+9ZZ/JgLZ7ePNxpPOnNg7V4evNgLZYzedi/f7+a\nPn26Uqp8rI4ePVp3HqzFcmYsWIunlL48WIvlzFiwFs+ZPFSdc52dk6rGcyYP1j4PnJmTqsZzJg9V\nYzmTg+LiYhUaGmqxrWfPnury5cvKZDKp4cOHq7y8PKfivfzyy+rq1atKKaUWLVqkNmzYoDuWUkod\nOnRIRUZGqtdee01zv+4ULyoqSp09e1YppdTevXvVP/7xD92xQkND1cWLF5XJZFKRkZEqPz9fc9+s\nrRn0jgdrsZwZC9XZy4krLW6Oio6ONuf/+vXrKjg4+LY2paWlSimlcnJy1KuvvqqUUiosLEz98MMP\nSimlzp8/r0aMGGH3uRw7ZOCEM2fO4O/vD0D79u3Jy8vTHatbt25MnDjRfN/V1dWpvqWkpBAZGUmT\nJk2cigNw7NgxfHx8iI6OZvTo0bz44otOxXv44YcpKyvDZDJhNBpxc3P8S4gWLVrwt7/9zXz/888/\np2PHjgAEBARw4sQJp+ItWbKExx9/HCg/8lW7dm3dsX766ScWLVrEjBkzHOrTneL94x//4MqVKwwZ\nMoSdO3ea/249sR5//HF+/vlnlFIUFRU5lAtr71ln8mAtnt48WIvlTB6sxdObB2uxnMlDly5dSEhI\nAOD777+ncePGuvNgLZYzY8FaPL15sBbLmbFgLZ4zeag65zo7J1WN50weqsZydk6qGs+ZPFSN5UwO\n8vPz+fXXXxk6dCiDBg3ik08+oaSkhBYtWmAwGPDz8+PkyZO64+Xm5pKWlmY+enrr1i3NebAW6/Ll\ny2RmZjJ+/HjNfbpTvLNnz1JYWMjBgweJiooiNzeXp556SnffKvJQWlrKzZs3HVqTWFsz6B0P1mI5\nMxaqM5NSFjdb3nvvPUJCQixuN27cMH8r4eHhwY0bN27bz83NjaVLlzJq1ChCQkIAMBqNdver6p4t\ntI1GI56enub7rq6u3Lp1S1csDw8PPD09MRqNTJgwgddee013v95//30aNmxo/keAs3766Sfy8vJY\nvnw5c+fOZerUqU6dtF+vXj0KCgro3r07s2bNIioqyuEYXbt2tZiAlVIYDAZA+xvFVrzKHyCbNm2y\n/zXKHWKVlZUxc+ZMZsyYgYeHh0N9ulPfCgoKqF+/PuvXr6dp06asXr1ad6yKr2a7d+/OtWvX+K//\n+i/Nsay9Z53Jg7V4evNQNdbEiROdyoO1vunNg7VYzuQByifPadOmkZCQQNeuXZ3KQ9VYzoyFqvFe\nfvllp/JQtW/OjAVr8fTmwdqc60wOrMXTm4eqsUwmk1M5sNY3vXmwFsuZsVCnTh2GDRvGmjVrmDt3\nLrGxsRandTiah6rxpk6daj7ndf/+/eTk5BAWFqY71uzZs5k3b56uA2tV402ZMoWLFy/SqVMnNm7c\nyL///W+2b9+uu29t2rRh9OjRBAcH07RpU1q3bq25b3daM+gZD9Zi/e53vwP0z0nVlVLK4mZLREQE\nu3btsrh5eXlRVFQEQFFREfXr17e676RJkzh69Chr1qzh22+/xdPTU9N+ld2zhXblzkH5BKbn6GyF\n//7v/2bQoEGEhobSo0cP3XG2bdvGiRMniIqK4vz580ybNo2rV6/qjuft7Y2fnx/u7u60bt2a2rVr\nU1hYqDve+vXr8fPzY9++fWRlZTF9+nRu3rypOx5gca6X1jeKPXv27GHOnDmkpqbe9oMCrT7//HMu\nX75MfHw8kydP5ssvv2T+/Pkt7gybAAAFm0lEQVRO9cvb25uXXnoJgJdeesmpb1Lmz5/P5s2b2bt3\nL2FhYSxYsMCh/au+Z53Ng7UxoDcPlWO1atXK6TxU7Zszeagay9k8QPmRwX379jFr1iyL8aQnD5Vj\n/fLLL06PhYp40dHR5OfnO5WHyn3z8vJyeixUjpeYmKgrD9bm3MpzpKM5uNMcricPVWP16NGDL774\nQncOrPXNxcVFVx6sxZo+fbrusfDwww/Ts2dPDAYDDz/8MF5eXvz888/mxx3NQ9V43t7eXL16lfXr\n17NmzRreeecdzUdTq8ZydXWloKCASZMmkZSUxKlTp0hNTdXdt4pzsZ977jkMBgOBgYGa82Ctb6+/\n/jq7d+8mOzubli1bsnbtWs19s7ZmqLywdiQPd1p//Bafz/9pfH19OXz4MABHjhyhQ4cOFo+fPHmS\nuXPnAlC7dm3c3NwwGAx297Pmni20fX19OXLkCAC5ubn4+PjojvXjjz8ydOhQYmJi6NOnj1P92rx5\nM5s2bSItLY3HH3+clJQU878A9ejQoQNHjx5FKcWVK1f49ddf8fb21h2vfv365q8pGjRowK1btygr\nK9MdD+CPf/wjOTk5QPkb5dlnn3UqXlZWlvk1bN68ue44Tz31FLt37yYtLY0lS5bwyCOPMHPmTKf6\n1qFDB/Og+OSTT3jkkUd0x2rQoIH5W5kmTZpw/fp1zftae886kwdr8fTmoWosZ/NgrW9682AtljN5\n2LFjB6tWrQKgbt26GAwG2rVrpysP1mLt379f91ioGq9x48Z8+OGHuvJgrW8dO3bUPRasxfP29taV\nB2tzbkBAgO6xYC3eiRMndOWhaqyKxZPesWCtb4GBgbryYC1Ws2bNdI+FrVu3mhfmFZ9V9erV49tv\nv0UpxbFjxxzKQ9V4RqORbdu2cfr0adavX+/QAq9qLIPBwN69e0lLS2PGjBk899xzjBw5Une8oqIi\nnnjiCU6fPg2U5+HRRx/VFau0tJRmzZpRr149wPE8WFszdOrUSdd4sBbryJEjv8nnc3WjlOXNUf36\n9ePixYv069ePzMxM8w9/Fy5cyKeffkrHjh0xmUxERkYyYMAABgwYQPPmzRkzZgy7d+8mMjKSs2fP\nMnDgQLvPZVDOnNfggIqrjnzxxRcopUhKStJ9dY/ExEQ+/PBDi69nVq9eTZ06dZzqY1RUFPHx8U5f\ndWThwoXk5OSglGLSpElOnZZSVFTEjBkzuHr1KqWlpQwaNEjXEfx//etfTJ48mS1btnDp0iVmzZpF\naWkprVu3JjEx0eGv4yripaen06lTJ5o2bWr+V/ef/vQnJkyYoKtvtrbpiVdQUEBcXBy//vornp6e\nLF68mAYNGuiKdfr0afMvy2vVqkVCQgLNmjXTFMfae3bmzJkkJibqykPVeGVlZVy8eJGHHnrI4TzY\nGk968mAt3oIFC3TlwVqsiRMn6s7DL7/8QmxsLD/++CO3bt1ixIgRtGnTRtd4sBZrxowZuseCtXhd\nunQBHB8P1mI9/vjjuseCtXje3t6681ChYs51cXFxek6qiDd79mwGDBjg1JxUuW8VnwfOzEmV49Wp\nU8epOalyrIrzx/XkoKSkhNjYWL7//nsMBgNTp07FxcWFpKQkysrK8PPzc+jqGVXjvfbaawwdOpQ/\n/vGP5iPZ3bt3p3///rr65uvrC0BOTg4ZGRkOXXXEWrx69eoxd+5cysrKaNasGQsWLMDd3V1XrGvX\nrpGamkrt2rXx8vJiwYIFDuW06pqhWbNmusdD1VhTpkxxeixUR4Fz37K4f3BO9H3qiX33bKEthBBC\nCCHE/yX37NQRIYQQQggh/i+RhbYQQgghhBB3gSy0hRBCCCGEuAtkoS2EEEIIIcRdIAttIYQQQggh\n7gJZaAshhBBCCHEXyEJbCCGEEEKIu0AW2kIIIYQQQtwF/w+l7tRBvwa3jQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x2327a29d630>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "TOTAL_EPISODES = 2000 if isFast else 3000\n",
    "\n",
    "def run(agent):\n",
    "    s = env.reset()\n",
    "    R = 0\n",
    "\n",
    "    while True:\n",
    "        # Uncomment the line below to visualize the cartpole\n",
    "        # env.render()\n",
    "\n",
    "        # CNTK: explicitly setting to float32\n",
    "        a = agent.act(s.astype(np.float32))\n",
    "\n",
    "        s_, r, done, info = env.step(a)\n",
    "\n",
    "        if done: # terminal state\n",
    "            s_ = None\n",
    "\n",
    "        agent.observe((s, a, r, s_))\n",
    "        agent.replay()\n",
    "\n",
    "        s = s_\n",
    "        R += r\n",
    "\n",
    "        if done:\n",
    "            return R\n",
    "\n",
    "agent = Agent()\n",
    "\n",
    "episode_number = 0\n",
    "reward_sum = 0\n",
    "while episode_number < TOTAL_EPISODES:\n",
    "    reward_sum += run(agent)\n",
    "    episode_number += 1\n",
    "    if episode_number % BATCH_SIZE_BASELINE == 0:\n",
    "        print('Episode: %d, Average reward for episode %f.' % (episode_number,\n",
    "                                                               reward_sum / BATCH_SIZE_BASELINE))\n",
    "        if episode_number%200==0:\n",
    "            plot_weights([(agent.brain.params['W1'], 'Episode %i $W_1$'%episode_number)], figsize=(14,5))\n",
    "        if reward_sum / BATCH_SIZE_BASELINE > REWARD_TARGET:\n",
    "            print('Task solved in %d episodes' % episode_number)\n",
    "            plot_weights([(agent.brain.params['W1'], 'Episode %i $W_1$'%episode_number)], figsize=(14,5))\n",
    "            break\n",
    "        reward_sum = 0\n",
    "agent.brain.model.save('dqn.mod')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you run it, you should see something like\n",
    "```\n",
    "Episode: 20, Average reward for episode 20.700000.\n",
    "Episode: 40, Average reward for episode 20.150000.\n",
    "Episode: 60, Average reward for episode 21.100000.\n",
    "...\n",
    "Episode: 960, Average reward for episode 20.150000.\n",
    "Episode: 980, Average reward for episode 26.700000.\n",
    "Episode: 1000, Average reward for episode 32.900000.\n",
    "Task solved in 1000 episodes\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Task 1.1**\n",
    "Rewrite the model without using the layer lib.\n",
    "\n",
    "**Task 1.2**\n",
    "Play with different [learners](https://cntk.ai/pythondocs/cntk.learner.html#module-cntk.learner). Which one works better? Worse? Think about which parameters you would need to adapt when switching from one learner to the other."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Running the DQN model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0\n",
      "1\n",
      "2\n",
      "3\n",
      "4\n",
      "5\n",
      "6\n",
      "7\n",
      "8\n",
      "9\n"
     ]
    }
   ],
   "source": [
    "env = gym.make('CartPole-v0')\n",
    "\n",
    "num_episodes = 10  # number of episodes to run\n",
    "\n",
    "modelPath = 'dqn.mod'\n",
    "root = C.load_model(modelPath)\n",
    "\n",
    "for i_episode in range(num_episodes):\n",
    "    print(i_episode)\n",
    "    observation = env.reset()  # reset environment for new episode\n",
    "    done = False\n",
    "    while not done: \n",
    "        if not 'TEST_DEVICE' in os.environ:\n",
    "            env.render()\n",
    "        action = np.argmax(root.eval([observation.astype(np.float32)]))\n",
    "        observation, reward, done, info  = env.step(action)    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 2: Policy gradient (PG)\n",
    "**Goal:**\n",
    "$$\n",
    "\\text{maximize } E [R | \\pi_\\theta]\n",
    "$$\n",
    "\n",
    "**Approach:**\n",
    "1. Collect experience (sample a bunch of trajectories through $(s,a)$ space)\n",
    "2. Update the policy so that _good_ experiences become more probable\n",
    "\n",
    "**Difference to DQN:**\n",
    " * we don't consider single $(s,a,r,s')$ transitions, but rather use whole episodes for the gradient updates\n",
    " * our parameters directly model the policy (output is an action probability), whereas in DQN they model the value function (output is raw score)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Rewards\n",
    "\n",
    "Remember, we get +1 reward for every time step, in which we still were in the game.\n",
    "\n",
    "The problem: we normally do not know, which action led to a continuation of the game, and which was actually a bad one. Our simple heuristic: actions in the beginning of the episode are good, and those towards the end are likely bad (they led to losing the game after all)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def discount_rewards(r, gamma=0.999):\n",
    "    \"\"\"Take 1D float array of rewards and compute discounted reward \"\"\"\n",
    "    discounted_r = np.zeros_like(r)\n",
    "    running_add = 0\n",
    "    for t in reversed(range(0, r.size)):\n",
    "        running_add = running_add * gamma + r[t]\n",
    "        discounted_r[t] = running_add\n",
    "    return discounted_r"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x2327a7ad128>"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAT8AAACLCAYAAAAXip58AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAADnRJREFUeJzt3X9MVfX/B/DnFbqC92KMRbONrmGT\nZTKX2Ir8hQwUKw1CfpO3xBiSRvqH48eI9KuJLNtKNgJt2nZ1CR/AcmuTUbFILOc0FAkqUFyisKuy\n4bnivdfL+/vHZ968H03lco6K7+fjL+55353X6/rjufe55764OiGEABGRZMY96AaIiB4Ehh8RSYnh\nR0RSYvgRkZQYfkQkJd8H3QAAXLt2DadOnUJwcDB8fHwedDtE9IhwuVywWq0IDw+Hn5+fx9pDEX6n\nTp1CZmbmg26DiB5Re/fuxYsvvuhx7J7C78SJE9i2bRssFgvOnj2LgoIC6HQ6TJ06FR999BHGjfvn\n6vnatWtYv349Ll26BIPBgLKyMgQFBd3x/MHBwe4GJ02aNNLXRUR0W319fcjMzHRnzM3uGn47d+7E\ngQMH4O/vDwAoLS3F2rVr8fLLL6OkpAQ//PADFi5c6H7+119/jbCwMLz//vv47rvvUFFRgeLi4jvW\nuHGpO2nSJISEhIzoxRER3c3t3k676w0Pk8mE8vJy9+P29na89NJLAID58+fj8OHDHs8/duwY5s2b\n517/5ZdfRtU0EZEW7hp+cXFx8PX9Z4MohIBOpwMAGAwGXLlyxeP5iqIgICDgX9eJiB4GI77hcfP7\nezabDRMnTvRYNxqNsNls/7p+N8n/9/VIW7pn/ylJ1+zcRDS2jPhzfs8//zyOHDkCAGhubr7lDkpE\nRAR++ukn9/qsWbNUaJOISF0jDr/8/HyUl5cjNTUVTqcTcXFxAICsrCw4HA6kp6fjr7/+Qnp6Oqqr\nq7FmzRrVmyYiGq17uuwNCQlBTU0NACA0NBR79uy55Tm7du1y/7x9+3aV2iMi0gbH24hISgw/IpIS\nw4+IpMTwIyIpMfyISEoMPyKSEsOPiKT0UPw+vweJ43REcuLOj4ikxPAjIikx/IhISgw/IpISw4+I\npMTwIyIpMfyISEoMPyKSEsOPiKQk/YTH/abVRAmnSYhGxqvwq6+vx/79+wEAdrsdHR0daGlpcX9T\n2+bNm3H8+HEYDAYAQEVFhfvrLImIHgZehV9iYiISExMBABs3bsSyZcs8vqKyvb0dX375JYKCgtTp\nkohIZaN6z6+trQ1dXV1ITU11HxseHsbZs2dRUlKCtLQ01NbWjrpJIiK1jeo9v6qqKqxevdrj2NWr\nV/HWW29hxYoVcLlcMJvNCA8Px3PPPTeqRomI1OT1zm9wcBCnT59GZGSkx3F/f3+YzWb4+/vDaDQi\nMjISnZ2do26UiEhNXoff0aNHMXv27FuO9/T0ICMjAy6XC06nE8ePH8f06dNH1SQRkdq8vuw9c+YM\nQkJC3I93794Nk8mEmJgYLF26FCkpKXjssccQHx+PqVOnqtIsEZFavA6/d9991+PxihUr3D9nZ2cj\nOzvb+66IiDTGCQ8ikhInPB5xnCghuj3u/IhISgw/IpISw4+IpMTwIyIpMfyISEoMPyKSEsOPiKTE\n8CMiKTH8iEhKDD8ikhLH20hVHKejsYI7PyKSEsOPiKTE8CMiKTH8iEhKDD8ikpLXd3sTEhIQEBAA\nAAgJCUFpaal7raamBvv27YOvry9yc3MRHR09+k6JiFTkVfjZ7XYAgMViuWXNarXCYrGgrq4Odrsd\nGRkZmDNnDvR6/eg6JSJSkVeXvZ2dnRgaGkJWVhbMZjNaW1vdaydPnsTMmTOh1+sREBAAk8nE7+0l\nooeOVzs/Pz8/rFy5EsnJyejp6UF2djYOHjwIX19fKIrivhwGAIPBAEVRVGuYiEgNXoVfaGgoJk+e\nDJ1Oh9DQUAQGBsJqteKpp56C0WiEzWZzP9dms3mEIZGaOFFC3vLqsre2thZbt24FAPT390NRFAQH\nBwMAZsyYgWPHjsFut+PKlSvo7u5GWFiYeh0TEanAq51fUlISCgsLkZ6eDp1Ohy1btsBiscBkMiEm\nJgbLly9HRkYGhBBYt24dxo8fr3bfRESj4lX46fV6fPrppx7HIiIi3D+npKQgJSVldJ0REWmIH3Im\nIikx/IhISgw/IpISw4+IpMTwIyIpMfyISEr8Dg+iEeBEyaODOz8ikhLDj4ikxPAjIikx/IhISgw/\nIpISw4+IpMTwIyIpMfyISEoMPyKSEsOPiKTE8TaihxjH6bTjVfg5nU4UFRWht7cXDocDubm5iImJ\nca/v3r0btbW1CAoKAgBs3LgRU6ZMUadjIiIVeBV+Bw4cQGBgID755BMMDAzgzTff9Ai/9vZ2lJWV\nITw8XLVGiYjU5FX4LV68GHFxce7HPj4+Huvt7e3YsWMHrFYrFixYgJycnNF1SUSkMq/Cz2AwAAAU\nRUFeXh7Wrl3rsf76668jIyMDRqMRa9asQVNTE6Kjo0ffLRGRSry+23vhwgWYzWbEx8dj6dKl7uNC\nCLz99tsICgqCXq9HVFQUfv/9d1WaJSJSi1fhd/HiRWRlZWH9+vVISkryWFMUBUuWLIHNZoMQAkeO\nHOF7f0T00PHqsreyshKDg4OoqKhARUUFACA5ORlDQ0NITU3FunXrYDabodfr8corryAqKkrVpomI\nRsur8CsuLkZxcfG/rickJCAhIcHrpoiItMYJDyKSEic8iMhNpokS7vyISEoMPyKSEsOPiKTE8CMi\nKTH8iEhKDD8ikhLDj4ikxPAjIikx/IhISpzwIKIHQqtpEuDeJkq48yMiKTH8iEhKDD8ikhLDj4ik\nxPAjIikx/IhISl6F3/DwMEpKSpCamorly5fj7NmzHus1NTVITExESkoKmpqaVGmUiEhNXn3O7/vv\nv4fD4UB1dTVaW1uxdetWfPHFFwAAq9UKi8WCuro62O12ZGRkYM6cOdDr9ao2TkQ0Gl7t/I4dO4Z5\n8+YBAF544QWcOnXKvXby5EnMnDkTer0eAQEBMJlM6OzsVKdbIiKVeLXzUxQFRqPR/djHxwfXr1+H\nr68vFEVBQECAe81gMEBRlDuez+VyAQD6+vrgUAa8aemenDt37pZjj0q929ViPdZ7mOvdj/97fX19\nAP7JmJt5FX5GoxE2m839eHh4GL6+vrdds9lsHmF4O1arFQCQmZnpTTv3LObbzzU9/4Os9yi/NtZj\nvdHWs1qtmDx5sscxr8IvIiICTU1NeO2119Da2oqwsDD32owZM/DZZ5/BbrfD4XCgu7vbY/12wsPD\nsXfvXgQHB8PHx8ebloiIbuFyuWC1WhEeHn7Lmk4IIUZ6wuHhYWzYsAF//vknhBDYsmULmpubYTKZ\nEBMTg5qaGlRXV0MIgZycHMTFxanyQoiI1OJV+BERjXX8kDMRSYnhR0RSYvgRkZTG5G9yvnHD5Y8/\n/oBer8fmzZtvuY2thRMnTmDbtm2wWCya1nE6nSgqKkJvby8cDgdyc3MRExOjWT2Xy4Xi4mKcOXMG\nPj4+KC0thclk0qweAFy6dAmJiYnYtWsXnn32WU1rAUBCQoL7I1chISEoLS3VrFZVVRV+/PFHOJ1O\npKenIzk5WbNaAFBfX4/9+/cDAOx2Ozo6OtDS0oKJEyeqXsvpdKKgoAC9vb0YN24cNm3apOnfn8Ph\nQGFhIf7++28YjUaUlJTgmWeeUefkYgxqaGgQ+fn5QgghfvvtN7Fq1SrNa+7YsUMsWbJEJCcna16r\ntrZWbN68WQghxOXLl0VUVJSm9RobG0VBQYEQQohff/1V8z9Ph8Mh3nvvPbFo0SLR1dWlaS0hhLh2\n7ZqIj4/XvI4Q//3zy8nJES6XSyiKIrZv335f6t6wYcMGsW/fPs3O39jYKPLy8oQQQhw6dEisWbNG\ns1pCCGGxWERxcbEQQoju7m6RlZWl2rnH5GXvncbrtGIymVBeXq55HQBYvHgxPvjgA/djrT/7GBsb\ni02bNgEAzp8/jyeeeELTemVlZUhLS8OTTz6paZ0bOjs7MTQ0hKysLJjNZrS2tmpW69ChQwgLC8Pq\n1auxatUqLFiwQLNa/6utrQ1dXV1ITU3VrEZoaChcLheGh4ehKIp7uEErXV1dmD9/PgBgypQp6O7u\nVu3cY/Ky907jdVqJi4v715EgtRkMBgD/fZ15eXlYu3at5jV9fX2Rn5+PxsZGbN++XbM69fX1CAoK\nwrx587Bjxw7N6tzMz88PK1euRHJyMnp6epCdnY2DBw9q8u9lYGAA58+fR2VlJc6dO4fc3FwcPHgQ\nOp1O9Vr/q6qqCqtXr9a0xoQJE9Db24tXX30VAwMDqKys1LTetGnT0NTUhNjYWJw4cQL9/f1wuVyq\nbAjG5M7vTuN1j4oLFy7AbDYjPj4eS5cuvS81y8rK0NDQgA8//BBXr17VpEZdXR0OHz6M5cuXo6Oj\nA/n5+e7xRq2EhobijTfegE6nQ2hoKAIDAzWrGRgYiLlz50Kv12PKlCkYP348Ll++rEmtmw0ODuL0\n6dOIjIzUtM5XX32FuXPnoqGhAd9++y0KCgpgt9s1q7ds2TIYjUaYzWY0NTVh+vTpql0Jjcnwi4iI\nQHNzMwDcMl73KLh48SKysrKwfv16JCUlaV7vm2++QVVVFQDA398fOp1Os0vtvXv3Ys+ePbBYLJg2\nbRrKysoQHBysSa0bamtrsXXrVgBAf38/FEXRrOasWbPw888/QwiB/v5+DA0NITAwUJNaNzt69Chm\nz56teZ2JEye6bxw9/vjjuH79+m1/aYBa2traMGvWLFgsFsTGxuLpp59W7dxjcru0cOFCtLS0IC0t\nzT1e9yiprKzE4OAgKioqUFFRAQDYuXMn/Pz8NKm3aNEiFBYWIjMzE9evX0dRURHGjx+vSa0HISkp\nCYWFhUhPT4dOp8OWLVs0u1KIjo7G0aNHkZSUBCEESkpK7su8+pkzZxASEqJ5nXfeeQdFRUXIyMiA\n0+nEunXrMGHCBM3qTZ48GZ9//jl27dqFgIAAfPzxx6qdm+NtRCSlMXnZS0Q0Wgw/IpISw4+IpMTw\nIyIpMfyISEoMPyKSEsOPiKTE8CMiKf0/4k4gmJwqa3AAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x2327a0c3048>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "discounted_epr = discount_rewards(np.ones(10))\n",
    "f, ax = plt.subplots(1, figsize=(5,2))\n",
    "sns.barplot(list(range(10)), discounted_epr, color=\"steelblue\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We normalize the rewards so that they tank below zero towards the end. gamma controls how late the rewards tank."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x2327a4dfe48>"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAATcAAACLCAYAAAAEXd6IAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAACaJJREFUeJzt221Ilfcfx/HP8YhZO5VEjh6YZgNB\n6kFY0Bi5G2Z3Y9UoD9NJEkasm1GNGN7QRMosoScVhBZE4eK/rfs9SowFpVFEpHSgBbYGZUvsBuK0\nTD3ntwd/EmPxH//j9buyn+/Xs6NwfX5afPxe5zrfgDHGCAAck/SmDwAANlBuAJxEuQFwEuUGwEmU\nGwAnJfsR0tvbq0gkovT0dAWDQT8iAYwCsVhMPT09mjlzplJTU1/5ni/lFolEVFJS4kcUgFHo6NGj\nmjNnzitf86Xc0tPTBw8wZcoUPyIBjAIPHjxQSUnJYMcM5Uu5vbwVnTJlijIyMvyIBDCKvO7tLh4o\nAHAS5QbASb7clg4V3vYfa9c+Vl1s7doA3i5MbgCcRLkBcBLlBsBJlBsAJ1FuAJxEuQFwEuUGwEmU\nGwAnUW4AnES5AXCS7+tXfmPdCxidmNwAOIlyA+CkYZVbR0eHVq5c6dVZAMAzCb/ndvDgQf3yyy8a\nO3asl+cBAE8kPLllZmZq3759Xp4FADyTcLktXLhQycnOP2wF8JbigQIAJ1FuAJxEuQFw0rDKLSMj\nQz///LNXZwEAz/BEwGOsewEjA7elAJxEuQFwEuUGwEmUGwAnUW4AnES5AXAS5QbASZQbACdRbgCc\nRLkBcBLrV2851r2A12NyA+Akyg2Akyg3AE6i3AA4iXID4CTKDYCTKDcATqLcADiJcgPgJMoNgJNY\nv8L/xda6F6te8BqTGwAnJTy5xeNx1dTU6NatW0pJSVFtba2ysrK8PBsAJCzhye3cuXPq6+vTTz/9\npC1btmjXrl1engsAhiXhcrt27Zry8/MlSbNmzVIkEvHsUAAwXAnflkajUYVCocHXwWBQAwMDSk7+\n35f0+41j8t7uPL8fYJD3ducNlfDkFgqF9OzZs8HX8Xj8X4sNAPyScLnl5eXpwoULkqT29nbl5OR4\ndigAGK6ER6358+erra1NRUVFMsaorq7Oy3MBwLAkXG5JSUnatm2bl2cBAM/wIV4ATuIJAEY01rKQ\nKCY3AE6i3AA4iXID4CTKDYCTKDcATqLcADiJcgPgJMoNgJMoNwBOotwAOIn1K2AI1r3cweQGwEmU\nGwAnUW4AnES5AXAS5QbASZQbACdRbgCcRLkBcBLlBsBJlBsAJ7F+BbxBrHvZM6zJraWlRVu2bPHq\nLADgmYQnt9raWrW2tio3N9fL8wCAJxKe3PLy8lRTU+PhUQDAO/86uR07dkxHjhx55Wt1dXX67LPP\ndOXKFWsHA4Dh+NdyC4fDCofDfpwFADzDR0EAOIlyA+CkYX3Obe7cuZo7d65XZwEAzzC5AXAS5QbA\nSaxfAaPIaFr3YnID4CTKDYCTKDcATqLcADiJcgPgJMoNgJMoNwBOotwAOIlyA+Akyg2Ak1i/AmDN\nm1z3YnID4CTKDYCTKDcATvLlPbdYLCZJevDggR9xAEaJl53ysmOG8qXcenp6JEklJSV+xAEYZXp6\nepSVlfXK1wLGGGM7uLe3V5FIROnp6QoGg7bjAIwSsVhMPT09mjlzplJTU1/5ni/lBgB+44ECACdR\nbgCcRLkBcBLlBsBJI3K3NB6Pq6amRrdu3VJKSopqa2v/8ZjXho6ODu3evVtNTU1Wc/r7+1VVVaWu\nri719fVp3bp1+vTTT63lxWIxbd26VXfu3FEwGNTOnTuVmZlpLe+lR48eafny5Tp06JDee+89q1lf\nfPGFxo8fL0nKyMjQzp07reY1Njbq119/VX9/v4qLixUOh61lnTx5UqdOnZIkvXjxQjdv3lRbW5sm\nTJjgeVZ/f78qKirU1dWlpKQkbd++3eq/XV9fnyorK3X37l2FQiFVV1dr2rRp3lzcjEDNzc2mvLzc\nGGPM9evXzdq1a61nHjhwwHz++ecmHA5bzzp+/Lipra01xhjz+PFj89FHH1nNa2lpMRUVFcYYYy5f\nvuzL77Ovr8+sX7/eLFiwwHR2dlrN6u3tNcuWLbOaMdTly5fN119/bWKxmIlGo2bv3r2+ZdfU1Jgf\nf/zR2vVbWlrMxo0bjTHGtLa2mm+++cZaljHGNDU1ma1btxpjjLl9+7YpKyvz7Noj8rb02rVrys/P\nlyTNmjVLkUjEemZmZqb27dtnPUeSFi1apE2bNg2+tv3Zv4KCAm3fvl2SdP/+fU2ePNlqniTV19er\nqKhI7777rvWs3377Tc+fP1dZWZlKS0vV3t5uNa+1tVU5OTnasGGD1q5dq48//thq3ks3btxQZ2en\nvvzyS2sZ2dnZisViisfjikajSk62e3PX2dmpDz/8UJI0ffp03b5927Nrj8jb0mg0qlAoNPg6GAxq\nYGDA6i964cKFunfvnrXrD/XOO+9I+u/PuXHjRm3evNl6ZnJyssrLy9XS0qK9e/dazTp58qQmTZqk\n/Px8HThwwGqWJKWmpmr16tUKh8P6448/tGbNGp09e9ba/5cnT57o/v37amho0L1797Ru3TqdPXtW\ngUDASt5LjY2N2rBhg9WMcePGqaurS4sXL9aTJ0/U0NBgNS83N1fnz59XQUGBOjo61N3drVgs5skf\n/BE5uYVCIT179mzwdTwet/4XxG9//vmnSktLtWzZMi1ZssSXzPr6ejU3N+v777/XX3/9ZS3nxIkT\nunTpklauXKmbN2+qvLx8cAXPhuzsbC1dulSBQEDZ2dlKS0uzmpeWlqZ58+YpJSVF06dP15gxY/T4\n8WNreZL09OlT/f7773r//fet5hw+fFjz5s1Tc3Ozzpw5o4qKCr148cJa3ooVKxQKhVRaWqrz589r\nxowZnt3JjMhyy8vL04ULFyRJ7e3tysnJecMn8tbDhw9VVlam7777ToWFhdbzTp8+rcbGRknS2LFj\nFQgErN4KHz16VD/88IOampqUm5ur+vp6paenW8s7fvy4du3aJUnq7u5WNBq1mjd79mxdvHhRxhh1\nd3fr+fPnSktLs5YnSVevXtUHH3xgNUOSJkyYMPhgZuLEiRoYGHjtUrpXbty4odmzZ6upqUkFBQWa\nOnWqZ9cekePQ/Pnz1dbWpqKiIhljVFdX96aP5KmGhgY9ffpU+/fv1/79+yVJBw8e/MdunFcWLFig\nyspKlZSUaGBgQFVVVRozZoyVrDehsLBQlZWVKi4uViAQUF1dndVJ/5NPPtHVq1dVWFgoY4yqq6ut\nv296584dZWRkWM2QpFWrVqmqqkpfffWV+vv79e2332rcuHHW8rKysrRnzx4dOnRI48eP144dOzy7\nNrulAJw0Im9LAWC4KDcATqLcADiJcgPgJMoNgJMoNwBOotwAOIlyA+CkvwHhkgHORuxDBAAAAABJ\nRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x2327a502b38>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "discounted_epr_cent = discounted_epr - np.mean(discounted_epr)\n",
    "discounted_epr_norm = discounted_epr_cent/np.std(discounted_epr_cent)\n",
    "f, ax = plt.subplots(1, figsize=(5,2))\n",
    "sns.barplot(list(range(10)), discounted_epr_norm, color=\"steelblue\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x23279db20f0>"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAATcAAADBCAYAAACqnveCAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAEQBJREFUeJzt3W1IlNnDBvBrUtRsNIkMP5imgfwl\nWdqMpQ3thUxr6W1LSRNFjEgzrDZcddZEVtNkW9gMXDVwWyb512b28kkxEkpDcCNFwVo0g9QUS8Gd\nUmccz/Nh0aee6qm9u89kx+v3bWb0vo6jXJ5zv8xtEEIIEBEpZt6nHgARkQwsNyJSEsuNiJTEciMi\nJbHciEhJzo4IGR8fR0dHB7y9veHk5OSISCKaA+x2O4aGhhASEgI3N7fXXtNUbjabDSaTCX19fbBa\nrUhNTcWmTZve+fUdHR2Ij4/XEkVE9F5VVVVYvXr1a89pKrcbN27Ay8sLP/30E0ZGRvDtt9/+v+Xm\n7e09MwAfHx8tkUREbxgYGEB8fPxMx7xKU7lt2bIFUVFRM4/ft9Scft3Hxwe+vr5aIomI3ultHaSp\n3BYsWAAAsFgsSE9Px9GjRz/4e2N+/K+WyA9yOTduzuUR0dtpPqDw9OlTpKWlYd++fdi+fbueY6J/\nQZXyZnGT3jSV27Nnz5CcnIzc3Fx8/fXXeo+JaAbLlLTSdJ5bWVkZRkdHUVpaioSEBCQkJGB8fFzv\nsRERaaZp5paTk4OcnBy9x0JEpBuHnMRL9LngMlgdvPyKiJTEciMiJbHciEhJ3OdG9AlxH588nLkR\nkZJYbkSkJJYbESmJ5UZESmK5EZGSWG5EpCSWGxEpieVGREriSbxEc8hcOmmYMzciUhLLjYiUxHIj\nIiWx3IhISSw3IlISy42IlMRyIyIlsdyISEksNyJSEsuNiJTEciMiJbHciEhJLDciUhLLjYiU9FHl\n1tbWhoSEBL3GQkSkG82f53bu3DncuHED8+fP13M8RES60Dxz8/Pzw9mzZ/UcCxGRbjTP3KKiotDb\n26vnWIhIMZ/yk395QIGIlMRyIyIlsdyISEkfVW6+vr74448/9BoLEZFuOHMjIiWx3IhISSw3IlIS\ny42IlMRyIyIlsdyISEksNyJSEsuNiJTEciMiJbHciEhJLDciUhLLjYiUxHIjIiWx3IhISSw3IlIS\ny42IlMRyIyIlsdyISEksNyJSEsuNiJTEciMiJbHciEhJLDciUhLLjYiUxHIjIiWx3IhISSw3IlKS\ns9ZvnJqaQl5eHh4+fAgXFxcUFBTA399fz7EREWmmeeZ28+ZNWK1WXLp0CcePH8epU6f0HBcR0UfR\nPHO7d+8ewsPDAQArV65ER0fHO7/WbrcDAAYGBmC1jGiNfK/e3t43nmPe55H3tizmMe99eQMDAwD+\nt2NeZRBCCC0b/+GHHxAZGYn169cDADZs2ICbN2/C2fnNvvzzzz8RHx+vJYaI6L2qqqqwevXq157T\nPHMzGo148eLFzOOpqam3FhsAhISEoKqqCt7e3nByctIaSUT0GrvdjqGhIYSEhLzxmuZyW7VqFRoa\nGvDNN9+gtbUVQUFB7/xaNze3N1qViEgP7zqQqXlZOn209K+//oIQAoWFhVi+fPlHDZKISC+ay42I\naDbjSbxEpCSWGxEpSfMBBZk+1dUPbW1tOH36NMxms9Qcm80Gk8mEvr4+WK1WpKamYtOmTdLy7HY7\ncnJy0NPTAycnJxQVFcHPz09a3rTnz59j9+7dqKyslL4/dteuXfDw8AAA+Pr6oqioSGpeeXk5bt26\nBZvNhri4OMTExEjLqqmpwdWrVwEAExMT6OzsRFNTEzw9PXXPstlsyMrKQl9fH+bNm4f8/Hypvzur\n1Yrs7Gw8efIERqMRubm5WLZsmT4bF7NQXV2dyMzMFEIIcf/+fZGSkiI9s6KiQmzbtk3ExMRIz6qu\nrhYFBQVCCCGGh4fF+vXrpebV19eLrKwsIYQQzc3NDnk/rVarOHTokIiMjBRdXV1Ss8bHx8XOnTul\nZryqublZHDx4UNjtdmGxWERJSYnDsvPy8sTFixelbb++vl6kp6cLIYRobGwUhw8flpYlhBBms1nk\n5OQIIYTo7u4WycnJum17Vi5L/83VD3rx8/PD2bNnpecAwJYtW3DkyJGZx7LP/YuIiEB+fj4AoL+/\nH4sXL5aaBwDFxcWIjY3FkiVLpGc9ePAAY2NjSE5ORmJiIlpbW6XmNTY2IigoCGlpaUhJScGGDRuk\n5k1rb29HV1cX9u7dKy0jICAAdrsdU1NTsFgs7zx3VS9dXV1Yt24dACAwMBDd3d26bXtWLkstFguM\nRuPMYycnJ0xOTkp9o6Oiot55CYneFixYAOCfnzM9PR1Hjx6Vnuns7IzMzEzU19ejpKREalZNTQ0W\nLVqE8PBwVFRUSM0C/jmPcv/+/YiJicHjx49x4MAB1NbWSvt7GRkZQX9/P8rKytDb24vU1FTU1tbC\nYDBIyZtWXl6OtLQ0qRnu7u7o6+vD1q1bMTIygrKyMql5wcHBaGhoQEREBNra2jA4OAi73a7LP/xZ\nOXP7N1c/fK6ePn2KxMRE7Ny5E9u3b3dIZnFxMerq6nDixAm8fPlSWs6VK1dw9+5dJCQkoLOzE5mZ\nmRgaGpKWFxAQgB07dsBgMCAgIABeXl5S87y8vBAWFgYXFxcEBgbC1dUVw8PD0vIAYHR0FI8ePcKa\nNWuk5pw/fx5hYWGoq6vD9evXkZWVhYmJCWl5e/bsgdFoRGJiIhoaGrBixQrdVjKzstxWrVqF27dv\nA8B7r374HD179gzJycnIyMhAdHS09Lxr166hvLwcADB//nwYDAapS+GqqipcuHABZrMZwcHBKC4u\nhre3t7S86urqmU+lGRwchMVikZoXGhqKO3fuQAiBwcFBjI2NwcvLS1oeALS0tGDt2rVSMwDA09Nz\n5sDMwoULMTk5+daL0vXS3t6O0NBQmM1mREREYOnSpbpte1ZOhzZv3oympibExsbOXP2gkrKyMoyO\njqK0tBSlpaUAgHPnzsHNzU1KXmRkJLKzsxEfH4/JyUmYTCa4urpKyfoUoqOjkZ2djbi4OBgMBhQW\nFkqd6W/cuBEtLS2Ijo6GEAK5ubnS95v29PTA19dXagYAJCUlwWQyYd++fbDZbDh27Bjc3d2l5fn7\n++PMmTOorKyEh4cHTp48qdu2HXKFwvj4ODo6OnjhPBHp6tUL5//v5MAhM7eOjg5+5BERSaPrRx79\nG9P7P6qqquDj4+OISCKaAwYGBhAfH//WfawOKbfppaiPj49D9hsQ0dzytt1dDj+gEPPjf6Vt+3Ju\n3JzLI6K3m5VHS+nDsUyJ3o7lRv+KrDJlkZLeZuVJvEREH4szN5rVOFMkrThzIyIlsdyISEksNyJS\nEve5Eb2C+/jUwZkbESlJ88ztU93EhYjoQ2ieud28eRNWqxWXLl3C8ePHZz4skIhoNtBcbp/iJi5E\nRB9K87JU601cHL1jlXnMm815jj6AoXreqzTP3ObCTVyI6POludxUv4kLEX3eNE+1VL+JCxF93jSX\n27x58/Djjz/qORaiOYcn98rDk3iJSEksNyJSEsuNiJTEciMiJbHciEhJLDciUhLLjYiUxHIjIiWx\n3IhISSw3IlISy42IlMRyIyIlsdyISEksNyJSEsuNiJTEciMiJfGmB0RzyFz6cEzO3IhISSw3IlIS\ny42IlMRyIyIlsdyISEksNyJSEsuNiJTEciMiJbHciEhJLDciUhLLjYiUxHIjIiWx3IhISSw3IlIS\ny42IlMRyIyIlsdyISEmaPon377//RkZGBiwWC2w2G7KysvDll1/qPTYiIs00ldtvv/2GNWvWICkp\nCY8ePcLx48dx9epVvcdGRKSZpnJLSkqCi4sLAMBut8PV1VXXQRERfaz3ltvly5fx+++/v/ZcYWEh\nvvjiCwwNDSEjIwMmk0naAImItHhvucXExCAmJuaN5x8+fIjvvvsO33//Pb766ispgyMi0krTsrSr\nqwtHjhzBL7/8gv/85z96j4mI6KNpKreff/4ZVqsVJ0+eBAAYjUb8+uuvug6MiOhjaCo3FhkRzXY8\niZeIlMRyIyIlsdyISEksNyJSEsuNiJTEciMiJbHciEhJms5zIyL6EJdz4z5ZNmduRKQklhsRKckh\ny1K73Q4AGBgYcEQcEc0R050y3TGvcki5DQ0NAQDi4+MdEUdEc8zQ0BD8/f1fe84ghBCyg8fHx9HR\n0QFvb284OTnJjiOiOcJut2NoaAghISFwc3N77TWHlBsRkaPxgAIRKYnlRkRKYrkRkZJYbkSkpFl5\n+dXU1BTy8vLw8OFDuLi4oKCg4I3DvDK0tbXh9OnTMJvNUnNsNhtMJhP6+vpgtVqRmpqKTZs2Scuz\n2+3IyclBT08PnJycUFRUBD8/P2l5054/f47du3ejsrISy5cvl5q1a9cueHh4AAB8fX1RVFQkNa+8\nvBy3bt2CzWZDXFzcW+8Qp5eampqZm55PTEygs7MTTU1N8PT01D3LZrMhKysLfX19mDdvHvLz86X+\n7qxWK7Kzs/HkyRMYjUbk5uZi2bJl+mxczEJ1dXUiMzNTCCHE/fv3RUpKivTMiooKsW3bNhETEyM9\nq7q6WhQUFAghhBgeHhbr16+XmldfXy+ysrKEEEI0Nzc75P20Wq3i0KFDIjIyUnR1dUnNGh8fFzt3\n7pSa8arm5mZx8OBBYbfbhcViESUlJQ7LzsvLExcvXpS2/fr6epGeni6EEKKxsVEcPnxYWpYQQpjN\nZpGTkyOEEKK7u1skJyfrtu1ZuSy9d+8ewsPDAQArV65ER0eH9Ew/Pz+cPXtWeg4AbNmyBUeOHJl5\nLPvcv4iICOTn5wMA+vv7sXjxYql5AFBcXIzY2FgsWbJEetaDBw8wNjaG5ORkJCYmorW1VWpeY2Mj\ngoKCkJaWhpSUFGzYsEFq3rT29nZ0dXVh79690jICAgJgt9sxNTUFi8UCZ2e5i7uuri6sW7cOABAY\nGIju7m7dtj0rl6UWiwVGo3HmsZOTEyYnJ6W+0VFRUejt7ZW2/VctWLAAwD8/Z3p6Oo4ePSo909nZ\nGZmZmaivr0dJSYnUrJqaGixatAjh4eGoqKiQmgUAbm5u2L9/P2JiYvD48WMcOHAAtbW10v5eRkZG\n0N/fj7KyMvT29iI1NRW1tbUwGAxS8qaVl5cjLS1Naoa7uzv6+vqwdetWjIyMoKysTGpecHAwGhoa\nEBERgba2NgwODsJut+vyD39WztyMRiNevHgx83hqakr6fxBHe/r0KRITE7Fz505s377dIZnFxcWo\nq6vDiRMn8PLlS2k5V65cwd27d5GQkIDOzk5kZmbOXIInQ0BAAHbs2AGDwYCAgAB4eXlJzfPy8kJY\nWBhcXFwQGBgIV1dXDA8PS8sDgNHRUTx69Ahr1qyRmnP+/HmEhYWhrq4O169fR1ZWFiYmJqTl7dmz\nB0ajEYmJiWhoaMCKFSt0W8nMynJbtWoVbt++DQBobW1FUFDQJx6Rvp49e4bk5GRkZGQgOjpaet61\na9dQXl4OAJg/fz4MBoPUpXBVVRUuXLgAs9mM4OBgFBcXw9vbW1pedXU1Tp06BQAYHByExWKRmhca\nGoo7d+5ACIHBwUGMjY3By8tLWh4AtLS0YO3atVIzAMDT03PmwMzChQsxOTn51ovS9dLe3o7Q0FCY\nzWZERERg6dKlum17Vk6HNm/ejKamJsTGxkIIgcLCwk89JF2VlZVhdHQUpaWlKC0tBQCcO3fujWvj\n9BIZGYns7GzEx8djcnISJpMJrq6uUrI+hejoaGRnZyMuLg4GgwGFhYVSZ/obN25ES0sLoqOjIYRA\nbm6u9P2mPT098PX1lZoBAElJSTCZTNi3bx9sNhuOHTsGd3d3aXn+/v44c+YMKisr4eHhgZMnT+q2\nbV5bSkRKmpXLUiKij8VyIyIlsdyISEksNyJSEsuNiJTEciMiJbHciEhJLDciUtL/ADahaAamiQxl\nAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x2327a4ebbe0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "discounted_epr = discount_rewards(np.ones(10), gamma=0.5)\n",
    "discounted_epr_cent = discounted_epr - np.mean(discounted_epr)\n",
    "discounted_epr_norm = discounted_epr_cent/np.std(discounted_epr_cent)\n",
    "f, ax = plt.subplots(2, figsize=(5,3))\n",
    "sns.barplot(list(range(10)), discounted_epr, color=\"steelblue\", ax=ax[0])\n",
    "sns.barplot(list(range(10)), discounted_epr_norm, color=\"steelblue\", ax=ax[1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Model: Policy Gradient\n",
    "\n",
    "$$\n",
    "l_1 = relu( x W_1 + b_1) \\\\\n",
    "l_2 = l_1 W_2 + b_2 \\\\\n",
    "\\pi(a|s) = sigmoid(l_2)\n",
    "$$\n",
    "\n",
    "Note: in policy gradient approach, the output of the dense layer is mapped into to a 0-1 range via the sigmoid function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "TOTAL_EPISODES = 2000 if isFast else 10000\n",
    "\n",
    "D = 4  # input dimensionality\n",
    "H = 10 # number of hidden layer neurons\n",
    "\n",
    "observations = C.sequence.input_variable(STATE_COUNT, np.float32, name=\"obs\")\n",
    "\n",
    "W1 = C.parameter(shape=(STATE_COUNT, H), init=C.glorot_uniform(), name=\"W1\")\n",
    "b1 = C.parameter(shape=H, name=\"b1\")\n",
    "layer1 = C.relu(C.times(observations, W1) + b1)\n",
    "\n",
    "W2 = C.parameter(shape=(H, ACTION_COUNT), init=C.glorot_uniform(), name=\"W2\")\n",
    "b2 = C.parameter(shape=ACTION_COUNT, name=\"b2\")\n",
    "score = C.times(layer1, W2) + b2\n",
    "# Until here it was similar to DQN\n",
    "\n",
    "probability = C.sigmoid(score, name=\"prob\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Running the PG model\n",
    "\n",
    "**Policy Search**: The optimal policy search can be carried out with either gradient free approaches or by computing gradients over the policy space ($\\pi_\\theta$) which is parameterized by $\\theta$. In this tutorial, we use the classic forward (`loss.forward`) and back (`loss.backward`) propagation of errors over the parameterized space $\\theta$. In this case, $\\theta = \\{W_1, b_1, W_2, b_2\\}$, our model parameters.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Episode: 20. Average reward for episode 24.550000.\n",
      "Episode: 40. Average reward for episode 18.700000.\n",
      "Episode: 60. Average reward for episode 20.750000.\n",
      "Episode: 80. Average reward for episode 19.300000.\n",
      "Episode: 100. Average reward for episode 19.750000.\n",
      "Episode: 120. Average reward for episode 20.850000.\n",
      "Episode: 140. Average reward for episode 20.150000.\n",
      "Episode: 160. Average reward for episode 19.650000.\n",
      "Episode: 180. Average reward for episode 15.150000.\n",
      "Episode: 200. Average reward for episode 20.700000.\n",
      "Episode: 220. Average reward for episode 18.500000.\n",
      "Episode: 240. Average reward for episode 20.350000.\n",
      "Episode: 260. Average reward for episode 17.700000.\n",
      "Episode: 280. Average reward for episode 15.900000.\n",
      "Episode: 300. Average reward for episode 21.700000.\n",
      "Episode: 320. Average reward for episode 17.900000.\n",
      "Episode: 340. Average reward for episode 17.850000.\n",
      "Episode: 360. Average reward for episode 16.050000.\n",
      "Episode: 380. Average reward for episode 19.450000.\n",
      "Episode: 400. Average reward for episode 19.150000.\n",
      "Episode: 420. Average reward for episode 18.950000.\n",
      "Episode: 440. Average reward for episode 19.400000.\n",
      "Episode: 460. Average reward for episode 21.550000.\n",
      "Episode: 480. Average reward for episode 19.850000.\n",
      "Episode: 500. Average reward for episode 17.500000.\n",
      "Episode: 520. Average reward for episode 18.350000.\n",
      "Episode: 540. Average reward for episode 23.050000.\n",
      "Episode: 560. Average reward for episode 21.400000.\n",
      "Episode: 580. Average reward for episode 25.350000.\n",
      "Episode: 600. Average reward for episode 22.000000.\n",
      "Episode: 620. Average reward for episode 22.700000.\n",
      "Episode: 640. Average reward for episode 21.200000.\n",
      "Episode: 660. Average reward for episode 24.050000.\n",
      "Episode: 680. Average reward for episode 21.100000.\n",
      "Episode: 700. Average reward for episode 22.500000.\n",
      "Episode: 720. Average reward for episode 25.350000.\n",
      "Episode: 740. Average reward for episode 24.400000.\n",
      "Episode: 760. Average reward for episode 27.850000.\n",
      "Episode: 780. Average reward for episode 27.200000.\n",
      "Episode: 800. Average reward for episode 23.250000.\n",
      "Episode: 820. Average reward for episode 26.350000.\n",
      "Episode: 840. Average reward for episode 24.800000.\n",
      "Episode: 860. Average reward for episode 26.350000.\n",
      "Episode: 880. Average reward for episode 26.450000.\n",
      "Episode: 900. Average reward for episode 21.750000.\n",
      "Episode: 920. Average reward for episode 22.500000.\n",
      "Episode: 940. Average reward for episode 26.350000.\n",
      "Episode: 960. Average reward for episode 25.500000.\n",
      "Episode: 980. Average reward for episode 32.050000.\n",
      "Task solved in: 980 \n"
     ]
    }
   ],
   "source": [
    "input_y = C.sequence.input_variable(1, np.float32, name=\"input_y\")\n",
    "advantages = C.sequence.input_variable(1, np.float32, name=\"advt\")\n",
    "\n",
    "loss = -C.reduce_mean(C.log(C.square(input_y - probability) + 1e-4) * advantages, axis=0, name='loss')\n",
    "\n",
    "lr = 0.001\n",
    "lr_schedule = C.learning_parameter_schedule_per_sample(lr) \n",
    "sgd = C.sgd([W1, W2], lr_schedule)\n",
    "\n",
    "gradBuffer = dict((var.name, np.zeros(shape=var.shape)) for var in loss.parameters if var.name in ['W1', 'W2', 'b1', 'b2'])\n",
    "\n",
    "xs, hs, label, drs = [], [], [], []\n",
    "running_reward = None\n",
    "reward_sum = 0\n",
    "episode_number = 1\n",
    "\n",
    "observation = env.reset()\n",
    "\n",
    "while episode_number <= TOTAL_EPISODES:\n",
    "    x = np.reshape(observation, [1, STATE_COUNT]).astype(np.float32)\n",
    "\n",
    "    # Run the policy network and get an action to take.\n",
    "    prob = probability.eval(arguments={observations: x})[0][0][0]\n",
    "    action = 1 if np.random.uniform() < prob else 0\n",
    "\n",
    "    xs.append(x)  # observation\n",
    "    # grad that encourages the action that was taken to be taken\n",
    "\n",
    "    y = 1 if action == 0 else 0  # a \"fake label\"\n",
    "    label.append(y)\n",
    "\n",
    "    # step the environment and get new measurements\n",
    "    observation, reward, done, info = env.step(action)\n",
    "    reward_sum += float(reward)\n",
    "\n",
    "    # Record reward (has to be done after we call step() to get reward for previous action)\n",
    "    drs.append(float(reward))\n",
    "\n",
    "    if done:\n",
    "        # Stack together all inputs, hidden states, action gradients, and rewards for this episode\n",
    "        epx = np.vstack(xs)\n",
    "        epl = np.vstack(label).astype(np.float32)\n",
    "        epr = np.vstack(drs).astype(np.float32)\n",
    "        xs, label, drs = [], [], []  # reset array memory\n",
    "\n",
    "        # Compute the discounted reward backwards through time.\n",
    "        discounted_epr = discount_rewards(epr)\n",
    "        # Size the rewards to be unit normal (helps control the gradient estimator variance)\n",
    "        discounted_epr -= np.mean(discounted_epr)\n",
    "        discounted_epr /= np.std(discounted_epr)\n",
    "\n",
    "        # Forward pass\n",
    "        arguments = {observations: epx, input_y: epl, advantages: discounted_epr}\n",
    "        state, outputs_map = loss.forward(arguments, outputs=loss.outputs,\n",
    "                                          keep_for_backward=loss.outputs)\n",
    "\n",
    "        # Backward psas\n",
    "        root_gradients = {v: np.ones_like(o) for v, o in outputs_map.items()}\n",
    "        vargrads_map = loss.backward(state, root_gradients, variables=set([W1, W2]))\n",
    "\n",
    "        for var, grad in vargrads_map.items():\n",
    "            gradBuffer[var.name] += grad\n",
    "\n",
    "        # Wait for some batches to finish to reduce noise\n",
    "        if episode_number % BATCH_SIZE_BASELINE == 0:\n",
    "            grads = {W1: gradBuffer['W1'].astype(np.float32),\n",
    "                     W2: gradBuffer['W2'].astype(np.float32)}\n",
    "            updated = sgd.update(grads, BATCH_SIZE_BASELINE)\n",
    "\n",
    "            # reset the gradBuffer\n",
    "            gradBuffer = dict((var.name, np.zeros(shape=var.shape))\n",
    "                              for var in loss.parameters if var.name in ['W1', 'W2', 'b1', 'b2'])\n",
    "\n",
    "            print('Episode: %d. Average reward for episode %f.' % (episode_number, reward_sum / BATCH_SIZE_BASELINE))\n",
    "\n",
    "            if reward_sum / BATCH_SIZE_BASELINE > REWARD_TARGET:\n",
    "                print('Task solved in: %d ' % episode_number)\n",
    "                break\n",
    "\n",
    "            reward_sum = 0\n",
    "\n",
    "        observation = env.reset()  # reset env\n",
    "        episode_number += 1\n",
    "probability.save('pg.mod')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Solutions**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "((4, 10), (10,), (10, 2), (2,), (2,))"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "observation = C.sequence.input_variable(STATE_COUNT, np.float32, name=\"s\")\n",
    "\n",
    "W1 = C.parameter(shape=(STATE_COUNT, H), init=C.glorot_uniform(), name=\"W1\")\n",
    "b1 = C.parameter(shape=H, name=\"b1\")\n",
    "layer1 = C.relu(C.times(observation, W1) + b1)\n",
    "\n",
    "W2 = C.parameter(shape=(H, ACTION_COUNT), init=C.glorot_uniform(), name=\"W2\")\n",
    "b2 = C.parameter(shape=ACTION_COUNT, name=\"b2\")\n",
    "model = C.times(layer1, W2) + b2\n",
    "W1.shape, b1.shape, W2.shape, b2.shape, model.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "((4, 10), (10,), (10, 2), (2,), (2,))"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Correspoding layers implementation - Preferred solution\n",
    "def create_model(input):\n",
    "    with C.layers.default_options(init=C.glorot_uniform()):\n",
    "        z = C.layers.Sequential([C.layers.Dense(H, name=\"layer1\"),\n",
    "                                 C.layers.Dense(ACTION_COUNT, name=\"layer2\")])\n",
    "        return z(input)\n",
    "    \n",
    "model = create_model(observation)\n",
    "\n",
    "model.layer1.W.shape, model.layer1.b.shape, model.layer2.W.shape, model.layer2.b.shape, model.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}