Merge branch 'master' of https://github.com/microsoft/tensorwatch

ISSUE_TEMPLATE.md

@@ -1,18 +0,0 @@
-# Read This First
-
-## If you are reporting a bug
-* Make sure to write **all reproduction steps**
-* Include full error message in text form
-* Search issues for error message before filing issue
-* Attach screenshot if applicable
-* Include code to run if applicable
-
-## If you have question
-* Add clear and concise title
-* Add OS, TensorWatch version, Python version if applicable
-* Include context on what you are trying to achieve
-* Include details of what you already did to find answers
-
-**What's better than filing issue? Filing a pull request :).**
-
------------------------------------- (Remove above before filing the issue) ------------------------------------

README.md

@@ -1,8 +1,10 @@
 # Welcome to TensorWatch
 
-TensorWatch is a debugging and visualization tool designed for deep learning and reinforcement learning. It fully leverages Jupyter Notebook to show real time visualizations and offers unique capabilities to query the live training process without having to sprinkle logging statements all over. You can also use TensorWatch to build your own UIs and dashboards. In addition, TensorWatch leverages several excellent libraries for visualizing model graph, review model statistics, explain prediction and so on.
+TensorWatch is a debugging and visualization tool designed for deep learning and reinforcement learning from Microsoft Research. It works in Jupyter Notebook to show real-time visualizations of your machine learning training and perform several other key visualizations of your models and data. 
 
-TensorWatch is under heavy development with a goal of providing a research platform for debugging machine learning in one easy to use, extensible and hackable package.
+TensorWatch is designed to be flexible and extensible so you can even build your own custom visualizations, UIs, and dashboards. Besides traditional what-you-see-is-what-you-log approach, it also has a unique capability to execute arbitrary query against your live ML training process, return a stream as a result of the query and view this stream using your choice of a visualizer (we call this [Lazy Logging Mode](#lazy-logging-mode])).
+
+TensorWatch is under heavy development with a goal of providing a platform for debugging machine learning in one easy to use, extensible, and hackable package.
 
 <img src="docs/images/teaser.gif" alt="TensorWatch in Jupyter Notebook" width="400"/>
 
@@ -18,45 +20,64 @@ TensorWatch supports Python 3.x and is tested with PyTorch 0.4-1.x. Most feature
 
 ### Quick Start
 
-Here's simple code that logs some metric every second:
+Here's simple code that logs an integer and its square as a tuple every second to TensorWatch:
 ```
 import tensorwatch as tw
 import time
 
+# streams will be stored in test.log file
 w = tw.Watcher(filename='test.log')
-s = w.create_stream(name='my_metric')
+
+# create a stream for logging
+s = w.create_stream(name='metric1')
+
+# generate Jupyter Notebook to view real-time streams
 w.make_notebook()
 
 for i in range(1000):
+    # write x,y pair we want to log
     s.write((i, i*i)) 
+
     time.sleep(1)
 ```
-When you run this code you will notice a Jupyter Notebook file `test.ipynb` gets created in your script folder. From command prompt type `jupyter notebook` and click on `test.ipynb`. Click on Run button to see live line graph as values gets written in your script.
+When you run this code, you will notice a Jupyter Notebook file `test.ipynb` gets created in your script folder. From command prompt type `jupyter notebook` and click on `test.ipynb`. Chose *Cell > Run all* in the menu to see real-time line graph as values get written in your script.
 
-Here's the output in Jupyter Notebook:
+Here's the output you will see in Jupyter Notebook:
 
 <img src="docs/images/quick_start.gif" alt="TensorWatch in Jupyter Notebook" width="250"/>
 
-Please [Tutorials](#tutorials) and [notebooks](https://github.com/microsoft/tensorwatch/tree/master/notebooks) for more information.
+To dive deeper in to various other features, please see [Tutorials](#tutorials) and [notebooks](https://github.com/microsoft/tensorwatch/tree/master/notebooks).
+
+### How does this work?
+
+When you write to TensorWatch stream, the values get serialized and sent to TCP/IP socket as well as the file you specified. From Jupyter Notebook, we load previously logged values from the file and then listen to TCP/IP socket for any future values. The visualier listens to the stream and renders values as it arrives.
+
+Ok, so that's a bit of a very simplified description. TensorWatch architecture is actually much more powerful. Almost everything in TensorWatch is a *stream*. Files, sockets, console and even visualizers themselves are stream. A cool thing about TensorWatch streams is that they can listen to other streams. This allows TensorWatch to create a *data flow graph*. This means a visualizer can listen to many streams simultaneously, each of which could be file, socket or some other stream. You can recursively extend this to build arbitrary data flow graphs. TensorWatch decouples streams from how they get stored and how they get visualized.
+
 
 ## Visualizations
-In above example, the line graph is used as default visualization. However TensorWatch supports many other visualizations including histogram, pie charts, scatter charts, bar charts and 3D versions of many of these plots. You can simply log your data, specify the chart type you had like to visualize and let TensorWatch take care of the rest. You can also easily create custom visualizations specific to your data.
+In the above example, the line graph is used as the default visualization. However, TensorWatch supports many other visualizations including histogram, pie charts, scatter charts, bar charts and 3D versions of many of these plots. You can log your data, specify the chart type you want and let TensorWatch take care of the rest. 
+
+One of the significant strengths of TensorWatch is to be able to combine, compose, and create custom visualizations effortlessly. For example, you can choose an arbitrary number of streams to get visualized in the same plot. Or you can have the same stream get visualized in many different plots  *simultaneously*. Or you can place an arbitrary set of visualizations side-by-side. Or you can create your own custom visualization simply by creating a new Python class that implements few methods.
+
+## Comparing Results of Multiple Runs
+Each TensorWatch stream may contain a metric of your choice. By default, TensorWatch saves all streams in a single file, but you could also choose to save each stream in seprate file or not save at all (for example, stream over sockets or in console, zero hit to disk!). Later you can open these streams and direct them to one or more visualizations. This design allows you to quickly compare the results from your different experiments in your choice of visualizations easily. 
 
 ## Training within Jupyter Notebook
-Many times you might prefer to do data analysis, ML training and testing from within Jupyter Notebook instead of from a separate script. TensorWatch can help you do sophisticated visualizations effortlessly for your code running within Jupyter Notebook.
+Many times you might prefer to do data analysis, ML training, and testing - all from within Jupyter Notebook instead of from a separate script. TensorWatch can help you do sophisticated real-time visualizations effortlessly from code that runs within Jupyter Notebook end-to-end.
 
-## Querying the Process (Lazy Log Mode)
-One of the unique capability TensorWatch offers is to be able to query the live running training process, retrieve the result of this query as stream and direct this stream to your preferred visualization - all of these without explicitly logging any data before hand! We call this new way of debugging *lazy logging mode*.
+## Lazy Logging Mode
+An unique feature in TensorWatch is the ability to query the live running process, retrieve the result of this query as a stream and direct this stream to your preferred visualization(s). You don't need to log any data beforehand. We call this new way of debugging and visualization a *lazy logging mode*.
 
-For example, below we show input and output image pairs randomly sampled during the training of an autoencoder on fruits dataset. These images were not logged, instead stream of these sampled images were returned on the fly as a response to the user query:
+For example, below, we show input and output image pairs randomly sampled during the training of an autoencoder on fruits dataset. These images were not logged beforehand in the script. Instead, the user sends query as Python lambda expression which results in a stream of images that gets displayed in Jupyter Notebook:
 
 <img src="docs/images/fruits.gif" alt="TensorWatch in Jupyter Notebook" width="200"/>
 
-See [Lazy Logging Tutorial](https://github.com/microsoft/tensorwatch/blob/master/docs/lazy_logging.md)
+See [Lazy Logging Tutorial](https://github.com/microsoft/tensorwatch/blob/master/docs/lazy_logging.md).
 
-### Pre-Training and Post-Training
+### Pre-Training and Post-Training Tasks
 
-TensorWatch builds on several excellent libraries including [hiddenlayer](https://github.com/waleedka/hiddenlayer), [torchstat](https://github.com/Swall0w/torchstat), [Visual Attribution](https://github.com/yulongwang12/visual-attribution) to allow performing debugging and analysis activities in one consistent package and interface.
+TensorWatch leverages several excellent libraries including [hiddenlayer](https://github.com/waleedka/hiddenlayer), [torchstat](https://github.com/Swall0w/torchstat), [Visual Attribution](https://github.com/yulongwang12/visual-attribution) to allow performing usual debugging and analysis activities in one consistent package and interface.
 
 For example, you can view the model graph with tensor shapes with one liner:
 
@@ -66,36 +87,40 @@ You can view statistics for different layers such as flops, number of parameters
 
 <img src="docs/images/model_stats.png" alt="Model statistics for Alexnet" width="600"/>
 
-[See notebook](https://github.com/microsoft/tensorwatch/blob/master/notebooks/network_arch.ipynb)
+[See notebook](https://github.com/microsoft/tensorwatch/blob/master/notebooks/network_arch.ipynb).
 
 You can view dataset in lower dimensional space using techniques such as t-sne:
 
 <img src="docs/images/tsne.gif" alt="t-sne visualization for MNIST" width="400"/>
 
-[See notebook](https://github.com/microsoft/tensorwatch/blob/master/notebooks/data_exploration.ipynb)
+[See notebook](https://github.com/microsoft/tensorwatch/blob/master/notebooks/data_exploration.ipynb).
 
 ### Prediction Explanations
-We have a goal to provide various tools for explaining predictions over time to help debugging models. Currently we offer several explainers for convolutional networks including [Lime](https://github.com/marcotcr/lime). For example, below highlights the areas that causes Resnet50 model to make prediction for class 240:
+We wish to provide various tools for explaining predictions to help debug models. Currently, we offer several explainers for convolutional networks, including [Lime](https://github.com/marcotcr/lime). For example, below highlights the areas that cause Resnet50 model to make a prediction for class 240 for Imagenet dataset:
 
 <img src="docs/images/saliency.png" alt="CNN prediction explanation" width="300"/>
 
-[See notebook](https://github.com/microsoft/tensorwatch/blob/master/notebooks/cnn_pred_explain.ipynb)
+[See notebook](https://github.com/microsoft/tensorwatch/blob/master/notebooks/cnn_pred_explain.ipynb).
 
 
 ## Tutorials
 
-- [Simple Logging Tutorial](https://github.com/microsoft/tensorwatch/blob/master/docs/lazy_logging.md)
+- [Simple Logging Tutorial](https://github.com/microsoft/tensorwatch/blob/master/docs/simple_logging.md)
 
 - [Lazy Logging Tutorial](https://github.com/microsoft/tensorwatch/blob/master/docs/lazy_logging.md)
 
 ## Contribute
 
-We would love your contributions, feedback and feature requests! Please [file a Github issue](https://github.com/microsoft/tensorwatch/issues/new) or send us a pull request. Review the [Microsoft Code of Conduct](https://opensource.microsoft.com/codeofconduct/) and [learn more](https://github.com/microsoft/tensorwatch/blob/master/CONTRIBUTING.md).
+We would love your contributions, feedback, questions, and feature requests! Please [file a Github issue](https://github.com/microsoft/tensorwatch/issues/new) or send us a pull request. Please review the [Microsoft Code of Conduct](https://opensource.microsoft.com/codeofconduct/) and [learn more](https://github.com/microsoft/tensorwatch/blob/master/CONTRIBUTING.md).
 
 ## Contact
 
 Join the TensorWatch group on [Facebook](https://www.facebook.com/groups/378075159472803/) to stay up to date or ask any questions.
 
+## Credits
+
+TensorWatch utilizes several open source libraries for many of its features. These includes: [hiddenlayer](https://github.com/waleedka/hiddenlayer), [torchstat](https://github.com/Swall0w/torchstat), [Visual-Attribution](https://github.com/yulongwang12/visual-attribution), [pyzmq](https://github.com/zeromq/pyzmq), [receptivefield](https://github.com/fornaxai/receptivefield), [nbformat](https://github.com/jupyter/nbformat). Please see `install_requires` section in [setup.py](setup.py) for upto date list.
+
 ## License
 
 This project is released under the MIT License. Please review the [License file](LICENSE.txt) for more details.

docs/simple_logging.md

@@ -1,7 +1,40 @@
 
-# Welcome to the Simple Logging Tutorial
+# Simple Logging Tutorial
 
-We would start by running the script [sum_log.py](https://github.com/microsoft/tensorwatch/blob/master/test/simple_log/sum_log.py) available at `test/simple_log` directory. This is just simple code that just sums N numbers. We added couple of TensorWatch statements to log the values we want. Now while this script is running, you can execute this notebook ([download it here](https://github.com/microsoft/tensorwatch/blob/master/notebooks/simple_logging.ipynb)).
+In this tutorial, we will use a simple script that maintains two variables. Our goal is to log the values of these variables using TensorWatch and view it in real-time in variety of ways from Jupyter Notebook demonstrating various features offered by TensorWatch. We would execute this script from a console and its code looks like this:
+
+```
+# available in test/simple_log/sum_log.py
+
+import time, random
+import tensorwatch as tw
+
+# we will create two streams, one for 
+# sums of integers, other for random numbers
+w = tw.Watcher()
+st_isum = w.create_stream('isums')
+st_rsum = w.create_stream('rsums')
+
+isum, rsum = 0, 0
+for i in range(10000):
+    isum += i
+    rsum += random.randint(i)
+
+    # write to streams
+    st_isum.write((i, isum))
+    st_rsum.write((i, rsum))
+
+    time.sleep(1)
+```
+
+## Basic Concepts
+
+The `Watcher` object allows you to create streams. You can then write any values in to these streams. By default, the `Watcher` object opens up TCP/IP sockets so a client can request these streams. As values are being written on one end, they get serialized and sent to any client(s) on the other end. We will use Jupyter Notebook to create a client, open these streams and feed them to various visualizers.
+
+## Create the Client
+You can either create a new Jupyter Notebook or get the existing one in the repo at `notebooks/simple_logging.ipynb`. 
+
+Let's first do imports:
 
 
 ```python
@@ -9,53 +42,61 @@ We would start by running the script [sum_log.py](https://github.com/microsoft/t
 import tensorwatch as tw
 ```
 
-First we create client that would connect to the Watcher running in out script. By default connection is made over TCP/IP sockets. We then specify the name of the log stream that we had created in the script.
-
+Next, open the streams that we had created in above script:
 
 ```python
-client = tw.WatcherClient()
-stream = client.open_stream('my_log')
+cli = tw.WatcherClient()
+st_isum = cli.open_stream('isums')
+st_rsum = cli.open_stream('rsums')
 ```
 
-Now lets visualize this stream in text format.
+Now lets visualize isum stream in textual format:
 
 
 ```python
-text_vis = tw.Visualizer(stream, vis_type='text')
+text_vis = tw.Visualizer(st_isum, vis_type='text')
 text_vis.show()
 ```
 
-<img src="images/simple_logging/text_cell.png" width=150/>
+You should see a growing table in real-time like this, each row is tuple we wrote to the stream:
+
+<img src="images/simple_logging/text_cell.png" width=120/>
 
 
-That worked out good! How about feeding same stream to line chart visualization?
+That worked out good! How about feeding the same stream simultaneously to plot a line graph?
 
 
 ```python
-line_plot = tw.Visualizer(stream, vis_type='line')
+line_plot = tw.Visualizer(st_isum, vis_type='line', xtitle='i', ytitle='isum')
 line_plot.show()
 ```
 
-
 <img src="images/simple_logging/line_cell.png" width=400/>
 
-Ooo... Now we have two simultaneous visualizations running in Jupyter Notebook in real-time fed from same stream! How about adding one more? This time we want to see real-time statistics of the values coming in our stream. Oh, BTW, we had also like to see next to our text visualization above so we set the cell parameter like this:
-
+Let's take step further. Say, we plot rsum as well but, just for fun, in the *same plot* as isum above. That's easy with the optional `host` parameter:
 
 ```python
-summary = tw.Visualizer(stream, vis_type='summary', cell=text_vis)
+line_plot = tw.Visualizer(st_rsum, vis_type='line', host=line_plot, ytitle='rsum')
 ```
+This instantly changes our line plot and a new Y-Axis is automatically added for our second stream. TensorWatch allows you to add multiple streams in same visualizer.
+
+Are you curious about statistics of these two streams? Let's display the live statistics of both streams side by side! To do this, we will use the `cell` parameter for the second visualization to place right next to the first one:
+
+```python
+istats = tw.Visualizer(st_isum, vis_type='summary')
+istats.show()
+
+rstats = tw.Visualizer(st_rsum, vis_type='summary', cell=istats)
+```
+
+The output looks like this (each panel showing statistics for the tuple that was logged in the stream):
 
 <img src="images/simple_logging/text_summary.png" width=300/>
 
+## Next Steps
 
-TensorWatch allows many different frameworks for visualizations. You can build your custom ones also fairly easily. Here let's see our stream visualized using Plotly library:
+We have just scratched the surface of what you can do with TensorWatch. You should check out [Lazy Logging Tutorial](lazy_logging.md) and other [notebooks](https://github.com/microsoft/tensorwatch/tree/master/notebooks) as well.
 
+## Questions?
 
-```python
-plotly_line = tw.Visualizer(stream, vis_type='plotly-line')
-plotly_line.show()
-```
-
-<img src="images/simple_logging/plotly_line.png" width=400/>
-
+File a [Github issue](https://github.com/microsoft/tensorwatch/issues/new) and let us know if we can improve this tutorial.

notebooks/data_exploration.ipynb

@@ -1,5 +1,28 @@
 {
  "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Using TensorWatch for Data Exploration\n",
+    "\n",
+    "In this tutorial, we will show how to quickly use TensorWatch for exploring data using dimention reduction technique called [t-sne](https://en.wikipedia.org/wiki/T-distributed_stochastic_neighbor_embedding). We plan to implement many other techniques in future (please feel to [contribute](https://github.com/microsoft/tensorwatch/blob/master/CONTRIBUTING.md)!)."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Installing regim\n",
+    "This tutorial will use small Python package called `regim`. It has few utility classes to quickly work with PyTorch datasets. To install regim, simply run:\n",
+    "\n",
+    "```\n",
+    "pip install regim\n",
+    "```\n",
+    "\n",
+    "Now we are done with that, let's do our imports:"
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": 1,
@@ -8,42 +31,80 @@
    },
    "outputs": [],
    "source": [
-    "import tensorwatch as tw"
+    "import tensorwatch as tw\n",
+    "from regim import DataUtils"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "First we will get MNIST dataset. The `regim` package has DataUtils class that allows to get entire MNIST dataset without train/test split and reshaping each image as vector of 784 integers instead of 28x28 matrix."
    ]
   },
   {
    "cell_type": "code",
    "execution_count": 2,
-   "metadata": {},
+   "metadata": {
+    "scrolled": true
+   },
    "outputs": [],
    "source": [
-    "from regim import *\n",
-    "ds = DataUtils.mnist_datasets(linearize=True, train_test=False)\n",
-    "ds = DataUtils.sample_by_class(ds, k=500, shuffle=True, as_np=True, no_test=True)"
+    "ds = DataUtils.mnist_datasets(linearize=True, train_test=False)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "MNIST dataset is big enough for t-sne to take long time on slow computers. So we will apply t-sne on sample of this dataset. The `regim` package has utility method that allows us to take `k` random samples for each class. We also set `as_np=True` to convert images to numpy array from PyTorch tensor. The `no_test=True` parameter instructs that we don't want to split our data as train and test. The return value is a tuple of two numpy arrays, one containing input images and other labels."
    ]
   },
   {
    "cell_type": "code",
    "execution_count": 3,
-   "metadata": {
-    "scrolled": false
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
-    "components = tw.get_tsne_components(ds)"
+    "inputs, labels = DataUtils.sample_by_class(ds, k=50, shuffle=True, as_np=True, no_test=True)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "We are now ready to supply this dataset to TensorWatch and in just one line we can get lower dimensional components. The `get_tsne_components` method takes a tuple of input and labels. The optional parameters `features_col=0` and `labels_col=1` tells which member of tuple is input features and truth labels. Another optional parameter `n_components=3` says that we should generate 3 components for each data point."
    ]
   },
   {
    "cell_type": "code",
    "execution_count": 4,
    "metadata": {
-    "scrolled": false
+    "scrolled": true
+   },
+   "outputs": [],
+   "source": [
+    "components = tw.get_tsne_components((inputs, labels))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Now that we have 3D component for each data point in our dataset, let's plot it! For this purpose, we use `ArrayStream` class from TensorWatch that allows you to convert any iterables in to TensorWatch stream. This stream then we supply to Visualizer class asking it to use `tsne` visualization type which is just fency 3D scatter plot."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "metadata": {
+    "scrolled": true
    },
    "outputs": [
     {
      "data": {
       "application/vnd.jupyter.widget-view+json": {
-       "model_id": "0a2f434c08cd4f3483a3d818713a7ad6",
+       "model_id": "f86416164bb04e729ea16cb71a666a70",
        "version_major": 2,
        "version_minor": 0
       },
@@ -60,9 +121,25 @@
    "source": [
     "comp_stream = tw.ArrayStream(components)\n",
     "vis = tw.Visualizer(comp_stream, vis_type='tsne', \n",
-    "                    hover_images=ds[0], hover_image_reshape=(28,28))\n",
+    "                    hover_images=inputs, hover_image_reshape=(28,28))\n",
     "vis.show()"
    ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Notice that as you hover over mouse on each point on graph, you will see MNIST image associated with that point. How does that work? We are supplying input images to `hover_image` parameter. As our images are 1 dimentional array of 784, we also set `hover_image_reshape` parameter to reshape it to 28x28. You can customize hover behavior by attaching new fuction to vis.[hover_fn](https://github.com/microsoft/tensorwatch/blob/master/tensorwatch/plotly/embeddings_plot.py#L28) member."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Questions?\n",
+    "\n",
+    "File a [Github issue](https://github.com/microsoft/tensorwatch/issues/new) and let us know if we can improve this tutorial."
+   ]
   }
  ],
  "metadata": {

setup.py

@@ -8,7 +8,7 @@ with open("README.md", "r") as fh:
 
 setuptools.setup(
     name="tensorwatch",
-    version="0.8.0",
+    version="0.8.1",
     author="Shital Shah",
     author_email="shitals@microsoft.com",
     description="Interactive Realtime Debugging and Visualization for AI",
@@ -23,6 +23,6 @@ setuptools.setup(
         "Operating System :: OS Independent",
     ),
     install_requires=[
-          'matplotlib', 'numpy', 'pyzmq', 'plotly', 'torchstat', 'receptivefield'
+          'matplotlib', 'numpy', 'pyzmq', 'plotly', 'torchstat', 'receptivefield', 'ipywidgets', 'sklearn', 'nbformat'
     ]
 )

tensorwatch.pyproj

@@ -167,7 +167,6 @@
       <SubType>Code</SubType>
     </Content>
     <Content Include="install_jupyterlab.bat" />
-    <Content Include="ISSUE_TEMPLATE.md" />
     <Content Include="LICENSE.TXT">
       <SubType>Code</SubType>
     </Content>

tensorwatch/embeddings/tsne_utils.py

@@ -3,6 +3,7 @@
 
 from sklearn.manifold import TSNE
 from sklearn.preprocessing import StandardScaler
+import numpy as np
 
 def _standardize_data(data, col, whitten, flatten):
     if col is not None:
@@ -29,7 +30,10 @@ def get_tsne_components(data, features_col=0, labels_col=1, whitten=True, n_comp
         labels = data[labels_col]
         for i, item in enumerate(comps):
             # low, high, annotation, text, color
-            item.extend((None, None, None, str(labels[i]), labels[i]))
+            label = labels[i]
+            if isinstance(labels, np.ndarray):
+                label = label.item()
+            item.extend((None, None, None, str(int(label)), label))
         return comps
     return tsne_results
 

tensorwatch/imagenet_utils.py

@@ -3,7 +3,7 @@
 
 from torchvision import transforms
 from . import pytorch_utils
-import json
+import json, os
 
 def get_image_transform():
     transf = transforms.Compose([ #TODO: cache these transforms?
@@ -21,6 +21,9 @@ def get_normalize_transform():
     return transforms.Normalize(mean=[0.485, 0.456, 0.406],
                                     std=[0.229, 0.224, 0.225])   
 
+def image2batch(image):
+    return pytorch_utils.image2batch(image, image_transform=get_image_transform())
+
 def predict(model, images, image_transform=None, device=None):
     logits = pytorch_utils.batch_predict(model, images, 
                                          input_transform=image_transform or get_image_transform(), device=device)
@@ -39,15 +42,18 @@ def probabilities2classes(probs, topk=5):
     top_probs = probs.topk(topk)
     # return (probability, class_id, class_label, class_code)
     return tuple((p,c, labels.index2label_text(c), labels.index2label_code(c)) \
-        for p, c in zip(top_probs[0][0].detach().numpy(), top_probs[1][0].detach().numpy()))
+        for p, c in zip(top_probs[0][0].data.cpu().numpy(), top_probs[1][0].data.cpu().numpy()))
 
 class ImagenetLabels:
-    def __init__(self, json_path='imagenet_class_index.json'):
+    def __init__(self, json_path=None):
         self._idx2label = []
         self._idx2cls = []
         self._cls2label = {}
         self._cls2idx = {}
-        with open(json_path, "r") as read_file:
+
+        json_path = json_path or os.path.join(os.path.dirname(__file__), 'imagenet_class_index.json')
+
+        with open(os.path.abspath(json_path), "r") as read_file:
             class_json = json.load(read_file)
             self._idx2label = [class_json[str(k)][1] for k in range(len(class_json))]
             self._idx2cls = [class_json[str(k)][0] for k in range(len(class_json))]

tensorwatch/pytorch_utils.py

@@ -21,17 +21,20 @@ def tensors2batch(tensors, preprocess_transform=None):
 def int2tensor(val):
     return torch.LongTensor([val])
 
+def image2batch(image, image_transform=None):
+    if image_transform:
+        input_x = image_transform(image)
+    else: # if no transforms supplied then just convert PIL image to tensor
+        input_x = transforms.ToTensor()(image)
+    input_x = input_x.unsqueeze(0) #convert to batch of 1
+    return input_x
+
 def image_class2tensor(image_path, class_index=None, image_convert_mode=None, 
                        image_transform=None):
-    
-    raw_input = image_utils.open_image(os.path.abspath(image_path), convert_mode=image_convert_mode)
-    if image_transform:
-        input_x = image_transform(raw_input)
-    else:
-        input_x = transforms.ToTensor()(raw_input)
-    input_x = input_x.unsqueeze(0) #convert to batch of 1
+    image_pil = image_utils.open_image(os.path.abspath(image_path), convert_mode=image_convert_mode)
+    input_x = image2batch(image_pil, image_transform)
     target_class = int2tensor(class_index) if class_index is not None else None
-    return raw_input, input_x, target_class
+    return image_pil, input_x, target_class
 
 def batch_predict(model, inputs, input_transform=None, device=None):
     if input_transform:

tensorwatch/saliency/gradcam.py

@@ -3,6 +3,9 @@ from .backprop import VanillaGradExplainer
 
 
 def _get_layer(model, key_list):
+    if key_list is None:
+        return None
+
     a = model
     for key in key_list:
         a = a._modules[key]
@@ -27,12 +30,13 @@ class GradCAMExplainer(VanillaGradExplainer):
         def backward_hook(m, grad_i, grad_o):
             self.intermediate_grad.append(grad_o[0].data.clone())
 
-        if self.use_inp:
-            self.target_layer.register_forward_hook(forward_hook_input)
-        else:
-            self.target_layer.register_forward_hook(forward_hook_output)
+        if self.target_layer is not None:
+            if self.use_inp:
+                self.target_layer.register_forward_hook(forward_hook_input)
+            else:
+                self.target_layer.register_forward_hook(forward_hook_output)
 
-        self.target_layer.register_backward_hook(backward_hook)
+            self.target_layer.register_backward_hook(backward_hook)
 
     def _reset_intermediate_lists(self):
         self.intermediate_act = []
@@ -43,14 +47,16 @@ class GradCAMExplainer(VanillaGradExplainer):
 
         _ = super(GradCAMExplainer, self)._backprop(inp, ind)
 
-        grad = self.intermediate_grad[0]
-        act = self.intermediate_act[0]
+        if len(self.intermediate_grad):
+            grad = self.intermediate_grad[0]
+            act = self.intermediate_act[0]
 
-        weights = grad.sum(-1).sum(-1).unsqueeze(-1).unsqueeze(-1)
-        cam = weights * act
-        cam = cam.sum(1).unsqueeze(1)
+            weights = grad.sum(-1).sum(-1).unsqueeze(-1).unsqueeze(-1)
+            cam = weights * act
+            cam = cam.sum(1).unsqueeze(1)
 
-        cam = torch.clamp(cam, min=0)
-
-        return cam
+            cam = torch.clamp(cam, min=0)
 
+            return cam
+        else:
+            return None

tensorwatch/saliency/saliency.py

@@ -52,7 +52,7 @@ def _get_layer_path(model):
         return ['avgpool'] #layer4
     elif model.__class__.__name__ == 'Inception3':
         return ['Mixed_7c', 'branch_pool'] # ['conv2d_94'], 'mixed10'
-    else: #unknown network
+    else: #TODO: guess layer for other networks?
         return None
 
 def get_saliency(model, raw_input, input, label, method='integrate_grad', layer_path=None):
@@ -75,11 +75,14 @@ def get_saliency(model, raw_input, input, label, method='integrate_grad', layer_
     exp = _get_explainer(method, model, layer_path)
     saliency = exp.explain(input, label, raw_input)
 
-    saliency = saliency.abs().sum(dim=1)[0].squeeze()
-    saliency -= saliency.min()
-    saliency /= (saliency.max() + 1e-20)
+    if saliency is not None:
+        saliency = saliency.abs().sum(dim=1)[0].squeeze()
+        saliency -= saliency.min()
+        saliency /= (saliency.max() + 1e-20)
 
-    return saliency.detach().cpu().numpy()
+        return saliency.detach().cpu().numpy()
+    else:
+        return None
 
 def get_image_saliency_results(model, raw_image, input, label,
                                methods=['lime_imagenet', 'gradcam', 'smooth_grad',
@@ -88,15 +91,17 @@ def get_image_saliency_results(model, raw_image, input, label,
     results = []
     for method in methods:
         sal = get_saliency(model, raw_image, input, label, method=method)
-        results.append(ImageSaliencyResult(raw_image, sal, method))
+
+        if sal is not None:
+            results.append(ImageSaliencyResult(raw_image, sal, method))
     return results
 
-def get_image_saliency_plot(image_saliency_results, cols = 2, figsize = None):
+def get_image_saliency_plot(image_saliency_results, cols:int=2, figsize:tuple=None):
     import matplotlib.pyplot as plt # delayed import due to matplotlib threading issue
 
     rows = math.ceil(len(image_saliency_results) / cols)
     figsize=figsize or (8, 3 * rows)
-    figure = plt.figure(figsize=figsize) #figsize=(8, 3)
+    figure = plt.figure(figsize=figsize)
     
     for i, r in enumerate(image_saliency_results):
         ax = figure.add_subplot(rows, cols, i+1)
@@ -106,7 +111,8 @@ def get_image_saliency_plot(image_saliency_results, cols = 2, figsize = None):
 
         #upsampler = nn.Upsample(size=(raw_image.height, raw_image.width), mode='bilinear')
         saliency_upsampled = skimage.transform.resize(r.saliency, 
-                                                      (r.raw_image.height, r.raw_image.width))
+                                                      (r.raw_image.height, r.raw_image.width),
+                                                      mode='reflect')
 
         image_utils.show_image(r.raw_image, img2=saliency_upsampled, 
                              alpha2=r.saliency_alpha, cmap2=r.saliency_cmap, ax=ax)

tensorwatch/watcher_client.py

@@ -47,7 +47,7 @@ class WatcherClient(WatcherBase):
             utils.debug_log("WatcherClient is closed", verbosity=1)
         super(WatcherClient, self).close()
 
-    def devices_or_default(self, devices:Sequence[str])->Sequence[str]: # overriden
+    def devices_or_default(self, devices:Sequence[str])->Sequence[str]: # overridden
         # TODO: this method is duplicated in Watcher and WatcherClient
 
         # make sure TCP port is attached to tcp device

test/simple_log/cli_sum_log.py

@@ -10,18 +10,20 @@ utils.set_debug_verbosity(4)
 
 def show_mpl():
     cli = tw.WatcherClient()
-    p = tw.LinePlot(title='Demo')
-    s1 = cli.create_stream(expr='lambda v:(v.i, v.sum)')
-    p.subscribe(s1, xtitle='Index', ytitle='sqrt(ev_i)')
-    p.show()
+    st_isum = cli.open_stream('isums')
+    st_rsum = cli.open_stream('rsums')
     
+    line_plot = tw.Visualizer(st_isum, vis_type='line', xtitle='i', ytitle='isum')
+    line_plot.show()
+
+    line_plot2 = tw.Visualizer(st_rsum, vis_type='line', host=line_plot, ytitle='rsum')
+
     tw.plt_loop()
 
 def show_text():
     cli = tw.WatcherClient()
-    s1 = cli.create_stream(expr='lambda v:(v.i, v.sum)')
-    text = tw.Visualizer(s1, vis_type='text')
-    text.show()
+    text_vis = tw.Visualizer(st_isum, vis_type='text')
+    text_vis.show()
     input('Waiting')
 
 #show_text()

test/simple_log/sum_lazy.py

@@ -1,17 +1,15 @@
-import time
+import time, random
 import tensorwatch as tw
-import random
 
-# create watcher, notice that we are not logging anything
+# create watcher object as usual
 w = tw.Watcher()
 
-weights = [random.random() for _ in range(5)]
+weights = None
 for i in range(10000):
     weights = [random.random() for _ in range(5)]
 
-    # we are just observing variables
-    # observation has no cost, nothing gets logged anywhere
-    # parameter name is your choice
+    # let watcher observe variables we have
+    # this has almost no performance cost
     w.observe(weights=weights)
 
     time.sleep(1)

test/simple_log/sum_log.py

@@ -1,14 +1,19 @@
-import time
+import time, random
 import tensorwatch as tw
 
-# create watcher and stream that we would use
-# for logging
+# we will create two streams, one for 
+# sums of integers, other for random numbers
 w = tw.Watcher()
-s = w.create_stream('my_log')
+st_isum = w.create_stream('isums')
+st_rsum = w.create_stream('rsums')
 
-sum = 0
+isum, rsum = 0, 0
 for i in range(10000):
-    sum += i
-    # write tuple to our log
-    s.write((i, sum))
+    isum += i
+    rsum += random.randint(0,i)
+
+    # write to streams
+    st_isum.write((i, isum))
+    st_rsum.write((i, rsum))
+
     time.sleep(1)

test/simple_log/sum_log2.py

@@ -1,11 +0,0 @@
-import time, random
-import tensorwatch as tw
-
-sum = 0
-rsum = 0
-for i in range(10000):
-    sum += i
-    rsum += random.random() * i
-    tw.observe(i=i, sum=sum, rsum=rsum)
-    time.sleep(1)
-

test/test.pyproj

@@ -5,7 +5,7 @@
     <ProjectGuid>9a7fe67e-93f0-42b5-b58f-77320fc639e4</ProjectGuid>
     <ProjectHome>
     </ProjectHome>
-    <StartupFile>simple_log\sum_lazy.py</StartupFile>
+    <StartupFile>post_train\saliency.py</StartupFile>
     <SearchPath>
     </SearchPath>
     <WorkingDirectory>.</WorkingDirectory>
@@ -25,9 +25,7 @@
   </PropertyGroup>
   <ItemGroup>
     <Compile Include="deps\live_graph.py" />
-    <Compile Include="simple_log\example.py">
-      <SubType>Code</SubType>
-    </Compile>
+    <Compile Include="simple_log\file_expr.py" />
     <Compile Include="simple_log\quick_start.py">
       <SubType>Code</SubType>
     </Compile>
@@ -68,7 +66,7 @@
     <Compile Include="post_train\saliency.py">
       <SubType>Code</SubType>
     </Compile>
-    <Compile Include="simple_log\cli_sum_log_2.py">
+    <Compile Include="simple_log\cli_file_expr.py">
       <SubType>Code</SubType>
     </Compile>
     <Compile Include="visualizations\confidence_int.py">
@@ -89,7 +87,6 @@
     <Compile Include="visualizations\pie_chart.py">
       <SubType>Code</SubType>
     </Compile>
-    <Compile Include="simple_log\sum_log_2.py" />
     <Compile Include="simple_log\sum_log.py">
       <SubType>Code</SubType>
     </Compile>