TensorFlowSharp 1.7 upgrade package (#746)

* some random change so that I can create this PR

* docs update for TensorFlowSharp new version

* changed the links to the new unitypackage file

* resolved conflicts, updated the pictures for CUDA 9.0

* fixed a typo

* resolved arthur's comment

* blurred the usernames

* modified the AWS doc

* resolved Vince's comment

docs/Basic-Guide.md

@@ -24,7 +24,7 @@ In order to use ML-Agents within Unity, you need to change some Unity settings f
 
 ![Project Settings](images/project-settings.png)
 
-[Download](https://s3.amazonaws.com/unity-ml-agents/0.3/TFSharpPlugin.unitypackage) the TensorFlowSharp plugin. Then import it into Unity by double clicking the downloaded file.  You can check if it was successfully imported by checking the TensorFlow files in the Project window under **Assets** > **ML-Agents** > **Plugins** > **Computer**. 
+[Download](https://s3.amazonaws.com/unity-ml-agents/0.4/TFSharpPlugin.unitypackage) the TensorFlowSharp plugin. Then import it into Unity by double clicking the downloaded file.  You can check if it was successfully imported by checking the TensorFlow files in the Project window under **Assets** > **ML-Agents** > **Plugins** > **Computer**. 
 
 **Note**: If you don't see anything under **Assets**, drag the `ml-agents/unity-environment/Assets/ML-Agents` folder under **Assets** within Project window.
 
@@ -86,9 +86,10 @@ launches our environment executable. This means:
 the 3DBall Scene is the only one checked. (If the list is empty, than only the 
 current scene is included in the build).
 6. Click **Build**:
-    a. In the File dialog, navigate to the `python` folder in your ML-Agents 
+    - In the File dialog, navigate to the `python` folder in your ML-Agents 
     directory.
-    b. Assign a file name and click **Save**.
+    - Assign a file name and click **Save**.
+    - (For Windows）With Unity 2018.1, it will ask you to select a folder instead of a file name. Create a subfolder within `python` folder and select that folder to build. In the following steps you will refer to this subfolder's name as `env_name`. 
 
 ![Build Window](images/mlagents-BuildWindow.png)
 

docs/Installation-Windows.md

@@ -6,7 +6,7 @@ To use ML-Agents, you install Python and the required Python packages as outline
 
 ## Step 1: Install Python via Anaconda
 
-[Download](https://www.anaconda.com/download/#windows) and install Anaconda for Windows. By using Anaconda, you can manage separate environments for different distributions of Python. Python 3 is required as we no longer support Python 2. In this guide, we are using Python version 3.5 (Anaconda comes with Python 3.6, we will create an environment with python 3.5) and Anaconda version 5.1 ([64-bit](https://repo.continuum.io/archive/Anaconda3-5.1.0-Windows-x86_64.exe) or [32-bit](https://repo.continuum.io/archive/Anaconda3-5.1.0-Windows-x86.exe) direct links).
+[Download](https://www.anaconda.com/download/#windows) and install Anaconda for Windows. By using Anaconda, you can manage separate environments for different distributions of Python. Python 3 is required as we no longer support Python 2. In this guide, we are using Python version 3.6 and Anaconda version 5.1 ([64-bit](https://repo.continuum.io/archive/Anaconda3-5.1.0-Windows-x86_64.exe) or [32-bit](https://repo.continuum.io/archive/Anaconda3-5.1.0-Windows-x86.exe) direct links).
 
 <p align="center">
     <img src="images/anaconda_install.PNG" 
@@ -37,7 +37,7 @@ You will create a new [Conda environment](https://conda.io/docs/) to be used wit
 To create a new Conda environment, open a new Anaconda Prompt (_Anaconda Prompt_ in the search bar) and type in the following command:
 
 ```
-conda create -n ml-agents python=3.5
+conda create -n ml-agents python=3.6
 ```
 
 You may be asked to install new packages. Type `y` and press enter _(make sure you are connected to the internet)_. You must install these required packages. The new Conda environment is called ml-agents and uses Python version 3.6.
@@ -51,7 +51,7 @@ You may be asked to install new packages. Type `y` and press enter _(make sure y
 To use this environment, you must activate it. _(To use this environment In the future, you can run the same command)_. In the same Anaconda Prompt, type in the following command:
 
 ```
-conda activate ml-agents
+activate ml-agents
 ```
 
 You should see `(ml-agents)` prepended on the last line.
@@ -59,7 +59,7 @@ You should see `(ml-agents)` prepended on the last line.
 Next, install `tensorflow`. Install this package using `pip` - which is a package management system used to install Python packages. Latest versions of Tensorflow won't work, so you will need to make sure that you install version 1.4.0. In the same Anaconda Prompt, type in the following command _(make sure you are connected to the internet)_:
 
 ```
-pip install tensorflow==1.4.0
+pip install tensorflow==1.7.1
 ```
 
 ## Step 3: Install Required Python Packages
@@ -69,7 +69,7 @@ ML-Agents depends on a number of Python packages. Use `pip` to install these Pyt
 If you haven't already, clone the ML-Agents Github repository to your local computer. You can do this using Git ([download here](https://git-scm.com/download/win)) and running the following commands in an Anaconda Prompt _(if you open a new prompt, be sure to activate the ml-agents Conda environment by typing `activate ml-agents`)_:
 
 ```
-git clone git@github.com:Unity-Technologies/ml-agents.git
+git clone https://github.com/Unity-Technologies/ml-agents.git
 ```
 
 If you don't want to use Git, you can always directly download all the files [here](https://github.com/Unity-Technologies/ml-agents/archive/master.zip).
@@ -87,25 +87,25 @@ pip install .
 
 ```
 
-This will complete the installation of all the required Python packages to run ML-Agents. 
+This will complete the installation of all the required Python packages to run ML-Agents.
 
 ## (Optional) Step 4: GPU Training using ML-Agents 
 
-Not required to use v0.3 for ML-Agents. This is a guide for advanced users who want to train using GPUs. Additionally, you will need to check if your GPU is CUDA compatible. Please check Nvidia's page [here](https://developer.nvidia.com/cuda-gpus).
+GPU is not required for ML-Agents and won't speed up the PPO algorithm a lot during training(but something in the future will benefit from GPU). This is a guide for advanced users who want to train using GPUs. Additionally, you will need to check if your GPU is CUDA compatible. Please check Nvidia's page [here](https://developer.nvidia.com/cuda-gpus).
 
-As of ML-Agents v0.3, only CUDA 8 and cuDNN 6 is supported.
+As of ML-Agents v0.3, only CUDA v9.0 and cuDNN v7.0.5 is supported.
 
 ### Install Nvidia CUDA toolkit
 
-[Download](https://developer.nvidia.com/cuda-toolkit-archive) and install the CUDA toolkit from Nvidia's archive. The toolkit includes GPU-accelerated libraries, debugging and optimization tools, a C/C++ (Step Visual Studio 2015) compiler and a runtime library and is needed to run ML-Agents. In this guide, we are using version 8.0.61 ([direct link](https://developer.nvidia.com/compute/cuda/8.0/Prod2/network_installers/cuda_8.0.61_win10_network-exe)). 
+[Download](https://developer.nvidia.com/cuda-toolkit-archive) and install the CUDA toolkit 9.0 from Nvidia's archive. The toolkit includes GPU-accelerated libraries, debugging and optimization tools, a C/C++ (Step Visual Studio 2017) compiler and a runtime library and is needed to run ML-Agents. In this guide, we are using version 9.0.176 (https://developer.nvidia.com/compute/cuda/9.0/Prod/network_installers/cuda_9.0.176_win10_network-exe)).
 
 Before installing, please make sure you __close any running instances of Unity or Visual Studio__.
 
-Run the installer and select the Express option. Note the directory where you installed the CUDA toolkit. In this guide, we installed in the directory `C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v8.0`
+Run the installer and select the Express option. Note the directory where you installed the CUDA toolkit. In this guide, we installed in the directory `C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v9.0`
 
 ### Install Nvidia cuDNN library
 
-[Download](https://developer.nvidia.com/cudnn) and install the cuDNN library from Nvidia. cuDNN is is a GPU-accelerated library of primitives for deep neural networks. Before you can download, you will need to sign up for free to the Nvidia Developer Program.
+[Download](https://developer.nvidia.com/cudnn) and install the cuDNN library from Nvidia. cuDNN is a GPU-accelerated library of primitives for deep neural networks. Before you can download, you will need to sign up for free to the Nvidia Developer Program.
 
 <p align="center">
     <img src="images/cuDNN_membership_required.png" 
@@ -113,7 +113,7 @@ Run the installer and select the Express option. Note the directory where you in
         width="500" border="10" />
 </p>
 
-Once you've signed up, go back to the cuDNN [downloads page](https://developer.nvidia.com/cudnn). You may or may not be asked to fill out a short survey. When you get to the list cuDNN releases, __make sure you are downloading the right version for the CUDA toolkit you installed in Step 1.__  In this guide, we are using version 6.0 for CUDA toolkit version 8.0 ([direct link](https://developer.nvidia.com/compute/machine-learning/cudnn/secure/v6/prod/8.0_20170307/cudnn-8.0-windows10-x64-v6.0-zip)).
+Once you've signed up, go back to the cuDNN [downloads page](https://developer.nvidia.com/cudnn). You may or may not be asked to fill out a short survey. When you get to the list cuDNN releases, __make sure you are downloading the right version for the CUDA toolkit you installed in Step 1.__  In this guide, we are using version 7.0.5 for CUDA toolkit version 9.0 ([direct link](https://developer.nvidia.com/compute/machine-learning/cudnn/secure/v7.0.5/prod/9.0_20171129/cudnn-9.0-windows10-x64-v7)).
 
 After you have downloaded the cuDNN files, you will need to extract the files into the CUDA toolkit directory. In the cuDNN zip file, there are three folders called `bin`, `include`, and `lib`. 
 
@@ -123,7 +123,7 @@ After you have downloaded the cuDNN files, you will need to extract the files in
         width="500" border="10" />
 </p>
 
-Copy these three folders into the CUDA toolkit directory. The CUDA toolkit directory is located at `C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v8.0`
+Copy these three folders into the CUDA toolkit directory. The CUDA toolkit directory is located at `C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v9.0`
 
 <p align="center">
     <img src="images/cuda_toolkit_directory.PNG" 
@@ -151,7 +151,7 @@ From here, click the __Environment Variables__ button. Click __New__ to add a ne
         width="500" border="10" />
 </p>
 
-For __Variable Name__, enter `CUDA_HOME`. For the variable value, put the directory location for the CUDA toolkit. In this guide, the directory location is `C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v8.0`. Press __OK__ once.
+For __Variable Name__, enter `CUDA_HOME`. For the variable value, put the directory location for the CUDA toolkit. In this guide, the directory location is `C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v9.0`. Press __OK__ once.
 
 <p align="center">
     <img src="images/system_variable_name_value.PNG" 
@@ -159,10 +159,10 @@ For __Variable Name__, enter `CUDA_HOME`. For the variable value, put the direct
         width="500" border="10" />
 </p>
 
-To set the two path variables, inside the same __Environment Variables__ window and under the second box called __System Variables__, find a variable called `PATH` and click __Edit__. You will add two directories to the list. For this guide, the two entries would look like:
+To set the two path variables, inside the same __Environment Variables__ window and under the second box called __System Variables__, find a variable called `Path` and click __Edit__. You will add two directories to the list. For this guide, the two entries would look like:
 
-    C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v8.0\lib\x64
-    C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v8.0\extras\CUPTI\libx64
+    C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v9.0\lib\x64
+    C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v9.0\extras\CUPTI\libx64
 
 Make sure to replace the relevant directory location with the one you have installed. _Please note that case sensitivity matters_.
 
@@ -173,10 +173,11 @@ Make sure to replace the relevant directory location with the one you have insta
 </p>
 
 ### Install TensorFlow GPU
-Next, install `tensorflow-gpu` using `pip`. You'll need version 1.4.0 as newer versions require CUDA 9 which is not yet supported. In an Anaconda Prompt with the Conda environment ml-agents activated, type in the following command _(make sure you are connected to the internet)_:
+Next, install `tensorflow-gpu` using `pip`. You'll need version 1.7.1. In an Anaconda Prompt with the Conda environment ml-agents activated, type in the following command to uninstall the tensorflow for cpu and install the tensorflow for gpu _(make sure you are connected to the internet)_:
 
 ```
-pip install tensorflow-gpu==1.4.0
+pip uninstall tensorflow
+pip install tensorflow-gpu==1.7.1
 ```
 
 Lastly, you should test to see if everything installed properly and that TensorFlow can identify your GPU. In the same Anaconda Prompt, type in the following command:
@@ -193,11 +194,6 @@ You should see something similar to:
 Found device 0 with properties ...
 ```
 
-Step Visual Studio 2015: CUDA 8.0 is not compatible with Visual Studio 2017, so you will need an older version. Uninstall Visual Studio 2017 that comes with Unity, download Visual Studio Enterprise 2015 and install it with the Windows SDK. If you don't want/ can't install Visual Studio Enterprise 2015, you will need: 
-    [Visual C++ Redistributable for Visual Studio 2015](https://www.microsoft.com/en-us/download/details.aspx?id=48145),
-    [Visual Studio Community 2015](https://www.visualstudio.com/vs/older-downloads/) + [Windows SDK](https://msdn.microsoft.com/en-us/library/mt683786.aspx).
-
-
 ## Acknowledgements
 
 We would like to thank [Jason Weimann](https://unity3d.college/2017/10/25/machine-learning-in-unity3d-setting-up-the-environment-tensorflow-for-agentml-on-windows-10/) and [Nitish S. Mutha](http://blog.nitishmutha.com/tensorflow/2017/01/22/TensorFlow-with-gpu-for-windows.html) for writing the original articles which were used to create this guide.

docs/Training-on-Amazon-Web-Service.md

@@ -1,20 +1,21 @@
 # Training on Amazon Web Service
 
-This page contains instructions for setting up an EC2 instance on Amazon Web Service for training ML-Agents environments. You can run "headless" training if none of the agents in the environment use visual observations. 
+This page contains instructions for setting up an EC2 instance on Amazon Web Service for training ML-Agents environments. Visual observations is not supported currently and you need to enable headless mode.
 
-## Pre-Configured AMI
-A public pre-configured AMI is available with the ID: `ami-30ec184a` in the `us-east-1` region. It was created as a modification of the Amazon Deep Learning [AMI](https://aws.amazon.com/marketplace/pp/B01M0AXXQB). 
+## Recommended AMI
+You can get started with an EC2 instance with the Deep Learning AMI (Ubuntu) listed under AWS Marketplace [AMI](https://aws.amazon.com/marketplace/pp/B077GCH38C). Choose the python3 environment within that ami which gives you the python3 and CUDA 9.0 environment.
 
 ## Configuring your own Instance
 
-1. To begin with, you will need an EC2 instance which contains the latest Nvidia drivers, CUDA8, and cuDNN.  There are a number of external tutorials which describe this, such as:
-    * [Getting CUDA 8 to Work With openAI Gym on AWS and Compiling TensorFlow for CUDA 8 Compatibility](https://davidsanwald.github.io/2016/11/13/building-tensorflow-with-gpu-support.html)
-    * [Installing TensorFlow on an AWS EC2 P2 GPU Instance](http://expressionflow.com/2016/10/09/installing-tensorflow-on-an-aws-ec2-p2-gpu-instance/)
-    * [Updating Nvidia CUDA to 8.0.x in Ubuntu 16.04 – EC2 Gx instance](https://aichamp.wordpress.com/2016/11/09/updating-nvidia-cuda-to-8-0-x-in-ubuntu-16-04-ec2-gx-instance/)
+To begin with, you will need an EC2 instance which contains the latest Nvidia drivers, CUDA9, and cuDNN.  There are a number of external tutorials which describe this (Note: You will need to tweak some steps in these tutorials for CUDA 9):
+
+* [Getting CUDA 8 to Work With openAI Gym on AWS and Compiling TensorFlow for CUDA 8 Compatibility](https://davidsanwald.github.io/2016/11/13/building-tensorflow-with-gpu-support.html)    
+* [Installing TensorFlow on an AWS EC2 P2 GPU Instance](http://expressionflow.com/2016/10/09/installing-tensorflow-on-an-aws-ec2-p2-gpu-instance/)   
+* [Updating Nvidia CUDA to 8.0.x in Ubuntu 16.04 – EC2 Gx instance](https://aichamp.wordpress.com/2016/11/09/updating-nvidia-cuda-to-8-0-x-in-ubuntu-16-04-ec2-gx-instance/)
 
 ## Installing ML-Agents
 
-2. Move `python` sub-folder of this ml-agents repo to the remote ECS instance, and set it as the working directory.
+1. Move `python` sub-folder of this ml-agents repo to the remote ECS instance, and set it as the working directory.
 2. Install the required packages with `pip3 install .`.
 
 ## Testing
@@ -24,7 +25,7 @@ To verify that all steps worked correctly:
 1. In the Unity Editor, load a project containing an ML-Agents environment (you can use one of the example environments if you have not created your own).
 2. Open the Build Settings window (menu: File > Build Settings).
 3. Select Linux as the Target Platform, and x86_64 as the target architecture.
-4. Check Headless Mode (unless you have enabled a virtual screen following the instructions below).
+4. Check Headless Mode. 
 5. Click Build to build the Unity environment executable.
 6. Upload the executable to your EC2 instance.
 7. Test the instance setup from Python using:
@@ -37,32 +38,3 @@ env = UnityEnvironment(<your_env>)
 Where `<your_env>` corresponds to the path to your environment executable.
  
 You should receive a message confirming that the environment was loaded successfully.
-
-## (Optional) Enabling a virtual screen
-
-_Instructions here are adapted from this [Medium post](https://medium.com/towards-data-science/how-to-run-unity-on-amazon-cloud-or-without-monitor-3c10ce022639) on running general Unity applications in the cloud._
-
-Current limitations of the Unity Engine require that a screen be available to render to when using visual observations. In order to make this possible when training on a remote server, a virtual screen is required. We can do this by installing Xorg and creating a virtual screen. Once installed and created, we can display the Unity environment in the virtual environment, and train as we would on a local machine. Ensure that `headless` mode is disabled when building linux executables which use visual observations.
-
-1. Run the following commands to install Xorg:
-
-    ```
-    sudo apt-get update
-    sudo apt-get install -y xserver-xorg mesa-utils
-    sudo nvidia-xconfig -a --use-display-device=None --virtual=1280x1024
-    ```
-
-2. Restart the EC2 instance.
-
-3. Make sure there are no Xorg processes running. To kill the Xorg processes, run `sudo killall Xorg`.  
-Note that you might have to run this command multiple times depending on how Xorg is configured.  
-If you run `nvidia-smi`, you will have a list of processes running on the GPU, Xorg should not be in the list. 
-
-4. Run:
-
-    ```
-    sudo /usr/bin/X :0 &
-    export DISPLAY=:0
-    ```
- 
-5. To ensure the installation was successful, run `glxgears`. If there are no errors, then Xorg is correctly configured.

docs/Using-TensorFlow-Sharp-in-Unity.md

@@ -15,7 +15,7 @@ _Notice: This feature is still experimental. While it is possible to embed train
 ## Requirements
 
 * Unity 2017.1 or above
-* Unity TensorFlow Plugin ([Download here](https://s3.amazonaws.com/unity-ml-agents/0.3/TFSharpPlugin.unitypackage))
+* Unity TensorFlow Plugin ([Download here](https://s3.amazonaws.com/unity-ml-agents/0.4/TFSharpPlugin.unitypackage))
 
 # Using TensorFlowSharp with ML-Agents
 
@@ -72,10 +72,8 @@ If you followed these instructions well, the agents in your environment that use
 
 # iOS additional instructions for building
 
-* Once you build for iOS in the editor, Xcode will launch.
-* In **General** > **Linked Frameworks and Libraries**:
-  * Add a framework called `Accelerate.framework`
-  * Remove the library `libtensorflow-core.a`
+* Once you build the project for iOS in the editor, open the .xcodeproj file within the project folder using Xcode.
+* Set up your ios account following the [iOS Account setup page](https://docs.unity3d.com/Manual/iphone-accountsetup.html). 
 * In **Build Settings** > **Linking** > **Other Linker Flags**:
   * Double click on the flag list to expand the list
   * Add `-force_load`

python/requirements.txt

@@ -1,4 +1,4 @@
-tensorflow==1.4.0
+tensorflow==1.7.1
 Pillow>=4.2.1
 matplotlib
 numpy>=1.11.0