Feature/mla 2394 update web docs (#37)

* Added mkdocs and refactored docs.

* Updated docs.

* Added gh action to manually public docs. Updated mkdocs.yml.

.github/workflows/publish_docs.yaml

@@ -0,0 +1,24 @@
+name: Publish HTML Docs
+
+on:
+  workflow_dispatch:
+
+jobs:
+  publish:
+    name: Publish Docs to GH Pages
+    runs-on: [self-hosted, Linux, X64]
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v2
+        with:
+          token: ${{ secrets.PUBLIC_GH_TOKEN }}
+      - name: Setup Python 3.7
+        uses: actions/setup-python@v2
+        with:
+          python-version: 3.7
+      - name: Publish docs
+        run: |
+          pip install mkdocs
+          git remote add public git@github.com:Unity-Technologies/ml-agents.git
+          mkdocs gh-deply --clean -r public
+

README.md

@@ -1,189 +0,0 @@
-<img src="docs/images/image-banner.png" align="middle" width="3000"/>
-
-# Unity ML-Agents Toolkit
-
-[![docs badge](https://img.shields.io/badge/docs-reference-blue.svg)](https://github.com/Unity-Technologies/ml-agents/tree/release_19_docs/docs/)
-
-[![license badge](https://img.shields.io/badge/license-Apache--2.0-green.svg)](LICENSE.md)
-
-([latest release](https://github.com/Unity-Technologies/ml-agents/releases/tag/latest_release))
-([all releases](https://github.com/Unity-Technologies/ml-agents/releases))
-
-**The Unity Machine Learning Agents Toolkit** (ML-Agents) is an open-source
-project that enables games and simulations to serve as environments for
-training intelligent agents. We provide implementations (based on PyTorch)
-of state-of-the-art algorithms to enable game developers and hobbyists to easily
-train intelligent agents for 2D, 3D and VR/AR games. Researchers can also use the
-provided simple-to-use Python API to train Agents using reinforcement learning,
-imitation learning, neuroevolution, or any other methods. These trained agents can be
-used for multiple purposes, including controlling NPC behavior (in a variety of
-settings such as multi-agent and adversarial), automated testing of game builds
-and evaluating different game design decisions pre-release. The ML-Agents
-Toolkit is mutually beneficial for both game developers and AI researchers as it
-provides a central platform where advances in AI can be evaluated on Unity’s
-rich environments and then made accessible to the wider research and game
-developer communities.
-
-## Features
-
-- 18+ [example Unity environments](docs/Learning-Environment-Examples.md)
-- Support for multiple environment configurations and training scenarios
-- Flexible Unity SDK that can be integrated into your game or custom Unity scene
-- Support for training single-agent, multi-agent cooperative, and multi-agent
-  competitive scenarios via several Deep Reinforcement Learning algorithms (PPO, SAC, MA-POCA, self-play).
-- Support for learning from demonstrations through two Imitation Learning algorithms (BC and GAIL).
-- Easily definable Curriculum Learning scenarios for complex tasks
-- Train robust agents using environment randomization
-- Flexible agent control with On Demand Decision Making
-- Train using multiple concurrent Unity environment instances
-- Utilizes the [Unity Inference Engine](docs/Unity-Inference-Engine.md) to
-  provide native cross-platform support
-- Unity environment [control from Python](docs/Python-LLAPI.md)
-- Wrap Unity learning environments as a [gym](docs/Python-Gym-API.md)
-
-See our [ML-Agents Overview](docs/ML-Agents-Overview.md) page for detailed
-descriptions of all these features.
-
-## Releases & Documentation
-
-**Our latest, stable release is `Release 19`. Click
-[here](https://github.com/Unity-Technologies/ml-agents/tree/release_19_docs/docs/Readme.md)
-to get started with the latest release of ML-Agents.**
-
-The table below lists all our releases, including our `main` branch which is
-under active development and may be unstable. A few helpful guidelines:
-- The [Versioning page](docs/Versioning.md) overviews how we manage our GitHub
-  releases and the versioning process for each of the ML-Agents components.
-- The [Releases page](https://github.com/Unity-Technologies/ml-agents/releases)
-  contains details of the changes between releases.
-- The [Migration page](docs/Migrating.md) contains details on how to upgrade
-  from earlier releases of the ML-Agents Toolkit.
-- The **Documentation** links in the table below include installation and usage
-  instructions specific to each release. Remember to always use the
-  documentation that corresponds to the release version you're using.
-- The `com.unity.ml-agents` package is [verified](https://docs.unity3d.com/2020.1/Documentation/Manual/pack-safe.html)
-  for Unity 2020.1 and later. Verified packages releases are numbered 1.0.x.
-
-|        **Version**         |   **Release Date**   |                                          **Source**                                          |                                            **Documentation**                                            |                                             **Download**                                              |                   **Python Package**                    |                                    **Unity Package**                                     |
-|:--------------------------:|:--------------------:|:--------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------:|:-----------------------------------------------------------------------------------------------------:|:-------------------------------------------------------:|:----------------------------------------------------------------------------------------:|
-|    **main (unstable)**     |          --          |             [source](https://github.com/Unity-Technologies/ml-agents/tree/main)              |            [docs](https://github.com/Unity-Technologies/ml-agents/tree/main/docs/Readme.md)             |             [download](https://github.com/Unity-Technologies/ml-agents/archive/main.zip)              |                           --                            |                                            --                                            |
-|       **Release 19**       | **January 14, 2022** |        **[source](https://github.com/Unity-Technologies/ml-agents/tree/release_19)**         |     **[docs](https://github.com/Unity-Technologies/ml-agents/tree/release_19_docs/docs/Readme.md)**     |        **[download](https://github.com/Unity-Technologies/ml-agents/archive/release_19.zip)**         | **[0.28.0](https://pypi.org/project/mlagents/0.28.0/)** |                                            --                                            |
-|       **Release 18**       |  **June 09, 2021**   |        **[source](https://github.com/Unity-Technologies/ml-agents/tree/release_18)**         |     **[docs](https://github.com/Unity-Technologies/ml-agents/tree/release_18_docs/docs/Readme.md)**     |        **[download](https://github.com/Unity-Technologies/ml-agents/archive/release_18.zip)**         | **[0.27.0](https://pypi.org/project/mlagents/0.27.0/)** | **[2.1.0](https://docs.unity3d.com/Packages/com.unity.ml-agents@2.1/manual/index.html)** |
-| **Verified Package 1.0.8** |   **May 26, 2021**   | **[source](https://github.com/Unity-Technologies/ml-agents/tree/com.unity.ml-agents_1.0.8)** | **[docs](https://github.com/Unity-Technologies/ml-agents/blob/release_2_verified_docs/docs/Readme.md)** | **[download](https://github.com/Unity-Technologies/ml-agents/archive/com.unity.ml-agents_1.0.8.zip)** | **[0.16.1](https://pypi.org/project/mlagents/0.16.1/)** | **[1.0.8](https://docs.unity3d.com/Packages/com.unity.ml-agents@1.0/manual/index.html)** |
-
-If you are a researcher interested in a discussion of Unity as an AI platform,
-see a pre-print of our
-[reference paper on Unity and the ML-Agents Toolkit](https://arxiv.org/abs/1809.02627).
-
-If you use Unity or the ML-Agents Toolkit to conduct research, we ask that you
-cite the following paper as a reference:
-
-Juliani, A., Berges, V., Teng, E., Cohen, A., Harper, J., Elion, C., Goy, C.,
-Gao, Y., Henry, H., Mattar, M., Lange, D. (2020). Unity: A General Platform for
-Intelligent Agents. _arXiv preprint
-[arXiv:1809.02627](https://arxiv.org/abs/1809.02627)._
-https://github.com/Unity-Technologies/ml-agents.
-
-## Additional Resources
-
-We have a Unity Learn course,
-[ML-Agents: Hummingbirds](https://learn.unity.com/course/ml-agents-hummingbirds),
-that provides a gentle introduction to Unity and the ML-Agents Toolkit.
-
-We've also partnered with
-[CodeMonkeyUnity](https://www.youtube.com/c/CodeMonkeyUnity) to create a
-[series of tutorial videos](https://www.youtube.com/playlist?list=PLzDRvYVwl53vehwiN_odYJkPBzcqFw110)
-on how to implement and use the ML-Agents Toolkit.
-
-We have also published a series of blog posts that are relevant for ML-Agents:
-
-- (July 12, 2021)
-  [ML-Agents plays Dodgeball](https://blog.unity.com/technology/ml-agents-plays-dodgeball)
-- (May 5, 2021)
-  [ML-Agents v2.0 release: Now supports training complex cooperative behaviors](https://blogs.unity3d.com/2021/05/05/ml-agents-v2-0-release-now-supports-training-complex-cooperative-behaviors/)
-- (December 28, 2020)
-  [Happy holidays from the Unity ML-Agents team!](https://blogs.unity3d.com/2020/12/28/happy-holidays-from-the-unity-ml-agents-team/)
-- (November 20, 2020)
-  [How Eidos-Montréal created Grid Sensors to improve observations for training agents](https://blogs.unity3d.com/2020/11/20/how-eidos-montreal-created-grid-sensors-to-improve-observations-for-training-agents/)
-- (November 11, 2020)
-  [2020 AI@Unity interns shoutout](https://blogs.unity3d.com/2020/11/11/2020-aiunity-interns-shoutout/)
-- (May 12, 2020)
-  [Announcing ML-Agents Unity Package v1.0!](https://blogs.unity3d.com/2020/05/12/announcing-ml-agents-unity-package-v1-0/)
-- (February 28, 2020)
-  [Training intelligent adversaries using self-play with ML-Agents](https://blogs.unity3d.com/2020/02/28/training-intelligent-adversaries-using-self-play-with-ml-agents/)
-- (November 11, 2019)
-  [Training your agents 7 times faster with ML-Agents](https://blogs.unity3d.com/2019/11/11/training-your-agents-7-times-faster-with-ml-agents/)
-- (October 21, 2019)
-  [The AI@Unity interns help shape the world](https://blogs.unity3d.com/2019/10/21/the-aiunity-interns-help-shape-the-world/)
-- (April 15, 2019)
-  [Unity ML-Agents Toolkit v0.8: Faster training on real games](https://blogs.unity3d.com/2019/04/15/unity-ml-agents-toolkit-v0-8-faster-training-on-real-games/)
-- (March 1, 2019)
-  [Unity ML-Agents Toolkit v0.7: A leap towards cross-platform inference](https://blogs.unity3d.com/2019/03/01/unity-ml-agents-toolkit-v0-7-a-leap-towards-cross-platform-inference/)
-- (December 17, 2018)
-  [ML-Agents Toolkit v0.6: Improved usability of Brains and Imitation Learning](https://blogs.unity3d.com/2018/12/17/ml-agents-toolkit-v0-6-improved-usability-of-brains-and-imitation-learning/)
-- (October 2, 2018)
-  [Puppo, The Corgi: Cuteness Overload with the Unity ML-Agents Toolkit](https://blogs.unity3d.com/2018/10/02/puppo-the-corgi-cuteness-overload-with-the-unity-ml-agents-toolkit/)
-- (September 11, 2018)
-  [ML-Agents Toolkit v0.5, new resources for AI researchers available now](https://blogs.unity3d.com/2018/09/11/ml-agents-toolkit-v0-5-new-resources-for-ai-researchers-available-now/)
-- (June 26, 2018)
-  [Solving sparse-reward tasks with Curiosity](https://blogs.unity3d.com/2018/06/26/solving-sparse-reward-tasks-with-curiosity/)
-- (June 19, 2018)
-  [Unity ML-Agents Toolkit v0.4 and Udacity Deep Reinforcement Learning Nanodegree](https://blogs.unity3d.com/2018/06/19/unity-ml-agents-toolkit-v0-4-and-udacity-deep-reinforcement-learning-nanodegree/)
-- (May 24, 2018)
-  [Imitation Learning in Unity: The Workflow](https://blogs.unity3d.com/2018/05/24/imitation-learning-in-unity-the-workflow/)
-- (March 15, 2018)
-  [ML-Agents Toolkit v0.3 Beta released: Imitation Learning, feedback-driven features, and more](https://blogs.unity3d.com/2018/03/15/ml-agents-v0-3-beta-released-imitation-learning-feedback-driven-features-and-more/)
-- (December 11, 2017)
-  [Using Machine Learning Agents in a real game: a beginner’s guide](https://blogs.unity3d.com/2017/12/11/using-machine-learning-agents-in-a-real-game-a-beginners-guide/)
-- (December 8, 2017)
-  [Introducing ML-Agents Toolkit v0.2: Curriculum Learning, new environments, and more](https://blogs.unity3d.com/2017/12/08/introducing-ml-agents-v0-2-curriculum-learning-new-environments-and-more/)
-- (September 19, 2017)
-  [Introducing: Unity Machine Learning Agents Toolkit](https://blogs.unity3d.com/2017/09/19/introducing-unity-machine-learning-agents/)
-- Overviewing reinforcement learning concepts
-  ([multi-armed bandit](https://blogs.unity3d.com/2017/06/26/unity-ai-themed-blog-entries/)
-  and
-  [Q-learning](https://blogs.unity3d.com/2017/08/22/unity-ai-reinforcement-learning-with-q-learning/))
-
-### More from Unity
-
-- [Unity Robotics](https://github.com/Unity-Technologies/Unity-Robotics-Hub)
-- [Unity Computer Vision](https://unity.com/computer-vision)
-- [Unity Game Simulation](https://unity.com/products/game-simulation)
-
-## Community and Feedback
-
-The ML-Agents Toolkit is an open-source project and we encourage and welcome
-contributions. If you wish to contribute, be sure to review our
-[contribution guidelines](com.unity.ml-agents/CONTRIBUTING.md) and
-[code of conduct](CODE_OF_CONDUCT.md).
-
-For problems with the installation and setup of the ML-Agents Toolkit, or
-discussions about how to best setup or train your agents, please create a new
-thread on the
-[Unity ML-Agents forum](https://forum.unity.com/forums/ml-agents.453/) and make
-sure to include as much detail as possible. If you run into any other problems
-using the ML-Agents Toolkit or have a specific feature request, please
-[submit a GitHub issue](https://github.com/Unity-Technologies/ml-agents/issues).
-
-Please tell us which samples you would like to see shipped with the ML-Agents Unity
-package by replying to
-[this forum thread](https://forum.unity.com/threads/feedback-wanted-shipping-sample-s-with-the-ml-agents-package.1073468/).
-
-
-Your opinion matters a great deal to us. Only by hearing your thoughts on the
-Unity ML-Agents Toolkit can we continue to improve and grow. Please take a few
-minutes to
-[let us know about it](https://unitysoftware.co1.qualtrics.com/jfe/form/SV_55pQKCZ578t0kbc).
-
-For any other questions or feedback, connect directly with the ML-Agents team at
-ml-agents@unity3d.com.
-
-## Privacy
-
-In order to improve the developer experience for Unity ML-Agents Toolkit, we have added in-editor analytics.
-Please refer to "Information that is passively collected by Unity" in the
-[Unity Privacy Policy](https://unity3d.com/legal/privacy-policy).
-
-## License
-
-[Apache License 2.0](LICENSE.md)

docs/Background-Machine-Learning.md

@@ -111,9 +111,7 @@ every step, but only when a robot arrives at a success or failure situation), is
 a defining characteristic of reinforcement learning and precisely why learning
 good policies can be difficult (and/or time-consuming) for complex environments.
 
-<p align="center">
-  <img src="images/rl_cycle.png" alt="The reinforcement learning cycle."/>
-</p>
+<div style="text-align: center"><img src="../images/rl_cycle.png" alt="The reinforcement learning lifecycle."></div>
 
 [Learning a policy](https://blogs.unity3d.com/2017/08/22/unity-ai-reinforcement-learning-with-q-learning/)
 usually requires many trials and iterative policy updates. More specifically,

docs/CODE_OF_CONDUCT.md

@@ -0,0 +1 @@
+{!../CODE_OF_CONDUCT.md!}

docs/CONTRIBUTING.md

@@ -0,0 +1 @@
+{!../com.unity.ml-agents/CONTRIBUTING.md!}

docs/Installation.md

@@ -10,11 +10,11 @@ The ML-Agents Toolkit contains several components:
   contains experimental C#/Unity components that are not yet ready to be part
   of the base `com.unity.ml-agents` package. `com.unity.ml-agents.extensions`
   has a direct dependency on `com.unity.ml-agents`.
-- Three Python packages:
+- Two Python packages:
   - [`mlagents`](../ml-agents/) contains the machine learning algorithms that
     enables you to train behaviors in your Unity scene. Most users of ML-Agents
     will only need to directly install `mlagents`.
-  - [`mlagents_envs`](../ml-agents-envs/) contains a Python API to interact with
+  - [`mlagents_envs`](../ml-agents-envs/) contains a set of Python APIs to interact with
     a Unity scene. It is a foundational layer that facilitates data messaging
     between Unity scene and the Python machine learning algorithms.
     Consequently, `mlagents` depends on `mlagents_envs`.

docs/LICENSE.md

@@ -0,0 +1 @@
+{!../LICENSE.md!}

docs/Learning-Environment-Examples.md

@@ -1,6 +1,6 @@
 # Example Learning Environments
 
-<img src="images/example-envs.png" align="middle" width="3000"/>
+<img src="../images/example-envs.png" align="middle" width="3000"/>
 
 The Unity ML-Agents Toolkit includes an expanding set of example environments
 that highlight the various features of the toolkit. These environments can also

docs/Learning-Environment-Executable.md

@@ -194,5 +194,5 @@ graphics display in the Unity executable. There are two ways to achieve this:
 
 If you want to train with graphics (for example, using camera and visual observations), you'll
 need to set up display rendering support (e.g. xvfb) on you server machine. In our
-[Colab Notebook Tutorials](Readme.md#python-tutorial-with-google-colab), the Setup section has
+[Colab Notebook Tutorials](ML-Agents-Toolkit-Documentation.md#python-tutorial-with-google-colab), the Setup section has
 examples of setting up xvfb on servers.

docs/Limitations.md

@@ -2,6 +2,6 @@
 
 See the package-specific Limitations pages:
 
-- [`com.unity.mlagents` Unity package](../com.unity.ml-agents/Documentation~/com.unity.ml-agents.md#known-limitations)
-- [`mlagents` Python package](../ml-agents/README.md#limitations)
-- [`mlagents_envs` Python package](../ml-agents-envs/README.md#limitations)
+- [`com.unity.mlagents` Unity package](com.unity.ml-agents.md)
+- [`mlagents` Python package](ML-Agents-README.md)
+- [`mlagents_envs` Python package](ML-Agents-Envs-README.md)

docs/ML-Agents-Envs-README.md

@@ -0,0 +1 @@
+{!../ml-agents-envs/README.md!}

docs/ML-Agents-Overview.md

@@ -179,9 +179,8 @@ The ML-Agents Toolkit contains five high-level components:
 - **Gym Wrapper** (not pictured). A common way in which machine learning
   researchers interact with simulation environments is via a wrapper provided by
   OpenAI called [gym](https://github.com/openai/gym). We provide a gym wrapper
-  in the `ml-agents-envs` package and
-  [instructions](Python-Gym-API.md) for using it with existing machine
-  learning algorithms which utilize gym.
+  in the `ml-agents-envs` package and [instructions](Python-Gym-API.md) for using
+  it with existing machine learning algorithms which utilize gym.
 - **PettingZoo Wrapper** (not pictured) PettingZoo is python API for
   interacting with multi-agent simulation environments that provides a
   gym-like interface. We provide a PettingZoo wrapper for Unity ML-Agents
@@ -190,7 +189,7 @@ The ML-Agents Toolkit contains five high-level components:
   algorithms.
 
 <p align="center">
-  <img src="images/learning_environment_basic.png"
+  <img src="../images/learning_environment_basic.png"
        alt="Simplified ML-Agents Scene Block Diagram"
        width="600"
        border="10" />
@@ -225,7 +224,7 @@ can have the same Behavior. This does not mean that at each instance they will
 have identical observation and action _values_.
 
 <p align="center">
-  <img src="images/learning_environment_example.png"
+  <img src="../images/learning_environment_example.png"
        alt="Example ML-Agents Scene Block Diagram"
        width="700"
        border="10" />
@@ -247,7 +246,7 @@ Channels_ is to exchange data with Python about _Environment Parameters_. The
 following diagram illustrates the above.
 
 <p align="center">
-  <img src="images/learning_environment_full.png"
+  <img src="../images/learning_environment_full.png"
        alt="More Complete Example ML-Agents Scene Block Diagram"
        border="10" />
 </p>
@@ -467,7 +466,7 @@ episodes of demonstrations can reduce training steps by more than 4 times. See
 Behavioral Cloning + GAIL + Curiosity + RL below.
 
 <p align="center">
-  <img src="images/mlagents-ImitationAndRL.png"
+  <img src="../images/mlagents-ImitationAndRL.png"
        alt="Using Demonstrations with Reinforcement Learning"
        width="700" border="0" />
 </p>

docs/ML-Agents-README.md

@@ -0,0 +1 @@
+{!../ml-agents/README.md!}

docs/ML-Agents-Toolkit-Documentation.md

@@ -0,0 +1,80 @@
+# Unity ML-Agents Toolkit Documentation
+
+## Installation & Set-up
+
+- [Installation](Installation.md)
+  - [Using Virtual Environment](Using-Virtual-Environment.md)
+
+## Getting Started
+
+- [Getting Started Guide](Getting-Started.md)
+- [ML-Agents Toolkit Overview](ML-Agents-Overview.md)
+  - [Background: Unity](Background-Unity.md)
+  - [Background: Machine Learning](Background-Machine-Learning.md)
+  - [Background: PyTorch](Background-PyTorch.md)
+- [Example Environments](Learning-Environment-Examples.md)
+
+## Creating Learning Environments
+
+- [Making a New Learning Environment](Learning-Environment-Create-New.md)
+- [Designing a Learning Environment](Learning-Environment-Design.md)
+  - [Designing Agents](Learning-Environment-Design-Agents.md)
+- [Using an Executable Environment](Learning-Environment-Executable.md)
+- [ML-Agents Package Settings](Package-Settings.md)
+
+## Training & Inference
+
+- [Training ML-Agents](Training-ML-Agents.md)
+  - [Training Configuration File](Training-Configuration-File.md)
+  - [Using TensorBoard to Observe Training](Using-Tensorboard.md)
+  - [Profiling Trainers](Profiling-Python.md)
+- [Unity Inference Engine](Unity-Inference-Engine.md)
+
+## Extending ML-Agents
+
+- [Creating Custom Side Channels](Custom-SideChannels.md)
+- [Creating Custom Samplers for Environment Parameter Randomization](Training-ML-Agents.md#defining-a-new-sampler-type)
+
+## Python Tutorial with Google Colab
+
+- [Using a UnityEnvironment](https://colab.research.google.com/github/Unity-Technologies/ml-agents/blob/release_19_docs/colab/Colab_UnityEnvironment_1_Run.ipynb)
+- [Q-Learning with a UnityEnvironment](https://colab.research.google.com/github/Unity-Technologies/ml-agents/blob/release_19_docs/colab/Colab_UnityEnvironment_2_Train.ipynb)
+- [Using Side Channels on a UnityEnvironment](https://colab.research.google.com/github/Unity-Technologies/ml-agents/blob/release_19_docs/colab/Colab_UnityEnvironment_3_SideChannel.ipynb)
+
+## Help
+
+- [Migrating from earlier versions of ML-Agents](Migrating.md)
+- [Frequently Asked Questions](FAQ.md)
+- [ML-Agents Glossary](Glossary.md)
+- [Limitations](Limitations.md)
+
+## API Docs
+
+- [API Reference](API-Reference.md)
+- [Python API Documentation](Python-LLAPI-Documentation.md)
+- [How to use the Python API](Python-LLAPI.md)
+- [How to use the Unity Environment Registry](Unity-Environment-Registry.md)
+- [Wrapping Learning Environment as a Gym (+Baselines/Dopamine Integration)](Python-Gym-API.md)
+
+## Translations
+
+To make the Unity ML-Agents Toolkit accessible to the global research and Unity
+developer communities, we're attempting to create and maintain translations of
+our documentation. We've started with translating a subset of the documentation
+to one language (Chinese), but we hope to continue translating more pages and to
+other languages. Consequently, we welcome any enhancements and improvements from
+the community.
+
+- [Chinese](../localized_docs/zh-CN/)
+- [Korean](../localized_docs/KR/)
+
+## Deprecated Docs
+
+We no longer use them ourselves and so they may not be up-to-date. We've decided
+to keep them up just in case they are helpful to you.
+
+- [Windows Anaconda Installation](Installation-Anaconda-Windows.md)
+- [Using Docker](Using-Docker.md)
+- [Training on the Cloud with Amazon Web Services](Training-on-Amazon-Web-Service.md)
+- [Training on the Cloud with Microsoft Azure](Training-on-Microsoft-Azure.md)
+- [Using the Video Recorder](https://github.com/Unity-Technologies/video-recorder)

docs/Python-Gym-API.md

@@ -12,7 +12,7 @@ Unity environment via Python.
 ## Installation
 
 The gym wrapper is part of the `mlgents_envs` package. Please refer to the
-[mlagents_envs installation instructions](../ml-agents-envs/README.md).
+[mlagents_envs installation instructions](ML-Agents-Envs-README.md).
 
 
 ## Using the Gym Wrapper

docs/Python-PettingZoo-API.md

@@ -7,6 +7,9 @@ interfacing with a Unity environment via Python.
 
 ## Installation and Examples
 
+The PettingZoo wrapper is part of the `mlgents_envs` package. Please refer to the
+[mlagents_envs installation instructions](ML-Agents-Envs-README.md).
+
 [[Colab] PettingZoo Wrapper Example](https://colab.research.google.com/github/Unity-Technologies/ml-agents/blob/develop-python-api-ga/ml-agents-envs/colabs/Colab_PettingZoo.ipynb)
 
 This colab notebook demonstrates the example usage of the wrapper, including installation,

docs/Readme.md

@@ -1,80 +1,181 @@
-# Unity ML-Agents Toolkit Documentation
+# Unity ML-Agents Toolkit
 
-## Installation & Set-up
+[![docs badge](https://img.shields.io/badge/docs-reference-blue.svg)](https://github.com/Unity-Technologies/ml-agents/tree/release_19_docs/docs/)
 
-- [Installation](Installation.md)
-  - [Using Virtual Environment](Using-Virtual-Environment.md)
+[![license badge](https://img.shields.io/badge/license-Apache--2.0-green.svg)](../LICENSE.md)
 
-## Getting Started
+([latest release](https://github.com/Unity-Technologies/ml-agents/releases/tag/latest_release))
+([all releases](https://github.com/Unity-Technologies/ml-agents/releases))
 
-- [Getting Started Guide](Getting-Started.md)
-- [ML-Agents Toolkit Overview](ML-Agents-Overview.md)
-  - [Background: Unity](Background-Unity.md)
-  - [Background: Machine Learning](Background-Machine-Learning.md)
-  - [Background: PyTorch](Background-PyTorch.md)
-- [Example Environments](Learning-Environment-Examples.md)
+**The Unity Machine Learning Agents Toolkit** (ML-Agents) is an open-source
+project that enables games and simulations to serve as environments for
+training intelligent agents. We provide implementations (based on PyTorch)
+of state-of-the-art algorithms to enable game developers and hobbyists to easily
+train intelligent agents for 2D, 3D and VR/AR games. Researchers can also use the
+provided simple-to-use Python API to train Agents using reinforcement learning,
+imitation learning, neuroevolution, or any other methods. These trained agents can be
+used for multiple purposes, including controlling NPC behavior (in a variety of
+settings such as multi-agent and adversarial), automated testing of game builds
+and evaluating different game design decisions pre-release. The ML-Agents
+Toolkit is mutually beneficial for both game developers and AI researchers as it
+provides a central platform where advances in AI can be evaluated on Unity’s
+rich environments and then made accessible to the wider research and game
+developer communities.
 
-## Creating Learning Environments
+## Features
+- 18+ [example Unity environments](Learning-Environment-Examples.md)
+- Support for multiple environment configurations and training scenarios
+- Flexible Unity SDK that can be integrated into your game or custom Unity scene
+- Support for training single-agent, multi-agent cooperative, and multi-agent
+  competitive scenarios via several Deep Reinforcement Learning algorithms (PPO, SAC, MA-POCA, self-play).
+- Support for learning from demonstrations through two Imitation Learning algorithms (BC and GAIL).
+- Easily definable Curriculum Learning scenarios for complex tasks
+- Train robust agents using environment randomization
+- Flexible agent control with On Demand Decision Making
+- Train using multiple concurrent Unity environment instances
+- Utilizes the [Unity Inference Engine](Unity-Inference-Engine.md) to
+  provide native cross-platform support
+- Unity environment [control from Python](Python-LLAPI.md)
+- Wrap Unity learning environments as a [gym](Python-Gym-API.md) environment
+- Wrap Unity learning environments as a [PettingZoo](Python-PettingZoo-API.md) environment
 
-- [Making a New Learning Environment](Learning-Environment-Create-New.md)
-- [Designing a Learning Environment](Learning-Environment-Design.md)
-  - [Designing Agents](Learning-Environment-Design-Agents.md)
-- [Using an Executable Environment](Learning-Environment-Executable.md)
-- [ML-Agents Package Settings](Package-Settings.md)
+See our [ML-Agents Overview](ML-Agents-Overview.md) page for detailed
+descriptions of all these features.
+## Releases & Documentation
 
-## Training & Inference
+**Our latest, stable release is `Release 19`. Click
+[here](https://github.com/Unity-Technologies/ml-agents/tree/release_19_docs/docs/Readme.md)
+to get started with the latest release of ML-Agents.**
 
-- [Training ML-Agents](Training-ML-Agents.md)
-  - [Training Configuration File](Training-Configuration-File.md)
-  - [Using TensorBoard to Observe Training](Using-Tensorboard.md)
-  - [Profiling Trainers](Profiling-Python.md)
-- [Unity Inference Engine](Unity-Inference-Engine.md)
+The table below lists all our releases, including our `main` branch which is
+under active development and may be unstable. A few helpful guidelines:
+- The [Versioning page](Versioning.md) overviews how we manage our GitHub
+  releases and the versioning process for each of the ML-Agents components.
+- The [Releases page](https://github.com/Unity-Technologies/ml-agents/releases)
+  contains details of the changes between releases.
+- The [Migration page](Migrating.md) contains details on how to upgrade
+  from earlier releases of the ML-Agents Toolkit.
+- The **Documentation** links in the table below include installation and usage
+  instructions specific to each release. Remember to always use the
+  documentation that corresponds to the release version you're using.
+- The `com.unity.ml-agents` package is [verified](https://docs.unity3d.com/2020.1/Documentation/Manual/pack-safe.html)
+  for Unity 2020.1 and later. Verified packages releases are numbered 1.0.x.
 
-## Extending ML-Agents
+| **Version** | **Release Date** | **Source** | **Documentation** | **Download** | **Python Package** | **Unity Package** |
+|:-------:|:------:|:-------------:|:-------:|:------------:|:------------:|:------------:|
+| **main (unstable)** | -- | [source](https://github.com/Unity-Technologies/ml-agents/tree/main) | [docs](https://github.com/Unity-Technologies/ml-agents/tree/main/docs/Readme.md) | [download](https://github.com/Unity-Technologies/ml-agents/archive/main.zip) | -- | -- |
+| **Release 19** | **January 14, 2022** | **[source](https://github.com/Unity-Technologies/ml-agents/tree/release_19)** | **[docs](https://github.com/Unity-Technologies/ml-agents/tree/release_19_docs/docs/Readme.md)** | **[download](https://github.com/Unity-Technologies/ml-agents/archive/release_19.zip)** | **[0.28.0](https://pypi.org/project/mlagents/0.28.0/)** | **[2.2.1](https://docs.unity3d.com/Packages/com.unity.ml-agents@2.2/manual/index.html)** |
+| **Verified Package 1.0.8** | **May 26, 2021** | **[source](https://github.com/Unity-Technologies/ml-agents/tree/com.unity.ml-agents_1.0.8)** | **[docs](https://github.com/Unity-Technologies/ml-agents/blob/release_2_verified_docs/docs/Readme.md)** | **[download](https://github.com/Unity-Technologies/ml-agents/archive/com.unity.ml-agents_1.0.8.zip)** | **[0.16.1](https://pypi.org/project/mlagents/0.16.1/)** | **[1.0.8](https://docs.unity3d.com/Packages/com.unity.ml-agents@1.0/manual/index.html)** |
 
-- [Creating Custom Side Channels](Custom-SideChannels.md)
-- [Creating Custom Samplers for Environment Parameter Randomization](Training-ML-Agents.md#defining-a-new-sampler-type)
+If you are a researcher interested in a discussion of Unity as an AI platform,
+see a pre-print of our
+[reference paper on Unity and the ML-Agents Toolkit](https://arxiv.org/abs/1809.02627).
 
-## Python Tutorial with Google Colab
+If you use Unity or the ML-Agents Toolkit to conduct research, we ask that you
+cite the following paper as a reference:
 
-- [Using a UnityEnvironment](https://colab.research.google.com/github/Unity-Technologies/ml-agents/blob/release_19_docs/colab/Colab_UnityEnvironment_1_Run.ipynb)
-- [Q-Learning with a UnityEnvironment](https://colab.research.google.com/github/Unity-Technologies/ml-agents/blob/release_19_docs/colab/Colab_UnityEnvironment_2_Train.ipynb)
-- [Using Side Channels on a UnityEnvironment](https://colab.research.google.com/github/Unity-Technologies/ml-agents/blob/release_19_docs/colab/Colab_UnityEnvironment_3_SideChannel.ipynb)
+Juliani, A., Berges, V., Teng, E., Cohen, A., Harper, J., Elion, C., Goy, C.,
+Gao, Y., Henry, H., Mattar, M., Lange, D. (2020). Unity: A General Platform for
+Intelligent Agents. _arXiv preprint
+[arXiv:1809.02627](https://arxiv.org/abs/1809.02627)._
+https://github.com/Unity-Technologies/ml-agents.
 
-## Help
+## Additional Resources
 
-- [Migrating from earlier versions of ML-Agents](Migrating.md)
-- [Frequently Asked Questions](FAQ.md)
-- [ML-Agents Glossary](Glossary.md)
-- [Limitations](Limitations.md)
+We have a Unity Learn course,
+[ML-Agents: Hummingbirds](https://learn.unity.com/course/ml-agents-hummingbirds),
+that provides a gentle introduction to Unity and the ML-Agents Toolkit.
 
-## API Docs
+We've also partnered with
+[CodeMonkeyUnity](https://www.youtube.com/c/CodeMonkeyUnity) to create a
+[series of tutorial videos](https://www.youtube.com/playlist?list=PLzDRvYVwl53vehwiN_odYJkPBzcqFw110)
+on how to implement and use the ML-Agents Toolkit.
 
-- [API Reference](API-Reference.md)
-- [Python API Documentation](Python-LLAPI-Documentation.md)
-- [How to use the Python API](Python-LLAPI.md)
-- [How to use the Unity Environment Registry](Unity-Environment-Registry.md)
-- [Wrapping Learning Environment as a Gym (+Baselines/Dopamine Integration)](Python-Gym-API.md)
+We have also published a series of blog posts that are relevant for ML-Agents:
 
-## Translations
+- (July 12, 2021)
+  [ML-Agents plays Dodgeball](https://blog.unity.com/technology/ml-agents-plays-dodgeball)
+- (May 5, 2021)
+  [ML-Agents v2.0 release: Now supports training complex cooperative behaviors](https://blogs.unity3d.com/2021/05/05/ml-agents-v2-0-release-now-supports-training-complex-cooperative-behaviors/)
+- (December 28, 2020)
+  [Happy holidays from the Unity ML-Agents team!](https://blogs.unity3d.com/2020/12/28/happy-holidays-from-the-unity-ml-agents-team/)
+- (November 20, 2020)
+  [How Eidos-Montréal created Grid Sensors to improve observations for training agents](https://blogs.unity3d.com/2020/11/20/how-eidos-montreal-created-grid-sensors-to-improve-observations-for-training-agents/)
+- (November 11, 2020)
+  [2020 AI@Unity interns shoutout](https://blogs.unity3d.com/2020/11/11/2020-aiunity-interns-shoutout/)
+- (May 12, 2020)
+  [Announcing ML-Agents Unity Package v1.0!](https://blogs.unity3d.com/2020/05/12/announcing-ml-agents-unity-package-v1-0/)
+- (February 28, 2020)
+  [Training intelligent adversaries using self-play with ML-Agents](https://blogs.unity3d.com/2020/02/28/training-intelligent-adversaries-using-self-play-with-ml-agents/)
+- (November 11, 2019)
+  [Training your agents 7 times faster with ML-Agents](https://blogs.unity3d.com/2019/11/11/training-your-agents-7-times-faster-with-ml-agents/)
+- (October 21, 2019)
+  [The AI@Unity interns help shape the world](https://blogs.unity3d.com/2019/10/21/the-aiunity-interns-help-shape-the-world/)
+- (April 15, 2019)
+  [Unity ML-Agents Toolkit v0.8: Faster training on real games](https://blogs.unity3d.com/2019/04/15/unity-ml-agents-toolkit-v0-8-faster-training-on-real-games/)
+- (March 1, 2019)
+  [Unity ML-Agents Toolkit v0.7: A leap towards cross-platform inference](https://blogs.unity3d.com/2019/03/01/unity-ml-agents-toolkit-v0-7-a-leap-towards-cross-platform-inference/)
+- (December 17, 2018)
+  [ML-Agents Toolkit v0.6: Improved usability of Brains and Imitation Learning](https://blogs.unity3d.com/2018/12/17/ml-agents-toolkit-v0-6-improved-usability-of-brains-and-imitation-learning/)
+- (October 2, 2018)
+  [Puppo, The Corgi: Cuteness Overload with the Unity ML-Agents Toolkit](https://blogs.unity3d.com/2018/10/02/puppo-the-corgi-cuteness-overload-with-the-unity-ml-agents-toolkit/)
+- (September 11, 2018)
+  [ML-Agents Toolkit v0.5, new resources for AI researchers available now](https://blogs.unity3d.com/2018/09/11/ml-agents-toolkit-v0-5-new-resources-for-ai-researchers-available-now/)
+- (June 26, 2018)
+  [Solving sparse-reward tasks with Curiosity](https://blogs.unity3d.com/2018/06/26/solving-sparse-reward-tasks-with-curiosity/)
+- (June 19, 2018)
+  [Unity ML-Agents Toolkit v0.4 and Udacity Deep Reinforcement Learning Nanodegree](https://blogs.unity3d.com/2018/06/19/unity-ml-agents-toolkit-v0-4-and-udacity-deep-reinforcement-learning-nanodegree/)
+- (May 24, 2018)
+  [Imitation Learning in Unity: The Workflow](https://blogs.unity3d.com/2018/05/24/imitation-learning-in-unity-the-workflow/)
+- (March 15, 2018)
+  [ML-Agents Toolkit v0.3 Beta released: Imitation Learning, feedback-driven features, and more](https://blogs.unity3d.com/2018/03/15/ml-agents-v0-3-beta-released-imitation-learning-feedback-driven-features-and-more/)
+- (December 11, 2017)
+  [Using Machine Learning Agents in a real game: a beginner’s guide](https://blogs.unity3d.com/2017/12/11/using-machine-learning-agents-in-a-real-game-a-beginners-guide/)
+- (December 8, 2017)
+  [Introducing ML-Agents Toolkit v0.2: Curriculum Learning, new environments, and more](https://blogs.unity3d.com/2017/12/08/introducing-ml-agents-v0-2-curriculum-learning-new-environments-and-more/)
+- (September 19, 2017)
+  [Introducing: Unity Machine Learning Agents Toolkit](https://blogs.unity3d.com/2017/09/19/introducing-unity-machine-learning-agents/)
+- Overviewing reinforcement learning concepts
+  ([multi-armed bandit](https://blogs.unity3d.com/2017/06/26/unity-ai-themed-blog-entries/)
+  and
+  [Q-learning](https://blogs.unity3d.com/2017/08/22/unity-ai-reinforcement-learning-with-q-learning/))
 
-To make the Unity ML-Agents Toolkit accessible to the global research and Unity
-developer communities, we're attempting to create and maintain translations of
-our documentation. We've started with translating a subset of the documentation
-to one language (Chinese), but we hope to continue translating more pages and to
-other languages. Consequently, we welcome any enhancements and improvements from
-the community.
+### More from Unity
 
-- [Chinese](localized/zh-CN/)
-- [Korean](localized/KR/)
+- [Unity Simulation Pro](https://unity.com/products/unity-simulation-pro)
+- [Unity Robotics](https://github.com/Unity-Technologies/Unity-Robotics-Hub)
+- [Unity Computer Vision](https://unity.com/computer-vision)
 
-## Deprecated Docs
+## Community and Feedback
 
-We no longer use them ourselves and so they may not be up-to-date. We've decided
-to keep them up just in case they are helpful to you.
+The ML-Agents Toolkit is an open-source project and we encourage and welcome
+contributions. If you wish to contribute, be sure to review our
+[contribution guidelines](CONTRIBUTING.md) and
+[code of conduct](../CODE_OF_CONDUCT.md).
 
-- [Windows Anaconda Installation](Installation-Anaconda-Windows.md)
-- [Using Docker](Using-Docker.md)
-- [Training on the Cloud with Amazon Web Services](Training-on-Amazon-Web-Service.md)
-- [Training on the Cloud with Microsoft Azure](Training-on-Microsoft-Azure.md)
-- [Using the Video Recorder](https://github.com/Unity-Technologies/video-recorder)
+For problems with the installation and setup of the ML-Agents Toolkit, or
+discussions about how to best setup or train your agents, please create a new
+thread on the
+[Unity ML-Agents forum](https://forum.unity.com/forums/ml-agents.453/) and make
+sure to include as much detail as possible. If you run into any other problems
+using the ML-Agents Toolkit or have a specific feature request, please
+[submit a GitHub issue](https://github.com/Unity-Technologies/ml-agents/issues).
+
+Please tell us which samples you would like to see shipped with the ML-Agents Unity
+package by replying to
+[this forum thread](https://forum.unity.com/threads/feedback-wanted-shipping-sample-s-with-the-ml-agents-package.1073468/).
+
+
+Your opinion matters a great deal to us. Only by hearing your thoughts on the
+Unity ML-Agents Toolkit can we continue to improve and grow. Please take a few
+minutes to
+[let us know about it](https://unitysoftware.co1.qualtrics.com/jfe/form/SV_55pQKCZ578t0kbc).
+
+For any other questions or feedback, connect directly with the ML-Agents team at
+ml-agents@unity3d.com.
+
+## Privacy
+
+In order to improve the developer experience for Unity ML-Agents Toolkit, we have added in-editor analytics.
+Please refer to "Information that is passively collected by Unity" in the
+[Unity Privacy Policy](https://unity3d.com/legal/privacy-policy).

docs/com.unity.ml-agents.md

@@ -0,0 +1 @@
+{!../com.unity.ml-agents/Documentation~/com.unity.ml-agents.md!}

docs/extra.css

@@ -0,0 +1,3 @@
+.wy-nav-top, .wy-side-nav-search {
+    background: #439b47;
+}

docs/index.md

@@ -0,0 +1,2 @@
+<img src="images/U_MachineLearningAgents_Logo_Black_RGB.png" align="middle" width="3000"/>
+{!README.md!}