* Dropped support for python 3.6
* Pinning python 3.9.9 for tests due to typing issues with 3.9.10
* Testing new bokken image.
* Testing new bokken image.
* Updated yamato standalone build test.
* Updated yamato standalone build test.
* Updated standalone build test.
* Updated yamato configs to use mla bokken vm.
* Bug fixes for yamato yml files.
* Fixed com.unity.ml-agents-test.yml
* Bumped min python version to 3.7.2
* pettingzoo api prototype
* add example
* update file names
* support multiple behavior names
* fix multi behavior action index
* add install in colab
* add setup
* update colab
* fix __init__
* clone single branch
* import tags only
* import in init
* catch import error
* update colab
* move colab and add readme
* handle agent dying
* add tests
* update doc
* add info
* add action mask
* fix action mask
* update action masks in colab
* change default env
* set version
* fix hybrid action
* fix colab for hybrid actions
* add note on auto reset
* Updated colab name.
* Update README.md
* Following petting_zoo registry API (#5557)
* init petting_zoo registry
* cherrypick Custom trainer editor analytics (#5511)
* cherrypick "Update dotnet-format to address breaking changes introduced by upstream changes (#5528)"
* Update colab to match pettingZoo import api
* ToRevert: pull exp-petting-registry branch
* Add init file to tests
* Install pettingzoo-unity requirements for pytest
* update pytest command
* Add docstrings and comments
* update coverage to pettingzoo folder
* unset log level
* update env string
* Two small bugfixes (#5589)
1. Add the missing `_cumulative_rewards` property
2. Update `agent_selection` to not error out when an agent finishes an episode.
* Updated gym to 0.21.0 and petting zoo to 1.13.1, fixed bugs with AEC wrapper for gym and PZ updates. API tests are passing.
* Some refactoring.
* Finished inital implementation of parallel. Tests not passing.
* Finished parallel API implementation and refactor. All PZ tests passing.
* Cleanup.
* Refactoring.
* Pinning numpy version.
* add metadata and behavior_specs initialization
* addressing behaviour_spec issues
* Bumped PZ version to 1.14.0. Fixed failing tests.
* Refactored gym-unity and petting-zoo into ml-agents-envs
* Added TODO to pydoc-config.yaml
* Refactored gym and pz to be under a subpackage in mlagents_env package
* Refactored ml-agents-envs docs.
* Minor update to PZ API doc.
* Updated mlagents_envs docs and colab.
* Updated pytest gh workflow to remove ref to gym and pz.
* Refactored to remove some test coupling between trainers and envs.
* Updated installation doc.
* Update ml-agents-envs/README.md
Co-authored-by: Andrew Cohen <andrew.cohen@unity3d.com>
* Updated failing yamato jobs.
* pettingzoo api prototype
* add example
* update file names
* support multiple behavior names
* fix multi behavior action index
* add install in colab
* add setup
* update colab
* fix __init__
* clone single branch
* import tags only
* import in init
* catch import error
* update colab
* move colab and add readme
* handle agent dying
* add tests
* update doc
* add info
* add action mask
* fix action mask
* update action masks in colab
* change default env
* set version
* fix hybrid action
* fix colab for hybrid actions
* add note on auto reset
* Updated colab name.
* Update README.md
* Following petting_zoo registry API (#5557)
* init petting_zoo registry
* cherrypick Custom trainer editor analytics (#5511)
* cherrypick "Update dotnet-format to address breaking changes introduced by upstream changes (#5528)"
* Update colab to match pettingZoo import api
* ToRevert: pull exp-petting-registry branch
* Add init file to tests
* Install pettingzoo-unity requirements for pytest
* update pytest command
* Add docstrings and comments
* update coverage to pettingzoo folder
* unset log level
* update env string
* Two small bugfixes (#5589)
1. Add the missing `_cumulative_rewards` property
2. Update `agent_selection` to not error out when an agent finishes an episode.
* Updated gym to 0.21.0 and petting zoo to 1.13.1, fixed bugs with AEC wrapper for gym and PZ updates. API tests are passing.
* Some refactoring.
* Finished inital implementation of parallel. Tests not passing.
* Finished parallel API implementation and refactor. All PZ tests passing.
* Cleanup.
* Refactoring.
* Pinning numpy version.
* add metadata and behavior_specs initialization
* addressing behaviour_spec issues
* Bumped PZ version to 1.14.0. Fixed failing tests.
* Refactored gym-unity and petting-zoo into ml-agents-envs
* Added TODO to pydoc-config.yaml
* Refactored gym and pz to be under a subpackage in mlagents_env package
* Refactored ml-agents-envs docs.
* Minor update to PZ API doc.
* Updated mlagents_envs docs and colab.
* Updated pytest gh workflow to remove ref to gym and pz.
* Refactored to remove some test coupling between trainers and envs.
* Updated installation doc.
* Update ml-agents-envs/README.md
Co-authored-by: Andrew Cohen <andrew.cohen@unity3d.com>
* Updated CHANGELOG.
* Updated Migration guide.
* Doc updates based on CR.
* Updated github workflow for colab tests.
* Updated github workflow for colab tests.
* Updated github workflow for colab tests.
* Fixed yamato import error.
Co-authored-by: Ruo-Ping Dong <ruoping.dong@unity3d.com>
Co-authored-by: Miguel Alonso Jr <miguelalonsojr>
Co-authored-by: jmercado1985 <75792879+jmercado1985@users.noreply.github.com>
Co-authored-by: Maryam Honari <honari.m94@gmail.com>
Co-authored-by: Henry Peteet <henry.peteet@unity3d.com>
Co-authored-by: mahon94 <maryam.honari@unity3d.com>
Co-authored-by: Andrew Cohen <andrew.cohen@unity3d.com>
* Updated Project and DevProject to 2020.3 LTS.
* Updated supported LTS version to 2020.3.
* Updated changelog.
* Bumped up CI Unity version from 2020.2 to 2020.3.
* Update CHANGELOG.md
* Removing VS Studio dependency from manifest.json.
* Update packages-lock.json
* Updated versions for main. Also updated validate versions script to remove exp.1 from package version.
* Reverted validate versions.
* Updated versions.
* Bumped package version for patch.
* Update DQN GridWorld colab example to be capable of outputting an ONNX model that works for inference.
-Allow GridWorld to work without a GoalSensor. Defaults to the GreenPlus target in that case.
-Add GridWorldColab.onnx file, trained and generated with the colab.
* Upgrade DQN GridWorld Colab to produce model that can work with (modified) GridWorld Scene.
-permute observation images from NHWC to NCHW right after env_step. Everything following is NCHW.
-update input_shape indexes and remove equivalent permute from the VisualQNetwork
-fix reference to global qnet inside generate_trajectories (use the passed in version)
-create wrapper network to add required constants and dummy input masks required by the
runtime inference (Barracuda)
-export the wrapped network to ONNX format compatible with Barracuda
* Add additional GridWorld scene called GridWorldColab that has the goal sensors removed (target is always green plus) and masks disabled. The purpose of this is to have a scene compatible with the DQN GridWorld Colab example.
* Minor edit in Colab docs.
* Delete unnecessary .lighting file.
* Update changelog.
* Update colab/Colab_UnityEnvironment_2_Train.ipynb
Co-authored-by: Maryam Honari <honari.m94@gmail.com>
* Fix error in colab metadata name.
* Add suggested comments and fix incorrect parameter ordering.
* Update docs for Elo curriculum completion measure.
Co-authored-by: Maryam Honari <honari.m94@gmail.com>
* Progress on propagating the setting to the action model.
* Added the _sample_action logic and tests.
* Add information to the changelog.
* Prioritize the CLI over the configuration file.
* Update documentation for config file.
* CR refactor.
* Update docs/Training-Configuration-File.md
Co-authored-by: Miguel Alonso Jr. <76960110+miguelalonsojr@users.noreply.github.com>
Update com.unity.ml-agents/CHANGELOG.md
Co-authored-by: Miguel Alonso Jr. <76960110+miguelalonsojr@users.noreply.github.com>
Update com.unity.ml-agents/CHANGELOG.md
Co-authored-by: Miguel Alonso Jr. <76960110+miguelalonsojr@users.noreply.github.com>
Update com.unity.ml-agents/CHANGELOG.md
Co-authored-by: Maryam Honari <honari.m94@gmail.com>
Update ml-agents/mlagents/trainers/settings.py
Co-authored-by: Maryam Honari <honari.m94@gmail.com>
Update ml-agents/mlagents/trainers/cli_utils.py
Co-authored-by: Maryam Honari <honari.m94@gmail.com>
* Fix CR requests
* Add tests for discrete.
* Update ml-agents/mlagents/trainers/torch/distributions.py
Co-authored-by: Maryam Honari <honari.m94@gmail.com>
* Added more stable test.
* Return deterministic actions for training (#5615)
* Added more stable test.
* Fix the tests.
* Fix pre-commit
* Fix help line to pass precommit.
* support for deterministic inference in onnx (#5593)
* Init: actor.forward outputs separate deterministic actions
* changelog
* Renaming
* Add more tests
* Package changes to support deterministic inference (#5599)
* Init: actor.forward outputs separate deterministic actions
* fix tensor shape for discrete actions
* Add test and editor flag
- Add tests for deterministic sampling
- update editor and tooltips
* Reverting to "Deterministic Inference"
* dissect tests
* Update docs
* Update CHANGELOG.md
* Fix the deterministic showing up all the tiime (#5621)
Co-authored-by: Chingiz Mardanov <chingiz.mardanov@unity3d.com>
Co-authored-by: cmard <87716492+cmard@users.noreply.github.com>
* Progress on propagating the setting to the action model.
* Added the _sample_action logic and tests.
* Add information to the changelog.
* Prioritize the CLI over the configuration file.
* Update documentation for config file.
* CR refactor.
* Update docs/Training-Configuration-File.md
Co-authored-by: Miguel Alonso Jr. <76960110+miguelalonsojr@users.noreply.github.com>
Update com.unity.ml-agents/CHANGELOG.md
Co-authored-by: Miguel Alonso Jr. <76960110+miguelalonsojr@users.noreply.github.com>
Update com.unity.ml-agents/CHANGELOG.md
Co-authored-by: Miguel Alonso Jr. <76960110+miguelalonsojr@users.noreply.github.com>
Update com.unity.ml-agents/CHANGELOG.md
Co-authored-by: Maryam Honari <honari.m94@gmail.com>
Update ml-agents/mlagents/trainers/settings.py
Co-authored-by: Maryam Honari <honari.m94@gmail.com>
Update ml-agents/mlagents/trainers/cli_utils.py
Co-authored-by: Maryam Honari <honari.m94@gmail.com>
* Fix CR requests
* Add tests for discrete.
* Update ml-agents/mlagents/trainers/torch/distributions.py
Co-authored-by: Maryam Honari <honari.m94@gmail.com>
* Added more stable test.
* Return deterministic actions for training (#5615)
* Added more stable test.
* Fix the tests.
* Fix pre-commit
* Fix help line to pass precommit.
* support for deterministic inference in onnx (#5593)
* Init: actor.forward outputs separate deterministic actions
* changelog
* Renaming
* Add more tests
* Package changes to support deterministic inference (#5599)
* Init: actor.forward outputs separate deterministic actions
* fix tensor shape for discrete actions
* Add test and editor flag
- Add tests for deterministic sampling
- update editor and tooltips
* Reverting to "Deterministic Inference"
* dissect tests
* Update docs
* Update CHANGELOG.md
Co-authored-by: Chingiz Mardanov <chingiz.mardanov@unity3d.com>
Co-authored-by: cmard <87716492+cmard@users.noreply.github.com>
* Added normalization to critic during training for poca, ppo, and sac.
* Fixed critic normalization bug. Added new tests to cover.
* Updated CHANGELOG.
* Fixed typo in test_trainers.py
* Fixing the analytics side-channel for curriculum learning.
* Made a more robust test.
* Update the changelog.
* Update com.unity.ml-agents/CHANGELOG.md
Co-authored-by: Maryam Honari <honari.m94@gmail.com>
Co-authored-by: Maryam Honari <honari.m94@gmail.com>
* Added training area replicator to com.unity.ml-agents package.
* Added num_areas to Unity RL Initialization proto. Added cli and config file support for num_areas.
* Changed training area replicator to size grid automatically from number of areas.
* Added tests for the training area replicator.
* Added setup for tests for the training area replicator.
* Added comments and updated create tutorial for training area replicator.
* Updated CHANGELOG.
* Fixed some failing tests.
* Update com.unity.ml-agents/CHANGELOG.md
Co-authored-by: Henry Peteet <henry.peteet@unity3d.com>
* Update docs/Learning-Environment-Create-New.md
Co-authored-by: Henry Peteet <henry.peteet@unity3d.com>
* Update com.unity.ml-agents/Runtime/Areas/TrainingAreaReplicator.cs
Co-authored-by: Henry Peteet <henry.peteet@unity3d.com>
* Addressed CR comments.
Co-authored-by: Miguel Alonso Jr <miguelalonsojr>
Co-authored-by: Henry Peteet <henry.peteet@unity3d.com>
### Change(s)
Update the SubprocessEnvManager to restart workers when the underlying Unity environments crash.
When a worker receives an ENV_EXITED signal it will now:
1. Record all failures coming through the step queue and drop all other messages.
2. Purge any pending trajectories as they may belong to a crashed worker or be corrupted.
3. Restart all failed workers (up to a configurable limit)
This behavior can be limited via a rate limit, max lifetime limit, or both. The configuration options for both are shown below with their default values.
⚠️ Each of these options applies to a single environment, if num_envs > 1 then the limit will apply separately to each replica (num_envs = 2 will spawn 2 Unity environments which can each be restarted 10 times).
```yaml
env_settings:
# Can restart 10 times over the lifetime of the experiment.
max_lifetime_restarts: 10
# Rate limit of 1 failure per 60s
restarts_rate_limit_n: 1
restarts_rate_limit_period_s: 60
```
They can of course be passed via CLI arguments as well
```bash
--max-lifetime-restarts
--restarts-rate-limit-n
--restarts-rate-limit-period-s
```
### Disabling this feature
* Rate limiting can be turned off by setting `--restarts-rate-limit-n=-1`
* Lifetime limiting can be turned off by setting `--max-lifetime-restarts=-1`
* Hash128 is not a cryptographic hash, replace with HMAC-SHA256.
* Extend TrainingAnalytics side channel to expose configuration details
* Change member function scopes and hash demo_paths
* Extract tbiEvent hashing method and add test coverage
* Added initial (untested) support for separating schedule for beta, LR, and epsilon.
* Added support for beta and epsilon schedule in test config conversion
* Updated docs. Set epsilon to be linear by default.
* Updated configs to add beta and epsilon schedule where appropriate.
* Addressed CR change requests.
* Minor update.
* Updated docs.
* Added changes to CHANGELOG.
* Updated CHANGELOG.
* Fixed typo in Training-Configuration-File.md
* Reverted config changes to remove beta and epsilon schedules.
* mypy error fixed from pre-commit.
* Fixed a bug that was causing a settings test to fail.
* Addressed CR comments.
Co-authored-by: Miguel Alonso Jr <miguel.alonso@miguel.alonso-00MM>
Co-authored-by: Miguel Alonso Jr <miguelalonsojr>
Maryam Honari
Jason Bowman
Miguel Alonso Jr
Xun Cao
Miguel Alonso Jr
Xun Cao
Jason Rupert
Henry Peteet
Maryam Honari
cmard
Vincent-Pierre BERGES
andrewcoh