* update relase tag references
* add table in readme (#5816)
* nit updates (#5823)
* prepare main for dev
* update python to v0.31.0-dev0, add unreleased section
* yamato promotion min editor bumped to 2020.3 (#34)
* Fix pypi publish github action (#35)
* update logo (#39)
* Move SB3 dependencies to colab (#29)
* Basic example running with multiple envs in sb3
* Merge & cleanup
* trains 3dball
* move SB3 helpers into colab
* remove sb3/supersuit hard reqs
* remove sb3 funcs
* remove unused imports
* remove sb3 example directory and script
* remove vec env tests
* clean colab
* fix link
Co-authored-by: Henry Peteet <henry.peteet@unity3d.com>
Co-authored-by: mahon94 <maryam.honari@unity3d.com>
* bump to barracuda 3.0.0 (#30)
* bump to barracuda 3.0.0 & Updated submodule com.unity.ml-agents
* Feature/mla 2394 update web docs (#37)
* Added mkdocs and refactored docs.
* Updated docs.
* Added gh action to manually public docs. Updated mkdocs.yml.
* fix markdown docs linking issue (#40)
* Updated submodule com.unity.ml-agents (#41)
* Add sonarqube yamato job (#38)
* Pinning markdown link check to 3.8.7 on nightly gha. (#43)
* Update submodules
* pin black version to 22.3.0 (#46)
* fix-sonar-bugs (#44)
* Update submodules
* bump version to 0.29.0 (#42)
* bump version to 0.29.0
* remove dev tag
* add checks for pytorch-gpu install
* pin black version to 22.3.0
* Update submodules
* Update automated trigger references to support develop branch (#36)
* Update automated trigger references to support develop branch
* Update pull request template links to refer to develop branch.
* Fix release branch name for github actions
* Update pre-commit pattern
* Removing test that no longer applys to this repo
* Removed dead links in pull request template. (#48)
* Use pinned version of package-ci/mac image (#50)
* Add shared critic configurability for PPO (#45)
* Update submodules
* Updated Unity version to 2020.3.33f1. (#53)
* Update submodules (#56)
Co-authored-by: GitHub Actions - update submodules <bot@noreply.github.com>
* Add poca reference and update to latex refs (#57)
* add poca ref in docs (#58)
* Update submodules
* Update Training-Configuration-File.md (#59)
* Update Training-Configuration-File.md
Doc says "behavior_cloning" when it should say "behavioral_cloning".
* Update Learning-Environment-Design-Agents.md
Minor correction to wording. (https://github.com/Unity-Technologies/ml-agents/issues/5745)
* rename ml-agents/trainers/torch to torch_entities (#55)
* rename ml-agents/trainers/torch to torch_entities
* fix mock patch
* Update submodules
* Update docs to explain Alternating Ray Order switch (mla-2537) (#61)
* Update docs to explain Alternating Ray Order switch (mla-2537)
* Update submodules
* Update submodules
* Updated validate meta files. (#65)
* Fix failing gha pytest (#66)
* Updated pytest.
* Updated SURVEY.md to remove dead survey link.
* Updated bug_report.md to update dead link.
* Update submodules
* Fixed upm ci bug in yamato tests. (#67)
* Update submodules
* remove torch shadow in trainers tests (#69)
* extend stale bot to 90 days (#70)
* Update submodules
* Dev bump min ver (#74)
* Bumped minimum version of Unity.
* Bumped minimum version of python and pytorch. fixed issues with deprecation warnings for x.T (tensor transpose).
* Fixed np.bool deprecation warning. This is a port of an external PR: https://github.com/Unity-Technologies/ml-agents/pull/5795. (#71)
* Updated submodule.
* Bumped max ver of python.
* Bumped to 2021.3 in yamato tests.
* Fixed version issue with torch in test constraints.
* update sysroot version
* update toolchain version
Co-authored-by: zhuo <zhuo@unity3d.com>
* Removed old backwards compatibility test since tensorflow/2019.4 are no longer s upported. (#76)
* Develop custom trainers (#73)
* Make create_policy more generic (#54)
* add on/off policy classes and inherit from
* trainers as plugins
* remove swap files
* clean up registration debug
* clean up all pre-commit
* a2c plugin pass precommit
* move gae to trainer utils
* move lambda return to trainer util
* add validator for num_epoch
* add types for settings/type methods
* move create policy into highest level api
* move update_reward_signal into optimizer
* move get_policy into Trainer
* remove get settings type
* dummy_config settings
* move all stats from actor into dict, enables arbitrary actor data
* remove shared_critic flag, cleanups
* refactor create_policy
* remove sample_actions, evaluate_actions, update_norm from policy
* remove comments
* fix return type get stat
* update poca create_policy
* clean up policy init
* remove conftest
* add sharedecritic to settings
* fix test_networks
* fix test_policy
* fix test network
* fix some ppo/sac tests
* add back conftest.py
* improve specification of trainer type
* add defaults fpr trainer_type/hyperparam
* fix test_saver
* fix reward providers
* add settings check utility for tests
* fix some settings tests
* add trainer types to run_experiment
* type check for arbitary actor data
* cherrypick rename ml-agents/trainers/torch to torch_entities (#55)
* make all trainers types and setting visible at module level
* remove settings from run_experiment console script
* fix test_settings and upgrade config scripts
* remove need of trainer_type argument up to trainefactory
* fix gohst trainer behavior id in policy Queue
* fix torch shadow in tests
* update trainers, rl trainers tests
* update tests to match the refactors
* fixing behavior name in ghost trainer
* update ml-agents-envs test configs
* separating the plugin package changes
* bring get_policy back for sake of ghost trainer
* add return types and remove unused returns
* remove duplicate methods in poca (_update_policy, add_policy)
Co-authored-by: mahon94 <maryam.honari@unity3d.com>
* Online/offline custom trainer examples with plugin system (#52)
* add on/off policy classes and inherit from
* trainers as plugins
* a2c trains
* remove swap files
* clean up registration debug
* clean up all pre-commit
* a2c plugin pass precommit
* move gae to trainer utils
* move lambda return to trainer util
* add validator for num_epoch
* add types for settings/type methods
* move create policy into highest level api
* move update_reward_signal into optimizer
* move get_policy into Trainer
* remove get settings type
* dummy_config settings
* move all stats from actor into dict, enables arbitrary actor data
* remove shared_critic flag, cleanups
* refactor create_policy
* remove sample_actions, evaluate_actions, update_norm from policy
* remove comments
* fix return type get stat
* update poca create_policy
* clean up policy init
* remove conftest
* add sharedecritic to settings
* fix test_networks
* fix test_policy
* fix test network
* fix some ppo/sac tests
* add back conftest.py
* improve specification of trainer type
* add defaults fpr trainer_type/hyperparam
* fix test_saver
* fix reward providers
* add settings check utility for tests
* fix some settings tests
* add trainer types to run_experiment
* type check for arbitary actor data
* cherrypick rename ml-agents/trainers/torch to torch_entities (#55)
* make all trainers types and setting visible at module level
* remove settings from run_experiment console script
* fix test_settings and upgrade config scripts
* remove need of trainer_type argument up to trainefactory
* fix gohst trainer behavior id in policy Queue
* fix torch shadow in tests
* update trainers, rl trainers tests
* update tests to match the refactors
* fixing behavior name in ghost trainer
* update ml-agents-envs test configs
* fix precommit
* separating the plugin package changes
* bring get_policy back for sake of ghost trainer
* add return types and remove unused returns
* remove duplicate methods in poca (_update_policy, add_policy)
* add a2c trainer back
* Add DQN cleaned up trainer/optimizer
* nit naming
* fix logprob/entropy types in torch_policy.py
* clean up DQN/SAC
* add docs for custom trainers,TODO: refrence tutorial
* add docs for custom trainers,TODO: refrence tutorial
* add clipping to loss function
* set old importlim-metadata version
* bump precomit hook env to 3.8.x
* use smooth l1 loss
Co-authored-by: mahon94 <maryam.honari@unity3d.com>
* add tutorial for validation
* fix formatting errors
* clean up
* minor changes
Co-authored-by: Andrew Cohen <andrew.cohen@unity3d.com>
Co-authored-by: zhuo <zhuo@unity3d.com>
* Trainer qa fix (#78)
* grammer fixes
* fix nit comments from QA
* add info about on/off policy
* add more context to the code block
* more context and minor fix
* pip3
Co-authored-by: zhuo <zhuo@unity3d.com>
* Develop merge submodule (#77)
* Update com.unity.ml-agents/CHANGELOG.md
Co-authored-by: Arthur Juliani <awjuliani@gmail.com>
* Update changelog
* [MLA-1762] reduce memory allocations from DiscreteActionOutputApplier (#4922)
* add option for Burst inference (#4925)
* surface specific GRPC errors more visibly (#4930)
* Add additional logic to avoid load being called on every advance (#4934)
* [MLA-1767] Refactor communicator connection exceptions (#4935)
* Update changelog for release 13. (#4938)
* Update master versions for release 13. (#4945)
* Release 13 versions. (#4946)
- updated release tag validation script to automate the updating of files with release tags that need to be changed as part of the pre-commit operation.
* Update docs to pass doc validation. (#4953)
* update defines, compile out Initialize body on non-desktop (#4957)
* Adding references to the Extensions package to help promote it. (#4967)
* Adding references to the Extensions package to help promote it.
* Updating header formatting to match the rest of the page
* Adding references to the Extensions package to help promote it. (#4967) (#4968)
Co-authored-by: Marwan Mattar <marwan@unity3d.com>
Co-authored-by: Chris Elion <chris.elion@unity3d.com>
* InputActuatorComponent to allow the generation of an action space from an InputActionAsset (#4881)
* InputActuatorComponent to allow the generation of an action space from an InputActionAsset (#4881) (#4974)
* pass sensor dimension flags to analytics (#4954)
* Communicator factory (#4965)
* Update barracuda, swtich Agents in Sorter use Burst. (#4979)
* Update barracuda, swtich Agents in Sorter use Burst. (#4979) (#4981)
* Set ignore done=False in GAIL (#4971)
* MultiAgentGroup Interface (#4923)
* add SimpleMultiAgentGroup
* add group reward field to agent and proto
* Make TrainingAnalyticsSideChannel internal (#4999)
* [MLA-1783] built-in actuator type (#4950)
* Add component menues for some sensors and actuators. (#5001)
* Add component menues for some sensors and actuators. (#5001) (#5002)
* Merge master -> release_13_branch-to-master
* Fix RpcCommunicator merge.
* master -> main. (#5010)
* Adding a name field to BufferSensorComponent
* Adding a note to the CHANGELOG about var len obs
* Adding a helper method for creating observation placeholder names and removed the _h and _c placeholders
* Adding a custom editor for BufferSensorComponent
* adding inheritdoc
* Update cattrs dependencies to support python3.9 (#4821)
* Fix issue with queuing input events that stomp on others. (#5034)
* Update cattrs dependencies to support python3.9 (#4821)
* Fix issue with queuing input events that stomp on others. (#5034)
* Update versions for release 14 hotfix. (#5040)
* master -> main. (#5010) (#5044)
* Update changelog. (#5045)
* [MLA-1809] catch mismatched observation sizes (#5030)
* Update changelog. (#5055)
* Fix xml docs. (#5057)
* pass sensor name through to ObservationSpec (#5036)
* Remove unused allocation (#5068)
* Automatically generate samples based on placement of mlagents-sample.json files in our examples. (#5077)
* Update barracuda to 1.3.2-preview. (#5084)
* POCA trainer (#5005)
Co-authored-by: Ervin Teng <ervin@unity3d.com>
Co-authored-by: Ruo-Ping Dong <ruoping.dong@unity3d.com>
Co-authored-by: Chris Elion <chris.elion@unity3d.com>
Co-authored-by: Vincent-Pierre BERGES <vincentpierre@unity3d.com>
* [docs] Documentation for POCA and cooperative behaviors (#5056)
Co-authored-by: Ruo-Ping Dong <ruoping.dong@unity3d.com>
* [docs] Update changelog (#5095)
* Add multiAgentGroup capabilities flag (#5096)
* Add multiAgentGroup capabilities flag
* Add proto
* Fix compiler error
* Add warning for multiagent group
* Add comment
* Fix spelling mistake
* Fix ghost curriculum and make steps private (#5098)
* use get step to determine curriculum
* add to CHANGELOG
* Make step in trainer private (#5099)
Co-authored-by: Ervin T <ervin@unity3d.com>
* [cherry-pick] Fix ghost curriculum and make steps private (#5098)
* use get step to determine curriculum
* add to CHANGELOG
* Make step in trainer private (#5099)
Co-authored-by: Ervin T <ervin@unity3d.com>
* Update changelog for samples. (#5103)
* Update changelog for samples. (#5103) (#5106)
* Update versions on main (#5102)
Increment versions after release 15 branch split
* [release_15] Release 15 update versions (#5101)
* Update versions
* Fix for validate release links
* Update release tag and docs
* [docs] Add missing package docs (#5108)
* Add missing package docs
* Fix grammar
* Update comment on Dispose
* Fix typo in doc
* [docs] Update readme and changelog (#5107)
* Make analytics module an optional dependency. (#5109)
Co-authored-by: Chris Elion <chris.elion@unity3d.com>
* Redo dotnet format (#5119)
* [docs] Add links to example envs for Buffer Sensor and MultiAgentGroup (#5116)
* Update changelog with links to example envs
* Cooperative push block link update
* [cherry-pick] Integrate Group Manager to soccer/retrain with POCA (#5115) (#5121)
* Integrate Group Manager to soccer/retrain with POCA (#5115)
* Add Soccer env to changelog
Co-authored-by: andrewcoh <54679309+andrewcoh@users.noreply.github.com>
* [cherry-pick][docs] Add Dungeon Escape Environment (#5133)
* Add DungeonEscape POCA Environment (#5128)
* Add DungeonEscape assets from working branch
* Add Dungeon Escape docs
* Create dungeon_escape.png
* Add to docs
Co-authored-by: Hunter-Unity <hunter@unity3d.com>
* update pre-commit instructions for CONTRIBUTING.md (#5130)
* Remove duplicated changelog
* Fix another duplicated line in the changelog...
* Removing Obsolete methods from the package (#5024)
* Removing Obsolete methods from the package
* Missing depecration and modified changelog
* Readding the obsolete BrainParameter methods, will need a larger discussion on these
* Removing Action Masker, readding the warining when using a non-implemented Heuristic, Removing NumAction from Brain Parameters
* removing documentation and some calls to deprecated methods in the extensions package
* Editing the Changelog to put the unreleased on top
* non-IEnumerable interface for action masking (#5060)
* V2 staging new model version (#5080)
* Make modelCheck have flavors of error messages
* ONNX exporter v3
* Using a better CheckType and a switch statement
* Removing unused message
* More tests
* Use an enum for valid versions and use GetVersion on model directly
* Maybe the model export version a static constant in Python
* Use static constructor for FailedCheck
* Use static constructor for FailedCheck
* Modifying the docstrings
* renaming LegacyDiscreteActionOutputApplier
* removing testing code
* better warning message
* Nest the CheckTypeEnum into the FailedCheck class
* Update com.unity.ml-agents/Runtime/Inference/BarracudaModelParamLoader.cs
* Adding a line explaining that legacy tensor checks are for versions 1.X only
* Modifying the changelog
* Exporting all the branches size instead of omly the sum (#5092)
* addressing comments
* Update com.unity.ml-agents/Runtime/Inference/BarracudaModelParamLoader.cs
Co-authored-by: Chris Elion <chris.elion@unity3d.com>
* readding tests
* Adding a comment around the new DiscreteOutputSize method
* Clearer warning : Model contains unexpected input > Model requires unknown input
* Fixing a bug in the case where the discrete action tensor does not exist
Co-authored-by: Chris Elion <chris.elion@unity3d.com>
* Make IActuator implement IHeuristicProvider. (#5110)
* Adding more tests for v2 models (#5138)
* Adding more tests for v2 models
* formatting
* fixing formatting
* [MLA-1634] Add ObservationSpec and update ISensor interfaces (#5127)
* Add custom package settings (#5027)
* cleanup: yamato triggers, changelog, docstrings
* clean up merge
* fix compile error
* update c# version to 2.0.0-exp.1 (#5160)
Co-authored-by: Chris Elion <chris.elion@unity3d.com>
* Move physics/sensor tests that fail in edit mode depending on what's in the scene to playmode tests. (#5162)
* update package version
* remove defines that are now always true
* changelog
* Remove all old analytics defines. (#5168)
* [MLA-1634] Compression spec (#5164)
* Low hanging fruit tests for coverage. (#5170)
* Update Policies to conditionally compile their analytics method. (#5171)
* Turns physics modules into optional dependencies. (#5112)
* [MLA-1634] Remove SensorComponent.GetObservationShape() (#5172)
* PR Feedback. (#5173)
* change default barracuda behavior (#5175)
* [MLA-1824] make SensorComponent return ISensor[] (#5181)
* Make SensorComponent return an array
* split match3 sensors, partial retrain
* docstrings, migration, changelog, cleanup
* Adding the goal conditioning sensors with the new observation specs (#5159)
* Fixing networks.py for the merge
* fix compile error
* Adding the goal conditioning sensors with the new observation specs
* addressing feedback
* I forgot to change the m_observationType
* Renaming Goal to GoalSignal (#5190)
* Renaming GOAL to GOAL_SIGNAL
* VectorSensorComponent to use new API
* Adding docstrings
* verbose pytest on github action
Co-authored-by: Chris Elion <chris.elion@unity3d.com>
* [MLA-1879] culture-invariant sorting for sensors and actuators (#5194)
* Goal conditioning grid world : Example of goal conditioning (#5193)
* Aded the Goal conditioned GridWorld to replace regular gridworld
* adding missing files
* Code improvements
* Documentation change on gridworld
* resolving conflicts
* new model
* Addressing comments
* comments and renames
* Update docs/Learning-Environment-Examples.md
Co-authored-by: Ervin T. <ervin@unity3d.com>
* adding reference to gridworld in docs about goal signal
Co-authored-by: Chris Elion <chris.elion@unity3d.com>
Co-authored-by: Ervin T. <ervin@unity3d.com>
* [MLA-1584] Match3 variable board size (#5189)
* [bug-fix] When agent isn't training, don't clear update buffer (#5205)
* Don't clear update buffer, but don't append to it either
* Update changelog
* Address comments
* Make experience replay buffer saving more verbose
* Changelog for goal conditioning (#5195)
* Changelog for goal conditioning
* adding a line about the changes to the gridworld example
* Addressing comments
* Change capitalization
* [bug fix] Fix warning using demo recorder (#5216)
* Making VectorActionSize and VectorActionSpaceType internal (#5214)
Made sure the editor does not complain;
* Turning some logger.info into logger.debug and remove some logging overhead when not using debug (#5211)
* turning some logger.info into logger.debug and remove some logging overhead when not using debug
* Addressing comments
* Adding to changelog
* rearrange match3 docs (#5215)
* rearrange match3 docs
* changelog and migration from previous PR
* [bug-fix] Fix POCA LSTM, pad sequences in the back (#5206)
* Pad buffer at the end
* Fix padding in optimizer value estimate
* Fix additional bugs and POCA
* Fix groupmate obs, add tests
* Update changelog
* Improve tests
* Address comments
* Fix poca test
* Fix buffer test
* Increase entropy for Hallway
* Add EOF newline
* Fix Behavior Name
* Address comments
* [MLA-1880] Raycast sensor interface improvements (#5222)
* WIP
* remove debug info struct
* cleanup + add to test
* changelog
* fix unit tests
* PR feedback
* Load individual elements if state dict load fails (#5213)
Co-authored-by: Vincent-Pierre BERGES <vincentpierre@unity3d.com>
Co-authored-by: Ervin T. <ervin@unity3d.com>
* [MLA-1159] Add virtual methods to DecisionRequester (#5223)
* [Bug Fix] Serialization of vectorActionSpace (#5220)
* [Bug Fix] Serialization of vectorActionSpace
* adding a test
* Clear ActionBuffers before Heuristic calls (#5227)
* Add ObservationType to analytics ObservationSpec (#5218)
* [MLA-1909] Match3 and Camera/RenderTexture sensor GC improvements (#5233)
* avoid empty set iteration, avoid Debug.AssertFormat (#5246)
* avoid empty set iteration, avoid Debug.AssertFormat
* changelog
* Update to barracuda 1.3.3 and changes to the model inputs and outputs for LSTM (#5236)
* Initial commit
* making it work with hallway. Added a new model version
* addressing feedback
* Adding a comment
* formatting
* modifying the changelog
* Adding descriptions on the model version descriptions
* Update com.unity.ml-agents/Runtime/Inference/GeneratorImpl.cs
Co-authored-by: Chris Elion <chris.elion@unity3d.com>
* addressing comments
Co-authored-by: Chris Elion <chris.elion@unity3d.com>
* [Release 16] Release 16 Merge Back to Main (#5255)
Update versions and documentation for Release 16.
Co-authored-by: Vincent-Pierre BERGES <vincentpierre@unity3d.com>
Co-authored-by: andrewcoh <54679309+andrewcoh@users.noreply.github.com>
Co-authored-by: Chris Elion <chris.elion@unity3d.com>
* fix link, run link check in quiet mode (#5257)
* [MLA-1912] Move Match3 to com.unity.ml-agents (#5259)
* Move GridSensor into main package (#5256)
* move OneHotGridSensor into main package
* changelog and migration guide
* remove old doc
* check if physics module presents
* Add advanced installation guide for com.unity.ml-agents (#5260)
* Fix errors when creating new RayCast sensor (#5261)
* check if tag and angle is null
* add test
* changelog
* fix all PVS and doc generation warnings (#5262)
* Fix --results-dir (#5269)
* Fix GridSensorComponent bug (#5270)
* LSTM models from 1.x will be incompatible with MLA 2.x (#5254)
* LSTM models from 1.x will be incompatible with MLA 2.x
* Adding a test and a new v2 model
* Make the Model Runner raise an error if using 1.0 model with LSTM
* adding a new model for hallway trained with 2.0
* reword error messages
* Only raise if error, not if warning
* Addressing comments: The legacy Barrauda memory generator and applier were removed. All code that checked for (memories + v1.X) have been removed since these will no longer be supported
* Modifying the changelog and the migrating guide with this change
* Fixing the merge issues
Co-authored-by: Chris Elion <chris.elion@unity3d.com>
* [MLA-1939] Match3 Custom Editor (#5263)
* Rider suggested cleanup, part 1 (#5265)
* Fix the attention module embedding size (#5272)
* Fix the attention module embedding size
* editing the changelog
* [bug-fix] Fix null ref exception in the demo recorder (#5274)
* Fix null ref exception in the demo recorder
* Update changelog
* Clearer logic
* Modifying the changelog for release 17 (#5277)
* Modifying the changelog for release 17
* Change version to 2.0.0-exp.1
* Removing the [Unreleased] section
* check for missing AbstractBoard, display warning (#5276)
* Updating the barracuda 1.4.0 (#5291)
* [debug] Require all behavior names to have a matching YAML entry (#5210)
* Add strict check to settings.py
* Remove warning from trainer factory, add test
* Add changelog
* Fix test
* Update changelog
* Remove strict CLI options
* Remove strict option, rename, make strict default
* Remove newline
* Update comments
* Set default dict to actually default to a default dict
* Fix tests
* Fix tests again
* Default trainer dict to requiring all fields
* Fix settings typing
* Use logger
* Add default_settings to error
* Version bump main (#5279)
* Version bump for the main branch following branching of release_17
* Messed up the extensions package version
* Bring back root reference in grid sensor (#5300)
* [MLA-1952] Add optional seed for gym action spaces (#5303)
* add optional seed for action spaces
* add optional seed for action spaces
* changelog
* undo packages-lock.json change
* Release 17 version bumps and docs version bumps (#5280)
* Migration guide - list removed APIs with replacements (#5307)
* list removed APIs with replacements
* fixes
* [docs] Update changelog (#5308)
* [MLA-1952] Add optional seed for gym action spaces (#5303) (#5315)
* Fix stacked grid sensor (#5335)
* Update to Unity Package licenses (#5340)
* Update to Unity Package licenses
Per Dan
* Updating the main repo license dates.
* Adding a fully connected visual encoder for super small visual input + tests (#5351)
* initial commit for a fully connected visual encoder
* adding a test
* addressing comments
* Fixing error with minimal size of fully connected network
* adding documentation and changelog
* Make OverlapChecker an interface (#5324)
* Add stacking option to VectorSensorComponent (#5376)
* [WIP] [Fix] Fixing collect observation called on done (#5375)
* [WIP] [Fix] Fixing collect observation called on done
* Update com.unity.ml-agents/Runtime/Agent.cs
* ⚠️ Modifying the test of stacking sensor when the agent is done
* modifying the documentation for BufferSensor to specify to call AddObservation in the CollectObservations method
* Fix NullReferenceException for Behavior Parameters without Agent (#5382)
* check agent existence. add warning box
* changelog
* [WIP] 2.0 verified to main (#5385)
* Warn if no joints on Root Body (#5387)
* Modifying the changelog for the continuous action SAC target entropy fix (#5394)
* Editing the GridSensor documentation for 2D use case (#5396)
* Editing the GridSensor documentation for 2D use case
* changing chagelog
* [MLA-2017] Move colab notebooks to github (#5399)
* Fix the reporting of histogram stats and adding a test (#5410)
* Fix the reporting of histogram stats and adding a test
* Appending to the Changelog
* Update changelog before Release 18 branch (#5412)
* Update package versions for main branch (#5413)
* [Release 18] Update versions and links (#5414)
* add changelog entries for PRs that were missing them before. (#5416)
* fix cref error found by doc validation (#5421)
* fix release versions and changelog section
* [bug-fix] Fix when group terminal steps are deleted, robust test (#5441)
* Fix when terminal steps are deleted, robust test
* Update changelog
* Fix test comment
* Fix torch device option in argparser (#5467)
* [FIX] prioritize cli options over yaml (#5495)
* prioritize cli over yaml in checkpointSettings
* prioritize resume if both set in one place
* fixed test_commandline_args
* addressing comments:renaming, commenting
* more tests and referenced in change_log
* Only update the native library and the c# calls that need to be made to clean up channels. (#5283)
* Fix Mac backcompat test (#5519)
* Initialize-from custom checkpoints (#5525)
* init from any checkpoint including older ones
* moving init_path logic ahead to learn.py
* fixing pytest to take the full path
* doc & changelog
* Update gym version to 0.20.0 (#5540)
* Fixed failing test due to gym 0.20.0 release. Updated gym-unity package to fix gym version to 0.20.0.
* Updated CHANGELOG.
* Fix VAIL (#5546)
* Custom trainer editor analytics (#5511)
* Custom trainer editor analytics
* inherit the default_training_analytics_side_channel
* Feature/mla 2205 separate schedule lr beta epsilon (#5538)
* Added initial (untested) support for separating schedule for beta, LR, and epsilon.
* Added support for beta and epsilon schedule in test config conversion
* Updated docs. Set epsilon to be linear by default.
* Updated configs to add beta and epsilon schedule where appropriate.
* Addressed CR change requests.
* Minor update.
* Updated docs.
* Added changes to CHANGELOG.
* Updated CHANGELOG.
* Fixed typo in Training-Configuration-File.md
* Reverted config changes to remove beta and epsilon schedules.
* mypy error fixed from pre-commit.
* Fixed a bug that was causing a settings test to fail.
* Addressed CR comments.
Co-authored-by: Miguel Alonso Jr <miguel.alonso@miguel.alonso-00MM>
Co-authored-by: Miguel Alonso Jr <miguelalonsojr>
* Harden user PII protection logic and extend TrainingAnalytics to expose detailed configuration parameters. (#5512)
* Hash128 is not a cryptographic hash, replace with HMAC-SHA256.
* Extend TrainingAnalytics side channel to expose configuration details
* Change member function scopes and hash demo_paths
* Extract tbiEvent hashing method and add test coverage
* Restart crashed Unity environments (#5553)
### Change(s)
Update the SubprocessEnvManager to restart workers when the underlying Unity environments crash.
When a worker receives an ENV_EXITED signal it will now:
1. Record all failures coming through the step queue and drop all other messages.
2. Purge any pending trajectories as they may belong to a crashed worker or be corrupted.
3. Restart all failed workers (up to a configurable limit)
This behavior can be limited via a rate limit, max lifetime limit, or both. The configuration options for both are shown below with their default values.
⚠️ Each of these options applies to a single environment, if num_envs > 1 then the limit will apply separately to each replica (num_envs = 2 will spawn 2 Unity environments which can each be restarted 10 times).
```yaml
env_settings:
# Can restart 10 times over the lifetime of the experiment.
max_lifetime_restarts: 10
# Rate limit of 1 failure per 60s
restarts_rate_limit_n: 1
restarts_rate_limit_period_s: 60
```
They can of course be passed via CLI arguments as well
```bash
--max-lifetime-restarts
--restarts-rate-limit-n
--restarts-rate-limit-period-s
```
### Disabling this feature
* Rate limiting can be turned off by setting `--restarts-rate-limit-n=-1`
* Lifetime limiting can be turned off by setting `--max-lifetime-restarts=-1`
* Develop add training area replicator (#5568)
* Added training area replicator to com.unity.ml-agents package.
* Added num_areas to Unity RL Initialization proto. Added cli and config file support for num_areas.
* Changed training area replicator to size grid automatically from number of areas.
* Added tests for the training area replicator.
* Added setup for tests for the training area replicator.
* Added comments and updated create tutorial for training area replicator.
* Updated CHANGELOG.
* Fixed some failing tests.
* Update com.unity.ml-agents/CHANGELOG.md
Co-authored-by: Henry Peteet <henry.peteet@unity3d.com>
* Update docs/Learning-Environment-Create-New.md
Co-authored-by: Henry Peteet <henry.peteet@unity3d.com>
* Update com.unity.ml-agents/Runtime/Areas/TrainingAreaReplicator.cs
Co-authored-by: Henry Peteet <henry.peteet@unity3d.com>
* Addressed CR comments.
Co-authored-by: Miguel Alonso Jr <miguelalonsojr>
Co-authored-by: Henry Peteet <henry.peteet@unity3d.com>
* Fixing the analytics side-channel for curriculum learning. (#5586)
* Fixing the analytics side-channel for curriculum learning.
* Made a more robust test.
* Update the changelog.
* Update com.unity.ml-agents/CHANGELOG.md
Co-authored-by: Maryam Honari <honari.m94@gmail.com>
Co-authored-by: Maryam Honari <honari.m94@gmail.com>
* Method to return stacked observations (#5547)
* Method to return stacked observations
* Added testing and returning staked observation flat.
* Update the comment lines.
* Remove brainstorm commits.
* Upgrade Barracuda to 2.3.1-preview (#5591)
* Upgrade to 2.3.1-preview
* Fix for critic normalization bug (#5595)
* Added normalization to critic during training for poca, ppo, and sac.
* Fixed critic normalization bug. Added new tests to cover.
* Updated CHANGELOG.
* Fixed typo in test_trainers.py
* Deterministic actions python training (#5619)
* Progress on propagating the setting to the action model.
* Added the _sample_action logic and tests.
* Add information to the changelog.
* Prioritize the CLI over the configuration file.
* Update documentation for config file.
* CR refactor.
* Update docs/Training-Configuration-File.md
Co-authored-by: Miguel Alonso Jr. <76960110+miguelalonsojr@users.noreply.github.com>
Update com.unity.ml-agents/CHANGELOG.md
Co-authored-by: Miguel Alonso Jr. <76960110+miguelalonsojr@users.noreply.github.com>
Update com.unity.ml-agents/CHANGELOG.md
Co-authored-by: Miguel Alonso Jr. <76960110+miguelalonsojr@users.noreply.github.com>
Update com.unity.ml-agents/CHANGELOG.md
Co-authored-by: Maryam Honari <honari.m94@gmail.com>
Update ml-agents/mlagents/trainers/settings.py
Co-authored-by: Maryam Honari <honari.m94@gmail.com>
Update ml-agents/mlagents/trainers/cli_utils.py
Co-authored-by: Maryam Honari <honari.m94@gmail.com>
* Fix CR requests
* Add tests for discrete.
* Update ml-agents/mlagents/trainers/torch/distributions.py
Co-authored-by: Maryam Honari <honari.m94@gmail.com>
* Added more stable test.
* Return deterministic actions for training (#5615)
* Added more stable test.
* Fix the tests.
* Fix pre-commit
* Fix help line to pass precommit.
* support for deterministic inference in onnx (#5593)
* Init: actor.forward outputs separate deterministic actions
* changelog
* Renaming
* Add more tests
* Package changes to support deterministic inference (#5599)
* Init: actor.forward outputs separate deterministic actions
* fix tensor shape for discrete actions
* Add test and editor flag
- Add tests for deterministic sampling
- update editor and tooltips
* Reverting to "Deterministic Inference"
* dissect tests
* Update docs
* Update CHANGELOG.md
Co-authored-by: Chingiz Mardanov <chingiz.mardanov@unity3d.com>
Co-authored-by: cmard <87716492+cmard@users.noreply.github.com>
* Revert "Deterministic actions python training (#5619)" (#5622)
This reverts commit 9ea962027d4c581de7c45a07d669f96453236c5c.
* Deterministic actions python training (#5626)
* Progress on propagating the setting to the action model.
* Added the _sample_action logic and tests.
* Add information to the changelog.
* Prioritize the CLI over the configuration file.
* Update documentation for config file.
* CR refactor.
* Update docs/Training-Configuration-File.md
Co-authored-by: Miguel Alonso Jr. <76960110+miguelalonsojr@users.noreply.github.com>
Update com.unity.ml-agents/CHANGELOG.md
Co-authored-by: Miguel Alonso Jr. <76960110+miguelalonsojr@users.noreply.github.com>
Update com.unity.ml-agents/CHANGELOG.md
Co-authored-by: Miguel Alonso Jr. <76960110+miguelalonsojr@users.noreply.github.com>
Update com.unity.ml-agents/CHANGELOG.md
Co-authored-by: Maryam Honari <honari.m94@gmail.com>
Update ml-agents/mlagents/trainers/settings.py
Co-authored-by: Maryam Honari <honari.m94@gmail.com>
Update ml-agents/mlagents/trainers/cli_utils.py
Co-authored-by: Maryam Honari <honari.m94@gmail.com>
* Fix CR requests
* Add tests for discrete.
* Update ml-agents/mlagents/trainers/torch/distributions.py
Co-authored-by: Maryam Honari <honari.m94@gmail.com>
* Added more stable test.
* Return deterministic actions for training (#5615)
* Added more stable test.
* Fix the tests.
* Fix pre-commit
* Fix help line to pass precommit.
* support for deterministic inference in onnx (#5593)
* Init: actor.forward outputs separate deterministic actions
* changelog
* Renaming
* Add more tests
* Package changes to support deterministic inference (#5599)
* Init: actor.forward outputs separate deterministic actions
* fix tensor shape for discrete actions
* Add test and editor flag
- Add tests for deterministic sampling
- update editor and tooltips
* Reverting to "Deterministic Inference"
* dissect tests
* Update docs
* Update CHANGELOG.md
* Fix the deterministic showing up all the tiime (#5621)
Co-authored-by: Chingiz Mardanov <chingiz.mardanov@unity3d.com>
Co-authored-by: cmard <87716492+cmard@users.noreply.github.com>
* Changelog cleanup. (#5627)
* Release cleanup for changelog
* Update CHANGELOG.md
Co-authored-by: Maryam Honari <honari.m94@gmail.com>
* Fix analytics error when compiling XBOX and PS5 (#5628)
* Add ENABLE_CLOUD_SERVICES_ANALYTICS Guards
To use analytics safely you must wrap usage with
`#if ENABLE_CLOUD_SERVICES_ANALYTICS`
See https://docs.unity3d.com/ScriptReference/Analytics.Analytics.html
* Update DQN Colab example to support export for Barracuda inference (#5643)
* Update DQN GridWorld colab example to be capable of outputting an ONNX model that works for inference.
-Allow GridWorld to work without a GoalSensor. Defaults to the GreenPlus target in that case.
-Add GridWorldColab.onnx file, trained and generated with the colab.
* Upgrade DQN GridWorld Colab to produce model that can work with (modified) GridWorld Scene.
-permute observation images from NHWC to NCHW right after env_step. Everything following is NCHW.
-update input_shape indexes and remove equivalent permute from the VisualQNetwork
-fix reference to global qnet inside generate_trajectories (use the passed in version)
-create wrapper network to add required constants and dummy input masks required by the
runtime inference (Barracuda)
-export the wrapped network to ONNX format compatible with Barracuda
* Add additional GridWorld scene called GridWorldColab that has the goal sensors removed (target is always green plus) and masks disabled. The purpose of this is to have a scene compatible with the DQN GridWorld Colab example.
* Minor edit in Colab docs.
* Delete unnecessary .lighting file.
* Update changelog.
* Update colab/Colab_UnityEnvironment_2_Train.ipynb
Co-authored-by: Maryam Honari <honari.m94@gmail.com>
* Fix error in colab metadata name.
* Add suggested comments and fix incorrect parameter ordering.
* Update docs for Elo curriculum completion measure.
Co-authored-by: Maryam Honari <honari.m94@gmail.com>
* Changelog prepped for release (#5629)
* Changelog prepped for release
* Added missing PRs to the changelog.
* Added missing CHANGELOG item.
* enable 3D Ball sample import button in package manager for unity sdk (#5665)
* enable 3D Ball sample import button in package manager for unity sdk
* add Samples folder
* add meta file
* fix git ignore config
* Updated versions for main. Also updated validate versions script to r… (#5668)
* Updated versions for main. Also updated validate versions script to remove exp.1 from package version.
* Reverted validate versions.
* Updated versions.
* Bumped package version for patch.
* Removed unreleased section. (#5667)
* Updated release versions and release links. (#5669)
* Updates for package validation errors. (#5671)
* Release 19 branch lts deprecation (#5673)
* Updated Project and DevProject to 2020.3 LTS.
* Updated supported LTS version to 2020.3.
* Updated changelog.
* Bumped up CI Unity version from 2020.2 to 2020.3.
* Update CHANGELOG.md
* Removing VS Studio dependency from manifest.json.
* Update packages-lock.json
* Update CHANGELOG.md
* Bumped package version for patch.
* Fixed build-docker and updated CHANGELOG to add unreleased section. (#5682)
* Fixed build-docker and updated CHANGELOG to add unreleased section.
* Updated backward compat mac training yamato test. (#5685)
* Develop deprecate python 3.6 (#5)
* Dropped support for python 3.6
* Pinning python 3.9.9 for tests due to typing issues with 3.9.10
* Testing new bokken image.
* Testing new bokken image.
* Updated yamato standalone build test.
* Updated yamato standalone build test.
* Updated standalone build test.
* Updated yamato configs to use mla bokken vm.
* Bug fixes for yamato yml files.
* Fixed com.unity.ml-agents-test.yml
* Bumped min python version to 3.7.2
* Updated failing yamato jobs.
* Updated github workflow for colab tests.
* Updated github workflow for colab tests.
* Updated github workflow for colab tests.
* Develop python api ga (#6)
* Dropped support for python 3.6
* Pinning python 3.9.9 for tests due to typing issues with 3.9.10
* Testing new bokken image.
* Testing new bokken image.
* Updated yamato standalone build test.
* Updated yamato standalone build test.
* Updated standalone build test.
* Updated yamato configs to use mla bokken vm.
* Bug fixes for yamato yml files.
* Fixed com.unity.ml-agents-test.yml
* Bumped min python version to 3.7.2
* pettingzoo api prototype
* add example
* update file names
* support multiple behavior names
* fix multi behavior action index
* add install in colab
* add setup
* update colab
* fix __init__
* clone single branch
* import tags only
* import in init
* catch import error
* update colab
* move colab and add readme
* handle agent dying
* add tests
* update doc
* add info
* add action mask
* fix action mask
* update action masks in colab
* change default env
* set version
* fix hybrid action
* fix colab for hybrid actions
* add note on auto reset
* Updated colab name.
* Update README.md
* Following petting_zoo registry API (#5557)
* init petting_zoo registry
* cherrypick Custom trainer editor analytics (#5511)
* cherrypick "Update dotnet-format to address breaking changes introduced by upstream changes (#5528)"
* Update colab to match pettingZoo import api
* ToRevert: pull exp-petting-registry branch
* Add init file to tests
* Install pettingzoo-unity requirements for pytest
* update pytest command
* Add docstrings and comments
* update coverage to pettingzoo folder
* unset log level
* update env string
* Two small bugfixes (#5589)
1. Add the missing `_cumulative_rewards` property
2. Update `agent_selection` to not error out when an agent finishes an episode.
* Updated gym to 0.21.0 and petting zoo to 1.13.1, fixed bugs with AEC wrapper for gym and PZ updates. API tests are passing.
* Some refactoring.
* Finished inital implementation of parallel. Tests not passing.
* Finished parallel API implementation and refactor. All PZ tests passing.
* Cleanup.
* Refactoring.
* Pinning numpy version.
* add metadata and behavior_specs initialization
* addressing behaviour_spec issues
* Bumped PZ version to 1.14.0. Fixed failing tests.
* Refactored gym-unity and petting-zoo into ml-agents-envs
* Added TODO to pydoc-config.yaml
* Refactored gym and pz to be under a subpackage in mlagents_env package
* Refactored ml-agents-envs docs.
* Minor update to PZ API doc.
* Updated mlagents_envs docs and colab.
* Updated pytest gh workflow to remove ref to gym and pz.
* Refactored to remove some test coupling between trainers and envs.
* Updated installation doc.
* Update ml-agents-envs/README.md
Co-authored-by: Andrew Cohen <andrew.cohen@unity3d.com>
* Updated failing yamato jobs.
* pettingzoo api prototype
* add example
* update file names
* support multiple behavior names
* fix multi behavior action index
* add install in colab
* add setup
* update colab
* fix __init__
* clone single branch
* import tags only
* import in init
* catch import error
* update colab
* move colab and add readme
* handle agent dying
* add tests
* update doc
* add info
* add action mask
* fix action mask
* update action masks in colab
* change default env
* set version
* fix hybrid action
* fix colab for hybrid actions
* add note on auto reset
* Updated colab name.
* Update README.md
* Following petting_zoo registry API (#5557)
* init petting_zoo registry
* cherrypick Custom trainer editor analytics (#5511)
* cherrypick "Update dotnet-format to address breaking changes introduced by upstream changes (#5528)"
* Update colab to match pettingZoo import api
* ToRevert: pull exp-petting-registry branch
* Add init file to tests
* Install pettingzoo-unity requirements for pytest
* update pytest command
* Add docstrings and comments
* update coverage to pettingzoo folder
* unset log level
* update env string
* Two small bugfixes (#5589)
1. Add the missing `_cumulative_rewards` property
2. Update `agent_selection` to not error out when an agent finishes an episode.
* Updated gym to 0.21.0 and petting zoo to 1.13.1, fixed bugs with AEC wrapper for gym and PZ updates. API tests are passing.
* Some refactoring.
* Finished inital implementation of parallel. Tests not passing.
* Finished parallel API implementation and refactor. All PZ tests passing.
* Cleanup.
* Refactoring.
* Pinning numpy version.
* add metadata and behavior_specs initialization
* addressing behaviour_spec issues
* Bumped PZ version to 1.14.0. Fixed failing tests.
* Refactored gym-unity and petting-zoo into ml-agents-envs
* Added TODO to pydoc-config.yaml
* Refactored gym and pz to be under a subpackage in mlagents_env package
* Refactored ml-agents-envs docs.
* Minor update to PZ API doc.
* Updated mlagents_envs docs and colab.
* Updated pytest gh workflow to remove ref to gym and pz.
* Refactored to remove some test coupling between trainers and envs.
* Updated installation doc.
* Update ml-agents-envs/README.md
Co-authored-by: Andrew Cohen <andrew.cohen@unity3d.com>
* Updated CHANGELOG.
* Updated Migration guide.
* Doc updates based on CR.
* Updated github workflow for colab tests.
* Updated github workflow for colab tests.
* Updated github workflow for colab tests.
* Fixed yamato import error.
Co-authored-by: Ruo-Ping Dong <ruoping.dong@unity3d.com>
Co-authored-by: Miguel Alonso Jr <miguelalonsojr>
Co-authored-by: jmercado1985 <75792879+jmercado1985@users.noreply.github.com>
Co-authored-by: Maryam Honari <honari.m94@gmail.com>
Co-authored-by: Henry Peteet <henry.peteet@unity3d.com>
Co-authored-by: mahon94 <maryam.honari@unity3d.com>
Co-authored-by: Andrew Cohen <andrew.cohen@unity3d.com>
* Refactoring for extraction to separate repo.
* removing unused github workflows.
* Updates for pre-commit and github actions.
* Fixed bug with validate_meta_files.py missing a file to exclude.
* Updated precommit gh action.
* Refactored yamato pack.
* Added yamato test yaml.
* Updated to CHANGELOG and VectorSensor from mono repo before swap.
* Pin dotnet version for GHA formatting.
* Refactor for MLA pro support. (#1)
* Refactored for MLA pro support.
* Fixed bug in yamato test.
* Updated the CHANGELOG.
* Fixed trigger logic in yamato test.
* Fixed failing tests. (#2)
* Fixed failing tests.
* Adding meta files.
* Fix execution order (#4)
* Fixed execution order.
* Fixed bug with CRT build.
* precommit formatting update.
* Updated agent Awake docstring.
* Set execution order to be negative to make sure scripts run before default time. (#5)
* Added #if to exclude communicator registration in the Agent awake script (#6)
* Add visibility to pro package (#7)
* add coverage publish promotion yamato jobs (#3)
* bump barracuda to 3.0.0 (#9)
upgrade to barracuda 3.0.0
* Updated README. (#8)
* add validation test dep. to publish job (#12)
* add validation test dep. to publish job
* fix package path
* Update automated trigger rules to support oneflow develop branch (#11)
* Update automated trigger rules to support oneflow develop branch
* Fix release branch name for github actions
* Release 2.3.0 exp.1 (#13)
* upgrade package to 2.3.0-exp.1
* rebase on develop branch
* update changelog
* remove unreleased section (#16)
* Updated experimental prerelease version and changelog. (#18)
* Added submodule update gha. (#19)
* fix bugs from sonar-scanner (#21)
* add ml-agents-pro as a upstream for submodule update (#22)
* Added support for standalone use of Academy with refactored communicator factory. (#24)
* clean commandline Arguments (#25)
* clean commandline Arguments
* fix precommit
* update unrelease section in changelog (#27)
* MLA-2537 Reorder RayPerceptionSensor output to reflect its spatial structure. (#26)
* Changes to support in order rays. GUI for switching to this. Updated tests.
* Updated README. (#29)
* Fixed samples. (#30)
* Added camera sensor component checkbox for runtime enablement. update… (#31)
* Added camera sensor component checkbox for runtime enablement. updated camera sensor to only be active when enabled for runtime.
* Updated yamato job.
* Updated CHANGELOG.
* Fixed issue with Academy not building for webgl builds. (#32)
* Dev bump min ver (#33)
* Bumped min version of Unity to 2021.3.
* Updated min version of python in pre-commit. Updated changelog.
* Moving files into packages/com.unity.ml-agents directoy.
* Remove com.unity.ml-agents submodule.
* Added back in ml-agents package.
* Removed packages temp folder.
* Updated yamato protobuf gen.
* Updated yamato to remove submodule update.
* Updated linux tool chain.
* Updated Project package dependencies.
* Updated flake8 repo link in precommit.
Co-authored-by: Vincent-Pierre BERGES <vincentpierre@unity3d.com>
Co-authored-by: Arthur Juliani <awjuliani@gmail.com>
Co-authored-by: Chris Elion <chris.elion@unity3d.com>
Co-authored-by: andrewcoh <54679309+andrewcoh@users.noreply.github.com>
Co-authored-by: Chris Goy <christopherg@unity3d.com>
Co-authored-by: Marwan Mattar <marwan@unity3d.com>
Co-authored-by: Ruo-Ping Dong <ruoping.dong@unity3d.com>
Co-authored-by: Ervin Teng <ervin@unity3d.com>
Co-authored-by: Hunter-Unity <hunter@unity3d.com>
Co-authored-by: Maryam Honari <honari.m94@gmail.com>
Co-authored-by: Miguel Alonso Jr <76960110+miguelalonsojr@users.noreply.github.com>
Co-authored-by: Miguel Alonso Jr <miguel.alonso@miguel.alonso-00MM>
Co-authored-by: Jason Bowman <jasonb@unity3d.com>
Co-authored-by: Henry Peteet <henry.peteet@unity3d.com>
Co-authored-by: cmard <87716492+cmard@users.noreply.github.com>
Co-authored-by: Chingiz Mardanov <chingiz.mardanov@unity3d.com>
Co-authored-by: Jason Rupert <92821409+jrupert-unity@users.noreply.github.com>
Co-authored-by: Xun Cao <xun.cao@outlook.com>
Co-authored-by: jmercado1985 <75792879+jmercado1985@users.noreply.github.com>
Co-authored-by: mahon94 <maryam.honari@unity3d.com>
Co-authored-by: Andrew Cohen <andrew.cohen@unity3d.com>
Co-authored-by: Xun Cao <xun.cao@unity3d.com>
Co-authored-by: Jason Rupert <jason.rupert@unity3d.com>
* Release 20 changelog (#80)
* clean changelog
* match extention package dependency
* fix typos
Co-authored-by: Xun Cao <xun.cao@unity3d.com>
Co-authored-by: Brandon Henry <brandonh@unity3d.com>
Co-authored-by: Andrew Cohen <andrew.cohen@unity3d.com>
Co-authored-by: Henry Peteet <henry.peteet@unity3d.com>
Co-authored-by: Miguel Alonso Jr <miguel.alonso@unity3d.com>
Co-authored-by: GitHub Actions - update submodules <bot@noreply.github.com>
Co-authored-by: Jason Bowman <jasonb@unity3d.com>
Co-authored-by: Florence Rolland <florence@unity3d.com>
Co-authored-by: Jason Rupert <jason.rupert@unity3d.com>
Co-authored-by: zhuo <zhuo@unity3d.com>
Co-authored-by: Vincent-Pierre BERGES <vincentpierre@unity3d.com>
Co-authored-by: Arthur Juliani <awjuliani@gmail.com>
Co-authored-by: Chris Elion <chris.elion@unity3d.com>
Co-authored-by: andrewcoh <54679309+andrewcoh@users.noreply.github.com>
Co-authored-by: Chris Goy <christopherg@unity3d.com>
Co-authored-by: Marwan Mattar <marwan@unity3d.com>
Co-authored-by: Ruo-Ping Dong <ruoping.dong@unity3d.com>
Co-authored-by: Ervin Teng <ervin@unity3d.com>
Co-authored-by: Hunter-Unity <hunter@unity3d.com>
Co-authored-by: Maryam Honari <honari.m94@gmail.com>
Co-authored-by: Miguel Alonso Jr <76960110+miguelalonsojr@users.noreply.github.com>
Co-authored-by: Miguel Alonso Jr <miguel.alonso@miguel.alonso-00MM>
Co-authored-by: cmard <87716492+cmard@users.noreply.github.com>
Co-authored-by: Chingiz Mardanov <chingiz.mardanov@unity3d.com>
Co-authored-by: Jason Rupert <92821409+jrupert-unity@users.noreply.github.com>
Co-authored-by: Xun Cao <xun.cao@outlook.com>
Co-authored-by: jmercado1985 <75792879+jmercado1985@users.noreply.github.com>
* Dropped support for python 3.6
* Pinning python 3.9.9 for tests due to typing issues with 3.9.10
* Testing new bokken image.
* Testing new bokken image.
* Updated yamato standalone build test.
* Updated yamato standalone build test.
* Updated standalone build test.
* Updated yamato configs to use mla bokken vm.
* Bug fixes for yamato yml files.
* Fixed com.unity.ml-agents-test.yml
* Bumped min python version to 3.7.2
* pettingzoo api prototype
* add example
* update file names
* support multiple behavior names
* fix multi behavior action index
* add install in colab
* add setup
* update colab
* fix __init__
* clone single branch
* import tags only
* import in init
* catch import error
* update colab
* move colab and add readme
* handle agent dying
* add tests
* update doc
* add info
* add action mask
* fix action mask
* update action masks in colab
* change default env
* set version
* fix hybrid action
* fix colab for hybrid actions
* add note on auto reset
* Updated colab name.
* Update README.md
* Following petting_zoo registry API (#5557)
* init petting_zoo registry
* cherrypick Custom trainer editor analytics (#5511)
* cherrypick "Update dotnet-format to address breaking changes introduced by upstream changes (#5528)"
* Update colab to match pettingZoo import api
* ToRevert: pull exp-petting-registry branch
* Add init file to tests
* Install pettingzoo-unity requirements for pytest
* update pytest command
* Add docstrings and comments
* update coverage to pettingzoo folder
* unset log level
* update env string
* Two small bugfixes (#5589)
1. Add the missing `_cumulative_rewards` property
2. Update `agent_selection` to not error out when an agent finishes an episode.
* Updated gym to 0.21.0 and petting zoo to 1.13.1, fixed bugs with AEC wrapper for gym and PZ updates. API tests are passing.
* Some refactoring.
* Finished inital implementation of parallel. Tests not passing.
* Finished parallel API implementation and refactor. All PZ tests passing.
* Cleanup.
* Refactoring.
* Pinning numpy version.
* add metadata and behavior_specs initialization
* addressing behaviour_spec issues
* Bumped PZ version to 1.14.0. Fixed failing tests.
* Refactored gym-unity and petting-zoo into ml-agents-envs
* Added TODO to pydoc-config.yaml
* Refactored gym and pz to be under a subpackage in mlagents_env package
* Refactored ml-agents-envs docs.
* Minor update to PZ API doc.
* Updated mlagents_envs docs and colab.
* Updated pytest gh workflow to remove ref to gym and pz.
* Refactored to remove some test coupling between trainers and envs.
* Updated installation doc.
* Update ml-agents-envs/README.md
Co-authored-by: Andrew Cohen <andrew.cohen@unity3d.com>
* Updated failing yamato jobs.
* pettingzoo api prototype
* add example
* update file names
* support multiple behavior names
* fix multi behavior action index
* add install in colab
* add setup
* update colab
* fix __init__
* clone single branch
* import tags only
* import in init
* catch import error
* update colab
* move colab and add readme
* handle agent dying
* add tests
* update doc
* add info
* add action mask
* fix action mask
* update action masks in colab
* change default env
* set version
* fix hybrid action
* fix colab for hybrid actions
* add note on auto reset
* Updated colab name.
* Update README.md
* Following petting_zoo registry API (#5557)
* init petting_zoo registry
* cherrypick Custom trainer editor analytics (#5511)
* cherrypick "Update dotnet-format to address breaking changes introduced by upstream changes (#5528)"
* Update colab to match pettingZoo import api
* ToRevert: pull exp-petting-registry branch
* Add init file to tests
* Install pettingzoo-unity requirements for pytest
* update pytest command
* Add docstrings and comments
* update coverage to pettingzoo folder
* unset log level
* update env string
* Two small bugfixes (#5589)
1. Add the missing `_cumulative_rewards` property
2. Update `agent_selection` to not error out when an agent finishes an episode.
* Updated gym to 0.21.0 and petting zoo to 1.13.1, fixed bugs with AEC wrapper for gym and PZ updates. API tests are passing.
* Some refactoring.
* Finished inital implementation of parallel. Tests not passing.
* Finished parallel API implementation and refactor. All PZ tests passing.
* Cleanup.
* Refactoring.
* Pinning numpy version.
* add metadata and behavior_specs initialization
* addressing behaviour_spec issues
* Bumped PZ version to 1.14.0. Fixed failing tests.
* Refactored gym-unity and petting-zoo into ml-agents-envs
* Added TODO to pydoc-config.yaml
* Refactored gym and pz to be under a subpackage in mlagents_env package
* Refactored ml-agents-envs docs.
* Minor update to PZ API doc.
* Updated mlagents_envs docs and colab.
* Updated pytest gh workflow to remove ref to gym and pz.
* Refactored to remove some test coupling between trainers and envs.
* Updated installation doc.
* Update ml-agents-envs/README.md
Co-authored-by: Andrew Cohen <andrew.cohen@unity3d.com>
* Updated CHANGELOG.
* Updated Migration guide.
* Doc updates based on CR.
* Updated github workflow for colab tests.
* Updated github workflow for colab tests.
* Updated github workflow for colab tests.
* Fixed yamato import error.
Co-authored-by: Ruo-Ping Dong <ruoping.dong@unity3d.com>
Co-authored-by: Miguel Alonso Jr <miguelalonsojr>
Co-authored-by: jmercado1985 <75792879+jmercado1985@users.noreply.github.com>
Co-authored-by: Maryam Honari <honari.m94@gmail.com>
Co-authored-by: Henry Peteet <henry.peteet@unity3d.com>
Co-authored-by: mahon94 <maryam.honari@unity3d.com>
Co-authored-by: Andrew Cohen <andrew.cohen@unity3d.com>
* Updated Project and DevProject to 2020.3 LTS.
* Updated supported LTS version to 2020.3.
* Updated changelog.
* Bumped up CI Unity version from 2020.2 to 2020.3.
* Update CHANGELOG.md
* Removing VS Studio dependency from manifest.json.
* Update packages-lock.json
* Update DQN GridWorld colab example to be capable of outputting an ONNX model that works for inference.
-Allow GridWorld to work without a GoalSensor. Defaults to the GreenPlus target in that case.
-Add GridWorldColab.onnx file, trained and generated with the colab.
* Upgrade DQN GridWorld Colab to produce model that can work with (modified) GridWorld Scene.
-permute observation images from NHWC to NCHW right after env_step. Everything following is NCHW.
-update input_shape indexes and remove equivalent permute from the VisualQNetwork
-fix reference to global qnet inside generate_trajectories (use the passed in version)
-create wrapper network to add required constants and dummy input masks required by the
runtime inference (Barracuda)
-export the wrapped network to ONNX format compatible with Barracuda
* Add additional GridWorld scene called GridWorldColab that has the goal sensors removed (target is always green plus) and masks disabled. The purpose of this is to have a scene compatible with the DQN GridWorld Colab example.
* Minor edit in Colab docs.
* Delete unnecessary .lighting file.
* Update changelog.
* Update colab/Colab_UnityEnvironment_2_Train.ipynb
Co-authored-by: Maryam Honari <honari.m94@gmail.com>
* Fix error in colab metadata name.
* Add suggested comments and fix incorrect parameter ordering.
* Update docs for Elo curriculum completion measure.
Co-authored-by: Maryam Honari <honari.m94@gmail.com>
* Progress on propagating the setting to the action model.
* Added the _sample_action logic and tests.
* Add information to the changelog.
* Prioritize the CLI over the configuration file.
* Update documentation for config file.
* CR refactor.
* Update docs/Training-Configuration-File.md
Co-authored-by: Miguel Alonso Jr. <76960110+miguelalonsojr@users.noreply.github.com>
Update com.unity.ml-agents/CHANGELOG.md
Co-authored-by: Miguel Alonso Jr. <76960110+miguelalonsojr@users.noreply.github.com>
Update com.unity.ml-agents/CHANGELOG.md
Co-authored-by: Miguel Alonso Jr. <76960110+miguelalonsojr@users.noreply.github.com>
Update com.unity.ml-agents/CHANGELOG.md
Co-authored-by: Maryam Honari <honari.m94@gmail.com>
Update ml-agents/mlagents/trainers/settings.py
Co-authored-by: Maryam Honari <honari.m94@gmail.com>
Update ml-agents/mlagents/trainers/cli_utils.py
Co-authored-by: Maryam Honari <honari.m94@gmail.com>
* Fix CR requests
* Add tests for discrete.
* Update ml-agents/mlagents/trainers/torch/distributions.py
Co-authored-by: Maryam Honari <honari.m94@gmail.com>
* Added more stable test.
* Return deterministic actions for training (#5615)
* Added more stable test.
* Fix the tests.
* Fix pre-commit
* Fix help line to pass precommit.
* support for deterministic inference in onnx (#5593)
* Init: actor.forward outputs separate deterministic actions
* changelog
* Renaming
* Add more tests
* Package changes to support deterministic inference (#5599)
* Init: actor.forward outputs separate deterministic actions
* fix tensor shape for discrete actions
* Add test and editor flag
- Add tests for deterministic sampling
- update editor and tooltips
* Reverting to "Deterministic Inference"
* dissect tests
* Update docs
* Update CHANGELOG.md
* Fix the deterministic showing up all the tiime (#5621)
Co-authored-by: Chingiz Mardanov <chingiz.mardanov@unity3d.com>
Co-authored-by: cmard <87716492+cmard@users.noreply.github.com>
* Progress on propagating the setting to the action model.
* Added the _sample_action logic and tests.
* Add information to the changelog.
* Prioritize the CLI over the configuration file.
* Update documentation for config file.
* CR refactor.
* Update docs/Training-Configuration-File.md
Co-authored-by: Miguel Alonso Jr. <76960110+miguelalonsojr@users.noreply.github.com>
Update com.unity.ml-agents/CHANGELOG.md
Co-authored-by: Miguel Alonso Jr. <76960110+miguelalonsojr@users.noreply.github.com>
Update com.unity.ml-agents/CHANGELOG.md
Co-authored-by: Miguel Alonso Jr. <76960110+miguelalonsojr@users.noreply.github.com>
Update com.unity.ml-agents/CHANGELOG.md
Co-authored-by: Maryam Honari <honari.m94@gmail.com>
Update ml-agents/mlagents/trainers/settings.py
Co-authored-by: Maryam Honari <honari.m94@gmail.com>
Update ml-agents/mlagents/trainers/cli_utils.py
Co-authored-by: Maryam Honari <honari.m94@gmail.com>
* Fix CR requests
* Add tests for discrete.
* Update ml-agents/mlagents/trainers/torch/distributions.py
Co-authored-by: Maryam Honari <honari.m94@gmail.com>
* Added more stable test.
* Return deterministic actions for training (#5615)
* Added more stable test.
* Fix the tests.
* Fix pre-commit
* Fix help line to pass precommit.
* support for deterministic inference in onnx (#5593)
* Init: actor.forward outputs separate deterministic actions
* changelog
* Renaming
* Add more tests
* Package changes to support deterministic inference (#5599)
* Init: actor.forward outputs separate deterministic actions
* fix tensor shape for discrete actions
* Add test and editor flag
- Add tests for deterministic sampling
- update editor and tooltips
* Reverting to "Deterministic Inference"
* dissect tests
* Update docs
* Update CHANGELOG.md
Co-authored-by: Chingiz Mardanov <chingiz.mardanov@unity3d.com>
Co-authored-by: cmard <87716492+cmard@users.noreply.github.com>
* Added normalization to critic during training for poca, ppo, and sac.
* Fixed critic normalization bug. Added new tests to cover.
* Updated CHANGELOG.
* Fixed typo in test_trainers.py
* Fixing the analytics side-channel for curriculum learning.
* Made a more robust test.
* Update the changelog.
* Update com.unity.ml-agents/CHANGELOG.md
Co-authored-by: Maryam Honari <honari.m94@gmail.com>
Co-authored-by: Maryam Honari <honari.m94@gmail.com>
* Added training area replicator to com.unity.ml-agents package.
* Added num_areas to Unity RL Initialization proto. Added cli and config file support for num_areas.
* Changed training area replicator to size grid automatically from number of areas.
* Added tests for the training area replicator.
* Added setup for tests for the training area replicator.
* Added comments and updated create tutorial for training area replicator.
* Updated CHANGELOG.
* Fixed some failing tests.
* Update com.unity.ml-agents/CHANGELOG.md
Co-authored-by: Henry Peteet <henry.peteet@unity3d.com>
* Update docs/Learning-Environment-Create-New.md
Co-authored-by: Henry Peteet <henry.peteet@unity3d.com>
* Update com.unity.ml-agents/Runtime/Areas/TrainingAreaReplicator.cs
Co-authored-by: Henry Peteet <henry.peteet@unity3d.com>
* Addressed CR comments.
Co-authored-by: Miguel Alonso Jr <miguelalonsojr>
Co-authored-by: Henry Peteet <henry.peteet@unity3d.com>
### Change(s)
Update the SubprocessEnvManager to restart workers when the underlying Unity environments crash.
When a worker receives an ENV_EXITED signal it will now:
1. Record all failures coming through the step queue and drop all other messages.
2. Purge any pending trajectories as they may belong to a crashed worker or be corrupted.
3. Restart all failed workers (up to a configurable limit)
This behavior can be limited via a rate limit, max lifetime limit, or both. The configuration options for both are shown below with their default values.
⚠️ Each of these options applies to a single environment, if num_envs > 1 then the limit will apply separately to each replica (num_envs = 2 will spawn 2 Unity environments which can each be restarted 10 times).
```yaml
env_settings:
# Can restart 10 times over the lifetime of the experiment.
max_lifetime_restarts: 10
# Rate limit of 1 failure per 60s
restarts_rate_limit_n: 1
restarts_rate_limit_period_s: 60
```
They can of course be passed via CLI arguments as well
```bash
--max-lifetime-restarts
--restarts-rate-limit-n
--restarts-rate-limit-period-s
```
### Disabling this feature
* Rate limiting can be turned off by setting `--restarts-rate-limit-n=-1`
* Lifetime limiting can be turned off by setting `--max-lifetime-restarts=-1`
* Added initial (untested) support for separating schedule for beta, LR, and epsilon.
* Added support for beta and epsilon schedule in test config conversion
* Updated docs. Set epsilon to be linear by default.
* Updated configs to add beta and epsilon schedule where appropriate.
* Addressed CR change requests.
* Minor update.
* Updated docs.
* Added changes to CHANGELOG.
* Updated CHANGELOG.
* Fixed typo in Training-Configuration-File.md
* Reverted config changes to remove beta and epsilon schedules.
* mypy error fixed from pre-commit.
* Fixed a bug that was causing a settings test to fail.
* Addressed CR comments.
Co-authored-by: Miguel Alonso Jr <miguel.alonso@miguel.alonso-00MM>
Co-authored-by: Miguel Alonso Jr <miguelalonsojr>
* prioritize cli over yaml in checkpointSettings
* prioritize resume if both set in one place
* fixed test_commandline_args
* addressing comments:renaming, commenting
* more tests and referenced in change_log
* initial commit for a fully connected visual encoder
* adding a test
* addressing comments
* Fixing error with minimal size of fully connected network
* adding documentation and changelog
Maryam Honari
Miguel Alonso Jr
Henry Peteet
Miguel Alonso Jr
Jason Rupert
Maryam Honari
cmard
Vincent-Pierre BERGES
andrewcoh
Ruo-Ping Dong
Ervin T
Chris Elion