Unity-Technologies/ml-agents - ml-agents - Git

2 строки

150 B

Plaintext

Исходник Постоянная ссылка Обычный вид История

pull development changes to `main` branch (#81) * yamato promotion min editor bumped to 2020.3 (#34) * Fix pypi publish github action (#35) * update logo (#39) * Move SB3 dependencies to colab (#29) * Basic example running with multiple envs in sb3 * Merge & cleanup * trains 3dball * move SB3 helpers into colab * remove sb3/supersuit hard reqs * remove sb3 funcs * remove unused imports * remove sb3 example directory and script * remove vec env tests * clean colab * fix link Co-authored-by: Henry Peteet <henry.peteet@unity3d.com> Co-authored-by: mahon94 <maryam.honari@unity3d.com> * bump to barracuda 3.0.0 (#30) * bump to barracuda 3.0.0 & Updated submodule com.unity.ml-agents * Feature/mla 2394 update web docs (#37) * Added mkdocs and refactored docs. * Updated docs. * Added gh action to manually public docs. Updated mkdocs.yml. * fix markdown docs linking issue (#40) * Updated submodule com.unity.ml-agents (#41) * Add sonarqube yamato job (#38) * Pinning markdown link check to 3.8.7 on nightly gha. (#43) * Update submodules * pin black version to 22.3.0 (#46) * fix-sonar-bugs (#44) * Update submodules * bump version to 0.29.0 (#42) * bump version to 0.29.0 * remove dev tag * add checks for pytorch-gpu install * pin black version to 22.3.0 * Update submodules * Update automated trigger references to support develop branch (#36) * Update automated trigger references to support develop branch * Update pull request template links to refer to develop branch. * Fix release branch name for github actions * Update pre-commit pattern * Removing test that no longer applys to this repo * Removed dead links in pull request template. (#48) * Use pinned version of package-ci/mac image (#50) * Add shared critic configurability for PPO (#45) * Update submodules * Updated Unity version to 2020.3.33f1. (#53) * Update submodules (#56) Co-authored-by: GitHub Actions - update submodules <bot@noreply.github.com> * Add poca reference and update to latex refs (#57) * add poca ref in docs (#58) * Update submodules * Update Training-Configuration-File.md (#59) * Update Training-Configuration-File.md Doc says "behavior_cloning" when it should say "behavioral_cloning". * Update Learning-Environment-Design-Agents.md Minor correction to wording. (https://github.com/Unity-Technologies/ml-agents/issues/5745) * rename ml-agents/trainers/torch to torch_entities (#55) * rename ml-agents/trainers/torch to torch_entities * fix mock patch * Update submodules * Update docs to explain Alternating Ray Order switch (mla-2537) (#61) * Update docs to explain Alternating Ray Order switch (mla-2537) * Update submodules * Update submodules * Updated validate meta files. (#65) * Fix failing gha pytest (#66) * Updated pytest. * Updated SURVEY.md to remove dead survey link. * Updated bug_report.md to update dead link. * Update submodules * Fixed upm ci bug in yamato tests. (#67) * Update submodules * remove torch shadow in trainers tests (#69) * extend stale bot to 90 days (#70) * Update submodules * Dev bump min ver (#74) * Bumped minimum version of Unity. * Bumped minimum version of python and pytorch. fixed issues with deprecation warnings for x.T (tensor transpose). * Fixed np.bool deprecation warning. This is a port of an external PR: https://github.com/Unity-Technologies/ml-agents/pull/5795. (#71) * Updated submodule. * Bumped max ver of python. * Bumped to 2021.3 in yamato tests. * Fixed version issue with torch in test constraints. * update sysroot version * update toolchain version Co-authored-by: zhuo <zhuo@unity3d.com> * Removed old backwards compatibility test since tensorflow/2019.4 are no longer s upported. (#76) * Develop custom trainers (#73) * Make create_policy more generic (#54) * add on/off policy classes and inherit from * trainers as plugins * remove swap files * clean up registration debug * clean up all pre-commit * a2c plugin pass precommit * move gae to trainer utils * move lambda return to trainer util * add validator for num_epoch * add types for settings/type methods * move create policy into highest level api * move update_reward_signal into optimizer * move get_policy into Trainer * remove get settings type * dummy_config settings * move all stats from actor into dict, enables arbitrary actor data * remove shared_critic flag, cleanups * refactor create_policy * remove sample_actions, evaluate_actions, update_norm from policy * remove comments * fix return type get stat * update poca create_policy * clean up policy init * remove conftest * add sharedecritic to settings * fix test_networks * fix test_policy * fix test network * fix some ppo/sac tests * add back conftest.py * improve specification of trainer type * add defaults fpr trainer_type/hyperparam * fix test_saver * fix reward providers * add settings check utility for tests * fix some settings tests * add trainer types to run_experiment * type check for arbitary actor data * cherrypick rename ml-agents/trainers/torch to torch_entities (#55) * make all trainers types and setting visible at module level * remove settings from run_experiment console script * fix test_settings and upgrade config scripts * remove need of trainer_type argument up to trainefactory * fix gohst trainer behavior id in policy Queue * fix torch shadow in tests * update trainers, rl trainers tests * update tests to match the refactors * fixing behavior name in ghost trainer * update ml-agents-envs test configs * separating the plugin package changes * bring get_policy back for sake of ghost trainer * add return types and remove unused returns * remove duplicate methods in poca (_update_policy, add_policy) Co-authored-by: mahon94 <maryam.honari@unity3d.com> * Online/offline custom trainer examples with plugin system (#52) * add on/off policy classes and inherit from * trainers as plugins * a2c trains * remove swap files * clean up registration debug * clean up all pre-commit * a2c plugin pass precommit * move gae to trainer utils * move lambda return to trainer util * add validator for num_epoch * add types for settings/type methods * move create policy into highest level api * move update_reward_signal into optimizer * move get_policy into Trainer * remove get settings type * dummy_config settings * move all stats from actor into dict, enables arbitrary actor data * remove shared_critic flag, cleanups * refactor create_policy * remove sample_actions, evaluate_actions, update_norm from policy * remove comments * fix return type get stat * update poca create_policy * clean up policy init * remove conftest * add sharedecritic to settings * fix test_networks * fix test_policy * fix test network * fix some ppo/sac tests * add back conftest.py * improve specification of trainer type * add defaults fpr trainer_type/hyperparam * fix test_saver * fix reward providers * add settings check utility for tests * fix some settings tests * add trainer types to run_experiment * type check for arbitary actor data * cherrypick rename ml-agents/trainers/torch to torch_entities (#55) * make all trainers types and setting visible at module level * remove settings from run_experiment console script * fix test_settings and upgrade config scripts * remove need of trainer_type argument up to trainefactory * fix gohst trainer behavior id in policy Queue * fix torch shadow in tests * update trainers, rl trainers tests * update tests to match the refactors * fixing behavior name in ghost trainer * update ml-agents-envs test configs * fix precommit * separating the plugin package changes * bring get_policy back for sake of ghost trainer * add return types and remove unused returns * remove duplicate methods in poca (_update_policy, add_policy) * add a2c trainer back * Add DQN cleaned up trainer/optimizer * nit naming * fix logprob/entropy types in torch_policy.py * clean up DQN/SAC * add docs for custom trainers,TODO: refrence tutorial * add docs for custom trainers,TODO: refrence tutorial * add clipping to loss function * set old importlim-metadata version * bump precomit hook env to 3.8.x * use smooth l1 loss Co-authored-by: mahon94 <maryam.honari@unity3d.com> * add tutorial for validation * fix formatting errors * clean up * minor changes Co-authored-by: Andrew Cohen <andrew.cohen@unity3d.com> Co-authored-by: zhuo <zhuo@unity3d.com> * Trainer qa fix (#78) * grammer fixes * fix nit comments from QA * add info about on/off policy * add more context to the code block * more context and minor fix * pip3 Co-authored-by: zhuo <zhuo@unity3d.com> * Develop merge submodule (#77) * Update com.unity.ml-agents/CHANGELOG.md Co-authored-by: Arthur Juliani <awjuliani@gmail.com> * Update changelog * [MLA-1762] reduce memory allocations from DiscreteActionOutputApplier (#4922) * add option for Burst inference (#4925) * surface specific GRPC errors more visibly (#4930) * Add additional logic to avoid load being called on every advance (#4934) * [MLA-1767] Refactor communicator connection exceptions (#4935) * Update changelog for release 13. (#4938) * Update master versions for release 13. (#4945) * Release 13 versions. (#4946) - updated release tag validation script to automate the updating of files with release tags that need to be changed as part of the pre-commit operation. * Update docs to pass doc validation. (#4953) * update defines, compile out Initialize body on non-desktop (#4957) * Adding references to the Extensions package to help promote it. (#4967) * Adding references to the Extensions package to help promote it. * Updating header formatting to match the rest of the page * Adding references to the Extensions package to help promote it. (#4967) (#4968) Co-authored-by: Marwan Mattar <marwan@unity3d.com> Co-authored-by: Chris Elion <chris.elion@unity3d.com> * InputActuatorComponent to allow the generation of an action space from an InputActionAsset (#4881) * InputActuatorComponent to allow the generation of an action space from an InputActionAsset (#4881) (#4974) * pass sensor dimension flags to analytics (#4954) * Communicator factory (#4965) * Update barracuda, swtich Agents in Sorter use Burst. (#4979) * Update barracuda, swtich Agents in Sorter use Burst. (#4979) (#4981) * Set ignore done=False in GAIL (#4971) * MultiAgentGroup Interface (#4923) * add SimpleMultiAgentGroup * add group reward field to agent and proto * Make TrainingAnalyticsSideChannel internal (#4999) * [MLA-1783] built-in actuator type (#4950) * Add component menues for some sensors and actuators. (#5001) * Add component menues for some sensors and actuators. (#5001) (#5002) * Merge master -> release_13_branch-to-master * Fix RpcCommunicator merge. * master -> main. (#5010) * Adding a name field to BufferSensorComponent * Adding a note to the CHANGELOG about var len obs * Adding a helper method for creating observation placeholder names and removed the _h and _c placeholders * Adding a custom editor for BufferSensorComponent * adding inheritdoc * Update cattrs dependencies to support python3.9 (#4821) * Fix issue with queuing input events that stomp on others. (#5034) * Update cattrs dependencies to support python3.9 (#4821) * Fix issue with queuing input events that stomp on others. (#5034) * Update versions for release 14 hotfix. (#5040) * master -> main. (#5010) (#5044) * Update changelog. (#5045) * [MLA-1809] catch mismatched observation sizes (#5030) * Update changelog. (#5055) * Fix xml docs. (#5057) * pass sensor name through to ObservationSpec (#5036) * Remove unused allocation (#5068) * Automatically generate samples based on placement of mlagents-sample.json files in our examples. (#5077) * Update barracuda to 1.3.2-preview. (#5084) * POCA trainer (#5005) Co-authored-by: Ervin Teng <ervin@unity3d.com> Co-authored-by: Ruo-Ping Dong <ruoping.dong@unity3d.com> Co-authored-by: Chris Elion <chris.elion@unity3d.com> Co-authored-by: Vincent-Pierre BERGES <vincentpierre@unity3d.com> * [docs] Documentation for POCA and cooperative behaviors (#5056) Co-authored-by: Ruo-Ping Dong <ruoping.dong@unity3d.com> * [docs] Update changelog (#5095) * Add multiAgentGroup capabilities flag (#5096) * Add multiAgentGroup capabilities flag * Add proto * Fix compiler error * Add warning for multiagent group * Add comment * Fix spelling mistake * Fix ghost curriculum and make steps private (#5098) * use get step to determine curriculum * add to CHANGELOG * Make step in trainer private (#5099) Co-authored-by: Ervin T <ervin@unity3d.com> * [cherry-pick] Fix ghost curriculum and make steps private (#5098) * use get step to determine curriculum * add to CHANGELOG * Make step in trainer private (#5099) Co-authored-by: Ervin T <ervin@unity3d.com> * Update changelog for samples. (#5103) * Update changelog for samples. (#5103) (#5106) * Update versions on main (#5102) Increment versions after release 15 branch split * [release_15] Release 15 update versions (#5101) * Update versions * Fix for validate release links * Update release tag and docs * [docs] Add missing package docs (#5108) * Add missing package docs * Fix grammar * Update comment on Dispose * Fix typo in doc * [docs] Update readme and changelog (#5107) * Make analytics module an optional dependency. (#5109) Co-authored-by: Chris Elion <chris.elion@unity3d.com> * Redo dotnet format (#5119) * [docs] Add links to example envs for Buffer Sensor and MultiAgentGroup (#5116) * Update changelog with links to example envs * Cooperative push block link update * [cherry-pick] Integrate Group Manager to soccer/retrain with POCA (#5115) (#5121) * Integrate Group Manager to soccer/retrain with POCA (#5115) * Add Soccer env to changelog Co-authored-by: andrewcoh <54679309+andrewcoh@users.noreply.github.com> * [cherry-pick][docs] Add Dungeon Escape Environment (#5133) * Add DungeonEscape POCA Environment (#5128) * Add DungeonEscape assets from working branch * Add Dungeon Escape docs * Create dungeon_escape.png * Add to docs Co-authored-by: Hunter-Unity <hunter@unity3d.com> * update pre-commit instructions for CONTRIBUTING.md (#5130) * Remove duplicated changelog * Fix another duplicated line in the changelog... * Removing Obsolete methods from the package (#5024) * Removing Obsolete methods from the package * Missing depecration and modified changelog * Readding the obsolete BrainParameter methods, will need a larger discussion on these * Removing Action Masker, readding the warining when using a non-implemented Heuristic, Removing NumAction from Brain Parameters * removing documentation and some calls to deprecated methods in the extensions package * Editing the Changelog to put the unreleased on top * non-IEnumerable interface for action masking (#5060) * V2 staging new model version (#5080) * Make modelCheck have flavors of error messages * ONNX exporter v3 * Using a better CheckType and a switch statement * Removing unused message * More tests * Use an enum for valid versions and use GetVersion on model directly * Maybe the model export version a static constant in Python * Use static constructor for FailedCheck * Use static constructor for FailedCheck * Modifying the docstrings * renaming LegacyDiscreteActionOutputApplier * removing testing code * better warning message * Nest the CheckTypeEnum into the FailedCheck class * Update com.unity.ml-agents/Runtime/Inference/BarracudaModelParamLoader.cs * Adding a line explaining that legacy tensor checks are for versions 1.X only * Modifying the changelog * Exporting all the branches size instead of omly the sum (#5092) * addressing comments * Update com.unity.ml-agents/Runtime/Inference/BarracudaModelParamLoader.cs Co-authored-by: Chris Elion <chris.elion@unity3d.com> * readding tests * Adding a comment around the new DiscreteOutputSize method * Clearer warning : Model contains unexpected input > Model requires unknown input * Fixing a bug in the case where the discrete action tensor does not exist Co-authored-by: Chris Elion <chris.elion@unity3d.com> * Make IActuator implement IHeuristicProvider. (#5110) * Adding more tests for v2 models (#5138) * Adding more tests for v2 models * formatting * fixing formatting * [MLA-1634] Add ObservationSpec and update ISensor interfaces (#5127) * Add custom package settings (#5027) * cleanup: yamato triggers, changelog, docstrings * clean up merge * fix compile error * update c# version to 2.0.0-exp.1 (#5160) Co-authored-by: Chris Elion <chris.elion@unity3d.com> * Move physics/sensor tests that fail in edit mode depending on what's in the scene to playmode tests. (#5162) * update package version * remove defines that are now always true * changelog * Remove all old analytics defines. (#5168) * [MLA-1634] Compression spec (#5164) * Low hanging fruit tests for coverage. (#5170) * Update Policies to conditionally compile their analytics method. (#5171) * Turns physics modules into optional dependencies. (#5112) * [MLA-1634] Remove SensorComponent.GetObservationShape() (#5172) * PR Feedback. (#5173) * change default barracuda behavior (#5175) * [MLA-1824] make SensorComponent return ISensor[] (#5181) * Make SensorComponent return an array * split match3 sensors, partial retrain * docstrings, migration, changelog, cleanup * Adding the goal conditioning sensors with the new observation specs (#5159) * Fixing networks.py for the merge * fix compile error * Adding the goal conditioning sensors with the new observation specs * addressing feedback * I forgot to change the m_observationType * Renaming Goal to GoalSignal (#5190) * Renaming GOAL to GOAL_SIGNAL * VectorSensorComponent to use new API * Adding docstrings * verbose pytest on github action Co-authored-by: Chris Elion <chris.elion@unity3d.com> * [MLA-1879] culture-invariant sorting for sensors and actuators (#5194) * Goal conditioning grid world : Example of goal conditioning (#5193) * Aded the Goal conditioned GridWorld to replace regular gridworld * adding missing files * Code improvements * Documentation change on gridworld * resolving conflicts * new model * Addressing comments * comments and renames * Update docs/Learning-Environment-Examples.md Co-authored-by: Ervin T. <ervin@unity3d.com> * adding reference to gridworld in docs about goal signal Co-authored-by: Chris Elion <chris.elion@unity3d.com> Co-authored-by: Ervin T. <ervin@unity3d.com> * [MLA-1584] Match3 variable board size (#5189) * [bug-fix] When agent isn't training, don't clear update buffer (#5205) * Don't clear update buffer, but don't append to it either * Update changelog * Address comments * Make experience replay buffer saving more verbose * Changelog for goal conditioning (#5195) * Changelog for goal conditioning * adding a line about the changes to the gridworld example * Addressing comments * Change capitalization * [bug fix] Fix warning using demo recorder (#5216) * Making VectorActionSize and VectorActionSpaceType internal (#5214) Made sure the editor does not complain; * Turning some logger.info into logger.debug and remove some logging overhead when not using debug (#5211) * turning some logger.info into logger.debug and remove some logging overhead when not using debug * Addressing comments * Adding to changelog * rearrange match3 docs (#5215) * rearrange match3 docs * changelog and migration from previous PR * [bug-fix] Fix POCA LSTM, pad sequences in the back (#5206) * Pad buffer at the end * Fix padding in optimizer value estimate * Fix additional bugs and POCA * Fix groupmate obs, add tests * Update changelog * Improve tests * Address comments * Fix poca test * Fix buffer test * Increase entropy for Hallway * Add EOF newline * Fix Behavior Name * Address comments * [MLA-1880] Raycast sensor interface improvements (#5222) * WIP * remove debug info struct * cleanup + add to test * changelog * fix unit tests * PR feedback * Load individual elements if state dict load fails (#5213) Co-authored-by: Vincent-Pierre BERGES <vincentpierre@unity3d.com> Co-authored-by: Ervin T. <ervin@unity3d.com> * [MLA-1159] Add virtual methods to DecisionRequester (#5223) * [Bug Fix] Serialization of vectorActionSpace (#5220) * [Bug Fix] Serialization of vectorActionSpace * adding a test * Clear ActionBuffers before Heuristic calls (#5227) * Add ObservationType to analytics ObservationSpec (#5218) * [MLA-1909] Match3 and Camera/RenderTexture sensor GC improvements (#5233) * avoid empty set iteration, avoid Debug.AssertFormat (#5246) * avoid empty set iteration, avoid Debug.AssertFormat * changelog * Update to barracuda 1.3.3 and changes to the model inputs and outputs for LSTM (#5236) * Initial commit * making it work with hallway. Added a new model version * addressing feedback * Adding a comment * formatting * modifying the changelog * Adding descriptions on the model version descriptions * Update com.unity.ml-agents/Runtime/Inference/GeneratorImpl.cs Co-authored-by: Chris Elion <chris.elion@unity3d.com> * addressing comments Co-authored-by: Chris Elion <chris.elion@unity3d.com> * [Release 16] Release 16 Merge Back to Main (#5255) Update versions and documentation for Release 16. Co-authored-by: Vincent-Pierre BERGES <vincentpierre@unity3d.com> Co-authored-by: andrewcoh <54679309+andrewcoh@users.noreply.github.com> Co-authored-by: Chris Elion <chris.elion@unity3d.com> * fix link, run link check in quiet mode (#5257) * [MLA-1912] Move Match3 to com.unity.ml-agents (#5259) * Move GridSensor into main package (#5256) * move OneHotGridSensor into main package * changelog and migration guide * remove old doc * check if physics module presents * Add advanced installation guide for com.unity.ml-agents (#5260) * Fix errors when creating new RayCast sensor (#5261) * check if tag and angle is null * add test * changelog * fix all PVS and doc generation warnings (#5262) * Fix --results-dir (#5269) * Fix GridSensorComponent bug (#5270) * LSTM models from 1.x will be incompatible with MLA 2.x (#5254) * LSTM models from 1.x will be incompatible with MLA 2.x * Adding a test and a new v2 model * Make the Model Runner raise an error if using 1.0 model with LSTM * adding a new model for hallway trained with 2.0 * reword error messages * Only raise if error, not if warning * Addressing comments: The legacy Barrauda memory generator and applier were removed. All code that checked for (memories + v1.X) have been removed since these will no longer be supported * Modifying the changelog and the migrating guide with this change * Fixing the merge issues Co-authored-by: Chris Elion <chris.elion@unity3d.com> * [MLA-1939] Match3 Custom Editor (#5263) * Rider suggested cleanup, part 1 (#5265) * Fix the attention module embedding size (#5272) * Fix the attention module embedding size * editing the changelog * [bug-fix] Fix null ref exception in the demo recorder (#5274) * Fix null ref exception in the demo recorder * Update changelog * Clearer logic * Modifying the changelog for release 17 (#5277) * Modifying the changelog for release 17 * Change version to 2.0.0-exp.1 * Removing the [Unreleased] section * check for missing AbstractBoard, display warning (#5276) * Updating the barracuda 1.4.0 (#5291) * [debug] Require all behavior names to have a matching YAML entry (#5210) * Add strict check to settings.py * Remove warning from trainer factory, add test * Add changelog * Fix test * Update changelog * Remove strict CLI options * Remove strict option, rename, make strict default * Remove newline * Update comments * Set default dict to actually default to a default dict * Fix tests * Fix tests again * Default trainer dict to requiring all fields * Fix settings typing * Use logger * Add default_settings to error * Version bump main (#5279) * Version bump for the main branch following branching of release_17 * Messed up the extensions package version * Bring back root reference in grid sensor (#5300) * [MLA-1952] Add optional seed for gym action spaces (#5303) * add optional seed for action spaces * add optional seed for action spaces * changelog * undo packages-lock.json change * Release 17 version bumps and docs version bumps (#5280) * Migration guide - list removed APIs with replacements (#5307) * list removed APIs with replacements * fixes * [docs] Update changelog (#5308) * [MLA-1952] Add optional seed for gym action spaces (#5303) (#5315) * Fix stacked grid sensor (#5335) * Update to Unity Package licenses (#5340) * Update to Unity Package licenses Per Dan * Updating the main repo license dates. * Adding a fully connected visual encoder for super small visual input + tests (#5351) * initial commit for a fully connected visual encoder * adding a test * addressing comments * Fixing error with minimal size of fully connected network * adding documentation and changelog * Make OverlapChecker an interface (#5324) * Add stacking option to VectorSensorComponent (#5376) * [WIP] [Fix] Fixing collect observation called on done (#5375) * [WIP] [Fix] Fixing collect observation called on done * Update com.unity.ml-agents/Runtime/Agent.cs * :warning: Modifying the test of stacking sensor when the agent is done * modifying the documentation for BufferSensor to specify to call AddObservation in the CollectObservations method * Fix NullReferenceException for Behavior Parameters without Agent (#5382) * check agent existence. add warning box * changelog * [WIP] 2.0 verified to main (#5385) * Warn if no joints on Root Body (#5387) * Modifying the changelog for the continuous action SAC target entropy fix (#5394) * Editing the GridSensor documentation for 2D use case (#5396) * Editing the GridSensor documentation for 2D use case * changing chagelog * [MLA-2017] Move colab notebooks to github (#5399) * Fix the reporting of histogram stats and adding a test (#5410) * Fix the reporting of histogram stats and adding a test * Appending to the Changelog * Update changelog before Release 18 branch (#5412) * Update package versions for main branch (#5413) * [Release 18] Update versions and links (#5414) * add changelog entries for PRs that were missing them before. (#5416) * fix cref error found by doc validation (#5421) * fix release versions and changelog section * [bug-fix] Fix when group terminal steps are deleted, robust test (#5441) * Fix when terminal steps are deleted, robust test * Update changelog * Fix test comment * Fix torch device option in argparser (#5467) * [FIX] prioritize cli options over yaml (#5495) * prioritize cli over yaml in checkpointSettings * prioritize resume if both set in one place * fixed test_commandline_args * addressing comments:renaming, commenting * more tests and referenced in change_log * Only update the native library and the c# calls that need to be made to clean up channels. (#5283) * Fix Mac backcompat test (#5519) * Initialize-from custom checkpoints (#5525) * init from any checkpoint including older ones * moving init_path logic ahead to learn.py * fixing pytest to take the full path * doc & changelog * Update gym version to 0.20.0 (#5540) * Fixed failing test due to gym 0.20.0 release. Updated gym-unity package to fix gym version to 0.20.0. * Updated CHANGELOG. * Fix VAIL (#5546) * Custom trainer editor analytics (#5511) * Custom trainer editor analytics * inherit the default_training_analytics_side_channel * Feature/mla 2205 separate schedule lr beta epsilon (#5538) * Added initial (untested) support for separating schedule for beta, LR, and epsilon. * Added support for beta and epsilon schedule in test config conversion * Updated docs. Set epsilon to be linear by default. * Updated configs to add beta and epsilon schedule where appropriate. * Addressed CR change requests. * Minor update. * Updated docs. * Added changes to CHANGELOG. * Updated CHANGELOG. * Fixed typo in Training-Configuration-File.md * Reverted config changes to remove beta and epsilon schedules. * mypy error fixed from pre-commit. * Fixed a bug that was causing a settings test to fail. * Addressed CR comments. Co-authored-by: Miguel Alonso Jr <miguel.alonso@miguel.alonso-00MM> Co-authored-by: Miguel Alonso Jr <miguelalonsojr> * Harden user PII protection logic and extend TrainingAnalytics to expose detailed configuration parameters. (#5512) * Hash128 is not a cryptographic hash, replace with HMAC-SHA256. * Extend TrainingAnalytics side channel to expose configuration details * Change member function scopes and hash demo_paths * Extract tbiEvent hashing method and add test coverage * Restart crashed Unity environments (#5553) ### Change(s) Update the SubprocessEnvManager to restart workers when the underlying Unity environments crash. When a worker receives an ENV_EXITED signal it will now: 1. Record all failures coming through the step queue and drop all other messages. 2. Purge any pending trajectories as they may belong to a crashed worker or be corrupted. 3. Restart all failed workers (up to a configurable limit) This behavior can be limited via a rate limit, max lifetime limit, or both. The configuration options for both are shown below with their default values. ⚠️ Each of these options applies to a single environment, if num_envs > 1 then the limit will apply separately to each replica (num_envs = 2 will spawn 2 Unity environments which can each be restarted 10 times). ```yaml env_settings: # Can restart 10 times over the lifetime of the experiment. max_lifetime_restarts: 10 # Rate limit of 1 failure per 60s restarts_rate_limit_n: 1 restarts_rate_limit_period_s: 60 ``` They can of course be passed via CLI arguments as well ```bash --max-lifetime-restarts --restarts-rate-limit-n --restarts-rate-limit-period-s ``` ### Disabling this feature * Rate limiting can be turned off by setting `--restarts-rate-limit-n=-1` * Lifetime limiting can be turned off by setting `--max-lifetime-restarts=-1` * Develop add training area replicator (#5568) * Added training area replicator to com.unity.ml-agents package. * Added num_areas to Unity RL Initialization proto. Added cli and config file support for num_areas. * Changed training area replicator to size grid automatically from number of areas. * Added tests for the training area replicator. * Added setup for tests for the training area replicator. * Added comments and updated create tutorial for training area replicator. * Updated CHANGELOG. * Fixed some failing tests. * Update com.unity.ml-agents/CHANGELOG.md Co-authored-by: Henry Peteet <henry.peteet@unity3d.com> * Update docs/Learning-Environment-Create-New.md Co-authored-by: Henry Peteet <henry.peteet@unity3d.com> * Update com.unity.ml-agents/Runtime/Areas/TrainingAreaReplicator.cs Co-authored-by: Henry Peteet <henry.peteet@unity3d.com> * Addressed CR comments. Co-authored-by: Miguel Alonso Jr <miguelalonsojr> Co-authored-by: Henry Peteet <henry.peteet@unity3d.com> * Fixing the analytics side-channel for curriculum learning. (#5586) * Fixing the analytics side-channel for curriculum learning. * Made a more robust test. * Update the changelog. * Update com.unity.ml-agents/CHANGELOG.md Co-authored-by: Maryam Honari <honari.m94@gmail.com> Co-authored-by: Maryam Honari <honari.m94@gmail.com> * Method to return stacked observations (#5547) * Method to return stacked observations * Added testing and returning staked observation flat. * Update the comment lines. * Remove brainstorm commits. * Upgrade Barracuda to 2.3.1-preview (#5591) * Upgrade to 2.3.1-preview * Fix for critic normalization bug (#5595) * Added normalization to critic during training for poca, ppo, and sac. * Fixed critic normalization bug. Added new tests to cover. * Updated CHANGELOG. * Fixed typo in test_trainers.py * Deterministic actions python training (#5619) * Progress on propagating the setting to the action model. * Added the _sample_action logic and tests. * Add information to the changelog. * Prioritize the CLI over the configuration file. * Update documentation for config file. * CR refactor. * Update docs/Training-Configuration-File.md Co-authored-by: Miguel Alonso Jr. <76960110+miguelalonsojr@users.noreply.github.com> Update com.unity.ml-agents/CHANGELOG.md Co-authored-by: Miguel Alonso Jr. <76960110+miguelalonsojr@users.noreply.github.com> Update com.unity.ml-agents/CHANGELOG.md Co-authored-by: Miguel Alonso Jr. <76960110+miguelalonsojr@users.noreply.github.com> Update com.unity.ml-agents/CHANGELOG.md Co-authored-by: Maryam Honari <honari.m94@gmail.com> Update ml-agents/mlagents/trainers/settings.py Co-authored-by: Maryam Honari <honari.m94@gmail.com> Update ml-agents/mlagents/trainers/cli_utils.py Co-authored-by: Maryam Honari <honari.m94@gmail.com> * Fix CR requests * Add tests for discrete. * Update ml-agents/mlagents/trainers/torch/distributions.py Co-authored-by: Maryam Honari <honari.m94@gmail.com> * Added more stable test. * Return deterministic actions for training (#5615) * Added more stable test. * Fix the tests. * Fix pre-commit * Fix help line to pass precommit. * support for deterministic inference in onnx (#5593) * Init: actor.forward outputs separate deterministic actions * changelog * Renaming * Add more tests * Package changes to support deterministic inference (#5599) * Init: actor.forward outputs separate deterministic actions * fix tensor shape for discrete actions * Add test and editor flag - Add tests for deterministic sampling - update editor and tooltips * Reverting to "Deterministic Inference" * dissect tests * Update docs * Update CHANGELOG.md Co-authored-by: Chingiz Mardanov <chingiz.mardanov@unity3d.com> Co-authored-by: cmard <87716492+cmard@users.noreply.github.com> * Revert "Deterministic actions python training (#5619)" (#5622) This reverts commit 9ea962027d4c581de7c45a07d669f96453236c5c. * Deterministic actions python training (#5626) * Progress on propagating the setting to the action model. * Added the _sample_action logic and tests. * Add information to the changelog. * Prioritize the CLI over the configuration file. * Update documentation for config file. * CR refactor. * Update docs/Training-Configuration-File.md Co-authored-by: Miguel Alonso Jr. <76960110+miguelalonsojr@users.noreply.github.com> Update com.unity.ml-agents/CHANGELOG.md Co-authored-by: Miguel Alonso Jr. <76960110+miguelalonsojr@users.noreply.github.com> Update com.unity.ml-agents/CHANGELOG.md Co-authored-by: Miguel Alonso Jr. <76960110+miguelalonsojr@users.noreply.github.com> Update com.unity.ml-agents/CHANGELOG.md Co-authored-by: Maryam Honari <honari.m94@gmail.com> Update ml-agents/mlagents/trainers/settings.py Co-authored-by: Maryam Honari <honari.m94@gmail.com> Update ml-agents/mlagents/trainers/cli_utils.py Co-authored-by: Maryam Honari <honari.m94@gmail.com> * Fix CR requests * Add tests for discrete. * Update ml-agents/mlagents/trainers/torch/distributions.py Co-authored-by: Maryam Honari <honari.m94@gmail.com> * Added more stable test. * Return deterministic actions for training (#5615) * Added more stable test. * Fix the tests. * Fix pre-commit * Fix help line to pass precommit. * support for deterministic inference in onnx (#5593) * Init: actor.forward outputs separate deterministic actions * changelog * Renaming * Add more tests * Package changes to support deterministic inference (#5599) * Init: actor.forward outputs separate deterministic actions * fix tensor shape for discrete actions * Add test and editor flag - Add tests for deterministic sampling - update editor and tooltips * Reverting to "Deterministic Inference" * dissect tests * Update docs * Update CHANGELOG.md * Fix the deterministic showing up all the tiime (#5621) Co-authored-by: Chingiz Mardanov <chingiz.mardanov@unity3d.com> Co-authored-by: cmard <87716492+cmard@users.noreply.github.com> * Changelog cleanup. (#5627) * Release cleanup for changelog * Update CHANGELOG.md Co-authored-by: Maryam Honari <honari.m94@gmail.com> * Fix analytics error when compiling XBOX and PS5 (#5628) * Add ENABLE_CLOUD_SERVICES_ANALYTICS Guards To use analytics safely you must wrap usage with `#if ENABLE_CLOUD_SERVICES_ANALYTICS` See https://docs.unity3d.com/ScriptReference/Analytics.Analytics.html * Update DQN Colab example to support export for Barracuda inference (#5643) * Update DQN GridWorld colab example to be capable of outputting an ONNX model that works for inference. -Allow GridWorld to work without a GoalSensor. Defaults to the GreenPlus target in that case. -Add GridWorldColab.onnx file, trained and generated with the colab. * Upgrade DQN GridWorld Colab to produce model that can work with (modified) GridWorld Scene. -permute observation images from NHWC to NCHW right after env_step. Everything following is NCHW. -update input_shape indexes and remove equivalent permute from the VisualQNetwork -fix reference to global qnet inside generate_trajectories (use the passed in version) -create wrapper network to add required constants and dummy input masks required by the runtime inference (Barracuda) -export the wrapped network to ONNX format compatible with Barracuda * Add additional GridWorld scene called GridWorldColab that has the goal sensors removed (target is always green plus) and masks disabled. The purpose of this is to have a scene compatible with the DQN GridWorld Colab example. * Minor edit in Colab docs. * Delete unnecessary .lighting file. * Update changelog. * Update colab/Colab_UnityEnvironment_2_Train.ipynb Co-authored-by: Maryam Honari <honari.m94@gmail.com> * Fix error in colab metadata name. * Add suggested comments and fix incorrect parameter ordering. * Update docs for Elo curriculum completion measure. Co-authored-by: Maryam Honari <honari.m94@gmail.com> * Changelog prepped for release (#5629) * Changelog prepped for release * Added missing PRs to the changelog. * Added missing CHANGELOG item. * enable 3D Ball sample import button in package manager for unity sdk (#5665) * enable 3D Ball sample import button in package manager for unity sdk * add Samples folder * add meta file * fix git ignore config * Updated versions for main. Also updated validate versions script to r… (#5668) * Updated versions for main. Also updated validate versions script to remove exp.1 from package version. * Reverted validate versions. * Updated versions. * Bumped package version for patch. * Removed unreleased section. (#5667) * Updated release versions and release links. (#5669) * Updates for package validation errors. (#5671) * Release 19 branch lts deprecation (#5673) * Updated Project and DevProject to 2020.3 LTS. * Updated supported LTS version to 2020.3. * Updated changelog. * Bumped up CI Unity version from 2020.2 to 2020.3. * Update CHANGELOG.md * Removing VS Studio dependency from manifest.json. * Update packages-lock.json * Update CHANGELOG.md * Bumped package version for patch. * Fixed build-docker and updated CHANGELOG to add unreleased section. (#5682) * Fixed build-docker and updated CHANGELOG to add unreleased section. * Updated backward compat mac training yamato test. (#5685) * Develop deprecate python 3.6 (#5) * Dropped support for python 3.6 * Pinning python 3.9.9 for tests due to typing issues with 3.9.10 * Testing new bokken image. * Testing new bokken image. * Updated yamato standalone build test. * Updated yamato standalone build test. * Updated standalone build test. * Updated yamato configs to use mla bokken vm. * Bug fixes for yamato yml files. * Fixed com.unity.ml-agents-test.yml * Bumped min python version to 3.7.2 * Updated failing yamato jobs. * Updated github workflow for colab tests. * Updated github workflow for colab tests. * Updated github workflow for colab tests. * Develop python api ga (#6) * Dropped support for python 3.6 * Pinning python 3.9.9 for tests due to typing issues with 3.9.10 * Testing new bokken image. * Testing new bokken image. * Updated yamato standalone build test. * Updated yamato standalone build test. * Updated standalone build test. * Updated yamato configs to use mla bokken vm. * Bug fixes for yamato yml files. * Fixed com.unity.ml-agents-test.yml * Bumped min python version to 3.7.2 * pettingzoo api prototype * add example * update file names * support multiple behavior names * fix multi behavior action index * add install in colab * add setup * update colab * fix __init__ * clone single branch * import tags only * import in init * catch import error * update colab * move colab and add readme * handle agent dying * add tests * update doc * add info * add action mask * fix action mask * update action masks in colab * change default env * set version * fix hybrid action * fix colab for hybrid actions * add note on auto reset * Updated colab name. * Update README.md * Following petting_zoo registry API (#5557) * init petting_zoo registry * cherrypick Custom trainer editor analytics (#5511) * cherrypick "Update dotnet-format to address breaking changes introduced by upstream changes (#5528)" * Update colab to match pettingZoo import api * ToRevert: pull exp-petting-registry branch * Add init file to tests * Install pettingzoo-unity requirements for pytest * update pytest command * Add docstrings and comments * update coverage to pettingzoo folder * unset log level * update env string * Two small bugfixes (#5589) 1. Add the missing `_cumulative_rewards` property 2. Update `agent_selection` to not error out when an agent finishes an episode. * Updated gym to 0.21.0 and petting zoo to 1.13.1, fixed bugs with AEC wrapper for gym and PZ updates. API tests are passing. * Some refactoring. * Finished inital implementation of parallel. Tests not passing. * Finished parallel API implementation and refactor. All PZ tests passing. * Cleanup. * Refactoring. * Pinning numpy version. * add metadata and behavior_specs initialization * addressing behaviour_spec issues * Bumped PZ version to 1.14.0. Fixed failing tests. * Refactored gym-unity and petting-zoo into ml-agents-envs * Added TODO to pydoc-config.yaml * Refactored gym and pz to be under a subpackage in mlagents_env package * Refactored ml-agents-envs docs. * Minor update to PZ API doc. * Updated mlagents_envs docs and colab. * Updated pytest gh workflow to remove ref to gym and pz. * Refactored to remove some test coupling between trainers and envs. * Updated installation doc. * Update ml-agents-envs/README.md Co-authored-by: Andrew Cohen <andrew.cohen@unity3d.com> * Updated failing yamato jobs. * pettingzoo api prototype * add example * update file names * support multiple behavior names * fix multi behavior action index * add install in colab * add setup * update colab * fix __init__ * clone single branch * import tags only * import in init * catch import error * update colab * move colab and add readme * handle agent dying * add tests * update doc * add info * add action mask * fix action mask * update action masks in colab * change default env * set version * fix hybrid action * fix colab for hybrid actions * add note on auto reset * Updated colab name. * Update README.md * Following petting_zoo registry API (#5557) * init petting_zoo registry * cherrypick Custom trainer editor analytics (#5511) * cherrypick "Update dotnet-format to address breaking changes introduced by upstream changes (#5528)" * Update colab to match pettingZoo import api * ToRevert: pull exp-petting-registry branch * Add init file to tests * Install pettingzoo-unity requirements for pytest * update pytest command * Add docstrings and comments * update coverage to pettingzoo folder * unset log level * update env string * Two small bugfixes (#5589) 1. Add the missing `_cumulative_rewards` property 2. Update `agent_selection` to not error out when an agent finishes an episode. * Updated gym to 0.21.0 and petting zoo to 1.13.1, fixed bugs with AEC wrapper for gym and PZ updates. API tests are passing. * Some refactoring. * Finished inital implementation of parallel. Tests not passing. * Finished parallel API implementation and refactor. All PZ tests passing. * Cleanup. * Refactoring. * Pinning numpy version. * add metadata and behavior_specs initialization * addressing behaviour_spec issues * Bumped PZ version to 1.14.0. Fixed failing tests. * Refactored gym-unity and petting-zoo into ml-agents-envs * Added TODO to pydoc-config.yaml * Refactored gym and pz to be under a subpackage in mlagents_env package * Refactored ml-agents-envs docs. * Minor update to PZ API doc. * Updated mlagents_envs docs and colab. * Updated pytest gh workflow to remove ref to gym and pz. * Refactored to remove some test coupling between trainers and envs. * Updated installation doc. * Update ml-agents-envs/README.md Co-authored-by: Andrew Cohen <andrew.cohen@unity3d.com> * Updated CHANGELOG. * Updated Migration guide. * Doc updates based on CR. * Updated github workflow for colab tests. * Updated github workflow for colab tests. * Updated github workflow for colab tests. * Fixed yamato import error. Co-authored-by: Ruo-Ping Dong <ruoping.dong@unity3d.com> Co-authored-by: Miguel Alonso Jr <miguelalonsojr> Co-authored-by: jmercado1985 <75792879+jmercado1985@users.noreply.github.com> Co-authored-by: Maryam Honari <honari.m94@gmail.com> Co-authored-by: Henry Peteet <henry.peteet@unity3d.com> Co-authored-by: mahon94 <maryam.honari@unity3d.com> Co-authored-by: Andrew Cohen <andrew.cohen@unity3d.com> * Refactoring for extraction to separate repo. * removing unused github workflows. * Updates for pre-commit and github actions. * Fixed bug with validate_meta_files.py missing a file to exclude. * Updated precommit gh action. * Refactored yamato pack. * Added yamato test yaml. * Updated to CHANGELOG and VectorSensor from mono repo before swap. * Pin dotnet version for GHA formatting. * Refactor for MLA pro support. (#1) * Refactored for MLA pro support. * Fixed bug in yamato test. * Updated the CHANGELOG. * Fixed trigger logic in yamato test. * Fixed failing tests. (#2) * Fixed failing tests. * Adding meta files. * Fix execution order (#4) * Fixed execution order. * Fixed bug with CRT build. * precommit formatting update. * Updated agent Awake docstring. * Set execution order to be negative to make sure scripts run before default time. (#5) * Added #if to exclude communicator registration in the Agent awake script (#6) * Add visibility to pro package (#7) * add coverage publish promotion yamato jobs (#3) * bump barracuda to 3.0.0 (#9) upgrade to barracuda 3.0.0 * Updated README. (#8) * add validation test dep. to publish job (#12) * add validation test dep. to publish job * fix package path * Update automated trigger rules to support oneflow develop branch (#11) * Update automated trigger rules to support oneflow develop branch * Fix release branch name for github actions * Release 2.3.0 exp.1 (#13) * upgrade package to 2.3.0-exp.1 * rebase on develop branch * update changelog * remove unreleased section (#16) * Updated experimental prerelease version and changelog. (#18) * Added submodule update gha. (#19) * fix bugs from sonar-scanner (#21) * add ml-agents-pro as a upstream for submodule update (#22) * Added support for standalone use of Academy with refactored communicator factory. (#24) * clean commandline Arguments (#25) * clean commandline Arguments * fix precommit * update unrelease section in changelog (#27) * MLA-2537 Reorder RayPerceptionSensor output to reflect its spatial structure. (#26) * Changes to support in order rays. GUI for switching to this. Updated tests. * Updated README. (#29) * Fixed samples. (#30) * Added camera sensor component checkbox for runtime enablement. update… (#31) * Added camera sensor component checkbox for runtime enablement. updated camera sensor to only be active when enabled for runtime. * Updated yamato job. * Updated CHANGELOG. * Fixed issue with Academy not building for webgl builds. (#32) * Dev bump min ver (#33) * Bumped min version of Unity to 2021.3. * Updated min version of python in pre-commit. Updated changelog. * Moving files into packages/com.unity.ml-agents directoy. * Remove com.unity.ml-agents submodule. * Added back in ml-agents package. * Removed packages temp folder. * Updated yamato protobuf gen. * Updated yamato to remove submodule update. * Updated linux tool chain. * Updated Project package dependencies. * Updated flake8 repo link in precommit. Co-authored-by: Vincent-Pierre BERGES <vincentpierre@unity3d.com> Co-authored-by: Arthur Juliani <awjuliani@gmail.com> Co-authored-by: Chris Elion <chris.elion@unity3d.com> Co-authored-by: andrewcoh <54679309+andrewcoh@users.noreply.github.com> Co-authored-by: Chris Goy <christopherg@unity3d.com> Co-authored-by: Marwan Mattar <marwan@unity3d.com> Co-authored-by: Ruo-Ping Dong <ruoping.dong@unity3d.com> Co-authored-by: Ervin Teng <ervin@unity3d.com> Co-authored-by: Hunter-Unity <hunter@unity3d.com> Co-authored-by: Maryam Honari <honari.m94@gmail.com> Co-authored-by: Miguel Alonso Jr <76960110+miguelalonsojr@users.noreply.github.com> Co-authored-by: Miguel Alonso Jr <miguel.alonso@miguel.alonso-00MM> Co-authored-by: Jason Bowman <jasonb@unity3d.com> Co-authored-by: Henry Peteet <henry.peteet@unity3d.com> Co-authored-by: cmard <87716492+cmard@users.noreply.github.com> Co-authored-by: Chingiz Mardanov <chingiz.mardanov@unity3d.com> Co-authored-by: Jason Rupert <92821409+jrupert-unity@users.noreply.github.com> Co-authored-by: Xun Cao <xun.cao@outlook.com> Co-authored-by: jmercado1985 <75792879+jmercado1985@users.noreply.github.com> Co-authored-by: mahon94 <maryam.honari@unity3d.com> Co-authored-by: Andrew Cohen <andrew.cohen@unity3d.com> Co-authored-by: Xun Cao <xun.cao@unity3d.com> Co-authored-by: Jason Rupert <jason.rupert@unity3d.com> * Release 20 changelog (#80) * clean changelog * match extention package dependency * fix typos Co-authored-by: Xun Cao <xun.cao@unity3d.com> Co-authored-by: Brandon Henry <brandonh@unity3d.com> Co-authored-by: Andrew Cohen <andrew.cohen@unity3d.com> Co-authored-by: Henry Peteet <henry.peteet@unity3d.com> Co-authored-by: Miguel Alonso Jr <miguel.alonso@unity3d.com> Co-authored-by: GitHub Actions - update submodules <bot@noreply.github.com> Co-authored-by: Jason Bowman <jasonb@unity3d.com> Co-authored-by: Florence Rolland <florence@unity3d.com> Co-authored-by: Jason Rupert <jason.rupert@unity3d.com> Co-authored-by: zhuo <zhuo@unity3d.com> Co-authored-by: Vincent-Pierre BERGES <vincentpierre@unity3d.com> Co-authored-by: Arthur Juliani <awjuliani@gmail.com> Co-authored-by: Chris Elion <chris.elion@unity3d.com> Co-authored-by: andrewcoh <54679309+andrewcoh@users.noreply.github.com> Co-authored-by: Chris Goy <christopherg@unity3d.com> Co-authored-by: Marwan Mattar <marwan@unity3d.com> Co-authored-by: Ruo-Ping Dong <ruoping.dong@unity3d.com> Co-authored-by: Ervin Teng <ervin@unity3d.com> Co-authored-by: Hunter-Unity <hunter@unity3d.com> Co-authored-by: Maryam Honari <honari.m94@gmail.com> Co-authored-by: Miguel Alonso Jr <76960110+miguelalonsojr@users.noreply.github.com> Co-authored-by: Miguel Alonso Jr <miguel.alonso@miguel.alonso-00MM> Co-authored-by: cmard <87716492+cmard@users.noreply.github.com> Co-authored-by: Chingiz Mardanov <chingiz.mardanov@unity3d.com> Co-authored-by: Jason Rupert <92821409+jrupert-unity@users.noreply.github.com> Co-authored-by: Xun Cao <xun.cao@outlook.com> Co-authored-by: jmercado1985 <75792879+jmercado1985@users.noreply.github.com> 2022-11-19 00:21:12 +03:00			`# see https://docs.github.com/en/github/creating-cloning-and-archiving-repositories/about-code-owners for more information`
			`* @unity/behavior-authoring`