* Aded the Goal conditioned GridWorld to replace regular gridworld
* adding missing files
* Code improvements
* Documentation change on gridworld
* resolving conflicts
* new model
* Addressing comments
* comments and renames
* Update docs/Learning-Environment-Examples.md
Co-authored-by: Ervin T. <ervin@unity3d.com>
* adding reference to gridworld in docs about goal signal
Co-authored-by: Chris Elion <chris.elion@unity3d.com>
Co-authored-by: Ervin T. <ervin@unity3d.com>
* Add pushblock collab
* Make SimpleMultiAgentGroup public
* Remove GoalDetectTrigger
* Remove GDT meta file
* Remove some comments
* Add training configuration
* Rename behavior
* Add to docs
* Change the reward structure in docs
* Add back GoalDetectTrigger
Co-authored-by: HH <brandonh@unity3d.com>
* Removing some scenes, All the Static and all the non variable speed environments. Also removed Bouncer, PushBlock, WallJump and reacher. Removed a bunch of visual environements as well. Removed 3DBallHard and FoodCollector (kept Visual and Grid FoodCollector)
* readding 3DBallHard
* readding pushblock and walljump
* Removing tennis
* removing mentions of removed environments
* removing unused images
* Renaming Crawler demos
* renaming some demo files
* removing and modifying some config files
* new examples image?
* removing Bouncer from build list
* replacing the Bouncer environment with Match3 for llapi tests
* Typo in yamato test
VisualFoodCollector is now an example environment of using a mix of visual and vector observation and is able to train with default config file.
Co-authored-by: Vincent-Pierre BERGES <vincentpierre@unity3d.com>
* initial commit
* works with Pyramids
* added unit tests and a separate config file
* Adding first batch of documentation
* adding in the docs that rnd is only for PyTorch
* adding newline at the end of the config files
* adding some docs
* Code comments
* no normalization of the reward
* Fixing the tests
* [skip ci]
* [skip ci] Make sure RND will only work for Torch by editing the config file
* [skip ci] Additional information in the Documentation
* Remove the _has_updated_once flag
* init
* Add reward manager and hurryUpReward
* fix hurry reward/ add awful first training
* Turn off head height and hurry rew
* changed max speed to 15. added small hh rew
* add NaN check for reward manager. start vel penalty
* add bpVel pen
* add new BPVelPen nn file
* remove outdated nn file
* add randomize speed bool
* try rewad product
* change coeff to 1
* try avg vel of all bp for reward
* move outside loop
* try linear inverselerp for vel
* add avg rew matchspeed15 nn file. looks much better
* save scene
* no hand penalty, random walk speed
* fix inverse lerp
* try new reward falloff
* cleanup
* added new nn file. don't allow hand contact
* update obsv
* remove hh rew. add trained no-hh model
* add new nn file
* new curve
* add new models. try no reset
* add hh rew
* clamp hh
* zero rewards if ground contact
* switch to approved with moving target
* try new dot
* add shifted dot and reg dot nn file
* add WalkerStaticVariableSpeedScene and PPO config
* add a NaN debug for action values
* start dynamic cleanup and more debug for NaNs
* more cleanup
* add WalkerDynamicVarialbeSpeed scene and update prefabs
* add trained static walker nn file
* About to do cleanup
* add all scenes
* reduce numpy ver
* add new no hh nn models and update prefabs
* add hh rew
* try 15k strength. reset jdcontroller to master
* remove h rew 10k strength
* increase to 30M
* trying to figure out shuffle foot regression. added 10k no hh model
* about to train 20k strength, no hh, no rolling targ 30M
* fixed shuffle step regr with 20k no hh
* update prefabs with new models. walkerstatic failed to train
* saved scene
* implemented distToTarget Instead of targetPos
* add dist observ nn files
* more cleanup
* reduce maxSpeed to 10, update prefabs
* max dist 50 avg core vel
* cleanup
* use all bp for avg vel
* reimplement cube relTargetPos
* update prefabs
* add relPos clamped to 100m models
* cleanup
* more prefab cleanup
* more cleanup
* remove unused prefabs
* remove unused code
* replace demo files
* remove demorecorder
* reset ppo learn.py to master
* reset these to master
* Update Learning-Environment-Examples.md
* cleanup from PR review
* more cleanup
* add code comments
* observe velocity delta
* add additional velocity observations
* add hh
* add trained models. remove hh rew
* remove multiple walk dir methods because its confusing
* update walker static vs prefabs
* add new trained models and romove old ones
* add new demo files
* reset script to master
* custom setter for TargetWalkingSpeed
* update benchmarks based on new models
* cleanup per PR suggestions
* Introduced the Constant Parameter Sampler that will be useful later as samplers and floats can be used interchangeably
* Refactored the settings.py to refect the new format of the config.yaml
* First working version
* Added the unit tests
* Update to Upgrade for Updates
* fixing the tests
* Upgraded the config files
* Fixes
* Additional error catching
* addressing some comments
* Making the code nicer with cattr
* Added and registered an unstructure hook for PrameterRandomization
* Updating C# Walljump
* Adding comments
* Add test for settings export (#4164)
* Add test for settings export
* Update ml-agents/mlagents/trainers/tests/test_settings.py
Co-authored-by: Vincent-Pierre BERGES <vincentpierre@unity3d.com>
Co-authored-by: Vincent-Pierre BERGES <vincentpierre@unity3d.com>
* Including environment parameters for the test for settings export
* First documentation update
* Fixing a link
* Updating changelog and migrating
* adding some more tests for the conversion script
* fixing bugs and using samplers in the walljump curriculum
* Changing the format of the curriculum file as per discussion
* Addressing comments
* Update ml-agents/mlagents/trainers/settings.py
Co-authored-by: Ervin T. <ervin@unity3d.com>
* Update docs/Migrating.md
Co-authored-by: Chris Elion <chris.elion@unity3d.com>
* addressing comments
Co-authored-by: Ervin T <ervin@unity3d.com>
Co-authored-by: Chris Elion <chris.elion@unity3d.com>
* added Target and OCube controllers. updated crawler envs
* update walker prefab
* add refs to prefab
* Update Crawler.prefab
* update platform, ragdoll, ocube prefabs
* reformat file
* reformat files
* fix behavior name
* add final retrained crawler and walker nn files
* collect hip ocube rot in world space
* update crawler observations and update prefabs
* change to 20M steps
* update crwl prefab to 142 observ
* update obsvs to 241. add expvel reward
* change walkspeed to 3
* add new crawler and walker nn files
* adjust rewards
* enable other pairs
* add RewardManager
* cleanup about to do final training
* cleanup add nn files for increased facing rew reduced height rew
* try no facing rew
* add vel only policy, try dy target
* inc torq on cube
* added dynamic cube nn. gonna try 40M steps
* add 40M step test, more cleanup
* change back to 20M steps
* Update WalkerStatic.unity
* add no vel pen nn file
* .005 head height rew
* remove extra walker in scene
* Update WalkerWithTargetPair.prefab
* Update WalkerStatic.unity
* more cleanup add new nn file with less head height reward
* added Target and OCube controllers. updated crawler envs
* update walker prefab
* add refs to prefab
* Update Crawler.prefab
* update platform, ragdoll, ocube prefabs
* reformat file
* reformat files
* fix behavior name
* add final retrained crawler and walker nn files
* collect hip ocube rot in world space
* update crawler observations and update prefabs
* change to 20M steps
* update crwl prefab to 142 observ
* update obsvs to 241. add expvel reward
* change walkspeed to 3
* add new crawler and walker nn files
* adjust rewards
* enable other pairs
* add RewardManager
* cleanup about to do final training
* cleanup add nn files for increased facing rew reduced height rew
* try no facing rew
* add vel only policy, try dy target
* inc torq on cube
* added dynamic cube nn. gonna try 40M steps
* add 40M step test, more cleanup
* change back to 20M steps
* Update WalkerStatic.unity
* add no vel pen nn file
* .005 head height rew
* remove extra walker in scene
* Update WalkerWithTargetPair.prefab
* Update WalkerStatic.unity
* more cleanup add new nn file with less head height reward
* cleanup
* remove comment
* more cleanup
* correct format
* Update ProjectVersion.txt
* change to Log()
* cleanup
* use the starting y position instead of a hard coded height
* test old fromtorot
* add 236 model
* testing new 236 nn files
* add final walker nn files
* cleanup
* crawler cleanup
* update crawler observ size
* add final crawler nn files
* fixed formatting ssues
* about to implement orientation cube
* oCube spawining works. ready to train
* working. about to try com
* ready for training
* add random rot on episode start
* feet now alternate but runs backwards
* still running with right leg in front
* increased joint strength to 40k
* removed texture example
* reduced maxAngVel, enabled enhanced determinism, cont spec
* rebuilt walker ragdoll to scale 1
* rebuilt ragdoll ready
* update walker pair prefab
* fixed bp heirarchy
* added trained model, renamed scene, usecollisioncallbacks
* updated dynamic platforms
* added dynamic walker tf file. max speed 5
* DynamicWalker working. has working nn file
* collect local rotations
* added new dynamic nn file
* hip facing reward
* Create WalkerDynamic.yaml
* fix hip rotation
* about to clean up code
* added dirIndicator and orentCubeGizmo
* clean up
* cleanup
* updated WalkerStatic scene with new ragdoll
* cleanup
* updated walker dynamic demo file. cleanup
* iterate through list not dict to collect observations
* increase gravity to 1.5
* try 100M steps on walkerdynamic
* 100M steps
* add dir vector obsv
* 2e7 steps
* testing new nn models
* testing bigger batch size
* try 8x mem for cloud
* 8x batch size for cloud test
* epoch 10
* hyptest
* cp
* increase timescale for cloudtraining
* cp
* try new cluster
* cp
* 200k buff cloud
* cleanup & put direction indicator in separate script
* update configs
* about to implement orientation cube
* oCube spawining works. ready to train
* working. about to try com
* ready for training
* add random rot on episode start
* feet now alternate but runs backwards
* still running with right leg in front
* increased joint strength to 40k
* removed texture example
* reduced maxAngVel, enabled enhanced determinism, cont spec
* rebuilt walker ragdoll to scale 1
* rebuilt ragdoll ready
* update walker pair prefab
* fixed bp heirarchy
* added trained model, renamed scene, usecollisioncallbacks
* updated dynamic platforms
* added dynamic walker tf file. max speed 5
* DynamicWalker working. has working nn file
* collect local rotations
* added new dynamic nn file
* hip facing reward
* Create WalkerDynamic.yaml
* fix hip rotation
* about to clean up code
* added dirIndicator and orentCubeGizmo
* clean up
* cleanup
* updated WalkerStatic scene with new ragdoll
* cleanup
* updated walker dynamic demo file. cleanup
* iterate through list not dict to collect observations
* increase gravity to 1.5
* try 100M steps on walkerdynamic
* 100M steps
* add dir vector obsv
* 2e7 steps
* testing new nn models
* testing bigger batch size
* try 8x mem for cloud
* 8x batch size for cloud test
* epoch 10
* hyptest
* cp
* increase timescale for cloudtraining
* cp
* try new cluster
* cp
* 200k buff cloud
* cleanup & put direction indicator in separate script
* update configs
* update configs to new class format
* added final nn files
* more cleanup
* new walker image for docs
* Update walker docs
* remove old gitignore item
* cleanup
* Delete trainer_config.yaml
* Update CHANGELOG.md
* remove code comment
* changed property to float
* rename variable
* remove header
* rename function
* added code comment and consolidated similar properties
* removed unused asset
* make maxAngularVelocity a constant
* cleeanup remove tab
* cleanup - remove unneeded header attr
* added code comments
* auto-format doc to remove unwanted tabs
* add new trained model. increase max step for dynamic
* add code comments. update oCube system. cleanup
* move orientation cube to shared prefabs
* refactored reward function variables
* removed header
* add SAC configs
* added new dynamic walker nn file
* remove old config
* add new config
* fix project ver
Thomas Simonini
Hunter-Unity
Ruo-Ping Dong
Ervin T
Vincent-Pierre BERGES
andrewcoh
Hunter-Unity
Christopher Goy
Chris Elion
Andrew Cohen
Jaden Travnik