1d1a20c5a1
Fix the openfold training. ( #4657 )
...
This PR removes the bias created as placeholders, which causes a crash
in openfold's training pipeline.
---------
Co-authored-by: Conglong Li <conglong.li@gmail.com>
2023-11-09 02:04:12 +00:00
c58146471e
Openfold fix ( #4368 )
...
* update
* format
---------
Co-authored-by: Conglong Li <conglong.li@gmail.com>
2023-09-20 01:44:16 -07:00
f876d81d34
DeepSpeed4Science ( #4357 )
...
* zero++ tutorial PR (#3783 )
* [Fix] _conv_flops_compute when padding is a str and stride=1 (#3169 )
* fix conv_flops_compute when padding is a str when stride=1
* fix error
* change type of paddings to tuple
* fix padding calculation
* apply formatting check
---------
Co-authored-by: Cheng Li <pistasable@gmail.com>
Co-authored-by: Olatunji Ruwase <olruwase@microsoft.com>
* fix interpolate flops compute (#3782 )
* use `Flops Profiler` to test `model.generate()` (#2515 )
* Update profiler.py
* pre-commit run --all-files
* Delete .DS_Store
* Delete .DS_Store
* Delete .DS_Store
---------
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
Co-authored-by: Cheng Li <pistasable@gmail.com>
* revert PR #3611 (#3786 )
* bump to 0.9.6
* ZeRO++ chinese blog (#3793 )
* zeropp chinese blog
* try better quality images
* make title larger
* even larger...
* various fix
* center captions
* more fixes
* fix format
* remove staging trigger (#3792 )
* DeepSpeed-Triton for Inference (#3748 )
Co-authored-by: Stephen Youn <styoun@microsoft.com>
Co-authored-by: Arash Bakhtiari <arash@bakhtiari.org>
Co-authored-by: Cheng Li <pistasable@gmail.com>
Co-authored-by: Ethan Doe <yidoe@microsoft.com>
Co-authored-by: yidoe <68296935+yidoe@users.noreply.github.com>
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
* ZeRO++ (#3784 )
Co-authored-by: HeyangQin <heyangqin@microsoft.com>
Co-authored-by: GuanhuaWang <alexwgh333@gmail.com>
Co-authored-by: cmikeh2 <connorholmes@microsoft.com>
Co-authored-by: Ammar Ahmad Awan <ammar.awan@microsoft.com>
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
Co-authored-by: Michael Wyatt <michaelwyatt@microsoft.com>
Co-authored-by: Olatunji Ruwase <olruwase@microsoft.com>
Co-authored-by: Reza Yazdani <reyazda@microsoft.com>
* adding zero++ to navigation panel of deepspeed.ai (#3796 )
* Add ZeRO++ Japanese blog (#3797 )
* zeropp chinese blog
* try better quality images
* make title larger
* even larger...
* various fix
* center captions
* more fixes
* fix format
* add ZeRO++ Japanese blog
* add links
---------
Co-authored-by: HeyangQin <heyangqin@microsoft.com>
Co-authored-by: Conglong Li <conglong.li@gmail.com>
* Bug Fixes for autotuner and flops profiler (#1880 )
* fix autotuner when backward is not called
* fix format
---------
Co-authored-by: Olatunji Ruwase <olruwase@microsoft.com>
* Missing strided copy for gated MLP (#3788 )
Co-authored-by: Ammar Ahmad Awan <ammar.awan@microsoft.com>
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
Co-authored-by: Logan Adams <114770087+loadams@users.noreply.github.com>
* Requires grad checking. (#3789 )
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
* bump to 0.10.0
* Fix Bug in transform.cu (#3534 )
* Bug fix
* Fixed formatting error
---------
Co-authored-by: Logan Adams <114770087+loadams@users.noreply.github.com>
* bug fix: triton importing error (#3799 )
Co-authored-by: Stephen Youn <styoun@microsoft.com>
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
* DeepSpeed4Science (#569 )
* Integrating evoformer attention
* add cutlass version check
* Updaate error message
* add benchmark
* Update
* Update evoformer_attn.py
* Update run_evoformer_test.py
* Update evoformer_attn.py
* Update run_evoformer_test.py
* support more GPU archs
* add copyright
* add tests
* Fix bugs
* Update benchmark
* update
* Fix nvcc macro
* clean code
* fix formatting
* fix yaml import
* skip unit test when not compatible
* fix yaml requirement
* revert changes
* update tutorial
* update
* fix formatting
* fix format
* skip evoformer attn in pre-compile-ops
* revert changes
* update tutorial
* fix cutlass check
* update tutorial
* refactor tutorial
* revise
* Updated the Megatron-DS section (#565 )
* Updated the Megatron-DS section
* minor fix
* minor fix
* minor fix
* separate evoformer tutorial
* Revised the ds4science landing page (#566 )
* Updated the Megatron-DS section
* minor fix
* minor fix
* minor fix
* Revised the landing page
* Revised the landing page
* Removing unused file
* fix links image position
* modify main page
* fix doc
---------
Co-authored-by: Shiyang Chen <csycfl@gmail.com>
Co-authored-by: Minjia Zhang <33713995+minjiaz@users.noreply.github.com>
---------
Co-authored-by: Heyang Qin <heyangqin@microsoft.com>
Co-authored-by: Bill Luo <50068224+zhiruiluo@users.noreply.github.com>
Co-authored-by: Cheng Li <pistasable@gmail.com>
Co-authored-by: Olatunji Ruwase <olruwase@microsoft.com>
Co-authored-by: Guorun <84232793+CaffreyR@users.noreply.github.com>
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
Co-authored-by: stephen youn <13525892+stephen-youn@users.noreply.github.com>
Co-authored-by: Stephen Youn <styoun@microsoft.com>
Co-authored-by: Arash Bakhtiari <arash@bakhtiari.org>
Co-authored-by: Ethan Doe <yidoe@microsoft.com>
Co-authored-by: yidoe <68296935+yidoe@users.noreply.github.com>
Co-authored-by: GuanhuaWang <alexwgh333@gmail.com>
Co-authored-by: cmikeh2 <connorholmes@microsoft.com>
Co-authored-by: Ammar Ahmad Awan <ammar.awan@microsoft.com>
Co-authored-by: Michael Wyatt <michaelwyatt@microsoft.com>
Co-authored-by: Reza Yazdani <reyazda@microsoft.com>
Co-authored-by: Masahiro Tanaka <81312776+tohtana@users.noreply.github.com>
Co-authored-by: Logan Adams <114770087+loadams@users.noreply.github.com>
Co-authored-by: Joe Mayer <114769929+jomayeri@users.noreply.github.com>
Co-authored-by: Ramya Ramineni <62723901+rraminen@users.noreply.github.com>
Co-authored-by: Shiyang Chen <csycfl@gmail.com>
Co-authored-by: Minjia Zhang <33713995+minjiaz@users.noreply.github.com>
2023-09-18 22:16:08 +00:00
57a27b0803
add type checker ignore to resolve that pylance can't resolved noqa annotation ( #4102 )
...
Co-authored-by: Logan Adams <114770087+loadams@users.noreply.github.com>
2023-08-08 13:30:23 +00:00
b361c72761
Update DeepSpeed copyright license to Apache 2.0 ( #3111 )
...
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
2023-03-30 17:14:38 -07:00
91d63e0228
update formatter version and style settings ( #3098 )
2023-03-27 07:55:19 -04:00
0acf7e9c48
[RFC] add device abstraction to allow other device than CUDA be used ( #2221 )
...
Co-authored-by: Olatunji Ruwase <olruwase@microsoft.com>
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
2023-03-07 09:40:17 -08:00
da84e60d98
add missing license info to top of all source code ( #2889 )
...
Co-authored-by: Michael Wyatt <michaelwyatt@microsoft.com>
Co-authored-by: Conglong Li <conglong.li@gmail.com>
Co-authored-by: Olatunji Ruwase <olruwase@microsoft.com>
2023-02-27 11:20:41 -08:00
316c4a43e0
Add flake8 to pre-commit checks ( #2051 )
2022-07-25 16:48:08 -07:00
be789b1665
Fix many typos ( #1423 )
...
* Fix typos in docs/
* Fix typos in code comments and output strings
* Fix typos in the code itself
* Fix typos in tests/
Co-authored-by: Olatunji Ruwase <olruwase@microsoft.com>
2021-10-01 19:56:32 -07:00
a128f34e7d
[benchmarks] flatten/unflatten benchmarks ( #919 )
...
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
2021-04-07 13:06:28 -07:00
cctry
Conglong Li
Earlee
Michael Wyatt
Jeff Rasley
Ma, Guokai
Alex Hedges
Stas Bekman