0ad4fd880b
Update zero.md tutorial ( #495 )
...
* Update zero.md
Update to ZeRO tutorial to specify the use of activation checkpointing
* Update zero-offload.md
Use activation checkpointing with ZeRO-Offload
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
2020-11-11 15:29:02 -08:00
be1147c08a
PLD release ( #513 )
...
* Progressive layer dropping docs (#499 )
* test
* Adding tutorial and news page for pld
* updating the tutorial and posts of PLD
* update the finetune tutorial
* Update PLD tutorial (#512 )
* Update installation instructions
* Format fix
* ZeRO tutorial
* Format fixes
* ZeRO-Offload
* ZeRO and ZeRO-Offload tutorials
* Update navigation page
* Format fixes
* Add yuxhe feedback
* Fix blog post link
* Fix OneBit-Adam link
Tweak scheduler example
* Fix date link
* Add DeepSpeed_Adam
* Add PLD tutorial to navigation
Co-authored-by: Shaden Smith <Shaden.Smith@microsoft.com>
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
* updating the pld docs
* DeepSpeed implementation of PLD (#508 )
* DeepSpeed implementation of PLD
* Format fixes
* Formatting fixes
* Fix broken url
* Address PR feedback
* Bump DSE
Co-authored-by: Minjia Zhang <33713995+minjiaz@users.noreply.github.com>
Co-authored-by: Shaden Smith <Shaden.Smith@microsoft.com>
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
Co-authored-by: Minjia Zhang <minjiaz@microsoft.com>
2020-11-10 12:53:50 -08:00
e082d4752e
updating pld docs ( #517 )
2020-11-10 06:33:35 -08:00
41fb24b3f0
Fix PLD news url ( #515 )
...
* PLD documentation
* Formatting fixes
* Fix url bug
2020-11-09 12:28:14 -08:00
e351090c6c
PLD documentation ( #514 )
...
* PLD documentation
* Formatting fixes
2020-11-09 12:20:02 -08:00
f5aa2547d8
Add CPUAdam optimizer for zero-offload in deepspeed engine ( #484 )
...
* add adamW to CPU-ADAM implementation
* supporting cpu-adam optimizer for zero-offload on deepspeed side
* bump DSE to match cpu-adam updates
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
2020-10-30 09:01:04 -07:00
d720fdb685
updating website dependencies ( #475 )
2020-10-19 16:11:22 -07:00
e25f2a23e0
fixing typo ( #460 )
2020-10-12 09:36:18 -07:00
23fc48f320
Add DeepSpeed_Adam optimizer ( #468 )
...
* Update installation instructions
* Format fix
* ZeRO tutorial
* Format fixes
* ZeRO-Offload
* ZeRO and ZeRO-Offload tutorials
* Update navigation page
* Format fixes
* Add yuxhe feedback
* Fix blog post link
* Fix OneBit-Adam link
Tweak scheduler example
* Fix date link
* Add DeepSpeed_Adam
Co-authored-by: Shaden Smith <Shaden.Smith@microsoft.com>
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
2020-10-10 09:48:42 -07:00
2efea69446
gan tutorial ( #462 )
...
* gan tutorial
* formatting fix
* adding pointer to repo; adding navigation link
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
2020-10-06 22:17:34 -07:00
6d176c45fa
link fix part two :-) ( #441 )
2020-09-25 07:48:10 -07:00
0ca82156e1
Update pipeline.md ( #439 )
2020-09-24 21:57:15 -07:00
192cf7c8e2
Update azure.md ( #437 )
2020-09-24 15:28:06 -07:00
5d40f006bb
Fix urls in tutorial ( #436 )
...
* url fix
* revert absolute path but keep some actual fix
* add real readme
2020-09-24 14:10:56 -07:00
c66f388156
Fix few typos in the docs ( #418 )
2020-09-17 06:40:05 -07:00
5812e84544
readthedocs yaml configuration ( #410 )
...
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
2020-09-16 18:57:43 -07:00
7d91be9765
Minor doc fixes ( #417 )
...
* Update installation instructions
* Format fix
* ZeRO tutorial
* Format fixes
* ZeRO-Offload
* ZeRO and ZeRO-Offload tutorials
* Update navigation page
* Format fixes
* Add yuxhe feedback
* Fix blog post link
* Fix OneBit-Adam link
Tweak scheduler example
* Fix date link
Co-authored-by: Shaden Smith <Shaden.Smith@microsoft.com>
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
2020-09-16 13:27:40 -07:00
c82756cd15
readthedocs upgrade ( #402 )
2020-09-10 15:44:47 -07:00
d15015e969
Update ZeRO-Offload blog post link ( #401 )
...
Co-authored-by: Shaden Smith <Shaden.Smith@microsoft.com>
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
2020-09-10 13:25:29 -07:00
5dc4d6c72f
Update news site with press release link
2020-09-10 13:10:34 -07:00
ea92ed29bb
Update _config.yml
2020-09-10 13:07:08 -07:00
4b1df25ae9
bump DSE and doc tweak
2020-09-10 19:32:05 +00:00
6bb5c69f48
Website edits ( #398 )
...
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
2020-09-10 09:18:53 -07:00
a8a8b3d288
Landing page updates ( #395 )
...
Co-authored-by: Shaden Smith <ShadenTSmith@gmail.com>
Co-authored-by: Shaden Smith <Shaden.Smith@microsoft.com>
2020-09-10 02:04:17 -07:00
c76769c4ff
Adding sparse attention news index item ( #376 )
...
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
2020-09-10 01:49:42 -07:00
59ce90d0d4
Minjiaz/zero offload ( #382 )
...
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
2020-09-10 01:46:06 -07:00
be4b94be26
Sparse attention: updating code tag in documentation ( #394 )
...
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
2020-09-10 00:39:47 -07:00
2dea61f285
ZeRO tutorials ( #384 )
...
Co-authored-by: Shaden Smith <Shaden.Smith@microsoft.com>
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
2020-09-10 00:22:55 -07:00
093f09ff27
Update documentation for 1-bit Adam ( #388 )
...
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
2020-09-10 00:05:07 -07:00
65c2f974d8
Pipeline parallel training engine. ( #392 )
...
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
2020-09-09 23:14:55 -07:00
41db1c2f03
ZeRO-Offload release ( #391 )
...
* ZeRO-Offload (squash) (#381 )
Co-authored-by: Olatunji Ruwase <olruwase@microsoft.com>
Co-authored-by: Reza Yazdani <reyazda@microsoft.com>
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
Co-authored-by: Jie <37380896+jren73@users.noreply.github.com>
Co-authored-by: Arash Ashari <arashari@microsoft.com>
Co-authored-by: Reza Yazdani <reyazda@microsoft.com>
Co-authored-by: Samyam Rajbhandari <samyamr@microsoft.com>
Co-authored-by: Shaden Smith <Shaden.Smith@microsoft.com>
Co-authored-by: arashashari <arashashari@ArashMSLaptop.redmond.corp.microsoft.com>
Co-authored-by: RezaYazdaniAminabadi <44502768+RezaYazdaniAminabadi@users.noreply.github.com>
Co-authored-by: Reza Yazdani <reyazda@microsoft.com>
Co-authored-by: Samyam Rajbhandari <samyamr@microsoft.com>
Co-authored-by: Shaden Smith <Shaden.Smith@microsoft.com>
2020-09-09 17:14:12 -07:00
161e8e60c6
fixing a link issue with SA tutorial ( #387 )
...
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
2020-09-09 15:04:17 -07:00
01726ce2b8
Add 1-bit Adam support to DeepSpeed ( #380 )
...
* 1-bit adam (#353 )
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
Co-authored-by: Your Name <you@example.com>
Co-authored-by: tanghl1994 <htang14@ur.rochester.edu>
Co-authored-by: Hank <tanghl1994@gmail.com>
Co-authored-by: root <root@node2x12b.cs.rochester.edu>
Co-authored-by: Ammar Ahmad Awan <awan.ammar@microsoft.com>
2020-09-09 14:37:37 -07:00
b73894dedc
adding sparse attention to feature index page ( #377 )
2020-09-08 19:15:44 -07:00
9dadf38dd6
Update Sparse Attention Tutorial ( #357 )
...
* adding BingSqaud e2e test
* updating the draft test; bring final step under try section
* finalizinf test for base deepspeed and deepspeed with ZeRO
* applying the comment (thanks Jeff); fixed formatting
* update Sparse Attention Tutorial
* fixed few issues and applied comments for better organization and readability
* updated sparse attention tutorial with making how to use section incremental; applying more comments
Co-authored-by: arashashari <arashashari@ArashMSLaptop.redmond.corp.microsoft.com>
2020-09-06 11:20:48 -07:00
9e83ef21ea
Update installation instructions ( #362 )
2020-09-06 08:59:48 -07:00
ac12833ea7
Jekyll installation instructions ( #351 )
2020-09-03 23:35:03 -07:00
6deac82ca6
Adding link to Sparse Attention in Navigation page ( #355 )
...
* adding link to Sparse Attention in Navigation page
2020-09-03 15:01:23 -07:00
e5bbc2e559
Sparse attn + ops/runtime refactor + v0.3.0 ( #343 )
...
* Sparse attn + ops/runtime refactor + v0.3.0
Co-authored-by: Arash Ashari <arashari@microsoft.com>
Co-authored-by: Arash Ashari <arashari@microsoft.com>
2020-09-01 18:06:15 -07:00
903a41a59d
updates website gems after kramdown alert ( #311 )
2020-08-08 15:51:54 -07:00
29c5fe2611
Add webinar link ( #309 )
...
Add webinar on-demand links and update readme
2020-08-07 14:40:31 -07:00
97c5427372
Fixing a typo ( #303 )
2020-07-28 14:24:12 -07:00
7ae8f8bc9b
DeepSpeed webinar announcement ( #301 )
2020-07-24 22:13:11 -07:00
366d88164d
Update amp docs ( #287 )
...
* add amp docs
2020-07-13 13:04:07 -07:00
6379292c62
Improving deepspeed.ai website ( #269 )
...
* syntax/typo fix
* add README for documentation
* fix links
* update navigation
* typo fix
* docs readme fix
2020-06-23 17:13:01 -07:00
664fa30cec
Ai scale ( #271 )
2020-06-19 18:52:42 -07:00
2e6d93e0e5
new transformer pre-ln image ( #268 )
2020-06-17 13:01:57 -07:00
9fcdc88595
adds gtag for website analytics ( #266 )
2020-06-16 17:41:24 -07:00
6e87251cd3
add the fine-tuning results ( #260 )
...
* add the fine-tuning results
* updating tutorial and blog-post
* updated the tutorials and links
2020-06-16 11:00:31 -07:00
2a1c5db1b0
Features links ( #252 )
...
* links and formatting
2020-06-04 12:58:49 -07:00
Samyam Rajbhandari
Olatunji Ruwase
Minjia Zhang
Reza Yazdani
Shaden Smith
niumanar
Haibin Lin
Conglong Li
Gowtham Prudhvi
Jeff Rasley
Arash Ashari
Ammar Ahmad Awan
Emmanuel Kahembwe