DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.
machine-learning
deep-learning
pytorch
gpu
compression
billion-parameters
data-parallelism
inference
mixture-of-experts
model-parallelism
pipeline-parallelism
trillion-parameters
zero
Обновлено 2024-11-07 07:38:21 +03:00
Tutel MoE: An Optimized Mixture-of-Experts Implementation
Обновлено 2024-10-28 06:23:06 +03:00
This package implements THOR: Transformer with Stochastic Experts.
Обновлено 2021-10-08 00:19:22 +03:00