DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.
machine-learning
deep-learning
pytorch
gpu
compression
billion-parameters
data-parallelism
inference
mixture-of-experts
model-parallelism
pipeline-parallelism
trillion-parameters
zero
Обновлено 2024-09-17 19:29:18 +03:00
Tutel MoE: An Optimized Mixture-of-Experts Implementation
Обновлено 2024-09-13 14:49:34 +03:00
This package implements THOR: Transformer with Stochastic Experts.
Обновлено 2021-10-08 00:19:22 +03:00