DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.
machine-learning
deep-learning
pytorch
gpu
compression
billion-parameters
data-parallelism
inference
mixture-of-experts
model-parallelism
pipeline-parallelism
trillion-parameters
zero
Обновлено 2024-11-20 04:04:47 +03:00
Tutel MoE: An Optimized Mixture-of-Experts Implementation
Обновлено 2024-11-18 06:00:53 +03:00