DeepSpeed: Extreme-scale model training for everyone
In February, we announced DeepSpeed, an open-source deep learning training optimization library, and ZeRO (Zero Redundancy Optimizer), a novel memory optimization technology in the library, which vastly advances large model training by improving scale, speed, cost, and usability. DeepSpeed has enabled researchers to create Turing Natural Language Generation (Turing-NLG), the largest language model with 17 […]
The post DeepSpeed: Extreme-scale model training for everyone appeared first on Microsoft Research.
Continue reading DeepSpeed: Extreme-scale model training for everyone
