GitHub - NVIDIA/TransformerEngine: A library for accelerating Transformer models on NVIDIA GPUs, including using 8-bit floating point (FP8) precision on Hopper GPUs, to provide better performance with lower memory utilization in both
![NVIDIA, Arm, and Intel Publish FP8 Specification for Standardization as an Interchange Format for AI | NVIDIA Technical Blog NVIDIA, Arm, and Intel Publish FP8 Specification for Standardization as an Interchange Format for AI | NVIDIA Technical Blog](https://developer-blogs.nvidia.com/wp-content/uploads/2022/09/Language-Model-AI-Inference-1.png)
NVIDIA, Arm, and Intel Publish FP8 Specification for Standardization as an Interchange Format for AI | NVIDIA Technical Blog
Dylan Patel on Twitter: "They literally are able to train GPT-3 with FP8 instead of FP16 with effectively no loss in accuracy. It's just nuts! https://t.co/H4Lr9yuP3h" / Twitter
![Erfolge mit Bfloat16 - Flexpoint, Bfloat16, TensorFloat32, FP8: Dank KI zu neuen Gleitkommazahlen - Golem.de Erfolge mit Bfloat16 - Flexpoint, Bfloat16, TensorFloat32, FP8: Dank KI zu neuen Gleitkommazahlen - Golem.de](https://scr3.golem.de/screenshots/2209/gleitkommzahlen/thumb620/05_New-Hopper-FP8-Precisions.jpg)
Erfolge mit Bfloat16 - Flexpoint, Bfloat16, TensorFloat32, FP8: Dank KI zu neuen Gleitkommazahlen - Golem.de
![NVIDIA, Arm, and Intel Publish FP8 Specification for Standardization as an Interchange Format for AI | NVIDIA Technical Blog NVIDIA, Arm, and Intel Publish FP8 Specification for Standardization as an Interchange Format for AI | NVIDIA Technical Blog](https://developer-blogs.nvidia.com/wp-content/uploads/2022/08/image7-5.png)
NVIDIA, Arm, and Intel Publish FP8 Specification for Standardization as an Interchange Format for AI | NVIDIA Technical Blog
![NVIDIA, Arm, and Intel Publish FP8 Specification for Standardization as an Interchange Format for AI | NVIDIA Technical Blog NVIDIA, Arm, and Intel Publish FP8 Specification for Standardization as an Interchange Format for AI | NVIDIA Technical Blog](https://developer-blogs.nvidia.com/wp-content/uploads/2022/09/Language-Model-AI-Training-1.png)
NVIDIA, Arm, and Intel Publish FP8 Specification for Standardization as an Interchange Format for AI | NVIDIA Technical Blog
![NVIDIA, Arm, and Intel Publish FP8 Specification for Standardization as an Interchange Format for AI | NVIDIA Technical Blog NVIDIA, Arm, and Intel Publish FP8 Specification for Standardization as an Interchange Format for AI | NVIDIA Technical Blog](https://developer-blogs.nvidia.com/wp-content/uploads/2022/09/egx-data-center-kv-2048x1024-1.jpg)