NVIDIA Unveils H200 Tensor Core GPU, Boosting AI Training and Inference Performance
NVIDIA has announced the launch of the H200 Tensor Core GPU, the next-generation AI accelerator under its Hopper architecture. Designed for large-scale AI model training and generative AI inference, the GPU will begin shipping to data centers in Q1 2024. The H200 offers higher memory capacity and bandwidth, enabling faster data processing.
H200 Core Spec Upgrades
The H200 is equipped with 141GB of HBM3e high-speed memory, delivering a memory bandwidth of 4.8 TB/s, a significant improvement over the previous generation H100’s 80GB HBM3 and 3.35 TB/s bandwidth. This upgrade makes the H200 more efficient at handling large language models (LLMs). According to NVIDIA’s official data, the H200 provides a 1.9x increase in throughput for Llama 2 70B model inference with FP8 and a 1.4x increase with FP16. Its performance in knowledge distillation tasks is boosted by up to 1.6x.
Optimized for Generative AI Applications
The H200 supports precision technologies like the Transformer Engine, FP8, and FP4, and is optimized for generative AI workloads. It excels in training models like GPT-3 175B, significantly reducing training cycles. NVIDIA states that the H200 is seamlessly compatible with the H100, supporting DGX H100/H200 systems and OVX computing systems for data center-scale AI deployments. These features make the H200 an ideal choice for training trillion-parameter models.
Market Availability and Ecosystem Support
The first H200 GPUs will begin shipping in February 2024 through NVIDIA partners, including OEM partners from Asia. NVIDIA partners like Cisco have already announced support for the H200 in their UCS C3 Super Blades systems. Officials also emphasized that the H200 will work in conjunction with Blackwell architecture GPUs to form a more powerful AI computing platform, driving the evolution of AI infrastructure.
The launch of the H200 continues NVIDIA’s leadership in the AI hardware space, focusing on addressing the performance bottlenecks of memory-intensive AI tasks.
