NVIDIA H200

The NVIDIA H200 is an evolution of the H100 that addresses one of its key limitations: memory capacity. By upgrading from 80GB HBM3 to 141GB HBM3e, the H200 provides 76% more memory while maintaining the same 990 FP16 TFLOPS of compute performance.

This memory upgrade is particularly significant for large language model inference, where the entire model must fit in GPU memory for efficient serving. Models that required multi-GPU setups on the H100 can run on fewer H200 GPUs, reducing both cost and latency.

The H200 also benefits from higher memory bandwidth — 4.8 TB/s compared to the H100's 3.35 TB/s — which improves performance on memory-bandwidth-bound operations common in transformer inference. For training workloads, the H200 offers modest improvements through better memory utilisation, though the compute performance remains identical to the H100.

In the cloud market, the H200 commands a premium over the H100 but can deliver better total cost of ownership for memory-intensive workloads by reducing the number of GPUs required.

Common Use Cases

Manufacturer

NVIDIA

Architecture

Hopper

Accelerator Type

GPU

Primary Use

training

Memory (VRAM)

141 GB

FP16 Performance

990 TFLOPS

Thermal Design Power

700W