NVIDIA H100
Hopper architecture · 80GB memory · 990 FP16 TFLOPS · 700W TDP
Cloud Pricing Today
About the NVIDIA H100
The NVIDIA H100, launched in 2022 as part of the Hopper architecture family, has become the backbone of cloud AI infrastructure. It was the first data centre GPU to feature the fourth-generation Tensor Core with FP8 precision support, enabling significantly faster transformer model training compared to its predecessor, the A100.
The H100 delivers 990 FP16 TFLOPS — a 3.2x improvement over the A100's 312 TFLOPS — while maintaining the same 700W TDP envelope. Its 80GB of HBM3 memory provides 3.35 TB/s of memory bandwidth, critical for memory-bound workloads like large language model training.
In the cloud GPU market, the H100 occupies the sweet spot between cost and performance. While newer Blackwell-generation GPUs like the B200 and GB200 offer higher raw performance, the H100 remains the most widely available high-end training GPU, with deep availability across all major cloud regions. This availability advantage, combined with mature software ecosystem support through CUDA and cuDNN, makes it the default choice for most AI training workloads.
The H100 supports NVLink 4.0 with 900 GB/s bidirectional bandwidth, enabling efficient multi-GPU and multi-node training configurations. It also introduced the Transformer Engine, which automatically applies mixed FP8/FP16 precision to accelerate transformer-based models — the dominant architecture in modern AI.
Common Use Cases
Key Facts
- Manufacturer
- NVIDIA
- Architecture
- Hopper
- Accelerator Type
- GPU
- Primary Use
- training
- Memory (VRAM)
- 80 GB
- FP16 Performance
- 990 TFLOPS
- Thermal Design Power
- 700W
Frequently Asked Questions
How much does an H100 cost per hour in the cloud?
The NVIDIA H100 cloud pricing varies by region and pricing model. The blended average across spot, on-demand, and reserved pricing is typically between $4–$8 per hour, though prices vary significantly by region and provider. Spot pricing can be 50–70% lower than on-demand rates.
What is the H100 used for?
The H100 is primarily used for training large AI models, including large language models (LLMs), generative AI systems, and scientific computing workloads. It is also used for high-performance inference where low latency is critical.
How much VRAM does the H100 have?
The NVIDIA H100 has 80GB of HBM3 memory with 3.35 TB/s of memory bandwidth. This is sufficient for training most large language models, though very large frontier models may require multi-GPU configurations or the higher-memory H200 (141GB).
What is the difference between H100 and A100?
The H100 delivers 3.2x the FP16 performance of the A100 (990 vs 312 TFLOPS) and introduces FP8 precision support via the Transformer Engine. Both GPUs have 80GB variants, but the H100 uses faster HBM3 memory (3.35 TB/s vs 2.0 TB/s). The H100 typically costs 2–3x more per hour in the cloud.
Is the H100 better for training or inference?
The H100 excels at both training and inference, but it is most cost-effective for training workloads where its high TFLOPS and memory bandwidth provide the greatest advantage. For pure inference, lower-cost GPUs like the L4 or L40S may offer better price/performance.
Related Accelerators
Compare NVIDIA H100
3.2x FP16 performance (990 vs 312 TFLOPS). Same 80GB VRAM but faster HBM3 memory. H100 costs ~2-3x more per hour but completes training jobs significantly faster.
Same 990 TFLOPS compute, but H200 has 76% more memory (141GB vs 80GB HBM3e). H200 is better for memory-bound workloads like large model inference. H200 is more expensive and less widely available.
B200 delivers ~1.8x the FP16 performance (1,800 vs 990 TFLOPS) with 192GB HBM3e. Blackwell architecture introduces second-gen Transformer Engine. B200 costs more but offers better performance per dollar for large-scale training.
AMD's MI300X offers competitive performance (1,300 FP16 TFLOPS) with significantly more memory (192GB vs 80GB). The MI300X can be cost-competitive but has a smaller software ecosystem compared to NVIDIA's CUDA platform.
Calculate NVIDIA H100 ROI
Estimate payback period, annual returns, and 3-year ROI with live Signwl pricing data.
Track NVIDIA H100 pricing over time
Get access to historical pricing data, regional analysis, and custom alerts.