Modern processing GPUs are designed to deliver incredible performance at any cost, so their power consumption and cooling requirements are quite huge. Nvidia’s latest H100 compute GPU based on the Hopper architecture can consume up to 700W in an attempt to deliver up to 60 FP64 Tensor TFLOPS, so it was clear from the start that we were dealing with a pretty SXM5 module design. monstrous. Still, Nvidia has never shown this up close.
Our colleagues at ServeTheHome, who were lucky enough to visit one of Nvidia’s offices and see for themselves an H100 SXM5 module, posted a photo of the compute GPU on Thursday. These SXM5 boards are designed for Nvidia’s DGX H100 and DGX SuperPod High Performance Computing (HPC) systems, as well as third-party designed machines. These modules will not be available separately in retail, so seeing them is a rare opportunity.
Nvidia’s H100 SXM5 module features a fully enabled GH100 processing GPU with 80 billion transistors and 8448/16896 FP64 / FP32 cores and 538 Tensor cores (see H100 specification and performance details in the tables below). The GH100 GPU comes with 96GB of HBM3 memory, although thanks to ECC support and other factors, users can access 80GB of ECC-enabled HBM3 memory connected via a 5120-bit bus. The particular GH100 compute GPU pictured is revision A1 marked U8A603.L06 and packaged in week 53 of 2021 (i.e., December 28 to December 31).
Nvidia’s GH100 measures 814mm ^ 2, making it one of the largest chips ever made. In fact, the die sizes of Nvidia’s recent computing GPUs were mainly limited by the lattice size of modern semiconductor manufacturing tools, which is around 850mm ^ 2. Since the chip made using custom TSMC N4 process technology (which belongs to the N5 node family) is made up of 80 billion transistors operating at around 1.40 ~ 1.50GHz, the GPU is extremely power hungry. Nvidia rates its thermal design power at 700W (but that number can change), so it requires an extremely sophisticated voltage regulation module (VRM) capable of delivering enough power to feed the beast.
In fact, the H100 SXM5 module is equipped with a VRM which has 29 high current inductors each equipped with two power stages and three inductors with one power stage. Inductors can survive high temperatures for extended periods of time and are supplied in metal shells to facilitate cooling of the VRM.
The dimensions of the SXM5 module are unknown, but they hardly differ significantly from previous generation Nvidia modules for processing GPUs. Meanwhile, Nvidia has changed the connector layout for the SXM5 (check out ServeTheHome), possibly due to the higher power consumption and faster PCIe Gen5 and NVLink data rates supported by its GH100.
Nvidia will start commercial shipments of its Hopper H100 compute GPUs sometime in the second half of this year and this is when it announces the final specifications of these products and their final TDP.