Grace, Grace Hopper Superchips: New Details, Delayed To Late 2023

Nvidia used GTC 2023 to highlight how its CPU-based Grace superchip and its Grace Hopper superchip, which combines a CPU and GPU, will accelerate new workloads.

The company also disclosed that while the two chip modules are now sampling with customers, it has delayed their availability within systems to the second half of the year after promising multiple times in 2022 that systems would arrive in the first half of 2023.

Server vendors expected to support the Grace Superchip consist of Asus, Atos, Gigabyte, Hewlett Packard Enterprise, QCT, Supermicro, Wistron and ZT.

While Nvidia CEO Jensen Huang emphasized at GTC 2023 that traditional CPUs aren’t well suited for many accelerated computing applications, he said the company designed the Grace CPU, which consists of 72 Arm-compatible cores, to “excel at single-threaded execution and memory processing” and provide “high energy efficiency at cloud data center.” These facets make the CPU “excellent for cloud and scientific computing applications,” Huang added.

The Grace superchip combines two Grace CPUs to provide a total of 144 cores, which are connected over a 900 GB/s low-power chip-to-chip coherent interface. For memory, the superchip has “server-class” LPDDR5X, which Nvidia said is the first data center CPU to do so.

Huang said Nvidia tested Grace on a popular Google benchmark, which measures performance for cloud microservices, as well as a suite of Apache Spark benchmarks that test memory-intensive processing, which the CEO called “foundation for cloud data centers.”

Compared to the newest generation of x86 CPUs, Grace is, on average, 30 percent faster at microservices and 20 percent faster at data processing while “using only 60 percent of the power measured at the full server node,” according to Huang, who didn’t identify the rival CPU.

“[Cloud service providers] can outfit a power-limited data center with 1.7 times more Grace servers, each delivering 25 percent higher throughput,” he said.

While Nvidia is designed the Grace superchip for cloud and scientific computing, its tailoring the Grace Hopper superchip for “processing giant data sets, like AI databases for recommender systems and large language models,” according to Huang.

It’s well-suited for these data-heavy applications because of how the Grace Hopper brings together a Grace CPU and a Hopper GPU onto one module and connects them with a high-speed interface. This interface is “seven times faster than PCI Express,” Huang said, which means the superchip is better suited for these applications than a discrete CPU and discrete GPU that communicate over PCIe.

“This is Nvidia’s inference platform, one architecture for diverse AI workloads and maximum data center acceleration and elasticity,” he said.