Arm Exec: New AGI CPU Has Big On-Prem Potential—But Limited Channel Play For Now

With its first silicon product, Arm is targeting new data centers powering agentic AI workloads, which Mohamed Awad says require a new class of ‘ruthlessly optimized’ CPU that is ‘super-efficient but highly performant’ and can ‘scale in a meaningful way.’

The executive behind Arm’s push to sell its first silicon product into data centers said he sees a big opportunity to push its new AGI CPU for on-premises deployments with the thousands of customers who have embraced Arm in the cloud.

But Mohamed Awad, who is executive vice president of Arm’s cloud AI business unit, said in an interview with CRN that the British chip designer doesn’t see a big play for solution providers yet, even as it signs up major OEMs like Lenovo and Supermicro.

“I think with the fullness of time, that’s certainly an area of opportunity. Near term, our focus is really on the big hits,” he said in late March, referring to large and flashy customers like Meta, which it has named as a lead partner and co-developer for the AGI CPU.

“We’re much more interested in deep, meaningful partner relationships than we are in a broad, boil-the-ocean type strategy,” added Awad.

This strategy makes sense from Arm’s perspective as well as for solution providers, according to Alexey Stolyar, who is CTO of International Computing Concepts, a Northbrook, Ill.-based systems integrator that partners with Intel, AMD and Nvidia.

While Stolyar said he likes to offer cutting-edge solutions, his company’s limited resources mean that his team needs to prioritize products that customers are asking for, especially given the extra work Arm-based solutions may require at first.

“If I don’t know how to sell it, and customers don’t want it, I’m never going to sell it, right? So if there’s an interest from customers, we’re going to go and figure out how to get the resources because there’s opportunity,” said the systems integration executive, whose company ranked No. 1 on CRN’s Fast Growth 150 last year.

Arm Seeks To Reap From Agentic AI Opportunity

Announced last month, the AGI CPU represents a significant shakeup to the company’s 36-year-old business model. This has mainly consisted of Arm licensing chip designs and instruction set architectures to a wide variety of companies developing their own processors, such as Amazon Web Services with its Arm-based Graviton CPUs.

With its first silicon product, Arm is targeting new data centers powering agentic AI workloads, which Awad said require a new class of “ruthlessly optimized” CPU that is “super-efficient but highly performant” and can “scale in a meaningful way.”

“What you’ll find is that many of the offerings out there will give you one or two of those, but not all three. And what Arm AGI CPU does is it brings all three of those together,” he said.

The AGI CPU packs up to 136 cores based on Arm’s Neoverse V3 architecture with what it said is leading performance measured by the core, system-on-chip, server blade and rack. These cores are packed within a 300-watt thermal envelope. This allows it to scale to more than 45,000 cores in a liquid-cooled rack or over 8,000 in an air-cooled rack, Arm said.

The company claimed that the CPU can deliver more than double the rack performance of x86 processors, enabling capital expenditure savings that could reach up to $10 billion per gigawatt of AI data center capacity.

“We wanted it to scale to a high number of potential cores/threads, but we wanted to do so without sacrificing performance. We wanted incredible efficiency, so we put the whole thing in a 300-watt package. That’s very different than what I would suggest legacy CPUs tend to do,” Awad said, referring to x86 CPUs that are now reaching 500 watts.

Why Arm Sees The Need For A New Class Of CPU

Arm thinks there’s a significant opportunity for the AGI CPU to because of its expectation that agentic AI workloads will require data centers to possess more than four times greater CPU capacity for every gigawatt.

Whereas generative AI routed queries through CPUs and then GPUs to create a response that would get passed back through the CPUs, agentic AI makes the motions through a data center “much more sophisticated,” according to Awad.

“What’s happening is that the user is out of the loop, meaning you as a user might spawn off a much more sophisticated task that is flowing up through the CPUs, but now there’s a whole new class of CPUs there, and that new class of CPUs is now spawning off multiple, different threads to multiple, different GPUs,” he said.

These CPUs are then retrieving tokens, “managing those responses from those tokens” and “deciding which models to then go call to get the next-level answer,” Awad added. “Or it’s even becoming recursive in that it’s calling itself again to go do another set of tasks, etc.”

He said this activity will, in turn, “put a lot of pressure on the databases, the storage, the networking, the rest of the system, which is all managed by CPUs as well.”

It’s this line of thinking that is leading Arm to believe AI data centers will need quadruple the CPU capacity, but Awad said he considered that a “conservative estimate.”

“That’s where the idea of, hey, if you could get 2x the performance in the same power footprint, that becomes super compelling,” he said.

Arm Attracts SAP, F5 As It Chases After SaaS, Appliance Vendors

While Meta is cited as the AGI CPU’s largest and most influential customer at launch, Arm has announced “commercial momentum” with several other companies, including ChatGPT maker OpenAI as well as AI chip designers like Cerebras and Rebellions.

Companies like these represent customers who are developing their own AI accelerator chips and need to pair them with a CPU for the host node, according to Awad.

Another group of customers sought by Arm for AGI CPU adoption are software-as-a-service companies. Awad cited German ERP vendor SAP as a company in this category that is a “big customer” of AWS’ Graviton CPU instances but in some cases requires “that same level of efficiency and performance for an on-prem hybrid installation.”

“There is no Arm-based offering that gives them that, so AGI CPU solves that problem for them,” said the executive.

A third customer group targeted by Arm is what Awad called appliance vendors, which includes networking OEMs and other kinds of hardware vendors that “have always used Arm on the lower end of their SKU map” and could benefit the AGI CPU’s benefits. The executive cited application delivery and security F5 as part of this group.

“They’re going to use it at that end as well. And what that allows them to do is unify their software code base and get the performance in the efficiency,” he said.

Arm Seeks To Win Over 10,000 Who Embrace Arm In The Cloud

At a broader level, Awad said Arm sees an opportunity to win over the 10,000 companies that are “using Arm in the cloud every day for their real production work,” whether that is with AWS’ Graviton CPUs, Google Cloud’s Axion processors or other hyperscaler offerings.

These opportunities would represent new deployments of on-prem data centers.

“I think every one of those who is also running a hybrid environment wants that same level of [operational expenditure] savings. They want that same level of performance-per-watt benefit. They want that same efficiency, but they want it on-prem to complement their cloud-based usage, and AGI CPU gives them that,” he said.

This means that Arm sees enterprises, midmarket firms and even SMBs as “potential targets,” according to the executive. But he cautioned that the company is “certainly not chasing every one of those today.”

For Channel Play, Arm Will Look At What Customers Want

When it comes to the channel opportunity for Arm’s AGI CPU, Awad said the company’s engagement with solution providers is limited right now.

While Awad recognizes the channel as an opportunity in the long run, he said the company’s focus on “deep, meaningful partnerships” over a broader push with solution providers is part of a go-to-market strategy he called “ruthlessly optimized.”

“We’ll continue to expand that out with time,” he said.

This means that Arm will look at preferences by customers first to determine what, if any, need there is for support from the channel, according to Awad.

“We’re really more focused on who are the end customers that we want to go engage, and then bringing them into the fold, and then figuring out who the right partners are to support them, as opposed to the other way around,” he said.

Arm’s Challenge With A Broad Channel Play

Even then, the executive said he has concerns about how a big expansion into the channel could dilute Arm’s core product because of how the AGI CPU has been designed not as a general-purpose CPU but is rather “broadly applicable within the AI data center.”

In focusing on performance, efficiency and scale for agentic AI workloads with the AGI CPU, Arm eschewed certain elements found in traditional x86 processors, like multithreading or legacy software compatibility, with Awad joking that it won’t run Lotus Notes.

That could make it challenging for Arm to sell the product through a broad group of channel partners whose customers are accustomed to a wider set of features for traditional IT needs, according to the executive.

“The small [or] medium business is going to need a lot more built into the silicon in order for them to be able to digest it,” he said. “And that doesn’t come without overhead: overhead in terms of power, overhead in terms of cost, overhead in terms of silicon area, and frankly, the big guys don’t necessarily want to pay for all that.”