VMware published results of a commissioned benchmarking study last week as evidence that its public cloud Infrastructure-as-a-Service is better than Amazon Web Services and Microsoft Azure from a price-performance standpoint.
Durham, N.C.-based market research firm Principled Technologies, which conducted the study, found that VMware's vCloud Hybrid Service offers twice the CPU power of Microsoft Azure and three times the storage I/O performance of AWS.
VMware, armed with the benchmarking results, said AWS and Microsoft customers would have to pay significantly more to get the same levels of storage and compute power that's available on vCHS.
The study has generated lots of debate in the cloud industry. Some experts have questioned whether Principled Technologies made an accurate like-for-like comparison between the different clouds, while others have dismissed the study as a VMware marketing stunt.
One VMware partner told CRN he doesn't put much stock in the study because VMware's comparisons don't reflect the way AWS and Microsoft are positioning their public cloud IaaS offerings.
"VMware built their cloud to make old legacy workloads and architectures scream. Amazon -- and likely Microsoft -- would say you are an idiot to transplant old applications into the cloud. Instead, you should rethink your app architecture and design it to take advantage of what cloud has to offer," said the partner, who asked not to be named to avoid damaging his relationship with the vendor.
Mathew Lodge, vice president of cloud services at VMware, said last week that a virtual machine running on vCloud Hybrid Service, with 140GB of storage, was able to maintain a minimum of 3,300 IOPS during the benchmarking. This, he said, is three times what AWS can offer in terms of storage performance.
However, the VMware partner told CRN he doesn't consider this a meaningful comparison because AWS customers don't use its public cloud this way.
"Amazon's take would be that a customer would never need 3300 IOPS out of a single VM, because you should design your application to be distributed across multiple data centers and geographies," the partner said.
AWS has also shown it can scale its cloud to handle huge workloads, such as Netflix streaming, and VMware's cloud doesn't yet have this kind of marquee use case, the partner said.
"There's no way you are doing that in vCHS, because VMware's monolithic architecture wasn’t built to scale out in that manner," the partner said.
NEXT: Why Pricing Is Such A Big Deal In The Cloud Market