Moving Beyond 10 Gigabit Ethernet: A Vexing Transition

The transition that is currently underway involves the shift from 1 gigabit Ethernet to 10 gigabit Ethernet (GbE) switching. A third of all servers now use the faster devices, but customers are still choosing from a limited choice of vendors, five years after 10-GbE products first went on sale. That's because five of the 10 largest switch makers don't yet offer 48-port single rack unit 10G Base-T products, the configuration most enterprises prefer. Many customers have not yet finalized their 10-GbE migration plans, having settled on neither a timetable nor a form factor for their purchases.

We expect many of these issues will be resolved by the end of 2013. But because of the insatiable appetite for faster networks, as soon as the 10-GbE transition is in its final stages, the industry will be grappling with the move to 40-GbE units. And that, we predict, will be the most critical transition in its history.

What makes the 10-GbE to 40-GbE shift so vexing? Largely the changing nature of enterprise IT, which, from the perspective of vendors, is dramatically shrinking the available market size. Many enterprise customers are shifting toward cloud and software-as-a-service (SaaS) architectures, in effect outsourcing their data center rather than building it themselves. In fact, over the next three years, many businesses will literally be buying their last server and their last switch in the data center; companies will have completed the shift away from premise-based data center IT solutions once they've outlived their usefulness. One of the results of this new IT approach will be a decline in overall capital spending, along with an unprecedented concentration of purchasing power in the hands of a relatively small number of cloud customers. These trends are immediately apparent in Dell'Oro Group projections for the data center Ethernet switch market.

For example, in 2012, major cloud customers such as Google, Facebook, Amazon, Rackspace and Verizon Terremark accounted for less than a quarter of the industry's $7.5 billion in sales. We project the overall market will potentially grow by 21 percent, to $12 billion, by 2017. But, most of that growth will be provided by a concentrated few cloud customers. Remarkably, total annual enterprise spending on switches will be lower in five years than it is today, even though the market will have substantially increased.

In addition to these obvious marketing hurdles, there is also a technical challenge confronting switch vendors as they contemplate 40 GbE.

Already, they are struggling to support two platforms: 1 GbE and 10 GbE. But the 40-GbE rollout will mean adding 40-GbE product lines to their SKU lists, further taxing engineering resources that are already under stress. Making matters worse is the fact that there is not yet an ISO standard for a small enough 40-GbE port form factor. The current ISO standard for 40-GbE ports, called Quad (4-channel) Small Form-factor Pluggable, or QSFP, does not have a small enough size to be effective in server access.

And, hovering over all these issues is the emergence of software-defined networks, which is expected to not only change the architecture of enterprise data centers but also result in fewer switches and more controllers -- another challenge to switch vendors. Much has been written about SDN, and its exact impact remains uncertain. But at a minimum, we expect the move to SDN to further strengthen the hand of big cloud customers, which, because of their high-profit margins, often have more hardware and software design resources at their disposal than all but the biggest traditional vendors.

NEXT: Increasing Pressure

The actual transition to 40-GbE switches will likely occur in three main waves. Each wave will increase the pressure on vendors to either find new customers outside of the big cloud providers or else exit the data center switching market altogether.

The first wave, already underway, involves a small number of the largest cloud providers building their own customized "white box" 40-GbE switches using merchant silicon. Because the bulky size of current QSFP ports requires that 48-port units be either two rack units high or an end-of-row chassis -- both undesirable for cost and space reasons -- we believe major cloud data centers are using proprietary port designs for their current 40-GbE devices. They are often also developing their own SDN protocols, based on the unique requirements of their particular installations.

Much the same thing happened six years back when major cloud data centers first began making the transition from 1 GbE to 10 GbE. Back then, they had to implement their own SFP+ direct attach copper solutions to reach the densities and price points necessary to justify the new breed of faster switches.

The extent to which the big cloud companies are making their own switches in-house, rather than relying on a traditional vendor, may surprise many people. For example, one large cloud provider is known to have a yearly 10-GbE run rate that's bigger than that of every vendor in the market, save Cisco, meaning they make more top-of-rack 10-GbE switch ports than even Dell.

Large cloud companies like this approach not only for its economics but also because it allows them to customize their switches with the features they need, which often results in performance advantages relative to their competitors.

The second wave in the 10-GbE to 40-GbE migration will occur when the industry standardizes on a 40-GbE port form factor that will allow at least 48 server-facing ports in 1 RU. We believe some of these 40-GbE ports will be used for server access and others for uplinks, similar to what occurred in the early days of 10 GbE.

In the third wave, vendors that are shipping 40-GbE servers will begin to include uplink speeds as fast as 100-GbE, power and density permitting. We expect some vendors to do so via standard modules, while others will look for alternative technologies not yet fully available in the enterprise market, such as embedded optics and photonics.

So what does all this mean for vendors and solution providers? Two things. First, vendors need to understand that every one of their customers will be asking themselves the same question every time they launch a new application: "Should I buy new hardware to run this, or should I look to cloud or SaaS alternatives?" Of course, many enterprise customers will be choosing the latter, accelerating the switch market's consolidation.

Second, Ethernet switch vendors that want to continue offering data center solutions have a long road ahead of themselves to remain competitive. They're going to need to make fundamental changes in both their product design and their marketing, finding ways to differentiate themselves from the often-compelling proposition of the cloud or SaaS approach. Doing so will involve both engineering imagination and expert business execution, but we believe the best networking companies are up for those challenges.

Alan Weckel is vice president, enterprise and data center market research at Dell'Oro Group, which focuses on the networking and telecommunications industries.

PUBLISHED JUNE 19, 2013