Linux Clustering Evolves High-Performance Computing


Linux clustering is widely regarded as the third stage of evolution in high-performance computing.

The dawn of high-performance computing came in the 1970s with the development of the Cray 1 and other custom-built supercomputers running proprietary operating systems. The early 1990s saw the use of semi-custom microprocessor supercomputers, running Unix on clustered RISC platforms. A milestone was reached in 1997, when the TOP500 list, a biannual listing of the world's most powerful supercomputers, first included cluster systems. Clusters are now in the top 30, and in the next TOP500 list,due out next month,clusters are expected to dominate the top 10.

Clusters running the open-source Linux OS offer a wide range of benefits over traditional supercomputers. Besides sporting a lower purchase price, Linux clusters are often less expensive to operate, said Dr. Mark Seager, assistant department head for terascale systems at the Lawrence Livermore National Laboratory in Livermore, Calif., which purchased a 1,152-server cluster from Sandy, Utah-based integrator Linux NetworX. Seager estimated that the cost to power and cool the cluster will be half that of a comparable supercomputer.

Clusters also are often quicker and easier to set up, said Steve Hill, CEO of Linux NetworX. Hill expects the Livermore cluster to be deployed and fully optimized within six months, compared with the two years typically required for traditional supercomputers.

Other advantages of Linux clusters include high stability, reliability, redundancy and availability, solution providers said. What's more, clusters offer high scalability, since end users can buy a small cluster and add additional processors and computing power as needed.

Linux is clearly the OS of choice for clusters over Microsoft Windows. Last year, Windows clusters represent just $30 million of the estimated $508 million high-performance computing cluster market this year, compared with $225 million for Linux and $240 million for Unix, according to research firm IDC. Industry analysts predict that Linux will continue to dominate Windows in high-performance computing clusters because of its lower price, increasing roster of high-performance computing Linux software and the fact that most high-performance computing applications run on Unix and, therefore, can easily be transferred to Linux. Analysts also say Linux cluster users appreciate the fact that they're not locked in to a single vendor.

Linux clusters will quickly replace traditional supercomputers in nearly every high-performance computing area, according to industry observers. Yet the market is continuing to evolve, and eventually grid computing will become the fourth stage of high-performance computing technology, they say.