Shazam! Oak Ridge Supercomputer Now World's Fastest For Open Science

With an increase of computing power to a peak 1.64 petaflops or quadrillion mathematical calculations per second, the Cray XT Jaguar supercomputer at the Department of Energy's Tennessee-based Oak Ridge National Laboratory is now the world's first petaflop system dedicated to open research.

The Jaguar system now has an 84-cabinet quad-core Cray XT4 system and 200 new Cray XT5 cabinets, which also use quad-core processors. Both parts of the system have 2 GB of memory per core, which provide users with a total of 362 TB of high-speed memory in the combined system.

The two systems will be combined by linking each to the Scalable I/O Network (SION), which links the systems together and to the Spider file system. The XT5 system also has 214 service and I/O nodes, which provide up to 240 GBps of bandwidth to SION and 200 Gbps to external networks. Additionally, the XT4 has 116 service and I/O nodes providing 44 GBps of bandwidth to SION and 100 Gbps to external networks.

Both the XT4 and XT5 boards have four nodes. There is a single Advanced Micro Devices quad-core Opteron 1354 "Budapest" processor coupled with 8 GB of DDR2-800 memory on the XT4 nodes. The XT5 is a double density version of the XT4 with twice the processing power, memory and memory bandwidth on each node. The XT5 node also has two Opteron 2356 "Barcelona" processors linked with dual HyperTransport connections.

Each node runs Cray's version of the SuSE Linux operating system, and Cray has tuned the Linux kernel so that it removes unnecessary services from the compute nodes. This means that the operating system minimizes interruptions to the application codes running on the system. The SuSE Linux operating system on the nodes combines with the system services, networking software, communications, I/O and mathematical libraries, as well as compilers, debuggers and performance tools to form the Cray Linux Environment.

In addition, the NCCS core LAN network consists of two Cisco 6500 series routers along with a Force10 E1200 router. The core network provides more than 100 10GE ports for intraswitch connections.

"Jaguar is one of science's newest and most formidable tools for advancement in science and engineering," said Dr. Raymond Orbach, DOE's Undersecretary for Science, said in a statement. "It will enable researchers to simulate physical processes on a scale never seen before and approach convergence for dynamical processes never thought possible. High-end computation will become the critical third pillar for scientific discovery, along with experiment and theory."