National Center for Computational Sciences

Jaguar to Host Next-Generation Climate Models

NOAA collaboration continues NCCS climate work

The National Center for Computational Sciences (NCCS) at Oak Ridge National Laboratory (ORNL) will provide the resources to research advanced climate models and simulations for the National Oceanic and Atmospheric Administration (NOAA).

The collaboration is the result of a three year memorandum of understanding between the Department of Energy’s (DOE) Office of Science and NOAA which will make available more than 10 million hours on three of the DOE’s top supercomputers to study climate change. The NCCS will provide its Cray XT4 known as Jaguar, further boosting its profile as a premier system for climate simulation. Besides exploring climate models researchers will perform weather prediction research on the DOE machines.

“This is a great step forward in further developing the relationship between DOE and NOAA in the important area of climate research,” said NCCS Director James Hack. “It provides a solid foundation for the discussions ORNL has been having with NOAA aimed at cross agency partnerships to better coordinate our respective climate science efforts.”

The Community Climate Systems Model, developed by the DOE and National Science Foundation, is the other major global climate modeling effort supported by the NCCS. The presence of both model development efforts at ORNL will provide opportunities for the respective activities to leverage experience on the laboratory’s new petascale platforms.

The NCCS has a strong history of climate research, as its Cheetah system was responsible for 40 percent of the data used in the United Nations’ Intergovernmental Panel on Climate Change report that shared a Nobel Peace Prize with former Vice President Al Gore.

Jaguar Poised to Pounce On Petascale

First cabinets arrive at ORNL

The National Center for Computational Sciences (NCCS) at Oak Ridge National Laboratory (ORNL) has begun receiving the first cabinets of its upcoming Cray XT5 petascale supercomputer, a system that will soon be among the most powerful in the world.

Named Jaguar, the new machine will feature liquid-cooled cabinets and quad-core AMD OpteronTM 2.3 gigahertz processors and will be housed in ORNL’s Computational Sciences Building, the sight of the laboratory’s current 263-teraflop Cray XT4. The cabinets for the new machine will use a Cray-designed R134-a refrigerant and chilled water cooling system to sustain the estimated peak power demand of more than 6.5MW. Introducing the petascale to open science has required substantial upgrades to the NCCS infrastructure, but the effort will no doubt be worth it.

Jaguar will continue the NCCS’s leadership research in numerous scientific arenas such as climate modeling, astrophysics, and fusion energy. While the current terascale system is a computational giant in its own right, the new Jaguar will be nearly four times as powerful, enabling a new era in simulation science.

For example, petascale simulations of high-temperature superconductors will explain the differences in transition temperatures between superconducting materials, greatly increasing the capacity for electronic storage; climate scientists will have the ability to better integrate models for the global ocean, sea ice, land, and atmosphere, better preparing policymakers to deal with the ramifications of climate change; and fusion scientists are working out details of the 100-million-degree (centigrade) ITER reactor, increasing our understanding of such issues as ion and electron turbulence and helping us to realize a secure energy future.

Other scientific disciplines that stand to benefit include astrophysics, numerous areas of chemistry and biology, and computer science.

The new Jaguar will be the only open science petascale system in the world when it comes online in 2009, allowing the scientific community to address the world’s most pressing dilemmas through collaborative research and information sharing. The arrival of the first Jaguar cabinets represents a significant milestone in the achievement of this goal.

The National Institute for Computational Sciences (NICS), likewise located at ORNL, is also due to bring a petascale system online in 2009. With two petascale machines under one roof ORNL will be the world’s most powerful computing complex and the epicenter of scientific progress via simulation.

Storage System Tops 3 Petabytes

HPSS storage on track to double again in the coming year

Oak Ridge National Laboratory’s High-Performance Storage System (HPSS) reached a significant milestone recently, surpassing 3 petabytes of storage.

That amount, enough to hold the contents of the Library of Congress 300 times over, is double the system’s storage this time last year. According to Stan White of the National Center for Computational Sciences (NCCS), the system’s storage is on track to double again in the coming year.

“The curve has been quite impressive,” White noted.

The decade-old HPSS is ORNL’s principal archival storage system, holding simulation data for both of ORNL’s major supercomputing centers: the NCCS and the National Institute for Computational Sciences (NICS). The storage is located in four silos in two locations. White noted that the system is due to receive 16 tape drives that can read tapes holding as much as 1 terabyte of data uncompressed.

NCCS Staffers Lead Technology Development Efforts

Shipman and Graham announced as new group leads

Two members of the NCCS Technology Integration Group are moving up to take formal leadership roles as the center pushes the limits of computing technology.

Galen Shipman will take over as group leader, stepping in for Shane Canon, who recently left the group to return to Lawrence Berkeley National Laboratory. Rich Graham will step in as leader of the newly created Application Productivity Software Group, working to develop technologies that allow researchers to take full advantage of the center’s leadership computers.

According to NCCS Director Jim Hack, the two are now in a position to make the most of their unique talents. “Each one of these guys is in a role that draws on their professional strengths, the most ideal matches for what we need in the way of leadership for those activities,” Hack said.

Shipman came to the NCCS in September 2007 from Los Alamos National Laboratory (LANL), where he was acting team leader in the Network Research Group within the laboratory’s Advanced Computing Laboratory (ACL). As leader of the Technology Integration Group, he will lead efforts to bring the next-generation Lustre file system, known as Spider, to the center.

“Galen is going to be an absolutely fabulous person for bringing the Spider file system to full operational status,” Hack said. “He has stepped in after Shane’s departure and has helped to ensure that the Spider file system project stayed on schedule. I’m looking forward to working with him very much.”

An early version of Spider has already been implemented, and the full version will be available later in the year in conjunction with the upcoming NCCS petascale system. In the longer term, Shipman and his group will work toward a common networking and file system infrastructure for the center that will eliminate the need to provide these services for each new system.

Graham also came to the NCCS from LANL, where he was acting group leader of the ACL. Among his responsibilities, Graham is coordinating a community-wide effort to update the decade-old Message Passing Interface standard for highly parallel computing.

The Application Productivity Software Group will focus on developing advanced software tools such as debuggers, performance tools, development environments, and middleware such as schedulers, communication libraries, and input/output libraries. It was created in order to focus efforts on tool development at the NCCS.

“We realized that we might be missing an opportunity if we didn’t take a look at Rich’s unique strengths in tools development,” Hack said. “So in consultation with the directorate we decided to create a group that would be charged with that mission.”

HPC Users Meet to Share Petascale Science Initiatives

2008 Fall Creek Falls Conference a success

“Delivering Science on Petascale Computers” was the theme of the 4th annual Fall Creek Falls Conference, held Sept. 7-10 at Montgomery Bell State Park, near Dickson, Tennessee, west of Nashville. The conference brings together experts from universities, national laboratories, and industry, to address the challenges of modeling and simulation, as high performance computing (HPC) has moved from terascale to petascale—a quadrillion floating point operations per second.

More than 100 participants at this year’s conference made presentations and participated in panels organized around climate change, including coupled high resolution modeling of the Earth System Project, electrical energy storage (supercapacitors), the biosciences, data analytics, quantum Monte Carlo calculations at the petascale, applications in biophysics, theoretical and computational nanoscience, and computational astrophysics.

There was a special session on the hardware and software of the emerging petascale systems, including critical data warehousing issues, as the systems generate an avalanche of new data. Scientific visualization tools were also discussed. An update on the race to petascale was presented by Arthur (Buddy) Bland of ORNL’s Leadership Computing Facility, Ray Bair of Argonne National Laboratory, and Andy White of Los Alamos National Laboratory.

A late afternoon poster session illustrated new research that HPC at petascale will make a reality: depth map building beyond the scanning electron microscope using a low-cost Graphics Processing Unit (GPU); new system software solutions for providing reliability and resiliency for extreme scale HPC systems; integrating a broad range of modeling tools to develop the structure/property relationships for nanostructured materials; and new codes to eliminate data bottlenecks for problems in matrix algebra.

The Fall Creek Falls Conference, hosted by the Computing and Computational Sciences Directorate (CCSD) at Oak Ridge National Laboratory ORNL, was first launched in 2004 as an opportunity for experts from its National Center for Computational Sciences (NCCS), university partners, NCCS researcher-users, other centers, industry and government to discuss how best to use the new computing resources provided by the DOE Office of Science.