National Center for Computational Sciences

Learning curve enhanced by first-hand approach and knowledgeable support

A new generation of scientists and engineers is training this summer at the National Center for Computational Sciences (NCCS) in the use and care of high performance computing and its impact on scientific discovery.

“This is a great way for someone to learn supercomputing,” says Arnold Tharrington, a computational scientist in the Scientific Computing Group at NCCS. “How many students can say they’ve worked on a supercomputer?”

The NCCS, located at the Department of Energy’s (DOE) Oak Ridge National Laboratory (ORNL), has 12 interns this summer with backgrounds including computer science, physics, and mechanical and chemical engineering. Each has been assigned a mentor and a project that provides hands-on experience with the lightning fast hardware of the center’s Cray XT4/XT5 supercomputer. Jaguar has a theoretical peak computing speed of 1.6 petaflops, or a quadrillion floating point operations (calculations) per second.

Nathan Livesey, an entering freshman in chemical engineering at Tennessee Tech University, works on cooling the machines as they crunch scientific data.“They use a lot of energy and create a lot of heat,” says Livesey, describing a complex cooling system that maintains computer-friendly temperatures.

This method of temperature control is similar to that of other computers, but on a much larger scale. Hot spots are generated in different areas of the room, depending on which processors are running. Cooling is provided via pipelines, which pump cold water from a chiller room into air control handlers in the computer room. There are 23 handlers available to blow cold air into the floor, where perforated tiles near the computer cabinets allow the air to escape and cool the machines. Livesey must adjust the workload of these air handlers to reach optimal cooling capacity and ensure practical energy consumption.

Efficiency is key. When the air handlers work in this manner, they use less power and chilled water, saving thousands of dollars per month in electricity.

Efficiency is also vital in the case of molecular dynamics (MD), a form of computer simulation which has many scientific research applications. MD codes model the motions and behaviors of atoms that otherwise could not be observed, such as how proteins function inside the body. Different codes exist—some better suited for certain simulations—but all model atomic interactions far too complex for modern instruments to measure. The results of these experiments can be applied to any number of areas, including biology, nanomaterials science, turbulence, energy storage research, or the geosciences, to name a few. Justin Vaughner, a senior in mechanical engineering at Alabama A&M, is learning to test these codes to discover which ones run most efficiently on Jaguar.

The Jaguar XT5 is capable of running simulations on over 150,000 processing cores, which creates challenges for any code.

“All kinds of issues pop up when you work with such a large scale, “explains Arnold Tharrington, Vaughner’s mentor. Multiple processors carrying out calculations simultaneously, known as parallel processing, is the source of Jaguar’s impressive processing speed. The cores work by communicating with one another. Yet more cores mean more speed until a point, where they begin to interfere with one another. Striking a balance involves extensive trial and error.

Vaughner tests MD codes, beginning with only 128 or 256 cores, and works towards eventually running tens of thousands of processors smoothly and simultaneously. Modifying these codes to employ as many of Jaguar’s cores as possible is vital in equipping scientists with the best tools for future experiments.

Education for future generations

While Vaughner and Tharrington work out the kinks of the molecular dynamics codes, Jesse Mays and Stephanie Poole, computer science and engineering majors, respectively, are developing educational models that can teach computational science to students from kindergarten up to the postdoctoral level.

Mays and Poole are currently working on a project which is the result of a partnership between NCCS and the Appalachian Regional Commission (ARC), a body of educators and business people whose goal is to support the educational development of the Appalachian region. The project seeks to open up college and career options to underprivileged and minority high school students, explains Bobby Whitten, a member of NCCS’s User and Assistance Outreach Group and mentor to Mays and Poole.

The partnership between NCCS and ARC, now in its second year, accepts 26 high school students and 26 college and high school instructors annually from a 205,000 acre stretch of land covering 15 states. The two week workshop, which ran this year from July 11 to 24, engaged the students and instructors with activities and projects in areas such as biology, astronomy, computational science, and chemistry. With the help of Mays, Poole, and their mentors, six of these students learned the basics of hardware, software, and programming and how they are applied in a high performance climate. They also received a crash course in parallel processing.

“Take a complex problem—if you ran it on one computer, it would take hours,” explains Mays, a senior at Morehouse College in Atlanta. “The idea is to run it on multiple computers to get it done faster.” In this way, problems which once might take a lifetime or more to solve, can today be done on Jaguar in days or even hours.

As a demonstration, the students connected six Mac Minis together to effectively create a mini supercomputer. They then were able to calculate pi at a speed of 19.5 gigaflops, or 19.5 billion calculations per second, meaning these six interconnected Mac Minis would be the fastest supercomputer in the world in the summer of 1989.

The goal of the ARC project is to give interested high school students the knowledge and first-hand experience needed to consider a career in the sciences or computer technology, or at least to become an informed observer of today’s advances in the field.

And those involved behind the scenes gain as well.

“It’s filled in the gaps with my programming,” says Poole, who will begin studying computer engineering this fall at Pellissippi State Technical Community College. Her involvement with NCCS has been useful in bridging the gap between a high school and college education. “I have a deeper understanding of computers than I did before.”

— by Wes Wade

Wes Wade is a science writing intern with the National Center for Computational Sciences.