National Center for Computational Sciences

Quantum Spin Doctors Dissect Exotic States of Matter

Jaguar sheds light on Bose glass

When German physicist Max Planck created quantum theory in 1900, he was not trying to revolutionize the world. He was just trying to provide a theoretical foundation for the way a heated object radiates energy and thereby to improve the efficiency of light bulbs.

Modern scientists, too, cannot know which research will change the world, but history suggests the next scientific revolution will focus on complex systems rather than isolated bits of matter. If so, it may stem from computer simulations such as those performed by Tommaso Roscilde of France’s École Normale Supérieure de Lyon and recently of Germany’s Max-Planck Institute. A team led by Roscilde is using Oak Ridge National Laboratory’s (ORNL’s) Cray XT4 Jaguar supercomputer to explore the quantum mechanical phenomena that give us superconductors and superfluids.

Roscilde and his teammates—Stephan Haas of the University of Southern California and Rong Yu of ORNL—are using Jaguar through the Department of Energy’s INCITE (Innovative and Novel Computational Impact on Theory and Experiment) program. A grant for 800,000 processor hours in 2007 allowed the team to simulate a lattice of atoms in a quantum magnet to examine two extraordinary quantum phases, or states of matter.

In the first, called Bose-Einstein condensation, atoms throughout the material occupy the same state, with the same momentum, range of probable locations, and spin. In a quantum system, this is the closest they can get to being in the same place at the same time. By introducing impurities into the material, the team is also able to create the second phase, known as Bose glass. In a Bose glass, the impurities force the condensation into separate islands throughout the lattice, with atoms sharing the same state only with other nearby atoms.

Roscilde’s team is using a technique called Quantum Monte Carlo to simulate disorder in a quantum magnet and thereby create Bose glass. The team hopes its efforts will allow its collaborators—Vivien Zapf and Marcelo Jaime at the National High Magnetic Field Laboratory at Los Alamos National Laboratory—to perform the first experimental confirmation of Bose glass.

The work is at the cutting edge of condensed-matter science.

“I find that in this particular instance of a study of a solid-state system, you’re really trying to tailor matter to a level of control that was unthinkable a few decades ago or even a few years ago,” Roscilde said. “What you have is a system where, in principle, you can tune the system among completely exotic phases that have no analog in classical systems.”

Roscilde, like Planck a century before, cannot say whether his work will produce benefits beyond a deeper understanding of the universe. He noted also that a device based on quantum systems would likely be very different from existing technologies.

“You have to think hard what to make out of these systems,” he said. “It’s not just that once you know them, you know what to make out of them. You have to totally think of new functionalities.”

NICS Unleashes “Kraken” Supercomputer

New system prepares for transformational science

The National Institute for Computational Sciences (NICS) is the newest member of an elite supercomputing community. Dedicated on April 3, the organization—formed through a National Science Foundation (NSF) grant to the University of Tennessee (UT) and its partners—is on its way to delivering a soon-to-be petascale system that promises substantial contributions in the effort to solve the world’s greatest scientific challenges, such as understanding the fundamentals of matter and unlocking the secrets to the origin of our universe.

The system, a Cray XT4 dubbed Kraken (after a gargantuan sea creature in Norse mythology), will come online in midsummer and is expected to feature more than 18,000 2.3 GHz AMD high-performance cores delivering 170 teraflops of performance. A new Cray-designed interconnect, featuring Cray SeaStar2 chips and high-speed links, will greatly increase reliability and provide for excellent scaling while eliminating the related cost and complications of external switches.

NICS is seeking “large, tightly coupled applications” to take advantage of the newly designed Cray interconnect, said NICS Project Director Phil Andrews. Currently a dozen large-scale applications are poised to run at NICS, spanning a diverse range of scientific fields including climate, fusion energy, biology, lattice QCD, and astrophysics. “ENZO cosmology simulations exhibit near-ideal scaling to 8,000 cores on the XT4,” said Michael Norman, a professor of physics at the University of California–San Diego. “Clearly even larger simulations are possible. This opens up all kinds of new frontiers in understanding cosmic evolution.”

Climate also figures to play a large role in Kraken’s research potential. As climate change continues to gain prominence both in the policy and scientific arenas, powerful systems such as Kraken will be increasingly important in all types of climate simulations, from CO2 cycles to the role of ocean currents. Just as previous efforts in eastern Tennessee substantially to the recent Nobel Prize given to the United Nations’ Intergovernmental Panel on Climate Change, Kraken also will greatly contribute to man’s understanding of his impact on the planet.

The Cray XT4 will ultimately evolve into a Baker system featuring more than 10,000 compute sockets; 100 trillion bytes of memory; and 2,300 trillion bytes of disk space. It will provide more than 700 million CPU hours per year and 1 petaflop of performance, making it the nation’s most powerful academic supercomputer.

Kraken is specifically designed for sustained application performance, scalability, and reliability and will incorporate key elements of the Cray Cascade system to prepare the user community for highly productive petascale science and engineering. The Cray XT4 will continue to operate in support of users until the Baker system is in full production.

The system and the related NICS organization are the result of an NSF Track II award of $65 million to UT and its partners to provide for next-generation high-performance computing. The award was won in an open competition among high-performance computing (HPC) resource institutions vying to facilitate America’s continued competitiveness through the next generation of supercomputers.

The NSF used a series of system-performance-related benchmarks as a key factor in the selection process, setting the stage for the future of simulation research by employing a system that is usable and reliable and well-suited to computationally-intensive scientific issues, such as protein shape and function as well as climate modeling.

As the foundation for NICS (a collaboration of universities, research institutions, and HPC industry leaders), the new system will be fully linked to the NSF-supported TeraGrid, a network of supercomputers across the country that is the world’s largest computational platform for open scientific research.

The NSF award places UT among a select group of supercomputing facilities, including the University of Illinois at Urbana-Champaign and the Texas Advanced Computing Center, likewise an NSF-funded facility. Due to the collaborative relationship between UT and ORNL, NICS promises to deliver state-of-the-art scientific research.

“Combined with the more traditional approaches of theory and experiment, scientific computation is a profound tool for insight and solution, as researchers move their problems for modeling and simulation from existing terascale systems to petascale systems later this year and onward to exascale [quintillion calculations per second] systems in the next decade,” stated Thomas Zacharia, vice president for science and technology at UT and the associate lab director for computing and computational sciences at ORNL.

NICS offers researchers a great opportunity to begin to port and scale code on a system that will ultimately move to the petascale. For more information, visit the NICS website at www.nics.tennessee.edu.

Weigand Receives DOE’s First Schlesinger Award

ORNL Researcher recognized for ASCI

Gil Weigand of ORNL’s Computing and Computational Sciences Directorate received the inaugural James R. Schlesinger Award from Secretary of Energy Samuel Bodman.

The Secretary lauded Weigand for his “passion for excellence along with his ability to foster and implement the practices and values that are necessary for the protection of our nation.”

Gil is credited with conceiving and implementing the Department of Energy’s (DOE’s) Accelerated Strategic Computing Initiative (ASCI), which pooled government programs and national laboratories to build the world’s best high-performance supercomputers. HPC and simulation at the ASCI level now pervade all areas of science and engineering.

“Simulation and modeling has arrived. It is wholly acknowledged today as a scientific methodology on an equal par with theory and experiment,” Gil said at the ceremony.

Schlesinger, who was present for the award ceremony, was the first Secretary of Energy. The Schlesinger Award is the highest award in the newly established Secretarial Honor Awards Program and the highest nonmonetary award bestowed by the agency.

NCCS Continues Outreach Success

HPC conference brings students, faculty together

The NCCS continues to provide quality outreach and education initiatives to the wider HPC community through programs like the High Performance Computing and Applications Conference.

The meeting, hosted by the NCCS, invited undergraduate, graduate, and postdoctoral students from universities across the southeastern United States to submit posters, abstracts, and papers for peer review. The students’ presentations were then reviewed by a team of volunteers, and the feedback passed on to the students. The purpose of the conference, said Bobby Whitten of the NCCS User Assistance and Outreach Group, was “to provide information that educators can incorporate into their curricula as well as provide students with a foundation in the basics of parallel programming.”

Approximately 12 students and 12 faculty members attended from universities such as Western Kentucky, Clemson, and the Georgia Polytechnic Institute. Presentations included a method for the modeling of galactic collisions and the use of HPC for monitoring and evaluating natural disasters.

“The conference was a great success,” said Whitten. “All of the participants seemed to enjoy the open, friendly environment and the opportunity to share ideas with their peers from various institutions. Faculty members appreciated the opportunity to collaborate on teaching techniques and research possibilities.”

The conference took place April 11–12 at the American Museum of Science and Energy
in Oak Ridge, Tennessee.

Kothe Guest Speaker at HPC User Forum

NCCS Director of Science discusses ORNL’s supercomputing future

The NCCS’s Doug Kothe was a keynote speaker at this year’s HPC User Forum in Norfolk, Virginia. Kothe’s talk, entitled “National Lab HPC Directions at ORNL,” explored the future of HPC at ORNL, one of America’s top supercomputing centers. Already home to one of the world’s premier supercomputers, ORNL will soon launch a petascale system that promises to greatly enhance the science of computer simulation.

Besides being a keynote speaker, Kothe was also named to the Steering Committee. The HPC User Forum is a regular gathering of industry, government, and academia to discuss technology and software trends, market dynamics, and possible collaborations to advance the state-of-the-art in HPC. This year’s theme was computational fluid dynamics. The conference was held April 14–16 and included representatives from Boeing, NASA, and General Motors.