National Center for Computational Sciences

Tracking a Neutron’s Odyssey through a Fast Reactor

Supercomputers at Oak Ridge help redesign the nuclear reactor

In an age of rapid technological advancement and growing ecological awareness, the more than 60-year-old design of the nuclear reactor seems to have been completely disregarded.

Reactors have been around since 1942, when the renowned Enrico Fermi and colleagues created Chicago Pile No. 1 under the abandoned stands of a stadium at the University of Chicago. Despite the obvious promise of nuclear power as a source of electricity, the public has had difficulty warming to the technology, especially since the 1979 accident at Pennsylvania’s Three Mile Island power plant and the far more serious 1986 disaster at Ukraine’s Chernobyl facility. Public worries have focused on safety, expense, and the fate of spent reactor fuel—a witch’s brew of highly radioactive materials that will remain lethally dangerous for tens of thousands of years.

In the decades since Fermi’s achievement, researchers have strived to improve reactor design and minimize the need for waste storage, but physical experiments can be prohibitively expensive, and computer simulations have been unable to tackle the enormous complexity of the nuclear processes involved. The computational limitations, however, are being overcome by researchers such as Argonne National Laboratory’s Dinesh Kaushik and computers such as Oak Ridge National Laboratory’s (ORNL’s) Cray XT5 Jaguar, the world’s most powerful scientific supercomputer. Kaushik and colleagues are using 25 million processor hours on Jaguar to increase our understanding of these processes. Not only will the team’s work help to understand traditional light-water reactors, it will also help in the development of a viable fast reactor, a design likely to use existing reactor waste as part of its fuel, rendering the waste far less dangerous and permanent.

The physical process responsible for both the nightmarish destructiveness of early nuclear weapons and the controlled energy of an electric power plant is known as nuclear fission. In it, neutrons bombard the fuel, causing susceptible—or fissile—nuclei to split; when they do, they release energy and more neutrons. The energy is responsible for generating electricity and the neutrons for continuing the process in a chain reaction. As designed, the fuel loaded into a conventional light-water reactor cannot maintain a chain reaction by itself because the concentration of fissile nuclei in the fuel is too low. Instead, the chain reaction relies on a material surrounding the fuel, known as a moderator, which slows the neutrons and increases the likelihood they will split other nuclei.

Fast reactors, on the other hand, do not use a moderator to slow neutrons. Instead, they use fuel with a higher concentration of fissile materials—either uranium-235 or plutonium. The extra neutrons produced by this more highly enriched fuel help it maintain a chain reaction.

The extra neutrons also serve at least two other purposes: They lodge in less fissile isotopes, causing them to become more fissile, and they bombard the fragments of nuclei already split, causing them to split further into isotopes that are stable (or at least less dangerous). As a result, fast reactors show promise not only in generating power, but also in converting and reducing our inventory of existing spent reactor fuel.

So far, however, commercially viable fast reactors remain beyond our grasp, and the enormous expense of design experiments has significantly slowed research in this area over the past 15 years. To find a workable design, researchers must first be able to accurately predict the behavior of fast neutrons inside the reactor. These neutrons have a wide range of energies, velocity vectors, and complex spatial distribution, all of which help determine how they scatter within the reactor and interact with the fuel, the reactor walls, other neutrons, and so forth.

Kaushik’s team is developing and testing neutron transport algorithms for fast reactors. Its UNIC code, being developed through the Department of Energy’s Advanced Modeling and Simulation program, carries out state-of-the-art numerical simulations that incorporate progressively more detailed descriptions of a nuclear reactor core and associated processes. Through these high-fidelity simulations, the team hopes to greatly reduce the existing uncertainties and biases in reactor-design calculations.

By coupling a realistic range of factors, the project will address complicated thermal and structural issues and may help prevent processes that lead to reactor accidents.
One example would be a pulse of neutrons that causes a small explosion inside the reactor, with the heat generated by the explosion eventually deforming the reactor walls, thereby altering the flow of neutrons in the reactor and creating an unwanted feedback loop.

Neutron transport in a fast reactor is mathematically described by the Boltzmann transport equation, which follows the movement of a particle through a fluid. To track a neutron, the Boltzmann equation must address seven independent variables: three in space, two in direction of motion (angle), one in energy, and one in time. Until now supercomputers lacked the power to implement models in space, angle, and energy that were sufficiently fine to reflect the complexity of the system.

A nuclear reactor does its job on a wide range of length scales, from the size of a nucleus to that of the reactor vessel. Neutrons and other particles within the reactor interact in complex patterns with a wide range of energies. To model the complex geometry of it all, billions of spatial elements, hundreds of angles, and thousands of energy groups are necessary. Without a petascale computer such as Jaguar, it can’t be done.

For several years computational scientists have responded to this overwhelming complexity with advanced approximations, replacing details in the various spatial, angle, energy, and time systems with averages. These averaging methods, known as homogenization, lack the detail needed to explain the localized behavior of neutrons in the reactor and resolve questions about the neutralization of transuranic waste in spent fuel. Kaushik and his collaborators will use the petascale power of Jaguar to progressively reduce this averaging and move toward more detailed, realistic simulations of fast reactors.

As they are refined, these algorithms will solve successively bigger problems, beginning with practice problems incorporating a fixed number of parameters. Later they will move toward more realistic fast-reactor geometries, incorporating different reactor configurations and a large number of energy groups and directions of motion.

"The code as a whole should allow the existing reactor analysis work to transition smoothly from the existing homogenization approaches to less crude homogenization and eventually to fully heterogeneous descriptions, as the computer technology allows," Kaushik explained. "We allow the reactor analyst to choose the level of approximation that is to be imposed rather than be limited to what is currently available. With the allotted computational time, we will demonstrate this transition ability within the limits of the computational abilities of the system."

To date the UNIC code has run on 131,072 of Jaguar’s 180,000-plus processor cores for two reactor problems, Kaushik said.

"With the 25 million processor core hours, we are doing 40 to 50 runs for various mesh sizes, a large number of energy groups, and higher angular resolutions. The time allocation will allow us to carry out more realistic reactor simulations, resulting in less uncertainty in the crucial reactor design and operational parameters.

"Over the coming months, we aim to improve the per-processor performance while maintaining the high parallel efficiency by employing better algorithms," he added. "Combining these additional algorithmic improvements with larger parallel machines in the near future should allow us to realize our long-term goal and start solving problems with more geometric detail and more energy groups."

The researchers will compare the results of the current work with existing methods and validate them against experiment.

Kaushik’s project is one of two aimed at using Jaguar to provide accurate simulation of the enormously complex processes involved in a nuclear reactor. Tom Evans of ORNL and colleagues are approaching this challenge by extending an existing parallel transport solver called Denovo, which uncouples the multi-level, phase space parameters of the Boltzmann equations.

Fast reactors may have a major role to play in the contemporary world—as both an efficient, less expensive source of electrical power and a potentially significant solution to the problem of nuclear waste disposal.

Cray XT5 Undergoes Upgrade

Oak Ridge’s Jaguar supercomputer to reach speeds of over 2 petaflops

The Cray XT Jaguar, housed at the National Center for Computational Sciences (NCCS) at ORNL, is undergoing an upgrade that will increase the peak performance of the machine from approximately 1.6 quadrillion calculations per second (or petaflops) to over 2 petaflops.

The process of upgrading the machine is tentatively scheduled to occur in fives phases over the course of 14 weeks, ending in November of this year.

Cray’s XT Jaguar is composed of the Jaguar XT5 and XT4 partitions, creating the world’s second fastest supercomputer for open research. Each of the XT5 partition’s 18,688 compute nodes presently contains two quad-core AMD Opteron (Barcelona) processors, resulting in over 149,000 processing cores. Funded with Recovery Act money from the Department of Energy, the upgrade will replace each of the XT5 partition’s quad-core processors with six-core AMD Opteron processors, code-named Istanbul. The result will bring Jaguar XT5’s total number of processors to over 224,000.

The upgrade will proceed on a rolling basis, keeping large portions of the machine available for user access for most of the process. Each of the five phases of the upgrade is scheduled to take between two to four weeks, and involves removing dozens of Jaguar’s 200 cabinets, each containing thousands of processing cores, for replacement. The XT4 partition will be available throughout the XT5 upgrade. Once the upgrade is completed, the XT5’s new processors will be tested thoroughly to ensure reliable and improved performance.

Biophysics Workshop Draws Top Researchers, Students
Problems, issues, and future for biological, physical, and computational communities discussed

A three day summer workshop in the biological, physical, and computational sciences attracted more than 100 students and leading scientists to downtown Knoxville, detailing current trends and possibilities in the fields.

The workshop, which took place August 3–5 at the UT Conference Center, targeted senior undergraduate, graduate, and postdoctoral students in sciences such as physics, mathematics, computer science, chemistry, and biology.

"First, we want to expose the students from technical fields to opportunities in the biosciences," said Tamah Fridman, event coordinator and research scientist at the Joint Institute for Computational Sciences (JICS) located at ORNL. "Simultaneously, we want to create an interface through which the biological community can reach those in the exact sciences."

Eighteen scientists were invited to speak, including Hashim Al Hashimi, Associate Professor of Chemistry and Biophysics at the University of Michigan, who delivered a presentation entitled "How Making Movies of Biology at the Atomic Scale can Help Cure Disease," and Jeremy Smith, director of the ORNL Center for Molecular Biophysics and the UT/ORNL’s Governor’s Chair for UT/ORNL’s Center for Molecular Biophysics, who presented "Computer Simulation and Neutron Scattering in Biology."

"To say that I was incredibly fortunate to attend would be an understatement," said Boloye Gomero, a first year graduate student in computer science at the University of Tennessee. "I must say that my interest in biophysics has increased quite a bit. This was a wonderful opportunity and I am glad to have attended."

Involved in organizing the first time event were JICS, co-managed by ORNL and the University of Tennessee (UT), the National Science Foundation’s (NSF) National Institute for Computational Sciences (NICS), managed by UT, the NCCS, and the Energy and Engineering Sciences Directorate at ORNL. Additional sponsors included Bruker Biospin and Rigaku, science technology development and manufacturing groups, and Cambridge Isotope Laboratories.

Those involved hope to make the workshop an annual event, said Fridman.

"Getting the chance to hear such well-renowned and well-respected researchers—such as Benoit Roux, Roald Sagdeev, and Gregory Petsko—was especially compelling," said Donald Frederick, member of NCCS’s User and Assistance Outreach Group. "I would imagine [the workshop] would be of great interest to new researchers.

OLCF Science Writers Recognized for Outstanding Achievements

Sweep ‘best feature’ awards in international competition

Science writers at the OLCF recently swept the gold, silver, and bronze categories for best feature article in a web-based/electronic publication in an international competition, the 2008 Magnum Opus Awards for Outstanding Achievement in Custom Media. To win these awards, presented annually in conjunction with the prestigious Missouri School of Journalism, the OLCF writers took on such competitors as The Walt Disney Company; Wyeth; Cargill, Inc.; Toyota Motor Sales, USA, Inc.; and Rodale Custom Publishing.

Scott Jones won the gold for "NCCS System Models Hummingbird Flight." His article describes the work of researchers from Digital Rocket Science who used the Phoenix supercomputer to dissect the dynamics of hummingbirds flapping their wings to better understand the aerodynaimcs of flight.

Dawn Levy took the silver for "Resolution Revolution," which describes the critical role of climate simulation software applications such as the Community Climate System Model (CCSM) in advanced climate prediction. The CCSM is a megamodel that couples four independent models of Earth’s atmosphere, oceans, lands, and sea ice, which could not run without the powerful supercomputing at the OLCF.

Leo Williams won the bronze for "Invisible Means of Support," which describes the work of computational astrophysicists on the Cray XT4 Jaguar supercomputer at the OLCF who made the largest simulation ever of dark matter evolving over billions of years to surround a galaxy.