National Center for Computational Sciences

Jaguar

Access Calendars CLE 2.1 Upgrade XT5 Changes File Systems Data Transfer Compiling

Debugging & Optimization Running Jobs FAQ Open Issues Software Version-Changes Roadmap Presentations & Publications

Jaguar is the primary system in the ORNL Leadership Computing Facility (OLCF).

XT5 Partition

The XT5 partition contains 18,688 compute nodes in addition to dedicated login/service nodes. Each compute node contains two quad-core AMD Opteron 2356 (Barcelona) processors running at 2.3 GHz, 16GB of DDR2-800 memory, and a SeaStar 2+ router. The resulting partition contains 149,504 processing cores, more than 300TB of memory, over 6 PB of disk space, and a peak performance of 1.38 petaflop/s (1.38 quadrillion floating point operations per second).

XT4 Partition

The XT4 partition contains 7,832 compute nodes in addition to dedicated login/service nodes. Each compute node contains a quad-core AMD Opteron 1354 (Budapest) processor running at 2.1 GHz, 8 GB of DDR2-800 memory (some nodes use DDR2-667 memory), and a SeaStar2 router. The resulting partition contains 31,328 processing cores, more than 62 TB of memory, over 600 TB of disk space, and a peak performance of 263 teraflop/s (263 trillion floating point operations per second).

The SeaStar2+ router (XT5 partition) has a peak bandwidth of 57.6GB/s, while the SeaStar2 router (XT4 partition) has a peak bandwidth of 45.6GB/s. The routers are connected in a 3D torus topology, which provides an interconnect with very high bandwidth, low latency, and extreme scalability.

The operating system for Jaguar is the Cray Linux Environment. This consists of a full-featured version of Linux on the service nodes and the Compute Node Linux micro-kernel on the compute nodes. The micro-kernel is designed to minimize partition overhead allowing scalable, low-latency global communications.

Jaguar Roadmap

Currently, the XT4 and XT5 partitions are accessed via separate login nodes (Jaguar and Jaguarpf, respectively) and have independent lustre filesystems and batch queues. There are several new features that are entering production. These include:

• a high-performance global Lustre filesystem mounted to both XT partitions
• access to both partitions via a set of external login nodes, and
• a common scheduler that manages the resources of both partitions from a single point.

Information on these upgrades will be posted here and to NCCS e-mail lists.