HPSS
| HPSS |
The mass storage facility at Oak Ridge National Laboratory (ORNL) currently consists of tape and disk storage components, Linux servers, and High Performance Storage System (HPSS) software. As of October, 2009, we have over 7.5 PB stored in over 16 million files.
Incoming data is written to disk, then later migrated to tape for long term archival. Tape storage is provided by robotic tape libraries.
The National Center for Computational Sciences (NCCS) has 3 SL8500 tape libraries, each holding up to 10,000 cartridges. The libraries house a total of twenty-four T10K-A tape drives (500 GB cartridges, uncompressed) and thirty-six T10K-B tape drives (1 TB cartridges, uncompressed). Each drive has a bandwidth of 120 MB/s.
ORNL Mass Storage History
ORNL’s work in mass storage began in the early 1990s to support the Atmospheric Radiation Measurement project and to provide storage for simulation results generated on the NCCS’s Paragon supercomputers. To support those projects, ORNL acquired and ran the NSL UniTree storage management product.
In 1993 it became clear that NSL UniTree would eventually become inadequate for the projected needs of the NCCS. At that time a follow-on to NSL UniTree, known as HPSS, was being designed by IBM and a collaboration of Department of Energy (DOE) national laboratories (Sandia, Livermore, and Los Alamos). ORNL joined that collaboration and took on responsibility for the storage system management (SSM) portion of the product, for which the ORNL HPSS development team continues to be responsible.
ORNL continued with NSL UniTree production use until 1997, at which time the conversion to HPSS was completed.
Also in 1997 HPSS won an R&D 100 Award based on an entry initiated and prepared at ORNL.
In 1998 a proposal to establish a storage testbed was developed and sent to the Mathematics, Information and Computer Sciences (MICS) office of DOE; that testbed was named Probe. It was funded in fiscal year 1999 as a collaboration between ORNL and the National Energy Research Scientific Computing Center (known as NERSC).
In late 2002 the ORNL storage group changed the tape robotics to StorageTek Powderhorn libraries using 9840 and 9940 tape drives and cartridges. This vastly increased the NCCS’ storage capacity in response to the relentless accumulation of user data.
In 2006, total data stored in HPSS at ORNL surpassed 1 petabyte for the first time. After taking 8.5 years to store 1 petabyte, the second petabyte took less than 2 years and the third took only 6 months, so it’s clear that users are storing data at an increasing rate.
As storage, network, and computing technologies continue to change, ORNL’s storage system evolves to take advantage of new equipment that is both more capable and more cost-effective. 2010 will see the addition of another SL8500 which will hold 10 petabytes of data with the current generation of tape drives. With the new tape library, ORNL will have room to store and use up to 40,000 tape cartridges.
