File Systems

Notice:
Compute nodes can see only the Lustre work space.

The NFS-mounted home, project, and software directories are not accessible to the compute nodes.

  • Executables must be executed from within the Lustre work space /tmp/work/$USER (XT4 and XT5) or /lustre/scr144/$USER (XT4 only).
  • Batch jobs can be submitted from the home or work space. If submitted from a user’s home area, the user should cd into the local Lustre work space directory prior to running the executable through aprun. An error similar to the following may be returned if this is not done: 
            aprun: [NID 94]Exec /tmp/work/userid/a.out failed: chdir /autofs/na1_home/userid
        No such file or directory
  • Input must reside in the local Lustre work space /tmp/work/$USER (XT4 and XT5) or /lustre/scr144/$USER (XT4 only).
  • Output must also be sent to the Lustre file system /tmp/work/$USER (XT4 and XT5) or /lustre/scr144/$USER (XT4 only).

Home Directories

Each user is provided a home directory to store frequently used items such as source code, binaries, and scripts. Please see the home directories page in the general support section for more details.

Project Directories

Each project is provided a directory to share project common files. Please see the project directories page in the general support section for more details.

Work Directories

Work space is available on each NCCS high-performance computing (HPC) system for temporary files and for staging large files from and to the High Performance Storage System (HPSS).

Spider

The Spider center-wide file system is now the operational work file system on Jaguar’s XT5 (jaguarpf) and XT4 (jaguar) partition, Lens, Smoky, and the dedicated data transfer nodes. It is the largest-scale Lustre file system in the world, with over 26,000 clients, and it is the fastest Lustre file system in the world, with a demonstrated bandwidth of 240 GB/s. Spider currently provides 5 PB of disk space.

More information on Spider including best practices can be found here .

The Spider center-wide file system is the operational work file system on Jaguar’s XT5 (jaguarpf) and XT4 (jaguar) partition, Lens, Smoky, and the data transfer nodes. Please see the spider page for more information.

Please see the work directories page in the general support section for more details.

HPSS

The HPSS provides archival storage for NCCS users. Please see the HPSS section for more details.