National Center for Computational Sciences

HDF

Description

The hierarchical data format (HDF) project involves the development and support of software and file formats for scientific data management. The HDF software includes input/output (I/O) libraries and tools for analyzing, visualizing, and converting scientific data.

There are two HDF formats, HDF (4.x and previous releases) and HDF5. These formats are completely different and NOT compatible.

Some of the HDF 4.x limitations are as follows:

A single file cannot store more than 20,000 complex objects, and a single file cannot be larger than 2 GB.
The data models are less consistent than they should be. There are more object types than necessary, and data types are too restricted.
The library source is old and overly complex, does not support parallel I/O effectively, and is difficult to use in threaded applications.

HDF5 includes the following improvements:

A new file format designed to address some of the deficiencies of HDF 4.x, particularly the need to store larger files and more objects per file.
A simpler, more comprehensive data model that includes only two basic structures:
1. a multidimensional array of record structures and
2. a grouping structure.
A simpler, better-engineered library and API, with improved support for parallel I/O, threads, and other requirements imposed by modern systems and applications.

HDF5 is not compatible with HDF 4.x, but conversion software is available for converting HDF 4.x data to HDF5, and vice versa.

For more information, see the HDF home page.

Use

Jaguar: HDF5 is available on Jaguar as a module. See the modules page for more information on modules. There are parallel and serial versions of HDF5. The modules are suffixed with _par or _ser for parallel or serial, respectively. Any binaries provided in the parallel module must be run with yod; however, any binaries provided in the serial module are run without yod.; Once an HDF module has been loaded, the following examples can be used to compile and link your program (hdf5example.f90) to the HDF libraries.
Phoenix: HDF5 is available on Phoenix as a module. By default the HDF modules will set up an environment for building multistreaming vector processor (MSP)-mode executables. See the modules page for more information on modules.; Once an HDF module has been loaded, the following examples can be used to compile and link your program (hdf5example.f90) to the HDF libraries.