Optimizing mass storage organization and access for multi-dimensional scientific data

Abstract

A critical issue for scientific investigators is ready access to the large volume of data generated by large-scale supercomputer simulations and physical experiments. The authors describe the current status of a collaborative effort which focuses on managing data produced by climate modeling applications. The project is aimed at significantly improving the accessibility and ease of use of large scientific databases, in the context of a hierarchical mass storage system. The following aspects of the work are considered in detail: physical database design, metadata, storage, the mass storage system interface, and reduced data sets.<>