An Adaptive Sub-sampling Method for In-memory Compression of Scientific Data

Abstract

A  current challenge in scientific computing is how to curb the growth of simulation datasets without  losing valuable information. While  wavelet based methods are popular, they require that data be decompressed before it can analyzed,for example, when identifying time-dependent structures in turbulent flows. We present Adaptive Coarsening, an adaptive subsampling compression strategy that enables the compressed data product to be directly manipulated in memory without requiring costly decompression.We demonstrate compression factors of up to 8 in turbulent flow simulations in three dimensions.Our compression strategy produces a non-progressive multiresolution representation, subdividing the dataset into fixed sized regions and compressing each region independently.