Proper orthogonal decomposition based parallel compression for visualizing big data on the K computer

Abstract

The development of supercomputers has greatly help us to carry on large-scale computing for dealing with various problems through simulating and analyzing them. Visualization is an indispensable tool to understand the properties of the data from supercomputers. Especially, interactive visualization can help us to analyze data from various viewpoints and even to find out some local small but important features. However, it is still difficult to interactively visualize such kind of big data directly due to the slow file I/O problem and the limitation of memory size. For resolving these problems, we proposed a parallel compression method to reduce the data size with low computational cost. Furthermore, the fast linear decompression process is another merit for interactive visualization. Our method uses proper orthogonal decomposition (POD) to compress data because it can effectively extract important features from the data and the resulting compressed data can also be linearly decompressed. Our implementation achieves high parallel efficiency with a binary load-distributed approach, which is similar to the binary-swap image composition used in parallel volume rendering [2]. This approach allows us to effectively utilize all the processing nodes and reduce the interprocessor communication cost throughout the parallel compression calculations. Our test results on the K computer demonstrate superior performance of our design and implementation.