Dynamic Workflow Engine of Atmospheric Big Remote Sensing Data Processing Powered by Heterogenous Platform for High Performance Computing

Abstract

The development of big remote sensing data related technologies and applications poses a big challenge that massive computing capability is needed to support big data processing. In order to solve this challenge, this paper proposes an architecture of heterogeneous platform of high performance computing, which employs the computer hardware resources to improve the efficiency of big remote sensing data processing by optimizing scheduling strategies and designing high-performance algorithms. Furthermore, the proposed platform can dynamically incorporated with a workflow engine regarding big remote sensing data processing. These algorithms are modular to meet the flexible combination of different processes.