A distributed learning architecture for big imaging problems in astrophysics

Abstract

Future challenges in Big Imaging problems will require that traditional, "black-box" machine learning methods, be revisited from the perspective of ongoing efforts in distributed computing. This paper proposes a distributed architecture for astrophysical imagery, which exploits the Apache Spark framework for the efficient parallelization of the learning problem at hand. The use case is related to the challenging problem of deconvolving a space variant point spread function from noisy galaxy images. We conduct benchmark studies considering relevant datasets and analyze the efficacy of the herein developed parallelization approaches. The experimental results report 58% improvement in time response terms against the conventional computing solutions, while useful insights into the computational trade-offs and the limitations of Spark are extracted.