MPI Acceleration of Image Classification: Are We Seeing the Resurgence of MPI in Solving Big Data Problems?

Abstract

Recent work has shown the effectiveness of the MPI programming paradigm in accelerating image classification via the Stochastic Gradient Descent optimization technique. Applications such as Caffe, Torch and Tensor Flow, that use Graphic Processing Unit accelerators within the SMP node, have been extended to use MPI across nodes with scalable speedups. In this talk, we will briefly review convolutional neural networks and the stochastic gradient technique to explore optimized solutions for the image classification problem. Next, I will review opportunities and challenges in parallel and distributed asynchronous stochastic gradient descent and the benefits from using MPI libraries. I will also present possible future directions for MPI based deep learning and other Big Data applications.