Adaptation of RF and CNN on Spark

Abstract

Biological images are used in many applications, most of which are important in medical field. For example, MRI scans and CT scans result in high resolution images that are critical for diagnosis of cancers and other malfunction of organs. Nowadays, high resolution ultrasound images can provide details to examine blood vessel blockage. Another type of biological images are those of mixed patterns of proteins in microscope human protein atlas images.Due to the enormous amount of image data available even in a single medical organization, Machine Learning and Deep Learning technology have been used to assist in the image data analysis.Spark is a computing framework that has been proved to speed up data analysis dramatically. However, Spark Scala doesn't fully support Deep learning algorithms. In this paper, we present a case study of adapting the Random Forest (RF) and Convolutional Neural Network (CNN) to the Spark Scala framework. These algorithms were applied to multi-classes multilabel classification on a biological dataset from Kagglers. The experimental results show that both RF and CNN can be implemented with Spark Scala and achieve extremely high throughput performance.