A Ditributed Algorithm for Quality Assessment of Biological Sequencing Based on MapReduce

Abstract

DNA sequencing technology has played an important role on life sciences, especially Illumina’s sequencer. It was used for more and more biological genomic and transcriptomic projects. Faced with the huge amount of biological sequencing data, it is a problem how to assess its quality quickly. In this paper, we developed a distributed algorithm based on MapReduce, which can assess the quality of biological sequencing in parallel. In order to validate the algorithm, different data sizes (1G - 20G) were used to test by different computing nodes (1 - 20) in Hadoop platform. The results show that the parallel efficiency improves continuously following with the increase of data size and computing nodes. And the algorithm has better parallel efficiency when data size and computing nodes greater than 5Gb and 10 processors. This work effectively saves the time of quality assessment of biological sequencing.