Automatic Arabic Text Summarization for Large Scale Multiple Documents Using Genetic Algorithm and MapReduce

Abstract

Multi document summarization focuses on extracting the most significant information from a collection of textual documents. Most summarization techniques require the data to be centralized, which may not be feasible in many cases due to computational and storage limitations. The huge increase of data emerging by the progress of technology and the various sources makes automatic text summarization of large scale of data a challenging task. We propose an approach for automatic text summarization of large scale Arabic multiple documents using Genetic algorithm and MapReduce parallel programming model. The approach insures scalability, speed and accuracy in summary generation. It eliminates sentence redundancy and increases readability and cohesion factors between the sentences of summaries. The experiments resulted in acceptable precision and recall scores. This indicates that the system successfully identifies the most important sentences. In Addition to all to that, the approach provided up to 10x speedup score, which is faster than on a single machine. Therefore, it can deal with large-scale datasets successfully. Finally, the efficiency score of the proposed approach indicates that the large data set utilizes the available resources up to 62%.