Intelligent agents based improved map-reduce method for multiple databases

Abstract

Big Data became in the last years a key basis of competition and innovation. But it is difficult to process the whole amount of data using traditional database and software techniques. To overcome this issue, search methods were developed to faster achieve information from large databases. In this paper, we propose an approach based on the use of agent technology and big data concept in order to improve the process time of Map-Reduce programming model. Finally, the approach is validated by a case study in which we achieved a 1.7x speedup comparing to traditional Map-Reduce technique for 26Mb of data.