An efficient and scalable SPARQL query processing framework for big data using MapReduce and hybrid optimum load balancing

Abstract

The increasing RDF (Resource Description Framework) data volume requires a Hadoop platform for processing queries over large datasets. In this work, SPARQL (Simple Protocol and Rdf Query Language) queries are evaluated with Hadoop based on the objective of minimizing the number of joins through data partitioning for performing map/reduce jobs. The query evaluation time and the number of cross node joins are minimized with the proposed partitioning techniques. Extended vertical partitioning is proposed for distributed data stores based on objects’ explicit information for splitting predicates. For accessing the RDF data, hybrid monarch butterfly with beetle swarm load balancing optimization with Map-reduce (Hybrid Optimum Load Balancing) is applied. The proposed SPARQL query processing is evaluated over large RDF datasets. The proposed approach’s evaluation results are analyzed with the existing approaches, indicating the proposed framework’s efficiency. By using the proposed approach, an accuracy of 97 % is obtained.