SHOE: A SPARQL Query Engine Using MapReduce

Abstract

As the reasoning aspects and the knowledge based processing capabilities of RDF (Resource Description Framework) have been widely adopted in W3C Recommendation, the ontology layer and query languages of the Semantic Web stack achieve a certain level of maturity. There exists an increasing need for high performance, read-only semantic analysis for the massive RDF data. In this demo we will present a Map-Reduce based SPARQL processing engine, called SHOE (SPARQL on Hadoop with Optimization Encoding), to handle billions of RDF triples. SHOE consists of three major components (1) the RDF data loader, (2) the partition generator and (3) the query processor. While this demonstration mainly focuses on enhancing SPARQL processing in the Hadoop platform, the underlying encoding and partitioning optimization strategies can be utilized by the common Map-Reduce frameworks in the share-nothing environment.