Composing Data Parallel Code for a SPARQL Graph Engine

Abstract

The emergence of petascale triple stores have motivated the investigation of alternates to traditional table-based relational methods. Since triple stores represent data as structured tuples, graphs are a natural data structure for encoding their information. The use of graph data structures, rather than tables, requires us to rethink the methods used to process queries on the store. We are developing a scalable, in-memory SPARQL graph engine that scales to hundreds of nodes while maintaining constant query throughput. Our framework comprises a SPARQL to data parallel C compiler, a library of parallel graph methods, and a custom multithreaded runtime layer for multinode commodity systems. Rather than transforming SPARQL queries into a series of select and join operations on tables, our front end compiles the queries into data parallel C code with calls to graph methods that walk internal data structures, constructing answers in their wake. In this paper, we describe the compilation process and give examples of the generated C code parallelized with OpenMP. We present performance numbers for the SP2Bench SPARQL benchmark queries on a 48-core shared-memory system. With respect to conventional relational database systems such as Virtuoso, our approach uses less memory and provides higher performance.