Distributed Inference Approach on Massive datasets using MapReduce

Abstract

Contemporary computer systems and applications generate high volume of data every day. Gaining knowledge from this ever-growing high velocity and high volume data is crucial to have insights and business intelligence. Using semantic web approaches for generating inferences to gain knowledge have been quite successful. When processing large amounts of data, a centralised method for finding inferences in ontologies will be ineffective. Therefore, to solve this problem, a distributed strategy is needed. The major challenges on large scale data are the difficulty in deriving suitable triples for appropriate inferences, to reduce the time spent in processing of inference and the requirement of scalable computation capabilities for large dataset. Also, storage space for increasing data must be addressed efficiently. This paper proposes a distributed conjecture approach to address the above issues by construction of SIM (Sparse Index Method) and ATC (Assertional Triples Construction) and to efficiently process the users’ queries.