Taming velocity and variety simultaneously in big data with stream reasoning: tutorial

Abstract

Many "big data" applications must tame velocity (processing data in-motion) and variety (processing many different types of data) simultaneously. The research on knowledge representation and reasoning has focused on the variety of data, devising data representation and processing techniques that promote integration and reasoning on available data to extract implicit information. On the other hand, the event and stream processing community has focused on the velocity of data, producing systems that efficiently operate on streams of data on-the-fly according to pre-deployed processing rules or queries. Several recent works explore the synergy between stream processing and reasoning to fully capture the requirements of modern data intensive applications, thus giving birth to the research domain of stream reasoning. This tutorial paper offers an overview of the theoretical and technological achievements in stream reasoning, highlighting the key benefits and limitations of existing approaches, and discussing the open challenges and the opportunities for future research. The paper mainly targets researchers and practitioners in the area of event and stream processing. The paper aims to stimulate the discussion on stream reasoning and to further promote the integration of reasoning techniques within event and stream processing systems in three ways: (i) by presenting an active research domain, where researchers on event and stream processing can apply their expertise; (ii) by discussing techniques and technologies that can help advancing the state of the art in event and stream processing; (iii) by identifying the open problems in the field of stream reasoning, and drawing attention to promising research directions.