Wrapping a NoSQL Datastore for Stream Analytics

Abstract

With the advent of the Industrial Internet of Things (IIoT) and Industrial Analytics, numerous application scenarios emerge, where business and mission-critical decisions depend upon large scale analytics of sensor streams. However, very large volumes of data from data streams generated at a high rate pose substantial challenges in providing scalable analytics from existing Database Management Systems (DBMS). While scalability can be provided by high-performance distributed datastores, due to the simple query operations, access to high-level query-based data analytics is usually limited. This work combines high-level query-based data analytics capabilities with high-performance distributed scalability by applying a wrapper-mediator approach. The Amos II extensible main-memory DBMS provides online query processing data analytics engine in front of the MongoDB distributed NoSQL datastore to support large-scale distributed data analytics over persisted data streams. Thus, the implemented system enables query-based online data stream analytics over persisted data streams stored/logged in distributed NoSQL datastores.