Striim: A streaming analytics platform for real-time business decisions

Real-time decisions and insights over real-time data have become the essential mantra of success for many enterprises. The real-time data is generated from a multitude of sources and they come in a streaming fashion with high volume and velocity. The data could be machine generated e.g. clickstream data, logs, sensor data from IoT devices or human generated e.g. social data, mission critical transactional data. This is causing a technological shift from storage driven architectures to event driven architectures for enterprises to be able to capture, integrate and analyze these large sets of data for real-time decision making. Striim is a novel end-to-end analytics platform that enables business users to easily develop and deploy analytical applications that can generate real-time insights over real-time streaming data; business users and developers use a SQL-like declarative language (that has been extended to include streaming semantics) to write application logic in Striim. Striim provides high-throughput, low-latency event processing on commodity hardware with a scale-out architecture. In this paper, we describe the architecture of Striim and discuss some of the key aspects of the platform (a) built-in real-time data capture including streaming change data capture from transactional databases (ii) a natively built storage and query engine that uses modern data structures like skip lists to store streaming window data and performs query optimization, planning and run-time code generation (iii) enabling application de-coupling using persisted streams.