Real-time big data stream analytics and complex event detection: modular visual framework, data science platform, and industry applications

In many industry applications, larger and larger amounts of data become available, allowing to gain deeper insights, to generate more accurate forecats, to optimize and automate processes, and to thereby create significant value. Often the data is not static, but arrives continuously in large data streams, which ideally are processed and leveraged in real-time. This talk will present a modular and flexible platform for real-time big data stream processing, complex event detection, data science and machine learning with an easy-to-use visual process design user interface, seamlessly integrating the most relevant big data and stream processing frameworks (Hadoop, Spark, Spark Streaming, Kafka, Flink, etc.) within one unifying platform and user interface, based on the widely used data science platform RapidMiner. This talk will also provide an overview of applications of this framework across many industries like machine failure prediction and prevention and predictive maintenance in industrial production in the manufacturing industry, criticial event detection, prediction and prevention in the chemical indutry, product quality prediction and optimization as well as energy consumption and cost reduction in the steel and metal industry, data-driven process optimization in the food and beverage industry, various use cases in the automotive and aviation industry, maritime data analysis to detect complex events like piracy or illegal fishing and to avoid collisions, drug effectiveness prediction for cancer drug development and biomedical research, financial time series analysis and forecasting for the investment industry. The latter three use cases are addresed in the European reseach projects INFORE, which will also be shortly introduced.