A Performance Driven Micro Services-Based Architecture/System for Analyzing Noisy IoT Data

Abstract

The Internet of Things (IoT) technology presents a complex and challenging paradigm where a huge amount of noisy raw sensor data is collected in order to observe and detect critical events occurring on the system, and generate alarms when required. The biggest challenge of the IoT systems is that the systems collect a massive amount of uncertain data from diverse IoT devices connected through the network. In addition, some events are inferred from other events and uncertainty is propagated from parent events to the inferred events, which additionally contributes to overall system uncertainty. The observed complex events are a complex relationship of primitive events that are produced by IoT devices and collected in IoT systems. A survey performed on existing prior arts on quantifying uncertainty for complex events concluded that proposed existing solutions are unable to scale under heavy loads of incoming data. This paper presents a micro-service based notification methodology that uses complex event recognition (both complex event processing and probabilistic programming) to handle IoT systems uncertainty. In addition, the paper analyzes and recommends existing big data platforms for processing complex events in IoT systems. The current focus of our work includes research and development of the optimized deadline-based and cost-effective resource allocation algorithm in Apache Spark for Uncertain IoT Notification systems.