Programming event processors with thingflow

Abstract

We present ThingFlow, a software architecture to write event-processing pipelines for Internet-of-things (IoT) applications. Such applications typically involve sensing, stateful transformations of event streams, state estimation, and control/actuation, learning, and data analytics. They are currently programmed using a zoo of languages, systems, and APIs with ad hoc data passing protocols. ThingFlow provides three abstractions: sensors which can sample changing physical values, streams of (potentially real-valued) data, and stateful filters, which transform input streams to output streams, and can be composed. ThingFlow programs consist of compositions of filters scheduled and run by an explicit, asynchronous, scheduler. ThingFlow is implemented as a Python API whose core is compact enough to run on very limited microcontrollers such as the ESP8266, while providing support for sensors, message streams over the network, machine learning, and cloud backends. We show the expressiveness, versatility, and simplicity of ThingFlow on a number of examples from the IoT domain, incorporating sensing, filtering, actuation, data analysis, and learning.