Building blocks for IoT testing - a benchmark of IoT apps and a functional testing framework

Abstract

IoT security is a topic that offers numerous opportunities for improvement and development. In this paper, we first present a set of open-source mock IoT applications along with the necessary infrastructure specifically designed to emulate a real IoT system. With our app set, users can add their own applications, automation rules, and communication flows with little technical effort, and test different scenarios to reproduce bugs that are not specific to the use of a single device. Second, we describe a functional testing framework for the IoT that is inspired by behavior-driven development (BDD), a testing methodology that serves as a proof-of-concept for how the application set can be used in different test scenarios. The application set and the functional testing framework are independent of each other. Our goal is to help IoT developers and testers find new testing techniques and benchmarking them in a reproducible, comparable, and less biased environment. We believe that they form the basis for a better understanding of how to test systems composed of heterogeneous devices to find issues and vulnerabilities that arise mainly from their interaction and data persistence management.