An approach to assess robustness of MQTT-based IoT systems

The Internet of Things (IoT) has become vital in modern life, connecting a wide range of devices, sensors, and other computing objects. As the number of devices and interconnections grows, IoT systems are also increasingly exposed to unexpected conditions. Thus, it becomes critical that they operate in a robust manner (i.e., they are able to deliver correct service, even in the presence of invalid inputs or stressful conditions). Despite this, the literature shows that practical robustness evaluation techniques have been mostly disregarded in this domain, with developers also tending to focus on core functionality, driven by reasons such as the pressure to time-to-market. This paper describes an approach for assessing the robustness of MQTT-based IoT systems, i.e., systems that use MQTT to exchange messages between devices. The proposed approach is based on injecting faults into messages carried by MQTT between IoT elements with the goal of activating residual application-level faults that may have escaped during the system development. To illustrate our approach, we applied it to two real case studies, Smart Rural and MInA, in which we were able to trigger several different types of failures. Overall, the results highlight the usefulness of the proposed approach and demonstrate the potential of a dedicated robustness assessment approach particularly tailored for IoT systems.