Advanced Monkey Testing for connected autonomous systems

Sensors, actuators, machine learning, communication and robotics are paving the way for the introduction of autonomous systems. Autonomous Systems in safety-critical applications require resilient operation of the intended functionality throughout the mission. Especially they must be safe and highly available. However, it is not possible to fully anticipate evolving threats, vulnerabilities and faults during the lifetime of those systems. This requires a resilient systems architecture of the autonomous system. Therefore, a thorough testing and evaluation of such systems is mandatory. In this paper, we present a monkey testing framework for evaluating resilience capabilities of autonomous systems. The framework contains a set of agents with specific role concepts and strategy sets. The framework can be applied to virtual, physical and hybrid testbeds. Due to its modularity the framework is extensible, scalable and also adaptable to different autonomous systems (e.g. mobile robot, manipulator). The monkey testing framework is able to work pseudo-randomized and thus reproducible on a connected system. A logging mechanism annotates the data so that the data can be used for machine learning (e.g. anomaly detection algorithm, selfhealing). We applied the framework on a mobile robotic system in virtual scenarios.