Metamorphic Model-Based Testing of Autonomous Systems

Abstract

Testing becomes difficult when we cannot easily determine whether or not the system under test delivers the correct result. Autonomous systems are a case in point because it is difficult to determine whether a safety-critical autonomous system's behavior meets its specifications. To address the problem of testing autonomous drones, we have developed a framework for automated testing of a simulated autonomous drone system using metamorphic testing principles combined with model-based testing. Based on the results from using the framework to test the drone in the simulator using obstacles that do not move during flight, we have determined that this is a cost beneficial solution allowing for comprehensive testing without having to develop complex testing infrastructure to determine detailed test oracles. Our test cases are automatically generated from a set of testing models where each model encodes a certain scenario that can be varied according to metamorphic principles.