Investigating Controller Evolution and Divergence through Mining and Mutation*

Successful cyber-physical system controllers evolve as they are refined, extended, and adapted to new systems and contexts. This evolution occurs in the controller design and also in its software implementation. Model-based design and controller synthesis can help to synchronize this evolution of design and software, but such synchronization is rarely complete as software tends to also evolve in response to elements rarely present in a control model, leading to mismatches between the control design and the software. In this paper we perform a first-of-it-skind study on the evolution of two popular open-source safety-critical autopilot control software – ArduPilot, and Paparazzi, to better understand how controllers evolve and the space of potential mismatches between control design and their software implementation. We then use that understanding to prototype a technique that can generate mutated versions of code to mimic evolution to assess its impact on a controller’s behavior.We find that 1) control software evolves quickly and controllers are rewritten in their entirety over their lifetime, implying that the design, synthesis, and implementation of controllers should also support incremental evolution, 2) many software changes stem from an inherent mismatch between continuous physical models and their corresponding discrete software implementation, but also from the mishandling of exceptional conditions, and limitations and distinct data representation of the underlying computing architecture, 3) small code changes can have a dramatic effect in a controller’s behavior, implying that further support is needed to bridge these mismatches as carefully verified model properties may not necessarily translate to its software implementation.