Toward Exploratory Understanding of Software using Test Suites

Abstract

Changing software without correctly understanding it often leads to confusion, as developers do not understand how the change corresponds to the new observed behaviour of the system. Today, many software systems are equipped with a test suite. Test suites document code and give feedback on changed program behaviour. We explored ways to use test suites for software comprehension and implemented a tool that provides additional visualisation and gives immediate feedback on software changes. Information about changes in the software and their implications to the test suite are collected using mutation testing. The tool uses this information to present relevant test cases for developers, and additionally prioritise test executions for immediate feedback. Our research indicates that entropy metrics can find test cases that are relevant for a specific context in the source code. Additionally, simple test case prioritisation strategies can already lead to a significant decrease in feedback time. Based on our case study we argue that test suites are not only useful for regression testing but can be used to generate meaningful information for software comprehension activities.