Refining Fitness Functions for Search-Based Program Repair

Abstract

Debugging is a time-consuming task for software engineers. Automated Program Repair (APR) has proved successful in automatically fixing bugs for many real-world applications. Search-based APR generates program variants that are then evaluated on the test suite of the original program, using a fitness function. In the vast majority of search-based APR work only the Boolean test case result is taken into account when evaluating the fitness of a program variant. We pose that more fine-grained fitness functions could lead to a more diverse fitness landscape, and thus provide better guidance for the APR search algorithms. We thus present 2Phase, a fitness function that also incorporates the output of test case failures, and compare it with ARJAe, that shares the same principles, and the standard fitness, that only takes the Boolean test case result into consideration. We conduct the comparison on 16 buggy programs from the QuixBugs benchmark using the Gin genetic improvement framework. The results show no significant difference in the performance of all three fitness functions considered. However, Gin was able to find 8 correct fixes, more than any of the APR tools in the recent QuixBugs study.