Please hold on: more time = more patches? Automated program repair as anytime algorithms

Abstract

Current evaluations of automatic program repair (APR) techniques focus on tools' effectiveness, while little is known about the practical aspects of using APR tools, such as how long one should wait for a tool to generate a bug fix. In this work, we empirically study whether APR tools are any time algorithms (e.g., the more time they have, the more fixes they generate, so it makes sense to trade off longer time for better quality). Our preliminary experiment shows that the amount of plausible patches, given exponentially greater time, only increases linearly or not at all.