Can We Automatically Fix Bugs by Learning Edit Operations?

Abstract

There has been much work done in the area of automated program repair, specifically through using machine learning methods to correct buggy code. Whereas some degree of success has been attained by those efforts, there is still considerable room for growth with regard to the accuracy of results produced by such tools. In that vein, we implement Hephaestus, a novel method to improve the accuracy of automated bug repair through learning to apply edit operations. Hephaestus leverages neural machine translation and attempts to produce the edit operations needed to correct a given buggy code segment to a fixed version. We examine the effects of using various forms of edit operations in the completion of this task. Our study found that all models which learned from edit operations were not as effective at repairing bugs as models which learned from fixed code segments directly. This evidences that learning edit operations does not offer an advantage over the standard approach of translating directly from buggy code to fixed code. We conduct an analysis of this lowered efficiency and explore why the complexity of the edit operations-based models may be suboptimal. Interestingly, even though our Hephaestus model exhibited lower translation accuracy than the baseline, Hephaestus was able to perform successful bug repair. This success, albeit small, leaves the door open for other researchers to innovate unique solutions in the realm of automatic bug repair.