Deep learning or classical machine learning? An empirical study on line-level software defect prediction

Background

Line-level software defect prediction (LL-SDP) serves as a valuable tool for developers to detect defective lines with minimal human effort. Recently, GLANCE was proposed as a readily implementable baseline for assessing the efficacy of newly proposed LL-SDP models.

Problem

While DeepLineDP, a cutting-edge LL-SDP model rooted in deep learning, has demonstrated state-of-the-art performance, it has not yet been compared against GLANCE.

Objective

We aim to empirically compare DeepLineDP with GLANCE to obtain a comprehensive understanding of how deep learning contributes to solving the LL-SDP challenge.

Method

We compare GLANCE against DeepLineDP to assess the extent to which DeepLineDP surpasses GLANCE in predicting defective files and identifying problematic lines. In order to obtain a reliable conclusion, we use the same dataset and performance metrics utilized by DeepLineDP.

Result

Our experimental findings indicate that DeepLineDP does not outperform GLANCE in LL-SDP. This suggests that the application of deep learning, in this context, does not yield the anticipated significant improvements.

Conclusion

This finding underscores the need for further research in deep learning-based LL-SDP to attain the state-of-the-art performance that remains elusive for less advanced techniques.