Nested deep learning with learned network embeddings for software defect prediction

Abstract

Existing software (SW) defect prediction approaches and the models are majorly based on features extracted from the code of the software to build defect datasets for predictive modeling. However, these models fail to sufficiently capture the complex, latent dependencies within the software components, which acts as a hindrance in achieving higher predictive accuracy. This study introduces an improved defect prediction model, the Nested Deep Learning (NDL) model, that leverages network embeddings from call graphs for enhanced representation of intricate hierarchical class dependencies and interactions. This work evaluates six network-embedding algorithms by applying them to call graphs of 10 real software projects, generating embeddings of dimensions 32 and 128. A total of 50 NDL models—with and without dropout layers—are developed, and a comparative evaluation of these models is conducted against traditional classifier-based models. This evaluation demonstrated the superiority of the NDL model with dropout, achieving a mean AUC of 0.87, an 8.98 % improvement over the traditional classifier-based models. Among the evaluated embedding methods, LINE embeddings outperformed others, and integrating network embeddings with software metrics led to a 15.85 % AUC improvement over using software metrics alone. The optimal configuration—combining software metrics with LINE embeddings (dimension 128) in an NDL model with three deep learning layers and dropout—achieved a mean AUC of 0.93, surpassing all other configurations by 3.33–14.81 %. This study is the first to validate the effectiveness of a nested deep learning framework for modeling call graph dependencies through network embeddings, providing a scalable and robust approach for improving software defect prediction.