Enhancing BERT-Based Language Model for Multi-label Vulnerability Detection of Smart Contract in Blockchain

Smart contracts are decentralized applications that hold a pivotal role in blockchain-based systems. Smart contracts are composed of error-prone programming languages, so it is affected by many vulnerabilities (e.g., time dependence, outdated version, etc.), which can result in a substantial economic loss within the blockchain ecosystem. Therefore, many vulnerability detection tools are designed to detect the vulnerabilities in smart contracts such as Slither, Mythrill and so forth. However, these tools require high processing time and cannot achieve good accuracy with complex smart contracts nowadays. Consequently, many studies have shifted towards using Deep Learning (DL) techniques, which consider bytecode to determine vulnerabilities in smart contracts. However, these mechanisms reveal three main limitations. First, these mechanisms focus on multi-class problems, assuming that a given smart contract contains only a single vulnerability while the smart contract can contain more than one vulnerability. Second, these approaches encounter ineffective word embedding with large input sequences. Third, the learning model in these mechanisms is forced to classify into one of pre-defined labels even when it cannot make decisions accurately, leading to misclassifications. Therefore, in this paper, we propose a multi-label vulnerability classification mechanism using a language model. To deal with the ineffective word embedding, the proposed mechanism not only takes into account the implicit features derived from the language models (e.g., SecBERT, etc.) but also auxiliary features extracted from other word embedding techniques (e.g., TF-IDF, etc.). Besides, a trustworthy neural network model is proposed to reduce the misclassification rate of vulnerability classification. In detail, an additional neuron is added to the output of the model to indicate whether the model is able to make decisions accurately or not. The experimental results illustrate that the trustworthy model outperforms benchmarks (e.g., binary relevance, label powerset, classifier chain, etc.), achieving up to approximately 98% f1-score while requiring low execution time with 26 ms.