Examining the effectiveness of transformer-based smart contract vulnerability scan

Smart contracts can be used for various scenarios, from automating transactions in decentralized finance to managing supply chain logistics, or ensuring the integrity of digital assets. However, smart contracts may expose vulnerabilities that may be exploited, which can lead to financial losses and disruptions in decentralized applications. These vulnerabilities involve decision logic, branching, sequencing, and interaction with other Ethereum addresses, and therefore are challenging to detect. In this work, we study the effectiveness of smart contract vulnerability detection using deep learning. We propose VASCOT, a Vulnerability Analyzer for Smart COntracts using Transformers, which performs sequential analysis of Ethereum Virtual Machine (EVM) bytecode and incorporates a sliding window mechanism to overcome input length constraints. To assess VASCOT’s detection efficacy, we construct a dataset of 16,469 verified Ethereum contracts deployed in 2022, and annotate it using trace analysis with concrete validation to mitigate false positives. VASCOT’s performance is then compared against a state-of-the-art LSTM-based vulnerability detection model on both our dataset and an older public dataset. Our findings highlight the strengths and limitations of each model, providing insights into their detection capabilities and generalizability.