CodeSpeak: Improving smart contract vulnerability detection via LLM-assisted code analysis

Abstract

Smart contracts play a crucial role in blockchain technology, but their security remains vulnerable to various threats. While deep learning approaches have shown promise in vulnerability detection, they often require complex graph constructions that complicate the detection process. Large language models (LLMs) offer powerful code comprehension capabilities, but their direct application to vulnerability detection often yields inconsistent or unreliable results. To address these challenges, we introduce CodeSpeak, a novel framework that enhances smart contract vulnerability detection by leveraging LLM-assisted code analysis. Our approach first eliminates redundant code statements to focus on security-critical sections. We then leverage LLMs with designed domain-specific instructions that simulate security expert auditing practices. These instructions serve as intermediate representations that bridge the gap between natural language and vulnerability patterns. CodeSpeak processes this analysis by LLMs and creates structured prompt templates with these results, which are used to train a detection model. Compared to deep learning approaches, this framework offers a more intuitive solution while maintaining high detection effectiveness. Extensive experiments conducted on four types of vulnerabilities (Reentrancy, Timestamp, Overflow/Underflow, and Delegatecall) demonstrate the effectiveness of our approach. Our framework also demonstrates strong adaptability to new vulnerability types with minimal training samples, and provides a cost-effective solution for practical deployment. Moreover, a user study with developers shows CodeSpeak reduces detection time by 98.7% compared to manual analysis while maintaining superior accuracy. These improvements highlight the potential of LLM-assisted code analysis in smart contract security assessment.