SolBERT: Advancing solidity smart contract similarity analysis via self-supervised pre-training and contrastive fine-tuning

Abstract

Context:

Reliable and effective similarity analysis for the smart contracts facilitates the maintenance and quality assurance of the smart contract ecosystem. However, existing signature-based methods and code representation learning-based methods suffer from limitations such as heavy-weight program analysis payloads or suboptimal contract encodings.

Objective:

This paper aims to design a fully unsupervised language model for better capturing the syntactic and semantic richness of Solidity code, and utilizes it for advancing the effectiveness of smart contract similarity analysis.

Methods:

Inspired by the impressive semantic learning capability of pre-trained language models (PLMs), we propose SolBERT, a PLM specifically tailored for enhancing Solidity smart contracts similarity detection. To ensure it produces high-quality encodings, SolBERT leverages BERT-style pre-training with the masked language modeling (MLM) and token type prediction (TTP) tasks applied on code-structure-aware token sequences derived from the contracts’ abstract syntax trees (ASTs) through structure-retaining tree linearization and light-weight normalization to learn a base model. On this basis, self-supervised contrastive fine-tuning and unsupervised whitening operations are further performed to optimize contract encoding generation.

Results:

Experiments are conducted on three contract similarity-related tasks, including contract clone detection, bug detection, and code clustering. The results indicate that SolBERT significantly outperforms state-of-the-art approaches with average absolute gains of 21.33% and 21.50% in terms of F1, and 17.78% and 26.60% in terms of accuracy for the clone detection and bug detection tasks, respectively; and an average absolute gain of 17.97% for code clustering task. When applying both contrastive fine-tuning and whitening optimizations, SolBERT also shows superior performance than the case of lacking any of them.

Conclusion:

The proposed approach, SolBERT, can serve as a reliable and powerful smart contract encoder, better capturing the syntactic and semantic aspects of the Solidity code. The results and findings also validate the effectiveness and positive synergistic effect of SolBERT’s encoding optimization operations.