Evaluation Framework for Smart Contract Fuzzers

With the widespread application of smart contracts in economics and asset management, the security of smart contracts has been widely addressed by academia and industry. Fuzz is an effective technique for vulnerability detection. Several fuzzers are currently available for smart contracts, how to choose the most appropriate tools to test smart contracts is a problem that needs to be solved. To this end, we propose an evaluation framework for a smart contract fuzzers, which sets eight evaluation indicators from five aspects to comprehensively evaluate the usability, transparency, detection ability, branch coverage, and design of oracle of the smart contract fuzzers. In order to verify the scientificity and rationality of the framework, we selected six state-of-the-art (SOTA) smart contract fuzzers for evaluation. By evaluating the usability of six fuzzers, the level of difficulty in using them was verified; by evaluating the transparency of six fuzzers, the usability of the tool's output information during use was verified; the branch coverage and rationality of oracle design of the six fuzzers was validated by evaluating their detection ability on the dataset. The final evaluation results validated the effectiveness of our proposed framework in guiding users to choose smart contract fuzzers.