Hallucination detection in LLM code generation: A sampling-based consensus verification approach

Abstract

Large Language Models (LLMs) have revolutionized the code generation task, but their output often contains "hallucinations" - code snippets that look reasonable but are actually wrong (such as API misuse or logic errors). Existing detection methods mainly rely on dynamic code execution, which requires complex runtime environment configurations. This paper proposes HalluCodeDetector, a new static analysis framework based on sampling consistency verification. The method is based on the following assumption: when LLM correctly understands the problem, its random output shows high consistency in syntactic structure, data flow, and API usage patterns. The process of the method is as follows: for a given problem, we let LLM repeatedly generate multiple code samples and evaluate their semantic/functional consistency, a new metric (MRCM) is used to calculate the average similarity between candidate response and other samples to quantify the possibility of hallucination. Experiments on HumanEval+ and MBPP benchmarks demonstrate that HalluCodeDetector achieves AUROC=0.76, outperforming baseline methods like LYNX by 15.2%, and with lower time overhead. Our method provides a secure, efficient, and generalizable solution for improving the reliability of LLM-generated code.