CM-SQL: A cross-model consistency framework for text-to-SQL

Abstract

In recent years, large language models (LLMs) have been widely applied to the task of Text-to-SQL. Currently, most LLM-based Text-to-SQL methods primarily adopt the following approaches to improve the accuracy of generated SQL: (1) schema linking; and (2) leveraging the model’s self-consistency to check, modify, and select the generated SQL. However, due to issues such as hallucinations in LLMs, the database schema generated during the schema linking phase may contain errors or omissions. On the other hand, LLMs often exhibit overconfidence when evaluating the correctness of their outputs. To address these issues, we propose a cross-model consistency SQL generation framework (CM-SQL), which generates SQL outputs from different perspectives by feeding two database schemas into two LLMs. The framework combines the stability of fine-tuned models with the powerful reasoning capabilities of LLMs to evaluate the generated SQL. Additionally, we propose a local modification strategy to correct erroneous SQL. Finally, the outputs of the evaluation module and the LLM are used to select candidate SQLs, yielding the final SQL. We evaluated the proposed framework on the BIRD dev dataset using GPT-4o-mini and DeepSeek-V2.5, achieving an execution accuracy of 65.65 %. On the test set of the Spider dataset, the execution accuracy reached 87.6%, significantly outperforming most methods based on the same LLMs. Furthermore, our performance is comparable to many approaches that rely on more expensive models, such as GPT-4.