Retriever-generator-verification: A novel approach to enhancing factual coherence in open-domain question answering

Abstract

In recent research on open-domain question answering (ODQA), significant advances have been achieved by merging document retrieval techniques with large language models (LLMs) to answer questions. However, current ODQA methods present two challenges: the introduction of noise during retrieval and granularity errors during generation. To address these challenges, we propose the Retriever-Generator-Verification (RGV) framework, which consists of the Evidence Document Generator (EDG), the Candidate Entities Generator (CEG), and the Candidate Subgraphs Validator and Ranker (CSVR). EDG combines retrieval and generative mechanisms to construct comprehensive reference documents, ensuring broad coverage of potential answers. CEG then extracts and expands multi-dimensional candidate answer entities from these reference documents, capturing finer-grained information. Finally, CSVR verifies the candidate subgraphs against external knowledge sources and ranks them based on relevance, refining the final answers to enhance their accuracy and reliability. By systematically integrating these components, the RGV framework improves the completeness of retrieved information while effectively mitigating noise during retrieval and granularity errors during generation, thereby enhancing the overall reliability of ODQA. We assessed the efficacy of our method on three widely used datasets, and the experimental results demonstrate that our method exhibits competitive performance in benchmark tests. Compared to the state-of-the-art method, our approach achieves a 2.3% improvement in F1 score on the WebQSP dataset and a 1.3% increase in Hits@1 on the CWQ dataset.