Towards evidence-aware retrieval-augmented generation via self-corrective chain-of-thought

Abstract

To address challenges in reconciling static internal knowledge of large language models (LLMs) with dynamic external information without sacrificing inference efficiency, we propose SC-RAG (self-corrective retrieval-augmented generation). This novel framework introduces evidence extraction using a hybrid retriever (combining semantic and unsupervised aspect-based retrieval for enhanced knowledge quality) and an evidence-aware self-correction mechanism via chain-of-thought (CoT) to activate relevant internal LLM knowledge. Experiments conducted on the LaMP (comprising a total of 7 datasets and 144k samples) and HotpotQA (comprising 113k samples) benchmarks demonstrate that SC-RAG significantly outperforms current state-of-the-art methods by 1.0% to 30.3% across various evaluation metrics. Furthermore, SC-RAG achieves these improvements while concurrently reducing inference time by up to 14.3%, offering a more efficient and accurate solution for retrieval-augmented generation.