RECoT: Relation-enhanced Chains-of-Thoughts for knowledge-intensive multi-hop questions answering

Abstract

Open Domain question answering is designed to enable a computer to understand and answer any question on a wide range of topics. The prevalent retrieval-reading paradigm helps large language models (LLMs) when retrieving relevant text from external knowledge sources using questions, however the multi-hop question answering approach based on Chains-of-Thoughts (CoT) may perform poorly when it comes to complex questions. This is because there can be errors in generating sentences at each hop, and these errors accumulate, leading to significant deviations in the final result. In order to solve the above problems, this paper first extracted the relational triples of complex problems. Next, triples are used to select the most representative sentence at each step during CoT generation as the query for the next-hop retrieval.

The RECoT with GPT-3 results in significant improvements with F1 score up 5.1 points in downstream QA on 2WikiMultihopQA datasets and up 2.9 points on HotpotQA datasets. In addition, improvements in results can be obtained even with smaller models such as Flan-T5-large without additional training. In conclusion, RECoT reduced model hallucination and accelerated more accurate CoT reasoning to guide retrieval to get improved results. Code is publicly available at https://github.com/XD-BDIV-NLP/RECoT.