FoRKER: Focused reasoner with knowledge editing and self-reflection

Multi-hop question answering (MHQA) is a complex question answering (QA) benchmark that requires agents to integrate information from diverse sources and utilize cross-referencing reasoning to answer intricate questions. Existing MHQA-handling frameworks typically employ a retrieve-read paradigm. However, these efforts rooted in the retrieve-read paradigm are still constrained by: 1) unstable document retrieval performance, 2) weak knowledge refinement capabilities, and 3) the absence of a reflection mechanism for error awareness. To address these limitations, we propose FoRKER (Focused Reasoner with Knowledge Editing and Self-Reflection), which is a plug-and-play framework. Specifically, we develop a novel progressive focusing mechanism to pinpoint highly relevant document resources and introduce knowledge editing techniques to further eliminate noise interference within textual information. Additionally, we design a novel prompting method, named Chain-of-Evidence (CoE), which is designed to augment the reasoning capabilities of FoRKER. Notably, the integration of Self-Reflection technology further endows FoRKER with the ability to learn and improve from its mistakes. Extensive experiments on widely-used datasets demonstrate that FoRKER achieves new state-of-the-art results in information retrieval and reading comprehension, while also exhibiting effective generalization. Exhilaratingly, on the MusiqueQA dataset, FoRKER demonstrates a 20 % improvement in Answering scores compared to the advanced competitors.