Enhancing long-form question answering via reflection with question decomposition

Abstract

Long-Form Question Answering (LFQA) requires multi-paragraph responses that explain, contextualize and justify an answer rather than returning a single fact. Large proprietary language models can meet this bar, but privacy, cost and hardware limits often force practitioners to rely on much smaller, locally hosted models — whose outputs are typically shallow or incomplete. We introduce Decomposition-Reflection, a training-free prompting framework that (i) decomposes a user question into the complementary sub-questions, (ii) answers each one, and (iii) runs a lightweight self-reflection loop after every stage to enhance the comprehensiveness, entailment and factuality of the results before synthesizing the final response. Across three LFQA benchmarks, the proposed approach raises ROUGE and LLM-based factuality scores over strong chain-of-thought and self-refinement baselines. Ablation study confirms that removing either decomposition or reflection sharply degrades coverage and entailment, underscoring the importance of both components.