Defining the problem: The impact of OCR quality on retrieval-augmented generation performance and strategies for improvement

Abstract

Despite considerable progress in Retrieval-Augmented Generation (RAG) and Optical Character Recognition (OCR) technologies, only a limited amount of research has examined how OCR-derived data influences RAG performance. Thus, this study presents a document-based question-answering dataset derived from unstructured image documents across financial domains and investigates the impact of OCR-generated data on RAG outcomes. Although high OCR accuracy was achieved, especially for handwritten content, using raw OCR outputs directly in the RAG substantially increased the error rates. To address this, we propose a simple yet effective method of transforming OCR outputs into a structured tabular format, with the results showing a marked improvement in RAG performance without altering the OCR quality. The approach proved robust in correcting OCR errors, representing data in structured formats, and integrating alternative retriever and reranker techniques, and highlighted that RAG performance is more sensitive to the structure of input data than to OCR accuracy alone. This study presents a practical solution for optimizing RAG systems by utilizing structured representations of OCR-extracted data, thereby providing new insights for integrating OCR and RAG.