DC-RRG: Diagnosis-centered cascaded radiology report generation

Radiology report generation (RRG) aims to automatically generate clinical reports from chest Xray images, offering potential benefits in aiding diagnosis and reducing clinician workload. However, this task remains challenging due to the need for generating diagnostically accurate and comprehensive reports. Previous methods neglect explicit and fine-grained diagnosis guidance, which is critical for addressing these challenges. To this end, we introduce a novel Diagnosis Centered method for Radiology Report Generation (DC-RRG), explicitly incorporating diagnostic observations to guide report generation. Specifically, our method utilizes a cascaded inference framework, initially using a large language model to generate diagnoses, which subsequently guide the generation of comprehensive reports. We design multi-modal prompts (i.e., textual instructions, visual features, and medical knowledge features) to extend the Multi-Modal Large Language Model (MLLM) to generate accurate and comprehensive diagnostic reports. Additionally, we also develop a progressive training strategy that aligns modalities in two stages: first using reports for coarse clinical priors, then using diagnoses for fine-grained details. Extensive experiments on MIMIC-CXR and IU-Xray datasets demonstrate that DC-RRG surpasses all previous methods and achieves new state-of-the-art results.