Retrieval-based adaptive fusion strategy for medical report generation

Abstract

Background and Objective:

Retrieval-based medical report generation methods attempt to improve efficiency by reusing historical reports, but their fixed feature-concatenation strategies often introduce cross-case redundancy. Moreover, most methods are designed for low-resolution X-ray images, and their evaluation metrics rely on textual similarity and overlook the implications of misdescription. To address these, we first constructed a high-resolution CT-report dataset, comprising 9 categories of chest CT scans and corresponding reports from 505 patients. Then, we propose RAFS, a retrieval-based adaptive fusion strategy, to dynamically balance contributions from generation and retrieval modules. Finally, we propose DICE, a dual-perspective integrated clinical evaluation including consensus-based positive scoring and penalties of misdescription.

Methods:

RAFS integrates an attention module to calculate the similarity between the current generated word’s hidden state and the retrieved text, passing the result through a fully connected layer to obtain retrieval probabilities. After, obtained attention weights are feed the Sigmoid function and its result for fusing the generation probabilities and retrieval probabilities.

Results:

RAFS achieves superior performance with BLEU-4, METEOR, ROUGE_L, CIDEr and the average of DICE scores of 45.8, 32.9, 59.1, 79.3 and 64.6 in the CT report generation task, outperforming existing methods. methods.

Conclusion:

RAFS significantly enhances the clinical interpretability of generated reports, with future work dedicated to optimizing the characterization of local pathological lesions.