A mixed-scale dynamic attention transformer for pediatric pneumonia diagnosis

Abstract

Pediatric pneumonia is a leading cause of morbidity and mortality in children under five, emphasizing the urgent need for automated diagnostic systems. While deep learning has shown promise in natural image classification, pediatric pneumonia imaging presents unique challenges, such as subtle symptoms, smaller anatomical structures, and the need for fine-grained feature extraction. To address this, We propose a Mixed-Scale Dynamic Attention Transformer aided by large language models (LLMs), which consists of three key modules: (1) Dynamic Local Attention Module: Dynamically focuses on nearby regions with fine-grained attention and applies coarse-grained attention to distant areas, effectively capturing both local and global spatial dependencies. (2) Hierarchical Multi-Scale Unit Module: Integrates and enhances multi-scale channel information, adapting to varying spatial scales to better detect subtle pneumonia-related features. (3) Attention Amplification Module: Leverages a frozen large language model (e.g., GPT, LLaMA) to amplify attention on critical pneumonia features by utilizing its rich semantic insights and global contextual understanding. Evaluations on pediatric chest X-ray datasets, including Pneumonia Physician, Guangzhou Women and Children’s Medical Center, and NIH CXR14, demonstrate the proposed method’s superior performance across key metrics such as accuracy, AUC, precision, recall, and F1-score.