A 3D multi-modal multi-scale end-to-end classifier for Alzheimer's disease diagnosis

Abstract

This study presents a novel 3D multi-modal multi-scale end-to-end classifier to enhance Alzheimer's Disease (AD) diagnosis by integrating MRI, PET, age, and MMSE cognitive test scores. Leveraging a ResNet-inspired architecture with trainable multi-scale convolutional scaling, the classifier categorizes subjects into four classes—Normal Control (NC), Stable Mild Cognitive Impairment (sMCI), Progressive Mild Cognitive Impairment (pMCI), and AD—capturing both structural and functional brain pathology. A tailored fusion strategy (MA_PC) processes MRI with age and PET with MMSE in parallel branches, optimizing complementary information use. Extensive experiments using the ADNI dataset, a five-fold cross-validation scheme, and an unseen test set demonstrate that MA_PC with convolutional scaling achieves superior performance, outperforming commonly used fusion strategies as well as pre-trained 3D ResNets designed for medical imaging applications. A comparative analysis reveals that 4-class classification consistently surpasses a 3-class approach (NC, MCI, AD), highlighting the model's ability to distinguish subtle AD progression stages. These findings highlight the critical role of advanced data fusion and scaling methods in enhancing AD diagnosis accuracy and underscore the potential of multi-modal CNNs in advancing medical imaging research.