Attention-Based Dual-Knowledge Distillation for Alzheimer’s Disease Stage Detection Using MRI Scans

Abstract

This study presents an efficient attention-guided dual-knowledge distillation (D-KD) framework for classifying Alzheimer’s disease (AD) stages using magnetic resonance imaging (MRI) scans based on detection of the subtle anatomical differences. Current challenges involve identifying precise discriminating features in low computational complexity without compromising classification accuracy. In this work, a dual-teacher model consisting of vision transformer (ViT) and swin transformer (ST) for capturing global and local features, respectively, is utilized to distill comprehensive knowledge into a lightweight ViT-based student model, ensuring accurate classification efficacy with reduced computational demands. For validation of the proposed idea, two well-known benchmark MRI datasets, Alzheimer’s Disease Neuroimaging Initiative (ADNI) and AIBL, have been considered for multiclass classification, using an online-training knowledge distillation approach, where teacher and student networks are trained concurrently. The proposed model has achieved accuracies (Ac) up to 98.24% and 97.07% on ADNI and AIBL, respectively, with a significant performance improvement of 15.6% with respect to existing works. The analysis shows that by leveraging the complementary strengths of ViT and ST, the D-KD strategy enhances generalization in data-limited scenarios and provides a reliable, resource-efficient solution for MRI-based AD diagnosis.