15 Years of optimizers in medical deep learning: A systematic review

Abstract

Optimization algorithms are pivotal in training deep learning (DL) models for medical imaging, determining how efficiently models learn, generalize, and perform across modalities. This systematic review analyzed 69 peer-reviewed studies (2010–2025) on optimizer performance in classification, segmentation, and object detection tasks using MRI, CT, X-ray, ultrasound, histopathology, and ECG data, following PRISMA 2020 guidelines. Adaptive optimizers such as Adam and its variants were most common, offering rapid convergence in CNN-based classification, whereas SGD and momentum-based methods yielded stronger generalization in large-scale segmentation. Emerging techniques—Sharpness-Aware Minimization (SAM), Ranger, and AdamW—improved robustness under domain shift or noisy conditions. Hybrid and metaheuristic optimizers provided marginal accuracy gains but at higher computational cost. Common limitations included inconsistent hyperparameter reporting, limited external validation, and dataset bias toward North American cohorts. Optimizer effectiveness was found to be task- and architecture-dependent: adaptive methods suit small or noisy datasets, while momentum-based and hybrid methods enhance generalization for complex imaging. Future studies should emphasize standardized evaluation, transparent reporting, and diverse data to enable equitable and reproducible deployment of medical AI.