Robust and explainable framework to address data scarcity in diagnostic imaging

Abstract

Deep learning has significantly advanced automatic medical diagnostics, releasing human resources from clinical pressure, yet the persistent challenge of data scarcity in this area hampers its further improvements and applications. To address this gap, we introduce a novel ensemble framework called ‘Efficient Transfer and Self-supervised Learning based Ensemble Framework’ (ETSEF). ETSEF leverages features from multiple pre-trained deep learning models to efficiently learn powerful representations from a limited number of data samples. To the best of our knowledge, ETSEF is the first strategy that combines two pre-training methodologies (Transfer Learning and Self-supervised Learning) with ensemble learning approaches. Various data enhancement techniques, including data augmentation, feature fusion, feature selection, and decision fusion, have also been deployed to maximise the efficiency and robustness of the ETSEF model. Five independent medical imaging tasks, including endoscopy, breast cancer detection, monkeypox detection, brain tumour detection, and glaucoma detection, were tested to demonstrate ETSEF’s effectiveness and robustness. Facing limited sample numbers and challenging medical tasks, ETSEF has demonstrated its effectiveness by improving diagnostic accuracy by up to 13.3% compared to strong ensemble baseline models and up to 14.4% compared with recent state-of-the-art methods. Moreover, we emphasise the robustness and trustworthiness of the ETSEF method through various vision-explainable artificial intelligence techniques, including Grad-CAM, SHAP, and t-SNE. Compared to large-scale deep learning models, ETSEF can be flexibly deployed and maintain superior performance for challenging medical imaging tasks, demonstrating potential for application in areas lacking training data. The code is available at Github ETSEF.