Enhancing deep learning interpretability for hand-crafted feature-guided histologic image classification via weak-to-strong generalization.

Abstract

Purpose: Deep learning (DL) models have achieved promising performance in histologic whole-slide image analysis for various clinical applications. However, their black-box nature hinders the interpretability of DL features for clinical adoption. By contrast, hand-crafted features (HCFs) directly calculated from images offer strong interpretability but with reduced predictive power of DL models. The relationship between DL features and HCFs remains insufficiently explored. We aim to enhance the interpretability and performance of DL models using a weak-to-strong generalization (WSG) framework that integrates HCFs into the learning process.

Approach: The proposed WSG framework leverages an interpretable and HCF-based "weak" teacher model that supervises a "strong" DL student model to learn and improve itself by generalizing from weaker forms of reasoning to stronger ones, for classification tasks. An adaptive bootstrap WSG loss function is designed to optimize the transfer of knowledge from hand-crafted to deep-learned features, enabling systematic analysis of feature interactions. Innovatively, mutual information (MI) between HCFs and DL features learned by student models is analyzed to gain insights into their correlations and the interpretability of DL features. The framework is evaluated using extensive experiments on three public datasets with diverse combinations of teacher and student models for tumor classification.

Results: The WSG framework achieves consistent improvements in classification performance across all evaluated models. Qualitative saliency-map analysis indicates that WSG supervision enables student models to concentrate on diagnostically relevant regions, thereby improving interpretability. Furthermore, quantitative analysis reveals a notable increase in MI between hand-crafted and deep-learned features following WSG training compared with that without WSG training, indicating more effective integration of expert knowledge into the learned representations.

Conclusions: Our study elucidates the key HCFs that drive DL model predictions in histologic image classification. The findings demonstrate that integrating HCFs into DL model training via the WSG framework can enhance both the interpretability and the model's predictive performance, supporting their broader clinical adoption.