Automatic colorectal cancer detection using machine learning and deep learning based on feature selection in histopathological images

Abstract

Colorectal cancer (CRC) accounts for 10% of global cancer cases and is the third most prevalent type, with a significant increase anticipated in the coming years. This trend underscores the need for precise diagnostics, as effective treatment depends on accurate histopathological analysis of hematoxylin and eosin (H&E) stained biopsies. However, manual evaluation of biopsies is labor-intensive and prone to errors due to staining variations and inconsistencies, complicating the work of pathologists. To address these challenges, advanced automated image analysis, incorporating deep learning (DL) and machine learning (ML) techniques, has substantially improved computer-aided diagnosis systems. This paper proposes a composite model that combines DL and ML to enhance the accuracy of CRC diagnosis. The model aims to increase diagnostic precision, reduce computational complexity, and prevent overfitting for reliable performance. It employs a cascaded design involving feature extraction with MobileNetV2 and DenseNet121 using transfer learning (TL), dataset balancing via the Synthetic Minority Over-sampling Technique (SMOTE), key feature selection through a Chi-square test, and classification by ML algorithms with hyperparameter tuning. The proposed model demonstrates superior performance on the Extended Bioimaging Histopathological Image Segmentation (EBHI-Seg) and multi-class datasets, achieving high accuracy, precision, recall, F1-score, and area under the curve (AUC), demonstrating that the suggested model is superior to other methods already in use¹.