Accurate diagnosis of non-small cell and small cell lung cancer by using machine learning models trained with physical science features extracted from pathological images

Abstract

Background: Accurate differentiation between non-small cell lung cancer (NSCLC) and small cell lung cancer (SCLC) is crucial for optimising treatment strategies and improving patient outcomes in lung cancer management. Early and precise classification supports tailored therapeutic decisions and enhances prognosis prediction.

Purpose: This study develops a novel method to use machine learning models trained with physical science features extracted from pathological images to classify lung cancer into NSCLC and SCLC with high accuracy and robustness.

Methods: Physical science features were employed to acquire quantitative cellular microarchitecture of cancer cells from histopathological images. Random Forest algorithm was applied to identify the most informative 20 features. Next, the selected top features were used to train and evaluate four machine learning classifiers: Support Vector Machine (SVM), Gradient Boosting, Logistic Regression, and Decision Tree. The dataset included pathological images from 240 histologically confirmed lung cancer cases, divided randomly into training and validation sets (80% train, 20% test). Then, model performance was evaluated using accuracy, recall, F1 score, and area under the receiver operating characteristic curve (AUC), with robustness validation via fivefold cross-validation.

Results: Logistic Regression achieved the highest overall performance, with a median accuracy near 90% and an AUC consistently above 0.90 across fivefold cross-validation. SVM and Gradient Boosting followed closely, each surpassing 0.90 in AUC, demonstrating reliable discrimination between NSCLC and SCLC. Decision Tree showed broader variability, though it maintained acceptable recall for SCLC. Random Forest feature selection revealed refractive index percentiles and polarisation histograms as top contributors to model performance.

Conclusions: Machine learning models trained on physical science features have the potential to serve as a highly accurate and robust framework for differentiating NSCLC from SCLC.