Robust vs. Non-robust radiomic features: the quest for optimal machine learning models using phantom and clinical studies.

Purpose: This study aimed to select robust features against lung motion in a phantom study and use them as input to feature selection algorithms and machine learning classifiers in a clinical study to predict the lymphovascular invasion (LVI) of non-small cell lung cancer (NSCLC). The results of robust features were also compared with conventional techniques without considering the robustness of radiomic features.

Methods: An in-house developed lung phantom was developed with two 22mm lesion sizes based on a clinical study. A specific motor was built to simulate motion in two orthogonal directions. Lesions of both clinical and phantom studies were segmented using a Fuzzy C-means-based segmentation algorithm. After inducing motion and extracting 105 radiomic features in 4 feature sets, including shape, first-, second-, and higher-order statistics features from each region of interest (ROI) of the phantom image, statistical analyses were performed to select robust features against motion. Subsequently, these robust features and a total of 105 radiomic features were extracted from 126 clinical data. Various feature selection (FS) and multiple machine learning (ML) classifiers were implemented to predict the LVI of NSCLC, followed by comparing the results of predicting LVI using robust features with common conventional techniques not considering the robustness of radiomic features.

Results: Our results demonstrated that selecting robust features as input to FS algorithms and ML classifiers surges the sensitivity, which has a gentle negative effect on the accuracy and the area under the curve (AUC) of predictions compared with commonly used methods in 12 of 15 outcomes. The top performance of the LVI prediction was achieved by the NB classifier and RFE FS without considering the robustness of radiomic features with 95% area under the curve of AUC, 67% accuracy, and 100% sensitivity. Moreover, the top performance of the LVI prediction using robust features belonged to the NB classifier and Boruta feature selection with 92% AUC, 86% accuracy, and 100% sensitivity.

Conclusion: Robustness over various influential factors is critical and should be considered in a radiomic study. Selecting robust features is a solution to overcome the low reproducibility of radiomic features. Although setting robust features against motion in a phantom study has a minor negative impact on the accuracy and AUC of LVI prediction, it boosts the sensitivity of prediction to a large extent.