An eXplainable machine learning framework for predicting the impact of pesticide exposure in lung cancer prognosis

Abstract

Lung cancer, the second most prevalent and lethal cancer, is caused by aberrant and uncontrolled cell division in the lungs. Once lung cancer spreads to surrounding tissues or organs, the likelihood of recovery declines; hence, early illness detection is vital. Machine learning has shown significant potential in several healthcare applications. Examining various factors and trends in the data, the machine learning model can predict lung cancer menace by pinpointing those more susceptible to the illness. Among the various causes of lung cancer, pesticide is a major contributor. ‘Pesticide’ refers to any chemical used in agriculture to manage pests like weeds and insects. Numerous health hazards, including the possibility of developing cancer, have been linked to exposure to specific pesticides. Our objective is to obtain the trust of medical professionals and patients depending on how interpretable machine learning models are in healthcare. This paper deals with implementing the proposed study by utilizing a public dataset from a Thai case study to predict the risk of lung cancer caused by pesticide exposure. Since the dataset was highly imbalanced, a hybrid normalization technique was utilized, combining the Synthetic Minority Oversampling Technique (SMOTE) and Edited Nearest Neighbor (ENN). We applied a two-stage feature selection technique combined with Extra Tree Classifier and Principal Component Analysis. An eXplainable XGBoost Classifier is developed to predict lung cancer risk based on pesticide exposure. The robustness of the model is reflected in the results, with accuracy, sensitivity, and F1-Score as 99.00%, 98.87%, and 98.57%, respectively. Two public datasets were utilized to generalize the model, and the model performed well on both datasets. The model achieved accuracy, sensitivity, and F1-Score of 99.00%, 99.00%, and 99.33% on the ‘Lung Cancer Prediction’ dataset. The model is trained and tested on the ‘survey lung cancer’ dataset and obtained an accuracy, sensitivity, and F1-Score of 99.00%, 99.00%, 99.00%, respectively. The proposed model outperformed existing state-of-the-art methodologies regarding quality metrics. An illustration is done on the XAI (eXplainable Artificial Intelligence) model by utilizing SHapley Additive exPlanations (SHAP), thereby identifying the most relevant features contributing to the lung cancer menace.