Explainable machine learning analysis of longitudinal mental health trajectories after breast cancer diagnosis

Abstract

Mental health impairment after breast cancer diagnosis may persist for months or years. The present work leverages on novel machine learning techniques to identify distinct trajectories of mental health progression in an 18-month period following BC diagnosis and develop an explainable predictive model of mental health progression using a large list of clinical, sociodemographic and psychological variables. The modelling process was conducted in two phases. The first modeling step included an unsupervised clustering to define the number of trajectory clusters, by means of a longitudinal K-means algorithm. In the second modeling step an explainable ML framework was developed, on the basis of Extreme Gradient Boosting (XGBoost) model and SHAP values, in order to identify the most prominent variables that can discriminate between good and unfavorable mental health progression and to explain how they contribute to model's decisions. The trajectory analysis revealed 5 distinct trajectory groups with the majority of patients following stable good (56%) or improving (21%) trends, while for others mental health levels either deteriorated (12%) or remained at unsatisfactory levels (11%). The model's performance for classifying patient mental health into good and unfavorable progression achieved an AUC of $0.82\pm 0.04$. The top ranking predictors driving the classification task were the higher number of sick leave days, aggressive cancer type (triple-negative) and higher levels of negative affect, anxious preoccupation, helplessness, arm and breast symptoms, as well as lower values of optimism, social and emotional support and lower age.