An Explainable Machine Learning Approach to Predict Suicidal Thoughts in University Students: The SMS-ME Project

Abstract

The increasing prevalence of suicidal ideation among university students, especially those enrolled in medical and healthcare courses, necessitates innovative approaches for early detection and prevention. The southern part of Italy presents a distinct cultural, social, and economic context and is under-represented in research. This study leverages the capabilities of machine learning methods and explainable artificial intelligence (XAI) to address this challenge, developing a predictive model for suicidal thoughts within a large sample of university students in Sicily, the southernmost and largest Italian region. The XGBoost algorithm was implemented for model training on survey data from 3278 students, including sociodemographic factors, academic performance metrics, and mental health indicators such as the Depression, Anxiety, and Stress Scale short version (DASS-21) and the Eating Attitude Test short version (EAT-26). The optimized XGBoost achieved promising performance, with AUROC ranging from 0.719 to 0.808 and balanced accuracy between 0.666 and 0.780 across internal and external test sets. Probability calibration was generally reasonable (Brier 0.151–0.185; ECE 0.035–0.120), supporting the reliability of predicted risk estimates. Performance in general varied by degree program, suggesting cohort-specific characteristics. Our findings underscore the potential of machine learning models to identify at-risk students and XAI for understating model prediction, emphasizing the importance of early intervention and targeted mental health support in university settings. The model′s performance and explainability highlight the potential in developing a student-centric and self-assessment framework while helping to mitigate potential ethical concerns associated with predictive mental health tools.