Development and internal validation of machine learning models for identifying suicidal behavior in inpatients with mood disorders.

Abstract

Background: Suicidal behavior (SB) is a major concern in mood disorders. Models developed from retrospective records are vulnerable to ambiguous targets, data leakage, and inappropriate handling of class imbalance. We developed and internally validated a model to identify the risk of suicidal behavior in patients with mood disorders upon admission.

Methods: The dataset included 1, 099 inpatients with ICD-10 mood disorders admitted to Beijing Anding Hospital in 2022. Exclusion of 20 records with more than 20% row-level missingness and one record without an outcome left 1, 078 patients, including 126 with SB. Current SB comprised documented suicidal ideation, planning, or attempt within the three months before index admission, including the admission assessment. Previous SB and previous suicide attempt (SA) referred to events more than three months before admission. Elastic Net selected 14 predictors from 65 candidates in the development set. We compared 13 classifiers from six model families. All preprocessing and resampling were confined to training folds, and evaluation used only observed patients. Assessment included repeated fivefold cross-validation, probability calibration, bootstrap confidence intervals, precision-recall and decision-curve analyses, clinical baselines, and sensitivity analyses.

Results: The analytic cohort comprised 952 SB-negative and 126 SB-positive patients (11.69%). The development set contained 864 patients (97 SB-positive), and the observed test set contained 214 patients (29 SB-positive). Calibrated linear discriminant analysis (LDA) provided the most consistent balance of discrimination and calibration. In repeated cross-validation, mean AUC was 0.721 ± 0.052, mean AUPRC was 0.268 ± 0.060, and mean Brier score was 0.095. In the observed test set, AUC was 0.704 (0.597-0.803), AUPRC was 0.283 (0.198-0.453), and Brier score was 0.111. At the training-derived threshold of 0.111, sensitivity was 0.552 and specificity was 0.659. The leading SHAP contributions were onset-polarity category, previous SA, age, and previous SB, but all attributions were associational.

Conclusions: Model performance was acceptable. These internally validated findings are exploratory and require prospective assessment, standardized outcome measurement, and external validation before clinical use.