Establishing the temporal stability of machine learning models that detect online gambling-related harms

Abstract

Artificial Intelligence (AI) models can detect at-risk online gamblers by analyzing patterns in their betting behaviour, but their performance over time has not been assessed. Linking online gamblers’ self-reported gambling problems to their transactional gambling data, we investigated the temporal stability of online gambling behaviours at two timepoints (2019–2022). We then assessed the impacts of shifting gambling behaviours on the performance of two previously-validated AI harm detection models.

Adult users of a Canadian online gambling website (n₂₀₁₉ = 9,145, n₂₀₂₂ = 11,258) completed the Problem Gambling Severity Index (PGSI), a validated questionnaire examining past-year gambling problems. Population Stability Index assessed the temporal stability of 10 previously-validated indicators of problematic online gambling. Two AI models were then revalidated using overall and threshold-dependent performance metrics.

All measured indicators of problematic online gambling behaviour shifted significantly from 2019 to 2022 (all p < 0.001). Existing AI models showed significant changes in Area Under the Precision-Recall Curve (PGSI 5+ ΔAUPRC = +2.87%; t(20401) = 2.83, p = 0.004; PGSI 8+ ΔAUPRC = +7.06%; t(20401) = 7.21, p < 0.001). Adjusting these models’ decision thresholds realigned their classification performance with originally-validated levels. These results support the use of AI for online gambling harm detection.