Novel application of deep learning to evaluate conversations from a mental health text support service

Abstract

The Shout text support service supports individuals experiencing mental health distress through anonymous text conversations. As one of the first research projects on the Shout dataset and one of the first significant attempts to apply advanced deep learning to a text messaging service, this project is a proof-of-concept demonstrating the potential of using deep learning to text messages. Several areas of interest to Shout are identifying texter characteristics, emphasising high suicide-risk participants, and understanding what can make conversations helpful to texters. Therefore, from a mental health perspective, we look at (1) characterising texter demographics strictly based on the vocabulary used throughout the conversation, (2) predicting an individual’s risk of suicide or self-harm, and (3) assessing conversation success by developing robust outcome metrics. To fulfil these aims, a series of Machine Learning models were trained using data from post-conversation surveys to predict the different levels of suicide risk, whether a conversation was helpful, and texter characteristics, such as demographic information. The results show that language models based on Deep Learning significantly improve understanding of this highly subjective dataset. We compare traditional methods and basic meta-features with the latest developments in Transformer-based architectures and showcase the advantages of mental health research.