A Multimodal Feature Fusion-Based Method for Individual Depression Detection on Sina Weibo

Abstract

Existing studies have shown that various types of information on the online social network (OSN) can help predict the early stage of depression. However, studies using machine learning methods to accomplish depression detection tasks still do not have high classification performance, suggesting that there is much potential for improvement in their feature engineering. In this paper, we first construct a dataset on Sina Weibo (a leading OSN with the largest number of active users in the Chinese community), namely the Weibo User Depression Detection Dataset (WU3D). It includes more than 10,000 depressed users and 20,000 normal users, both of which are manually labeled and rechecked by specialists. Then, we extract text-based word features using the popular pretrained model XLNet and summarize nine statistical features related to user text, social behavior, and pictures. Moreover, we construct a deep neural network classification model, i.e. Multimodal Feature Fusion Network (MFFN), to fuse the above-extracted features from different information sources and further accomplish the classification task. The experimental results show that our approach achieves an F1-Score of 0.9685 on the test dataset, which has a good performance improvement compared to the existing works. In addition, we verify that our multimodal detecting approach is more robust than multimodel ensemble ones. Our work could also provide new research methods for depression detection on other OSN platforms.