Adaptive attention-aware fusion for human-in-the-loop behavioral health detection

Abstract

Identifying behavioral health is paramount for law enforcement officers to provide appropriate follow-up community care. In the current practice, law enforcement offices manually identify these behavioral health cases to allow the designation of the relevant follow-up resources. In this work, we develop a tool to automatically detect behavioral health cases from police public narrative reports by identifying behavioral health indicator signals. We propose a novel adaptive attention-aware fusion model for detecting behavioral health signals in sensitive police reports. Our model leverages contextual and semantic information from the reports and relevant behavioral health cues as keywords from a pre-trained attention-weighted keyword-based model. Our model also employs label self-attention mechanisms to correlate label embeddings with the report and keyword representations. Furthermore, we propose a novel clustering-based uncertainty-enabled informative sampling query strategy to integrate humans-in-the-loop in the active learning framework to reduce required annotation from experts. This querying strategy selects the most informative and diverse samples for expert annotation. Our experimental results showed that the proposed model outperforms state-of-the-art classifiers on a dataset of 300 manually annotated ground truth police reports, achieving an accuracy of 87.58% and an F1-score of 85.67%. Applying our querying strategy to our proposed model increased the detection of behavioral health, achieving an accuracy of 92% and an F1-score of 91.1%. Also, our proposed model achieves an accuracy score of 93.75% and an F1-score of 93.61% on unseen samples. Lastly, our proposed model demonstrates its interpretability by extracting the keywords associated with each behavioral health category.