Categorizing Mental Stress: A Consistency-Focused Benchmarking of ML and DL Models for Multi-Label, Multi-Class Classification via Taxonomy-Driven NLP Techniques

Abstract

Mental stress, a critical concern worldwide, necessitates precise and nuanced characterization. This study introduces a novel approach to effectively characterize mental stress through a multi-label, multi-class classification framework through natural language processing techniques. Building on existing literature, discussions with psychologists and other mental health practitioners, we developed a taxonomy of 27 distinctive markers spread across 4 label categories; aiming to create a preliminary screening tool leveraging textual data.

The core objective is to identify the most suitable model for this complex task, encompassing comprehensive evaluation of various machine learning and deep learning algorithms. we experimented with support vector machines (SVM), random forest (RF) and long short-term memory (LSTM) algorithms incorporating various feature combinations involving Term Frequency-Inverse Document Frequency (TF-IDF) and Latent Dirichlet Allocation (LDA). The best performer of this comparative study was further evaluated against an LLM.

The potential of large language models (LLMs), including their language understanding and prediction capabilities, is another key focus. We explore how these models could augment and advance mental health research, offering new perspectives and insights into the characterization of mental stress.

Our findings show that the top model, an LSTM with TF-IDF and LDA (class weights assigned) outperformed the PaLM model with a coefficient of variation as low as 0.87% across all labels. Despite the PaLM model’s superior average performance, it exhibited higher variability among different labels.