How robust are linguistic markers of aging? The case of aging-related social media text

Abstract

Recent research suggests that linguistic biomarkers are effective in detecting probable cognitive impairment (PCI). This study extends this line of research by examining whether linguistic markers, which have been previously validated in structured, controlled clinical settings, are sufficiently robust to effectively detect PCI in unstructured, spontaneous, everyday social media posts. We hypothesized that classic linguistic markers, along with novel large language model (LLM)-derived measures, can robustly, reliably differentiate PCI from healthy controls (HC) based on social media posts. Both clinical interviews and social media posts were preprocessed to extract relevant content. We constructed natural language processing (NLP) pipelines to compute linguistic markers, and used LLMs to derive text similarity and surprisal metrics. Machine learning (ML) methods with explicit linguistic features were then used for predictive modeling. Additionally, we prompted LLM without explicit linguistic features to infer and detect PCI using zero-shot method. ML classifiers, especially gradient boosting, achieved high precision, recall, and overall accuracy in distinguishing PCI from HC across both data types (i.e., clinical interviews and social media posts). We also found that LLM-derived features and prompting methods provided moderate additional insights, and classic linguistic markers remained the most influential predictors. These findings demonstrated a reliable approach to transfer linguistic markers from clinical, controlled settings to spontaneous, unstructured social media posts. This study adds to the growing evidence that robust, AI-enhanced linguistic markers can advance early, noninvasive detection for cognitive decline in everyday language settings of social media posts.