Textless, Tiny, and Private: Hashed n-Grams With LLM-Teacher Distillation and Local Differential Privacy for Mental-Health Screening on Social Media

Abstract

We present a textless, compute-light pipeline for user-level mental-health screening on social media that combines salted hashed character $n$-grams, local differential privacy (LDP), and large language model (LLM)-teacher concept distillation into a tiny, calibrated logistic classifier. The encoder stores no vocabulary or raw text; instead, all posts for a user are aggregated on device into a salted hashed presence vector, and a subsampled randomized-response mechanism is applied once per user to provide an explicit user-level LDP guarantee for this representation. To recover interpretability without online LLMs, we invoke frozen LLMs (one for Chinese and one for English) on a small, stratified subset to label clinically grounded concepts (e.g., hopelessness, self-blame, sleep disturbance, and help-seeking); compact students are then trained on hashed features to predict these concepts across the full corpus. On two large Weibo user-level corpora, our Hashed + differential privacy (DP ) + Concepts model achieves calibrated utility competitive with a dense-embedding baseline while retaining a CPU-only footprint. Relative to a hashed baseline without concepts, we observe consistent gains in macro-AUPRC and substantially lower calibration error after isotonic mapping. Privacy audits show that, at our default user-level privacy budget, LDP plus salt rotation reduces user-level membership-inference attack success, resists reconstruction compared to embedding inversion, and substantially lowers cross-deployment linkage area under the curve (AUC) toward chance in our empirical setting. We release code and audit scripts to facilitate responsible reuse.