EntroLLM: Leveraging Entropy and Large Language Model Embeddings for Enhanced Risk Prediction with Wearable Device Data.

Abstract

Wearable devices collect complex structured data with high-dimensional and time-series features that are challenging for traditional models to handle efficiently. We propose EntroLLM, a new method that combines entropy measures and the low-dimensional representation (embedding) generated from large language models (LLMs) to enhance risk prediction using wearable device data. In EntroLLM, the entropy quantifies the variability of a subject's physical activity patterns, while the LLM embedding approximates the latent temporal structure. We evaluate the feasibility and performance of EntroLLM using NHANES data to predict overweight status using demographics and physical activity collected from wearable devices. Results show that combining entropy with GPT-based embedding improves model performance compared to baseline models and other embedding techniques, leading to an average increase in AUC from 0.56 to 0.64. EntroLLM showcases the potential of combining entropy and LLM-based embedding and offers a promising approach to wearable device data analysis for predicting health outcomes.