Multi-modal clinical AI: An edge-aware interpretable framework fusing textual-temporal data

Abstract

Modern clinical AI faces growing challenges in integrating high-frequency features from clinical sensors with unstructured text under latency-sensitive and localized data processing requirements. In response, this paper introduces a unified, lightweight edge-centric framework designed for next-generation clinical AIoT that represents a fundamental paradigm shift from architectures that inherently conflate independent feature dynamics. Initially pre-trained on a cloud layer, the system utilizes a novel deep learning mechanism that decomposes recurrent processing into independent units to effectively trace the memory of a single variable while capturing global semantic guidance. Consequently, this design allows the model to jointly reason over textual and temporal data, capturing short and long-term dependencies without informational dilution. Furthermore, to ensure robust deployment, we incorporate a favorable, effective validation mechanism to certify the structural fidelity of representations generated at the edge. Ultimately, a retrained lightweight encoder facilitates early detection and rapid local inference close to the data source. This localized processing and the effective data compression reduce the communication overhead and restrict raw data exposure. Subsequently, the deployed decoder drives scalable deep fusion and model updates, achieving significant performance with accuracy, precision, recall, and F1 metric values of (0.921, 0.920, 0.910, 0.915) for hypertension, (0.932, 0.910, 0.920, 0.915) for diabetes, and (0.925, 0.920, 0.920, 0.910) for heart attack.