TranSenseFusers: A temporal CNN-Transformer neural network family for explainable PPG-based stress detection

Stress is a common everyday emotional state in modern society contributing to both physical and mental illnesses. Thus, detecting and managing the degree of stress is crucial to improve well-being. Wearable devices equipped with biosensors, such as PhotoPlethysmoGraphy (PPG), can measure reliably a person’s affective state. However, PPG-based approaches suffer from the presence of Motion Artifacts (MA) affecting their overall performance. Classical machine learning and deep learning approaches have been proposed over the years for PPG-based stress detection, exploiting signal processing techniques to remove the recorded noise, but lack explainability or their performance fails to generalize across subjects. In the current work, we present a novel architecture, TranSenseFuser comprised of temporal convolutions followed by feature-level or sequence-level multi-head attention to improve sensor fusion’s effectiveness and exploit the provided attention maps as a form of explainability. The developed models are evaluated on highly benchmarked public dataset, namely WESAD, achieving state-of-the-art results (up to 98.46% accuracy and 97.03% F1-score) using different window sizes and cross-validation set-ups. Moreover, we demonstrate the explainability of the model towards filtering out the motion artifacts by visualizing the obtained attention maps and quantify the performance of this artifact segmentation feature in a zeros-shot manner.