Joint Global and Local Feature Learning Based on Facial StO2 for Stress Recognition

Stress is an emotional state that is inevitable in social life, it is of great importance to accurately identify different types of stress. In this paper, we use human facial tissue oxygen saturation (StO2) to identify four states of an individual’s baseline, emotional stress, high-intensity physical stress and low-intensity physical stress. For the stress classification, we proposed a network called GLNet that combines global and local StO2 features. Specifically, GLNet learns local features from facial regions that provide the effective stress information and global features from the whole face. Then, the two features are fused at decision-level and classified. In addition, a new data augmentation method was proposed to address the data imbalance, which generates new data by constructing a mixed network called MixNet that fuses the depth features of multiple data. Experimental results show that MixNet can effectively alleviate the problem of data imbalance and GLNet combined with the data expanded by MixNet achieved the best performance compared to previous methods of StO2 stress recognition.