Fuzzy performance estimation of real-world driver’s stress recognition models based on physiological signals and deep learning approach

Driver’s mental stress is known as a prime factor in road crashes. The devastation of these crashes often results in losses of humans, vehicles, and infrastructure. Likewise, persistent mental stress could develop mental, cardiovascular, and abdominal disorders. Preceding research in this domain mostly focuses on feature engineering and conventional machine learning (ML) approaches. These approaches recognize different stress levels based on handcrafted features extracted from various modalities including physiological, physical, and contextual data. Acquiring the good quality features from these modalities using feature engineering is often a difficult job. The recent developments in the form of deep learning (DL) algorithms have relieved feature engineering by automatically extracting and learning resilient features. Conventional DL models, however, frequently over-fit due to large number of parameters. Thus, large networks face gradient vanishing issues causing an increase in learning failure and generalization errors. Furthermore, it is often hard to acquire a large dataset for training a deep learning model from scratch. To overcome these problems for driver’s stress recognition domain, this paper proposes fast and computationally efficient deep transfer learning models based on Xception pre-trained neural networks. These models classify the driver’s Low, Medium, and High stress levels through electrocardiogram (ECG), heart rate (HR), galvanic skin response (GSR), electromyogram (EMG), and respiration (RESP) signals. Continuous Wavelet Transform (CWT) acquires the scalograms for ECG, HR, GSR, EMG, and RESP signals separately. Then unimodal Xception models are trained based on these scalograms to classify the three stress levels. The proposed Xception models have achieved 97.2%, 86.4%, 82.7%, 71.9%, and 68.9% average validation accuracies based on ECG, RESP, HR, GSR, and EMG signals, respectively. The fuzzy EDAS (evaluation based on distance from average solution) approach also evaluates the performance of proposed models based on accuracy, recall, precision, F-score, and specificity. For the driver’s three stress levels, fuzzy EDAS performance estimation shows that the proposed ECG, RESP, and HR based Xception models achieved 1st, 2nd, and 3rd positions, respectively.