Vision-Based Fatigue Detection In Drivers Using Multi-Facial Feature Fusion

Abstract

Fatigued driving has been reported to be a major cause of road accidents claiming millions of lives worldwide. Studies have shown that most road accidents occur either at night or early morning when the driver is already fatigued and there is insufficient light to notice obstacles. Some of the automated fatigue detection systems use physiological signals like EEG, ECG, and blood pressure movements. But, in most cases, the invasive nature of obtaining these signals makes them non-ideal. The recently developed computer vision based fatigue detection systems are too bulky or have limited accuracy due to prediction using single facial features or low-light conditions. Hence, the proposed method first enhances low-light images by improving the overall saturation and creating a uniform image using Gamma Correction. The enhanced images are then fed to a modified Multi-Task Cascaded Convolutional Neural Network for face detection and facial landmark extraction. Finally, the extracted eye state and mouth state features are fed to the LSTM network for fatigue classification. The output of this model decides whether the driver is fatigued or alert. The Mirror subset of the publicly available YawDD data set has been used for effective training and evaluation of the proposed model. The model achieved an exceptionally high F1 score of 0.98 and a Recall of 0.99 on the validation set.