Driver's Companion-Drowsiness Detection and Emotion Based Music Recommendation System

Recent advancements and development in Deep Learning models and frameworks have opened doors to many possible applications of the same in order to tackle much more complex problems. In this paper, we tackle two major problems-drowsiness detection and emotion detection and create a combined system that can detect both emotional and physical state of user, and respond appropriately by alerting the user when they are showing signs of fatigue, and detecting their emotions in order to recommend appropriate entertainment in the form of music that can positively influence their experience. The driver drowsiness model includes the use of face landmark shape predictor and dlib library to detect the state of eyes in real time. If the eyelid is left closed for a few seconds, an alert is generated. For expression detection, a Convolutional Neural Network (CNN) trained on the FER2013 dataset is used for feature extraction and classification to one of 7 emotions. The music recommendation model is built based on Russell's model which classifies emotions based on valence and energy values. We successfully created a system that can accurately judge drowsiness in user as well as detect emotion with an accuracy of 83% and recommend songs based on user's emotional state.