On-Board Deployability of a Deep Learning-Based System for Distraction and Inattention Detection

Abstract

Driver distraction recognition is gaining increasing interest in improving traffic safety, as well as in automated driving. This article reports the experience we have gained developing a driver distraction detection (DDD) system within the Hi-Drive research project on driving automation. Targeting on-board deployability, we have faced several leading-edge research issues that have not been addressed together in published research works. We propose a compact sensory configuration and a limited computational resource system architecture, also exploiting careful manual and automated labeling, trying to find a tradeoff among conflicting needs in terms of accuracy, privacy preservation, energy efficiency, and costs. Our system detects two levels of insufficient attention, which are keys not only for designing a proper driver warning and information management strategy but also for better managing the transition among different automation levels. Our experiments confirmed on real-world data, and in the three-class task, the importance of distinguishing users among training, validation, and testing to prevent overestimating model performance by overfitting individual participant patterns present in all three subsets. We analyzed the complexity of the three-class problem, which is also related to the relatively low representation of the intermediate distraction class in the dataset. We showed that the size of the classifiable time window is a critical performance factor and found that a 5 s length seems to achieve the best tradeoff between latency, time resolution, and the need for capturing sufficient temporal information to detect distraction. Another empirical finding comes from a SHAP values-based explainability analysis and concerns the importance of vehicular signals for the detection task, particularly in the three-class problem. This is significant, as such signals are inexpensively available in vehicles, and their processing does not add further privacy concerns. Finally, assessing performance on three state-of-the-art embedded platforms, we observed that the developed deep learning models are able to effectively run on limited-resource, on-board deployable devices, meeting real-time performance requirements, also on a mainstream, low-cost microcontroller. We argue that these findings open significant perspectives toward an effective and efficient field deployment of DDD electronic systems.