Intelligent multi-sensor fusion and anomaly detection in vehicles via deep learning

Abstract

Deep learning techniques are predominantly used to identify vehicular events such as harsh cornering, harsh braking, and rapid acceleration by analyzing signal data. However, deploying deep learning models demands high-quality, large-scale data, and the processes of data acquisition, labeling, extraction, and processing are often overlooked in the literature. In this article, we focus on detailed dataset creation, including labeling and feature analysis, alongside the development of AI models. Real-time data collection is conducted on experimental roads using numerous vehicles equipped with AI-enabled edge units. The raw data collected, however, is unsuitable for training deep learning models due to redundant features, noisy attributes, and a lack of labeled anomalous events. To address this, we employ multiple preprocessing and postprocessing techniques to generate high-quality datasets, analyzing the specific impacts of each signal feature on anomalous events. Since real-time collected data lacks labels, a thorough labeling process is required for each data point. An autoencoder-based labeling process is applied to the final dataset, where the autoencoder detects and labels anomalous behaviors based on data timestamps. Following the labeling, a hybrid deep learning model incorporating Convolutional Neural Networks (CNN), Long Short Term Memory (LSTM), attention, and Fully Connected Neural Networks (FCDNN) layers is trained and tested for detecting anomalous driving events. The results demonstrate that the proposed method outperforms the state-of-the-art solutions by reaching high accuracy rates: 99.69% for harsh cornering events and 98.24% for rapid acceleration and harsh braking events, with corresponding F1 scores of 90.14% and 81.22%, respectively.