Driving Anomaly Detection Using Contrastive Multiview Coding to Interpret Cause of Anomaly

Modern advanced driver assistant systems (ADAS) rely on various types of sensors to monitor the vehicle status, driver's behaviors and road condition. The multimodal systems in the vehicle include sensors, such as accelerometers, pressure sensors, cameras, lidar and radars. When looking at a given scene with multiple modalities, there should be congruent in-formation among different modalities. Exploring the congruent information across modalities can lead to appealing solutions to create robust multimodal representations. This work proposes an unsupervised approach based on contrastive multiview coding (CMC) to capture the correlations in representations extracted from different modalities, learning a more discriminative rep-resentation space for unsupervised anomaly driving detection. We use CMC to train our model to extract view-invariant factors by maximizing the mutual information between mul-tiple representations from a given view, and increasing the distance of views from unrelated segments. We consider the vehicle driving data, driver's physiological data, and external environment data consisting of distances to nearby pedestrians, bicycles, and vehicles. The experimental results on the driving anomaly dataset (DAD) indicate that the CMC representation is effective for driving anomaly detection. The approach is efficient, scalable and interpretable, where the distances in the contrastive embedding for each view can be used to understand potential causes of the detected anomalies.