Semantic Tagging of CAN and Dash Camera Data from Naturalistic Drives

Abstract

The goal of this paper is to automate the creation of naturalistic driving data sets of dash camera footage that is tagged with information captured from the vehicle's Controller Area Network (CAN) bus, using only a standard dash camera and CAN reader. The paper describes pairing and synchronizing dash camera videos with CAN bus data gathered from a vehicle with advanced driver assistance features. That data is then used to label the dash camera videos with telemetric information. Further, with the synchronized videos and CAN bus data, it is possible to identify video clips with meaningful events such as following a lead vehicle, cars passing in front of the vehicle, braking, turns, etc. This method of data-gathering and data set creation is significantly cheaper and more scalable than other driving data sets, while having competitive quality in terms of telemetric attributes. This could significantly increase the quantity, diversity, and in turn, quality of driving data sets in the future.