Pedal error prediction by driver foot gesture analysis: A vision-based inquiry

Pedal errors have been reported as a cause of fatal traffic accidents. However it is not well understood why and when these pedal errors happen as well as how to mitigate them. In this paper, we study pedal error events in a real-world stop-and-go driving experiment, in which we quantitatively analyze foot behavior with measurements from embedded vehicle sensors (e.g. brake or acceleration pedal state) as well as a video input looking at the driver's foot. Our analysis shows some initial insights in factors influencing pedal errors (beside other possible causes like driver age, gender, and driver workload), including Sequential Effects and Cue Modality. We also develop a new vision-based approach for driver foot behavior analysis and use it to predict brake and acceleration pedal presses. Experimental results over twelve different subjects show that the proposed approach correctly detects pedal misapplications approximately 200ms before the actual press. This indicates the potential of applying this approach to predict and mitigate pedal errors in real-world driving.