Using computer vision and street-level videos for pedestrian-vehicle tracking and behaviour analysis

Abstract

The extraction and exploitation of Surrogate Safety Measures (SSMs) have gained prominence in recent years, aided by rapid emerging technologies. SSMs can be applied in all aspects of safety, enabling new insights for analysing road user behaviour. In this context, this paper presents an integrated framework for tracking pedestrians and vehicles in complex urban environments, with a focus on analysing behaviours in relation to the traffic light status and the computation of the Time-to-Collision (TTC). Utilising advanced computer vision object detection and the feature extraction models of YOLOv8 and ResNet-50, this framework integrates Kalman filtering, homography transformations, and object re-identification to achieve high accuracy. The data used for this study were roadside video recordings from the Athens centre (Greece). Notably, the results show accuracy rates of 50 % to 70 % in detecting traffic light statuses and identified a 23 % discrepancy on average between manual and automated counts of illegal crossings. One of the key strengths and contributions of the study is the utilisation and transformation of street-level data provided by smartphone camera recordings, which emphasise the ease of transferability of the proposed approach without the requirements of specialised, costly, or heavy equipment. The analysis of pedestrian compliance, particularly during intergreen phases, provides novel knowledge on pedestrian behaviour and highlights opportunities to improve intersection design for safety. This study underscores the potential of computer vision detection systems to provide reliable, real-time data that takes the road network conditions into account, ultimately contributing to safer urban traffic management and informed policy decisions.