Roads Infrastructure Digital Twin: Advancing Situational Awareness Through Bandwidth-Aware 360° Video Streaming and Multi-View Clustering

Abstract

Future-facing cities increasingly integrate smart and autonomous objects for their smooth functioning and operations, which ultimately benefit city dwellers and the ecosystem at large. In such highly complex and digital environments, the increased situational awareness is very important for the safety of road participants. In this paper, we propose a new bandwidth-aware framework that maximizes the situational awareness of a given region, using mobile digital boxes and $360^{\circ }$ cameras, mounted on connected vehicles, taking into account the constrained uplink capacity. The proposed framework leverages the multi-view spectral clustering approach and the K-Means++ algorithms to ensure efficient clustering of vehicles based on their GPS coordinates. The clustering step is crucial for larger spatial coverage and, thus, higher situational awareness. Vehicle selection and video quality attribution, under limited uplink constraints, are then performed per cluster to fairly cover the region. Extensive simulations and comparisons against state-of-the-art solutions have been conducted to evaluate the performance of the proposed framework, in terms of region coverage rate and normalized mutual information score, at both small- and large-scale deployments. The results obtained demonstrate the superiority of the proposed approach.