A Framework for Queryable Video Analysis: A Case-Study on Transport Modelling

Abstract

Analysing video data requires the use of different models trained to retrieve or process data for a particular task. In this paper, we introduce an approach to represent the visual context within a video as queryable information. Through the use of computer vision techniques, we can detect and classify objects. Our system processes these classifications in order to construct a queryable data-set referred to as the real world model. The advantage of this approach is that through the formalisation of the information, we can create generic queries to retrieve information. This approach allows for processing to be done on edge devices such as embedded cameras while only transmitting detected information reducing the transmission bandwidth as well as infrastructural costs. The final recognition data is processed on the cloud. The analysed case study works on video traffic representation - an experiment around the transport domain. We evaluate and validate our approach by posing several queries to the system that generates information on the traffic situation, such as car counting and traffic flow. The results show that our approach can add context to classifications with a high degree of accuracy in some of the cases, achieving 95% car counting accuracy during the day. Fine tuning approaches are also discussed with reference to the video traffic representation case while keeping to the same proposed methodology.