Vision-based recognition and geolocation of colored steel constructions in cities to promote urban management

Abstract

The ability to accurately identify and geolocate colored steel constructions (CSCs) at a city scale addresses critical challenges in urban management, particularly concerning unauthorized and temporary structures that pose safety risks. This study introduces an end-to-end pipeline for recognizing CSC information in cities using unmanned aerial vehicle (UAV) to promote efficient urban management. The proposed pipeline involves several techniques, including UAV-based data acquisition, deep learning-powered image processing, photogrammetry, and map visualization. The pipeline encompasses UAV flight planning, image collection, detection and segmentation of CSCs, and street location visualization, producing comprehensive output regarding the CSC area, distribution density, and precise location. The accuracy level was validated through the ArUco marker-based experiments. Case studies were performed in two real-world cities to test the pipeline, indicating the encouraging accuracy, efficiency, and robustness of the proposed method. Furthermore, the two-stage instance segmentation model developed in this study requires only an object detection dataset for training, which avoids collecting pixel-level mask annotation datasets. Hence, users can easily train object detection models for other objects to replace the model used in this study, thereby facilitating the recognition and localization of other high-risk objects within urban areas.