Distantly supervised reinforcement localization for real-world object distribution estimation

Abstract

Predicting the distribution of objects in the real world from monocular images is a challenging task due to the disparity between object distributions in perspective images and reality. Many researchers focus on predicting object distributions by converting perspective images into Bird’s-Eye View (BEV) images. In scenarios where camera parameter information is unavailable, the prediction of vanishing lines becomes critical for performing inverse perspective transformations. However, accurately predicting vanishing lines necessitates accounting for variations in object size, which cannot be effectively captured through simple regression models. Therefore, this paper proposes a size variation-aware method, utilizing expert knowledge from object detection to build a reinforcement learning framework for predicting vanishing lines in traffic scenes. Specifically, this method leverages size information from trained detectors to convert perspective images into BEV images without the need for additional camera intrinsic parameters. First, we design a novel reward mechanism that utilizes prior knowledge of scale differences between similar objects in perspective images, allowing the network to automatically update and learn specific vanishing line positions. Second, we propose a fast inverse perspective transformation method, which accelerates the training speed of the proposed approach. To evaluate the effectiveness of the method, experiments are conducted on two traffic flow datasets. The experimental results demonstrate that the proposed algorithm accurately predicts vanishing line positions and successfully transforms perspective images into BEV images. Furthermore, the proposed algorithm performs competitively with directly supervised methods. The code is available at: https://github.com/HotChieh/DDRL.