WSPM3D: Weakly Supervised Prompted Monocular 3-D Object Detection Using Image-Level Labels

Abstract

Weakly supervised monocular 3D object detection aims to predict 3D object locations and attributes using annotations on RGB images. Existing methods typically rely on large-scale 2D bounding box annotations. However, large-scale annotations remain costly. Moreover, inaccurate 2D boxes often lead to poor supervision and significantly degrade 3D detection performance. To address this limitation, we propose a novel Weakly-Supervised Prompted Monocular 3D (WSPM3D) Object Detection framework that relies solely on image-level labels and unlabeled point clouds. First, a point-prompt generation (PPG) module is introduced to extract semantic activation cues from image-level labels. Then, an occlusion-aware segmentation module (SAM2-SwinT) is developed to produce stable and consistent 2D pseudo-boxes using semantic cues from PPG. Furthermore, we introduce a Distance-Aware Refinement (DAR) module to enhance the quality of pseudo-boxes for distant objects, so that these enhanced 2D pseudo-boxes can guide the extraction of Candidate Object LiDAR Points (COLPs) to implicitly boost the performance of 3D object detection. Finally, two auxiliary geometric losses are proposed to prevent depth localization drift and align the predicted and LiDAR-derived BEV heatmaps, offering robust geometric supervision via sparse point clouds. Experimental results on the KITTI and nuScenes datasets demonstrate that our framework achieves superior monocular 3D object detection performance while greatly reducing the cost and potential errors of manual annotations.