RevFB-BEV: Memory-Efficient Network With Reversible Swin Transformer for 3D BEV Object Detection

Abstract

The perception of Bird's Eye View (BEV) has become a widely adopted approach in 3D object detection due to its spatial and dimensional consistency. However, the increasing complexity of neural network architectures has resulted in higher training memory, thereby limiting the scalability of model training. To address these challenges, we propose a novel model, RevFB-BEV, which is based on the Reversible Swin Transformer (RevSwin) with Forward-Backward View Transformation (FBVT) and LiDAR Guided Back Projection (LGBP). This approach includes the RevSwin backbone network, which employs a reversible architecture to minimise training memory by recomputing intermediate parameters. Moreover, we introduce the FBVT module that refines BEV features extracted from forward projection, yielding denser and more precise camera BEV representations. The LGBP module further utilises LiDAR BEV guidance for back projection to achieve more accurate camera BEV features. Extensive experiments on the nuScenes dataset demonstrate notable performance improvements, with our model achieving over a $4\times$ reduction in training memory and a more than $12\times$ decrease in single-backbone training memory. These efficiency gains become even more pronounced with deeper network architectures. Additionally, RevFB-BEV achieves 68.1 mAP (mean Average Precision) on the validation set and 68.9 mAP on the test set, which is nearly on par with the baseline BEVFusion, underscoring its effectiveness in resource-constrained scenarios.