A 15.9 mW 96.5 fps Memory-Efficient 3D Reconstruction Processor with Dilation-based TSDF Fusion and Block-Projection Cache System

Abstract

A real-time dense 3D reconstruction on lightweight AR headsets is challenging since its memory access surpasses the available memory bandwidth. To solve this problem, the proposed processor integrates two key building blocks - Dilation-based TSDF (D-TSDF) fusion and Block-Projection (BP) engine. D-TSDF projects the depth map in the reverse order of voxel-to-pixel coordinate transformation and dilates it, leading to 96.61% External Memory Access (EMA) reduction with minimum map quality degradation. Second, a specialized BP engine compresses high-resolution occupancy grid by decomposing the 3D bitmap into 2D and 1D vectors, achieving $\times \mathbf{166.09}$ reduced memory bandwidth. The proposed processor is implemented in 28nm CMOS technology occupying 1.27 mm2 area. As a result, 96.45 fps 3D reconstruction is possible while consuming only 15.94 mW power.