CUBE360: Learning Cubic Field Representation for Monocular Panoramic Depth Estimation

Abstract

Panoramic depth estimation presents significant challenges due to the severe distortion caused by equirectangular projection (ERP) and the limited availability of panoramic RGB-D datasets. Inspired by the recentsuccess of neural rendering, we propose a self-supervised method, named CUBE360, that learns a cubic field composed of multiple Multi-Plane Images (MPIs) from a single panoramic image for continuous depth estimation at any view direction. Our CUBE360 employs cubemap projection to transform an ERP image into six faces and extract the MPIs for each, thereby reducing the memory consumption required for MPIs processing of high-resolution data. An attention-based blending module is then employed to learn correlations among the MPIs of cubic faces, constructing a cubic field representation with color and density information at various depth levels. Furthermore, a dual-sampling strategy is introduced to render novel views from the cubic field at both cubic and planar scales. The entire pipeline is trained using photometric loss calculated from rendered views within a self-supervised learning (SSL) approach, enabling training without depth annotations. Experiments on synthetic and real-world datasets demonstrate the superior performance of CUBE360 compared to previous SSL methods.