The Effect of Non-Reference Point Cloud Quality Assessment (NR-PCQA) Loss on 3D Scene Reconstruction from a Single Image

Abstract

This paper proposes a two-stage approach for 3D scene reconstruction from a single image. The first stage involves a monocular depth estimation model, and the second stage involves a point cloud model that recovers depth shift and the focal length from the generated depth map. The paper investigates the use of various pre-trained state-of-the-art transformer models and compares them to existing work without transformers. The loss function is improved by adding a No-Reference point cloud quality assessment (NR-PCQA) to account for the quality of the generated point cloud structure. The paper reports results on four datasets using Locally Scale Invariant RMSE (LSIV) as the metric of evaluation. The paper shows that transformer models outperform previous methods, and transformer models that took into account NR-PCQA outperformed those that did not.