CT-PCQA: A Convolutional Neural Network and Transformer combined Method for Point Cloud Quality Assessment

Abstract

Point clouds serve as a prevalent means of 3D content representation, finding wide applications across multiple fields. However, their extensive and intricate data often encounter various distortions due to limitations in storage and bandwidth. Notably, compression and simplification, commonly employed during point cloud transmission, significantly damage their quality. To address this challenge, the development of effective methodologies for quantifying distortion in point clouds becomes imperative. In this paper, we propose a novel approach for Point Cloud Quality Assessment (PCQA) named as CT-PCQA by combining Convolutional Neural Network (CNN) and Transformer methods. Our method involves generating multi-projections through a cube-like projection process, catering to both full-reference (FR) and no-reference (NR) PCQA tasks. We leverage the strengths of CNN and Transformer by extracting quality-aware features using popular vision backbones. For FR quality representation, we compute the similarity between the feature maps of reference and distorted projections. For NR quality representation, we simply employ average pooling on the feature maps of distorted projections. Subsequently, these quality representations are regressed into visual quality scores using fully-connected (FC) layers. Our participation in the ICIP 2023 PCVQA Grand Challenge yielded significant results, securing the top spot in four out of the five competition tracks. Furthermore, experimental results demonstrate that the proposed method achieves state-of-the-art performance across various databases. The related code will be released at https://github.com/zyj-2000/CT-PCQA.