Video Quality Assessment based on Quality Aggregation Networks

Abstract

A reliable video quality assessment (VQA) algorithm is essential for evaluating and optimizing video processing pipelines. In this paper, we propose a quality aggregation network (QAN) for full-reference VQA, which models the characteristics of human visual perception of video quality in both spatial and temporal domain. The proposed QAN is composed of two mod-ules, the spatial quality aggregation (SQA) network and the tem-poral quality aggregation (TQA) network. Specifically, the SQA network models the quality of video frames using 3D CNN, taking both spatial and temporal masking effects into consideration for the modeling of the perception of human visual system (HVS). In the TQA network, considering the memory effect of HVS facing the temporal variation of frame-level quality, an LSTM-based temporal quality pooling network is proposed to capture the nonlinearities and temporal dependencies involved in the process of quality evaluation. According to the experimental results on two well-established VQA databases, the proposed model could outperform the state-of-the-art metrics. The code of the proposed method is available at: https://github.com/lorenzowu/QAN.