Metrics for Evaluating Video Streaming Quality in Lossy IEEE 802.11 Wireless Networks

Abstract

Peak Signal-to-Noise Ratio (PSNR) is the simplest and the most widely used video quality evaluation methodology. However, traditional PSNR calculations do not take the packet loss into account. This shortcoming, which is amplified in wireless networks, contributes to the inaccuracy in evaluating video streaming quality in wireless communications. Such inaccuracy in PSNR calculations adversely affects the development of video communications in wireless networks. This paper proposes a novel video quality evaluation methodology. As it not only considers the PSNR of a video, but also with modifications to handle the packet loss issue, we name this evaluation method MPSNR. MPSNR rectifies the inaccuracies in traditional PSNR computation, and helps us to approximate subjective video quality, Mean Opinion Score (MOS), more accurately. Using PSNR values calculated from MPSNR and simple network measurements, we apply linear regression techniques to derive two specific objective video quality metrics, PSNR-based Objective MOS (POMOS) and Rates-based Objective MOS (ROMOS). Through extensive experiments and human subjective tests, we show that the two metrics demonstrate high correlation with MOS. POMOS takes the averaged PSNR value of a video calculated from MPSNR as the only input. Despite its simplicity, it has a Pearson correlation of 0.8664 with the MOS. By adding a few other simple network measurements, such as the proportion of distorted frames in a video, ROMOS achieves an even higher Pearson correlation (0.9350) with the MOS. Compared with the PSNR metric from the traditional PSNR calculations, our metrics evaluate video streaming quality in wireless networks with a much higher accuracy while retaining the simplicity of PSNR calculation.