Should WebRTC Prioritise Intelligibility over Speech Quality?

Abstract

Network delay remains a challenge for real-time voice communication on the web. Jitter buffer algorithms have been widely deployed in popular platforms such as webRTC to reduce the impact of delay with playout adjustments. A trade off must be made between speech loss and voice degradations as adjustments can either drop segments resulting in a loss of speech intelligibility or change the rate of playout and impact the pitch or natural sound of the speech. Both options can negatively influence a listener's quality of experience (QoE). Optimising this trade-off requires knowledge of how intelligibility and quality are perceived and priorities when a listener syntheses both factors into a fused QoE judgement. This study conducted two subjective experiments to evaluate intelligibility and quality independently along with a short descriptive analysis to address the interplay between the two factors. The study uses a dataset that simulated listener-end speech under extreme but realistic network delay conditions using webRTC's standard jitter buffer and a variation that prioritised minimisation of packet loss. The results show that intelligibility is a key dimension in quality judgement for the scenarios tested. As a result, this study calls for attention when comparing the quality scores as the overlooked non-traditional quality attributes are proven to be actively contributing to the overall QoE. The descriptive analysis also indicates there is inconsistency in the interpretation of ‘quality’ among the assessors. This finding questions the methodology used in standard QoE subjective experiment designs and proposes adopting a more flexible approach to measure subjective QoE.