Training listeners for multi-channel audio quality evaluation in MUSHRA with a special focus on loop setting

Abstract

Audio quality evaluation for audio material of intermediate and high quality requires expert listeners. In comparison to non-experts, these are not only more critical in their ratings, but also employ different strategies in their evaluation. In particular they concentrate on shorter sections of the audio signal and compare more to the reference than inexperienced listeners. We created a listener training for detecting coding artifacts in multi-channel audio quality evaluation. Our training is targeted at listeners without technical background. For this training, expert listeners commented on smaller sections of an audio signal they focused on in the listening test and provided a description of the artifacts they perceived. The non-expert listeners participating in the training were provided with general advice for helpful strategies in MUSHRA tests (Multi Stimulus Tests with Hidden Reference and Anchor), with the comments on specific sections of the stimulus by the experts, and with feedback after rating. Listener's performance improved in the course of the training session. Afterwards they performed the same test without the training material and a further test with different items. Performance did not decrease in these tests, showing that they could transfer what they had learned to other stimuli. After the training they also set more loops and compared more to the reference.