Assessing unreliability in OTT video QoE subjective evaluations using clustering with idealized data

In this paper, we describe an Over-The-Top (OTT) video Quality of Experience (QoE) subjective evaluation experiment that was carried out to examine variations in the way subjects assess viewing experiences. The experiment focuses on different level of impairment and failure types, using 5-point measurement scales. Clustering is used to differentiate between unreliable and reliable participants, where reliability is defined in terms of criteria such as consistency of rating and ability to distinguish between qualitative differences in level of impairments. The results show that clustering a data set that is augmented with unreliable pseudo-participants can provide a new and improved perspective on individual differences in video QoE assessment.