Learning referee evaluation and assessing action quality from coarse to fine in diving sport

Abstract

Intelligently assessing the quality of athletic performances in sports scenarios remains a fascinating challenge in computer vision. However, unraveling the subtle distinctions between two similar actions in videos and mapping those video representations to quality scores remain significant obstacles. To address these challenges, this work redefines the paradigm of quality score estimation from traditional relative quality score prediction to relative referee score prediction. To make this shift, a cross-feature fusion module rooted in Transformer-based video representation is introduced, to improve pairwise video feature learning in the realm of action quality assessment. Then, a novel contrastive action parsing decoder module generates mid-level representations to effectively connect visual features with detailed quality scores. Both modules utilize cross-attention mechanisms; the former refines the pairwise video features to represent the differences between video pairs, while the latter updates the input queries corresponding to each referee’s evaluation. Finally, to achieve precise quality score estimation, we introduce a meticulous coarse-to-fine decision process, integrating a score classifier and offset regressor. After validation on challenging diving datasets, including MTL-AQA, FineDiving, and TASD-2, the experimental results show that the proposed approach demonstrates effectiveness and feasibility when compared with state-of-the-art methods.