Formation-Guided Multimodal Representation Learning for Group Action Quality Assessment

Abstract

Group action quality assessment (GAQA) aims to assess the quality of coordinated movements performed by multiple individuals. Most existing methods often overlook structural formation patterns that are essential for understanding coordination in multi-person GAQA. In this paper, we propose a formation-based multimodal scoring network that incorporates formation transcripts and visual cues for comprehensive evaluation. Specifically, we first leverage the sequence reasoning capabilities of large language models to capture both the technical and artistic semantics from formation transitions. Then, we introduce a shared feature extraction module to learn modality-invariant representations across visual and formation modalities, and a modality-specific extraction module to retain unique and complementary modality-specific cues. Extensive experiments on three challenging datasets (i.e., LOng-form GrOup, OlympicFS, and FS1000) demonstrate the effectiveness of our method.