Sight vs. Sound Judgments of Music Performance Depend on Relative Performer Quality: Cross-cultural Evidence From Classical Piano and Tsugaru Shamisen Competitions

Which information dominates in evaluating performance in music? Both experts and laypeople consistently report believing that sound should be the most important domain when judging music competitions, but experimental studies of Western participants rating video-only vs. audio-only versions of 6-second excerpts of Western classical performances have shown that in at least some cases visual information can play a stronger role. However, whether this phenomenon applies generally to music competitions or is restricted to specific repertoires or contexts is disputed. In this Registered Report, we focus on testing the generalizability of sight vs. sound effects by replicating previous studies of classical piano competitions with Japanese participants, while also expanding the same paradigm using new examples from competitions of a traditional Japanese folk musical instrument: the Tsugaru shamisen. For both classical piano and Tsugaru shamisen, we ask participants to choose the winner between the 1st- and 2nd- placing performers in 5 competitions and the 1st-place and low-ranking performers in 5 competitions (i.e., 40 performers total from 10 piano and 10 shamisen competitions). We tested the following three predictions twice each (once for piano and once for shamisen): 1) an interaction was predicted between domain (video-only vs. audio-only) and variance in quality (choosing between 1st and 2nd place vs. choosing between 1st and low-placing performers); 2) visuals were predicted to trump sound when variation in quality is low (1st vs. 2nd place); and 3) sound was predicted to trump visuals when variation in quality is high (1st vs. low-placing). Our experiments (n = 155 participants) confirmed our first predicted interaction between audio/visual domain and relative performer quality for both piano and shamisen conditions, suggesting that this interaction is cross-culturally general. In contrast, the second prediction was only supported for the piano stimuli and the third prediction was only supported for the shamisen condition, suggesting culturally dependent factors in the specific balance between sight and sound in the judgment of musical performance. Our results resolve discrepancies and debates from previous sight-vs-sound studies by replicating and extending them to include non-Western participants and musical traditions. Our findings may also have practical applications to evaluation criteria for performers, judges, and organizers of competitions, concerts, and auditions.