From writing to drawing: Examining visual composition in danmu-mediated textual communication

Danmu (anonymous superimposed video comments) is a popular form of communication on Chinese and Japanese video sharing sites. While previous studies primarily focused on the verbal aspects of danmu comments, there is a growing interest in exploring their multimodal features. This study investigates the unique potential of danmu comments to communicate visual meaning, interact with on-screen content, and thereby shape audience perception. Informed by a social semiotic approach to multimodality and relevant pragmatic theories, the study analyzed 50 screenshots of visually significant danmu comments to understand the resources used by commenters to craft visual comments and the relationship between these comments and the screen. Our findings revealed that four key resources were utilized to create visual comments: arrows, kaomoji, context-specific special characters and symbols, and ASCII art. Additionally, five types of relationships were identified between visual danmu comments and the screen, including deictic, emphasizing, complementing, extending, and independent. This study provides an up-to-date examination of the possibilities for visual expression in textual communication and extends previous research on semiotic resources in social media. It also discusses the role of danmu visual play as internet memes and the emergence of danmu visual grammar.